Pragathi K
Senior Data Engineer (Using Hadoop Ecosystems & Azure, Aws, Gcp Cloudtechnoliges @Elevance Health (Formerly Anthem)
Signup · Get unlimited contacts
WORK HISTORY
Senior Data Engineer (Using Hadoop Ecosystems & Azure, Aws, Gcp Cloudtechnoliges @Elevance Health (Formerly Anthem)
Change streams will be enabled for all the required collections. For any change to document like insert, update and delete document db will capture the changes into the log. These changes hours by default in the database.A DMS Task runs continuously for each collection, reading these changes.Changes are written to CSV files every hour or when the file size reaches 10 MB (whichever comes first).Files are stored in an S3 bucket: edp-pszone-dev-member.
EDUCATION
Campbellsville University
MSIT, Computer Science
Jawaharlal Nehru Technological University
B. tech, Electrical and Electronics Engineering
ABOUT PRAGATHI K
At Anywhere Real Estate Inc, our team leverages my expertise in Apache Spark and Databricks to develop and implement cutting-edge data processing solutions. Mastery in ETL, particularly with tools like Sqoop and Data Stage, enables us to efficiently ingest data from diverse sources, enhancing operational efficiency and decision-making. Proficiency in Python and experience with cloud platforms, including GCP and AWS, underpin our robust data engineering strategies.Collaboration is key in managing live applications and CDC processes, which I lead with a strategic approach to meet dynamic business needs. My contributions have been pivotal in designing end-to-end data pipelines and optimizing Pyspark codes, processing vast datasets up to 8 TB. These efforts support our mission to deliver data-driven insights that propel Anywhere Real Estate Inc. forward in a competitive marketplace.
This profile is compiled from publicly available professional sources. Unifers is not affiliated with or endorsed by LinkedIn. Request removal of this profile.