Yash Gupta
Data Engineer @ConnectandHeal
- Role
- Data Engineer at Connect and Heal - CNH Care
- Location
- New Delhi, DL, IN
- LinkedIn followers
- 500 followers
About Yash Gupta
Data Engineer at Connect and Heal with a Bachelor of Technology in Computer Science from…
Experience
Data Engineer
Jun 2024 — Present · Bengaluru, IN
Led RDS Migration & Cost Optimization: Designed and implemented an end-to-end framework for migrating RDS databases between clusters, ensuring data quality and integrity. Successfully consolidated multiple RDS clusters into a single optimized cluster per environment, reducing costs significantly for each environment i.e Production, Stage, Dev, and Preprod- Scalable Data Pipeline Development: Built a high-performance data pipeline to efficiently query Apache Hudi datasets and securely share the results with the client’s cloud storage on a daily basis- Data Lakehouse Monitoring & Alerting: Developed real-time monitoring dashboards and implemented automated alerts to enhance data observability and ensure proactive issue resolution within the Data Lakehouse- AWS Workflow Automation: Designed and automated AWS Lambda & DynamoDB workflows, handling pagination complexities and large payloads for seamless backup retrieval and processing- Optimized Data Pipelines & Workflow Management: Maintained and optimized Data Lakehouse workflows built on AWS S3, MWAA, Glue, DMS, Apache Hudi, Spark, and EMR. Ensured stability and efficiency across the medallion architecture, resolving critical failures and enhancing incremental upserts in Hudi tables.
Education
Presidium School
High School Diploma, Science
Shiv Nadar University
Bachelor of Technology - BTech, Computer Science
2020 — 2024
Find verified contacts for anyone on LinkedIn
Unifers gives sales teams verified emails and direct dials, enriched profiles, and outreach that lands in the inbox.
Free plan included · No credit card required
This profile is compiled from publicly available professional sources. Unifers is not affiliated with or endorsed by LinkedIn. Request removal of this profile.