Varun Shah
Data Engineer | Databricks, Spark, AWS
- Role
- Senior Software Engineer at Flexera
- Location
- Mumbai, MH, IN
- LinkedIn followers
- 500 followers
About Varun Shah
Experience working on Bigdata tools in the AWS cloud platform, mainly Spark and Airflow, along with other AWS services like Dynamodb, RDS, lambda, Glue, Redshift. Developed complex workflow pipelines using Apache Airflow, processing a large amount of data using Apache Spark, optimizing the same, and ensuring that it follows the best practices. Also, experienced in bigdata tools like Hive, Sqoop. My programming languages include Scala and python.
Experience
Senior Software Engineer
Feb 2024 — Present · IN
Build and enhanced 24x7 running data pipelines on databricks for customers> Optimize SQL queries, reducing latency from 48 hours to under 1 hour> Lead development of Monitoring tools - SLA, Spark metrics, and other pipelines to enrich the data pipelines observability> Led team of 6 members for ensuring quality and timely delivery of features
Education
Dwarkadas J. Sanghvi College of Engineering
Bachelor of Technology, Information Technology
2015 — 2019
Find verified contacts for anyone on LinkedIn
Unifers gives sales teams verified emails and direct dials, enriched profiles, and outreach that lands in the inbox.
Free plan included · No credit card required
This profile is compiled from publicly available professional sources. Unifers is not affiliated with or endorsed by LinkedIn. Request removal of this profile.