Varun Jain
Data Engineer @ZS
Signup · Get unlimited contacts
WORK HISTORY
Data Engineer @ZS
IN
Creating and debugging large-scale data pipelines using Airflow as the data orchestration tool to facilitate the flow of data from Source to MDM (Informatica). Performed overlap analysis across multiple datasets to draw insights by performing complex SQL queries. Testing python scripts and debugging code in Pyspark shell on EMR. Used RDS to configure cluster configurations and system components for AWS.
EDUCATION
Maharaja Surajmal Institute Of Technology
Bachelor of Technology - BTech
Delhi School Of Music - India
Grade 3
ABOUT VARUN JAIN
A tech enthusiast with a keen interest in Data Engineering. Experience in designing and optimizing large-scale ETL data pipelines using Airflow for seamless data integration from diverse sources to a data lake. Proficient in developing Spark SQL scripts in Pyspark for enhanced data processing and analysis. Adept at data modeling and collaborating closely with stakeholders to provide comprehensive insights into data flow within the entire data infrastructure. Techstack worked on-Data processing platform: Apache SparkLanguages: Python, SQLCloud: AWS Services such as Airflow, EMR, EC2, S3, Redshift, Glue, Lambda, AthenaVersion Control: Git/GitHub
This profile is compiled from publicly available professional sources. Unifers is not affiliated with or endorsed by LinkedIn. Request removal of this profile.