Rajneesh Dubey
Senior Data Engineer @ ZS | AWS, Apache Spark, EMR, Redshift, Hive, ETL
- Role
- Senior Data Engineer at ZS
- Location
- Gurugram, HR, IN
- LinkedIn followers
- 500 followers
About Rajneesh Dubey
As a Data Engineer at ZS, I leverage Apache Spark, AWS, Hive and Hadoop to design, develop, and maintain scalable data pipelines and platforms for various clients in the healthcare, life sciences, and pharmaceutical industries. I have over 4.5 years of experience in data engineering, including five months as an intern at ZS, where I received multiple accolades for my performance and contributions. I hold a Bachelor of Technology degree in computer science from Guru Gobind Singh Indraprastha University, where I gained a strong foundation in data structures, algorithms, and programming languages. I am passionate about solving complex data problems and delivering innovative solutions that enhance the quality and value of data-driven insights. I am also committed to continuous learning and improvement, and I seek to collaborate with diverse and talented teams that share my vision and values.
Experience
Senior Data Engineer
Jan 2024 — Present · Gurugram, IN
Led a 4+ team members to develop and maintain an active data warehousing and reporting solution for a pharma client, utilizing Hadoop, Hive, Spark, and Python. Implemented efficient ETL pipelines for batch processing of 1.1 TBs of daily data from diverse sources, facilitating daily data exports to Salesforce and Redshift.•Orchestrated the seamless migration of an existing Hadoop cluster to Redshift, optimizing performance and leveraging Tableau for enhanced reporting capabilities. Utilized AWS services including EMR, S3, and Lambda to streamline data storage and processing, resulting in improved efficiency and cost savings.•Spearheaded the performance optimization of Spark jobs within an ongoing project, employing advanced code-level and cluster-level optimization techniques. Utilized strategies such as Broadcast hash joins, repartitioning, coalescing, and salting to optimize data distribution and minimize shuffling. Implemented partition pruning to reduce unnecessary data scanning and minimize wide transformations, ensuring efficient resource utilization within the cluster.
Education
Guru Gobind Singh Indraprastha University
Bachelor of Technology - BTech, Computer Science
2017 — 2021
Central Board of Secondary Education
Senior secondary education
2014 — 2016
Find verified contacts for anyone on LinkedIn
Unifers gives sales teams verified emails and direct dials, enriched profiles, and outreach that lands in the inbox.
Free plan included · No credit card required
This profile is compiled from publicly available professional sources. Unifers is not affiliated with or endorsed by LinkedIn. Request removal of this profile.