Ashutosh Pandey
Immediate Joiner || Data Engineer at TCS || Pyspark || SQL || Python || Talend || AWS || Apache Spark || Hive || Passionate About Data Engineering Solutions
- Role
- Data Engineer at Tata Consultancy Services
- Location
- Mumbai, MH, IN
- LinkedIn followers
- 500 followers
About Ashutosh Pandey
Education never ends, Watson. It is a series of lessons, with the greatest for the…
Experience
Data Engineer
Jul 2022 — Present · Mumbai, IN
Built end-to-end ETL pipelines using Talend and PySpark, loading data from JSON and CSV sourcesinto Hive tables for efficient storage and querying.• Developed Pyspark scripts for cleaning, transforming and analyzing batch data, enabling the generation of key metrics, aggregates and business critical indicators for client use in analytics, reporting anddecision making.• Optimized Talend jobs, bringing down runtimes by 99%, resulting in significantly faster data processing and improved operational efficiency.• Implemented SCD Type 4 logic to manage historical data as per client requirements, ensuring dataaccuracy and consistency.• Developed and optimized 500+ complex SQL queries to automate data flow between two distinctmodules, eliminating manual data uploads and streamlining client processes, resulting in significant time savings and improved efficiency.• Implemented PySpark scripts to pre-process and transform migrated data from a rigid columnar format in source to a dynamic row-based format as required by the client, while reducing manual efforts and improving the speed of the migration process by 85%.• Collaborated with cross-functional teams and demonstrated strong initiative to align goals, ensuring the development of a reliable and fault-tolerant software that met client expectations.
Education
Thakur College Of Engineering and Technology
Bachelor of Engineering - BE
2018 — 2022
Find verified contacts for anyone on LinkedIn
Unifers gives sales teams verified emails and direct dials, enriched profiles, and outreach that lands in the inbox.
Free plan included · No credit card required
This profile is compiled from publicly available professional sources. Unifers is not affiliated with or endorsed by LinkedIn. Request removal of this profile.