Astha Agarwal

About Astha Agarwal

Experienced Data Engineer in designing and implementing scalable data solutions, including real-time dashboards, automated reporting systems, and robust ETL pipelines. Proficient in PySpark, Python, SQL, Kafka, and AWS, with a strong foundation in data validation and testing practices.Skilled at optimizing data workflows, reducing manual efforts, and improving reporting accuracy and timeliness. Committed to building high-quality, efficient data systems that support business intelligence and decision-making.

Experience

1. Data Engineer

   IBM

   Aug 2023 — Present · Bengaluru, IN

   As a Data Engineer, I led the design and implementation of scalable, real-time data solutions that significantly optimized data processing workflows and improved business decision-making- Spearheaded the end-to-end development of a real-time data dashboard by integrating Apache Kafka and Apache Druid, improving data accessibility and operational visibility- Built and deployed automated reporting systems using Python, Pandas, and NumPy, streamlining data extraction, transformation, and storage- Designed and maintained robust PySpark pipelines to handle and transform large-scale datasets efficiently- Achieved a 90% reduction in manual data processing efforts through process automation, leading to more accurate and timely stakeholder reporting- Enhanced data accuracy and reliability by seamlessly integrating log data into Druid, ensuring high-quality insights for analytics teams.Tech Stack: PySpark, Python, Kafka, Druid, Pandas, NumPy, SQL, MySQL

Education

• DIT UNIVERSITY

  Bachelor's degree, Electrical and Electronics Engineering