Durgabhavani Pothineni

About Durgabhavani Pothineni

Machine learning Data engineer with Master\'s degree in Computer Science and over 5+ years…

Experience

1. Ml Data Engineering & Analytics

   Dollar General

   Oct 2023 — Present · Atlanta, GA, US

   Designed and built end-to-end data pipelines using PySpark, Apache Airflow (Composer), and Dataproc, enabling reliable ingestion, transformation, and processing of large-scale datasets on GCP.🞄 Developed Spark 3.x pipelines to process and optimize billions of rows of structured and semi-structured data, ensuring scalability and high performance for business-critical ML workflows.🞄 Implemented Spark SQL transformations and advanced optimization techniques (partitioning, caching, broadcast joins, vectorization) to improve query execution and reduce processing time for large datasets.🞄 Collaborated with data scientists to operationalize predictive models in production, integrating ML workflows with Vertex AI for demand forecasting, and customer analytics.🞄 Created reporting feeds and Looker dashboards that deliver real-time insights into sales, forecasting accuracy, and ensuring stakeholders have visibility into critical metrics.🞄 Conducted performance tuning and scalability testing across GCP services (BigQuery, Dataproc, Composer) to ensure workflows could handle high-volume data.🞄 Partnered with cross-functional teams to design data architectures and implement best practices for distributed systems and big data pipelines, aligning with enterprise-scale needs.🞄 Actively contributed to model implementation & optimization, translating mathematical models into scalable PySpark/SQL code to accelerate execution and improve forecasting accuracy.

Education

• R.V.R. & J.C. Institute of Engineering & Technology

  Bachelor of Technology

• Georgia State University

  Master's degree