Manisha Jonnagaddala

Data Engineer @Nowigence

Albany, NY, US

Architected and deployed scalable ETL pipelines using PySpark and Apache Airflow, enabling efficient data integration across distributed systems and optimizing data flow.• Designed and implemented a data lake architecture on AWS S3, ensuring seamless access and storage for both structured and semi-structured data, enhancing downstream analytics.• Enhanced data quality and reliability by implementing automated validation and error-checking in processing pipelines, emphasizing data integrity and reducing manual verification tasks.• Optimized SQL and NoSQL database interactions by engineering efficient queries, reducing critical business metric query execution times.• Integrated over 10 data sources into a unified data warehouse, facilitating easier access for business analytics and improving decision-making processes.• Led cloud migration efforts, moving on-premises data systems to AWS, significantly improving system scalability and reducing infrastructure costs.• Utilized Kafka streams for real-time data ingestion and event streaming, increasing data availability for immediate reporting and business actions.• Constructed machine learning data pipelines for advanced analytics, supporting the development of predictive models for business decision-making.• Developed monitoring solutions via Grafana and AWS CloudWatch, ensuring real-time visibility into system performance and minimizing downtime issues.• Optimized AWS resource management to minimize cloud costs, enhancing system performance and cost efficiency.• Mentored junior data engineers, fostering best practices in data engineering, including data pipeline and cloud architecture design.