Jyothi Janapureddi

Data Engineer @Next

Leicester, GB

Automated scheduling and execution of Databricks notebooks for data processing workflows, leveraging Azure Data Factory for orchestration- Led efforts in a cloud migration project, successfully transitioning data pipelines and storage systems from AWS to Azure while minimizing downtime and ensuring data integrity- Building scalable ETL pipelines using Databricks to process and transform large datasets, enhancing performance and maintainability- Utilizing PySpark SQL to create dashboards for monitoring job performance, data pipeline status, and table updates, improving visibility and operational efficiency- Streamlining data housekeeping processes by managing and optimizing storage in Azure Data Lake, ensuring compliance with data retention policies and cost efficiency- Collaborating with analysts and data scientists to deliver data solutions tailored to business needs, improving decision-making capabilities.