Sai Rama Krishna Gangavarapu

Data Engineer @MetLife

US

Engineered a real-time data ingestion pipeline using Apache Kafka and AWS Kinesis to process 2 million daily policy transactions, reducing data latency fromhours to under 60 seconds for actuarial teams.• Architected a Delta Lake house on Databricks leveraging Unity Catalog, enabling schema enforcement and time travel for 15 TB of historical claimsdata, improving data reliability for analytics by 70%.• Developed automated data quality frameworks using Great Expectations within Airflow DAGs, identifying and resolving 500+ data anomalies monthly acrosscritical financial datasets.• Migrated legacy on-premise SQL Server ETL processes to Azure Data Factory and Synapse Analytics, reducing data processing costs by permonth.• Optimized complex PySpark jobs by implementing adaptive query execution and predicate pushdown, cutting down average job runtime from 45 minutes to 20minutes.• Implemented a CI/CD pipeline for Databricks notebooks using Azure DevOps, reducing deployment time for new code from 3 days to 4 hours and ensuringenvironment consistency.• Built a scalable feature store for ML models in Azure Machine Learning, accelerating model training cycles by 30% for the data science team.• Automated the provisioning of data infrastructure using Terraform modules, standardizing deployments across 3 environments and reducing setup time from 2 days to3 hours.• Designed and deployed a company-wide data catalog with Azure Purview, automating data lineage discovery for over 500 datasets and improvinggovernance compliance.• Orchestrated end-to-end data pipelines with Apache Airflow, managing dependencies for 50+ daily jobs and improving overall workflow reliability by 90%.

California State University - East Bay

Master of Science - MS, Computer Science

R.V.R. & J.C. College of Engineering

Bachelor of Technology - BTech, Computer Science