Hemanth Kosuri

About Hemanth Kosuri

I build data pipelines that don’t just run — they power regulatory decisions, real-time…

Experience

1. Data Engineer

   Amgen

   Sep 2024 — Present · US

   Engineered high-throughput ETL pipelines processing 10–12 TB/day using AWS Glue, Amazon S3, and Apache Spark on EMR, enabling scalable ingestion and transformation of large-scale clinical and manufacturing datasets into a centralized data lake. Built robust real-time streaming pipelines using Apache Kafka (MSK) and Spark Structured Streaming, processing 1M–2M events/hour with <5 second latency, enabling near real-time analytics and operational monitoring. Optimized Spark workloads by implementing partitioning strategies, caching mechanisms, and broadcast joins, improving job performance by 40–45% and reducing overall compute costs by 20–25%. Designed and managed centralized metadata using AWS Glue Data Catalog across 500+ datasets, ensuring schema consistency, improved data discoverability, and governance compliance. Developed scalable dimensional data models (star schema) in Amazon Redshift supporting 20+ enterprise reporting and analytics use cases, including regulatory and operational dashboards. Implemented automated data quality validation frameworks using Python and SQL across 100+ pipelines, including schema checks, null validations, and reconciliation logic, reducing data defects by 30%. Automated infrastructure provisioning and deployment pipelines using Terraform and Jenkins CI/CD, reducing release cycles by 70%(2–3 days to <6 hours) and improving deployment reliability. Established monitoring and alerting using AWS CloudWatch, logs, and metrics to ensure 99.9% pipeline uptime, faster incident detection, and improved operational stability