Lahari C.

About Lahari C.

Data Engineer with around 5 years of experience, proficient in Python, T-SQL, PL/SQL, and Shell scripting (Bash), with hands-on expertise in automating over 120+ data processing scripts and reducing ETL execution time by 25% through optimized queries and reusable code components. • Built scalable and secure data pipelines using AWS Redshift, S3, Lambda, and EMR, processing over 3TB of structured and semi-structured data daily across production and staging environments. • Designed cloud-based data warehouse solutions on Snowflake, Google BigQuery, and Azure Synapse, achieving a 35% improvement in query response time and enabling faster access to business insights. • Developed 50+ robust ETL workflows using Apache Airflow, dbt, Matillion, and Fivetran to automate data ingestion, transformation, and delivery from various cloud and on-premise sources. • Integrated Kafka, Hadoop, Flink, and Google Dataflow for high-volume streaming and batch data pipelines, enabling near real-time analytics and supporting 99.9% uptime in distributed processing. • Managed and optimized data storage across PostgreSQL, MySQL, Oracle, and MongoDB, ensuring high availability and reducing query latency by tuning indexes and partitioning large datasets. • Configured NoSQL solutions such as MongoDB and DynamoDB, and leveraged Amazon S3 and Azure Blob Storage for cost-efficient archival and backup strategies with 99.99% data durability. • Implemented CI/CD pipelines using Docker, Kubernetes, Jenkins, and GitLab CI, reducing deployment time by 40% and enabling seamless integration for analytics and ETL components. • Automated infrastructure provisioning using Terraform and Infrastructure as Code (IaC) principles, accelerating cloud deployment cycles and ensuring consistency across multiple environments.

Experience

1. Data Engineer

   Northern Trust

   Jan 2024 — Present · US

Education

• Andhra University

  Bachelor's degree

• University at Buffalo

  Master's degree