Tarun Vamsi Krishna Chavatapalli
Data Engineer | Batch & Real-Time Data Pipelines | Spark, Kafka, Airflow | Cloud-Native (AWS | GCP)
- Role
- Data Engineer at Spark Technologies Pvt. Ltd
- Location
- Dallas-Fort Worth, TX, US
- LinkedIn followers
- 500 followers
About Tarun Vamsi Krishna Chavatapalli
Data Engineer with 3+ years of experience building batch and real-time data pipelines, cloud-native architectures, and data lakehouse solutions. Experienced in Spark, Kafka, Airflow, Python, SQL, and cloud platforms (AWS, GCP).I design, implement, and operate production-grade pipelines that support analytics, reporting, and business-critical workflows. I focus on pipeline reliability, performance tuning, and automation while ensuring secure, scalable, and maintainable data systems.Key achievements:Built real-time Spark Streaming pipelines integrated with Kafka, processing high-volume event data.Migrated legacy MapReduce ETL workloads to modern Spark pipelines on AWS EMR and GCP Dataproc.Automated workflows with Airflow, standardizing orchestration and improving operational reliability.Delivered data solutions with row-level security using BigQuery authorized views and Dataproc automation.Developed dashboards and monitoring solutions using Grafana, ELK, and OpenSearch for production observability.
Experience
Data Engineer
Jan 2025 — Present · Dallas, TX, US
Managed Spark pipelines on AWS EMR for batch processing and GCP Dataproc for scalable analytics.Built ingestion workflows using AWS Glue and Lambda, reducing manual pipeline tasks.Migrated Oozie jobs to Airflow on GCP Composer with automated alerts and retries.Automated extraction of historical data from Netezza to S3 for Spark processing.Developed monitoring dashboards with Grafana, ELK, and OpenSearch.Implemented secure data access and automated Dataproc cluster management for multiple teams.
Education
Aditya Educational Institutions
Bachelor of Technology - BTech, ece
2017 — 2021
Sri Chaitanya College of Education
Intermediate
2015 — 2017
Ravindra Bharathi school
SSC
2013 — 2015
University of North Texas
Master's degree, Advanced Data Analytics
Cisco Networking Academy
Computer Networking
2018 — 2021
Find verified contacts for anyone on LinkedIn
Unifers gives sales teams verified emails and direct dials, enriched profiles, and outreach that lands in the inbox.
Free plan included · No credit card required
This profile is compiled from publicly available professional sources. Unifers is not affiliated with or endorsed by LinkedIn. Request removal of this profile.