Tanishka Kapoor

About Tanishka Kapoor

I am a Data Engineer with hands-on experience in building scalable, real-time data pipelines and orchestration frameworks across hybrid cloud environments (GCP and Hadoop). My work focuses on designing efficient, automated workflows using Airflow, Kafka, and Spark to transform raw datasets into trusted, analytics-ready formats. With expertise in both real-time streaming and batch processing, I help teams unlock the value of their data through automation, quality, and scalability.Key Strengths:Data Pipeline Engineering: Designed and optimized end-to-end Airflow DAGs for validating, and storing ESG and enterprise data across GCP and HDP platforms.Real-Time & Batch Processing: Developed Kafka-based pipelines for real-time data quality metrics, backed by Parquet-based storage on GCS and HDFS for long-term analytics.Data Quality & Governance: Integrated Great Expectations within data pipelines to enable rule-based, automated data validation and proactive anomaly detection across 8+ use cases.Cloud-Native Development: Led deployments on GCP using, Docker, Cloud Run, CloudSQL, and Dataproc; also built Spark-based Kafka consumers in on-prem environments.API & ETL Development: Built REST APIs and metadata-driven ETL frameworks for dynamic data quality rule execution and seamless CloudSQL integration.

Experience

1. Senior Software Engineer

   Airtel Digital

   Jan 2026 — Present · Gurugram, IN

Education

• Mody University

  Bachelor of Technology, Computer Science

  2017 — 2021