Shivansh Mishra

About Shivansh Mishra

Azure Data Engineer with 3.9+ years of experience in building scalable batch and streaming data pipelines using Azure Databricks, PySpark, SQL, Delta Lake, and Azure Data Factory.Experienced in designing metadata-driven ETL frameworks, Medallion Architecture, incremental data processing, and real-time streaming solutions using Azure Event Hub and Structured Streaming.Worked on enterprise-scale data platforms processing millions of records with low-latency SLA delivery. Skilled in Spark optimization, Delta Merge, CI/CD using Databricks Asset Bundles, Git, and Azure DevOps.Hands-on experience with:• Azure Databricks• Python• PySpark• SQL• Delta Lake• ADF• Event Hub• DAB• Git & CI/CD• Power BI• MSBI• ServiceNow • Azure DevopsOpen to opportunities in Data Engineering, Databricks, and Cloud Data Platforms.

Experience

1. Data Engineer

   Tata Consultancy Services

   Aug 2022 — Present · Bengaluru, IN

   Developed scalable batch and streaming pipelines using Azure Databricks and PySpark.Built metadata-driven ETL frameworks supporting Bronze, Silver, and Gold layers.Processed millions of records across multiple clusters with ~15-minute SLA.Implemented Delta Merge and Spark optimization techniques improving pipeline performance.Developed real-time data ingestion using Structured Streaming and Azure Event Hub.Contributed to CI/CD implementation using Databricks Asset Bundles, Git, and Azure DevOps.

Education

• PSIT Kanpur (Pranveer Singh Institute of Technology)

  Bachelor of Technology - BTech, Electronic and communication engineering

• The Model I C Kalyanpur Kanpur

  Intermediate

  2015 — 2017

• Prabhat Public H S School

  Highschool

  2013 — 2015