Manish Dash

About Manish Dash

Manish is a Senior Data Engineer with over 6 yoe in building scalable batch and streaming pipelines on Azure and Databricks. Skilled in designing config-driven, reusable ETL/ELT frameworks using ADF, Databricks, Delta Lake and ADLS, integrating data from various sources into Lakehouse architecture.Proficient in Python, PySpark, Spark SQL, PL/SQL, and Azure Synapse, with experience in Medallion Architecture, Data Modeling, Data Quality, and Data Governance. Skilled in developing automated test scripts, tracking code quality and coverage, SCD implementations and complex data transformations. Enables automation and workflows using Azure Functions, Logic Apps, GitHub, and Purview.Certifications- Databricks: Data Engineer Associate, Lakehouse Fundamentals- Microsoft: DP-203 (Azure Data Engineer Associate), PL-300 (Power BI Data Analyst Associate), DP-900, AZ-900

Experience

1. Senior Associate 1

   Publicis Sapient

   Jan 2024 — Present · Bengaluru, IN

   Client: Shell; Energy and Commodities - Designing and implementing end-to-end batch and streaming data pipelines using Azure Data Factory and Databricks, ingesting data from APIs, Kafka, RDBMS, and file systems into a unified Delta Lakehouse- Building reusable, config-driven frameworks using Python, PySpark, and SQL to streamline data workflows, with custom transformations, SCD implementations, and layered storage (raw, enriched, curated) on ADLS using Delta Lake- Implementing robust data quality with Great Expectations library, code quality and coverage using SonarQube, Pytest-based unit tests, and custom logging frameworks; exposing curated data via external Delta tables for downstream analytics.

Education

• Young Blood Public School

  Secondary

  2012 — 2013

• Jupiter +2 Science College

  Senior Secondary, Science

  2013 — 2015

• Silicon Institute of Technology (SIT), Bhubaneswar

  Bachelor of Technology - BTech, Electrical, Electronics and Communications Engineering

  2015 — 2019