Sandhya Paghdar

WORK HISTORY

EDUCATION

ABOUT SANDHYA PAGHDAR

Azure Data Engineer with 7+ years of IT experience, with 5 years of hands-on Azure expertise in building scalable, end-to-end data pipelines, Lakehouse architectures, and real-time analytics solutions.* Designed batch and streaming pipelines using ADF (Gen2), Databricks Delta Live Tables, and Structured Streaming, processing 10M+ records per week from REST APIs, Snowflake, and IoT sources.* Architected Lakehouse solutions with Delta Lake (mature) and Unity Catalog (GA mid-2022), enabling RBAC, data lineage, and secure cross-workspace access.* Developed high-performance PySpark jobs and Databricks SQL notebooks, improving query speed by 40% via OPTIMIZE, Z-ORDER, and VACUUM.* Delivered Power BI dashboards with Direct Lake (Fabric, 2024), RLS, and Azure AD integration, serving 100+ business users with governed, real-time insights.* Integrated MLflow for model tracking and deployment, embedding real-time predictions into Power BI dashboards for operational decision-making.* Automated CI/CD using GitLab, Databricks Asset Bundles (DAB, 2024), and Terraform, standardizing code, infra, and deployment pipelines.* Built real-time IoT data pipelines with <60s latency, enabling predictive maintenance and operational dashboards using Power BI.* Migrated legacy ETL to ADF and Synapse (SQL DW), reducing manual effort by 40%, and implemented CDC-based incremental loads for efficient refresh cycles.* Mentored junior engineers on Databricks DevOps, Unity Catalog governance, and Lakehouse best practices, contributing to team capability building.* Piloted Microsoft Fabric (early 2024) for unified analytics, integrating with Synapse and Power BI for modern reporting solutions.* Experienced in Agile delivery using Jira and Confluence, ensuring timely, high-quality project execution with cross-functional collaboration.