Pavan Kesav

About Pavan Kesav

EDI Developer with 6+ years of experience in designing, implementing, and supporting…

Experience

1. Data Engineer

   Kroger

   Mar 2023 — Present · US

   Developed end-to-end data pipelines in Databricks (Python & PySpark) to ingest and transform JSON, CSV, and Parquet files into Azure Data Lake Storage Gen2 under Bronze, Silver, and Gold layers.• Designed and implemented SCD Type 2 framework using Delta Merge and window functions for historical data tracking.• Integrated Snowflake as the enterprise data warehouse and optimized warehouse clustering for faster Power BI queries.• Automated data ingestion and transformation through Azure Data Factory (ADF) with parameterized and metadata-driven pipelines.• Built and maintained metadata lookup tables and logging frameworks to enable automated pipeline auditing and error tracking.• Developed Databricks notebooks for file validation, data profiling, and Delta table creation with ACID transactions.• Implemented Unity Catalog for centralized data governance and fine-grained RBAC controls across business units.• Applied data masking and column-level security to restrict sensitive PII and customer data.• Collaborated with Azure DevOps team to build CI/CD pipelines for Databricks notebooks and SQL scripts.• Integrated ADF triggers and event-based pipelines for real-time ingestion from external API and FTP sources.• Conducted data quality validation using PySpark rules framework and maintained logs in control tables.• Built Power BI dashboards on Gold layer data for sales and supply-chain KPIs, reducing reporting latency by 40%.

Education

• Andhra University College of Engineering

  Bachelor's degree, Computer Science

• University of Dayton

  Master's degree, Business analytics