Abhishek Kumar

About Abhishek Kumar

Expertise (Databricks, Delta Lake,PySpark,Azure)Experience designing Lakehouse pipelinesPerformance tuning Spark jobsCI/CD (DevOps)Real-time streaming (Kafka/Event Hub)

Experience

1. Data Engineer

   TP

   Feb 2023 — Present · Gurugram, IN

   Spearheaded migration from Azure Synapse to Databricks, achieving 45% cost reduction and 50% efficiency improvement by leveraging Python multithreading/multiprocessing.• Processed structured/unstructured data (CSV, JSON, Parquet, Excel) into Delta Lake tables using PySpark and Spark SQL, driving actionable business insights.• Led Workday API integrations and data migrations, optimizing workflows for operational efficiency.• Designed and developed scalable ETL/ELT pipelines with Azure Synapse Analytics, Data Factory, Functions, and Fabric, integrating REST APIs, SharePoint, and SQL sources.• Optimized Azure Fabric solutions to reduce Genesis process costs by 20% through advanced Python scripting.• Configured Databricks environments and ADLS Gen2, ensuring secure, scalable data storage and processing.• Created AI initiative solutions for process optimization, enhancing data workflows with predictive analytics and automation.• Delivered Azure cloud solutions for access control, networking, and DevOps, supporting seamless data operations.• Accelerated data processing by 30% through Delta Lake optimizations and tailored Spark SQL queries.

Education

• INDERAPRASTHA ENGINEERING COLLEGE, GHAZIABAD

  Bachelor of Technology - BTech, Mechanical Engineering

  2017 — 2021