Vaibhav Gupta

About Vaibhav Gupta

Certified Azure Data Engineer with 5+ years of experience in building scalable and cost-efficient data solutions. Proven expertise in Azure Data Factory (ADF), Databricks, Spark (PySpark), Delta Lake, and Data Lake Storage (ADLS). Adept at designing metadata-driven, modular data pipelines, and delivering impactful solutions in fast-paced environments. Passionate about automation, performance optimization, and clean, reliable data.Cloud Platforms: Azure (ADF, ADB, ADLS, Logic Apps, Synapse, Event Grid, Service Bus, DevOps)Big Data & ETL Tools: Databricks, Spark (PySpark, Scala), Hadoop, Hive, Kafka, Delta Lake, SqoopProgramming & Scripting: Python, Scala, SQLDatabases: SQL Server, MySQL, Snowflake, Azure SQL DBData Warehousing: Dimensional Modelling, SCD Type 2, Data LakehouseDevOps & CI/CD: Git, Azure DevOps PipelinesData Governance: Unity Catalog, RBAC, Metadata ManagementCertifications: Azure Data Engineer Associate, Azure Fundamentals, Azure Data FundamentalsPROJECT HIGHLIGHTS• Streaming Framework: Built near real-time streaming ingestion using Autoloader for retail client—reduced latency by 70%.• Metadata-Driven Pipelines: Designed reusable metadata tables to dynamically control ADF pipeline behaviour, simplifying change management.• Cloud Cost Optimization: Re-architected pipeline architecture and job scheduling strategy, reducing monthly cloud cost by

Experience

1. Data Engineer

   Tiger Analytics

   Jul 2024 — Present

Education

• Meerut public school

  Intermediate , Pcm

• KRISHNA INSTITUTE OF ENGINEERING AND TECHNOLOGY, GHAZIABAD

  Bachelor of Technology, electrical and electronics engineering

  2015 — 2019