Pankaj Jain

About Pankaj Jain

I’m a Senior Data Engineering Leader with expertise in Big Data, Cloud Platforms (AWS, GCP), Real-Time Data Pipelines, and a strong foundation in Database Development and Administration across SQL Server, MySQL, and Oracle.My passion lies in building scalable, cloud-native data platforms that empower businesses with real-time insights, foster data democratization, and are ready for AI/ML integration.I have spent years building and optimizing complex transactional systems, leading cloud-native architectures, and delivering real-time, data-driven solutions at scale.Core Strengths:Designing CDC-based real-time pipelines (MySQL → Kafka → BigQuery) to support real-time analytics.Architecting and optimizing cloud environments on AWS and GCP, with a focus on cost-efficiency and performance.Developing self-service BI platforms using Power BI, Looker Studio, and SSRS for analytics and reporting teams.Building Data Lakehouses using AWS Hudi, Athena, and PySpark, integrated with Redshift and S3.Strong experience in Database Development for high-transaction systems in SQL Server, MySQL, and Oracle.Skilled in DBA tasks such as performance tuning, high availability (HA/DR), replication, and query optimization.Working knowledge of NoSQL systems like MongoDB for flexible, scalable data storage.While I’ve successfully led teams and delivered large-scale data initiatives, I remain deeply committed to hands-on development, technical problem-solving, and building efficient, scalable data systems.I am particularly focused on real-time analytics, cloud-native architectures, and enabling data democratization across organizations. I enjoy collaborating with teams to deliver high-impact solutions that support data-driven decision-making.

Experience

1. Senior Manager Data & Platform Engineer

   Ecom Express Limited

   Jan 2017 — Present · Gurugram, IN

   Led Ecom’s data journey over 8 years – database architecture → cloud-native data engineering.Built real-time pipelines: MySQL → Debezium → Kafka (MSK) → BigQuery.Migrated legacy systems to BigQuery, improving performance by 60%.Architected Data Lakehouse (AWS Hudi + Athena + Redshift).Engineered 100+ real-time Hudi tables using PySpark on EMR.Automated ELT pipelines with Airflow DAGs, implemented CI/CD (Bitbucket, Jenkins, Docker).Reduced AWS cloud costs by $40K/month.Supported BI tools: Power BI, Looker Studio, Redshift Spectrum.Enabled data democratization for analytics/reporting teams.Led monitoring systems (Grafana, Prometheus, AWS SNS).Designed HA/DR for MySQL, SQL Server, MongoDB.

Skills

• Sql
• Asp.net
• Pl/Sql
• Web Services
• Vb.net
• Ajax
• Windows Communication Foundation (Wcf)
• Javascript
• .net Framework
• C#
• .net
• Microsoft Sql Server
• Wcf
• Oracle
• Requirements Analysis
• Xml
• Design Patterns
• Jquery
• C++