Shubham Verma

Data Eng Mgmt and Governance Sr Analyst @Accenture in India

Gurugram, IN

As a Senior Data Engineer at Accenture Solutions Pvt. Ltd, I architect and deliver cloud-based data engineering and analytics solutions for Indian Oil Corporation Ltd.(IOCL), ensuring scalability, automation, and performance optimization across high-volume enterprise systems.I develop and maintain end-to-end ETL pipelines using AWS Glue, Lambda, S3, EMR, Redshift, Athena, MSK, SQS, and API Gateway, processing 10M+ records daily. I design PySpark-based batch and streaming workflows for incremental and real-time data ingestion, transformation, and analytics.Implemented a data quality and validation framework leveraging AWS Glue Catalog and Python automation for schema checks, threshold validation, and exception handling—reducing pipeline failures by 40% and ensuring 100% data completeness. Integrated AWS CloudWatch with Slack and SNS alerts for proactive error monitoring and system health tracking.I manage Apache Airflow DAGs for workflow orchestration, scheduling 50+ automated data jobs, and deploy data pipelines through CI/CD using Git, Jenkins, and GitHub Actions, improving deployment efficiency and reducing manual errors.Experienced in Redshift Spectrum, Athena query optimization, and partition pruning, achieving up to 35% faster query performance. I also ensure robust data governance, access control, and encryption via AWS IAM, KMS, and CloudTrail.Collaborating with cross-functional teams, I translate business requirements into scalable data solutions, enabling real-time analytics and KPI dashboards. My focus on automation, reliability, and cost optimization continues to enhance data-driven decision-making for large-scale enterprise operations.

Indira Gandhi National Open University

Master of Computer Applications - MCA

Maharshi Dayanand University Rohtak

BCA