Sanchay Yadav

About Sanchay Yadav

Data Engineer at Amazon with nearly 4 years of experience building large-scale data infrastructure. Currently processing 650M+ records per hour and managing 2.6PB Redshift infrastructure serving employees.Recently reduced retail catalog defects by 20% through an LLM-powered detection pipeline processing 200M daily records. Working on a retrospective experimentation platform enabling causal impact measurement without pre-defined control groups.Strong background in AWS (Glue, Redshift, EMR, S3), PySpark, Python, dimensional modeling, and data quality frameworks. Experience collaborating with cross-functional teams including science, product, and international vendor teams.Skills: Python | PySpark | SQL | AWS | Apache Spark | Data Modeling | ETL | Data Quality | LLM Integration | Statistical Analysis

Experience

1. Data Engineer 2

   Amazon

   Apr 2025 — Present · Bengaluru, IN

   Collaborating with Catalog teams on datasets processing 650M+ records per hour, building the first comprehensive dataset tracking catalog updates with cross-team applicability.Building retrospective A/B testing platform enabling causal impact measurement without pre-defined control groups through historical data matching and statistical significance testing.Implementing SCD2 dimensional modeling for Retail Business Services unified ASIN data lake using AWS Glue and Apache Iceberg.Managing 2.6PB Redshift data warehouse infrastructure serving employees across the organization.Optimizing production Spark jobs processing complex Ion data formats with heavy UDF usage.Leading adoption of CI/CD tool for ETL pipelines across the organization.Mentoring team members and interns on data engineering best practices, code reviews, and technical design.

Education

• SRM IST Chennai

  Bachelor of Technology - BTech, Artificial Intelligence and Machine Learning

  2018 — 2022

Skills

• Analytics
• Public Relations
• Business Strategy
• Marketing Communications
• Product Management
• Business Planning
• Digital Media
• Marketing
• Customer Insight
• Marketing Management
• Market Analysis
• New Product Roll Out
• Brand Health Tracking
• Team Management
• Market Planning
• Sales Operations
• Crm
• Strategy
• Business Development
• Channel Sales
• Sales
• Product Marketing
• Social Media Marketing
• Brand Management
• Marketing Strategy
• Integrated Marketing
• Vendor Management
• Leadership
• Management
• Trade Marketing
• Media Planning
• Marketing Research
• Product Development
• Market Research
• Advertising
• Competitive Analysis