Chinmay Bapat

About Chinmay Bapat

Chinmay is an Engineering Manager in the Amazon SageMaker AI Inference team at AWS, where he leads engineering efforts focused on building scalable infrastructure for generative AI inference. His work enables customers to deploy and serve large language models and other AI models efficiently at scale.

Experience

1. Software Development Manager

   Amazon Web Services (AWS)

   Oct 2023 — Present

   Gen AI Infrastructure Leadership: Direct the delivery of critical Gen AI capabilities, including LoRA support and Voice Agents. Collaborate with LLM serving optimization teams to unblock new revenue streams by enabling customers to deploy thousands of custom models in Bedrock and SageMaker AI with minimal TTFT while only paying for compute actually used. Architected robust detection and recovery mechanisms for critical workloads across the latest NVIDIA GPUs as well as Amazon Trainium / Inferentia accelerators. Engineered the platform to ensure high reliability for models served using vLLM, DJL, and Triton.Cost & Efficiency Strategy:Spearheaded cost-optimization initiatives, architecting multi-model serving and smart routing strategies that maximize KV cache reuse and hardware utilization (NVIDIA/Trainium) to drive down customer TCO.Team Building & Talent Development: Manage a high-performing organization of 15 engineers. Directly hired 6 key members and mentored 4 engineers to promotion (Junior to Mid/Senior levels), fostering a culture of continuous growth and high talent density.Strategic Customer Impact: Partner with Product, Science, and Technical Account Managers to define the product roadmap. Directly manage relationships with strategic enterprise customers representing $10M+ in annual revenue, resolving critical escalations and driving platform enhancements that prevented churn and secured long-term adoption.Operational Excellence & Culture: Transformed the team’s operational culture by instituting rigorous COE (Correction of Error) processes and leading weekly operations reviews. Established mechanisms to identify repeated failures and drive long-term architectural fixes, maintaining 99.99% availability for a platform serving 3 million+ requests/second.

Education

• Indian Institute of Technology, Madras

  Master of Technology (M.Tech.), Computer Science and Engineering

  2014 — 2015

• Indian Institute of Technology, Madras

  Bachelor of Technology (B.Tech.), Computer Science and Engineering

  2010 — 2015

Skills

• Teamwork
• C
• Programming
• C++
• Java
• English
• Python
• Core Java
• Windows
• Public Speaking