Devanshu Gupta

About Devanshu Gupta

I’m a Software Engineer (Machine Learning) with experience building and deploying large scale, AI-driven systems powering real-time personalization at Amazon Prime Video. My expertise lies in designing low-latency, high concurrency backend systems and production ML pipelines, with a strong focus on embedding-based retrieval, ranking systems, and LLM powered applications. I specialize in distributed systems, microservices architecture, and AWS based infrastructure, with hands-on experience in Python, Java, Kotlin and scalable data processing frameworks.Throughout my experience at Amazon and previously at Tata Consultancy Services, I’ve contributed to high-impact systems serving 100K+ users, optimizing performance and scalability in production environments. I’ve led initiatives such as developing LLM-driven voice systems with sub-300ms latency, improving system performance by 3x through parallelization, and building intelligent caching layers that enhanced user experience by reducing redundancy. My work bridges the gap between research and production ranging from fine tuning LLMs and designing evaluation frameworks to shipping robust, experiment-driven systems that deliver measurable business impact.I bring a strong foundation in system design, performance optimization, and applied machine learning, with a focus on building reliable, scalable AI solutions that directly influence product metrics. With a Master’s in Computer Science from Arizona State University, my interests lie at the intersection of backend engineering, large-scale ML systems, and intelligent applications.

Experience

1. Software Engineer (Machine Learning)

   Amazon

   May 2025 — Present · NY, US

   Working on content delivery, personalization, and next-gen viewing experiences on Prime Video. Focussed on Ownership, Delivering results and Collaboration- Led development of low-latency LLM-powered backend service (Python, AWS Lambda, API Gateway, DynamoDB) for Prime Video–Alexa voice experiences, handling 100K+ daily queries under 300ms SLA, enabling real-time conversational recommendations- Fine-tuned 7B parameter LLM (LoRA, PyTorch, Hugging Face) on 100K+ domain-specific conversational samples, improving offline model accuracy by 18% and enhancing contextual relevance in production responses- Built a FAISS-powered retrieval and ranking system using KNN-based similarity search and NDCG-optimized reranking, serving 1M+ users within 500ms latency and increasing CTR by 16%- Architected an title-aware caching layer with ElastiCache (Memcache) and DynamoDB to eliminate duplicate recommendations, reducing redundancy by 35% and enhancing large-scale content discovery performance- Led end to end feature engineering and model iteration lifecycle including offline evaluation and (A/B) testing improving NDCG by 12% across production experiments and deployments at scale effectively- Designed scalable LLM evaluation framework (Athena, Python, LLM-as-judge, automated validation pipelines) processing 50K+ conversations/day, increasing defect detection by 60% and improving model quality monitoring- Collaborated cross-functionally with Applied Scientists, Product Managers, and Backend teams, translating research models into production-grade ML systems with measurable business impact.

Education

• Arizona State University

  Master's degree, Computer Science

• University Institute of Technology, RGPV

  Bachelor of Technology, Computer Science

  2017 — 2021