Victor A.

About Victor A.

Engineering manager for Google Search’s deep learning inference team. Responsible for optimizing and productionizing some of Google’s largest and most complex production ranking, recommender, and large language models.I have worked on a range of projects- Leading a team to build and deploy the production inference system for Featured Snippets (https://developers.google.com/search/docs/appearance/featured-snippets). This high-throughput, low-latency system was one of the most reliable production inference systems of its scale in all of Google. This system was used for the first deployment of a BERT-style model, on a new ML accelerator, in the world. Featured snippets model productionization times decreased from weeks to months to minutes as a result of this effort- For a number of years, I helped to build the annual business case for Search’s inference HW purchases for Google’s CFO. I helped guide TPU HW co-design for Search inference and evaluated new generations of TPUs (and other ASICs). I also evaluate most HW requests for Search inference launches- I lead a team which has developed infrastructural and architectural optimizations for large embedding, BERT-style transformers, and LLMs. Although we are organizationally situated in Search, our optimizations and tools are used by YouTube, Ads, Translate, and Google Brain. My team works closely with HW resource managers, SREs, and other infrastructure teams in Search. We also build tools to make model optimization and productionization more self-service. I also mentor and help to develop Search’s growing ML infrastructure organization.

Experience

1. Member of Technical Staff

   Microsoft AI

   May 2025 — Present

   Model Inference at Scale!

Education

• Advanced Technologies Academy

  Computer Science

• Stanford University

  Bachelor of Science (BSc), Mathematics

  2006 — 2010