Karthik Malaisamy

About Karthik Malaisamy

I’m a product leader working at the intersection of LLM inference infrastructure, routing systems, and platform economics. At Microsoft, I help define how M365 Copilot allocates capacity, routes workloads, and scales enterprise AI reliably and efficiently. My work has included shipping geo-local inference processing in India, the UK, Singapore, and the UAE to support sovereignty-sensitive enterprise adoption; building request-aware routing based on workload shape, user profile, latency tolerance, and business priority; and improving the economics of large-scale inference through batching, load balancing, and smarter capacity orchestration.Over the last few years, I’ve helped drive outcomes like $100M+ in annualized GPU savings, 40%+ lower throttling for high-priority workloads, and 22% higher utilization through LLM-engine load balancing. I’ve also worked on the monetization side of AI infrastructure, including usage-based billing and enterprise controls that helped scale Pay-As-You-Go adoption 10x.What makes this work especially interesting to me is that inference is not just a systems problem. It is where infrastructure, product strategy, enterprise requirements, and business model all meet. I enjoy building products that align service performance with real customer needs, whether that means latency-sensitive routing, sovereign processing, or making the unit economics of AI work at scale.Before product, I was a software engineer, and that still shapes how I work. I’m comfortable going deep on architecture, partnering closely with infra and platform teams, and turning complex technical tradeoffs into clear product decisions.I’m especially excited by teams building the next generation of inference platforms, model serving systems, and enterprise AI infrastructure.

Experience

1. Senior Product Manager

   Microsoft

   May 2022 — Present · Redmond, WA, US

   Lead product strategy for AI inference infrastructure, routing, and platform economics for M365 Copilot.• Built request-aware routing based on workload shape, user profile, latency tolerance, and business priority, aligning service performance with business goals and differentiated user experiences.• Shipped geo-local inference processing across multiple international regions, enabling sovereignty-sensitive enterprise deployments and improving Microsoft\'s position in competitive AI deals.• Launched LLM-engine load balancing capabilities to improve throughput and utilization across workloads with different latency targets.• Drove major inference-efficiency gains through token routing, dynamic batching, and capacity orchestration, reducing cost and increasing effective GPU capacity.• Built real-time utilization and metering capabilities to support large-scale model launches while protecting latency for paid and priority workloads.• Developed enterprise billing controls and usage-based governance capabilities that expanded monetization and helped customers adopt AI workloads with greater transparency.

Education

• P.S.G Sarvajana Higher Secondary School

  Higher secondary

• WU (Vienna University of Economics and Business)

  Winter semester

• College of Engineering, Guindy

  Bachelor of Engineering (BEng)

• Indian Institute of Management Ahmedabad

  Master of Business Administration - MBA

Skills

• Microsoft Office
• Public Speaking
• Agile Methodologies
• Web Development
• C++
• Business Analysis
• Javascript
• C
• Data Analysis
• Business Planning
• Leadership
• Html
• Marketing Strategy
• Fedora
• Ubuntu
• Java
• Python
• Sql
• Linux
• Market Research
• Microsoft Excel
• Teamwork
• Mysql