Muthu Palaniappan A.
Software Development Engineer Ii @Amazon
Signup · Get unlimited contacts
WORK HISTORY
Software Development Engineer Ii @Amazon
Palo Alto, CA, US
Optimized LLM inference for Amazon’s shopping assistant Rufus, increasing system throughput by 30%. • Maintained a high-availability inference stack handling ~1500 TPS, ensuring scalability and reliability. • Led a team to develop benchmarking tools, uncovering optimization opportunities for model deployments.
EDUCATION
University of Southern California
Master of Science - MS, Computer Science
College of Engineering, Guindy
Bachelor of Technology - BTech, Information Technology
ABOUT MUTHU PALANIAPPAN A.
As an SDE at Amazon, I focus on optimizing Large Language Model (LLM) inference systems to ensure high throughput and low latency for Amazon\'s shopping assistant, Rufus. By employing advanced techniques such as continuous batching and streaming responses, our team has enhanced system scalability and reliability, supporting production-level traffic of approximately 1,500 TPS. Additionally, I have led efforts in developing benchmarking and profiling tools, identifying critical opportunities for performance gains in model and checkpoint deployments. I hold an MS in Computer Science from the University of Southern California and a BTech in Information Technology from the College of Engineering, Guindy. My expertise includes working with vLLM, AWS Neuron, and Approximate Nearest Neighbor (ANN) search. Passionate about solving complex technical challenges, I strive to contribute to advancements in LLM evaluation and deployment strategies to improve user experiences.
This profile is compiled from publicly available professional sources. Unifers is not affiliated with or endorsed by LinkedIn. Request removal of this profile.