Dan Sun

About Dan Sun

Lead Bloomberg cloud native AI Inference / Traffic Team to architect and build model hosting and AI gateway platform that can serve predictive and generative models at scale on GPUs and handle billions of requests a day. As the co-founder and maintainer of KServe, Envoy AI Gateway OSS project, I have been contributed to cloud native open source infrastructure that powers modern cloud platforms since 2017.I am also a regular speaker on platform and cloud native infrastructure topics. Always interested in collaborating and connecting with engineers working on solving model inference and routing problems.

Experience

1. Software Engineering Team Lead

   Bloomberg

   Jun 2010 — Present · New York, NY, US

   2021 - AI Inference Team Lead- Lead AI Traffic team which is responsible for managing inference traffic to cloud frontier models and on prem self hosted models- Lead AI Inference team which is responsible for the internal ML Inference platform serving hundreds of production models with low latency built on top of KServe- Co-Founder and lead for Envoy AI Gateway, which manages Unified Access to Generative AI Services built on Envoy Gateway- Co-Founder and lead for KServe, a standardized Distributed Generative and Predictive AI Inference Platform for Scalable, Multi-Framework Deployment on Kubernetes.

Education

• Washington University in St. Louis

  MS, Computer Science

  2006 — 2008

Skills

• Java
• C++
• Data Mining
• Software Project Management
• Linux
• C
• Object Oriented Design
• Big Data
• Software Engineering
• Algorithms
• Machine Learning
• Agile Methodologies
• Stl
• C
• Perl
• Python
• Scrum
• Trading Systems
• Distributed Systems
• Git
• C#
• Oop
• Multithreading
• Unix
• Artificial Intelligence