Jianbao Tao

About Jianbao Tao

Experience: 12+ years of backend engineering, ranging from offline data pipelines, to search verticals, and most recently to ads model serving infra (an area of RecSys inference) at Meta.Interest: AI model/software/hardware co-design for training/inference of LLM and RecSys modelsSkills:CUDA kernel programming, PyTorchC++/Python

Experience

1. Staff Software Engineer

   Meta

   Jan 2022 — Present · Menlo Park, CA, US

   Jun 2023 - present, Ads ML Serving PlatformScope: Retrieval stage GPU-powered ads models serving.I have a track record of driving major advancements and solving complex engineering challenges in ads model serving infra. Examples:* Design and implemented AED (ad embedding on device). Notable complexities: GPU-based hash table to allow O(10-100k) concurrent lookup, HBM/DRAM hybrid storage for ad embeddings to allow O(100M) ads in memory, vectorized memcpy that is 16x faster than CUDA native memcpy. Impact: alleviated memory bottleneck and CPU bottleneck for major retrieval stage ads models and hence unblocked their launches. AED has become a foundational building block for ads retrieval stage GPU model serving. Without it, several ongoing high-profile projects, such as TorchRetrieval, Neutron Star on AMD, Genre 3.0, and Stage Consolidation, would be blocked.* Major contributor for Neutron Star launch (first GPU-based model serving system in ads serving infra). I served as predictor service TL and solved a few hard problems (eg, invented a novel solution to derisk Manifold IO throttling for NS model serving; mitigated fallback SLA blocker)* The AMD model serving launch in ads retrieval stage model serving. Served as TL and personally solved arguably the hardest problem, which was the CPU bottleneck on AMD.Jan 2022 - May 2023, Metrics Computation Infra

Education

• University of Colorado Boulder

  Doctor of Philosophy (Ph.D.), Astrophysical and Planetary Sciences

  2007 — 2012

• University of Science and Technology of China

  Bachelor of Science (BS), Earth and Space Sciences

  2000 — 2004

Skills

• Matlab
• Grails
• Mpi
• Gradle
• Apache Spark
• Data Mining
• Plasma Physics
• Cython
• Algorithms
• Data Structures
• Css
• Groovy
• Jquery
• Fortran
• Physics
• Computer Science
• Apache Pig
• Simulations
• Javascript
• C++
• Idl
• Machine Learning
• Scala
• Python
• Latex
• Numerical Analysis
• Big Data
• Intellij Idea
• Html
• Data Analysis
• Online Advertising
• C
• Programming
• Hadoop
• Space Physics
• Hive
• Oozie
• Information Retrieval
• Signal Processing
• Pattern Recognition