Yanshu Z.

About Yanshu Z.

Experienced Data ScientistCurrent: Experimentation Platform (Research & Analysis) Research Analysis: CUPED method efficacy, Power Analysis of Gen AI cloud experiments, Alert System Methodology for Metric Movement and SRM DetectionPrevious: Machine Learning in Recommender SystemDecision tree, gradient boosting tree, Catboost, XGBoost, logistic regression, generalized linear model;Collaborative Filtering, FFM, DIN.• Trained supervised machine learning models to predict customer churn probability. •Defined metrics and designed experiments to estimate causal effects• Devised methodology for calculating the website traffic that is attributable to product advertising, implemented a Multi-Arm-Bandit algorithm to optimize ads performance.Industry Experience: Tech, E-commerce, Financial Service

Experience

1. Data & Applied Scientist

   Microsoft

   Jun 2022 — Present

   Data Scientist at Microsoft, Experimentation Testing Enablement & Education-Taught advanced experimentation and metric design classes for cross-functional teams (PMs, analysts, engineers), covering CUPED, delta method, power analysis, and best practices in large-scale A/B testing-Designed GenAI agent evaluation metrics based on client use cases, enabling more tailored and actionable insights from experimentation results-Investigated and resolved Sample Ratio Mismatch (SRM) issues across multiple product scorecards, improving the reliability and trustworthiness of experimentation outcomes.Experimentation Research & Methodology-Research on CUPED variance reduction efficacy, quantifying impact on experiment sensitivity and statistical power in terms of both methodological performance and real-world implementation practices, including diagnosing low or negative efficacy scenarios and establishing best practices for experiment design-Power analysis for GenAI experiments, evaluating multiple LLM metrics across different randomization units and optimized designs to improve experiment power and efficiency-Experiment alert system methodology for automated metric movement detection and Sample Ratio Mismatch (SRM) monitoring.

Education

• SMU Cox School of Business

  Master’s Degree, Business Analytics

  2017 — 2018

• Nanjing Audit University

  Bachelor’s Degree, Accounting

  2011 — 2015

• SMU Cox School of Business

  Bachelor's degree, Accounting

  2015