Renyi Qu

About Renyi Qu

I work on AI.

Experience

1. Research Engineer

   Vectara

   Jun 2024 — Present · Palo Alto, CA, US

   Reduced hallucination rate of proprietary Large Language Model (LLM) from 8.2% to 0.9%, achieving Top 4 on Hallucination Leaderboard by fine-tuning with Direct Preference Optimization (DPO) and inference with Best-of-N sampling • Trained proprietary LLM v2 for multilingual Retrieval Augmented Generation (RAG), outperforming v1 (+80% ROUGE-L;+21% TREC-RAG,+65% in Arabic). Applied Low-Rank Adaptation (LoRA), Distillation, ReAct prompting, Distributed Data Parallel (DDP) • Enabled multimodal table and text understanding capabilities in proprietary LLM to support RAG on unstructured financial documents, achieving +16% fuzzy matching accuracy via supervised fine-tuning (SFT) on diverse table serialization strategies • Engineered and debugged unsupervised evaluation metrics in the Open RAG Evaluation framework for RAG systems, achieving 90% alignment with label-based metrics, enabling scalable, label-free RAG evaluation • Developed a multimodal RAG benchmark with PDF-based datasets to enable robust RAG system evaluation, leveraging an automated, multi-stage LLM ensemble pipeline for query generation and post-processing • Built an automated hyperparameter optimization pipeline for proprietary RAG system using Ray, achieving +43% mean UMBRELA retrieval score through comparative evaluation of Grid Search, Bayesian Optimization, and Evolution Strategies • Built a new language detection module in Java using FastText and HunSpell, boosting accuracy from 62% to 98%

Education

• Rosslyn Academy

  High School Diploma

• Tampere University of Technology 1965-2018

  Bachelor's degree, Information Technology

• Keio University

  Bachelor's degree, Statistics

• University of Pennsylvania

  Master's degree, Data Science