Saurabh Yadav

About Saurabh Yadav

Backend engineer focused on production LLM inference systems. Experience fine-tuning and deploying instruction models (LoRA, Hugging Face) and building low-latency backend services using Go and Python. Skilled in prompt engineering, inference validation, and system integrations that reduce latency and improve model accuracy.

Experience

1. Backend Developer (Watsonx Ai and Watson Machine Learning (Wml))

   IBM

   May 2024 — Present · Bengaluru, IN

   Designed and developed the MCP Deployment Server, a core backend component enabling model deployment, management, and inference across IBM Cloud using the Model Context Protocol (MCP) Fine-tuned the IBM/granite-3.3B Instruct model by generating synthetic data from Jane’s API, improving model response quality on targeted tasks. Built a custom LoRA adapter (a lightweight tuning method) using gbcli to make the model learn faster and better for specific tasks. Used smart prompt engineering and data enhancements to help the model give better, more accurate responses. Continuously improved the model’s accuracy to 98% by refining the training data and adjusting fine-tuning process. Performed inference testing and validation, ensuring model robustness and alignment with business requirements Implemented Envoy Proxy to maintain continuous synchronization with Redis for updated metadata, enabling efficient routing of inference requests to the runtime manager for predictions. Implemented the GoDog FVT Framework, streamlining the deployment, testing, and prediction processes for Watsonx and WML models. Migrated keys from etcd to Redis, reducing record retrieval time by 100ms and enhancing storage performance by leveraging Redis’ support for multiple data types.

Education

• Children Senior Secondary School, Belaisa Azamgarh

  Intermediate

• Indian Institute of Information Technology, Pune

  Bachelor of Technology - BTech, Electrical, Electronics and Communications Engineering

  2020 — 2024