Hari Hud

About Hari Hud

Experienced Senior Software Engineer with 9+ years in developing robust software solutions. Expertise in MLOps, backend development, CI/CD, cloud infrastructure, DevOps, and automation. Proven track record in designing and implementing scalable systems and frameworks, enhancing efficiency, and ensuring security. Skilled in driving continuous improvement and delivering high-quality software projects.For the past 6+ years, I have been with Nvidia, contributing to various projects within the platform development team. Currently, I am working on building a unified MLOps platform for model development, data preparation, model training, and evaluations. My main focus is on creating an evaluation tool and evaluator microservice for assessing models like NeMo, Megatron, and LLaMA.One of my key contributions was developing ClusterForge, a modern platform built on NVIDIA\'s Kaizen Framework, Temporal, Go, Java, and Kubernetes. I was responsible for developing REST APIs, a workflow engine, and Ansible playbooks to manage BCP clusters on both NVIDIA\'s infrastructure and external data centers.I also worked on a DevSecOps project, where I built a security service to help developers identify and resolve security vulnerabilities before release. This included container scanning, open-source dependency scanning, static code analysis, secret detection, and infrastructure-as-code security checks.Another significant project I contributed to was AKUC, a non-disruptive Kubernetes upgrade controller designed to upgrade Kubernetes clusters without interrupting workloads.Additionally, I developed a managed CI/CD platform to ensure seamless integration, deployment, security, and testing of applications throughout their lifecycle. I have experience in developing command-line interfaces (CLIs) using various programming languages and implementing infrastructure as code.Before joining Nvidia, I spent 3 years at GSLab Pune as a Cloud Orchestration Engineer, where I focused on developing workflows for VM lifecycle management using Python, Django, OpenStack, and VMware vRA/vRO.Overall, I am passionate about software development and have a strong background in platform development, cloud orchestration, CI/CD, infrastructure as code, MLOps, and DevOps.

Experience

1. Senior System Software Engineer, Llm Mlops & Speech Ai

   NVIDIA

   Jun 2023 — Present · Pune, IN

   Built a unified LLM & RAG evaluation framework at NVIDIA, streamlining fragmented tools into a single extensible platform. Supports academic/custom benchmarks, LLM-as-a-judge, safety/security evals, and diverse model backends (NeMo, HuggingFace, Llama, OpenAI, DeepSeek, etc.).• Worked on data acquisition from external vendors for Physical AI and robotics training, and on web scraping pipelines for LLM training datasets.• Led the design and development of NVIDIA Eval Factory — a unified platform to build, certify, and operationalize standardized LLM evaluation benchmarks across the company. • Built an automated ASR evaluation pipeline on Kratos (Kubeflow), reducing evaluation time from 2–3 weeks to just 1–2 days.• Drove a 90%+ reduction in dataset processing time by parallelizing the TTS data pipeline (vertical and horizontal scaling) and enabling archive-based uploads. This optimization allowed 28 English datasets to be processed in just 2–3 days instead of months, a 10× improvement. This pipeline was later used to process 100+ datasets across multiple languages.• Designed and developed an Evaluator Microservice using Python FastAPI and PostgreSQL to externalize API-based LLM evaluation capabilities for enterprise users.• Created an OpenAI API-compatible Inference API Server for NVIDIA\'s TensorRT-LLM Engine.• Improved evaluation observability and reliability by designing and implementing a progress tracking solution within the Evaluator Microservice.• Designed and developed CI/CD pipelines and testing frameworks for MLOps projects.• Integrated the LLM eval-tool into customers\' CI/CD pipelines to automate model evaluations.• Authored the NTech paper titled \"NeMo Eval Tool: A Unified Evaluation Framework for LLMs and Retrievers/RAGs,\" which was selected as a Software Paper for NVIDIA\'s NTech India presentation• Filed and received acceptance for U.S. patent for Evaluator Microservice

Education

• ZPPS Gadiwat, Chhatrapati Sambhajinagar (Aurangabad, MH)

  SSC

• PES POLYTECHNIC, Chhatrapati Sambhajinagar (Aurangabad, MH)

  Diploma, Computer Science

• Shri Guru Gobind Singhji Institute of Engineering and Technology, Vishnupuri, Nanded

  B.Tech, Information Technology