Rishitha Gunturu

Analyst @TD Securities

CA

Led a team of DevOps engineers in deploying and managing LLMs (Large Language Models) and embedded AI models on Red Hat OpenShift AI.•Implemented and optimized scalable vector database solutions, including Qdrant DB, to support AI/ML workloads in cloud-native environments. •Integrated and deployed Llama Guard to enforce guardrails in LLM-based applications, ensuring security, compliance, and responsible AI usage. Automated deployment, monitoring, and scaling of Llama Guard within Kubernetes/OpenShift environments optimizing performance and reliability for AI/ML workflows.•Implemented and optimized KServe and vLLM for scalable and high-performance LLM model serving in Kubernetes/OpenShift environments. Designed automated deployment, monitoring, and inference optimization strategies to enhance efficiency and reliability in AI/ML workflows.•Designed automated deployment, monitoring, and performance tuning strategies using Kubernetes and OpenShift, ensuring high availability and low-latency retrieval for LLM-based applications.•Optimized Kubernetes-based AI workloads using OpenShift Service Mesh, GPU acceleration (NVIDIA GPUs), and distributed training techniques for enhanced model performance.•Led a team of SPT offshore DevOps engineers, ensuring smooth collaboration across time zones.•Designed and implemented robust CI/CD pipelines using Azure DevOps to automate and optimize deployments to Red Hat OpenShift. Enabled seamless integration and deployment for Python, Java, and Golang-based applications, improving efficiency, scalability, and reliability.•Designed and implemented MySQL database solutions to securely store user credentials, chat history, and session data for LLM-based applications. •Integrated and customized open-source Open Web UI for seamless user interaction with LLM-based applications.•Designed and developed a custom user interface for LLM-based applications, enhancing user interaction and streamlining workflows.