Chandra Kumar Reddy

Lead Llm Engineer @Verizon Connect

Architected modular RAG pipelines using LangChain, LlamaIndex, and FAISS, enabling context-enriched generation from CRM records, historical call notes, and other structured/unstructured sources. Engineered advanced prompt chaining logic, integrating OpenAI GPT-4 and Anthropic Claude with contextual memory, system role design, and fallback routing for reliable multi-turn conversations. Fine-tuned LLaMA-2 models with LoRA and PEFT, optimized for latency and GPU efficiency, targeting use cases such as intent recognition, summarization, and classification. Containerized model inference and retrieval services in Docker and deployed via Kubernetes, exposing secure REST APIs with dynamic scaling and authenticated gateways. Implemented LangSmith-based observability to trace prompt-response flows, capture intermediate reasoning, and provide runtime debugging and auditability. Built cloud-native infrastructure with Terraform and GitOps (ArgoCD) for reproducible, compliant, and secure multi-environment deployments. Developed PII redaction pipelines leveraging regex, moderation APIs, and classification-based filtering for privacy-preserving input/output handling. Designed evaluation pipelines using BLEU, ROUGE, and perplexity to benchmark prompt performance and reduce hallucinations. Created interactive model testing dashboards in Streamlit and Gradio, enabling product teams to run experiments and capture human preference feedback. Integrated Azure OpenAI with governance controls, content filters, and rate limit policies through custom middleware. Led prompt engineering reviews, architecture retrospectives, and mentored junior engineers on LLM lifecycle management and secure GenAI deployment practices.