Madhu Kanukula

About Madhu Kanukula

As a Staff AI/ML Applied Research Scientist, I specialize in design, develop, and deploy scalable AI/ML systems that drive business transformation and innovation. With 13+ years of experience, I bridge the gap between machine learning, software engineering, and strategic execution, ensuring AI innovations deliver measurable value.What I Do- AI Agent Frameworks – Architecting autonomous, task-driven AI agents for reasoning, planning, and workflow automation- LLM Inferencing & Optimization - Enhancing performance, cost-efficiency, and scalability of AI-powered applications, including AI Agents and multi-agent systems- RAG Solution Design & Development - Building retrieval-augmented generation (RAG) pipelines to improve accuracy, reduce hallucinations, and enable dynamic knowledge integration in enterprise AI systems- End-to-End ML Pipeline Development - Taking models from research to production with robust MLOps, monitoring, and continuous improvement- Predictive Analytics & Optimization - Leveraging data-driven insights to enhance decision-making and automate high-impact processes- I thrive on solving complex AI challenges whether scaling generative AI systems, optimizing RAG architectures, improving model efficiency, or deploying intelligent agents for business automation- My approach combines deep technical expertise with a sharp focus on business outcomes, helping organizations harness AI for innovation, efficiency, and growth.Let’s connect and explore how AI Agents, RAG, and cutting-edge ML can unlock new opportunities!

Experience

1. Lead Ai Engineer

   IBM

   Oct 2023 — Present · Bengaluru, IN

   Architected enterprise-grade AI systems by integrating LLMs, semantic search, knowledge graphs, and agentic workflows to automate complex business processes end-to-end.• Designed and delivered a semantic product catalogue search engine using transformers and vector retrieval,boosting product view rates by 43% and increasing sales conversions by 13% for large e-commerce platforms.• Built a GenAI-powered IT Helpdesk Assistant integrated with Microsoft Teams for a major banking client, reducing IT ticket load by ~17% and significantly lowering TAT through automated troubleshooting and guided responses.• Architected a DORA & legal contract compliance analysis platform, enabling automated clause extraction, risk scoring, and audit reporting reducing legal review cycles from 9 months to a few weeks.• Led development of a multimodal intelligence pipeline (video-text, audio-text, persona-based summaries, topic extraction) to generate instant insights from long-form conference content, improving analysis speed andcontent accessibility.• Developed and optimized LLM inference pipelines with dynamic batching, caching, and GPU-aware scheduling, achieving 40% reduction in latency while maintaining high throughput and accuracy.• Designed distributed training workflows using GPU clusters and parallelization strategies, accelerating model training and fine-tuning cycles for large transformer models.• Integrated GenAI and retrieval-based pipelines into enterprise workflows, enabling up to 60% operational cost savings through intelligent automation and augmentation.• Led and mentored cross-functional AI/ML teams, establishing standards for model serving, prompt engineering, LLM fine-tuning, and scalable ML-Ops across complex production environments.

Education

• Jawaharlal Nehru Technological University

  Bachelor of Technology - BTech, Information Technology

  2007 — 2011

• Birla Institute of Technology and Science, Pilani

  Master of Technology - MTech, Data Science & Engineering