Devang S.
Senior Software Engineer Ii - Agentic Ai & Data Pipeline (Kafka Spark) @LexisNexis
Signup · Get unlimited contacts
WORK HISTORY
Senior Software Engineer Ii - Agentic Ai & Data Pipeline (Kafka Spark) @LexisNexis
Ottawa, ON, CA
Built Agentic AI and Retrieval-Augmented Generation (RAG) pipelines with Learning-to-Rank (LTR) to automate legal research, replacing manual paralegal workflowswith AI agents that surface the most relevant cases and improve response reli-ability.• Designed LLM feedback and quality-control loops using prompt engineering andautomated hallucination detection, cross-verifying cited case intent against sourcedocuments and improving accuracy using RAGAS metrics and practice-area hi-erarchies.• Integrated a memory-based (mem0-style) architecture into existing RAG systemsto store and retrieve user profiles and feedback, enabling personalized responsesand continuous improvement in LLM output quality.• Developed modular MCP tools to integrate data from multiple internal and exter-nal sources into an internal LLM console, enabling teams to validate AI responsesbefore production integration.• Created an Agentic AI human-in-the-loop relevance testing framework usingJenkins and an internal LLM Console, supporting structured evaluation of AIresponses before release.• Optimized AWS EKS data and AI pipelines, achieving a 45% cost reduction throughscale-to-zero architecture (Knative Serving/Eventing with Kafka) and migrationto Graviton nodes; extending this work with KEDA to target an additional 25%operational cost savings.• Led and mentored engineering teams, delivering production-grade solutions acrossKubernetes, Kafka, microservices, and observability stacks (Prometheus, Grafana).• Improved performance of a large-scale data profiling service by 27% using Spark,pandas-profiling, and Delta Live Tables (Databricks).• Maintained and standardized Helm charts for Kafka (Strimzi), Spark, batch work-loads, and stateless microservices on AWS EKS, ensuring reliable and repeatabledeployments at scale.
EDUCATION
National Institute of Technology Rourkela
Master of Technology (M.Tech.), Computer Science
Trent University
Master of Science - MS, Applied Modelling and Quantitative Methods
Sardar Vallabhbhai Patel Institute of Technology
Bachelor of Engineering (B.E.), Information Technology
SKILLS
ABOUT DEVANG S.
Building Agentic AI & RAG Systems | LTR, Search & Retrieval Systems | Scalable Data & ML Pipelines• Senior Software Engineer with 6+ years of experience building large-scale data and AI platforms, including 3+ years delivering production-grade Agentic AI and RAG systems in legal and fintech domains.• Expert in LLM-driven architectures (MultiAgent Systems, RAG, feedback loops, hallucination detection) that automate complex research workflows and significantly reduce manual effort while improving response reliability.• Strong background in distributed data engineering and streaming systems, leveraging Kafka, Spark, and search technologies to process high-volume, low-latency data at scale.• Proven track record of designing cloud-native platforms on AWS and Kubernetes, with a focus on observability, resilience, and cost optimization, delivering measurable performance gains and infrastructure cost reduction.• Technical leader and published researcher, known for mentoring engineers, collaborating across product and business teams, and contributing to peer-reviewed publications and industry talks on cloud, data, and AI systemsComputer Science Skill: • Agentic AI: MCP, RAG, Mem0, LangChain, guardrails-ai, KServe, RayServe• Agent Orchestration: LangChain-LangGraph, Semantic Kernel, AutoGen, CrewAI• Programming: Python, Java, Linux Shell Script• Streaming: Spark, Kafka, Apache Pulsar. Flink• Databases: SQL, MongoDB, Snowflake, Cassandra, Neo4j, opensearch, Solr• Cloud & DevOps: Docker, Kubernetes (AWS EKS), Terraform, Helm Charts, ArgoCD
This profile is compiled from publicly available professional sources. Unifers is not affiliated with or endorsed by LinkedIn. Request removal of this profile.