Nikhil K.

Data Engineer @American Medical Association

What I’m Working On:• Agentic AI for Data ProcurementLed the development of a LLM-driven automation engine that interacts with U.S. state medical boards via email. The system intelligently interprets incoming email responses—ranging from attachments and secure download links to payment instructions—and dynamically triggers the appropriate data ingestion workflows on Databricks using MCP protocol.• Sanctions Intelligence – Fine-Tuning LLMFine-tuned a foundation model to understand and classify physician sanctions based on 150K+ PDF documents and proprietary AMA codes. Scaled from a single GPU to multiple GPU\'s for training using pytorch and deepspeed. The goal is to achieve 99% accuracy and replace thousands of hours of manual review, enhancing compliance and operational efficiency.• Retrieval-Augmented Generation (RAG) for PDF ParsingImplemented RAG pipelines to extract granular details about physicians and their sanctions from unstructured PDF sources—turning buried information into actionable intelligence.• Data Platform Engineering on DatabricksManaging and optimizing large-scale data workflows, and job orchestration on Databricks to support real-time and batch data processing across multiple healthcare and regulatory use cases.• Genie Chat Assistant on Microsoft TeamsIntegrating a Text-to-SQL AI assistant on AMA structured data to enable faster insights for sales and marketing teams. • CI/CD Pipeline Architecture - Driving the design and implementation of automated deployment pipelines for Databricks environments.• Data Governance Platform - Partnering with enterprise stakeholders to reimagine data governance strategies, building an integrated dashboard solution that manages data assets, enforces policies, and streamlines data access workflows. Whether it’s fine-tuning models, building retrieval pipelines, or designing AI agents that autonomously interact with humans and systems, I’m driven by one goal: automate the un-automatable!