Venkateshwar Balakrishnan

About Venkateshwar Balakrishnan

Data engineer with 4+ years of experience transforming complex datasets into business…

Experience

1. Data Engineer

   Psolv

   Jan 2025 — Present

   Enhanced in-house ETL workflow tool by implementing pre-execution validation for PySpark scripts, reducing job failures. Developed schema checks to detect column errors before execution and used Python AST to catch syntax issues early, improving reliability and debugging efficiency.Spearheading an LLM-based data quality system to automate ETL/ELT pipeline healing with minimal human intervention. Ollama - Qwen2.5 Coder, Gemma3 models, GPT models and Llama Models.Architected Lumina, a Retrieval-Augmented Generation (RAG) system integrating BERT embeddings, FAISS, and GPT-4 to translate natural language queries into SQL on Amazon Redshift, with automated error correction and retry mechanisms, and built onboarding microservices (FastAPI, AWS S3, vector databases) to capture client metadata and business processes for context-rich workflows.Healthcare data:Built scalable Spark pipelines to process healthcare Machine-Readable Files (MRFs) from major insurers like Cigna and UHC, ensuring CMS Transparency in Coverage compliance.Extracted and indexed insurance metadata including CPT codes, plan IDs, and negotiated rates for in-network and allowed-amount files.Processed 50+ TBs of in-network and out-of-network contract details using AWS services like EC2, Batch Jobs, Lambda, EKS (Kubernetes), ECR.