Eniyan Paramasivam
Senior Lead Data Engineer | Cloud (AWS, Azure, Databricks) | Data Lakehouse, Delta Lake, PySpark, SQL | AI & RAG Solutions | MLflow, CI/CD, Data Governance
- Role
- Lead Databricks Data Engineer at Tata Consultancy Services
- Location
- Chennai, IN
- LinkedIn followers
- 500 followers
About Eniyan Paramasivam
I’m a Senior Lead Data Engineer with over 18 years of experience designing, building, and optimizing data engineering, AI, and cloud solutions across AWS, Azure, and Databricks ecosystems. My work focuses on transforming legacy data platforms into modern, scalable, and governed Data Lakehouse architectures that power advanced analytics and AI-driven decision-making. I specialize in building Retrieval-Augmented Generation (RAG)-based AI assistants by integrating Azure OpenAI (Mosaic AI), Databricks, and Microsoft Teams, enabling natural-language insights from enterprise data. My experience also spans MLflow-based experiment tracking, prompt optimization, and cost analysis, ensuring efficient and scalable AI deployments. On the engineering side, I’ve led multiple end-to-end implementations using Databricks (Delta Lake, Photon, Unity Catalog) and cloud-native services such as AWS (S3, Glue, EMR, Redshift, Lambda, Step Functions) and Azure (ADF, ADLS Gen2, Synapse, Purview). I’ve architected ETL/ELT pipelines using PySpark, Python, SQL, and DBT, and built real-time data processing frameworks leveraging Kafka, Auto Loader, and REST APIs. I’m deeply committed to data governance, automation, and performance optimization — implementing Terraform- and GitLab-based CI/CD, cost-efficient architectures, and security frameworks with IAM, KMS, and Key Vault. I thrive at the intersection of technology and strategy, collaborating with cross-functional teams to deliver enterprise-scale, data-driven solutions that reduce cost, improve reliability, and accelerate innovation. Core Expertise: Databricks | AWS | Azure | Delta Lake | PySpark | Python | SQL | ADF | Glue | Synapse | DBT | Kafka | MLflow | Terraform | GitLab CI/CD | RAG | Azure OpenAI | Data Governance | Cost Optimization | Leadership
Experience
Lead Databricks Data Engineer
Aug 2023 — Present · Tamil Nadu, IN
Education
Dominic Savio higher secondary school, Tirupattur, India
Bachelor of Science - BSC
2004 — 2007
Find verified contacts for anyone on LinkedIn
Unifers gives sales teams verified emails and direct dials, enriched profiles, and outreach that lands in the inbox.
Free plan included · No credit card required
This profile is compiled from publicly available professional sources. Unifers is not affiliated with or endorsed by LinkedIn. Request removal of this profile.