Shanmuk Chadaram
Consultant - Data Engineering | Databricks | Spark | Python | Cloud Migrations | GenAI Agents & RAG Systems
- Role
- Consultant at ADP
- Location
- Hyderabad, TG, IN
- LinkedIn followers
- 500 followers
About Shanmuk Chadaram
Senior Data Engineer with 8 years of experience in the Finance, Sales and Health Care, Insurance and Claims Domain. Full life-cycle Project Delivery Experience from product definition to implementation including requirements gathering and Specification writing, Design documentation, unit and system testing, Optimization and performance analysis, quality assurance, automating manual tasks, Code release into production environment, preparing release documents for Enhancements and monitoring the jobs.Primary Skills: Databricks, Pyspark, Informatica Powercenter, Streamsets, Teradata, Oracle, Control-m, Unix, Shell Scripting, pythonSecondary Skills : Talend, AWS S3, AWS Redshift, Python, Pandas, Tableau, Greenplum, Apache Kafka, Gitlab CICD, Mainframes CA-7 Scheduler
Experience
Consultant
Oct 2025 — Present · Hyderabad, IN
Spearheading large-scale data migration initiatives, transitioning complex legacy SSIS packages, snaploogic tasks into a modernized Databricks environment to enhance scalability and pipeline performance.Architected and deployed a highly scalable fuzzy logic matching system utilizing Apache Spark and Databricks.Engineered advanced blocking and salting techniques within Spark to efficiently process, deduplicate, and match workloads exceeding one billion records.Designing and optimizing robust ETL pipelines, focusing on compute efficiency, data quality, and platform reliability.Integrating Generative AI capabilities as a specialized layer within data operations to resolve platform inefficiencies and automate engineering workflows.Developing intelligent, AI-driven diagnostic agents that leverage Unity Catalog lineage and repository metadata to perform automated root-cause analysis on Databricks job failures.Building Proof of Concept (PoC) RAG (Retrieval-Augmented Generation) systems utilizing Databricks Mosaic AI to streamline metadata management and platform observability.
Education
National Institute of Technology, Kurukshetra, Haryana
Master of Technology - MTech, Instrumentation Technology/Technician
Avanthi Institute of Engineering & Technology, Tamaram Village, Makavarapalem mandal, Narsipatnam , Pin-531113. (CC-81)
Bachelor of Technology - BTech, Electrical, Electronics and Communications Engineering
Find verified contacts for anyone on LinkedIn
Unifers gives sales teams verified emails and direct dials, enriched profiles, and outreach that lands in the inbox.
Free plan included · No credit card required
This profile is compiled from publicly available professional sources. Unifers is not affiliated with or endorsed by LinkedIn. Request removal of this profile.