Madhava A

About Madhava A

I\'m a results-driven Lead Data Engineer with deep expertise in building scalable…

Experience

1. Senior Data Engineer

   Citi

   Jul 2025 — Present

   Directed the design, development, and modernization of Citi’s enterprise analytics platform on Cloudera Hadoop, delivering governed, high-performance datasets for operational and executive reporting across business lines.• Led the migration of legacy Hadoop/Cloudera workloads to Databricks on AWS, enabling scalable cloud-native data engineering and reducing infrastructure costs.• Applied advanced Databricks performance optimization techniques including Liquid Clustering, Z-Ordering, and Adaptive Query Execution (Layout Adapter) to improve query efficiency and cut pipeline runtimes by 30–40%.• Implemented PySpark optimization strategies such as broadcast joins, partition pruning, caching, and skew mitigation to enhance cluster utilization and stability.• Modernized orchestration by migrating workflows from Autosys to Databricks Workflows, achieving tighter integration with Databricks jobs and simplifying operational monitoring.• Transitioned CI/CD pipelines from TeamCity/UrbanCode Deploy to Jenkins, automating build and release processes with stronger governance and deployment reliability.• Built data validation and reconciliation frameworks to ensure accuracy, lineage, and auditability of datasets consumed by credit risk, fraud detection, and capital adequacy models.• Partnered with finance, risk, and compliance teams to deliver analytics-ready, audit-compliant datasets that drive stress testing, liquidity forecasting, and regulatory insights. Environment: PySpark, SQL, Hive, Cloudera (Hue, Impala), Autosys, Bitbucket, Databricks Workflows, Bitbucket, Jenkins, Team Citi, UrbanCode Deploy,ETL Pipelines, Data Validation, Data Cleansing, Data Management, Reporting & Insights, AWS cloud services.