Tharun Kumar Reddy Karukula

About Tharun Kumar Reddy Karukula

I\'m a Senior Data Engineer with 5 years of experience building secure, scalable, and high-performance data platforms on Microsoft Azure.Currently at Infosys working with Microsoft\'s Payments Reconciliation team, I\'m leading the migration of an entire enterprise analytics platform from legacy distributed computing to Azure Databricks and Delta Lake — covering account reconciliation, submission processing, currency conversion, chargeback handling, and financial reporting. We achieved 100% data parity with production outputs across every processing stream.What I do day-to-day:Design and build Bronze → Silver → Gold Lakehouse pipelines processing millions of daily payment transactionsBuild reusable PySpark frameworks for data ingestion, transformation, validation, and quality assuranceOrchestrate production workloads using Azure Data Factory with automated scheduling, retry policies, and dependency chainingModernize CI/CD infrastructure using Managed SDP with Bicep, ARM Templates, and EV2 deploymentsCreate SOX compliance dashboards with proactive alerting and automated incident managementBuild monitoring dashboards in Microsoft Fabric, Power BI, and Azure Workbooks for operational visibilityOnboard new payment partners with end-to-end data integration pipelinesPreviously, I delivered mainframe-to-data-lake migrations for banking KYC compliance using Informatica BDM, Spark Scala, Hadoop, and Hive — ensuring 24/7 data availability and regulatory compliance.Tech I work with: PySpark, Azure Databricks, Delta Lake, Azure Data Factory, Azure Synapse, Microsoft Fabric, Power BI, Cosmos DB, Python, SQL, Spark Scala, Bicep, ARM, Azure DevOpsCertified: Microsoft Fabric Analytics Engineer AssociateAlways open to connecting with fellow data engineers, architects, and anyone passionate about building data platforms at scale. Feel free to reach out!

Experience

1. Specialist Programmer L2

   Infosys

   Jan 2025 — Present

   Working with Microsoft\'s Payments Inference & Reconciliation team.• Led the end-to-end migration of the entire EPA Core platform from legacy distributed computing to Azure Databricks PySpark — covering account reconciliation, submission processing, currency conversion, chargeback handling, data cleansing, and financial reporting — achieving 100% row-level data parity with production outputs across all streams.• Architected and implemented Bronze–Silver–Gold Delta Lake pipelines ingesting data from Cosmos structured streams, Azure Data Lake, and Excel sources, processing millions of daily payment transactions across both real-time and batch workflows.• Designed reusable PySpark frameworks for standardized data ingestion, transformation, and automated data validation, reducing manual verification effort by 40%+.• Built and maintained Azure Data Factory (ADF) orchestration pipelines with scheduled triggers, dependency chaining, and automated retry policies for production workloads.• Upgraded deployment infrastructure to Managed SDP, establishing end-to-end CI/CD pipelines with automated build, test, and release stages for Databricks notebooks, ADF ARM templates, Bicep/IaC deployments, and EV2 rollouts across environments.• Developed ADLS Growth Monitor dashboard using Microsoft Fabric (OneLake, Lakehouse, DirectLake) for real-time storage insights and weekly capacity trend reporting.• Created Power BI dashboards and Azure Workbooks with KPIs on pipeline health, execution statistics, and stakeholder reporting.• Built and monitored SOX compliance dashboards with proactive alerting for early anomaly detection; established automated ICM incident creation for pipeline failures.• Onboarded new payment partners into the reconciliation platform with end-to-end ingestion, mapping, and validation configurations.• Optimized Spark job performance through partition pruning, caching strategies, and resource tuning, achieving 30%+ reduction in pipeline costs and runtime.

Education

• Sri Chandrasekharendra Saraswathi Viswa Mahavidyalaya, Kancheepuram

  BE, Computer science and engineering

  2016 — 2020