Varun Reddy Talla

About Varun Reddy Talla

Microsoft‑certified Senior Azure Data Engineer who turns raw, high‑velocity data into real‑time, decision‑ready intelligence. Over the last 4 + years I’ve architected and operated lakehouse platforms that stream 100 GB +/day through Azure Event Hubs, Databricks Structured Streaming and Delta Lake, shrinking latency from 4 hours to 5 minutes while sustaining 99.9 % uptime.What I Do Best->Real‑Time Pipelines: Build CDC‑enabled Spark jobs that land, cleanse and curate data in minutes, not hours->Warehouse Modernisation: Migrate legacy SSIS/Teradata workloads to Azure Synapse & ADF, re‑model into star‑schemas and serverless lakehouses, cutting query time 60 % and platform spend 40 %->AI‑Driven Analytics: Embed SynapseML & Azure ML models (segmentation, anomaly detection, forecasting) directly in pipelines; deliver sub‑200 ms scoring to apps and APIs->Power BI Enablement: Publish certified Delta datasets and live DirectQuery dashboards that turned 200 users into 1 200 + active decision‑makers->Governance & DevOps: Enforce lineage and RBAC with Azure Purview & Key Vault; automate releases via Azure DevOps and Terraform, complete with data‑quality unit tests and blue/green rollouts->Observability: Instrument everything with Log Analytics, Monitor, Application Insights and Great Expectations, keeping bad‑data incidents below 0.5 %.Signature Wins Scaled a lakehouse from 10 TB to 250 TB with zero downtime using adaptive compaction & partition pruning. Saved ~$100 K /yr by negotiating spot‑instance strategies and auto‑scaling cluster policies. Lifted recommendation accuracy 25 % by enriching feature stores with sentiment mined via Cognitive Services. Mentored 15 + data engineers, launching an internal “Databricks Guild” that standardised PySpark patterns and code reviews.Why Connect With MeI combine hands‑on engineering depth with a product mindset: business outcomes come first, tech choices follow. Whether you’re untangling legacy ETL, aiming for real‑time insights, or weaving AI into your data fabric, I’ll help you ship reliable, cost‑efficient solutions at scale.Always learning, always sharing—let’s connect and build smarter data systems together.

Experience

1. Azure Data Engineer

   Walmart

   Jan 2023 — Present · Bentonville, AR, US

   Senior Azure Data Engineer with 4 + years turning Walmart‑scale data into live, trusted intelligence. I design and operate a lakehouse that ingests 100 GB + daily of click‑stream, supply‑chain and POS data through Azure Event Hubs, Databricks Structured Streaming, Delta Lake and PySpark, cutting latency from 4 h to 5 min while holding 99.9 % uptime. I have migrated 50 + legacy Oracle/SSIS ETL jobs into Azure Synapse Analytics and Data Factory, rebuilt them as star‑schema and lake zones, and wired everything to Azure DevOps CI/CD with data‑quality unit tests, blue‑green releases and Great Expectations monitors exposed in Azure Monitor & Log Analytics—slashing query time 60 % and platform spend 40 %.I embed AI where it matters: SynapseML flags pipeline anomalies in real time; Azure ML hosted segmentation scores 10 M + customers in < 200 ms; Cognitive Services mines sentiment from millions of reviews, enriching features that lifted recommendation accuracy 25 %. Downstream, I publish certified Power BI datasets and DirectQuery dashboards that put real‑time KPIs in the hands of merchants, finance and logistics teams, growing active insight users six‑fold.Fluent in governance (Purview, Key Vault, RBAC, Private Endpoints) and infrastructure‑as‑code (Terraform, ARM), I translate business goals into reliable Spark code, cost‑efficient cluster strategies and audit‑ready data flows. If your challenge involves streaming scale, warehouse modernisation or AI‑driven analytics, I bring a proven record of building secure, high‑impact Azure data platforms.

Education

• Wilmington University

  Master's degree, Information System Technology

• Guru Nanak Institutions(GNI)

  Bachelor of Technology - BTech, Mechanical Engineering