Subbarao Bathina

About Subbarao Bathina

Data Engineer with 8+ years of experience designing and delivering enterprise-scale data solutions across banking and regulated industries. I specialize in building and optimizing data lakes, data warehouses, and ETL pipelines using Spark (Scala/PySpark), Python, SQL, and Unix Shell scripting, with proven expertise in Hadoop ecosystems and cloud platforms (Azure, AWS).I have led projects in data governance, lineage, and compliance reporting, ensuring accuracy, integrity, and audit readiness across global environments. My background includes performance tuning of Spark pipelines to meet strict SLAs, large-scale data migrations, and publishing curated datasets for BI platforms such as Tableau, Power BI, Qlik, SAS, and Crystal Reports (with familiarity in Epic).Passionate about turning data into trusted insights, I thrive in cross-functional Agile teams, bringing strong communication, problem-solving, and leadership skills to align technical delivery with business objectives.

Experience

1. Data Engineer

   EPAM Systems

   Jun 2023 — Present · US

   Built a real-time fraud detection platform using Azure Event Hubs, Databricks, and Synapse Analytics to process live credit card transaction streams across regions.•Developed Spark Structured Streaming pipelines ingesting from Azure SQL, ADLS, and external APIs, applying schema validation via Azure Schema Registry.•Automated complex ETL workflows using ADF, with error tracking via dead-letter queues and logging for both batch and streaming paths.•Integrated third-party Visa and Mastercard APIs using secure OAuth authentication, processed JSON in Python, and routed data into real-time Spark flows.•Applied Spark performance tuning techniques like dynamic memory allocation and executor scaling to ensure low- latency processing.•Created Power BI dashboards with DirectQuery to Synapse, offering fraud analysis views by region, card type, and transaction risk level.•Used Azure Repos for structured code collaboration, pull request workflows, and sprint progress tracking throughAzure Boards.