Girish Patil

About Girish Patil

I’m a Sr, Data Engineer passionate about designing and optimizing ETL, data ingestion, and transformation pipelines that drive business value. With extensive experience in AWS, Azure, and big data technologies, I specialize in building scalable, high-performance data solutions for large-scale enterprises.I\'ve developed end-to-end data pipelines, automated data ingestion from diverse sources, and optimized cloud-based ETL workflows across finance, transportation, and analytics domains. From architecting settlement engines to implementing governance models with Apache Atlas & Ranger, I ensure secure, efficient, and automated data workflows.As a collaborative engineer, I thrive in cross-functional teams, tackling complex data challenges and enabling businesses to make data-driven decisions with confidence.

Experience

1. Sr Data Engineer

   Enquero

   Sep 2022 — Present

   Developed a global settlement engine in PySpark, integrating with an ERP system to process payments based on complex rules and aggregation logic across multiple regions.Optimized AWS Glue jobs using multithreading and Spark optimization techniques, improving performance and efficiency.Automated report generation by extracting data from Tableau, converting it into PDFs, and sharing it with users.Built sales and transportation dashboards using PySpark and AWS, integrating data from in-house and third-party sources for advanced analytics.Designed and maintained ELT data pipelines using Azure Data Factory and Databricks, ensuring seamless orchestration and processing of large datasets.Developed data transformations using Databricks (PySpark) for efficient processing, aggregation, and data cleansing, improving pipeline performance.Implemented Azure Data Factory pipelines, including activity mapping, data flows, triggers, and linked services, supporting seamless transformations across environments.Performed extensive debugging, validation, and error handling on large datasets using Azure Databricks and AWS Glue, ensuring data integrity and operational efficiency.Developed and optimized data pipelines leveraging AWS S3 for storage, AWS Athena for serverless querying, and AWS Step Functions for workflow automation.Collaborated with DevOps teams to implement CI/CD pipelines and test-driven development in Azure, streamlining data workflows and supporting incremental loads.Skills: PySpark · Databricks · Azure Data Factory · AWS Glue · SQL · Python · Big Data · ETL · Data Engineering · Spark Optimization · API Integration

Education

• Centre for Development of Advanced Computing

  PG-DAC

  2017 — 2017

• University of Mumbai

  Bachelor of Engineering (B.E.)

  2011 — 2015