Amol Patil

About Amol Patil

Data engineer with 5years of experience in creating ETL data pipeline Extract, transform and loading data form one source to destination.Proficient I working with Python,SQL, Pyspark and machine learning.Currently working on Azure Databricks, Azure Data factory, ADLSCreating pipeline in azure data factory and working in databricksGood knowledge of developing and implementing effective data solution.also having good hand on machine learning as well of building a statistical model by using different machine learning algorithm.

Experience

1. Data Engineer

   EY

   Jul 2023 — Present

   Designed and developed ETL/ELT pipelines using Azure Data Factory (ADF) for ingesting data from Oracle DB, APIs, flat files, and cloud sources. Managed data storage and governance using Azure Data Lake Storage (ADLS) with optimal folder structure, access control, and versioning. Built and optimized PySpark/SQL notebooks in Databricks for data transformation, cleansing, and feature engineering. Developed data models and analytical datasets in Azure Synapse for reporting and dashboard consumption. Worked closely with business teams from Insurance (claims, policies, customer data) and FMCG (sales, supply chain, distribution) to understand data requirements and provide scalable solutions. Performed data validation, quality checks, and implemented CI/CD pipelines using Azure DevOps for smooth deployment. Optimized performance of pipelines, storage, and queries to reduce cost and improve efficiency. Collaborated in an Agile environment, participating in sprint planning, solution design, and release management.

Education

• Government College of Engineering Aurangabad

  Master of Computer Applications - MCA

  2014 — 2017