Syed Babji

About Syed Babji

In my current role as an Senior Data Engineer, I play a pivotal role in projects that revolves around [Azure Data Factory (ADF), Azure Databricks, and Azure Synapse] and [MongoDB, AWS Data Lake, S3, AWS Glue, AWS DataSync, Data Platform, Pyspark]. My key contributions include:Lentra Data Platform (primary):Responsibilities:• Exposure to AWS Glue, DataSync, EC2, S3, EMR, MongoDB.• Design and implement ELT processes to load Tbs of data into the data lakes.• Spearheaded the archival data migration project from on-premises mongodb Dighi servers to AWS S3 data lakes, utilizing AWS DataSync, mongodb queries, Python (pymongo)and shell scripts for efficient data transfer and storage.• Implemented PII (Personally Identifiable Information) masking solutions using PySpark and AWS Glue over CDC real time data, ensuring compliance with data privacy regulations.• Loaded and transformed large sets of semi structured data using pyspark in AWS Glue.• Orchestrated the data transfer process using AWS DataSync, Python scripts (pymongo), shell scripts with mongosh commands, and AWS CLI commands.• Collaboration with Data Scientists and other data teams.• Loaded and transformed large sets of semi structured data likes JSON, Avro, Parquet, Binary.KMBL EWS Azure (secondary):Responsibilities:• Exposure to Azure Data Factory (ADF), Azure Databricks, Azure Synapse and Azure Data Lakes GEN2.• Responsible for managing day-to-day data pipelines using Azure Data Factory, Databricks, and Synapse for an Early Warning system.• Designed and implemented data pipelines to extract, transform, and load (ETL) bank data from onpremises servers to Azure Data Lake Storage Gen2, utilizing Azure Data Factory.• Developed and optimized Databricks notebooks (Pyspark) for processing raw data and storing the processed data in designated zones within Azure Data Lake Storage.• Optimising SQL queries in Azure Synapse to support data analysis and reporting tools like powerBI• Used JIRA for bug tracking and Bitbucket to check-in and checkout code changes

Experience

1. Senior Consultant

   EY

   Dec 2024 — Present · Bengaluru, IN

Education

• ST.JOSEPH'S BOYS HIGH SCHOOL

  SECONDARY EDUCATION

  2010 — 2014

• NARAYANA JUNIOR COLLEGE

  INTERMEDIATE, PCM

  2014 — 2016

• K.S.R.M. College of Engineering (MBA)

  Bachelor of Technology, Electronics and communication engineering

  2016 — 2020