Ali Sayyed

About Ali Sayyed

Accomplished Azure Data Engineer offering more than 3 years of experience in designing…

Experience

1. Azure Data Engineer

   Infosys

   Dec 2023 — Present · Pune, IN

   Worked on Databricks for data preparation and complex transformations according to business requirements and creating dashboards on Power BI.• Experience in working with unity catalogue, created Metastore & schema for silver and gold tables and access connector to connect Databricks and ADLS’s container.• Developed Delta live tables (DLT), SCD-I and SCD-II logics, incremental data processing pipelines for Change Data Capture (CDC) workloads, leveraging Delta live table’s versioning, merge, time travel, logs.• Created ETL pipelines on ADF for data migration from source systems to SSMS.• Worked on stored procedures and integrated it with Chester for client\'s usage.• Worked on automation of pipelines using scheduled and event triggers on adf pipelines. • Added error notification functionality to our master pipelines.• Created Databricks notebooks which handles large sized files using partition By approach, file conversion from different formats to excel for smooth data ingestion and implemented many other scenarios.• Worked on many source systems like SAPBI, sharepoint, SFA, outlook, Promote, shared drive,etc.• Hands-on experience on MS SQL Server, Azure Data Factory and PySpark.• Implemented various loading strategy like append and upsert in case of incremental refresh and overwrite in case of batch file processing.• Conducted thorough data validation across end reports and Power BI outputs, including the development of comprehensive validation documents and artifacts to ensure data integrity and accuracy.