Amritpal Singh

About Amritpal Singh

I have around 5 years of experience working as an Azure Data Engineer. I have good…

Experience

1. Senior Data Engineer

   Decisiontree Analytics & Services

   Sep 2023 — Present · Gurugram, IN

   Developed data pipelines in Azure Data Factory [ADF] for ETL operations on Salesforce, Shopify, Iterable, Amazon and Netsuite data.>>Worked on Delta Lake Architecture.>>Created delta tables & Upsert logic using PySpark in Azure Synapse Analytics>>Utilized Azure DevOps to migrate pipelines from lower to higher environment using CI/CD Pipelines.>>Managed Pipeline Orchestration, Error Handling, Monitoring and Error Logging in Azure Data Factory.>>Leveraged REST APIs to fetch data for Azure pipelines.>>Have used Azure Key Vault to store secrets and have used key vault along with Azure Synapse and Azure Data Factory for secure connections.>>Worked with Azure Data Lake Storage (ADLS) and Azure Blob Storage.>>Used Azure Logic Apps to trigger pipeline notification on emails and slack.>>Performed complex data transformations using PySpark in Azure Synapse.>>Utilized Copy Activity, Lookup, Data Flows, Triggers, Delete Activity, Get MetaData, If Condition, For Each, Execute Activity, Web Activity, Set Variable, Parameters, Linked Services, Datasets etc. in Azure data factory.>>Developed complex SQL queries to carry out transformation and create views on top of underlying tables.

Education

• Kendriya Vidyalaya

  12th Class

  2015 — 2016

• Acharya Institutes

  Bachelor of Engineering - BE

  2016 — 2020

• Kendriya Vidyalaya

  10th Class

  2013 — 2014