Amitpal Singh

About Amitpal Singh

Experienced and detail-oriented Big Data/Data Engineer with 3.6 years+ of hands-on experience in designing, developing, and maintaining robust data pipelines for enterprise-grade systems. Proven ability to work across both cloud platforms and on-premises environments, delivering scalable, high-performance, and secure data solutions. Adept at building distributed data processing systems, optimizing large-scale ETL/ELT processes, and managing structured and semi structured data efficiently. Big Data Tools: Apache Spark, Hadoop, PySpark, Spark SQL, YARN, Hive Programming Languages: Python, Scala, SQL Data Engineering: ETL/ELT Processes, Spark Optimization, Data Modelling Tools & Platforms: Git (Version Control), Azure, Microsoft Fabric Cloud Platforms & Services: Azure Data Factory (ADF), ADLS Gen2, Azure SQL Database, Azure Key Vault, Data Warehouses, Lakehouse Architecture, Fabric Pipelines, Fabric Dataflows, Delta Lake (on Fabric) Workflow Orchestration: Apache Oozie, Cron

Experience

1. Digital Specialist Engineer L1

   Infosys

   Apr 2024 — Present

   Designed and developed end-to-end data pipelines on Azure & Microsoft Fabric, implementing the medallion architecture (bronze, silver, gold layers) to ensure structured and scalable data processing.• Built robust data ingestion workflows to fetch and process data from multiple sources—including HCM Cloud, Flow Extract, and PeopleSoft—handling over 300 Product Variant Objects (PVOs) and 20+ extract files, and efficiently loading them through to the silver layer.• Created and transformed fact and dimension tables to support downstream analytics, enabling the Power BI team to build semantic models for comprehensive business reporting.• Implemented security best practices by defining security roles for various Azure Active Directory (AAD) groups, including object-level security and data masking to protect sensitive information.• Developed an automated, scalable notebook solution capable of loading 300+ PVOs into the silver layer, significantly improving operational efficiency and reducing manual intervention.• Established a metadata-driven framework and implemented detailed logging for each PVO, enhancing data traceability, monitoring, and governance across the pipeline.• Collaborated with analytics teams to complete a proof-of-concept (POC) Power BI report, demonstrating the effectiveness of the data pipeline in supporting business intelligence and reporting needs.

Education

• Baba Farid Institute of Technology, Dehradun

  BCA, Computer Science

  2015 — 2018

• Graphic Era Deemed to be University

  Master of Computer Applications - MCA, Computer Science