Harminder Sardar

About Harminder Sardar

I’m a Senior Data Engineer with 12+ years of experience building scalable, cloud-native…

Experience

1. Senior Data Engineer

   Wells Fargo

   Nov 2024 — Present · New York, NY, US

   Architected and deployed scalable cloud-native data pipelines on AWS (Glue, EMR, Lambda, S3), enabling real-time and batch analytics at enterprise scale • Re-engineered legacy ETL workflows into optimized PySpark pipelines, significantly improving performance, scalability, and maintainability • Designed end-to-end data ingestion frameworks integrating SQL Server, APIs, and cloud storage, ensuring high data quality and reliability • Built distributed data processing pipelines using Apache Spark on EMR and Databricks for large-scale structured and semi-structured data • Developed real-time streaming pipelines using Kafka and AWS Kinesis, enabling near real-time analytics and faster decision-making • Implemented Change Data Capture (CDC) and Slowly Changing Dimensions (SCD) for incremental processing and historical tracking • Integrated data lake and warehouse solutions using Redshift, Snowflake, and Delta Lake for unified analytics • Developed machine learning pipelines using Amazon SageMaker to support predictive analytics and customer insights • Automated infrastructure provisioning using Terraform, AWS CDK, and CloudFormation, reducing manual effort and improving consistency • Built CI/CD pipelines for data workflows using Git-based version control, improving deployment speed and reliability • Implemented data governance and security using AWS Lake Formation and IAM policies to ensure compliance • Optimized data models and queries, improving dashboard performance and end-user experience • Automated data validation and quality checks, ensuring accuracy and reliability of enterprise datasets