Joel Sarpong

About Joel Sarpong

With over seven years of experience leading the architecture and delivery of enterprise-scale data platforms across Azure and AWS, I bring a strategic, outcome-driven approach to building secure, scalable, and compliant data ecosystems. I specialize in defining cloud data architecture standards and driving the adoption of modern data engineering practices using Azure Databricks, Apache Spark, and cloud-native analytics services.I have a proven track record of optimizing large-scale data processing, automating complex workflows, and enabling seamless integration across diverse data sources to support advanced analytics and executive decision-making. Operating at the intersection of technology and business, I partner with senior stakeholders and cross-functional leadership teams to translate organizational objectives into resilient data strategies that deliver measurable business value, operational efficiency, and long-term platform sustainability.

Experience

1. Data Engineer

   ShopRite

   Apr 2020 — Present

   Extensive experience designing enterprise-level data ingestion pipelines, transformations, data management, and data governance solutions.Designed and implemented scalable data solutions using Azure Databricks, Azure Data Lake Storage (ADLS), and Azure Data Factory (ADF), resulting in a 30% improvement in overall data architecture efficiency.Achieved a 35% reduction in operational costs by optimizing Databricks job cluster configurations and improving resource utilization.Built and orchestrated batch ETL workflows using Apache Airflow, enabling robust dependency management and scheduling.Implemented Medallion Architecture (Bronze/Silver/Gold) with Delta Lake to enhance modularity, data quality, validation, and traceability.Led the deployment of Azure Databricks Lakehouse architecture, leveraging PySpark and Delta Live Tables (DLT) to support scalable analytics and increase data processing throughput by 30%.Developed ADF pipelines to extract, ingest, and load data from multiple sources into ADLS and Azure SQL.Performed data curation, transformation, and validation using Azure Databricks.Implemented data quality checks using Great Expectations for schema validation, null checks, and anomaly detection in nightly runs.Configured CI/CD pipelines in Azure DevOps to automate deployments across dev, test, and prod environments for Databricks and ADF assets.Partnered with data governance teams to implement data lineage and metadata tracking using OpenMetadata, improving traceability and impact analysis.Collaborated closely with product and business stakeholders to translate requirements into technical solutions, achieving 95% stakeholder satisfaction.Produced comprehensive technical documentation, including workflow diagrams, data dictionaries, and solution design artifacts for both technical and non-technical audiences.Experienced in end-to-end solution design, from requirements gathering through implementation and production support.