Sujit Giridharan

About Sujit Giridharan

I help enterprises turn massive, messy data into fast, reliable pipelines that people actually trust.As a Data Solutions Architect, I design and build scalable data platforms using Databricks, Delta Lake, Apache Spark, and cloud infrastructure on GCP and Azure. My sweet spot is tackling the hardest performance problems — the kind where tables have billions of rows, jobs run for hours, and teams have stopped believing things can get better. They can. I recently re-architected a critical SAP financial pipeline (ACDOCA — 5.68 billion rows) and cut MERGE processing time from 43 minutes to 18 seconds. That wasn\'t magic; it was a systematic approach: smart partitioning, scoped MERGE operations, broadcast joins, and relentless tuning.Over 7+ years in data engineering, I\'ve built end-to-end solutions across retail operations, supply chain, financial analytics, and inventory management — processing SAP data at enterprise scale for organizations that can\'t afford downtime or bad numbers.What I bring to the table- Architecting three-layer data platforms on Databricks and Delta Lake using Medallion Architecture- Performance optimization for billion-row Delta tables using partitioning, Z-ordering, liquid clustering, and incremental loading- Deep SAP domain expertise- Data quality frameworks, CDC pipelines, schema evolution, and automated audit trails- Cloud cost optimization — right-sizing Databricks clusters to save thousands monthly without sacrificing speed- ETL/ELT pipeline development using PySpark, Spark SQL, Python, and SQL across GCP, Azure, and AWS.I hold an MS in Data Science from the University of North Texas and a BS in Computer Science and Engineering.If your data pipelines are slow, expensive, or fragile — let\'s talk.

Experience

1. Data Solutions Architect

   Grocery Outlet

   Feb 2025 — Present · Emeryville, CA, US

   Lead the design and development of enterprise-scale data platforms on Databricks, Delta Lake, and Apache Spark, processing SAP data containing 5.68B+ rows across retail operations, supply chain, finance, and inventory systems on GCP and Azure- Reduced ACDOCA pipeline MERGE processing time from 43 minutes to 18 seconds by implementing partition-based scoped MERGE operations, broadcast joins, and systematic performance tuning — a 99.3% improvement- Architect and maintain 40+ data pipelines following Medallion Architecture (Bronze → Silver → Gold) with incremental loading, Change Data Capture (CDC), and automated data quality validation frameworks- Optimized Databricks cluster infrastructure through right-sizing analysis, reducing monthly cloud compute costs by 40%+ while maintaining processing SLAs for billion-row tables- Built a comprehensive data catalog spanning 40+ retail data sources with full data lineage, metadata management, and self-service discovery for business stakeholders-Designed and implemented partitioning strategies, Z-ordering, and liquid clustering for Delta Lake tables (posdw_tlogf: 3.96B rows, acdoca: 5.68B rows), enabling sub-minute query performance on previously hour-long operations- Developed automated file processing pipelines handling thousands of CSV files hourly from SAP ERP modules (MM, SD, FI/CO, WM) with schema evolution handling and audit trail tracking- Created semantic data models and Power BI dashboards supporting self-service analytics across merchandising, procurement reporting, inventory analytics, and financial analysis- Provided technical leadership and mentorship to data engineering team on Databricks best practices, Spark optimization, and enterprise integration patterns.

Education

• University of North Texas

  Master's degree, Data Science

• INSTITUTE OF AERONAUTICAL ENGINEERING

  Bachelor of Technology (B.Tech.), Computer Science and Engineering