Soyed Muniruzzaman J

Experience

1. Senior Data Warehouse Engineer

   CHS Inc.

   Jan 2024 — Present · US

   As a Senior Data Engineer at CHS, I lead and mentor a team in designing, developing, and optimizing scalable data pipelines using Apache Spark, Databricks, and Kubernetes across multiple cloud platforms. I architected and implemented a hybrid-cloud data strategy, integrating AWS (Redshift, Glue), Azure (Synapse, ADF), GCP (BigQuery, Dataflow), and Snowflake, enabling enterprise-wide analytics. I designed and optimized ETL workflows using Airflow, dbt, and Terraform, automating data ingestion, transformation, and deployment across various platforms. Additionally, I established data governance frameworks with Apache Atlas, AWS Lake Formation, and Azure Purview to ensure data security, lineage tracking, and compliance with industry regulations such as HIPAA and GDPR. My work includes implementing real-time data streaming solutions with Kafka, AWS Kinesis, and Google Pub/Sub to enhance event-driven architectures. I collaborate with business stakeholders, data scientists, and DevOps teams to define data strategies, build scalable data models, and enable AI/ML-driven insights. I also established best practices for CI/CD in data engineering using GitHub Actions, Jenkins, and Cloud Build, streamlining deployment and reducing operational downtime. A key initiative I led involved modernizing legacy data systems by migrating on-prem databases to cloud-based data lakes and warehouses, significantly improving performance and scalability.