Max Shumouski

About Max Shumouski

Senior Data Engineer / BI Developer with over 7 years of experience delivering end-to-end…

Experience

1. Sr Bi Developer Data Engineer - Sql T-sql Python Ssis Adf Fabric Power Bi

   Zapier

   Jan 2022 — Present · San Francisco County, CA, US

   Designed, implemented, and managed scalable SQL Server database structures, including normalized schemas, complex relationships, and indexed views, ensuring efficient data handling and high availability for critical operations.• Developed advanced SQL scripts to support extensive data analysis and enhanced reporting capabilities, significantly improving the accuracy and speed of business insights.• Built reusable T-SQL procedures and triggers to enforce business logic and ensure consistent data quality across integrated systems.• Conducted in-depth performance tuning and optimization of SQL queries, employing indexing strategies and partitioning techniques to handle high-volume datasets effectively.• Architected robust data pipelines using Azure Data Factory (ADF), integrating diverse sources such as SQL Server, Azure Blob Storage, and REST APIs to facilitate seamless data flow and transformation into Data Lake for Medallion Architecture.• Enabled large-scale distributed data processing in Azure Databricks by leveraging Apache Spark and Python, utilizing libraries like PySpark, Pandas, and NumPy to execute advanced data transformations.• Implemented complex logic in ADF Mapping Data Flows, including dynamic lookups, filtering, and aggregations, to support real-time analytics processes.• Automated cross-platform data synchronization using Azure Streaming Analytics, parameterized workflows, and Kafka for real-time data streaming, reducing manual intervention and improving consistency.• Designed adaptable Azure Data Factory pipelines utilizing dynamic parameters to accommodate diverse data integration scenarios and ensure seamless scalability.• Configured and managed Azure Data Lake for efficient storage and retrieval of structured, unstructured, and semi-structured data.• Established CI/CD pipelines through Azure DevOps to streamline the development, deployment, and monitoring of data workflows.