Soumya N
Senior Data Engineer
- Role
- Senior Azure Data Engineer at 84.51˚
- Location
- Chicago, IL, US
- LinkedIn followers
- 500 followers
About Soumya N
Network & Cloud Engineer with 3+ years of experience in enterprise networking, data…
Experience
Senior Azure Data Engineer
Jul 2022 — Present · Chicago, IL, US
Designed and implemented scalable data pipelines in Azure Data Factory and Databricks, ingesting and transforming batch and streaming data from diverse sources including SQL, REST APIs, and Kafka- Built and optimized real-time data processing workflows using Spark Structured Streaming and Delta Lake, enhancing performance and ensuring high data availability- Developed a robust medallion architecture with Delta Lake and implemented SCD Type 2 for accurate historical tracking using dbt and Databricks- Integrated Snowflake for advanced analytics, utilizing Snowpipe for real-time ingestion and Time Travel for historical data access and analysis- Implemented RBAC and ACL policies in Unity Catalog and ADLS Gen2, ensuring secure and governed access to data assets- Automated deployments using Azure DevOps and managed CI/CD pipelines for Azure Data Factory, Synapse, and Databricks projects- Leveraged Azure Key Vault and AAD OAuth2 for secure handling of secrets and API authentication across data pipelines- Built interactive Power BI and Tableau dashboards integrated with SSRS/SSAS, enabling self-service analytics and business intelligence- Collaborated with cross-functional teams in Agile environments, delivering data solutions that improved processing efficiency and reduced infrastructure costs by 30%.
Education
Eastern Illinois University
Master of Science - MS
2015 — 2016
Find verified contacts for anyone on LinkedIn
Unifers gives sales teams verified emails and direct dials, enriched profiles, and outreach that lands in the inbox.
Free plan included · No credit card required
This profile is compiled from publicly available professional sources. Unifers is not affiliated with or endorsed by LinkedIn. Request removal of this profile.