Shubham More
- Role
- Senior Data Engineer at Wipro
- Location
- Pune Division, MH, IN
- LinkedIn followers
- 500 followers
About Shubham More
I help organizations turn raw enterprise data into reliable, analytics-ready datasets by…
Experience
Senior Data Engineer
Jun 2022 — Present · Pune, IN
Designed and optimized Azure Databricks Spark / PySpark pipelines processing 10TB+ data daily, achieving a 40% improvement in end-to-end performance through partitioning, caching, and Spark tuning.• Automated end-to-end ETL workflows using Azure Data Factory (ADF) with parameterized pipelines and triggers, reducing manual intervention by 70% and improving data freshness and reliability.• Built and optimized Azure Data Lake Storage Gen2 (ADLS)–based data lake and Delta Lake–backed lakehouse architecture to support scalable ingestion and analytics workloads.• Implemented incremental and CDC-based ingestion patterns, enabling efficient data processing and reduced compute costs.• Designed robust monitoring, logging, and error-handling frameworks across ADF pipelines and Databricks jobs for operational stability.• Mentored and guided a team of 3 junior engineers on Azure data engineering, Spark best practices, and production-grade pipeline design, improving team productivity and delivery quality.• Collaborated with architects, analytics teams, and business stakeholders to deliver analytics-ready datasets for BI and downstream reporting.
Find verified contacts for anyone on LinkedIn
Unifers gives sales teams verified emails and direct dials, enriched profiles, and outreach that lands in the inbox.
Free plan included · No credit card required
This profile is compiled from publicly available professional sources. Unifers is not affiliated with or endorsed by LinkedIn. Request removal of this profile.