Snehasish Dutta
Professional Data Engineer @Lidl E-Commerce
Signup · Get unlimited contacts
WORK HISTORY
Professional Data Engineer @Lidl E-Commerce
Berlin, DE
Architected event-driven pipelines using Spark Scala/PySpark, ingesting 6M+ records daily from Google Pubsub, Kafka, SAP, and Snowflake into Delta Lake medallion architecture• Delivered mission-critical pipelines for orders, product-inventory, stock, and pricing; built real-time stock notification system using Spark Streaming generating €1.5M revenue• Enforced AI-driven schema discovery and automated schema creation; streamlined code reviews; simplified onboarding of new Pubsub events• Engineered CI/CD pipelines using Azure DevOps; maintained Spring Boot stream applications consuming from Pubsub; driving event-driven architecture adoption• Spearheading platform evolution: evaluating Google Pubsub source with Iceberg sink, Apache Flink, Lakeflow, and Golang-based Franz for cost optimization
EDUCATION
VTU Group
Bachelor of Engineering, Computational Science
Visvesvaraya Technological University
Engineer's Degree, Computer Sciene and Engineering
The Scottish Church Collegiate School
High School, Mathematics and Computer Science
SKILLS
ABOUT SNEHASISH DUTTA
Enthusiastic and Zealous about BigData Technologies
This profile is compiled from publicly available professional sources. Unifers is not affiliated with or endorsed by LinkedIn. Request removal of this profile.