Akash Dwivedi
- Role
- Data Engineer at Expedia Group
- Location
- Gurugram, IN
- LinkedIn followers
- 500 followers
About Akash Dwivedi
Driving Enterprise-Scale AI Innovation at Expedia Group As a Senior Machine…
Experience
Data Engineer
Dec 2020 — Present
Developed the technical architecture and frameworks for a real-time customer experience reporting platform, leveraging Python, Kafka, Spark, and Cassandra. • Successfully migrated the central state store of real-time processors from Redis to Scylla, resulting in an annual cost savings of $8M. • Worked on migrating the underlying data warehouse from Snowflake to Starburst, uses Presto to enhance query speed and divide the pipeline into two parts (cold and hot data). Migrated real-time reports to Dremio and batch reports to Starburst, resulting in additional cost savings. • Scaled the real-time processors from an initial capacity of 8CPS to 25-30CPS by centralizing fragmented code and converting monolithic services to microservices. • Set up a scalable and simplified Kafka Connect solution from scratch, with dynamic schema update and PEM-based authentication for improved connectivity between two clusters.
Skills
- C
- Powerpoint
- Windows
- C++
- Microsoft Word
- Leadership
- Html
- Microsoft Office
- Microsoft Excel
- Programming
- Python
- Django
- Mysql
- Php
- Cascading Style Sheets (Css)
- Linux
Find verified contacts for anyone on LinkedIn
Unifers gives sales teams verified emails and direct dials, enriched profiles, and outreach that lands in the inbox.
Free plan included · No credit card required
This profile is compiled from publicly available professional sources. Unifers is not affiliated with or endorsed by LinkedIn. Request removal of this profile.