Sumit Kumar Mishra
Data Engineer @Skyscanner
Signup · Get unlimited contacts
WORK HISTORY
Data Engineer @Skyscanner
Glasgow, GB
Designed and reviewed the architecture for user behavioral datasets and session dataset to enable performance visualization of Skyscanner platform.• Developed sessions domain pipeline to incorporate user behavior, conversion, booking and revenue from different sources to help improve user journey.• Developed transactional and aggregated fact datasets to centralize and address all consumer requirements.• Developed strategies and implemented enhancements to enable rapid identification of issues within the session dataset, providing appropriate solutions with impact analysis, given the dataset\'s critical importance to the organization.• Leveraged the liquid clustering and change data feed features in Databricks to efficiently manage approximately 25 million data records per day.• Leveraged the Unity Catalog feature in Databricks to allow data governance and monitor any alerting or freshness issue using Monte Carlo.• Using Docker to containerize the AWS ECR images to deploy the PySpark jobs and the corresponding Python code via GitHub.
EDUCATION
Biju Patnaik University of Technology, Odisha
Bachelor of Engineering - BE, B. Tech
ABOUT SUMIT KUMAR MISHRA
A self-motivated & focused ETL developer who has around 11 years of experience in IT field delivering Data-warehousing, data migration, change management, business intelligence using various tools. 11 years of industry Experience with 2 years of Experience as a Data Scientist using ML algorithms and Computer Vision to build FinTech product POC’s and presenting, selling them to clients. 11 years of industry Experience with 2 years of Experience as a Data Engineering using Hadoop, BigData, Spark for doing ETL and processing data before loading it to ElasticSearch NoSQL database and finally to Quantexa for identifying and capturing Fraud Network by performing Entity Resolution. Having comprehensive understanding of project management life cycle including Technical leadership I am involved in low level design, managing team,co-coordinating between teams to reduce the gap of understanding, technical lead,maintaining quality and timeline of delivery, tool development, process implementation,Estimation, Change management, end to end project implementation.To explore data sets and identify data transformation and data quality needs for targetedApplications.Gathering data, cleansing and transforming the data, data visualization and data modelling using python libraries such as numpy, pandas, nltk, matplotlib for data scientist team to create their own model.Creating dash board for visualization of data by business using power BI. Good knowledge of transformation of data using power BI query editor (like creation of intelligence time series, column transformation for data types and data, text transformation, append queries and merge queries), creating structured data models using model view, creating measures wherever required to keep it separate from original data set for easy identification use at later point of time, creating medium of communication in data model using Data Analysis Expression (DAX). Understanding optimization of data model and how to make it light weight.
This profile is compiled from publicly available professional sources. Unifers is not affiliated with or endorsed by LinkedIn. Request removal of this profile.