Shubhankar Ghosh
Data Engineer
- Role
- Data Engineer at Cognizant
- Location
- Kolkata, WB, IN
- LinkedIn followers
- 500 followers
About Shubhankar Ghosh
Experienced in distributed data processing and analysis using PySpark. Data ingestion and wrangling with PySpark. Spark SQL for data querying and manipulation.Developed robust ETL pipelines using PySpark on Databricks notebooks, enabling ingestion and transformation of large-scale datasets from multiple sources (structured and unstructured).Working on Design and configure Azure data factory pipeline and its data flow to migrate middleware data integration.Implemented Synapse integration with Azure Databricks notebook to make a dynamic partition of data and store it in database table for further reporting analysis.Experience in development of CI/CD pipeline to manage multiple applications into different environments.Proficient in cleaning, transforming, extracting insight and creating visualization using python libraries (NumPy, Pandas, Matplotlib, Seaborns) and Power BI Dashboards and report creation.
Experience
Data Engineer
Oct 2021 — Present · Kolkata, IN
Collaborate with Architects and Principal Engineers to design scalable, resilient data solutions that handle 100+ TB daily workloads Develop and optimize enterprise data pipelines using Azure Data Factory, Airflow, and Databricks for batch and streaming processes Lead the implementation of ML pipelines, working closely with Data Scientists to streamline model training and deployment workflows Design and implement robust data models in Snowflake and Azure Synapse, ensuring performance optimization and scalability Establish data governance frameworks and security protocols, ensuring GDPR compliance and industry best practices Develop complex SQL queries and Python/PySpark code for sophisticated data transformations Architect real-time streaming solutions using Delta Live Tables and ADLS Gen2, improving data freshness by 60% Implement comprehensive monitoring and observability solutions for data pipelines using Azure Monitor and Log Analytics Lead cross-functional teams in delivering critical data products, mentoring junior engineers and facilitating knowledge transfer Drive technical workshops and requirement gathering sessions, translating business needs into actionable technical solutions
Education
Amrita Vishwa Vidyapeetham
BCA
2021 — 2024
Ramakrishna mission shilpamandira
Diploma in Electronics and Telecommunication
2011 — 2014
Find verified contacts for anyone on LinkedIn
Unifers gives sales teams verified emails and direct dials, enriched profiles, and outreach that lands in the inbox.
Free plan included · No credit card required
This profile is compiled from publicly available professional sources. Unifers is not affiliated with or endorsed by LinkedIn. Request removal of this profile.