Sukanya Ghosh
AI Data Engineer | Big Data | Apache Spark |Agentic AI | SQL | Gen AI | LLMs | Databricks | Apache Iceberg
- Role
- Data Engineer at Dell Technologies
- Location
- Kolkata, WB, IN
- LinkedIn followers
- 500 followers
About Sukanya Ghosh
I am a Data Engineer and Data Architect with 11+ years of experience in software…
Experience
Data Engineer
Jun 2021 — Present · Kolkata, IN
Designed and deployed 50+ end-to-end data pipelines using PySpark, Kafka, Hive, SQL, Iceberg, and MongoDB.>> Optimized ETL workflows, reducing data processing time by 75% through PySpark tuning and indexing.>> Built solutions using Python and Iceberg on the IOMETE data lake.>> Created reusable utilities for Apache Iceberg on Hadoop, testing various methods to determine compatibility with the Hadoop ecosystem using Spark.sql. This included testing data compaction, distribution, partitioning, and sorting during data writing.>> Led on-premise to cloud migration, enhancing scalability with Databricks.>> Developed a GenAI-powered classification system for customer feedback, leveraging PySpark and OpenAI LLMs.>> Automated data workflows using Apache Airflow, ensuring reliable execution.>> Reduced SQL-heavy job execution time from 10+ hours to 2 hours by refactoring with PySpark.>> Mentored junior engineers, providing training on Spark optimizations and best practices.>> Created user documentation and trained customer support teams on new data tools and processes.
Education
CAMELLIA INSTITUTE OF TECHNOLOGY
Bachelor of Technology (B.Tech.), Electronics and Instrumentation engineering
2009 — 2013
Dumdum Motijheel Girls High School
10, General studies
2007
DumDum Motijheel Girls high school(h.s)
10+2, Mathematics,Computer Science, Physics , chemistry
2009
Skills
- Lombardi Teamworks
- Java
- Web Design
- Teamwork
- Teaching
- Html
- Websphere Integration Developer
- Microsoft Office
- Requirements Analysis
- Business Process Management
- Ibm Bpm
Find verified contacts for anyone on LinkedIn
Unifers gives sales teams verified emails and direct dials, enriched profiles, and outreach that lands in the inbox.
Free plan included · No credit card required
This profile is compiled from publicly available professional sources. Unifers is not affiliated with or endorsed by LinkedIn. Request removal of this profile.