Suchita J.
Data Engineer @N.V.Data Systems Private Limited
Signup · Get unlimited contacts
WORK HISTORY
Data Engineer @N.V.Data Systems Private Limited
Bengaluru, IN
Worked on designing and developing scalable end-to-end data engineering solutions using modern cloud and big data technologies. Experienced in handling both batch and real-time data processing with optimized and production-ready pipelines. Project 1: Real-Time Data Pipeline using Kafka & DatabricksBuilt a real-time data ingestion pipeline using Apache Kafka to capture streaming data such as customer transactions and clickstream events.Processed streaming data using Databricks (Apache Spark Structured Streaming) for near real-time analytics.Implemented data transformation and cleansing logic to ensure data quality and consistency.Stored processed data in Azure Data Lake Storage (ADLS) using Delta format.Optimized pipeline performance using partitioning and caching techniques. Project 2: End-to-End Data Pipeline using Microsoft FabricDesigned and implemented an end-to-end pipeline using Microsoft Fabric to ingest data from multiple Retail APIs (Sales, Product, Customer).Used Data Pipelines (Lookup, ForEach, Web, Copy activities) for orchestration and ingestion of JSON data.Implemented Medallion Architecture (Bronze, Silver, Gold layers) for structured data processing.Performed data transformation using PySpark / Notebooks and created analytical datasets.Enabled reporting and insights using optimized data models.Key ContributionsDeveloped scalable and reusable data pipelines handling both batch and streaming data.Improved data processing efficiency by implementing performance optimization techniques.Ensured data reliability with error handling, logging, and monitoring mechanisms.Collaborated with cross-functional teams to deliver business-ready data solutions.Worked on schema handling and data validation for maintaining high data quality.
EDUCATION
Dr. Babasaheb Ambedkar Technological University (DBATU)
Bachelor of Technology
ABOUT SUCHITA J.
Hey, I’m Suchita Jadhav.I am a Azure Data Engineer with 3+years of experience in building scalable data solutions using modern cloud platforms. I specialize in designing end-to-end data pipelines using Microsoft Fabric and Databricks to transform raw data into actionable business insights.In my current project, I have designed and implemented a complete data pipeline for a retail domain using Microsoft Fabric Lakehouse architecture. I worked on ingesting data from REST APIs (Sales, Product, Inventory, Customer) using Data Pipelines with activities like Lookup, ForEach, Web, and Copy to store raw JSON data.I implemented Medallion Architecture (Bronze, Silver, Gold layers) to process and structure data efficiently. Using Databricks and PySpark, I performed data cleansing, transformation, deduplication, and flattening of complex JSON data into Delta tables.I also worked on:• Incremental data loading using watermark logic• Handling schema changes in evolving data sources• Performance optimization of Spark jobs• Data validation and quality checksTechnical Skills:Azure Data Factory (ADF), Microsoft Fabric, Databricks, ADLS, PySpark, SQL, Python, Delta LakeI am passionate about building efficient data pipelines and continuously learning new technologies in the data engineering space. Currently, I am open to new opportunities where I can contribute to scalable and impactful data-driven solutions.
This profile is compiled from publicly available professional sources. Unifers is not affiliated with or endorsed by LinkedIn. Request removal of this profile.