Gagan J
Actively seeking roles in Data Engineering | Data Engineer | Distributed Systems| Kafka | PySpark | Databricks| AWS | Snowflake | Airflow | Real-time & Batch Processing
- Role
- Senior System Engineer at Infosys
- Location
- Bengaluru, IN
- LinkedIn followers
- 500 followers
About Gagan J
I\'m a Data Engineer with 3.3+ years of hands-on experience designing and building…
Experience
Senior System Engineer
Aug 2022 — Present · Bengaluru, IN
Project 3: Snowflake Data Warehouse Migration (Ongoing)• Leading migration of ~5 TB enterprise data from on-prem systems to Snowflake.• Migrated heterogeneous data sources including Parquet and CSV formats into curated Snowflake layers.• Implemented SCD Type 2 for historical tracking and designed analytical models using Star Schema.• Applied partitioning and clustering strategies to improve query performance and reduce storage cost.Project 2: Near Real-Time Cloud Migration (On-Prem → Azure)• Engineered low-latency pipelines to migrate ~1.1M+ records with an average 813 ms latency.• Processed structured and semi-structured data such as XML, JSON during ingestion.• Built fault-tolerant pipelines with monitoring and retry mechanisms to ensure reliable cutover into Azure.Project 1: Kafka Support• Supported Kafka platform for high-volume transactional data pipelines, handling formats like Avro and JSON for streaming ingestion.• Enabled at-least-once semantics by fine-tuning producer/consumer configurations for reliable message delivery.• Designed and implemented CDC workflows using Win logs, ensuring accurate and real-time propagation of changes downstream.
Find verified contacts for anyone on LinkedIn
Unifers gives sales teams verified emails and direct dials, enriched profiles, and outreach that lands in the inbox.
Free plan included · No credit card required
This profile is compiled from publicly available professional sources. Unifers is not affiliated with or endorsed by LinkedIn. Request removal of this profile.