Prafful Tak

About Prafful Tak

My name is Prafful Tak, and I am a B.Tech graduate of batch-2024 from IIT Jodhpur. I’m a Software Engineer working on large-scale data platforms, real-time systems, and backend infrastructure, currently at Jupiter Money.I specialize in building and scaling data-intensive systems, from ingestion to transformation to real-time serving. At Jupiter, I own end-to-end data pipelines handling 30+ TB of data across 100+ databases, powering analytics, machine learning, and business-critical workflows.My work sits at the intersection of data engineering, distributed systems, and product impact:Built and scaled Medallion architecture (Bronze → Silver → Gold) with 1400+ pipelines using Kafka, Databricks, and DBT. Designed and owned the Customer Segmentation Engine (CSE) powering real-time personalization across onboarding, payments, lending, and growth systems. Improved system reliability and latency by eliminating data gaps and scaling high-throughput real-time services. Developed high-scale pipelines processing 50M+ SMS/day and 15M+ events/day, driving measurable business impact (credit cards, loans, engagement). Led initiatives across CI/CD, cost optimization, data governance, and partner integrations, improving system efficiency and developer velocity.I enjoy solving problems involving scale, reliability, and real-time decisioning, and I’m particularly interested in backend systems, data platforms, and distributed architectures.

Experience

1. Sde-1 (Data Platform)

   Jupiter

   Apr 2025 — Present · Bengaluru, IN

   Owned end-to-end data platform operations, managing ingestion and processing of 30+ TB data across 100+ MySQL, PostgreSQL, and YugabyteDB databases using Debezium CDC → Kafka → S3 → Databricks (Delta Lake), with table-level Spark ETL pipelines orchestrating dataset refreshes• Scaled and maintained 1400+ Kafka sink connectors, transforming data across 1400+ Bronze → 400+ Silver → 700+ Gold tables via DBT under a Medallion architecture, powering analytics, ML, and business workflows• Engineered a config-driven ETL framework for ingesting Kafka-based service events and DynamoDB Streams (via Lambda CDC), supporting schema evolution and transformations; onboarded 250+ tables, significantly reducing integration effort• Owned and operated the feature pipeline for Customer Segmentation Engine (CSE), managing 400+ feature tables and building aggregation + feature-master pipelines for real-time serving with zero-downtime Elasticsearch switching• Scaled CSE and realtime services for rule-based segmentation across onboarding, payments, lending, and growth use cases• Improved realtime reliability and latency by eliminating crash loops, building DLQ-based fault tolerance, fixing critical bugs (5.5-hour data gaps, 2-hour latency), and enabling stable horizontal scalability under high throughput• Shipped CI/CD guardrails reducing feature onboarding TAT by 97%(2 days → same-day) with zero pipeline failures via mandatory DBT checks, parallel PR validation, and Airflow schema enforcement• Processed 15M+ events/day through a high-throughput pipeline with selective routing to CleverTap and Amplitude, eliminating event drops and improving real-time analytics accuracy• Built secure partner data pipelines (10+ partners) via SFTP with SHA-256 hashing, enabling privacy-compliant data sharing and reducing PII exposure risks• Implemented enterprise-grade data governance via Databricks Unity Catalog with RBAC and schema isolation, ensuring zero unauthorized access across 250+ users

Education

• Noble Public School - 2

  12th Standard - PCM

• Arihant Academy Higher Secondary School - India

  10th Standard

• Indian Institute of Technology Jodhpur

  Bachelor of Technology - B.Tech

• Resonance Eduventures Limited

  IIT-JEE, PCM