K Vijay

About K Vijay

I build data systems that don’t break when scale hits.I’m a Senior Data Engineer who designs and delivers end-to-end data pipelines that turn messy, high-volume data into reliable, production-ready systems. My focus is on building platforms that support real-time analytics, reporting, and machine learning without bottlenecks.In my recent work, I’ve built scalable pipelines using Databricks, Spark, and Kafka, integrating platforms like Snowflake and AWS to handle both batch and streaming workloads. I’ve improved pipeline performance, reduced manual effort through automation, and ensured data is clean, consistent, and ready for downstream teams. I also work closely with data modeling, governance, and orchestration—using tools like Airflow, DBT, and Unity Catalog—to make sure systems are not just fast, but also reliable and maintainable.The problems I enjoy solving most are around scale, performance, and reliability—whether it’s optimizing slow pipelines, designing real-time streaming architectures, or building lakehouse solutions that support multiple use cases.My core strengths include ETL/ELT pipeline design, real-time data processing, distributed systems with Spark, and building cloud-native data platforms across AWS and Azure.Right now, I’m looking for Senior Data Engineer roles where I can work on large-scale data platforms, real-time systems, and high-impact business problems.If you’re hiring or want to connect, feel free to reach out at v••••••••@gmail.com/+12•••••••65 or message me here.

Experience

1. Sr Data Engineer

   JCPenney

   Jan 2023 — Present · Irving, TX, US

   Built scalable ETL pipelines using Databricks, Spark, and Airflow, resulting in 40% faster data processing and improved reliability.Integrated Snowflake with AWS S3 and Redshift, resulting in 30% faster query performance for analytics teams.Automated workflows using DBT and Airflow, reducing manual effort by 35% and improving pipeline consistency.Developed real-time streaming pipelines using Kafka and Spark, resulting in near real-time insights (<5 min latency).Optimized data processing using PySpark and SQL, reducing pipeline runtimes by 30% and lowering compute costs.Implemented CI/CD pipelines using Jenkins and Docker, resulting in 40% faster deployments and fewer production issues.Enhanced data governance using Unity Catalog, reducing data inconsistencies by 30% and improving lineage tracking.Designed scalable data models (Star/Snowflake schemas), improving query efficiency by 25% for reporting workloads.Optimized Cloud Services which reducing infrastructure costs by ~20%.Built reusable Python utilities and frameworks, improving developer productivity by 25% across teams.

Education

• University of Central Missouri

  Masters, Computer Science