Subhash Reddy

About Subhash Reddy

I am a Data Engineer focused on building scalable, reliable, and high-performance data systems. I work across the full data lifecycle—ingestion, transformation, modeling, orchestration, governance, and analytics enablement. My expertise spans modern big data and cloud technologies, including Apache Spark, Airflow, Kafka, and distributed data processing using Python, SQL, and Scala.I specialize in architecting ETL/ELT pipelines, real-time streaming solutions, and cloud data platforms across AWS, GCP, and Azure. I enjoy designing efficient data models, optimizing query and job performance, and implementing strong data governance practices such as lineage, cataloging, masking, and access control.I am passionate about automation, reliability, and observability—whether through Airflow, CI/CD pipelines, Terraform, monitoring tools, or reusable data quality frameworks. I focus on delivering clean, trusted, and well-structured data that supports analytics, BI, machine learning, and large-scale decision-making.I continuously stay updated with modern data engineering tools and cloud technologies and enjoy solving complex data challenges with a mix of technical depth, structured thinking, and a strong commitment to quality.

Experience

1. Data Engineer

   PayPal

   Oct 2024 — Present · US

   Currently building and optimizing distributed data pipelines using Spark, Scala, and Python on Hadoop and GCP. Developing real-time feature engineering workflows with PySpark, Kafka, and S3, and orchestrating ETL processes through custom Airflow DAGs. Designing scalable BigQuery data models, implementing data security with masking and encryption, and creating reusable ingestion frameworks. Enhancing pipeline reliability with monitoring and alerting, strengthening RBAC governance, automating schema evolution, and tuning Spark jobs for high-performance processing.

Education

• St. Francis College

  Master's degree