Praveen Kumar

About Praveen Kumar

With over 8 years of experience in Data Engineering, I specialize in designing, building, and optimizing scalable data solutions across diverse cloud platforms like GCP and Azure. I have honed my skills in implementing robust ETL pipelines, streamlining data workflows, and ensuring seamless integration for large-scale data systems. Key Expertise- Proficient in Python, PySpark, and SQL for advanced data manipulation and complex transformations- Hands-on experience with cloud-native technologies such as Azure Data Factory, and Snowflake for data processing and warehousing- Strong foundation in data modeling, schema design, and query optimization for both relational and NoSQL databases- Adept at building real-time data pipelines using Apache Kafka, Spark, Java, enabling low-latency, high-throughput analytics. Accomplishments- Led cloud migration projects, moving on-premises databases to Azure and GCP with minimal downtime, enhancing scalability and performance- Designed and implemented scalable data lakes and warehouses with tools like Azure Synapse Analytics, GCP BigQuery, and Snowflake, supporting business-critical analytics- Automated CI/CD pipelines with Terraform, Docker, and Kubernetes to streamline infrastructure deployment and application delivery- Built interactive dashboards in Tableau and Power BI, transforming raw data into actionable insights for stakeholders. Passion for Innovation: I thrive on solving complex data challenges, whether by integrating machine learning models into data pipelines, optimizing distributed systems, or mentoring junior engineers to foster a culture of innovation.

Experience

1. Senior Data Engineer

   Rite Aid

   Aug 2022 — Present · Philadelphia, PA, US

   I have extensive experience in designing and implementing real-time data streaming solutions using Google Pub/Sub to process large datasets with low-latency and high-throughput ingestion for analytics. I have integrated diverse data sources into Google BigQuery, leveraging both batch and streaming ingestion techniques to enable real-time reporting. My expertise includes developing and maintaining ETL pipelines using Python and PySpark, automating data extraction, transformation, and loading into centralized data warehouses and data lakes. I have optimized PySpark clusters, resolved production issues such as data purging, data pruning, shuffle partitioning, and predicate pushdowns, and successfully implemented Spark SQL and DataFrame APIs for large-scale data processing and analytics. Additionally, I have implemented real-time data streaming with Apache Kafka and Flume, managed distributed coordination with Apache Zookeeper, and utilized Hadoop tools like HDFS, Hive, and Oozie for data processing and scheduling, while integrating machine learning models using Databricks. I have also managed Kafka topics and partitions for efficient message brokering and administered Zookeeper ensembles for distributed synchronization. My backend expertise includes developing high-performance microservices using Spring Boot, implementing RESTful APIs, and integrating applications with databases like PostgreSQL, MySQL, and MongoDB, message brokers like Kafka, and caching systems. I have also worked extensively with Google Dataform to automate data transformations into BigQuery and optimized its performance through SQL partitioning and clustering strategies, ensuring faster query processing and cost efficiency.

Education

• JNTUH College of Engineering Hyderabad

  Bachelor of Technology - BTech

  2011 — 2015