Niranjan Reddy

Data Engineer @Mastercard

St Louis, MO, US

Designed high-throughput pipelines using Spark Streaming, which processed over two million data every day with ease and at peak efficiency.•Improved ETL procedures using Apache Airflow, increasing processing effectiveness by 20% and enhancing workflow-wide system dependability.•Optimized MapReduce, HDFS, and Hive setups to improve Hadoop operations and reduce reporting delays by 60%• Created and enhanced data pipelines using Apache Spark and AWS EMR, increasing data processing effectiveness by 35%.•Used Apache Airflow and AWS Glue to manage and automate ETL processes, guaranteeing accurate and timely data delivery.•Established AWS S3 data lake topologies that improved data scalability and made sophisticated analytics possible.•Developed and refined Redshift schemas, which improved cross-departmental reporting and cut query speeds by 30%.

University of Illinois Springfield

3.87