Modou Gaye

Senior Data Engineer @Keysight Technologies

Created new ETL pipelines, transformations using PySpark Manage data warehouse; code development, testing and solution deployment through the environments. Manage a Data Lake in AWS and providing a competitive advantage to customers by generating insights from large data sets. Manage EMR pyspark data pipelines that use Athena as a data source to transform the data and output it to RDS which is then used as a data source on QuickSight for insights and visualisation. Productionising an ETL based on Jupyter Notebook to an EMR data pipeline. Manage RDS databases; code development, testing and solution deployment through the environments. Creating workflows using AWS EMR/Glue and Lambda to automate tasks. Transformed an ETL written in Pandas to PySpark Motivating in curiosity, design and delivery of product enhancements and new innovative solutions. Helping to determine the best technologies and strategies to use.