Sai Prabhath

Experience

1. Lead Java Python Developer Aws Data Engineer

   Apple

   Aug 2020 — Present

   Worked on building PySpark algorithms for different aggregations of data based on the specifications. Involved with performance and process enhancement of the PySpark framework. Developed integration checks around the PySpark framework for Processing of large datasets. Worked on migration of PySpark framework into AWS Glue for enhanced processing. Wrote various automation scripts for automation of data processing on AWS Glue. Worked on Scripting CloudFormation Templates to auto provisioning of resources. Worked of various Quality control checks for data processing using Spark SQL. Build a data processing algorithm for incremental processing of data that’s aggregated each week. Involved in the complete software development lifecycle (SDLC) to develop the application. Worked on data transformation like cleaning, partitioning of data for enhanced processing of data. Worked on setting up of SEE5 data mining server on AWS for pattern finding in the data Extensively worked on AWS Athena database to provide various client reports. Involved with migration of AI based project from Google Cloud Platform into AWS. Build Schedule based AWS Lambdas for automatic build and run of data processing pipelines Worked on building a standalone UI tool using wxPython for creating custom requirement JSON templets to feed into AWS Glue for ETL processing of data. Worked on setting of automated loading of data into SQL database using AWS Glue and Step Functions. Worked on integrating different AWS components like EC2 and Lambdas to work with AWS Athena. Scheduled an ETL process through S3 Event Trigger to load the data from S3 to tables in AWS Redshift using AWS lambda.