Surya Teja Nallamothu

Senior Data Engineer @AgFirst Farm Credit Bank

Columbia, SC, US

Imported data from various sources like HDFS/HBase into Spark RDD and developed a data pipeline using Kafka and Storm to store data into HDFS.• Created YAML files for each data source, including the definition of Glue tables for stack creation.• Designed and setup an Enterprise Data Lake to provide support for various use cases, including storage, processing, analytics, and reporting of voluminous, rapidly changing data by using various AWS services.• Used various AWS services including S3, EC2, AWS Glue, Athena, Redshift, EMR, SNS, SQS. • Extracted data from multiple source systems S3, RedShift, RDS, and created Multiple tables/databases in Glue Catalog by creating Glue Crawlers.• Created AWS Glue Crawlers for crawling the source data in S3 and RDS.• Created Multiple Glue ETL jobs in Glue Studio and then Processed the data by using different transformations and then loaded it into S3, Redshift, and RDS.• Developed Glue ETL jobs in Glue Studio, applied different transformations, and loaded data into S3, Redshift, and RDS.• Created multiple recipes in Glue Data Brew and then used them in various Glue ETL jobs.• Designed and developed ETL processes in AWS Glue to migrate data from external sources into AWS Redshift.

JNTUH College of Engineering Hyderabad

Bachelor's degree