Rakesh G
- Role
- Data Engineer at Fannie Mae
- Location
- Reston, VA, US
- LinkedIn followers
- 500 followers
About Rakesh G
A results-driven and technically adept IT professional with around 6 of experience…
Experience
Data Engineer
Dec 2016 — Present
Worked on Enterprise Data lake(EDL) to lift & shift data from on-prem to AWS cloud using EMR, AWS Lambda(Python), Spark. Designed & developed generic scalable data ingestion framework to extract terabytes of data from tables and ingest the data to AWS S3 using multi-part upload. Also, developed on-prem & cloud(AWS) data reconciliation framework using hive & spark-sql utilizing the dataframe operations. Developed AWS lambdas using Python & step functions to orchestrate data pipelines. Implemented HIVE dynamic partitions to load incremental data. Setup SNS notifications and cloud watch events to monitor and alert data loads. Built Hive external tables on top on S3 dataset in parquet format for end users to query the data & analyzed data issues reported by end users. Used AWS Glue as Hive metastore for hive external tables. Created S3 event based AWS Lambda to load data to postgresSql AWS RDS service. Triaged data loss issues during the lift & shift process on EDL & on-prem. Tuned Spark-sql queries that caused broadcast issues. Developed daily and monthly ETL processes which were automated using custom UNIX shell scripts. Implemented parallel application by replicating the components datasets and processing modules into number of partitions. Worked on loading terabytes of data into Netezza DB using nzsql’s. Worked on XML components and generated xml files in enterprise wide common format and vended the files to various end consumers. Developed, tested and configured Autosys jils and shell and python scripts. Designed & developed reconciliation framework to compare source & target data and report as needed. Used IDQ to report data quality issues/anomalies to business operations. Built several custom in-house tools to monitor and enhance production operations. Created Health check jobs scripts to monitor the status of DB’s/ESB Adapters, ETL. Participated in Performance tuning reviews of the production environment.
Education
SRM IST Chennai
Bachelor's degree, Computer Science
Wilmington University
Master's degree, Computer and Information Systems Security/Information Assurance
Find verified contacts for anyone on LinkedIn
Unifers gives sales teams verified emails and direct dials, enriched profiles, and outreach that lands in the inbox.
Free plan included · No credit card required
This profile is compiled from publicly available professional sources. Unifers is not affiliated with or endorsed by LinkedIn. Request removal of this profile.