Hemanth Sai
- Role
- Senior Data Engineer at Nomura
- Location
- Overland Park, KS, US
- LinkedIn followers
- 500 followers
Experience
Senior Data Engineer
Aug 2022 — Present · NJ, US
Project Description: Designed and Developed efficient pipelines for a leading Investment Banking firm, incorporating various data feeds, and performing ETL (Extract, Transform, Load) processes, and then seamlessly storing the output in respective database tables on AWS (Amazon Web Services) cloud platform. Responsibilities: • Collaborated with Business Analysts to understand the business logics and ETL transformations that would be needed for the raw data from Upstream to Downstream. • Automated file arrival from remote Unix machines to AWS EC2 and transferring raw files to corresponding AWS S3 buckets using Unix scripting and AWS Data Sync. • Ran the entire file arrival process using Autosys. • Developed transformations using AWS Glue with Spark. • Leveraged DynamoDB to store ETL and Business Logics as dynamic content, ensuring high scalability and performance. • Utilised Glue tables to seamlessly read structured and semi-structured data, ensuring flexibility in data retrieval processes. Employed the boto3 Python SDK to access and interact with AWS components and services via the command-line interface (CLI), streamlining operations and workflows. • Proficiently worked with multiple file formats, including CSV, JSON, and XML, within AWS S3 buckets to optimise data storage and retrieval processes. • Validated ETL and Business Logics in AWS Athena to ensure mappings saved in DynamoDB were reflected in downstream tables. • Verified the accuracy of ETL and Business Logics stored in DynamoDB by validating mappings in AWS Athena, ensuring downstream tables reflected correct data. • Monitored all cloud activities using AWS CloudWatch. • Developed sophisticated orchestration solutions utilizing AWS Step Functions, demonstrating expertise in cloud-based workflow automation, and enhancing operational efficiency. • Automated to check the data quality using PySpark Scripts to check the partitions existence and their statistics.
Find verified contacts for anyone on LinkedIn
Unifers gives sales teams verified emails and direct dials, enriched profiles, and outreach that lands in the inbox.
Free plan included · No credit card required
This profile is compiled from publicly available professional sources. Unifers is not affiliated with or endorsed by LinkedIn. Request removal of this profile.