Hemanth Reddy

About Hemanth Reddy

An Experienced and Motivated professional currently working as Senior Data Engineer.• Around 10+ years of IT experience as a Data Engineer and worked in the Analysis, design, development, testing and implementation of ETL Informatica developer & Business Intelligence solutions using Data Warehouse/ Data Mart Design, ETL, SQL SERVER, MSBI, Power BI and Azure Data Engineer.• Solid experience in using Hadoop ecosystem components like Hadoop, Hive, Pig, Sqoop, HBase, Cassandra, Spark, Spark Streaming, Spark SQL, Oozie, Zookeeper, Kafka, Flume, MapReduce framework, Yarn, Scala and Hue.• Experience in Data ingestion to one or more Azure services (Azure Data Lake, Azure Storage, Azure SQL, Azure DW) and processing the data in Azure Databricks.• Designing and Developed Oracle PL/SQL and Shell Scripts, performed Data Import/Export, Data Conversions, and Data Cleansing. • Strong experience in Software Development Life Cycle (SDLC) including Requirements Analysis, Design Specification, and Testing as per Cycle in both Waterfall and Agile methodologies. • Ability to work effectively in cross-functional team environments, excellent communication, and interpersonal skills.Contact Information :Email - h••••••••@gmail.comPhone -(913) 229

Experience

1. Sr Data Engineer

   Nomura

   Feb 2023 — Present · NJ, US

   Project Description: Designed and Developed efficient pipelines for a leading Investment Banking firm, incorporating various data feeds, and performing ETL (Extract, Transform, Load) processes, and then seamlessly storing the output in respective database tables on AWS (Amazon Web Services) cloud platform.Responsibilities: • Collaborated with Business Analysts to understand the business logics and ETL transformations that would be needed for the raw data from Upstream to Downstream. • Automated file arrival from remote Unix machines to AWS EC2 and transferring raw files to corresponding AWS S3 buckets using Unix scripting and AWS Data Sync. • Ran the entire file arrival process using Autosys. • Developed transformations using AWS Glue with Spark. • Leveraged DynamoDB to store ETL and Business Logics as dynamic content, ensuring high scalability and performance. • Utilised Glue tables to seamlessly read structured and semi-structured data, ensuring flexibility in data retrieval processes. Employed the boto3 Python SDK to access and interact with AWS components and services via the command-line interface (CLI), streamlining operations and workflows. • Proficiently worked with multiple file formats, including CSV, JSON, and XML, within AWS S3 buckets to optimise data storage and retrieval processes. • Validated ETL and Business Logics in AWS Athena to ensure mappings saved in DynamoDB were reflected in downstream tables.• Verified the accuracy of ETL and Business Logics stored in DynamoDB by validating mappings in AWS Athena, ensuring downstream tables reflected correct data. • Monitored all cloud activities using AWS CloudWatch.• Developed sophisticated orchestration solutions utilizing AWS Step Functions, demonstrating expertise in cloud-based workflow automation, and enhancing operational efficiency.• Automated to check the data quality using PySpark Scripts to check the partitions existence and their statistics.

Education

• JNTUH College of Engineering Hyderabad

  Bachelor of Technology - BTech, Computer Science