Raviteja Karnati

Data Engineer @Hyundai Motor America

Prosper, TX, US

Working on package configuration to set up automated ETL load processing for one-time and incremental Data Loads. •Wrote complex ETL code to extract the data from Source EPIC Clarity system and designed and developed ETL mappings to implement data transformation and loading as per the Business requirement. •Automation of the ETL code end-end and make sure all the data is updated on a daily basis. •Participate in development, defect resolution on existing ETL processes. •Maintain and enhance ETL code, work with the QA and DBA team to fix performance issues. •Working on DW Projects in an Agile based Development Environment. •Worked with Offshore resources on multiple projects. •Created numerous pipelines to get the data from disparate source systems by using different Activities like Move & Transform, Copy, filter, for each, etc. •Develop required ETL processes, performance tune ETL programs/scripts. •Maintain and enhance ETL code, work with the QA and DBA team to fix performance issues. •Automated jobs using different triggers like Events, Schedules, and Tumbling. •Created, and provisioned different clusters, notebooks, jobs, and autoscaling. •Created infrastructure for optimal extraction, transformation, and loading of data from a wide variety of data sources. •Identify and resolve all problems and ensure appropriate tracking of issues on the same. •Analyze all database statistics and recommend changes if required. •Monitor all internal and external data feeds for systems. •Designing and implementing highly performant data ingestion pipelines from multiple sources using Apache Spark and/or Databricks Delta Lake. •Orchestrated data integration pipelines using various Activities like Get Metadata, Lookup, For Each,Wait, Execute Pipeline, Set Variable, Filter, until, etc. •Understand the data model document and analyze it to identify the source systems and transformations required to ingest the data into Data Lake.