Bharath Kumar S

Data Engineer @UnitedHealth Group

Responsibilities:• Involved in creating data ingestion pipelines for collecting health care and providers data from various external sources like FTP Servers and S3 buckets.• Involved in migrating existing Teradata Datawarehouse to AWS S3 based data lakes.• Involved in migrating existing traditional ETL jobs to Spark and Hive Jobs on new cloud data lake.• Wrote complex spark applications for performing various de-normalization of the datasets and creating a unified data analytics layer for downstream teams.• Primarily responsible for fine-tuning long-running spark applications, writing custom spark udfs, troubleshooting failures, etc,• Involved in building a real-time pipeline using Kafka and Spark streaming for delivering event messages to downstream application team from an external rest-based application.• Involved in creating Hive scripts for performing adhoc data analysis required by the business teams.• Worked extensively on migrating on prem workloads to AWS Cloud.• Worked on utilizing AWS cloud services like S3, EMR, Redshift, Athena and Glue Metastore.• Used broadcast variables in spark, effective & efficient Joins, caching and other capabilities for data processing.• Involved in continuous Integration of applications using Jenkins.• Responsible for debugging and troubleshooting the running applications in production.Environment: AWS EMR, Spark, Hive, HDFS, Sqoop, Kafka, Oozie, HBase, Scala, MapReduce.