Kiran Kumar

Experience

1. Data Engineer

   Highmark Health

   Aug 2020 — Present · Highmark Health, PA, US

   Designed, and build scalable distributed data solutions using with AWS & planned migration plan for existing on-premises Cloudera Hadoop distribution to AWS based on business requirement.Implemented simple to complex transformation on Streaming Data and Datasets.Worked on analyzing Hadoop cluster and different big data analytic tools including Hive, Spark, Python, Sqoop, Oozie.Worked with legacy on-premises VMs based on UNIX distributions. Worked with batch data as well as 3rd Party data through FTP.Worked with HDFS that stores distributed data. Configured Oozie along with Sqoop to ingest relational data.Wrote YML files for Kafka Producers for ingesting streaming data. Assigned partitions to customers. Installed Kafka manager for consumer logs and for monitoring Kafka Metrics.Developed Scala scripts using both Data frames/SQL/Data sets and RDD/MapReduce in Spark for Data Aggregation, queries and writing data back into OLTP system through Sqoop.Developed PySpark Streaming by consuming static and streaming data from different sources.Used Spark Streaming to stream data from external sources using Kafka service and responsible for migrating the code base from Cloudera Platform to Amazon EMR and evaluated Amazon eco systems components like RedShift, Dynamo DB.Having good knowledge in NOSQL databases like Dynamo DB, Mongo DB, Cassandra. Setting up and administering DNS system in AWS cloud using Route53.Performed configuration, deployment, and support of cloud services in Amazon Web Services (AWS).

Education

• GITAM Deemed University

  Bachelor of Technology

  2015