Prudhvi Raj

About Prudhvi Raj

A -can-do- attitude and a strong passion for technology and willingness to learn new skills. Skilled in Data Engineering. Interested in solving technical problems as well as helping people out on the way. Always seeking opportunities to learn more and contribute to something meaningful. Looking forward to challenging roles that allow me to constantly develop my skills and build on them.

Experience

1. Sr Data Engineer

   Verizon

   Dec 2018 — Present · Irving, TX, US

   Analyzed large and critical datasets using HDFS, HBase, MapReduce, Hive, Hive UDF, pyspark,Pig, Sqoop, Zookeeper and Spark.• Working on design and implementation of highly concurrent, scalable,distributed, and resilient application to process 200 TB+data including computation of BitSet and HuperLogLog using Akka, Kafka, Spark, Hbase.• Loaded and transformed large sets of structured, semi structured and unstructured data using Hadoop/Big Data concepts.• Performed Data transformations in HIVE and used partitions, buckets for performance improvements.• Developing Spark scripts, UDF\'s using both Spark DSL and Spark SQL query for data aggregation, querying, and writing data back into RDBMS through Sqoop.• Used Snowflake to separate compute usage from storage in their pricing structure.• Developed Spark code usingScala and Spark-SQL/Streaming for faster processing of data.• Ingested data into HDFS using SQOOP and scheduled an incremental load to HDFS.• Using Hive to analyze data ingested into HBase by using Hive-HBase integration and compute various metrics for reporting on the dashboard.• Developed python code to gather the data from HBase and designs the solutions to implement using pyspark.• Great hands on experience with pyspark for using spark libiries by using python scripting for data analysis.• Utilized Apache Spark with Python to develop and execute Big Data Analytics and Machine learning applications, executed machine Learning use cases under Spark ML and Mllib• Used to store relational database and works well with Tableau, ExcPYSPARKel and many other tools familiar to end users.• Data sources are extracted, transformed and loaded to generate CSV data files with Python programming and SQL queries.• Worked on NiFi data Pipeline to process large set of data and configured Lookup’s for Data Validation and Integrity.