Shivakrishna Bera

About Shivakrishna Bera

Over 11 years of overall experience in data analysis, data modelling and implementation of enterprise class systems spanning Big Data, Data Integration, Object Oriented programming and Advanced Analytics.Analyzed the Cassandra/SQL scripts and designed the solution to implement using Scala.Expertise in Big Data Technologies and Hadoop Ecosystem tools like Flume, Sqoop, Hbase, ZooKeeper, Oozie, MapReduce, Hive, PIG and YARN.Extensive experience in Scala working with the fraud detection models and machine learning.Experience in using Pyspark, the Spark Python API for extracting data from cloud infrastructure and ultimately storing the data in a persistent manner.Good experience in using the AWS ecosystem services EC2, S3, Lambda, IAM, Glue and Athena.Extracted and updated the data into MONGODB using MONGO import and export command line utility interface.Developed Collections in MongoDB and performed aggregations on the collections.Hands on experience in installation, configuration, management and deployment of Big Data solutions and the underlying infrastructure of Hadoop Cluster using Cloudera and Horton works distributions.Experience in different Hadoop distributions like Cloudera (Cloudera distribution CDH3, 4 and 5), Horton works Distributions (HDP) and AWS Elastic Map Reduce (EMR).Responsible for setting up the Docker, Elastic search as part of the Streaming data purpose.Extensive experience in Spark/Scala, Expert in creating Pig and Hive UDFs using Java to analyze the data efficiently.Experience in creating Map Reduce codes in Java as per the business requirements.Developed Map reduce jobs to automate the transfer of data from Hbase.Strong experience in working with Elastic Map Reduce and setting up environments on Amazon AWS.11

Experience

1. Data Engineer

   Capital One

   Oct 2023 — Present · Richmond, VA, US

Education

• Kakatiya University

  Bachelors of Computer Science, Computer Science

  2006 — 2010