Venkata Jeevan

About Venkata Jeevan

Having 10+ years of practical Data Engineer with 8+ years in Big Data/Hadoop/PySprak technology development.• Experience in developing applications that perform large scale distributed data processing using big data ecosystem tools like HDFS, YARN, Sqoop, Flume, Kafka, MapReduce, Pig, Hive, Sprak, PySprak SQL, PySprak Streaming, HBase, Cassandra, MongoDB, Mahout, Oozie, and AWS.• Good working knowledge of Amazon Web Services (AWS) Cloud Platform which includes services like EC2, S3, VPC, ELB, IAM, DynamoDB, Cloud Front, Cloud Watch, Route 53, Elastic Beanstalk (EBS), Auto Scaling, Security Groups, EC2 Container Service (ECS), Code Commit, Code Pipeline, Code Build, Code Deploy, Dynamo DB, Auto Scaling, Security Groups, Red shift, CloudWatch, CloudFormation, CloudTrail, Ops Works, Kinesis, IAM, SQS, SNS, SES.• Good functional experience in using various Hadoop distributions like Hortonworks, Cloudera, and EMR• In - depth understanding of SnowFlake cloud technology.• Good understanding in using data ingestion tools- such as Kafka, Sqoop and Flume.• Experienced in performing in-memory real time data processing using Apache Sprak.• Good experience in developing multiple Kafka Producers and Consumers as per business requirements.• Extensively worked on PySprak components like PySprak SQL and PySprak Streaming.• Configured PySprak Streaming to receive real time data from Kafka and store the stream data to HDFS and process it using PySprak.• Developed quality code adhering to coding standards and best practices.• Experience in migrating map reduce programs into PySprak RDD transformations, actions to improve performance.• Involved in collecting and aggregating large amounts of log data using Apache Flume and staging data in HDFS for further analysis.• Extensive working experience with data warehousing technologies such as HIVE.• Expertise in writing Hive and Pig queries for data analysis to meet the business requirement.• Extensively worked on Hive and Sqoop for sourcing and transformations.• Extensive work experience in creating UDFs, UDAFs in Pig and Hive.• Good experience in using Impala for data analysis.• Experience on NoSQL databases such as HBase, Cassandra, MongoDB, and DynamoDB.• Implemented CRUD operations using CQL on top of Cassandra file system.• Experience in creating data-models for client’s transactional logs, analyzed the data from Cassandra tables for quick searching, sorting, and grouping using the Cassandra Query Language (CQL).

Experience

1. Senior Data Engineer

   Fidelity Investments

   Jun 2022 — Present · Atlanta, GA, US

   Worked with Data Engineers, Data Architects, to define back-end requirements for data products (aggregations, materialized views, tables – visualization).• Leveraged Google Cloud Platform Services to process and manage the data from streaming and file-based sources• Worked on GCP services like Compute engine, cloud load balancing, cloud storage, cloud SQL, stack driver monitoring and cloud deployment manager.• Experienced in Converting existing AWS Infrastructure to Server less architecture (AWS Lambda, Kinesis), deploying via Terraform and AWS Cloud Formation templates.• Architect and design serverless application CI/CD by using AWS Serverless (Lambda) application model• Experienced with machine learning algorithm such as logistic regression, random forest, XGboost, KNN, SVM, neural network, linear regression, lasso regression and k – means.• Creating Google Cloud Storage buckets, maintaining and utilizing the policy management of these buckets and used GCS coldline for storage and backup on Google cloud• Built the machine learning model include: SVM, random forest, XGboost to score and identify the potential new business case with Python Scikit-learn.• Responsible for importing data from PostgreSQL to HDFS, HIVE using SQOOP. Experienced in migrating HiveQL into Impala to minimize query response time.• Implemented Avro and parquet data formats for apache Hive computations to handle custom business requirements.• Developed Spark applications using Scala and Spark-SQL for data extraction, transformation, and aggregation from multiple file formats. Using Kafka and integrating with the Spark Streaming. Developed data analysis tools using SQL and Python code.• Developed Mappings using Transformations like Expression, Filter, Joiner and Lookups for better data messaging and to migrate clean and consistent data

Education

• National Institute of Technology Karnataka

  B.Tech, Computer Science