Bhanu Prakash

About Bhanu Prakash

Motivated Data Engineer having 8+ years of professional experience in Data Engineering, Analytics, Data Modeling, Data Science, Data Architecture, Programming Analysis and Database Design of OLTP and OLAP systems with sound knowledge in Cloud Technologies (AWS, Azure).• Domain expertise in E-commerce, Healthcare, Enterprise Systems, Financial Sector. Skilled in working with cross-functional teams to design, develop and implement data-driven solutions with large data sets to solve complex business problems and executed data-driven action-oriented solutions with storytelling capabilities using agile methodologies.• Performed data engineering functions: data extraction, transformation, loading, and integration to support enterprise data infrastructures - data warehouse, operational data stores, and master data management.• Good experience in creating and designing data ingest pipelines using technologies such as Apache Storm- Kafka and Experienced in writing live Real-time Processing using Spark Streaming with Kafka as a data pipeline system.• Strong programming capability using Python and Hadoop framework utilizing Cloudera Hadoop Ecosystem projects (HDFS, Hadoop, Sqoop, Hive, HBase, Oozie, Impala, Zookeeper, etc.)• Practicing consulting on Snowflake Data Platform Solution Architecture, Design, Development and deployment focused to bring the data driven culture across the enterprises• Experienced in Agile development, specification, and performance tuning on Oracle databases by leveraging explain plans, tuning SQL queries, and excellent experience in writing Complex SQL queries to validate data movement between different layers in the data warehouse environment.• Expert in Building reports using SQL Server Reporting Services (SSRS, Crystal Reports, Power BI, and Business Objects.• Experience in designing a Terraform and deploying it in cloud deployment manager to spin up resources like cloud virtual networks, Compute Engines in public and private subnets along with AutoScaler in Google Cloud Platform.• Experience in Designing, Architecting and implementing scalable cloud-based web applications using AWS • Developed pipeline for POC to compare performance/efficiency while running pipeline using the AWS EMR Spark cluster.• Highly Skilled at Python coding using SQL, NumPy, Pandas, and Spark/Pyspark for Data Analysis and Model building, deploying, and operating highly available, scalable, and fault-tolerant systems using Amazon Web Services (AWS).

Experience

1. Sr Data Engineer

   Resmed

   Jun 2021 — Present · San Diego, CA, US

   Developed Qlik Bigdata jobs to load large volume of data into S3 data lake and then into Snowflake. • Developed snow pipes for continuous injection of data using event handler from AWS (S3 bucket). • Developed SnowSql scripts to deploy new objects and update changes into Snowflake. • Developed a Python script to integrate DDL changes between on-prem Talend warehouse and snowflake. • Working with AWS stack S3, EC2, EMR, Athena, Glue, Redshift, DynamoDB, RDS, Aurora, IAM, and Lambda. • Involved in migrating master-data from Hadoop to AWS. • Processing with Amazon EMR big data across a Hadoop cluster of virtual servers on Amazon Elastic Compute Cloud (EC2) and Amazon Simple Storage Service (S3). • Imported data from AWS S3 into Spark RDD, performed transformations and actions on RDD\'s. • Developed Apache Spark applications by using Scala for data processing from various streaming sources • Responsible for design and development of Spark SQL Scripts based on Functional Specifications • Worked on the large-scale Hadoop YARN cluster for distributed data processing and analysis using Spark, Hive, and Cassandra • Involved in converting Cassandra/Hive/SQL queries into Spark transformations using RDD\'s and Scala • Involved in loading data from rest endpoints to Kafka producers and transferring the data to Kafka brokers • Used Apache Kafka functionalities like distribution, partition, replicated commit log service for messaging • Exported the analyzed data to the relational databases using Sqoop for visualization and to generate reports for the BI team • Developed solutions to pre-process large sets of structured, semi-structured data, with different file formats like Text, Avro, Sequence, XML, JSON, and Parquet Environment: Snowflake, SnowSQL, Hadoop, MapReduce, HDFS, Yarn, Hive, Sqoop, Oozie, Spark, Scala, AWS, EC2, S3, EMR, Kafka, Pig, Linux, Shell Scripting