Bhargavi C.

About Bhargavi C.

With over 10 years of professional experience in the IT industry, I specialize in Data Warehousing and Decision Support Systems. My expertise lies in implementing Full Lifecycle Data Warehousing Projects and working extensively with Hadoop/Big Data technologies for data storage, querying, processing, and analysis.I have hands-on experience with various Hadoop ecosystem tools such as HDFS, MapReduce, Hive, Sqoop, Oozie, Spark, Flume, and Kafka, including a deep understanding of components like HDFS, Job Tracker, Task Tracker, Name Node, Data Node, YARN, and MapReduce programming paradigm. Proficient in Python, Scala, SQL, and HiveQL languages, I excel in creating Hive tables, loading data using Sqoop, and processing it with HiveQL.Furthermore, I have expertise in RDD architecture and Spark operations, with knowledge of Spark Streaming for data ingestion. I\'ve utilized data ingestion tools like Sqoop across RDBMS, web applications, and HDFS, and have worked on AWS EC2, EMR, and S3 for cluster creation and data management.My experience extends to Spark Core, Spark RDD, Dataset, and Spark Deployment Architectures. I\'m skilled in stream processing with tools like Storm and Spark Streaming, and proficient in data cleansing using Spark Functions.Additionally, I\'ve worked with Azure Databricks cloud for data organization and visualization through notebooks and dashboards. I\'ve implemented Microsoft BI/Azure BI solutions like Azure Data Factory, Power BI, Azure Databricks, and Azure Analysis Services.Familiar with Spark Context, Spark SQL, Dataframe, and Dataset, I\'ve also worked with relational databases like Oracle, Teradata, MySQL, and Postgres. I have experience in importing/exporting data using Sqoop between HDFS and relational database systems, and working with various file formats like Parquet, Avro, ORC, JSON, and Flat files.I specialize in Dimensional Modeling, Data Migration, Data Cleansing, Data Profiling, and ETL Processes for data warehouses, and have hands-on experience in creating Hive tables, partitions, and buckets. I\'ve extended Hive Core functionality by writing UDFs for Data Analysis and have worked with various Hadoop distributions like Cloudera (CDH5) and Hortonworks.Proficient in Unix and Linux command line and shell scripting, I also possess knowledge of creating Data Pipelines using Kafka, Python, and Spark Streaming integration for end-to-end data processing solutions.

Experience

1. Lead Big Data Developer

   Marsh McLennan

   Mar 2022 — Present · NY, US

   Worked as a data engineer in part of team, using different ingestion methods loaded the data into raw transient.• Designed and Setup Enterprise Data Lake to provide support for various uses cases including analytics, processing, storing, and reporting of voluminous, rapidly changing data.• Responsible for maintaining quality reference data in source by performing operations such as cleaning, transformation and ensuring integrity in a relational environment by working closely with the stakeholders and solution architect.• Designed and developed security framework to provide fine grained access to objects in AWS Lambda.• Build complex ETL jobs that transform data visually with data flows or by using compute services Azure Databricks, and Azure SQL Database• Set up and worked on Kerberos authentication principals to establish secure network communication on cluster and testing of Hive and Pig to access cluster for new users.• Performed end-to-end Architecture and implementation assessment of various AWS services like EMR, Redshift and S3.• Used AWS EMR to transform and move large amounts of data into and out of other AWS data stores and databases, such as Amazon Simple Storage Services (Amazon S3) and DynamoDB.• Implemented AWS Step functions to automate and orchestrate the Amazon Sage maker related tasks such as publishing data to S3 deploying it for prediction.

Education

• Wilmington University

  Master's degree, Information Technology( Web Design)

  2016 — 2018

• JNTUH College of Engineering Hyderabad

  Bachelor's degree

  2008 — 2012