Swapnil Sonawane

About Swapnil Sonawane

Roles and responsibilities :• Use S3 to store objects and perform analysis on S3 using Athena and Redshift spectrum.• Worked on HDP and Cloudera data platform for big data processing and transformation.• Create data tools for analytics and data scientist team members that assist them in building and optimizing our product.• Used Dynamo DB to store, process and analyze retail customer data.• Used Kubernetes container-orchestration system for automating application deployment, scaling, and management.• Create and maintain optimal data pipeline architecture using AWS Glue.• AWS redshift as warehouse for storing historical data and perform analysis on data.• Writing Spark code for data migration from legacy system to Data Lake.• Writing shell script for pre-data processing and configuration for triggering load job.• Worked on EMR AWS for big data processing and analysis.• Designed and implemented Streaming Application using AWS kinesis.• Created & worked on Sqoop jobs with incremental load to populate Hive External tables.• Use of Partitions, Bucketing concepts in Hive and designed both Managed and External tables in Hive to optimize performance.• Involved in processing of audit data from an FTP location and pushing into HDFS and process the data using Map Reduce.

Experience

1. Senior Data Engineer

   Marsh McLennan

   Mar 2024 — Present · Mumbai, IN

Education

• Somaiya Vidyavihar University

  HSC

  2005 — 2007

• Thirani Highschool

  SSC

  1996 — 2005

• PDEA College Of Engineering

  Bachelor of Engineering - BE

  2007 — 2011

• Sinhgad College of Engineering

  Master's degree

Skills

• Java
• Xml
• Sql
• Software Development
• Mysql
• C
• Oracle
• Requirements Analysis
• Core Java
• Testing