Jigar Modi

About Jigar Modi

With around 8 years of experience in Data Engineering and Data Analysis, I have expertise…

Experience

1. Data Engineer

   Equifax

   Jan 2022 — Present

   Analyze, design, and build Modern data solutions using Azure PaaS service to support visualization of data. Understand current Production state of application and determine the impact of new implementation on existing business processes.• Extract Transform and Load data from Sources Systems to Azure Data Storage services using a combination of Azure Data Factory, T-SQL, Spark SQL, and U-SQL Azure Data Lake Analytics. Data Ingestion to one or more Azure Services -(Azure Data Lake, Azure Storage, Azure SQL, Azure DW) and processing the data in Azure Databricks.• Developed HQL, Mappings, tables, external tables in Hive for analysis across different banners and worked on partitioning, optimization, compilation, and execution.• Created Pipelines in ADF using Linked Services/Datasets/Pipeline to perform ETL on data from different sources like Azure.• SQL, Blob storage, Azure SQL Data warehouse, write-back tool and backwards.• Developed Spark applications using PySpark and Spark-SQL for data extraction, transformation, and aggregation from multiple file formats for analyzing & transforming the data to uncover insights into the customer usage patterns.• Responsible for estimating the cluster size, monitoring, troubleshooting of the Spark data bricks cluster.• Design, develop and maintain SSIS packages to consume web services data for Analysis and Reporting of APS Decision Making System.• Gathered business requirements in meetings for successful implementation and POC (Proof-of-Concept) of Hadoop and its ecosystem.• Extensive role in fixing performance issues by tuning slow running queries and run-away queries using different methods, by evaluating statics, joins, indexes and code changes.• O Created alarms, alerts, notifications for Spark Jobs to email and slack group message job status and log in Cloud Watch.• Develop Databricks Python notebooks to Join, filter, pre-aggregate, and process the files stored in Azure data lake storage.

Education

• LDRP Institute of Technology & Research, Gujarat Technological University

  Bachelor's degree

• Saint Peter's University

  Master of Science - MS