Prashant Mehta
Azure Data Engineer @Nutanix
Signup · Get unlimited contacts
WORK HISTORY
Azure Data Engineer @Nutanix
Toronto, ON, CA
Experienced Azure Data Engineer at Nutanix using various Microsoft Azure services.• Designed and developed scalable ELT pipelines using Azure Data Factory to ingest and process large volumes of Energy Sector data from various sources.• Utilized various Optimization techniques to improve performance.• Developed and enforced RBAC policies using Unity Catalog in Azure Databricks to effectively manage and secure access to data assets, ensuring data governance.• Used Medallion Architecture to move data between bronze, silver and gold layers.• Implemented Databricks Auto Loader to streamline data movement across pipeline layers.• Creating fact and dimensions tables and star and Snowflex schema for Data Modeling.• Designed a Proof of Concept leveraging Delta Live Tables to automate ETL processes, ensuring efficient data quality validation and ensuring robust data integrity and flexible schema adjustments.• Efficient in SQL and Python programming for data manipulation, analysis, and integration within Azure data solutions.• Utilized email alerts and logs to monitor and obtain critical information about data processing and system performance.
EDUCATION
Algoma University
Postgraduate Degree
ABOUT PRASHANT MEHTA
A highly skilled Azure Data Engineer with 6 years of hands-on experience leveraging a wide range of Azure tools and services for data storage, integration, analytics, processing, and management. Successfully contributed to three major projects in the Energy, Textile and manufacturing sectors, preparing data for data analysts and data science teams. Applied optimization techniques (Optimize, Z-order, Vacuum) to reduce processing time by 30%. Implemented Unity Catalog for enhanced data governance, ensuring secure and organized access across the organization. Utilized Databricks Auto Loader to automate data transfers between layers, significantly improving pipeline efficiency and reliability. Developed a proof of concept using Delta Live Tables to automate ETL processes, ensuring seamless data quality checks and schema evolution. Designed, developed, and maintained ELT pipelines with Azure Data Factory (ADF) to automate data integration from diverse sources including S3, SAP, file-based sources, and on-premises data. Applied PySpark for data transformations on Databricks, optimizing Spark performance to enhance processing efficiency. Implemented Medallion Architecture for streamlined data processing across bronze, silver, and gold layers, ensuring optimal data quality, reliability, and performance. Experienced in managing tickets through Jira and Azure DevOps Boards. Proficient in visualization tools such as Power BI and Matplotlib for analysis and reporting. Strong communication and collaboration skills foster a data-driven culture, ensuring effective teamwork and decisionmaking across the organization.
This profile is compiled from publicly available professional sources. Unifers is not affiliated with or endorsed by LinkedIn. Request removal of this profile.