Anurag P

About Anurag P

Email: a••••••••@gmail.com Phone No: 98••••••81 With 12 years of experience in Data Engineering, I specialize in architecting and implementing scalable, high-performance data pipelines using Azure, AWS, and Snowflake. My expertise spans cloud data migration, ETL/ELT development, big data processing, and real-time streaming** solutions. Microsoft Azure & Big Data Expertise - Handson experience with **Azure Data Factory (ADF), Azure Databricks, Azure Synapse Analytics, Azure Data Lake (ADLS), Azure SQL Database**, and **PolyBase** for efficient data integration- Designed and built **dynamic ADF pipelines**, leveraging advanced components like **Mapping Data Flows, Lookup, Join, Conditional Split, Window, and Sort**- Migrated **on-premises systems and SSIS packages** to Azure, ensuring seamless data transition and optimization. Snowflake & Data Warehousing - Proficient in Snowflake SQL, SnowSQL, Snowpipe, Snowpark, Materialized Views, Zero Copy Cloning, and Storage Integrations** for optimized cloud data warehousing- Designed **data models with Erwin** for seamless integration and scalability. AWS & Big Data Ecosystem - Implemented **data lakes, real-time streaming, and scalable solutions using AWS Glue, Redshift, Athena, S3, RDS, DynamoDB, Lambda**, and EMR- Developed real-time **streaming pipelines** using **Apache Flink, Spark Streaming, and Azure Stream Analytics. Power BI & Data Visualization - Designed and implemented Power BI dashboards and reports for real-time business intelligence- Expertise in Power BI star schema, snowflake schema, slicers, drill-throughs, measures, and calculated columns- Created interactive visualizations to support data-driven decision-making. Automation & Orchestration - Expertise in Azure DevOps, Apache Airflow, Oozie, and Flume to manage complex ETL workflows- Integrated DevOps practices for CI/CD, automation, and scalable deployments. I thrive in data-driven environments, delivering actionable insights and optimized data solutions for business growth. Passionate about cloud data engineering, performance tuning, and modernizing data architecture, I am always exploring the latest advancements in big data, AI, and cloud platforms. Let’s connect to discuss data engineering, cloud technologies, or innovative data solutions!#DataEngineering #Azure #Azure Data Factory# Azure Databricks #AWS #Snowflake #Power BI #BigData #ETL #CloudComputing #SQL #Databricks #DataPipelines #StreamingAnalytics

Experience

1. Azure Data Engineer

   Merck

   Mar 2022 — Present · VA, US

   Devised, executed, and managed data pipelines responsible for transferring and transforming data across diverse systems, databases, and applications, employing Azure data services like Azure Data Factory and Azure Databricks.• Crafted data cleansing and validation scripts on Azure Databricks utilizing Pyspark and Spark SQL.• Engineered and oversaw scalable data storage solutions within Azure, inclusive of Azure Data Lake Storage • Leveraged Azure Databricks for data transformation and exploration, integrating it with APIs for dynamic data retrieval.• Designed and implemented end-to-end data integration solutions using Azure Data Factory.• Developed ETL data pipelines for efficient and automated data movement across on-premises and cloud environments.• Utilized Azure Data Factory\'s data flow capabilities to transform and cleanse data at scale, improving data quality.• Integrated Azure Data Factory with various Azure services, including Azure Blob Storage and Azure SQL Database.• Produced Databricks notebooks featuring delta format tables and initiated the implementation of the lake house architecture.• Developed control flow structures within data pipelines for multiple datasets, contingent on the corresponding data lake dataset sizes.• Utilized Azure Data Factory and Azure Functions to amalgamate data from various RESTful APIs into a centralized Azure SQL database.• Optimized data models in Snowflake, including schema design, indexing, and partitioning for efficient query performance.• Implemented data loading strategies, utilizing Snowflake loading for large-scale datasets.

Education

• GRIET

  Bachelor's degree

  2008 — 2012