Nagesh Pawankar
Azure Data Engineer @Tata Consultancy Services
Signup · Get unlimited contacts
WORK HISTORY
Azure Data Engineer @Tata Consultancy Services
Nagpur, IN
Designed and implemented a scalable Azure Databricks data engineering framework to ingest, validate, and process healthcare and IoT data from hospitals, APIs, and connected medical devices- Built an end-to-end Azure-based data platform for processing patient vitals and lab test data, supporting high-volume, multi-source healthcare datasets- Developed and optimized PySpark ETL pipelines in Azure Databricks to cleanse, validate, and transform structured and semi-structured data (JSON, CSV, XLSX; 1–3 GB files)- Implemented Delta Lake tables following Medallion Architecture (Bronze, Silver, Gold) to enable incremental data loads, schema evolution, and performant analytics- Enforced healthcare-specific data quality rules, including missing-value checks and medical range validations, with audit trails and metadata management stored in Azure SQL Database- Automated data ingestion and workflow orchestration using Azure Data Factory (ADF), integrating Databricks notebooks for scheduled and reliable batch processing- Collaborated with cross-functional teams to deliver a cloud-native data engineering solution ensuring data accuracy, scalability, and compliance for healthcare analytics and reporting.
EDUCATION
GOVERNMENT COLLEGE OF ENGINEERING, JALGAON.
B. Tech, Electrical Engineering
ABOUT NAGESH PAWANKAR
Azure Data Engineer with ~4 years of experience designing and building scalable data platforms on Microsoft Azure, with a strong focus on Azure Databricks, PySpark, and Azure Data Factory.I have hands-on experience building end-to-end data pipelines to ingest, validate, and transform large-scale healthcare, clinical, and genomics datasets. My work includes processing large scientific instrument output files (1–3 GB per file), implementing robust data quality checks, and delivering analytics-ready datasets for downstream reporting and decision-making.I have implemented Medallion architecture (Bronze, Silver, Gold) using ADLS Gen2, Delta Lake, and Parquet to support incremental processing, schema enforcement, and optimized querying. I regularly work with PySpark and Python to standardize raw data, build aggregation logic, and curate Gold-layer datasets consumed by Power BI dashboards.My experience also includes automating data ingestion workflows using Azure Data Factory, orchestrating Databricks notebooks for reliable end-to-end processing, and implementing monitoring and alerting using Azure Monitor to improve pipeline reliability. I have worked with Azure DevOps to version, deploy, and manage Databricks notebooks through CI/CD pipelines across environments.I enjoy building reliable, scalable, and well-monitored data platforms and collaborating with analytics and business teams to deliver high-quality, trustworthy data.Core skills:Python | SQL | PySpark | Azure Databricks | Azure Data Factory | ADLS Gen2 | Delta Lake | Python | Azure SQL Database | Azure DevOps | Data Quality & Validation | Medallion Architecture
This profile is compiled from publicly available professional sources. Unifers is not affiliated with or endorsed by LinkedIn. Request removal of this profile.