Navya Sharma

Data Engineer @TD

Toronto, ON, CA

Contributed to a large-scale data migration and cloud modernization initiative for RBC\'s Wealth Management division, moving over 120TB of structured (SQL Server, Oracle) and unstructured (Excel, PDFs) data into Azure.• Developed metadata-driven ETL pipelines using Azure Data Factory (ADF) and Databricks (PySpark) for ingestion, cleansing, and transformation.• Built scalable Bronze, Silver, and Gold layers following Medallion Architecture principles using Delta Lake and Synapse Analytics.• Automated incremental ingestion with Delta Merge and Change Data Capture (CDC), improving data refresh times by 25%.• Implemented optimized PySpark transformations using AQE, broadcast joins, Z-Ordering, and Auto Optimize to enhance performance.• Created Live Tables in Databricks to support near-real-time data analytics workflows.• Developed data validation and feature engineering modules using PySpark, Pandas, and NumPy libraries.• Built ADF pipelines with dynamic parameters and integrated Azure Logic Apps for pipeline success/failure alerts.• Applied data governance policies including GDPR compliance, PII masking, and lineage tracking through Azure Purview.• Implemented Role-Based Access Control (RBAC) and secure credential management using Azure Key Vault.• Enabled predictive analytics pipelines by creating curated features for Azure Machine Learning batch scoring.• Supported CI/CD deployments for ADF, Databricks, and Synapse assets using Azure DevOps.• Participated in Agile Scrum ceremonies and internal knowledge-sharing sessions to drive best practices in cloud-native data engineering.