Shilpa Manjunath Murthy
Senior Data Engineer @Fasken
Signup · Get unlimited contacts
WORK HISTORY
Senior Data Engineer @Fasken
Toronto, ON, CA
Led the migration from Azure Synapse to Microsoft Fabric, designed and developed PySpark /Spark SQL Notebooks using Medallion architecture adopting data reliability, scalability for seamless data transition and platform modernization. Designed and implemented complex Data Factory pipelines, utilizing activities such as Lookup, For Each, and Metadata-driven Copy to ingest SQL tables and stored procedures into Lakehouse environments. Optimized incremental data loading strategies for terabyte-scale datasets, resulting in a 30% improvement in data ingestion performance, tuned warehouse performance through partitioning, indexing, and workload management, using Spark SQL - improving dashboard load times by 40%. Developed real time data integrity dashboards, created Event house, KQL database – developed KQL queries to read Semantic Model logs, enabled proactive monitoring and achieved 100% improvement in data quality metrics. Developed and enforced data governance policies; implemented Row Level Security (RLS), column masking, and RBAC to secure Finance and Intapp Time system Engineered Gen1 and Gen2 Data Flows, Data factory and notebook pipelines with Deduplication, Idempotent, incremental loading, schema evolution, SCD Type2 data patterns to integrate diverse data sources including SharePoint folders with alerts and monitoring features, financial application data from SQL, Rest API calls. Enhanced SQL and stored procedure performance during SSRS migration, reducing query latency and improving report responsiveness. Applied dimensional data modeling best practices, optimized Spark SQL, and built performant historical views boosting Power BI report efficiency and Delta Lake query performance.
EDUCATION
Malnad College of Engineering
Bachelor's degree, Electrical and Electronics Engineering
Simplilearn Alumni
Masters in Data Scientist, SAS, Python and Tableau Big data programming
ABOUT SHILPA MANJUNATH MURTHY
15 years of IT experience- including 7 years of Data Engineering Experience in designing, development, architecting, ETL Pipelinesusing ADF. Handing simple to complex pipelines meta data driven architectureFocused on : Building Scalable, Resilient Pipelines both low -code to high code use cases, using PySpark notebooks, Data Factory, Dataflow Gen2 - pipeline activities. Optimizing SQL, Spark, delta tables for distributed processing using Medallion architecture.Designed and developed Fabric Semantic models on both import and direct query. using Star Schema Data modelling techniques, SCD Type 1, Type 2. Designed Power BI dashboard for data integration checks that built on Fabric Semantic modelsMonitoring data quality, data integrity check to ensure 100% data quality.Data Governance: Implemented Security patterns like RLS, CLS, column masking techniques on Azure SQL, Fabric Data warehouseData Validations: Sanity checks / Reconciliations are performed on every new semantic models before release to Production.Cost / Resource Optimization on Fabric/Power BI Workload Awards: 1. Employee of month Award : 2022 Feb Utthunga Technology2. 5 Star Recognition Award - August 2024 - Fasken LLP.Certification : Microsoft Data Engineer Associate DP-700
This profile is compiled from publicly available professional sources. Unifers is not affiliated with or endorsed by LinkedIn. Request removal of this profile.