Shweta Terkar

About Shweta Terkar

I\'m a seasoned Data Engineer with over 7 years of experience in developing robust data solutions across multiple platforms including AWS, Azure, and on-premises environments. Currently enhancing data workflows at Amazon in Seattle, I specialize in optimizing ETL processes, improving data ingestion, and designing efficient data pipelines to empower organizations with timely insights.At McKesson, I work as a Senior Data Engineer building scalable healthcare data platforms focused on molecular and clinical data. I design and implement end-to-end Medallion architecture (Bronze → Silver → Gold) in Azure Databricks using Spark Declarative Pipelines and Delta Live Tables, supporting both batch and incremental processing patterns using serverless compute My role involves developing Python-driven schema mapping and transformation frameworks to standardize complex, vendor-specific molecular and clinical datasets — including biomarkers, genomic variants, and observation data — into unified, analytics-ready models. I focus heavily on data quality, validation, traceability, and production-grade ingestion patterns to ensure reliable downstream analytics and research use.I have built and optimized materialized snapshot layers and spark declarative pipeline-managed datasets with strong dependency tracking, atomic updates, and orchestration-ready outputs. I also integrate data workflows with Azure Blob Storage for scalable cloud ingestion and curated data delivery.Additionally, I’ve implemented advanced search capabilities using Elasticsearch, including keyword/text search, query rules, vector search, and hybrid search, improving data discovery and retrieval performance for data-driven applications.At Amazon, I spearheaded projects using advanced AWS services and Python automation to handle high-volume, multi-regional data operations efficiently. Previously at Principal Global Services and Cognizant Technology Solutions, I led initiatives to improve data validation, automate report distribution, and significantly enhance system performance through strategic data modeling and SQL optimization.I thrive on solving complex data challenges and am passionate about leveraging technology to improve business outcomes through strategic data analysis and solution design.Let\'s connect to discuss big data innovations and opportunities!

Experience

1. Senior Data Engineer

   McKesson

   Jan 2025 — Present

   Designed and implemented Medallion architecture (Bronze → Silver → Gold) in Azure Databricks using Spark Declarative Pipelines / Delta Live Tables, supporting batch + incremental processing patterns. • Built and optimized Materialized Tables (batch snapshot layer) and live pipeline-managed datasets with reliable dependency tracking, atomic updates, and orchestration-ready outputs. • Developed Python-heavy schema mapping pipelines for molecular healthcare data, implementing dynamic mappings/transformations and validations to standardize vendor-specific structures into unified analytic schema. • Processed healthcare / molecular data (clinical observations, biomarkers, genomic-related fields) ensuring production-grade data quality, traceability, and repeatable ingestion patterns. • Integrated data workflows with Azure Blob Storage for source ingestion and curated outputs, enabling scalable cloud storage + downstream consumption.* Implemented Elasticsearch search capabilities including keyword/text search, query rules, vector search using Kibana, and hybrid search for improved retrieval quality.

Education

• Smt Kashibai Navale college of engineering Pune

  Bachelor of Engineering - BE, Computer Science

  2013 — 2017

• Grand Valley State University

  Master's degree, Computer Science, major - data analytics

  2021 — 2022