Hetal Vaghela

About Hetal Vaghela

I\'m a Data Engineer with 4+ years of experience building scalable, reliable data platforms across healthcare and financial services, with a strong focus on Databricks Lakehouse architecture and cloud-native data engineering. I specialize in designing and optimizing end-to-end batch and real-time data pipelines using Python, SQL, and Apache Spark. My work centers on building high-performance solutions on Databricks using Delta Lake, Structured Streaming, and metadata-driven ingestion, while ensuring strong data governance through frameworks like Unity Catalog and data quality validation. I\'ve consistently improved pipeline performance and reduced processing times by up to 40% by optimizing Spark workloads, data models, and cloud infrastructure. At CVS Health, I engineered enterprise-scale pipelines processing millions of healthcare records, transitioning legacy ETL workflows to streaming architectures integrated with Apache Kafka. I implemented data quality frameworks, CI/CD automation, and fine-grained access controls to ensure high data reliability and compliance. I also built RAG pipelines and LLM-enabled workflows using Claude and Snowflake Cortex AI, and delivered interactive dashboards in Amazon QuickSight and Tableau to visualize critical operational metrics for clinical and pharmacy leadership. Earlier, I built financial data platforms integrating Spark, Databricks, and Snowflake, supporting credit risk and portfolio analytics through scalable data models and analytics-ready datasets. I developed financial reporting dashboards in Google Data Studio and Power BI, enabling real-time insights for business stakeholders. What drives me is turning complex, high-volume data into trusted, business-ready insights especially in regulated environments where performance, governance, and data accuracy are critical. Core Skills: Python · SQL · Apache Spark · Databricks (Delta Lake, DLT, Autoloader, MLflow, Vector Search) · Snowflake (Cortex AI, Cortex LLM Functions) · AWS (S3, Glue, EMR, Kinesis, Redshift, QuickSight) · Azure (Databricks, Data Factory, Data Lake, DevOps) · Apache Kafka · Airflow · RAG Pipelines · LLMOps · Claude API · Data Modeling · CI/CD · Data Governance · Data Quality Frameworks · Lakehouse Architecture · Tableau · Power BI · Amazon QuickSight · Google Data Studio

Experience

1. Data Engineer

   ServiceNow

   Nov 2023 — Present · MA, US

   Designed and implemented scalable ETL pipelines using Python and SQL within the Hadoop ecosystem, improving structured healthcare data ingestion efficiency by over 40%.Modernized legacy ETL workflows by migrating SSIS processes to AWS Glue, reducing infrastructure maintenance and enhancing pipeline scalability and performance.Built real-time streaming pipelines with Apache Kafka to process pharmacy inventory and order data, improving data freshness and compliance by 30%.Optimized large-scale batch processing using Apache Spark, cutting overall pipeline latency by 30% and increasing reliability for downstream analytics.Utilized Amazon Athena to execute ad hoc SQL queries directly on S3-stored datasets, improving business intelligence turnaround time by 20%.Engineered CI/CD pipelines leveraging DevOps best practices to automate deployment of data workflows, reducing production errors by 25% and accelerating releases.Developed dynamic dashboards in Amazon QuickSight, enabling real-time visualization of prescription trends and inventory levels to support faster, data-driven decision-making.Led cross-functional Agile collaborations, improving development velocity by 20% while ensuring alignment with business and compliance requirements.

Education

• Gujarat Technological University (GTU)

  Bachelor's degree, Information Technology

• UMass Boston

  Master's degree, Information Technology