Vyshnavi M

Senior Big Data Engineer @Somatus

McLean, VA, US

As a Senior Big Data Engineer, I led the development of scalable, secure, and compliant data pipelines for healthcare analytics and operations. I migrated over 10TB of legacy healthcare data to Google Cloud (BigQuery, Cloud SQL), reducing ETL latency by 60%. Using Ab Initio, I built robust ingestion pipelines for structured and semi-structured clinical data (HL7, X12, CCD) and implemented metadata governance with Google Cloud Data Catalog.I engineered real-time data processing workflows using Apache Kafka, Google Pub/Sub, and Dataflow to capture patient vitals and device data, supporting real-time care and alerting. My work ensured HIPAA/GDPR compliance by leveraging IAM roles, Secret Manager, and VPC controls for data security.To handle high-volume workloads, I designed and tuned Cassandra data models for low-latency reads/writes, and containerized ETL jobs on GKE, using Terraform and Ansible for provisioning. I also developed Spark (Java/PySpark) jobs on Dataproc, optimized for autoscaling and monitored through Cloud Logging and Cloud Monitoring, improving SLA adherence.In addition, I built centralized knowledge graphs using Neo4j and GraphDB, connecting 2B+ relationships among patients, diagnoses, and providers to drive insights for clinical teams. I automated data validation, schema checks, and alerting across the pipeline using Cloud Composer (Airflow) and GCP monitoring tools.I worked in Agile Kanban sprints, using Jira for tracking and GitHub for CI/CD, ensuring rapid and reliable deployments.Tools/Tech: GCP (BigQuery, Pub/Sub, Cloud SQL, Composer, GKE), Apache Kafka, Ab Initio, Cassandra, Spark, PySpark, Terraform, Ansible, Neo4j, Airflow, GitHub, HL7, X12