Shiva B.
Data Engineer @ Indeed | Real-time + batch ETL, data quality, monitoring, Data Modeling | Kafka • Spark • Glue • Redshift | GDPR-ready data quality
- Role
- Data Engineer at Indeed
- Location
- Dallas-Fort Worth, TX, US
- LinkedIn followers
- 500 followers
About Shiva B.
Open To I am a Senior Data Engineer focused on analytics outcomes. I build…
Experience
Data Engineer
Apr 2022 — Present · Austin, TX, US
At Indeed, I built, manage and optimize large-scale ETL pipelines to process millions of jobseeker profiles. I design real-time streaming pipelines using Kafka and Spark Structured Streaming, processing millions events daily to keep jobseeker profiles. Additionally, I maintain batch ETL workflows that process up to 1TB of data daily using AWS Glue, Redshift, and S3, ensuring efficient data transformation and availability for analysis.My role also involves SQL query optimization, implementing data validation frameworks in SQL to ensure data quality by detecting and resolving issues like missing values, incorrect types, and anomalies. This helps maintain high data accuracy for downstream analytics. I closely collaborate with product managers, data scientists, and engineers to design and implement data models that align with business goals, ensuring data is structured for easy access and analysis.In addition to pipeline development, I utilize Datadog to monitor the health of pipelines, track anomalies, and meet service-level agreements (SLAs). A critical aspect of my role is managing PII data in compliance with GDPR.
Education
University of the Cumberlands Graduate
Master of Science - MS
2021
Find verified contacts for anyone on LinkedIn
Unifers gives sales teams verified emails and direct dials, enriched profiles, and outreach that lands in the inbox.
Free plan included · No credit card required
This profile is compiled from publicly available professional sources. Unifers is not affiliated with or endorsed by LinkedIn. Request removal of this profile.