Shiva S.
Senior Data Engineer | Machine Learning, AWS, Snowflake, Databricks, PySpark, Kafka, DBT| Building Scalable, AI-Ready Data Platforms
- Role
- Senior Data Engineer at Sas Info
- Location
- Seattle, WA, US
- LinkedIn followers
- 500 followers
About Shiva S.
I’m a Senior Data Engineer with 7+ years of experience building large-scale, cloud-native…
Experience
Senior Data Engineer
Apr 2024 — Present
Engineered fault-tolerant data platforms on AWS (S3, Glue, Lambda), achieving high availability via partitioning and optimized S3 storage classes. Designed and deployed scalable ETL pipelines using AWS Glue, Lambda, and PySpark to support analytical reporting and subscriber metric dashboards. Partnered with cross-functional teams to ingest, normalize, and validate data across platforms, building a compliant data lake using Unity Catalog and IAM-based access controls. Integrated ML inference workflows using SageMaker endpoints into streaming dashboards, enabling automated decision-making and personalization features. This solution was aligned with Generative AI exploration initiatives, enabling seamless interaction with NLP-based customer feedback analysis and summarization tools. Deployed CI/CD for ETL pipelines with Docker, Git, Jenkins; integrated CloudWatch/Datadog for automated monitoring, ensuring system stability. Optimized Spark jobs in Databricks with caching, partition pruning, parallelism, and adapting every execution, improving large-scale processing performance and reducing costs.
Find verified contacts for anyone on LinkedIn
Unifers gives sales teams verified emails and direct dials, enriched profiles, and outreach that lands in the inbox.
Free plan included · No credit card required
This profile is compiled from publicly available professional sources. Unifers is not affiliated with or endorsed by LinkedIn. Request removal of this profile.