Siddharth Srivastava
Senior Data Engineer | AWS Cloud | Python | PySpark | Databricks | Snowflake | Data Lakehouse | ETL/ELT | Apache Iceberg | Airflow | Data Modelling | Machine Learning
- Role
- Senior Big Data Engineer at Annalect
- Location
- Gurugram, HR, IN
- LinkedIn followers
- 500 followers
About Siddharth Srivastava
I am a results-driven Senior / Lead Data Engineer with over 9 years of experience in…
Experience
Senior Big Data Engineer
Jan 2023 — Present · Gurugram, IN
Working as an AWS big data engineer for company\'s in house product \"Omni\".Developed Python-based data connectors for Airbyte\'s open-source framework, automating extraction of data from social media platforms to S3Implemented stateful data ingestion using DynamoDB, optimizing EC2 costs while processing multiple asynchronous API requestsCreated dockerized development environment for AWS Glue and EMR services, significantly reducing local testing costs and deployment timeArchitected data lakehouse solutions using Apache Iceberg to improve query performanceBuilt automated data quality framework with Great Expectations to detect anomalies pre-ingestion and reduce data incidentsUtilized tech stack including Python, PySpark, AWS Glue, Lambda, S3, DynamoDB, EC2, SQS, SNS, AWS CDK, AWS Athena, Apache Iceberg, Airbyte, Airflow, Great Expectations, EMR serverless, and Docker
Education
KIIT - Kalinga Institute of Industrial Technology
Bachelor of Technology - BTech, Civil Engineering
Find verified contacts for anyone on LinkedIn
Unifers gives sales teams verified emails and direct dials, enriched profiles, and outreach that lands in the inbox.
Free plan included · No credit card required
This profile is compiled from publicly available professional sources. Unifers is not affiliated with or endorsed by LinkedIn. Request removal of this profile.