Surya Siram
Senior Data Engineer @Jet2.com and Jet2holidays
Signup · Get unlimited contacts
WORK HISTORY
Senior Data Engineer @Jet2.com and Jet2holidays
Leeds, GB
EDUCATION
Jawaharlal Nehru Technological University, Kakinada
B.TECH, Computer Science
SKILLS
ABOUT SURYA SIRAM
Designed and implemented a real-time streaming pipeline to transfer event-structured data from a legacy DB2 in-house application to a BigQuery data warehouse via on-prem Kafka, Pub/Sub, and Dataflow services, handling the ingestion of million records daily from 55 locations.• Developed GCP batch data pipelines to load trading and risk application data from cloud storage buckets to BigQuery using Dataflow jobs, achieving a 100% reduction in manual effort.• Skilled in managing large datasets with PySpark, performing data cleaning, data transformation, and optimizing functionality with effective techniques.• Orchestrated the entire batch data pipeline using Airflow and Cloud Functions services.• Created complex SQL queries, views, and materialized views with auto-refresh every 30 minutes, supporting the BI team in generating reports and dashboards in Power BI and GCP Looker.• Implemented AWS batch data pipelines to ingest user log data from S3 buckets into the Redshift data warehouse using AWS Glue jobs, orchestrated workflows with AWS CloudWatch and Lambda functions, achieving a 100% reduction in manual effort.
This profile is compiled from publicly available professional sources. Unifers is not affiliated with or endorsed by LinkedIn. Request removal of this profile.