Sai Gowtham Kallu

Lead Data Engineer @7-Eleven

Orchestrated a scalable and fault-tolerant data processing pipeline on GCP, leveraging Cloud Dataflow, Cloud Pub/Sub and Cloud Bigtable. Implemented a real-time data ingestion pipeline using PostgreSQL and Apache Kafka to process and analyse streaming data.Experienced in working with various databases such as SQL Server, CosmosDB, PostgreSQL, Oracle, Aurora, and DynamoDB.Automated testing and documentation processes within DBT to improve deployment efficiency and stakeholder understanding.Utilized Hasura\'s real-time capabilities to create subscriptions for live data updates, enhancing user experience in applications requiring instant data reflection.Architected and deployed a secure and scalable data lake solution on GCP, utilizing Cloud Storage, Big Query and Data Catalog. Designed an optimized data ingestion framework using Cloud Composer and Cloud Functions for efficient and automated data loading from diverse sources. Implemented robust data governance and access controls with IAM and Cloud DLP, ensuring data privacy compliance and enabling self-service data exploration.Developed and maintained data transformation workflows using DBT, ensuring data quality and consistency across pipelines.Extensive expertise in cloud platforms such as AWS, Azure, and GCP, and skilled in managing data warehouses like Snowflake, Google Big Query, and RedShift.Integrated Hasura with PostgreSQL databases, optimizing data modelling and access patterns to streamline ETL processes and improve data retrieval times.Designed and developed ETL processes using AWS Glue to automate the ingestion, transformation, and enrichment of healthcare data from various sources, enabling efficient data integration and analysis.