Sai Sandeep K
Big Data Engineer @RBC
Signup · Get unlimited contacts
WORK HISTORY
Big Data Engineer @RBC
Toronto, ON, CA
Cloud Transformation: Led the migration of legacy data from SQL Server and Teradata into Amazon S3, creating a scalable, cloud-native foundation for enterprise analytics.Performance Engineering: Developed high-performance Spark applications using Scala, Spark SQL, and Spark Streaming for complex batch and real-time processing.Query Optimization: Designed consumption views that significantly reduced execution time for analytical queries and improved scalability by converting complex Hive/SQL queries into Spark RDD transformations.Streaming & Ingestion: Built and maintained robust Kafka-based pipelines to ingest server logs and event data into Spark Streaming.Enterprise Integration: Integrated diverse data sources—including Cassandra, Kinesis, and NiFi—into the Enterprise Data Lake, ensuring secure and high-volume data handling.Automation: Orchestrated end-to-end ETL workflows using Apache Oozie and managed distributed jobs on AWS EMR.Data Governance: Ensured data quality and reliability through leaf-level validation and cross-database reconciliation.
ABOUT SAI SANDEEP K
Results-oriented Data Engineer with 5+ years of experience in building scalable, high-performance data platforms across AWS and Azure ecosystems. Specialized in designing real-time and batch data pipelines using Spark, Kafka, Snowflake, and Databricks.Proven expertise in- Developing ETL/ELT pipelines using AWS Glue, Azure Data Factory, PySpark- Architecting Snowflake DWH solutions and integrating with BI tools (Power BI, Tableau)- Real-time data streaming with Kafka, Kinesis, and Spark Streaming- Implementing CDC and data governance (Azure Purview, Apache Atlas)- Delivering 30–60% cost and performance improvements in production environmentsDriven by impact, collaboration, and innovation — let’s connect if you\'re looking to solve data challenges at scale.
This profile is compiled from publicly available professional sources. Unifers is not affiliated with or endorsed by LinkedIn. Request removal of this profile.