Shashidhar Reddy G

Sr Data Engineer @Charles Schwab

TX, US

Designed and implemented ETL pipelines for data extraction, transformation, and loading into target systems.Developed ETL pipelines to move data to and from Snowflake data store using combination of Python and Snowflake Snow SQLGood knowledge of defining, developing, and deploying Star Schema, Snowflake Schema and dimensional Data Modelling using SQL Server.Created stored procedures in Snowake and implemented Change Data Capture (CDC) using table streams Skills.Implemented data pipelines and workflows to transfer data between Snowflake Data Lake and Azure services, ensuring seamless data integration and synchronization across platforms.Successfully managed large-scale data warehouses on (ADLS), optimizing query performance, managing data distribution, and implementing data compression techniques for efficient storage and retrieval.Integrated Azure Data Factory with Azure services, including Azure Blob Storage, Azure Data Lake Storage, Azure SQL Database, and Azure Synapse Analytics, to enable seamless data integration and processing.Tuned Spark-SQL jobs written in Scala for optimal performance in YARN-managed Hadoop clusters, including dynamic resource allocation and DAG optimization.Created reusable Scala-based libraries for accessing and writing to HDFS, Hive, and external systems (e.g, Cassandra, Kafka).Orchestrated Databricks jobs with Azure Data Factory pipelines, enabling automated data movement and end-to-end workflow scheduling.Developed and maintained scalable ETL pipelines in Azure Databricks using PySpark for processing large-scale structured and semi-structured data.Used Databricks Repos and Git integration for version control and CI/CD practices in collaborative development environments.Leveraged Databricks notebooks to perform interactive data exploration, transformation, and model development in Python and SQL.

University of North Texas

Masters in Information Systems