Venkata Sudheendra Eluri

Snowflake Data Engineer @WM

Developed data pipelines to transform and load data from various sources into Snowflake and AWS Redshift using PySpark.• Designed and optimized AWS Redshift tables by configuring distribution styles and sort keys for improved performance.• Built streaming data pipelines with AWS Kinesis Data Streams and Firehose for real-time data processing.• Created and managed ETL workflows using AWS Glue for data integration and transformation.• Utilized SparkSQL to handle JSON data, create Schema RDDs, and load data into Hive tables.• Developed Python workflows for API data ingestion and transformations from data warehouse tables.• Led data migration projects from legacy databases (Oracle, PostgreSQL) to Snowflake and Redshift, ensuring smooth data extraction, transformation, and loading.• Managed Snowflake environments, including Multi-Cluster Warehouses, scaling policies, and programmatic warehouse suspension using the Python Connector.• Implemented Snowflake utilities such as SnowSQL, Snowpipe, Streams, Tasks, and Snowsight.• Applied Snowflake security features, including column-level dynamic data masking, row-level access policies, and object tagging.• Automated ELT pipelines using Snowflake\'s External Stage, Snowpipe, Streams, and Tasks.• Worked with business units to gather requirements for building Data pipelines, ETL workflows and Dashboards.• Created and managed ETL workflows using AWS Glue for data integration and transformation.• Managed Snowflake environments, including Multi-Cluster Warehouses, scaling policies, and programmatic warehouse suspension using the Python Connector.• Orchestrated Data pipelines using Stonebranch Scheduler and AWS Step functions depending on the business needs.• Created robust data pipelines using dbt to automate ETL processes, streamlining data integration from multiple sources into data warehouses and enhancing data model performance.