Sai Kowshik

Sr Big Data Engineer @GEICO

Chevy Chase, MD, US

Migrate data from traditional database systems to Azure SQL Databases• Recreating existing application logic and functionality in the Azure Data Lake, Data Factory, SQL Database and SQL Datawarehouse environment. • Used various sources to pull data into Power BI such as SQL Server, Excel, Oracle, SQL Azure etc.• Implement ETL and data movement solutions using Azure Data Factory, SSIS• Created several Databricks Spark jobs with PySpark to perform several tables to table operations. • Used Spark SQL for reading data from external sources and processes the data using PySpark computation framework.• Performing ETL from multiple sources such as Kafka, NIFI, Teradata, DB2 using Hadoop spark.• Developed ETL scripts using PySpark and Python and used SQL queries to validate the data.• Developed Spark applications using Pyspark and Spark-SQL for data extraction, transformation and aggregation from multiple file formats• Extract Transform and Load data from Sources Systems to Azure Data Storage services using a combination of Azure Data Factory, T-SQL, Spark SQL and U-SQL Azure Data Lake Analytics. • Completed online data transfer from AWS S3 to Azure Blob by using Azure Data Factory.• Used Azure Migrate to get started migrating your AWS EC2 instances over to Azure.• Design and implement database solutions in Azure SQL Data Warehouse, Azure SQL• Created tables on top of the data using Azure Data bricks• Completed online data transfer from AWS S3 to Azure Blob by using Azure Data Factory.• Designed and implemented a robust Data Warehousing Solution, loading data using parameterized Azure Data Factory pipelines from diverse sources to Azure SQL Database. • Created Airflow Scheduling scripts in Python• Created program in python to handle PL/SQL functions like cursors and loops which are not supported by snowflake.