Shravani M

Sr Azure Data Engineer @Credit Suisse

Raleigh, NC, US

Analyze, design and build Modern data solutions using Azure PaaS service to support visualization of data. Understand current Production state of application and determine the impact of new implementation on existing business processes. Worked in Agile Methodology and used JIRA for maintain the stories about project. Extract Transform and Load data from Sources Systems to Azure Data Storage services using a combination of Azure Data Factory, T-SQL, Spark SQL and U-SQL Azure Data Lake Analytics. Data Ingestion to one or more Azure Services -(Azure Data Lake, Azure Synapse, Azure Storage, Azure SQL, Azure DW) and processing the data in Azure Databricks. Created Pipelines in ADF using Linked Services/Datasets/Pipeline/ to Extract, Transform and load data from different sources like Azure SQL, Blob storage, Azure SQL Data warehouse, write-back tool and backwards. Developed Python scripts to automate the ETL process using Apache Airflow and CRON scripts in the Unix operating system as well. Worked on developing the real time process using Event Hub’s and transforming the data using Scala notebooks. Written Sqoop Queries to import data into Hadoop from SQL Server table. Developed Spark streaming application to pull data from cloud to hive table. Developed scripts in Hive to perform transformations on the data and load to target systems for reporting. Worked on using Azure Automation Account to schedule the Power shell (run books). As a Big Data implementation engineer responsible for developing, troubleshooting and implementing programs. Worked extensively with Dimensional modeling, Data migration, Data cleansing, ETL Processes for data warehouses. Knowledge on creating various repositories and version control using GIT.