Mahendra K

Lead Data Engineer Cloud Engineer @U.S. Renal Care

Dallas, TX, US

Designing and Configuring Azure Cloud relational servers and databases analyzing current and future business requirements.• Designing and developing CI/CD process using Azure Devops.• Designing and creating Data Marts, Databases, Indexes, Views, Aggregations, Stored Procedures, Partitions and Data Integrity.• Migrating Data from On-Perm SQL Server to Cloud databases Azure Synapse Analytics(DW) and Azure SQL DB.• Performing research to identify source and nature of data required for ETL solutions.• Developing Tabular Models on Azure Analytics services• Developing Azure Data Factory pipelines to extract and manipulate data from Azure Blob storage/Azure Data Lake/ SQL Server on cloud.• Experience in developing Spark applications using Spark-SQL in Databricks for data extraction, transformation, and aggregation from multiple file formats for Analyzing & transforming the data to uncover insights into the customer usage patterns. Extract Transform and Load data from sources Systems to Azure Data Storage services using a combination of Azure Data factory, T-SQL, Spark SQL, and U-SQL Azure Data Lake Analytics. Data ingestion to one or more Azure services (Azure Data Lake, Azure Storage, Azure SQL, Azure DW) and processing the data in Azure Databricks• Develop Spark applications using pyspark and spark SQL for data extraction, transformation, and aggregation from multiple file formats for analyzing and transforming the data uncover insight into the customer usage patterns• Hands on experience on developing SQL Scripts for automation• Responsible for estimating the cluster size, monitoring, and troubleshooting of the Spark databricks cluster• Ability to apply the spark DataFrame API to complete Data manipulation within spark session• Good understanding of Spark Architecture including spark core, spark SQL, DataFrame, Spark streaming, Driver Node, Worker Node, Stages, Executors and Tasks, Deployment modes, the Execution hierarchy, fault tolerance, and collection

JNTUH College of Engineering Hyderabad

B.tech, Computer Science