Rahman Mohammed

Senior Python Data Engineer @CNA Insurance

IL, US

Developed data engineering processes in Azure using components like Azure Data Factory, Azure Data Lake Analytics, HDInsight, Databricks, etc.• Created numerous pipelines in Azure using Azure Data Factory v2 to get the data from disparate source systems by using different Azure Activities like Move & Transform, Copy, filter, for each, Databricks, etc.• Created Linked services to connect the external resources to ADF. Automated jobs using different triggers like Events, Schedules, and Tumbling in ADF.• Used JSON schema to define table and column mapping from S3 data to Redshift• Developed complex SQL/USQL/PySpark code for Data Engineering pipelines in Azure Data Lake Analytics and Azure Data Factory.• Designed and developed Batch processing and real-time processing solutions using ADF, Databricks clusters, and Stream Analytics.• Worked on data pre-processing and cleaning the data to perform feature engineering and data imputation techniques for the missing values in the dataset using Python. Designed and developed a new solution to process the NRT data by using Azure stream analytics, Azure Event Hub, and Service Bus Queue.• Loaded HIVE tables with data wrote hive queries that run on MapReduce and Created a customized BI tool for management teams that perform query analytics using HiveQL.• Messages we receive from Marketo to Azure Service Bus and experience in processing messages in separate Services.

JNTUH College of Engineering Hyderabad

Bachelor's degree