Abdul Raheem
Senior Data Engineer @AgFirst Farm Credit Bank
Signup · Get unlimited contacts
WORK HISTORY
Senior Data Engineer @AgFirst Farm Credit Bank
Columbia, SC, US
Created Datasets/ Data-Frames from RDDs using reflection and programmatic inference of schema over RDD.Developed Python Spark programs for processing HDFS Files using RDDs, Pair RDDs, Spark SQL, SparkStreaming, Data-Frames, Accumulators, Broadcast variables.Created Pipelines in ADF using Linked Services/Datasets/Pipeline/ to Extract, Transform and load data fromdifferent sources like Azure SQL, Blob storage, Azure SQL Data warehouse, write-back tool and backwards.Work with support teams in resolving operational & performance issues.Developed PYSPARK programs using various Transformations and operations.Worked on MySQL database writing queries using Python MySQL connector and MySQL Package.Responsible for managing large databases using Panda data frames and MySQL.Used Pandas library for statistical Analysis, regular expressions, and python collections.Created Datasets/ Data-Frames from RDDs using reflection and programmatic inference of schema over RDD.
EDUCATION
Jawaharlal Nehru Technological University, Hyderabad
Bachelor of Technology - BTech, Computer Science
ABOUT ABDUL RAHEEM
Around 8 years of Experience as a Data Engineer which includes Analysis, Design, Development, and Implementation.• Analyzing and understanding source systems and business requirements to Design the Enterprise Data warehousing and Business Intelligence Solutions, DataMart and Operational Data Store • Experience in designing & developing applications using Big Data technologies HDFS, Map Reduce, Sqoop, Hive, PySpark & Spark SQL, Hbase, Python, Snowflake, S3 storage, Airflow.• Experience in job workflow scheduling and monitoring tools like Airflow and Autosys.• Experienced in Designing, Developing, Documenting, Testing ETL jobs and mappings in Server and Parallel jobs using Data Stage to populate tables in Data Warehouse and Data marts. • Worked in Production support team for maintaining the mappings, sessions and workflows to load the data in Data Warehouse.• Hands-on experience in using Hadoop ecosystem components like Hadoop, Hive, Pig, Sqoop, HBase, Cassandra, Spark, Spark Streaming, Spark SQL, Oozie, Zookeeper, Kafka, Flume, MapReduce framework, Yarn, Scala, and Hue. • Extensively worked on AWS services like EC2, S3, EMR, RDS, SageMaker, Athena, Glue Data Catalog, RDS(Aurora), Redshift, DynamoDB, and Elastic Cache (Memcached & Redis) & Quick Sight and other services of the AWS family. • Worked closely with the Enterprise Data Warehouse team and Business Intelligence Architecture team to understand repository objects that support the business requirement and process.• Extensive knowledge in working with Azure cloud platforms (HDInsight, Datalake, Databricks, Blob Storage, Data Factory, Synapse, SQL, SQL DB, DWH, and Data Storage Explorer). • Extensive experience in working with NoSQL databases and its integration Dynamo DB, Cosmo DB, Mongo DB, Cassandra, and HBase. • Experience in building data pipelines using Azure Data factory, Azure data bricks and loading data to Azure data Lake, Azure SQL Database, Azure SQL Data warehouse and controlling and granting database access.
This profile is compiled from publicly available professional sources. Unifers is not affiliated with or endorsed by LinkedIn. Request removal of this profile.