Hussain Q

About Hussain Q

A Data Engineer with over all 8+ years of experience working with ETL, Big Data, Java Python/Scala, Relational Database Management Systems (RDBMS), and enterprise-level cloud-based computing and applications.• Comprehensive experience on Hadoop ecosystem utilizing technologies like MapReduce, Hive, HBase, Spark, Sqoop, Kafka, Oozie, Zookeeper and AWS. • Created partitions and bucketing as well as designed tables in Hive to optimize performance. • Experience with developing User Defined Functions (UDFs) in Apache Hive usin¬g Java, Scala, and Python.• Hands on experience working in GCP services like Big Query, Cloud Storage (GCS), cloud function, cloud dataflow, Pub/sub, Cloud Shell, GSUTIL, Big Query, Data Proc, Operations Suite (Stack driver).• Performed in-memory data processing for batch, real-time, and advanced analytics using Apache Spark (Spark Core, Spark SQL, and Streaming).• Expertise in using Cloud based managed services for data warehousing in Confidential Azure (Azure Data Lake Storage, Azure Data Factory). • Ingested data into Hadoop from various data sources like Oracle, MySQL, and Teradata using Sqoop tool. • Experienced in Agile and Waterfall methodologies in Project execution.• Strong knowledge in NoSQL column-oriented databases like HBase and their integration with Hadoop.• Experience in setting up Hadoop clusters on cloud platforms like AWS and GCP.

Experience

1. Sr Data Engineer Gcp

   Eaze

   Feb 2021 — Present · CA, US

   Responsible for building scalable distributed data solutions using Hadoop• Installing and configuring nagios check postgresql.pl plugins for Monitoring PostgreSQL instances effectively.• Created and developed landing pages for the Confidential marketing department using HTML, CSS, and jQuery.• SQL query tuning, build Stored procs for the core-processing engine, including Oracle Advanced queuing.• Experience in Developing ETL solutions using Spark SQL in Azure Databricks for data extraction, transformation and aggregation from multiple file formats and data sources for analyzing & transforming the data to uncover insights into the customer usage patterns.• Build data pipelines using airflow in GCP for ETL related jobs using different airflow operators.• Developed Pre-processing job using Spark Data frames to flatten JSON documents to flat file. • Written T-SQL queries for the retrieval of the data• Setup environment for data science team, installation of Anaconda, Jupiter hub, spark/python, tensorflow/Keras/hail etc.• Experience on Cloud Databases and Data warehouses ( SQL Azure and Confidential Redshift/RDS).• Load D-Stream data into Spark RDD and do in memory data Computation to generate output response.• Writing Complex Transact SQL (T-SQL) Queries, Sub queries, Co-related sub queries, Dynamic SQL queries etc. • Developed daily email blasts using HTML and CSS• Analytic tools such as TensorFlow, PyTorch, Cafe2, CNTK, Keras, Azure ML Studio, MSFT R server, Cortana Intelligence Suite, IOT, Stream Analytics, IVR analytics, R, SAS, Python, Java, C/C++, Splunk,

Education

• Jamia Hamdard University

  bachelor's , Computer Science (CSE)