Yuba Raj Pokhrel

About Yuba Raj Pokhrel

Senior Data Engineer with 10+ years of experience building scalable data platforms and…

Experience

1. Senior Aws Data Engineer

   CG COMMUNICATIONS LTD.

   Jul 2021 — Present

   Designed and deployed a data-driven web application using Python and Flask, integrated with Oracle for real-time data access, and hosted on AWS EC2 with static assets stored in S3-Developed Python-based ETL pipelines to extract and transform data from MS SQL Server, Oracle, PostgreSQL, MySQL, MongoDB, and Cassandra, leveraging advanced SQL queries, Common Table Expressions (CTEs), and complex joins for efficient data retrieval-Engineered scalable data pipelines on Databricks/AWS EMR using Apache Spark and PySpark to process large volumes of structured and semi-structured data-Integrated Amazon S3 for data lake management, ensuring reliable storage of pipeline output-Automated secure data delivery using Paramiko via SFTP, and implemented end-to-end workflows to join, validate, and apply business rules to customer datasets, generating decision-ready reports for financial and operational use cases on AWS infrastructure-Orchestrated ETL workflows with Apache Airflow, designing DAGs with task dependencies, retries, SLA monitoring, and alerts, reducing manual interventions by over 60% and improving pipeline reliability-Developed REST APIs in Flask for real-time data access and JSON response delivery, supporting dashboards, analytics, and internal system integrations- Built data visualization dashboards using Matplotlib, Highcharts, and Python reporting scripts to monitor KPIs, trends, and operational metrics-Performed data cleaning, transformation, aggregation, and exploratory analysis with Pandas and NumPy, supporting reporting, analytics, and model development- Conducted web scraping using BeautifulSoup and Requests to collect third-party and public datasets for integration into internal pipelines-Automated recurring workflows for credit scoring, customer profiling, and risk validation, enabling pipeline monitoring, failure alerts, and restart capabilities.