Nishar Ramesh

Gcp Data Engineer @HCLTech

CA

Developed data pipelines using Google Cloud Dataflow and Apache Beam, ensuring efficient ETL processes for both batch and real-time data processing, leading to improved data accuracy and reduced processing timesDesigned and optimized data storage solutions with Google Big Query, Google Cloud Storage, and Google Bigtable, ensuring high availability, scalability, and cost efficiency for large-scale data management and analytics projectsMaintained real-time data streaming applications using Google Cloud Pub/Sub, enabling low-latency data processing and seamless integration across various systems to support business-critical operationsUtilized Stack driver Monitoring and Logging to monitor data workflows and application performance, setting up alerts and dashboards to proactively identify and resolve issues, ensuring system reliability and optimal performanceAutomated data workflows with Cloud Composer (Apache Airflow), orchestrating complex data processing tasks and improving the reliability and efficiency of ETL pipelines, which resulted in a slight increase in data processing speedLeveraged Google Cloud Dataprep for data cleaning and transformation, enabling self-service data preparation for analysts and data scientists, reducing the time spent on data preparation and increasing productivitySpearheaded the design and implementation of scalable ETL pipelines using PySpark on Google Cloud\'s DataProc, enabling the efficient processing of large datasetsImplemented Google Cloud Data Loss Prevention (DLP) strategies to safeguard sensitive data across various projects and conducted regular audits and compliance checks, ensuring that all data processing adhered to industry standards and regulatory requirementsAutomated data quality checks and workflow monitoring using Google Cloud tools, ensuring continuous data integrity and minimizing downtime in data processing environments