Raj Anand

About Raj Anand

Responsibilities:Develop and maintain data pipelines on AWS platform using services such as AWS Glue, Amazon Redshift, AWS Lambda, Amazon S3, Amazon RDS, Amazon EC2, Amazon SES, and Amazon SNS.Perform daily operations including schema modifications, data validation, and data staging utilizing AWS services.Implement data ingestion processes from Amazon RDS to Amazon S3 data lake for staging.Design and create data processing pipelines using PySpark on AWS, facilitating data movement from source systems to Amazon S3 and from Amazon S3 to Amazon Redshift.Ensure data quality and integrity through the development of validation processes, including the creation of Lambda functions scheduled to verify source and destination data counts.Proactively identify and address data discrepancies by implementing Lambda functions to detect duplicate records in Amazon Redshift and sending alert messages accordingly.Monitor and maintain data pipeline health by developing Lambda functions to detect pipeline failures and execution delays, ensuring timely resolution. Implement email notification system for immediate alerts on pipeline failures.Collaborate with cross-functional teams to understand data requirements, troubleshoot issues, and optimize data workflows for performance and efficiency.Stay current with AWS technologies and best practices, incorporating new tools and methodologies to enhance data engineering processes.Key Achievements:Successfully implemented automated data validation processes, reducing manual effort by 100% and improving data accuracy.Enhanced pipeline performance by optimizing PySpark code, resulting in a 85% reduction in processing time.Implemented proactive monitoring solutions, reducing pipeline downtime by 0% and improving data availability.Skills:Proficient in AWS services including AWS Glue, Amazon Redshift, AWS Lambda, Amazon S3, Amazon RDS, Amazon EC2, Amazon SES, and Amazon SNS.Strong programming skills in Python for data processing and automation tasks.Experience with big data technologies such as PySpark for data transformation and processing.Knowledge of data validation techniques and tools for ensuring data quality.Familiarity with SQL for querying and manipulating data in Amazon Redshift and other relational databases.Excellent problem-solving skills and ability to troubleshoot complex data issues.Strong communication and collaboration skills for working effectively in cross-functional teams.

Experience

1. Delivery Lead

   Acuity Analytics

   Feb 2026 — Present · Gurugram, IN

Education

• St. Atulanand Convent school varanasi

  10 & 12

  1997 — 2003

• University of Rajasthan

  Bachelor of Engineering - BE, Information Technology

  2004 — 2008

• Rajiv Gandhi Prodyogiki Vishwavidyalaya

  M.Tech.(CSE), Computer Science & Engineering

  2010 — 2012