Suriya Valarmathi
Data Engineer @Tata Consultancy Services
Signup · Get unlimited contacts
WORK HISTORY
Data Engineer @Tata Consultancy Services
Toronto, ON, CA
Pipeline Development ∙ Built and maintained automated PySpark data pipelines for AML transaction data ingestion from multiple source systems into AWS S3 data lakes ∙ Developed Python scripts for data extraction via REST APIs, loading financial data into centralized Hive repositories on AWS EMR clusters ∙ Designed and executed SQL and HiveQL queries for data transformation, aggregation, and reconciliation of AML compliance datasetsCloud & Infrastructure ∙ Deployed and managed data workflows on AWS, ensuring pipeline reliability, scalability, and cost efficiency ∙ Implemented partitioning and bucketing strategies in Hive to optimize storage and query performance on large-scale transaction datasets ∙ Monitored Spark job execution, tuned configurations (executor memory, parallelism), and resolved bottlenecks to meet SLA requirementsAML & Compliance ∙ Developed data validation and reconciliation frameworks in Python to ensure accuracy and completeness of AML risk scoring datasets ∙ Supported case management workflows by delivering clean, audit-ready data to downstream AML detection engines ∙ Documented data flows, Hive schemas, and PySpark transformation logic to support regulatory audit and review processesCollaboration & Governance ∙ Worked with data scientists and compliance officers to define data models supporting AML typologies and suspicious activity detection ∙ Maintained version-controlled pipeline deployments using CI/CD practices integrated with AWS CodePipeline ∙ Enforced data quality standards and lineage tracking across the AML data platform
EDUCATION
Sri Venkateswara College of Engineering And Technology, Nh-5, Etcherla, PIN-532402 (CC-MT)
Bachelor of Engineering - BE, Electrical, Electronics and Communications Engineering
ABOUT SURIYA VALARMATHI
More than 8 years of experience in Data Engineering in various domains like Healthcare, Insurance and Banking.Extensive experience in the Healthcare, Insurance, and Banking sectors, delivering tailored data solutions.Proficient in using AWS services including DMS, RDS, Amazon Redshift, EC2, EMR, S3, and Snowflake for building and optimizing data pipelines.Designed and deployed scalable data architectures, ensuring high availability and performance.Strong experience in data migration, transformation, and ETL processes using AWS tools.Hands-on expertise in automation testing using Unix shell scripting and Python, enhancing data process reliability.Deep understanding of AWS security best practices, including IAM roles, policies, and encryption methods.Proven ability to collaborate with cross-functional teams to deliver complex data projects on time.Continuously learning and adapting to new AWS features and services to stay ahead in the evolving cloud landscape.Significant experience in building and managing data solutions using Azure Data Lake Storage Gen2 and Azure Data Lake Analytics.Successfully created and deployed end-to-end data pipelines in Azure Databricks, optimizing for performance and cost.In-depth knowledge of the Analytics Development Life Cycle, ensuring robust data quality assurance across all project phases.Strong hands-on experience in data analysis using tools like NumPy, pandas, and other Python libraries, driving actionable insights.Expertise in managing and monitoring Azure data solutions, ensuring continuous integration and deployment (CI/CD) with tools like Azure DevOps.Solid understanding of Azure security, governance, and compliance standards, ensuring data protection and regulatory adherence.Actively involved in supporting team members and collaborating with pre-sales teams to deliver successful Azure-based data projects.Committed to continuous professional development, staying updated on the latest Azure advancements and best practices.
This profile is compiled from publicly available professional sources. Unifers is not affiliated with or endorsed by LinkedIn. Request removal of this profile.