Sooraj Antony

About Sooraj Antony

Lead Data Engineer and Architect with 15 years of expertise in designing and deploying cloud-native Big Data platforms that transform complex business problems into scalable, data-driven solutions. Proven track record of modernizing legacy systems, optimizing data architectures, and leading cross-functional engineering teams to deliver high-impact results.Core Expertise:Cloud & Big Data Architecture – Deep expertise in AWS ecosystem (EMR, Lambda, MWAA, Glue, S3, DynamoDB) with hands-on experience architecting serverless data lakes and event-driven pipelines. Successfully migrated multiple on-premise systems to AWS, reducing infrastructure costs by 30% while improving scalability and reliability.Data Engineering at Scale – Proficient in building end-to-end ETL/ELT pipelines using PySpark, Hive, Pig, and Airflow, processing billions of records daily for Fortune 500 clients. Specialized in performance tuning, resolving data skew, and optimizing workflows to achieve 65% execution time reductions.Leadership & Delivery – Lead and mentor engineering teams using Agile/Kanban methodologies, improving team velocity by 40%. Excel at translating stakeholder requirements into technical roadmaps, conducting code reviews, and ensuring delivery excellence through DataOps best practices.Technical Innovation – Designed and implemented automated CI/CD pipelines with GitLab, integrated Infrastructure as Code using Terraform, and developed rule-based data governance frameworks that reduced manual validation efforts by 70%. Experienced in developing FastAPI services and integrating AI/ML capabilities into data products.Database & Analytics – Strong foundation in SQL optimization across Netezza, Postgres, Redshift, and Oracle, with expertise in OLAP analytical functions and performance tuning for complex data warehouse solutions.Certifications: AWS Solutions Architect Associate | AWS Data Analytics Specialist | Oracle SQL Expert | Azure FundamentalsPassionate about building resilient, cost-efficient data platforms that enable AI-driven insights and empower organizations to make data-informed decisions. Open to opportunities in Big Data Products, data architecture, engineering leadership, and cloud data platform innovations.

Experience

1. Manager, Technology

   Nielsen

   Jan 2024 — Present · Kochi, IN

Education

• Lady of Hope Anglo Indian High School

  SSLC(10th)

  1990 — 2003

• Mahatma Gandhi University

  B.Tech, Electronics & Communication

  2005 — 2009

Skills

• Sqoop
• Cubes
• Data Warehousing
• Python
• Sql
• Oracle
• Requirements Analysis
• Git
• Etl
• Talend
• Bi Publisher
• Data Integration
• Data Modeling
• Unix Shell Scripting
• Pl/Sql
• Apache Pig
• Olap
• Hive
• Hadoop
• Netezza
• Big Data
• Perl