Sai Teja
Data Analytics Engineer Data Engineer @American Express
Signup · Get unlimited contacts
WORK HISTORY
Data Analytics Engineer Data Engineer @American Express
Irving, TX, US
Implemented scalable data pipelines in AWS using EMR, EC2, and Glue to process multi-terabyte datasets, significantly reducing overall processing time.Designed and developed data transformations using PySpark on AWS Databricks to prepare and enrich financial datasets for downstream analytics.Created a real-time data ingestion system using AWS Lambda and S3 to capture live data feeds and automate storage for immediate processing.Built Python-based data quality checks and validation layers are integrated with AWS services to ensure data integrity and compliance throughout the pipeline.Designed data lake architecture on AWS, using PySpark and Scala for aggregation and transformation to support regulatory and reporting needs.Automated scaling of ETL jobs using AWS CloudWatch and Lambda, dynamically adjusting compute resources based on workload size and thresholds.Currently leading a POC integrating GenAI with our data pipeline to auto-generate data quality issue summaries and remediation suggestions using Amazon Bedrock and custom LLM prompts.Developed an internal chatbot prototype using OpenAI APIs that allows business users to query the data lake in natural language, returning insights from Redshift and S3 without needing SQL.Evaluating the use of GenAI to automate documentation of complex ETL workflows by extracting metadata from Terraform, Argo, and PySpark code, helping improve onboarding and audit readiness.Automated scaling of ETL jobs using AWS CloudWatch and Lambda, dynamically adjusting compute resources based on workload size and thresholds.Extracted and transformed large datasets from a Cloudera Hadoop environment using PySpark and SQL to generate benchmarking insights for financial clients.Developed complex SQL logic on AWS Redshift to support advanced
EDUCATION
St. Peter's Engineering College
BTech - Bachelor of Technology, cse
Franklin University
Masters degree, Information Technology
ABOUT SAI TEJA
I serve as a Senior Data Engineer at American Express, where I specialize in cloud-based data solutions. My core competencies include building scalable data infrastructures, implementing real-time data ingestion systems, and ensuring data integrity through automated quality checks. Leveraging tools like AWS, EMR, EC2, and Glue, I\'ve significantly reduced processing times for multi-terabyte datasets.At American Express, our team has developed innovative data transformations using PySpark on AWS Databricks, improving financial analytics capabilities. I\'ve also been instrumental in designing data lake solutions that support stringent compliance and reporting requirements. My experience with both AWS and Azure platforms has been pivotal in delivering robust data architectures and real-time insights, reflecting my commitment to excellence and a culture of continuous improvement. Through collaboration, we aim to empower the organization with data-driven decisions and maintain our competitive edge in the financial services industry.
This profile is compiled from publicly available professional sources. Unifers is not affiliated with or endorsed by LinkedIn. Request removal of this profile.