Vivian K

About Vivian K

Data Engineer with 5 years of experience in designing and optimizing ETL pipelines, big data processing, and machine learning integration across retail, banking, and e-commerce domains. Proven expertise in building scalable data solutions using Apache Spark, AWS, Azure, and Snowflake, enabling advanced analytics and real-time insights. Data Engineering & ETL – Designed and implemented high-performance ETL pipelines processing 10M+ daily transactions, integrating payment gateways, APIs, and cloud-based data lakes. Optimized query performance in Snowflake & Redshift, reducing processing time by 40%. Machine Learning & AI – Developed and deployed AI-powered solutions, including a GPT-3 chatbot that handled customer queries/month, improving resolution time by 45%. Built a predictive model for failure detection, leveraging Random Forest & Python to enhance forecasting accuracy. Big Data & Cloud – Expertise in AWS (Glue, Lambda, S3, DynamoDB, KMS) and Azure (AKS, OpenAI, Power BI) to manage and process large datasets with high availability and security. Implemented real-time fraud detection with Kafka and ML models, enhancing anomaly detection. Business Intelligence & Analytics – Developed interactive dashboards in Tableau & Power BI, providing data-driven insights on transaction trends, payment performance, and fraud detection. Passionate about leveraging data-driven solutions to drive business impact, ensuring scalability, security, and efficiency. Always open to collaborating on innovative data engineering and AI projects. Tech Stack: Python, SQL, Apache Spark, AWS, Azure, Snowflake, Kafka, Tableau, Power BI.

Experience

1. Data Engineer

   Kroger

   Jun 2023 — Present

   Designed and developed an end-to-end ETL pipeline to process and transform 10 million daily transactions from POS systems, mobile wallets, and e-commerce platforms into a centralized data lake. Integrated payment gateways using APIs, ensuring seamless data ingestion and maintaining data integrity across multiple retail channels. Leveraged Apache Spark and AWS Glue to optimize data processing pipelines, achieving a 40% reduction in processing time. Implemented a real-time fraud detection system in collaboration with data scientists, utilizing machine learning models trained on historical transaction data. Deployed Apache Kafka to process and monitor real-time payment streams, flagging anomalies for immediate fraud prevention. Built data models in Snowflake and Amazon Redshift, optimizing query performance by 30% and enabling advanced analytics. Created interactive dashboards in Tableau and Power BI, providing actionable insights on metrics such as transaction trends, payment method usage, and revenue performance. Improved system reliability by analyzing payment failure trends and enhancing system components to reduce failure rates. Utilized AWS services like S3, Lambda, DynamoDB, and CloudWatch to ensure scalability, high availability, and performance monitoring. Enabled horizontal scaling of data pipelines to handle seasonal spikes, ensuring zero downtime during high-traffic periods. Secured sensitive payment data by implementing role-based access control (RBAC) and using AWS KMS for encryption, achieving compliance with PCI DSS standards. Collaborated with stakeholders to align data models and reporting solutions with business goals, ensuring effective delivery of insights. Documented the data pipeline architecture, data flow, and fraud detection processes to facilitate future maintenance and scalability.

Education

• The University of Texas at Arlington

  Master of Science - MS

  2021 — 2023

• JNTUH College of Engineering Hyderabad

  Bachelor of Technology

  2015 — 2019