Patel Yash

Data Engineer @Revent Labs

Performed ETL environment setup using aws kinesis, lambda, s3 and Snowflake for data event processing and storage. Wrote SQL queries to identify and validate data inconsistencies in the data warehouse against the source system. Also,optimized data retrieval processes in a high data volume environment by re-writing the SQL queries. Created python spark jobs on aws glue to retrieve data from Snowflake data warehouse for reporting purposes. Set up monitoring platform for aws services being used using Prometheus and grafana for cost and uptime management. Automated whole data and ETL infrastructure using IaC, code commit, code build, code deploy. Containerised script applications used for data reporting. Used Opensearch service for cloudwatch logs parsing and ingestion for different aws services and other on premapplications. Hands-on Experience with Docker and Docker Hub/ Artifactory Repository. Creating Docker Images and managing Dockercontainers on Kubernetes environments.