Vishal Gupta

About Vishal Gupta

Over 5 years of experience as a Data Engineer/SQL Developer in Design, Development, Big Data Implementation and analysis. • Experience in utilizing PySpark for Ingestion, storage, querying, processing, and analysis of big data. • Experience in Agile Software development process, Test Driven Development and Scrum. • Strong working experience on Amazon Web Services including S3, EC2 and EMR. • Experience in front-end UI development skills using HTML5, CSS3, Java Script. • Hands on experience in configuring and deploying Applications using different web/application servers such as Terraform and CircleCI • Worked with source code version control systems like GIT for providing common platform for all the developers. • Designed, developed, and implemented customized temporary tables, queries, and reports utilizing SQL. • Extensive experience in Hadoop components like Pyspark, Airflow and Hive • Experience in Cloud data migration using AWS and Snowflake. • Strong Linux/Unix based experience for process management. • Solid experience in writing SQL queries and procedures to extract data from various source tables. • Knowledge on developing complex Tableau reports and dashboards. • Technical knowledge on Tableau Desktop • Experience in Data Analysis using Snow SQL on Snowflake. • Experience in process automation through Airflow. • Experience in master core functionalities such as DAGs, Operators, Tasks, Workflows etc in Airflow. • Strong experience in analyzing large amounts of data sets writing Pyspark scripts and Hive queries. • Knowledge on AWS Glue to prepare the Data for Analysis through Automated Extract, Transform and Load (ETL) processes. Knowledge on AWS Redshift Data warehouse. • Extensive experience in UNIX performance monitoring and Load balancing to ensure stable performance. • Attended Performance Optimization training by Snowflake Team from Snowflake. • Working experience on creating DNS Server names through Terraform. • Implementation of AWS Lambda functions and integration with SNS for email notification to get the data from Sprinklr Api

Experience

1. Data Analyst

   Capital One

   Jan 2022 — Present · VA, US

   Implement AWS Lambdas to drive real-time monitoring dashboard of Kinesis streams. • Experience with full development cycle of a Data Warehouse, including requirements gathering, design, implementation and maintenance. Integrated data from multiple sources and transformed them into a unified graph database model. Analyzed data to identify trends, patterns, and insights that helped improve business decision-making. Collaborated with cross-functional teams, including developers, analysts, and business stakeholders, to understand requirements and deliver solutions that met business needs. Utilized graph database modelling tools such as Neo4j, Graph DB, and Amazon Neptune to design and develop graph data models. Design and implement data pipeline infrastructure that ingests, stores, and processes large amounts of data • Worked with management, developers, quality engineers, and product managers to gather requirements and define workflow for a new project, then implement in JIRA. • Improvement performance of existing ETL processes and SQL queries for weekly CRM summary data. • Utilized Jitterbit tool to get the data from Salesforce objects to S3. • Worked on Consumer data and processed using Pyspark data frames and loaded the data to S3. • Used Enterprise Snowflake Datawarehouse to populate the data for reporting team. • Responsible for Creating roles and assigning to individuals for snowflake access. • Used Apache Airflow extensively for orchestration purpose. • Created Hive tables and Okera views and used data masking of sensitive attributes in okera views. • Created and Maintained Airflow dags to automate the process of loading data to Snowflake and Hive. • Extensive experience on different filetypes which include Json, Csv, Text, Parquet. • Creation of filetypes and external stages in Snowflake. • Used Cerberus as a Safe deposit boxes for Key management tool. • Used CloudFormation to provision all the s3 bucket policies.e deployment purposes

Education

• University of New Orleans

  Master's degree, Computer Science

  2020 — 2022

• Amity University

  Bachelor of Science - BS, Engineering Management

  2013 — 2016