Vishal N.

About Vishal N.

Built & optimized ETL/ELT processes using Python & Spark, reducing processing time by 50% via code optimization & parallel processing.• Pioneered scalable and fault-tolerant pipelines using Apache Kafka and Amazon Kinesis for real-time data ingestion into Redshift, reducing data processing latency by 25% through parallel processing.• Conducted data validation and quality checks using Informatica and DBT, ensuring data accuracy and reducing errors by 15%.• Crafted Apache Flink\'s windowing and event-time processing with Databricks\' Delta Lake and performed Time series analysis using Tableau to identify customer retention trends and churn rate, resulting in a 10% reduction in churn rate.• Engineered real-time data pipelines with Data Dog, ensuring efficient integration and real-time analytics, resulting in a 30% increase in data availability for the analytics team.• Implemented serverless data processing using AWS Lambda, reducing infrastructure management overhead by 80% and enabling faster deployment of data pipelines.• Published developed reports to Power BI Service and migrated 20+ reports created in (SSRS, Excel) to Power BI.

Experience

1. Data Engineer

   Tekion Corp

   Jul 2024 — Present

   Designing and optimizing data pipelines, processing 500K+ records daily using Apache Spark and Kafka for real-time analytics.Building and maintaining scalable ETL processes, improving data ingestion efficiency by 40% using AWS Glue and Snowflake.Developing and managing data models and warehouses, ensuring seamless integration with business intelligence tools like Tableau and Power BI.Enhancing database performance, reducing query execution time by 30% through indexing and partitioning in PostgreSQL and Redshift.Automating data validation and quality checks, achieving 99.9% accuracy by implementing data governance best practices.Collaborating with cross-functional teams, including data scientists and software engineers, to deliver data-driven insights for 5+ key business domains.Implementing data security and compliance measures, ensuring adherence to GDPR and HIPAA regulations.Optimizing cloud-based data solutions, leveraging AWS services like S3, Lambda, and Redshift to support petabyte-scale datasets.Writing and reviewing efficient SQL queries and Python scripts, reducing data processing time by 25% through code optimization.Monitoring and troubleshooting data pipelines, maintaining a 99.8% uptime by proactively identifying and resolving performance bottlenecks.

Education

• University of Alabama at Birmingham

  Master of Science - MS, Computer Science

• KLE Technological University - Hubballi (India)

  Bachelor's degree, Computer Science

• Vikas Pre-University College - India

  Pre-University, Physics, Chemistry, Mathematics and Biology