Vinay Kumar

Data Engineer @Trinity Health

Livonia, MI, US

Proficient in integrating Hive and Base for streamlined operations.• Skilled in crafting Spark programs using Scale for Data Transformations, Datasets, Data frames, and Spark SQL queries, including spark streaming and windowed streaming applications.• Implemented on-demand tables on S3 files using Lambda functions and AWS Glue with Python and Pyspark.• Developed a PyUnit framework for unit testing and executed complex Transact SQL queries and SSIS packages for data warehouse loading.• Created an Admin API for managing Kafka objects, along with data ingestion modules using AWS Glue for S3 loading and reporting via Athena and Quick sight.• Designed and executed data quality scripts using SQL and Hive for data validation post-load.• Produced various data visualizations using Python and Tableau and demonstrated proficiency in monitoring and managing Hardtop clusters via Cloud era.• Engaged in testing, bug fixing, and scrum/agile methodologies utilizing tools like JIRA.• Collaborated with Data Architects, Business Analysts, and users to grasp business and functional requirements.