Karthick V

Sr Data Engineer @NRECA

I Analyzed the SQL scripts and designed it by using PySpark SQL for faster performance. I\'m Involved in creating Hive tables, and then applied Hive QL on those tables for data validation. Imported and exported data using Sqoop from HDFS to RDBMS and vice-versa. Extensively used Azure Key vault as central repository for maintaining secrets and referenced the secrets in Azure Data Factory and also in Azure Data bricks notebooks. Designed and implemented streaming solutions using Kafka or Azure Stream Analytics and implemented database solutions in Azure SQL Data Warehouse, Azure SQL Performed job functions using Spark API’s in Scala for real time analysis and for fast querying purposes. Analyzed and validated the snowflake data based on expectations. Developed Oracle, PL/SQL scripts to validate and load data into interface tables.