Narotham K C
- Role
- Data Engineer at CVS Health
- Location
- Memphis, TN, US
- LinkedIn followers
- 500 followers
About Narotham K C
IT proficient with Around 7 years of engagement, spent significant time in Big Data…
Experience
Data Engineer
Nov 2023 — Present · US
Designed and built ETL pipelines to extract, transform, and load data from Hadoop to Google cloud platform(GCP) using pySpark (cloud publishing framework) and data proc clusters.• Automated workflows using Apache Airflow DAGs for seamless orchestration of data pipelines.• Scheduled and monitored ETL jobs using Tidal Automation tool, ensuring on-time data processing and delivery.• Developed bash scripting files to optimize processing speed and enhance pipeline performance.• Utilized Collibra for comprehensive data validation, ensuring accuracy and consistency during data migration.• Provided production support by trouble shooting job failures and ensuring high system uptime.• Collaborated with the Application Insights team to implement alerting systems for job monitoring and failure detection.• Migrated large amount of data from Hadoop to on perm GCP using tools like distcp and gsutil.• Converted on-perm hive hql’s to bigquery sql’s.• Leveraged Inforworks Core Ingestion and Replicator to automate and streamline data ingestion and replication process.• Actively participated in all phases of the project life cycle, following Agile scrum methodology to analyze, develop, and deliver system enhancements.• Addressed and implemented enhancement requests from users to improve data systems and workflows.• Loaded and transformed large datasets of structured, semi-structured using Hadoop/Big data technologies to ensure data migration and storage.• Worked with various data sources including RDBMS, raw files, and windows shared paths for comprehensive data ingestion.• Collaborated with data governance team to ensure adherence to the latest rules and standards.• Worked on optimizing existing pipelines and addressing user feedback for continuous system improvements.• Utilized Rally to track and manage data validation, ensuring transparency and accountability.
Find verified contacts for anyone on LinkedIn
Unifers gives sales teams verified emails and direct dials, enriched profiles, and outreach that lands in the inbox.
Free plan included · No credit card required
This profile is compiled from publicly available professional sources. Unifers is not affiliated with or endorsed by LinkedIn. Request removal of this profile.