Sampath Kumar Arpula
Certified GCP & AWS Data Engineer | Expert in Apache Kafka, Spark, Airflow, Scala, Python, Hadoop |BigQuery, Snowflake, Redshift | Open to C2C and C2H Roles
- Role
- Senior Data Engineer at Cardinal Health
- Location
- Columbia, MO, US
- LinkedIn followers
- 500 followers
About Sampath Kumar Arpula
Senior Data Engineer with a strong passion for technology and a commitment to driving…
Experience
Senior Data Engineer
Oct 2022 — Present · Dublin, OH, US
Led a comprehensive data migration and integration project for a healthcare network, transferring data from on-premises Oracle databases to cloud environments, ensuring security and efficiency.• Migrated historical data using Apache Sqoop, extracting data from Oracle databases and staging in HDFS and cloud storage for batch processing.• Utilized Apache Kafka for real-time data streaming, ingesting continuous updates of pharmaceutical data into distributed storage for further processing.• Employed Apache Spark for both batch and real-time data processing, transforming streaming and historical data for seamless integration.• Developed custom PySpark functions in Python to standardize diverse data formats, improving data quality and consistency across datasets.• Optimized ETL pipelines using PySpark’s in-memory processing, partitioning, and Python scripts, reducing data processing times and enhancing pipeline responsiveness.• Implemented automated ETL workflows and data pipelines with Apache Airflow, ensuring efficient processing of both batch and real-time healthcare data.• Data from GCP\'s BigQuery was loaded into interactive dashboards in Looker for real-time monitoring of key metrics, incorporating various data sources. The dashboards also provided projections for future trends.
Education
Jawaharlal Nehru Technological University
Bachelor of Technology - BTech
University of Missouri-Columbia
Master of Science - MS
Find verified contacts for anyone on LinkedIn
Unifers gives sales teams verified emails and direct dials, enriched profiles, and outreach that lands in the inbox.
Free plan included · No credit card required
This profile is compiled from publicly available professional sources. Unifers is not affiliated with or endorsed by LinkedIn. Request removal of this profile.