Pradeep R.
Senior Data Engineer | Snowflake | Big Data | ETL Pipelines | Cloud (AWS, Azure, GCP)
- Role
- Senior Data Engineer at Capital One
- Location
- Charlotte, NC, US
- LinkedIn followers
- 500 followers
About Pradeep R.
Senior Data Engineer with expertise in Snowflake, AWS, Azure, Databricks, Spark, Airflow,…
Experience
Senior Data Engineer
Apr 2024 — Present · McLean, VA, US
Migrated 200M+ credit card records to Delta Lake, reducing query time from 4 hours to 45 minutes using Spark and Snowflake. Automated ETL pipelines with Airflow and AWS Glue, eliminating 30% of manual work and lowering compliance risk. Built record-level validation scripts in Python to reconcile billions of transactions, reducing data mismatches in UAT by 95%. Resolved high-priority incidents in Spark and Snowflake under 1-hour SLA, keeping marketing analytics pipelines stable. Optimized partitioning and broadcast joins in Spark, cutting job runtime from 90 minutes to under 25 minutes. Integrated Snowflake with AWS services, scaling data storage by 40% without cost increase. Automated lineage tracking with Databricks and Glue Catalog, improving audit readiness by 60%. Developed monitoring dashboards in CloudWatch to detect pipeline failures, cutting downtime by 50%. Refactored legacy PL/SQL ETL into PySpark jobs, improving developer productivity by 35%. Collaborated with compliance teams to design secure pipelines, achieving 100% audit pass rate.
Education
Sacred Heart University
Master of Science - MS, Computer Science and Information Technology
JNTUH College of Engineering Hyderabad
Bachelor of Technology - BTech, Computer Science and Engineering
2019
Find verified contacts for anyone on LinkedIn
Unifers gives sales teams verified emails and direct dials, enriched profiles, and outreach that lands in the inbox.
Free plan included · No credit card required
This profile is compiled from publicly available professional sources. Unifers is not affiliated with or endorsed by LinkedIn. Request removal of this profile.