Naveen Yadav
Data Engineer | Bigdata Engineer | Bigdata Developer| Works at Amazon | Hadoop | Hive | Scala | Python | Spark | AWS | AWS Glue | AWS EMR | AWS Redshift | AWS LAMBDA | AWS IAM | AWS |SQL | Shell Scripting | DSA
- Role
- Data Engineer (Business Intelligence Analyst) at Amazon
- Location
- Bengaluru, KA, IN
- LinkedIn followers
- 500 followers
About Naveen Yadav
I am Naveen, a distinguished Big Data professional with proven expertise in architecting…
Experience
Data Engineer (Business Intelligence Analyst)
Aug 2022 — Present · Karnataka, IN
Designed and implemented complex ETL pipelines using Spark, Hadoop, and AWS technologies, processing over 100TB of data daily with 99.9% reliability.• Architected and deployed enterprise data lake solution using AWS Lake Formation, S3, and Glue, reducing data processing time by 60% and eliminating data silos.• Developed high-performance data processing frameworks using Apache Spark and Hadoop, achieving 40% faster data processing while maintaining data integrity.• Engineered scalable data pipelines using PySpark on EMR clusters, optimizing resource utilization and reducing processing time by 55%.• Implemented advanced data transformations using Spark SQL, DataFrames, and RDDs, improving query performance by 45% and reducing cluster costs by 40%.• Built real-time data ingestion pipelines using AWS Kinesis and Lambda, processing 1M+ events per minute with sub-second latency.• Optimized Hive queries and table partitioning strategies, resulting in 30% faster query execution and improved resource utilization.• Established robust data governance framework using AWS Lake Formation and IAM, ensuring secure data access and compliance across multiple teams.• Created automated data quality validation frameworks using Python and Spark, maintaining 99% data accuracy across all pipelines.• Integrated multiple data sources using Sqoop and custom connectors, building a unified data platform serving 500+ daily active users.• Mentored junior engineers in big data technologies and best practices, reducing onboarding time by 40% for new team members.• Collaborated with cross-functional teams to define data models and implement data warehousing solutions, improving data accessibility by 70%.• Led the migration of legacy data processing systems to cloud-native solutions, resulting in 65% cost reduction and improved scalability.
Find verified contacts for anyone on LinkedIn
Unifers gives sales teams verified emails and direct dials, enriched profiles, and outreach that lands in the inbox.
Free plan included · No credit card required
This profile is compiled from publicly available professional sources. Unifers is not affiliated with or endorsed by LinkedIn. Request removal of this profile.