Pranay Kumar Padidhapalli

WORK HISTORY

EDUCATION

ABOUT PRANAY KUMAR PADIDHAPALLI

AWS Data Engineer | 7+ Years of Experience in Scalable Data Solutions & Cloud InfrastructureI\'m an experienced AWS Data Engineer with over 7 years of success in building scalable, reliable, and high-performance data solutions in the cloud. I specialize in architecting modern ETL/ELT pipelines, optimizing data storage and processing, and automating cloud infrastructure.My toolkit includes a wide range of AWS services—Glue, Redshift, Lambda, Kinesis, S3, RDS, SQS, SNS, EMR, VPC, and CloudWatch—combined with orchestration tools like Apache Airflow, Step Functions, and Informatica. I’m fluent in Python, PySpark, SQL, and Scala, and have a strong focus on performance tuning, cost optimization, and infrastructure-as-code with Terraform and CloudFormation.In the big data world, I’ve worked extensively with Hadoop tools like HDFS, Hive, Pig, HBase, Sqoop, and Oozie—handling massive datasets and designing secure, scalable data lakes. I also bring deep experience with the Spark ecosystem (Core, SQL, Streaming) and real-time data processing using Kafka, Flume, Cassandra, and Elasticsearch. Security is a priority in every solution I build, leveraging tools like Kerberos and Apache Ranger for authentication and access control.Key Wins:Processed over 1M real-time events/sec with AWS Kinesis & PySparkCut data processing time by 40% by optimizing ELT pipelines using Glue & RedshiftSaved 35% in storage costs through S3 lifecycle policies & Glacier tieringImproved Redshift query performance by 30% with Spectrum optimizationBoosted CI/CD pipeline efficiency by 35% with Jenkins, Docker, and KubernetesIndustries & Passion: I’ve delivered impactful data solutions across finance, insurance, and retail domains. I\'m passionate about data engineering and cloud architecture—turning complex data into actionable insights that drive real business outcomes.