Priyank Shah

About Priyank Shah

Data Engineer | AWS & Python | Scalable Data Pipelines & OrchestrationI am a Data Engineer with hands-on experience of 4+ years building cloud-native, production-grade data pipelines on AWS. I specialize in transforming raw, high-volume data into analytics-ready datasets that power business decision-making.Over the years, I have designed and orchestrated complex ETL workflows using AWS services such as S3, Glue, Lambda, Step Functions, and Airflow-style DAG orchestration patterns, with a strong focus on scalability, reliability, and automation.My work includes optimizing high-volume pipelines, automating data validation and re-drivability, and maintaining pipeline uptime, resulting in measurable improvements in performance and cost efficiency. I have also built backend services using FastAPI, Django to support secure data access and orchestration.I enjoy working at the intersection of data engineering, cloud architecture, and system design, and I love working in the environment where I can own data platforms end-to-end and help teams build reliable, scalable data systems.Tech Stack: AWS (S3, Glue, Lambda, Step Functions, Redshift), Apache Airflow, Python, SQL, Spark, FastAPI, CI/CD#DataEngineering #AWS #Python #ETL #CloudComputing #FastAPI #Redshift #Spark #CI/CD #DevOps #DataPipelines #SQL

Experience

1. Data Engineer

   Radian

   Jul 2023 — Present · Franklin, TN, US

   Spearheaded the standardization of diverse client data, from nearly 10 clients, into a centralized data lake, ensuring consistency and improving the processing time of data across different sources- Led the design and execution of a modern ETL architecture to handle high-volume data with a focus on scalability and seamless data ingestion, supporting the company’s transition to handling high-mix and high-volume data- Engineered and implemented a modern ETL architecture leveraging AWS services (S3, Lambda, RDS, Glue, Step Functions, SNS, SQS), improving logging and re-drivability from failure- Optimized error handling in data ingestion and automated job reruns to maintain 99.9% uptime, ensuring seamless and reliable data flow for multiple business tools- Designed and orchestrated scalable ETL workflows using AWS Step Functions and Airflow-style DAG patterns, enabling reliable scheduling, dependency management, and automated recovery from failures- Built middleware using FastAPI to manage user sessions, seamlessly integrating with AWS Cognito for authentication and session tracking, resulting in a 40% improvement in user experience and system security- Leveraged AWS CloudWatch for logging and monitoring jobs- Led the implementation of GitHub as the organization’s version control system, driving an 80% improvement in team collaboration and code quality through the establishment of best practices and regular code reviews- Aim to add CI/CD using GithubActions to automate the code deployment.

Education

• CHARUSAT

  BTech - Bachelor of Technology, Computer Science

• University of Maryland - Robert H. Smith School of Business

  Master of Science - MS, Information Systems