Pramod P.

About Pramod P.

Extensive experience in IT industry comprising of data management, design, development, Release & Deployment and Cloud implementation.• Designed and implemented CI/CD pipelined and automated solutions.• Developed Python Scripts to automate various system tasks, OS patches for Dev environments, and deployment of applications to testing/prod environments using Python OS modules.• Experience in building and architecting multiple Data pipelines, end to end ETL pipelines for data ingestion and transformation in AWS.• Extract Transform and Load data from Nested AWS sources to AWS DataLake in relational format.• Orchestrate the Glue job runs hourly and end-of-day, based on upstream events, developed lambda and Step-Function scripts to store upstream events in DynamoDB, trigger Gluejobs for the respective batches and publish events for downstream consumers using EventBridge.• Strong Experience in implementing Data warehouse solutions in Amazon Redshift• Hands on experience on architecting the ETL transformation layers and writing Pyspark jobs in Glue for data processing.• Expertise in Creating, Debugging, Scheduling and Monitoring ETL jobs using Amazon Managed Airflow.• Extensively worked on JENKINS for continuous integration and End-to-End automation for all build and deployments.• Built S3 buckets and managed policies for S3 buckets and used S3 bucket and Glacier for storage and backup on AWS.• In depth, practical experience with Cloud methodologies (IaaS, PaaS, SaaS, VM), on industry-leading cloud vendor offerings, experienced in Code as a Service (CaaS) cloud implementation on AWS and Azure using TERRAFORM and Cloud formation Template.• Passionate in designing and implementing complex, multi-tier, end-to-end solutions using a wide variety of private and public cloud offerings, including AWS EC2, S3, ELB, RDS, IAM, EKS, ECS, Lambda, API Gateway, SNS, SQS, DynamoDB and Auto Scaling. • Experience with Business Intelligence using PowerBI and AWS Quicksight.

Experience

1. Sr Aws Data Engineer

   State Farm

   Feb 2021 — Present

   Designed and setup Enterprise DataLakeHouse to provide support for various uses cases including Storing, Processing, Analytics and Reporting.• Transformed unstructured and semi-structured data from legacy systems into dimensional models (star/snowflake) for Analytics and BI. • Leveraged AWS Glue, S3, Athena and Lake Formation to build a centralized AWS Data Lake, integrating data from multiple vendor sources.• Developed PySpark ETL pipelines to transform nested upstream data into flattened relational models, registering them as Apache Iceberg tables with schema evolution, partitioning, Time Travel and roll-back capabilities.• Created and maintained Redshift Cluster, built Databases and Tables for legacy Statefarm data migrated from Mainframe.• Developed business logic on EMR to consolidate data from multiple business lines into Datalake, transforming and loading into S3.• Orchestrated ETL pipelines using Apache Airflow DAGs to schedule and monitor data ingestion, transformation, and loading into Redshift.• Migrated mainframe legacy data into RDS PostgreSql in Star schema modal, and supply data for Custom UI and BI tools. • Hands-on experience of writing notebooks in databricks using python/Spark SQL for complex data transformations and schema operations. • Designed Flink pipelines to consume streaming data from kafka and apply business logic to transform and serialize raw data.• Utilized Python and Boto3 to build clients for DynamoDB, EventBridge, and SNS to insert and retrieve records, send email notifications, and publish events to CloudWatch.• Designed and deployed REST APIs using Amazon API-Gateway, integrating with AWS Lambda.• Migrated BI reports from Cognos to PowerBI, to let agents track their revenue.• Built Docker images with all necessary packages and Jars for running Pyspark and Hudi operations.• Developed Terraform Modules for Various AWS services such as Lambda, IAM Roles, Policies, Glue Jobs, S3, Lake Formation, EMR, Airflow, Redshift.

Education

• San Francisco Bay University

  Master of Science - MS, Computer Science

• University of the Cumberlands

  Master's degree, Computer and Information Systems Security/Information Assurance