Rushikesh Misal

About Rushikesh Misal

Highly skilled Cloud Data Engineer with good hands-on experience, in data engineering on…

Experience

1. Assistant Consultant

   Tata Consultancy Services

   Jun 2021 — Present · Pune, IN

   Projects and Responsibilities:RDS-Aurora to S3 Data Lake Extraction•Objective: Implement a solution for extracting RDS PostgreSQL workload to S3 Data Lake•Role: Configured end to end AWS pipeline using terraform code and validate source and destination data.NRT Data Movement from Aurora to No-SQL DB-Mark logic•Objective: Develop a cloud layer to handle NRT claims data processing.•Role: Enhanced data movement solutions between NRT and Marklogic Data Management Platforms.CDA data movement from S3 to EMR •Objective: Create a comprehensive ETL solution to manage and analyze raw CDA data.•Role: The design and implementation of a robust data pipeline.S3 to Data Mart Transformation Pipeline•Objective: Build a pipeline to process transactional data stored in S3, transform it based on business use cases, and load it into a data mart for reporting.•Role: Ingested data from AWS S3, performed data validation, applied transformations using PySpark, and loaded clean data into Redshift and S3. Integrated monitoring and alerting via SNS and implemented dashboards for reporting.DynmoDB to Athena Product Sales Projection•Objective: Enable real-time data streaming and sales projection reporting by capturing product events from DynamoDB to Athena via a streaming architecture.•Role: Configured change data capture from DynamoDB using streams, piped data through Kinesis Firehose into S3, and created Glue jobs for transformation. Leveraged Athena for querying and analysis.