Manish Maddeshiya

Data Engineer @Capgemini

Noida, IN

American Family Health Insurance(AMFAM) Regulatory & Finance Data Pipelines Capgemini • Designed and implemented Airflow DAGs for scheduling and orchestrating regulatory and financial data workflows. • Developed batch processing logic for insurance and financial reports using Python, SQL, and DBT. • Automated BigQuery table versioning and metadata tracking with Python-based ETL jobs. • Utilized GCP Cloud Storage and Pub/Sub for handling JSON-based source data ingestion. • Managed Terraform scripts to provision infrastructure for cloud-based data processing. • Implemented GitLab CI/CD pipelines for version control and deployment automation. • Resolved Docker daemon errors in GitLab pipeline execution for containerized jobs. • Optimized data pipeline performance by tuning SQL queries, partitioning & indexing. • Integrated Cloud Logging & Monitoring (CloudWatch, GCP Logging) for tracking job execution. • Built and optimized streaming pipelines using Apache Dataflow for real-time data processing-Warner Bros Theatrical Data Pipeline Capgemini Capgemini • Worked on a data engineering project for Warner Bros in the Theatrical Data domain, using a Snowflake-based architecture with bronze, silver, and gold layers to manage and transform large datasets. • Utilized Qlik Replicate to ingest data from multiple sources into the bronze layer. • Conducted data transformations in the silver layer using SQL and Python for data cleaning, aggregation, and formatting. • Set up the gold layer for final reporting and analytics, which was used by various business teams for decision-making. • Managed and monitored data pipelines using Redwood, ensuring successful daily data loads and error resolution. • Collaborated with cross-functional teams to ensure data quality, availability, and timely access.

KRISHNA INSTITUTE OF ENGINEERING AND TECHNOLOGY, GHAZIABAD

Bachelor's degree