Suraj Panwar

About Suraj Panwar

Accomplished Data & Observability Engineer with over four years of experience delivering…

Experience

1. Data & Observability Engineer

   Altrata

   Sep 2022 — Present · Chennai, IN

   Working at the intersection of data engineering and observability to design, develop, and optimize scalable data pipelines while building reliable monitoring systems to ensure performance, availability, and data integrity.Key Responsibilities & Achievements: Data Engineering:Designed and implemented scalable ETL pipelines using Databricks, PySpark, and SQL for processing structured and semi-structured data.Automated repetitive data processes, resulting in a 75% reduction in manual checks and improved data accuracy.Achieved a 15% decrease in data processing time through performance tuning and workflow optimization.Implemented cost-optimization strategies, leading to a 20% reduction in cloud workspace costs.Created dynamic dashboards and data models in Power BI and Tableau, enabling real-time business insights. Observability Engineering:Built and maintained end-to-end observability systems using DataDog, including custom logs, metrics, and monitors for pipeline and application health.Developed real-time alerting pipelines integrated with Microsoft Teams, significantly improving incident response time.Engineered a JSON-based dashboard automation system to cut delivery time from days to hours.Reduced monitoring costs from $60/day to $1/day by leveraging API-based solutions for data collection and alerting.Partnered with product and engineering teams to define KPIs, onboard custom metrics, and implement proactive monitoring strategies.