Dayashankar Das

Data Engineer @AU SMALL FINANCE BANK

Mumbai, IN

Designed and architected end-to-end data pipelines for both batch and real-time processing, enabling seamless data ingestion, transformation, and storage using AWS services such as S3, EMR, Glue, Redshift, and Spark Streaming.• Led the development and deployment of ETL and ELT processes, ensuring data is efficiently ingested from diverse sources and transformed for analytical use across data lake and warehouse systems.• Collaborated with cross-functional teams including business analysts, data scientists, and engineering teams to translate technical requirements into scalable data solutions.• Owned the AWS-based data platform infrastructure, ensuring reliability, performance, and cost-efficiency by continuously monitoring and optimizing resource usage.• Developed and enforced best practices and reusable patterns for data sourcing, transformation, storage, and access across projects, aligning with organizational data strategy.• Implemented real-time data streaming pipelines using Apache Spark Structured Streaming for low-latency ingestion and transformation of operational data.• Conducted data validation, reconciliation, and quality checks to maintain the integrity, accuracy, and consistency of data throughout the pipeline lifecycle.• Delivered technical presentations and solution walkthroughs to stakeholders and leadership to ensure alignment and transparency of data engineering initiatives.• Provided production support and maintenance for Data Warehouse (DW) and Data Lake (DL) environments, including proactive monitoring, incident escalation, and resolution.• Continuously improved system performance and efficiency by identifying cost-saving opportunities, optimizing job runtimes, and streamlining workflows.

University of Mumbai

Bachelor of Engineering - IT