Warren Ross

Data Architect @Ministry of Business, Innovation and Employment

Wellington, NZ

Designed and implemented a customised data analysis platform from scratch:o Fully containerised application components using Docker/Kubernetes;o Completely integrated open source application stack including Jupter Hub/Lab, Apache Airflow, Apache Spark, PostgreSQL, Traefik, Elasticsearch, and Kibana;o Customised language kernels including Python, R, PySpark, and SparkR;o Logically tiered data flow from ingestion to presentation featuring granular role-based access control (RBAC) throughout;o Customised security integration with Active Directory via PAM, Kerberos, and Workflow integration with Azure DevOps Boards and Repos (git-based) providing comprehensive, secure source-code management for both Infrastructure-as-Code and data engineering/analytics work streams;o Integrated with multiple cloud platforms: MS Azure for development work and CI/CD pipelines, Datacom Cloud Services for Government (DCSG) for production deployment;o Powerful data workflow management with Apache Airflow and custom-build ETL logging system provides end-to-end visibility of data pipelines;o Enormous range of interactive data visualisations and outputs, such as dashboards, reports, and geographical views, provided through multiple application layers including MS Power BI, Jupyter Lab, and Kibana;o Zero-license approach to design (aside from MS Power BI): every open source software component used in the platform is free from commercial licensing – all we pay for is compute and storage;o One collaborative platform for all data specialist roles, used by data engineers, data scientists, data analysts, and business users.

University of South Africa/Universiteit van Suid-Afrika

BA, English Literature / Psychology