Runtime Decision Models under Uncertainty using Surprise-Based Learning for Supply Management

Abstract

upply management is a complex and critical aspect of operations that requires decision-making under uncertainty and changing con- ditions. In many real-world settings, such as healthcare, supply systems are subject to dynamic demand, resource constraints, and non-stationary environmental behaviour, making it difficult to rely on fixed or predefined models and requiring timely and responsive decision-making. In this paper, we propose a model-based approach that enables the runtime adaptation of supply management models by continuously revising their assumptions about environmental dynamics. The approach integrates Markov Decision Processes (MDPs) with Surprise-Based Learning (SBL) to dynamically update transition probabilities based on observed behaviour. We further incorporate human-in-the-loop feedback to support informed and context-aware decision-making. We evaluate the approach in the context of hospital supply management, assessing its correctness, usefulness, and adaptability compared to a fixed baseline. The re- sults show that enabling runtime adaptation of model assumptions leads to improved responsiveness and more robust decision sup- port under varying conditions, as reflected in the system’s ability to maintain more stable supply levels and ordering behaviour com- pared to a fixed baseline.