Runtime Decision Models under Uncertainty using Surprise-Based Learning for Supply Management

Abstract

Supply management is a complex and critical aspect of operations that requires decision-making under uncertainty and changing conditions. In many real-world settings, such as healthcare, supply systems are subject to dynamic demand, resource constraints, and non-stationary environmental behaviour, making it difficult to rely on fixed or predefined models and requiring timely and responsive decision-making. In this paper, we propose a model-based approach that enables the runtime adaptation of supply management models by continuously revising their assumptions about environmental dynamics. The approach integrates Markov Decision Processes (MDPs) with Surprise-Based Learning (SBL) to dynamically update transition probabilities based on observed behaviour. We further incorporate human-in-the-loop feedback to support informed and context-aware decision-making. We evaluate the approach in the context of hospital supply management, assessing its correctness, usefulness, and adaptability compared to a fixed baseline. The results show that enabling runtime adaptation of model assumptions leads to improved responsiveness and more robust decision support under varying conditions, as reflected in the system’s ability to maintain more stable supply levels and ordering behaviour compared to a fixed baseline.