Publication Details

An Oracle-Guided Approach to Constrained Policy Synthesis Under Uncertainty

ANDRIUSHCHENKO Roman, ČEŠKA Milan, JUNGES Sebastian, KATOEN Joost-Pieter and MACÁK Filip. An Oracle-Guided Approach to Constrained Policy Synthesis Under Uncertainty. Journal of Artificial Intelligence Research, vol. 2025, no. 82, pp. 433-469. ISSN 1076-9757. Available from: https://www.jair.org/index.php/jair/article/view/16593
Czech title
Syntéza kontrolérů se strukturálními omezeními v Markovských rozhodovacích procesech.
Type
journal article
Language
english
Authors
Andriushchenko Roman, Ing. (DITS FIT BUT)
Češka Milan, doc. RNDr., Ph.D. (DITS FIT BUT)
Junges Sebastian (RWTH Aachen University)
Katoen Joost-Pieter (RWTH)
Macák Filip, Ing. (DITS FIT BUT)
URL
Keywords

Markov decision processes, model-based reasoning, search, decision making under uncertainty

Abstract

Dealing with aleatoric uncertainty is key in many domains involving sequential decision making, e.g., planning in AI, network protocols, and symbolic program synthesis. This paper presents a general-purpose model-based framework to obtain policies operating in uncertain environments in a fully automated manner. The new concept of coloured Markov Decision Processes (MDPs) enables a succinct representation of a wide range of synthesis problems. A coloured MDP describes a collection of possible policy configurations with their structural dependencies. The framework covers the synthesis of (a) programmatic policies from probabilistic program sketches and (b) finite-state controllers representing policies for partially observable MDPs (POMDPs), including decentralised POMDPs as well as constrained POMDPs. We show that all these synthesis problems can be cast as exploring memoryless policies in the corresponding coloured MDP. This exploration uses a symbiosis of two orthogonal techniques: abstraction refinement-using a novel refinement method-and counter-example generalisation. Our approach outperforms dedicated synthesis techniques on some problems and significantly improves an earlier version of this framework.

Published
2025
Pages
433-469
Journal
Journal of Artificial Intelligence Research, vol. 2025, no. 82, ISSN 1076-9757
Publisher
AI Access Foundation
DOI
BibTeX
@ARTICLE{FITPUB13365,
   author = "Roman Andriushchenko and Milan \v{C}e\v{s}ka and Sebastian Junges and Joost-Pieter Katoen and Filip Mac\'{a}k",
   title = "An Oracle-Guided Approach to Constrained Policy Synthesis Under Uncertainty",
   pages = "433--469",
   journal = "Journal of Artificial Intelligence Research",
   volume = 2025,
   number = 82,
   year = 2025,
   ISSN = "1076-9757",
   doi = "10.1613/jair.1.16593",
   language = "english",
   url = "https://www.fit.vut.cz/research/publication/13365"
}
Back to top