Publication Details
Inductive Synthesis of Finite-State Controllers for POMDPs
partially observable Markov decision processes, finite-state controllers,
inductive synthesis, counter-examples, abstraction
We present a novel learning framework to obtain finite-state controllers (FSCs)
for partially observable Markov decision processes and illustrate its
applicability for indefinite-horizon specifications. Our framework builds on
oracle-guided inductive synthesis to explore a design space compactly
representing available FSCs. The inductive synthesis approach consists of two
stages: The outer stage determines the design space, i.e., the set of FSC
candidates, while the inner stage efficiently explores the design space. This
framework is easily generalisable and shows promising results when compared to
existing approaches. Experiments indicate that our technique is (i) competitive
to state-of-the-art belief-based approaches for indefinite-horizon properties,
(ii) yields smaller FSCs than existing methods for several POMDP models, and
(iii) naturally treats multi-objective specifications.
@inproceedings{BUT178215,
author="ANDRIUSHCHENKO, R. and ČEŠKA, M. and JUNGES, S. and KATOEN, J.",
title="Inductive Synthesis of Finite-State Controllers for POMDPs",
booktitle="Conference on Uncertainty in Artificial Intelligence",
year="2022",
series="Proceedings of Machine Learning Research",
volume="180",
number="180",
pages="85--95",
publisher="Proceedings of Machine Learning Research",
address="Eindhoven",
issn="2640-3498"
}