Robust $Q$-learning Algorithm for Markov Decision Processes under Wasserstein Uncertainty

arXiv2210.00898MaRDI QIDQ6412730

Author name not available (Why is that?)

Publication date: 30 September 2022

Abstract: We present a novel $Q$-learning algorithm to solve distributionally robust Markov decision problems, where the corresponding ambiguity set of transition probabilities for the underlying Markov decision process is a Wasserstein ball around a (possibly estimated) reference measure. We prove convergence of the presented algorithm and provide several examples also using real data to illustrate both the tractability of our algorithm as well as the benefits of considering distributional robustness when solving stochastic optimal control problems, in particular when the estimated distributions turn out to be misspecified in practice.

Has companion code repository: https://github.com/juliansester/wasserstein-q-learning

This page was built for publication: Robust $Q$-learning Algorithm for Markov Decision Processes under Wasserstein Uncertainty

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q6412730)