Hierarchical algorithms for discounted and weighted Markov decision processes
type: Item
instance of: scholarly article
title: Hierarchical algorithms for discounted and weighted Markov decision processes
zbMATH Open document ID: 1069.90106
author: Mohammed Abbad
author: Cherki Daoui
published in: Mathematical Methods of Operations Research
publication date: 2003-12-16T00:00:00Z
review text: In the recent paper, \textit{M. Abbad} and \textit{H. Boustique} [Oper. Res. Lett. 31, No. 6, 473--476 (2003; Zbl 1052.90097)] described a state decomposition and an algorithm for an average-reward Markov Decision Process (MDP) with finite state and action sets. The advantage of this algorithm is that, for a multichain problem, it solves a sequence of smaller problems. In fact, a version of this decomposition and this algorithm were described in 1973 in the paper by \textit{J. Bather} [Adv. Appl. Probab. 5, 541--553 (1973; Zbl 0275.90050)] which is not mentioned in Abbad and Boustique (loc. cit.). The current paper applies the same decomposition to compute optimal policies for discounted MDPs. It also describes an algorithm to find an ultimately stationary \(\epsilon\)-optimal policy for a criterion that is a linear combination of average rewards per unit time and discounted rewards. Though the authors assume that the average and discounted rewards are calculated for the same reward function, this assumption is not essential.
reviewed by: Eugene A. Feinberg
Mathematics Subject Classification ID: 90C40
Mathematics Subject Classification ID: 90C39
zbMATH DE Number: 2018327
zbMATH Keywords: Markov decision process
zbMATH Keywords: discounted rewards
zbMATH Keywords: weighted criterion
zbMATH Keywords: decomposition
MaRDI profile type: Publication
full work available at URL: s001860300290
OpenAlex ID: W2030072125
DOI: 10.1007/S001860300290
description: scientific article; zbMATH DE number 2018327