Necessary and sufficient conditions for a bounded solution to the optimality equation in average reward Markov decision chains
type: Item
instance of: scholarly article
title: Necessary and sufficient conditions for a bounded solution to the optimality equation in average reward Markov decision chains
zbMATH Open document ID: 0645.90099
DOI: 10.1016/0167-6911(88)90043-6
author: Rolando Cavazos-Cadena
published in: Systems \& Control Letters
publication date: 1988-01-01T00:00:00Z
review text: Consider a discrete time Markov decision process with countable state space S. In addition to the standard assumptions of compact action sets and continuous transition probabilities, suppose that the Markov chain determined by each stationary policy f has a single positive recurrent class R(f), which is entered with probability one and which contains at least one member of a fixed, finite subset G of S. The main theorem gives, under these assumptions, five necessary and sufficient conditions (including a simultaneous Doeblin condition with set G) for the average reward optimality equation to have a bounded measurable solution for an arbitrary bounded measurable reward function. The establishment of necessity is an uncommon feature; sufficient conditions are discussed in \textit{L. C. Thomas} [``Connectedness conditions for denumerable state Markov decision processes'', in: Recent developments in Markov decision processes, R. Hartley, L. C. Thomas, D. J. White (eds.), Academic Press (1980; Zbl 0547.90064)].
Mathematics Subject Classification ID: 90C40
zbMATH DE Number: 4053391
zbMATH Keywords: optimal stationary policies
zbMATH Keywords: discrete time Markov decision process
zbMATH Keywords: countable state space
zbMATH Keywords: simultaneous Doeblin condition
zbMATH Keywords: average reward optimality equation
zbMATH Keywords: bounded measurable reward function
MaRDI profile type: Publication
full work available at URL: 0167-6911(88)90043-6
OpenAlex ID: W2094140762
cites work: Q4606219
cites work: Two competing queues with linear costs and geometric service requirements: the μc-rule is often optimal
cites work: Necessary conditions for the optimality equation in average-reward Markov decision processes
cites work: Existence of optimal stationary policies in average reward Markov decision processes with a recurrent state
cites work: A note on simultaneous recurrence conditions on a set of denumerable stochastic matrices
cites work: Q5599448
cites work: Q4771778
cites work: Q4131338
cites work: Q5615108
cites work: A new condition for the existence of optimal stationary policies in average cost Markov decision processes
description: scientific article; zbMATH DE number 4053391