An expected average reward criterion
type: Item
instance of: scholarly article
title: An expected average reward criterion
zbMATH Open document ID: 0663.90095
DOI: 10.1016/0304-4149(87)90055-X
author: K.-J. Bierth
published in: Stochastic Processes and their Applications
publication date: 1987-01-01T00:00:00Z
review text: Consider a Markovian decision model with finite state space and a fixed initial state i. Let \[ \bar U(i,\pi)=E_{i\pi}[\overline{\lim}\frac{1}{n}\sum^{n-1}_{t=0}r(X_ t,\quad A_ t)]\quad and\quad \bar U(i)=\sup_{\pi \in \Delta}\bar U(i,\pi), \] where \(r(X_ t,A_ t)\) is a bounded random reward when action \(A_ t\) at time t is chosen in state \(X_ t\) at time t, \(\Delta\) is the set of all policies \(\pi\). Similarly we have \b{U}(i,\(\pi)\) and \b{U}(i). The paper shows that all results for the criteria \[ \bar V(i,\pi)=\overline{\lim}_{n\to \infty}E_{i\pi}[\frac{1}{n}\sum^{n- 1}_{t=0}r(X_ t,A_ t)] \] and \b{V}(i,\(\pi)\) can be carried over to the new criteria \(\bar U\) and \b{U} without the assumption of a special gain function as in \textit{S. Demko} and \textit{T. P. Hill} [ibid. 17, 349- 357 (1984; Zbl 0537.90095)]. There exists some deterministic Markov policy which is (strongly) \(\epsilon\)-optimal for any one of the four criteria. Under some conditions including that the set of actions in state i is a compact metric space, for any \(\epsilon >0\) there exists some deterministic stationary policy which is (strongly) \(\epsilon\)- optimal for any criterion. The author also proves that \b{U}\(=\bar U=\underline V=\bar V\).
Mathematics Subject Classification ID: 90C40
Mathematics Subject Classification ID: 90C39
zbMATH DE Number: 4085450
zbMATH Keywords: expected average reward
zbMATH Keywords: strong \(\epsilon \) -optimal decision
zbMATH Keywords: finite state space
zbMATH Keywords: deterministic stationary policy
MaRDI profile type: Publication
cites work: Optimal decision procedures for finite markov chains. Part I: Examples
cites work: Q5545069
cites work: Q4183968
cites work: On maximizing the average time at a goal
cites work: Q5343895
cites work: Q3254057
cites work: Q3862198
cites work: On Controlled Finite State Markov Processes with Compact Control Sets
cites work: Q4169844
cites work: An $\varepsilon $-Optimal Control of a Finite Markov Chain with an Average Reward Criterion
cites work: Denumerable Undiscounted Semi-Markov Decision Processes with Unbounded Rewards
cites work: Denumerable state semi-Markov decision processes with unbounded costs, average cost criterion
cites work: An Example in Denumerable Decision Processes
cites work: Q4771778
cites work: Q3266141
cites work: Q4165953
cites work: Estimation and control in Markov chains
cites work: Q5615108
cites work: Q3762100
Recommended article: Q4003930
Recommended article: Q3148831
Recommended article: General Discounting Versus Average Reward
Recommended article: Reward revision and the average reward Markov decision process
Recommended article: Q3737133
Recommended article: Q4501317
Recommended article: Q3755254
Recommended article: Q4705461
Recommended article: Q3365301
description: scientific article; zbMATH DE number 4085450