Non-homogeneous Markov decision processes with a constraint
type: Item
instance of: scholarly article
title: Non-homogeneous Markov decision processes with a constraint
zbMATH Open document ID: 0911.90342
author: Tetsuo Iida
published in: Journal of Mathematical Analysis and Applications
publication date: 1999-04-20T00:00:00Z
review text: This paper considers a finite state and action non-homogeneous Markov decision process with finite state and action sets. There are two reward functions. The goal is to maximize average rewards per unit time for one function subject to the constraint that the total discounted reward for another function is equal to a given value. For unconstrained problems, \textit{W. Hopp, J. Bean} and \textit{R. Smith} [Oper. Res. 35, No. 6, 875-883 (1987; Zbl 0651.90090)] introduced the notion of periodic forecast horizon optimality which was later studied by \textit{J. Bean, R. Smith} and \textit{J. Lasserre} [Math. Oper. Res. 9, 391-401 (1990)] under the name algorithmic optimality. A policy \(\pi\) is algorithmic optimal if there is a sequence of optimal \(N\)-horizon policies that converge to \(\pi\) as \(N\to\infty\). In this paper, the author defines algorithmic optimal policies for problems with the described constraint and provides conditions under which these policies are average optimal among policies satisfying this constraint.
Mathematics Subject Classification ID: 90C40
zbMATH DE Number: 1115517
zbMATH Keywords: constrained Markov decision process
zbMATH Keywords: algorithmic optimality
zbMATH Keywords: average optimality
reviewed by: Eugene A. Feinberg
MaRDI profile type: Publication
full work available at URL: jmaa.1997.5610
OpenAlex ID: W1991031672
cites work: Conditions for the Existence of Planning Horizons
cites work: Denumerable state nonhomogeneous Markov decision processes
cites work: Concepts of Forecast and Decision Horizons: Applications to Dynamic Stochastic Optimization Problems
cites work: Variance-Penalized Markov Decision Processes
cites work: Q4271761
cites work: A New Optimality Criterion for Nonhomogeneous Markov Decision Processes
cites work: Constrained Undiscounted Stochastic Dynamic Programming
cites work: Q4098444
cites work: Q3912356
cites work: Technical Note—Dynamic Programming and Probabilistic Constraints
cites work: Utility, probabilistic constraints, mean and variance of discounted rewards in Markov decision processes
cites work: Mean, variance and probabilistic criteria in finite Markov decision processes: A review
DOI: 10.1006/JMAA.1997.5610
Recommended article: Q3347664
Recommended article: Non-randomized policies for constrained Markov decision processes
Recommended article: On constrained Markov decision processes
Recommended article: Q4271761
Recommended article: Q4264741
Recommended article: Constrained continuous-time Markov decision processes on the finite horizon
Recommended article: Constrained Markov decision processes with uncertain costs
Recommended article: Constrained Markovian decision processes: The dynamic programming approach
description: scientific article; zbMATH DE number 1115517