Markov control process with the expected total cost criterion: Optimality, stability, and transient models
type: Item
instance of: scholarly article
title: Markov control process with the expected total cost criterion: Optimality, stability, and transient models
zbMATH Open document ID: 0964.93086
DOI: 10.1023/A:1006368714127
author: Guadalupe Carrasco
author: Rubén Pérez-Hernández
author: Onésimo Hernández-Lerma
published in: Acta Applicandae Mathematicae
publication date: 2000-08-29T00:00:00Z
review text: The authors study discrete-time Markov Control Processes (MCPs) on Borel spaces under the Expected Total Cost (ETC) criterion \[ V(\pi, x)= E^\pi_x\Biggl[ \sum^\infty_{t= 0} c(x_ t,a_t)\Biggr], \] where \(c(x_t, a_t)\) is the cost-per-stage function and is possibly unbounded [for the basic concepts and notations of the MCPs, cf. \textit{O. Hernández-Lerma} and \textit{J. B. Lasserre}, Discrete-time Markov control processes: Basic optimality criteria, Springer-Verlag, New York (1995; Zbl 0840.93001)]. A lot of optimality questions are answered affirmatively here. Conditions for a control policy to be ETC-optimal and conditions for the ETC-value function to be a solution to the dynamic programming equation are well provided. It is also shown that the finiteness of the ETC function may lead to two kinds of stability: Lagrange stability and stability with probability one. In addition, transient control models [cf. \textit{S. R. Pliska}, Dynamic programming and its applications, Proc. Int. Conf., Vancouver 1977, 335-349 (1978; Zbl 0458.90082)] are fully analyzed. In fact, with the authors' new results, the paper provides a fairly complete, up-dated, survey-like presentation of the ETC criterion for MCPs.
reviewed by: Chengxun Wu
Mathematics Subject Classification ID: 93E20
Mathematics Subject Classification ID: 90C40
zbMATH DE Number: 1436944
zbMATH Keywords: policy iteration
zbMATH Keywords: discrete-time Markov control processes
zbMATH Keywords: expected total cost
zbMATH Keywords: dynamic programming
zbMATH Keywords: Lagrange stability
zbMATH Keywords: stability with probability one
zbMATH Keywords: transient control models
MaRDI profile type: Publication
full work available at URL: a:1006368714127
OpenAlex ID: W1920513863
description: scientific article; zbMATH DE number 1436944