Nonstationary value-iteration and adaptive control of discounted semi- Markov processes
type: Item
instance of: scholarly article
title: Nonstationary value-iteration and adaptive control of discounted semi- Markov processes
zbMATH Open document ID: 0581.90096
DOI: 10.1016/0022-247X(85)90253-7
author: Onésimo Hernández-Lerma
published in: Journal of Mathematical Analysis and Applications
publication date: 1985-01-01T00:00:00Z
review text: We consider in this paper discounted-reward, denumerable state space, semi-Markov decision processes which depend on unknown parameters. The problems we are interested in are: Given that the true parameter value is unknown, (I) give an iterative scheme to determine the total maximal discounted reward, and (II) find an asymptotically discount optimal (adaptive) policy. Our solutions are inspired by the nonstationary value iteration (NVI) scheme of \textit{A. Federgruen} and \textit{P. J. Schweitzer} [J. Optimization Theory Appl. 34, 207-241 (1981; Zbl 0426.90091)] combined with the ideas of \textit{M. Schäl} [in: Optimization, theory and algorithms, Conf. Confolant/France 1981, Lect. Notes Pure Appl. Math. 86, 239-253 (1983; Zbl 0525.93071)] concerning the ''principle of estimation and control'' for the adaptive control of semi-Markov processes.
Mathematics Subject Classification ID: 90C40
zbMATH DE Number: 3930759
zbMATH Keywords: discounted-reward
zbMATH Keywords: denumerable state space
zbMATH Keywords: semi-Markov decision processes
zbMATH Keywords: unknown parameters
zbMATH Keywords: nonstationary value iteration
zbMATH Keywords: principle of estimation and control
zbMATH Keywords: adaptive control of semi-Markov processes
MaRDI profile type: Publication
cites work: Dynamic programming and stochastic control
cites work: Nonstationary Markov decision problems with converging parameters
cites work: Q4150452
cites work: Adaptive control of service in queueing systems
cites work: Adaptive control of discounted Markov decision chains
cites work: Q5599448
cites work: The average-optimal adaptive control of a Markov renewal model in presence of an unknown parameter
cites work: Strongly consistent estimation in a controlled Markov renewal model
cites work: Q3875736
cites work: Q5649557
cites work: On Dynamic Programming with Unbounded Rewards
cites work: Estimation and control in Markov chains
cites work: Q5615108
cites work: Conditions for optimality in dynamic programming and for the limit of n-stage optimal policies to be optimal
cites work: Estimation and control in discounted stochastic dynamic programming
cites work: Q4772533
full work available at URL: 0022-247x(85)90253-7
OpenAlex ID: W1975110715
description: scientific article; zbMATH DE number 3930759