Nonstationary Markov decision problems with converging parameters

DOI10.1007/BF00935474zbMath0426.90091MaRDI QIDQ1136706

Awi Federgruen, Paul J. Schweitzer

Publication date: 1981

Published in: Journal of Optimization Theory and Applications (Search for Journal in Brave)

zbMATH Keywords

convergence ratesasymptotic behavioroptimal policiesapproximations for parametersdiscounted versionmodified value-iteration methodnon-stationary Markov decision problemsundiscounted version

Mathematics Subject Classification ID

Minimax problems in mathematical programming (90C47) Rate of convergence, degree of approximation (41A25)

Related Items

Monotone value iteration for discounted finite Markov decision processes ⋮ Finite-state approximations for denumerable multidimensional state discounted Markov decision processes ⋮ A unified approach to adaptive control of average reward Markov decision processes ⋮ Adaptive policy-iteration and policy-value-iteration for discounted Markov decision processes ⋮ Adaptive policies for discrete-time stochastic control systems with unknown disturbance distribution ⋮ Central limit theorem for the estimator of the value of an optimal stopping problem ⋮ On confidence intervals from simulation of finite Markov chains ⋮ Nonparametric adaptive control of discrete-time partially observable stochastic systems ⋮ Discretization procedures for adaptive Markov control processes ⋮ A value-iteration scheme for undiscounted multichain Markov renewal programs ⋮ Adaptive discounted control for piecewise deterministic Markov processes ⋮ The rate of convergence for backwards products of a convergent sequence of finite Markov matrices ⋮ Nonparametric estimation and adaptive control in a class of finite Markov decision chains ⋮ Computationally efficient algorithms for on-line optimization of Markov decision processes ⋮ Unnamed Item ⋮ Recursive adaptive control of Markov decision processes with the average reward criterion ⋮ Computing Optimal Policies for Markovian Decision Processes Using Simulation ⋮ Nonstationary value iteration in controlled Markov chains with risk-sensitive average criterion ⋮ Unnamed Item ⋮ Adaptive control of discounted Markov decision chains ⋮ Nonstationary value-iteration and adaptive control of discounted semi- Markov processes ⋮ Existence of optimal policy for time non-homogeneous discounted Markovian decision programming

Cites Work

• Unnamed Item
• Unnamed Item
• Unnamed Item
• Unnamed Item
• The rate of convergence for backwards products of a convergent sequence of finite Markov matrices
• Exponential convergence of products of stochastic matrices
• Contraction mappings underlying undiscounted Markov decision problems
• Dynamic programming, Markov chains, and the method of successive approximations
• Iterative solution of the functional equations of undiscounted Markov renewal programming
• Applying a New Device in the Optimization of Exponential Queuing Systems
• Note—A Test for Nonoptimal Actions in Undiscounted Finite Markov Decision Chains
• The rate of convergence of certain nonhomogeneous Markov chains
• A general markov decision method II: Applications
• Towards consensus: some convergence theorems on repeated averaging
• Geometric convergence of value-iteration in multichain Markov decision problems
• The Asymptotic Behavior of Undiscounted Value Iteration in Markov Decision Problems
• Markov-Renewal Programming. I: Formulation, Finite Return Models
• On the Iterative Method of Dynamic Programming on a Finite Space Discrete Time Markov Process
• Contraction Mappings in the Theory Underlying Dynamic Programming
• Étude asymptotique des systèmes markoviens à commande
• Letter to the Editor—A Test for Suboptimal Actions in Markovian Decision Problems
• Discrete Dynamic Programming with a Small Interest Rate
• Discrete Dynamic Programming with Sensitive Discount Optimality Criteria
• On Finding the Maximal Gain for Markov Decision Processes
• Perturbation theory and finite Markov chains
• Multichain Markov Renewal Programs
• Some Bounds for Discounted Sequential Decision Processes
• Markov Renewal Programs with Small Interest Rates
• An Optimal Policy for Operating a Multipurpose Reservoir
• Tests for Suboptimal Actions in Discounted Markov Programming
• Technical Note—Elimination of Suboptimal Actions in Markov Decision Problems
• Quasi-Newton Methods for Unconstrained Optimization
• Stochastic Games