A time aggregation approach to Markov decision processes
type: Item
instance of: scholarly article
title: A time aggregation approach to Markov decision processes
zbMATH Open document ID: 1026.93054
DOI: 10.1016/S0005-1098(01)00282-5
author: Zhiyuan Ren
author: Shalabh Bhatnagar
author: Cao, Xiren
author: Michael C. Fu
author: Steven I. Marcus
published in: Automatica
publication date: 2002-09-05T00:00:00Z
review text: An infinite horizon average cost control problem for discrete time ergodic Markov chains is considered. A time aggregation approach is proposed, by which policy iteration of the original problem is replaced by a series of policy iterations on nonintersecting subsets of the state space, using the associated embedded Markov chains and equivalent performance functions. Single sample path-based estimation algorithms are presented. The results are illustrated by numerical and simulation examples.
Mathematics Subject Classification ID: 93E20
Mathematics Subject Classification ID: 90C40
Mathematics Subject Classification ID: 93C55
Mathematics Subject Classification ID: 93E10
zbMATH DE Number: 1797073
zbMATH Keywords: sample path estimation algorithms
zbMATH Keywords: discrete time ergodic Markov chains
reviewed by: Henrikas Pragarauskas
MaRDI profile type: Publication
cites work: Aggregation of the policy iteration method for nearly completely decomposable Markov chains
cites work: Q4257216
cites work: The relations among potentials, perturbation analysis, and Markov decision processes
cites work: Single sample path-based optimization of Markov chains
cites work: A unified approach to Markov decision problems and performance sensitivity analysis
cites work: Dependability for systems with a partitioned state space: Markov and semi-Markov theory and computational implementation
cites work: Multilayer control of large Markov chains
cites work: Simulation-based optimization of Markov reward processes
cites work: Q3326564
cites work: Using Randomization to Break the Curse of Dimensionality
cites work: Between MDPs and semi-MDPs: A framework for temporal abstraction in reinforcement learning
cites work: Average cost temporal-difference learning
cites work: \({\mathcal Q}\)-learning
cites work: Performance gradient estimation for the very large finite Markov chains
description: scientific article; zbMATH DE number 1797073