Improved iterative computation of the expected discounted return in Markov and semi-Markov chains

DOI10.1007/BF01919243zbMath0442.90104OpenAlexW2063100889MaRDI QIDQ3885559

Evan L. Porteus

Publication date: 1980

Published in: Zeitschrift für Operations Research (Search for Journal in Brave)

Full work available at URL: https://doi.org/10.1007/bf01919243

zbMATH Keywords

convergencetest problemsfinite Markov chainscomputational experiencesemi-Markov chainsiterative computationexpected discounted returnnorm reducing extrapolations

Mathematics Subject Classification ID

Numerical mathematical programming methods (65K05) Markov renewal processes, semi-Markov processes (60K15) Markov and semi-Markov decision processes (90C40)

Related Items (7)

Serial and parallel value iteration algorithms for discounted Markov decision processes ⋮ Computational comparison of policy iteration algorithms for discounted Markov decision processes ⋮ Generic rank-one corrections for value iteration in Markovian decision problems ⋮ On efficiency of linear programming applied to discounted Markovian decision problems ⋮ (Approximate) iterated successive approximations algorithm for sequential decision processes ⋮ Conditions for characterizing the structure of optimal strategies in infinite-horizon dynamic programs ⋮ Second order bounds for Markov decision processes

Cites Work

• Unnamed Item
• Unnamed Item
• Unnamed Item
• Unnamed Item
• Unnamed Item
• Unnamed Item
• Unnamed Item
• Unnamed Item
• Unnamed Item
• Unnamed Item
• Unnamed Item
• Unnamed Item
• Estimates for finite-stage dynamic programs
• A successive approximation algorithm for an undiscounted Markov decision process
• Discounted Markov games: Generalized policy iteration method
• Contraction mappings underlying undiscounted Markov decision problems
• Dynamic programming, Markov chains, and the method of successive approximations
• A modified dynamic programming method for Markovian decision problems
• Iterative solution of the functional equations of undiscounted Markov renewal programming
• Bounds and Transformations for Discounted Finite Markov Decision Chains
• A decision exclusion algorithm for a class of Markovian Decision Processes
• Discounting, Ergodicity and Convergence for Markov Decision Processes
• A set of successive approximation methods for discounted Markovian decision problems
• Technical Note—Accelerated Computation of the Expected Discounted Return in a Markov Chain
• Modified Policy Iteration Algorithms for Discounted Markov Decision Problems
• Geometric convergence of value-iteration in multichain Markov decision problems
• Markov-Renewal Programming. I: Formulation, Finite Return Models
• Contraction Mappings in the Theory Underlying Dynamic Programming
• On computing the expected discounted return in a markov chain
• Technical Note—On the Asymptotic Convergence Rate of Cost Differences for Markovian Decision Processes
• Technical Note—Undiscounted Markov Renewal Programming Via Modified Successive Approximations
• Some Bounds for Discounted Sequential Decision Processes
• A Generalization of the Additive Correction Methods for the Iterative Solution of Matrix Equations

This page was built for publication: Improved iterative computation of the expected discounted return in Markov and semi-Markov chains