On the Convergence of Policy Iteration in Stationary Dynamic Programming

DOI10.1287/moor.4.1.60zbMath0411.90072OpenAlexW2042680115MaRDI QIDQ4198357

Shelby Brumelle, Martin L. Puterman

Publication date: 1979

Published in: Mathematics of Operations Research (Search for Journal in Brave)

Full work available at URL: https://doi.org/10.1287/moor.4.1.60

zbMATH Keywords

rate of convergenceerror boundscompact action spaceprogramming in abstract spacespolicy iteration methodpartially ordered normed linear spacesfinite state Markovian decision problemNewton-Kantorovich iteration procedurestationary dynamic programming

Mathematics Subject Classification ID

Minimax problems in mathematical programming (90C47) Newton-type methods (49M15) Normed linear spaces and Banach spaces; Banach lattices (46B99) Dynamic programming (90C39) Programming in abstract spaces (90C48) Markov and semi-Markov decision processes (90C40) Rate of convergence, degree of approximation (41A25)

Related Items (42)

Multilevel techniques for the solution of HJB minimum-time control problems ⋮ Undiscounted control policy generation for continuous-valued optimal control by approximate dynamic programming ⋮ Rates of convergence for the policy iteration method for mean field games systems ⋮ Numerical approximation of equations involving minimal/maximal operators by successive solution of obstacle problems ⋮ Approximations and Optimal Control for State-Dependent Limited Processor Sharing Queues ⋮ Approximating Optimal feedback Controllers of Finite Horizon Control Problems Using Hierarchical Tensor Formats ⋮ Discrete dynamic programming and viscosity solutions of the Bellman equation ⋮ Survey of linear programming for standard and nonstandard Markovian control problems. Part I: Theory ⋮ (Approximate) iterated successive approximations algorithm for sequential decision processes ⋮ On linear and super-linear convergence of natural policy gradient algorithm ⋮ On the convergence of policy iteration for controlled diffusions ⋮ Value-Gradient Based Formulation of Optimal Control Problem and Machine Learning Algorithm ⋮ Optimal investment strategies for pension funds with regulation-conform dynamic pension payment management in the absence of guarantees ⋮ Optimal polynomial feedback laws for finite horizon control problems ⋮ Policy iteration method for time-dependent mean field games systems with non-separable Hamiltonians ⋮ Unique Tarski Fixed Points ⋮ A note on generalized second-order value iteration in Markov decision processes ⋮ Exponential Convergence and Stability of Howard's Policy Improvement Algorithm for Controlled Diffusions ⋮ A semi-Lagrangian scheme for a modified version of the Hughes' model for Pedestrian flow ⋮ Recent Results in the Approximation of Nonlinear Optimal Control Problems ⋮ Unnamed Item ⋮ A semi-Lagrangian algorithm in policy space for hybrid optimal control problems ⋮ A discrete Hughes model for pedestrian flow on graphs ⋮ A mean field games model for finite mixtures of Bernoulli and categorical distributions ⋮ Optimal consumption under uncertainty, liquidity constraints, and bounded rationality ⋮ Domain decomposition based parallel Howard's algorithm ⋮ Policy iteration and Newton-Raphson methods for Markov decision processes under average cost criterion ⋮ Multigrid methods for two‐player zero‐sum stochastic games ⋮ Unnamed Item ⋮ A Fixed Point Approach to Undiscounted Markov Renewal Programs ⋮ A neural network-based policy iteration algorithm with global \(H^2\)-superlinear convergence for stochastic games on domains ⋮ A numerical method for pricing European options with proportional transaction costs ⋮ The primal-dual active set method for a class of nonlinear problems with \(T\)-monotone operators ⋮ A policy iteration method for mean field games ⋮ NUMERICAL METHODS FOR DIFFERENTIAL GAMES BASED ON PARTIAL DIFFERENTIAL EQUATIONS ⋮ Tensor Decomposition Methods for High-dimensional Hamilton--Jacobi--Bellman Equations ⋮ Optimal price-threshold control for battery operation with aging phenomenon: a quasiconvex optimization approach ⋮ Applications of Markov chain approximation methods to optimal control problems in economics ⋮ Continuous vs. discrete time: some computational insights ⋮ Two-scale methods for convex envelopes ⋮ An Accelerated Value/Policy Iteration Scheme for Optimal Control Problems and Games ⋮ The variational calculus and approximation in policy space for Markovian decision processes

This page was built for publication: On the Convergence of Policy Iteration in Stationary Dynamic Programming