Projected equation methods for approximate solution of large linear systems

DOI10.1016/j.cam.2008.07.037zbMath1165.65010OpenAlexW2019172585MaRDI QIDQ1012492

Huizhen Yu, Dimitri P. Bertsekas

Publication date: 21 April 2009

Published in: Journal of Computational and Applied Mathematics (Search for Journal in Brave)

Full work available at URL: https://doi.org/10.1016/j.cam.2008.07.037

zbMATH Keywords

algorithmssimulationdynamic programmingMarkov chainsiteration methodlinear equationsJacobi methodBellman's equationvalue iterationoptimal stopping problemslinear least squares problemstemporal differencesMarkovian decision problemsnonlinear fixed point problemsprojected equations

Mathematics Subject Classification ID

Computational methods in Markov chains (60J22) Numerical solutions to overdetermined systems, pseudoinverses (65F20) Monte Carlo methods (65C05) Dynamic programming in optimal control and differential games (49L20) Applications of Markov chains and discrete-time Markov processes on general state spaces (social mobility, learning theory, industrial processes, etc.) (60J20) Numerical analysis or methods applied to Markov chains (65C40) Iterative numerical methods for linear systems (65F10)

Related Items

Approximate policy iteration: a survey and some new methods, Q-learning and policy iteration algorithms for stochastic shortest path problems, A sketched finite element method for elliptic models, Temporal difference-based policy iteration for optimal control of stochastic systems, Approximate dynamic programming via direct search in the space of value function approximations, Proximal algorithms and temporal difference methods for solving fixed point problems, Off-policy temporal difference learning with distribution adaptation in fast mixing chains, Transmission scheduling for multi-process multi-sensor remote estimation via approximate dynamic programming, Unnamed Item

Cites Work

• Unnamed Item
• Unnamed Item
• Unnamed Item
• Unnamed Item
• Unnamed Item
• A generalized Kalman filter for fixed point approximation and efficient temporal-difference learning
• Technical update: Least-squares temporal difference learning
• Average cost temporal-difference learning
• Least squares policy evaluation algorithms with linear function approximation
• Basis function adaptation in temporal difference reinforcement learning
• A Learning Algorithm for Risk-Sensitive Cost
• An analysis of temporal-difference learning with function approximation
• Optimal stopping of Markov processes: Hilbert space theory, approximation algorithms, and an application to pricing high-dimensional financial derivatives
• A Retrospective and Prospective Survey of the Monte Carlo Method
• A Note on the Inversion of Matrices by Random Walks
• “Monte Carlo” Methods for the Iteration of Linear Operators
• Monte Carlo strategies in scientific computing