A sparse Markov chain approximation of LQ-type stochastic control problems.

DOI10.3934/mcrf.2016007zbMath1350.49031OpenAlexW2207863317MaRDI QIDQ326794

Ralf Banisch, Carsten Hartmann

Publication date: 12 October 2016

Published in: Mathematical Control and Related Fields (Search for Journal in Brave)

Full work available at URL: https://doi.org/10.3934/mcrf.2016007

zbMATH Keywords

dynamic programmingstochastic optimal controlMarkov chain approximationGalerkin discretization schemelogarithmic transformationMarkov state model

Mathematics Subject Classification ID

Dynamic programming in optimal control and differential games (49L20) Dynamic programming (90C39) Markov chains (discrete-time Markov processes on discrete state spaces) (60J10) Finite element, Rayleigh-Ritz and Galerkin methods for boundary value problems involving PDEs (65N30) Optimal stochastic control (93E20) Markov and semi-Markov decision processes (90C40) Discrete approximations in optimal control (49M25) Existence of optimal solutions to problems involving randomness (49J55)

Related Items

Overcoming the timescale barrier in molecular dynamics: Transfer operators, variational principles and machine learning ⋮ Transition Path Theory for Langevin Dynamics on Manifolds: Optimal Control and Data-Driven Solver ⋮ Learning-based importance sampling via stochastic optimal control for stochastic reaction networks

Uses Software

• Gromacs
• LINCS

Cites Work

• Unnamed Item
• Unnamed Item
• Unnamed Item
• Unnamed Item
• The minimal entropy martingale measure (MEMM) for a Markov-modulated exponential Lévy model
• Numerical methods for high dimensional Hamilton-Jacobi equations using radial basis functions
• Optimal control of molecular dynamics using Markov state models
• Numerical methods for controlled regime-switching diffusions and regime-switching jump diffusions
• A radial basis collocation method for Hamilton-Jacobi-Bellman equations
• Methods of contemporary mathematical statistical physics. Lecture notes presented at the 5th Prague summer school on mathematical statistical physics, Prague, Czech Republik, September 2006
• Multi-grid methods for Hamilton-Jacobi-Bellman equations
• Exit probabilities and optimal stochastic control
• Connections between stochastic control and dynamic games
• Meshless methods: An overview and recent developments
• Numerical solution of Hamilton-Jacobi-Bellman equations by an upwind finite volume method
• An efficient algorithm for Hamilton-Jacobi equations in high dimension
• The theory of stochastic processes. II: Translated from the Russian by S. Kotz
• An adaptive sparse grid semi-Lagrangian scheme for first order Hamilton-Jacobi Bellman equations
• Convergence of Markov chain approximation on generalized HJB equation and its applications
• Controlled Markov processes and viscosity solutions
• Towards a theory of transition paths
• Optimal control of multiscale systems using reduced-order models
• Ergodicity for SDEs and approximations: locally Lipschitz vector fields and degenerate noise.
• Moving least-squares approximations for linearly-solvable stochastic optimal control problems
• Chemical Reactions as $\Gamma$-Limit of Diffusion
• Importance Sampling for Multiscale Diffusions
• Estimating the Eigenvalue Error of Markov State Models
• Optimal Control of Robot Guided Laser Material Treatment
• On the Approximation Quality of Markov State Models
• Stochastic Control and Exit Probabilities of Jump Processes
• A Survey of Some Applications of Probability and Stochastic Control Theory to Finite Difference Methods for Degenerate Elliptic and Parabolic Equations
• Meshless Galerkin methods using radial basis functions
• Domain decomposition algorithms for solving hamilton-jacobi-bellman equations
• Some Estimates for Finite Difference Approximations
• Brownian motion in a field of force and the diffusion model of chemical reactions
• Finite Elements
• The finite element approximation of Hamilton-Jacobi-Bellman equations
• Stochastic differential equations. An introduction with applications.