Actor-Critic Method for High Dimensional Static Hamilton--Jacobi--Bellman Partial Differential Equations based on Neural Networks

DOI10.1137/21M1402303zbMath1481.65203arXiv2102.11379WikidataQ115246863 ScholiaQ115246863MaRDI QIDQ5021407

Jiequn Han, Mo Zhou, Jian-feng Lu

Publication date: 13 January 2022

Published in: SIAM Journal on Scientific Computing (Search for Journal in Brave)

Full work available at URL: https://arxiv.org/abs/2102.11379

zbMATH Keywords

stochastic controlHamilton-Jacobi-Bellman equationsactor-critic methodshigh dimensional partial differential equations

Mathematics Subject Classification ID

Artificial neural networks and deep learning (68T07) Numerical mathematical programming methods (65K05) Nonlinear elliptic equations (35J60) Optimal stochastic control (93E20) Numerical solutions to stochastic differential and integral equations (65C30) Probabilistic methods, particle methods, etc. for initial value and initial-boundary value problems involving PDEs (65M75) Hamilton-Jacobi equations in optimal control and differential games (49L12)

Related Items

SympOCnet: Solving Optimal Control Problems with Applications to High-Dimensional Multiagent Path Planning Problems, Deep reinforcement learning of viscous incompressible flow, Approximating Optimal feedback Controllers of Finite Horizon Control Problems Using Hierarchical Tensor Formats, Data-Driven Tensor Train Gradient Cross Approximation for Hamilton–Jacobi–Bellman Equations, Solving non-linear Kolmogorov equations in large dimensions by using deep learning: a numerical comparison of discretization schemes, Deep reinforcement learning in finite-horizon to explore the most probable transition pathway, Approximative Policy Iteration for Exit Time Feedback Control Problems Driven by Stochastic Differential Equations using Tensor Train Format

Uses Software

• TensorFlow
• Adam
• GitHub

Cites Work

• Unnamed Item
• Unnamed Item
• Unnamed Item
• Unnamed Item
• Unnamed Item
• Adapted solution of a backward stochastic differential equation
• Deep learning-based numerical methods for high-dimensional parabolic partial differential equations and backward stochastic differential equations
• Solving high-dimensional Hamilton-Jacobi-Bellman PDEs using neural networks: perspectives from the theory of controlled diffusions and measures on path space
• Online actor-critic algorithm to solve the continuous-time infinite horizon optimal control problem
• Natural actor-critic algorithms
• Fronts propagating with curvature-dependent speed: Algorithms based on Hamilton-Jacobi formulations
• Approximate solutions to the time-invariant Hamilton-Jacobi-Bellman equation
• Galerkin approximations of the generalized Hamilton-Jacobi-Bellman equation
• Numerical solution of Hamilton-Jacobi-Bellman equations by an upwind finite volume method
• Overapproximating reachable sets by Hamilton-Jacobi projections
• Solving Dirichlet problems numerically using the Feynman-Kac representation
• Mitigating the curse of dimensionality: sparse grid characteristics method for optimal feedback control and HJB equations
• Weak approximation of killed diffusion using Euler schemes.
• On reachability and minimum cost optimal control
• Neural networks-based backward scheme for fully nonlinear PDEs
• Recurrent neural networks for stochastic control problems with delay
• Solving high-dimensional eigenvalue problems using deep neural networks: a diffusion Monte Carlo like approach
• A derivative-free method for solving elliptic partial differential equations with deep neural networks
• Convergence of the deep BSDE method for coupled FBSDEs
• Machine learning for semi linear PDEs
• Probabilistic interpretation for systems of quasilinear parabolic partial differential equations
• User’s guide to viscosity solutions of second order partial differential equations
• OnActor-Critic Algorithms
• On the convergence rate of approximation schemes for Hamilton-Jacobi-Bellman Equations
• Polynomial Approximation of High-Dimensional Hamilton--Jacobi--Bellman Equations and Applications to Feedback Control of Semilinear Parabolic PDEs
• HJB-POD-Based Feedback Design for the Optimal Control of Evolution Problems
• Solving high-dimensional partial differential equations using deep learning
• Adaptive Deep Learning for High-Dimensional Hamilton--Jacobi--Bellman Equations
• Tensor Decomposition Methods for High-dimensional Hamilton--Jacobi--Bellman Equations
• Algorithms for solving high dimensional PDEs: from nonlinear Monte Carlo to machine learning
• A time-dependent Hamilton-Jacobi formulation of reachable sets for continuous dynamic games
• Deep optimal stopping
• Numerical solution of Hamilton–Jacobi–Bellman equations by an exponentially fitted finite volume method