A neural network-based policy iteration algorithm with global \(H^2\)-superlinear convergence for stochastic games on domains

DOI10.1007/s10208-020-09460-1zbMath1472.82030arXiv1906.02304OpenAlexW3027682703WikidataQ114852444 ScholiaQ114852444MaRDI QIDQ2031059

Christoph Reisinger, Kazufumi Ito, Yu-Fei Zhang

Publication date: 8 June 2021

Published in: Foundations of Computational Mathematics (Search for Journal in Brave)

Full work available at URL: https://arxiv.org/abs/1906.02304

zbMATH Keywords

neural networkspolicy iteration\(q\)-superlinear convergenceinexact semismooth Newton methodHamilton-Jacobi-Bellman-Isaacs equationsglobal conergence

Mathematics Subject Classification ID

Existence of solutions for minimax problems (49J35) Finite difference methods for initial value and initial-boundary value problems involving PDEs (65M06) Stability and convergence of numerical methods for initial value and initial-boundary value problems involving PDEs (65M12) Stochastic games, stochastic differential games (91A15) Neural nets applied to problems in time-dependent statistical mechanics (82C32) Uniqueness problems for PDEs: global uniqueness, local uniqueness, non-uniqueness (35A02) Strong solutions to PDEs (35D35) Finite difference methods applied to problems in statistical mechanics (82M20)

Related Items (8)

Extensions of the deep Galerkin method ⋮ Optimal polynomial feedback laws for finite horizon control problems ⋮ Unnamed Item ⋮ Linear Convergence of a Policy Gradient Method for Some Finite Horizon Continuous Time Control Problems ⋮ State-dependent Riccati equation feedback stabilization for nonlinear PDEs ⋮ Deep ReLU network expression rates for option prices in high-dimensional, exponential Lévy models ⋮ Time-extremal navigation in arbitrary winds on conformally flat Riemannian manifolds ⋮ Regularity and Stability of Feedback Relaxed Controls

Uses Software

• Adam
• DGM

Cites Work

• Unnamed Item
• Unnamed Item
• Unnamed Item
• Unnamed Item
• Unnamed Item
• Unnamed Item
• Unnamed Item
• The stochastic reach-avoid problem and set characterization for diffusions
• Deep learning-based numerical methods for high-dimensional parabolic partial differential equations and backward stochastic differential equations
• Backward stochastic differential equations with jumps and related nonlinear expectations
• Least-squares finite element methods
• Galerkin approximations of the generalized Hamilton-Jacobi-Bellman equation
• The Deep Ritz Method: a deep learning-based numerical algorithm for solving variational problems
• DGM: a deep learning algorithm for solving partial differential equations
• Convergence of the deep BSDE method for coupled FBSDEs
• On the dynamic programming principle for uniformly nondegenerate stochastic differential games in domains and the Isaacs equations
• Generalized Hamilton--Jacobi--Bellman Equations with Dirichlet Boundary Condition and Stochastic Exit Time Optimal Control Problem
• Penalty Methods for the Solution of Discrete HJB Equations—Continuous Control and Obstacle Problems
• Robust Feedback Control of Nonlinear PDEs by Numerical Approximation of High-Dimensional Hamilton--Jacobi--Isaacs Equations
• An Efficient Policy Iteration Algorithm for Dynamic Programming Equations
• Some Convergence Results for Howard's Algorithm
• Stochastic differential equations with reflecting boundary conditions
• On the Convergence of Policy Iteration in Stationary Dynamic Programming
• Semi–Smooth Newton Methods for Variational Inequalities of the First Kind
• The Primal-Dual Active Set Strategy as a Semismooth Newton Method
• Smoothing Methods and Semismooth Methods for Nondifferentiable Operator Equations
• $H^2$-Convergence of Least-Squares Kernel Collocation Methods
• Polynomial Approximation of High-Dimensional Hamilton--Jacobi--Bellman Equations and Applications to Feedback Control of Semilinear Parabolic PDEs
• Convergence Properties of Policy Iteration
• Sobolev and Viscosity Solutions for Fully Nonlinear Elliptic and Parabolic Equations
• Generalizations of the Dennis--Moré Theorem
• Generalized Halton sequences in 2008
• Successive Galerkin approximation algorithms for nonlinear optimal and robust control
• Exponential Convergence and Stability of Howard's Policy Improvement Algorithm for Controlled Diffusions
• Deep Neural Networks Algorithms for Stochastic Control Problems on Finite Horizon: Convergence Analysis
• Error Estimates of Penalty Schemes for Quasi-Variational Inequalities Arising from Impulse Control Problems
• Discontinuous Galerkin Finite Element Approximation of Hamilton--Jacobi--Bellman Equations with Cordes Coefficients
• Set-valued analysis

This page was built for publication: A neural network-based policy iteration algorithm with global \(H^2\)-superlinear convergence for stochastic games on domains