Consistency of a simple multidimensional scheme for Hamilton-Jacobi-Bellman equations
type: Item
instance of: scholarly article
title: Consistency of a simple multidimensional scheme for Hamilton-Jacobi-Bellman equations
zbMATH Open document ID: 1067.60045
author: Rémi Munos
author: Hasnaa Zidani
published in: Comptes Rendus. Mathématique. Académie des Sciences, Paris
publication date: 2005-04-28T00:00:00Z
review text: The authors consider a multi-valued controlled Markov diffusion on \(X=\mathbb R^n\) \((n\geq 1)\), given as \[ x(t)=x + \int_0^tf(x(s),u(s))ds + \int_0^t\sigma (X(s),u(s))\,dW_s, \] where \((W_t)_{0\leq t\leq \infty}\) is a standard Brownian motion. The control \(u\) is a Lebesgue measurable function with values in a compact set. Further, they assume an infinite horizon discounted payoff and the value function as the minimum value of the payoff, then they consider the associated Hamilton-Jacobi-Bellman (HJB) equation. The authors propose an algorithm for the HJB equation, a Markov decision process approximation, using discretised time steps and diffusion steps. Its consistency is proved, which in turn implies its convergence. The algorithm has the advantage of being easy to implement in any dimension.
reviewed by: Evelyn Buckwar
Mathematics Subject Classification ID: 60H10
Mathematics Subject Classification ID: 65C30
Mathematics Subject Classification ID: 65C40
Mathematics Subject Classification ID: 49L20
Mathematics Subject Classification ID: 93E20
Mathematics Subject Classification ID: 60H35
zbMATH DE Number: 2162047
zbMATH Keywords: stochastic optimal control
zbMATH Keywords: Hamilton-Jacobi-Bellman equation of second order
zbMATH Keywords: Markov Decision Process approximation
zbMATH Keywords: dynamic programming
zbMATH Keywords: convergence
MaRDI profile type: Publication
full work available at URL: j.crma.2005.02.001
OpenAlex ID: W2031520946
cites work: Consistency of Generalized Finite Difference Schemes for the Stochastic HJB Equation
cites work: A fast algorithm for the two dimensional HJB equation of stochastic control
cites work: An approximation scheme for the optimal control of diffusion processes
cites work: Q2703816
cites work: Q3923524
cites work: Variable resolution discretization in optimal control
DOI: 10.1016/J.CRMA.2005.02.001
Recommended article: Hamilton–Jacobi–Bellman Equations on Multi-domains
Recommended article: Convergence results for Hamilton-Jacobi-Bellman equations in variable domains
Recommended article: Existence and multiplicity for Hamilton-Jacobi-Bellman equation
Recommended article: A relaxation scheme for Hamilton-Jacobi-Bellman equations
Recommended article: Some non monotone schemes for Hamilton-Jacobi-Bellman equations
Recommended article: Simplifying numerical analyses of Hamilton-Jacobi-Bellman equations
Recommended article: Multi-grid methods for Hamilton-Jacobi-Bellman equations
Recommended article: On the Convergence of Finite Element Methods for Hamilton--Jacobi--Bellman Equations
Recommended article: On Hamilton-Jacobi-Bellman equations with convex gradient constraints
Recommended article: On the convergence rate of approximation schemes for Hamilton-Jacobi-Bellman Equations
description: scientific article; zbMATH DE number 2162047