Penalty methods for the solution of discrete HJB equations -- continuous control and obstacle problems (Q2903003)

The authors present penalty methods for the solution of discrete Hamilton-Jacobi-Bellman (HJB) equations. The paper draws the problem from optimal stochastic control that is formulated and studied in the domain of mathematical analysis/numerical analysis. In particular, the authors start with the formulation of two kinds of nonlinear equations which are solved numerically. The results include estimates of the penalization error for a class of penalty terms. In the proofs they apply variations of Newton's method, in order to obtain globally convergent iterative solvers for the penalized equations. It's interesting that an analysis is provided subject to certain conditions which can guarantee quadratic convergence of the iterative solvers. Numerical evidence is provided with a concrete example on an incomplete market investment problem. An analysis on discrete systems is also included in the present study. The paper concludes with two appendices on policy iteration for the HJB equation and policy iteration for the HJB obstacle problem.