Convergence rates of Markov chain approximation methods for controlled diffusions with stopping
type: Item
instance of: scholarly article
title: Convergence rates of Markov chain approximation methods for controlled diffusions with stopping
zbMATH Open document ID: 1210.60081
author: Qingshuo Song
author: G. Yin
published in: Journal of Systems Science and Complexity
publication date: 2010-11-03T00:00:00Z
review text: The numerical methods using Markov chain approximations are considered for a one-dimensional controlled diffusion process \[ X^{x,u}(t)=x+\int_0^t b(X^{x,u}(s), u(s))ds +\int_0^t\sigma(X^{x,u}(s))dW(s) \] with initial data \(x\), given control \(u=u(t)\) and a standard real-valued Wiener process \(W(t)\). For \(B>0\) the first exit time from \((-B,B)\), i.e., \(\tau_B^{x,u}=\inf \{t:X^{x,u}(t) \notin (-B,B)\}\) is studied. The value function describes the optimal first exit time \(V(x,B)=\inf_{u}\tau_B^{x,u}\). The explicit representation of the dependence of \(V(x,B)\) on \(B\) is a novelty of the analysis of the problem. The considered numerical methods use a purely probabilistic approach. To prove the convergence of the algorithm by a Markov chain approximation method, a tangency problem might arise due to discontinuity of the first exit time with respect to discretized \(B\). In the considered setup, it is demonstrated that the tangency problem will not arise in the sense of convergence in probability and in \(L^1\), the analysis of the dependence of \(V(x,B)\) on \(B\) is used. In addition, controlled diffusions with a discount factor are considered.
Mathematics Subject Classification ID: 60J22
Mathematics Subject Classification ID: 65C40
Mathematics Subject Classification ID: 60G40
Mathematics Subject Classification ID: 60J60
Mathematics Subject Classification ID: 93E20
zbMATH DE Number: 5809974
zbMATH Keywords: controlled diffusion
zbMATH Keywords: dynamic programming equation
zbMATH Keywords: Markov chain approximation
zbMATH Keywords: rate of convergence
reviewed by: Alex V. Kolnogorov
MaRDI profile type: Publication
full work available at URL: s11424-010-0148-5
OpenAlex ID: W2086418723
cites work: Numerical Methods for Stochastic Control Problems in Continuous Time
cites work: Q2703816
cites work: On the convergence rate of approximation schemes for Hamilton-Jacobi-Bellman Equations
cites work: The rate of convergence of finite-difference approximations for parabolic bellman equations with Lipschitz coefficients in cylindrical domains
cites work: ON THE RATE OF CONVERGENCE OF APPROXIMATION SCHEMES FOR BELLMAN EQUATIONS ASSOCIATED WITH OPTIMAL STOPPING TIME PROBLEMS
cites work: On the rate of convergence of finite-difference approximations for Bellman's equations with variable coefficients
cites work: Some Estimates for Finite Difference Approximations
cites work: Rate of convergence of finite difference approximations for degenerate ordinary differential equations
cites work: Numerical methods for controlled regime-switching diffusions and regime-switching jump diffusions
cites work: Rates of Convergence of Numerical Methods for Controlled Regime-Switching Diffusions with Stopping Times in the Costs
cites work: Consistency issues for numerical methods for variance control, with applications to optimization in finance
cites work: Q4086303
cites work: Q3911791
cites work: Q5600531
cites work: Q4002114
cites work: SOME NEW RESULTS IN THE THEORY OF CONTROLLED DIFFUSION PROCESSES
DOI: 10.1007/S11424-010-0148-5
description: scientific article; zbMATH DE number 5809974