Numerical comparison of controls and verification of optimality for stochastic control problems
type: Item
instance of: scholarly article
title: Numerical comparison of controls and verification of optimality for stochastic control problems
zbMATH Open document ID: 0965.93099
DOI: 10.1023/A:1004659107996
published in: Journal of Optimization Theory and Applications
publication date: 2001-05-09T00:00:00Z
review text: The purpose of this paper is to provide a numerical method for comparing the efficiency of various controls and to present a verification approach to the solution to stochastic control problems. A Linear Programming (LP) formulation involving the moments of the stationary distribution of the process is used. Two approaches were adopted. The first was to specify the control for the process and run the LP to obtain various characteristics of the controlled process including the cost associated with the process. Doing this with several controls allows a comparison of the costs. The second approach was to specify a candidate for the optimal control and derive a modified LP which all controlled processes satisfied and which the candidate control satisfied with tightness for particular constraints. This differs from dynamic programming in which one guesses the value function, verifies that it is a solution to the Hamilton-Jacobi-Bellman equation, and as a consequence determines the optimal control. For numerical computation, the problem is reduced to finite dimensions by considering only a finite number of moments. The proposed approach deals directly with the stochastic process of the control problem rather than with some approximating process.
reviewed by: Vyacheslav A. Vasiliev
Mathematics Subject Classification ID: 93E20
Mathematics Subject Classification ID: 93C05
Mathematics Subject Classification ID: 93E25
zbMATH DE Number: 1533382
zbMATH Keywords: stochastic control
zbMATH Keywords: linear programming
zbMATH Keywords: numerical comparison
zbMATH Keywords: optimal control
zbMATH Keywords: moments
zbMATH Keywords: verification approach
zbMATH Keywords: numerical computation
author: K. Helmes
author: Richard H. Stockbridge
MaRDI profile type: Publication
cites work: Linear programming and sequential decisions
cites work: On sequential decisions and Markov chains
cites work: On Linear Programming in a Markov Decision Problem
cites work: Linear Programming in a Markov Chain
cites work: A Linear Programming Problem Connected with Optimal Stationary Control in a Dynamic Decision Problem
cites work: Average cost Markov decision processes: Optimality conditions
cites work: Time-average control of martingale problems: Existence of a stationary solution
cites work: Time-average control of martingale problems: A linear programming formulation
cites work: Existence of Markov Controls and Characterization of Optimal Markov Controls
cites work: Occupation measures for controlled Markov processes: Characterization and optimality
cites work: Approximation of Infinite-Dimensional Linear Programming Problems which Arise in Stochastic Control
cites work: Q3721531
cites work: Some solvable stochastic control problemst†
cites work: Q4836494
cites work: Q5798359
cites work: Q4086303
cites work: Q4032143
full work available at URL: a:1004659107996
OpenAlex ID: W1507296508
description: scientific article; zbMATH DE number 1533382