Optimal consumption and exploration: A case study in piecewise-deterministic Markov modelling
type: Item
instance of: scholarly article
title: Optimal consumption and exploration: A case study in piecewise-deterministic Markov modelling
zbMATH Open document ID: 0959.91046
DOI: 10.1023/A:1018918027153
author: Mohammad Farid
author: Mark H. A. Davis
published in: Annals of Operations Research
publication date: 1999-12-02T00:00:00Z
review text: The optimal decision problem for the consumption and exploration of resources are studied in the framework of Piecewise-Deterministic Markov Process (PDP) optimal control theory [cf. \textit{M. H. A. Davis}, Markov models and optimization, Monographs on Statistics and Applied Probability 49, Chapman-Hall (1993; Zbl 0780.60002), in particular, Chap. 4]. It is assumed that the inter-arrival random times \((\{T_i\})\) of the new resources are exponentially distributed, that the amount of resource found at each \(T_i\) is of a known distribution and that the exploration costs are paid for from the value of current reserves. Under some general but mild assumptions, this problem is transformed into a deterministic control problem, the value function of the PDP optimal control is solved as the unique viscosity solution of the associated Hamilton-Jacobi-Bellman (HJB) equation. Simulation results by an iterative method for solving the value function using the HJB equation are well presented under guaranteed convergence.
reviewed by: Chengxun Wu
Mathematics Subject Classification ID: 91B60
Mathematics Subject Classification ID: 93E20
zbMATH DE Number: 1373572
zbMATH Keywords: stochastic optimal control
zbMATH Keywords: non-renewable resources
zbMATH Keywords: Hamilton-Jacobi-Bellman equation
zbMATH Keywords: piecewise-deterministic Markov process
zbMATH Keywords: optimal decision
zbMATH Keywords: viscosity solution
MaRDI profile type: Publication
full work available at URL: a:1018918027153
OpenAlex ID: W4776762
description: scientific article; zbMATH DE number 1373572