A convex analytic approach to Markov decision processes
type: Item
instance of: scholarly article
title: A convex analytic approach to Markov decision processes
zbMATH Open document ID: 0628.90090
DOI: 10.1007/BF00353877
author: Vivek Borkar
publication date: 1988-01-01T00:00:00Z
review text: This paper develops a new framework for the study of Markov decision processes in which the control problem is viewed as an optimization problem on the set of canonically induced measures on the trajectory space of the joint state and control process. This set is shown to be compact convex. One then associates with each of the usual cost criteria (infinite horizon discounted cost, finite horizon, control up to an exit time) a naturally defined occupation measure such that the cost is an integral of some function with respect to this measure. These measures are shown to form a compact convex set whose extreme points are characterized. Classical results about existence of optimal strategies are recovered from this and several applications to multicriteria and constrained optimization problems are briefly indicated.
Mathematics Subject Classification ID: 90C40
Mathematics Subject Classification ID: 90C48
zbMATH DE Number: 4023056
zbMATH Keywords: canonically induced measures
zbMATH Keywords: optimization in measure space
zbMATH Keywords: occupation measure
zbMATH Keywords: existence of optimal strategies
zbMATH Keywords: multicriteria
zbMATH Keywords: constrained optimization
MaRDI profile type: Publication
cites work: Q5560061
cites work: Dynamic programming and stochastic control
cites work: On Minimum Cost Per Unit Time Control of Markov Chains
cites work: Q5508589
full work available at URL: https://doi.org/10.1007/bf00353877
OpenAlex ID: W2004640191
published in: Zeitschrift für Wahrscheinlichkeitstheorie und Verwandte Gebiete
description: scientific article; zbMATH DE number 4023056