CTMDP and its relationship with DTMDP
type: Item
instance of: scholarly article
title: CTMDP and its relationship with DTMDP
zbMATH Open document ID: 0701.90093
author: Qiying Hu
published in: Chinese Science Bulletin
publication date: 1990-01-01T00:00:00Z
review text: The discounted continuous time Markov decision problem (shortly, CTMDP) discussed here is of countable state space, non-empty action sets and non-uniformly bounded transition rates. Under weak conditions we proved that the discounted CTMDP is equivalent to a discounted discrete time Markov decision problem with discount factor having the form \(\beta\) (i) in the following meanings: i) their optimality equations are equal to each other: ii) their discounted objectives are equal to each other in the stochastic stationary policy set \(\Pi_ s\). Here, sup \(\beta\) (i) may be equal to 1. But when the transition rate q(j\(| i,a)\) of CTMDP is uniformly bounded, \(\beta\) (i) can be taken as a constant. By this equivalence most results known for discounted DTMDP can be easily generalized to discounted CTMDP. In addition a previous result in proving the optimality equation of discounted CTMDP is corrected.
Mathematics Subject Classification ID: 90C40
zbMATH DE Number: 4149919
zbMATH Keywords: discounted continuous time Markov decision
zbMATH Keywords: countable state space
zbMATH Keywords: discount factor
zbMATH Keywords: optimality equation
MaRDI profile type: Publication
description: scientific article; zbMATH DE number 4149919