On the uniqueness of solutions to the Poisson equations for average cost Markov chains with unbounded cost functions
type: Item
instance of: scholarly article
title: On the uniqueness of solutions to the Poisson equations for average cost Markov chains with unbounded cost functions
zbMATH Open document ID: 1037.60068
author: Sandjai Bhulai
author: F. M. Spieksma
published in: Mathematical Methods of Operations Research
publication date: 2003-12-16T00:00:00Z
review text: Markov chains (MC) with state space \(X\) and transition matrix \(P\) are treated, to which a cost function \(c: X\to{\mathbb R}\) is associated. \(X\) is supposed as countable so that \(c\) may be unbounded. For all investigations as a basic tool, Banach spaces \(B_w(X)\) of \(w\)-bounded functions \(u: X\to{\mathbb R}\) with the norm \(\| u\| _w:=\sup_{x\in X}\frac{| u(x)| }{w(x)},\;w: X\to[1,\infty)\), will act, and \(c\in B_w(X)\) is postulated. Now it is asked under what conditions the Poisson equations \[ \varphi=P\varphi,\quad\varphi+v=c+Pv\tag{*} \] will have exactly one solution in \(B_w(X)\). At beginning of reading, the reader may think to have to do with Markov decision processes as the Poisson equations also play a role in Howard's policy improvement algorithm. But \(P\) and \(c\) are fixed. For the following, some ergodicity and recurrence properties are supposed. If the MC is geometrically \(w\)-ergodic (\(w\)-{GE}) and stable with finitely many closed classes, then \(\varphi\) is unique and \(v\) is unique up to a constant in every closed class. For this result, originally found by \textit{R. Dekker} [Ph.D. thesis, Leiden, 1985], an alternative probabilistic proof is given. The property of geometric \(w\)-recurrence (\(w\)-({GR}(\(M\))), i.e. \(\exists\varepsilon>0:| | _MP| | _w\leq1-\varepsilon,\;_MP\) being a taboo transition matrix for some \(M\subset X\), is easier to check than \(w\)-{GE} and sufficient for \(w\)-{GE}. Three queueing examples (multi-server queue, independent single server queue, priority queue with switching costs) are considered in greater detail, and appropriate taboo sets \(M\) are given so that (*) will have a unique solution.
Mathematics Subject Classification ID: 60J10
Mathematics Subject Classification ID: 90C40
Mathematics Subject Classification ID: 90B22
Mathematics Subject Classification ID: 60K25
Mathematics Subject Classification ID: 60J22
zbMATH DE Number: 2018326
zbMATH Keywords: Markov chain
zbMATH Keywords: weighted supremum norm
zbMATH Keywords: geometric ergodicity
zbMATH Keywords: Poisson equations
zbMATH Keywords: taboo transition matrix
zbMATH Keywords: multi-server queue
zbMATH Keywords: priority queue
MaRDI profile type: Publication
full work available at URL: s001860300292
OpenAlex ID: W2045362220
DOI: 10.1007/S001860300292
description: scientific article; zbMATH DE number 2018326