Moments' analysis in homogeneous Markov reward models (Q1041298)

Let \(X=(X_t:\, t\geq 0)\) be a continuous-time Markov process with finite state space \(S\), and let \(f:S\to \mathbb{R}\) be a reward function. The (normalized) accumulated reward over time interval \((0,t)\) is defined by \[ Y_t=\frac{1}{t\|f\|_{\infty}} \int_0^t f(X_s)\, ds\, . \] This paper deals with numerical computation of the moments of \(Y_t\). The proposed algorithm depends on the uniformization method which reduces the computation of moments to computation of certain coefficients \(U_i(n,r)\). By analyzing convergence of those coefficients as \(n\to \infty\) (\textit{stationarity detection}), the authors derive an improved procedure for the numerical computation of first moments. The paper concludes with an example dealing with fault-tolerant multiprocessor system.