Linear programming and constrained average optimality for general continuous-time Markov decision processes in history-dependent policies (Q2884585)

The authors study the constrained average optimality for continuous-time Markov decision processes in the class of randomized history-dependent policies. The states and actions are in general Polish spaces, and the transition rates are allowed to be bounded. The optimality criterion is expected average costs, multiple constraints are imposed on similar expected average costs, and all costs may be unbounded both from above and from below. Basing on the improved concept of a stable policy and using the analogue of the forward Kolmogorov equation the authors show the existence of a constrained optimal policy. Then, they develop a linear program (LP), which is equivalent to the constrained optimality problem and is used to obtain a constrained optimal policy. Further, it is established the dual program (DP) to LP and showed that LP and DP are solvable. Finally, the authors use a cash flow model and a controlled birth and death system to illustrate the applications of the results of the paper. Ample set of cited references contain 39 items.