Infinite horizon Markov decision processes with unknown or variable discount factors (Q1821705)

The standard approach to discounted Markov decision processes is to assume a fixed discount factor, and, perhaps, apply a sensitivity analysis if the estimate is subject to error. In this paper the discount factor is assumed to be endowed with certain stochastic properties, and the Markov decision problem is then reformulated. Bounds for loss of optimality are calculated if an estimate of the discount factor is used in a standard model.