Adaptive control of a partially observed discrete time Markov process (Q1384646)

The authors consider the Markov process \((X_n,n\in N)\) on the measurable space \((E,{\mathcal E})\), where \(E\) is either a closed subset of \(R^d\) or \(E\) is a countable set. The transition operator depends on an unknown, fixed parameter \(a^0\in {\mathcal A} \subset R^{K_0}\). The process \((X_n,\;n\in N)\) is completely observed in a fixed recurrent domain \(\Gamma\), and is partially observed in \(\Gamma^C\). An admissible control is adapted to the observations, and the cost is desired to minimize the average cost per unit time. The solution of an adaptive control problem is given by constructing an approximately self-optimal strategy. A different form of partial observations is also considered. Namely, in this case there is no subset of complete observations but there is a sequence of stopping times \((\tau_n,\;n\in N)\) such that the random variables \((X_{\tau_n},\;n\in N)\) are independent and identically distributed.