A remark on control of partially observed Markov chains (Q757323)

Let be \(X_ n\), \(n\geq 0\) a Markov chain with values in the state space \(S=\{1,2,3,...\}\), \(Z_ n\), \(n\geq 0\) a control process with values in a compact metric control space D and \(Y_ n\), \(n\geq 0\) an observation process with values in a finite or countable observation space \(H=\{1,2,...\}\). Suppose further p: \(S\times D\times S\times H\to [0,1]\) is a map satisfying \(\sum_{j,k}p(i,u,j,k)=1\) for all \(i\in S\), \(u\in D\), and the evolution is described by \(P(X_{n+1}=j\), \(Y_{n+1}=k| \sigma (X_ m,Z_ m,Y_ m,m\leq n)=p(X_ n,Z_ n,j,k)\). The problem is to choose \(Z_ n\) to minimize some prescribed cost functional under the constraint \(Z_ n\) and \(\{X_ m,m\leq n,Z_ m,m<n\}\) are conditionally independent given \(Y_ m\), \(m\leq n\) for \(n\geq 0\). The author introduces a new variable of control taking values in the finite measures on the state space. He considers the evolution under a new probability measure under which the observation process has a much simpler statistic.