Policy structure for discrete time Markov chain disorder problems (Q1077336)

A Markov chain disorder problem (MCDP) is a partially observed Markov decision problem where a policy of response must be made to an unobservable transition to an undesirable state. Models for discrete time MCDP and standard functional equations for the minimal expected discounted cost are given. The functional equations yield stationary Markov policies which achieve the desired minima. Sufficient conditions are given for the policies to have a ''likelihood consistency property'', and such policies can be used when an ''optimal'' policy is unattainable or undesirable.