The Optimal Control of Partially Observable Markov Processes over the Infinite Horizon: Discounted Costs

DOI10.1287/opre.26.2.282zbMath0379.60067OpenAlexW2044375425WikidataQ29030996 ScholiaQ29030996MaRDI QIDQ4158268

Edward J. Sondik

Publication date: 1978

Published in: Operations Research (Search for Journal in Brave)

Full work available at URL: https://doi.org/10.1287/opre.26.2.282

Mathematics Subject Classification ID

Applications of Markov chains and discrete-time Markov processes on general state spaces (social mobility, learning theory, industrial processes, etc.) (60J20) Mathematical programming (90Cxx)

Related Items

Ambiguous partially observable Markov decision processes: structural results and applications ⋮ PARTIALLY OBSERVABLE MARKOV DECISION PROCESSES AND PERIODIC POLICIES WITH APPLICATIONS ⋮ An optimal inspection and replacement policy under incomplete state information ⋮ A model of project evaluation with limited attention ⋮ On arrival driven queueing models: Admission control, traffic policing, abandonments, and correlated arrivals ⋮ A nonhomogeneous hidden Markov model of response dynamics and mailing optimization in direct marketing ⋮ On the construction of \(\epsilon\)-optimal strategies in partially observed MDPs ⋮ A survey of algorithmic methods for partially observed Markov decision processes ⋮ On the average cost optimality equation and the structure of optimal policies for partially observable Markov decision processes ⋮ On the computation of the optimal cost function for discrete time Markov models with partial observations ⋮ Cross-entropic learning of a machine for the decision in a partially observable universe ⋮ On-line parameter estimation for a failure-prone system subject to condition monitoring ⋮ Availability maximization under partial observations ⋮ Admission Control Policies in a Finite Capacity Geo/Geo/1 Queue Under Partial State Observations ⋮ LQG dynamic games with a control-sharing information pattern ⋮ An efficient heuristic for a partially observable Markov decision process of machine replacement ⋮ Cops and invisible robbers: the cost of drunkenness ⋮ The skyline algorithm for POMDP value function pruning ⋮ Markov Decision Processes with Incomplete Information and Semiuniform Feller Transition Probabilities ⋮ A nonlinear programming model for partially observable Markov decision processes: Finite horizon case ⋮ BOUNDED-PARAMETER PARTIALLY OBSERVABLE MARKOV DECISION PROCESSES: FRAMEWORK AND ALGORITHM ⋮ Rationality and intelligence ⋮ Planning in partially-observable switching-mode continuous domains ⋮ A systematic approach to determining mean-variance tradeoffs when managing randomly varying populations ⋮ Formalization of methods for the development of autonomous artificial intelligence systems ⋮ Approximability and efficient algorithms for constrained fixed-horizon POMDPs with durative actions ⋮ Partially observable Markov decision process-based optimal maintenance planning with time-dependent observations ⋮ Solving zero-sum one-sided partially observable stochastic games ⋮ Inventory control with modulated demand and a partially observed modulation process ⋮ Future memories are not needed for large classes of POMDPs ⋮ Monotone control laws for noisy, countable-state Markov chains ⋮ INDEXABILITY AND OPTIMAL INDEX POLICIES FOR A CLASS OF REINITIALISING RESTLESS BANDITS ⋮ Off-policy evaluation in partially observed Markov decision processes under sequential ignorability ⋮ Partially observable Markov decision model for the treatment of early prostate cancer ⋮ Computation of approximate optimal policies in a partially observed inventory model with rain checks ⋮ Exploiting symmetries for single- and multi-agent partially observable stochastic domains ⋮ A unified framework for stochastic optimization ⋮ Strong planning under partial observability ⋮ Convergence of probability measures and Markov decision models with incomplete information ⋮ Managing mobile production-inventory systems influenced by a modulation process ⋮ Markov-Entscheidungs-Prozesse mit abhängigen Aktionen für optimale Reparaturmaßnahmen bei unvollständiger Information. (Markov decision processes with dependent actions for optimal repair policies under incomplete information) ⋮ Optimal control of infinite horizon partially observable decision processes modelled as generators of probabilistic regular languages ⋮ Reinforcement learning with limited reinforcement: using Bayes risk for active learning in POMDPs ⋮ Planning and acting in partially observable stochastic domains ⋮ Conformant plans and beyond: principles and complexity ⋮ Filters and parameter estimation for a partially observable system subject to random failure with continuous-range observations ⋮ Heuristic anytime approaches to stochastic decision processes ⋮ Probabilistic Acceptors for Languages over Infinite Words ⋮ Finite-state, discrete-time optimization with randomly varying observation quality ⋮ Minimum principles in motor control. ⋮ Optimal condition based maintenance with imperfect information and the proportional hazards model ⋮ Undiscounted Markov decision chains with partial information; an algorithm for computing a locally optimal periodic policy ⋮ Partially Observable Total-Cost Markov Decision Processes with Weakly Continuous Transition Probabilities ⋮ Locally-Connected Interrelated Network: A Forward Propagation Primitive ⋮ An Uncertainty-Based Belief Selection Method for POMDP Value Iteration ⋮ On replacement policies for additive systems with several working levels ⋮ Partially observable Markov decision processes with imprecise parameters ⋮ A Fenchel-Moreau-Rockafellar type theorem on the Kantorovich-Wasserstein space with applications in partially observable Markov decision processes ⋮ Monitoring machine operations using on-line sensors ⋮ A dynamic epistemic framework for reasoning about conformant probabilistic plans ⋮ A tutorial on partially observable Markov decision processes ⋮ Stochastic dynamic programming with factored representations ⋮ A New Computational Approach to Cost Variance Iuvestigation Problems ⋮ Constrained Multiagent Markov Decision Processes: a Taxonomy of Problems and Algorithms ⋮ Transformation of partially observable Markov decision processes into piecewise linear ones ⋮ A survey of solution techniques for the partially observed Markov decision process ⋮ Optimal cost and policy for a Markovian replacement problem ⋮ A leader-follower partially observed, multiobjective Markov game ⋮ Value of information for a leader-follower partially observed Markov game ⋮ On the undecidability of probabilistic planning and related stochastic optimization problems ⋮ Optimal sensor scheduling for hidden Markov model state estimation