Finite state Markovian decision processes
From MaRDI portal
Publication:2561156
zbMath0262.90001MaRDI QIDQ2561156
Publication date: 1970
Published in: Mathematics in Science and Engineering (Search for Journal in Brave)
Decision theory (91B06) Research exposition (monographs, survey articles) pertaining to statistics (62-02) Markov and semi-Markov decision processes (90C40) Research exposition (monographs, survey articles) pertaining to operations research and mathematical programming (90-02)
Related Items
Markov decision programming with constraints ⋮ Asymptotic expansions for dynamic programming recursions with general nonnegative matrices ⋮ Deciding probabilistic automata weak bisimulation: theory and practice ⋮ Discounted Markov decision processes with fuzzy costs ⋮ Matching, search, and bargaining ⋮ Utility, probabilistic constraints, mean and variance of discounted rewards in Markov decision processes ⋮ A variance minimization problem for a Markov decision process ⋮ Quadratic programming and the single-controller stochastic game ⋮ Performance analysis of probabilistic timed automata using digital clocks ⋮ Continue, quit, restart probability model ⋮ Perspectives of approximate dynamic programming ⋮ Impact of supply risks on procurement decisions ⋮ Optimal control in light traffic Markov decision processes ⋮ On efficiency of linear programming applied to discounted Markovian decision problems ⋮ Trading performance for stability in Markov decision processes ⋮ Communicating MDPs: Equivalence and LP properties ⋮ Strategy improvement for concurrent reachability and turn-based stochastic safety games ⋮ An efficient heuristic for a partially observable Markov decision process of machine replacement ⋮ Finite-horizon variance penalised Markov decision processes ⋮ A semimartingale characterization of average optimal stationary policies for Markov decision processes ⋮ Constrained Markov decision processes with first passage criteria ⋮ On confidence intervals from simulation of finite Markov chains ⋮ On the life and work of Cyrus Derman ⋮ Derman's book as inspiration: some results on LP for MDPs ⋮ Q-learning and policy iteration algorithms for stochastic shortest path problems ⋮ Asymptotically optimal Bayesian sequential change detection and identification rules ⋮ Discounting axioms imply risk neutrality ⋮ Variance minimization for constrained discounted continuous-time MDPs with exponentially distributed stopping times ⋮ Training and repair policies for stand-by systems ⋮ Inventory replenishment control under supply uncertainty ⋮ Exact decomposition approaches for Markov decision processes: a survey ⋮ A uniform framework for modeling nondeterministic, probabilistic, stochastic, or mixed processes and their behavioral equivalences ⋮ First passage problems for nonstationary discrete-time stochastic control systems ⋮ A game-theoretic analysis of bargaining with reputations ⋮ The blast furnaces problem ⋮ Mathematical modeling of distributed catastrophic and terrorist risks ⋮ On strategy improvement algorithms for simple stochastic games ⋮ Separable value functions for infinite horizon average reward Markov decision processes ⋮ Optimal control of Markov chains admitting strong and weak interactions ⋮ A decomposition algorithm for limiting average Markov decision problems. ⋮ Explicit solution of the average-cost optimality equation for a pest-control problem ⋮ Optimal maintenance of systems with Markovian mission and deterioration ⋮ A model of far-sighted electoral competition ⋮ Dynamic programming and the secretary problem ⋮ Optimal threshold probability in undiscounted Markov decision processes with a target set. ⋮ Adaptive control of constrained Markov chains: Criteria and policies ⋮ Nonlinear programming and stationary equilibria in stochastic games ⋮ Sensitivity of constrained Markov decision processes ⋮ Polynomial time decision algorithms for probabilistic automata ⋮ Learning about variable demand in the long run ⋮ Separable Markovian decision problems. The linear programming method in the multichain case ⋮ An exponential lower bound for Cunningham's rule ⋮ Saddle-point calculation for constrained finite Markov chains ⋮ Finite approximation of the first passage models for discrete-time Markov decision processes with varying discount factors ⋮ Estimating the value of a discounted reward process ⋮ Structural properties of optimal tool replacement policy in a machining center ⋮ On polynomial cases of the unichain classification problem for Markov decision processes ⋮ An optimal maintenance policy of a discrete time Markovian deterioration system ⋮ Policy iteration and Newton-Raphson methods for Markov decision processes under average cost criterion ⋮ Some remarks on the new optimality criterion of Mine and Tabata ⋮ On the convergence of the average expected return in dynamic programming ⋮ Maximizing the probability of attaining a target prior to extinction ⋮ Plastic-elastic torsion, optimal stopping and free boundaries ⋮ Stochastic control of paging in a two-level computer memory ⋮ Optimal software testing in the setting of controlled Markov chains ⋮ Optimal isolation policies for deterministic and stochastic epidemics ⋮ On stopped decision processes with discrete time parameter ⋮ Optimal threshold probability and expectation in semi-Markov decision processes ⋮ Optimal maintenance of two stochastically deteriorating machines with an intermediate buffer ⋮ Stochastic evolution and control of an economic activity ⋮ Stochastische dynamische Optimierung als Spezialfall linearer Optimierung in halbgeordneten Vektorräumen ⋮ Optimal immunization rules for an epidemic with recovery ⋮ Optimal systems for equipment maintenance and replacement under Markovian deterioration ⋮ Markov decision models for the optimal maintenance of a production unit with an upstream buffer ⋮ Contraction mappings underlying undiscounted Markov decision problems ⋮ Probabilistic mobile ambients ⋮ Fast convergence to state-action frequency polytopes for MDPs ⋮ A note on the hypercube model ⋮ Optimal maintenance of a production-inventory system with idle periods ⋮ Constrained denumerable state non-stationary MDPs with expected total reward criterion ⋮ Optimal strategies for an inventory system with cost functions of general form ⋮ On the optimal control of M/G/1 systems under the cycle criterion ⋮ On asymptotic optimization of a class of nonlinear stochastic hybrid systems ⋮ Percentiles and Markovian decision processes ⋮ Optimal control of a dam under seasonal electricity prices ⋮ On maximizing the average time at a goal ⋮ On the block upper-triangularity of undiscounted multi-chain Markov decision problems ⋮ Paging against a distribution and IP networking ⋮ Stochastic control theory and operational research ⋮ Sensitivity analysis in discounted Markovian decision problems ⋮ Mean-variance criteria in an undiscounted Markov decision process ⋮ Derivation of optimal stocking policies for grazing in arid regions. I. Methodology ⋮ Optimal maintenance strategies for systems with partial repair options and without assuming bounded costs ⋮ Finite state Markov decision models with average reward criteria ⋮ Generalized polynomial approximations in Markovian decision processes ⋮ Optimal policies for controlled Markov chains with a constraint ⋮ Some comments on a theorem of Hardy and Littlewood ⋮ Revised simplex algorithm for finite Markov decision processes ⋮ Optimal stopping problems for multiarmed bandit processes with arms' independence ⋮ Combinatorial structure and randomized subexponential algorithms for infinite games ⋮ The power of two choices for random walks ⋮ Model Checking of Biological Systems ⋮ Regular Policies in Abstract Dynamic Programming ⋮ Algebraic optimization of sequential decision problems ⋮ Geometry and convergence of natural policy gradient methods ⋮ Machine maintenance with workload considerations ⋮ Controlled Markov Fields with Finite State Space on Graphs ⋮ Markov decision processes ⋮ Adaptive policy for two finite Markov chains zero-sum stochastic game with unknown transition matrices and average payoffs ⋮ An analysis of transient Markov decision processes ⋮ Process algebra for performance evaluation ⋮ On the chance to visit a goal set infinitely often ⋮ The linear time-branching time spectrum of equivalences for stochastic systems with non-determinism ⋮ A Convex Analytic Approach to Risk-Aware Markov Decision Processes ⋮ Markov Decision Processes with Asymptotic Average Failure Rate Constraint ⋮ Control of a hybrid stochastic system ⋮ Temporal logics for the specification of performance and reliability ⋮ Symbolic model checking for probabilistic timed automata ⋮ Performance Model Checking Scenario-Aware Dataflow ⋮ Managing stochastic inventory systems with free shipping option ⋮ A control policy of an inventory system with compound poisson demand ⋮ Algorithms for stochastic games ? A survey ⋮ Certified Impossibility Results and Analyses in Coq of Some Randomised Distributed Algorithms ⋮ Block-scaling of value-iteration for discounted Markov renewal programming ⋮ Replacement process decomposition for discounted Markov renewal programming ⋮ On some algorithms for limiting average Markov decision processes ⋮ Optimal Preventive Maintenance of a Production-Inventory System When the Action of “Idling” Is Permissible ⋮ On stochastic optimality of policies in first passage problems ⋮ Biased random walks ⋮ Controlled Markov processes on the infinite planning horizon: Weighted and overtaking cost criteria ⋮ Markov: A methodology for the solution of infinite time horizon markov decision processes ⋮ Constrained Semi-Markov decision processes with average rewards ⋮ Multi-objective discounted Markov decision processes with expectation and variance criteria ⋮ Generalized Markovian decision processes ⋮ Survey of linear programming for standard and nonstandard Markovian control problems. Part I: Theory ⋮ Linear programming formulation of MDPs in countable state space: The multichain case ⋮ Error bounds for stochastic shortest path problems ⋮ Unnamed Item ⋮ \textsc{ULTraS} at work: compositionality metaresults for bisimulation and trace semantics ⋮ First Passage Optimality and Variance Minimisation of Markov Decision Processes with Varying Discount Factors ⋮ Optimal repair allocation in a series system expected discounted operation time criterion ⋮ Risk-Sensitive Reinforcement Learning via Policy Gradient Search ⋮ A Simple P-Matrix Linear Complementarity Problem for Discounted Games ⋮ Stochastic shortest path problems with associative accumulative criteria ⋮ The Linear Program approach in multi-chain Markov Decision Processes revisited ⋮ Partially observable Markov decision model for the treatment of early prostate cancer ⋮ First passage models for denumerable semi-Markov decision processes with nonnegative discounted costs ⋮ Deterministic discrete dynamic programming with discount factor greater than one: Some further results and algorithms ⋮ On nonzero-sum game considered on solutions of a hybrid system with frequent random jumps ⋮ A value iteration method for undiscounted multichain Markov decision processes ⋮ On optimal replacement policy ⋮ STRONG AVERAGE OPTIMALITY FOR CONTROLLED NONHOMOGENEOUS MARKOV CHAINS* ⋮ Towards solving 2-TBSG efficiently ⋮ The LP approach in average reward MDPs with multiple cost constraints: The countable state case ⋮ Verification of the randomized consensus algorithm of Aspnes and Herlihy: a case study ⋮ Scheduling service in tandem queues attended by a single server ⋮ Fuzzy optimality relation for perceptive MDPs-the average case ⋮ Constructive logical characterizations of bisimilarity for reactive probabilistic systems ⋮ Towards general axiomatizations for bisimilarity and trace semantics ⋮ Concurrent reachability games ⋮ A Subexponential Lower Bound for Zadeh’s Pivoting Rule for Solving Linear Programs and Games ⋮ On Markov games ⋮ Task-structured probabilistic I/O automata ⋮ Code aware resource management ⋮ Customizing exponential semi-Markov decision processes under the discounted cost criterion ⋮ A decision exclusion algorithm for a class of Markovian Decision Processes ⋮ A Generalisation of Stationary Distributions, and Probabilistic Program Algebra ⋮ Relating strong behavioral equivalences for processes with nondeterminism and probabilities ⋮ Dynamic multi-appointment patient scheduling for radiation therapy ⋮ Solution procedures for multi-objective markov decision processes ⋮ Illustrated review of convergence conditions of the value iteration algorithm and the rolling horizon procedure for average-cost MDPs ⋮ Revisiting bisimilarity and its modal logic for nondeterministic and probabilistic processes ⋮ A survey of maintenance models: The control and surveillance of deteriorating systems ⋮ Transient policies in discrete dynamic programming: Linear programming including suboptimality tests and additional constraints ⋮ Optimization of moves and measurements in networks with stochastic costs ⋮ MARKOVIAN DETERIORATION WITH UNCERTAIN INFORMATION — A MORE GENERAL MODEL ⋮ Remarks on the hypercube model ⋮ On the Fixed Points of the Optimal Reward Operator in Stochastic Dynamic Programming with Discount Factor Greater than One ⋮ First passage game ⋮ Solving stochastic dynamic programming problems by linear programming — An annotated bibliography ⋮ Optimal dynamic rules for assigning customers to servers in a heterogeneous queuing system ⋮ Some approaches to solving inventory control problems ⋮ On Solving Finite State Multi-Armed Bandit Problem by Linear Programming ⋮ Solution of continuous-time markovian decision models using infinite linear programming ⋮ Computing Optimal Policies for Markovian Decision Processes Using Simulation ⋮ A Markovian Decision Process with hidden states and hidden costs ⋮ Optimal state-dependent pricing policies for a class of stochastic multiunit service systems ⋮ Suboptimal inspection policies for imperfectly observed realistic systems ⋮ Solving a general discounted dynamic program by linear programming ⋮ Optimal maintenance models for systems subject to failure–A Review ⋮ Average Reward Markov Decision Processes with Multiple Cost Constraints ⋮ Adaptive optimization and the harvest of biological populations ⋮ Allocation of distinguishable servers ⋮ A parametric characterization and an \(\epsilon\)-approximation scheme for the minimization of a quasiconcave program ⋮ Mean, variance and probabilistic criteria in finite Markov decision processes: A review ⋮ Group-by-Group Probabilistic Bisimilarities and Their Logical Characterizations ⋮ Average optimality for Markov decision processes in borel spaces: a new condition and approach ⋮ On the solvability of Bellman's functional equation for a Markovian decision process ⋮ A finite algorithm for the switching control stochastic game ⋮ Dynamic Repair Allocation for a k−Out−of−n System Maintained by Distinguishable Repairmen ⋮ Finite state multi-armed bandit problems: Sensitive-discount, average-reward and average-overtaking optimality ⋮ On the Optimality of Trunk Reservation in Overflow Processes ⋮ Optimal preventive maintenance of a production system with an intermediate buffer ⋮ Unnamed Item ⋮ Robust shortest path planning and semicontractive dynamic programming ⋮ Repeated bargaining with opportunities for learning ⋮ Optimal policy for minimizing risk models in Markov decision processes ⋮ Preventive replacement for multi-parts systems ⋮ Maintenance of a device with age-dependent exponential failures ⋮ Remarks on Testing Probabilistic Processes ⋮ Quantitative program logic and expected time bounds in probabilistic distributed algorithms. ⋮ Unnamed Item ⋮ First passage Markov decision processes with constraints and varying discount factors ⋮ Joint optimization of \(\overline{X}\) control chart and preventive maintenance policies: a discrete-time Markov chain approach
This page was built for publication: Finite state Markovian decision processes
