Conditions for optimality in dynamic programming and for the limit of n-stage optimal policies to be optimal

DOI10.1007/BF00532612zbMath0316.90080OpenAlexW323116121MaRDI QIDQ4077764

Manfred Schäl

Publication date: 1975

Published in: Zeitschrift für Wahrscheinlichkeitstheorie und verwandte Gebiete (Search for Journal in Brave)

Full work available at URL: https://doi.org/10.1007/bf00532612

Mathematics Subject Classification ID

Dynamic programming in optimal control and differential games (49L20) Optimal stochastic control (93E20) Markov and semi-Markov decision processes (90C40) Decision theory for games (91A35)

Related Items (only showing first 100 items - show all)

Dynamic moral hazard without commitment ⋮ Semicontinuous nonstationary stochastic games ⋮ A polynomial time bound for Howard's policy improvement algorithm ⋮ Sufficient conditions for optimality of a \((z,c^ -,c^ +)\)-sampling plan in multistage Bayesian acceptance sampling ⋮ Conditions for the solvability of the linear programming formulation for constrained discounted Markov decision processes ⋮ Optimal assignment policy of a single server attended by two queues ⋮ Recurrence conditions for Markov decision processes with Borel state space: A survey ⋮ Density estimation and adaptive control of Markov processes: Average and discounted criteria ⋮ Scheduling in a multi-class series of queues with deterministic service times ⋮ Optimale Innovationspolitik bei unvollständiger Information. (Optimal innovation policy under incomplete information) ⋮ Normal fields of functionals and optimal measurable sections ⋮ Optimal learning with costly adjustment ⋮ Recursive utility and optimal growth under uncertainty ⋮ Value iteration in average cost Markov control processes on Borel spaces ⋮ A strategic market game with secured lending ⋮ Convex analytic approach to constrained discounted Markov decision processes with non-constant discount factors ⋮ Constrained Markov control processes with randomized discounted cost criteria: infinite linear programming approach ⋮ Geometry of information structures, strategic measures and associated stochastic control topologies ⋮ On compactness of the space of policies in stochastic dynamic programming ⋮ Markov decision processes with iterated coherent risk measures ⋮ Optimality of monotonic policies for two-action Markovian decision processes, with applications to control of queues with delayed information ⋮ Sample path methods in the control of queues ⋮ The policy iteration algorithm for average continuous control of piecewise deterministic Markov processes ⋮ Two-person zero-sum stochastic games with varying discount factors ⋮ Markov control models with unknown random state-action-dependent discount factors ⋮ Robust Markov control processes ⋮ Convex Analysis in Decentralized Stochastic Control, Strategic Measures, and Optimal Solutions ⋮ Dynamic programming for discrete-time stochastic systems of a general type ⋮ Markov decision processes with state-dependent discount factors and unbounded rewards/costs ⋮ On theory and algorithms for Markov decision problems with the total reward criterion ⋮ Markov renewal decision processes with finite horizon ⋮ Conditions for characterizing the structure of optimal strategies in infinite-horizon dynamic programs ⋮ On the expected total reward with unbounded returns for Markov decision processes ⋮ On discounted dynamic programming with unbounded returns ⋮ Discounted dynamic programming with unbounded returns: application to economic models ⋮ Sequential stochastic control (single or multi-agent) problems nearly admit change of measures with independent measurement ⋮ \(n\)-person dynamic strategic market games ⋮ A natural extension of the MacQueen extrapolation ⋮ Bias and overtaking equilibria for zero-sum stochastic differential games ⋮ A note on the \({\sigma}\)-compactness of sets of probability measures on metric spaces ⋮ Optimal policies in multiproduct inventory models ⋮ On two-state quality control under Markovian deterioration ⋮ Optimal dividend policy in discrete time ⋮ How to stay in a set or Koenig's lemma for random paths ⋮ Adaptive control of constrained Markov chains: Criteria and policies ⋮ Denumerable semi-Markov decision chains with small interest rates ⋮ On variable discounting in dynamic programming: applications to resource extraction and other economic models ⋮ Average optimality for risk-sensitive control with general state space ⋮ Policy iteration algorithms for zero-sum stochastic differential games with long-run average payoff criteria ⋮ Optimal dynamic load distribution in a class of flow-type flexible manufacturing systems ⋮ Stochastic dynamic programming with non-linear discounting ⋮ Maximizing the probability of visiting a set infinitely often for a countable state space Markov decision process ⋮ Zero-sum Markov games with random state-actions-dependent discount factors: existence of optimal strategies ⋮ Stochastic optimal growth with bounded or unbounded utility and with bounded or unbounded shocks ⋮ Adaptive control of stochastic systems with unknown disturbance distribution: discounted criteria ⋮ A note on negative dynamic programming for risk-sensitive control ⋮ Dynamic programming with state-dependent discounting ⋮ A linear-quadratic Gaussian approach to dynamic information acquisition ⋮ Some structured dynamic programs arising in economics ⋮ Stochastic games with unbounded payoffs: applications to robust control in economics ⋮ Policy iteration for continuous-time average reward Markov decision processes in Polish spaces ⋮ Continuous-time Markov decision processes with state-dependent discount factors ⋮ The discounted method and equivalence of average criteria for risk-sensitive Markov decision processes on Borel spaces ⋮ Compactness of the space of non-randomized policies in countable-state sequential decision processes ⋮ Semi-Markov control models with partially known holding times distribution: discounted and average criteria ⋮ Stochastic growth with short-run prediction of shocks ⋮ Nonstationary discrete-time deterministic and stochastic control systems: bounded and unbounded cases ⋮ Manufacturing lead-time rules: customer retention versus tardiness costs ⋮ On stopped decision processes with discrete time parameter ⋮ Average cost optimal policies for Markov control processes with Borel state space and unbounded costs ⋮ Nonstationary discrete-time deterministic and stochastic control systems with infinite horizon ⋮ Generalised discounting in dynamic programming with unbounded returns ⋮ Measurable selection theorems for optimization problems ⋮ A dynamic game approach to distributionally robust safety specifications for stochastic systems ⋮ A general stochastic fixed-point theorem for continuous random operators on stochastic domains ⋮ A survey of Markov decision models for control of networks of queues ⋮ Dynamic programming method for deterministic discrete processes of general form ⋮ Semicontinuous nonstationary stochastic games. II ⋮ On piecewise deterministic Markov control processes: Control of jumps and of risk processes in insurance ⋮ Adaptive control of diffusion processes with a discounted reward criterion ⋮ Sensitivity analysis of multisector optimal economic dynamics ⋮ Characterizations of overtaking optimality for controlled diffusion processes ⋮ MDPs with setwise continuous transition probabilities ⋮ A transformation method for stochastic control problems with partial observations ⋮ Limiting optimal discounted-cost control of a class of time-varying stochastic systems ⋮ Zum Problem des zweiarmigen Bernoulli-Banditen mit einer bekannten Erfolgswahrscheinlichkeit und unendlich vielen Spielen ⋮ Convex analytic method revisited: further optimality results and performance of deterministic policies in average cost stochastic control ⋮ On a stopping rule for a class of sequential decision problems ⋮ A strategic market game with active bankruptcy ⋮ Adaptive control of discounted Markov decision chains ⋮ Semi-Markov decision processes with variance minimization criterion ⋮ Discounted robust control for Markov diffusion processes ⋮ Control of arrivals to two queues in series ⋮ Zero-sum semi-Markov games with state-action-dependent discount factors ⋮ A note on Optimal control of a queueing system with two heterogeneous servers ⋮ Nonstationary value-iteration and adaptive control of discounted semi- Markov processes ⋮ Overtaking optimality for controlled Markov-modulated diffusions ⋮ Lock and no-lock mortgage plans: Is it only a matter of risk shifting? ⋮ Stationary policies and Markov policies in Borel dynamic programming ⋮ Nonzero-sum stochastic differential games with additive structure and average payoffs

Cites Work

• Unnamed Item
• Unnamed Item
• Unnamed Item
• Unnamed Item
• Unnamed Item
• Unnamed Item
• Unnamed Item
• Unnamed Item
• On dynamic programming: Compactness of the space of policies
• On stopped decision processes with discrete time parameter
• A selection theorem for optimization problems
• Instationäre dynamische Optimierung bei schwachen Voraussetzungen über die Gewinnfunktionen
• Bayesian dynamic programming
• Markovian Decision Processes with Compact Action Spaces
• Discounted Dynamic Programming
• Negative Dynamic Programming
• On continuous dynamic programming with discrete time-parameter
• On the Dubins and Savage Characterization of Optimal Strategies
• Topologies on Spaces of Subsets

This page was built for publication: Conditions for optimality in dynamic programming and for the limit of n-stage optimal policies to be optimal