Average Cost Markov Decision Processes with Weakly Continuous Transition Probabilities

DOI10.1287/moor.1120.0555zbMath1297.90173arXiv1202.4122OpenAlexW1966208686MaRDI QIDQ2925348

Eugene A. Feinberg, Nina V. Zadoianchuk (Zadoyanchuk), Pavlo O. Kasyanov

Publication date: 21 October 2014

Published in: Mathematics of Operations Research (Search for Journal in Brave)

Full work available at URL: https://arxiv.org/abs/1202.4122

zbMATH Keywords

Markov decision processoptimal policyoptimality inequalityaverage cost per unit time

Mathematics Subject Classification ID

Dynamic programming (90C39) Markov and semi-Markov decision processes (90C40)

Related Items

LP based upper and lower bounds for Cesàro and Abel limits of the optimal values in problems of control of stochastic discrete time systems ⋮ Unbounded dynamic programming via the Q-transform ⋮ A Mixed Value and Policy Iteration Method for Stochastic Control with Universally Measurable Policies ⋮ STOCHASTIC SETUP-COST INVENTORY MODEL WITH BACKORDERS AND QUASICONVEX COST FUNCTIONS ⋮ A useful technique for piecewise deterministic Markov decision processes ⋮ Markov Decision Processes with Incomplete Information and Semiuniform Feller Transition Probabilities ⋮ Reduction of total-cost and average-cost MDPs with weakly continuous transition probabilities to discounted mdps ⋮ Optimality Conditions for Partially Observable Markov Decision Processes ⋮ Examples concerning Abel and Cesàro limits ⋮ Near optimality of quantized policies in stochastic control under weak continuity conditions ⋮ Formalization of methods for the development of autonomous artificial intelligence systems ⋮ Continuity of discounted values and the structure of optimal policies for <scp>periodic‐review</scp> inventory systems with setup costs ⋮ A survey of average cost problems in deterministic discrete-time control systems ⋮ A note on the existence of optimal stationary policies for average Markov decision processes with countable states ⋮ Fatou's Lemma in Its Classical Form and Lebesgue's Convergence Theorems for Varying Measures with Applications to Markov Decision Processes ⋮ Average Cost Optimality Inequality for Markov Decision Processes with Borel Spaces and Universally Measurable Policies ⋮ Convergence of probability measures and Markov decision models with incomplete information ⋮ Continuity of minima: local results ⋮ Average Cost Markov Decision Processes with Semi-Uniform Feller Transition Probabilities ⋮ Constrained Markov decision processes in Borel spaces: from discounted to average optimality ⋮ Solutions of the average cost optimality equation for Markov decision processes with weakly continuous kernel: the fixed-point approach revisited ⋮ Unnamed Item ⋮ Planning for the long run: programming with patient, Pareto responsive preferences ⋮ Uniform Fatou's lemma ⋮ On the vanishing discount factor approach for Markov decision processes with weakly continuous transition probabilities ⋮ Berge's theorem for noncompact image sets ⋮ Partially Observable Total-Cost Markov Decision Processes with Weakly Continuous Transition Probabilities ⋮ Fatou's Lemma for Weakly Converging Measures under the Uniform Integrability Condition ⋮ On the Minimum Pair Approach for Average Cost Markov Decision Processes with Countable Discrete Action Spaces and Strictly Unbounded Costs ⋮ MDPs with setwise continuous transition probabilities ⋮ On structural properties of optimal average cost functions in Markov decision processes with Borel spaces and universally measurable policies ⋮ Convex analytic method revisited: further optimality results and performance of deterministic policies in average cost stochastic control ⋮ On Convergence of Value Iteration for a Class of Total Cost Markov Decision Processes ⋮ On the reduction of total‐cost and average‐cost MDPs to discounted MDPs ⋮ Average optimality for continuous-time Markov decision processes under weak continuity conditions ⋮ Berge's maximum theorem for noncompact image sets ⋮ Structure of optimal policies to periodic-review inventory models with convex costs and backorders for all values of discount factors ⋮ Continuity of equilibria for two-person zero-sum games with noncompact action sets and unbounded payoffs ⋮ On the optimality equation for average cost Markov decision processes and its validity for inventory control

Cites Work

• Unnamed Item
• Compactness of the space of non-randomized policies in countable-state sequential decision processes
• A counterexample on the optimality equation in Markov decision chains with the average cost criterion
• Average optimality in dynamic programming on Borel spaces -- unbounded costs and controls
• Fatou's lemma and Lebesgue's convergence theorem for measures
• On Sequential Decisions and Markov Chains
• OPTIMALITY OF FOUR-THRESHOLD POLICIES IN INVENTORY SYSTEMS WITH CUSTOMER RETURNS AND BORROWING/STORAGE OPTIONS
• Average Optimality in Dynamic Programming with General State Space
• A Counterexample on the Semicontinuity of Minima
• Fatou's Lemma for Weakly Converging Probabilities
• Discrete Dynamic Programming
• Optimality Inequalities for Average Cost Markov Decision Processes and the Stochastic Cash Balance Problem
• Markovian Sequential Replacement Processes
• Non-Discounted Denumerable Markovian Decision Models
• Arbitrary State Markovian Decision Processes
• On the Nonexistence of $|varepsilon$-Optimal Randomized Stationary Policies in Average Cost Markov Decision Models
• Optimal decision procedures for finite markov chains. Part I: Examples