Robust Control of Markov Decision Processes with Uncertain Transition Matrices

DOI10.1287/opre.1050.0216zbMath1165.90674OpenAlexW1965878388MaRDI QIDQ5322137

No author found.

Publication date: 18 July 2009

Published in: Operations Research (Search for Journal in Brave)

Full work available at URL: https://semanticscholar.org/paper/6db16608fccddef51202af84112b34cfebfbe20a

Mathematics Subject Classification ID

Minimax problems in mathematical programming (90C47) Dynamic programming in optimal control and differential games (49L20) Markov and semi-Markov decision processes (90C40)

Related Items

Explicit explore, exploit, or escape \((E^4)\): near-optimal safety-constrained reinforcement learning in polynomial time ⋮ Data-driven remanufacturing planning with parameter uncertainty ⋮ Parameter synthesis in Markov models: a gentle survey ⋮ A few lessons learned in reinforcement learning for quadcopter attitude control ⋮ Stochastic control for organ donations: a review ⋮ Joint chance-constrained Markov decision processes ⋮ Distributionally Robust Strategy Synthesis for Switched Stochastic Systems ⋮ Interval Markov Decision Processes with Continuous Action-Spaces ⋮ Robust Control for Dynamical Systems with Non-Gaussian Noise via Formal Abstractions ⋮ A survey of nonlinear robust optimization ⋮ Robust optimization in countably infinite linear programs ⋮ Variable demand and multi-commodity flow in Markovian network equilibrium ⋮ Scenario-Based Verification of Uncertain MDPs ⋮ Ambiguous partially observable Markov decision processes: structural results and applications ⋮ Probabilistic Timed Automata with One Clock and Initialised Clock-Dependent Probabilities ⋮ Robust decomposable Markov decision processes motivated by allocating school budgets ⋮ Tight Approximations of Dynamic Risk Measures ⋮ Oracle-Based Robust Optimization via Online Learning ⋮ Partially observable Markov decision processes incorporating ⋮ Quantitative verification and strategy synthesis for stochastic games ⋮ Dynamic programming for deterministic discrete-time systems with uncertain gain ⋮ Optimal Information Blending with Measurements in the L² Sphere ⋮ Reachability analysis of uncertain systems using bounded-parameter Markov decision processes ⋮ Deterministic policies based on maximum regrets in MDPs with imprecise rewards ⋮ Lipschitzness is all you need to tame off-policy generative adversarial imitation learning ⋮ Data-Driven Pricing for a New Product ⋮ Robust Markov Decision Processes with Data-Driven, Distance-Based Ambiguity Sets ⋮ Robust stability, ℋ₂ analysis and stabilisation of discrete-time Markov jump linear systems with uncertain probability matrix ⋮ Poisoning finite-horizon Markov decision processes at design time ⋮ Uncertainty quantification for Markov chain models ⋮ Minimax and risk averse multistage stochastic programming ⋮ Constrained Markov decision processes with uncertain costs ⋮ Estimating permanent price impact via machine learning ⋮ Quantile Markov Decision Processes ⋮ Robust Markov control processes ⋮ Z-relation-based multistage decision making ⋮ Variance-constrained actor-critic algorithms for discounted and average reward MDPs ⋮ Robust solutions to Stackelberg games: addressing bounded rationality and limited observations in human cognition ⋮ Risk-Averse Stochastic Programming: Time Consistency and Optimal Stopping ⋮ Minimax Q-learning control for linear systems using the Wasserstein metric ⋮ ON ROBUST MULTI-PERIOD PRE-COMMITMENT AND TIME-CONSISTENT MEAN-VARIANCE PORTFOLIO OPTIMIZATION ⋮ Risk-aware controller for autonomous vehicles using model-based collision prediction and reinforcement learning ⋮ Coefficients of ergodicity for Markov chains with uncertain parameters ⋮ Unnamed Item ⋮ Robustness to Incorrect System Models in Stochastic Control ⋮ A graph-theoretic-based method for analyzing conduction problems ⋮ Efficient solutions to factored MDPs with imprecise transition probabilities ⋮ Using mathematical programming to solve factored Markov decision processes with imprecise probabilities ⋮ Online First-Order Framework for Robust Convex Optimization ⋮ Learning parametric policies and transition probability models of Markov decision processes from data ⋮ Unnamed Item ⋮ Robust topological policy iteration for infinite horizon bounded Markov decision processes ⋮ Computation of weighted sums of rewards for concurrent MDPs ⋮ Technical Note—Time Inconsistency of Optimal Policies of Distributionally Robust Inventory Models ⋮ Distributionally Robust Partially Observable Markov Decision Process with Moment-Based Ambiguity ⋮ On the Complexity of Reachability in Parametric Markov Decision Processes ⋮ A dynamic inventory rationing problem with uncertain demand and production rates ⋮ Robust bounds and optimization at the large deviations scale for queueing models via Rényi divergence ⋮ A survey of decision making and optimization under uncertainty ⋮ Unnamed Item ⋮ Unnamed Item ⋮ Policy iteration for robust nonstationary Markov decision processes ⋮ An analysis of model-based interval estimation for Markov decision processes ⋮ Discrete time Markov chains with interval probabilities ⋮ Light robustness in the optimization of Markov decision processes with uncertain parameters ⋮ Reinforcement learning with limited reinforcement: using Bayes risk for active learning in POMDPs ⋮ Rectangular Sets of Probability Measures ⋮ Robust Actuarial Risk Analysis ⋮ A dynamic programming approach to adjustable robust optimization ⋮ Policy-based branch-and-bound for infinite-horizon multi-model Markov decision processes ⋮ Tutorial on risk neutral, distributionally robust and risk averse multistage stochastic programming ⋮ Time (in)consistency of multistage distributionally robust inventory models with moment constraints ⋮ Asymptotic optimality of the generalized \(c\mu\) rule under model uncertainty ⋮ Likelihood robust optimization for data-driven problems ⋮ Selected topics in robust convex optimization ⋮ Robust response-guided dosing ⋮ Robust analysis of discounted Markov decision processes with uncertain transition probabilities ⋮ Robust Adversarial Risk Analysis: A Level-k Approach ⋮ Extended Laplace principle for empirical measures of a Markov chain ⋮ Continuous-Time Robust Dynamic Programming ⋮ Robust Adaptive Routing Under Uncertainty ⋮ Discrete Approximation and Quantification in Distributionally Robust Optimization ⋮ Quantifying Distributional Model Risk via Optimal Transport ⋮ Data Uncertainty in Markov Chains: Application to Cost-Effectiveness Analyses of Medical Innovations ⋮ Robust Analysis in Stochastic Simulation: Computation and Performance Guarantees ⋮ Distributionally robust optimal control and MDP modeling ⋮ IMPRECISE MARKOV CHAINS AND THEIR LIMIT BEHAVIOR ⋮ Partially observable Markov decision processes with imprecise parameters ⋮ Robust Control of Partially Observable Failing Systems ⋮ ROBUST ASSET ALLOCATION WITH BENCHMARKED OBJECTIVES ⋮ Robust Sensitivity Analysis for Stochastic Systems ⋮ Reinforcement Learning in Robust Markov Decision Processes ⋮ Robust MDPs with k-Rectangular Uncertainty ⋮ ROBUST DYNAMIC PRICING OVER INFINITE HORIZON IN THE PRESENCE OF MODEL UNCERTAINTY ⋮ Distributionally robust optimization for sequential decision-making ⋮ Sensitivity Analysis in Markov Decision Processes with Uncertain Reward Parameters ⋮ Gittins' theorem under uncertainty ⋮ Learning and planning in partially observable environments without prior domain knowledge ⋮ Risk-averse policy optimization via risk-neutral policy optimization ⋮ Distributionally robust modeling of optimal control ⋮ Robust control of the multi-armed bandit problem ⋮ Concurrent MDPs with Finite Markovian Policies ⋮ Distributionally Robust Markov Decision Processes and Their Connection to Risk Measures ⋮ Distributionally Robust Inventory Control When Demand Is a Martingale ⋮ Formulation and properties of a divergence used to compare probability measures without absolute continuity ⋮ Toward theoretical understandings of robust Markov decision processes: sample complexity and asymptotics ⋮ Algorithmic aspects of mean-variance optimization in Markov decision processes