Risk-averse dynamic programming for Markov decision processes

DOI10.1007/s10107-010-0393-3zbMath1207.49032OpenAlexW2001009060MaRDI QIDQ607497

Ruszczyński, Andrzej

Publication date: 22 November 2010

Published in: Mathematical Programming. Series A. Series B (Search for Journal in Brave)

Full work available at URL: https://doi.org/10.1007/s10107-010-0393-3

zbMATH Keywords

policy iterationvalue iterationdynamic risk measuresMarkov risk measuresmin-max Markov modelsnonsmooth Newton's method

Mathematics Subject Classification ID

Related Items (only showing first 100 items - show all)

Dynamic Programming Deconstructed: Transformations of the Bellman Equation and Computational Efficiency ⋮ Risk measurement and risk-averse control of partially observable discrete-time Markov systems ⋮ Equivalence between time consistency and nested formula ⋮ A combined SDDP/Benders decomposition approach with a risk-averse surface concept for reservoir operation in long term power generation planning ⋮ Tight Approximations of Dynamic Risk Measures ⋮ Equilibrium, uncertainty and risk in hydro-thermal electricity systems ⋮ Reference points and learning ⋮ Analyzing operational risk-reward trade-offs for start-ups ⋮ Time-inconsistent multistage stochastic programs: martingale bounds ⋮ On time stochastic dominance induced by mixed integer-linear recourse in multistage stochastic programs ⋮ Building up time-consistency for risk measures and dynamic optimization ⋮ Risk aversion in multistage stochastic programming: a modeling and algorithmic perspective ⋮ Models for Optimization of Power Systems ⋮ Minimum Average Value-at-Risk for Finite Horizon Semi-Markov Decision Processes in Continuous Time ⋮ Recursive risk measures under regime switching applied to portfolio selection ⋮ Optimal Information Blending with Measurements in the L² Sphere ⋮ Equal risk pricing and hedging of financial derivatives with convex risk measures ⋮ Time consistent multi-period worst-case risk measure in robust portfolio selection ⋮ A quantitative comparison of risk measures ⋮ Markov Decision Problems Where Means Bound Variances ⋮ From Infinite to Finite Programs: Explicit Error Bounds with Applications to Approximate Dynamic Programming ⋮ Resource planning strategies for healthcare systems during a pandemic ⋮ Discrete-time risk-aware optimal switching with non-adapted costs ⋮ Markov decision processes with iterated coherent risk measures ⋮ A generalization of Bellman's equation with application to path planning, obstacle avoidance and invariant set estimation ⋮ Risk-sensitive control of Markov decision processes: a moment-based approach with target distributions ⋮ Dynamic programming with value convexity ⋮ Stability of a class of risk-averse multistage stochastic programs and their distributionally robust counterparts ⋮ Time-Coherent Risk Measures for Continuous-Time Markov Chains ⋮ Effective Scenarios in Multistage Distributionally Robust Optimization with a Focus on Total Variation Distance ⋮ Minimax and risk averse multistage stochastic programming ⋮ Quantile Markov Decision Processes ⋮ Dynamic Risked Equilibrium ⋮ Mean-variance problems for finite horizon semi-Markov decision processes ⋮ Scenario decomposition of risk-averse multistage stochastic programming problems ⋮ Dynamic consistency for stochastic optimal control problems ⋮ Analysis of stochastic dual dynamic programming method ⋮ Risk forms: representation, disintegration, and application to partially observable two-stage systems ⋮ Martingale characterizations of risk-averse stochastic optimization problems ⋮ Variance-constrained actor-critic algorithms for discounted and average reward MDPs ⋮ Time consistency of dynamic risk measures ⋮ Unnamed Item ⋮ Dynamic sampling algorithms for multi-stage stochastic programs with risk aversion ⋮ Regularized Decomposition of High-Dimensional Multistage Stochastic Programs with Markov Uncertainty ⋮ Markov risk mappings and risk-sensitive optimal prediction ⋮ An axiomatic approach to Markov decision processes ⋮ Optimal investment policy in the time consistent mean-variance formulation ⋮ Probabilistically distorted risk-sensitive infinite-horizon dynamic programming ⋮ Monotone trends in inventory-price control under time-consistent coherent risk measure ⋮ Time-consistent, risk-averse dynamic pricing ⋮ Risk-averse model predictive control ⋮ Two-stage portfolio optimization with higher-order conditional measures of risk ⋮ Evaluating policies in risk-averse multi-stage stochastic programming ⋮ Stability Analysis of Optimization Problems with $k$th order stochastic and distributionally robust dominance constraints induced by full random recourse ⋮ Minimizing spectral risk measures applied to Markov decision processes ⋮ Markov decision processes with recursive risk measures ⋮ On tight bounds for function approximation error in risk-sensitive reinforcement learning ⋮ On capacity expansion planning under strategic and operational uncertainties based on stochastic dominance risk averse management ⋮ Composite time-consistent multi-period risk measure and its application in optimal portfolio selection ⋮ Time-consistent investment policies in Markovian markets: a case of mean-variance analysis ⋮ Erratum to ``Risk-averse dynamic programming for Markov decision processes ⋮ Simulation optimization of risk measures with adaptive risk levels ⋮ Risk-averse stochastic path detection ⋮ Bounds on risk-averse mixed-integer multi-stage stochastic programming problems with mean-CVaR ⋮ Risk management for forestry planning under uncertainty in demand and prices ⋮ Time consistent multi-period robust risk measures and portfolio selection models with regime-switching ⋮ Decomposability and time consistency of risk averse multistage programs ⋮ Nested conditional value-at-risk portfolio selection: a model with temporal dependence driven by market-index volatility ⋮ A dynamic programming approach to adjustable robust optimization ⋮ On the time-consistent stochastic dominance risk averse measure for tactical supply chain planning under uncertainty ⋮ A trade execution model under a composite dynamic coherent risk measure ⋮ A time-consistent Benders decomposition method for multistage distributionally robust stochastic optimization with a scenario tree structure ⋮ Gas storage valuation in incomplete markets ⋮ Time (in)consistency of multistage distributionally robust inventory models with moment constraints ⋮ An online algorithm for the risk-aware restless bandit ⋮ Robust optimal control using conditional risk mappings in infinite horizon ⋮ Resource allocation for contingency planning: an inexact proximal bundle method for stochastic optimization ⋮ A multistage risk-averse stochastic programming model for personal savings accrual: the evidence from Lithuania ⋮ Time consistency and risk averse dynamic decision models: definition, interpretation and practical consequences ⋮ Time consistent policy of multi-period mean-variance ⋮ Structure of risk-averse multistage stochastic programs ⋮ Time-Consistent Decisions and Temporal Decomposition of Coherent Risk Functionals ⋮ Multilevel Optimization Modeling for Risk-Averse Stochastic Programming ⋮ A survey of time consistency of dynamic risk measures and dynamic performance measures in discrete time: LM-measure perspective ⋮ Time-consistent approximations of risk-averse multistage stochastic optimization problems ⋮ Distributionally robust optimal control and MDP modeling ⋮ Socially responsible merchant operations: comparison of shutdown-averse CVaR and anticipated regret policies ⋮ Robust Control of Partially Observable Failing Systems ⋮ Robust MDPs with k-Rectangular Uncertainty ⋮ Gittins' theorem under uncertainty ⋮ Mean-semivariance optimality for continuous-time Markov decision processes ⋮ Risk-averse autonomous systems: a brief history and recent developments from the perspective of optimal control ⋮ Discrete-time mean field games with risk-averse agents ⋮ An active-set strategy to solve Markov decision processes with good-deal risk measure ⋮ Distributionally robust modeling of optimal control ⋮ Equilibrium routing under uncertainty ⋮ Computational Methods for Risk-Averse Undiscounted Transient Markov Models ⋮ Peril, prudence and planning as risk, avoidance and worry ⋮ On variance reduction of mean-CVaR Monte Carlo estimators ⋮ Process-based risk measures and risk-averse control of discrete-time systems

Cites Work

• Unnamed Item
• Unnamed Item
• Unnamed Item
• Unnamed Item
• Unnamed Item
• Unnamed Item
• Unnamed Item
• Unnamed Item
• On a time consistency concept in risk averse multistage stochastic programming
• Dynamic monetary risk measures for bounded discrete-time processes
• Duality in stochastic linear and dynamic programming
• Stochastic optimal control. The discrete time case
• Newton's method for a class of nonsmooth functions
• Nonsmooth equations in optimization. Regularity, calculus, methods and applications
• Convex measures of risk and trading constraints
• Optimal long term growth rate of expected utility of wealth
• From stochastic dominance to mean-risk models: Semideviations as risk measures
• Coherent multiperiod risk adjusted values and Bellman's principle
• Dynamic coherent risk measures
• Generalized deviations in risk analysis
• Time consistent dynamic risk measures
• Markov decision processes with a new optimality criterion: Discrete time
• Risk-Sensitive Control and an Optimal Investment Model
• Coherent Measures of Risk
• Convex risk measures and the dynamics of their penalty functions
• RISK MEASURES AND CAPITAL REQUIREMENTS FOR PROCESSES
• VALUATIONS AND DYNAMIC CONVEX RISK MEASURES
• DYNAMIC INDIFFERENCE VALUATION VIA CONVEX RISK MEASURES
• Discounted MDP’s: Distribution Functions and Exponential Utility Maximization
• A Utility Criterion for Markov Decision Processes
• Temporal Resolution of Uncertainty and Dynamic Choice Theory
• Variational Analysis
• Minimax Control of Discrete-Time Stochastic Systems
• Dual Stochastic Dominance and Related Mean-Risk Models
• Polyhedral Risk Measures in Stochastic Programming
• Optimization of Convex Risk Functions
• Conditional Risk Mappings
• Stationary Ordinal Utility and Impatience
• A SHORT NOTE ON SECOND‐ORDER STOCHASTIC DOMINANCE PRESERVING COHERENT RISK MEASURES
• On the Theory of Dynamic Programming
• Set-valued analysis
• Stochastic finance. An introduction in discrete time
• On consistency of stochastic dominance and mean-semideviation models

This page was built for publication: Risk-averse dynamic programming for Markov decision processes