scientific article; zbMATH DE number 1325008
From MaRDI portal
Publication:4255598
zbMath0928.93002MaRDI QIDQ4255598
Onésimo Hernández-Lerma, Jean-Bernard Lasserre
Publication date: 19 August 1999
Title: zbMATH Open Web Interface contents unavailable due to conflicting licenses.
linear programmingdynamic programmingoptimal policiesdiscrete-time Markov control processesexpected total cost criterionMarkov control modelssample path optimalityundiscounted cost criteriaweighted-norm spaces
Discrete-time Markov processes on general state spaces (60J05) Dynamic programming in optimal control and differential games (49L20) Dynamic programming (90C39) Research exposition (monographs, survey articles) pertaining to systems and control theory (93-02) Optimal stochastic control (93E20) Markov and semi-Markov decision processes (90C40)
Related Items
On the optimality equation for average cost Markov control processes with Feller transition probabilities, Nonzero-sum constrained discrete-time Markov games: the case of unbounded costs, LP based upper and lower bounds for Cesàro and Abel limits of the optimal values in problems of control of stochastic discrete time systems, Finite horizon continuous-time Markov decision processes with mean and variance criteria, Zero-sum discounted reward criterion games for piecewise deterministic Markov processes, Zero-sum stochastic games with average payoffs: new optimality conditions, The Lagrange approach to infinite linear programs, Conditions for the solvability of the linear programming formulation for constrained discounted Markov decision processes, Impulsive control for continuous-time Markov decision processes: a linear programming approach, A two-state partially observable Markov decision process with three actions, Zero-sum continuous-time Markov games with unbounded transition and discounted payoff rates, A perturbation approach to a class of discounted approximate value iteration algorithms with Borel spaces, Uniform ergodicity of continuous-time controlled Markov chains: a survey and new results, Remarks on sensitive equilibria in stochastic games with additive reward and transition structure, Average optimal strategies for zero-sum Markov games with poorly known payoff function on one side, Convex analytic approach to constrained discounted Markov decision processes with non-constant discount factors, Optimality in Feller semi-Markov control processes, Approximation of noncooperative semi-Markov games, A semimartingale characterization of average optimal stationary policies for Markov decision processes, Constrained Markov decision processes with first passage criteria, Constrained optimality for finite horizon semi-Markov decision processes in Polish spaces, On symmetric stochastic games of resource extraction with weakly continuous transitions, Variance minimization for constrained discounted continuous-time MDPs with exponentially distributed stopping times, The policy iteration algorithm for average continuous control of piecewise deterministic Markov processes, Filtering \(\mathcal{S}\)-coupled algebraic Riccati equations for discrete-time Markov jump systems, A mean-variance optimization problem for discounted Markov decision processes, First passage problems for nonstationary discrete-time stochastic control systems, Mean-variance problems for finite horizon semi-Markov decision processes, Markov control models with unknown random state-action-dependent discount factors, Strong \(n\)-discount and finite-horizon optimality for continuous-time Markov decision processes, A probability criterion for zero-sum stochastic games, Robust Markov control processes, Nonzero-sum games for continuous-time Markov chains with unbounded transition and average payoff rates, Near optimality of quantized policies in stochastic control under weak continuity conditions, The transformation method for continuous-time Markov decision processes, Markov decision processes with state-dependent discount factors and unbounded rewards/costs, Stochastic optimal growth model with risk sensitive preferences, Average sample-path optimality for continuous-time Markov decision processes in Polish spaces, Discounted dynamic programming with unbounded returns: application to economic models, Controlled linear system with delayed relay output under impulse random disturbances., New average optimality conditions for semi-Markov decision processes in Borel spaces, A version of the Euler equation in discounted Markov decision processes, Average control of Markov decision processes with Feller transition probabilities and general action spaces, Total reward criteria for unconstrained/constrained continuous-time Markov decision processes, A risk minimization problem for finite horizon semi-Markov decision processes with loss rates, Semi-Markov control processes with unknown holding times distribution under an average cost criterion, Discounted continuous-time Markov decision processes with unbounded rates and randomized history-dependent policies: the dynamic programming approach, Numerical analysis of generalised max-plus eigenvalue problems., Optimal threshold probability in undiscounted Markov decision processes with a target set., Discounted continuous-time constrained Markov decision processes in Polish spaces, On variable discounting in dynamic programming: applications to resource extraction and other economic models, Modeling stochastic dominance as infinite-dimensional constraint systems via the Strassen theorem, Approximation of Markov decision processes with general state space, Optimal timing of decisions: a general theory based on continuation values, Hitting times in Markov chains with restart and their application to network centrality, Discrete-time hybrid control in Borel spaces, Discrete-time hybrid control in Borel spaces: average cost optimality criterion, Continuous-time Markov decision processes with risk-sensitive finite-horizon cost criterion, Constrained Markov decision processes in Borel spaces: from discounted to average optimality, Attack allocation on remote state estimation in multi-systems: structural results and asymptotic solution, Solutions of the average cost optimality equation for Markov decision processes with weakly continuous kernel: the fixed-point approach revisited, Finite approximation of the first passage models for discrete-time Markov decision processes with varying discount factors, Quantitative model-checking of controlled discrete-time Markov processes, Stochastic games for continuous-time jump processes under finite-horizon payoff criterion, Continuous-time constrained stochastic games under the discounted cost criteria, Linear programming approach to optimal impulse control problems with functional constraints, Stochastic games with unbounded payoffs: applications to robust control in economics, Policy iteration for continuous-time average reward Markov decision processes in Polish spaces, Continuous-time Markov decision processes with state-dependent discount factors, An excursion-theoretic approach to stability of discrete-time stochastic hybrid systems, Singular perturbation for the discounted continuous control of piecewise deterministic Markov processes, New optimality conditions for average-payoff continuous-time Markov games in Polish spaces, Maximizing the probability of attaining a target prior to extinction, Nonstationary discrete-time deterministic and stochastic control systems: bounded and unbounded cases, On a continuous solution to the Bellman-Poisson equation in stochastic games, Optimal threshold probability and expectation in semi-Markov decision processes, Variance minimization for continuous-time Markov decision processes: two approaches, Markov decision processes with quasi-hyperbolic discounting, Generalised discounting in dynamic programming with unbounded returns, Constrained stochastic games with the average payoff criteria, Markov control processes with pathwise constraints, A survey of recent results on continuous-time Markov decision processes (with comments and rejoinder), Average optimality for continuous-time Markov decision processes in Polish spaces, Equilibrium storage with multiple commodities, A separation principle for partially observed control of singular stochastic processes, Revisiting optimal investment strategies of value-maximizing insurance firms, Discounted cost optimality problem: Stability with respect to weak metrics, Two person zero-sum semi-Markov games with unknown holding times distribution on one side: A discounted payoff criterion, Characterizations of overtaking optimality for controlled diffusion processes, Zero-sum ergodic semi-Markov games with weakly continuous transition probabilities, Zero-sum games for pure jump processes with risk-sensitive discounted cost criteria, Nonzero-sum risk-sensitive average stochastic games: The case of unbounded costs, Limiting optimal discounted-cost control of a class of time-varying stochastic systems, Nonzero-sum stochastic games with probability criteria, Stochastic optimal growth with unbounded shock, Nonatomic total rewards Markov decision processes with multiple criteria, Minimax control for discrete-time time-varying stochastic systems, Ergodicity conditions for zero-sum games, Nonzero-sum stochastic differential games with additive structure and average payoffs, Discount-sensitive equilibria in zero-sum stochastic differential games, Continuous-time zero-sum games for markov decision processes with discounted risk-sensitive cost criterion on a general state space, Risk-Sensitive Average Optimality for Discrete-Time Markov Decision Processes, Some Limit Properties of Markov Chains Induced by Recursive Stochastic Algorithms, Stability Estimation of Transient Markov Decision Processes, Generalized maximum entropy estimation, Unnamed Item, Constrained Continuous-Time Markov Control Processes with Discounted Criteria, APPROXIMATE DYNAMIC PROGRAMMING TECHNIQUES FOR THE CONTROL OF TIME-VARYING QUEUING SYSTEMS APPLIED TO CALL CENTERS WITH ABANDONMENTS AND RETRIALS, Blackwell Optimality for Controlled Diffusion Processes, On Linear Programming for Constrained and Unconstrained Average-Cost Markov Decision Processes with Countable Action Spaces and Strictly Unbounded Costs, Finite-horizon piecewise deterministic Markov decision processes with unbounded transition rates, Risk sensitive control of pure jump processes on a general state space, Zero-sum semi-Markov games with a probability criterion, Regular Policies in Abstract Dynamic Programming, Zero-sum infinite-horizon discounted piecewise deterministic Markov games, Formalization of methods for the development of autonomous artificial intelligence systems, Zero-sum stochastic games with the average-value-at-risk criterion, A dynamic analytic method for risk-aware controlled martingale problems, Discrete-time switching control in random walks, Short Communication: Existence of Markov Equilibrium Control in Discrete Time, Robustness to Incorrect System Models in Stochastic Control, Some advances on constrained Markov decision processes in Borel spaces with random state-dependent discount factors, A note on the existence of optimal stationary policies for average Markov decision processes with countable states, Extreme Occupation Measures in Markov Decision Processes with an Absorbing State, Risk-sensitive first passage stochastic games with unbounded costs, Absorbing Markov decision processes, Realizable Strategies in Continuous-Time Markov Decision Processes, A Verification Theorem for Threshold-Indexability of Real-State Discounted Restless Bandits, Multiconstrained Finite-Horizon Piecewise Deterministic Markov Decision Processes with Unbounded Transition Rates, A Universal Dynamic Program and Refined Existence Results for Decentralized Stochastic Control, Average Cost Optimality Inequality for Markov Decision Processes with Borel Spaces and Universally Measurable Policies, A Convex Programming Approach for Discrete-Time Markov Decision Processes under the Expected Total Reward Criterion, Nonzero-sum games for continuous-time Markov chains with unbounded discounted payoffs, Another Set of Conditions for Strongn(n = −1, 0) Discount Optimality in Markov Decision Processes, Controlled Random Walk: Conjecture and Counter-Example, Q-Learning for Distributionally Robust Markov Decision Processes, Markov decision processes associated with two threshold probability criteria, Bisimulation for Markov Decision Processes through Families of Functional Expressions, Zero-sum games for continuous-time jump Markov processes in Polish spaces: discounted payoffs, Asymptotic Normality of Discrete-Time Markov Control Processes, A Fixed Point Approach to Solve the Average Cost Optimality Equation for Semi-Markov Decision Processes with Feller Transition Probabilities, The optimal control of nonlinear diffusion equations with rough initial data, Optimal Distributed Uplink Channel Allocation: A Constrained MDP Formulation, Unnamed Item, New sufficient conditions for average optimality in continuous-time Markov decision processes, Nonzero-Sum Expected Average Discrete-Time Stochastic Games: The Case of Uncountable Spaces, Continuous-Time Markov Decision Processes with Unbounded Transition and Discounted-Reward Rates, On Reducing a Constrained Gradual-Impulsive Control Problem for a Jump Markov Model to a Model with Gradual Control Only, Risk-sensitive semi-Markov decision processes with general utilities and multiple criteria, A Sensitivity‐Based Construction Approach to Variance Minimization of Markov Decision Processes, Maximizing terminal utility by controlling risk exposure; a discrete-time dynamic control approach, Discrete-time average-cost mean-field games on Polish spaces, On the Minimum Pair Approach for Average Cost Markov Decision Processes with Countable Discrete Action Spaces and Strictly Unbounded Costs, A perturbation approach to approximate value iteration for average cost Markov decision processes with Borel spaces and bounded costs, First passage risk probability optimality for continuous time Markov decision processes, Time-varying Markov decision processes with state-action-dependent discount factors and unbounded costs, Unnamed Item, Unnamed Item, Robustness to Incorrect Priors in Partially Observed Stochastic Control, Optimal Impulse Control of Dynamical Systems, Constrained Markov Decision Processes with Expected Total Reward Criteria, Optimality of admission control in an M∕M∕1∕N queue with varying services, Impulsive Control for Continuous-Time Markov Decision Processes, On Convergence of Value Iteration for a Class of Total Cost Markov Decision Processes, Average optimality for continuous-time Markov decision processes under weak continuity conditions, Approximations of Countably Infinite Linear Programs over Bounded Measure Spaces, On the First Passage $g$-Mean-Variance Optimality for Discounted Continuous-Time Markov Decision Processes, Off-Policy Estimation of Long-Term Average Outcomes With Applications to Mobile Health, Unnamed Item, A Convex Analytic Approach to Risk-Aware Markov Decision Processes, Discrete-time constrained stochastic games with the expected average payoff criteria, Unnamed Item, Ergodic risk-sensitive control of Markov processes on countable state space revisited, Dynamic reinsurance in discrete time minimizing the insurer's cost of capital, Distributionally Robust Markov Decision Processes and Their Connection to Risk Measures, Continuous-time zero-sum games for Markov chains with risk-sensitive finite-horizon cost criterion, A Moreau-Yosida regularization for Markov decision processes, Zero and non-zero sum risk-sensitive Semi-Markov games, Unnamed Item, Infinite horizon controlled diffusions with randomly varying and state-dependent discount cost rates, Time and Ratio Expected Average Cost Optimality for Semi-Markov Control Processes on Borel Spaces, Unnamed Item, Unnamed Item, The Expected Total Cost Criterion for Markov Decision Processes under Constraints: A Convex Analytic Approach, Markov Decision Processes with Variance Minimization: A New Condition and Approach, Bias optimality and strong \(n\) \((n= -1,0)\) discount optimality for Markov decision processes, Unbounded dynamic programming via the Q-transform, Bayesian estimation of the mean holding time in average semi-Markov control processes, Unnamed Item, On Some Ergodic Impulse Control Problems with Constraint, Randomized and Relaxed Strategies in Continuous-Time Markov Decision Processes, Discounted stochastic games for continuous-time jump processes with an uncountable state space, Risk-sensitive discounted cost criterion for continuous-time Markov decision processes on a general state space, Continuous-time zero-sum games for Markov decision processes with discounted risk-sensitive cost criterion, Minimum Average Value-at-Risk for Finite Horizon Semi-Markov Decision Processes in Continuous Time, A consumption and investment problem via a Markov decision processes approach with random horizon, A Mixed Value and Policy Iteration Method for Stochastic Control with Universally Measurable Policies, Markov--Nash Equilibria in Mean-Field Games with Discounted Cost, Regime switching optimal growth model with risk sensitive preferences, Constrained continuous-time Markov decision processes on the finite horizon, Constrained Markov control processes with randomized discounted cost criteria: infinite linear programming approach, From Infinite to Finite Programs: Explicit Error Bounds with Applications to Approximate Dynamic Programming, Aggregated occupation measures and linear programming approach to constrained impulse control problems, Continuous-time zero-sum stochastic game with stopping and control, A useful technique for piecewise deterministic Markov decision processes, Equilibria in altruistic economic growth models, Risk-sensitive continuous-time Markov decision processes with unbounded rates and Borel spaces, Bias optimality of admission control in a non-stationary repairable queue, Another set of verifiable conditions for average Markov decision processes with Borel spaces, Nonzero-sum risk-sensitive continuous-time stochastic games with ergodic costs, Red light green light method for solving large Markov chains, First Passage Optimality and Variance Minimisation of Markov Decision Processes with Varying Discount Factors, Asymptotic optimality and rates of convergence of quantized stationary policies in continuous-time Markov decision processes, Bounding Stationary Averages of Polynomial Diffusions via Semidefinite Programming, Minimizing Ruin Probabilities by Reinsurance and Investment: A Markovian Decision Approach, Estimation of the Optimality Deviation in Discounted Semi-Markov Control Models, A New Condition and Approach for Zero-Sum Stochastic Games with Average Payoffs, Empirical estimation in average Markov control processes, A survey of average cost problems in deterministic discrete-time control systems, An average-value-at-risk criterion for Markov decision processes with unbounded costs, Discrete-time zero-sum games for Markov chains with risk-sensitive average cost criterion, First passage models for denumerable semi-Markov decision processes with nonnegative discounted costs, On discounted dynamic programming with unbounded returns, Bias and Overtaking Optimality for Continuous-Time Jump Markov Decision Processes in Polish Spaces, Value iteration algorithm for mean-field games, Continuous-time Markov decision processes under the risk-sensitive first passage discounted cost criterion, On the limit perfect public equilibrium payoff set in repeated and stochastic games, Discrete-time control with non-constant discount factor, Optimal dividend policy in discrete time, A policy improvement method for constrained average Markov decision processes, Sample-path optimality and variance-maximization for Markov decision processes, Stochastic dynamic programming with non-linear discounting, An Overview for Markov Decision Processes in Queues and Networks, Average optimality for continuous-time Markov decision processes with a policy iteration approach, Stochastic optimal growth with bounded or unbounded utility and with bounded or unbounded shocks, Markov decision processes with recursive risk measures, Average optimality inequality for continuous-time Markov decision processes in Polish spaces, Adaptive control of stochastic systems with unknown disturbance distribution: discounted criteria, Zero-sum stochastic games with partial information and average payoff, Constrained continuous-time Markov decision processes with average criteria, Inequalities for the ruin probability in a controlled discrete-time risk process, Exit time risk-sensitive control for systems of cooperative agents, Finite-horizon optimality for continuous-time Markov decision processes with unbounded transition rates, Weak Feller property of non-linear filters, Nonstationary discrete-time deterministic and stochastic control systems with infinite horizon, Risk-sensitive average equilibria for discrete-time stochastic games, Another set of conditions for Markov decision processes with average sample-path costs, Finite horizon risk-sensitive continuous-time Markov decision processes with unbounded transition and cost rates, Multiobjective Stopping Problem for Discrete-Time Markov Processes: Convex Analytic Approach, Constrained discounted Markov decision processes with Borel state spaces, Zero-sum games for continuous-time Markov chains with unbounded transition and average payoff rates, New discount and average optimality conditions for continuous-time Markov decision processes, Bounds for the Ruin Probability of a Discrete-Time Risk Process, Risk-sensitive finite-horizon piecewise deterministic Markov decision processes, Ergodic Control, Bias, and Sensitive Discount Optimality for Markov Diffusion Processes, Convergence of Markov decision processes with constraints and state-action dependent discount factors, Continuous-time zero-sum games with probability criterion, A Fenchel-Moreau-Rockafellar type theorem on the Kantorovich-Wasserstein space with applications in partially observable Markov decision processes, The Vanishing Discount Approach for the Average Continuous Control of Piecewise Deterministic Markov Processes, Optimality of Mixed Policies for Average Continuous-Time Markov Decision Processes with Constraints, Duality in optimal impulse control, On structural properties of optimal average cost functions in Markov decision processes with Borel spaces and universally measurable policies, Unnamed Item, Absorbing Continuous-Time Markov Decision Processes with Total Cost Criteria, Average optimality for Markov decision processes in borel spaces: a new condition and approach, On the value function of the M/Cox(r)/1 queue, Convex analytic method revisited: further optimality results and performance of deterministic policies in average cost stochastic control, Mean-semivariance optimality for continuous-time Markov decision processes, On the reduction of total‐cost and average‐cost MDPs to discounted MDPs, The Expected Total Cost Criterion for Markov Decision Processes under Constraints, Dynamic programming for ergodic control with partial observations., Stochastic approximations of constrained discounted Markov decision processes, Semi-Markov decision processes with variance minimization criterion, Zero-sum risk-sensitive stochastic games with unbounded payoff functions and varying discount factors, Zero-sum semi-Markov games with state-action-dependent discount factors, Batch policy learning in average reward Markov decision processes, Computational Methods for Risk-Averse Undiscounted Transient Markov Models, First passage Markov decision processes with constraints and varying discount factors, Zero-sum average cost semi-Markov games with weakly continuous transition probabilities and a minimax semi-Markov inventory problem
