Basic ideas for event-based optimization of Markov systems

DOI10.1007/s10626-004-6211-4zbMath1130.90054OpenAlexW2062472527MaRDI QIDQ1773104

Cao, Xiren

Publication date: 25 April 2005

Published in: Discrete Event Dynamic Systems (Search for Journal in Brave)

Full work available at URL: https://doi.org/10.1007/s10626-004-6211-4

zbMATH Keywords

aggregationperturbation analysispolicy iterationperformance potentialsPOMDPspolicy gradientsMarkov decision processes (MDPs)

Mathematics Subject Classification ID

Markov chains (discrete-time Markov processes on discrete state spaces) (60J10) Discrete event control/observation systems (93C65) Applications of Markov chains and discrete-time Markov processes on general state spaces (social mobility, learning theory, industrial processes, etc.) (60J20) Markov and semi-Markov decision processes (90C40)

Related Items (14)

A reinforcement-learning approach for admission control in distributed network service systems ⋮ Event-based optimization approach for solving stochastic decision problems with probabilistic constraint ⋮ Potential-based least-squares policy iteration for a parameterized feedback control system ⋮ Language for Description of Worlds. Part 1: Theoretical Foundation ⋮ Finding optimal memoryless policies of POMDPs under the expected average reward criterion ⋮ Potentials based optimization with embedded Markov chain for stochastic constrained system ⋮ Performance optimization of queueing systems with perturbation realization ⋮ A tutorial on event-based optimization -- a new optimization framework ⋮ Event-based optimization of admission control in open queueing networks ⋮ An Overview for Markov Decision Processes in Queues and Networks ⋮ Policy iteration for customer-average performance optimization of closed queueing systems ⋮ Policy Gradient Approach of Event‐Based Optimization and Its Online Implementation ⋮ A survey of recent results on continuous-time Markov decision processes (with comments and rejoinder) ⋮ A Sensitivity‐Based Construction Approach to Variance Minimization of Markov Decision Processes

Cites Work

• Unnamed Item
• Unnamed Item
• Unnamed Item
• Unnamed Item
• Unnamed Item
• A unified approach to Markov decision problems and performance sensitivity analysis with discounted and average criteria: multichain cases
• Perturbation analysis and optimization of queueing networks
• Single sample path-based optimization of Markov chains
• Realization probabilities. The dynamics of queuing systems
• The relations among potentials, perturbation analysis, and Markov decision processes
• A time aggregation approach to Markov decision processes
• \({\mathcal Q}\)-learning
• A unified approach to Markov decision problems and performance sensitivity analysis
• A single sample path-based performance sensitivity formula for Markov chains
• Stochastic optimization of regenerative systems using infinitesimal perturbation analysis
• Perturbation realization, potentials, and sensitivity analysis of Markov processes
• CONVERGENCE OF SIMULATION-BASED POLICY ITERATION
• Simulation-based optimization of Markov reward processes
• The no free lunch theorems: complexity and security
• Potential-Based Online Policy Iteration Algorithms for Markov Decision Processes
• The Potential Structure of Sample Paths and Performance Sensitivities of Markov Systems
• Recent advances in hierarchical reinforcement learning

This page was built for publication: Basic ideas for event-based optimization of Markov systems