Stationary Distributions of Continuous-Time Markov Chains: A Review of Theory and Truncation-Based Approximations
From MaRDI portal
Publication:5150207
DOI10.1137/19M1289625zbMath1469.60243arXiv1909.05794OpenAlexW3128789216MaRDI QIDQ5150207
Juan Kuntz, Philipp Thomas, Mauricio Barahona, Guy-Bart Stan
Publication date: 10 February 2021
Published in: SIAM Review (Search for Journal in Brave)
Full work available at URL: https://arxiv.org/abs/1909.05794
linear programmingerror boundsstochastic reaction networkschemical master equationoptimal approximationsboundedness in probabilityergodic distributionsFoster-Lyapunov criteriareducible Markov chainscensored chainfinite state projection algorithmlevel-dependent quasi-birth-death processestruncation-and-augmentation scheme
Computational methods in Markov chains (60J22) Applications of mathematical programming (90C90) Linear programming (90C05) Numerical analysis or methods applied to Markov chains (65C40) Continuous-time Markov processes on discrete state spaces (60J27)
Related Items (7)
Numerical Methods for Integral Equations of the Second Kind with NonSmooth Solutions of Bounded Variation ⋮ Abstraction-guided truncations for stationary distributions of Markov population models ⋮ First passage risk probability minimization for piecewise deterministic Markov decision processes ⋮ Singular perturbation for a two-class processor-sharing queue with impatience ⋮ Truncated and sparse power methods with partially updating for large and sparse higher-order PageRank problems ⋮ Transition graph decomposition for complex balanced reaction networks with non-mass-action kinetics ⋮ Approximations of Countably Infinite Linear Programs over Bounded Measure Spaces
Cites Work
- State space truncation with quantified errors for accurate solutions to discrete chemical master equation
- Analysis and computation of the stationary distribution in a special class of Markov chains of level-dependent \(\mathrm{M}/\mathrm{G}/1\)-type and its application to \(\mathrm{BMAP}/\mathrm{M}/\infty\) and \(\mathrm{BMAP}/\mathrm{M}/c+\mathrm{M}\) queues
- Continuous-time block-monotone Markov chains and their block-augmented truncations
- Product-form stationary distributions for deficiency zero chemical reaction networks
- Markovian dynamics on complex reaction networks
- Denumerable Markov processes and the associated contraction semigroups on l
- Multi-dimensional asymptotically quasi-Toeplitz Markov chains and their application in queueing theory
- Solving the chemical master equation for monomolecular reaction systems analytically
- Stochastic methods. A handbook for the natural and social sciences
- Augmented truncation approximations of discrete-time Markov chains
- On polyhedra of Perron-Frobenius eigenvectors
- Monotone infinite stochastic matrices and their augmented truncations
- Computing the stationary distribution for infinite Markov chains
- Continuous-time Markov chains. An applications-oriented approach
- A matrix continued fraction algorithm for the multiserver repeated order queue.
- A retrial BMAP/PH/N system
- Inference from iterative simulation using multiple sequences
- An individual-based approach to SIR epidemics in contact networks
- Exponential and uniform ergodicity of Markov processes
- Computation of the steady state distribution for multi-server retrial queues with phase type service process
- A matrix continued fraction approach to multiserver retrial queues
- Non-explosivity of stochastically modeled reaction networks that are complex balanced
- Results on stochastic reaction networks with non-mass action kinetics
- Error bounds for augmented truncation approximations of continuous-time Markov chains
- Approximating Markov chains and \(V\)-geometric ergodicity via weak perturbation theory
- Non-negative matrices and Markov chains.
- Dynamical properties of discrete reaction networks
- Perturbation analysis for continuous-time Markov chains
- Stochastic simulation: Algorithms and analysis
- A LINEAR PROGRAMMING APPROACH TO THE STEADY-STATE ANALYSIS OF REFLECTED BROWNIAN MOTION
- A Mathematical View of Interior-Point Methods in Convex Optimization
- Lectures on Modern Convex Optimization
- Efficient Steady State Analysis of Multimodal Markov Chains
- Tensor Approximation of Stationary Distributions of Chemical Reaction Networks
- Bounding Stationary Averages of Polynomial Diffusions via Semidefinite Programming
- Approximation and inference methods for stochastic biochemical kinetics—a tutorial review
- Importance Sampling for Stochastic Simulations
- Stochastic Neural Field Theory and the System-Size Expansion
- A Stochastic Model for Order Book Dynamics
- Infinite level-dependent QBD processes and matrix-analytic solutions for stochastic chemical kinetics
- Stability of Markovian processes II: continuous-time processes and sampled chains
- Stability of Markovian processes III: Foster–Lyapunov criteria for continuous-time processes
- Two Types ofRG-Factorizations of Quasi-birth-and-death Processes and Their Applications to Stochastic Integral Functionals
- Computational Identification of Irreducible State-Spaces for Stochastic Reaction Networks
- Equilibrium distribution of block-structured Markov chains with repeating rows
- SOME FURTHER PATHOLOGICAL EXAMPLES IN THE THEORY OF DENUMERABLE MARKOV PROCESSES
- A note on the classification of Q-processes when Q is not regular
- Modeling and Simulating Chemical Reactions
- Quantitative Sociodynamics
- Bounding Stationary Expectations of Markov Processes
- Augmented truncations of infinite stochastic matrices
- Bounds for the Positive Eigenvectors of Nonnegative Matrices and for their Approximations by Decomposition
- Approximation of the invariant probability measure of an infinite stochastic matrix
- Criteria for ergodicity, exponential ergodicity and strong ergodicity of Markov processes
- Birth-and-death processes on the integers with phases and general boundaries
- Probability with Martingales
- Stability of Markovian processes I: criteria for discrete-time Chains
- Open, Closed, and Mixed Networks of Queues with Different Classes of Customers
- Discrete time methods for simulating continuous time Markov chains
- Stochastic Complementation, Uncoupling Markov Chains, and the Theory of Nearly Reducible Systems
- Truncation approximations of invariant measures for Markov chains
- Markov Chains
- A logarithmic reduction algorithm for quasi-birth-death processes
- Computation of Steady-State Probabilities for Infinite-State Markov Chains with Repeating Rows
- Computing bounds on steady state availability of repairable computer systems
- Markov Chains
- Refinable bounds for large Markov chains
- Stochastic block–monotonicity in the approximation of the stationary distribution of infinite markov chains
- Perfect simulation using dominating processes on ordered spaces, with application to locally stable point processes
- Bound computation of dependability and performance measures
- Modelling with the Master Equation
- ERROR BOUNDS FOR LAST-COLUMN-BLOCK-AUGMENTED TRUNCATIONS OF BLOCK-STRUCTURED MARKOV CHAINS
- Some Network Conditions for Positive Recurrence of Stochastically Modeled Reaction Networks
- Applied Probability and Queues
- The censored Markov chain and the best augmentation
- Coupling for jump processes
- Calculating the equilibrium distribution in level dependent quasi-birth-and-death processes
- Fast simulation of rare events in queueing and reliability models
- Stationarity detection in the initial transient problem
- Exact sampling with coupled Markov chains and applications to statistical mechanics
- Bounding the equilibrium distribution of Markov population models
- Kronecker-Based Infinite Level-Dependent QBD Processes
- Computing Stationary Expectations in Level-Dependent QBD Processes
- Error bounds for augmented truncation approximations of Markov chains via the perturbation method
- Error Bounds for Augmented Truncations of Discrete-Time Block-Monotone Markov Chains under Geometric Drift Conditions
- Moment growth bounds on continuous time Markov processes on non-negative integer lattices
- Accurate Chemical Master Equation Solution Using Multi-Finite Buffers
- Stationarity Equations in Continuous Time Markov Chains
- COUNTABLE STATE MARKOV PROCESSES: NON-EXPLOSIVENESS AND MOMENT FUNCTION
- Bounding availability of repairable systems
- Networks of Waiting Lines
- Computing the Distributions of Economic Models via Simulation
- A New Method of Approximation in Markov-Chain Theory and its Application to Some Problems in the Theory of Random Time Substituion
- Truncation procedures for non-negative matrices
- Stochastic approach to chemical kinetics
- The calculation of limit probabilities for denumerable Markov processes from infinitesimal properties
- Computing Densities for Markov Chains via Simulation
- Approximating the stationary distribution of an infinite stochastic matrix
- Systèmes markoviens et stationnaires. Cas dénombrable
- On the Stochastic Matrices Associated with Certain Queuing Processes
- Unnamed Item
- Unnamed Item
- Unnamed Item
- Unnamed Item
- Unnamed Item
- Unnamed Item
- Unnamed Item
- Unnamed Item
- Unnamed Item
- Unnamed Item
- Unnamed Item
- Unnamed Item
- Unnamed Item
- Unnamed Item
This page was built for publication: Stationary Distributions of Continuous-Time Markov Chains: A Review of Theory and Truncation-Based Approximations
