Finite Continuous Time Markov Chains

DOI10.1137/1106012zbMath0112.09802OpenAlexW2091169817WikidataQ110853055 ScholiaQ110853055MaRDI QIDQ3849299

John G. Kemeny, J. Laurie Snell

Publication date: 1962

Published in: Theory of Probability & Its Applications (Search for Journal in Brave)

Full work available at URL: https://doi.org/10.1137/1106012

zbMATH Keywords

probability theory

Related Items

An iterative aggregation-disaggregation algorithm for solving linear equations, On the effects of noise and speed on computations, Expected hitting times for a random walk on a connected graph, Stationary distributions of perturbed Markov chains, Reversed absorbing Markov chain: A sample path approach, An exact solution to a class of queueing systems with multipurpose counters, Random walks on quasisymmetric functions, Cyclical dynamics of industrial production and employment: Markov chain-based estimates and tests, Detection of embeddings in binary Markov chains, On the complexity of partially observed Markov decision processes, A parametric uncertainty analysis method for queues with vacations, PH-graphs for analyzing shortest path problems with correlated traveling times, A Markovian procedure for assessing the state of a system, Moments of transition-additive random variables defined on finite, regenerative random processes, Saturated chains of subsets and a random walk, Computing the characteristics of a thinned Markov sequence, Bidirectional resource networks: A new flow model, Finiteness in the Beggar-My-Neighbor card game, Spread of (mis)information in social networks, Mutation, selection, and ancestry in branching models: a variational approach, Transition matrices for well-conditioned Markov chains, Comparative statics and stochastic dynamics of age-structured populations, Combining decomposition and reduction for state space analysis of a self-stabilizing system, Markovian interpretations of dual retrieval processes, Partially additive categories and flow-diagram semantics, Loss of selectively neutral alleles in small populations and regular mating systems, Random walks on trees, Modeling dependence dynamics through copulas with regime switching, Measurement and comparison of soil structures, Stochastically stable equilibria in \(n\)-person binary coordination games, Guidelines for choosing the transition matrix in Monte Carlo methods using Markov chains, On the inverse mean first passage matrix problem and the inverse \(M\)-matrix problem, Recurrence and transience for finite probabilistic tables, On the first passage time of a simple random walk on a tree, Parametric forms for generalized inverses of Markovian kernels and their applications, Characterisation of ergodic upper transition operators, The detection of particular genotypes in finite populations I. Natural selection effects, Lumpability in compartmental models, An introduction to multivariate Krawtchouk polynomials and their applications, Generalized inverses and their application to applied probability problems, Inequalities for functions of transition matrices, The expected hitting times for finite Markov chains, Seven things to remember about hidden Markov models: A tutorial on Markovian models for time series, The first-passage times of phase semi-Markov processes, Single-crossover dynamics: finite versus infinite populations, Variances of first passage times in a Markov chain with applications to mixing times, Transport in two-dimensional maps: Concepts, examples, amd a comparison of the theory of Rom-Kedar and Wiggins with the Markov model of MacKay, Meiss, Ott, and Percival, Calculation of characteristics of binormal sequential criteria and plans, Saddle-point calculation for constrained finite Markov chains, The Brownian approximation for rate-control throttles and the \(G/G/1/C\) queue, Efficient simulation of hierarchical stochastic activity network models, A necessary condition for weak lumpability in finite Markov processes, nano\(\kappa\): A calculus for the modeling and simulation of nano devices, Adaptive \(m\)-ary segmentation and canonical recoding algorithms for multiplication of large binary numbers, A Markov model for switching regressions, Systems factorial technology provides new insights on global-local information processing in autism spectrum disorders, A fluid analysis framework for a Markovian process algebra, On the theory of biased bisection operations and their inverses, Sum rules for hitting times of Markov chains, Measuring the confinement of probabilistic systems, Equivalence classes of functions of finite Markov chains, Information networks: A probabilistic model for hierarchical message transfer, On the calculation of the probability of current family-planning status in a cohort of women, Stack distance density and page fault probability in virtual memory systems, The distribution of sojourn times in finite absorbing Markov chains, Some stochastic properties of ``semi-magic and ``magic Markov chains, A mathematical model for paced serial addition, A lower bound for probabilistic algorithms for finite state machines, Quasi-stationary distributions as centrality measures for the giant strongly connected component of a reducible graph, Covering times of random walks on bounded degree trees and other graphs, Functions of random walks on hyperplane arrangements, A worm algorithm for the fully-packed loop model, Tempus fugit: How to plug it, A discrete time model for detection of randomly presented stimuli, A definition of measures over language space, Modelling spatio-temporal movement of tourists using finite Markov chains, Sojourn times for conditioned Markov chains in genetics, An alternative expression for the mean first passage matrix, An informational transition in conditioned Markov chains: applied to genetics and evolution, Asymptotic growth and stability in populations with time dependent vital rates, Variance estimation in the central limit theorem for Markov chains, Coupling and mixing times in a Markov chain, An \(M/M/2\) queueing system with heterogeneous servers and multiple vacations, A feedback algorithm for determining search parameters for Monte Carlo optimization, Probabilistic weak simulation is decidable in polynomial time, Carries, shuffling, and symmetric functions, A higher order Markov model for analyzing covariate dependence, Distance measures in terms of substitution processes, Analysis of a versatile multi-class delay-loss system with a superimposed Markovian arrival process, \(H_\infty \) estimation for discrete-time piecewise homogeneous Markov jump linear systems, Multiplicative processes reaching stationarity in finite time, The state reduction and related algorithms and their applications to the study of Markov chains, graph theory, and the optimal stopping problem, What can and what cannot be inferred from pairwise sequence comparisons?, Generalized eigenvectors and sets of nonnegative matrices, Estimating equilibrium probabilities for band diagonal Markov chains using aggregation and disaggregation techniques, Sensitivity of the stationary distribution vector for an ergodic Markov chain, Graph-theoretic approach to symbolic analysis of linear descriptor systems, Equivalence notions and model minimization in Markov decision processes, A lattice path lemma and an application in enzyme kinetics, Random walks on the triangular prism and other vertex-transitive graphs, The rapid mixing of random walks defined by an \(n\)-cube, On a two-level multigrid solution method for finite Markov chains, Spectral analysis of finite Markov chains with spherical symmetries, On the mean and variance of cover times for random walks on graphs, Limit distributions of sums of random variables defined on a finite homogeneous Markov chain. I, Multivariate approximation in total variation. I: Equilibrium distributions of Markov jump processes, Comparative branching-time semantics for Markov chains, A note on absorption probabilities in one-dimensional random walk via complex-valued martingales, A preorder relation for Markov reward processes, On regularly perturbed fundamental matrices, Stability of discrete linear inclusion, On weak lumpability of denumerable Markov chains, Geometric Approaches to the Estimation of the Spectral Gap of Reversible Markov Chains, An all-or-none model for noncorrection routines with elimination of incorrect responses, Muddling through: Noisy equilibrium selection, Directed graphs, 2D state models, and characteristic polynomials of irreducible matrix pairs, Stationary ensembles in threshold networks, Nonparametric tests and nested sequential sampling plans for change-point detection, Coupled Markov chain model: Characterization of membrane channel currents with multiple conductance sublevels as partially coupled elementary pores, Stability, performance and sensitivity analysis of I.I.D. jump linear systems, On weak lumpability of a finite Markov chain, Discrete time Markov chain model for age of information, A sufficient condition for a unique invariant distribution of a higher-order Markov chain, Flux through a Markov chain, A rapidly mixing stochastic system of finite interacting particles on the circle, The effect of serial correlation on the in-control average run length of cumulative score charts, Approximating labelled Markov processes, Robust power series algorithm for epistemic uncertainty propagation in Markov chain models, Reward algorithms for semi-Markov processes, Parameter estimation: Known vector signals in unknown Gaussian noise, Exact results for the probability and stochastic dynamics of fixation in the Wright-Fisher model, Perturbation bounds for Markov chains with general state space, Testing lumpability in Markov chains., Learning by matching patterns., Model reduction for a class of nonstationary Markov jump linear systems, Rank and statistical mapping of a strongly connected automaton, Algebraic Schwarz methods for the numerical solution of Markov chains, Aggregation of stochastic automata networks with replicas, Improved bounds for a condition number for Markov chains, The first Laurent series coefficients for singularly perturbed stochastic matrices, A unified approach to Markov decision problems and performance sensitivity analysis, Perturbation analysis of continuous‐time absorbing Markov chains, Potentials for denumerable Markov chains, The generating functions of hitting times for random walk on trees, Hitting times for random walks on vertex-transitive graphs, Algorithms for improving efficiency of discrete Markov chains, Occupancy time in sets of states for demographic models, Strong stability and perturbation bounds for discrete Markov chains, Bisimulation relations for weighted automata, An evolutionary analysis of the volunteer's dilemma, On reducing the statespace of hidden Markov models for the identity by descent process, Arbitrary elementary landscapes \& AR(1) processes, An asymptotic maximum principle for essentially linear evolution models, Continuous state models for finite state machines†, On the Markov property of a finite hidden Markov chain, The characteristic landscape equation for an AR(2) landscape, Adaptive policy for two finite Markov chains zero-sum stochastic game with unknown transition matrices and average payoffs, Weak bisimulation is sound and complete for pCTL\(^*\), New perturbation bounds for denumerable Markov chains, Bisimulation and cocongruence for probabilistic systems, Lot-sizing two-echelon assembly systems with random yields and rigid demand, Quantum dynamics of human decision-making, Mixing times with applications to perturbed Markov chains, Uniform random sampling of planar graphs in linear time, Process algebra for performance evaluation, One-dimensional kinetic diluted Ising model. Exact solution for correlation functions, Sensitivity of finite Markov chains under perturbation, Derivation of learning process statistics for a general Markov model, Interwoven redundant logic, Synthesis of probability transformers, Some convergence theorems for stochastic learning models with distance diminishing operators, Asymptotic reliability analysis for a class of complex systems, On the global limits and oscillations of a system of nonlinear differential equations describing a flow on a probabilistic network, Embedding fields: A theory of learning with physiological implications, Linear programming algorithms for semi-Markovian decision processes, Error prediction in Markov models of building/HVAC systems, A probabilistic approach to navigation in Hypertext, The application of some mathematical methods in medical diagnostics, Scalar timing and semi-Markov chains in free-operant avoidance, IMPRECISE MARKOV CHAINS AND THEIR LIMIT BEHAVIOR, Comparative analysis of automata with a common input alphabet, A generalization of the classical occupancy problem, Probabilistic game automata, Open migration systems with uniform inflow, First passage step count number versus first passage time in age- dependent dynamics, A simple stochastic gene substitution model, Stability of Product Form G-Networks, Matrix analysis of a Markov chain small-world model, Markov chain sensitivity measured by mean first passage times, The evolution of aggregated Markov chains, Stationary distributions and mean first passage times of perturbed Markov chains, Language-based performance prediction for distributed and mobile systems, Controlled Markov set-chains under average criteria, On the computational power of probabilistic and quantum branching program, Entropic measures, Markov information sources and complexity, Asymptotic variance rate of the output of a transfer line with no buffer storage and cycle-dependent failures, Formal requirements of Markov state models for paired associate learning, Investigating the underlying Markovian dynamics of ECG rhythms by information flow, Statistical inference on markov process of neuronal impulse sequences, Learning correlated equilibria in population games.