Markov Chains

Introductory exposition (textbooks, tutorial papers, etc.) pertaining to probability theory (60-01) Markov chains (discrete-time Markov processes on discrete state spaces) (60J10)

Related Items

A graph-algorithmic approach for the study of metastability in Markov chains, Interacting generalized Friedman's urn systems, Probabilistic state estimation for labeled continuous time Markov models with applications to attack detection, Gradient flows in metric random walk spaces, Fast reactions with non-interacting species in stochastic reaction networks, Stochastic learning-based weak estimation of multinomial random variables and its applications to pattern recognition in non-stationary environments, Direct statistical inference for finite Markov jump processes via the matrix exponential, Combinatorics of a disordered two-species ASEP on a torus, Observer-based quantized control for discrete-time switched systems with infinitely distributed delay, Moment bounds and geometric ergodicity of diffusions with random switching and unbounded transition rates, Evolution of cooperation in a multidimensional phenotype space, Maximal Parrondo's paradox for classical and quantum Markov chains, Conditional multivariate distributions of phase-type for a finite mixture of Markov jump processes given observations of sample path, A hybrid stochastic model and its Bayesian identification for infectious disease screening in a university campus with application to massive COVID-19 screening at the university of Liège, On a Markovian game model for competitive insurance pricing, Markovian imprecise jump processes: extension to measurable variables, convergence theorems and algorithms, Balancing quarantine and self-distancing measures in adaptive epidemic networks, Multivariate reversed hazard rates and inactivity times of systems, Indicator variogram models: do we have much choice?, The emergence of ``super-groups in dynamical networks, Contemporaneous statistics for estimation in stochastic actor-oriented co-evolution models, Convergence time and phase transition in a non-monotonic family of probabilistic cellular automata, Imprecise continuous-time Markov chains, A bag-of-paths framework for network data analysis, The effect of clumped population structure on the variability of spreading dynamics, Computation of epidemic final size distributions, Forest gap dynamics and the Ising model, No arbitrage in continuous financial markets, Tunneling and metastability of continuous time Markov chains. II: The nonreversible case, The effect of residual \(\operatorname{Ca}^{2 +}\) on the stochastic gating of \(\operatorname{Ca}^{2 +} \)-regulated \(\operatorname{Ca}^{2 +}\) channel models, Mean field dynamics of stochastic cellular automata for random and small-world graphs, Privacy and safety analysis of timed stochastic discrete event systems using Markovian trajectory-observers, Commute times for a directed graph using an asymmetric Laplacian, Dynamics, stability and inheritance of somatic DNA methylation imprints, Approximation of exit times for one-dimensional linear diffusion processes, The common ancestor process revisited, Stochastic approach to molecular interactions and computational theory of metabolic and genetic regulations, Assessing and accounting for time heterogeneity in stochastic actor oriented models, Markov processes on the adeles and representations of Euler products, Stationary equilibria of mean field games with finite state and action space, Non-existence of stabilizing policies for the critical push-pull network and generalizations, Markov chain approximations for symmetric jump processes, Modelling of limit order books by general compound Hawkes processes with implementations, Inference for partially observed epidemic dynamics guided by Kalman filtering techniques, Flux large deviations of independent and reacting particle systems, with implications for macroscopic fluctuation theory, Escape rate of Markov chains on infinite graphs, Past states of continuous-time Markov models for ecological communities, Pattern formation in auxin flux, Long term behaviour of a reversible system of interacting random walks, Compartmental model diagrams as causal representations in relation to DAGs, Discrete kinetic models for molecular motors: asymptotic velocity and Gaussian fluctuations, Stochastic chemical reaction networks for robustly approximating arbitrary probability distributions, Tunneling of the Kawasaki dynamics at low temperatures in two dimensions, Approximating persistence in a general class of population processes, Quantitative analysis of probabilistic lossy channel systems, Greedy pathlengths and small world graphs, The survival of the weakest in networks, On dynamic compromise, Open quantum random walks and quantum Markov chains, On fast computation of directed graph Laplacian pseudo-inverse, Extinction times for a birth--death process with two phases, Stability of the nonlinear filter for slowly switching Markov chains, Fitting timeseries by continuous-time Markov chains: a quadratic programming approach, Cover time in edge-uniform stochastically-evolving graphs, Perturbation analysis of quantum reset models, Sparse randomized shortest paths routing with Tsallis divergence regularization, Stationary distributions via decomposition of stochastic reaction networks, The total variation flow in metric random walk spaces, Absorption in invariant domains for semigroups of quantum channels, Entropy-information inequalities under curvature-dimension conditions for continuous-time Markov chains, Sustainability of RNA-interference in rule based modelling, Determinantal structures in space-inhomogeneous dynamics on interlacing arrays, Rewriting theory for the life sciences: a unifying theory of CTMC semantics, Bounding inferences for large-scale continuous-time Markov chains: a new approach based on lumping and imprecise Markov chains, Discrete-time implementation of continuous-time filters with application to regime-switching dynamics estimation, Characterization of Brownian Gibbsian line ensembles, Stationary distributions and convergence for \(M/M/1\) queues in interactive random environment, Dynamics of a birth-death process based on combinatorial innovation, Notes on Markov embedding, Design of nucleic acid strands with long low-barrier folding pathways, The entropy method under curvature-dimension conditions in the spirit of Bakry-Émery in the discrete setting of Markov chains, Bidirectional sliding of two parallel microtubules generated by multiple identical motors, Nash equilibria in a class of Markov stopping games with total reward criterion, A Markovian approach to the Prandtl-Tomlinson frictional model, A three-player singled out game, Applications of the fundamental matrix to mean absorption and conditional mean absorption problems, Sampling strategies for uncertainty reduction in categorical random fields: formulation, mathematical analysis and application to multiple-point simulations, Dynamic stochastic block models: parameter estimation and detection of changes in community structure, Hitting probabilities and hitting times for stochastic fluid flows, Birth and death processes in interactive random environments, Feedback control of switched stochastic systems using randomly available active mode information, Mahler/zeta correspondence, Beatable imitation in symmetric games with perturbed payoffs, Linear-quadratic stochastic leader-follower differential games for Markov jump-diffusion models, Model checking single agent behaviours by fluid approximation, On a class of quantum channels, open random walks and recurrence, Study of a degenerate elliptic equation in an optimal consumption problem under partial information, Positive consensus of multi-agent systems with hierarchical control protocol, Adiabatic Lindbladian evolution with small dissipators, Real-time order tracking for supply systems with multiple transportation stages, Integrals for continuous-time Markov chains., On the price of risk of the underlying Markov chain in a regime-switching exponential Lévy model, An asymptotic study for path reversal., Metastable states, quasi-stationary distributions and soft measures, Operator theoretic framework for optimal placement of sensors and actuators for control of nonequilibrium dynamics, Simplification of reversible Markov chains by removal of states with low equilibrium occupancy, A theoretical investigation of the effect of proliferation and adhesion on monoclonal conversion in the colonic crypt, Computationally exact methods for stochastic periodic dynamics: spatiotemporal dispersal and temporally forced transmission, Diagnostic tools for evaluating and updating hidden Markov models, A solution to the reversible embedding problem for finite Markov chains, Pricing exotic derivatives exploiting structure, Synthesizing efficient systems in probabilistic environments, Universality behaviour for polarity formation in channel-type inclusion compounds, An analysis of the exponential decay principle in probabilistic trust models, Coalescent process with fluctuating population size and its effective size, Gradient and passive circuit structure in a class of non-linear dynamics on a graph, Shattering and non-uniqueness in fragmentation models -- an analytic approach, On Hille-type approximation of degenerate semigroups of operators, Trajectory optimization under kinematical constraints for moving target search, The infection time of graphs, Random talk: Random walk and synchronizability in a moving neighborhood network, Translated Poisson approximation for Markov chains, Computing Freidlin's cycles for the overdamped Langevin dynamics. Application to the Lennard-Jones-38 cluster, An ergodic theorem for filtering with applications to stability, Random walks and the effective resistance sum rules, Runs of Markov chains and streaks in baseball, Controlled risk processes in discrete time: lower and upper approximations to the optimal probability of ruin, Reversible MCMC on Markov equivalence classes of sparse directed acyclic graphs, On parameter estimation in population models, Dynamic regular registers in systems with churn, On parameter estimation in population models. II: Multi-dimensional processes and transient dynamics, Efficient methods for studying stochastic disease and population dynamics, The conditional ancestral selection graph with strong balancing selection, Stochastic model of microtubule dynamics, Minimal counterexamples for linear-time probabilistic verification, Random walks and diffusion on networks, The topological susceptibility from grand canonical simulations in the interacting instanton liquid model: strongly associating fluids and biased Monte Carlo, Convergence time analysis of quantized gossip consensus on digraphs, A customer based supplier selection process that combines quality function deployment, the analytic network process and a Markov chain, Stochastic observability in network state estimation and control, Some edge correction methods for marked spatio-temporal point process models, A random walk model for infection on graphs: spread of epidemics \& rumours with mobile agents, Markov property of continuous dislocation band propagation, Critical exponents for long-range \(\mathrm O(n)\) models below the upper critical dimension, Target containment control of multi-agent systems with random switching interconnection topologies, Tunneling and metastability of continuous time Markov chains, An experimental investigation of kernels on graphs for collaborative recommendation and semisupervised classification, Stochastic imitative game dynamics with committed agents, An evolutionary approach to learning in a changing environment., A multi-cluster time aggregation approach for Markov chains, Stochastic modelling of the eukaryotic heat shock response, The cost of probabilistic agreement in oblivious robot networks, Single-crossover dynamics: finite versus infinite populations, Memoryless search algorithms in a network with faulty advice, The evolution of bet-hedging adaptations to rare scenarios, Stochastic reaction networks with input processes: analysis and application to gene expression inference, Modeling biological systems in stochastic concurrent constraint programming, Cubical token systems, Upper escape rate for weighted graphs via metric graphs, Harnack inequality for subordinate random walks, The Eyring-Kramers law for Markovian jump processes with symmetries, Irreducible recurrence, ergodicity, and extremality of invariant measures for resolvents, Open quantum random walks: ergodicity, hitting times, gambler's ruin and potential theory, Estimating the basic reproductive number during the early stages of an emerging epidemic, The recombination equation for interval partitions, Graphical criteria for positive solutions to linear systems, Learning from dependent observations, A new bound for the spectral radius of nonnegative tensors, Strategy recovery for stochastic mean payoff games, Quantum walks: a comprehensive review, Probabilistic approximations of ODEs based bio-pathway dynamics, Random walks on discrete cylinders with large bases and random interlacements, A stochastic min-driven coalescence process and its hydrodynamical limit, Gallavotti-Cohen-type symmetry related to cycle decompositions for Markov chains and biochemical applications, Hybrid dynamics of stochastic programs, Applying Brownian motion to the study of birth-death chains, Metastability of reversible finite state Markov processes, Spectra of large random trees, Controlling number of particles in fragmentation equations, Mathematical modeling of monoclonal conversion in the colonic crypt, Model checking Markov population models by stochastic approximations, Invasion of infectious diseases in finite homogeneous populations, Maximum likelihood estimation for social network dynamics, On estimation for Brownian motion governed by telegraph process with multiple off states, Common ancestor type distribution: a Moran model and its deterministic limit, Calcium-dependent inactivation and the dynamics of calcium puffs and sparks, Existence condition of strong stationary times for continuous time Markov chains on discrete graphs, Incomplete penetrance: the role of stochasticity in developmental cell colonization, Dynamic contracting with persistent shocks, Genetic clustering of social networks using random walks, The complexity of relating quantum channels to master equations, Random walks on a finite graph with congestion points, Strictly substochastic semigroups with application to conservative and shattering solutions to fragmentation equations with mass loss, Polarity formation by a higher order interaction Markov-like chain, Measure and probability for concurrency theorists, Matrix Lie theory and measure chains, Google PageRank as mean playing time for pinball on the reverse web, Hybrid behaviour of Markov population models, Smoothed model checking for uncertain continuous-time Markov chains, A characterization for mixtures of semi-Markov processes., Understanding measure-driven algorithms solving irreversibly ill-conditioned problems, Dynamics of finite inhomogeneous particle systems with exclusion interaction, Bounded model checking for interval probabilistic timed graph transformation systems against properties of probabilistic metric temporal graph logic, Optimal deterministic controller synthesis from steady-state distributions, Inverse Problems for Discrete Heat Equations and Random Walks for a Class of Graphs, Better bounds on the adaptivity gap of influence maximization under full-adoption feedback, Applying monoid duality to a double contact process, Local limit laws for symbol statistics in bicomponent rational models, Probabilistic verification of diagnosability for a certain class of timed stochastic systems, A versatile stochastic dissemination model, Diagnosability of fault patterns with labeled stochastic Petri nets, Spine for interacting populations and sampling, Unnamed Item, A predictive Bayesian approach to EWMA and CUSUM charts for time-between-events monitoring, Locality of the interaction affects dynamics in probabilistic cellular automata, Open Quantum Random Walks and Quantum Markov chains on Trees I: Phase transitions, Random population dynamics under catastrophic events, TAIL BEHAVIOR OF STOPPED LÉVY PROCESSES WITH MARKOV MODULATION, Unnamed Item, Unnamed Item, Branching Structures Within Random Walks and Their Applications, Checking Individual Agent Behaviours in Markov Population Models by Fluid Approximation, An Algebraic Approach to Product-form Stationary Distributions for Some Reaction Networks, Conditional Monte Carlo for Reaction Networks, Similar States in Continuous-Time Markov Chains, Modeling of chemical reaction systems with detailed balance using gradient structures, On equal-input and monotone Markov matrices, Fuzzy Markov chains modeling of aggregation processes, The Effect of Time-Varying Jamming Interference on Networked Stabilization, Hawkes processes in insurance: risk model, application to empirical data and optimal investment, Dividend optimisation: a behaviouristic approach, Design of biased random walks on a graph with application to collaborative recommendation, Compatibility, embedding and regularization of non-local random walks on graphs, Markov chains with doubly stochastic transition matrices and application to a sequence of non-selective quantum measurements, Estimating local outbreak risks and the effects of non-pharmaceutical interventions in age-structured populations: SARS-CoV-2 as a case study, Synthesizing and Tuning Chemical Reaction Networks with Specified Behaviours, Stoichiometric and dynamical autocatalysis for diluted chemical reaction networks, On Binomial Observations of Continuous-Time Markovian Population Models, Metzler/zeta correspondence, Stochastic reaction networks within interacting compartments, Network dynamics with a nested node set: Sociability in seven villages in Senegal, Attraction domain analysis for steady states of Markovian open quantum systems, Hitting time expressions for quantum channels: beyond the irreducible case and applications to unitary walks, Recurrence of a class of quantum Markov chains on trees, A bipartite version of the Erdős–McKay conjecture, Mathematical analysis of the limiting behaviors of a chromatin modification circuit, Speed function for biased random walks with traps, Computational Translation Framework Identifies Biochemical Reaction Networks with Special Topologies and Their Long-Term Dynamics, Probabilistic Model Checking for Continuous-Time Markov Chains via Sequential Bayesian Inference, G-Circulant Quantum Markov Semigroups, Exploring the Limits of Controlled Markovian Quantum Dynamics with Thermal Resources, The narrow recurrence of Markov chains, Large deviations for the empirical measure and empirical flow of Markov renewal processes with a countable state space, A stochastic model of group wealth responses to insurance mechanisms in low-income communities, Concatenation of Nonhonest Feller Processes, Exit Laws, and Limit Theorems on Graphs, Quantum Markov chains on the line: matrix orthogonal polynomials, spectral measures and their statistics, On a \(\psi\)-mixing property for entangled Markov chains, Beyond Wilson-Cowan dynamics: oscillations and chaos without inhibition, Affinely spanned quasi-stochastic algebraic monoids, Multimodal dynamics of nonhomogeneous absorbing Markov chains evolving at stochastic transition rates, Stochastic monotonicity and comparability of Markov chains with block-monotone transition matrices and their applications to queueing systems, Characterization of \(H\)-Brownian Gibbsian line ensembles, Exponential decay of matrix Φ-entropies on Markov semigroups with applications to dynamical evolutions of quantum ensembles, Moran process version of the tug-of-war model: behavior revealed by mathematical analysis and simulation studies, Li-Yau inequalities for general non-local diffusion equations via reduction to the heat kernel, Tightness of discrete Gibbsian line ensembles, A duscrete-time model of high-frequency stock returns, PageRank, a Look at Small Changes in a Line of Nodes and the Complete Graph, PageRank, Connecting a Line of Nodes with a Complete Graph, Recurrence and Transience of Continuous-Time Open Quantum Walks, Unnamed Item, GENERAL SEMI-MARKOV MODEL FOR LIMIT ORDER BOOKS, Stochastically modeled weakly reversible reaction networks with a single linkage class, Model-based Clustering of Sequential Data with an Application to Contraceptive Use Dynamics, Local approximation of Markov chains in time and space, Stationary Distributions of Continuous-Time Markov Chains: A Review of Theory and Truncation-Based Approximations, Estimating dementia-free life expectancy for Parkinson's patients using Bayesian inference and microsimulation, A 3D Fractional Step Computational Modeling of Nerve Impulse Transmission Through an Axonal Membrane: Incorporating Calcium Buffer and Extrusion, Long-Time Behaviour in a Model of Microtubule Growth, On the Approximation of Stochastic Concurrent Constraint Programming by Master Equation, Limit Theorems for a Generalized Feller Game, Stochastic Programs and Hybrid Automata for (Biological) Modeling, Stochastic Modeling of Density-Dependent Diploid Populations and the Extinction Vortex, Convergence and Monotonicity for a Model of Spontaneous Infection and Transmission, Linear Structures for Concurrency in Probabilistic Programming Languages, Data-based modeling of the failure rate of repairable equipment, Matching random colored points with rectangles, Approximate Verification of the Symbolic Dynamics of Markov Chains, An oscillation criterion for linear difference equations with oscillating coefficients, A Lagrangian Approach to Weakly Coupled Hamilton--Jacobi Systems, Cycle characterization of the Aubry set for weakly coupled Hamilton–Jacobi systems, Analysis of multivariable controllers using degree of freedom data, An algorithmic approach to South Indian classical music, Haplotypes: the joint distribution of alleles at linked loci, Open quantum random walks, quantum Markov chains and recurrence, Mean-field Analysis of Multipopulation Dynamics with Label Switching, Reliability, signature, and relative quality functions of systems under time‐homogeneous load‐sharing models, Chemomechanical coupling of molecular motors: thermodynamics, network representations, and balance conditions, Atomistic origins of continuum dislocation dynamics, Dynamical Systems and Stochastic Programming: To Ordinary Differential Equations and Back, Stationary Distributions and Condensation in Autocatalytic Reaction Networks, On Linear Hypocoercive BGK Models, Path-integral analysis of fluctuation theorems for general Langevin processes, Asymptotic hitting probabilities for the Bolthausen-Sznitman coalescent, A Metric on Directed Graphs and Markov Chains Based on Hitting Probabilities, Bayesian inversion for electromyography using low-rank tensor formats, Active learning of continuous-time Bayesian networks through interventions*, Stochastic Concurrent Constraint Programming and Differential Equations, Branching processes with interactions: subcritical cooperative regime, Accuracy of Multiscale Reduction for Stochastic Reaction Systems, Rule Algebras for Adhesive Categories, Multipartite quantum systems: an approach based on Markov matrices and the Gini index