Regime Switching Stochastic Approximation Algorithms with Application to Adaptive Discrete Stochastic Optimization

DOI10.1137/S1052623403423709zbMath1112.62330OpenAlexW2061412229MaRDI QIDQ4651989

Vikram Krishnamurthy, C. Ion, G. George Yin

Publication date: 23 February 2005

Published in: SIAM Journal on Optimization (Search for Journal in Brave)

Full work available at URL: https://doi.org/10.1137/s1052623403423709

zbMATH Keywords

trackingstochastic approximationswitching diffusionregime switching modeltime-varying parameterMarkovian parameterregime switching ODEs

Mathematics Subject Classification ID

Applications of Markov chains and discrete-time Markov processes on general state spaces (social mobility, learning theory, industrial processes, etc.) (60J20) Stochastic approximation (62L20)

Related Items (36)

Stability of regime-switching diffusions ⋮ Information diffusion in social sensing ⋮ Existence, uniqueness and exponential ergodicity under Lyapunov conditions for McKean-Vlasov SDEs with Markovian switching ⋮ Asymptotically optimal dividend policy for regime-switching compound Poisson models ⋮ Change-point monitoring for online stochastic approximations ⋮ Sequences of random matrices modulated by a discrete-time Markov chain* ⋮ Random periodic solutions for a class of hybrid stochastic differential equations ⋮ On the monotonicity property of the generalized eigenvalue for weakly-coupled cooperative elliptic systems ⋮ Unnamed Item ⋮ System identification: regime switching, unmodeled dynamics, and binary sensors ⋮ Asymptotic properties of hybrid random processes modulated by Markov chains ⋮ Controllability and adaptation of linear time-invariant systems under irregular and Markovian sampling ⋮ Iterative weak approximation and hard bounds for switching diffusion ⋮ Optimal control of the risk process in a regime-switching environment ⋮ A stochastic maximum principle for switching diffusions using conditional mean-fields with applications to control problems ⋮ A numerical method for annuity-purchasing decision making to minimize the probability of financial ruin for regime-switching wealth models ⋮ Regularity and recurrence of switching diffusions ⋮ Asymptotic optimality for consensus-type stochastic approximation algorithms using iterate averaging ⋮ Adaptive stepsize selection for tracking in a regime-switching environment ⋮ Networks of biosensors: decentralized activation and social learning ⋮ Invariant density, Lyapunov exponent, and almost sure stability of Markovian-regime-switching linear systems ⋮ Commuting birth-and-death processes ⋮ Asymptotic properties of consensus-type algorithms for networked systems with regime-switching topologies ⋮ Balanced realizations of regime-switching linear systems ⋮ On laws of large numbers for systems with mean-field interactions and Markovian switching ⋮ Asymptotic properties of Markov-modulated random sequences with fast and slow timescales ⋮ Weak convergence of Markov-modulated random sequences ⋮ How does a stochastic optimization/approximation algorithm adapt to a randomly evolving optimum/root with jump Markov sample paths ⋮ Stability of random-switching systems of differential equations ⋮ Tracking and identification of regime-switching systems using binary sensors ⋮ Switching diffusions with mean-field interactions: limit results, maximum principle, and non-Markov systems ⋮ Non-asymptotic error bounds for constant stepsize stochastic approximation for tracking mobile agents ⋮ Persistent tracking and identification of regime-switching systems with structural uncertainties: unmodeled dynamics, observation bias, and nonlinear model mismatch ⋮ Stability and stochastic stabilization of numerical solutions of regime-switching jump diffusion systems ⋮ Ergodic risk-sensitive control for regime-switching diffusions ⋮ Sign-error adaptive filtering algorithms involving Markovian parameters

This page was built for publication: Regime Switching Stochastic Approximation Algorithms with Application to Adaptive Discrete Stochastic Optimization