Convergence rate of linear two-time-scale stochastic approximation.

DOI10.1214/105051604000000116zbMath1094.62103arXivmath/0405287OpenAlexW1985291828MaRDI QIDQ1879892

John N. Tsitsiklis, Vijay R. Konda

Publication date: 15 September 2004

Published in: The Annals of Applied Probability (Search for Journal in Brave)

Full work available at URL: https://arxiv.org/abs/math/0405287

Mathematics Subject Classification ID

Central limit and other weak theorems (60F05) Stochastic approximation (62L20)

Related Items (29)

Recursive regression estimation based on the two-time-scale stochastic approximation method and Bernstein polynomials ⋮ A Two-Timescale Stochastic Algorithm Framework for Bilevel Optimization: Complexity Analysis and Application to Actor-Critic ⋮ Sequential online subsampling for thinning experimental designs ⋮ Online calibrated forecasts: memory efficiency versus universality for learning in games ⋮ Convergence rate and averaging of nonlinear two-time-scale stochastic approximation algo\-rithms ⋮ Change-point monitoring for online stochastic approximations ⋮ Risk-Sensitive Reinforcement Learning via Policy Gradient Search ⋮ Variance-constrained actor-critic algorithms for discounted and average reward MDPs ⋮ DIMIX: Diminishing Mixing for Sloppy Agents ⋮ Geometrical Insights for Implicit Generative Modeling ⋮ Two-time-scale nonparametric recursive regression estimator for independent functional data ⋮ Asymptotic behavior of multiscale stochastic partial differential equations with Hölder coefficients ⋮ A Two-Time-Scale Stochastic Optimization Framework with Applications in Control and Reinforcement Learning ⋮ Towards multi‐agent reinforcement learning‐driven over‐the‐counter market simulations ⋮ Two-timescale stochastic gradient descent in continuous time with applications to joint online parameter estimation and optimal sensor placement ⋮ Weak convergence of dynamical systems in two timescales ⋮ Gradient-Based Adaptive Stochastic Search for Simulation Optimization Over Continuous Space ⋮ Non asymptotic controls on a recursive superquantile approximation ⋮ Generative adversarial networks are special cases of artificial curiosity (1990) and also closely related to predictability minimization (1991) ⋮ Stochastic compositional gradient descent: algorithms for minimizing compositions of expected-value functions ⋮ Averaging principle and normal deviations for multiscale stochastic systems ⋮ Stochastic approximation algorithms for superquantiles estimation ⋮ Empirical Dynamic Programming ⋮ GADE: a generative adversarial approach to density estimation and its applications ⋮ Computing VaR and CVaR using stochastic approximation and adaptive unconstrained importance sampling ⋮ Networks of reinforced stochastic processes: asymptotics for the empirical means ⋮ Fundamental design principles for reinforcement learning algorithms ⋮ Finite-Time Analysis and Restarting Scheme for Linear Two-Time-Scale Stochastic Approximation ⋮ Actor-Critic Algorithms with Online Feature Adaptation

Cites Work

• New method of stochastic approximation type
• Convergence and convergence rate of iterative stochastic algorithms. I: General case
• Stochastic approximation methods for constrained and unconstrained systems
• Stochastic approximation with two time scales
• Stochastic Approximation with Averaging of the Iterates: Optimal Asymptotic Rate of Convergence for General Processes
• Applications of Singular Perturbation Techniques to Control Problems
• Acceleration of Stochastic Approximation by Averaging
• OnActor-Critic Algorithms
• Actor-Critic--Type Learning Algorithms for Markov Decision Processes
• Unnamed Item
• Unnamed Item
• Unnamed Item
• Unnamed Item

This page was built for publication: Convergence rate of linear two-time-scale stochastic approximation.