Simple and Optimal Methods for Stochastic Variational Inequalities, II: Markovian Noise and Policy Evaluation in Reinforcement Learning

DOI10.1137/20M1381691zbMath1493.90205arXiv2011.08434OpenAlexW3106200437MaRDI QIDQ5081106

Tianjiao Li, Georgios Kotsalis, Guanghui Lan

Publication date: 1 June 2022

Published in: SIAM Journal on Optimization (Search for Journal in Brave)

Full work available at URL: https://arxiv.org/abs/2011.08434

zbMATH Keywords

variational inequalityaccelerationreinforcement learningtemporal difference learningoperator extrapolationstochastic policy evaluation

Mathematics Subject Classification ID

Analysis of algorithms and problem complexity (68Q25) Stochastic programming (90C15) Complementarity and equilibrium problems and variational inequalities (finite dimensions) (aspects of mathematical programming) (90C33) Stochastic approximation (62L20)

Related Items (3)

Variable sample-size operator extrapolation algorithm for stochastic mixed variational inequalities ⋮ Smooth monotone stochastic variational inequalities and saddle point problems: a survey ⋮ \(\mathcal{H}_\infty\) tracking learning control for discrete-time Markov jump systems: a parallel off-policy reinforcement learning

Cites Work

• Unnamed Item
• Unnamed Item
• Unnamed Item
• Unnamed Item
• Unnamed Item
• Unnamed Item
• Stochastic optimal control. The discrete time case
• Information-based complexity of linear operator equations
• The minimax learning rates of normal and Ising undirected graphical models
• First-order and stochastic optimization methods for machine learning
• Mixing time estimation in reversible Markov chains from a single sample path
• Deterministic and stochastic primal-dual subgradient algorithms for uniformly convex minimization
• Stable Optimal Control and Semicontractive Dynamic Programming
• Markov Chains and Stochastic Stability
• An analysis of temporal-difference learning with function approximation
• Introduction to Stochastic Search and Optimization
• OnActor-Critic Algorithms
• 10.1162/1532443041827907
• Convergence Rates for Markov Chains
• Optimal Stochastic Approximation Algorithms for Strongly Convex Stochastic Composite Optimization I: A Generic Algorithmic Framework
• Ergodic Mirror Descent
• Statistical Inference via Convex Optimization

This page was built for publication: Simple and Optimal Methods for Stochastic Variational Inequalities, II: Markovian Noise and Policy Evaluation in Reinforcement Learning