SVRG meets AdaGrad: painless variance reduction

DOI10.1007/s10994-022-06265-xarXiv2102.09645OpenAlexW3131097734MaRDI QIDQ6097116

Mark Schmidt, Sharan Vaswani, Reza Babanezhad, Benjamin Dubois-Taine, Simon Lacoste-Julien

Publication date: 12 June 2023

Published in: Machine Learning (Search for Journal in Brave)

Full work available at URL: https://arxiv.org/abs/2102.09645

zbMATH Keywords

stochastic optimizationconvex optimizationvariance reductionadaptive gradient methods

Mathematics Subject Classification ID

Learning and adaptive systems in artificial intelligence (68T05)

Cites Work

• Unnamed Item
• Unnamed Item
• Minimizing finite sums with the stochastic average gradient
• Introductory lectures on convex optimization. A basic course.
• Just interpolate: kernel ``ridgeless regression can generalize
• Minimization of functions having Lipschitz continuous first partial derivatives
• Theory of Classification: a Survey of Some Recent Advances
• Two-Point Step Size Gradient Methods
• Katyusha: the first direct acceleration of stochastic gradient methods
• Stochastic Dual Coordinate Ascent Methods for Regularized Loss Minimization
• Exact and inexact subsampled Newton methods for optimization