On the regularization effect of stochastic gradient descent applied to least-squares

DOI10.1553/etna_vol54s610zbMath1475.65022arXiv2007.13288OpenAlexW3202832451MaRDI QIDQ2055514

Stefan Steinerberger

Publication date: 1 December 2021

Published in: ETNA. Electronic Transactions on Numerical Analysis (Search for Journal in Brave)

Full work available at URL: https://arxiv.org/abs/2007.13288

zbMATH Keywords

regularizationleast-squaresKaczmarz methodstochastic gradient descent

Mathematics Subject Classification ID

Numerical solutions to overdetermined systems, pseudoinverses (65F20) Numerical mathematical programming methods (65K05) Quadratic programming (90C20) Methods of reduced gradient type (90C52)

Related Items (1)

Approximate Solutions of Linear Systems at a Universal Rate

Cites Work

• Unnamed Item
• Unnamed Item
• Unnamed Item
• Two-subspace projection method for coherent overdetermined systems
• Randomized block Kaczmarz method with projection for solving least squares
• Minimizing finite sums with the stochastic average gradient
• Acceleration of randomized Kaczmarz method via the Johnson-Lindenstrauss lemma
• Randomized Kaczmarz solver for noisy linear systems
• A randomized Kaczmarz algorithm with exponential convergence
• The rate of convergence for the method of alternating projections. II
• On convergence rate of the randomized Kaczmarz method
• A new greedy Kaczmarz algorithm for the solution of very large linear systems
• A derandomization approach to recovering bandlimited signals across a wide range of random sampling rates
• Adaptive estimation of a quadratic functional by model selection.
• Convergence rates for Kaczmarz-type algorithms
• Paved with good intentions: analysis of a randomized block Kaczmarz method
• On the rate of convergence of the alternating projection method in finite dimensional spaces
• Randomized Kaczmarz with averaging
• Randomized Extended Kaczmarz for Solving Least Squares
• Randomized Methods for Linear Constraints: Convergence Rates and Conditioning
• An accelerated randomized Kaczmarz algorithm
• Convergence Properties of the Randomized Extended Gauss--Seidel and Kaczmarz Methods
• Randomized Iterative Methods for Linear Systems
• Preasymptotic convergence of randomized Kaczmarz method
• On the regularizing property of stochastic gradient descent
• A weighted randomized Kaczmarz method for solving linear systems
• Semi-convergence properties of Kaczmarz’s method
• Randomized Kaczmarz Converges Along Small Singular Vectors
• On Adaptive Sketch-and-Project for Solving Linear Systems
• A Markov Chain Theory Approach to Characterizing the Minimax Optimality of Stochastic Gradient Descent (for Least Squares)
• Phase retrieval via randomized Kaczmarz: theoretical guarantees

This page was built for publication: On the regularization effect of stochastic gradient descent applied to least-squares