An Optimal Scheduled Learning Rate for a Randomized Kaczmarz Algorithm

DOI10.1137/22M148803XMaRDI QIDQ5885816

Oscar Mickelin, Nicholas F. Marshall

Publication date: 30 March 2023

Published in: SIAM Journal on Matrix Analysis and Applications (Search for Journal in Brave)

Full work available at URL: https://arxiv.org/abs/2202.12224

zbMATH Keywords

stochastic gradient descentlearning raterandomized Kaczmarz

Mathematics Subject Classification ID

Numerical solutions to overdetermined systems, pseudoinverses (65F20) Random operators and equations (aspects of stochastic analysis) (60H25) Iterative numerical methods for linear systems (65F10)

Related Items (1)

Approximate Solutions of Linear Systems at a Universal Rate

Uses Software

• Regularization tools
• DLMF
• TensorFlow
• Adam
• ADADELTA

Cites Work

• Unnamed Item
• Unnamed Item
• Single projection Kaczmarz extended algorithms
• Randomized block Kaczmarz method with projection for solving least squares
• Acceleration of randomized Kaczmarz method via the Johnson-Lindenstrauss lemma
• On the acceleration of Kaczmarz's method for inconsistent linear systems
• Randomized Kaczmarz solver for noisy linear systems
• A randomized Kaczmarz algorithm with exponential convergence
• Strong underrelaxation in Kaczmarz's method for inconsistent systems
• Coordinate descent algorithms
• Paved with good intentions: analysis of a randomized block Kaczmarz method
• Regularization tools version \(4.0\) for matlab \(7.3\)
• Randomized Kaczmarz with averaging
• Fundamentals of Computerized Tomography
• Randomized Extended Kaczmarz for Solving Least Squares
• An accelerated randomized Kaczmarz algorithm
• Convergence Properties of the Randomized Extended Gauss--Seidel and Kaczmarz Methods
• Row-Action Methods for Huge and Sparse Systems and Their Applications
• High-Dimensional Probability
• Sampled Tikhonov regularization for large linear inverse problems
• Randomized Kaczmarz Converges Along Small Singular Vectors
• Sampled limited memory methods for massive linear inverse problems
• Faster Randomized Block Kaczmarz Algorithms
• Phase retrieval via randomized Kaczmarz: theoretical guarantees
• Two Algorithms Related to the Method of Steepest Descent
• A Stochastic Approximation Method
• Quantile-based Random Kaczmarz for corrupted linear systems of equations
• Stochastic gradient descent, weighted sampling, and the randomized Kaczmarz algorithm

This page was built for publication: An Optimal Scheduled Learning Rate for a Randomized Kaczmarz Algorithm