High‐dimensional limit theorems for SGD: Effective dynamics and critical scaling

DOI10.1002/cpa.22169arXiv2206.04030OpenAlexW4387343669MaRDI QIDQ6182180

Aukosh Jagannath, Reza Gheissari, Gérard Ben Arous

Publication date: 23 January 2024

Published in: Communications on Pure and Applied Mathematics (Search for Journal in Brave)

Full work available at URL: https://arxiv.org/abs/2206.04030

Mathematics Subject Classification ID

Probability theory and stochastic processes (60-XX) Statistical mechanics, structure of matter (82-XX)

Cites Work

• Unnamed Item
• Unnamed Item
• Unnamed Item
• Unnamed Item
• Unnamed Item
• Unnamed Item
• Unnamed Item
• Unnamed Item
• The eigenvalues and eigenvectors of finite, low rank perturbations of large random matrices
• Algorithmic thresholds for tensor PCA
• The largest eigenvalues of finite rank deformation of large Wigner matrices: Convergence and nonuniversality of the fluctuations
• Optimality and sub-optimality of PCA. I: Spiked random matrix models
• On the distribution of the largest eigenvalue in principal components analysis
• Stochastic algorithms
• Statistical limits of spiked tensor models
• Fundamental limits of detection in the spiked Wigner model
• Bridging the gap between constant step size stochastic gradient descent and Markov chains
• Bounding flows for spherical spin glass dynamics
• Mean field analysis of neural networks: a central limit theorem
• The largest eigenvalue of rank one deformation of large Wigner matrices
• Phase transition of the largest eigenvalue for nonnull complex sample covariance matrices
• Multidimensional diffusion processes.
• Statistical thresholds for tensor PCA
• Asymptotic behavior of stochastic approximation and large deviations
• Functional and random central limit theorems for the Robbins-Munro process
• Analysis of recursive stochastic algorithms
• High-Dimensional Statistics
• High-Dimensional Probability
• Stochastic Approximation and Large Deviations: Upper Bounds and <scp>w.p.1</scp> Convergence
• A mean field view of the landscape of two-layer neural networks
• The Landscape of the Spiked Tensor Model
• Phase retrieval via randomized Kaczmarz: theoretical guarantees
• Fast spectral algorithms from sum-of-squares proofs: tensor decomposition and planted sparse vectors
• A Stochastic Approximation Method
• Dynamics of stochastic gradient descent for two-layer neural networks in the teacher–student setup*

This page was built for publication: High‐dimensional limit theorems for SGD: Effective dynamics and critical scaling