High-dimensional dynamics of generalization error in neural networks
From MaRDI portal
Publication:2057778
DOI10.1016/j.neunet.2020.08.022zbMath1475.68258arXiv1710.03667OpenAlexW3083720136WikidataQ100457880 ScholiaQ100457880MaRDI QIDQ2057778
Madhu S. Advani, Andrew M. Saxe, Haim Sompolinsky
Publication date: 7 December 2021
Published in: Neural Networks (Search for Journal in Brave)
Full work available at URL: https://arxiv.org/abs/1710.03667
Random matrices (probabilistic aspects) (60B20) Learning and adaptive systems in artificial intelligence (68T05)
Related Items (17)
Surprises in high-dimensional ridgeless least squares interpolation ⋮ Align, then memorise: the dynamics of learning with feedback alignment* ⋮ Align, then memorise: the dynamics of learning with feedback alignment* ⋮ Generalization error rates in kernel regression: the crossover from the noiseless to noisy regime* ⋮ An analytical theory of curriculum learning in teacher–student networks* ⋮ The inverse variance–flatness relation in stochastic gradient descent is critical for finding flat minima ⋮ Large-dimensional random matrix theory and its applications in deep learning and wireless communications ⋮ Free dynamics of feature learning processes ⋮ An instance-dependent simulation framework for learning with label noise ⋮ Landscape and training regimes in deep learning ⋮ A statistician teaches deep learning ⋮ Unnamed Item ⋮ Unnamed Item ⋮ A Unifying Tutorial on Approximate Message Passing ⋮ A phase transition for finding needles in nonlinear haystacks with LASSO artificial neural networks ⋮ A random matrix analysis of random Fourier features: beyond the Gaussian kernel, a precise phase transition, and the corresponding double descent* ⋮ Prediction errors for penalized regressions based on generalized approximate message passing
Uses Software
Cites Work
- Unnamed Item
- Unnamed Item
- The singular values and vectors of low rank perturbations of large rectangular random matrices
- The eigenvalues and eigenvectors of finite, low rank perturbations of large random matrices
- Spectral analysis of large dimensional random matrices
- Dynamics of learning near singularities in radial basis function networks
- Universality of covariance matrices
- On early stopping in gradient descent learning
- Statistical Mechanics of Learning
- Statistical mechanics of complex neural systems and high dimensional data
- Generalization in a linear perceptron in the presence of noise
- Universal approximation bounds for superpositions of a sigmoidal function
- Learning and generalization in a linear perceptron stochastically trained with noisy data
- 10.1162/153244303321897690
- A mean field view of the landscape of two-layer neural networks
- Benign overfitting in linear regression
- Reconciling modern machine-learning practice and the classical bias–variance trade-off
- Dynamics of Learning Near Singularities in Layered Networks
This page was built for publication: High-dimensional dynamics of generalization error in neural networks
