Nonlinear random matrix theory for deep learning

DOI10.1088/1742-5468/ab3bc3zbMath1459.60012OpenAlexW2996376886MaRDI QIDQ5854102

Jeffrey Pennington, Pratik Worah

Publication date: 16 March 2021

Published in: Journal of Statistical Mechanics: Theory and Experiment (Search for Journal in Brave)

Full work available at URL: https://doi.org/10.1088/1742-5468/ab3bc3

Mathematics Subject Classification ID

Random matrices (probabilistic aspects) (60B20) Neural networks for/in biological studies, artificial life and related topics (92B20) Random matrices (algebraic aspects) (15B52) Neural nets and related approaches to inference from stochastic processes (62M45)

Related Items (9)

Deep learning: a statistical viewpoint ⋮ Surprises in high-dimensional ridgeless least squares interpolation ⋮ On the universality of noiseless linear estimation with respect to the measurement matrix ⋮ Products of many large random matrices and gradients in deep neural networks ⋮ Large-dimensional random matrix theory and its applications in deep learning and wireless communications ⋮ A spin glass model for the loss surfaces of generative adversarial networks ⋮ Eigenvalue distribution of some nonlinear models of random matrices ⋮ A note on the Pennington-Worah distribution ⋮ Unnamed Item

Uses Software

• ImageNet
• AlexNet
• WaveNet

Cites Work

• Unnamed Item
• The spectrum of kernel random matrices
• THE SPECTRUM OF RANDOM INNER-PRODUCT KERNEL MATRICES
• DISTRIBUTION OF EIGENVALUES FOR SOME SETS OF RANDOM MATRICES

This page was built for publication: Nonlinear random matrix theory for deep learning