Infinite-width limit of deep linear neural networks

DOI10.1002/cpa.22200MaRDI QIDQ6587580

Lénaïc Chizat, Xavier Fernández-Real, Alessio Figalli, Maria Colombo

Publication date: 14 August 2024

Published in: Communications on Pure and Applied Mathematics (Search for Journal in Brave)

zbMATH Keywords

deep learningdeep networksinfinite-width

Mathematics Subject Classification ID

Artificial neural networks and deep learning (68T07) Random matrices (probabilistic aspects) (60B20)

Cites Work

• An iterative construction of solutions of the TAP equations for the Sherrington-Kirkpatrick model
• Representations for partially exchangeable arrays of random variables
• Approximation and estimation bounds for artificial neural networks
• Bayesian learning for neural networks
• Products of many large random matrices and gradients in deep neural networks
• High-Dimensional Probability
• A mean field view of the landscape of two-layer neural networks
• Learning deep linear neural networks: Riemannian gradient flows and convergence to global minimizers
• Mean Field Analysis of Neural Networks: A Law of Large Numbers
• A mathematical theory of semantic development in deep neural networks
• The Dynamics of Message Passing on Dense Graphs, with Applications to Compressed Sensing
• Disentangling feature and lazy training in deep neural networks
• The Continuous Formulation of Shallow Neural Networks as Wasserstein-Type Gradient Flows
• Deep Linear Networks for Matrix Completion—an Infinite Depth Limit
• Gradient descent on infinitely wide neural networks: global convergence and generalization
• Gradient flows on graphons: existence, convergence, continuity equations

This page was built for publication: Infinite-width limit of deep linear neural networks

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q6587580)