Analysis of a two-layer neural network via displacement convexity

DOI10.1214/20-AOS1945zbMath1464.62401arXiv1901.01375OpenAlexW3111778284MaRDI QIDQ1996787

Adel Javanmard, Marco Mondelli, Andrea Montanari

Publication date: 26 February 2021

Published in: The Annals of Statistics (Search for Journal in Brave)

Full work available at URL: https://arxiv.org/abs/1901.01375

zbMATH Keywords

neural networksconvergence ratestochastic gradient descentWasserstein gradient flowdisplacement convexityfunction regression

Mathematics Subject Classification ID

Estimation in multivariate analysis (62H12) Point estimation (62F10) General nonlinear regression (62J02) Neural nets and related approaches to inference from stochastic processes (62M45)

Related Items

The Continuous Formulation of Shallow Neural Networks as Wasserstein-Type Gradient Flows, Birth–death dynamics for sampling: global convergence, approximations and their asymptotics, A rigorous framework for the mean field limit of multilayer neural networks, A blob method for inhomogeneous diffusion with applications to multi-agent control and sampling, Convergence rates for shallow neural networks learned by gradient descent, Stochastic gradient descent with noise of machine learning type. II: Continuous time analysis, Unnamed Item, A selective overview of deep learning, Mean-field Langevin dynamics and energy landscape of neural networks

Uses Software

• scar

Cites Work

• Greedy function approximation: A gradient boosting machine.
• Contractions in the 2-Wasserstein length space and thermalization of granular media
• Interacting diffusions approximating the porous medium equation and propagation of chaos
• Vlasov equations
• Stochastic differential equations with reflecting boundary condition in convex regions
• A convexity principle for interacting gases
• Kinetic equilibration rates for granular media and related equations: entropy dissipation and mass transportation estimates
• The landscape of empirical risk for nonconvex losses
• Mean field analysis of neural networks: a central limit theorem
• Optimal transport for applied mathematicians. Calculus of variations, PDEs, and modeling
• Stochastic differential equations with reflecting boundary conditions
• Superresolution via Sparsity Constraints
• Universal approximation bounds for superpositions of a sigmoidal function
• The sample complexity of pattern classification with neural networks: the size of the weights is more important than the size of the network
• Boosting With theL₂Loss
• Theoretical Insights Into the Optimization Landscape of Over-Parameterized Shallow Neural Networks
• Simulation of the Solution of a Viscous Porous Medium Equation by a Particle Method
• Neural Network Learning
• A mean field view of the landscape of two-layer neural networks
• Breaking the Curse of Dimensionality with Convex Neural Networks
• Generalized Additive and Index Models with Shape Constraints
• Towards a Mathematical Theory of Super‐resolution
• Optimal Transport
• Approximation by superpositions of a sigmoidal function
• Entropy dissipation methods for degenerate parabolic problems and generalized Sobolev inequalities
• Unnamed Item
• Unnamed Item
• Unnamed Item
• Unnamed Item
• Unnamed Item
• Unnamed Item
• Unnamed Item
• Unnamed Item