A rigorous framework for the mean field limit of multilayer neural networks

DOI10.4171/msl/42arXiv2001.11443MaRDI QIDQ6062704

Huy-Tuan Pham, Phan-Minh Nguyen

Publication date: 6 November 2023

Published in: Mathematical Statistics and Learning (Search for Journal in Brave)

Full work available at URL: https://arxiv.org/abs/2001.11443

zbMATH Keywords

optimizationglobal convergenceneural networkmean field limitdeep learning

Mathematics Subject Classification ID

Nonparametric regression and quantile regression (62G08) Artificial neural networks and deep learning (68T07) Nonconvex programming, global optimization (90C26) Interacting particle systems in time-dependent statistical mechanics (82C22)

Cites Work

• Unnamed Item
• Unnamed Item
• Machine learning from a continuous viewpoint. I
• Analysis of a two-layer neural network via displacement convexity
• Sparse optimization on measures with over-parameterized gradient descent
• Gradient descent optimizes over-parameterized deep ReLU networks
• A mean field view of the landscape of two-layer neural networks
• Particle dual averaging: optimization of mean field neural network with global convergence rate analysis*
• Mean Field Analysis of Deep Neural Networks
• Mean Field Analysis of Neural Networks: A Law of Large Numbers
• Wide neural networks of any depth evolve as linear models under gradient descent ^*
• Approximation by superpositions of a sigmoidal function

This page was built for publication: A rigorous framework for the mean field limit of multilayer neural networks