Mean Field Analysis of Deep Neural Networks

DOI10.1287/moor.2020.1118zbMath1493.68333arXiv1903.04440OpenAlexW3155817928MaRDI QIDQ5076694

Justin A. Sirignano, Konstantinos V. Spiliopoulos

Publication date: 17 May 2022

Published in: Mathematics of Operations Research (Search for Journal in Brave)

Full work available at URL: https://arxiv.org/abs/1903.04440

zbMATH Keywords

weak convergencemean-field limitdeep learningdeep neural network

Mathematics Subject Classification ID

Central limit and other weak theorems (60F05) Artificial neural networks and deep learning (68T07) Neural nets applied to problems in time-dependent statistical mechanics (82C32)

Related Items (9)

The Continuous Formulation of Shallow Neural Networks as Wasserstein-Type Gradient Flows ⋮ Revisiting Landscape Analysis in Deep Neural Networks: Eliminating Decreasing Paths to Infinity ⋮ Neural Parametric Fokker--Planck Equation ⋮ On the Benefit of Width for Neural Networks: Disappearance of Basins ⋮ Large Sample Mean-Field Stochastic Optimization ⋮ A rigorous framework for the mean field limit of multilayer neural networks ⋮ Normalization effects on deep neural networks ⋮ Unnamed Item ⋮ Mean field analysis of neural networks: a central limit theorem

Uses Software

• CIFAR
• DeepFace
• GNMT
• DGM

Cites Work

• High-density limits of hierarchically structured branching-diffusing populations
• Machine learning strategies for systems with invariance properties
• Wandering random measures in the Fleming-Viot model
• Hierarchically interacting Fleming-Viot processes with selection and mutation: Multiple space time scale analysis and quasi-equilibria
• Approximation and estimation bounds for artificial neural networks
• Multilevel mutation-selection systems and set-valued duals
• Multilayer feedforward networks are universal approximators
• DGM: a deep learning algorithm for solving partial differential equations
• Mean field analysis of neural networks: a central limit theorem
• A multilevel branching model
• Stochastic Gradient Descent in Continuous Time
• A mean field view of the landscape of two-layer neural networks
• Mean Field Analysis of Neural Networks: A Law of Large Numbers
• Kurdyka–Łojasiewicz–Simon inequality for gradient flows in metric spaces
• Reynolds averaged turbulence modelling using deep neural networks with embedded invariance
• Approximation by superpositions of a sigmoidal function
• Unnamed Item
• Unnamed Item
• Unnamed Item
• Unnamed Item
• Unnamed Item
• Unnamed Item
• Unnamed Item

This page was built for publication: Mean Field Analysis of Deep Neural Networks