Particle dual averaging: optimization of mean field neural network with global convergence rate analysis*

DOI10.1088/1742-5468/ac98a8OpenAlexW3214193129MaRDI QIDQ5055425

Taiji Suzuki, Denny Wu, Atsushi Nitanda

Publication date: 13 December 2022

Published in: Journal of Statistical Mechanics: Theory and Experiment (Search for Journal in Brave)

Full work available at URL: https://arxiv.org/abs/2012.15477

zbMATH Keywords

machine learningdeep learningstochastic particle dynamics

Mathematics Subject Classification ID

Statistical mechanics, structure of matter (82-XX)

Related Items

A rigorous framework for the mean field limit of multilayer neural networks, Convergence rates of gradient methods for convex optimization in the space of measures

Uses Software

• Scikit

Cites Work

• Unnamed Item
• Unnamed Item
• Unnamed Item
• Unnamed Item
• Unnamed Item
• Unnamed Item
• Primal-dual subgradient methods for convex problems
• Smooth minimization of non-smooth functions
• Poincaré and logarithmic Sobolev inequalities by decomposition of the energy landscape
• Sample estimate of the entropy of a random vector
• Exponential convergence of Langevin distributions and their discrete approximations
• Generalization of an inequality by Talagrand and links with the logarithmic Sobolev inequality
• Logarithmic Sobolev inequalities and stochastic Ising models
• Adaptive estimation of a quadratic functional by model selection.
• Mixing time guarantees for unadjusted Hamiltonian Monte Carlo
• Gradient descent optimizes over-parameterized deep ReLU networks
• Mirror descent algorithms for minimizing interacting free energy
• Nonparametric regression using deep neural networks with ReLU activation function
• Mean field analysis of neural networks: a central limit theorem
• Couplings and quantitative contraction rates for Langevin dynamics
• Nonasymptotic convergence analysis for the unadjusted Langevin algorithm
• Ergodicity for SDEs and approximations: locally Lipschitz vector fields and degenerate noise.
• The Variational Formulation of the Fokker--Planck Equation
• Scaling Limit of the Stein Variational Gradient Descent: The Mean Field Regime
• A mean field view of the landscape of two-layer neural networks
• When do neural networks outperform kernel methods?*
• Interacting Langevin Diffusions: Gradient Structure and Ensemble Kalman Sampler
• Breaking the Curse of Dimensionality with Convex Neural Networks
• Understanding Machine Learning
• Convergence of unadjusted Hamiltonian Monte Carlo for mean-field models