Loss landscapes and optimization in over-parameterized non-linear systems and neural networks
From MaRDI portal
Publication:2134108
DOI10.1016/j.acha.2021.12.009OpenAlexW4226038297MaRDI QIDQ2134108
Chaoyue Liu, Libin Zhu, Mikhail Belkin
Publication date: 6 May 2022
Published in: Applied and Computational Harmonic Analysis (Search for Journal in Brave)
Full work available at URL: https://arxiv.org/abs/2003.00307
Related Items (6)
An adaptively weighted stochastic gradient MCMC algorithm for Monte Carlo simulation and global optimization ⋮ On the Benefit of Width for Neural Networks: Disappearance of Basins ⋮ Approximation bounds for norm constrained neural networks with applications to regression and GANs ⋮ Localized ensemble Kalman inversion ⋮ Newton-MR: inexact Newton method with minimum residual sub-problem solver ⋮ Discovering a reaction-diffusion model for Alzheimer's disease by combining PINNs with symbolic regression
Uses Software
Cites Work
- Gradient descent optimizes over-parameterized deep ReLU networks
- Condition
- Theoretical Insights Into the Optimization Landscape of Over-Parameterized Shallow Neural Networks
- A mean field view of the landscape of two-layer neural networks
- Global Minima of Overparameterized Neural Networks
- Gradient Descent with Identity Initialization Efficiently Learns Positive-Definite Linear Transformations by Deep Residual Networks
- Reconciling modern machine-learning practice and the classical bias–variance trade-off
- Condition Numbers of Gaussian Random Matrices
- Wide neural networks of any depth evolve as linear models under gradient descent *
- A jamming transition from under- to over-parametrization affects generalization in deep learning
- Unnamed Item
- Unnamed Item
- Unnamed Item
- Unnamed Item
This page was built for publication: Loss landscapes and optimization in over-parameterized non-linear systems and neural networks
