On the existence of minimizers in shallow residual ReLU neural network optimization landscapes

DOI10.1137/23M1556241MaRDI QIDQ6652417

Arnulf Jentzen, Sebastian Kassing, Steffen Dereich

Publication date: 12 December 2024

Published in: SIAM Journal on Numerical Analysis (Search for Journal in Brave)

zbMATH Keywords

neural networksbest approximationapproximative compactnessshallow networksReLU activation

Mathematics Subject Classification ID

Artificial neural networks and deep learning (68T07) Best approximation, Chebyshev systems (41A50)

Cites Work

• Title not available (Why is that?)
• Title not available (Why is that?)
• Title not available (Why is that?)
• On the convergence of the proximal algorithm for nonsmooth functions involving analytic features
• Best approximation by linear combinations of characteristic functions of half-spaces.
• Convergence and convergence rate of stochastic gradient search in the case of multiple and non-isolated extrema
• Topological properties of the set of functions generated by neural networks of fixed size
• Best \(k\)-layer neural network approximations
• Landscape analysis for shallow neural networks: complete classification of critical points for affine target functions
• Cooling down stochastic differential equations: Almost sure convergence
• Stochastic subgradient method converges on tame functions
• Global Minima of Overparameterized Neural Networks
• Spurious Valleys in Two-layer Neural Network Optimization Landscapes
• The Łojasiewicz Inequality for Nonsmooth Subanalytic Functions with Applications to Subgradient Dynamical Systems
• Convergence of the Iterates of Descent Methods for Analytic Cost Functions
• Convex Analysis
• Mathematical Pictures at a Data Science Exhibition
• Stochastic gradient descent with noise of machine learning type. I: Discrete time analysis
• On minimal representations of shallow ReLU networks
• Central limit theorems for stochastic gradient descent with averaging for stable manifolds
• On the omnipresence of spurious local minima in certain neural network training problems

This page was built for publication: On the existence of minimizers in shallow residual ReLU neural network optimization landscapes

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q6652417)