A geometric approach of gradient descent algorithms in linear neural networks

DOI10.3934/mcrf.2022021zbMath1524.68312arXiv1811.03568OpenAlexW4285139792MaRDI QIDQ6099180

Yacine Chitour, Zhenyu Liao, Romain Couillet

Publication date: 19 June 2023

Published in: Mathematical Control and Related Fields (Search for Journal in Brave)

Full work available at URL: https://arxiv.org/abs/1811.03568

zbMATH Keywords

normal hyperbolicitygradient descentdeep learning

Mathematics Subject Classification ID

Artificial neural networks and deep learning (68T07) Nonconvex programming, global optimization (90C26) Gradient-like behavior; isolated (locally maximal) invariant sets; attractors, repellers for topological dynamical systems (37B35)

Related Items (1)

Global convergence of the gradient method for functions definable in o-minimal structures

Cites Work

• Unnamed Item
• Unnamed Item
• Unnamed Item
• Unnamed Item
• Regression modeling strategies. With applications to linear models, logistic regression, and survival analysis
• Lectures on partial hyperbolicity and stable ergodicity
• First-order methods almost always avoid strict saddle points
• Some NP-complete problems in quadratic and nonlinear programming
• Gradient Descent Only Converges to Minimizers: Non-Isolated Critical Points and Invariant Regions
• Explicit bounds for the Łojasiewicz exponent in the gradient inequality for polynomials
• Convergence of the Iterates of Descent Methods for Analytic Cost Functions
• Convergence Conditions for Nonlinear Programming Algorithms
• Invariant manifolds

This page was built for publication: A geometric approach of gradient descent algorithms in linear neural networks