Extending the Step-Size Restriction for Gradient Descent to Avoid Strict Saddle Points

DOI10.1137/19M129509XzbMath1484.90088arXiv1908.01753OpenAlexW3111569753MaRDI QIDQ5027014

Scott G. McCalla, Hayden Schaeffer

Publication date: 3 February 2022

Published in: SIAM Journal on Mathematics of Data Science (Search for Journal in Brave)

Full work available at URL: https://arxiv.org/abs/1908.01753

zbMATH Keywords

optimizationnonconvexgradient descentstrict saddles

Mathematics Subject Classification ID

Nonconvex programming, global optimization (90C26) Normal forms, center manifold theory, bifurcation theory for infinite-dimensional dissipative dynamical systems (37L10)

Cites Work

• Nonconvergence to unstable points in urn models and stochastic approximations
• Introductory lectures on convex optimization. A basic course.
• A geometric analysis of phase retrieval
• The zero set of a real analytic function
• First-order methods almost always avoid strict saddle points
• Cubic regularization of Newton method and its global performance
• Complete Dictionary Recovery Over the Sphere I: Overview and the Geometric Picture
• Complete Dictionary Recovery Over the Sphere II: Recovery by Riemannian Trust-Region Method
• On the use of directions of negative curvature in a modified newton method
• Trust Region Methods
• Accelerated Methods for NonConvex Optimization
• Theoretical Insights Into the Optimization Landscape of Over-Parameterized Shallow Neural Networks
• Gradient Descent Only Converges to Minimizers: Non-Isolated Critical Points and Invariant Regions
• Complexity Analysis of Second-Order Line-Search Algorithms for Smooth Nonconvex Optimization
• Convergence of the Iterates of Descent Methods for Analytic Cost Functions

This page was built for publication: Extending the Step-Size Restriction for Gradient Descent to Avoid Strict Saddle Points