An Inertial Newton Algorithm for Deep Learning
From MaRDI portal
Publication:5159400
zbMath1482.68211arXiv1905.12278MaRDI QIDQ5159400
Camille Castera, Cédric Févotte, Edouard Pauwels, Jérôme Bolte
Publication date: 27 October 2021
Full work available at URL: https://arxiv.org/abs/1905.12278
Artificial neural networks and deep learning (68T07) Nonconvex programming, global optimization (90C26) Stochastic programming (90C15)
Related Items
Asymptotic behavior of Newton-like inertial dynamics involving the sum of potential and nonpotential terms ⋮ The rate of convergence of optimization algorithms obtained via discretizations of heavy ball dynamical systems for convex optimization problems ⋮ Newton-type inertial algorithms for solving monotone equations Governed by sums of potential and nonpotential operators ⋮ Convergence of iterates for first-order optimization algorithms with inertia and Hessian driven damping ⋮ Subgradient Sampling for Nonsmooth Nonconvex Minimization ⋮ An Improved Unconstrained Approach for Bilevel Optimization ⋮ A fast and simple modification of Newton's method avoiding saddle points ⋮ Inertial Newton algorithms avoiding strict saddle points ⋮ First order inertial optimization algorithms with threshold effects associated with dry friction ⋮ Conservative parametric optimality and the ridge method for tame min-max problems ⋮ Convergence of inertial dynamics driven by sums of potential and nonpotential operators with implicit Newton-like damping ⋮ Fast optimization via inertial dynamics with closed-loop damping ⋮ Continuous Newton-like Methods Featuring Inertia and Variable Mass ⋮ Unnamed Item ⋮ On the effect of perturbations in first-order optimization methods with inertia and Hessian driven damping
Uses Software
Cites Work
- Unnamed Item
- Unnamed Item
- Unnamed Item
- Unnamed Item
- Unnamed Item
- Unnamed Item
- Unnamed Item
- Unnamed Item
- A Stochastic Quasi-Newton Method for Large-Scale Optimization
- Proximal alternating linearized minimization for nonconvex and nonsmooth problems
- A dynamical system associated with Newton's method for parametric approximations of convex minimization problems
- On gradients of functions definable in o-minimal structures
- A second-order gradient-like dissipative dynamical system with Hessian-driven damping. Application to optimization and mechanics.
- Conservative set valued fields, automatic differentiation, stochastic gradient methods and deep learning
- Convergence of constant step stochastic gradient descent for non-smooth non-convex functions
- Convergence rate of inertial proximal algorithms with general extrapolation and proximal coefficients
- Newton-type methods for non-convex optimization under inexact Hessian information
- Stochastic subgradient method converges on tame functions
- On the Use of Stochastic Hessian Information in Optimization Methods for Machine Learning
- Proximal Alternating Minimization and Projection Methods for Nonconvex Problems: An Approach Based on the Kurdyka-Łojasiewicz Inequality
- Clarke Subgradients of Stratifiable Functions
- Characterizations of Łojasiewicz inequalities: Subgradient flows, talweg, convexity
- Analysis of recursive stochastic algorithms
- Stochastic Methods for Composite and Weakly Convex Optimization Problems
- Optimization Methods for Large-Scale Machine Learning
- An investigation of Newton-Sketch and subsampled Newton methods
- Convergence and Dynamical Behavior of the ADAM Algorithm for Nonconvex Stochastic Optimization
- Stochastic Approximations and Differential Inclusions
- Learning representations by back-propagating errors
- The Łojasiewicz Inequality for Nonsmooth Subanalytic Functions with Applications to Subgradient Dynamical Systems
- Some methods of speeding up the convergence of iteration methods
- A Stochastic Approximation Method
