Convergence of constant step stochastic gradient descent for non-smooth non-convex functions

DOI10.1007/s11228-022-00638-zzbMath1497.65099arXiv2005.08513OpenAlexW3025065608MaRDI QIDQ2158840

Sholom Schechtman, Pascal Bianchi, Walid Hachem

Publication date: 26 July 2022

Published in: Set-Valued and Variational Analysis (Search for Journal in Brave)

Full work available at URL: https://arxiv.org/abs/2005.08513

zbMATH Keywords

differential inclusionsstochastic approximationClarke subdifferentialbackpropagation algorithmnon-convex and non-smooth optimization

Mathematics Subject Classification ID

Numerical optimization and variational techniques (65K10) Stochastic programming (90C15) Ordinary differential inclusions (34A60)

Related Items (7)

Survey Descent: A Multipoint Generalization of Gradient Descent for Nonsmooth Optimization ⋮ Lyapunov stability of the subgradient method with constant step size ⋮ Subgradient Sampling for Nonsmooth Nonconvex Minimization ⋮ An Improved Unconstrained Approach for Bilevel Optimization ⋮ Conservative parametric optimality and the ridge method for tame min-max problems ⋮ An Inertial Newton Algorithm for Deep Learning ⋮ Perturbed iterate SGD for Lipschitz continuous loss functions

Cites Work

• Unnamed Item
• Unnamed Item
• Unnamed Item
• Unnamed Item
• Unnamed Item
• Unnamed Item
• Solution of nonconvex nonsmooth stochastic optimization problems
• Stochastic generalized gradient method for nonconvex nonsmooth stochastic optimization
• Geometric categories and o-minimal structures
• Conservative set valued fields, automatic differentiation, stochastic gradient methods and deep learning
• Convergence of a stochastic subgradient method with averaging for nonsmooth nonconvex constrained optimization
• Stochastic subgradient method converges on tame functions
• Clarke Subgradients of Stratifiable Functions
• Markov Chains and Stochastic Stability
• An Invitation to Tame Optimization
• Generic Differentiability of Lipschitzian Functions
• Poincaré's recurrence theorem for set-valued dynamical systems
• ERGODIC PROPERTIES OF WEAK ASYMPTOTIC PSEUDOTRAJECTORIES FOR SET-VALUED DYNAMICAL SYSTEMS
• Stochastic Approximations with Constant Step Size and Differential Inclusions
• Constant step stochastic approximations involving differential inclusions: stability, long-run convergence and applications
• Stochastic Approximations and Differential Inclusions
• Principle of Averaging for Parabolic and Elliptic Differential Equations and for Markov Processes with Small Diffusion

This page was built for publication: Convergence of constant step stochastic gradient descent for non-smooth non-convex functions