From the Ravine Method to the Nesterov Method and Vice Versa: A Dynamical System Perspective

DOI10.1137/22M1474357zbMath1503.37098arXiv2201.11643OpenAlexW4221149512MaRDI QIDQ5097023

Jalal Fadili, Hedy Attouch

Publication date: 19 August 2022

Published in: SIAM Journal on Optimization (Search for Journal in Brave)

Full work available at URL: https://arxiv.org/abs/2201.11643

zbMATH Keywords

convergence ratesLyapunov analysisproximal algorithmsRavine methodNesterov accelerated gradient methodHessian driven dampinghigh resolution ODE

Mathematics Subject Classification ID

Numerical mathematical programming methods (65K05) Convex programming (90C25) Numerical optimization and variational techniques (65K10) Management decision making, including multiple objectives (90B50) Applications of functional analysis in optimization, convex analysis, mathematical programming, economics (46N10) Dynamical systems in optimization and economics (37N40)

Related Items (3)

Accelerated gradient methods combining Tikhonov regularization with geometric damping driven by the Hessian ⋮ From Halpern's fixed-point iterations to Nesterov's accelerated interpretations for root-finding problems ⋮ Continuous Newton-like Methods Featuring Inertia and Variable Mass

Cites Work

• Unnamed Item
• Unnamed Item
• Unnamed Item
• Unnamed Item
• Unnamed Item
• A Fast Iterative Shrinkage-Thresholding Algorithm for Linear Inverse Problems
• Optimized first-order methods for smooth convex minimization
• Fast convex optimization via inertial dynamics with Hessian driven damping
• On the convergence of the iterates of the ``fast iterative shrinkage/thresholding algorithm
• A second-order gradient-like dissipative dynamical system with Hessian-driven damping. Application to optimization and mechanics.
• Introductory lectures on convex optimization. A basic course.
• Rate of convergence of inertial gradient dynamics with time-dependent viscous damping coefficient
• Asymptotic stabilization of inertial gradient dynamics with time-dependent viscosity
• Continuous Newton-like inertial dynamics for monotone inclusions
• Understanding the acceleration phenomenon via high-resolution differential equations
• First-order optimization algorithms via inertial systems with Hessian driven damping
• Asymptotic behavior of Newton-like inertial dynamics involving the sum of potential and nonpotential terms
• Tikhonov regularization of a second order dynamical system with Hessian driven damping
• An extension of the second order dynamical system that models Nesterov's convex gradient method
• Convergence rate of inertial forward-backward algorithm beyond Nesterov's rule
• Backward-forward algorithms for structured monotone inclusions in Hilbert spaces
• Fast convergence of inertial dynamics and algorithms with asymptotic vanishing viscosity
• First-order frameworks for continuous Newton-like dynamics governed by maximally monotone operators
• Newton-type inertial algorithms for solving monotone equations Governed by sums of potential and nonpotential operators
• The Rate of Convergence of Nesterov's Accelerated Forward-Backward Method is Actually Faster Than $1/k^2$
• Accelerated and Inexact Forward-Backward Algorithms
• A Continuous Dynamical Newton-Like Approach to Solving Monotone Inclusions
• Evolution equations for maximal monotone operators: asymptotic analysis in continuous and discrete time
• On the Long Time Behavior of Second Order Differential Equations with Asymptotically Small Dissipation
• On the long time behavior of second order differential equations with asymptotically small dissipation
• On the Convergence of the Proximal Point Algorithm for Convex Minimization
• New Proximal Point Algorithms for Convex Minimization
• THE HEAVY BALL WITH FRICTION METHOD, I. THE CONTINUOUS DYNAMICAL SYSTEM: GLOBAL EXPLORATION OF THE LOCAL MINIMA OF A REAL-VALUED FUNCTION BY ASYMPTOTIC ANALYSIS OF A DISSIPATIVE DYNAMICAL SYSTEM
• The Differential Inclusion Modeling FISTA Algorithm and Optimality of Convergence Rate in the Case b $\leq3$
• Convergence Rates of Inertial Forward-Backward Algorithms
• Asymptotic for a second-order evolution equation with convex potential andvanishing damping term
• A Dynamical Approach to an Inertial Forward-Backward Algorithm for Convex Minimization
• Rate of convergence of the Nesterov accelerated gradient method in the subcritical case α ≤ 3
• Finite Convergence of Proximal-Gradient Inertial Algorithms Combining Dry Friction with Hessian-Driven Damping
• Newton-like Inertial Dynamics and Proximal Algorithms Governed by Maximally Monotone Operators
• Optimal Convergence Rates for Nesterov Acceleration
• Some methods of speeding up the convergence of iteration methods

This page was built for publication: From the Ravine Method to the Nesterov Method and Vice Versa: A Dynamical System Perspective