Steepest descent with momentum for quadratic functions is a version of the conjugate gradient method

DOI10.1016/S0893-6080(03)00170-9zbMath1082.68093WikidataQ51728840 ScholiaQ51728840MaRDI QIDQ1887125

Amit Bhaya, Eugenius Kaszkurewicz

Publication date: 23 November 2004

Published in: Neural Networks (Search for Journal in Brave)

zbMATH Keywords

ConvergenceControl Lyapunov functionContinuous optimizationBackpropagationMomentumSteepest descentBilinear systemConjugate gradient algorithm

Mathematics Subject Classification ID

Learning and adaptive systems in artificial intelligence (68T05)

Related Items (17)

A secant-based Nesterov method for convex functions ⋮ A vector regularization method to solve linear inverse problems ⋮ Sparse approximations of matrix functions via numerical integration of ODEs ⋮ Runge-Kutta-like scaling techniques for first-order methods in convex optimization ⋮ ASD+M: automatic parameter tuning in stochastic optimization and on-line learning ⋮ An efficient primal-dual method for the obstacle problem ⋮ Momentum acceleration-based matrix splitting method for solving generalized absolute value equation ⋮ Distributed solving linear algebraic equations with switched fractional order dynamics ⋮ Semistability of Steepest Descent with Momentum for Quadratic Functions ⋮ A Study on the Optimal Double Parameters for Steepest Descent with Momentum ⋮ Near optimal step size and momentum in gradient descent for quadratic functions ⋮ A primal-dual optimization strategy for elliptic partial differential equations ⋮ Accelerating backpropagation using effective parameters at each step and an experimental evaluation ⋮ Dynamic search trajectory methods for global optimization ⋮ An online gradient method with momentum for two-layer feedforward neural networks ⋮ Comparison of ARIMA, neural networks and hybrid models in time series: tourist arrival forecasting ⋮ On parameter acceleration methods for saddle point problems

Cites Work

• Unnamed Item
• Unnamed Item
• Unnamed Item
• Unnamed Item
• Unnamed Item
• Unnamed Item
• Stabilization of bilinear systems by quadratic feedback controls
• Feedback stabilization of a second-order system: A nonmodal approach
• On asymptotically stabilizing feedback control of bilinear systems
• On the Continuous Realization of Iterative Processes
• Matrix differential equations: a continuous realization process for linear algebra problems
• Numerical Optimization
• THE HEAVY BALL WITH FRICTION METHOD, I. THE CONTINUOUS DYNAMICAL SYSTEM: GLOBAL EXPLORATION OF THE LOCAL MINIMA OF A REAL-VALUED FUNCTION BY ASYMPTOTIC ANALYSIS OF A DISSIPATIVE DYNAMICAL SYSTEM
• Some methods of speeding up the convergence of iteration methods

This page was built for publication: Steepest descent with momentum for quadratic functions is a version of the conjugate gradient method