Incremental gradient algorithms with stepsizes bounded away from zero
type: Item
instance of: scholarly article
title: Incremental gradient algorithms with stepsizes bounded away from zero
zbMATH Open document ID: 0915.65061
DOI: 10.1023/A:1018366000512
published in: Computational Optimization and Applications
publication date: 1999-06-15T00:00:00Z
review text: For the problem \[ \text{minimize}\quad \sum^k_{j= 1} f_j(x)\quad\text{subject to}\quad x\in\mathbb{R}^n, \] where \(f_1,\dots, f_k: \mathbb{R}^n\to \mathbb{R}\) are continuously differentiable functions, the author proposes the following incremental gradient algorithm: Choose any \(x^0\in \mathbb{R}^n\). Having \(x^i\), check a stopping criterion. If not satisfied, compute \(x^{i+ 1}= T(x^i,\eta_i)\), where \(T:\mathbb{R}^n\times \mathbb{R}^+\to \mathbb{R}^n\) is given by \[ T(x,\eta):= x-\eta \sum^k_{j=1}\nabla f_j(z^j) \] and \[ z^1= x,\quad z^{j+1}= z^j- \eta\nabla f_j(z^j),\quad j=1,\dots, k-1. \] The author presents convergence results for a class of incremental gradient algorithms with stepsizes bounded away from zero. Applications on neural network training are discussed.
Mathematics Subject Classification ID: 65K05
Mathematics Subject Classification ID: 90C06
Mathematics Subject Classification ID: 90C30
zbMATH DE Number: 1230440
zbMATH Keywords: incremental gradient algorithm
zbMATH Keywords: convergence
zbMATH Keywords: neural network training
author: M. V. Solodov
reviewed by: Dorel I. Duca
MaRDI profile type: Publication
full work available at URL: a:1018366000512
OpenAlex ID: W2113372738
description: scientific article; zbMATH DE number 1230440