Regularized Step Directions in Nonlinear Conjugate Gradient Methods

Author name not available (Why is that?)

Publication date: 12 October 2021

Abstract: Conjugate gradient minimization methods (CGM) and their accelerated variants are widely used. We focus on the use of cubic regularization to improve the CGM direction independent of the steplength computation. In this paper, we propose the Hybrid Cubic Regularization of CGM, where regularized steps are used selectively. Using Shanno's reformulation of CGM as a memoryless BFGS method, we derive new formulas for the regularized step direction. We show that the regularized step direction uses the same order of computational burden per iteration as its non-regularized version. Moreover, the Hybrid Cubic Regularization of CGM exhibits global convergence with fewer assumptions. In numerical experiments, the new step directions are shown to require fewer iteration counts, improve runtime, and reduce the need to reset the step direction. Overall, the Hybrid Cubic Regularization of CGM exhibits the same memoryless and matrix-free properties, while outperforming CGM as a memoryless BFGS method in iterations and runtime.

Has companion code repository: https://github.com/cassiebuhler/conmincg

This page was built for publication: Regularized Step Directions in Nonlinear Conjugate Gradient Methods

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q6380131)