Modeling Hessian-vector products in nonlinear optimization: new Hessian-free methods

DOI10.1093/IMANUM/DRAB022zbMATH Open1498.65096arXiv1912.10523OpenAlexW3140491508MaRDI QIDQ5075167

Author name not available (Why is that?)

Publication date: 10 May 2022

Published in: IMA Journal of Numerical Analysis (Search for Journal in Brave)

Abstract: In this paper, we suggest two ways of calculating interpolation models for unconstrained smooth nonlinear optimization when Hessian-vector products are available. The main idea is to interpolate the objective function using a quadratic on a set of points around the current one and concurrently using the curvature information from products of the Hessian times appropriate vectors, possibly defined by the interpolating points. These enriched interpolating conditions form then an affine space of model Hessians or model Newton directions, from which a particular one can be computed once an equilibrium or least secant principle is defined. A first approach consists of recovering the Hessian matrix satisfying the enriched interpolating conditions, from which then a Newton direction model can be computed. In a second approach we pose the recovery problem directly in the Newton direction. These techniques can lead to a significant reduction in the overall number of Hessian-vector products when compared to the inexact or truncated Newton method, although simple implementations may pay a cost in linear algebra or number of function evaluations.

Full work available at URL: https://arxiv.org/abs/1912.10523

zbMATH Keywords

quadratic interpolationNewton directionHessian recoveryHessian-vector productsnonlinear/nonconvex optimization

Mathematics Subject Classification ID

Numerical mathematical programming methods (65K05) Numerical optimization and variational techniques (65K10)

Related Items (1)

A new framework for the computation of Hessians

Recommendations

• Title not available (Why is that?) 👍 👎
• Approximating Hessians in unconstrained optimization arising from discretized problems 👍 👎
• Inexact Hessian-vector products in reduced-space differential-equation constrained optimization 👍 👎
• Accelerated conjugate gradient algorithm with finite difference Hessian/vector product approximation for unconstrained optimization 👍 👎
• A new framework for the computation of Hessians 👍 👎
• Matrix-Free Convex Optimization Modeling 👍 👎
• Optimization of unconstrained functions with sparse hessian matrices-newton-type methods 👍 👎
• Projected Hessian Updating Algorithms for Nonlinearly Constrained Optimization 👍 👎
• Nonlinear Conjugate Gradient Methods for Vector Optimization 👍 👎
• New Basic Hessian Approximations for Large-Scale Nonlinear Least-Squares Optimization 👍 👎

This page was built for publication: Modeling Hessian-vector products in nonlinear optimization: new Hessian-free methods

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q5075167)