Trust-region algorithms for training responses: machine learning methods using indefinite Hessian approximations

DOI10.1080/10556788.2019.1624747zbMath1440.90092arXiv1807.00251OpenAlexW2811026747WikidataQ127744831 ScholiaQ127744831MaRDI QIDQ5113710

Joshua D. Griffin, Jennifer B. Erway, Roummel F. Marcia, Riadh Omheni

Publication date: 16 June 2020

Published in: Optimization Methods and Software (Search for Journal in Brave)

Full work available at URL: https://arxiv.org/abs/1807.00251

zbMATH Keywords

large-scale optimizationquasi-Newton methodsmachine learningtrust-region methodsnon-convexlimited-memory symmetric rank-one update

Mathematics Subject Classification ID

Numerical mathematical programming methods (65K05) Large-scale problems in mathematical programming (90C06) Numerical optimization and variational techniques (65K10) Newton-type methods (49M15) Methods of quasi-Newton type (90C53) Linear equations (linear algebraic aspects) (15A06)

Related Items

Globally Convergent Multilevel Training of Deep Residual Networks, A limited-memory trust-region method for nonlinear optimization with many equality constraints

Uses Software

• L-BFGS
• LSTRS
• GQTPAR
• MNIST
• TensorFlow
• Spearmint
• Hyperopt
• AdaGrad
• Adam
• HOGWILD
• EMNIST
• TuPAQ
• HSL-VF05
• ADADELTA

Cites Work

• Unnamed Item
• Unnamed Item
• Unnamed Item
• Unnamed Item
• Unnamed Item
• Unnamed Item
• A Stochastic Quasi-Newton Method for Large-Scale Optimization
• On solving L-SR1 trust-region subproblems
• Sample size selection in optimization methods for machine learning
• A quasi-Newton algorithm for nonconvex, nonsmooth optimization with global convergence guarantees
• On the limited memory BFGS method for large scale optimization
• Convergence of quasi-Newton matrices generated by the symmetric rank one update
• Representations of quasi-Newton matrices and their use in limited memory methods
• A New Matrix-Free Algorithm for the Large-Scale Trust-Region Subproblem
• Minimizing a Quadratic Over a Sphere
• On Efficiently Computing the Eigenvalues of Limited-Memory Quasi-Newton Matrices
• Computing a Trust Region Step
• Algorithm 873
• Iterative Methods for Finding a Trust-region Step
• A Subspace Minimization Method for the Trust-Region Step
• The Conjugate Gradient Method and Trust Regions in Large Scale Optimization
• Testing a Class of Methods for Solving Minimization Problems with Simple Bounds on the Variables
• Updating Quasi-Newton Matrices with Limited Storage
• Computing Optimal Locally Constrained Steps
• Trust Region Methods
• Optimization Methods for Large-Scale Machine Learning
• Solving the Trust-Region Subproblem using the Lanczos Method
• A robust multi-batch L-BFGS method for machine learning
• A Stochastic Approximation Method
• The elements of statistical learning. Data mining, inference, and prediction