Use of static surrogates in hyperparameter optimization

DOI10.1007/s43069-022-00128-wzbMath1483.68305arXiv2103.07963OpenAlexW4213337141MaRDI QIDQ2120124

Dounia Lakhmiri, Sébastien Le Digabel

Publication date: 31 March 2022

Published in: SN Operations Research Forum (Search for Journal in Brave)

Full work available at URL: https://arxiv.org/abs/2103.07963

zbMATH Keywords

blackbox optimization (BBO)derivative-free optimization (DFO)hyperparameter optimization (HPO)

Mathematics Subject Classification ID

Derivative-free methods and methods using generalized derivatives (90C56) Learning and adaptive systems in artificial intelligence (68T05) Sensitivity, stability, parametric optimization (90C31)

Uses Software

• NOMAD
• APPSPACK
• MNIST
• CIFAR
• Hyperopt
• PyTorch
• GitHub
• Fashion-MNIST
• STL-10 dataset
• Hyperband
• HyperNOMAD

Cites Work

• Unnamed Item
• Unnamed Item
• Hyperband: A Novel Bandit-Based Approach to Hyperparameter Optimization
• Fast Bayesian hyperparameter optimization on large datasets
• Generalized pattern searches with derivative information
• Algorithm 909
• Algorithm 856
• Introduction to Derivative-Free Optimization
• Derivative-Free and Blackbox Optimization
• HyperNOMAD
• Mesh Adaptive Direct Search Algorithms for Constrained Optimization
• A Simplex Method for Function Minimization