Hyperband: A Novel Bandit-Based Approach to Hyperparameter Optimization

DOI10.48550/arXiv.1603.06560zbMath1468.68204arXiv1603.06560MaRDI QIDQ72746

Giulia DeSalvo, Kevin Jamieson, Lisha Li, Afshin Rostamizadeh, Ameet Talwalkar, Giulia Desalvo, Kevin Jamieson, Ameet Talwalkar, Afshin Rostamizadeh, Lisha Li

Publication date: 21 March 2016

Full work available at URL: https://arxiv.org/abs/1603.06560

zbMATH Keywords

model selectiononline optimizationdeep learninghyperparameter optimizationinfinite-armed bandits

Mathematics Subject Classification ID

Artificial neural networks and deep learning (68T07) Learning and adaptive systems in artificial intelligence (68T05) Approximation methods and heuristics in mathematical programming (90C59) Problem solving in the context of artificial intelligence (heuristics, search strategies, etc.) (68T20) Online algorithms; streaming algorithms (68W27)

Related Items (43)

Use of static surrogates in hyperparameter optimization ⋮ Best Arm Identification for Contaminated Bandits ⋮ Automated porosity estimation using CT-scans of extracted core data ⋮ Multiobjective Tree-Structured Parzen Estimator ⋮ Estimating shape parameters of piecewise linear-quadratic problems ⋮ Unnamed Item ⋮ Escaping local minima with local derivative-free methods: a numerical investigation ⋮ Automated Reinforcement Learning (AutoRL): A Survey and Open Problems ⋮ HEBO: An Empirical Study of Assumptions in Bayesian Optimisation ⋮ An efficient modified Hyperband and trust-region-based mode-pursuing sampling hybrid method for hyperparameter optimization ⋮ Computationally efficient integrated design and predictive control of flexible energy systems using multi‐fidelity simulation‐based Bayesian optimization ⋮ Supervised Machine Learning Techniques: An Overview with Applications to Banking ⋮ Investigation of the Lombard effect based on a machine learning approach ⋮ Recurrent and convolutional neural networks in structural dynamics: a modified attention steered encoder-decoder architecture versus LSTM versus GRU versus TCN topologies to predict the response of shock wave-loaded plates ⋮ Accelerated Componentwise Gradient Boosting Using Efficient Data Representation and Momentum-Based Optimization ⋮ Naive automated machine learning ⋮ Research on spatio-temporal network prediction model of parallel-series traffic flow based on transformer and GCAT ⋮ Local surrogate responses in the Schwarz alternating method for elastic problems on random voided domains ⋮ Unnamed Item ⋮ AutonoML: Towards an Integrated Framework for Autonomous Machine Learning ⋮ Automated Deep Learning: Neural Architecture Search Is Not the End ⋮ Business processes resource management using rewriting logic and deep-learning-based predictive monitoring ⋮ A data-driven explainable case-based reasoning approach for financial risk detection ⋮ Goal-oriented sensitivity analysis of hyperparameters in deep learning ⋮ Improving the efficiency of reinforcement learning for a spacecraft powered descent with Q-learning ⋮ A machine learning approach for efficient uncertainty quantification using multiscale methods ⋮ Bayesian deep convolutional encoder-decoder networks for surrogate modeling and uncertainty quantification ⋮ Hyperparameter optimization in learning systems ⋮ Unnamed Item ⋮ Bayesian optimization of pump operations in water distribution systems ⋮ mlr3hyperband ⋮ Imbalanced regression and extreme value prediction ⋮ Data-driven algorithm selection and tuning in optimization and signal processing ⋮ Combining Bayesian optimization and Lipschitz optimization ⋮ Joint detection of malicious domains and infected clients ⋮ Benchmark and Survey of Automated Machine Learning Frameworks ⋮ Unnamed Item ⋮ Unnamed Item ⋮ Unnamed Item ⋮ Unnamed Item ⋮ Learning and meta-learning of stochastic advection–diffusion–reaction systems from sparse measurements ⋮ A taxonomy of weight learning methods for statistical relational learning ⋮ Automatic model training under restrictive time constraints

Uses Software

• Spearmint
• CIFAR
• Hyperopt
• SMAC
• auto-sklearn
• Hyperband

Cites Work

• Fast Bayesian hyperparameter optimization on large datasets
• Bayesian Optimization in a Billion Dimensions via Random Embeddings
• Efficient Multi-Start Strategies for Local Search Algorithms
• Pure Exploration in Multi-armed Bandits Problems
• Information-Theoretic Regret Bounds for Gaussian Process Optimization in the Bandit Setting
• Unnamed Item
• Unnamed Item
• Unnamed Item
• Unnamed Item
• Unnamed Item
• Unnamed Item
• Unnamed Item
• Unnamed Item

This page was built for publication: Hyperband: A Novel Bandit-Based Approach to Hyperparameter Optimization