“BOHB: Robust and Efficient Hyperparameter Optimization at Scale” was submitted in July 2018 by Stefan Falkner, Aaron Klein, and Frank Hutter. It set out to combine the two leading families of tuning methods so as to get the best of both.
Bayesian optimization is sample-efficient, learning a model of which configurations are promising, but the time to train modern models makes plain Bayesian optimization slow to get started. Hyperband, by contrast, is cheap to start and parallelizes well because it kills weak configurations early, but it chooses configurations at random and so never learns from past trials. BOHB (Bayesian Optimization and HyperBand) replaces Hyperband’s random sampling with a Bayesian model that guides which configurations to try, while keeping Hyperband’s early-stopping schedule. The result has strong anytime performance, finding good configurations quickly and continuing to improve, and it scales across parallel workers.
BOHB became a widely used default in AutoML toolkits and influenced many later multi-fidelity optimization methods.
For a business reader, BOHB is a practical example of engineering progress: rather than a single breakthrough, it carefully fuses two existing techniques to remove each other’s weaknesses.