“Practical Bayesian Optimization of Machine Learning Algorithms” was submitted in June 2012 by Jasper Snoek, Hugo Larochelle, and Ryan P. Adams. It tackled what the authors called the black art of hyperparameter tuning, the trial-and-error process of choosing learning rates, regularization strengths, and similar settings that strongly affect how well a model performs.
The paper frames tuning as a Bayesian optimization problem. A Gaussian process is used to build a probabilistic model of how validation performance depends on the hyperparameters, and an acquisition function decides which configuration to try next, balancing exploring uncertain regions against exploiting promising ones. The key practical contributions were methods that account for the fact that some configurations take much longer to evaluate than others, and that experiments can be run in parallel. On contemporary models including latent Dirichlet allocation, structured support vector machines, and convolutional neural networks, the approach matched or beat expert human tuning.
This paper is widely credited with bringing Bayesian optimization into mainstream machine-learning practice and inspired tools such as Spearmint and many later AutoML systems.
For a business reader, it is the foundation of automated tuning, replacing a tedious manual chore with a principled, sample-efficient search.