Bayesian optimization is a strategy for finding the best inputs to a function that is expensive to evaluate and whose form you do not know, such as the validation accuracy of a model as a function of its hyperparameters. Rather than trying inputs blindly, it builds a probabilistic surrogate model of the function, often a Gaussian process, which gives both a prediction and an estimate of uncertainty at every untested point.
An acquisition function then uses that surrogate to decide where to sample next, balancing exploitation (trying points the surrogate predicts will be good) against exploration (trying points where the surrogate is uncertain and might be hiding something better). After each real evaluation, the surrogate is updated and the cycle repeats, so the method needs relatively few expensive evaluations to make progress. Snoek, Larochelle, and Adams’ 2012 paper “Practical Bayesian Optimization of Machine Learning Algorithms” made this approach practical for tuning machine-learning models, including handling parallel trials and varying evaluation costs.
Bayesian optimization is now a core technique in hyperparameter tuning, AutoML, experimental design, and even drug and materials discovery, anywhere each test is costly.
For a business reader, it is the principled answer to a common problem: how to find the best configuration when each thing you try costs real time or money.