No Free Lunch Theorems for Optimization

“No Free Lunch Theorems for Optimization” by David Wolpert and William Macready appeared in IEEE Transactions on Evolutionary Computation, volume 1, in 1997. It proved a result that is humbling and clarifying in equal measure: averaged over all possible problems, every optimization algorithm performs exactly the same.

The intuition is that an algorithm can only do better than random on some problems by doing worse on others. Any assumption that helps on the problems it suits will hurt on the problems it does not. If you average over the full universe of possible objective functions - including all the bizarre, structureless ones - the gains and losses cancel, and no method has an edge. A matching no-free-lunch result applies to supervised machine learning: no learning algorithm is universally better than any other across all possible datasets.

The practical lesson is not that algorithm choice does not matter, but the opposite. Real-world problems are not drawn uniformly from all possible problems; they have structure. An algorithm wins by matching its built-in assumptions to that structure. The theorem is a permanent caution against claims that some method is “best” in general.

Why business readers should care: there is no single best AI or analytics method for every situation, and vendors who claim otherwise are contradicting a proven theorem. The right choice depends on the specific structure of your problem and data.

Sources

Last verified June 7, 2026