Reverend Bayes on Inference Engines: A Distributed Hierarchical Approach

Judea Pearl presented this paper at the 1982 AAAI conference, addressing a practical obstacle to using Bayesian reasoning in artificial intelligence. Updating beliefs across many interrelated variables by brute force is exponentially expensive, which made Bayesian methods seem impractical for real inference engines. Pearl proposed a way to spread the effect of new evidence efficiently through a structured network of hypotheses.

His scheme, now known as belief propagation, assigns each variable a small set of belief parameters and has neighboring variables exchange messages that summarize what each one implies about the other. Each node updates its own belief from the messages it receives and passes new messages onward. For networks shaped like trees, Pearl showed this local message passing converges in a single sweep to exactly the answer that full Bayesian calculation would give, but at a fraction of the cost.

The paper is foundational because it reframed probabilistic inference as communication between parts of a graph rather than a monolithic calculation. That perspective became central to graphical models and led directly to algorithms still in use for decoding error-correcting codes, parsing language, and reasoning under uncertainty.

For a general reader, the lesson is that hard global computations can sometimes be replaced by many simple local ones, an idea that recurs throughout modern AI and that made probabilistic reasoning fast enough to deploy.