Parallel Distributed Processing (Rumelhart and McClelland)

David Rumelhart, James McClelland, and the PDP Research Group published “Parallel Distributed Processing: Explorations in the Microstructure of Cognition” through MIT Press in 1986. The two-volume work, with Volume 1 on foundations and Volume 2 on psychological and biological models, is the book that relaunched neural networks as a serious approach to understanding the mind after the field had stagnated for over a decade.

The central commitment, called parallel distributed processing or connectionism, is that cognition emerges from the interactions of large numbers of simple, neuron-like units whose connection strengths encode knowledge in a distributed way. No single unit holds a concept; representations are patterns of activity spread across many units, and learning means adjusting the connections. This contrasted sharply with the symbol-manipulation view of mind that dominated cognitive science at the time.

The volumes collected chapters on distributed representations, competitive learning, harmony theory, and Boltzmann machines. Most consequential was the chapter by Rumelhart, Hinton, and Williams that gave a clear, accessible presentation of learning internal representations by error backpropagation, which moved the algorithm from obscurity into wide use.

For a general reader, these books are the intellectual headwaters of the deep learning era: the case they made, that useful internal representations can be learned from data by simple networks, is the case that AI has spent the decades since proving on an industrial scale.