Alexey Grigorievich Ivakhnenko (1913-2007) was a Soviet and Ukrainian mathematician and cyberneticist whom some historians call the grandfather of deep learning. In 1965, with Valentin Lapa, he introduced the Group Method of Data Handling (GMDH), a technique for automatically building multilayer networks of polynomial units, adding and pruning layers to fit data. On Juergen Schmidhuber’s documented history of the field, Ivakhnenko and Lapa “introduced the first general, working learning algorithms for deep multi-layer perceptrons.”
GMDH built its networks layer by layer from training examples, keeping the units that improved prediction and discarding the rest - a self-organizing approach to depth. Schmidhuber records that a 1971 paper described “a deep learning net with 8 layers” trained by the method, and notes that Ivakhnenko had “connectionism with adaptive hidden layers two decades before the name ‘connectionism’ became popular in the 1980s.”
Ivakhnenko’s work sits awkwardly with the standard Western story of deep learning, which usually starts in the 1980s. Schmidhuber pointedly observes that this pioneering work was “repeatedly republished without attribution” by researchers who later shared a Turing Award. Whatever one makes of the priority dispute, Ivakhnenko is a clear case of a major idea arriving early, in the wrong place and language to be widely credited, and being rediscovered decades later.