“PyTorch: An Imperative Style, High-Performance Deep Learning Library” was presented at NeurIPS in 2019 by Adam Paszke, Sam Gross, Francisco Massa, and a large team from Facebook AI Research and the broader community. It is the canonical written description of the design philosophy behind what became the most widely used deep-learning research framework.
The paper’s central argument is that a framework does not have to trade ease of use for speed. PyTorch uses an imperative, define-by-run style in which the model is an ordinary Python program: control flow, debugging, and inspection all work the way they do in normal Python, and the computation graph is built dynamically as the code runs. This contrasted with the static-graph approach of early TensorFlow, where you first defined a graph and then executed it. The authors detail how careful engineering, a custom automatic-differentiation system, a caching memory allocator, and asynchronous GPU execution, let this user-friendly style still reach the performance needed for serious training.
PyTorch’s dynamic, Pythonic style became the dominant idiom in machine-learning research, and the paper documents the engineering choices that made that possible.
For a business reader, this paper marks the design that shaped how most modern AI models are written and trained, including the open-source ecosystem most teams build on.