“Pre-train, Prompt, and Predict: A Systematic Survey of Prompting Methods in Natural Language Processing,” submitted to arXiv on July 28, 2021 by Pengfei Liu, Weizhe Yuan, Jinlan Fu, Zhengbao Jiang, Hiroaki Hayashi, and Graham Neubig, was the first comprehensive attempt to organize the fast-growing literature on prompting into a single framework. It gave the field a shared vocabulary at the moment prompting was becoming the dominant way to use large language models.
The survey frames prompting as a distinct paradigm. In the older “pre-train, fine-tune” approach, a pre-trained model is adapted to each new task by training it further on a labeled dataset, which changes the model’s weights and requires substantial task-specific data. In the “pre-train, prompt, predict” approach, the model’s weights stay frozen. Instead the input is rewritten into a template - a textual string with unfilled slots - and the language model fills in the blanks. Tasks are reformulated as language modeling problems, which is what lets the same frozen model handle few-shot and even zero-shot tasks with no labeled training data.
The paper catalogs the design choices that make up a prompting method: the shape of the prompt template, whether answers are generated freely or chosen from a fixed set, and how multiple prompts can be combined or tuned. By laying these out systematically, it turned a scattered collection of tricks into an engineering discipline.
Why business readers should care: this survey marks the point where “prompting” stopped being folklore and became a named method with a map. Understanding that prompting is an alternative to fine-tuning - cheaper, faster, but with its own trade-offs - is the starting point for deciding how to deploy language models.