This paper by Brandon T. Willard and Remi Louf, released in July 2023, is the foundation of the open-source Outlines library and a clean account of how to make a language model produce output that always conforms to a required structure, such as valid JSON, a date format, or a grammar. Coaxing structure through prompting alone is unreliable; the model can still emit malformed text.
The authors reformulate generation as transitions through the states of a finite-state machine. They precompute an index over the model’s vocabulary that, at every step, marks which next tokens would keep the output on a valid path according to a regular expression or context-free grammar. Tokens that would violate the structure are masked out before sampling. Because the hard work is done once in the index, the constraint adds minimal overhead at generation time, and the approach is model-agnostic.
The result is generation that is guaranteed to be well-formed, which is essential when a model’s output feeds another program, an API call, or a database.
For a business, guided generation is what makes language models dependable as components in software pipelines: instead of hoping the model returns parseable output, you can guarantee it, removing a whole class of brittle failures.