“TabPFN: A Transformer That Solves Small Tabular Classification Problems in a Second” by Noah Hollmann, Samuel Mueller, Katharina Eggensperger, and Frank Hutter was posted to arXiv in 2022, with a later version published in the journal Nature in 2025. It applies the foundation-model idea to tabular data and produced a striking result.
TabPFN is a transformer that has been pre-trained once on millions of synthetic tabular datasets generated from a prior over plausible data-generating processes. To use it, a practitioner does not train anything: they feed the model their labeled training rows and the rows they want predicted, all at once, and the transformer produces predictions in a single forward pass through in-context learning, much as a language model answers from examples in its prompt. On small tabular classification problems, with a few thousand rows, it matched or beat tuned gradient-boosting pipelines while running in about a second, a reported speedup of up to 230 times once tuning time is counted.
TabPFN challenged the long-standing assumption that tree ensembles are unbeatable on small tabular data, at least in the small-data regime it targets.
Why business readers should care: TabPFN can deliver strong predictions on small datasets instantly, without the data-science labor of model selection and tuning.