BLOOM, the BigScience Large Open-science Open-access Multilingual Language Model, was released in July 2022 by the BigScience workshop, a year-long collaboration of more than a thousand researchers coordinated largely by Hugging Face. At 176 billion parameters, it was a decoder-only Transformer of roughly the same scale as GPT-3, but built and shared openly rather than behind a commercial API.
What set BLOOM apart was its breadth and openness. It was trained on the ROOTS corpus spanning 46 natural languages and 13 programming languages, an explicit response to the English-centric nature of most large models. Training ran for several months on the Jean Zay public supercomputer in France using government-funded research compute. The model, code, and a detailed account of the process were released under the BigScience Responsible AI License (RAIL), which permits broad use while prohibiting certain harmful applications.
BLOOM, alongside Meta’s OPT-175B earlier the same year, marked the arrival of GPT-3-scale models that anyone could download and study. Its emphasis on multilingual coverage and transparent, community-driven development made it a landmark for open and participatory AI research.