On July 23, 2024, Meta released Llama 3.1, headlined by a 405-billion-parameter model that Meta described as the first frontier-level open source AI model. Meta claimed it rivaled the top closed models on general knowledge, math, tool use, and multilingual translation, and shipped it with a 128,000-token context window and support for eight languages, alongside upgraded 8B and 70B versions.
The scale of the effort was substantial. Llama 3.1 405B was trained on more than 15 trillion tokens using over 16,000 NVIDIA H100 GPUs. Crucially, Meta released the weights for download, arguing in the announcement that open models keep power from concentrating in a few hands and let developers fully customize, fine-tune, and self-host rather than depending on a closed API.
Llama 3.1 405B closed much of the gap between open and closed frontier models. By putting a model competitive with the best proprietary systems into the open ecosystem, it strengthened the case that open weights could keep pace at the frontier rather than trailing a generation behind.