VSI-Bench
A visual-spatial benchmark testing whether multimodal models can understand and recall spaces from video.
How AI is measured - each tied to the paper or site that defined it.
A visual-spatial benchmark testing whether multimodal models can understand and recall spaces from video.
A benchmark that separates honesty from accuracy, measuring whether language models lie under pressure rather than just whether they are correct.
The 2025 successor to ARC-AGI, a harder reasoning benchmark that frontier AI systems initially scored in the single digits on.
OpenAI benchmark of 1,266 questions that force a web agent to dig persistently for hard-to-find facts.
A Meta hallucination benchmark built on a clear taxonomy, with tasks that regenerate to resist leakage.
OpenAI benchmark scoring AI on real economically valuable work across 44 occupations in nine GDP sectors.
A 46-task terminal benchmark with dense partial-credit grading where the best agent scores just 15.2 percent.
A Wharton, CMU, Penn, and Harvard team benchmarked frontier AI on business cases across 18 disciplines; models already score highly on instructor rubrics.
A 191-task benchmark where five frontier models break 65 to 86 percent of already-broken cryptographic schemes.
A three-benchmark index scores models on unverifiable conceptual argument; Opus 5 leads at 73.6 against an estimated ceiling of 91.
A 60-task benchmark of project-level scientific research where agent scores fall from 50.91 to 26.62 once methodological guidance is removed.
Twenty whole-repository migrations where only 28 of 520 frontier agent runs passed all three evaluation stages.
A cross-domain science benchmark where the best agent configuration finished only 20 of 97 end-to-end workflows.
Google DeepMind tested a Gemini model against confidential benchmarks inside a cryptographically sealed enclave.
Ai2 releases BenchMIRT, an item-response-theory tool that audits LLM benchmarks per question, trained on 100 models and 34,000-plus questions
1,600 sandbox tasks across seven languages and eight cultures, where frontier models reach only 49.2 percent.
Epoch AI's 60-photo IKEA error-spotting test: top score rose from 28 percent in November 2025 to 80 percent for GPT-6 Astra in September 2026.
1,215 synthetic conversations with rubrics from 80-plus licensed clinicians; GPT-6 Astra leads at 57.3 percent, Claude Opus 5.5 third.
Given 3,000 GPU-hours to rediscover a 2026 post-training method, GPT-5.6 Sol reached 15 percent of its gains and Claude Fable 5 none in scope.
Epoch's SpireClimb benchmark shows models winning Slay the Spire 90%+ at base difficulty and up to 20% at its hardest, from near zero six months ago.
GPT-6 Astra passes 87.4% of SQLite tests across 20 languages vs 26.8% for GPT-5.6 Sol; GPT-6.1 Sol hits 97.9% on COBOL migration.
Ai2's StateBench grades long-context state tracking in four levels; frontier models fall to 13 percent or lower on the hardest, longest problems.
Epoch's Bayesian ECI refit gives credible intervals and head-to-head odds: GPT-6 Astra has a 72 percent chance of out-ranking Claude Opus 5.5.
FrontierPhysics has 56 research tasks written by 42 physicists; across 22 models no setup passed more than 46 of its 168 rollouts.
TCSExam's 100 expert-reviewed tasks are checked by Lean or executable checkers; GPT-6 Astra leads the full set at 71.9.
Ai2's ToyBlocks pairs simulated block tasks and 1,600 hours of demonstrations with real-robot tests; a Molmo 2 plus ManiFlow agent spells 50 percent.
On 100 tasks built from raw lab data with Ligo, GPT-6 Astra earned 66.7% of available credit against 43.8% for GPT-5.6 Sol.
Epoch's new index scores chips in GB300 equivalents for agentic serving; GB300 gives about 25 times H100's capacity with 2.5 times the FLOP/s.
GDP.xlsx pairs 70 workplace requests with real workbooks and strict rubrics; Gemini 4 Argon tops 20 configurations at 38.3% pass@1.
DeepMind released SatWeave satellite imagery and the PARDIG global precipitation map, both openly available on Google Cloud Storage.