On September 11, 2026, researchers released “Tasks over Application Manuals” (TAM, arXiv:2609.13005), a benchmark built from two real-world, rule-heavy domains: ICD-10-CM clinical coding and U.S. federal sentencing guideline calculations. Each task requires an agent to navigate manuals containing tens of thousands of interdependent rules to reach a single correct answer, rather than answering a self-contained question.
Testing retrieval-augmented generation, ReAct-style prompting, and an agent-harness baseline built on GPT-5, the authors report exact-match accuracy of just 1 percent on ICD-10-CM coding and 15.5 percent on federal sentencing calculations. That is far below the near-saturated scores frontier models post on short-horizon coding and math benchmarks, despite the underlying models being the same class of system.
The paper’s argument is that existing agent benchmarks systematically overstate real-world readiness because they test short, self-contained tasks. Procedure-heavy work in medicine, law, tax, and compliance instead requires reliably chaining together thousands of interdependent rules over a long horizon, a capability current retrieval-plus-prompting techniques do not reliably provide, even when the underlying model is otherwise capable.
This is a direct warning for anyone piloting agents on regulated, rule-bound back-office work - clinical coding, sentencing, tax, insurance adjudication - where a 1 percent exact-match rate means the agent is nowhere close to usable without heavy human review. It does not say agents can never do this work; it says the current toolkit of retrieval plus prompting is not the mechanism that gets there, and that short-task benchmarks are the wrong instrument for predicting performance on it.