On November 9, 2026, Epoch AI introduced the Inference Performance Index (IPI), a way to compare AI hardware by how many requests a chip can serve at once across a range of required output speeds, rather than by FLOP/s alone. Version 1 targets agentic workloads, combines results from four representative models, and expresses scores in GB300 equivalents with GB300 set to 1. Performance is modeled with NVIDIA’s AISimulate from arithmetic throughput, memory capacity and bandwidth, and interconnect, over 50 to 400 tokens per second per request, assuming a 262,144-token cached context.
The headline result is that GB300 provides about 25 times H100’s inference capacity per chip on these workloads with only 2.5 times the 8-bit FLOP/s, so a FLOP/s comparison would understate its advantage roughly tenfold. Listed scores are 0.12 for H200 HGX, 0.54 for B300 HGX, about 0.55 for AMD’s MI355X, 1 for GB300 NVL72 and 2.18 for Vera Rubin VR200. GB300 NVL72 has about 1.9 times the per-GPU capacity of B300 HGX, which Epoch attributes to its 72-GPU high-bandwidth group versus eight. Measured GB300 throughput on Kimi K3 tracked predictions at 50% of modeled speed-of-light performance within about 12% on average.
Why it matters: Most compute estimates, including counts of H100 equivalents, are FLOP-based. If inference capacity scales with memory and interconnect far faster than FLOP/s, those estimates understate how much serving capacity new hardware adds.
What it does not show: Scores are modeled hardware potential and ignore software maturity. Epoch notes that a geometric mean would substantially lower Hopper’s scores and that scores are only comparable within one IPI version.