Epoch AI published a paper proposing a statistical framework that stitches benchmarks together into a common scale, aimed at a structural problem: most benchmarks saturate too quickly to study long-run trends in AI capability.
The reasoning is simple enough to be uncomfortable. A really bad model scores 0% on a benchmark — but so does an extremely bad one, so the benchmark cannot distinguish them. A really good model scores 100%, and so does an extremely good one. A benchmark therefore only separates models in its middle range, and since models improve quickly, each benchmark provides a usable signal for a short window before it stops measuring anything. Stitching overlapping benchmarks lets that narrow window be extended into a longer curve, with consequences the authors tie to estimates of algorithmic progress and to forecasting.
Treating the measurement instrument as the object of study is the move behind evals moving from leaderboards to operating diagnostics, and it depends on the integrity conditions in double-blind frontier evaluations and the minimum integrity stack for an agent leaderboard. A stitched scale inherits every weakness of the benchmarks it joins.
This is a paper with published source code rather than a settled method, and the stitched trend depends on modelling assumptions about how benchmarks overlap. The condition to watch is whether independent groups reproduce the same long-run curve from different benchmark sets, because a scale that only one framework can compute is a forecast rather than a measurement.