What happened
METR’s November 20 update presents capability evaluations of public models relevant to AI R&D acceleration, through Opus 5.5 and GPT-6 Astra. METR says it remains significantly uncertain both about the capabilities necessary for drastic acceleration and about the rate at which relevant capabilities are improving. It defines drastic AI R&D acceleration as a situation in which capability gains arrive quickly enough that people cannot meaningfully understand, assess, and react to catastrophic risks from new systems.
The organization evaluated models on three task suites. Time Horizon 1.1 contains 222 tasks, including work from SWAA, HCAST, and RE-Bench. MirrorCode contains 75 command-line software-replication tasks. TH2 contains 36 tasks in 12 task families. Its verifiable subset includes optimization, machine-learning, puzzle, game, and autonomous-gameplay work; its hard-to-verify and judgment subset includes open-ended research, assessment of another agent’s work, and conceptual reasoning.
METR says it used a ReAct-style scaffold with bash, Python, and image-viewing tools. Models generally ran at xhigh or higher reasoning effort, and token and time limits were selected with the aim of reaching a performance plateau under additional inference scaling. The tested runs typically included 15 models, among them GPT-4o, o1, GPT-5, several Opus versions, GPT-6 Astra, and Fable models.
METR reports that models usually match or exceed skilled humans on its coding and optimization tasks when those tasks offer strong feedback signals. In settings with data from more than 10 domain specialists, models tend to place at a high percentile, though not always above the best experts. In contrast, METR finds that agents display worse judgment than human experts where good choices must be made without clear empirical feedback. In one task, model judges overestimated output quality when the output came from a similarly capable or stronger model.
Across the combined Time Horizon 1.1 and TH2 suite, METR estimates that time horizons double roughly every 3.6 months. TH2 is harder than Time Horizon 1.1, so a given model’s 50% horizon is lower on the combined suite than on Time Horizon 1.1 alone. METR says its public-frontier composite, the METR Capability Index, shows no strong acceleration or deceleration among the tested models. It does see uncertain developer-specific changes under some conditions, but says those findings are sensitive to methodological choices. The update also reports that agent swarms appear helpful on verifiable tasks with a clear metric and fast feedback, compared with best-of-k, but have unclear or negative effects on other tasks.
METR additionally examines task lifecycle cost: its estimate of the cost of creating, running, and scoring an evaluation. On its data, models do somewhat worse on tasks that are more costly to turn into reinforcement-learning environments, even after controlling for estimated task length. Yet METR reports similar progress rates on lower- and higher-cost task subsets, including improvement over time on its costliest tasks.
What the archive adds
New Runtime’s earlier coverage, Independent Assessment of AI R&D Acceleration at Anthropic in October 2026, dated 2026-11-05, covered METR’s assessment of internal AI-model performance growth. This is a new fact on a known story: it adds public-model results across multiple suites, the 3.6-month estimate, and more detail on the distinction between verifiable performance and judgment.
Trend signal
This supports Verification bandwidth is the scarce engineering resource. METR’s results distinguish tasks with rapid, clear feedback from work where tests provide limited evidence that an output will generalize. Better model performance does not by itself create a trustworthy basis for accepting outcomes in judgment-heavy workflows; teams still need capacity to inspect, test, and own those outcomes.
Why it matters
The headline doubling estimate should not be used as a generic productivity forecast. It is a trend measured on a particular combined task suite. The more operational lesson is the difference between work an agent can evaluate continuously and work where success is delayed, ambiguous, or dependent on expert context.
A team can often give a coding agent a tight loop: a test suite, an optimization target, a reproducible failure, or a measurable quality threshold. METR’s findings suggest that this remains the kind of environment where capability gains are easiest to capture. By contrast, a system that selects an architectural direction, assesses research quality, or signs off on a change with incomplete signals needs a stronger review design.
The update also matters for model evaluation and procurement. METR warns that inexpensive, containerized, easily scored evaluations may resemble reinforcement-learning environments used in training. A high score on such work may therefore be a less reliable proxy for performance on a team’s distinct production tasks. Benchmark results remain useful inputs, but they should be paired with representative trials and explicit acceptance criteria.
Limits and uncertainty
METR says central questions remain unresolved: what capabilities are required for drastic AI R&D acceleration and how quickly those capabilities are improving. Its estimate relies on task suites rather than a direct measurement of all AI R&D work. Direct human baselines inform only 9% of its estimates for human completion times; for many other tasks, METR uses procedures informed by public records, expert input, and extrapolation.
The organization also says its developer-specific trend findings are sensitive to methodological choices. TH2 task coverage is more targeted than the broader software-engineering coverage of Time Horizon 1.1, and some tasks are manually graded or judged by models. These are METR’s results and robustness checks, not an independent replication.
Skill to build now
Create two queues for one agent workflow this week. Put tasks with executable tests, objective score targets, or rapid reversible feedback in an assisted-autonomy queue. Put tasks that require choosing among poorly specified alternatives, judging quality beyond test coverage, or predicting real-world generalization in a review-required queue.
For each task, write down the acceptance signal before the agent starts and record whether that signal is fast, reliable, and difficult to game. Use parallel or swarm-style attempts only when the metric is clear and feedback is rapid; METR’s update reports unclear or negative effects outside that setting. Review the second queue with a domain owner rather than treating model self-evaluation as final approval.
Reader takeaway
METR’s 3.6-month figure is a useful indicator of rapid improvement on its measured task horizons, not a universal clock for AI productivity. Build around the more durable distinction: give agents work with strong, fast verification, and retain accountable expert review when quality cannot be cheaply demonstrated.
Open question
Will future evaluations show that models can make reliable judgment under slow or incomplete feedback, or will improvement continue to concentrate in tasks with easily testable objectives?