Verdict: partly held. The case for expert-supervised, verifiable calculations has gained support, but the evidence has moved beyond the original bounded-task question without independently establishing broad or autonomous discovery.
Original analysis: The physics speedup depends on a result experts can check, published 2026-10-03 (7 weeks ago).
What we said
New Runtime’s earlier coverage, published October 3, treated Matthew Schwartz’s two-week calculation as a promising but conditional example: the task had established methods and numerical checks, and an expert supplied judgment to guide and verify the work. It recommended testing similarly bounded calculations, keeping code and derivations inspectable, and tracking review and correction time as well as completion time. The note did not claim that AI had demonstrated autonomous discovery or that the reported speedup would generalize.
What happened since
- An October study tested agents on 50 unpublished theoretical-physics problems with verifiable answers. Its best setup answered 49 correctly with web access and 46 without; the authors said this supports agents’ ability to handle many concrete, known-method calculations, but distinguished those tasks from breakthrough discovery and noted possible memorization and answer-checking limits. That is evidence for the narrow setting New Runtime emphasize [1]
- An October paper argued that formalization can make AI-assisted physics results easier to check and reuse, while warning that a machine-checked proof does not by itself show that the formal definitions capture the intended physics. Its emphasis on inspectable derivations, expert review and community-reviewed formal libraries echoes the earlier note’s insistence on field-standard validation; it also makes clear that verification has more than one [2]
- On November 17, Anthropic reported two AI-assisted theoretical-physics results involving black holes: a calculation of the Hawking–Page transition in a quantum regime and a limit for a version of the modular bootstrap method. Anthropic says the bootstrap result was checked in Lean, multiple agents reached it independently, and a physicist checked the main results; it also reports a separate physicist’s solution by a different method for the first [3]
- The October 5 follow-up preserved the earlier account’s central caveat: Schwartz’s reported calculation was notable not just for speed but because an expert could judge the task and result. It repeated the reported draft, token and compute figures, which remain Schwartz’s account rather than independently verified performance measures. The passage available here ends before the follow-up’s full argument, so it does not establish that New Runtime [4]
What we missed
The earlier note was careful about the limits of a single, bounded calculation, but the landscape moved quickly: within weeks, a benchmark study offered broader evidence for known-method tasks, and a company announced claims of more novel results with formal checks and human review. Those developments do not prove that the original advice was wrong; they show that the question is no longer only whether a model can help an expert complete a checkable calculation. We still lack independent confirmation in the listed reporting of the newer claims, and the benchmark’s strong scores do not establish autonomous discovery, faithful physical formalization or time saved in real research groups.
Forecast check
- At least one independent group will publish an AI-assisted theoretical-physics calculation with an explicit account of validation and human effort. (45%, due 2027-10-03): unclear. The Anthropic report describes physics results and validation, but the reporting does not establish that an independent group published them. The forecast is due October 3, 2027, so it is not yet settled.
- Public reporting on AI-assisted research will increasingly include intermediate checks and human oversight alongside final outputs. (60%, due 2027-04-03): on track. The benchmark study reports answer-checking caveats, and Anthropic describes formal checking, independent routes and physicist review. These are early examples of reporting on checks and oversight, though the forecast remains open until April 3, 2027.
What to do now
Builders should keep the original discipline, but apply it to a wider range of claims. For bounded calculations, define expected intermediate outputs and independent checks before the model starts, preserve inspectable code and derivations, and record expert correction and review time alongside compute and calendar time. For a claimed novel result, ask separately whether the formal statement matches the intended physics, whether another method or group can reproduce it, and what part of the work depended on human direction. Treat benchmark accuracy as evidence about the tested problems, not a forecast of breakthrough capability. Treat company-reported discoveries as leads to evaluate, not settled demonstrations, until independent researchers can check both the result and the underlying reasoning.
Next check
The most informative next observation is whether an independent research group reproduces one of the reported novel physics results, with the physical definitions, validation route and human contribution made clear.