These are results on a defined test set, not evidence that a model independently completed a research project.
Implication: Researchers can treat these systems as potentially lower-cost aids for bounded theory questions, while retaining human review and validation of the underlying work.
This starts a distinct analytical record because no prior New Runtime record is supplied for this specific benchmark and cost comparison.
Thesis
The study suggests that frontier models are making some theoretical research questions cheaper to investigate, while measuring answer agreement rather than autonomous scientific discovery.
Observed facts
The study used 254 recent arXiv theory questions across seven domains in mathematics, physics, and economics, with human-authored answers unavailable to models when answering.
1The reported share of problems with answers matching the human authors rose from 2.8% for o1 to 34.6% for GPT-5.5.
1The researchers estimate approximately $5 per attempt and $15 per successful match for GPT-5.5.
1
Mechanism
The researchers compare model answers with human-authored ground truth for recent theory questions and estimate the cost per attempt and per matching answer.
Why now
The study spans model releases from December 2024 to April 2026, making the reported change a longitudinal comparison rather than a single-model snapshot.
Implications
Researchers can treat these systems as potentially lower-cost aids for bounded theory questions, while retaining human review and validation of the underlying work.
The reported cost and answer-match rate provide a test-set-specific basis for comparing utility, not a universal measure of scientific productivity.
Evidence boundary
The supplied source reports agreement against known human answers across seven domains; it does not show that models generated novel results, completed an end-to-end research workflow, or generalize to all scientific tasks.
Watch conditions
Future evaluations should test whether gains persist on independently held-out questions and translate into verified research outputs.