What happened
Artificial Analysis updated its Intelligence Index to version 5.0. The company’s methodology page presents the index as a composite evaluation across four categories: Knowledge Work, Coding, Science, and General. In v5.0, each category carries a 25% weight. That is a meaningful framing choice: the index is not simply a general reasoning score, but an aggregate intended to combine several kinds of model work.
The new version adds or elevates several evaluations. HLE-Diamond accounts for 10% of the index. Terminal-Bench-Science 0.1 accounts for 15%, bringing an agentic science evaluation into the weighted core. Cognition’s FrontierCode 2.0 also receives a 15% weight, adding a coding benchmark whose premise is closer to whether a proposed change satisfies repository-specific expectations for mergeability than whether it merely passes a narrow test. Artificial Analysis also upgrades AA-Omniscience to v1.1.
Other components have changed position or disappeared. GDP.pdf falls from 10% to 5%. CritPt is replaced by CritPt v2, using its Hard Subset in the index. The v5.0 description also says that GDPval-AA, SciCode, AA-LCR, and Humanity’s Last Exam are removed from the index. The methodology page continues to list many additional evaluations separately from the index itself, including task-specific measures for knowledge work, coding, science, and general capability. In other words, being documented by Artificial Analysis does not by itself mean an evaluation contributes to the headline composite score.
The most consequential methodological change is access to the test material. Artificial Analysis says that a majority of the index—about 60%—now uses private test sets or held-out answers. That choice changes what an index result represents. Private material can reduce the chance that a model has been trained directly on public benchmark questions or tuned around their known scoring rules. But it also means outside teams cannot independently inspect the full task set, rerun it, or establish whether a score transfers to their own environment.
For users reading the ranking, v5.0 should therefore be understood as a new composite rather than a continuous, directly interchangeable extension of earlier versions. Its scores reflect a revised mixture of tasks, weights, benchmark versions, and visibility into the underlying answers. It is useful as a screening signal; it is not evidence that one model will automatically win on a particular company’s codebase, tool permissions, or operational workflow.
What the archive adds
This is a new fact on a known benchmark story. The archive covered FrontierCode 2.0, including its expanded task scope and stricter mergeability criteria. It also covered an Artificial Analysis leaderboard for FrontierCode 1.1. The v5.0 update matters because it incorporates FrontierCode 2.0 into a broader index alongside science and general evaluations.
Trend signal
This supports verification bandwidth is the scarce engineering resource. FrontierCode’s mergeability-oriented framing and the index’s move toward held-out material both shift attention away from easily generated answers and toward evidence that output can survive a stronger check. The index does not solve verification for a team, but it makes that constraint more visible in model comparison.
Connections to other events
An earlier archive item described Terminal-Bench-Science 0.1 as a benchmark for scientific agents that emphasizes terminal environments and reproducible artifacts. Its inclusion at 15% in the new index connects agentic scientific work to the same composite comparison surface as coding, knowledge work, and general capability. The archive’s FrontierCode coverage supplies the parallel coding side: judging changes against repository-specific mergeability expectations rather than only unit-test completion.
Why it matters
Teams often need a fast way to narrow a long model shortlist. A composite index can help, but v5.0 makes the trade-off explicit. It is more oriented toward tasks that resemble agentic coding and science workflows, and private or held-out material may make the comparison less vulnerable to public benchmark optimization. At the same time, a team has less ability to inspect the test distribution and decide whether it resembles its own work.
The practical implication is to use the index for candidate selection, not final procurement or deployment decisions. A team that needs repository changes should separately test merge criteria, permissions, review burden, and rollback behavior. A team building scientific workflows should separately test whether outputs create inspectable artifacts in its own environment. The headline score cannot answer those questions because its task mix is necessarily general.
What happens next
- Model vendors will increasingly emphasize performance on held-out or private agent evaluations. (6 months, 65%). Watch for: more benchmark methodology pages that withhold task answers or test sets.
- Teams will pair composite leaderboard shortlists with internal task suites before selecting a production model. (12 months, 70%). Watch for: published evaluation guidance that separates discovery benchmarks from deployment gates.
Limits and uncertainty
The supplied material establishes the v5.0 composition and the stated use of private tests or held-out answers; it does not provide the full contents of those private sets, their contamination controls, confidence intervals, or error analysis. It also does not establish that the index’s rankings predict performance in a particular organization. The reported weighting shows how Artificial Analysis combines evaluations, not how much each benchmark resembles every production setting. Nor does the material establish why particular models may rise or fall after the revision. A score comparison across versions should therefore be treated cautiously.
Skill to build now
Create a small internal evaluation pack for the decision your team is actually making. Select five to ten representative tasks, define a pass condition before running models, and retain the artifacts needed to review the result: diffs, commands, citations, outputs, and failure notes. Run the same pack for the models surfaced by a general index. This turns a leaderboard into a shortlist and makes the final choice accountable to your own constraints rather than to an opaque aggregate.
Reader takeaway
Artificial Analysis v5.0 is not just another leaderboard refresh. It changes the task mixture toward coding and science agent evaluations and relies more heavily on non-public test material. Use it to find models worth testing, then make the production decision with evidence from your own tasks and review process.
Open question
Will greater reliance on private evaluations improve resistance to benchmark gaming enough to outweigh the reduced ability of independent teams to audit what the score measures?