What happened
Artificial Analysis has published an evaluation page and leaderboard for HLE-Diamond. HLE-Diamond is a refined 1,000-question version of Humanity's Last Exam, produced by the Center for AI Safety and Scale AI after a year of cleaning and revising the original questions with input from research communities. The benchmark is intended to test expert-level reasoning and knowledge using closed-ended academic questions.
The set is evenly divided into 500 reasoning questions and 500 knowledge questions. Of the 1,000 questions, 272 include an image. Artificial Analysis says it runs every question without tools. Each model receives the official HLE system prompt and answers each question once. The service does not display example questions because the dataset authors ask users not to share or redistribute HLE-Diamond questions.
Scoring is pass@1: one response is evaluated for each question, and the reported score is the share of questions judged correct. Artificial Analysis uses GPT-5.6 Luna (medium) as the judge, with the official HLE judge prompt, comparing a model response with the reference answer. The evaluator allows a small margin for numerical answers. Questions for which there is no valid answer count as wrong. Models capable of accepting images receive the image and text; models that cannot accept images receive only the text for the 272 image questions, but are still graded across all 1,000 questions.
The current leaderboard names Gemini 4 Carbon (High) as the highest-scoring listed model at 69.1%, followed by Claude Opus 5.5 (Max, Default Fallback) at 67.9% and Claude Opus 5.5 (Xhigh, Default Fallback) at 64.0%. Gemini 4 Carbon also leads the separate reasoning and knowledge partitions at 79.6% and 58.6%, respectively. The page presents more than an aggregate ranking: it includes score by partition, output tokens per task, cost per task, time per task, score versus cost, and score versus release date. Artificial Analysis describes the cost figure as an average per evaluation task, broken down where canonical token counts are available into input, cache-hit, cache-write, reasoning, and answer-token pricing. Time per task is a weighted average decode time and excludes time to first token and overhead.
The result is an independently operated presentation layer over HLE-Diamond rather than a new model release. Its value is in making a fixed, difficult test comparable across model variants and in exposing the resource trade-offs beside scores. The primary source is the Artificial Analysis HLE-Diamond benchmark page.
What the archive adds
This is a new fact on a known story: Artificial Analysis launches Intelligence Index v5, with 60% of the score on private test sets covered the broader v5 release and its revised composite methodology. HLE-Diamond is the specific evaluation surface now visible within that ecosystem, with its own per-task operational metrics.
Trend signal
This offers limited support for Agent economics moves to completed work. The leaderboard reports cost and time per benchmark task beside quality, rather than treating token price as the only economic measure. But an HLE-Diamond question is an academic test item, not a verified production outcome, so the page is an early proxy rather than evidence that the broader shift has been solved.
Why now
The benchmark page sits alongside Artificial Analysis's Intelligence Index v5 evaluation surface. The archive indicates that v5 expanded and changed the composite index, including a greater role for private or held-out tests. Publishing a dedicated HLE-Diamond view makes one of the constituent comparison surfaces inspectable on its own terms.
Connections to other events
The archive's Decision Index v1.0 coverage described a separate effort to evaluate bounded decisions through more than one aggregate measure. The HLE-Diamond page follows a similar design instinct: retain a headline score, but expose partitions and operating characteristics. It is not the same task class, however; HLE-Diamond tests closed-ended academic reasoning and knowledge without tools.
Why it matters
Model-selection discussions often collapse into a single public score or a token-price table. This leaderboard makes the trade-off more visible: a team can inspect an overall pass@1 result alongside reasoning versus knowledge results, output-token volume, estimated task cost, and decode time. That is useful when a system has a workload resembling difficult question answering and when latency or inference budget constrains deployment.
It should not be used as a direct procurement answer. Production agents commonly need tool use, access controls, multi-step execution, recovery from failures, and domain-specific evaluation. None of those are established by this test, because every HLE-Diamond question is answered once without tools. Builders should treat the leaderboard as a screening signal for model candidates, then run their own representative tasks and review their failure modes.
What happens next
- More model variants will receive HLE-Diamond results. (3 months, 75%). Watch for: additional entries and refreshed score-versus-cost views on the leaderboard.
- Teams will use partition-level results to qualify headline rankings. (6 months, 60%). Watch for: model comparisons that distinguish reasoning and knowledge performance rather than citing only the aggregate score.
Limits and uncertainty
The source establishes the evaluation procedure and the scores displayed at the time of publication; it does not establish that the leading model is best for software engineering, browsing, tool use, enterprise retrieval, or autonomous-agent work. A single pass@1 attempt can differ from workflows that use retries, ensembles, or external verification. The scoring also depends on an LLM judge, GPT-5.6 Luna (medium), even though it uses the official judge prompt and reference answers. The page does not provide the underlying questions, by the dataset authors' request, which limits independent inspection of individual failures. Models without image input are evaluated across image-containing questions after receiving only text, which also affects comparisons.
Skill to build now
Create a small internal model-selection sheet for one real workflow. Record success on representative tasks, median completion time, tokens or cost per completed task, and the specific failures a reviewer had to correct. Then compare those columns with public benchmark signals such as HLE-Diamond's reasoning, knowledge, cost, and time views. This makes public leaderboards a hypothesis generator instead of a substitute for evaluation.
Reader takeaway
HLE-Diamond gives builders a richer public comparison of frontier models on hard closed-ended academic questions: one score, two task partitions, and operational metrics. Use it to narrow a candidate set, not to declare a universal winner. The decisive test remains whether a model completes your constrained, tool-using work reliably enough to justify its cost and latency.
Open question
Will the benchmark's score, cost, and time views remain predictive once models are placed in agent harnesses that use tools, retries, and task-specific verification?