The Cyber Index ranks providers' refusal policies as much as models' defensive skill

What we checked

• How do the Cyber Index's evaluation results, safety blocks, and access constraints affect the interpretation of its leading model scores? confirmed. At launch, the top scores mostly reflected which models were willing to attempt the tasks. The public GPT-6 Sol, GPT-6 Astra, Claude Opus 5.5, Claude Fable 5.1 and Gemini 3.8 Flash declined tasks on safety grounds and trailed the leaders by 19-31 points. Declined tasks score zero. The trusted-access GPT-6 Sol (Daybreak Blue) had no safety blocks, scored 32 points above the public version and took first place. Artificial Analysis says the largest gain came on CyberGym-E2E, the component with the most refusals. [1][4][5]
• Does the Index treat refusals as a separate outcome, or does it fold them into the score? partly confirmed. Both. The methodology page says safety declines are 'recorded separately from scores', and refusal rates are published as their own metric. But the leaderboard page says a declined task scores zero, so refusals still pull the headline number down. [7][1]

What we found

• The launch leaders were Grok 4.7 (xhigh) and MiMo-V2.6-Pro, tied at 56, followed by GPT-6 Luna (max) at 53. Independent trade coverage repeats these numbers and adds GLM-5.3-Flash at 50. On CyberGym-E2E-AA it reports MiMo-V2.6-Pro at 79%, GPT-6 Luna at 78% and Grok 4.7 at 74%. Cost per task varies widely: Artificial Analysis cites $11.67 for Grok 4.7 (xhigh) as a more expensive comparison point. [1][2][3]
• Refusals are concentrated in CyberGym-E2E-AA, which asks for a crashing proof-of-concept input on C/C++ projects, followed by a patch and passing tests. According to the launch article, GPT-6 Astra, GPT-6 Sol, Claude Fable 5.1, Claude Opus 5.5 and two Qwen3.8 models 'refuse at least 98% of tasks' there, which 'makes frontier comparison difficult'. On the leaderboard, a task the model or provider declines on safety grounds scores zero. [4][1][2]
• On October 8, Artificial Analysis added trusted-access models, starting with GPT-6 Sol (Daybreak Blue, max). It 'hits no safety blocks across the entire Index', beats the public GPT-6 Sol (max) by 32 points, now leads the Index and costs $1.77 per task. The leaderboard marks such entries as 'trusted-access model (not publicly available)'. The text we could read did not include exact per-component scores; those are only in charts. [5][1]
• OpenAI's documentation shows Daybreak is gated behind enterprise approval. Teams need organization approval and project setup, then select behavior with an access_programs.cyber parameter (standard, daybreak_blue, daybreak_red). The docs say the parameter 'doesn't grant access' and that selecting a program doesn't guarantee every request will succeed. They also say reduced refusals on gpt-6.1-sol and gpt-6-astra require Daybreak Red. [6]
• The methodology matters when comparing scores. CWE-Bench-AA has 120 held-out OWASP Top 10 (2025) tasks scored pass@1 by a deterministic verifier, with a 2-hour limit per task. DeepsecBench-AA reports median F2 over three runs, judged by GPT-5.6 Sol. CyberGym-E2E-AA has 131 tasks with a 90-minute limit, and whether the patch fixes the intended bug is only diagnostic. Artificial Analysis warns that results aren't directly comparable to the original benchmark leaderboards. [7]
• Even the leading models show clear weaknesses. The best model found only 41% of the expert-verified issues in DeepsecBench-AA. On CyberGym, 42% of attempts hit the 90-minute limit, and 31% of passes patched an off-target crash instead of the intended bug. Partial fixes made up 55% of failed CWE-Bench-AA attempts. [4]

What it means

Treat the Cyber Index as a combined measure of capability and policy, not a pure capability ranking. If you're choosing a model for an internal vulnerability-triage or auto-patching pipeline, look at the score, the safety-block rate and the access tier together. A low score for a public frontier model may mostly mean it refused the tasks. A top score from a trusted-access variant doesn't mean you can get that behavior through a standard API key. Some lower-cost open-weight or less-restricted models, such as MiMo-V2.6-Pro and GPT-6 Luna, scored near the top without special access. That makes them practical defaults, but they still need your own sandboxing and governance.

Before you commit, run your own tasks on Stirrup or your own harness against your codebase and track refusals separately. Also check patch correctness yourself: the benchmark data shows models often fix a shallower, off-target crash and break existing functionality. If you need frontier models for memory-safety work, the realistic path is applying to programs like OpenAI's Daybreak. Plan for the enterprise onboarding and the per-request access parameter.

Still open

• What are the exact per-component scores for GPT-6 Sol (Daybreak Blue) and the public GPT-6 Sol? They appear only in charts we couldn't read as text.
• Will Anthropic, Google or other labs submit trusted-access variants? The October 8 article doesn't mention any.
• The leaderboard's text summary still named Grok 4.7, MiMo-V2.6-Pro and GPT-6 Luna as the top three. It's unclear whether trusted-access entries are left out of some default views.
• Are trusted-access runs set up identically to public runs (same prompts, harness and reasoning effort)? The trusted-access article doesn't say.