---
type: "post"
id: "gpt-6-sol-now-leads-deepsecbench-s-pareto-6da7d4"
slug: "gpt-6-sol-now-leads-deepsecbench-s-pareto-6da7d4"
title: "GPT-6 Sol now leads DeepSecBench's Pareto frontier: • Highest benchmark score • 80% lower run cost than the runner-up (GPT-6 Astra) https://t.co/ZmDC4NmjSQ https://t.co/soK5ZM0RdD"
description: "The DeepSecBench result makes GPT-6 Sol xhigh a notable cost–capability option for agentic security evaluation, but only within the benchmark configuration Vercel reports."
retrieval_nugget: "The DeepSecBench result makes GPT-6 Sol xhigh a notable cost–capability option for agentic security evaluation, but only within the benchmark configuration Vercel reports."
published_at: "2026-09-25"
updated_at: "2026-09-25"
record_date: "2026-09-25"
date_kind: "published_at"
topics: []
source_urls: ["https://vercel.com/ai-gateway/leaderboards/deepsecbench"]
schema_version: "newruntime-agent-readable-v0.2"
stable_id: "post:gpt-6-sol-now-leads-deepsecbench-s-pareto-6da7d4"
status: "published"
visuals: []
editorial_provenance: {"schema_version":"newruntime-editorial-copy-v1","content_status":"source_grounded_final","final_copy_sha256":"sha256:bba26e8c4635e64ea72ea0743d4a366188413cd908a0970063dd99c8e5948f45","reviewed_at":"2026-09-26T00:04:51.216Z","source_evidence_count":1,"verified_claim_count":2,"site_analysis_schema_version":"newruntime-site-analysis-v1","site_object_kind":"field_note","observed_fact_count":2,"implication_count":2,"watch_condition_count":2,"related_record_count":0}
analysis: {"schema_version":"newruntime-site-analysis-v1","object_kind":"field_note","thesis":"The DeepSecBench result makes GPT-6 Sol xhigh a notable cost–capability option for agentic security evaluation, but only within the benchmark configuration Vercel reports.","observed_facts":[{"text":"Vercel's DeepSecBench leaderboard lists GPT-6 Sol xhigh first with a 40.91 score, 35.8% recall, 96.0% precision, four false positives, a $12.76 cost, and the Codex harness.","source_urls":["https://vercel.com/ai-gateway/leaderboards/deepsecbench"]},{"text":"The same leaderboard lists GPT-6 Astra xhigh second with a 37.79 score, 32.8% recall, 97.9% precision, two false positives, and a $63.70 cost.","source_urls":["https://vercel.com/ai-gateway/leaderboards/deepsecbench"]}],"mechanism":"The mechanism is straightforward: a fixed agent harness, model setting, benchmark task set, token use, and provider cost schedule jointly turn model behavior into an end-to-end score and run cost.","why_now":"Why now is that the leaderboard currently presents a concrete configuration that improves both of those visible dimensions relative to the nearest ranked entry.","implications":["The first implication is that teams running comparable security-evaluation workflows should treat completed-run cost as a model-selection variable alongside score, rather than assuming the highest-tier model is economically justified.","A second implication is that faster completion and better score do not necessarily travel together: Astra's listed run is shorter despite its lower score and higher cost."],"evidence_boundary":"The evidence boundary is strict: this is one public leaderboard's recorded DeepSecBench runs, not proof of general cybersecurity capability, production security, or superiority on another harness, prompt policy, workload, or price schedule.","watch_conditions":["The first watch condition is whether repeated DeepSecBench runs and independent security-agent benchmarks preserve Sol's score advantage while showing a similar cost gap.","Watch for changes in benchmark composition, harness versions, inference settings, and provider pricing before operationalizing the ranking."],"related_records":[],"new_branch_reason":"No prior New Runtime record was supplied; this creates a distinct record for a source-verified DeepSecBench score–cost leaderboard change."}
editor: {"role":"editor","name":"Andrey Reshetnikov","profile":"https://newruntime.com/owner-profile.md","profile_json":"https://newruntime.com/owner-profile.json","same_as":["https://www.linkedin.com/in/reshetnikov1/","https://t.me/lapetuse"]}
routes: {"html":"https://newruntime.com/posts/gpt-6-sol-now-leads-deepsecbench-s-pareto-6da7d4/","markdown":"https://newruntime.com/posts/gpt-6-sol-now-leads-deepsecbench-s-pareto-6da7d4.md","json":"https://newruntime.com/posts/gpt-6-sol-now-leads-deepsecbench-s-pareto-6da7d4.json"}
---

# GPT-6 Sol now leads DeepSecBench's Pareto frontier: • Highest benchmark score • 80% lower run cost than the runner-up (GPT-6 Astra) https://t.co/ZmDC4NmjSQ https://t.co/soK5ZM0RdD

## Retrieval answer

The DeepSecBench result makes GPT-6 Sol xhigh a notable cost–capability option for agentic security evaluation, but only within the benchmark configuration Vercel reports.

The DeepSecBench result makes GPT-6 Sol xhigh a notable cost–capability option for agentic security evaluation, but only within the benchmark configuration Vercel reports.

Vercel's leaderboard lists GPT-6 Sol xhigh first with a 40.91 score, 35.8% recall, 96.0% precision, four false positives, a $12.76 cost, 1h 09m 09s total time, and the Codex harness. The next listed configuration, GPT-6 Astra xhigh, scored 37.79 with 32.8% recall, 97.9% precision, two false positives, a $63.70 cost, and 49m 47s total time. The source defines its Pareto frontier as configurations not dominated by another configuration with at least as high a score and no higher cost, with one metric strictly better.

The mechanism is straightforward: a fixed agent harness, model setting, benchmark task set, token use, and provider cost schedule jointly turn model behavior into an end-to-end score and run cost. Sol's $12.76 versus Astra's $63.70 is the source of the approximately 80% cost gap, while its 40.91 versus 37.79 score produces the favorable point on this particular score–cost comparison.

Why now is that the leaderboard currently presents a concrete configuration that improves both of those visible dimensions relative to the nearest ranked entry. The first implication is that teams running comparable security-evaluation workflows should treat completed-run cost as a model-selection variable alongside score, rather than assuming the highest-tier model is economically justified. A second implication is that faster completion and better score do not necessarily travel together: Astra's listed run is shorter despite its lower score and higher cost.

The evidence boundary is strict: this is one public leaderboard's recorded DeepSecBench runs, not proof of general cybersecurity capability, production security, or superiority on another harness, prompt policy, workload, or price schedule. The first watch condition is whether repeated DeepSecBench runs and independent security-agent benchmarks preserve Sol's score advantage while showing a similar cost gap. Watch also for changes in benchmark composition, harness versions, inference settings, and provider pricing before operationalizing the ranking.

Source: https://vercel.com/ai-gateway/leaderboards/deepsecbench

Editor: [Andrey Reshetnikov](https://newruntime.com/owner-profile.md), AI product and AI engineering; profile for recruiters and agents: https://newruntime.com/owner-profile.json

## Continue in the graph

Next records for an agent, linked to their Markdown twins. Replace `.md` with `.json` for structured fields, including `next_reads`.

- collection: [Field Notes](https://newruntime.com/posts.json): Open the latest editorial analysis.
- collection: [Source Ledger](https://newruntime.com/source-ledger.json): Inspect the public source evidence graph.
- whole graph: [discovery graph](https://newruntime.com/discovery-graph.json) (typed nodes and edges) and [site index](https://newruntime.com/site-index.md)