What happened
Cognition’s FrontierCode leaderboard now reports FrontierCode 2.0 results. The company dates the current revision to October 27, 2026 and says it rebuilds the suite to evaluate user experience. The revised suite contains 132 tasks, spans more repositories and categories than before, adds categories including security patches and performance optimizations, applies higher mergeability standards with more emphasis on thoroughness, and uses larger task scope.
The benchmark’s stated target is “mergeability”: whether a maintainer would actually merge a proposed pull request. Cognition says its criteria cover end-to-end code quality, including correctness, test quality, scope discipline, style, and adherence to codebase standards. Each task is written by an open-source maintainer from the repository represented in the suite. More than 20 developers contributed tasks, spending more than 40 hours per task, and define what mergeable means for their own repository.
FrontierCode does not present one metric as the full result. Its score is a weighted aggregate of rubric items; a solution that fails blocking criteria receives zero. The leaderboard plots that score against cost. Cost is the mean US-dollar spend per rollout at list API pricing, including public discounts. A line connects a model’s runs at different reasoning efforts, and the chart describes the top-left as the efficient region. The site also offers linear and logarithmic views of the score-versus-cost scale.
Cognition says the evaluation combines unit tests, rubrics, and additional verifiers. Because rubric grading includes judgment, the company describes a quality-control process involving adversarial testing, calibration, multi-stage review, and manual review of every task by a Cognition researcher. Internet access is restricted but not eliminated: models may use it as an engineer would, for example to read documentation or search error messages. Runs detected consulting solution-bearing material, such as the original pull request, are scored zero.
The site illustrates the distinction with a task from Mock Service Worker. It says Claude Opus 5.5 can fix the stated reset-handlers bug while missing a user’s finally-cleanup code; on that task, all five listed reasoning-effort settings show zero passes out of five. This is an example of the benchmark’s blocking-quality logic, not a general performance claim about the model.
FrontierCode 2.0 follows prior methodology revisions. Version 1.1, dated July 7, 2026, added rules intended to distinguish legitimate internet use from unfair solution use, zeroed flagged runs, audited blocker criteria, and deprecated the Diamond subset. The original benchmark, dated June 8, 2026, introduced mergeability as its grading standard. Cognition’s October 27 changelog says earlier 1.0 and 1.1 rankings remain on their respective posts while the live results reflect the new suite.
What the archive adds
This is a new benchmark revision on a known story. Our note on Artificial Analysis’ FrontierCode 1.1 leaderboard described coding-agent comparison as more task- and cost-aware, while remaining bounded by benchmark design. FrontierCode 2.0 changes the underlying task suite and standards, so its new live results should not be treated as a simple continuation of the old rankings.
Trend signal
This supports Agent economics moves to completed work. FrontierCode explicitly puts mean rollout cost beside an assessment of whether a pull request is mergeable, rather than asking readers to infer economic value from token prices. It also supports Verification bandwidth is the scarce engineering resource: the benchmark adds human-defined criteria, blockers, review, and adversarial quality control because generated code alone is insufficient evidence of usable work.
Why now
The timing is explained by the revision itself. Cognition says FrontierCode 2.0 was rebuilt to evaluate user experience, with bigger tasks, more categories, and stricter standards for thoroughness. The October 27 update also moves the live leaderboard to the new suite while preserving prior-version rankings elsewhere, making the methodology change visible rather than silently mixing results.
Connections to other events
The earlier Artificial Analysis FrontierCode 1.1 coverage made the same broad comparison surface visible: task performance and cost, not only raw model capability. The new Cognition revision is not an independent validation of that earlier leaderboard; it is a changed benchmark suite with its own grading and reporting rules. Together, they show why a coding-agent ranking must always be read with its task set, cost assumptions, and evaluator in view.
Why it matters
Teams deploying coding agents need a selection process that resembles their actual acceptance process. FrontierCode’s design makes a useful distinction: a patch can appear correct on the reported bug yet still fail because it omits cleanup, violates scope expectations, or does not meet a repository’s standards. Its cost figure is also closer to an operational question than token price is, because it measures spend per rollout across a model’s reasoning settings.
That does not turn the leaderboard into a procurement answer. A team’s repositories, tests, review rules, tool permissions, and failure tolerance will differ from the suite’s. But the update provides a practical evaluation template: define blocking requirements, measure complete attempts, inspect failures, and compare cost only after the definition of acceptable output is fixed.
Limits and uncertainty
The source describes FrontierCode’s construction and reporting but does not establish that its 132 tasks represent every software environment or every engineering workflow. The score is a weighted rubric aggregate, and Cognition itself notes that points are shaped by the evaluation harness. The supplied material does not provide a complete current ranking, confidence intervals, raw run traces for every model, or evidence that benchmark cost will match a particular team’s bills. Public list API pricing, discounts, reasoning effort, internet-use rules, and the benchmark’s zeroing policy can all affect comparison. The example involving Claude Opus 5.5 is one task-level illustration, not a broad conclusion about that model.
Skill to build now
Take one recurring repository task this week and write a mergeability checklist before asking an agent to solve it. Include the bug or feature requirement, tests that must pass, forbidden scope expansion, cleanup or compatibility expectations, and the evidence a reviewer needs. Run at least two model or reasoning configurations against the same task, record total spend per attempt, and mark failures against the checklist. That produces a local score-versus-cost view whose blockers match the code your team actually owns.
Reader takeaway
FrontierCode 2.0 is useful less as a permanent model winner table than as a reminder of what to measure. For coding agents, the relevant unit is a reviewable, mergeable change at a known attempt cost. Read its results as evidence from a specific, revised evaluation harness—not as a universal ranking of models.
Open question
Will Cognition publish enough task-level evidence and stability information for teams to determine whether score and cost differences persist outside this 132-task suite?