What happened
Cognition announced FrontierCode 2.0 on October 27, 2026 as the next generation of its coding benchmark series. The company frames the release around a limitation of common coding evaluations: models can perform well when a prompt specifies the intended approach, edge cases, and acceptance tests in detail, while still struggling with user intent, repository conventions, unstated constraints, and the longer-term consequences of a change.
The benchmark is intended to measure functional correctness alongside three less bounded forms of engineering work: interpreting and expanding an initial user prompt, making reasonable design and scope decisions, and integrating a change smoothly into an existing codebase. Cognition describes recurring implicit requirements such as reusing existing abstractions, following repository conventions, and writing nontrivial tests that catch bugs. FrontierCode 2.0 calls this dimension “thoroughness”: conditions an agent should infer from the prompt and codebase, rather than receive as an explicit instruction.
Compared with FrontierCode 1.1, the new set uses tasks from 55% more repositories and includes 43% more blocking criteria. Reference solutions modify 67% more files and 89% more lines of code on average, in codebases that are 25% larger. Tasks now span more categories, including security patches and performance optimizations. Cognition says it also audited prompts, reference solutions, and grading criteria more rigorously. In 80% of FrontierCode 2.0 tasks, the reference solution differs from the upstream pull request, versus 60% in FrontierCode 1.0; the company says review found subtle edge cases and, in one automated audit pipeline, identified a security vulnerability in an upstream implementation that was subsequently reviewed and reported to maintainers.
More than 20 open-source developers each spent over 40 hours per task building the evaluations from repositories they maintain, according to Cognition. Scores are weighted aggregates of rubric items; a solution that misses a blocking criterion receives zero. Cost is reported as mean US-dollar spend per rollout at list API pricing, including public discounts. Cognition says each model is shown at its best reasoning effort. Claude Opus 5.5 leads the published FrontierCode 2.0 results at 45.5%, followed by GPT-6 Astra at 42.8%; Cognition emphasizes that even the highest score remains below 50%.
What the archive adds
This is a new version of a known story rather than a restatement. The earlier Field Note, Benchmark grades mergeability using rubrics from 20+ open-source maintainers, not just unit tests., established FrontierCode’s mergeability thesis. Artificial Analysis launched a leaderboard for Cognition's FrontierCode 1.1 benchmark. covered comparison and cost awareness around the prior version. FrontierCode 2.0 adds a larger, more demanding benchmark and an explicit measure of inferred thoroughness.
Trend signal
The release supports Goal-scoped loops replace manual continuation, but indirectly. A coding agent can only be judged against a goal when the goal includes externally reviewable constraints beyond a plausible-looking patch. FrontierCode 2.0 operationalizes part of that problem through rubrics and blocking criteria, while showing that incomplete specifications remain difficult for current systems.
Why it matters
Teams should treat this as evidence about evaluation design, not as a procurement verdict. A coding agent that passes explicit tests can still select an unsuitable abstraction, fix only one instance of a repeated bug, omit a necessary test, or violate local conventions. Those failures are expensive because they move the burden from code generation to review, integration, and later maintenance.
The practical implication is to evaluate agents in repositories that resemble the work they will actually perform. Include repository-specific acceptance criteria, design constraints, regression tests, and review of cases that a competent engineer would infer without being told. FrontierCode’s low top score is also a caution against assuming an autonomous agent will reliably turn a brief ticket into a production-ready change without a verification process.
What happens next
- Coding-agent evaluations will increasingly publish repository-specific rubrics alongside pass rates. (6 months, 65%). Watch for: benchmarks adding explicit blocking criteria and mergeability-style scoring.
- Vendors will focus more on agent behavior under incomplete specifications. (12 months, 60%). Watch for: releases that measure convention-following, scope decisions, and regression coverage.
Limits and uncertainty
FrontierCode 2.0 is a benchmark produced and published by Cognition, so its task selection, rubrics, model configurations, and cost calculations should not be treated as independent evidence of commercial-agent performance. The supplied material reports aggregate scores but does not establish how results transfer to a particular company’s repositories, languages, development practices, security requirements, or human review process. It also does not provide enough information here to reproduce every evaluation or determine whether ranking differences are statistically robust. A 45.5% leading score indicates difficulty under this setup, not a universal failure rate for coding models.
Skill to build now
Create a small internal “mergeability” evaluation from three recently completed changes. For each, retain the original short request, identify requirements that were only implicit in the repository, and write blocking checks for regressions, conventions, and scope. Run candidate agents against the same environment, then review failures by category. This produces a decision-relevant complement to generic benchmark scores.
Reader takeaway
FrontierCode 2.0 argues that the hard part of coding-agent evaluation is no longer just whether code runs. It is whether an agent can infer and satisfy the engineering obligations surrounding a terse request. For builders, the relevant response is better repository-grounded evaluation and verification, not blind reliance on a leaderboard.
Open question
Can public benchmarks make their repository-specific rubrics and task construction transparent enough to distinguish genuine engineering judgment from optimization to a particular evaluation setup?