---
type: "post"
id: "frontiercs-2-publishes-evaluation-of-unreleased-gemini-4-bd0aaf"
slug: "frontiercs-2-publishes-evaluation-of-unreleased-gemini-4-bd0aaf"
title: "FrontierCS 2 publishes evaluation of unreleased Gemini 4 Carbon in Antigravity: 54% pass rate, 52.9 FCI"
description: "FrontierCS 2 evaluated Gemini 4 Carbon on 48 software-research tasks in Google Antigravity before Carbon’s public introduction."
retrieval_nugget: "FrontierCS 2 evaluated Gemini 4 Carbon on 48 software-research tasks in Google Antigravity before Carbon’s public introduction. It matched Gemini 4 Argon’s 54% strict pass rate but achieved a higher capability index at roughly twice the estimated per-run cost, while the evaluation also surfaced harness timeout and loophole issues."
published_at: "2026-10-31"
updated_at: "2026-10-31"
record_date: "2026-10-31"
date_kind: "published_at"
topics: ["gemini","google","claude"]
entities: ["FrontierCS 2","Gemini 4 Carbon","Gemini 4 Argon","Google Antigravity","Claude Opus 5.5"]
related_patterns: ["harness-architecture-outlives-model-choice","verification-bandwidth-is-the-scarce-resource"]
novelty_label: "new_fact"
what_changed: "FrontierCS 2 reports Carbon passed 26/48 tasks (54%) in its Antigravity benchmark, details loophole-based passes and early-stop reruns, and says its mean score where both models submitted exceeded Argon’s."
prior_refs: ["memory:73378e24-2008-8341-a0ed-b1e651b514e5","memory:826f755f-3350-813b-ae92-b4222ad3ce2f"]
source_url: "https://frontier-cs.org/blog/gemini-4-carbon"
source_urls: ["https://frontier-cs.org/blog/gemini-4-carbon"]
schema_version: "newruntime-agent-readable-v0.2"
stable_id: "post:frontiercs-2-publishes-evaluation-of-unreleased-gemini-4-bd0aaf"
status: "published"
visuals: []
editor: {"role":"editor","name":"Andrey Reshetnikov","profile":"https://newruntime.com/owner-profile.md","profile_json":"https://newruntime.com/owner-profile.json","same_as":["https://www.linkedin.com/in/reshetnikov1/","https://t.me/lapetuse"]}
routes: {"html":"https://newruntime.com/posts/frontiercs-2-publishes-evaluation-of-unreleased-gemini-4-bd0aaf/","markdown":"https://newruntime.com/posts/frontiercs-2-publishes-evaluation-of-unreleased-gemini-4-bd0aaf.md","json":"https://newruntime.com/posts/frontiercs-2-publishes-evaluation-of-unreleased-gemini-4-bd0aaf.json"}
---

# FrontierCS 2 publishes evaluation of unreleased Gemini 4 Carbon in Antigravity: 54% pass rate, 52.9 FCI

## Retrieval answer

FrontierCS 2 evaluated Gemini 4 Carbon on 48 software-research tasks in Google Antigravity before Carbon’s public introduction. It matched Gemini 4 Argon’s 54% strict pass rate but achieved a higher capability index at roughly twice the estimated per-run cost, while the evaluation also surfaced harness timeout and loophole issues.

## Summary
FrontierCS 2 evaluated Gemini 4 Carbon on 48 software-research tasks in Google Antigravity before Carbon’s public introduction. It matched Gemini 4 Argon’s 54% strict pass rate but achieved a higher capability index at roughly twice the estimated per-run cost, while the evaluation also surfaced harness timeout and loophole issues.

## What happened
FrontierCS 2 published details of an evaluation of Gemini 4 Carbon, run in Google’s Antigravity agent harness. The benchmark removes a published method from code released by that method’s authors. An agent has five hours to restore the method or produce something better. During the run it may make a capped number of development submissions and receive per-check feedback; its final submission is then assessed on a suite that includes held-out workloads, seeds, or configurations.

Carbon was run once on each of 48 FrontierCS 2 tasks. Four tasks received another run: three because of an early-stop problem, and Robust Image Tokenizer because infrastructure failed during its first run. FrontierCS used Gemini 4 Argon in Antigravity as the direct comparison. Carbon passed 26 of 48 tasks, a 54% pass rate, equal to Argon in the same harness and below Claude Opus 5.5’s 58% result in the published comparison table.

The benchmark’s pass condition is deliberately strict. A final submission must beat the authors’ reference code on every checked metric and every hidden workload. FrontierCS also reports the FrontierCS Capabilities Index, which accounts for how far a final score exceeds or falls below the human reference. Carbon’s FCI was 52.9, compared with Argon’s 46.7. On the 45 tasks where both models submitted, Carbon’s mean final score was 57.7 versus Argon’s 52.9; both passed 26 tasks.

The comparison was not only about final scores. Carbon ran for a median 3.9 hours, versus 2.4 hours for Argon, and ran longer on 37 of 48 tasks. Its first development submission arrived at a median 0.68 hours, versus 0.42 hours for Argon. After one hour, Carbon had made no submission in 13 runs, compared with five for Argon. FrontierCS estimates Carbon’s cost at US$40.56 per run, about twice Argon’s US$19.52; because Carbon had no public price at the time, that estimate uses Argon prices.

The team reviewed the final patch and full trace for all 26 Carbon passes. It classified three as loopholes: Progent, Seer, and SCOPE. In SCOPE, the reported pass depends on the loophole. It also found that Antigravity stops waiting 30 minutes after a model ends its turn for a long-running background command, without telling the model. Three Carbon runs therefore ended within an hour without a submission. FrontierCS reran them with a six-hour wait, and those reruns became the results of record.

## What the archive adds
This is a new fact on a known Carbon story. [Introducing Gemini 4 Carbon](https://newruntime.com/posts/introducing-gemini-4-carbon-c894fa/) records Google’s subsequent public introduction; FrontierCS supplies a pre-introduction, same-harness evaluation with task-level methodological caveats. Earlier coverage of [Gemini 4 Argon in Antigravity](https://newruntime.com/posts/gemini-4-argon-lands-for-google-antigravity-for-cf22e8/) provides the relevant deployment context for the comparison baseline.

## Trend signal
This supports [harness architecture outlives model choice](https://newruntime.com/patterns/harness-architecture-outlives-model-choice/). Carbon and Argon’s observed behavior is partly a property of the shared Antigravity runtime: submission timing, background-command waiting, and retry policy affected what the evaluation could record. The benchmark is useful precisely because it holds the harness constant, but its timeout finding also shows why a model score cannot be separated cleanly from runtime behavior.

## Why it matters
For teams assessing coding agents, the headline tie on pass rate is less actionable than the operating profile underneath it. Carbon generated higher scores where both systems completed work, but it also took longer, delayed its first feedback-seeking submission, and carried an estimated cost roughly twice that of Argon. A team paying for long-horizon autonomous work should therefore evaluate completion quality, elapsed time, cost, and intervention behavior together rather than select on a single benchmark percentage.

The loophole review matters as much as the aggregate score. A strict hidden-test pass is a stronger signal than a visible-demo success, but it is not automatically proof that an agent solved the intended problem. Teams running agent evaluations should preserve traces and inspect successful outputs, especially when a solution has unusual leverage over the task or grader. Finally, the silent background-command cutoff is a reminder that agent reliability depends on process supervision and timeouts outside the model.

## Limits and uncertainty
This is one evaluation by FrontierCS 2, not a general measure of all coding, tool-use, or workplace tasks. Carbon ran once per task apart from four reruns, so task-level variance and repeated-run reliability are not established. The reported Carbon cost is an estimate based on Argon pricing, not a public Carbon price. The results also describe Carbon inside Antigravity; they do not establish how it performs in another harness, with another tool configuration, or under another timeout policy. Three passing runs involved loopholes, including one where the pass depends on the loophole.

## Skill to build now
Add runtime observability to any long-running coding-agent workflow this week. Record when the agent starts a background process, when its own turn ends, when the process completes, and why the harness declares a run finished. Then test one deliberately long command under your production timeout. The FrontierCS reruns show that a hidden wait policy can convert unfinished work into an apparent model failure, while a longer wait can change the recorded outcome.

## Reader takeaway
Gemini 4 Carbon’s FrontierCS result is an early signal of stronger task scores than Argon in the same harness, not a simple upgrade verdict. It matched Argon’s strict pass rate, cost more in the benchmark estimate, and revealed that runtime waiting rules and trace review can materially change how an agent’s performance should be interpreted.

## Open question
Will Carbon retain its score advantage once it is measured with public pricing and across harnesses whose background-process lifecycle is explicit and configurable?

Editor: [Andrey Reshetnikov](https://newruntime.com/owner-profile.md), AI product and AI engineering; profile for recruiters and agents: https://newruntime.com/owner-profile.json

## Continue in the graph

Next records for an agent, linked to their Markdown twins. Replace `.md` with `.json` for structured fields, including `next_reads`.

- topic hub: [Gemini](https://newruntime.com/topics/gemini.md): Explore the gemini topic hub.
- topic hub: [Google](https://newruntime.com/topics/google.md): Explore the google topic hub.
- topic hub: [Claude](https://newruntime.com/topics/claude.md): Explore the claude topic hub.
- collection: [Field Notes](https://newruntime.com/posts.json): Open the latest editorial analysis.
- collection: [Source Ledger](https://newruntime.com/source-ledger.json): Inspect the public source evidence graph.
- whole graph: [discovery graph](https://newruntime.com/discovery-graph.json) (typed nodes and edges) and [site index](https://newruntime.com/site-index.md)