Field noteNov 14, 20265 min read1 source

Independent check finds Google’s Gemini 4 Hydrogen Legal Research ranking does not match the Vals board

In one minute

Google DeepMind’s Gemini 4 Hydrogen model card describes a model for fast coding and agentic workflows, with a context window of up to one million tokens and benchmark results across several task categories. An independent check reports mismatches between some launch-related benchmark claims and public leaderboards, making evaluation conditions important for teams selecting a model.

What changed: An independent check finds Gemini 4 Hydrogen’s legal-research ranking differs from the Vals board.

What happened

Google DeepMind published the Gemini 4 Hydrogen model card on November 12, 2026. It describes Hydrogen as adapted from Gemini 4 Carbon, but optimized for speed and efficiency in everyday coding and agentic workflows rather than the deep, long-horizon reasoning Google associates with Carbon and Gemini 4 Argon. The card says Hydrogen accepts text, images, audio and video, with a combined input-output context window of up to one million tokens. It lists up to 64,000 default synchronous output tokens and up to one million through background long decode.

The card names distribution through the Gemini app, Gemini API, Google Sheets, Google AI Studio, Gemini for business, Google Antigravity and Google Search. It says downstream providers can access the model through an API under applicable terms, and that no particular hardware or software is required. The card does not state a price. Google says Hydrogen is suited to iterative repository coding and debugging, open-ended agent execution, multimodal knowledge work, web development, and use as a sub-agent alongside frontier models.

Google reports November 2026 benchmark results across coding, knowledge work, computer use, multimodal understanding, science and math, and reasoning. Examples include 13.5% on SWE-sweep, 57.0% on Terminal-Bench 4.1, marked “Cheating-adjusted,” 71.6% on Agents’ Last Exam v1.1, 76.1% on AutomationBench-AA, 77.6% on the offline subset of OSWorld-2.1, and 58.8% on HLE-Diamond. The card compares some results with Claude Sonnet 5.5 and GPT-6.1 Terra and points to Google’s evaluation-methodology page for additional benchmark and methodology detail.

The card says Hydrogen may hallucinate, can sometimes be slow or time out, and may use more tokens at higher thinking levels. It gives a March 2026 knowledge cutoff. Google reports internal automated safety evaluations against Gemini 3.8 Flash, including a 7.00 percentage-point increase in image-to-text safety and a 5.57-point increase in unjustified refusals. It says a full Safety & Security Report will follow. Separately, an independent check dated November 13 reports that Vals Legal Research listed Hydrogen sixth at 53.85%, while 55.77% belonged to Gemini 4 Carbon. The model card itself does not discuss that reported discrepancy.

What the archive adds

New Runtime’s earlier coverage said Gemini 4 Hydrogen had become available in Cursor alongside Gemini 4 Carbon (November 12 note). This model card is a new fact on that known model story: it documents Google’s intended uses, distribution, benchmark claims and safety caveats. The reported leaderboard discrepancy adds a separate reason to scrutinize evaluation claims.

Trend signal

This does not yet fit a pattern New Runtime tracks. It does reinforce a practical editorial lesson: vendor-reported benchmark figures, public leaderboard rows and a team’s own workload tests answer different questions and should not be treated as interchangeable.

Why now

Google’s model card is dated November 12, 2026, and the independent check is dated November 13. The close timing makes the reported discrepancy immediately relevant to anyone comparing Hydrogen against alternatives using current leaderboard positions.

Why it matters

A benchmark score can influence model routing, procurement and expectations for coding or knowledge-work agents. If a result is self-computed, measured under a different setup, or attributed to the wrong model on a leaderboard, a team may be comparing unlike tests or selecting a model for the wrong reason. Hydrogen’s broad distribution and stated fit for everyday agentic work make those distinctions practical, not academic.

The useful response is not to discard Google’s card or treat the independent check as conclusive. Instead, separate vendor-reported results from independently reproduced leaderboard entries, check the benchmark’s version and evaluation conditions, and run representative tasks from your own workflow. That approach also helps expose trade-offs the card itself flags, including latency, timeouts, token use and higher unjustified-refusal rates in the reported safety evaluation.

Limits and uncertainty

The reported discrepancy has not been addressed in Google’s model card. Its claims about leaderboard placement should therefore be checked directly against the relevant benchmark pages and their timestamps. Google’s card points to further methodology but does not reproduce every evaluation detail alongside each score. It also says Hydrogen’s full Safety & Security Report will be published in the coming days, leaving its fuller safety evaluation unpublished at the time of the card. Reported benchmark scores do not establish how the model will perform on a particular team’s codebase, tools or prompts.

Skill to build now

Before changing a production model route, create a small comparison sheet for each benchmark your team uses: record the exact task version, model variant, score, date checked, evaluation conditions and whether the result is vendor-reported or independently reproduced. Then run a fixed set of representative internal tasks against the candidate models and track completion quality, latency, token use and review effort.

Reader takeaway

Hydrogen’s model card offers a useful map of Google’s intended use, distribution and reported performance, but benchmark numbers should not be treated as interchangeable facts. Verify leaderboard entries and test the model on your own work before using a score to drive a routing or purchasing decision.

Evidence / sources

  1. 1deepmind.google/models/model-cards/gemini-4-hydrogen
All Field Notes

Next reads

Read next

Open the graph
  1. 01topicGoogleExplore the google topic hub.
  2. 02topicGeminiExplore the gemini topic hub.
  3. 03archiveField NotesOpen the latest editorial analysis.
  4. 04sourcesSourcesSee the original sources this archive cites.
Who read this page?Machine requests, hidden until opened

Loading the privacy-safe route aggregate...

Open the JSON contract