---
type: "post"
id: "how-much-does-reasoning-effort-affect-score-and-b03b68"
slug: "how-much-does-reasoning-effort-affect-score-and-b03b68"
title: "How Much Does Reasoning Effort Affect Score and Cost?"
description: "Vals AI compared three reasoning settings across seven frontier models and eight benchmarks, finding that lower settings often cut run cost much more than they reduce benchmark score."
retrieval_nugget: "Vals AI compared three reasoning settings across seven frontier models and eight benchmarks, finding that lower settings often cut run cost much more than they reduce benchmark score. The results support testing reasoning effort per task rather than defaulting to maximum, but do not establish a universally best setting."
published_at: "2026-11-04"
updated_at: "2026-11-06"
record_date: "2026-11-04"
date_kind: "published_at"
topics: ["gemini","claude"]
entities: ["Vals AI","DeepSeek V4.1 Flash","Grok 4.7","Muse Spark 1.3","GPT-6 Astra","Claude Opus 5.5","GLM-5.3","Gemini 4 Argon"]
related_patterns: ["agent-economics-moves-to-completed-work"]
novelty_label: "new_fact"
what_changed: "Vals AI reports a separate seven-model benchmark: four models retained at least 80% of score for half the cost or less, and lower settings matched leaderboard scores in 22 of 56 model-benchmark combinations."
prior_refs: ["https://newruntime.com/posts/what-effort-level-should-i-choose-on-the-059045/"]
source_url: "https://www.vals.ai/blogs/reasoning-effort"
source_urls: ["https://www.vals.ai/blogs/reasoning-effort"]
schema_version: "newruntime-agent-readable-v0.2"
stable_id: "post:how-much-does-reasoning-effort-affect-score-and-b03b68"
status: "published"
visuals: []
editor: {"role":"editor","name":"Andrey Reshetnikov","profile":"https://newruntime.com/owner-profile.md","profile_json":"https://newruntime.com/owner-profile.json","same_as":["https://www.linkedin.com/in/reshetnikov1/","https://t.me/lapetuse"]}
routes: {"html":"https://newruntime.com/posts/how-much-does-reasoning-effort-affect-score-and-b03b68/","markdown":"https://newruntime.com/posts/how-much-does-reasoning-effort-affect-score-and-b03b68.md","json":"https://newruntime.com/posts/how-much-does-reasoning-effort-affect-score-and-b03b68.json"}
markdown_url: "https://newruntime.com/posts/how-much-does-reasoning-effort-affect-score-and-b03b68.md"
---

# How Much Does Reasoning Effort Affect Score and Cost?

## Retrieval answer

Vals AI compared three reasoning settings across seven frontier models and eight benchmarks, finding that lower settings often cut run cost much more than they reduce benchmark score. The results support testing reasoning effort per task rather than defaulting to maximum, but do not establish a universally best setting.

## Summary
Vals AI compared three reasoning settings across seven frontier models and eight benchmarks, finding that lower settings often cut run cost much more than they reduce benchmark score. The results support testing reasoning effort per task rather than defaulting to maximum, but do not establish a universally best setting.

## What happened
In a post dated November 3, 2026, Vals AI reports measurements for the top model from each of seven AI labs: DeepSeek V4.1 Flash, Grok 4.7, Muse Spark 1.3, GPT-6 Astra, Claude Opus 5.5, GLM-5.3, and Gemini 4 Argon. Each ran at a lowest, middle, and leaderboard setting. The leaderboard setting was usually the model’s highest: max for DeepSeek, Astra, Opus, and GLM; xhigh for Grok and Muse. For Gemini, whose leaderboard runs used no effort level, the researchers used high.

The evaluation covered eight benchmarks in the Vals Index: Finance Agent 2.1, EMB 1.1, Terminal-Bench 4, Vibe Code Bench, Code Migration 1.1, Legal Research, HLAB 1.1, and Tax Agent Bench. Each setting ran on the same fixed subset of each benchmark, about half its tasks. Vals says it chose subsets that rank models in the same order as the full benchmark. For most models, leaderboard-setting scores came from existing leaderboard runs, rescored on the subset; where no such run existed, including all Gemini runs, Vals ran the setting fresh.

The headline comparison is within each model: score and cost at lower effort as percentages of that model’s leaderboard setting. At the lowest setting, four of seven models retained at least 80% of their Vals Index score for half the cost or less. Claude Opus 5.5 retained 80% at 16% of the cost; Grok 4.7 retained 86% at 16%. Gemini 4 Argon retained 99% at 84% of cost. No model matched its overall leaderboard score at its lowest setting, although individual benchmark results often differed.

Across the 56 model-and-benchmark combinations, a lower setting scored as high as the leaderboard setting at lower cost in 22 cases. None of the cheaper settings scored significantly higher; 33 scored significantly lower. The effect of extra reasoning varied by benchmark: the median gain from lowest to highest was 19.2 points on Legal Research, 15.6 on Code Migration 1.1, and 12.1 on Terminal-Bench 4, while EMB 1.1, HLAB 1.1, and Finance Agent 2.1 each gained less than six points.

Vals also calculated a hindsight upper bound: for each task, it selected the cheapest setting that achieved that task’s best score. Those picks added a median six points over leaderboard settings at a median 64% of their cost. This is not a deployable policy as stated, because a real system must choose effort before seeing task results. The source does not specify an access change or price change; it is a benchmark study, not a new model release.

## What the archive adds
This is a new benchmark result on a known question, not a wholly new topic. Our earlier note, [What Effort Level Should I Choose? On the Economics of Reasoning Effort in LLM Agents](https://newruntime.com/posts/what-effort-level-should-i-choose-on-the-059045/), already examined the cost economics of reasoning effort. Vals adds cross-model and cross-benchmark measurements of how score and cost move together.

## Trend signal
The results support [Agent economics moves to completed work](https://newruntime.com/patterns/agent-economics-moves-to-completed-work/): model cost alone is not enough to choose a setting; teams need to assess the quality achieved on actual tasks. The benchmark’s aggregate score and run-cost comparisons are useful inputs, but they are not verified completion costs for production workflows.

## Why it matters
Reasoning effort is a policy choice with measurable quality and cost consequences, not simply a “more is better” dial. A team using a high setting for every request may be paying substantially more on tasks where a lower setting remains competitive. Conversely, Legal Research and several coding or agent benchmarks showed larger gains from higher effort, so applying one low setting globally could sacrifice meaningful performance.

The useful implication is to segment a workload and evaluate effort settings against the team’s own success criteria and costs. This study gives a reason to run that test, not a ready-made routing rule: its benchmark subsets, scoring system, and costs may not reflect a team’s data, verification burden, or production mix.

## Limits and uncertainty
The study covers seven models, three settings per model, and fixed subsets of eight benchmarks; it does not establish how results transfer to other models, tasks, prompts, or live workloads. The Vals Index combines benchmark results, and the reported cost comparisons are relative to each model’s leaderboard setting, not a universal dollar price or cross-model cost ranking.

The per-task “best setting” calculation uses hindsight, so it overstates what a system could achieve when choosing before observing results. The source does not show a deployed adaptive policy, end-to-end task costs, or the effects of retries and human review. Its subset-selection claim is that rankings match the full benchmarks; that does not guarantee identical score or cost behavior on every deployment workload.

## Skill to build now
Take a representative sample of your own tasks and run each at the reasoning settings your model API exposes. Record task-level quality against a reviewable success criterion alongside actual run cost, then compare settings by task type. Treat the results as a routing experiment, not proof that a single setting is best everywhere.

## Reader takeaway
Lower reasoning effort can preserve much of a model’s benchmark score at a fraction of its leaderboard-setting cost, but the trade-off changes by task. Measure settings on your own workload before choosing a default.

Editor: [Andrey Reshetnikov](https://newruntime.com/owner-profile.md), AI product and AI engineering; profile for recruiters and agents: https://newruntime.com/owner-profile.json

## Continue in the graph

Next records for an agent, linked to their Markdown twins. Replace `.md` with `.json` for structured fields, including `next_reads`.

- topic hub: [Gemini](https://newruntime.com/topics/gemini.md): Explore the gemini topic hub.
- topic hub: [Claude](https://newruntime.com/topics/claude.md): Explore the claude topic hub.
- collection: [Field Notes](https://newruntime.com/posts.json): Open the latest editorial analysis.
- collection: [Sources](https://newruntime.com/sources/): See the original sources this archive cites.
- whole graph: [discovery graph](https://newruntime.com/discovery-graph.json) (typed nodes and edges) and [site index](https://newruntime.com/site-index.md)