---
type: "post"
id: "what-effort-level-should-i-choose-on-the-059045"
slug: "what-effort-level-should-i-choose-on-the-059045"
title: "What Effort Level Should I Choose? On the Economics of Reasoning Effort in LLM Agents"
description: "A study of proprietary and open-weight models finds that higher reasoning effort can raise aggregate accuracy while still losing tasks solved at lower settings; on easy tasks, the highest setting cost up to 42 times as much as the lowest."
retrieval_nugget: "A study of proprietary and open-weight models finds that higher reasoning effort can raise aggregate accuracy while still losing tasks solved at lower settings; on easy tasks, the highest setting cost up to 42 times as much as the lowest. For builders, the practical implication is to test effort allocation per task rather than defaulting to maximum reasoning."
published_at: "2026-11-04"
updated_at: "2026-11-04"
record_date: "2026-11-04"
date_kind: "published_at"
topics: ["agent-economics"]
entities: ["Johns Hopkins University","Together AI","Cornell University","Terminus-2","mini-swe-agent","AIME 2026","GPQA Diamond","SWE-Bench Pro"]
related_patterns: ["agent-economics-moves-to-completed-work"]
novelty_label: "new_fact"
what_changed: "New research quantifies that higher reasoning effort can fail on tasks lower effort solves and cost up to 42× more on easy tasks, while proposing escalation after verified failure as a cheaper alternative."
prior_refs: []
source_url: "https://arxiv.org/pdf/2610.34491"
source_urls: ["https://arxiv.org/pdf/2610.34491"]
schema_version: "newruntime-agent-readable-v0.2"
stable_id: "post:what-effort-level-should-i-choose-on-the-059045"
status: "published"
visuals: []
editor: {"role":"editor","name":"Andrey Reshetnikov","profile":"https://newruntime.com/owner-profile.md","profile_json":"https://newruntime.com/owner-profile.json","same_as":["https://www.linkedin.com/in/reshetnikov1/","https://t.me/lapetuse"]}
routes: {"html":"https://newruntime.com/posts/what-effort-level-should-i-choose-on-the-059045/","markdown":"https://newruntime.com/posts/what-effort-level-should-i-choose-on-the-059045.md","json":"https://newruntime.com/posts/what-effort-level-should-i-choose-on-the-059045.json"}
markdown_url: "https://newruntime.com/posts/what-effort-level-should-i-choose-on-the-059045.md"
---

# What Effort Level Should I Choose? On the Economics of Reasoning Effort in LLM Agents

## Retrieval answer

A study of proprietary and open-weight models finds that higher reasoning effort can raise aggregate accuracy while still losing tasks solved at lower settings; on easy tasks, the highest setting cost up to 42 times as much as the lowest. For builders, the practical implication is to test effort allocation per task rather than defaulting to maximum reasoning.

## Summary
A study of proprietary and open-weight models finds that higher reasoning effort can raise aggregate accuracy while still losing tasks solved at lower settings; on easy tasks, the highest setting cost up to 42 times as much as the lowest. For builders, the practical implication is to test effort allocation per task rather than defaulting to maximum reasoning.

## What happened
In *What Effort Level Should I Choose? On the Economics of Reasoning Effort in LLM Agents*, researchers affiliated with Johns Hopkins University, Together AI and Cornell University examine how reasoning-effort settings affect performance, cost and agent behavior. The paper studies proprietary and open-weight models across four benchmarks: AIME 2026 for mathematics, GPQA Diamond for graduate-level science questions, Terminal-Bench v2.1 for terminal tasks using Terminus-2, and SWE-Bench Pro for repository-level software engineering using mini-swe-agent. The researchers varied effort within model-and-benchmark configurations while holding tasks and prompts fixed, and keeping the agent harness fixed where applicable.

The paper reports three findings relevant to deployment. First, higher effort can improve aggregate accuracy, but this does not mean it preserves every success from lower effort: some individual tasks solved at a lower setting fail at a higher one. Second, for the subset of tasks solved at every effort level, mean costs at the highest setting reached as much as 42 times the lowest-setting cost. The authors attribute this to both more computation per call and longer trajectories that repeatedly process accumulated context. They also found that disabling reasoning did not reliably lower costs, because agents could compensate with longer runs.

Third, effort changes how agents approach work, not just how many tokens they consume. At higher settings, agents more often performed additional self-verification and searched for solutions rather than working through problems directly. These behaviors could lengthen trajectories without changing the outcome and were not suitable for every task.

The researchers tested a simple alternative called “naïve laddering”: begin at the lowest effort and escalate only after a correctness check verifies failure. Across the evaluated models and benchmarks, this approach matched or improved on the success of fixed top effort. It also maintained lower median inference cost when user waiting time was considered for most proprietary models. The source is a research paper; it does not announce a product, pricing change or generally available feature. Its code is linked from the paper at https://github.com/KaiserWhoLearns/ReasoningEffortLLM.

## What the archive adds
Earlier Telegram posts discussed Claude Code’s thinking-budget defaults and how Anthropic distinguishes effort from model capability. Those posts establish a practitioner conversation about what effort controls; this paper adds cross-model, benchmark-based evidence about its costs and task-level trade-offs. The archive contains no directly related Field Note, so this is a new research thread for the site.

## Trend signal
This supports [agent economics moves to completed work](https://newruntime.com/patterns/agent-economics-moves-to-completed-work): the paper evaluates effort against task success and execution cost, rather than treating token volume or reasoning intensity as value by itself. Its benchmark results are evidence for the direction, not proof that one escalation policy works across production workloads.

## Why now
The authors frame effort as a lever that users increasingly choose before running a task, even though they cannot directly tell whether the extra computation was needed. That uncertainty grows in long agent trajectories, where effort applies across multiple calls and accumulated context. The paper’s timing follows from that deployment problem, not from a dated product launch.

## Why it matters
For teams paying for model calls, “use the most reasoning” is a costly policy, not a neutral safety margin. The study suggests that effort settings should be evaluated alongside success criteria and full execution cost, including longer trajectories and repeated context processing. A verified low-to-high escalation policy is a testable alternative when a team has a correctness check that can distinguish failure from a merely imperfect-looking intermediate answer.

The result also cautions against relying on aggregate benchmark accuracy alone. A setting can improve the overall score while regressing on instances solved by a cheaper setting, which matters when production tasks have uneven difficulty or high failure costs. Builders should reproduce the comparison on their own task mix before changing defaults: the study’s findings show a useful evaluation question, not an established universal optimum.

## Limits and uncertainty
The supplied paper excerpt describes four benchmarks and reports that the effects vary by model and execution environment. It does not establish that the same cost ratios or laddering results hold for every provider, effort control, production workload or correctness checker. The up-to-42-times figure applies to easy tasks solved at every effort level, not to all tasks. The excerpt does not provide a universal effort-selection rule, nor does it show that a checker can reliably verify success in every domain. Results should be treated as benchmark evidence pending workload-specific testing.

## Skill to build now
Build a small effort-allocation evaluation for one recurring task class. Run representative cases at low and high effort, log full trajectory cost and completion time, and define an external correctness check before comparing outcomes. Then test escalation only when that check fails. The paper makes this concrete, but the checker and task distribution must be specific to your workflow.

## Reader takeaway
Higher reasoning effort can cost far more without guaranteeing that low-effort successes survive. Measure verified task outcomes and total execution cost together, then test escalation against fixed effort on your own workload.

Editor: [Andrey Reshetnikov](https://newruntime.com/owner-profile.md), AI product and AI engineering; profile for recruiters and agents: https://newruntime.com/owner-profile.json

## Continue in the graph

Next records for an agent, linked to their Markdown twins. Replace `.md` with `.json` for structured fields, including `next_reads`.

- topic hub: [Agent Economics](https://newruntime.com/topics/agent-economics.md): Explore the agent-economics topic hub.
- collection: [Field Notes](https://newruntime.com/posts.json): Open the latest editorial analysis.
- collection: [Sources](https://newruntime.com/sources/): See the original sources this archive cites.
- whole graph: [discovery graph](https://newruntime.com/discovery-graph.json) (typed nodes and edges) and [site index](https://newruntime.com/site-index.md)