---
type: "post"
id: "how-do-ai-agents-waste-your-money-framework-4b5aa5"
slug: "how-do-ai-agents-waste-your-money-framework-4b5aa5"
title: "How Do AI Agents Waste Your Money? framework measures removable token waste across agent harnesses"
description: "An arXiv paper introduces a framework for locating removable text in AI-agent trajectories and measuring its token cost."
retrieval_nugget: "An arXiv paper introduces a framework for locating removable text in AI-agent trajectories and measuring its token cost. In experiments on two agent benchmarks, online pruning removed 41–46% of reasoning while largely preserving task performance, suggesting that harness and task design may matter as much as model choice for reducing waste."
published_at: "2026-11-06"
updated_at: "2026-11-06"
record_date: "2026-11-06"
date_kind: "published_at"
topics: ["agent-economics"]
entities: ["arXiv","How Do AI Agents Waste Your Money?","Huimin Zeng","Yuxuan Lu","Jesse Cai"]
related_patterns: ["agent-economics-moves-to-completed-work"]
novelty_label: "new_fact"
what_changed: "A new framework identifies removable token waste across agent harnesses and reports pruning 41–46% of reasoning while largely preserving task performance."
prior_refs: []
source_url: "https://arxiv.org/abs/2611.07238"
source_urls: ["https://arxiv.org/abs/2611.07238"]
schema_version: "newruntime-agent-readable-v0.2"
stable_id: "post:how-do-ai-agents-waste-your-money-framework-4b5aa5"
status: "published"
visuals: []
editor: {"role":"editor","name":"Andrey Reshetnikov","profile":"https://newruntime.com/owner-profile.md","profile_json":"https://newruntime.com/owner-profile.json","same_as":["https://www.linkedin.com/in/reshetnikov1/","https://t.me/lapetuse"]}
routes: {"html":"https://newruntime.com/posts/how-do-ai-agents-waste-your-money-framework-4b5aa5/","markdown":"https://newruntime.com/posts/how-do-ai-agents-waste-your-money-framework-4b5aa5.md","json":"https://newruntime.com/posts/how-do-ai-agents-waste-your-money-framework-4b5aa5.json"}
markdown_url: "https://newruntime.com/posts/how-do-ai-agents-waste-your-money-framework-4b5aa5.md"
---

# How Do AI Agents Waste Your Money? framework measures removable token waste across agent harnesses

## Retrieval answer

An arXiv paper introduces a framework for locating removable text in AI-agent trajectories and measuring its token cost. In experiments on two agent benchmarks, online pruning removed 41–46% of reasoning while largely preserving task performance, suggesting that harness and task design may matter as much as model choice for reducing waste.

## Summary
An arXiv paper introduces a framework for locating removable text in AI-agent trajectories and measuring its token cost. In experiments on two agent benchmarks, online pruning removed 41–46% of reasoning while largely preserving task performance, suggesting that harness and task design may matter as much as model choice for reducing waste.

## What happened
The paper “How Do AI Agents Waste Your Money?” was submitted to arXiv on 4 November 2026 by Huimin Zeng and ten co-authors. It addresses a gap the authors identify in existing work: agent systems can consume large numbers of tokens, but aggregate usage does not show which parts of a run were necessary. The paper proposes a unified framework intended to measure token waste across heterogeneous agent systems.

The framework records and normalizes complete agent trajectories. It then uses an LLM-as-a-Judge to identify spans that may be removable and maps those spans back to the original trajectory for token-level measurement. The authors do not treat a judge’s label alone as proof of waste. They validate candidate spans through direct interventions: remove the marked text, resume agent execution, and measure the effect on task outcomes and subsequent actions.

In online pruning experiments, the authors report removing 41–46% of the model’s reasoning while largely preserving performance across two distinct agent benchmarks. The abstract does not give the benchmark names, absolute success rates, or a monetary savings figure, so the percentage is a reduction in reasoning, not a demonstrated reduction in total spend by the same amount. The paper presents the method as a way to turn token waste into an actionable optimization signal rather than as a ready-made cost-saving product.

The authors’ analysis finds that waste varies more across tasks and agent settings than across models. It also finds different waste patterns in reasoning and visible execution, and distinct inefficiencies introduced by different harnesses even when model and task are held constant. They argue that researchers and practitioners can use these measurements to train more efficient models, shape reasoning strategies, and optimize harnesses. The arXiv page identifies the paper as version 1 and classifies it under Computation and Language; it does not state a price or commercial access terms for a tool.

## What the archive adds
New Runtime's earlier coverage has treated token use as a potentially misleading proxy for value, including [a note on agent work moving into harnesses and accepted artifacts](https://newruntime.com/posts/agent-work-is-moving-into-harnesses-factories-and-accepted-artifacts/) (26 August 2026). This paper adds a measurement method and intervention results to that broader discussion; it is a new research fact on a known topic, not a restatement of a product launch.

## Trend signal
This supports the [agent economics moves to completed work](https://newruntime.com/patterns/agent-economics-moves-to-completed-work/) pattern weakly: the paper questions token volume as a useful standalone cost measure and tests whether runs can retain outcomes after pruning. It does not itself calculate cost per verified completed task, so it points toward the pattern without establishing that economic unit.

## Why now
The paper arrived as agent teams increasingly confront long, tool-using runs whose cost cannot be understood from a model’s list price alone. Its timing matters because the proposed unit of analysis is a complete trajectory: the combination of reasoning, visible execution, tool use, task conditions, and harness behavior. That is where a team can distinguish expensive work from merely verbose work.

## Why it matters
Teams often respond to high agent bills by choosing a cheaper model or imposing token limits. This paper suggests another source of leverage: inspect the trajectory and harness that produce the usage. If a system repeatedly restates context, duplicates tool work, or takes unnecessary detours, a model swap may leave the underlying waste intact.

The more important contribution is methodological. Removing a span and resuming the run tests whether it was actually dispensable, rather than assuming that shorter reasoning is better. Builders can adapt that principle to their own evaluations: measure task success and downstream actions after a proposed change, not just the tokens removed. However, the reported reductions are research results, not a forecast of savings for a production agent or evidence that aggressive pruning is safe for every task.

This also changes how teams should assign ownership. Model vendors can improve inference efficiency, but a team operating an agent controls prompt assembly, tool contracts, retry behavior, memory injection, and the harness that sequences work. A token dashboard can reveal that usage rose; trajectory-level analysis can help locate the behavior responsible. For systems with expensive, repeated workflows, that distinction may be more useful than a single global budget.

## Limits and uncertainty
The paper is an arXiv preprint, not evidence of peer-reviewed publication. Its abstract reports results across two benchmarks but does not name them, give absolute task-success figures, or quantify changes in latency or monetary cost. “Largely preserving” performance is not a claim of identical outcomes. The results also do not establish that the same pruning rate will transfer to other models, tools, tasks, or production harnesses. The framework relies on an LLM judge to identify candidate spans, while the intervention provides a check on those labels; how robust that process is under different judge models and task distributions remains to be established.

## Skill to build now
Choose one recurring agent workflow and retain complete trajectories alongside its task outcome. Before changing a prompt, model, or token budget, form a specific removal hypothesis: for example, a repeated context block or an unnecessary step between tool calls. Run the changed trajectory through the same task set and compare task outcomes and subsequent actions, not only token counts. This mirrors the paper’s intervention principle and makes a cost optimization falsifiable.

## Reader takeaway
The useful question is not only how many tokens an agent used, but which parts of its trajectory were needed to finish the task. This paper offers a way to test that distinction, with promising benchmark results, but teams should validate any pruning against their own task outcomes before treating token reduction as a cost or quality win.

Editor: [Andrey Reshetnikov](https://newruntime.com/owner-profile.md), AI product and AI engineering; profile for recruiters and agents: https://newruntime.com/owner-profile.json

## Continue in the graph

Next records for an agent, linked to their Markdown twins. Replace `.md` with `.json` for structured fields, including `next_reads`.

- topic hub: [Agent Economics](https://newruntime.com/topics/agent-economics.md): Explore the agent-economics topic hub.
- collection: [Field Notes](https://newruntime.com/posts.json): Open the latest editorial analysis.
- collection: [Sources](https://newruntime.com/sources/): See the original sources this archive cites.
- whole graph: [discovery graph](https://newruntime.com/discovery-graph.json) (typed nodes and edges) and [site index](https://newruntime.com/site-index.md)