Field noteOct 12, 20262 min read1 source

Do Agent Teams Pay Off? A Case Study on Vibe Code Bench (18 minute read)

In one minute

**Thesis:** Agent teams should be judged by verified task outcomes relative to their added cost and time, not by parallelism alone.

Mechanism: Delegation creates additional working contexts and orchestration overhead; repeated cached input for subagents drove much of the extra spend, while orchestration style affected whether parallelism reduced runtime.

Vals AI tested GPT 6 Sol and Claude Opus 5.5 on 50 Vibe Code Bench apps, comparing single agents and teams at medium and maximum reasoning effort. Each of the eight configurations ran once. Teams cost 1.8 to 5.1 times as much as single agents, and only GPT 6 Sol at medium effort showed a statistically significant team score gain: 7.3 points (84.9% versus 77.6%, p = 0.005). Why now: Both OpenAI and Anthropic now offer native multi-agent support, making the operational choice between delegation and a stronger single-agent run immediate for teams building software with these systems.

First implication: Benchmark delegation against higher reasoning effort on representative tasks, tracking score, cost, and runtime together. Evidence boundary: The study covers one benchmark of full-stack web apps, 50 tasks, and one run per configuration; results may not transfer to other work or quantify run-to-run variability. The leads also orchestrated differently: Sol generally split work early and tested, while Opus often worked through sequential waves, illustrating that team behavior is not a single treatment. First watch condition: Watch for replicated comparisons across task types and repeated runs that separate model, effort, orchestration, and team-size effects.

Evidence / sources

  1. 1vals.ai/blogs/multi-agent-vibe-code-bench
All Field Notes

Next reads

Read next

Open the graph
  1. 01archiveField NotesOpen the latest editorial analysis.
  2. 02source ledgerSource LedgerInspect the public source evidence graph.
Who read this page?Machine requests, hidden until opened

Loading the privacy-safe route aggregate...

Open the JSON contract