---
type: "raw_signal"
id: "do-agent-teams-pay-off-a-case-study-315222"
slug: "do-agent-teams-pay-off-a-case-study-315222"
title: "Do Agent Teams Pay Off? A Case Study on Vibe Code Bench (18 minute read)"
description: "Vals AI compared single-agent and multi-agent setups on 50 Vibe Code Bench tasks: building complete web applications from product specifications."
retrieval_nugget: "Vals AI compared single-agent and multi-agent setups on 50 Vibe Code Bench tasks: building complete web applications from product specifications."
observed_at: "2026-10-12"
record_date: "2026-10-12"
date_kind: "observed_at"
why_it_matters: "The results are specific to this benchmark and its implementation. They do not establish that teams generally fail or succeed; they show that model, reasoning effort, orchestration, and budget affect the trade-off. Vals AI notes that the work used one benchmark of full-stack web apps and may not carry over to other kinds of work."
novelty: "notable"
verification_level: "source-linked"
signal_type: "research"
source_platform: "vals.ai"
topics: ["claude"]
entities: ["Vals AI","Vibe Code Bench","GPT 6 Sol","Claude Opus 5.5"]
related_patterns: ["agent-economics-moves-to-completed-work"]
source_url: "https://www.vals.ai/blogs/multi-agent-vibe-code-bench"
source_urls: ["https://www.vals.ai/blogs/multi-agent-vibe-code-bench"]
schema_version: "newruntime-agent-readable-v0.2"
stable_id: "signal:do-agent-teams-pay-off-a-case-study-315222"
status: "published"
visuals: []
editor: {"role":"editor","name":"Andrey Reshetnikov","profile":"https://newruntime.com/owner-profile.md","profile_json":"https://newruntime.com/owner-profile.json","same_as":["https://www.linkedin.com/in/reshetnikov1/","https://t.me/lapetuse"]}
routes: {"html":"https://newruntime.com/signals/do-agent-teams-pay-off-a-case-study-315222/","markdown":"https://newruntime.com/signals/do-agent-teams-pay-off-a-case-study-315222.md","json":"https://newruntime.com/signals/do-agent-teams-pay-off-a-case-study-315222.json"}
---

# Do Agent Teams Pay Off? A Case Study on Vibe Code Bench (18 minute read)

## Retrieval answer

Vals AI compared single-agent and multi-agent setups on 50 Vibe Code Bench tasks: building complete web applications from product specifications.

Vals AI compared single-agent and multi-agent setups on 50 Vibe Code Bench tasks: building complete web applications from product specifications. Each app was deployed with Docker Compose and graded by browser agents that exercised its workflows. The study tested GPT 6 Sol and Claude Opus 5.5 at medium and maximum reasoning effort, with each of the eight configurations run once.

Teams cost 1.8 to 5.1 times as much as single agents. Only one of the four within-model comparisons showed a statistically significant score difference: GPT 6 Sol’s medium-effort team scored 84.9%, versus 77.6% for its single agent, a 7.3-point gain (p = 0.005). The other differences, ranging from −0.3 to +3.4 points, were not statistically significant. Costs and runtimes varied substantially: Claude Opus 5.5 at maximum effort cost a mean $122 per app as a team and took a median 174.1 minutes; its medium-effort single-agent setup cost $4.08 and took 20.3 minutes.

The results are specific to this benchmark and its implementation. They do not establish that teams generally fail or succeed; they show that model, reasoning effort, orchestration, and budget affect the trade-off. Vals AI notes that the work used one benchmark of full-stack web apps and may not carry over to other kinds of work.

Editor: [Andrey Reshetnikov](https://newruntime.com/owner-profile.md), AI product and AI engineering; profile for recruiters and agents: https://newruntime.com/owner-profile.json

## Continue in the graph

Next records for an agent, linked to their Markdown twins. Replace `.md` with `.json` for structured fields, including `next_reads`.

- topic hub: [Claude](https://newruntime.com/topics/claude.md): Explore the claude topic hub.
- collection: [Raw Signals](https://newruntime.com/signals.json): Open every source-linked raw signal.
- collection: [Source Ledger](https://newruntime.com/source-ledger.json): Inspect the public source evidence graph.
- whole graph: [discovery graph](https://newruntime.com/discovery-graph.json) (typed nodes and edges) and [site index](https://newruntime.com/site-index.md)