{"type":"post","id":"do-agent-teams-pay-off-a-case-study-04444a","slug":"do-agent-teams-pay-off-a-case-study-04444a","title":"Do Agent Teams Pay Off? A Case Study on Vibe Code Bench (18 minute read)","description":"**Thesis:** Agent teams should be judged by verified task outcomes relative to their added cost and time, not by parallelism alone.","retrieval_nugget":"**Thesis:** Agent teams should be judged by verified task outcomes relative to their added cost and time, not by parallelism alone.","published_at":"2026-10-12","updated_at":"2026-10-12","record_date":"2026-10-12","date_kind":"published_at","topics":[],"source_urls":["https://www.vals.ai/blogs/multi-agent-vibe-code-bench"],"schema_version":"newruntime-agent-readable-v0.2","stable_id":"post:do-agent-teams-pay-off-a-case-study-04444a","status":"published","visuals":[],"editorial_provenance":{"schema_version":"newruntime-editorial-copy-v1","content_status":"source_grounded_final","final_copy_sha256":"sha256:c4807bdee8b4c25a24c37397389cfbd5dabd3d122ec00f89d69d9019c9d9160a","reviewed_at":"2026-10-12T14:06:23.017Z","source_evidence_count":1,"verified_claim_count":3,"site_analysis_schema_version":"newruntime-site-analysis-v1","site_object_kind":"field_note","observed_fact_count":3,"implication_count":2,"watch_condition_count":1,"related_record_count":0},"analysis":{"schema_version":"newruntime-site-analysis-v1","object_kind":"field_note","thesis":"Agent teams should be judged by verified task outcomes relative to their added cost and time, not by parallelism alone.","observed_facts":[{"text":"Vals AI tested GPT 6 Sol and Claude Opus 5.5 on 50 Vibe Code Bench apps at medium and maximum reasoning effort; each of the eight configurations ran once.","source_urls":["https://www.vals.ai/blogs/multi-agent-vibe-code-bench"]},{"text":"Teams cost 1.8 to 5.1 times as much as single agents; only GPT 6 Sol at medium effort had a statistically significant score gain, 7.3 points (p = 0.005).","source_urls":["https://www.vals.ai/blogs/multi-agent-vibe-code-bench"]},{"text":"The Vals AI report says Sol generally split work early and tested, while Opus often worked in sequential waves.","source_urls":["https://www.vals.ai/blogs/multi-agent-vibe-code-bench"]}],"mechanism":"Delegation creates additional working contexts and orchestration overhead; repeated cached input for subagents drove much of the extra spend, while orchestration style affected whether parallelism reduced runtime.","why_now":"Both OpenAI and Anthropic now offer native multi-agent support, making the operational choice between delegation and a stronger single-agent run immediate for teams building software with these systems.","implications":["Benchmark delegation against higher reasoning effort on representative tasks, tracking score, cost, and runtime together.","Do not treat a team’s peak score as a default choice without accounting for its cost and time."],"evidence_boundary":"The study covers one benchmark of full-stack web apps, 50 tasks, and one run per configuration; results may not transfer to other work or quantify run-to-run variability.","watch_conditions":["Watch for replicated comparisons across task types and repeated runs that separate model, effort, orchestration, and team-size effects."],"related_records":[],"new_branch_reason":"No prior New Runtime record was supplied; this creates a distinct analytical record around measured agent-team economics and benchmark boundaries."},"editor":{"role":"editor","name":"Andrey Reshetnikov","profile":"https://newruntime.com/owner-profile.md","profile_json":"https://newruntime.com/owner-profile.json","same_as":["https://www.linkedin.com/in/reshetnikov1/","https://t.me/lapetuse"]},"routes":{"html":"https://newruntime.com/posts/do-agent-teams-pay-off-a-case-study-04444a/","markdown":"https://newruntime.com/posts/do-agent-teams-pay-off-a-case-study-04444a.md","json":"https://newruntime.com/posts/do-agent-teams-pay-off-a-case-study-04444a.json"},"next_reads":[{"type":"collection","path":"/posts/","url":"https://newruntime.com/posts/","title":"Field Notes","reason":"Open the latest editorial analysis.","media_type":"text/html","json_url":"https://newruntime.com/posts.json"},{"type":"collection","path":"/source-ledger.json","url":"https://newruntime.com/source-ledger.json","title":"Source Ledger","reason":"Inspect the public source evidence graph.","media_type":"application/json"}]}