Observed facts: Cohere tested team configurations on ProgramBench, a benchmark of 200 programs rebuilt from observed behavior and scored against hidden behavioral tests. In a reported comparison, a Sol lead coordinating 16 Luna planning workers reached 61.8% reward, versus 63.2% for an all-Sol team delegating planning, implementation, and verification; reported cost was $155 versus $510. The study also reports that delegating exploration and planning alone matched the three-stage delegation setup at about half the cost.
Mechanism: The cost advantage comes from assigning cheaper workers to bounded exploration and planning while reserving the stronger model for task framing and coordination; compute is targeted at bottlenecks instead of spread across every stage.
Why now: Coding-agent teams are moving from demonstrations toward larger software tasks, making the gap between benchmark gains and operating cost increasingly relevant; this study offers controlled comparisons that earlier project examples could not isolate.
First implication: Teams should test stage-specific delegation and model heterogeneity before scaling homogeneous teams.
Evidence boundary: The evidence is Cohere's harness and ProgramBench reconstruction tasks with hidden behavioral tests; it does not establish transfer to ordinary repository work, other models, or production review requirements. The study’s harness, prompts, benchmark tasks, and reported configurations define the boundary of what can be claimed here.
First watch condition: Watch for independent replications on real-world repositories reporting verified completion quality and total cost under comparable team configurations.
A separate analytical record is warranted because this study supplies controlled evidence about agent-team allocation and measured cost, rather than merely describing another large coding-agent project.