Artificial Analysis describes a dataset of coding-agent trajectories generated in the OpenCode harness, where DeepSeek V3.2, GLM 4.7, and Kimi K2.5 were prompted to resolve issues in public repositories. The reported input sequence lengths span about 5,000 to 131,000 tokens, with a mean near 27,000.
The mechanism is trajectory replay with simulated tool latency: after each tool call, the benchmark inserts a delay sampled from a distribution partitioned by tool type. The stated range is under 0.1 seconds to 5 seconds, with a median of about one second. This changes the load shape from a stream of independent prompts to concurrent agents alternating between model work and tool waits. The methodology then compares total server throughput with per-request output speed across configurations.
Why now: as inference providers serve more multi-step coding-agent traffic, benchmark rankings based on isolated requests may not capture the interaction between concurrency, context reuse, scheduler behavior, and time spent waiting on tools. The practical implication is that teams choosing serving configurations should ask whether their evaluation workload resembles their agents’ actual trajectories, not just their token mix.
Evidence boundary: the cited page documents the benchmark’s design and metrics; it does not, by itself, establish that any system is faster, cheaper, or more suitable than another. Watch whether published leaderboard entries disclose comparable configurations and whether results remain stable across model and software-stack updates. This begins a distinct analytical record because no prior New Runtime record was supplied for this specific methodology.