Field noteNov 12, 20265 min read1 source

Mid-Harness, the source of the TerminalBench-Lite result, changes how the harness selects actions.

In one minute

A paper introducing Mid-Harness tests whether terminal agents can improve by sampling several possible actions and verifying them before execution. On TerminalBench-Lite, the authors report Pass@1 rising from 50.00% to 68.03% when a GPT-5.6 Sol verifier selects among eight actions generated by TMAX-9B. The result makes action selection a measurable harness-design question.

What changed: Mid-Harness reports that verifying eight sampled actions raises TerminalBench-Lite Pass@1 from 50.00% to 68.03%, showing how action selection can improve terminal-agent reliability.

What happened

In Mid-Harness: Scaling Actions Between Model and Harness for Terminal Agents, the authors study a reliability problem in terminal agents: a model may be able to generate a useful command, but the command it actually generates can still be poor. Their example is an incorrect package installation that changes the environment and makes later progress harder. The paper asks whether additional test-time computation placed between the model and the harness can improve the reliability of actions and the success of complete trajectories.

Mid-Harness samples candidate actions and verifies them before passing one to the harness for execution. The paper describes this as keeping the generator and harness unchanged while adding selection at their boundary. That distinction matters to the experiment: the intervention is not a new action-generating model or a redesigned terminal environment. It is an extra step for deciding which generated command should run.

The main TerminalBench-Lite result uses TMAX-9B as the action generator. With GPT-5.6 Sol as verifier, the reported Pass@1 increases from 50.00% for the base agent to 68.03% with eight sampled actions. The paper stresses that sampling alone is not enough: with weak verification, more candidate actions bring little benefit. A capable verifier can exploit useful alternatives already present in the generator's outputs. When TMAX-9B also serves as verifier, pairwise verification performs best among the verification mechanisms the authors evaluated.

The authors also report distilling responses from the stronger verifier into TMAX-9B. This further improves Pass@1 while leaving the action generator unchanged. In a separate comparison, combining action scaling with trajectory scaling reaches higher success at lower estimated token cost than generating more trajectories alone. The paper does not state a numerical cost estimate for that comparison. It says Mid-Harness also improves performance across additional models, benchmarks, and harnesses, without listing those results in its abstract. The paper was submitted to arXiv on September 30, 2026.

What the archive adds

New Runtime's HarnessTax analysis (October 22, 2026) argued that harness design is a model- and task-dependent control surface. Mid-Harness adds a new research result to that thread: selecting among candidate actions can change measured success even when the generator stays the same. This is a new fact on a known topic, not evidence that the method is already deployed in production.

Trend signal

This supports harness architecture outlives model choice: the paper's intervention changes what happens around a generator rather than replacing the generator. The result remains limited to an experimental setup, but it gives a concrete example of harness design affecting the outcome without changing the generating model.

Why now

The paper frames the work around test-time compute: increasing inference-time effort need not mean generating more full trajectories. Its experiments ask whether spending some of that effort on action selection can yield a better success-and-cost tradeoff. That makes the result relevant to teams comparing where to allocate inference budget in terminal-agent systems.

Why it matters

For a builder, the key implication is that a terminal agent's action policy need not be a single unreviewed model output. A harness can treat commands as candidates, inspect them, and execute only a selected action. That can matter especially when a command changes the environment and creates downstream consequences, as in the paper's package-install example.

The reported improvement is large enough to motivate a controlled evaluation, but it does not establish that adding a verifier will improve every workload. Teams would need to measure their own task success, execution failures, latency, and total inference cost. The paper's comparison with trajectory scaling suggests an alternative to simply running more complete attempts, while its dependence on verifier capability warns that candidate generation without effective selection may waste compute. The useful engineering question is therefore not merely how many actions to sample, but whether the selection step catches mistakes that matter on the team's tasks.

Limits and uncertainty

The headline numbers are the authors' reported results on TerminalBench-Lite; they are not independent replication results. The paper's abstract does not provide confidence intervals, per-task breakdowns, latency, or the numerical estimated token costs behind the action-and-trajectory comparison. It also does not specify operational details such as verifier prompts or the cost of verification. The authors say the approach improves results across additional settings, but the abstract does not identify those settings or quantify each gain. Production reliability and transfer to other terminal workloads remain unconfirmed.

Skill to build now

Take a small set of terminal-agent tasks your team can rerun, and log the generated command, the resulting environment change, and whether the task completed. For a subset, compare single-action execution with sampling several candidates and applying a verifier before execution. Track success alongside token use and latency. This turns the paper's central claim into a local test, rather than assuming a benchmark gain will transfer.

Reader takeaway

Mid-Harness treats command choice as a place to spend inference-time compute: generate alternatives, verify them, then execute one. Its TerminalBench-Lite result is promising, but the practical value depends on whether a verifier can reliably select better actions on your own tasks at acceptable cost.

Evidence / sources

  1. 1arxiv.org/abs/2609.39982
All Field Notes

Next reads

Read next

Open the graph
  1. 01topicAgent HarnessExplore the agent-harness topic hub.
  2. 02topicHarness EngineeringExplore the harness-engineering topic hub.
  3. 03topicVerificationExplore the verification topic hub.
  4. 04archiveField NotesOpen the latest editorial analysis.
  5. 05sourcesSourcesSee the original sources this archive cites.
Who read this page?Machine requests, hidden until opened

Loading the privacy-safe route aggregate...

Open the JSON contract