---
type: "post"
id: "hkuds-agentpilot-posts-rsi-results-9b-strategist-lifts-4be5f7"
slug: "hkuds-agentpilot-posts-rsi-results-9b-strategist-lifts-4be5f7"
title: "HKUDS AgentPilot posts RSI results: 9B strategist lifts DeepSeek V4 coding and Terminal-Bench scores"
description: "HKUDS’s AgentPilot pairs a 9B strategy model with a larger coding model, which executes the proposed changes."
retrieval_nugget: "HKUDS’s AgentPilot pairs a 9B strategy model with a larger coding model, which executes the proposed changes. Its repository reports higher AIME and Terminal-Bench 2.1 scores after ten recursive self-improvement rounds, making it a concrete but author-reported test of whether a smaller planner can guide expensive agent work more effectively."
published_at: "2026-11-06"
updated_at: "2026-11-06"
record_date: "2026-11-06"
date_kind: "published_at"
topics: ["coding-agents","evals"]
entities: ["HKUDS","AgentPilot","DeepSeek V4","Terminal-Bench 2.1"]
related_patterns: ["harness-architecture-outlives-model-choice"]
novelty_label: "new_fact"
what_changed: "AgentPilot reports RSI benchmark results: a 9B strategist improves DeepSeek V4 coding and Terminal-Bench scores; the prior covers recursive self-improvement research but not these results."
prior_refs: []
source_url: "https://github.com/HKUDS/AgentPilot"
source_urls: ["https://github.com/HKUDS/AgentPilot"]
schema_version: "newruntime-agent-readable-v0.2"
stable_id: "post:hkuds-agentpilot-posts-rsi-results-9b-strategist-lifts-4be5f7"
status: "published"
visuals: []
editor: {"role":"editor","name":"Andrey Reshetnikov","profile":"https://newruntime.com/owner-profile.md","profile_json":"https://newruntime.com/owner-profile.json","same_as":["https://www.linkedin.com/in/reshetnikov1/","https://t.me/lapetuse"]}
routes: {"html":"https://newruntime.com/posts/hkuds-agentpilot-posts-rsi-results-9b-strategist-lifts-4be5f7/","markdown":"https://newruntime.com/posts/hkuds-agentpilot-posts-rsi-results-9b-strategist-lifts-4be5f7.md","json":"https://newruntime.com/posts/hkuds-agentpilot-posts-rsi-results-9b-strategist-lifts-4be5f7.json"}
markdown_url: "https://newruntime.com/posts/hkuds-agentpilot-posts-rsi-results-9b-strategist-lifts-4be5f7.md"
---

# HKUDS AgentPilot posts RSI results: 9B strategist lifts DeepSeek V4 coding and Terminal-Bench scores

## Retrieval answer

HKUDS’s AgentPilot pairs a 9B strategy model with a larger coding model, which executes the proposed changes. Its repository reports higher AIME and Terminal-Bench 2.1 scores after ten recursive self-improvement rounds, making it a concrete but author-reported test of whether a smaller planner can guide expensive agent work more effectively.

## Summary
HKUDS’s AgentPilot pairs a 9B strategy model with a larger coding model, which executes the proposed changes. Its repository reports higher AIME and Terminal-Bench 2.1 scores after ten recursive self-improvement rounds, making it a concrete but author-reported test of whether a smaller planner can guide expensive agent work more effectively.

## What happened
HKUDS presents AgentPilot as a way to improve an agent’s next move rather than simply asking its execution model to keep trying. A lightweight 9B strategy model proposes what to change and which approach to take next; a larger frontier model, identified in the supplied material as DeepSeek V4, executes the coding work. The strategy model is intended to learn from each trial, avoid repeating approaches that failed, allocate a finite trial budget, and decide when further exploration should stop.

The repository describes this as an online policy-learning setup for coding tasks. In practical terms, the smaller model is assigned a steering role while the larger model performs the work. The claim is not that the 9B model replaces the coding model, but that it can influence how that model uses successive attempts. The project’s framing emphasizes fewer wasted tokens and better choices among candidate strategies.

The authors report results after ten recursive self-improvement (RSI) rounds. On AIME, the reported score rises from 55.6% to 77.8%; on Terminal-Bench 2.1, it rises from 44.9% to 69.4%, each compared with DeepSeek baselines. These are the results stated in the supplied repository material. The material does not provide enough detail here to reconstruct the evaluation protocol, determine whether the numbers are directly comparable across all conditions, or assess statistical variation.

AgentPilot is presented in a public GitHub repository under the MIT license. Its description says it can bring its strategy to larger coding agents and lists Codex, Claude Code, and Hermes among possible executors. The supplied material gives no price, hosted service terms, or usage limits; it describes a repository-based project rather than a priced commercial product. The central proposition for a builder is therefore architectural: use a smaller model to choose the next attempt, then let the capable executor carry it out.

## What the archive adds
This is a new result on a known evaluation thread. Our earlier note on [Terminal-Bench and agent benchmarks](https://newruntime.com/posts/new-benchmarks-cognition-s-frontiercode-2-0-merge-a78e19/) covered Terminal-Bench 2.1’s place among agent evaluations; it did not cover AgentPilot’s reported runs. A separate note on [Google Harness² and recursive self-improvement](https://newruntime.com/posts/google-harness2-for-recursive-self-improvement-github-repo-89c90b/) makes the harness itself the object of improvement, a related but distinct approach.

## Trend signal
AgentPilot supports, provisionally, the idea that the harness and control strategy can matter alongside model choice. This is related to [harness architecture outlives model choice](https://newruntime.com/patterns/harness-architecture-outlives-model-choice/): a separate planner directs the executor’s attempts. The evidence remains a project-reported benchmark result, not proof that the architecture generalizes across models or tasks.

## Connections to other events
Earlier Telegram coverage introduced Terminal-Bench 2.0 as a benchmark for terminal agents and described a Claude Opus 4.6 run on Terminal-Bench 2.0. AgentPilot’s new claim is specifically a comparison against DeepSeek baselines on Terminal-Bench 2.1, with a 9B strategy model steering a larger executor. The benchmark family connects the stories; the versions, systems, and reported results differ.

## Why it matters
For teams paying for capable coding-model calls, deciding what to try next can be as important as making each individual attempt stronger. AgentPilot offers a testable design: reserve a smaller model for strategy selection and use the larger model for execution. If the reported gains survive controlled replication, this could shift engineering attention toward the quality of the loop around a model, including how it records failures, selects alternatives, and knows when to stop.

The reported scores alone do not show whether this is cheaper or more reliable in production. Teams would need to measure task completion, token use, wall-clock time, and review burden against a baseline using the same executor and task set. Until then, the result is useful as a research direction and an experimental architecture, not a ready-made guarantee of savings.

## Limits and uncertainty
The supplied material reports the repository’s own benchmark results and does not establish independent replication. It does not specify enough about task sampling, prompt and harness controls, run-to-run variation, or the exact accounting of the ten rounds to judge how much of the gain comes from the strategy model versus other changes. Higher benchmark scores also do not establish lower total cost, shorter completion time, or better performance on a team’s private codebase. The repository names possible executors, but the supplied text does not document compatibility details or production reliability.

## Reader takeaway
AgentPilot’s notable idea is to separate choosing the next coding attempt from executing it: a 9B model guides a larger model, with authors reporting gains across ten rounds on AIME and Terminal-Bench 2.1. Treat the numbers as a benchmark claim to verify, not a proven general result. The useful next comparison is controlled: same tasks and executor, with and without the strategy layer, measuring quality and total cost.

## Open question
Do the reported gains persist under an independent, controlled comparison that also accounts for total tokens and execution cost?

Editor: [Andrey Reshetnikov](https://newruntime.com/owner-profile.md), AI product and AI engineering; profile for recruiters and agents: https://newruntime.com/owner-profile.json

## Continue in the graph

Next records for an agent, linked to their Markdown twins. Replace `.md` with `.json` for structured fields, including `next_reads`.

- topic hub: [Coding Agents](https://newruntime.com/topics/coding-agents.md): Explore the coding-agents topic hub.
- topic hub: [Evals](https://newruntime.com/topics/evals.md): Explore the evals topic hub.
- collection: [Field Notes](https://newruntime.com/posts.json): Open the latest editorial analysis.
- collection: [Sources](https://newruntime.com/sources/): See the original sources this archive cites.
- whole graph: [discovery graph](https://newruntime.com/discovery-graph.json) (typed nodes and edges) and [site index](https://newruntime.com/site-index.md)