What happened
The public model card from harness-hub publishes PEFT LoRA adapters for LiquidAI’s LFM2.5-24B-A2B. The release is specifically named for pi, a coding-agent harness. Its repository metadata identifies the base model as LiquidAI/LFM2.5-24B-A2B, lists PEFT and Safetensors as the packaging libraries, and tags the release with TRL, GRPO, asynchronous GRPO, Harbor, OpenEnv, and pi. The repository is public and ungated; it reports no inference providers.
The training setup is the central claim. Rather than grading a short isolated completion, harness-hub says a rollout was a full pi coding-agent session in a remote sandbox. The sessions worked on data-analysis tasks and were graded by a verifier. The card also describes a separate Claude Code variant, but the published result in this release concerns the pi adapter. Training used TRL’s asynchronous GRPO method and LoRA adapters, leaving the named base model intact rather than publishing a wholly retrained checkpoint.
The reported evaluation is bounded. On the first 100 tasks of FineEnvs/SmolDataEnvs-harbor-test, the pi adapter’s pass@1 was 4.2 percentage points below the base model. The card characterizes that difference as within noise, approximately plus or minus 5 points. For tasks that both the adapted and base systems solved, however, the adapted system made 39% fewer tool calls. This is not a claim that the adapter improves success rate; it is a claim about the number of tool interactions conditional on shared successful outcomes.
The release exposes some unfavorable details rather than presenting a single best score. harness-hub publishes the checkpoint from step 100 because accuracy fell at step 110. It also says lower-learning-rate runs and experiments with other harnesses moved little. For deployment, it provides GGUF versions of the adapter, described as about 37 MB, for llama.cpp. The card says those GGUF versions were not re-evaluated. It gives serving instructions for vLLM using the LFM2 tool-call parser and for llama-server using --jinja; those instructions are implementation guidance, not a claim of equivalent measured performance.
What the archive adds
This is a new fact on a known story. The earlier note on Hermit covered a framework for training harness adapters while leaving both the model and harness unchanged. harness-hub instead publishes an adapter tied to one named base model and the pi harness, along with a cost-oriented tool-call result.
Trend signal
This supports Harness architecture outlives model choice. The result only exists in a particular agent environment: full pi sessions, sandboxed tasks, verifier grading, a tool-call parser, and a selected checkpoint. The adapter is not presented as a general capability upgrade. It is an attempt to alter the behavior of a model inside a harness, which is precisely why its result must be evaluated at that boundary.
Why it matters
Teams operating agents pay for more than tokens. A tool call can trigger latency, a browser or code-execution action, a rate-limited API request, or a side effect that needs inspection. A 39% reduction in calls on jointly solved tasks would therefore matter if it holds under a team’s own workload and tool pricing. But this release also illustrates the appropriate engineering framing: optimize a measurable operational behavior without silently converting it into a quality claim.
The practical choice is not simply whether to use this adapter. Builders can treat it as an example of an evaluation design: run the whole harness, use an external verifier, report success separately from cost, and retain checkpoints whose trade-offs are understandable. The disclosed dip at step 110 is useful because it shows that more optimization steps did not automatically produce a better operating point.
Limits and uncertainty
The source reports a result on the first 100 tasks of one named test set, not on production repositories or a broad coding benchmark. The 39% figure applies only to tasks both systems solved, so it does not establish lower cost per attempted task or lower cost per completed task across all inputs. The reported pass@1 difference is described by the authors as within noise, but the supplied material does not provide confidence intervals, raw task outcomes, evaluation seeds, tool pricing, latency, or independent replication. Nor does it establish that the GGUF adapters preserve the measured behavior, since they were not re-evaluated.
Skill to build now
Add a harness-level cost table to one agent evaluation this week. For each task, record whether it was verified as solved, how many tool calls it made, and which tools it invoked. Compare a baseline and a changed configuration on the intersection of solved tasks, as this release does, but also report all-task completion and total cost separately. Keep the checkpoint or configuration identifier with every run. That makes a promising reduction in agent activity inspectable rather than anecdotal.
Reader takeaway
The notable part of this release is not a new general-purpose model. It is an open, harness-specific attempt to use reinforcement learning to reduce tool activity while retaining roughly comparable measured task success on a limited evaluation. Treat the 39% number as a hypothesis to reproduce in your own harness, not as a portable efficiency guarantee.
Open question
Will the reduction persist on real repositories and heterogeneous tools once failures, retries, latency, and verification work are included in the cost measure?