Field noteOct 20, 20262 min read1 source

ALTK-Evolve: How learned memory tops the τ²-bench leaderboard on customer service tasks without a frontier model

In one minute

**Thesis:** ALTK-Evolve suggests that an agent can improve policy-bound task reliability by converting experience into explicit, reusable behavioral guidance, without changing model weights.

Observed facts: IBM Research reports that GPT-4.1 retail Pass⁴ rose from 57.5% to 67.5% on held-out τ²-bench tasks when the full learned guideline set was appended to the prompt. In an airline test using GPT-5.5, the 103-guideline configuration raised Pass⁴ from 55% to 70%. The authors say the guidelines were distilled from training-task attempts and applied unchanged on held-out tasks.

Mechanism: The method distills behavioral rules from successful and failed training-task trajectories and appends those rules to the agent's system prompt; it does not update model weights.

Why now: The source tests both a non-reasoning GPT-4.1 agent and a GPT-5.5 airline setup, addressing whether learned guidance is useful only as a patch for a weaker model. The reported gains also extend to the GPT-5.5 airline setup.

Implications: The reported gains support treating memory as a policy-bearing interface between past attempts and future actions, rather than merely as a store of retrieved text. Teams can test whether distilled procedural rules improve repeatability without fine-tuning. The article also reports domain differences: a smaller high-support set was more reliable than the full set for GPT-4.1 airline tasks, so guidance volume may need tuning rather than maximizing.

Evidence boundary: These are results reported by the authors on τ²-bench, with 40 held-out retail tasks and 20 airline tasks in the GPT-4.1 experiments, plus 20 airline tasks for GPT-5.5. The benchmark uses simulated customers, written policies, and a strict task outcome; the source does not establish production customer-support impact. The reported benchmark results do not establish real-world production impact.

Watch condition: A replication should test whether the gains persist across customer-service systems, policy sets, and independently evaluated live workflows. Independent tests should check whether the gains persist outside τ²-bench.

Evidence / sources

  1. 1huggingface.co/blog/ibm-research/altk-evolve-bc
All Field Notes

Next reads

Read next

Open the graph
  1. 01topicMemoryExplore the memory topic hub.
  2. 02archiveField NotesOpen the latest editorial analysis.
  3. 03source ledgerSource LedgerInspect the public source evidence graph.
Who read this page?Machine requests, hidden until opened

Loading the privacy-safe route aggregate...

Open the JSON contract