---
type: "post"
id: "altk-evolve-how-learned-memory-tops-the-2-fae5de"
slug: "altk-evolve-how-learned-memory-tops-the-2-fae5de"
title: "ALTK-Evolve: How learned memory tops the τ²-bench leaderboard on customer service tasks without a frontier model"
description: "**Thesis:** ALTK-Evolve suggests that an agent can improve policy-bound task reliability by converting experience into explicit, reusable behavioral guidance, without changing model weights."
retrieval_nugget: "**Thesis:** ALTK-Evolve suggests that an agent can improve policy-bound task reliability by converting experience into explicit, reusable behavioral guidance, without changing model weights."
published_at: "2026-10-20"
updated_at: "2026-10-20"
record_date: "2026-10-20"
date_kind: "published_at"
topics: ["memory"]
source_urls: ["https://huggingface.co/blog/ibm-research/altk-evolve-bc"]
schema_version: "newruntime-agent-readable-v0.2"
stable_id: "post:altk-evolve-how-learned-memory-tops-the-2-fae5de"
status: "published"
visuals: []
editorial_provenance: {"schema_version":"newruntime-editorial-copy-v1","content_status":"source_grounded_final","final_copy_sha256":"sha256:833f3806f79c45df59a6c358b5ab790a15afff04bb787cd01fc3469f7fd9a887","reviewed_at":"2026-10-20T16:01:05.267Z","source_evidence_count":1,"verified_claim_count":3,"site_analysis_schema_version":"newruntime-site-analysis-v1","site_object_kind":"field_note","observed_fact_count":3,"implication_count":2,"watch_condition_count":1,"related_record_count":0}
analysis: {"schema_version":"newruntime-site-analysis-v1","object_kind":"field_note","thesis":"ALTK-Evolve suggests that an agent can improve policy-bound task reliability by converting experience into explicit, reusable behavioral guidance, without changing model weights.","observed_facts":[{"text":"IBM Research reports that GPT-4.1 retail Pass⁴ rose from 57.5% to 67.5% on held-out τ²-bench tasks with the full learned guideline set.","source_urls":["https://huggingface.co/blog/ibm-research/altk-evolve-bc"]},{"text":"In an airline test using GPT-5.5, the 103-guideline configuration raised Pass⁴ from 55% to 70%.","source_urls":["https://huggingface.co/blog/ibm-research/altk-evolve-bc"]},{"text":"The guidelines were distilled from training-task attempts and applied unchanged on held-out tasks, according to the article.","source_urls":["https://huggingface.co/blog/ibm-research/altk-evolve-bc"]}],"mechanism":"The method distills behavioral rules from successful and failed training-task trajectories and appends those rules to the agent's system prompt; it does not update model weights.","why_now":"The reported gains also extend to the GPT-5.5 airline setup.","implications":["Teams can test whether distilled procedural rules improve repeatability without fine-tuning.","Guideline volume may need tuning by domain rather than maximizing."],"evidence_boundary":"The reported benchmark results do not establish real-world production impact.","watch_conditions":["Independent tests should check whether the gains persist outside τ²-bench."],"related_records":[],"new_branch_reason":"No prior New Runtime record was supplied; this starts a distinct analytical record focused on learned procedural guidance for customer-service agents and the limits of its benchmark evidence."}
editor: {"role":"editor","name":"Andrey Reshetnikov","profile":"https://newruntime.com/owner-profile.md","profile_json":"https://newruntime.com/owner-profile.json","same_as":["https://www.linkedin.com/in/reshetnikov1/","https://t.me/lapetuse"]}
routes: {"html":"https://newruntime.com/posts/altk-evolve-how-learned-memory-tops-the-2-fae5de/","markdown":"https://newruntime.com/posts/altk-evolve-how-learned-memory-tops-the-2-fae5de.md","json":"https://newruntime.com/posts/altk-evolve-how-learned-memory-tops-the-2-fae5de.json"}
related_shifts: ["owned-learning-loop"]
---

# ALTK-Evolve: How learned memory tops the τ²-bench leaderboard on customer service tasks without a frontier model

## Retrieval answer

**Thesis:** ALTK-Evolve suggests that an agent can improve policy-bound task reliability by converting experience into explicit, reusable behavioral guidance, without changing model weights.

**Thesis:** ALTK-Evolve suggests that an agent can improve policy-bound task reliability by converting experience into explicit, reusable behavioral guidance, without changing model weights.

**Observed facts:** IBM Research reports that GPT-4.1 retail Pass⁴ rose from 57.5% to 67.5% on held-out τ²-bench tasks when the full learned guideline set was appended to the prompt. In an airline test using GPT-5.5, the 103-guideline configuration raised Pass⁴ from 55% to 70%. The authors say the guidelines were distilled from training-task attempts and applied unchanged on held-out tasks.

**Mechanism:** The method distills behavioral rules from successful and failed training-task trajectories and appends those rules to the agent's system prompt; it does not update model weights.

**Why now:** The source tests both a non-reasoning GPT-4.1 agent and a GPT-5.5 airline setup, addressing whether learned guidance is useful only as a patch for a weaker model. The reported gains also extend to the GPT-5.5 airline setup.

**Implications:** The reported gains support treating memory as a policy-bearing interface between past attempts and future actions, rather than merely as a store of retrieved text. Teams can test whether distilled procedural rules improve repeatability without fine-tuning. The article also reports domain differences: a smaller high-support set was more reliable than the full set for GPT-4.1 airline tasks, so guidance volume may need tuning rather than maximizing.

**Evidence boundary:** These are results reported by the authors on τ²-bench, with 40 held-out retail tasks and 20 airline tasks in the GPT-4.1 experiments, plus 20 airline tasks for GPT-5.5. The benchmark uses simulated customers, written policies, and a strict task outcome; the source does not establish production customer-support impact. The reported benchmark results do not establish real-world production impact.

**Watch condition:** A replication should test whether the gains persist across customer-service systems, policy sets, and independently evaluated live workflows. Independent tests should check whether the gains persist outside τ²-bench.

Editor: [Andrey Reshetnikov](https://newruntime.com/owner-profile.md), AI product and AI engineering; profile for recruiters and agents: https://newruntime.com/owner-profile.json

## Continue in the graph

Next records for an agent, linked to their Markdown twins. Replace `.md` with `.json` for structured fields, including `next_reads`.

- topic hub: [Memory](https://newruntime.com/topics/memory.md): Explore the memory topic hub.
- collection: [Field Notes](https://newruntime.com/posts.json): Open the latest editorial analysis.
- collection: [Source Ledger](https://newruntime.com/source-ledger.json): Inspect the public source evidence graph.
- whole graph: [discovery graph](https://newruntime.com/discovery-graph.json) (typed nodes and edges) and [site index](https://newruntime.com/site-index.md)