In the authors’ τ²-bench tests, the full guideline set raised GPT-4.1 retail Pass⁴ from 57.5% to 67.5% on held-out tasks. In airline tasks using GPT-5.5, 103 guidelines raised Pass⁴ from 55% to 70%. The results are benchmark-specific: τ²-bench uses simulated customers, documented policies, and tool actions, and the reported test sets are small. The article also reports that the best guideline volume varies by domain, and that gains come with more prompt tokens rather than longer conversations.
ALTK-Evolve: How learned memory tops the τ²-bench leaderboard on customer service tasks without a frontier model
In one minute
IBM Research describes ALTK-Evolve, a method that distills behavioral guidelines from an agent’s training-task attempts and appends them to its system prompt without changing model weights.
What changed: Adds a customer-service benchmark result: ALTK-Evolve with GPT-4.1 scored 67.5% Pass^4 on τ²-bench retail, topping the October 12 leaderboard snapshot without model-weight updates.