Field noteOct 26, 20261 min read1 source

Knowing How Is Not Knowing What (14 minute read)

In one minute

Yunze Xiao published an essay on RL environment design for computer-use agents.

What changed: It argues RL environments should test organizational know-how and goal achievement, not just software operation—an extension of prior critiques of benchmark design and agent evaluation.

It argues that executing a software procedure should not be treated as equivalent to completing an organizational task. In the examples, sending a headcount request does not secure an authorized position, and submitting an $8,000 refund does not satisfy a policy requiring a second approval above $5,000.

The proposed environment includes more than application interfaces and task instructions. Xiao calls for persistent organizational state: roles, authority, budgets, deadlines, documents with cross-references, messages, and domain rules that may be unwritten. Actions should change that state across tool calls. Evaluation should inspect the resulting world, including whether an authorized approval occurred and whether required resources exist.

The essay also proposes treating real company records as raw material rather than directly reusable environments, because archives may contain personal data, cannot readily be reset or varied, and record past events rather than a counterfactual definition of correct work. It does not release a benchmark, dataset, environment, model, or experimental results establishing that this approach improves agent performance.

Evidence / sources

  1. 1algoroxyolo.github.io/blog/2026/knowing-how-is-not-knowing-what
All Field Notes

Next reads

Read next

Open the graph
  1. 01topicAgent LoopsExplore the agent-loops topic hub.
  2. 02topicAutomationExplore the automation topic hub.
  3. 03archiveField NotesOpen the latest editorial analysis.
  4. 04source ledgerSource LedgerInspect the public source evidence graph.
Who read this page?Machine requests, hidden until opened

Loading the privacy-safe route aggregate...

Open the JSON contract