The LessWrong essay *The Expensive-Loop Problem* distinguishes “cheap-loop” environments, where experiments are quick, deterministic, and cheap, from expensive-loop environments constrained by real-world cost or delay. It says high-value work in science, business, public policy, medicine, and AI research often falls into the latter category. The essay makes this distinction and names proxy selection, theory-building, resource awareness, and error correction as relevant demands.
Thesis
Performance on cheap, rapidly verifiable tasks does not establish capability in domains where feedback is costly or delayed.
Observed facts
Mechanism
When direct evaluation is expensive, an agent must choose cheaper proxies, infer how they relate to the real objective, and revise them when they stop transferring; otherwise it can optimize the proxy rather than the target or waste resources on low-value experiments.
Why now
As AI systems are considered for work beyond easily scored coding or other computer-checkable tasks, the distinction helps explain why benchmark success may not predict performance where experiments are costly and feedback sparse.
Implications
First implication: teams evaluating agents for such work should test proxy choice, experiment value, and resource allocation—not only final scores on cheap checks.
Evidence boundary
This is an author’s conceptual argument, not a new empirical study or a measured model comparison.
Watch conditions
First watch condition: look for evaluations in expensive-loop settings that report how agents choose proxies, update theories, allocate resources, and correct errors when feedback is limited.