An agent says it has finished. That sentence is becoming less important.
As AI systems receive longer runtimes, broader tools, and permission to revise their own work, a completion message can no longer serve as proof. The executor knows what it attempted and can produce a persuasive account of the result. It does not automatically know whether the result fits the organization, generalizes beyond the visible test, respects an unstated constraint, or deserves to alter a durable record.
The emerging operating law is simple:
The more freedom an executor receives, the less authority its own completion verdict should carry.
This does not mean that every output needs a larger manual review queue. It means that autonomy grows by separating execution from acceptance. Attempts remain inside the agent loop. Held-out tests, independent graders, monitors, accountable judgment, and the right to write back into a system of record stay outside it.
That separation is now visible across coding, scientific workflows, model training, alignment research, and production code review. The individual implementations differ, but they are converging on the same architecture: the agent may own the attempt; the institution owns the definition and proof of success.
The completion message is not the evidence
Databricks built an internal coding-agent benchmark from recent, human-written changes to its own multi-million-line codebase. The tasks cover the languages and build systems its engineers actually use. Test files are withheld from the agent and patched back only after the agent declares completion. Databricks says it evaluated every benchmark sample by hand, rewrote weak tests when necessary, and deliberately avoided an LLM judge because fluent explanations can be rewarded even when the implementation is wrong.
The strongest detail is not a model score. Early agents could recover the original solution from Git history. Databricks had to seal the worktree off from the repository for the run. The benchmark became credible only when the institution controlled what the executor could see, what counted as a pass, and which evidence was admissible.
This changes what a completion event means. The agent’s statement creates a checkpoint. External machinery decides whether that checkpoint is accepted.
Scientific work makes the artifact primary
Terminal-Bench-Science pushes the same boundary into scientific workflows. Its first release contains 70 tasks drawn from the life, physical, earth, mathematical, and engineering sciences. Agents are graded on concrete artifacts—analyses, simulations, proofs, code, and data products—using reproducible task-specific tests. The leading evaluated system resolved 30% of the tasks.
The construction funnel is revealing. The project reports 920 proposals, 464 approvals for implementation, 386 opened pull requests, and only 70 tasks in the first release. Domain reviewers, technical reviewers, and a final bar raiser had to decide whether a task was scientifically meaningful, objectively verifiable, and still difficult for frontier systems.
The scarce object was not another agent answer. It was a well-formed acceptance environment.
That extends the existing pattern that scientific agents need artifact evidence. A useful scientific agent does not merely return a conclusion. It leaves a procedure, artifact, environment, and test that another researcher can inspect, rerun, or challenge.
The evaluator becomes a different system
Anthropic’s work on automated alignment researchers makes the separation explicit. Five research agents can iterate in parallel for up to 48 hours, propose a method, write code, train a model under a compute cap, and submit the resulting weights. A separate evaluator loads those weights and returns scores. Held-out data lives under a different operating-system account that the research process cannot read. A monitor checks the submitted code against integrity constraints.
The agent loop is long and adaptive, but the acceptance semantics do not live inside it. The researchers can optimize attempts; they cannot inspect the held-out benchmark, edit the evaluator, or grant themselves a capability pass.
The result is strong inside its boundary: across ten well-characterized alignment failures, selected methods improved held-out results while preserving the measured capability checks. It is not evidence that autonomous researchers can safely choose open-ended research goals or validate arbitrary discoveries. Anthropic’s own design depends on failures that can be specified, scored, isolated, and monitored.
That limitation matters. External evaluation is not a magic truth machine. It is a way to make a bounded institutional decision repeatable.
Verification rules are productive infrastructure
Thinking Machines Lab’s Text-to-SQL work shows that acceptance rules do more than catch bad outputs after the fact. They shape what a model learns.
An LLM auditor was precise when it flagged annotation errors, but caught only 24.5% of the errors human experts found. In a separate analysis, 32.8% of positive execution-match rewards were attached to SQL queries that were not semantically equivalent to the correct query. The output happened to match on one database instance, so a shallow success rule reinforced the wrong behavior.
The team changed the environment: expert-verified data, bounded semantic verification, and rule-based process rewards became part of training. The important unit was no longer a generated query or even a matching result. It was a result accepted under a more faithful definition of correctness.
This is where verification bandwidth becomes more than a bottleneck. A good verifier is a productive asset. It turns expert judgment into a reusable gradient, gate, or contract. A weak verifier scales the wrong behavior faster.
Production systems already use layered acceptance
At Uber, uReview reportedly analyzes more than 90% of roughly 65,000 weekly diffs. Its architecture does not trust a single review prompt. Specialized assistants generate comments; other stages grade, filter, validate, and deduplicate them. Developers can mark comments as useful, and whether a comment was addressed becomes another feedback signal.
Uber reports that 75% of rated comments are marked useful and more than 65% of posted comments are addressed. The system can catch local bugs at scale, but Uber also states that it is much weaker at judging overall system design because it lacks broader artifacts such as past pull requests, feature flags, schemas, and technical documentation.
That is an honest boundary. Acceptance depends on the context available to the evaluator. The more consequential the decision, the more the institution must own that context rather than assume the model inferred it.
The corresponding organizational shift from code production to verification ownership is already underway. Engineers increasingly own specifications, evidence, risk thresholds, and release consequences even when an agent produces most of the implementation.
Recursive improvement does not remove the gate
Cline’s account of a recursive harness-improvement campaign is a useful stress test. One prompt initiated roughly 17 hours of agent work that read traces, formed hypotheses, modified the harness, ran evaluations, and improved the reported Terminal-Bench result. The campaign consumed roughly one billion tokens. Invalid runs were excluded and repeated; the final pull request was reviewed by people before merge.
This is vendor-reported evidence from one bounded campaign, not proof of generally self-improving software. Yet the mechanism is important. Longer autonomy became possible because the run had a fixed goal, observable traces, a benchmark, attribution rules, invalidation criteria, and a final merge boundary. The agent improved the machinery that generated attempts. It did not acquire unilateral authority to redefine success or ship the result.
That is also why agent economics is moving toward completed work. Tokens, turns, and elapsed time matter, but their value depends on whether an external acceptance system can recognize a completed task and preserve the evidence behind it.
The new durable object is an acceptance contract
Across these cases, the evaluator is not one component. It is a contract connecting several layers:
- Task identity. What state is supposed to change?
- Admissible evidence. Which artifacts, traces, measurements, and sources count?
- Isolation. Which answers, tests, credentials, or records must remain unavailable to the executor?
- Evaluation. Which deterministic checks, statistical graders, model judges, and human decisions apply?
- Decision rights. Who may accept an exception, revise a threshold, or approve an under-specified result?
- Writeback. What accepted state enters code, production, research memory, or another system of record?
- Revision. What new failure would cause the contract itself to change?
The contract can survive a model swap. It can also expose when a harness improvement raises a benchmark score by exploiting the task rather than improving the underlying work. This is the beginning of a stable control plane around fluid execution.
When goals enter the org chart, the goal needs more than memory, tools, and a runtime. It needs this external acceptance contract. Otherwise the organization has created a persistent executor without a durable definition of completion.
From human-in-the-loop to institution-in-the-loop
“Human-in-the-loop” usually describes an interruption: the system asks a person to approve, correct, or complete a step. That remains necessary for many decisions, but it is not the larger change.
The emerging form is institution-in-the-loop. People do not need to inspect every low-risk attempt. They design and revise the rules that can change reality: task boundaries, evidence standards, isolation, thresholds, escalation paths, decision rights, and writeback. Machines can perform most attempts and some checks. The institution remains present through the acceptance environment even when no person clicks approve on every run.
This is a forecast, not a measured cross-industry norm. The current evidence is concentrated in software, benchmarks, model research, and other artifact-rich domains. Many valuable decisions are political, relational, physical, ambiguous, or difficult to reduce to stable graders. Automated acceptance can also become procedural theater: a large evidence bundle may preserve the wrong assumptions with more confidence.
The forecast should be revised if broader autonomy succeeds without independent evidence, if model self-evaluation becomes as reliable as protected external evaluation, or if organizations cannot maintain acceptance contracts as systems and tasks evolve.
For now, the practical direction is narrower. Before granting an agent a longer runtime or stronger permissions, design the evidence and writeback path first. Let the executor own attempts. Keep the authority to declare reality somewhere else.