{"schema_version":"newruntime-agent-readable-v0.2","type":"capability_delta","stable_id":"capability_delta:executable-outcome-checks-turn-agent-work-into-training-signals","slug":"executable-outcome-checks-turn-agent-work-into-training-signals","title":"Executable outcome checks turn agent work into training signals","description":"ThinkingBox uses executable checks of final database state for both agent benchmarks and reinforcement-learning rewards; paired with runtime traces and outcome evaluation, this makes completed work measurable beyond the model’s output.","retrieval_nugget":"ThinkingBox uses executable checks of final database state for both agent benchmarks and reinforcement-learning rewards; paired with runtime traces and outcome evaluation, this makes completed work measurable beyond the model’s output.","status":"published","observed_at":"2026-10-26","record_date":"2026-10-26","date_kind":"observed_at","before_capability":"Builders could evaluate agent responses, but the items do not describe final database state checks serving as both benchmarks and training rewards.","enabling_change":"ThinkingBox checks final database state with executable tests and uses them for benchmarking and reinforcement-learning rewards; runtime evaluation can also connect traces to outcomes.","after_capability":"Builders can score whether an agent achieved a task’s final state and use that result as an evaluation or training signal.","human_consequence":"Teams can assess agents by verified task outcomes rather than relying only on the plausibility of their responses.","verification_level":"source-inspected","buildability_class":"B2","bottleneck":"The checks must accurately encode the intended outcome; the items do not establish that this approach generalizes across tasks or systems.","related_shift":"verification-ownership","related_posts":[],"topics":["github"],"source_urls":["https://newruntime.com/signals/agent-quality-depends-on-closing-the-loop-between-execution-and-verifi-5d3319/","https://newruntime.com/signals/builders-are-turning-completed-outcomes-into-measurable-training-and-e-54937f/"],"featured":false,"supporting_evidence":[{"title":"Agent quality depends on closing the loop between execution and verification","url":"https://newruntime.com/signals/agent-quality-depends-on-closing-the-loop-between-execution-and-verifi-5d3319/","kind":"signal","observed_at":"2026-10-26"},{"title":"Builders are turning completed outcomes into measurable training and evaluation signals","url":"https://newruntime.com/signals/builders-are-turning-completed-outcomes-into-measurable-training-and-e-54937f/","kind":"signal","observed_at":"2026-10-26"}],"routes":{"html":"https://newruntime.com/deltas/executable-outcome-checks-turn-agent-work-into-training-signals/","markdown":"https://newruntime.com/deltas/executable-outcome-checks-turn-agent-work-into-training-signals.md","json":"https://newruntime.com/deltas/executable-outcome-checks-turn-agent-work-into-training-signals.json"},"source_format":"knowledge-ledger","published_at":"2026-11-02T06:04:01.021Z","auto_created":true,"editorial_provenance":{"desk":"knowledge-revision","provider":"codex","model":"gpt-6-luna","created_at":"2026-11-02T06:04:01.021Z","week_start":"2026-10-26","review":"auto-created from reviewed evidence; the owner may revise or retire it"}}