Verifiable AI Coding Workflow
A small agent-assisted coding workflow where the useful artifact is not generated code, but a checked path from idea to tests, diff, review, and handoff.
- Buildability
- B2 engineer-repeatable
- Status
- candidate
- Time
- 1-2 hours in a prepared repository
- Cost
- Low to medium, depending on model and test loop length
- Human approval
- Local file edits only; commits and deploys remain explicit approvals
- Last verified
- Oct 26, 2026
Outcome demo
Turn an implementation idea into a bounded coding task with checks and reviewable evidence. Wow moment: The agent does not just produce a patch; it leaves commands, changed files, acceptance checks, and a human-readable handoff. Current bottleneck: Requires a repo with runnable checks and a human willing to enforce task boundaries. This New Runtime record is an evidence-linked retrieval unit.
Can I build this?
- Code level
- Developer or technical operator
- Data touched
- Repository source, tests, local command output
- Write actions
- Local file edits only; commits and deploys remain explicit approvals
- Sources
- 4 public receipts
System map
- Human
- Defines scope, approves write actions, reviews output.
- Agent
- Plans, edits, verifies, and reports with evidence.
- Data
- Repository source, tests, local command output
- Actions
- Local file edits only; commits and deploys remain explicit approvals
What has to reproduce
The JSON route exposes proof, handoff, permissions, acceptance checks, and source URLs in one replayable contract.
- Verification
- locally reproduced
- Outputs
- 6 fixtures
- Replay
- 2 receipts
- Axes
- 7 assessed
Fixture outputs
- Project JSON contract
dist/projects/verifiable-ai-coding-workflow.jsonThe JSON route exposes proof, handoff, permissions, acceptance checks, and source URLs in one replayable contract.
Open route - Build brief handoff
dist/projects/verifiable-ai-coding-workflow/build-brief.mdThe build brief preserves scope, non-goals, build steps, questions, and approval gates without requiring the original chat.
Open route - Acceptance checklist
dist/projects/verifiable-ai-coding-workflow/acceptance.mdThe checklist separates required checks, permission scopes, reproduction commands, known limits, and failure conditions.
Open route - Clean-room replay fixture
scripts/project-proof-depth-v2-fixture-test.mjsThe fixture replays a bounded coding task in a temporary directory, proves the initial failure and final pass, and verifies that an unrelated file is unchanged.
- Independent replay kit
scripts/project-proof-independent-replay-kit.mjsThe kit starts failing, exposes no ready solution, protects every out-of-scope file by checksum, and refuses to emit a receipt without a non-author attestation and a passing final check.
- Independent agent replay receipt
src/data/project-proof-receipts/verifiable-ai-coding-workflow-independent-agent-v1.jsonA separately operated Codex received only the isolated kit, observed the initial failure, changed only src/add.mjs, passed the final check, and produced an attestation that the primary verifier accepted.
Open route
Replay commands
npm run project-proof:replay:testnpm run project-proof:replay:preparenpm run validate:contentnpm run buildnpm run validate:agent-indexes
Provider scopes
- Local repositorywrite
Focused source, test, or documentation files inside the shared worktree.
The workflow proves a bounded implementation loop through local edits and checks.
Human names the task and repository before edits start.
- Git remotepublish
Focused commit and push only when explicitly requested.
Some workflows end in a reviewable remote branch or production release, but this is not automatic.
Commit, push, deploy, and external writes require a separate explicit owner command.
- Test runner or local dev serverread
Named checks and local rendered routes needed for the accepted task.
Model output is not evidence until a command, artifact, or rendered page confirms the claim.
Human or repo docs identify which checks count for acceptance.
Seven-axis Buildability
non author replayed · 2026-08-06
- timehigh
- The clean-room task reaches a checked result in one bounded local run, while the public blueprint targets first value within one to two hours.
- BottleneckRepository setup and test availability still determine the real start time.
- code burdenmedium
- The workflow needs a developer-readable task and a focused source edit, but no bespoke application or new service.
- BottleneckA nontechnical owner still needs an agent or engineer to interpret failures.
- integration burdenhigh
- The replay uses only repository files, Node, a test runner, and the existing agent handoff contract.
- BottleneckRepositories without a runnable local check need an integration step before replay.
- operational burdenhigh
- The first proof is local and temporary, requires no server, and deletes no shared or production state.
- BottleneckLong-running or deployment tasks would need additional recovery and monitoring rules.
- permission clarityhigh
- Read, local-write, commit, push, deploy, and external-write boundaries are named separately in the build brief and acceptance contract.
- BottleneckA broader task can still create ambiguity unless exact files and approvals are fixed before editing.
- reproducibilityhigh
- A separately operated Codex received only the isolated public kit, reproduced the initial failure, changed the one allowed file, and passed the same acceptance check without inspecting the author solution.
- BottleneckA real external repository replay would test integration burden beyond this bounded fixture.
- failure recoveryhigh
- The replay records the initial failing check, preserves an unrelated file, performs no remote action, and leaves explicit residual-risk and approval sections.
- BottleneckProduction rollback and credential recovery are outside this local proof.
Replay evidence
Bounded receipts, with limitations kept visible.
automated clean room · 2026-08-06
npm run project-proof:replay:test- This replay proves the deterministic local contract and failure boundary, not a real external repository integration.
- No human or separately operated agent independently interpreted the brief in this first replay.
independent agent · 2026-08-06
node --test test/add.test.mjs- This proves a bounded non-author replay of the public workflow contract, not integration into a production repository.
- Reviewer identity is a local agent attestation and is not cryptographically verified.
- The workflow needs a repository with runnable checks; otherwise the blocker must be explicit and reproducible.
- It does not replace human review for broad architecture changes, credentials, payments, or publication.
- The proof is strongest for bounded tasks; exploratory research still needs a separate evidence workflow.
- The independent replay proves the bounded public workflow contract, not integration into a real external or production repository.
- The independent reviewer identity is a local agent attestation and is not cryptographically verified.
Fork with an agent
Use the build brief for execution context, the acceptance checklist for verification, and the JSON route for structured constraints.
Current state, as of Oct 20, 2026
Where this stands now
strengthening The workflow remains a practical way to turn coding ideas into bounded tasks with checks and reviewable evidence. Recent examples include LangChain’s conversion of documentation samples into runnable tests, divided between agent instructions and continuous integration.
The core workflow is still to define scope and acceptance checks, let an agent inspect and make a focused change, run a relevant check, and hand over evidence for human review. A passing command is evidence about what was checked, not universal proof.
Examples now span documentation samples converted into runnable, testable files, with work divided between agent instructions and continuous integration, and earlier build-review and testing workflows. For builders, the evidence supports adapting this pattern to a specific task while recording its environment, assumptions, and remaining human decisions.
What changed since the original text
- LangChain described turning inline documentation code samples into runnable, testable files, with work divided between agent instructions and continuous integration. 1
- Earlier examples documented build-review, browser-based testing, and repeatable security-testing workflows as implementations of bounded tasks with checks and reviewable outcomes. 2
Open questions
- How reliably do these workflows catch errors across different repositories and task types?
- What checks and human review are sufficient for a given task?
- Do the examples report outcomes beyond test execution and reviewable evidence?
Watch next: Watch for reported results showing whether these workflows catch defects or improve review outcomes across real tasks.
- Weekly update: LangChain described a workflow for testing documentation code samples update, Oct 12, 2026
- Weekly update: New AI agent workflows demonstrate bounded coding tasks with built‑in checks and review update, Sep 21, 2026
Rewritten automatically by the New Runtime knowledge desk from this record's weekly updates, releases and archive. The original text below is kept as written; where the two disagree, this section is newer.
Weekly knowledge desk
Weekly updates
3 reviewed updates to this project, newest first. Every update is checked against its evidence before it changes the record.
Guides turn coding-agent work into bounded tasks with verifiable outputs
A guide and companion skill describe isolating and specifying past coding-agent tasks and pairing them with a verifier. A separate account of a local supervisor emphasizes bounded work, verifiable outputs, and a human-controlled merge boundary, both illustrating the record’s workflow.
LangChain described a workflow for testing documentation code samples
LangChain described converting inline documentation code samples into runnable, testable files. A related item says the workflow divides work between agent instructions and continuous integration, a concrete example of a coding task with checks and reviewable evidence.
- How We Made Our Docs Test Themselves signal, Oct 16, 2026
- How We Made Our Docs Test Themselves post, Oct 16, 2026
New AI agent workflows demonstrate bounded coding tasks with built‑in checks and review
Recent signals show several concrete implementations of the claimed verifiable AI coding workflow. Zed’s Delta public beta (e116) lets one agent build a feature while another reviews it before teammates approve, providing a clear build‑review loop. Cerebras’ internal QA pipeline (e120) runs plain‑English test files through an LLM agent in an authenticated browser, turning ideas into executable checks. GitHub Security Lab’s Taskflow Agent (e186) packages fuzzing as a repeatable, agent‑orchestrated vulnerability‑discovery workflow, delivering reviewable evidence of test outcomes. Together these items confirm that the record’s claim is being realized in practice.
- Zed shows agent build and review flow in Delta public beta signal, Sep 24, 2026
- How an AI Agent Automates QA for the Cerebras Cloud Console signal, Sep 24, 2026
- AI-powered fuzzing with the GitHub Security Lab Taskflow Agent post, Sep 26, 2026
Minimal build
- Write a task brief with files in scope, non-goals, and acceptance checks.
- Give the agent repo-specific operating instructions.
- Let it inspect before editing.
- Require a focused diff and a real check command.
- Keep commit, push, deploy, and external writes as separate approvals.
Evidence
The current site update itself is a candidate use case: post content, docs, production drift, and build checks had to be kept separate instead of collapsed into one “make website better” prompt.
September 2026 update
Minimal build
Write scope, non-goals, expected artifacts, acceptance checks, and approval gates before inspection. Then capture intended edit path, focused diff, check output, and final handoff. Build-review flow, browser QA, and repeatable security testing each supply a concrete version of this loop.
Failure mode
A passing command is not universal proof. Record what was checked, the environment, untouched assumptions, and the human decision still required.