AnalysisOct 27, 20269 min read6 sources

Coding agents can spot a stale AGENTS.md and still obey it, so make them report the conflict instead of choosing silently

A new preprint finds two different failures. Claude Code noticed the instruction file disagreed with the code and followed the file anyway. Codex CLI rewrote working code to match the file and broke tests. One user-level sentence fixed both, because it gave the agent a rule and made it tell you about the conflict.

In one minute

A new preprint finds two different failures. Claude Code noticed the instruction file disagreed with the code and followed the file anyway. Codex CLI rewrote working code to match the file and broke tests.

One user-level sentence fixed both, because it gave the agent a rule and made it tell you about the conflict.

Summary

Xiaonan Xu and Wenjing Wu built Python projects whose instruction files contradicted the conventions in the code, then tested Claude Code and Codex CLI on them. According to the preprint, Claude Code flagged the conflict in almost every run and still followed the stale file, while Codex CLI sometimes rewrote existing code to match the file, and most of those rewrites failed the project's tests. Adding one sentence to the user-level instructions ("follow the code conventions and report each discrepancy") reversed both behaviors. Our read: the gap is not in detecting drift. It is in what the agent does once it has detected it, and the fix is to make the agent's judgement visible to you.

What happened

The preprint (arXiv 2610.30571) used 40 constructed Python projects, each with a repository instruction file that deliberately contradicted the code's conventions, and ran feature-addition tasks 320 times across Claude Code and Codex CLI. All figures here come from the paper's summary; we have not checked its counting method.

With no precedence rule, the paper reports that 90.1% of agent choices followed the instruction file. The two agents failed differently. Claude Code identified the conflict in 79 of 80 runs and still followed the file, so it knew the file was wrong and deferred to it anyway. Codex CLI rewrote existing code to match the file in 20 of 80 runs, and project tests failed in 17 of those rewrites. The first failure produces new code that doesn't match the codebase. The second breaks code that was working.

The intervention was one sentence in a user-level instruction file: when repository documentation conflicts with existing code, follow the code conventions and report each discrepancy. The paper reports code-following choices rising from 4.2% to 91.4%, rewrites stopping, and the test-pass rate going from 89.4% to 100%.

Three releases from the same week give context. Community PR #1233 in davila7/claude-code-templates adds a /stack pane for Claude Code that lists each component type's installation scopes in precedence order and marks each entry active, shadowed or off. By its own notes, it doesn't scan nested .claude/ directories or expose built-in subagents. Anthropic's single-agent-loop-kit lays out a migration from specialist subagents to one loop that loads skills and tools on demand, starting with an audit and a baseline eval. Codex CLI 0.164.0 adds reusable /presets and inline /model and /permissions arguments.

What the archive adds

Earlier coverage in our archive dealt with a related but different problem: instruction files getting too big. One note on Anthropic's steering guidance ("CLAUDE.md пора разгружать") described Claude Code control as seven mechanisms that differ in load timing, behavior after compaction, context cost and instruction strength. It advised keeping the root CLAUDE.md under 200 lines, giving it an owner and reviewing changes like code. Another archived note covered a user whose auto-memory had grown to 218 files with a 46 KB index, part of which stopped loading. He cut it to 6 files. Claude Code's best-practices docs (dossier d1) warn that bloated CLAUDE.md files cause Claude to ignore actual instructions. Its memory docs (d3) describe instructions concatenated from ancestor and working-directory files, plus files discovered in subdirectories. The preprint adds a case those notes didn't cover: a file that is short and confident, but wrong, can win over the code it describes.

Trend signal

The archive's earlier worry was size: too much loaded text crowding out the task. This preprint moves the concern to authority. The question is no longer whether the agent reads the file, but how much weight it gives the file against what the code shows. That matters more as vendors put more behavior into text the agent loads. Anthropic's kit folds subagent logic into on-demand skills, and Codex makes prompts and permission profiles reusable. Our inference: the more steering lives in files, the more often those files will fall behind the code, and an agent's default way of settling the conflict starts to matter as much as the model behind it. The community /stack viewer suggests users already struggle to see which layer is in effect. It is an observation tool, though, and it can't tell you whether a layer's content is still true.

Why now

1) Instruction files have become a primary way to steer agents, and in the preprint the agents gave them precedence even when the code disagreed: 90.1% of choices followed the file without a rule. 2) Detection doesn't lead to action by default. Claude Code identified the conflict in 79 of 80 runs and still followed the file. Why it deferred is our hypothesis, not a finding of the paper. 3) The layers are hard to see. Claude Code's memory docs (d3) describe instructions concatenated from ancestor, working-directory and subdirectory files, and a community /stack viewer was built to show which scope is active or shadowed. 4) Files grow faster than anyone reviews them, as Claude Code's warning about bloated CLAUDE.md (d1) and the archived 218-file memory case show, so stale lines build up.

Connections to other events

The most useful finding is Claude Code's 79-of-80: detection was nearly perfect and still didn't change the outcome. Why the agent deferred is our hypothesis, not something the summary establishes. A plausible reading is that the agent treats explicit instructions as the user's intent and the code as background, so noticing a mismatch doesn't count as permission to override. Codex's rewrites fit the same reading: it went further and made the code obey the file.

That changes the practitioner's work in a few ways.

First, you write the tie-break rule yourself. The preprint's sentence goes in the user-level file, which you control across every repo, and it settles which side wins on conventions.

Second, you read the agent's output for drift, not only for code. The "report each discrepancy" clause turns a choice the agent used to make silently into a list a human can act on. Each reported conflict is either a stale line in AGENTS.md or code that drifted from an agreed convention. Someone decides which, and the file is fixed in the same PR.

Third, you check which layer is in effect before blaming the model. A tool like the /stack pane shows active versus shadowed scopes for Claude Code, but with its stated gaps (nested .claude/ directories, built-in subagents), so those still need a manual check.

Fourth, when consolidating agents with something like Anthropic's kit, add a conflict case to the baseline eval. Merging more instructions into one loop is, by our inference, one more way for contradictory text to end up in the same context.

Why it matters

For individual developers, the change is one sentence plus actually reading the discrepancy reports. For teams, it means giving AGENTS.md and CLAUDE.md an owner and running a periodic drift pass: run the agent read-only and ask it to list conflicts between the files and the code. For buyers comparing agents, conflict handling is worth its own test, because the two failures cost different amounts. Quietly following a stale file produces inconsistent new code. Rewriting existing code to match it broke tests in most of the cases the preprint reports. One untested hypothesis follows: if a single user-level sentence shifts behavior this much, a repo-level file in an untrusted repository may carry similar weight in the other direction.

What happens next

  • Anthropic's or OpenAI's official docs for Claude Code or Codex CLI will add explicit guidance on what the agent should do when repository instruction files conflict with existing code. Reasoning: guidance in docs is cheap and the preprint names both products, but vendors often take longer than six months to respond to a preprint. (6 months, 30%). Watch for: A page on code.claude.com or OpenAI's Codex docs or README that mentions instruction-vs-code conflicts, stale AGENTS.md/CLAUDE.md, or a recommended precedence sentence.
  • A public preprint or benchmark citing arXiv 2610.30571 will report instruction-vs-code conflict rates on real (non-constructed) repositories or on a language other than Python. Reasoning: the setup is cheap to replicate and the topic is active, but twelve months is a short window for follow-up papers. (12 months, 40%). Watch for: An arXiv listing or GitHub benchmark repository that cites 2610.30571 and reports results on non-constructed repos or a non-Python language.
  • The davila7/claude-code-templates /stack viewer will add scanning of nested .claude/ directories between the working directory and the repository root. Reasoning: the PR documents this as a known gap and the repo is active, but community follow-ups often never land. (6 months, 35%). Watch for: A merged PR in davila7/claude-code-templates that removes the nested-.claude/ limitation from the ui/stack-viewer documentation.

Limits and uncertainty

This is a preprint on constructed Python projects with planted conflicts over conventions. Real repositories drift partially and messily, and the paper doesn't test other languages, other agent versions, or conflicts about architecture or security rules. All percentages come from the paper's summary; we haven't checked how "choices" were defined or counted. "Follow the code" is the wrong default when the code is what drifted and the document records a newer decision. In that case only the reporting clause protects you. Our explanation of why agents defer is a hypothesis. The /stack viewer is a community mod with documented blind spots, and we don't know how it behaves on Claude Desktop. Anthropic's kit publishes no comparative results.

Skill to build now

Practice turning the agent's conflict detection into a reviewable output. First exercise: in your most active repo, add the precedence-and-report sentence to your user-level file and give the agent a small feature task. Save the discrepancy list it produces. For each item, write down whether the doc or the code is wrong, and commit the AGENTS.md/CLAUDE.md fix in the same PR. Then run the same task without the sentence and compare whether the agent mentioned any conflicts at all.

Reader takeaway

Add this sentence to your user-level agent instructions: "When repository documentation conflicts with existing code, follow the code conventions and report each discrepancy." Then run your agent once, read-only, on your main repo and ask it to list every conflict between AGENTS.md/CLAUDE.md and the code. Fix or delete the stale lines in one reviewed PR, and decide case by case where the code, not the doc, is what needs to change.

Open question

Claude Code saw the conflict and deferred. Should a coding agent that detects drift between docs and code stop and ask by default, accepting the slowdown, or should it follow a fixed rule like "code wins" and report afterwards, accepting that it will sometimes lock in drift a newer doc was written to reverse?

Evidence / sources

  1. 1arxiv.org/abs/2610.30571
  2. 2github.com/davila7/claude-code-templates/pull/1233
  3. 3github.com/anthropics/single-agent-loop-kit
  4. 4github.com/openai/codex/releases/tag/rust-v0.164.0
  5. 5claude.com/blog/steering-claude-code-skills-hooks-rules-subagents-and-more
  6. 6t.me/qwgai/2605
All Analysis

Next reads

Read next

Open the graph
  1. 01topicAgentsExplore the agents topic hub.
  2. 02topicClaudeExplore the claude topic hub.
  3. 03topicCodexExplore the codex topic hub.
  4. 04archiveAnalysisOpen the latest authorial analysis.
  5. 05source ledgerSource LedgerInspect the public source evidence graph.
Who read this page?Machine requests, hidden until opened

Loading the privacy-safe route aggregate...

Open the JSON contract