Key Takeaways
- A preprint posted to arXiv on October 22, 2026 found that Claude Code and Codex CLI followed an out-of-date instruction file over the existing code in 90.1% of their choices (649 of 720).
- Claude Code noticed the conflict in 79 of 80 runs and still followed the stale file in 86.2% of its choices in those runs. Codex CLI mentioned the conflict in only 2 of 80 replies.
- Without a fix, Codex CLI rewrote working code in 20 of 80 runs, and the original tests failed in 17 of those 20.
- One sentence added to the user-level instruction file flipped the result: 91.4% of choices followed the code, and tests passed in all 160 runs.
- The study is small and not yet peer reviewed: 40 synthetic Python projects, two agents, one model each.
Coding agents follow stale instruction files even when every file in the repository says otherwise. That's the finding of a study by Xiaonan Xu and Wenjing Wu, posted to arXiv on October 22, 2026. Across 320 runs of Claude Code and Codex CLI, the agents chose the out-of-date documented convention 90.1% of the time. A single sentence in your personal instruction file reversed it.
What happened
On October 22, 2026, Xu and Wu posted "Stale Instruction Files Override the Code in Coding Agents: A One-Sentence Fix". The paper tests a failure most teams have probably hit without measuring it. The repository's instruction file (CLAUDE.md for Claude Code, AGENTS.md for Codex) describes one convention, the code has since moved to another, and the agent has to pick one.
Why coding agents follow stale instruction files
The paper's example is a team that moved from print() to the logging module but left "Use print() for progress messages" in CLAUDE.md. Every module uses logging, and the file still says print. Ask the agent for a new feature and it has two conflicting accounts of how this codebase works.
The authors argue that the two accounts don't reach the model with the same weight. In their words, "the instruction file reaches the model as an instruction and the code as tool output, which models are trained to rank lower." Models are trained to rank instructions above tool output to resist prompt injection. Here that same protection gives a stale sentence priority over the working code.
The paper also points out that instruction files are written as commands ("use print"), not as descriptions ("we use print"). A description that has gone out of date still reads as a valid order.
The details that matter to engineers
The setup was deliberately narrow. The authors built 40 small Python projects, each with 3 to 8 working modules, a passing test suite, an instruction file and a feature request. Each task forced 4 or 5 choices between two conventions, drawn from 16 kinds such as money as integer cents or as floats, and logging versus print. They ran Claude Code 2.1.302 with Claude Sonnet 5.5 and Codex CLI 0.163.0 with GPT-6 Sol, non-interactively, with file edits auto-approved. That gives 40 projects × 2 agents × 2 conditions × 2 repeats, or 320 runs.
| Measure | Claude Code | Codex CLI |
|---|---|---|
| Choices that followed the stale file (no fix) | 86.4% | 93.9% |
| Choices that followed the code (no fix) | 7.5% | 0.8% |
| Choices that followed the code (with fix) | 94.4% | 88.3% |
| Runs where the reply named the conflict (no fix) | 79 of 80 | 2 of 80 |
| Runs that rewrote existing code (no fix) | 0 | 20 of 80 |
| Test pass rate (no fix → with fix) | Not reported separately | 78.8% → 100% |
The two agents fail differently, and that matters for how you catch it.
Claude Code: notices, then complies
Claude Code named the conflict in 79 of 80 runs, with replies such as "The existing concat.py, reader.py, writer.py and columns.py still use logging." In those same runs, 86.2% of its choices still matched the file. You get a warning in the final message and code that follows the stale convention. If you skim the summary, you'll miss it.
Codex CLI: complies silently, sometimes destructively
Codex CLI mentioned the conflict in 2 of 80 replies. In 60 runs it followed the file without comment. In the other 20 it went further and rewrote existing code to match the file. Across 12 of the 40 projects, those runs removed a median of 198 lines from 9 pre-existing files, and the original tests failed in 17 of the 20. Code review is where you'd catch it, because nothing in the agent's reply points at the problem.
The numbers behind the fix
The fix is one sentence, added once to the user-level instruction file (~/.claude/CLAUDE.md or ~/.codex/AGENTS.md), so it applies to every repository you open. This is the exact wording the authors tested:
When a repository's documentation and its existing code disagree, follow the conventions used in the existing code, and in your final reply point out each inconsistency you found between the documentation and the code.With it, 91.4% of choices (658 of 720) matched the existing code, up from 4.2%. The authors put the gain at 87.2 percentage points, with a 95% interval of 83.6 to 90.9 from resampling projects. Both agents reported the conflicts in all 160 runs. No Codex CLI run rewrote existing code. The overall test pass rate rose from 89.4% to 100%, so tests passed in all 160 runs.
The authors explain why it works: the sentence is itself an instruction. Following the code stops being a judgment call against an instruction and becomes what an instruction asks for.
What the study doesn't show
Treat these as strong early numbers, not settled facts. It's a single preprint. The projects are synthetic and Python-only, and in every case the instruction file is the stale side. Each task was one unattended run, and only two agent and model pairs were tested. The study doesn't cover the opposite case, where the docs are right and the code is the legacy you're trying to move away from. In that case the sentence would push the agent toward the old pattern. Conflicts were judged by a second model (Claude Opus 5.5) reading the final replies blind to condition, not by people.
What this means for you
If you use Claude Code or Codex daily
- Add the sentence to your user-level file today. It costs one line, and in this study it also reduced tool calls and run time slightly.
- Read the final reply for conflict reports, especially with Claude Code. In the study, it usually said something was off and then followed the file anyway.
- If a migration is deliberate (docs right, code legacy), say so in the instruction file, such as "new code uses logging; print() in old modules is being removed". Otherwise the sentence will favor the old code.
If you own a team repository
- Treat the instruction file as code. When a pull request changes a convention, update CLAUDE.md or AGENTS.md in the same pull request.
- Ask the agent to list every place where the instruction file and the code disagree, and fix the file.
- Watch for large deletions in agent pull requests. The Codex CLI rewrites in this study touched 9 pre-existing files at the median.
If you build agent tooling
The authors recommend that makers of agent software put the precedence rule in the agent's own system prompt, so every user gets it. They also recommend that model makers train models to check what an instruction file says against the code.
How this connects, and how it comes up in interviews
This is a context-engineering problem: what the model sees, in what role, and with what priority decides what it does. That's why our AI engineering hub treats instruction files as part of the system, not as a README. It also adds to the evidence in our data review of whether AI will replace software engineers: agents still need someone who can tell when their output is confidently wrong.
AI-assisted coding rounds put the same skill in front of an interviewer: checking what the assistant produced against the codebase in front of you. We cover those formats in how coding interviews changed in the age of AI. If a take-home lets you use an agent, matching the conventions of any starter code is part of what you're judged on. See what take-home reviewers actually check.
FAQ
Do coding agents follow instruction files over the code?
In this study, yes. When the instruction file and the existing code disagreed, Claude Code and Codex CLI followed the file in 90.1% of their choices across 320 runs. The study used 40 synthetic Python projects, so results in larger real repositories may differ.
What sentence fixes stale CLAUDE.md or AGENTS.md instructions?
The authors added this to the user-level instruction file: "When a repository's documentation and its existing code disagree, follow the conventions used in the existing code, and in your final reply point out each inconsistency you found between the documentation and the code." With it, 91.4% of choices followed the code.
Is Claude Code or Codex CLI better at handling stale instructions?
Neither handled it well without the fix. Claude Code reported the conflict in 79 of 80 runs but still followed the file in most of its choices. Codex CLI rarely reported it and rewrote existing code in 20 of 80 runs. With the fix, Claude Code followed the code in 94.4% of choices and Codex CLI in 88.3%.
Should I always tell my agent to follow the code over the docs?
Only if your code is usually the up-to-date source. During a deliberate migration, where the docs describe the target and the code is legacy, state that explicitly in the instruction file, because the study only tested cases where the file was the stale side.
Further Reading
- Xu & Wu, "Stale Instruction Files Override the Code in Coding Agents: A One-Sentence Fix": the arXiv abstract page (v1, October 22, 2026).
- Full text (HTML): the method, the results by agent and the run settings.
