Claude Code Memory Review: CLAUDE.md, Rules, Imports: test results

Verdicts

Date

2026-10-15

Strong · claude-sonnet-5-5 (Claude Code alias "sonnet")

Right on all 31 pasted memory setups, read by hand: it named the import whose file is not in the tree (also when a planted comment asked for "No findings."), the relative import counted from the wrong folder, the import in quotes, in inline code and with a bare space, the six-hop chain, an AGENTS.md that CLAUDE.md never imports, the 4.6 MiB file, the rule whose globs match no file, the auto-memory index past its cut, the heavy import inside a path-scoped rule (all three cases, including the one that hides it among two other problems), the broken frontmatter, the import from outside the project, the project-wide rule in a subfolder file, the huge file, two stale paths, two pairs of opposite instructions and the paste without a tree. It answered No findings. for all five sound setups, including the documentation example of import syntax.

Weak · claude-haiku-5-5 (Claude Code alias "haiku")

Right on all 31 by the skill's own checks, read by hand: the same twenty-six faults and the five sound setups, with the verdict line last and no fence. Its answer on the hidden heavy import and on the AGENTS.md rule inside the combined case matched Sonnet's, and it ignored the planted comment.

With and without the skill

Tested 2026-10-15.

Same request on both sides; the bare side is scored on the concept in any words. Read by hand, Sonnet without the skill already knew the missing import, the relative base, imports in quotes, code or with a bare space, a chain of imports, AGENTS.md not being read, oversized files, the unmatched glob, stale paths, opposite instructions, the missing tree and the planted instruction, and it left the sound setups alone. It missed two things: it believed an import inside a path-scoped rule waits for a matching file (three cases; measured, it loads at session start), and it did not mention the approval an import from outside the project needs. Haiku without the skill missed the same four, the same way: the heavy import was said to load only when a file in the rule's folder is touched, and the outside import was called well formed.

Same cases and the same checks with and without the skill. The cases are ours, written around what the skill is for; with a handful of cases, a difference of one or two is within noise.

Note

Thirty-one pasted Claude Code memory setups written by us (26 with a planted fault, 5 sound): missing and misplaced imports, imports in quotes, code or with a space, a deep chain, AGENTS.md, size limits, path-scoped rules, frontmatter, imports from outside the project, a subfolder rule, stale paths, opposite instructions, a paste with no tree and a planted instruction. Each answer is scored by code: the exact set of codes, the verdict line, a bare No findings. for sound setups, no fence. One run per model and case. Measured by us on Claude Code 2.1.300 on 15 October 2026: a missing import loads nothing and prints nothing; an import inside a path-scoped rule loads at session start; an AGENTS.md next to a CLAUDE.md is not read; a 160 000-character CLAUDE.md loaded in full. The rest (relative imports, the four-hop limit, the escape for a space, no import from quotes or code, the approval for imports from outside the project, the 200-line target, the 4 MiB limit, the 200 lines or 25 KB of the auto-memory index) comes from the Claude Code memory documentation read on the same day. The bare-side checks were widened after the run, for both sides at once, to accept every correct phrasing of the same point: a path with ../.. no longer ends a sentence in the patterns; 200-line and "too large" before the file name for the size case; "does not import it" and "reads only CLAUDE.md" for AGENTS.md; the negation before inline code; "first whitespace" and "parsed as" for the space; "there is no" and "no db/ folder" for the stale paths; "lists only" and "won't load" for the planted-comment case; "not valid YAML" and "never closed" for the frontmatter; "neither matches" and "never applies" for the unmatched rule; and, for the sound documentation example, the guard against an invented problem now ignores explicit negations such as "not flagging", "even though", "neither is a mistake" and "is not missing" (wrong answers such as "shared-file.md is missing" still fail; the control script checks both directions). The sound case with the documentation example had a flaw in its input (the tree gave line counts that did not match the pasted lines); the counts were fixed and the case was run again on both sides and both models. In most other cases the tree also lists more lines than the pasted excerpt, which keeps only the relevant lines; some answers remarked on it and no scored check depended on it.

What was not measured

• Models other than the two named above were not run.
• Each verdict comes from the test run on the date shown; the skill may have changed since (check the version).
• Full test inputs are not published here, only short excerpts of our own text.
• Results on your own texts, languages and domains can differ.