Agent Context Audit

agent-context-audit — unhobble this repo's agent context

Goal: find where this repo's context (CLAUDE.md, docs, skills, tool designs)

hobbles a Claude 5-generation model — overconstrains it, contradicts itself,

repeats itself, or hides context the model actually needs — and leave behind a

findings report plus approved fixes.

Background: Anthropic removed over 80% of Claude Code's system prompt for

Claude 5 models with no measurable loss on coding evals. Older context was

written for models that needed rules; newer models need judgment, good

interfaces, and the facts they can't infer. This skill audits against that

shift, plus the "finding your unknowns" framework (the gap between the *map* —

your prompts/docs — and the *territory* — the actual codebase).

You are auditing first, fixing second. Do not edit anything until Step 4.

The six shifts (the audit rubric)

Every finding maps to one of these. Cite the shift number in the report.

• Rules → Judgment. Hard rules ("NEVER…", "ALWAYS…", "do not add

comments", "one-line docstrings max") that encode a *preference*, not a real

constraint, should become judgment framing ("write code that reads like the

surrounding code") — or be deleted if the model would infer it anyway.

Keep hard rules only where violation is genuinely costly (security, prod

data, irreversible actions, legal/billing).

• Examples → Interface design. Long tool-usage examples and few-shot

transcripts constrain exploration. Prefer expressive interfaces: good

parameter names, enums that hint at valid states, tight descriptions.

In tool/MCP definitions, an enum of pending | in_progress | completed

teaches more than three worked examples.

• Upfront context → Progressive disclosure. Anything long that's only

sometimes needed (review checklists, deploy runbooks, style deep-dives)

should move out of CLAUDE.md into a skill or linked file loaded on demand.

CLAUDE.md is loaded *every* session — it should carry only what every

session needs.

• Repetition → Concise, single-home instructions. The same instruction

appearing in CLAUDE.md *and* a skill *and* a tool description is a bug:

copies drift and eventually conflict. Each instruction gets exactly one

home — tool-usage guidance lives in the tool description, repo gotchas in

CLAUDE.md, team opinions in skills.

• Manual memory → Automatic memory. Sections telling the agent to

hand-maintain notes/changelogs in CLAUDE.md, or accumulated session-specific

trivia, are obsolete where auto-memory exists. Flag CLAUDE.md content that

is really *memory* (per-user, per-incident, time-bound) rather than

*repo truth*.

• Simple specs → Rich references. Where docs describe behavior in loose

prose, prefer pointing at the real thing: @-referenced source files, a

test suite, an HTML mockup, a rubric a verifier can score against. Code-based

specs beat prose paraphrases of code.

Cross-cutting failure modes to hunt alongside the shifts:

• Conflicts — instructions that clash across layers (e.g. "document

thoroughly" in one file, "DO NOT add comments" in another). Highest-value

findings; a conflict forces the model to deliberate or guess on every task.

• Staleness (map ≠ territory) — docs naming files, commands, flags, or

services that no longer exist, or missing ones that now do. Verify every

concrete claim you audit against the actual repo.

• Missing unknown-knowns — things obvious to the team but written nowhere:

the non-obvious build step, the directory you must never touch, the reason a

weird pattern exists. These are what CLAUDE.md is *for* ("repository

gotchas rather than obvious patterns").

Step 0 — Inventory the context surface

Collect everything that gets assembled into an agent's context here. Look for

the capability, not a specific filename:

• CLAUDE.md files — root, nested per-directory, ~/.claude/CLAUDE.md only

if the user asks for a global audit. Also AGENTS.md, .cursorrules,

.github/copilot-instructions.md if present (same disease, same cure).

• Skills — .claude/skills//SKILL.md, skills//SKILL.md, plugin

skills committed to the repo.

• Tool designs — MCP server definitions the repo owns (tool names,

descriptions, parameter schemas), custom slash commands, hooks, and any

agent definitions (.claude/agents/*.md).

• Codebase docs agents are pointed at — README, CONTRIBUTING, docs/

referenced from CLAUDE.md or skills.

Record rough sizes (lines/tokens) per artifact — total always-loaded weight is

itself a finding when large.

Step 1 — Audit each artifact against the rubric

For each artifact, walk the six shifts and cross-cutting modes. For every

finding record: file:line, quote, shift #, severity, proposed rewrite

(the actual replacement text — or "delete", with one line of why it's safe).

Severity:

• high — conflicts between layers; rules that block correct behavior;

stale facts an agent would act on.

• medium — overconstraint, redundancy, always-loaded bulk that belongs in

a skill.

• low — style, phrasing, minor bloat.

Verify before you flag: a claim of staleness must be checked against the repo

(does that script exist? does that command run?); a claim of redundancy must

cite both locations.

Step 2 — Probe for unknowns (the gaps docs don't show)

Auditing text only finds what's written. Now find what's missing:

• Blind-spot pass: skim the actual territory — build config, CI, scripts,

the weirdest-looking directories — and list load-bearing facts that appear in

no doc. Each is a candidate "unknown known" to add.

• Knowledge quiz: write 5–10 questions a fresh agent must answer to work

here safely ("how do I run one test?", "what must never be committed?",

"which service is the source of truth for X?"). Answer each *using only the

audited docs*. Unanswerable questions = gaps; wrong answers = stale docs.

• Git check: git log --oneline -20 -- <doc> — a CLAUDE.md untouched for

months in an active repo is presumptively stale; recent churny areas of the

codebase with no doc coverage are presumptive gaps.

Step 3 — Report

Deliver a findings report (markdown in the repo, e.g.

docs/agent-context-audit-YYYY-MM-DD.md, or just in the reply if the user prefers):

• Scorecard — per artifact: size, finding counts by severity, one-line

verdict (keep / trim / restructure / delete).

• Findings table — file:line, quote, shift #, severity, proposed rewrite.
• Gaps — missing unknown-knowns from Step 2, each with proposed text and

the home it belongs in (CLAUDE.md vs skill vs tool description).

• Projected result — estimated always-loaded context before → after.

Lead with the top 3–5 highest-value changes; don't bury a layer conflict under

twenty style nits.

Step 4 — Apply (with approval)

Ask which findings to apply (all high, everything, or cherry-pick). Then:

• Make the edits exactly as proposed in the report.
• When moving content out of CLAUDE.md into a skill, create the skill and leave

a one-line pointer behind.

• Keep each change reviewable — don't reflow or rewrite text you didn't flag.
• If /doctor is available in this Claude Code install, suggest the user also

run it as a second opinion on CLAUDE.md/skill sizing.

Anti-patterns for the auditor

• Deleting a hard rule that guards something genuinely irreversible — the

point is removing *fake* constraints, not real ones.

• Flagging brevity as a problem — short, dense CLAUDE.md files are the goal.
• Rewriting voice/style wholesale — preserve the team's phrasing where content

is sound.

• Reporting a finding without a concrete rewrite — every finding must be

actionable as written.

Sources: Anthropic, "The new rules of context engineering for Claude 5

generation models" (claude.com/blog); "A field guide to Claude Fable: finding

your unknowns" (claude.com/blog).