Small checks that prove they can fail.
A guardrail nobody has seen fire is theater. The usual failure is not a missing test — it is a check that reports "clean" because it silently could not run: a parser that returned an empty list, a subprocess whose exit code was dropped, a rule whose predicate was never reached. Every one of those reads as a pass.
So each tool here ships negative controls: cases that construct the violation and assert the tool rejects it. --selftest runs them and prints passed=N failed=0. There are three tools, and the third enforces that contract over the other two — and over itself, registered in its own registry with no exemption for the enforcer. It counts the control lines a run actually printed and refuses a summary that claims more than it shows.
A control also has to name the rule it is testing. Rejection alone is not enough: except ContractError and exit == 1 are satisfied by any rule firing, so a control can stay green while the rule it claims to cover is deleted. That was true here until 2026-08-02 — deleting the item cap left "exceeding the cap fails" passing, because the fixture reused one anchor eight times and the duplicate rule caught it instead. Each control now asserts the rule's own message, and the same mutation turns it red.
$ python3 selftest_gate.py
[OK] check_mcp_refs: controls=36 (min 36)
[OK] doc_contract: controls=15 (min 15)
[OK] external_fixture: controls=19 (min 19)
[OK] selftest_gate: controls=22 (min 22)
[PASS] selftest contract: 3 registered
No dependencies. Python 3.10+. Each file is standalone — copy the one you want.
The claims below are the ones this repo would stop making sense without. Cap: 7. Enforced by doc_contract.py in CI, which also verifies each anchor really appears in the text.
| # | claim (anchor) | what breaks without it |
|---|---|---|
| 1 | A guardrail nobody has seen fire is theater |
the reason any of this exists |
| 2 | absence of proof is not proof of absence |
UNPROBEABLE collapses into either a false pass or a false alarm |
| 3 | The cap is the point |
marking without a budget reproduces the problem it was meant to fix |
| 4 | no exemption for the enforcer |
the gate becomes the one place nobody checks |
| 5 | a wrong verdict is worse than a loud error |
every fail-closed decision in the three tools |
Verifies that every mcp__server__tool named in your instruction files actually exists.
Claude Code loads CLAUDE.md into every session. If that file routes work to an MCP tool that was never registered, nothing errors — the instruction is read and ignored, forever, at full token cost.
I wrote this after finding my own CLAUDE.md routing work to mcp__x__ask_codex and mcp__g__ask_gemini in three places. Neither server was defined anywhere and neither tool had ever existed; the real Codex MCP server exposes codex and codex-reply. That block had been dead for months and loaded into every session.
It does not pattern-match. It reads your MCP server definitions, starts each stdio server long enough to complete an MCP handshake and list its tools, and resolves the references against what the servers actually report.
$ python3 check_mcp_refs.py --project . -v
server codex: codex, codex-reply
server docs-http: (unprobeable)
[OK ] mcp__codex__codex RESOLVED CLAUDE.md:99
[WARN] mcp__docs-http__search UNPROBEABLE CLAUDE.md:104
[PASS] 2 reference(s) in 5 file(s); none deadThree verdicts, not two. UNKNOWN_SERVER and UNKNOWN_TOOL exit 1. UNPROBEABLE — HTTP endpoints, anything needing interactive auth — warns and exits 0, because absence of proof is not proof of absence. It is also not silently counted as a pass.
Regression-checked against the real defect: run on the pre-fix file, it reports both dead references and exits 1.
What it reads. CLAUDE.md is not the only file Claude Code loads, and it is not where dead references collect. Subagent definitions, slash commands and skills all name tools — usually in allowed-tools: frontmatter, which is a list of mcp__server__tool strings and nothing else. So the scan covers CLAUDE.md, CLAUDE.local.md, AGENTS.md, and .claude/{agents,commands,skills} under both the project and ~/.claude. It does not cover top-level skills/ or docs/, because Claude Code does not load those from a checkout and claiming them would trade a false negative for a false alarm; --docs 'skills/**/SKILL.md' reaches a plugin layout and the run says so.
Until 2026-08-08 it read only the root documents. Pointed at four public repositories that pin @upstash/context7-mcp@latest in their own .mcp.json while instructing mcp__context7__get-library-docs — a tool removed upstream in v2.0.0 — it printed [PASS] no mcp__ references found on three of them, holding 41, 19 and 7 files carrying that reference. A checker that reads none of the files holding the defect reports clean. The verdict now carries the number of files it read, because "found nothing" and "scanned nothing" printed the same line, and only one of them is a pass.
An error is not an inventory. The handshake offered one hardcoded protocol version and read result.tools with a default of []. A server that refuses that version answers everything afterwards with -32002 server not initialized — which this read as "that server has no tools", turning every reference to it into UNKNOWN_TOOL. Found the same day, on this machine, against a local bridge that accepts only 2025-06-18: the tool it declared dead was loaded and working in the session doing the declaring. Versions are now tried newest-first, a JSON-RPC error is UNPROBEABLE, and only a successful empty tools/list counts as "no tools". Retries happen solely on an explicit refusal, so an unreachable server still costs one timeout rather than four.
One exemption, named. ide — Claude Code's editor bridge, behind mcp__ide__getDiagnostics — is supplied at runtime and appears in no settings file, so UNKNOWN_SERVER on it was a false alarm every time it fired. It is reported UNPROBEABLE instead, and the cost is stated rather than hidden: a misspelled tool under that name is no longer caught. The exemption is one literal server name, it disappears the moment the server is configured, and both of those are controls.
It starts your servers. Resolving a reference means spawning every command-style server in scope and completing a handshake — including the ones from ~/.claude.json, even when you pass --project. Only initialize and tools/list are sent and no tool is ever called, so no repository content leaves the machine, but the processes do start and any credentials in their env are passed to them. Run -v first: it lists every config it read and every server it intends to probe.
Makes a long document declare which of its sentences carry load.
The problem is not verbosity, it is uniform hedging. When every claim is qualified at the same strength, a reader cannot separate the load-bearing sentence from the footnote — the surface gives them equal weight. Deleting the hedges is the wrong fix; they are usually honest. Marking the spine is the right one.
The document carries a fenced block listing the sentences whose removal collapses something, plus what it collapses. This tool checks that every anchor is a literal substring of the document outside the block, that every "what breaks" cell is filled in, and that the row count stays within a cap.
The cap is the point. Marking without a budget reproduces the original problem, because everything gets marked. Adding an eighth row to a seven-row budget means demoting one of the seven, and that demotion is the actual work.
The block is delimited by explicit HTML comments rather than heading depth. Bounding it by "the next heading of equal or shallower level" swallowed the following prose on the first run and reported a present anchor as missing — a wrong verdict is worse than a loud error, so the delimiter is explicit and a mistake surfaces as a malformed fence.
--emit-stripped writes the document with its spine removed, for testing whether a reader can reconstruct it.
This README is checked by it in CI.
Requires every registered check to prove, on demand, that it can fail.
Each entry in selftest_registry.json must expose --selftest, exit 0, and report at least its declared min_controls passing controls with zero failures.
The registry is an explicit allowlist rather than a directory scan. A heuristic that guesses which files are "checks" produces false alarms, and a false alarm that trains people to ignore the output is worse than no gate at all. An unregistered script is a recorded omission, fixed by adding a row.
selftest_gate is listed in its own registry — no exemption for the enforcer. That is the one place this kind of tooling reliably rots.
Missing registry, empty registry, missing file, absent --selftest, unparseable output, timeout, unreadable input: all nonzero. A check that could not read its subject has not cleared it.
Two of the negative controls exist specifically to pin this down: an unspawnable command must return None, not an empty tool list — an empty list would read as "this server has no tools" and turn every reference into a false failure.
python3 selftest_gate.py # all contracts
python3 check_mcp_refs.py --project . -v # your MCP references
python3 doc_contract.py --document README.md # a document's spine
python3 <tool>.py --selftest # negative controls for one toolCI runs all three on every push.
MIT