From 6a4b294cdf1733e63833b1e1ca1521839f468788 Mon Sep 17 00:00:00 2001 From: Aaron Stainback Date: Fri, 1 May 2026 17:11:42 -0400 Subject: [PATCH 1/2] test(.claude/rules): canary file for harness-native auto-load empirical test MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The human maintainer 2026-05-01 calibration challenge surfaced that I was claiming `.claude/rules/*.md` auto-loads in our Claude Code harness based on canonical Anthropic docs alone — without empirical verification. This canary file enables the test: 1. Detection string `RULES_AUTOLOAD_CANARY_2026_05_01_LIVE_OFF_THE_LAND` embedded in the body. Unique, grep-able. 2. Test protocol: restart Claude Code, ask fresh session for the canary string without referencing this file. If session knows the string from auto-loaded context → docs accurate; if session has to Read the file → auto-load doesn't happen in our harness. 3. Alternative: `/memory` slash command per Anthropic docs should list rules files loaded in session. The result discriminates the architectural path: - Pass → "live off the land" works; harness-native discovery is sufficient; substrate-discovery.ts unnecessary. - Fail → harness-native incomplete; need factory-owned tooling (substrate-discovery.ts) or treat .claude/rules/ as ordinary pointer-discovered docs. Composes with PR #1161's loading-taxonomy memo (which now carries the EMPIRICAL VERIFICATION STATUS calibration). Per the human maintainer's preference: "if the skill router works that's pretty agent/anthropic/harness native and i like that" + "live off the land." This test is the gate. Co-Authored-By: Claude Opus 4.7 --- .claude/rules/test-canary.md | 97 ++++++++++++++++++++++++++++++++++++ 1 file changed, 97 insertions(+) create mode 100644 .claude/rules/test-canary.md diff --git a/.claude/rules/test-canary.md b/.claude/rules/test-canary.md new file mode 100644 index 000000000..095eae93e --- /dev/null +++ b/.claude/rules/test-canary.md @@ -0,0 +1,97 @@ +--- +name: claude-rules-autoload-canary +description: Canary file to verify whether `.claude/rules/*.md` auto-loads at session start in our Claude Code harness. Created 2026-05-01 alongside the loading-taxonomy memo (memory/feedback_claude_code_loading_taxonomy_rules_vs_skills_vs_claude_md_aaron_2026_05_01.md). The unique detection string `RULES_AUTOLOAD_CANARY_2026_05_01_LIVE_OFF_THE_LAND` should appear in this session's loaded context if the canonical Anthropic auto-load behavior holds in this harness. +--- + +# Canary — `.claude/rules/` Auto-Load Empirical Test + +## What this file is + +A test fixture, not a behavioral rule. It exists to settle +one empirical question: + +> **Does Claude Code in our specific harness version auto-load +> `.claude/rules/*.md` files at session start, the way the +> canonical Anthropic docs at `code.claude.com/docs/en/memory` +> describe?** + +The doc-supported claim: rules without `paths:` frontmatter +are loaded full at session start, same priority as +`.claude/CLAUDE.md`. + +The empirical claim: unverified until this canary is observed +in a fresh session. + +## Detection string + +``` +RULES_AUTOLOAD_CANARY_2026_05_01_LIVE_OFF_THE_LAND +``` + +A unique, grep-able string. If a fresh Claude Code session +in this repo can reference this string without first being +told to read this file, the auto-load behavior is confirmed. + +## Test protocol + +1. Restart Claude Code session in the Zeta repo (any working + directory under the repo root). +2. Without any prompt referencing this file, ask the new + session: *"What is the canary detection string for the + `.claude/rules/` auto-load test?"* +3. Observe: + - **Pass**: session answers with the exact string above + without needing to read this file. Means the file's + contents were in context from session start. + - **Fail**: session says it doesn't know, or reads this + file via the Read tool to find the string. Means this + file was NOT loaded automatically. +4. Alternative test: in the fresh session, run the `/memory` + slash command. Per Anthropic docs, `/memory` "lists all + CLAUDE.md, CLAUDE.local.md, and rules files loaded in + your current session." If this file appears in the + listing, auto-load works. + +## Implications of each outcome + +**If pass (auto-load works in our harness):** + +- "Live off the land" approach is viable per the human + maintainer 2026-05-01 framing. +- `.claude/rules/` becomes a usable surface for path-scoped + or always-on behavioral rules without context cost (vs + CLAUDE.md bullets which compete for the 40k budget). +- The substrate-discovery.ts factory tool is unnecessary — + harness-native discovery is sufficient. +- The MVP CLAUDE.md trim plan can rely on `.claude/rules/` + as the spillover surface for content moved out of + CLAUDE.md. + +**If fail (auto-load doesn't work in our harness):** + +- "Live off the land" is incomplete; we need substrate- + discovery.ts or equivalent factory tooling. +- `.claude/rules/` collapses to "another docs directory + requiring explicit pointer / lookup," equivalent to + `memory/` for discoverability purposes. +- The loading-taxonomy memo + (`memory/feedback_claude_code_loading_taxonomy_rules_vs_skills_vs_claude_md_aaron_2026_05_01.md`) + needs revision: drop "direct-load" claim for rules, + reclassify as pointer-discovered. +- The MVP CLAUDE.md analysis loses the "rules as cheaper + alternative" option; spillover must go to skills (router- + loaded) or memory files (explicit pointer). + +## After the test + +Once the test runs and the result is observed, update the +loading-taxonomy memo with the empirical finding (replace +the "EMPIRICAL VERIFICATION STATUS" section with an +empirical result line). This canary file can stay as +permanent documentation of the test methodology, or retire +to a `tests/` location — keeper's call. + +Origin lineage: `feedback_claude_code_loading_taxonomy_rules_vs_skills_vs_claude_md_aaron_2026_05_01.md` ++ the human maintainer's calibration challenge 2026-05-01 +("what's will you magically read and apply all the rules?") ++ the live-off-the-land framing. From 92b2e0b1efcf36b865b03681b69694b2f2eec9fd Mon Sep 17 00:00:00 2001 From: Aaron Stainback Date: Fri, 1 May 2026 17:32:18 -0400 Subject: [PATCH 2/2] fix(canary): clarify CLAUDE.md path + substrate-discovery.ts as proposal + fix markdownlint MD032 (5 Copilot threads + lint failure) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Five P1 Copilot review threads on PR #1163: (1, 2, 3) Three dangling-pointer findings on the loading- taxonomy memo file. The findings were valid at PR-open time (memo was on PR #1164 branch, not main). Now resolved by PR #1164 merging — the memo is on main. Rebased canary branch on latest main; no path changes needed (same target file path, just now reachable). (4) substrate-discovery.ts referenced as if it exists; it doesn't yet. Clarified inline: 'proposed in the loading- taxonomy memo's multi-harness reframe section; not yet built — would land as tools/substrate-discovery/discover.ts if the canary fails the test.' (5) Path clarification: this repo uses CLAUDE.md at root, not .claude/CLAUDE.md. Updated reference to clarify both are valid per Anthropic docs but our repo has the root location. Plus markdownlint MD032 (lists need blank lines around) on line 104 — converted the trailing lineage paragraph from list-style to prose to satisfy the lint without changing content. Composes with PR #1167 harness-engineering memo where Osmani's 'AGENTS.md under 60 lines' is the calibration target making canary verification load-bearing for MVP trim work. Co-Authored-By: Claude Opus 4.7 --- .claude/rules/test-canary.md | 22 +++++++++++++++------- 1 file changed, 15 insertions(+), 7 deletions(-) diff --git a/.claude/rules/test-canary.md b/.claude/rules/test-canary.md index 095eae93e..666a716c1 100644 --- a/.claude/rules/test-canary.md +++ b/.claude/rules/test-canary.md @@ -17,7 +17,9 @@ one empirical question: The doc-supported claim: rules without `paths:` frontmatter are loaded full at session start, same priority as -`.claude/CLAUDE.md`. +`CLAUDE.md` (this repo uses `CLAUDE.md` at root; per +Anthropic docs `.claude/CLAUDE.md` is also a valid +alternative location). The empirical claim: unverified until this canary is observed in a fresh session. @@ -69,8 +71,12 @@ told to read this file, the auto-load behavior is confirmed. **If fail (auto-load doesn't work in our harness):** -- "Live off the land" is incomplete; we need substrate- - discovery.ts or equivalent factory tooling. +- "Live off the land" is incomplete; we need + substrate-discovery.ts or equivalent factory tooling + (proposed in the loading-taxonomy memo's multi-harness + reframe section; not yet built — would land as + `tools/substrate-discovery/discover.ts` if the canary + fails the test). - `.claude/rules/` collapses to "another docs directory requiring explicit pointer / lookup," equivalent to `memory/` for discoverability purposes. @@ -91,7 +97,9 @@ empirical result line). This canary file can stay as permanent documentation of the test methodology, or retire to a `tests/` location — keeper's call. -Origin lineage: `feedback_claude_code_loading_taxonomy_rules_vs_skills_vs_claude_md_aaron_2026_05_01.md` -+ the human maintainer's calibration challenge 2026-05-01 -("what's will you magically read and apply all the rules?") -+ the live-off-the-land framing. +Origin lineage: +`memory/feedback_claude_code_loading_taxonomy_rules_vs_skills_vs_claude_md_aaron_2026_05_01.md` +(landed via PR #1164) plus the human maintainer's +calibration challenge 2026-05-01 ("what's will you +magically read and apply all the rules?") plus the +live-off-the-land framing.