Skip to content

docs(claude.md): add §6.2 "Tests are specifications" bullet - #18

Merged
jensholdgaard merged 1 commit into
mainfrom
docs/claude-md-test-preservation
May 13, 2026
Merged

docs(claude.md): add §6.2 "Tests are specifications" bullet#18
jensholdgaard merged 1 commit into
mainfrom
docs/claude-md-test-preservation

Conversation

@jensholdgaard

@jensholdgaard jensholdgaard commented May 13, 2026

Copy link
Copy Markdown
Owner

Summary

Adds one bullet to CLAUDE.md §6.2 (Testing discipline):

Tests are specifications, not friction. A passing test must not be weakened or deleted without explicit approval. If a refactor or new feature makes an existing test inconvenient to keep working, the right move is to add a new test for the new behaviour alongside; if the old test now asserts something we no longer want, that is a contract change — stop, surface it explicitly, get approval, then change the test. "I edited the test to make it pass" is a code smell that silently rots the safety net. The risk is doubly large in a project that is intentionally fully AI-assisted: AI editors are prone to deleting or weakening inconvenient tests rather than diagnosing the underlying drift, and that pattern must not land here.

Why this rule, why now

The project is intentionally fully AI-assisted. Dave Farley's recent talk on AI-assisted development calls out "deletes inconvenient tests rather than diagnose the failure" as a specific recurring failure mode he's seen. CLAUDE.md §6.2 already lists which tests we write but does not lock in not deleting them. This bullet closes that gap before the integration PRs (cluster + miner widening + Parquet) start putting more load on the test surface.

Meta-RFC waiver (per CLAUDE.md §10)

§10 requires a meta: RFC for changes to this file. Maintainer (sole contributor) has informally waived for this small additive bullet; precedent is commit b50067d "docs: align CLAUDE.md with present-day repo (meta-RFC informally waived)". The footer is updated to record both the date and the waiver.

Verification

  • Doc-only change, no code touched.
  • CLAUDE.md is at the repo root, outside the mdBook source tree (docs/) — no mdbook build needed.
  • No cargo/clippy/test invocations affected.

Test plan

  • Diff review (single file, 19 insertions, 5 deletions, only §6.2 + footer touched)
  • Footer correctly cites the 2026-05-13 revision and the b50067d precedent

🤖 Generated with Claude Code

@jensholdgaard
docs(claude.md): add §6.2 "Tests are specifications" bullet
8043f1b 
A passing test must not be weakened or deleted without explicit
approval; if a refactor makes a test inconvenient to keep working,
add a new test alongside or surface the contract change for
approval — never edit the test into compliance.

Why this rule, why now: the project is intentionally fully
AI-assisted, and AI editors are unusually prone to deleting or
weakening inconvenient tests rather than diagnosing the
underlying drift. Dave Farley calls this out as a recurring
failure mode in his "AI without continuous delivery is a trap"
talk; the rule encodes the safety net at the project level so
neither human nor AI contributors can erode it incrementally.

Footer updated to record the 2026-05-13 revision and to note
the informal meta-RFC waiver (precedent: b50067d).

This is a doc-only change. No code, tests, or build behaviour
affected. mdbook source tree (`docs/`) is untouched, so no
mdbook build needed (CLAUDE.md sits at repo root, outside the
mdBook).
@jensholdgaard
jensholdgaard force-pushed the docs/claude-md-test-preservation branch from ab2453b to 8043f1b Compare May 13, 2026 20:32
@jensholdgaard
jensholdgaard requested a review from Copilot May 13, 2026 20:32
Copilot AI reviewed May 13, 2026
View reviewed changes

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds a new bullet to CLAUDE.md §6.2 codifying that passing tests must not be weakened or deleted without explicit approval, with rationale tied to AI-assisted development risks. Also updates the document footer to record the 2026-05-13 revision and the informal meta-RFC waiver.

Changes:

  • New §6.2 bullet "Tests are specifications, not friction"
  • Updated footer with revision history including the 2026-05-13 entry and waiver precedent

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

@jensholdgaard
jensholdgaard merged commit bf3de01 into main May 13, 2026
11 checks passed
@jensholdgaard jensholdgaard mentioned this pull request Jul 12, 2026
Merged
jensholdgaard added a commit that referenced this pull request Jul 12, 2026
@jensholdgaard @claude
docs(bench): rfc 0031 §9.13 — run #18 latency channel: rfc0031.7 pass…
5931b1b 
…es as written

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WQY9wfrfRggqSpMLH8Xj3Y
Copilot AI mentioned this pull request Jul 12, 2026
Merged
jensholdgaard added a commit that referenced this pull request Jul 12, 2026
@jensholdgaard @claude
docs(bench): rfc 0031 §9.13 — heading spans #18; pass claim scoped to…
d7ea1e4 
… counted runs

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WQY9wfrfRggqSpMLH8Xj3Y
@jensholdgaard jensholdgaard mentioned this pull request Jul 13, 2026
Closed
6 tasks
jensholdgaard added a commit that referenced this pull request Jul 13, 2026
@jensholdgaard @claude
docs(bench): RFC 0031 §9.13 comparative entry draft (runs #8–#18) — m…
b6a139b 
…aintainer-gated fold-in (#494)

* docs(bench): rfc 0031 §9 comparative entry draft (runs #8–#17)

§9.13 compiles the RFC 0031 comparative program's honest-metric era
(runs #8–#17 on corpus/otel-demo-v8 vs digest-pinned Loki 3.5.3):
L1 and L3 provisional must-win passes on both channels, L2
parity-plus storage-side with named levers, the time-window losses
published, the Loki flag deviations and nondeterminism recorded,
and the §7 freeze inputs listed as open maintainer decisions.
Fold-in is maintainer-gated; this is the draft.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* docs(bench): rfc 0031 §9.13 — auditability round: full digest, run #16 row, scoped determinism

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WQY9wfrfRggqSpMLH8Xj3Y

* docs(bench): rfc 0031 §9.13 — every quoted ratio carries its raw loki bytes

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WQY9wfrfRggqSpMLH8Xj3Y

* docs(bench): rfc 0031 §9.13 — l2 ledger carries its raw loki bytes too

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WQY9wfrfRggqSpMLH8Xj3Y

* docs(bench): rfc 0031 §9.13 — full reproduction rows, streaks audit from the entry alone

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WQY9wfrfRggqSpMLH8Xj3Y

* docs(bench): rfc 0031 §9.13 — l2 reproduction rows back the quoted band

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WQY9wfrfRggqSpMLH8Xj3Y

* docs(bench): rfc 0031 §9.13 — run #13's salvaged pairs are counted; say so

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WQY9wfrfRggqSpMLH8Xj3Y

* docs(bench): rfc 0031 §9.13 — deviation flags spelled exactly as passed

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WQY9wfrfRggqSpMLH8Xj3Y

* docs(bench): rfc 0031 §9.13 — bytes floor labeled as analog of the latency gate; .11 citation

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WQY9wfrfRggqSpMLH8Xj3Y

* docs(bench): rfc 0031 §9.13 — bloom provenance cites impl + amendment; full run table

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WQY9wfrfRggqSpMLH8Xj3Y

* docs(bench): rfc 0031 §9.13 — run #18 latency channel: rfc0031.7 passes as written

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WQY9wfrfRggqSpMLH8Xj3Y

* docs(bench): rfc 0031 §9.13 — heading spans #18; pass claim scoped to counted runs

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WQY9wfrfRggqSpMLH8Xj3Y

---------

Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
@jensholdgaard jensholdgaard mentioned this pull request Jul 13, 2026
Merged
jensholdgaard added a commit that referenced this pull request Jul 13, 2026
@jensholdgaard @claude
docs(rfc): RFC 0031 §7 — partial calibration freeze (M_L1/M_L3 @10 st…
fc31a8d 
…orage, F_L6 @3 latency) (#502)

* docs(rfc): rfc 0031 §7 — partial freeze: m_l1/m_l3 at 10 (storage), f_l6 at 3 (latency)

Frozen where three-plus consecutive equivalence-verified runs prove
headroom (L1 77×, L3 21×; window latency floor holds at 0.34/3.43/
1.32); deferred with NAMED conditions where the data says wait:
M_L2 freezes after RFC 0033 lands (until then processed-channel
gate at 10, storage informational), M_L4 and F_L7 freeze on first
measurement. Window bytes reclassified gated-floor → published
diagnostic — the loss is structural, small in absolute terms, and
publishing it honestly is the commitment.

Calibration evidence: benchmarks.md §9.13 (runs #8–#18). Maintainer
delegated the ordering and judgment 2026-07-13.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WQY9wfrfRggqSpMLH8Xj3Y

* docs(rfc): rfc 0031 §3.6/§7 — comparator channels defined; references qualified

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WQY9wfrfRggqSpMLH8Xj3Y

* docs(rfc): rfc 0031 — consistent benchmarks.md formatting

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WQY9wfrfRggqSpMLH8Xj3Y

* docs(rfc): rfc 0031 — channel key labels match benchmarks.md terminology

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WQY9wfrfRggqSpMLH8Xj3Y

* docs(rfc): rfc 0031 §3.6 — channel amendment explicitly supersedes the single-comparator wording

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WQY9wfrfRggqSpMLH8Xj3Y

* docs(rfc): rfc 0031 §3.6 — two-channel model stated in the primary bullet

Also: headChunkBytes described accurately (memory-served, counted so
unflushed data is not free) and the loss band cites both the full
record (0.003–0.018) and current code (0.007–0.018).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WQY9wfrfRggqSpMLH8Xj3Y

---------

Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
This was referenced Jul 13, 2026
Merged
Merged
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

Copilot code review Copilot
Copilot review effort
Applies to this pull request for everyone.Learn more about Copilot code review.
Copilot left review comments

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@jensholdgaard