Skip to content

feat(B-0170.4): seed convention-drift fixture + regression test#4085

Merged
AceHack merged 1 commit into
mainfrom
feat/b-0170-convention-drift-fixture-2026-05-17
May 17, 2026
Merged

feat(B-0170.4): seed convention-drift fixture + regression test#4085
AceHack merged 1 commit into
mainfrom
feat/b-0170-convention-drift-fixture-2026-05-17

Conversation

@AceHack
Copy link
Copy Markdown
Member

@AceHack AceHack commented May 17, 2026

Summary

5th eval-set fixture for the substrate-claim-checker. Covers the convention sub-class of the 7-class verify-then-claim taxonomy via a self-contained fixture pair (current ADR + sibling predecessor ADR support file). Synthetic exemplar; anchor PR #2512 (the PR that shipped check-convention.ts).

Continues the B-0170.4 eval-set thread:

Remaining check-types for B-0170.4 thread completion: semantic-equivalence-drift, empirical-output-drift, self-recursive-drift (none yet shipped as checkers).

Files

  • tools/substrate-claim-checker/fixtures/convention-drift-no-reciprocal-marker.md (new) — current ADR claiming supersession
  • tools/substrate-claim-checker/fixtures/_convention-drift-target-adr.md (new) — predecessor ADR support file (leading-underscore marks "fixture support, not a top-level fixture")
  • tools/substrate-claim-checker/fixtures/README.md — fixture-index row added
  • tools/substrate-claim-checker/fixtures.test.ts — regression test added; pins exact finding count (1), claim line (36), target path, and reason substring ("not reciprocated", "Superseded by")

The fixture pair is self-contained: check-convention.ts's 3-root resolution (fileDir / parentDir / repoRoot) finds the support file via fileDir without dragging in any real repo ADR pair.

Per PR #3611 review-thread discipline, the fixture's HTML-comment provenance intentionally avoids restating the literal Supersedes ADR ... claim so a regression in body-claim detection cannot be masked by a comment-side match.

Focused checks

  • bun test tools/substrate-claim-checker/fixtures.test.ts5 pass / 0 fail / 30 expects
  • bun test tools/substrate-claim-checker/ (full suite) → 117 pass / 0 fail / 274 expects (6 files)
  • Direct CLI run: bun tools/substrate-claim-checker/check-convention.ts tools/substrate-claim-checker/fixtures/convention-drift-no-reciprocal-marker.md → exits 1 with 1 convention-drift finding(s) on line 36 with reciprocal-marker reason string
  • Tree-count canary: git ls-tree HEAD = 53 entries (matches origin/main)

Test plan

  • Focused fixture test passes
  • Full substrate-claim-checker suite passes
  • Direct CLI invocation reports expected finding shape
  • No commit-tree corruption (53 root entries vs 53 on main)

Composes with

🤖 Generated with Claude Code

Adds the 5th eval-set fixture for the substrate-claim-checker, covering
the convention sub-class of the 7-class verify-then-claim taxonomy. The
fixture pair (current ADR + sibling predecessor ADR support file) makes
the broken half of the bidirectional ADR supersession convention
reproducible without depending on any real ADR pair in the repo.

Anchor: PR #2512 (the PR that shipped check-convention.ts) — synthetic
exemplar, same shape as the path-form-drift fixture's synthetic case.

Focused check outcomes:
- bun test tools/substrate-claim-checker/fixtures.test.ts → 5 pass / 0 fail
- bun test tools/substrate-claim-checker/ → 117 pass / 0 fail
- Direct CLI run reports 1 convention-drift finding on line 36 with the
  expected reciprocal-marker reason string

Composes with B-0170 (parent), B-0170.4 eval-set thread (PRs #3611,
#3624, #3696, #3749).

operative-authorization: aaron 2026-05-14: "- **Devil-pole** (edge-runner drive): keep pushing, discover, go hard, never-be-idle"

Co-Authored-By: Claude <noreply@anthropic.com>
Copilot AI review requested due to automatic review settings May 17, 2026 14:01
@AceHack AceHack enabled auto-merge (squash) May 17, 2026 14:01
Copy link
Copy Markdown

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds the fifth empirical eval-set fixture to tools/substrate-claim-checker, covering the convention drift checker (ADR supersession reciprocity) and wiring a regression test to ensure the checker continues to flag the missing reciprocal “Superseded by” marker pattern.

Changes:

  • Added a self-contained convention-drift fixture pair (current ADR + predecessor support ADR) under tools/substrate-claim-checker/fixtures/.
  • Extended fixtures.test.ts with a regression test that pins exact finding shape (count, claim line, target path, reason substrings).
  • Updated the fixtures index README to include the new convention-drift fixture row.

Reviewed changes

Copilot reviewed 4 out of 4 changed files in this pull request and generated no comments.

File Description
tools/substrate-claim-checker/fixtures/README.md Adds the convention-drift fixture row to the eval-set index.
tools/substrate-claim-checker/fixtures/convention-drift-no-reciprocal-marker.md New “current ADR” fixture containing a Supersedes ADR \...`` body claim.
tools/substrate-claim-checker/fixtures/_convention-drift-target-adr.md New predecessor ADR support file intentionally missing the reciprocal “Superseded by” marker.
tools/substrate-claim-checker/fixtures.test.ts Adds a convention-drift regression test that asserts exact finding count/line/target/reason substrings.

@AceHack AceHack merged commit be0c93e into main May 17, 2026
33 of 34 checks passed
@AceHack AceHack deleted the feat/b-0170-convention-drift-fixture-2026-05-17 branch May 17, 2026 14:05
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants