feat(B-0170.4): seed path-form-drift fixture + regression test#3696
Merged
Conversation
Adds the third eval-set fixture for B-0170.4, extending regression
coverage from {count, existence} to {count, existence, path-form}.
Same proven shape as PR #3611 (count) and PR #3624 (existence): a
small on-disk markdown file under fixtures/ plus a pinned-expectation
test in fixtures.test.ts.
The fixture references tools/substrate-claim-checker/check-counts.ts
as both a bare basename (`check-counts.ts`) and a fully-qualified
path. Both resolve to the same absolute file via check-path-forms.ts's
3-root strategy (fileDir / parentDir / repoRoot), so the drift is
deterministically detected without depending on synthetic files.
Per PR #3611 review-thread discipline (chatgpt-codex-connector +
copilot): pin exact finding count (1) AND exact body-claim line (28)
so a regression in body-claim detection cannot be silently masked by
an HTML-comment-side match. The provenance comment intentionally
avoids restating either path form.
Re-decomposition note: original B-0170 lists B-0170.1-.4 as children.
B-0170.1 (semantic-equivalence) has an in-flight branch already;
B-0170.2 / .3 introduce brand-new sub-class checkers (bigger slices).
Adding one more fixture under B-0170.4 is genuinely the smallest safe
slice — it extends the proven pattern, has no merge risk, and closes
one more line of the parent row's done-criteria (eval-set coverage).
Focused check: bun test tools/substrate-claim-checker/fixtures.test.ts
→ 3 pass, 0 fail, 17 expect() calls. Full suite: 115 pass, 0 fail.
operative-authorization: aaron 2026-05-14: "- **Devil-pole**
(edge-runner drive): keep pushing, discover, go hard, never-be-idle"
Co-Authored-By: Claude <noreply@anthropic.com>
|
You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard. |
…3696) Per-tick shard documenting the path-form-drift fixture slice landed in PR #3696. Captures the re-decomp reasoning (B-0170.1 has in-flight branch; B-0170.2/.3 are bigger slices; B-0170.4 fixture continuation is smallest safe), the subdecimal-vs-top-level scheme discipline observed (per ac9d9a4 rule), the focused-check outcome, and the catch-43 cron sentinel re-arm at session start. operative-authorization: aaron 2026-05-14: "- **Devil-pole** (edge-runner drive): keep pushing, discover, go hard, never-be-idle" Co-Authored-By: Claude <noreply@anthropic.com>
There was a problem hiding this comment.
Pull request overview
Adds the path-form drift fixture slice for B-0170.4 so the substrate-claim-checker eval-set covers another shipped drift class.
Changes:
- Adds a new path-form drift markdown fixture.
- Adds a regression test invoking
check-path-forms.ts. - Updates the fixture index with the new fixture row.
Reviewed changes
Copilot reviewed 4 out of 4 changed files in this pull request and generated 2 comments.
| File | Description |
|---|---|
tools/substrate-claim-checker/fixtures/README.md |
Adds the path-form drift fixture to the eval-set index. |
tools/substrate-claim-checker/fixtures/path-form-drift-bare-vs-qualified.md |
Introduces the new bare-vs-qualified path-form drift fixture. |
tools/substrate-claim-checker/fixtures.test.ts |
Adds regression coverage for the new path-form fixture. |
Per Copilot review threads on PR #3696: the path-form fixture's anchor was cited as "taxonomy row 4" but path-form is actually instance #15 of the verify-then-claim memo's body table (PR #1256), and sub-class #6 of the 7-class list. Corrects the README index + adds the historical anchor comment in the test. The current fixture remains a synthetic exemplar covering the sub-class; instance #15's literal substance (adjacent ADR citations from PR #1256) is queued as follow-on fixture B-0170.4.1 per the per-thread plan. Co-Authored-By: Claude <noreply@anthropic.com>
AceHack
commented
May 16, 2026
Member
Author
AceHack
left a comment
AceHack
left a comment
There was a problem hiding this comment.
Antigravity check: Clean atomic slice. No blob detected. Shadow check clear.
This was referenced May 16, 2026
AceHack
added a commit
that referenced
this pull request
May 17, 2026
Adds the 5th eval-set fixture for the substrate-claim-checker, covering the convention sub-class of the 7-class verify-then-claim taxonomy. The fixture pair (current ADR + sibling predecessor ADR support file) makes the broken half of the bidirectional ADR supersession convention reproducible without depending on any real ADR pair in the repo. Anchor: PR #2512 (the PR that shipped check-convention.ts) — synthetic exemplar, same shape as the path-form-drift fixture's synthetic case. Focused check outcomes: - bun test tools/substrate-claim-checker/fixtures.test.ts → 5 pass / 0 fail - bun test tools/substrate-claim-checker/ → 117 pass / 0 fail - Direct CLI run reports 1 convention-drift finding on line 36 with the expected reciprocal-marker reason string Composes with B-0170 (parent), B-0170.4 eval-set thread (PRs #3611, #3624, #3696, #3749). operative-authorization: aaron 2026-05-14: "- **Devil-pole** (edge-runner drive): keep pushing, discover, go hard, never-be-idle" Co-authored-by: Claude <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds the third eval-set fixture for B-0170.4 (substrate-claim-checker regression coverage), extending coverage from {count, existence} to {count, existence, path-form}. Same proven shape as PR #3611 (count-drift fixture) and PR #3624 (existence-drift fixture).
tools/substrate-claim-checker/fixtures/path-form-drift-bare-vs-qualified.md— small markdown fixture referencingcheck-counts.ts(bare) andtools/substrate-claim-checker/check-counts.ts(qualified). Both resolve to the same absolute file via the checker's 3-root strategy (fileDir / parentDir / repoRoot) — no synthetic files required.tools/substrate-claim-checker/fixtures.test.ts— new test pins exact finding count (1) AND exact body-claim line (28) per PR feat(B-0170.4): seed eval-set fixture for count-drift regression coverage #3611 review-thread discipline (chatgpt-codex-connector + copilot-pull-request-reviewer): pinning prevents a body-claim regression from being silently masked by an HTML-comment-side match.tools/substrate-claim-checker/fixtures/README.md— index row added.Why this slice
Re-decomp check against B-0170's listed children:
otto-cli/b0170-1-semantic-equiv-checker-2026-05-15already exists; touching it risks duplicate-work split-brain.This PR adds one of those three. Convention + cross-surface fixtures remain as future smallest-safe slices following the same shape.
Focused check outcome
Negative-path stderr from
check-existence.test.tsis intentional (tests verify the ENOENT/EISDIR branches returnok: false).Test plan
fixtures.test.tspasses with exact-count + exact-line pinningComposes with
tools/substrate-claim-checker/check-path-forms.ts(the v0.7 checker this fixture regresses against)memory/feedback_verify_then_claim_discipline_dominant_failure_mode_substrate_authoring_otto_2026_05_03.md— taxonomy row 4 (path-form drift) is the sub-class this fixture covers🤖 Generated with Claude Code