Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
20 changes: 20 additions & 0 deletions tools/substrate-claim-checker/fixtures.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,7 @@
import { describe, expect, test } from "bun:test";
import { join } from "node:path";
import { checkFile as checkCounts } from "./check-counts.ts";
import { checkFile as checkExistence } from "./check-existence.ts";

const fixtures = join(import.meta.dir, "fixtures");

Expand All @@ -35,3 +36,22 @@ describe("eval-set fixtures / count drift", () => {
expect(finding.claimIsMinimum).toBe(false);
});
});

describe("eval-set fixtures / existence drift", () => {
test("existence-drift-missing-doc.md — backtick-quoted path that doesn't exist anywhere is detected", () => {
const result = checkExistence(join(fixtures, "existence-drift-missing-doc.md"));
expect(result.ok).toBe(true);
// Same PR #3611 discipline applied to existence-drift: exact count
// and pin the body claim's line so a regression in body-claim
// detection cannot be masked by an HTML-comment match. The fixture
// path is intentionally synthetic (won't be created accidentally),
// so the test stays stable across substrate evolution.
expect(result.findings.length).toBe(1);
const finding = result.findings[0]!;
expect(finding.line).toBe(24);
expect(finding.pathClaim).toBe(
"docs/_fixture_existence_drift_target_b0170_2026_05_15.md",
);
expect(finding.severity).toBe("drift");
});
});
1 change: 1 addition & 0 deletions tools/substrate-claim-checker/fixtures/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -22,6 +22,7 @@ axis.
| Sub-class | Fixture | Anchor PR / history | Expected finding |
|---|---|---|---|
| count drift | `count-drift-9-vs-15.md` | PR #1259 (`review(pr-1257-postmerge): verify-then-claim count drift (9→18+) frontmatter + body + MEMORY.md`) | "9 drift instances" claim vs 15-row table |
| existence drift | `existence-drift-missing-doc.md` | PR #1252 — verify-then-claim memo body table instance #8 (`future-domain memo references docs/courier-ferry-protocol.md` / doesn't exist) | backtick-quoted synthetic path resolves at no candidate root |

Add a new row when a new fixture lands.

Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,24 @@
<!--
Eval-set fixture for substrate-claim-checker (B-0170).

Reproduces the existence-drift pattern surfaced in PR #1252,
where a future-domain memo referenced a docs/ markdown file
that did not actually exist on disk. The historical eval-set
catalogues this as instance #8 of the verify-then-claim memo's
body table (Existence drift sub-class).

The fixture references a clearly synthetic path that will
never be created accidentally, so the fixture stays stable
across substrate evolution.

NOTE: this comment intentionally avoids backtick-quoting
the exact fixture path. Per PR #3611 review threads
(chatgpt-codex-connector + copilot-pull-request-reviewer),
restating a path claim inside the HTML provenance comment
produces a spurious second finding and lets regressions
in body-claim detection slip past `fixtures.test.ts`.
-->

# Existence-drift catalogue (eval-set fixture)

The drift instance below cites `docs/_fixture_existence_drift_target_b0170_2026_05_15.md` as the canonical source.
Loading