Skip to content

review(pr-1255-postmerge): row-7 prose-form fix + drift catalogue extended to 15 instances#1257

Merged
AceHack merged 2 commits intomainfrom
free-memory/skill-design-memo-table-row-7-prose-fix-aaron-2026-05-03
May 3, 2026
Merged

review(pr-1255-postmerge): row-7 prose-form fix + drift catalogue extended to 15 instances#1257
AceHack merged 2 commits intomainfrom
free-memory/skill-design-memo-table-row-7-prose-fix-aaron-2026-05-03

Conversation

@AceHack
Copy link
Copy Markdown
Member

@AceHack AceHack commented May 3, 2026

Summary

P2 finding on PR #1255 (already merged): my "fix" of row 7 in the skill-design memo was a second semantic-equivalence drift on the same substitution attempt.

Two recursive failures on the same substitution:

  1. First attempt: ls docs/DECISIONS/ | grep -iEfind docs/DECISIONS/ -iname "*<pattern>*" (caught: find -iname only does shell glob, not regex alternation)
  2. Second attempt: → grep -ilrE "<pattern>" docs/DECISIONS/ (caught: grep -r searches file CONTENTS, not filenames)

Both attempts failed because I claimed semantic-equivalence without empirical verification. Exactly the failure mode the verify-then-claim discipline names; happening on the very memo defining the discipline.

Fix

  • Row 7 of the skill-design memo's procedure-table now describes the search in prose with a NOTE warning against the wrong substitution
  • Canonical command pulled OUT of the table cell into a fenced code block below the table — preserves working command without pipe-in-cell escapes

Drift catalogue extended

The verify-then-claim memo's drift catalogue grew from 9 → 15+ instances. New entries (post-naming):

Added new section identifying recurring sub-classes:

  • existence-drift / count-drift / semantic-equivalence-drift / empirical-output-drift / convention-drift / path-form-drift / self-recursive-drift

Headline update: 6 of 15 instances landed AFTER the discipline was named. Manual discipline insufficient. tools/substrate-claim-checker/ TS tool is the only path forward.

Test plan

🤖 Generated with Claude Code

…ended to 15 instances

P2 finding on PR #1255 (already merged): my "fix" of row 7 in
the skill-design memo (replacing `ls docs/DECISIONS/ | grep
-iE` with `grep -ilrE PATTERN docs/DECISIONS/`) was ALSO a
semantic-equivalence drift — `grep -r` searches file CONTENTS,
not filenames. The original ls|grep searches filenames.

Two recursive failures on the same substitution attempt:
- First: substituted ls|grep → find -iname (shell glob only)
- Second: substituted ls|grep → grep -ilrE (file-content search)

Both attempts failed because I claimed semantic-equivalence
without empirical verification. Exactly the failure mode the
verify-then-claim discipline names; happening on the very
memo defining the discipline.

Fixes:
- Row 7 of the skill-design memo's procedure-table now describes
  the search in prose ("ADR-filename search by regex pattern")
  with a NOTE warning against the wrong substitution (`grep
  -ilrE` searches contents not filenames)
- Canonical command pulled OUT of the table cell into a fenced
  code block below the table — preserves the working command
  without needing pipe-in-cell escapes

Verify-then-claim memo's drift catalogue extended:
- Was 9 instances; now 15+ (added #10-#15 covering all in-session
  drift caught after the memo was authored)
- New section: recurring sub-classes (existence-drift, count-drift,
  semantic-equivalence-drift, empirical-output-drift, convention-
  drift, path-form-drift, self-recursive-drift)
- Updated headline to reflect that 6 of 15 instances landed
  AFTER the discipline was named — strongest empirical urgency
  for mechanization

Manual discipline insufficient against trained-prior pull. The
substrate-claim-checker TS tool is the only path forward.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Copilot AI review requested due to automatic review settings May 3, 2026 00:42
@AceHack AceHack enabled auto-merge (squash) May 3, 2026 00:42
…ion drift caught; catalogue at 15+

Same substitution attempt (ls|grep → alternative single-command)
failed twice in succession. Drift catalogue extended with 7
recurring sub-classes. 6 of 15 instances landed AFTER the
verify-then-claim discipline was named — manual discipline
insufficient; mechanization required.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
@AceHack AceHack merged commit 3dce194 into main May 3, 2026
22 of 23 checks passed
@AceHack AceHack deleted the free-memory/skill-design-memo-table-row-7-prose-fix-aaron-2026-05-03 branch May 3, 2026 00:45
Copy link
Copy Markdown

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Updates two existing memory/ feedback memos to correct a previously-merged semantic-equivalence drift in an ADR-filename search procedure and to expand the “verify-then-claim” drift catalogue/taxonomy with additional instances.

Changes:

  • Rewrites the skill-design memo’s row-7 ADR search guidance into prose and moves the canonical ls | grep -iE example into a fenced code block (avoids markdown table pipe escaping and warns against the incorrect grep -r substitution).
  • Extends the verify-then-claim memo’s empirical drift catalogue from 9→15+ instances and adds a recurring sub-classes breakdown.
  • Updates surrounding narrative to reflect the new drift instances and the recursive failure mode.

Reviewed changes

Copilot reviewed 3 out of 3 changed files in this pull request and generated 3 comments.

File Description
memory/feedback_verify_then_claim_discipline_dominant_failure_mode_substrate_authoring_otto_2026_05_03.md Expands drift-instance catalogue and adds drift sub-class taxonomy.
memory/feedback_skills_as_carved_sentences_knowledge_in_docs_datavault_2_0_pattern_aaron_2026_05_03.md Fixes row-7 ADR-filename search guidance and adds canonical command as a fenced block.

@AceHack
Copy link
Copy Markdown
Member Author

AceHack commented May 3, 2026

All 3 post-merge findings addressed in follow-up PR #1259:

  • Frontmatter description count drift (9→18+) → fixed
  • Body line 91 count drift → fixed
  • MEMORY.md index entry count drift + manual-insufficient framing → updated

This is itself drift instance #19 (frontmatter ↔ body sub-pattern within count drift). Both substrate surfaces should match body content; sub-class added to substrate-claim-checker spec.

AceHack added a commit that referenced this pull request May 3, 2026
…ist + tool-status across memo

4 substantive findings on PR #1259 (in-flight):

1. **Section heading drift** — "## Empirical evidence (this
   session, 9+ PRs, 15+ distinct drift instances)" still said
   "15+" while body table has 20 rows + summary says 20.
   Updated heading to "20 distinct drift instances".

2. **Carved sentence stale at "9"** — line 115 still said
   "9 instances caught across 7 PRs". Updated to "20 instances
   across 9+ PRs" + named that instances #10-#20 landed after
   discipline-naming + named v0-shipped status.

3. **PR list incorrect** — frontmatter listed `#1247` (not in
   table) and excluded `#1249, #1257, #1259` (which ARE in
   table). Corrected to `#1245, #1248/#1249, #1250, #1252,
   #1253, #1254, #1255, #1256, #1257, #1259`.

4. **"Until tool ships" + "v0 shipped" contradiction** —
   reorganized §96 to put tool-status FIRST ("v0 shipped covering
   count-drift; v1+ extends to remaining 6 sub-classes; until
   v1+ ships covering all 7, the discipline outside count-drift
   is still manual").

2 tick-shard findings (0049Z + 0058Z) NOT addressed — tick
shards are append-only history preserving agent-belief-at-time.
The shards accurately recorded my belief at write-time; the
underlying memo is the canonical truth and is fixed in this PR.
A note in the next tick shard acknowledges the over-claims.

Drift instances #21 + #22 + #23 + #24 (this PR's own findings)
are not yet catalogued in the table — they will land in the
next sync pass to avoid recursing forever in this PR.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
AceHack added a commit that referenced this pull request May 3, 2026
…tmatter + body + MEMORY.md (#1259)

* review(pr-1257-postmerge): update verify-then-claim count drift (9→18+) in frontmatter + body + MEMORY.md

Copilot post-merge findings on PR #1257 (already merged):
the body of verify-then-claim memo says "15+ drift instances"
but the FRONTMATTER description and MEMORY.md index entry
still say "9 drift instances" — count drift between body
and metadata.

This is itself drift instance #19 (count drift, sub-class
already catalogued). Fixed in three places:

1. **Frontmatter description** updated 9 → 18+, names the
   PRs covered (#1245-#1256 and counting), names the 7 sub-
   classes catalogued, sharpens the manual-insufficient
   framing to reflect post-naming drift.

2. **Body line 91** ("9 drift instances above" → "18+ drift
   instances above across 7 recurring sub-classes").

3. **MEMORY.md index entry** updated to reflect 18+ count + 7
   sub-classes + manual-insufficient framing + the
   instances-#10-#18-landed-AFTER-naming evidence.

The frontmatter ↔ body drift is itself a recurring sub-class
within count-drift: when body content updates but metadata
doesn't, the index summary lies. The substrate-claim-checker
TS tool spec gets another check: scan frontmatter description
+ MEMORY.md entry against body content for count consistency.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

* hygiene(tick-history): 2026-05-03T00:49Z — frontmatter↔body↔MEMORY.md count drift caught (drift #19)

Body said 15+, frontmatter description + MEMORY.md said 9 —
count drift across surfaces. Each new tick produces new drift
instances even when the discipline cataloguing the drift was
authored last tick. Mechanization (substrate-claim-checker TS
tool) is the only path. Spec gets another concrete check:
cross-surface count consistency.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

* review(pr-1259): add table rows #16-#20 to match the "20" count claim

Copilot caught: frontmatter description + MEMORY.md said "18+
drift instances" but body table only had 15 rows — opposite-
direction count drift introduced by the very PR fixing the
prior count drift. **This is itself drift instance #20** —
self-recursive count drift; the count-fix introduces new
count drift in the opposite direction.

Fix: added 6 catalogue rows to the body table (#16-#20)
matching the claimed 20-instance count. Body now has 20 rows;
all three surfaces (frontmatter description + body table +
MEMORY.md index entry) consistent at 20.

The 6 new rows document drift instances #16-#20 — including
THIS PR's own drift as instance #20, demonstrating the
self-recursive sub-class explicitly.

Also updated:
- Sub-class section: self-recursive instances now [#10, #11, #19, #20]
- Body line 96: "20 drift instances above" + note that v0 of
  substrate-claim-checker shipped in PR #1260
- Frontmatter description: count → 20; instances range →
  #10-#20; v0 shipped reference
- MEMORY.md: count → 20; v0 shipped reference

This is the perfect worked example for the substrate-claim-
checker tool's value: the very count-drift-fix produced new
count drift, which the tool catches automatically. v0 (PR
#1260) would have caught this pre-publish.

Verified manually: `awk '/Drift instance/,/^$/'` + `grep -c
"^| [0-9]"` returns 20 rows; matches all 3 surfaces.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

* hygiene(tick-history): 2026-05-03T00:58Z — count-fix introduced opposite-direction drift; body extended to 20 rows

Even authoring a PR to fix count drift produces opposite-direction
count drift. Drift instance #20 self-recursively documents this
PR's own drift. Substrate-claim-checker v0 (PR #1260) would have
caught it pre-publish — empirical evidence v0 was the right
architectural answer.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

* review(pr-1259): synchronize section heading + carved sentence + PR list + tool-status across memo

4 substantive findings on PR #1259 (in-flight):

1. **Section heading drift** — "## Empirical evidence (this
   session, 9+ PRs, 15+ distinct drift instances)" still said
   "15+" while body table has 20 rows + summary says 20.
   Updated heading to "20 distinct drift instances".

2. **Carved sentence stale at "9"** — line 115 still said
   "9 instances caught across 7 PRs". Updated to "20 instances
   across 9+ PRs" + named that instances #10-#20 landed after
   discipline-naming + named v0-shipped status.

3. **PR list incorrect** — frontmatter listed `#1247` (not in
   table) and excluded `#1249, #1257, #1259` (which ARE in
   table). Corrected to `#1245, #1248/#1249, #1250, #1252,
   #1253, #1254, #1255, #1256, #1257, #1259`.

4. **"Until tool ships" + "v0 shipped" contradiction** —
   reorganized §96 to put tool-status FIRST ("v0 shipped covering
   count-drift; v1+ extends to remaining 6 sub-classes; until
   v1+ ships covering all 7, the discipline outside count-drift
   is still manual").

2 tick-shard findings (0049Z + 0058Z) NOT addressed — tick
shards are append-only history preserving agent-belief-at-time.
The shards accurately recorded my belief at write-time; the
underlying memo is the canonical truth and is fixed in this PR.
A note in the next tick shard acknowledges the over-claims.

Drift instances #21 + #22 + #23 + #24 (this PR's own findings)
are not yet catalogued in the table — they will land in the
next sync pass to avoid recursing forever in this PR.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

* hygiene(tick-history): 2026-05-03T01:06Z — 5-surface count-drift sub-pattern; prior shards over-claimed "all surfaces consistent"

Memos have 5 count-bearing surfaces (frontmatter + body table +
section heading + carved sentence + MEMORY.md), not just 3. Prior
shards (0049Z + 0058Z) claimed "all 3 surfaces consistent" when
the section heading + carved sentence still had stale counts.
Acknowledgment lands here in append-only history; substrate-claim-
checker v1+ spec gets enumeration of all count-bearing surfaces.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants