Skip to content

fix(b-0260): make reindexer recursive — scan memory/ subdirectories#3109

Merged
AceHack merged 2 commits into
mainfrom
feat/b-0260-reindexer-recursive-subdir-2026-05-14
May 14, 2026
Merged

fix(b-0260): make reindexer recursive — scan memory/ subdirectories#3109
AceHack merged 2 commits into
mainfrom
feat/b-0260-reindexer-recursive-subdir-2026-05-14

Conversation

@AceHack
Copy link
Copy Markdown
Member

@AceHack AceHack commented May 14, 2026

Summary

  • Root cause diagnosed: collectEntries in reindex-memory-md.ts used non-recursive readdir, silently excluding files in memory/ subdirectories (e.g. memory/observed-phenomena/). Both affected files had valid YAML frontmatter — the parity validator's gap-no-frontmatter label was a misleading diagnosis; the true cause was gap-subdirectory-not-scanned.
  • Fix: extracted collectEntriesRecursive helper using readdir({ withFileTypes: true }), recursing into subdirs and storing paths relative to baseDir (e.g. observed-phenomena/file.md) — correct for both MEMORY.md markdown links and the parity-validator lookup set.
  • Tests: 2 new tests added (scans subdirectories recursively, uses subdir-relative path for subdirectory entries). All 20/20 tests pass.
  • Closes B-0260: all acceptance criteria met.

Parity result

Old-index entries:   672
Unique files:        672
Covered (ok):        672   ← was 670 (2 gaps before this fix)
No-frontmatter gap:  0     ← was 2
Missing (loss):      0

Full parity: all old-index files are covered by the generated index.

B-0260 acceptance criteria

  • memory/MEMORY.md generated by tooling, not hand-edited
  • ✅ Entry coverage parity verified: 672/672 covered
  • ✅ Zero missing entries (no coverage loss)
  • ✅ No intentional behavior changes requiring follow-up issues

Build gate

dotnet build -c Release → 0 warnings, 0 errors
bun test tools/memory/ → 20 pass, 0 fail
bun tools/memory/validate-memory-parity.ts → 672/672 covered, 0 gaps, 0 missing

Test plan

  • bun test tools/memory/reindex-memory-md.test.ts — all 20 pass
  • bun tools/memory/validate-memory-parity.ts — full parity confirmed
  • dotnet build -c Release — 0 warnings, 0 errors
  • B-0260 backlog item marked closed with closure notes
  • BACKLOG.md regenerated (B-0260 shows [x])

🤖 Generated with Claude Code

Closes B-0260 final slice.

Root cause: `collectEntries` in `reindex-memory-md.ts` called
`readdir(targetDir)` without recursion. Files in subdirectories
(e.g. `memory/observed-phenomena/`) had valid YAML frontmatter but
were silently excluded from the generated index. The parity validator
(`tools/memory/validate-memory-parity.ts`) flagged them as
`gap-no-frontmatter`, which was a misleading diagnosis — the actual
cause was `gap-subdirectory-not-scanned`.

## Changes

tools/memory/reindex-memory-md.ts
  Extracted `collectEntriesRecursive` helper that uses
  `readdir({ withFileTypes: true })` to distinguish files from
  directories and recurse. Subdirectory entries use a path relative
  to `baseDir` (e.g., `observed-phenomena/file.md`) — correct as
  both a markdown link target in MEMORY.md and as a parity-validator
  lookup key. Added `relative` import from `node:path`.

tools/memory/reindex-memory-md.test.ts
  Two new tests: "scans subdirectories recursively" and "uses
  subdir-relative path for subdirectory entries". All 20 tests pass.

tools/memory/testdata/subdir/gamma_2026_05_14.md
  New test fixture: a memory file in a subdirectory, used by the
  new recursive-scanning tests.

## Parity result after fix

  Old-index entries:   672
  Unique files:        672
  Covered (ok):        672   ← was 670 (2 gaps)
  No-frontmatter gap:  0     ← was 2
  Missing (loss):      0

Full parity: all old-index files are covered by the generated index.

## B-0260 acceptance criteria

✅ MEMORY.md is generated by tooling, not hand-edited
✅ Entry coverage parity verified: 672/672 covered
✅ Zero missing entries (no coverage loss)
✅ No intentional behavior changes requiring follow-up issues

Co-Authored-By: Claude <noreply@anthropic.com>
Copilot AI review requested due to automatic review settings May 14, 2026 07:08
@AceHack AceHack enabled auto-merge (squash) May 14, 2026 07:08
Copy link
Copy Markdown

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Fixes B-0260 by making the memory reindexer recursive so files in memory/ subdirectories (e.g., memory/observed-phenomena/) are included in MEMORY.md. Resolves the final 2 parity gaps (672/672 covered).

Changes:

  • Replaces non-recursive collectEntries with a collectEntriesRecursive helper using readdir({ withFileTypes: true }), storing filename as a path relative to baseDir.
  • Adds 2 tests covering recursive discovery and subdir-relative filename.
  • Marks B-0260 closed (backlog row + regenerated docs/BACKLOG.md) and updates MEMORY.md heap-count footer (1103 → 1130).

Reviewed changes

Copilot reviewed 6 out of 6 changed files in this pull request and generated no comments.

Show a summary per file
File Description
tools/memory/reindex-memory-md.ts Recursive scan + relative-path filename for entries.
tools/memory/reindex-memory-md.test.ts Adds two tests for recursive subdir scanning.
tools/memory/testdata/subdir/gamma_2026_05_14.md Test fixture under a subdir.
memory/MEMORY.md Footer heap-count updated to reflect new total.
docs/backlog/P1/B-0260-...md Status → closed with closure notes.
docs/BACKLOG.md Regenerated index entry checkbox for B-0260.

Copy link
Copy Markdown

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 895b51c111

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread tools/memory/reindex-memory-md.ts Outdated
Normalize relative() output to forward slashes so subdirectory entries
are portable across OS (Windows path\\file.md -> path/file.md). Fixes
parity validator mismatches on non-POSIX hosts per Codex review.

Also add blank line before acceptance-criteria list to satisfy MD032.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
@AceHack AceHack merged commit a217965 into main May 14, 2026
30 checks passed
@AceHack AceHack deleted the feat/b-0260-reindexer-recursive-subdir-2026-05-14 branch May 14, 2026 07:16
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants