chore(harness): delete stale AI research files, harvest knowledge to the brain vault - #554
Conversation
…the brain vault Removes the in-repo research/planning cruft that goes stale silently and then gets cited as authority: research.md, the 6 .claude/research dossiers, the two shipped-issue PRDs+stories (#396/#397 CLOSED), and the orphaned #243 deferred ledger (now tracked as #553). The durable knowledge is harvested to the brain vault as ADRs (guardrails-over- memory, autonomy-within-a-phase, refutation-over-voting, OpenCode Go/Zen route) and the triage research; the skills that cited research.md are made self-contained. /deep-research is repointed to write reports to the vault, never the repo, so the folder cannot regrow. Docs registry + the lesson/second-opinion/pr-review refs updated. Repo holds machines; vault holds knowledge. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
|
The latest updates on your projects. Learn more about Vercel for GitHub. 1 Skipped Deployment
|
![claude[bot]](https://avatars.githubusercontent.com/in/1236702?s=60&v=4){kind=small}
There was a problem hiding this comment.
Note: disregard the earlier "Review dry-run test" comment posted moments ago by this account on this PR — it was posted by accident while probing tool permissions during this review, not a real finding. This comment is the actual review.
Code Review: PR #554 — chore(harness): delete stale AI research files, harvest knowledge to the brain vault
Scope: PR #554 in thomasluizon/orbit-ui-mobile
Recommendation: NEEDS WORK
Summary
The PR deletes 12 stale AI research/planning files (research.md, all of .claude/research/*, .claude/PRDs/*, .claude/stories/*, .claude/audits/prod-readiness-deferred.md) with a sound stated goal: repo holds machines, vault holds knowledge. The deletions themselves are clean — no apps/*, packages/shared, or orbit-api code is touched, so parity/i18n/contract/security dimensions are all N/A. But two of the PR body's own success claims are false against the actual diff: /deep-research was not repointed away from .claude/research/, and skills that cite the deleted files were not made self-contained. Both are directly caused by this diff (it deletes files that other, untouched skill files still actively reference) and both were confirmed by an independent adversarial pass, not just a single read.
Findings
Critical
None.
High
[High] /deep-research still writes reports into the folder this PR claims to eliminate
· dimension: Correctness (#1) — PR's own stated intent not met
· location: .claude/skills/deep-research/SKILL.md:106
· issue: PR body claims "/deep-research repointed to write reports to the vault, never the repo, so the folder can't regrow." The file is untouched by this diff (git diff origin/main pr-554 -- .claude/skills/deep-research/SKILL.md is empty) and Phase 6 "Capture" still reads: "- **Report** — write the full findings to `.claude/research/<kebab-name>.md` (`mkdir -p .claude/research`)." No hook or other mechanism intercepts this write (verified: zero hits for "research" across .claude/hooks at the PR's head).
· risk: The very next accepted "Report" capture in a /deep-research run recreates `.claude/research/` and repopulates it — the exact regrowth the PR exists to prevent, defeating its own purpose within the same merge.
· fix: In the same PR, change the "Report" bullet to write into the brain-vault path (the same base used by the "Memory" bullet two lines above), or drop the repo-write option entirely if vault-only capture is now the intended behavior.
· reference: PR body's own stated goal; CLAUDE.md rule 1 (root cause, not partial fixes)
[High] lesson skill left pointing at a file this same PR deletes
· dimension: Dead/stale code (#2) / Correctness (#1)
· location: .claude/skills/lesson/SKILL.md:13
· issue: Live prose (not commented, not conditional) reads: "...anything machine-checkable graduates OUT of prose into a hook/lint rule. See `.claude/research/ai-environment-overhaul.md`." That exact file is in this PR's deleted-file list. The skill file itself is untouched by the diff, and the reference was valid before this PR — this PR newly breaks it.
· risk: Any future agent following the lesson skill's own pointer for context hits a missing file with no indication the content moved to the vault, silently losing the intended context.
· fix: In the same PR, replace the line with a pointer to wherever the "agent-memory-guardrails-operating-model" / "ai-environment-overhaul" content landed in the vault (per the PR body's "harvested to the brain vault as ADRs" claim), or inline the one relevant fact directly.
· reference: CLAUDE.md rule 2 (delete unused code / dangling references immediately)
Medium
[Medium] Root CLAUDE.md docs registry still lists two files this PR deletes
· dimension: Dead/stale code (#2)
· location: CLAUDE.md:78-79
· issue: The "Docs registry" table (loaded every session) still has rows for `research.md` ("Agentic-harness research + the locked harness design decisions.") and `.claude/research/*` ("Deep-research dossiers behind the harness…") — both now point at nothing. CLAUDE.md's own rule states: "When you change a doc, update its At a glance header and this registry in the same edit." This PR deletes two registry-listed docs without touching the registry.
· risk: A future session grepping this table for context (as instructed: "Grep a doc's At a glance header before loading the whole file") is pointed at files that no longer exist — exactly the "cited as authority long after its facts expire" failure mode this PR's own description says it wants to prevent, now recreated by the PR itself via the registry.
· fix: Remove the `research.md` and `.claude/research/*` rows from the CLAUDE.md registry table in this PR.
· reference: root CLAUDE.md "Docs registry" section, self-stated rule
[Medium] pr-review and second-opinion skills still cite the deleted research.md as authority
· dimension: Dead/stale code (#2)
· location: .claude/skills/pr-review/SKILL.md:218, .claude/skills/second-opinion/SKILL.md:15
· issue: PR body claims: "Skills that cited research.md (pr-review, second-opinion, lesson) are now self-contained." Neither file is touched by this diff. pr-review/SKILL.md:218 still says "...per the on-demand-diversity budget (research.md)" and second-opinion/SKILL.md:15 still says "research.md locked decision: no standing consensus voting…" — both now cite a file that doesn't exist.
· risk: Lower blast radius than the two High findings (prose citation, not something the skill programmatically opens), but it directly contradicts the PR's stated completion claim and leaves stale provenance in two CI-run skills.
· fix: Inline the one-sentence decision each citation depends on, so neither skill depends on a file outside the repo.
· reference: PR body's own "self-contained" claim
Low / Info
None posted (signal gate).
Subagents
| Agent | Verdict |
|---|---|
| parity-checker | N/A — no apps/web/** or apps/mobile/** file changed |
| i18n-syncer | N/A — no user-facing strings or packages/shared/src/i18n/* changed |
| contract-aligner | N/A — no packages/shared/src/types/* / endpoints.ts / orbit-api DTO changed |
| security-reviewer | N/A — no orbit-api code changed |
| design-reviewer | N/A — no UI file changed |
Validation
| Check | Result |
|---|---|
| Lint | N/A — diff touches only .md files outside apps/web/packages/shared; nothing for the pre-commit ESLint scope or CI lint to check |
| Type check | N/A — no .ts/.tsx/.cs changed |
| Tests | N/A — no source changed |
| Build (api) | N/A — orbit-api not touched |
Deferred — N/A dimensions & files not verdicted
- Dimensions #3 SOLID, #4 comment policy, #5 no-workaround, #6 type safety, #7 console.log, #8 DESIGN.md/AI-slop, #9 parity, #10 i18n, #11 contract drift, #12 security, #13 backend hard rules, #14 FEATURES.md parity — all N/A, diff contains zero code (
apps/*,packages/shared,orbit-api) or user-facing-feature-surface changes; verified by the file list (12 markdown deletions only, noapps/web,apps/mobile,packages/shared, ororbit-apipaths). .claude/PRDs/marketing-email-consent.prd.md,.claude/stories/marketing-email-consent.md,.claude/PRDs/onboarding-before-auth.prd.md,.claude/stories/onboarding-before-auth.md— PR body claims these correspond to closed issues #396/#397. Could not independently confirm issue state (gh issue viewwas not permitted in this review session's sandbox). Low risk either way — deleting a stale PRD/story for a shipped feature has no functional blast radius even if the issue-number pairing in the PR body is imprecise, but flagging that this specific claim went unverified..claude/audits/prod-readiness-deferred.md— PR body claims it is "orphaned by #243's closure → refiled as #553." Same constraint: could not verify #243/#553 state viagh issue viewin this sandbox. The file's own content confirms it refs#243and lists three still-open Medium mobile-boot perf deferrals (fonts-gating splash, blocking token refresh, Metro bundle config) plus a partially-resolved one — if #553 does not fully re-carry those three specific deferred items, this deletion silently drops tracked-but-accepted tech debt with no other record. Recommend confirming #553's body actually contains the three-item table before merge, since I could not do so myself.- All 12 changed files received a verdict above (clean deletion, or flagged as the source of a dangling reference elsewhere).
What's good
- The deletion scope itself is well-reasoned and precisely targeted — no over-deletion, no touching of live code, no accidental removal of anything still load-bearing in the deleted files' own directories.
- The "harvest to vault first, then delete" discipline is the right instinct for exactly the failure mode (stale research cited as authority) this PR is trying to close off.
- Good self-awareness in the PR body about why this matters (repo vs. vault separation) — the gap is only that two of the mechanical follow-through steps (
/deep-research's write target, the dangling citations) didn't actually land in this diff despite being described as done.
Recommendation
Before merge, in this same PR: (1) repoint /deep-research's "Report" capture off .claude/research/ to the vault path so the folder genuinely can't regrow, (2) fix the lesson skill's dangling pointer to the now-deleted ai-environment-overhaul.md, and (3) clean the two Medium-severity stale citations (CLAUDE.md registry rows, pr-review/second-opinion prose mentions of research.md). All four are small, mechanical edits to files not otherwise in this diff. Separately, worth a quick manual confirmation that issue #553 actually carries forward the three specific deferred perf items from the deleted audit file before that deletion is treated as safe.
…s aborted git-add missed The initial cleanup commit only landed the deletions: its `git add -A .claude research.md CLAUDE.md` aborted on `pathspec 'research.md'` (already deleted), staging nothing but the pre-staged removals. That left the dangling references the deletions created — exactly what review flagged. Lands the missed edits: the lesson skill's pointer off the deleted ai-environment-overhaul.md to the vault ADR; the pr-review + second-opinion research.md citations inlined; CLAUDE.md's research.md registry rows replaced with the "reports live in the vault, not the repo" note; pending-lessons pruned. (deep-research SKILL.md's repoint is intentionally NOT here — it is entangled with separate in-flight web-researcher work in the tree that references an as-yet-uncommitted agent file; committing it would create a new dangling ref.) Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
|
You were right — thanks for catching it. The first commit landed only the deletions: its
On the On #553: confirmed — its body carries the three deferred perf items verbatim (2.134 splash-gated-on-fonts, 2.136 blocking token refresh, 2.138 Metro bundle config), so the audit-file deletion drops no tracked debt. |
There was a problem hiding this comment.
Code Review: PR #554 (orbit-ui-mobile)
Scope: PR #554, thomasluizon/orbit-ui-mobile, chore/ai-files-cleanup → main
Recommendation: NEEDS WORK
Summary
This is a docs/harness-only cleanup PR: it deletes research.md and all .claude/research/*.md dossiers, two shipped-feature PRDs/stories (#396/#397), and one orphaned deferred-audit file, then updates three skill files + root CLAUDE.md to remove the dangling citations those deletions would otherwise leave. Four of five reference updates are correct and complete. However, the PR's central justification — "/deep-research repointed to write reports to the vault, never the repo, so the folder can't regrow" — is false: .claude/skills/deep-research/SKILL.md Phase 6 is untouched by this diff and still instructs writing reports to .claude/research/<kebab-name>.md, which will silently regrow the exact clutter this PR removes the next time that skill's offered "Report" step is accepted.
Findings
Critical
None.
High
1. /deep-research's Report step was never repointed to the vault — the PR's stated fix is not actually made, and .claude/research/ will regrow.
- File:
.claude/skills/deep-research/SKILL.md:106 - Also contradicted by the PR's own new prose:
CLAUDE.md:81("/deep-researchwrites its reports there [the vault], never here [the repo].") - Defect: confirmed via
git show HEAD --statthat.claude/skills/deep-research/SKILL.mdhas zero changes in this PR. Its Phase 6 — Capture still reads:**Report** — write the full findings to.claude/research/.md(mkdir -p .claude/research)..claude/research/is not gitignored, so a file written there is trackable and can be re-committed on a future PR — reintroducing the same stale-dossier problem this PR exists to solve, with no CI/link-check gate to catch the recurrence. - Failure scenario: a future
/deep-researchrun (by a human or an agent) accepts the offered "Report" capture step, as it's designed to do;.claude/research/some-topic.mdis written and later committed, silently undoing the cleanup and leavingCLAUDE.md's new claim ("never here") false in the repo it lives in. - Fix: edit
.claude/skills/deep-research/SKILL.mdPhase 6's "Report" bullet to target the brain vault path (matching the pattern already used for the "Memory" bullet immediately above it), consistent with what the PR body and the newCLAUDE.mdsentence both already claim was done. - Independently re-verified this session (not just relayed from the skill's subagent pass): re-ran
git show HEAD --statto confirm the file is absent from the changed-files list, and read the live file content at the PR head directly — confirmed.
Medium
None.
Low / Info
None (one trivial vault-path formatting inconsistency between .claude/skills/lesson/SKILL.md:15 and CLAUDE.md:81 — 2 Areas/... vs brain/2 Areas/... — was found but dropped per the rubric's signal gate; it's not concretely actionable since neither path is tool-read).
Subagents
| Agent | Verdict |
|---|---|
| parity-checker | N/A — no apps/web/** or apps/mobile/** file changed |
| i18n-syncer | N/A — no user-facing strings or packages/shared/src/i18n/*.json changed |
| contract-aligner | N/A — no packages/shared/src/types/* / endpoints.ts / orbit-api DTOs changed |
| security-reviewer | N/A — no orbit-api code changed |
| design-reviewer | N/A — no UI files changed |
Validation
| Check | Result |
|---|---|
| Lint | N/A — diff touches only .claude/** and root CLAUDE.md (markdown/prose), no lint-covered source |
| Type check | N/A — no TS/C# source changed |
| Tests | N/A — no source changed |
| Build (api) | N/A — orbit-api untouched |
Deferred — N/A dimensions & files not verdicted
- Dimensions #8 (DESIGN.md/AI-slop), #9 (Parity), #10 (i18n), #11 (Contract drift/backward-compat), #12 (Security), #13 (Backend hard rules), #14 (FEATURES.md parity): all N/A — their gating surfaces (
apps/*UI, i18n JSON, shared types/DTOs,orbit-api, user-facing feature surface) are untouched by this diff. - All 17 changed files (10 deletions, 6 modifications, 1 root doc edit) were given a verdict; nothing was skipped.
- Not independently verified in this session: the PR body's claim that the deleted
.claude/PRDs/*and.claude/stories/*files correspond to CLOSED/shipped issues #396 and #397, and that the deletedprod-readiness-deferred.mdwas "refiled as #553." Low-impact even if imprecise — deleting stub planning docs for already-superseded features has no functional blast radius, unlike the deep-research finding above. - The "harvested to the brain vault as ADRs" claims (guardrails-over-memory, autonomy-within-a-phase, refutation-over-voting, OpenCode Go/Zen route) point outside the repo (a personal vault) and are structurally unverifiable from this checkout.
What's good
- Three of the four skill-file reference updates (
lesson/SKILL.md,pr-review/SKILL.md,second-opinion/SKILL.md) correctly replace theirresearch.mdcitations with inline, self-contained rationale — exactly the stated goal of making these skills CI-safe (no vault access at runtime). pending-lessons.md's graduation entry is internally consistent (dated 2026-07-16, matching today) and correctly removes the promoted lesson from the pending section.- No dangling references remain to the deleted PRDs, stories, or the deferred-audit file anywhere in the tree.
- The
CLAUDE.mddocs-registry table edit is clean — the two removed rows leave a well-formed table with no formatting breakage.
Recommendation
Fix the one High finding before merge: update .claude/skills/deep-research/SKILL.md Phase 6's "Report" bullet to point at the brain vault (mirroring the "Memory" bullet's pattern) instead of .claude/research/<kebab-name>.md. That's the only gap between what this PR claims to have done and what it actually did — once closed, the rest of the change is clean, well-scoped, and does what it says.
…ose the review gap) Review correctly flagged that CLAUDE.md claimed "/deep-research writes reports to the vault, never here" while the skill's Phase 6 still wrote to `.claude/research/`, so the folder could regrow. Lands only that Report-bullet hunk; the unrelated web-researcher edits in the same file belong to a separate in-flight change and are left unstaged. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
|
There was a problem hiding this comment.
Review Complete
Scope: PR #554 — chore(harness): delete stale AI research files, harvest knowledge to the brain vault (thomasluizon/orbit-ui-mobile, chore/ai-files-cleanup → main)
Recommendation: APPROVE
| Severity | Count |
|---|---|
| Critical (incl. |
0 |
| High | 0 |
| Medium | 0 |
| Low / Info | 2 |
What the diff is
Pure harness/docs chore, 18 files: deletes research.md, 6 .claude/research/* dossiers, 2 completed .claude/PRDs/* + their .claude/stories/* mirrors (shipped issues #396/#397), and an orphaned .claude/audits/prod-readiness-deferred.md; edits CLAUDE.md, .claude/pending-lessons.md, and 4 SKILL.md files (deep-research, lesson, pr-review, second-opinion) to stop citing the deleted paths. No apps/*, packages/*, or orbit-api file touched.
Verification performed
- Confirmed via
git diffbetween the merge-base (637a064d) and the PR head (18348d1d) that this matchesgh's reported file list exactly (18 files, 1769 deletions / 17 insertions). - Repo-wide
git grepon the post-PR tree found zero dangling references to any deleted path outside the intentional explanatory mentions added inCLAUDE.md:81and the four SKILL.md files. - Spot-checked the deleted research files for secrets (API keys, tokens, private keys) — none found; the one "secret leak" mention inside
research.mdwas itself just research prose about a different project's Slack-token issue, not a live credential. - Confirmed the four rewritten skills are genuinely self-contained now:
lesson,second-opinion, andpr-reviewinline the actual reasoning that used to live only inresearch.md(adherence-decay stats, refuter-vs-voting rationale, on-demand-diversity budget) rather than swapping one dangling citation for another. - All 5 subagent gates (parity-checker, i18n-syncer, contract-aligner, security-reviewer, design-reviewer) and the backward-compat guard are N/A — no
apps/*,packages/shared, ororbit-apisurface in the diff. - Validation (Phase 7 / /validate) skipped per CI adaptation — no lint/type/test-gated surface touched, and this PR's Build/Unit Tests/SonarCloud run as separate required checks.
Findings
Critical / High / Medium: None.
Low / Info:
- The rewritten SKILL.md sections correctly inline the previously-dangling reasoning rather than re-pointing at another repo file — the right fix for skills that must stay self-contained and run in CI.
.claude/pending-lessons.md's "opencode + Zen" entry move to## Graduatedfollows the file's own existing convention.
Deferred
- Dimensions 3, 4, 6–14 of the rubric (SOLID, comment policy, type safety, console.log, DESIGN.md, parity, i18n, contract drift, backend hard rules, FEATURES.md) — all N/A, diff has zero code surface.
- The PR body's claim that
prod-readiness-deferred.mdwas "orphaned by #243's closure → refiled as #553" was not independently verified via GitHub issue lookup. The deletion itself is safe regardless — zero remaining references confirmed. - Cross-repo dimensions (contract-aligner, orbit-api side of backward-compat guard): not verifiable in CI (orbit-api not checked out), and not applicable here regardless since no orbit-api or contract surface is touched.
1 participant
Deletes in-repo research/planning cruft that goes stale silently and gets cited as authority long after its facts expire.
Removed
research.md+ all 6.claude/research/*dossiers.claude/PRDs/*+.claude/stories/*for Onboarding before auth: pre-auth flow with signup as the final "save your plan" step #396 / Marketing email consent + Resend audience sync (enable product-update broadcasts) #397 (both CLOSED and shipped).claude/audits/prod-readiness-deferred.md(orphaned by Final pre-launch: SonarCloud full-surface → zero + loop /prod-readiness to clean (both repos) #243's closure → refiled as Mobile boot perf: the #243 accepted-deferred findings need a device-testable session (splash gated on fonts + on token refresh) #553)Preserved, not lost
research.md(pr-review,second-opinion,lesson) are now self-contained — they run in CI, which can't read the vault./deep-researchrepointed to write reports to the vault, never the repo, so the folder can't regrow.Also cleaned ~3.5 MB of gitignored local scratch (night-run/, dead 461 plans, duplicate audits) — not in this diff since it was never tracked.
Repo holds machines; vault holds knowledge.
🤖 Generated with Claude Code