determinize: memory-reference-existence CI lint + FACTORY-HYGIENE #59 (Amara Determinize 2/5)

.github/workflows/memory-reference-existence-lint.yml

@@ -0,0 +1,58 @@
+name: memory-reference-existence-lint
+
+# Every `](foo.md)` link target in memory/MEMORY.md MUST
+# resolve to an actual file under memory/. Amara's 4th-ferry
+# (PR #221 absorb) Determinize-stage item: prevent the
+# retrieval-drift class where prose cites paths that don't
+# exist.
+#
+# Third leg of memory-index hygiene:
+#   1. PR #220 — every memory/*.md change updates MEMORY.md
+#   2. AceHack #12 — MEMORY.md has no duplicate link targets
+#   3. THIS workflow — every MEMORY.md link target resolves
+#      to a real file
+#
+# Safe-pattern compliance (per FACTORY-HYGIENE row #43):
+#   - SHA-pinned actions/checkout
+#   - Explicit minimum permissions
+#   - No user-authored context; purely repo-file reads
+#   - Concurrency group + pinned runs-on
+#
+# See:
+#   - tools/hygiene/audit-memory-references.sh (the tool)
+#   - docs/aurora/2026-04-23-amara-memory-drift-alignment-
+#     claude-to-memories-drift.md (Amara ferry)
+
+on:
+  pull_request:
+    paths:
+      - "memory/**"
+      - "tools/hygiene/audit-memory-references.sh"
+      - ".github/workflows/memory-reference-existence-lint.yml"
+  push:
+    branches: [main]
+    paths:
+      - "memory/**"
+      - "tools/hygiene/audit-memory-references.sh"
+      - ".github/workflows/memory-reference-existence-lint.yml"
+  workflow_dispatch: {}
+
+permissions:
+  contents: read
+
+concurrency:
+  group: memory-reference-existence-lint-${{ github.ref }}
+  cancel-in-progress: false
+
+jobs:
+  lint:
+    name: lint memory/MEMORY.md reference-existence
+    runs-on: ubuntu-22.04
+    steps:
+      - uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
+
+      - name: run reference-existence lint
+        shell: bash
+        run: |
+          set -euo pipefail
+          tools/hygiene/audit-memory-references.sh --enforce

docs/FACTORY-HYGIENE.md

@@ -96,6 +96,7 @@ is never destructive; retiring one requires an ADR in
 | 51 | Cross-platform parity audit (bash / PowerShell / bun+TS twin check across macOS / Windows / Linux / WSL) | Detect-only now (landed 2026-04-22); cadenced detection every 5-10 rounds (same cadence as row #46); opportunistic on-touch every time an agent adds or edits a script under `tools/`. Enforcement deferred until baseline is green AND CI matrix runs `--enforce` on `macos-latest` / `windows-latest` / `ubuntu-latest` (WSL inherits ubuntu-latest for CI). | Dejan (devops-engineer) on cadenced detection; author of the script (self-check at author-time against the rule classes in the audit's decision-record header block). Kenji (Architect) on CI-matrix-enforcement sign-off when baseline is green. | both | `tools/hygiene/audit-cross-platform-parity.sh` classifies every script under `tools/` by rule class: (a) **pre-setup** (`tools/setup/**`) — both `.sh` AND `.ps1` required per Q1 dual-authoring rule (`memory/feedback_preinstall_scripts_forced_shell_meet_developer_where_they_live`); (b) **post-setup permanent-bash** (`thin wrapper over existing CLI` / `trivial find-xargs pipeline` / `stay bash forever`) — `.ps1` twin required per the Windows-twin obligation (`memory/feedback_stay_bash_forever_implies_powershell_twin_obligation.md`); (c) **post-setup transitional** (`bun+TS migration candidate` / `bash scaffolding`) — no twin obligation (long-term plan is one cross-platform bun+TS script); (d) **post-setup bun+TS** (`*.ts` under `tools/`) — no twin needed (cross-platform native via bun). `--summary` prints counts; `--enforce` flips exit 2 on gaps. **Why detect-only first:** baseline at first fire (2026-04-22) was 13 gaps — 12 pre-setup bash without `.ps1` twin (Q1 violation silently accumulating since `tools/setup/` existed) + 1 post-setup permanent-bash (`tools/profile.sh`) without `.ps1` twin. Turning enforcement on before triage would block every CI run. **Why this row exists:** Aaron 2026-04-22 *"missing mac/windows/linux/wsl parity (ubuntu latest) we can deffer but should have the hygene in place for when we want to enforce and it will be more obvious to you in the future that we are cross platform."* Cross-platform-first must be a *visible* factory property (audit exists, runs, prints the gap) before it becomes an enforced gate. Same pattern as FACTORY-HYGIENE rows #23 / #43 / #47. See `memory/feedback_cross_platform_parity_hygiene_deferred_enforcement.md`. **Classification (row #47):** **prevention-bearing** — the audit runs at author-time (opportunistic on-touch) and surfaces the gap before it lands, same as row #46. The audit itself is a detect-only mechanism but detect-only surfaces the obligation at author-time when the author runs it. Ships to project-under-construction: adopters inherit the parity audit + the decision-record-block pattern + the CI-matrix obligation once it's wired. | Audit output in repo root on each fire; cadenced runs appended to `docs/hygiene-history/cross-platform-parity-history.md` (per-fire schema per row #44); BACKLOG row per gap at triage time; ROUND-HISTORY row when a gap resolves. | `tools/hygiene/audit-cross-platform-parity.sh` (detection + decision-record header block) + `memory/feedback_cross_platform_parity_hygiene_deferred_enforcement.md` + `memory/feedback_stay_bash_forever_implies_powershell_twin_obligation.md` + `memory/feedback_preinstall_scripts_forced_shell_meet_developer_where_they_live` + `docs/POST-SETUP-SCRIPT-STACK.md` |
 | 54 | Backlog-refactor cadenced audit (overlap / staleness / priority-drift / knowledge-update sweep of `docs/BACKLOG.md`) | Cadenced detection every 5-10 rounds (same cadence as rows #5 / #23 / #38 / #46 meta-audits) + opportunistic on-touch when a tick adds a new BACKLOG row and the author notices adjacent rows that may overlap. Not exhaustive; bounded passes per firing are acceptable. | Architect (Kenji) on round-cadence sweeps; `backlog-scrum-master` skill if explicitly invoked; all agents (self-administered) on on-touch overlap-spot during authoring. | factory | Read `docs/BACKLOG.md` (or a scoped slice — P0/P1 first if full scan is too large) and apply the following passes: (a) **overlap cluster** — two or more rows describing the same concern from different angles get flagged; decide merge (single consolidated row) or sharpen (two rows with clear non-overlap scope boundaries); (b) **stale retire** — rows where context has died, implementation landed without retire-action, or assumption has been falsified by newer knowledge get explicitly retired with a "retired: <reason>" marker (not silent deletion — signal-preservation still applies); (c) **re-prioritize** — priority labels (P0/P1/P2/P3) re-examined against current knowledge; any row whose priority feels wrong after re-read gets a justified move with a one-line rationale; (d) **knowledge absorb** — rows written before a newer architectural insight landed get rewording / cross-refs to the new substrate (e.g., rows predating AutoDream cadence now cite the policy; rows predating scheduling-authority sharpening now note self-schedulability); (e) **document** — ROUND-HISTORY row per fire with pre-audit and post-audit row counts + what was merged / retired / re-prioritized / updated. **Why this row exists:** the human maintainer 2026-04-23 *"we probalby need some meta iteam to refactor the backlog base on current knowledge and look for overlap, this is hygene we could run from time to time so our backlog is not just a dump"*. The BACKLOG is the triage substrate for every future tick's "what to pick up" decision; without periodic meta-audit it becomes an append-only log rather than a living triage surface. **Classification (row #50):** **detection-only-justified** — accumulated drift (overlap, staleness, priority-drift, knowledge-update-gap) is inherently post-hoc; no author-time check can prevent rows from becoming overlapping with *future* rows not yet written. **Maintainer-scope boundary:** rows with explicit maintainer framing at their priority (e.g., P0 rows the human maintainer explicitly set) stay at that priority; re-prioritization applies within the agent-owned priority space only. Ships to project-under-construction: adopters inherit the cadenced-sweep discipline + the retire-with-marker convention + the ROUND-HISTORY documentation pattern. | ROUND-HISTORY row per fire with pre/post row counts + merged/retired/re-prioritized/updated actions; `docs/hygiene-history/backlog-refactor-history.md` (per-fire schema per row #44 — date, agent, rows touched, actions taken, pre/post counts, next-fire-expected-date). | `docs/BACKLOG.md` (target surface) + governing rule in per-user memory (not in-repo; lives at `~/.claude/projects/<slug>/memory/feedback_backlog_hygiene_cadenced_refactor_look_for_overlap_not_just_dump_2026_04_23.md`) + `.claude/skills/backlog-scrum-master/SKILL.md` (dedicated runner when invoked) + `.claude/skills/reducer/SKILL.md` (Rodney's Razor applied at backlog level) + sibling meta-audit rows #5, #23, #38, #46, #50 |
 | 52 | Tick-history bounded-growth audit (`docs/hygiene-history/loop-tick-history.md` line-count vs threshold) | Detect-only (landed 2026-04-22); cadenced detection once per round-close (same cadence as row #44 cadence-history sweep, since this is the canonical row #44 worked example auditing itself); opportunistic on-touch whenever the tick-history file is read or edited. Archive action itself remains manual for now; deferring automation to the larger BACKLOG row that also covers threshold-revision and append-without-reading refactor. | Dejan (devops-engineer) on cadenced detection; the tick itself (self-administered at tick-close) on the opportunistic on-touch — each tick's end-of-tick sequence can invoke this audit after the append + commit to get a `within bounds: 96/500 lines` visibility signal. | factory | `tools/hygiene/audit-tick-history-bounded-growth.sh` checks the file's line count against a threshold (default 500, overrideable via `--threshold N`) and exits 0 within bounds / 2 over threshold. The threshold is set lower than the stated 5000-line paper bound because the file is read on every tick-close append — a per-tick context cost that scales linearly with file size — and 5000 lines represents too large a context hit on a 1-minute cadence. The audit's header block carries a mini-ADR decision record for the 500-line choice (context / decision / alternatives / supersedes / expires-when). **Why this row exists:** Aaron 2026-04-22 tick-fire interrupt: *"does loop tick history grow unbounded? that's an issue if so you just read it"*. Honest state was stated-bound-no-enforcement: file header named 5000 lines, nothing checked it. This row closes the enforcement gap for the threshold-check half of the full BACKLOG row (archive-action + append-without-reading refactor remain deferred). **Self-referential closure:** the tick-history file IS the canonical row-#44 cadence-history-tracking worked example (named explicitly in row #44's "Durable output" citation). Until this row landed, the most-cadenced surface in the factory — the tick itself — had its fire-log surface unaudited for its own growth. Meta-audit triangle remains intact (existence #23 / activation #43 / fire-history #44), and row #49 adds a fourth: fire-history files themselves need bounded-growth audits because they grow at the cadence of the surface they track. **Classification (row #47):** **prevention-bearing** — the audit surfaces approaching-threshold warnings at 80% so the archive action can be planned, rather than reactive-only at over-threshold. Ships to project-under-construction indirectly: adopters inherit the pattern (fire-log files under their own `docs/hygiene-history/` need the same bounded-growth treatment), not this exact script. | Audit output on each fire; cadenced runs appended to `docs/hygiene-history/tick-history-bounded-growth-history.md` (per-fire schema per row #44); BACKLOG row when archival is due (archive-action itself queued as part of the larger tick-history enforcement BACKLOG row); ROUND-HISTORY row when threshold changes or archive action executes. | `tools/hygiene/audit-tick-history-bounded-growth.sh` (detection + mini-ADR header block) + `docs/hygiene-history/loop-tick-history.md` (target surface, canonical row #44 worked example) + BACKLOG row *"Loop-tick-history bounded-growth enforcement"* (larger follow-up: threshold revision + append-without-reading refactor + archive action) |
+| 59 | Memory-reference-existence CI check (every `](foo.md)` link target in `memory/MEMORY.md` MUST resolve to an actual file under `memory/`) | Every pull_request + push-to-main touching `memory/**` or the audit tool / workflow; workflow-dispatch manual run available | Automated (`.github/workflows/memory-reference-existence-lint.yml`); any contributor resolves on fail | factory | `tools/hygiene/audit-memory-references.sh --enforce` parses link targets of the form `](<path>.md)` in the supplied file (default `memory/MEMORY.md`), resolves each against a base dir (default `memory/`), and fails (exit 2 under `--enforce`) on any broken reference. Supports `--file PATH` and `--base DIR` for custom use. **Why this row exists:** Amara 2026-04-23 4th-ferry absorb (PR #221 Determinize-stage action) — her commit samples show repeated cleanup passes for memory paths that didn't exist; this is the retrieval-drift class she named. First-run baseline (2026-04-24): in-repo `memory/MEMORY.md` 44 refs all resolve; per-user MEMORY.md 391 refs all resolve (PR #220 memory-index-integrity CI has kept the substrate clean). **Third leg of memory-index hygiene:** row #58 (same-commit-pairing) + AceHack PR #12 (no duplicates) + this row (refs resolve) = three complementary checks. **Classification (row #47):** **prevention-bearing** — blocks merge before broken refs land. Ships to project-under-construction: adopters inherit the tool + workflow + three-leg hygiene pattern. | CI job result; first-run baseline captured in PR body. Optional fire-history file if longer-than-90-day retention wanted. | `.github/workflows/memory-reference-existence-lint.yml` + `tools/hygiene/audit-memory-references.sh` + sibling rows #58 (PR #220) + AceHack PR #12 duplicate-lint + `docs/aurora/2026-04-23-amara-memory-drift-alignment-claude-to-memories-drift.md` |
 | 58 | Memory-index-integrity CI check (PR/push that adds or modifies `memory/*.md` MUST also update `memory/MEMORY.md` in the same range) | Every pull_request + push-to-main touching `memory/**`; workflow-dispatch manual run available | Automated (`.github/workflows/memory-index-integrity.yml`); human-maintainer or any contributor resolves on fail | factory | Scope triggers: top-level `memory/*.md` add-or-modify (excluding `memory/README.md` and `memory/MEMORY.md` itself, and excluding `memory/persona/**` which has its own lifecycle). Check: if any trigger-qualifying file changed in the PR/push range, `memory/MEMORY.md` MUST also be in that range. Fail message cites NSA-001 (canonical incident: new memory landed without MEMORY.md pointer → undiscoverable from fresh session). Safe-pattern compliant per row #43 (SHA-pinned actions, explicit minimum permissions, no user-authored context interpolation, concurrency group, pinned runs-on). **Why this row exists:** Amara 2026-04-23 decision-proxy + technical review courier report (absorbed as PR #219) — action item #1 in her "10 immediate fixes" list, highest-value by her own ranking. Directly addresses the NSA-001 measured failure mode. **Classification (row #47):** **prevention-bearing** — the check runs at PR author-time, blocks merge before the memory substrate can diverge from its index. Ships to project-under-construction: adopters inherit the workflow unchanged; the `memory/**.md` and `memory/MEMORY.md` conventions are factory-generic. | CI job result + annotated fail message in PR checks + `docs/hygiene-history/memory-index-integrity-fires.md` (per-fire schema per row #44 — optional; CI log is durable for 90 days so fire-history file exists only if the human maintainer wants longer retention) | `.github/workflows/memory-index-integrity.yml` (detection + fail message) + `docs/hygiene-history/nsa-test-history.md` (NSA-001 canonical incident) + `docs/aurora/2026-04-23-amara-decision-proxy-technical-review.md` (ferry with proposal) + FACTORY-HYGIENE row #25 (pointer-integrity audit — covers dangling-pointer from the other direction) |
 | 55 | Machine-specific content scrubber (cadenced audit of in-repo tracked files for user-home paths, Claude Code harness paths, Windows user-profile paths, hostname leaks) | Detect-only (landed 2026-04-23); cadenced detection once per round-close (same cadence as rows #50 / #51 / #52 meta-audits) + opportunistic on-touch when a tick migrates per-user content to in-repo. Enforcement (`--enforce` exit-2) deferred until baseline is green. | Dejan (devops-engineer) on cadenced detection + CI-enforcement sign-off when baseline is green; the migrating agent (self-administered) on on-touch — every in-repo-first migration runs the audit before committing. | factory | `tools/hygiene/audit-machine-specific-content.sh` scans all tracked files (`git ls-files`) for machine-specific patterns: `/Users/<name>/`, `/home/<name>/`, `C:\Users\<name>`, `C:/Users/<name>`. Excludes: `docs/ROUND-HISTORY.md`, `docs/hygiene-history/**`, `docs/DECISIONS/**`, and the audit script itself. `--list` prints offending files; `--enforce` flips exit 2 on any gap. **Why this row exists:** Aaron 2026-04-23 Otto-27 — *"we can have a machine specific scrubber/lint hygene task for anyting that makes it in by default. just run on a cadence."* Following the Option D in-repo-first policy shift (per-user memory migrations to in-repo became the default), machine-specific content leakage becomes a real risk — content comfortably per-user now crosses the factory's public repo boundary. Baseline at first fire (2026-04-23) was 9 gaps: `/Users/` patterns in several SKILL.md files, 2 PDFs (metadata scan), a scratch-recon doc, a parallel-worktree research doc; `C:\Users\` pattern in 1 SKILL.md + `memory/feedback_path_hygiene.md` (anti-example reference — legitimate). **Classification (row #47):** **prevention-bearing** — the audit runs at author-time (on-touch during in-repo-first migrations) and surfaces the gap before it lands. Ships to project-under-construction: adopters inherit the audit + pattern list + exclusion-list discipline. | Audit output on each fire; cadenced runs appended to `docs/hygiene-history/machine-specific-content-audit-history.md` (per-fire schema per row #44 — date, agent, gaps count, files touched, actions taken, next-fire-expected-date); BACKLOG row per gap at triage time if cleanup doesn't fit on-touch. | `tools/hygiene/audit-machine-specific-content.sh` (detection + pattern list + exclusion list) + cross-refs: `memory/feedback_path_hygiene.md` + `memory/CURRENT-aaron.md` + `memory/CURRENT-amara.md` (in-repo-first migration boundary surfaces this audit's need) |