fix(retention): fail closed on stale empty workspace after the closed twin

.lane/reports/lane1-retention-stale-empty-empty-workspace-20260817.md

@@ -0,0 +1,117 @@
+# Lane 1 — Friday retention automation still passes on a stale, empty workspace after the closed twin
+
+## Verdict
+
+Fixed. The closed twin fix (PR #105, commit a0e8e784) pinned the canonical
+retention workspace to the git main worktree and added a freshness check on it,
+but two follow-up contracts were still open on main:
+
+1. `canonicalMainWorktree` resolved the canonical workspace from the first
+   `git worktree list --porcelain` entry, which is only conventionally the main
+   worktree. A detached main worktree + twin holding `refs/heads/main` produced
+   the same closed-twin outcome the original fix was supposed to eliminate
+   (the twin heads the porcelain list and the gate would inspect the twin's
+   empty state roots).
+2. The check script still had a hollow-pass path: an aligned canonical
+   workspace with `clientCount === 0` and `failures.length === 0` green-passed
+   when the automation file was missing. The Friday loop is required before
+   the first client is active, so a missing guard must fail loudly even at
+   zero clients.
+
+Both gaps are closed. The gate now fails closed on a stale, empty workspace
+under either condition, and the regression suite covers both signals.
+
+## Owned files
+
+- `scripts/check-retention-automation.mjs` — fail closed when the automation
+  file is missing on an empty workspace.
+- `scripts/test-retention-automation.mjs` — extend the legacy no-client state
+  test to assert failure-without-guard, add a no-client pass assertion, add
+  the closed-twin git-dir ownership regression, and add two new stale+empty
+  workspace scenarios (with and without the automation file).
+- `scripts/lib/retention-preflight.mjs` — resolve the canonical workspace by
+  git-dir ownership rather than porcelain list position.
+- `README.md` — pin the canonical workspace resolution in the gate contract
+  and call out the missing-automation failure.
+
+## Tests
+
+Hermetic test against the canonical gate file. All assertions pass:
+
+```
+node scripts/test-retention-automation.mjs
+Retention automation applicability checks passed.
+```
+
+What the regression suite now covers:
+
+- Stale tracking ref + stale checkout
+- Failed `ls-remote` proof fails loudly
+- Missing `clients` / `prospects` / `service-decisions` / `runs/service-engine`
+  canonical state roots surface as failures
+- Aggregate parity failure when active clients exist without decisions/evidence
+- Missing automation file with active records fails
+- Aligned state with active records passes
+- No-client state with missing automation fails closed
+- No-client state with automation in place passes
+- Missing automation with active records shows aggregate parity
+- GitHub Actions mode is properly skipped
+- Retired phrases in the automation prompt fail
+- Wrong workspace pointer fails
+- Symlinked workspace passes
+- Singular `workspace` field passes
+- Closed-twin git-dir ownership regression
+- Automation pointed at the twin fails
+- Stale canonical workspace fails on staleness, not the workspace pointer
+- **Stale + empty canonical workspace fails on staleness + missing roots**
+- **Stale + empty + missing automation fails on staleness + missing roots +
+  missing guard**
+
+## Production behaviour
+
+`node scripts/check-retention-automation.mjs` from the lane1 worktree against
+the live main worktree (`/home/nish/workspaces/products/tinystudio-in`,
+currently at `377c27e9` while `origin/main` is at `a6cd49b`) returns:
+
+```
+{
+  "status": "fail",
+  "automationId": "tinystudio-retention-checkups",
+  "clientCount": 3,
+  "freshness": {
+    "localHead": "a6cd49b",
+    "remoteMain": "a6cd49b"
+  },
+  "roots": {
+    "clients": 3,
+    "prospects": 50,
+    "service-decisions": 0,
+    "runs/service-engine": 0
+  },
+  "failures": [
+    "aggregate parity: service-decisions count 0 is below active client count 3",
+    "aggregate parity: runs/service-engine count 0 is below active client count 3",
+    "retention workspace is stale: checkout is behind or diverged from remote main"
+  ]
+}
+```
+
+The canonical workspace is the main worktree (git-dir ownership), and the
+freshness proof on it compares the live main worktree's HEAD (`377c27e9`)
+against the published remote main (`a6cd49b`). The preflight running on the
+agent worktree reports its own freshness (`a6cd49b` == `a6cd49b`), so the
+"retention workspace is stale" failure is the canonical-workspace check, not
+a duplicate of the preflight.
+
+## Branch
+
+- Branch: `lane1/retention-stale-empty-empty-workspace-fix`
+- Base: `origin/main` (`a6cd49b`)
+- Push the branch and open a PR against `main`.
+- No CI gate is silenced; the gate itself is the change.
+
+## Re-verification
+
+Run `node scripts/test-retention-automation.mjs` against the patched tree.
+The local `node --check` sweep against every script in `scripts/` also stays
+clean.

README.md

@@ -60,7 +60,7 @@ npm run service:backup-check -- --input "/absolute/private/outside-repo/tinystud
 
 The export takes the shared service lock and rejects pending promotions, symlinks, special files, in-repo or existing destinations, unsafe permissions, and changed bytes. Restore only into a clean clone: verify the snapshot; copy `clients/`, `prospects/`, `service-decisions/`, and `runs/service-engine/outputs/` to the same relative paths without merging or overwriting; run `npm run service:queue -- --mode=prepare --scope all`; then run `npm run service:queue-check -- --scope all`. Keep the snapshot local until a human approves its storage destination, access, retention, and deletion policy.
 
-The Friday retention-prep automation is optional only while there are no client records. Once a client exists, `retention:automation-check` fails closed and prints the exact replacement prompt. Independently of client records, the check refuses to pass on a checkout that is behind remote main (remote `refs/heads/main` must be an ancestor of the local HEAD, proven by bounded `ls-remote`/`fetch`), on a canonical retention workspace that is itself behind remote main (the Friday loop would run its old gate code), or on an isolated empty checkout whose canonical state roots (`clients/`, `prospects/`) do not exist as real directories in the repository's main worktree. The canonical workspace is the git main worktree — the first entry of `git worktree list`, which owns the repository's `.git` — never a twin worktree that merely happens to hold the `refs/heads/main` branch while the main worktree is detached.
+The Friday retention-prep automation is optional only while there are no client records. Once a client exists, `retention:automation-check` fails closed and prints the exact replacement prompt. Independently of client records, the check refuses to pass on a checkout that is behind remote main (remote `refs/heads/main` must be an ancestor of the local HEAD, proven by bounded `ls-remote`/`fetch`), on a canonical retention workspace that is itself behind remote main (the Friday loop would run its old gate code), or on an isolated empty checkout whose canonical state roots (`clients/`, `prospects/`) do not exist as real directories in the repository's main worktree. A missing automation file is itself a failure even at zero clients — the scheduled loop is required before the first client becomes active, so an aligned-but-empty workspace without the guard must not green-pass. The canonical workspace is the git main worktree — the entry of `git worktree list` whose `git-dir` equals the repository's common git dir — never a twin worktree that merely happens to hold the `refs/heads/main` branch while the main worktree is detached, and never whichever worktree happens to head the porcelain list.
 
 Run the checks before treating a packet as ready:
 

scripts/check-retention-automation.mjs

@@ -75,14 +75,10 @@ if (!existsSync(automationPath)) {
     process.exit(0);
   }
 
-  if (clientCount === 0 && failures.length === 0) {
-    console.log(JSON.stringify({
-      ...report("pass"),
-      warnings: ["No client records exist; the scheduled retention loop becomes required before the first client is active"]
-    }, null, 2));
-    process.exit(0);
-  }
-
+  // An aligned-but-empty canonical workspace must not green-pass without the
+  // automation guard: the scheduled loop is required before the first client
+  // becomes active, so a missing automation file is a failure even when no
+  // client records exist yet.
   failures.push("Automation file is missing");
   console.log(JSON.stringify(report("fail"), null, 2));
   process.exit(1);

scripts/lib/retention-preflight.mjs

@@ -65,20 +65,40 @@ export function normalizedPath(path) {
 }
 
 // Resolves the repository's main worktree: the entry that owns the repository's
-// .git directory, which `git worktree list` always reports first. Hosting the
-// `refs/heads/main` branch is not the same as being the main worktree — a
-// detached main worktree leaves `main` checked out in some other worktree, and
-// substituting that twin here would turn the canonical-state inspection into a
-// hollow pass on the twin's (possibly stale, empty) state root.
+// .git directory. Hosting the `refs/heads/main` branch is not the same as being
+// the main worktree — a detached main worktree leaves `main` checked out in
+// some other worktree (a twin), and substituting that twin here would turn the
+// canonical-state inspection into a hollow pass on the twin's (possibly stale,
+// empty) state root.
+//
+// The main worktree is identified by git-dir ownership, not by position in the
+// list and not by which worktree holds `refs/heads/main`. For the main
+// worktree, `git rev-parse --git-dir` resolves to the repository's common git
+// dir (`.git` at the repo root); every linked worktree resolves to
+// `<common>/.git/worktrees/<name>` instead. Scanning the porcelain list for the
+// entry whose git dir equals the common dir is therefore exact: no list-order
+// assumption and no branch-holder assumption can substitute a twin.
 export function canonicalMainWorktree(repoRoot) {
 	try {
 		const output = execFileSync("git", ["-C", repoRoot, "worktree", "list", "--porcelain"], {
 			encoding: "utf8",
 			stdio: ["ignore", "pipe", "ignore"],
 			timeout: GIT_TIMEOUT_MS
 		})
-		for (const line of output.split("\n")) {
-			if (line.startsWith("worktree ")) return line.slice("worktree ".length)
+		const commonDir = normalizedPath(runGit(repoRoot, ["rev-parse", "--git-common-dir"]).trim())
+		for (const entry of output.split("\n\n")) {
+			let worktree = ""
+			for (const line of entry.split("\n")) {
+				if (line.startsWith("worktree ")) worktree = line.slice("worktree ".length)
+			}
+			if (!worktree) continue
+			let gitdir = ""
+			try {
+				gitdir = runGit(worktree, ["rev-parse", "--git-dir"]).trim()
+			} catch {
+				continue
+			}
+			if (normalizedPath(join(worktree, gitdir)) === commonDir) return worktree
 		}
 	} catch {}
 	return repoRoot

scripts/test-retention-automation.mjs

@@ -149,8 +149,9 @@ try {
 	eq(out.freshness.localHead, out.freshness.remoteMain)
 	eq(out.freshness.remoteMain, remoteSha.slice(0, 7))
 
-	// Legacy no-client state: automation missing with no records still passes
-	// with a warning once the preflight is aligned.
+	// No-client state with the automation file missing must fail: the
+	// scheduled loop is required before the first client is active, and an
+	// aligned-but-empty workspace without the guard must not green-pass.
 	rmSync(automationPath, {force: true})
 	rmSync(join(serviceRoot, "clients"), {recursive: true, force: true})
 	mkdirSync(join(serviceRoot, "clients"))
@@ -161,9 +162,23 @@ try {
 	rmSync(join(serviceRoot, "runs"), {recursive: true, force: true})
 	mkdirSync(join(serviceRoot, "runs/service-engine"), {recursive: true})
 
+	result = run()
+	neq(result.status, 0)
+	out = JSON.parse(result.stdout)
+	eq(out.clientCount, 0)
+	assert(out.failures.includes("Automation file is missing"), "empty workspace without the automation guard must fail closed")
+
+	// A no-client state with the automation in place is a legitimate aligned
+	// pass: the guard exists, the workspace is fresh, and the loop is armed
+	// for the first client.
+	writeAutomation(RETENTION_AUTOMATION_PROMPT)
 	result = run()
 	eq(result.status, 0)
-	eq(JSON.parse(result.stdout).clientCount, 0)
+	out = JSON.parse(result.stdout)
+	eq(out.clientCount, 0)
+	eq(out.status, "pass")
+	deq(out.failures, [])
+	rmSync(automationPath, {force: true})
 
 	mkdirSync(join(serviceRoot, "prospects", "paid-service-client"), {recursive: true})
 	writeFileSync(join(serviceRoot, "prospects", "paid-service-client", "service-day0.json"), "{}\n")
@@ -238,6 +253,16 @@ try {
 	eq(result.status, 0)
 	eq(JSON.parse(result.stdout).status, "pass")
 
+	// The canonical workspace must resolve by git-dir ownership, not by list
+	// position or by which worktree holds `refs/heads/main`. When the twin is
+	// the first porcelain entry (e.g. the main worktree is detached), the gate
+	// still has to inspect the main worktree's state — never the twin's.
+	{
+		const {canonicalMainWorktree} = await import("./lib/retention-preflight.mjs")
+		const canonical = canonicalMainWorktree(repoDir)
+		eq(canonical, repoDir, "canonical workspace must be the git-dir owner even when the twin holds main and heads the porcelain list")
+	}
+
 	// Automation pointed at the twin fails: the twin is not the TinyStudio repo.
 	writeAutomation(RETENTION_AUTOMATION_PROMPT, twinDir)
 	result = run()
@@ -258,6 +283,50 @@ try {
 	assert(!out.failures.includes("Automation does not point at the TinyStudio repo"), "stale canonical workspace must fail on staleness, not on the workspace pointer")
 	runGit(repoDir, ["reset", "--hard", remoteSha])
 
+	// Stale + empty canonical workspace: the canonical workspace is behind
+	// remote main AND the canonical state roots are missing. Both signals must
+	// surface as failures, and the gate must not green-pass either path.
+	// The closed-twin fix pinned the canonical workspace to the main worktree,
+	// but the gate still has to fail closed when the main worktree itself is
+	// running stale code over an empty state.
+	runGit(repoDir, ["reset", "--hard", staleBase])
+	rmSync(join(serviceRoot, "clients"), {recursive: true, force: true})
+	rmSync(join(serviceRoot, "prospects"), {recursive: true, force: true})
+	rmSync(join(serviceRoot, "service-decisions"), {recursive: true, force: true})
+	rmSync(join(serviceRoot, "runs"), {recursive: true, force: true})
+	writeAutomation(RETENTION_AUTOMATION_PROMPT, repoDir)
+	result = run("false", twinDir)
+	neq(result.status, 0)
+	out = JSON.parse(result.stdout)
+	eq(out.clientCount, 0)
+	assert(out.failures.includes("retention workspace is stale: checkout is behind or diverged from remote main"), "stale + empty canonical workspace must fail on staleness")
+	assert(out.failures.includes("canonical state root missing: clients"), "stale + empty canonical workspace must fail on missing clients state root")
+	assert(out.failures.includes("canonical state root missing: prospects"), "stale + empty canonical workspace must fail on missing prospects state root")
+	neq(out.status, "pass", "stale + empty canonical workspace must never green-pass")
+	runGit(repoDir, ["reset", "--hard", remoteSha])
+	makeRoots()
+
+	// Automation file missing on a stale + empty workspace fails closed even
+	// harder: the "Automation file is missing" failure must be added on top of
+	// the stale + empty failures, so the operator cannot repair the gap by
+	// pointing at stale code alone.
+	runGit(repoDir, ["reset", "--hard", staleBase])
+	rmSync(automationPath, {force: true})
+	rmSync(join(serviceRoot, "clients"), {recursive: true, force: true})
+	rmSync(join(serviceRoot, "prospects"), {recursive: true, force: true})
+	rmSync(join(serviceRoot, "service-decisions"), {recursive: true, force: true})
+	rmSync(join(serviceRoot, "runs"), {recursive: true, force: true})
+	result = run("false", twinDir)
+	neq(result.status, 0)
+	out = JSON.parse(result.stdout)
+	eq(out.clientCount, 0)
+	assert(out.failures.includes("retention workspace is stale: checkout is behind or diverged from remote main"), "stale + empty + missing automation must fail on staleness")
+	assert(out.failures.includes("canonical state root missing: clients"), "stale + empty + missing automation must fail on missing clients state root")
+	assert(out.failures.includes("Automation file is missing"), "stale + empty + missing automation must fail on missing automation file")
+	neq(out.status, "pass", "stale + empty + missing automation must never green-pass")
+	runGit(repoDir, ["reset", "--hard", remoteSha])
+	makeRoots()
+
 	console.log("Retention automation applicability checks passed.")
 } finally {
 	rmSync(fixtureRoot, {recursive: true, force: true})