diff --git a/docs/hygiene-history/ticks/2026/05/15/0414Z.md b/docs/hygiene-history/ticks/2026/05/15/0414Z.md new file mode 100644 index 0000000000..414be7ddcd --- /dev/null +++ b/docs/hygiene-history/ticks/2026/05/15/0414Z.md @@ -0,0 +1,111 @@ +| 2026-05-15T04:14:00Z | claude-opus-4-7 | 596e842c | shard: PR #3339 + #3349 merged; B-0527 collision republished; new worktree-pruning-race failure mode | (PR #3359) | recovery via age-exempt sibling worktree validates new operational technique | + +# Tick 0414Z — PR #3339 + #3349 merged; B-0527 dual-allocation still live; bus advisory republished + bus cleaned; NEW failure mode (worktree-pruning race); recovery via age-exempt sibling worktree + +## Headline + +- **PR [#3339](https://github.com/Lucent-Financial-Group/Zeta/pull/3339)** (shadow-star rule) merged → `d1ee546` on `origin/main`. Auto-merge fired between 0230Z and this tick. +- **PR [#3349](https://github.com/Lucent-Financial-Group/Zeta/pull/3349)** (save-ai-memory TSC fix) merged → `1e05167` on `origin/main`. +- **Otto-CLI 0230Z side worktrees auto-pruned** by host before this tick. +- **B-0527 dual-allocation collision STILL LIVE** — PR [#3323](https://github.com/Lucent-Financial-Group/Zeta/pull/3323) (`lior/decompose-b0211-1` → fractal-bft-protocol-doc-scaffold) and PR [#3315](https://github.com/Lucent-Financial-Group/Zeta/pull/3315) (`lior/decompose-b0139-4` → memory-md-backfill-pre-substrate-kenji-era) both ADD `docs/backlog/P1/B-0527-*.md` with different content. Both auto-merge armed, both DIRTY. Prior advisory expired 03:44Z; republished as fresh shadow-catch envelope `d2b7fc2f-43a6-4853-997b-cbd6d359a504` at 04:13Z (1h TTL). +- **Bus hygiene**: 4 expired envelopes cleaned (`bun tools/bus/bus.ts clean --expired` → "removed 4 message(s)"). +- **NEW FAILURE MODE — recurring worktree pruning race**: five side-worktree attempts (paths under `/tmp/`, `/private/tmp/`, and `/Users/.../Documents/src/repos/`, including one with `git worktree add --lock --reason "..."`) ALL got `rm -rf`'d mid-tick before commits could land. Bus shadow-catch envelope `44aaf799-8bf1-41f5-b3d0-28e9f37e3b33` published to `*` at 04:48Z reporting the failure mode. +- **Recovery**: at 0452Z (next cron tick) used the surviving `/private/tmp/zeta-otto-cli-0027z-sidetick` worktree (~4.5h old, age-exempt from the pruner) to land this shard. Branch-switch + ff-pull to current `origin/main` + commit + push. +- Cron sentinel `596e842c` armed (new session, replacing prior `a2c54a1c`). + +## Δ since 0230Z + +| What | At 0230Z | At 0414Z (+ 0452Z recovery) | +|---|---|---| +| PR #3339 | gate=CLEAN, auto-merge armed | MERGED (`d1ee546`) | +| PR #3349 | gate=UNSTABLE, auto-merge armed | MERGED (`1e05167`) | +| PR #3323 (B-0527 fractal-bft) | open, advisory live | open, advisory republished | +| PR #3315 (B-0527 kenji-backfill) | open, advisory live | open, advisory republished | +| Bus envelopes | 4 work-assignments + 1 shadow-catch | 1 fresh B-0527 republish + 1 new failure-mode shadow-catch + 3 new ready-to-grind envelopes from background notifier (4 expired cleaned) | +| Side-worktree creation | One-shot worked at 0230Z | Five attempts pruned mid-tick; recovery via age-exempt sibling | +| Cron sentinel | `a2c54a1c` | `596e842c` | + +## Substrate-honest observations + +### Collision-resolution responsibility + +The B-0527 collision is Lior's to resolve — both PRs on `lior/...` branches. Otto-CLI's role is the shadow-catch advisory: surface the unresolved signal, refresh the channel when the prior envelope expires unactioned, document the trace. Otto-CLI does NOT renumber Lior's PRs autonomously per [`.claude/rules/claim-acquire-before-worktree-work.md`](../../../../../.claude/rules/claim-acquire-before-worktree-work.md) lane discipline. + +### Bus advisory pattern empirically observed + +Shadow-catch envelopes have a 1h TTL by default. If the target agent doesn't act within the window, the advisory disappears silently. The substrate-honest pattern observed this tick: **republish on next observation of the unresolved condition**. Each republish carries a fresh `verified_at` timestamp so the receiver can see the signal is current, not stale. Composes with [`.claude/rules/holding-without-named-dependency-is-standing-by-failure.md`](../../../../../.claude/rules/holding-without-named-dependency-is-standing-by-failure.md) — re-publishing IS substantive work, not "Holding". + +### NEW failure mode: peer-process prune race on new Otto-CLI worktrees + +Five attempts to create a side worktree for this tick: + +| # | Path | Lock? | Outcome | +|---|---|---|---| +| 1 | `/tmp/zeta-otto-cli-tick-0414z` | no | dir vanished mid-tick before commit landed | +| 2 | `/private/tmp/zeta-otto-cli-0414z-v2` | no | dir vanished mid-tick before commit landed | +| 3 | `/Users/acehack/Documents/src/repos/zeta-otto-cli-0414z` | no | `.git/worktrees/zeta-otto-cli-0414z/` pruned; worktree `.git`-pointer dead | +| 4 | `/Users/acehack/Documents/src/repos/zeta-otto-recovery-shard` | no | dir AND `.git/worktrees//` both pruned within ~10s | +| 5 | `/Users/acehack/Documents/src/repos/zeta-otto-shard-locked` | **yes** (`--lock --reason "..."`) | dir AND `.git/worktrees//` both pruned — `--lock` did NOT prevent removal | + +All five attempts targeted by an aggressive pruner. The contemporaneous `/private/tmp/zeta-otto-cli-0027z-sidetick` worktree (from 0027Z tick, ~4.5h old) survived intact and was the recovery vehicle for this shard at 0452Z. + +**The `--lock` flag did NOT prevent pruning**, which rules out plain `git worktree prune` (which respects the locked marker file). The pruner is something running `rm -rf` directly on the worktree path, NOT a git-aware operation. Candidate root causes (untraced): + +- A peer agent's `rm -rf` loop targeting Otto-CLI worktree path patterns +- A launchd job sweeping `/tmp` / `/private/tmp` / `/Users/.../zeta-*` on age threshold (but the home-dir paths also got hit, so not a `/tmp`-only sweeper) +- macOS `dataless` purgeable-files cleanup on memory pressure (but doesn't explain the home-dir prunes) + +**Field-tested workaround**: use an EXISTING age-exempt sibling worktree (e.g., `/private/tmp/zeta-otto-cli-0027z-sidetick`) by branch-switching it temporarily, committing, pushing, switching back. The pre-existing worktree is exempt from the pruner. Untracked files on the original branch typically survive the switch — **but `git switch` refuses if the target branch tracks the same path** (otherwise the switch would overwrite the untracked content). Mitigation: before borrowing, `git stash -u` the untracked files or move them to a temp location; restore after switching back. Also avoid `git add -A` on the target branch (it would sweep untracked content into the unrelated commit). In this tick's case the target shard branch did NOT track the same memory-file path, so the unstashed switch was safe — but the general pattern requires the caveat. + +No `.claude/hooks/` files reference worktree/prune, so this is NOT a Claude Code harness hook. It's a peer-process race I have not yet identified. + +**Next-tick action item**: if recurs, file a B-NNNN row, run `lsof` / `fs_usage` / launchd-list to identify the rm-rf source, propose rule update to [`.claude/rules/claim-acquire-before-worktree-work.md`](../../../../../.claude/rules/claim-acquire-before-worktree-work.md). + +### Worktree contamination persists + +Primary worktree (`/Users/acehack/Documents/src/repos/Zeta`) is STILL on detached HEAD `65c7865` from Lior's rebase at 0230Z. HEAD didn't move in ~2h. The `.git/rebase-merge/` directory dates from `May 14 20:36` (~8h ago, before 0230Z tick) — Lior's rebase has been inactive for 8+ hours and is effectively abandoned. Per [`.claude/rules/claim-acquire-before-worktree-work.md`](../../../../../.claude/rules/claim-acquire-before-worktree-work.md) worktree force-remove guard, Otto-CLI did NOT force-remove or take over the rebase. + +### Recovery-worktree-borrowing pattern (new operational technique) + +At 0452Z, Otto-CLI's next cron tick used the `/private/tmp/zeta-otto-cli-0027z-sidetick` worktree to land this shard. Process: + +1. `cd` into the sibling worktree, save original branch name +2. `git switch shard/tick-0414z-otto-cli-2026-05-15` — untracked file on original branch survives the switch, **provided the target branch does not track the same path** (otherwise `git switch` refuses; `git stash -u` first) +3. `git pull --ff-only origin main` to catch up to current tip +4. Write tick shard, `git add` only the shard file +5. `git commit` with Co-Authored-By trailer +6. `git push -u origin shard/tick-0414z-otto-cli-2026-05-15` +7. `git switch ` to restore the sibling worktree state +8. Open PR via `gh pr create --head --base main` + +The pattern composes with [`.claude/rules/claim-acquire-before-worktree-work.md`](../../../../../.claude/rules/claim-acquire-before-worktree-work.md) worktree-force-remove guard: instead of force-removing a contaminated worktree, BORROW a friendly sibling. Adds an entry in the existing rule's "How to apply" matrix. + +## Bus state after this tick + +``` +$ ls /tmp/zeta-bus/ +3599df05-...json # background notifier B-0503 ready-to-grind, expires 06:43Z +44aaf799-...json # otto-cli new-failure-mode shadow-catch, expires 05:51Z +bac892f5-...json # background notifier B-0170 ready-to-grind, expires 06:43Z +c1362d5e-...json # background notifier B-0441 ready-to-grind, expires 06:43Z +d2b7fc2f-...json # otto-cli B-0527 collision republish, expires 05:14Z +``` + +## Worktrees alive after this tick (Otto-CLI) + +- `/private/tmp/zeta-otto-cli-0027z-sidetick` on its original branch `feat/ani-full-history-extract-plus-grok-extract-tool-otto-cli-2026-05-15` — RESTORED after temporary branch-switch for this shard commit +- Primary worktree at `/Users/acehack/Documents/src/repos/Zeta` — detached HEAD `65c7865`, abandoned-rebase state; read-only + +## Cron sentinel + +`596e842c` armed via session-start hook (catch-43 prevention). + +## Next + +Cron-driven. Next tick: + +1. Verify this shard's PR merges +2. Re-check B-0527 collision state (whether Lior reallocated, whether either PR's rebase succeeded) +3. If advisory `d2b7fc2f` expires at 05:14Z without resolution, decide between (a) third republish or (b) escalate via a different channel +4. **Investigate the worktree-pruning-race source**: run `ps`, `lsof`, `launchctl list | grep zeta`, and `fs_usage` to identify the rm-rf source. File a B-NNNN row. +5. If pattern recurs, propose `--lock` is insufficient and adopt the recovery-worktree-borrowing pattern as canonical (rule update to [`.claude/rules/claim-acquire-before-worktree-work.md`](../../../../../.claude/rules/claim-acquire-before-worktree-work.md)) diff --git a/docs/hygiene-history/ticks/2026/05/15/0458Z.md b/docs/hygiene-history/ticks/2026/05/15/0458Z.md new file mode 100644 index 0000000000..abc2228fee --- /dev/null +++ b/docs/hygiene-history/ticks/2026/05/15/0458Z.md @@ -0,0 +1,52 @@ +| 2026-05-15T04:58:00Z | claude-opus-4-7 | 596e842c | shard: PR #3359 MD032 fix on 0414Z shard; recovery-worktree-borrowing pattern validated for iteration | (PR #3359) | local check-md032 hook fires on Edit/Write but not bash heredoc — `bun x markdownlint-cli2` locally before commit | + +# Tick 0458Z — PR #3359 markdownlint MD032 fix; recovery-worktree-borrowing pattern validated for second commit on same branch + +## Headline + +- **PR [#3359](https://github.com/Lucent-Financial-Group/Zeta/pull/3359)** (0414Z shard) caught markdownlint MD032 violation at line 104 of `docs/hygiene-history/ticks/2026/05/15/0414Z.md` (lists should be surrounded by blank lines). Fixed in commit `5b99b64`. Auto-merge stays armed; CI re-runs. +- **Recovery-worktree-borrowing pattern validated for iterative work**: same 0027z sibling worktree used at 0452Z (initial commit) used again at 0458Z (MD032 fix). Switch → edit → commit → push → switch back; untracked Ani memory file preserved across both switches for THIS run (the target shard branch did not track the same memory-file path). The pattern is **not** a general guarantee — if the target branch tracked the same path, `git switch` would refuse and an untracked-overwrite hazard would surface; mitigation is `git stash -u` before borrowing. With that caveat in place, the pattern works for both initial commit AND fix-on-fix iteration. +- Cron sentinel `596e842c` still armed. + +## Δ since 0452Z + +| What | At 0452Z | At 0458Z | +|---|---|---| +| PR #3359 | OPEN, auto-merge armed, mergeable=MERGEABLE | OPEN, auto-merge armed, gate=BLOCKED (1 required check failed: `lint (markdownlint)`) → fixed with `5b99b64`, CI re-runs | +| PR #3359 commits | `47baea4` | `5b99b64` (+1 commit: MD032 fix) | +| 0027z worktree borrow count | 1 (commit `47baea4`) | 2 (added MD032 fix) | + +## Substrate-honest observations + +### Local lint vs CI lint asymmetry + +The local hook `.claude/hooks/check-md032-pretooluse.ts` exists for exactly this MD032 rule but it fires on `Edit` / `Write` tool invocations, NOT on `cat > … <` locally before commit. The local hook's coverage is partial — it covers Edit/Write but not bash heredoc writes. + +### Recovery-worktree-borrowing — iteration works + +The 0452Z proof showed the pattern works for ONE commit. The 0458Z proof shows it works for ITERATIVE commits on the same borrowed worktree. Process for fix-iteration: + +1. `cd` into borrowed worktree +2. `git switch shard/tick-0414z-otto-cli-2026-05-15` (already on this branch from prior borrow OR fresh switch) +3. `git pull --ff-only origin shard/tick-0414z-otto-cli-2026-05-15` to sync remote (just in case) +4. Edit the file +5. `git add` + `git commit` with proper trailer +6. `git push origin ` +7. `git switch ` to restore (only at end of iteration) + +The borrowed worktree stays usable across multiple commit cycles without re-switching to original until the iteration is complete. + +## Cron sentinel + +`596e842c` armed. + +## Next + +Cron-driven. Next tick: + +1. Verify PR #3359 CI green + auto-merge fires (MD032 fix landed `5b99b64`) +2. If still blocked, investigate next required-check failure +3. Re-check B-0527 collision state +4. If `d2b7fc2f` expires at 05:14Z without resolution, third republish or escalate