fix(watch): keep supervision alive across delivered wakes and parked lanes - #1419
Closed
mattadams-dev wants to merge 14 commits into
Closed
mattadams-dev wants to merge 14 commits into
mattadams-dev wants to merge 14 commits into
Conversation
Captured 2026-07-31 from a live home's state/.watch-cycle-exits.log. It accumulates over days and cannot be regenerated, so it is committed as raw evidence before any analysis or fix is written against it.
… lanes a check-in cadence wip checkpoint: implementation only, regression tests still to come. The arm layer never started the next watcher cycle after delivering an actionable wake (successor=none on 499 of 499 captured cycles), so supervision ended on every wake and resumed only if some adapter above it happened to re-arm. Arming is now the first act of consuming a wake. Parked lanes each carried a standing per-cycle trigger, so a fresh watcher died within seconds of every start on whichever lane was next overdue. Due lanes now batch into one home-wide check-in rate-limited by .last-parked-checkin mtime.
… parked check-in cadence Six cases across the two suites that already own this behavior: - a delivered wake arms its successor before the wake is handled - an attached arm delivers the wake its successor cycle closed on - no successor is armed for a home with nothing left to supervise - the preserved 499-record capture still shows the defect a fresh cycle no longer reproduces - parked lanes batch into one check-in per cadence window, and the cadence still comes due - a parked lane whose wait has cleared is surfaced at once inside a closed cadence
…adence Corrects the ownership claim that continuity lives only in the per-harness adapters, and records the forked-session case where Claude's Stop hook is inert by design because state/.lock still names the live pre-fork harness.
fm_test_tmproot is called through a command substitution, so its array append landed in the subshell and every suite leaked its entire temp root. Registration now goes to a file that survives the subshell, and only the sourcing shell may run the cleanup. This became load-bearing once an arm can leave a detached successor: a leaked temp root kept a live watcher supervising a home no test owned any more, and the accumulated processes destabilised timing-sensitive cases in the same suite. Also keep the away-mode daemon on the unbatched parked wake form, since it classifies one window per printed reason.
tests/fixtures/<name>/ is the repo's fixture convention: the changed-test selector resolves a fixture through the directory its consuming suite names, and a file sitting directly under tests/fixtures/ has no mapping at all.
…p root A named per-owner path is also what lets the command-substitution subshell append to the registry its parent will read, and a suite that never takes a temp root now leaves nothing behind.
…e watcher comments
Author
|
Apologies for the noise - this was opened against upstream by mistake and has been reopened on our own fork (mattadams-dev#3); no action needed here. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Intent
Fix two defects in firstmate's supervision watcher, diagnosed by the captain from a preserved 499-record capture of a live home's watcher cycle-exit ledger (committed at tests/fixtures/watch-cycle-exits/cycle-exits.log because it accumulates over days and cannot be regenerated).
FRAMING THE CAPTAIN SET, WHICH MATTERS FOR REVIEW: the watchers were never failing. All 499 records show exit_code=0 and signal=none - the watcher is one-shot BY DESIGN and completes in order to deliver a wake. Earlier attempts treated this as a supervision failure and repaired it by hand; that framing was wrong. This is a redesign, not a fourth patch.
DEFECT 1: successor=none on 499 of 499 records. Nothing armed the next watcher cycle after a wake was delivered, so supervision ended on every delivered wake and only resumed if some adapter above the arm layer happened to re-arm. The captain specified the fix shape verbatim: 'arming the successor is the first act of consuming a wake, before handling what it woke you for.' That ordering IS the design - handling first and arming afterwards leaves a window as wide as the handling turn, which is exactly what was swallowing supervision. So in bin/fm-watch-arm.sh the successor is armed and verified before print_watch_output, deliberately. The successor is intentionally NOT the arm's child (double-forked) because it must outlive the arm that delivers the wake; that is a deliberate departure from the file's existing 'the child stays our child' comment, and it is documented in the header. It is still verified through the same live-process-plus-fresh-beacon honesty gate as an owned child, so it is not the fire-and-forget shell-& pattern the header warns about. Gated on supervision actually being needed and away mode being off, so an idle home is never left with a detached watcher.
Root cause of the secondary path, recorded but deliberately NOT fixed here: after a session fork, state/.lock still names the pre-fork harness pid, which stays alive, so the forked session's Claude Stop auto-arm hook can never claim the home and is permanently inert. Evidence from the live home: .lock=17907, a live claude process. The arm-layer fix makes continuity independent of that path; the lock behaviour itself belongs to the existing fm-upstream-issue-dossier backlog item, whose first line is already 'watcher re-arm after fork'.
A third exit shape the captain observed and told me to record rather than chase is resolved as a side effect: an ATTACHED arm could only report a genuine wake as 'watcher: FAILED - cycle ended without an actionable reason', because it could not see its watcher's output. The successor now writes its one reason to state/.watch-successor-output. before releasing the singleton, and the attached arm claims and delivers it.
DEFECT 2: 112 of 499 cycles (22%) exited within 5 seconds, all on actionable-stale, all driven by lanes deliberately parked awaiting a captain decision with idle ages in the thousands of seconds. Each parked lane carried its own independently-phased standing trigger, so a fresh watcher died within seconds of every start on whichever lane was next overdue. The captain specified the fix as his own pins-for-conditions rule: 'parked-awaiting-captain is not actionable staleness - give it a check-in cadence, not a standing trigger', and explicitly said NOT to solve it by suppressing the lane, because a declared wait can stop holding: the distinction is CADENCE, not silence. So due lanes now batch into one check-in wake, rate-limited home-wide by a state/.last-parked-checkin mtime - deliberately copying the shape the slow per-task checks already use (.last-check mtime) so the cadence survives watcher restarts, which the captain pointed at as the model. A lane whose wait has actually cleared still surfaces at once, independent of the cadence, and that is covered by its own test. Away mode keeps the unbatched one-shot form because the daemon classifies one window per printed reason.
This is guard-class code, so the captain applied the fleet mutation bar: each protection is proven by the mutation that breaks exactly its own test. That matrix is recorded in docs/verification/supervision.md - six mutations, each run twice (once to see which case it kills, once with that case removed to prove nothing else breaks). One row is honestly reported as shared by two cases guarding the same pause release.
Also in the branch, because it turned out to be load-bearing rather than incidental: tests/lib.sh's fm_test_tmproot is called through a command substitution, so its array append landed in the subshell and EVERY suite leaked its entire temp root. That was harmless until an arm could leave a detached successor - then a leaked temp root kept a live watcher supervising a home no test owned any more, and the accumulated processes destabilised a timing-sensitive case (test_watch_restart_attaches_to_healthy_peer) in the same suite. Registration now goes to a named per-owner registry file that survives the subshell, and only the sourcing shell may run the cleanup. After the fix both watcher suites run with zero stray watcher processes and zero leftover temp roots.
Known pre-existing failure, NOT caused by this branch and deliberately not fixed here: tests/fm-session-start.test.sh's Herdr husk-recovery case fails identically on the unmodified base commit f7d0d0a ('the later fleet read did not confirm the relaunched Herdr endpoint'). Verified by checking out the base commit and re-running. It is unrelated to supervision successor arming.
What Changed
bin/fm-watch-arm.shnow arms and verifies a successor watcher as the first act of consuming a wake, before the wake is printed, so supervision no longer ends on every delivered cycle. The successor is deliberately detached (double-forked) so it outlives the arm that delivers the wake, is still confirmed through the same live-process-plus-fresh-beacon gate as an owned child, and is armed only when supervision is actually needed and away mode is off. Each successor writes its one reason to a pid-keyedstate/.watch-successor-output.<pid>file, which an attached arm claims and delivers - an attached close carrying a genuine wake is no longer reported ascycle ended without an actionable reason. The outcome is recorded per cycle in thestate/.watch-cycle-exits.logledger.bin/fm-watch.shgives parked lanes (declared external waits and captain holds) a check-in cadence instead of a standing per-cycle trigger: every lane due in one poll is batched into a single wake, rate-limited home-wide by thestate/.last-parked-checkinmtime (FM_PARKED_CHECKIN_SECS, defaulting toFM_PAUSE_RESURFACE_SECS), and only the lanes carried in an emitted check-in get their per-lane cadence stamped. A lane whose pause has actually cleared still surfaces immediately, and away mode keeps the unbatched one-shot form the daemon's one-window-per-reason triage expects.tests/lib.shregisters temp roots through a named per-owner registry file that survives the command substitutionfm_test_tmprootis called through, so suites stop leaking temp roots (which, once an arm could leave a detached successor, kept live watchers supervising abandoned homes); the 499-record live cycle-exit capture is preserved attests/fixtures/watch-cycle-exits/cycle-exits.logbecause it accumulates over days and cannot be regenerated; new cases intests/fm-watcher-lock.test.shandtests/fm-watch-triage.test.shcover successor arming, attached-cycle delivery, the away-mode gate and the parked cadence; a pre-existing race intest_watch_restart_attaches_to_healthy_peer(the TERM-resistant peer was signalled before its handler was installed) is fixed; anddocs/verification/supervision.mdrecords the capture analysis plus an eight-mutation guard-class matrix, each mutation run at least twice, with the sharedclear_pause_staterow and a surviving mutant at the redundant pause-release site reported honestly.Note on diff size: this branch is cut from a point behind the gate's
main, so the delta also carries eight already-merged upstream commits (#1303, #1327, #1328, #1339, #1349, #1350, #1356, #1358). The supervision change itself is the delta fromf7d0d0a.Risk Assessment
✅ Low: The round-1 blocking defect is fixed with a test that can no longer pass over it, the two arm-layer race gaps are closed with a pid handshake I verified from source (exec preserves the pid, and nothing re-execs before the lock claim), the registry is now unpredictable and swept by every lib-sourcing suite, and only a trivial prune-reachability tidiness item remains.
Testing
I ran the two suites that own this contract repeatedly - fm-watcher-lock four times and fm-watch-triage three times, all clean - then proved the intent at the operator surface by driving the real arm and watcher against an identical staged home under base-commit bin and branch bin, capturing arm stdout, the cycle-exit ledger record, and whether any watcher still supervises the home afterwards. The base reproduces all three captured shapes exactly and the branch fixes each. I also found the away-mode half of the successor gate had no test, so I added a focused case and mutation-proved it kills exactly that protection, taking the suite to 35 cases. The change is shell and CLI only with no rendered surface, so evidence is CLI transcripts rather than screenshots. Every run ended with zero leftover test temp roots and zero stray watcher processes, confirming the tests/lib.sh registry fix. The only working-tree change left is the added test case; bin/fm-watch-arm.sh was restored byte-identical after the mutation check.
Evidence: DEFECT 1 - successor arming, base vs branch
Evidence: Third exit shape - attached arm delivers its successor cycle wake
Evidence: DEFECT 2 - parked check-in cadence, base vs branch
Evidence: Away-mode successor gate
Evidence: Mutation proof for the added away-mode case
Evidence: Preserved 499-record capture stats
Evidence: Evidence index
Evidence: Suite log - fm-watcher-lock 35/35
Evidence: Suite log - fm-watch-triage 42/42
Pipeline
Updates from git push no-mistakes
✅ **intent** - passed
✅ No issues found.
⏭️ **Rebase** - skipped
tests/fm-secondmate-harness.test.sh- merge conflict rebasing onto origin/mainbin/fm-watch.sh:357- The parked-lane accumulator loses its record separator, so the batched check-in only ever carries one lane.parked_due=$(printf '%s%s\t%s\n' "$parked_due" "$win" "$reason")runs through a command substitution, which strips the trailing newline; the next append then concatenates directly onto the previous line. With 3 due lanes the value isw1<TAB>reason1w2<TAB>reason2w3<TAB>reason3(verified by running the exact loop in bash).flush_parked_checkin(bin/fm-watch.sh:378) therefore performs exactly onereaditeration: only the first lane is enqueued viafm_wake_append stale "$win" "$reason", only its.paused-resurfaced-<key>marker is stamped, thecombined="$combined; $reason"join never executes, andwakeprints a tab-glued mash of every lane's reason attributed to the first window. This is the exact multi-lane scenario the fix targets (the capture's 112 sub-5s exits were independently-phased parked lanes), and it also silently violates the function's own documented contract that lanes not included in an emitted check-in stay unstamped and due - lanes 2..N are never stamped, so they remain permanently due. tests/fm-watch-triage.test.sh:709 does not catch it becausegrep -F "test:fm-park$i"matches the glued line andgrep -c '^stale:'is 1 either way. Fix: preserve the newline, e.g.parked_due="${parked_due}${win}"$'\t'"${reason}"$'\n', and tighten the test to assert one wake-queue record per lane and a;-joined reason.bin/fm-watch-arm.sh:385- When two arms are attached to the same watcher - a topology this file explicitly supports ("A live cycle already present means re-arm attaches", header line 42) - both reach the close path inattach_and_waitsimultaneously. Both passwatch_output_has_wake "$out", thenmv -f "$out" "$claimed"succeeds for one and fails for the other. The loser falls back toclaimed=$out, which no longer exists, soprint_watch_outputemits nothing and the function still doesreturn 0. The arm exits 0 with completely empty stdout - precisely the "clean empty completion that an adapter could mistake for a no-op" the function comment (line 361) says it must never produce, and which the non-wake path below deliberately turns into a typed nonzero failure. Fix: onmvfailure, do not treat the wake as claimed; fall through towait_for_healthy_successorso the arm either attaches to the successor the winner just armed or fails loudly.bin/fm-watch-arm.sh:325-arm_successorbinds its pending output file to whichever pid wins the singleton (HEALTHY_PID), not to the process it actually forked. When another watcher wins - a concurrentarm_successorfrom a second attached arm, or an adapter-launched successor (docs/architecture.md:63 notes Pi and OpenCode launch their own singleton successor from child-close handlers) - the arm still renames its own tmp to<winner-pid>. The winner's stdout goes elsewhere, so the published file ends up holding the loser watcher'swatcher: already running pid Nline instead of the winner's reason, while the winner's real reason (whose fd points at the unlinked tmp on the[ -e "$target" ]branch) never lands on disk. A later arm attaching to that pid then failswatch_output_has_wakeand can reportwatcher: FAILED - cycle ended without an actionable reasonfor a genuine wake - the third exit shape this change set out to resolve. The reason still reachesstate/.wake-queue, so nothing is lost durably; only the arm-layer delivery degrades. Minimal fix: capture the pid of the process this arm actually started and only publish the output file when the confirmed healthy pid matches it, recordingattached:<pid>otherwise. Cleaner shared boundary: have fm-watch.sh write its own reason tostate/.watch-successor-output.$WATCHER_PID, so the binding is authoritative regardless of who forked it.bin/fm-watch-arm.sh:305- Recording the residual bound of the arm-layer fix, which the intent explicitly authorizes as containment for the deferredstate/.lockfork defect. A successor is armed only from an arm process, so with a permanently inert adapter (the observed forked-Claude-session case, where the Stop auto-arm hook can never claim the home) the chain is exactly one cycle deep: the detached successor covers the handling turn, wakes, writes its reason to.watch-successor-output.<pid>andstate/.wake-queue, exits, and no further successor is armed. The blind window is bounded by the successor's own cycle rather than eliminated, so the intent's "makes continuity independent of that path" is stronger than what the arm layer alone delivers. No action requested - the durable fix is correctly deferred to the fm-upstream-issue-dossier item, and the unclaimed output file is reaped byprune_successor_outputsat FM_WATCH_SUCCESSOR_OUT_TTL.tests/lib.sh:73- The cleanup registry path is fully predictable (${TMPDIR:-/tmp}/fm-test-cleanup.<pid>) and its contents driverm -rfon any line starting with/(tests/lib.sh:84). On a shared host with a sticky/tmp, another user can pre-create that path as a symlink; the sticky bit prevents therm -fat line 74 from removing it, the>>infm_test_tmprootthen follows the symlink and writes as the test user, andfm_test_cleanuprecursively deletes every absolute path it reads back. The comment's two stated requirements (subshell-visible path, nothing created for a suite that takes no temp root) are both satisfiable safely:FM_TEST_CLEANUP_REGISTRY=$(mktemp "${TMPDIR:-/tmp}/fm-test-cleanup.XXXXXX")plusexportgives an unpredictable, atomically-created path that subshells still see, andfm_test_cleanupalready removes it so it leaves no litter.tests/fm-kimi-harness.test.sh:22-trap fm_test_cleanup EXITis now installed at source time, so any suite that installs its own EXIT trap afterwards replaces it. tests/fm-kimi-harness.test.sh:22 and tests/fm-afk-pi-herdr-return-e2e.test.sh:62 both do this without callingfm_test_cleanup. Their temp roots are still removed (eachrm -rf "$TMP_ROOT"directly), so the branch's zero-leftover-temp-roots claim holds, but the registry file itself is never unlinked, leaving one stray${TMPDIR:-/tmp}/fm-test-cleanup.<pid>per run. Addingfm_test_cleanupto both cleanup functions - which the library comment at tests/lib.sh:57 already prescribes - closes it.🔧 Fix: fix parked check-in batching and successor output binding
1 info still open:
bin/fm-watch-arm.sh:309-prune_successor_outputsis only ever called fromarm_successor(the sole call site, line 309), and it sits after thesuccessor_wanted || { printf 'none'; return 0; }early return on line 308. Once a home stops needing supervision - the exact state where leftover files accumulate and nothing new claims them - the sweeper becomes unreachable, so any unclaimed.watch-successor-output.<pid>(plus orphan.pending.*/.forkpid.*from an arm killed mid-arm_successor) stays instate/indefinitely rather than aging out at FM_WATCH_SUCCESSOR_OUT_TTL. Not a correctness or supervision risk - the files are small, AGENTS.md:114 marks them watcher internals, and every live path still cleans up its own - but it defeats the sweeper's stated purpose in a codebase that otherwise size-caps every state artifact. Moving theprune_successor_outputscall above thesuccessor_wantedgate makes it run on every arm cycle, including the ones that decline to arm.🔧 **Test** - 1 issue found → auto-fixed ✅
tests/fm-watcher-lock.test.sh:447- tests/fm-watcher-lock.test.sh:447test_watch_restart_attaches_to_healthy_peerwas flaky for a cause unrelated to the leaked-temp-root explanation recorded in docs/verification/supervision.md. The case stages a TERM-resistant peer withnode -e 'process.on("SIGTERM", ...)'and immediately launchesfm-watch-arm.sh --restart, whose first act is to TERM the recorded lock pid. Nothing waited for node to finish starting, so when node lost that race the TERM landed before the handler existed, the peer died, and the arm correctly started a fresh watcher - reported asrestart did not attach to the verified healthy peer. Reproduced 3 times in 18 runs on an idle 32-core host (load 0.15), with a diagnostic confirmingpeer DEAD (TERM landed before its handler). The case is unchanged since a323c2b, so it predates this branch. Fixed by having the peer publish a readiness marker from inside the handler-installed process and having the case wait for it before anything can signal the peer; 15/15 isolated runs and two clean full-suite runs since.bash tests/fm-watcher-lock.test.sh- full suite, two consecutive clean runs (34 cases each), includingtest_delivered_wake_arms_its_successor_before_it_is_handled,test_attached_arm_delivers_its_successor_cycle_wake,test_successor_is_not_armed_without_supervision_need,test_preserved_capture_still_shows_the_defect_a_fresh_cycle_no_longer_hasbash tests/fm-watch-triage.test.sh- full suite clean, includingtest_parked_lanes_batch_into_one_checkin_on_a_shared_cadence,test_a_cleared_park_is_noticed_promptly_inside_a_closed_cadence,test_afk_present_reverts_watcher_to_one_shotManual E2E before/after:git archive f7d0d0a bin | tar -x -C <evidence>/base-tree, then ranbin/fm-watch-arm.shandbin/fm-watch.shfrom BOTH the base tree and the branch against identically staged homes (/tmp/no-mistakes-evidence/01KYVWHDGTDH63YJJC091K85AP/supervision-e2e-demo.sh), capturing arm stdout, thestate/.watch-cycle-exits.logrecord, andps -o pid=,ppid=,args=proof of the surviving detached successorManual E2E parked-lane cadence: three overdue parked lanes run through three watcher cycles per variant with per-lane throttles cleared between cycles, then.last-parked-checkinaged past its window, then one lane's pause verb cleared inside a closed cadenceLeak check after every run:ls -d /tmp/fm-watcher-lock-tests.* /tmp/fm-watch-triage* /tmp/fm-test-cleanup.* /tmp/fm-supervision-e2e.*andps -eo pid,args | grep c5e834eacdf2.*fm-watch- all emptyFlake reproduction and fix verification fortest_watch_restart_attaches_to_healthy_peer: 18 runs before the fix (3 failures, with a peer-liveness diagnostic), 15 isolated runs after the fix (0 failures)🔧 Fix: wait for TERM-resistant peer handler before restarting arm
✅ Re-checked - no issues remain.
tests/fm-watcher-lock.test.sh - 4 consecutive full runs, 34/34 cases each, all cleantests/fm-watch-triage.test.sh - 3 consecutive full runs, 42/42 cases each, all cleanManual E2E defect-1 contrast via demo-successor-arming.sh against base-commit bin and branch binManual E2E attached-delivery contrast via demo-attached-delivery.shManual E2E defect-2 contrast via demo-parked-cadence.sh over four successive watcher cyclesManual verification of the away-mode successor gate with state/.afk presentAdded test_successor_is_not_armed_in_away_mode to tests/fm-watcher-lock.test.sh; suite now 35/35 okMutation proof: removed only the away-mode line from successor_wanted() in bin/fm-watch-arm.sh, reran the suite, then restored the fileRe-derived preserved capture stats from tests/fixtures/watch-cycle-exits/cycle-exits.logLeak hygiene after every run: zero fm-* test temp roots in /tmp and no test-owned watcher processes🔧 **Document** - 1 issue found → auto-fixed (3) ✅
docs/verification/supervision.md:222- The guard-class mutation matrix has no row for the away-mode half of the successor gate. The working tree adds test_successor_is_not_armed_in_away_mode to tests/fm-watcher-lock.test.sh (uncommitted), which guards a distinct protection - .afk suppressing the successor so a detached watcher never competes with the supervise-daemon - and the intent states the fleet mutation bar requires each protection be proven by the mutation that breaks exactly its own test. That new case also shifts the '33 cases clean' figure recorded for the three lock-suite rows, which was measured before it existed. Closing this needs the mutation matrix re-run (mutating bin/fm-watch-arm.sh and running both suites), which is outside a documentation phase, so I did not invent a row or adjust the recorded counts.🔧 Fix: no doc changes; mutation matrix re-measurement unfinished
1 error still open:
docs/verification/supervision.md:225- The guard-class mutation matrix was NOT updated this round: the re-measurement run was still on its first mutation when I was required to finalize, so I have suite baselines but no per-mutation kill results. I made no edit rather than publish a partially re-measured table, which the captain explicitly ruled worse than an obviously old one.WHAT IS ESTABLISHED BY MEASUREMENT: both suites were re-run clean against the current tree (HEAD 58dae2b). tests/fm-watcher-lock.test.sh emits 35 'ok -' cases, exit 0. tests/fm-watch-triage.test.sh emits 42 'ok -' cases, exit 0. Since the table's 'N cases clean' figures are baseline-minus-the-removed-case, this already proves the three lock-suite rows reading '33 cases clean' (lines 227, 228, 229, 232 - four rows total) are stale: 33 = a 34-case baseline minus one, and the baseline is now 35 because test_successor_is_not_armed_in_away_mode was added. Those rows must read 34 once confirmed by an actual mutation run. The triage figures (41 on lines 230/231, and 40 on line 233) are arithmetically consistent with the current 42-case baseline, but consistency is not measurement and they were not re-run.
WHAT IS BUILT AND READY: /tmp/no-mistakes-evidence/mutate.py implements the full matrix with the two-run method. All eight mutations are defined and every anchor was verified to match exactly once in its target file. The new away-mode row mutates successor_wanted in bin/fm-watch-arm.sh:297, replacing the '[ -e "$STATE/.afk" ] && return 1' guard with ':' while leaving fm_supervision_needed intact, so a successor is armed even in away mode. The shared-pause-release row targets clear_pause_state in bin/fm-watch.sh:398-403 (not the similar rm at line 477), because that is the release primitive both test_a_cleared_park_is_noticed_promptly_inside_a_closed_cadence and test_secondmate_unpause_clears_pause_tracking depend on. The harness removes each killed case's invocation line and re-runs, looping until the suite goes clean, so a mutation shared by more than one case is recorded honestly instead of being forced into a one-to-one claim. One run was observed working end to end before I stopped it.
TO CLOSE: run 'python3 /tmp/no-mistakes-evidence/mutate.py' (about 35 minutes; roughly 18 suite runs at ~2 minutes each) and transcribe matrix.json into the table, including the new away-mode row. Caveat: the harness lives in ephemeral /tmp and may not survive; if it is gone it must be rebuilt before the numbers can be re-measured.
SAFETY STATE: the interrupted run left mutation 1 applied to bin/fm-watch-arm.sh. I inspected the diff, confirmed it was exactly that mutation and nothing else, and reverted it. The worktree is verified clean at 58dae2b with zero modified files, so no behavioral change escaped this phase.
🔧 Fix: re-measure guard-class mutation matrix and add away-mode row
1 info still open:
bin/fm-watch.sh:477- Surviving mutant found while re-measuring the matrix, reported rather than fixed because this is a documentation phase and closing it means adding a test.While determining which release site the 'Stop releasing a lane whose pause verb has gone' row actually describes, I measured both candidates. The
elsebranch insidesurface_nonterminal_stale(bin/fm-watch.sh:477) - the site that literally matches that label, releasing.paused-<key>,.paused-rechecked-<key>and.paused-resurfaced-<key>whenstatus_is_paused_or_captain_heldis false - can be replaced with:and tests/fm-watch-triage.test.sh still passes in full: 42ok -lines, exit 0, zero cases killed. Measured in a private copy; log at /tmp/no-mistakes-evidence/01KYVWHDGTDH63YJJC091K85AP/matrix/keep-cleared-pause-verb.run1.log.The row was therefore published against the site that IS guarded,
clear_pause_state(bin/fm-watch.sh:403), which four cases depend on. That is honest and measured, but it means the record's 'one mutation per protection' framing now has a known hole: a lane that surfaces as non-terminal stale after its declared pause verb disappears has its pause markers cleared by an untested branch.Suggested follow-up, outside this change: a triage case driving a window through
surface_nonterminal_stalewith a status whose pause verb has been replaced by a non-paused one, asserting the three.paused-*markers are gone. Then the matrix can carry a genuine row for that protection instead of relying on the broaderclear_pause_statemutation.🔧 Fix: record surviving mutant at redundant pause-release site
✅ Re-checked - no issues remain.
✅ **Lint** - passed
✅ No issues found.
✅ **Push** - passed
✅ No issues found.