Conversation
… daemon housekeeping
…r and away-mode docs
sdivanl
force-pushed
the
fm/contrib-fm-5004-parked-stale
branch
from
September 21, 2026 11:45
7efe16f to
8bfac28
Compare
Contributor
Author
|
This pull request is rebased on current main and its full CI is green at head 8bfac28 (20 checks passed, 0 failed). The fork account has no merge permission on this repository, so it needs a maintainer merge. Could you merge it when you have a moment? Thank you. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Intent
Fix the defect this home reported upstream as firstmate issue #5004: a deliberately parked, finished task keeps registering as stale, so its idle endpoint is escalated as a possible wedge.
The observed behaviour, which this home hit live on 2026-09-20 with a finished investigation whose worker had been stopped while its task record stayed open: the stop path proves the stopped or already-stopped transition and retires the busy state, but leaves no durable record that the stop was deliberate, while the watcher's pause classification still admits only a declared pause and a captain-held task. A deliberately parked finished worker whose endpoint is idle therefore has no legitimate case to fall into and keeps being treated as stale and escalated.
Upstream triage accepted the report: verdict ready-for-pr, contract class restore, judged at commit 90cd351 (#5004 (comment)). It confirmed the same diagnosis and invited exactly this scope: write a durable deliberate-stop marker from the stop command, cleared on relaunch and cleanup, so the watcher treats a parked finished task like a declared pause - a long bounded recheck cadence and no escalation.
What Changed
state/<id>.deliberate-stopmarker (fm_control_deliberate_stop_*helpers inbin/fm-control-lib.sh), written bybin/fm-control.sh'sexitverb on every verified stop (includingalready-stoppedandendpoint-gone) and cleared atbin/fm-spawn.sh's relaunch commit point and bybin/fm-teardown.sh.bin/fm-watch.shandbin/fm-supervise-daemon.shto give a task carrying that marker the declared-pause treatment - a long boundedFM_PAUSE_RESURFACE_SECSrecheck anchored on the marker's mtime - whether its pane reads idle or busy and regardless of whether its last status line is terminal or non-terminal, instead of the stale/wedge escalation ladder.Risk Assessment
✅ Low: The change is a focused, marker-gated bug fix: every new control-flow branch is guarded by presence of the deliberate-stop file, non-marker tasks keep their prior behavior, and the marker lifecycle (write on verified stop, clear on relaunch delivery/teardown) and both supervision postures are covered by executable regression tests.
Testing
Drove the deliberate-stop fix against the real product in a private real-tmux environment. The committed tests/fm-deliberate-stop-live-e2e.test.sh ran three times with all seven scenarios passing: verified exit records the marker and an unprovable endpoint refuses, the real watcher absorbs a fresh deliberate stop then re-surfaces it as a bounded 'deliberately stopped' recheck (idle, busy-past-the-turn-bound, and churning-pane variants) and never as a wedge, removing the marker returns the pane to ordinary terminal-stale, real teardown retires the marker, and a real pi relaunch clears it while an aborted relaunch retains it. A supplemental driver then ran the real bin/fm-supervise-daemon.sh classify_stale/handle_wake/housekeeping paths against a real tmux backend: it parks a deliberately stopped task, anchors the pause marker on the stop mtime so the very next tick re-surfaces without a doubled window, drops a pre-aged wedge marker for a parked task without escalating, and still escalates the same marker as 'possible wedge' when no deliberate stop exists. The user-requested hermetic suites were re-run: fm-control, fm-control-relaunch, and fm-daemon pass fully; the five deliberate-stop watch-triage cases pass (the suite is >25 min and was stopped after ~80 passing cases with no failures); the deliberate-stop teardown case passes, though the teardown suite later aborts on a leaked-process-reap test that I confirmed also fails on the base commit, so it is a pre-existing sandbox limitation unrelated to this change. No reviewer-visible UI exists for this CLI/daemon change; evidence is command transcripts and rendered escalation lines. Worktree left clean at the target commit.fm-control exitagainst a verified real worker: it records the durable state/<id>.deliberate-stop marker; an absent/unprovable endpoint refuses the stop and records no markerfm-control relaunchlaunches a real replacement and clears the marker so the replacement is supervised normally; a relaunch whose replacement launch is refused retains the parked stopEvidence: Committed real-tmux live guard output (all 7 scenarios pass)
Source: Committed real-tmux live guard output (all 7 scenarios pass)
ok - live: a verified tmux stop records the deliberate-stop marker; an unprovable endpoint records none ok - live: the watcher absorbs a fresh deliberate stop and re-surfaces it on the bounded cadence ok - live: the watcher parks a deliberately stopped busy pane on the bounded recheck cadence ok - live: a churning deliberately parked pane still gets the bounded deliberate-stop recheck ok - live: removing the marker returns the finished task to ordinary terminal-stale supervision ok - live: teardown retires the deliberate-stop marker for a real task ok - live: a real relaunch clears the marker and an aborted relaunch retains the parked stopEvidence: Away-mode daemon real-tmux driver output (all 4 scenarios pass)
Source: Away-mode daemon real-tmux driver output (all 4 scenarios pass)
ok - D1 real-tmux: classify_stale parks a deliberately stopped task (pause, not wedge) ok - D2 real-tmux: handle_wake anchors the pause marker on the stop epoch and the very next housekeeping tick re-surfaces it as a deliberate-stop recheck ok - D3 real-tmux: housekeeping drops a pre-aged wedge marker for a deliberately stopped task without escalating ok - D4 adversarial real-tmux: the same pre-aged wedge marker with no deliberate stop still escalates as a possible wedgeEvidence: Targeted hermetic suites (control, relaunch, teardown, daemon)
Source: Targeted hermetic suites (control, relaunch, teardown, daemon)
Evidence: Round-3 live validation summary
Source: Round-3 live validation summary
Pipeline
Updates from git push no-mistakes
✅ **intent** - passed
✅ No issues found.
✅ **Rebase** - passed
✅ No issues found.
✅ **Review** - passed
✅ No issues found.
fm-control exitrecords the durable state/<id>.deliberate-stop marker, and a refused (unprovable) stop records none, Adversarial: a genuinely stopped-responding worker that was never deliberately stopped still wedge-escalates exactly as before,fm-control relaunchclears the marker so the replacement is supervised normally; an aborted relaunch retains the parked stop, Teardown/cleanup removes the deliberate-stop marker so no stale marker survives the task, Away-mode daemon classifies a deliberately parked task as pause and drops a pre-aged wedge marker without escalating, Away-mode daemon re-surfaces the parked task on the bounded pause cadence and anchors the first recheck on the stop epoch (no doubled window), A deliberately stopped worker whose pane still reads busy is parked on the bounded recheck, not wedge-escalated, A deliberately parked task whose idle pane keeps churning (new hash every poll) still receives the bounded recheck instead of rotting invisiblyfm-control exitrecords the durable state/<id>.deliberate-stop marker, and a refused (unprovable) stop records none.stale-since-*wedge timer, stale suppressor advanceddeliberately stopped ... not a wedgerecheck (real tmux)stale: live:fm-parked (deliberately stopped ... rechecked on a long cadence not a wedge ...), throttle recorded, no wedge timerstale: live:fm-parkedwith the marker removedfm-control relaunchclears the marker so the replacement is supervised normally; an aborted relaunch retains the parked stopbin/fm-teardown.shagainst a live task.bash ~/.no-mistakes/evidence/01M31QF0G2P1V63BT4BCCSZJ58/live-tmux-driver.sh(real private-socket tmux server + realbin/fm-watch.sh, four phases: fresh-stop absorb, past-cadence recheck, no-marker adversarial terminal stale, re-stop first-sight absorb)bash tests/.tmp-watch-targeted.test.sh(targeted subset oftests/fm-watch-triage.test.sh: deliberate-stop absorb, re-stop, cleared marker, busy-pane, churning-pane, plus terminal-stale and wedge-escalation baselines)bash tests/.tmp-daemon-targeted.test.sh(targeted subset oftests/fm-daemon.test.sh: deliberate-stop classify/housekeeping and first-window anchor)bash tests/.tmp-control-targeted.test.sh(targeted subset oftests/fm-control.test.sh: exit records the marker on verified stop, not on refusal)bash tests/.tmp-relaunch-targeted.test.sh(targeted subset oftests/fm-control-relaunch.test.sh: relaunch clears the marker, failed backlog commit still clears, aborted launch retains)bash tests/.tmp-teardown-targeted.test.sh(targeted subset oftests/fm-teardown.test.sh: teardown removes the marker)🔧 Fix applied.
1 info still open:
tests/fm-teardown.test.sh:3306- tests/fm-teardown.test.sh aborts in this sandbox at test_leaked_worktree_process_is_reaped ('leaked worktree process survived teardown'). I reproduced the same failure from a clean archive of the base commit 631bc26, so it is pre-existing and unrelated to the deliberate-stop change (which only adds state/<id>.deliberate-stop to teardown's cleanup lists). The sandbox does not support reaping the reparented process, and the suite's fail() exits before the remaining cases run. The deliberate-stop teardown case itself ran before the abort and passed.fm-control exitagainst a verified real worker: it records the durable state/<id>.deliberate-stop marker; an absent/unprovable endpoint refuses the stop and records no markerfm-control relaunchlaunches a real replacement and clears the marker so the replacement is supervised normally; a relaunch whose replacement launch is refused retains the parked stopbash tests/fm-deliberate-stop-live-e2e.test.sh(committed real-tmux live guard; ran 3x, all 7 scenarios pass)FM_REPO_ROOT=$PWD bash ~/.no-mistakes/evidence/01M31QF0G2P1V63BT4BCCSZJ58/live-daemon-deliberate-stop-driver.sh(real tmux + real fm-supervise-daemon.sh functions; ran 2x, 4 scenarios pass)bash tests/fm-control.test.sh(verified stop records marker; refused stop records none)bash tests/fm-control-relaunch.test.sh(clear on delivered relaunch, clear after failed backlog commit, retain on aborted wiring)bash tests/fm-daemon.test.sh(park instead of wedge-escalate; bounded cadence re-surface; stop-epoch anchor)bash tests/fm-watch-triage.test.sh(five deliberate-stop cases: parked, re-stop first-sight absorb, marker-cleared resume, busy-pane, churning-pane all pass; suite stopped after ~80 passing cases as it exceeds 25 minutes)bash tests/fm-teardown.test.sh(deliberate-stop teardown case passes; aborts later on the pre-existing leaked-process-reap sandbox limitation)✅ **Document** - passed
✅ No issues found.
✅ **Lint** - passed
✅ No issues found.
✅ **Push** - passed
✅ No issues found.