Skip to content

fix: renew idle Claude watcher cycles before hook timeout - #6129

Open
d1on wants to merge 11 commits into
kunchenguid:mainfrom
d1on:fm/fm-claude-idle-secondmate-8h-hook-blind
Open

d1on wants to merge 11 commits into
kunchenguid:mainfrom
d1on:fm/fm-claude-idle-secondmate-8h-hook-blind

Conversation

@d1on

@d1on d1on commented Sep 29, 2026 •

Copy link
Copy Markdown

Problem

An idle Claude primary (a main home or a secondmate home) that runs the plain Stop-hook arm stops being supervised after about eight hours.
Since #6124 the supervision host is the default for Claude primaries and already ends its own park before the hook timeout, so this affects homes that opt out of the host (config/supervision-host holding off), and any home that ran the plain arm before that change.
The Stop-owned auto-arm (bin/fm-claude-stop-autoarm.sh) runs as an asyncRewake Stop hook registered with timeout: 28800.
On an idle home the watcher cycle never closes, because no-change heartbeats are absorbed without ending the park.
At 28800 seconds Claude terminates the hook's process tree, and the #4474 trap records the failure and exits 2, but Claude does not deliver the exit 2 of a hook it terminated at its own timeout (measured in docs/verification/supervision.md).
No turn starts, so nothing re-arms, and the home stays unwatched until the next inbound message.
The supervision host bounds its own park below the timeout; the plain hook-owned arm did not.

Fixes #5718

Fix

  • The hook ends its own quiet park at a fixed 27000-second boundary, measured from the firing's start and checked every second.
  • At the boundary it stops only the watcher of its own cycle, through a new identity-bound bin/fm-watch-arm.sh --stop-if-watcher PID IDENTITY that leaves any other cycle's lock untouched, then starts a handling successor so the short renewal turn stays covered.
  • It delivers one check: cycle-renewal rewake while Claude still honors exit 2; the renewal turn drains, handles anything the drain presents, acknowledges, and ends, and that turn's end arms the next bounded park.
    When a real watcher reason reached the hook at the boundary, that reason is delivered as an ordinary wake without the renewal line.
  • Every wait after the boundary is bounded so the rewake always commits before the registration: reaping the arm escalates from TERM to KILL within five seconds, and the successor's confirmation wait stops at 28770 seconds elapsed.
  • The fix(bin): translate Stop hook timeout signals into durable auto-arm failure #4474 HUP/TERM/INT translation stays as defense in depth, and the supervision host keeps its own boundary.
  • Docs: docs/turnend-guard.md, docs/supervision-host.md, docs/watcher-continuity.md, and docs/supervision-protocols/claude.md describe the boundary and the renewal wake.

Tests

  • tests/fm-claude-stop-autoarm.test.sh drives the hook with a test-only elapsed clock, honored only under FM_TEST_SEAM=1:
    test_quiet_park_renews_at_the_park_boundary, test_boundary_real_event_has_no_renewal_label, test_park_boundary_reaps_term_resistant_arm_before_deadline, test_renewal_caps_unconfirmed_successor_wait, and test_park_clock_requires_the_test_marker.
  • tests/fm-watch-arm.test.sh runs a real arm and watcher: test_scoped_stop_preserves_other_cycle_lock.
  • Each scenario test was checked to fail once its guard is removed from the implementation; with the boundary removed, the hook never returns.
  • A real 28800-second Claude hook timeout cannot practically be driven live, so real-harness confirmation of the renewal wake will come from field observation after release.

Known edge cases left for a follow-up

These behave exactly as on main and are out of scope here, because a durable fix needs watcher coverage outside the hook-owned process group:

  • Overlapping-firing handoff: when a newer Stop firing supersedes a still-running firing whose arm started the watcher, the older firing's arm and watcher remain in its hook's process group.
  • Away-mode takeover during a park: if away mode begins while a hook-owned park is running, the boundary still ends that hook's arm and its watcher.

Pipeline

Updates from git push no-mistakes

✅ **intent** - passed

✅ No issues found.

✅ **Rebase** - passed

✅ No issues found.

🔧 **Review** - 1 issue found → auto-fixed (3) ✅
  • 🚨 bin/fm-watch-arm.sh:615 - USR1 lets a started arm exit without signalling its watcher, but that watcher was forked inside Claude's hook-owned process group. When the superseded hook exits, Claude can terminate the remaining process tree, killing the watcher the newer generation follows. The same lifetime problem affects the away-mode return at bin/fm-claude-stop-autoarm.sh:405-408: it leaves the arm in that process group before exiting. The new tests check survival under a fake harness, not after Claude ends the hook. The remedy needs an authorized way to transfer or replace coverage outside the hook-owned group; simply releasing the arm does not make either handoff durable.

🔧 Fix applied.
3 issues (1 error, 2 warnings) still open:

  • 🚨 bin/fm-watch-arm.sh:615 - USR1 lets a started arm exit without signalling its watcher, but that watcher was forked inside Claude's hook-owned process group. When the superseded hook exits, Claude can terminate the remaining process tree, killing the watcher the newer generation follows. The same lifetime problem affects the away-mode return at bin/fm-claude-stop-autoarm.sh:405-408: it leaves the arm in that process group before exiting. The new tests check survival under a fake harness, not after Claude ends the hook. The remedy needs an authorized way to transfer or replace coverage outside the hook-owned group; simply releasing the arm does not make either handoff durable.
  • ⚠️ bin/fm-watch-arm.sh:520 - The new --stop-if-watcher mode can still take the unscoped stale-lock cleanup path. If the recorded PID has been reused and the lock now describes a different cycle whose identity no longer matches that process, the PID check at line 509 passes, but this branch publishes downtime and removes that other cycle's lock. Return without changing state on a scoped identity mismatch; retain stale-lock recovery for the existing unscoped modes.
  • ⚠️ bin/fm-claude-stop-autoarm.sh:599 - A watcher can emit an actionable reason just after the arm-liveness check at line 398, as the boundary check at line 399 fires. The banner then includes that real reason and also prints cycle-renewal - no event, giving the handling turn a false no-event label. Emit the renewal line only when the boundary produced no actionable watcher reason.

🔧 Fix applied.
2 warnings still open:

  • ⚠️ bin/fm-claude-stop-autoarm.sh:599 - A watcher can emit an actionable reason just after the arm-liveness check at line 398, as the boundary check at line 399 fires. The banner then includes that real reason and also prints cycle-renewal - no event, giving the handling turn a false no-event label. Emit the renewal line only when the boundary produced no actionable watcher reason.
  • ⚠️ bin/fm-claude-stop-autoarm.sh:401 - The round-2 fix left a sibling path to a false no-event label. If the watcher delivers a real reason just before the boundary, TERM can interrupt the arm before it copies that reason to the hook output; the arm's signal handler exits without printing its child's output. At bin/fm-claude-stop-autoarm.sh:599, the empty output then selects check: cycle-renewal - no event rather than the real reason. Check the identity-bound delivery record before classifying a boundary close as no-event.

🔧 Fix applied.
✅ Re-checked - no issues remain.

⚠️ **Test** - 2 warnings
  • ⚠️ The real Claude Stop-hook renewal could not be demonstrated live. The required launch from this linked gate worktree does not qualify as a primary checkout, so the hook remained inert. A runbook-authorized genuine-primary lab is needed to observe the renewal wake.
  • ⚠️ live validation verdict: inconclusive (0 of 4 scenarios were driven live against the product); untested: An idle Claude second mate reaches its park boundary and receives a renewal wake with a covering successor, A watcher event at the boundary is delivered without a misleading renewal label, A resistant arm or unconfirmed successor cannot hold renewal past Claude’s hook deadline, A scoped watcher stop leaves a different watcher cycle intact
  • Live validation: ⚠️ inconclusive - 0 of 4 scenarios driven live against the product
Scenario Result Live Evidence
An idle Claude second mate reaches its park boundary and receives a renewal wake with a covering successor ⏸️ untested no The mandated launch from the linked gate worktree is not a genuine primary checkout. Provide a runbook-authorized genuine-primary lab for a live Stop-hook cycle.
A watcher event at the boundary is delivered without a misleading renewal label ⏸️ untested no The same primary-scope restriction prevented a live watcher cycle; provide a runbook-authorized genuine-primary lab.
A resistant arm or unconfirmed successor cannot hold renewal past Claude’s hook deadline ⏸️ untested no The linked-worktree primary-scope restriction prevented a live hook cycle; provide a runbook-authorized genuine-primary lab with a controlled elapsed-clock test.
A scoped watcher stop leaves a different watcher cycle intact ⏸️ untested no No live watcher cycle was established in the isolated lab because the required linked-worktree launch was outside primary scope; provide a runbook-authorized lab with an active watcher.
  • bash tests/fm-claude-stop-autoarm.test.sh
  • bash tests/fm-watch-arm.test.sh
  • Launched Claude 2.1.285 on a private fm-lab tmux socket, completed a turn, captured the pane and lab state, then tore down the lab.
✅ **Document** - passed

✅ No issues found.

✅ **Lint** - passed

✅ No issues found.

✅ **Push** - passed

✅ No issues found.

@greptile-apps

greptile-apps Bot commented Sep 29, 2026 •

Copy link
Copy Markdown

RetriggerConfidence Score: 4/5

[High risk] Adds a new watcher-stop mode tied to supervision timeout logic.

The PR does not appear safe to merge while a superseded Stop firing can still stop the current watcher.

Reviews (3) · Last reviewed commit: "no-mistakes(document): Clarify Claude qu..."

Comment thread bin/fm-claude-stop-autoarm.sh Outdated
Comment thread bin/fm-claude-stop-autoarm.sh Outdated
Comment thread bin/fm-claude-stop-autoarm.sh Outdated
Comment thread bin/fm-claude-stop-autoarm.sh Outdated
Comment on lines +401 to +403
if ! fm_autoarm_still_owner "$STATE" "$MY_GEN"; then
# Reap only our arm; an attached arm does not signal its watcher.
stop_own_arm

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Superseded arm stops shared watcher When a newer Stop firing attaches to the watcher started by the old arm, this cleanup sends TERM to that old arm. A started arm forwards TERM to its watcher, so the superseded firing stops the newer firing's only watcher instead of leaving its supervision intact. The supersession test covers an attached old arm, which does not forward TERM.

Knowledge Base Used: Watch and wake workflows

Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!

d1on added 11 commits September 29, 2026 23:29
An idle Claude primary's plain Stop-hook park never closes, because no-change
heartbeats are absorbed, so Claude TERMs the hook at its 28800-second timeout
and drops the exit 2 the timeout trap emits. The home then stays unsupervised
until the next inbound message.

The hook now ends its own quiet park at FM_CLAUDE_AUTOARM_PARK_SECONDS
(default 27000, below the registration): it stops the home's watcher, starts a
covering handling successor, and delivers one no-event `check: cycle-renewal`
rewake while Claude still honors exit 2. The renewal turn's end arms the next
bounded park. The timeout trap stays as defense in depth, away mode is left
alone, and the opt-in supervision host keeps its own boundary.

Fixes kunchenguid#5718
…-claude-stop-autoarm.sh: away mode is checked before boundary signalling, superseded firings reap their own arm silently, and boundary arm termination is bounded with KILL escalation. Added behavioral regressions in tests/fm-claude-stop-autoarm.test.sh. The test suite, shellcheck, bash syntax checks, and git diff checks pass
A superseded Claude Stop-hook generation reaped its own arm with TERM at the
park boundary, and a started arm forwards TERM to the watcher it owns, so the
superseded firing killed the watcher a newer generation was following.

bin/fm-watch-arm.sh now treats USR1 as a release: the arm records
arm-released in the cycle ledger and exits without signalling its watcher,
which keeps running as the home's singleton. The hook's superseded path
releases its arm with USR1 (bounded, with the existing KILL fallback, which
never reaches the watcher either). The arm header also documents the
--stop-if-watcher mode the park boundary uses.

The supersession test now uses a started arm stub that forwards TERM like the
real arm, and a real-process arm test covers release against a TERM control.

Refs kunchenguid#5718
@d1on
d1on force-pushed the fm/fm-claude-idle-secondmate-8h-hook-blind branch from 9ae8551 to 0274f96 Compare September 30, 2026 03:37
@kunchenguid

Copy link
Copy Markdown
Owner

Speaking as Kun's firstmate: triage on HEAD 0274f965190176b9e5be85fcb81d3de29dfaa985.

closesReadyForPr: VERIFIED against #5718. Issue asks for a voluntary park bound below Claude's 28800s Stop-hook timeout on the plain arm (Claude drops the #4474 exit-2 of a timed-out hook; idle homes go unsupervised). Body Fixes #5718 matches: park boundary at 27000s, identity-scoped --stop-if-watcher, check: cycle-renewal rewake, successor budget capped at 28770s elapsed. Not just a closing-link sort hint.

contract-class: new-default. Main tip header still says the plain arm does not shorten a quiet park; host already owns its own boundary and is the Claude default. This hardcodes PARK_SECONDS=27000 and a renewal wake on the shared run_arm path (including homes that opted out of the host). VISION peace-of-mind / "homes should stay supervised" does not make restore (FM-LEARN-4627). New always-on renewal wake class on the plain-arm path → new-default. No auto-merge.

VISION (brief): One captain/interface aligns (continuity without new captain surface). Authority aligns (no new consent assumption beyond existing Stop arm). Scripts/agents aligns (deterministic boundary in scripts). Restart aligns (ledger/successor unchanged). Delegation N/A. Fleet-outlives-vendor aligns (Claude-specific measured adapter debt, documented). Scope aligns (command-layer supervision continuity, not workshop). Align paragraph: deepens looking-away confidence for idle Claude homes; resist risk is new default-on wake noise on the opt-out plain arm.

Attestation: MATCH (0274f965…). workflow-zero: yes. CI: first-time fork runs approved this pass (36665842719, 36665233092, 36665233093); NM body-compliance SUCCESS; full CI queued. Greptile ignored (repo does not use it).

Waiting on: CI to finish. Even if green: new-default → no auto-merge; Firstmate flag only once otherwise-ready.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Claude Stop auto-arm still goes deaf after 8 hours on an idle home: the #4474 timeout trap commits the failure, but Claude drops its exit 2

2 participants