Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
33 changes: 33 additions & 0 deletions bin/fm-claude-stop-autoarm.sh
Original file line number Diff line number Diff line change
Expand Up @@ -35,6 +35,8 @@
# - Foreground arm: the owner runs bin/fm-watch-arm.sh in the FOREGROUND of
# this hook-owned process tree (never shell &); Claude owns the process
# group, so its timeout/session teardown kills arm and watcher together.
# HUP, TERM, and INT are translated through the ordinary durable failure
# handoff instead of leaving the generation frozen at arming.
# - Translation: while supervision is still needed and AFK remains inactive,
# an actionable arm close (signal:/stale:/check:/heartbeat) prints one
# rewake banner to stderr and exits 2, which wakes Claude even while idle
Expand Down Expand Up @@ -207,6 +209,37 @@ autoarm_record() { # <outcome>
fm_autoarm_write_owned "$STATE" "$MY_GEN" "$1" >/dev/null 2>&1 || true
}

# Claude terminates the complete async-hook process tree when the configured
# hook timeout expires. The arm is intentionally allowed to follow a healthy
# watcher until its next wake, so that wait cannot be shortened without adding
# artificial turns. Translate a host interruption through the ordinary durable
# failure protocol instead: the winning generation records a terminal outcome,
# creates the episode marker, and exits 2 so Claude delivers a recovery turn.
# A superseded generation remains silent, and an episode whose attended
# fail-open was already consumed must not restart automatic continuation.
# shellcheck disable=SC2329 # Invoked indirectly by the signal traps below.
handle_autoarm_signal() {
local signal=$1
trap - HUP TERM INT
[ -z "${OUT:-}" ] || rm -f "$OUT" 2>/dev/null || true
if [ -e "$FAILURE_ALARM" ]; then
autoarm_record failed-suppressed
exit 0
fi
if [ ! -e "$FAILURE_NOTICE" ]; then
printf 'firstmate watcher auto-arm INTERRUPTED by %s - the Stop-owned automatic supervision mechanism did not reach a terminal watcher outcome.\n' "$signal" >&2
printf 'Do not launch a manual background arm from this notice; investigate the automatic Stop hook and watcher startup before ending blind.\n' >&2
autoarm_commit failed "$FAILURE_NOTICE" && exit 2
exit 0
fi
autoarm_commit failed-suppressed && exit 2
exit 0
}

trap 'handle_autoarm_signal HUP' HUP
trap 'handle_autoarm_signal TERM' TERM
trap 'handle_autoarm_signal INT' INT

# X mode cadence: source the generated config so an X instance polls at its
# 30s cadence (fm-bootstrap.sh x_mode_setup contract).
# shellcheck source=/dev/null
Expand Down
1 change: 1 addition & 0 deletions docs/turnend-guard.md
Original file line number Diff line number Diff line change
Expand Up @@ -107,6 +107,7 @@ Two bounded residuals are accepted intent, each costing at most one extra contin
A legacy build's lock-holding claim (recognizable by its `autoarm` role file) still defers or reclaims under the legacy abandonment proof, with a live identity-verified stuck owner retired via TERM before its lock is removed and an unverified pid never signalled, so an upgrade mid-session can neither double-arm nor deadlock, and a failed reclaim re-blocks rather than allowing a blind stop.
Fresh `failed` and `failed-suppressed` outcomes enter or advance the failure progression instead of acting as unconditional recovery proof.
The auto-arm itself rechecks the healthy watcher predicate and retries a bounded number of times before reporting a genuine failure.
The foreground arm legitimately follows a healthy watcher until its next wake, so the hook catches HUP, TERM, and INT from host timeout or teardown and commits the ordinary durable failed outcome and failure-notice marker before exiting 2 for a recovery turn.
The first fresh exhausted-failure epoch preserves its handoff without consuming a blocked-stop count, while later fresh failed epochs advance the same monotonic progression instead of resetting it.
When none of those proofs appears, it re-blocks up to `FM_CLAUDE_TURNEND_BLOCK_BUDGET` times (default 3, below Claude's 8-block override).
In Claude mode, positive watcher recovery clears the block budget, failure notice, and attended alarm together under the existing budget lock before either hook reports ordinary recovery.
Expand Down
2 changes: 1 addition & 1 deletion docs/watcher-continuity.md
Original file line number Diff line number Diff line change
Expand Up @@ -118,7 +118,7 @@ The same suite covers ordinary same-process session replacement for `/new`, `/re
`tests/fm-watch-recovery-loop.test.sh` covers the once-per-generation announcement bound with the real Pi extension against a refused handling handshake, and a handling successor that must surface a real crew event instead of going blind.
`tests/fm-watcher-lock.test.sh` covers verified-successor attach, recovery publication before stale-lock removal, the typed self-eviction failure, bounded and successor-linked lifecycle rows, and a SIGSTOP counterfactual that distinguishes a live PID from a stale beacon before classifying termination.
`tests/fm-subagent-pretool-check.test.sh` proves Claude retains only the non-status Bash seatbelts.
`tests/fm-claude-stop-autoarm.test.sh` covers the auto-arm's scope, stale and live session owners, unchanged AFK and need boundaries, single-flight, bounded failure retries, benign live-watcher cycle ends, one-notice failure episodes, and exit-2 translation.
`tests/fm-claude-stop-autoarm.test.sh` covers the auto-arm's scope, stale and live session owners, unchanged AFK and need boundaries, single-flight, bounded failure retries, benign live-watcher cycle ends, one-notice failure episodes, exit-2 translation, and host-timeout HUP/TERM/INT translation into the same durable failure handoff.
It also covers generation-claim single-flight, stuck-claim supersession, superseded-owner silence, notice-marker refusal and retry, ownership-atomic episode reset, and the legacy upgrade shim; [`turnend-guard.md`](turnend-guard.md) owns those behavior contracts.
`FM_CLAUDE_LIVE_E2E=1 tests/fm-claude-stop-autoarm-live-e2e.test.sh` starts with the reproduced stale-lock state, runs session start first, completes two tokenless cycles, and checks the competing-live-owner negative control.
`tests/fm-turnend-guard.test.sh` covers the cooperative `--claude` guard, including monotonic failed-epoch progression, the integrated bounded fail-open, post-alarm continuation suppression, and positive recovery reset; [`turnend-guard.md`](turnend-guard.md#regression-coverage) lists that suite's full generation and legacy claim coverage.
Expand Down
36 changes: 36 additions & 0 deletions tests/fm-claude-stop-autoarm.test.sh
Original file line number Diff line number Diff line change
Expand Up @@ -683,6 +683,41 @@ test_single_flight_admits_exactly_one_owner() {
pass "auto-arm: concurrent firings admit one owner and one rewake translation"
}

# Claude terminates the complete async hook process tree when the declared hook
# timeout expires. The hook owner must turn that TERM into the same durable,
# rewake-triggering failure handoff as any other exhausted arm failure; leaving
# the generation at `arming` cannot recover without a later manual turn.
test_term_mid_arm_commits_failure_and_rewakes() {
local dir out hook_pid i status=0
dir=$(make_primary_dir "$TMP_ROOT/term-mid-arm")
: > "$dir/state/task.meta"
write_arm_fixture "$dir" blocking-actionable
out="$dir/state/autoarm.out"
run_autoarm_bg "$dir" "$out"

hook_pid=
i=0
while [ "$i" -lt 100 ]; do
hook_pid=$(epoch_field "$dir" owner_pid)
[ -n "$hook_pid" ] && [ -e "$dir/state/arm-ran" ] && break
sleep 0.02
i=$((i + 1))
done
[ -n "$hook_pid" ] || fail "auto-arm did not publish its generation owner before TERM"
[ -e "$dir/state/arm-ran" ] || fail "auto-arm did not enter the foreground arm before TERM"

kill -TERM "$hook_pid" 2>/dev/null || fail "could not TERM the foreground auto-arm owner"
wait "$RUN_AUTOARM_BG_PID" || status=$?

expect_code 2 "$status" "TERM mid-arm must preserve Claude's rewake-triggering hook exit"
assert_present "$dir/state/.claude-autoarm-failure-notified" "TERM mid-arm left no durable failure marker"
[ "$(epoch_outcome "$dir")" = failed ] \
|| fail "TERM mid-arm left a nonterminal ledger outcome: $(sed -n '1p' "$dir/state/.claude-autoarm-epoch")"
assert_contains "$(cat "$out")" "firstmate watcher auto-arm INTERRUPTED" \
"TERM mid-arm omitted the rewake failure banner"
pass "auto-arm: TERM mid-arm commits a durable failure and exits 2 for rewake"
}

# --- abandoned single-flight claim recovery (legacy shim) ----------------------
# The 2026-08-14 lapse: one cycle armed, beat its beacon, delivered a single
# rewake, and exited, leaving its owner lock behind with a live pid. The single
Expand Down Expand Up @@ -1219,6 +1254,7 @@ test_owner_mutex_contention_preserves_failure_episode_reset
test_arms_for_x_mode_poll_need_without_inflight
test_arms_for_registered_custom_check_without_inflight
test_single_flight_admits_exactly_one_owner
test_term_mid_arm_commits_failure_and_rewakes
test_abandoned_owner_claim_is_reclaimed_and_rearms
test_arming_claim_with_fresh_beacon_is_never_reclaimed
test_fresh_arming_claim_with_stale_beacon_is_never_reclaimed
Expand Down
Loading