Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
24 changes: 24 additions & 0 deletions bin/fm-watch-checkpoint.sh
Original file line number Diff line number Diff line change
Expand Up @@ -115,7 +115,31 @@ positive_or() { # <value> <default>

# shellcheck source=bin/fm-supervision-engine-lib.sh
. "$SCRIPT_DIR/fm-supervision-engine-lib.sh"
# shellcheck source=bin/fm-session-lock-lib.sh
. "$SCRIPT_DIR/fm-session-lock-lib.sh"

# This checkpoint is the Codex home's arm owner, so it performs the same guarded
# stale-owner reclaim the other arm owners perform before they park:
# bin/fm-claude-stop-autoarm.sh and bin/fm-turnend-guard-cursor.sh both delegate
# a lock whose recorded owner is not a live harness to bin/fm-lock.sh - the
# single acquisition owner - before they touch supervision state. Without it a
# replacement Codex session (a new harness pid recorded while the predecessor's
# anchor pid is dead) can never reclaim the home: bin/fm-supervision-host.sh
# refuses ownership before activation, starts no watcher cycle, and every later
# checkpoint in that session repeats the refusal until a session start reclaims
# the lock by hand. A live owner this session does not own, and a missing or
# malformed lock, stay exactly as uncertain as before and are never touched.
reclaim_dead_session_lock_owner() {
fm_session_lock_owned_by_self "$STATE" && return 0
fm_session_lock_inspect "$STATE"
[ "$FM_LOCK_INSPECT_STATE" = stale ] || return 0
"$SCRIPT_DIR/fm-lock.sh" >/dev/null 2>&1 || true
}

if fm_supervision_host_enabled "$CONFIG" codex; then
# The host proves ownership before it activates; a provably dead recorded
# owner is reclaimed first so a replaced session keeps supervision.
reclaim_dead_session_lock_owner
BOUND=$SECONDS_ARG
if [ -f "$STATE/.afk-contract" ] \
&& [ "$(FM_STATE_OVERRIDE="$STATE" "$SCRIPT_DIR/fm-afk-contract.sh" mode 2>/dev/null)" != quiet ]; then
Expand Down
2 changes: 1 addition & 1 deletion docs/scripts.md
Original file line number Diff line number Diff line change
Expand Up @@ -48,7 +48,7 @@ The shared no-mistakes gate lifecycle boundary is summarized in [architecture.md
| `fm-ensure-agents-md.sh` | Manually initialize project agent-memory files (see the helper's header and help) |
| `fm-guard.sh` | Warn on primary-checkout tangles, main-session pending wakes, and unhealthy supervision |
| `fm-primary-scope-lib.sh` | Shared marker-or-plain-checkout primary-home predicate for tracked hooks |
| `fm-session-lock-lib.sh` | Shared session-lock ownership from harness ancestry or a trusted Claude session id for fm-lock.sh and the Claude Stop auto-arm, plus the read-only lock inspection behind `fm-lock.sh status` and `fm-inbox.sh ready` |
| `fm-session-lock-lib.sh` | Shared session-lock ownership from harness ancestry or a trusted Claude session id for `fm-lock.sh`, arm owners, and read-only lock inspection |
| `fm-claude-stop-autoarm.sh` | Claude Stop `asyncRewake` hook owning tokenless watcher continuity with single-flight exit-2 rewake (docs/watcher-continuity.md) |
| `fm-turnend-guard.sh` | Shared primary turn-end guard predicate so no turn ends blind (docs/turnend-guard.md) |
| `fm-turnend-guard-grok.sh` | Grok Stop-hook adapter for the primary turn-end guard |
Expand Down
5 changes: 5 additions & 0 deletions docs/supervision-host.md
Original file line number Diff line number Diff line change
Expand Up @@ -77,6 +77,8 @@ Hook, plugin, extension, and checkpoint owners pass their harness as the primary
Grok's model-owned call relies on primary detection.
The host pins dispatched work to the primary's crew harness rather than the engine's.

The Claude auto-arm, the Cursor stop hook, and the Codex checkpoint own a home the way `bin/fm-lock.sh` records it, so before they run the host they reclaim a lock whose shared inspection proves a dead recorded owner through that same writer; absent, malformed, unknown, and live foreign locks are left untouched, and the OpenCode, omp, and Pi adapters instead refuse to arm a home with no live owner and name the recovery.

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Unknown lock behavior misstated If a lock records a live non-harness PID, this paragraph says Claude Stop and Cursor leave it untouched. Both hooks instead pass any numeric PID that is not a live verified harness to fm-lock.sh, which can reclaim the lock. Codex does leave it untouched. This difference matters when operators diagnose ownership or decide whether an unknown lock is safe from takeover.


Grok's arm command is fixed when the session-start block renders.
So adding or removing the file on a Grok home takes effect at the next session start.
The other owners read the file at every arm.
Expand Down Expand Up @@ -262,6 +264,9 @@ When the host loses session-lock ownership or its auto-arm generation, it stands
A host that starts without that ownership stands down before activation.
So it never stops the owner's host or watcher or releases its leases.

The host also stands down before activation when the lock is absent, malformed, unknown, or live-owned by a foreign session, because it cannot prove ownership.
The arm-owner reclaim rule above covers only a dead recorded owner before activation.

### A host that dies without a close

The host's owner retries it.
Expand Down
132 changes: 132 additions & 0 deletions tests/fm-watch-checkpoint.test.sh
Original file line number Diff line number Diff line change
Expand Up @@ -90,6 +90,8 @@ make_host_home() { # <name>
mkdir -p "$home/root/bin"
cp "$CHECKPOINT" "$home/root/bin/fm-watch-checkpoint.sh"
cp "$ROOT/bin/fm-supervision-engine-lib.sh" "$home/root/bin/fm-supervision-engine-lib.sh"
cp "$ROOT/bin/fm-session-lock-lib.sh" "$home/root/bin/fm-session-lock-lib.sh"
cp "$ROOT/bin/fm-cursor-lib.sh" "$home/root/bin/fm-cursor-lib.sh"
cat > "$home/root/bin/fm-supervision-host.sh" <<'SH'
#!/usr/bin/env bash
printf 'args=%s\nprimary=%s\npark=%s\nlimit=%s\n' "$*" "${FM_SUPERVISION_HOST_PRIMARY:-}" \
Expand Down Expand Up @@ -196,6 +198,132 @@ test_real_host_checkpoint_ends_quietly_at_its_bound() {
pass "checkpoint: the real host ends its park at the checkpoint bound as a quiet checkpoint"
}

# An unchanged, live-owned lock is left byte-identical, and an absent lock is
# never claimed: a transport reconnect that leaves ownership intact must not
# cause a takeover, and uncertainty stays uncertain.
# shellcheck disable=SC2016 # the fake harness's script expands in its own shell
test_checkpoint_leaves_an_owned_or_absent_lock_alone() {
local home home2 fakebin status
home=$(make_home reclaim-no-takeover)
: > "$home/config/supervision-host"
printf 'task=lab\n' > "$home/state/lab-task.meta"
fakebin="$TMP_ROOT/reclaim-owned-bin"
mkdir -p "$fakebin"
ln -s /bin/bash "$fakebin/codex"
status=0
FM_HOME="$home" FM_POLL=1 FM_SIGNAL_GRACE=1 FM_CHECK_INTERVAL=999999 FM_HEARTBEAT=999999 "$fakebin/codex" -c '
"$0/bin/fm-lock.sh" >/dev/null
before=$(cat "$FM_HOME/state/.lock")
"$0/bin/fm-watch-checkpoint.sh" --seconds 3 >/dev/null 2>&1
"$0/bin/fm-watch-checkpoint.sh" --seconds 3 >/dev/null 2>&1
after=$(cat "$FM_HOME/state/.lock")
[ "$before" = "$after" ] || { printf "lock changed: %s -> %s\n" "$before" "$after"; exit 7; }
[ "$before" = "$$" ] || { printf "lock is not this session: %s\n" "$before"; exit 8; }
' "$ROOT" || status=$?
expect_code 0 "$status" "two checkpoints in the owning session must leave the owner record byte-identical"
status=0
home2="$TMP_ROOT/reclaim-absent"; mkdir -p "$home2/state" "$home2/config" "$home2/data"
: > "$home2/config/supervision-host"
printf 'task=lab\n' > "$home2/state/lab-task.meta"
FM_HOME="$home2" FM_POLL=1 FM_SIGNAL_GRACE=1 FM_CHECK_INTERVAL=999999 FM_HEARTBEAT=999999 "$fakebin/codex" -c '
"$0" --seconds 3 >/dev/null 2>&1
' "$CHECKPOINT" || status=$?
assert_absent "$home2/state/.lock" "a checkpoint must never claim a home with no session lock"
pass "checkpoint: an owned or absent session lock is never rewritten or claimed"
}

# A replacement Codex session whose predecessor's recorded anchor pid is dead
# must reclaim the home through the checkpoint, exactly as the Claude Stop
# auto-arm (bin/fm-claude-stop-autoarm.sh) and the Cursor stop hook
# (bin/fm-turnend-guard-cursor.sh) do, instead of standing down on it forever.
# shellcheck disable=SC2016 # the fake harness's script expands in its own shell
test_checkpoint_reclaims_a_dead_session_lock_owner() {
local home fakebin status
home=$(make_home reclaim-dead-owner)
: > "$home/config/supervision-host"
printf 'task=lab\n' > "$home/state/lab-task.meta"
fakebin="$TMP_ROOT/reclaim-dead-bin"
mkdir -p "$fakebin"
ln -s /bin/bash "$fakebin/codex"
# The predecessor harness is gone; its recorded anchor pid is what it left.
printf '999999\n' > "$home/state/.lock"
status=0
FM_HOME="$home" FM_POLL=1 FM_SIGNAL_GRACE=1 FM_CHECK_INTERVAL=999999 FM_HEARTBEAT=999999 "$fakebin/codex" -c '
printf "%s\n" "$$" > "$FM_HOME/me"
"$0" --seconds 3
' "$CHECKPOINT" >"$home/out.txt" 2>"$home/err.txt" || status=$?
expect_code 124 "$status" "a replaced session must reclaim the dead owner and park quietly: $(cat "$home/out.txt" "$home/err.txt")"
[ "$(cat "$home/state/.lock" 2>/dev/null)" = "$(cat "$home/me" 2>/dev/null)" ] \
|| fail "the checkpoint did not reclaim the dead session-lock owner onto this harness"
assert_not_contains "$(cat "$home/out.txt")" "stood down" "a reclaimable dead owner must not stand the host down"
assert_grep ' boundary ' "$home/state/.supervision-host.log" "the reclaimed home must have run a real park"
pass "checkpoint: a replacement session reclaims a provably dead session-lock owner"
}

# The reclaim never crosses a live owner: another session's harness holding the
# lock keeps the checkpoint's stand-down and leaves the lock byte-identical.
# shellcheck disable=SC2016 # the fake harness's script expands in its own shell
test_checkpoint_never_steals_a_live_session_lock_owner() {
local home fakebin other i status
home=$(make_home reclaim-live-owner)
: > "$home/config/supervision-host"
printf 'task=lab\n' > "$home/state/lab-task.meta"
fakebin="$TMP_ROOT/reclaim-live-bin"
mkdir -p "$fakebin"
ln -s /bin/bash "$fakebin/codex"
"$fakebin/codex" -c 'sleep 20; :' &
other=$!
i=0
while [ "$i" -lt 50 ] && ! kill -0 "$other" 2>/dev/null; do
sleep 0.1
i=$((i + 1))
done
kill -0 "$other" 2>/dev/null || fail "fixture: the foreign harness did not start"
printf '%s\n' "$other" > "$home/state/.lock"
status=0
FM_HOME="$home" FM_POLL=1 FM_SIGNAL_GRACE=1 FM_CHECK_INTERVAL=999999 FM_HEARTBEAT=999999 "$fakebin/codex" -c '
"$0" --seconds 3
' "$CHECKPOINT" >"$home/out.txt" 2>"$home/err.txt" || status=$?
expect_code 1 "$status" "a live foreign owner must still stand the host down"
[ "$(cat "$home/state/.lock" 2>/dev/null)" = "$other" ] \
|| fail "the checkpoint stole a live session-lock owner"
assert_contains "$(cat "$home/out.txt")" "supervision-host stood down" "the stand-down must still say why"
kill -TERM "$other" 2>/dev/null || true
wait "$other" 2>/dev/null || true
pass "checkpoint: a live session-lock owner is never reclaimed by the checkpoint"
}

# shellcheck disable=SC2016 # the fake harness's script expands in its own shell
test_checkpoint_never_reclaims_a_live_unknown_session_lock_owner() {
local home fakebin unrelated i status
home=$(make_home reclaim-live-unknown-owner)
: > "$home/config/supervision-host"
printf 'task=lab\n' > "$home/state/lab-task.meta"
fakebin="$TMP_ROOT/reclaim-live-unknown-bin"
mkdir -p "$fakebin"
ln -s /bin/bash "$fakebin/codex"
sleep 20 &
unrelated=$!
i=0
while [ "$i" -lt 50 ] && ! kill -0 "$unrelated" 2>/dev/null; do
sleep 0.1
i=$((i + 1))
done
kill -0 "$unrelated" 2>/dev/null || fail "fixture: the unrelated process did not start"
printf '%s\n' "$unrelated" > "$home/state/.lock"
status=0
FM_HOME="$home" FM_POLL=1 FM_SIGNAL_GRACE=1 FM_CHECK_INTERVAL=999999 FM_HEARTBEAT=999999 "$fakebin/codex" -c '
"$0" --seconds 3
' "$CHECKPOINT" >"$home/out.txt" 2>"$home/err.txt" || status=$?
expect_code 1 "$status" "a live non-harness owner must stand the host down"
[ "$(cat "$home/state/.lock" 2>/dev/null)" = "$unrelated" ] \
|| fail "the checkpoint reclaimed an unknown live session-lock owner"
assert_contains "$(cat "$home/out.txt")" "supervision-host stood down" "the stand-down must still say why"
kill -TERM "$unrelated" 2>/dev/null || true
wait "$unrelated" 2>/dev/null || true
pass "checkpoint: a live non-harness session-lock owner is never reclaimed"
}

test_quiet_checkpoint_exits_124_cleanly
test_signal_passes_through_and_exits_zero
test_registered_check_uses_preserved_watcher_environment
Expand All @@ -204,3 +332,7 @@ test_host_checkpoint_bounds_the_park_by_posture
test_host_checkpoint_passes_a_handback_and_reports_a_stand_down
test_host_checkpoint_needs_the_file_and_honors_off
test_real_host_checkpoint_ends_quietly_at_its_bound
test_checkpoint_reclaims_a_dead_session_lock_owner
test_checkpoint_never_steals_a_live_session_lock_owner
test_checkpoint_never_reclaims_a_live_unknown_session_lock_owner
test_checkpoint_leaves_an_owned_or_absent_lock_alone