From 70a197fc1edc122a1327e5629ea4ba05a9d34fbc Mon Sep 17 00:00:00 2001 From: kunchenguid Date: Sat, 4 Jul 2026 12:02:02 -0700 Subject: [PATCH 1/5] fix(afk): make the away-mode daemon backend-aware for herdr bin/fm-supervise-daemon.sh discovered its supervisor pane and injected via raw tmux calls only, so /afk failed outright on a herdr-based fleet (TMUX_PANE unset, firstmate:0 fallback unresolvable). Discovery now resolves backend (tmux|herdr) and target independently, mirroring fm-backend.sh's own runtime auto-detection, with an explicit FM_SUPERVISOR_BACKEND override alongside the existing FM_SUPERVISOR_TARGET. zellij/orca refuse loudly at startup instead of misapplying tmux primitives. Injection (pane-exists probe, busy-guard, composer-guard, verified submit) now dispatches through bin/fm-backend.sh's generic primitives, adding a new fm_backend_composer_state dispatcher; the tmux path is byte-identical to before. Also fixes a pre-existing bug in fm_backend_target_exists's herdr arm (missing --session, so it silently misrouted once more than one herdr server was running) found while verifying this end to end against a real isolated herdr session. Classification, batching, max-defer, the marker contract, locks, and wake-queue handling are unchanged - this is a transport-layer fix. --- .agents/skills/afk/SKILL.md | 57 ++-- CONTRIBUTING.md | 1 + bin/fm-backend.sh | 34 +- bin/fm-supervise-daemon.sh | 200 +++++++++-- docs/herdr-backend.md | 32 ++ tests/fm-afk-inject-e2e.test.sh | 6 + tests/fm-afk-inject-herdr-e2e.test.sh | 470 ++++++++++++++++++++++++++ tests/fm-backend-herdr.test.sh | 28 ++ tests/fm-daemon.test.sh | 172 ++++++++++ 9 files changed, 956 insertions(+), 44 deletions(-) create mode 100755 tests/fm-afk-inject-herdr-e2e.test.sh diff --git a/.agents/skills/afk/SKILL.md b/.agents/skills/afk/SKILL.md index 022b57b9148..b9c77c963e8 100644 --- a/.agents/skills/afk/SKILL.md +++ b/.agents/skills/afk/SKILL.md @@ -79,17 +79,23 @@ opencode, pi, and grok). ## Busy-guard and composer guard The daemon never injects into an in-use pane. Two checks run before every -injection (shared with `fm-send.sh` via `bin/fm-tmux-lib.sh`): - -- **`pane_is_busy`** - the harness shows a busy footer (agent mid-turn). -- **`pane_input_pending`** - the cursor line holds real unsubmitted text (a +injection, dispatched through `bin/fm-backend.sh` for the supervisor's own +backend (tmux or herdr; see "Auto-discovered supervisor pane" below): + +- **`pane_is_busy`** - the harness shows a busy footer (agent mid-turn) on + tmux (shared with `fm-send.sh` via `bin/fm-tmux-lib.sh`); on herdr, tries the + native `agent.get`-backed busy state first and falls back to the same + regex-over-capture reader when that reports unknown. +- **`pane_input_pending`** - the composer holds real unsubmitted text (a human's half-typed line, or a previous injection whose Enter was swallowed). - The detector **strips the harness's composer box borders first**, so an idle - *bordered* composer (claude draws `│ > … │`) is correctly read as empty, not - pending. Without this, every idle claude pane looked like pending input and - the daemon deferred 100% of escalations (incident afk-invx-i5). - `FM_COMPOSER_IDLE_RE` still overrides empty-composer matching after border - stripping. + On tmux, the cursor-line detector **strips the harness's composer box + borders first**, so an idle *bordered* composer (claude draws `│ > … │`) is + correctly read as empty, not pending. Without this, every idle claude pane + looked like pending input and the daemon deferred 100% of escalations + (incident afk-invx-i5). `FM_COMPOSER_IDLE_RE` still overrides empty-composer + matching after border stripping. On herdr, the equivalent structural + border-row classifier (`fm_backend_herdr_composer_state`, + docs/herdr-backend.md) plays the same role. Either condition defers the injection; the buffered escalation survives in `state/.subsuper-escalations` and is retried on the next housekeeping tick. In @@ -108,7 +114,8 @@ So a guard false-positive becomes a visible stall, never an unbounded silent no- ## Submit model -The digest is typed **once** via `send-keys -l`, then submitted with Enter and +The digest is typed **once** (`send-keys -l` on tmux, `pane send-text` on +herdr - both literal, non-submitting sends), then submitted with Enter and **verified**: Enter is retried (Enter only, never a retype) until the composer clears. A submit "landed" only when the composer is confirmed empty afterward, using @@ -181,11 +188,12 @@ the marker lets firstmate distinguish it from a real captain message. durable `state/.subsuper-inject-wedged` marker, and a status-line flash. A composer false-positive surfaces as a visible stall, never an unbounded silent no-op. -- **Verified type-once submit model** - the digest is typed once via - `send-keys -l`, then submitted with Enter and verified. Enter is retried, - Enter only and never a retype, until the composer is confirmed empty. That - empty composer is the acknowledgement that the submit landed, using the same - dim-ghost-aware and border-aware detector so a ghost-only or bordered-empty +- **Verified type-once submit model** - the digest is typed once (`send-keys -l` + on tmux, `pane send-text` on herdr), then submitted with Enter and verified. + Enter is retried, Enter only and never a retype, until the composer is + confirmed empty. That empty composer is the acknowledgement that the submit + landed, using the same dim-ghost-aware and border-aware detector (tmux) or + structural border-row classifier (herdr) so a ghost-only or bordered-empty claude composer counts as submitted rather than a false swallowed Enter. - **Marker strip** - `strip_injection_marker` removes the sentinel prefix before classification or relay, so the digest text firstmate sees is clean. @@ -194,10 +202,19 @@ the marker lets firstmate distinguish it from a real captain message. - **Dedupe across signal/stale/scan** - `classify_signal` and `classify_stale` both check the seen-status marker before escalating, so a status escalated by one path is not re-escalated by another in the same digest. -- **Auto-discovered supervisor pane** - the daemon resolves its injection target - from `FM_SUPERVISOR_TARGET`, then `$TMUX_PANE`, then a `firstmate:0` fallback - with a warning. The resolution source is logged at startup so a - wrong-but-resolving fallback is detectable. +- **Auto-discovered supervisor pane** - the daemon resolves its own BACKEND + (tmux vs herdr) and TARGET independently, mirroring + `bin/fm-backend.sh`'s own runtime auto-detection. Backend: `FM_SUPERVISOR_BACKEND` + override, then `$TMUX_PANE` set (tmux), then `$HERDR_ENV=1` with + `$HERDR_PANE_ID` present (herdr), then a tmux fallback. Target: + `FM_SUPERVISOR_TARGET` override (a tmux target or a herdr + `":"` target), then `$TMUX_PANE`, then + `"${HERDR_SESSION:-default}:${HERDR_PANE_ID}"` under herdr, then a + `firstmate:0` fallback with a warning. Both resolution sources are logged at + startup so a wrong-but-resolving fallback is detectable. zellij and orca are + not yet supported as a supervisor backend; the daemon refuses loudly at + startup instead of misapplying tmux primitives to a pane that isn't one + (docs/herdr-backend.md "Away-mode daemon: herdr supervisor-pane support"). ## Reliability properties diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index 9e5883d356f..0310dcfdf0e 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -76,6 +76,7 @@ tests/fm-send-secondmate-marker.test.sh # fm-send from-firstmate marker for ki tests/fm-wake-daemon-lifecycle-e2e.test.sh # watcher + daemon lifecycle e2e: restart catch-up, batching, dedupe, stale-pane routing, and digest injection tests/fm-composer-ghost.test.sh # dim-ghost stripping, ghost-only composer detection, and escape-free peek tests tests/fm-afk-inject-e2e.test.sh # private-socket end-to-end test of the afk injection path (partial-input deferral, swallowed-Enter retry) +tests/fm-afk-inject-herdr-e2e.test.sh # real-herdr end-to-end test of the afk daemon's herdr transport, on an isolated throwaway HERDR_SESSION: partial-input deferral, swallowed-Enter retry, a normal digest, and the max-defer wedge alarm on a persistently pending composer tests/fm-bootstrap.test.sh # bootstrap dependency, feature-probe, and crew-dispatch reporting tests tests/fm-session-start.test.sh # fm-session-start.sh: ABSENT vs empty-vs-present digest files, lock-refusal read-only path skipping every mutating step, diagnostics-first section ordering, status-tail bounding, tmux/herdr endpoint liveness, and composition of the real fm-lock/fm-bootstrap/fm-wake-drain scripts tests/fm-grok-harness.test.sh # grok adapter spawn hook, token guard, teardown cleanup, and session-lock detection tests diff --git a/bin/fm-backend.sh b/bin/fm-backend.sh index 2fde1e89128..e74cbdf1665 100644 --- a/bin/fm-backend.sh +++ b/bin/fm-backend.sh @@ -417,6 +417,30 @@ fm_backend_busy_state() { # esac } +# fm_backend_composer_state: classify the composer/input row of as +# empty|pending|unknown - the SUBMIT-side classifier each adapter already uses +# internally to verify fm_backend_send_text_submit, exposed generically so a +# caller other than the send path (the away-mode daemon's supervisor-pane +# pending-input guard, bin/fm-supervise-daemon.sh) can ask the same question +# without duplicating per-backend composer-reading logic. tmux and herdr both +# expose a named classifier already (fm_tmux_composer_state, +# fm_backend_herdr_composer_state), as does orca +# (fm_backend_orca_composer_state); zellij's submit path uses an internal +# content-diff approach with no separately named classifier, so it reports +# unknown here - callers fall back to their own policy, exactly as an unknown +# fm_backend_busy_state already does. +fm_backend_composer_state() { # -> empty|pending|unknown + local backend=$1 + shift + fm_backend_source "$backend" || { printf 'unknown'; return 0; } + case "$backend" in + tmux) fm_tmux_composer_state "$@" ;; + herdr) fm_backend_herdr_composer_state "$@" ;; + orca) fm_backend_orca_composer_state "$@" ;; + *) printf 'unknown' ;; + esac +} + # fm_backend_target_exists: cheap, READ-ONLY existence check - does the # recorded TARGET endpoint still exist on BACKEND? Never starts a server or # session: for herdr this deliberately queries the pane directly instead of @@ -440,7 +464,15 @@ fm_backend_target_exists() { # [expected-label] session=${target%%:*} pane=${target#*:} [ -n "$session" ] && [ -n "$pane" ] && [ "$pane" != "$target" ] || return 1 - HERDR_SESSION="$session" herdr pane get "$pane" >/dev/null 2>&1 + # fm_backend_herdr_cli (not a raw HERDR_SESSION-only call): verified + # empirically (docs/herdr-backend.md "Session targeting") that the bare + # env var alone is NOT reliably honored once another herdr server is + # already bound on the machine - it silently queries whatever server IS + # running instead. fm_backend_herdr_cli appends the required --session + # flag on top, so this check is correctly scoped even when the caller's + # own ambient session (e.g. the primary firstmate's default session) is + # a DIFFERENT one than the target's. + fm_backend_herdr_cli "$session" pane get "$pane" >/dev/null 2>&1 ;; zellij) fm_backend_source zellij || return 1 diff --git a/bin/fm-supervise-daemon.sh b/bin/fm-supervise-daemon.sh index dad40386b4f..112a243dbe3 100755 --- a/bin/fm-supervise-daemon.sh +++ b/bin/fm-supervise-daemon.sh @@ -56,9 +56,24 @@ # Usage: fm-supervise-daemon.sh # Long-lived background loop. Normally started by the /afk skill, which # sets state/.afk first. Env knobs: -# FM_SUPERVISOR_TARGET supervisor tmux target (override; otherwise -# auto-discovered from TMUX_PANE, then -# firstmate:0 fallback) +# FM_SUPERVISOR_TARGET supervisor pane target (override; otherwise +# auto-discovered per backend - $TMUX_PANE +# under tmux, ":" from +# $HERDR_PANE_ID under herdr - then +# firstmate:0 fallback). Accepts either a +# tmux target or a herdr ":" +# target; which one it's read as is decided by +# FM_SUPERVISOR_BACKEND (below), independently. +# FM_SUPERVISOR_BACKEND supervisor pane BACKEND (tmux|herdr; +# override; otherwise auto-discovered the same +# way bin/fm-backend.sh's fm_backend_detect +# resolves the runtime firstmate itself is +# executing inside - $TMUX_PANE selects tmux, +# $HERDR_ENV=1 selects herdr - falling back to +# tmux). zellij and orca are not yet supported +# as a supervisor backend; the daemon refuses +# loudly at startup rather than trying tmux +# primitives against a non-tmux pane. # FM_INJECT_SKIP |-prefixes force-self-handle bypassing # classification (default "heartbeat"); empty # disables. Use sparingly: it overrides the @@ -122,6 +137,18 @@ FM_HOME="${FM_HOME:-${FM_ROOT_OVERRIDE:-$FM_ROOT}}" # --- tunables --------------------------------------------------------------- FM_SUPERVISOR_TARGET_DEFAULT="firstmate:0" +# Fallback BACKEND paired with the fallback target above: "firstmate:0" is a +# tmux session:window name, so the bare fallback (nothing configured, nothing +# detected) assumes tmux - matching this daemon's pre-herdr-support behavior +# byte-for-byte when run outside both tmux and herdr. +FM_SUPERVISOR_BACKEND_DEFAULT="tmux" +# Supervisor backends this daemon knows how to inject into today. zellij and +# orca are real backends elsewhere in firstmate (bin/fm-backend.sh) but this +# daemon has no verified composer/busy primitives wired up for them yet - see +# docs/herdr-backend.md and AGENTS.md section 4's harness-verification +# discipline. Selecting one refuses loudly at startup instead of silently +# running tmux primitives against a pane that is not a tmux pane. +FM_SUPERVISOR_SUPPORTED_BACKENDS="tmux herdr" INJECT_SKIP_DEFAULT="heartbeat" STALE_ESCALATE_SECS_DEFAULT=240 ESCALATE_BATCH_SECS_DEFAULT=90 @@ -247,11 +274,20 @@ _collapse_newlines() { # } # Auto-discover the supervisor pane at startup. Priority: -# 1. FM_SUPERVISOR_TARGET env (explicit override) — caller passes it in. +# 1. FM_SUPERVISOR_TARGET env (explicit override) — caller passes it in; +# may be a tmux target or a herdr ":" target (paired +# with discover_supervisor_backend, below, to know which). # 2. $TMUX_PANE — tmux sets this in every pane's environment; inherited by # the daemon when the /afk skill launches it from firstmate's own pane. -# 3. firstmate:0 — legacy fallback (may not resolve if the session is named -# differently). The caller logs a warning in that case. +# 3. $HERDR_ENV=1 + $HERDR_PANE_ID — herdr injects both into every process +# it manages a pane for (docs/herdr-backend.md); the daemon composes the +# ":" target string the herdr adapter expects from +# $HERDR_SESSION (defaulting to "default", mirroring +# bin/backends/herdr.sh's fm_backend_herdr_session) and $HERDR_PANE_ID. +# Checked after $TMUX_PANE so a tmux pane nested inside herdr still +# resolves to tmux, matching fm_backend_detect's innermost-first rule. +# 4. firstmate:0 — legacy tmux fallback (may not resolve if the session is +# named differently). The caller logs a warning in that case. # Returns the resolved target on stdout; returns 1 if only the fallback is left # AND the fallback does not resolve to a live pane. discover_supervisor_target() { @@ -263,10 +299,40 @@ discover_supervisor_target() { printf '%s' "$TMUX_PANE" return 0 fi + if [ "${HERDR_ENV:-}" = "1" ] && [ -n "${HERDR_PANE_ID:-}" ]; then + printf '%s:%s' "${HERDR_SESSION:-default}" "$HERDR_PANE_ID" + return 0 + fi printf '%s' "$FM_SUPERVISOR_TARGET_DEFAULT" return 1 } +# Auto-discover the supervisor's BACKEND at startup - independent of the +# target string above, so an explicit FM_SUPERVISOR_TARGET override still +# needs to know which primitives (tmux vs herdr) to dispatch through. Priority +# mirrors discover_supervisor_target and bin/fm-backend.sh's fm_backend_detect: +# 1. FM_SUPERVISOR_BACKEND env (explicit override). +# 2. $TMUX_PANE set — tmux. +# 3. $HERDR_ENV=1 (with $HERDR_PANE_ID present) — herdr. +# 4. FM_SUPERVISOR_BACKEND_DEFAULT (tmux) — matches the target fallback above. +# Returns the resolved backend on stdout; returns 1 if only the fallback is left. +discover_supervisor_backend() { + if [ -n "${FM_SUPERVISOR_BACKEND:-}" ]; then + printf '%s' "$FM_SUPERVISOR_BACKEND" + return 0 + fi + if [ -n "${TMUX_PANE:-}" ]; then + printf 'tmux' + return 0 + fi + if [ "${HERDR_ENV:-}" = "1" ] && [ -n "${HERDR_PANE_ID:-}" ]; then + printf 'herdr' + return 0 + fi + printf '%s' "$FM_SUPERVISOR_BACKEND_DEFAULT" + return 1 +} + # --- classification helpers (PURE: no side effects, testable) --------------- # last_status_line, status_is_captain_relevant, window_to_task, and # scan_captain_relevant_statuses come from bin/fm-classify-lib.sh (sourced above), @@ -409,8 +475,37 @@ mark_escalated_seen() { # # strips the harness's composer box borders, so a ghost-only or idle bordered # claude composer ("│ > … │") is correctly read as empty, not pending (incidents # afk-invx-i5 and composer-robust). -pane_is_busy() { fm_pane_is_busy "$@"; } # -pane_input_pending() { fm_pane_input_pending "$@"; } # +# pane_is_busy / pane_input_pending: BACKEND-AWARE now (previously tmux-only +# direct calls). defaults to tmux when omitted, so every existing +# caller/test that passes only is unaffected. Dispatch goes through +# bin/fm-backend.sh's generic per-backend primitives (fm_backend_busy_state, +# fm_backend_capture, fm_backend_composer_state) rather than hand-rolling a +# case statement here, mirroring the same fallback pattern +# stale_window_is_busy already uses for per-task panes: try the backend's +# native busy-state first, and fall back to the shared regex-over-capture +# reader when the backend reports "unknown" (tmux has no native busy-state +# primitive, so it always takes this fallback path - byte-identical to the +# pre-existing fm_pane_is_busy, since fm_backend_capture's tmux arm runs the +# exact same `tmux capture-pane -p -t -S -40`). +pane_is_busy() { # [backend] + local target=$1 backend=${2:-tmux} bs tail40 + bs=$(fm_backend_busy_state "$backend" "$target" 2>/dev/null) + case "$bs" in + busy) return 0 ;; + idle) return 1 ;; + esac + tail40=$(fm_backend_capture "$backend" "$target" 40 2>/dev/null) || return 1 + printf '%s' "$tail40" | grep -v '^[[:space:]]*$' | tail -6 \ + | grep -qiE "${FM_BUSY_REGEX:-$FM_TMUX_BUSY_REGEX_DEFAULT}" +} + +# pane_input_pending: dispatches through fm_backend_composer_state, which for +# tmux calls the exact same fm_tmux_composer_state this function called +# directly before - byte-identical for the default/omitted-backend case. +pane_input_pending() { # [backend] + local target=$1 backend=${2:-tmux} + [ "$(fm_backend_composer_state "$backend" "$target" 2>/dev/null)" = pending ] +} task_window_backend() { # local win=$1 state=$2 task meta @@ -466,7 +561,7 @@ escalate_flush() { # # the supervisor client's status line. Nothing is lost — the buffer and the # wake-queue both survive — but the stall stops being invisible. inject_wedge_alarm() { # - local state=$1 age=$2 marker target + local state=$1 age=$2 marker target backend marker="$state/.subsuper-inject-wedged" # Re-alarm at most once per max-defer window so a long wedge does not spam. if [ "$(_file_age "$marker")" -lt "${FM_MAX_DEFER_SECS:-$MAX_DEFER_SECS_DEFAULT}" ]; then @@ -479,7 +574,14 @@ inject_wedge_alarm() { # cat "$state/.subsuper-escalations" 2>/dev/null } > "$marker" 2>/dev/null || true target="${FM_SUPERVISOR_TARGET:-$FM_SUPERVISOR_TARGET_DEFAULT}" - tmux display-message -t "$target" "fm: away-mode escalations WEDGED ${age}s — see $marker" 2>/dev/null || true + backend="${FM_SUPERVISOR_BACKEND:-$FM_SUPERVISOR_BACKEND_DEFAULT}" + # Best-effort status-line flash. tmux's display-message is a client-side OSD + # with no herdr equivalent; the log line + durable marker above are already + # the primary, backend-independent signal, so a non-tmux backend just skips + # this cosmetic extra rather than attempting an unsupported call. + if [ "$backend" = tmux ]; then + tmux display-message -t "$target" "fm: away-mode escalations WEDGED ${age}s — see $marker" 2>/dev/null || true + fi } _oldest_line_age() { # -> seconds since the oldest buffered item first arrived (sidecar epoch) @@ -614,7 +716,7 @@ window_for_task() { # [state] # line, or a previous injection's unsent text), defer entirely - injecting # would merge with the human's text. inject_msg() { # [state] - local msg=$1 state target retries sleep_s verdict + local msg=$1 state target backend retries sleep_s verdict state="${2:-$(_state_root)}" # (1) Presence-gate: inject ONLY when afk is active. When afk is off, the # daemon self-handles and stays quiet; firstmate drives the normal always-on @@ -627,17 +729,23 @@ inject_msg() { # [state] msg=$(_collapse_newlines "$msg") msg="${FM_INJECT_MARK}${msg}" target="${FM_SUPERVISOR_TARGET:-$FM_SUPERVISOR_TARGET_DEFAULT}" - tmux display-message -p -t "$target" '#{pane_id}' >/dev/null 2>&1 || return 1 + # BACKEND-AWARE (previously a raw `tmux display-message` pane-exists probe): + # dispatches through bin/fm-backend.sh so a herdr supervisor pane is checked + # via the herdr adapter instead of always assuming tmux. Falls back to tmux + # when unset (sourced/test contexts that never ran fm_super_main's startup + # discovery), matching this function's pre-existing default assumption. + backend="${FM_SUPERVISOR_BACKEND:-tmux}" + fm_backend_target_exists "$backend" "$target" || return 1 # (3) Busy-guard: never inject into an in-use pane. Two checks: # a) pane_is_busy: the harness shows a busy footer (agent mid-turn). # b) pane_input_pending: the cursor line has real unsubmitted text after # dim/faint ghost text and borders are ignored (a human's half-typed line, # or a previous injection whose Enter was swallowed). - if pane_is_busy "$target"; then + if pane_is_busy "$target" "$backend"; then log "inject deferred: supervisor pane busy (agent mid-turn)" return 1 fi - if pane_input_pending "$target"; then + if pane_input_pending "$target" "$backend"; then log "inject deferred: supervisor pane has pending input (non-empty composer)" return 1 fi @@ -645,9 +753,12 @@ inject_msg() { # [state] # retype) via the shared submit primitive. Success = the composer is confirmed # EMPTY afterward (the text was consumed). An unconfirmed/unknown pane does NOT # count as delivered, so the buffer is preserved (strict) rather than cleared. + # Dispatches through fm_backend_send_text_submit (bin/fm-backend.sh): for + # backend=tmux this calls fm_backend_tmux_send_text_submit, a verbatim + # re-export of fm_tmux_submit_core - byte-identical to calling it directly. retries=${FM_INJECT_CONFIRM_RETRIES:-$INJECT_CONFIRM_RETRIES_DEFAULT} sleep_s=${FM_INJECT_CONFIRM_SLEEP:-$INJECT_CONFIRM_SLEEP_DEFAULT} - verdict=$(fm_tmux_submit_core "$target" "$msg" "$retries" "$sleep_s" "$sleep_s") + verdict=$(fm_backend_send_text_submit "$backend" "$target" "$msg" "$retries" "$sleep_s" "$sleep_s") if [ "$verdict" = empty ]; then return 0 # Composer cleared → submit confirmed. fi @@ -772,9 +883,46 @@ fm_super_main() { fi echo "$$" > "$PIDFILE" + # --- auto-discover the supervisor BACKEND (tmux vs herdr) first ----------- + # Priority: FM_SUPERVISOR_BACKEND override > $TMUX_PANE (tmux) > $HERDR_ENV=1 + # (herdr) > tmux fallback. Resolved before the target below, since target + # discovery composes a herdr ":" string using the same + # $HERDR_PANE_ID/$HERDR_SESSION markers this checks. Exporting the result + # into FM_SUPERVISOR_BACKEND makes inject_msg/pane_is_busy/pane_input_pending + # (which read that env var) dispatch through the right backend without an + # extra global thread-through. + local discovered_backend backend_source + backend_source="FM_SUPERVISOR_BACKEND" + if [ -z "${FM_SUPERVISOR_BACKEND:-}" ]; then + if [ -n "${TMUX_PANE:-}" ]; then + backend_source="TMUX_PANE" + elif [ "${HERDR_ENV:-}" = "1" ] && [ -n "${HERDR_PANE_ID:-}" ]; then + backend_source="HERDR_ENV" + else + backend_source="FALLBACK($FM_SUPERVISOR_BACKEND_DEFAULT)" + fi + fi + discovered_backend=$(discover_supervisor_backend) || true + FM_SUPERVISOR_BACKEND="$discovered_backend" + local BACKEND="$FM_SUPERVISOR_BACKEND" + + # --- refuse an unsupported supervisor backend loudly, before ever trying a + # tmux/herdr-specific call against it (zellij/orca have no verified + # composer/busy primitives wired up for this daemon yet - AGENTS.md section 4 + # harness-verification discipline). This is the "clear refusal" the task + # calls for, instead of a confusing "does not resolve to a tmux pane" error. + if ! fm_backend_list_contains "$FM_SUPERVISOR_SUPPORTED_BACKENDS" "$BACKEND"; then + echo "error: away-mode daemon does not support supervisor backend '$BACKEND' yet (supported: $FM_SUPERVISOR_SUPPORTED_BACKENDS); set FM_SUPERVISOR_BACKEND=tmux|herdr and FM_SUPERVISOR_TARGET to run firstmate's own pane under a supported backend" >&2 + log "startup failed: unsupported supervisor backend '$BACKEND' (source=$backend_source)" + fm_lock_release "$LOCK" 2>/dev/null || true + rm -f "$PIDFILE" 2>/dev/null || true + exit 1 + fi + # --- auto-discover the supervisor target (the pane running firstmate) ----- - # Priority: FM_SUPERVISOR_TARGET override > $TMUX_PANE (inherited from the - # pane that launched the daemon, normally firstmate's own) > firstmate:0 + # Priority: FM_SUPERVISOR_TARGET override > $TMUX_PANE (tmux; inherited from + # the pane that launched the daemon, normally firstmate's own) > + # $HERDR_PANE_ID (herdr, composed into ":") > firstmate:0 # fallback. Exporting the result into FM_SUPERVISOR_TARGET makes inject_msg # (which reads that env var) use the discovered pane without an extra global. local discovered target_source @@ -782,6 +930,8 @@ fm_super_main() { if [ -z "${FM_SUPERVISOR_TARGET:-}" ]; then if [ -n "${TMUX_PANE:-}" ]; then target_source="TMUX_PANE" + elif [ "${HERDR_ENV:-}" = "1" ] && [ -n "${HERDR_PANE_ID:-}" ]; then + target_source="HERDR_ENV(HERDR_PANE_ID)" else target_source="FALLBACK(firstmate:0)" fi @@ -789,15 +939,19 @@ fm_super_main() { if discovered=$(discover_supervisor_target); then : # resolved cleanly else - echo "warn: could not auto-discover supervisor pane (no FM_SUPERVISOR_TARGET or TMUX_PANE); falling back to '$discovered' — verify this is firstmate's pane" >&2 + echo "warn: could not auto-discover supervisor pane (no FM_SUPERVISOR_TARGET, TMUX_PANE, or HERDR_ENV/HERDR_PANE_ID); falling back to '$discovered' — verify this is firstmate's pane" >&2 fi FM_SUPERVISOR_TARGET="$discovered" local TARGET="$FM_SUPERVISOR_TARGET" # --- validate supervisor target at startup (a missing target is a typo) --- - if ! tmux display-message -p -t "$TARGET" '#{pane_id}' >/dev/null 2>&1; then - echo "error: supervisor target '$TARGET' does not resolve to a tmux pane; set FM_SUPERVISOR_TARGET" >&2 - log "startup failed: target '$TARGET' not found" + # Dispatches through bin/fm-backend.sh instead of a raw `tmux display-message` + # probe, so a herdr supervisor pane is checked via the herdr adapter; for + # backend=tmux this runs the exact same `tmux display-message -p -t "$TARGET" + # '#{pane_id}'` call as before. + if ! fm_backend_target_exists "$BACKEND" "$TARGET"; then + echo "error: supervisor target '$TARGET' does not resolve to a $BACKEND pane; set FM_SUPERVISOR_TARGET" >&2 + log "startup failed: target '$TARGET' not found (backend=$BACKEND)" fm_lock_release "$LOCK" 2>/dev/null || true rm -f "$PIDFILE" 2>/dev/null || true exit 1 @@ -805,7 +959,7 @@ fm_super_main() { local afk_status="off" afk_active "$STATE" && afk_status="on" - log "daemon starting (pid $$); target=$TARGET; target_source=$target_source; afk=$afk_status; inject_skip='${FM_INJECT_SKIP:-$INJECT_SKIP_DEFAULT}'; stale_escalate=${FM_STALE_ESCALATE_SECS:-$STALE_ESCALATE_SECS_DEFAULT}s; batch=${FM_ESCALATE_BATCH_SECS:-$ESCALATE_BATCH_SECS_DEFAULT}s" + log "daemon starting (pid $$); target=$TARGET; target_source=$target_source; backend=$BACKEND; backend_source=$backend_source; afk=$afk_status; inject_skip='${FM_INJECT_SKIP:-$INJECT_SKIP_DEFAULT}'; stale_escalate=${FM_STALE_ESCALATE_SECS:-$STALE_ESCALATE_SECS_DEFAULT}s; batch=${FM_ESCALATE_BATCH_SECS:-$ESCALATE_BATCH_SECS_DEFAULT}s" # --- shutdown: flush buffered escalations, reap child, release lock ------- local WATCHER_PID="" CUR_TMP="" @@ -861,7 +1015,7 @@ fm_super_main() { # has nowhere to go, and firstmate itself is the consumer of escalations. # Catch-up signals persist in state/*.status and flow on the next run, so # this delays rather than loses work. - if ! tmux display-message -p -t "$TARGET" '#{pane_id}' >/dev/null 2>&1; then + if ! fm_backend_target_exists "$BACKEND" "$TARGET"; then log "warn: supervisor target '$TARGET' gone; backing off ${INJECT_FAIL_SLEEP}s, will retry" # Flush is pointless with no pane; preserve any buffered escalations. sleep "$INJECT_FAIL_SLEEP" diff --git a/docs/herdr-backend.md b/docs/herdr-backend.md index db12d7bc797..c8a096d778a 100644 --- a/docs/herdr-backend.md +++ b/docs/herdr-backend.md @@ -336,6 +336,38 @@ This exercises the fm-spawn.sh-level behavior the adapter-primitive smoke test c All ten assertions passed on the real binary on the first run. As with every other real-herdr test in this document, the default session's own workspace state (label, tab count) was confirmed byte-identical immediately before and immediately after the run. +## Away-mode daemon: herdr supervisor-pane support + +`bin/fm-supervise-daemon.sh` (the `/afk` sub-supervisor) was tmux-only through 2026-07-03: it discovered its own injection target from `$TMUX_PANE`, and injected via raw `tmux display-message`/`tmux capture-pane`/`tmux send-keys` calls with no backend indirection. +On a herdr-based fleet (firstmate itself running with `HERDR_ENV=1`, no `$TMUX_PANE`), this failed outright at startup: `TMUX_PANE` is unset, so discovery fell through to the legacy `firstmate:0` fallback, which then failed the tmux pane-exists probe and refused to start. + +The fix is transport-layer only - discovery, injection, and the busy/composer guards now dispatch through the SAME `bin/fm-backend.sh` primitives every other backend-aware script already uses (`fm_backend_target_exists`, `fm_backend_busy_state`, `fm_backend_capture`, `fm_backend_send_text_submit`, and the new `fm_backend_composer_state` dispatcher added alongside this work). +Classification policy, batching, the max-defer escape, the `FM_INJECT_MARK` sentinel contract, locks, and wake-queue handling are all unchanged. + +**Discovery.** `FM_SUPERVISOR_TARGET` remains the explicit override, now accepting either a tmux target or a herdr `":"` target. +A new `FM_SUPERVISOR_BACKEND` override (`tmux`|`herdr`) resolves independently, mirroring `bin/fm-backend.sh`'s own `fm_backend_detect`: `$TMUX_PANE` set selects tmux (even nested inside herdr, matching the innermost-first rule); `$HERDR_ENV=1` with `$HERDR_PANE_ID` present selects herdr, composing the target as `"${HERDR_SESSION:-default}:${HERDR_PANE_ID}"`; absent both, the daemon falls back to tmux/`firstmate:0`, byte-identical to its pre-herdr-support behavior. +zellij and orca are not yet supported as a supervisor backend - the daemon refuses loudly at startup (`FM_SUPERVISOR_SUPPORTED_BACKENDS="tmux herdr"`) rather than misapplying tmux primitives to a pane that isn't a tmux pane. + +**Injection dispatch.** `inject_msg`'s pane-exists probe, busy-guard (`pane_is_busy`), composer-guard (`pane_input_pending`), and verified submit all take an optional `` argument (defaulting to `tmux` when omitted, so every pre-existing caller/test is unaffected) and route through the generic dispatchers instead of calling `tmux` directly. +For `backend=tmux` every dispatch resolves to the exact same underlying call as before (`fm_backend_capture`'s tmux arm runs the identical `tmux capture-pane -p -t -S -40`; `fm_backend_tmux_send_text_submit` re-exports `fm_tmux_submit_core` verbatim), so tmux behavior is unchanged byte-for-byte. +For `backend=herdr`, busy detection tries the native `agent.get`-backed `fm_backend_herdr_busy_state` first and falls back to the shared regex-over-capture reader on `unknown`, exactly mirroring the per-task stale-pane busy check `bin/fm-supervise-daemon.sh`'s `stale_window_is_busy` already used; composer/pending detection and the verified submit reuse `fm_backend_herdr_composer_state`/`fm_backend_herdr_send_text_submit` unchanged. +The wedge alarm's supervisor-client status-line flash (`tmux display-message ...`) is tmux-only cosmetic UI with no herdr equivalent; it is skipped for non-tmux backends, while the ERROR log line and the durable `state/.subsuper-inject-wedged` marker (the actual signal) are backend-independent and unaffected. + +**A pre-existing bug this surfaced: `fm_backend_target_exists`'s herdr arm.** Before this task, that function's herdr case called `HERDR_SESSION="$session" herdr pane get "$pane"` directly, WITHOUT the `--session` flag. +Per "Session targeting" above, `HERDR_SESSION` alone is not reliably honored once another herdr server is already bound on the machine - it silently falls back to whatever server IS running. +This function happened to look correct in every prior test because those tests only ever had ONE herdr server running at a time. +Verifying the away-mode daemon end to end against a real, isolated `HERDR_SESSION` - while the ambient default herdr session was also running (the normal shape of an actual firstmate fleet) - reproduced it directly: the daemon's own startup target-exists check spuriously refused a genuinely live pane in the isolated session because the ambient default session's socket answered instead. +Fixed by routing through `fm_backend_herdr_cli` (which appends `--session` on top of the env var) instead of the raw ad hoc call. +This fix is backend-plumbing, not daemon-specific: it also corrects the same liveness check other callers use (`bin/fm-session-start.sh`'s per-task endpoint-liveness digest read). + +**Empirical verification (real herdr, isolated session only).** `tests/fm-afk-inject-herdr-e2e.test.sh` mirrors `tests/fm-afk-inject-e2e.test.sh`'s three scenarios (human-partial-input deferral, swallowed-Enter retry, a normal single digest) plus a fourth (a persistently pending composer that never clears must alarm via `state/.subsuper-inject-wedged`, preserve the buffer, and never crash the daemon) against a real, throwaway, NEVER-default `HERDR_SESSION`, torn down with `herdr_safe_stop_and_delete` exactly like `tests/fm-backend-herdr-smoke.test.sh`. +The "supervisor pane" is a tiny deterministic bash loop drawing a bordered composer row (not a real harness), matching the structural classifier `fm_backend_herdr_composer_state` expects; a thin `herdr` PATH shim swallows exactly one `pane send-keys enter` call to simulate the swallowed-Enter scenario, since herdr's real CLI has no built-in way to drop a keystroke. + +Building that test surfaced one more real finding worth recording for anyone writing a similar herdr-driven composer script: `tput cols`, called from WITHIN a script launched into a herdr pane via `pane run`/`send-text`, reported a stale/default `80` regardless of the pane's actual width, while an interactively-typed one-off `tput cols` in the same pane correctly reported its real width (54, in the environment this was verified in). +A composer redraw that trusts `tput cols` for its own line-wrapping math can therefore silently overflow the pane's real width and wrap across two terminal rows - breaking the structural single-row border classifier's assumption (the digest looked "concatenated with itself" because the guard never fired: the composer read `unknown` instead of `pending`, so the busy/composer guard did not defer a second attempt). +The test's composer script works around this with a hardcoded conservative width rather than trusting `tput cols` in this execution context. +This is a test-harness-only concern - `fm_backend_herdr_composer_state` and `fm_backend_herdr_send_text_submit` themselves are unchanged and were reverified correct once the test's own composer script stayed within the pane's real width - but it is a sharp edge for any future herdr-launched interactive script that computes its own layout from `tput`. + ## Known gaps and follow-up notes - **No `events.subscribe` native push.** The busy-state semantic read (`agent.get`) is consumed through the EXISTING `fm-watch.sh` poll loop (same 15-second cadence as every other window), not a persistent async subscriber pushing events directly into the wake queue. diff --git a/tests/fm-afk-inject-e2e.test.sh b/tests/fm-afk-inject-e2e.test.sh index 3f9791735af..415c7db45af 100755 --- a/tests/fm-afk-inject-e2e.test.sh +++ b/tests/fm-afk-inject-e2e.test.sh @@ -19,6 +19,11 @@ # A tmux shim first on PATH redirects the daemon's bare `tmux` calls to the # private socket. The daemon points at a throwaway state dir (FM_STATE_OVERRIDE) # and the test pane (FM_SUPERVISOR_TARGET). Nothing touches the live fleet. +# FM_SUPERVISOR_BACKEND=tmux is passed explicitly (not left to auto-detection): +# this test's own process may itself be running inside herdr (HERDR_ENV=1 is +# inherited by every process herdr manages a pane for), which would otherwise +# leak into the spawned daemon subprocess and misdetect backend=herdr against +# what is actually a tmux pane on the private socket. # # Assert on submitted CONTENT (logged verbatim by the supervisor pane), not pane # appearance — terminal line-wrapping looks like newlines but isn't. @@ -154,6 +159,7 @@ start_daemon() { PATH="$TMUX_SHIM_DIR:$PATH" \ FM_STATE_OVERRIDE="$STATE_DIR" \ FM_SUPERVISOR_TARGET="$SUPERVISOR_PANE" \ + FM_SUPERVISOR_BACKEND=tmux \ FM_ESCALATE_BATCH_SECS=0 \ FM_HOUSEKEEPING_TICK=1 \ FM_POLL=1 \ diff --git a/tests/fm-afk-inject-herdr-e2e.test.sh b/tests/fm-afk-inject-herdr-e2e.test.sh new file mode 100755 index 00000000000..0e9ba8f8b60 --- /dev/null +++ b/tests/fm-afk-inject-herdr-e2e.test.sh @@ -0,0 +1,470 @@ +#!/usr/bin/env bash +# tests/fm-afk-inject-herdr-e2e.test.sh - real-herdr end-to-end test for the +# away-mode daemon's herdr transport (bin/fm-supervise-daemon.sh), the herdr +# counterpart of tests/fm-afk-inject-e2e.test.sh's private-socket tmux e2e. +# Mirrors tests/fm-backend-herdr-smoke.test.sh and tests/herdr-test-safety.sh's +# isolation patterns: everything runs on a throwaway, named, NEVER-default +# HERDR_SESSION, torn down with herdr_safe_stop_and_delete. Skips cleanly when +# herdr or jq is not installed. +# +# Unlike the tmux e2e (which redirects a bare `tmux` PATH shim to a private +# socket), herdr already supports named-session isolation via --session, so no +# PATH redirection is needed for the happy path - the daemon is simply pointed +# at FM_SUPERVISOR_BACKEND=herdr, FM_SUPERVISOR_TARGET=":", +# and HERDR_SESSION="". A thin herdr SHIM is still used, +# but only to simulate a swallowed Enter (Scenario B) - herdr's real CLI has no +# built-in way to drop a keystroke, so the shim intercepts exactly one +# `pane send-keys enter` call and forwards everything else to the real +# binary untouched. +# +# The "supervisor pane" is a tiny deterministic bash loop (not a real harness): +# it draws a bordered composer row ("│ > │") matching the structural +# classifier fm_backend_herdr_composer_state expects, and logs every submitted +# line (hex + text + injection/user classification) - the same technique +# tests/fm-afk-inject-e2e.test.sh uses for its tmux supervisor pane, so this +# test asserts on submitted CONTENT, not pane appearance. +set -u + +ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)" +DAEMON="$ROOT/bin/fm-supervise-daemon.sh" + +command -v herdr >/dev/null 2>&1 || { echo "skip: herdr not found"; exit 0; } +command -v jq >/dev/null 2>&1 || { echo "skip: jq not found (required by the herdr adapter)"; exit 0; } + +# shellcheck source=tests/herdr-test-safety.sh +. "$ROOT/tests/herdr-test-safety.sh" + +fail() { printf 'not ok - %s\n' "$1" >&2; cleanup_all; exit 1; } +pass() { printf 'ok - %s\n' "$1"; } + +SESSION="fm-afk-herdr-e2e-$$" +export HERDR_SESSION="$SESSION" +STATE_DIR= +HERDR_SHIM_DIR= +LOG_FILE= +DAEMON_PID= +SUPERVISOR_TARGET= +PANE_ID= +LOOP_SCRIPT= + +cleanup_all() { + if [ -n "${DAEMON_PID:-}" ]; then + afk_exit "${STATE_DIR:-}" 2>/dev/null || true + kill "$DAEMON_PID" 2>/dev/null || true + wait "$DAEMON_PID" 2>/dev/null || true + fi + herdr_safe_stop_and_delete "$SESSION" 2>/dev/null || true + rm -rf "${HERDR_SHIM_DIR:-}" 2>/dev/null || true + rm -rf "${STATE_DIR:-}" 2>/dev/null || true +} +trap cleanup_all EXIT + +# --- source the daemon (for afk_enter/afk_exit/FM_INJECT_MARK) + the backend - +# shellcheck source=bin/fm-supervise-daemon.sh +. "$DAEMON" +fm_backend_source herdr || fail "fm_backend_source herdr failed" + +# --- build the isolated session's supervisor pane ---------------------------- + +fm_backend_herdr_version_check || fail "version_check failed against the real installed herdr" + +STATE_DIR=$(mktemp -d "${TMPDIR:-/tmp}/fm-afk-herdr-e2e.XXXXXX") +mkdir -p "$STATE_DIR" +LOG_FILE="$STATE_DIR/submitted.log" +: > "$LOG_FILE" + +CONTAINER_RAW=$(fm_backend_herdr_container_ensure /tmp) || fail "container_ensure failed" +CONTAINER=${CONTAINER_RAW%%$'\t'*} +SEEDED_TAB_ID=${CONTAINER_RAW#*$'\t'} +TASK_IDS=$(fm_backend_herdr_create_task "$CONTAINER" "fm-afk-e2e-supervisor" /tmp "$SEEDED_TAB_ID") \ + || fail "create_task for the scratch supervisor pane failed" +read -r _TAB_ID PANE_ID < │" border so fm_backend_herdr_composer_state's structural +# classifier (a row whose trimmed content starts AND ends with the same border +# glyph) recognizes it, exactly like a real bordered-TUI harness composer. +LOOP_SCRIPT="$STATE_DIR/supervisor-loop.sh" +cat > "$LOOP_SCRIPT" <<'LOOP' +#!/usr/bin/env bash +MARK=$'\x1f' +LOG="$1" +OLD_STTY=$(stty -g 2>/dev/null || true) +[ -z "$OLD_STTY" ] || stty -echo -icanon min 1 time 0 2>/dev/null || true +cleanup() { + [ -z "$OLD_STTY" ] || stty "$OLD_STTY" 2>/dev/null || true +} +trap cleanup EXIT INT TERM + +_buf= +# redraw: keep the composer visually pinned to ONE terminal row regardless of +# _buf's length - a realistic bordered single-line composer horizontally +# scrolls to show the tail near the cursor rather than letting the terminal +# hard-wrap a too-long line across multiple rows (which would break the +# structural border-row classifier's one-row assumption: a batched escalation +# digest easily exceeds a narrow pane's column width). A hardcoded width +# (not `tput cols`) is used deliberately: verified empirically against a real +# herdr pane launched this same way that `tput cols` inside this script's own +# process reports 80 regardless of the pane's ACTUAL width (54, confirmed via +# a separate interactively-typed `tput cols`), so trusting it here silently +# let content overflow the real width and wrap across two rows. 40 is +# comfortably under every real pane width observed on this machine. +redraw() { + local avail=40 shown tail_n + if [ "${#_buf}" -gt "$avail" ]; then + tail_n=$((avail - 3)) + shown="...${_buf: -$tail_n}" + else + shown="$_buf" + fi + printf '\r\033[K│ > %s │' "$shown" +} +submit_line() { + local _line=$_buf _c _hex + if [ "${_line:0:1}" = "$MARK" ]; then + _c="injection" + else + _c="user" + fi + _hex=$(printf '%s' "$_line" | od -An -tx1 | tr -d ' \n') + printf '%s\t%s\t%s\n' "$_hex" "$_line" "$_c" >> "$LOG" + _buf= + printf '\r\033[K\n' + redraw +} + +redraw +while IFS= read -r -n 1 _ch; do + if [ -z "$_ch" ]; then + submit_line + continue + fi + case "$_ch" in + $'\r'|$'\n') submit_line ;; + $'\177'|$'\b') _buf=${_buf%?}; redraw ;; + *) _buf="${_buf}${_ch}"; redraw ;; + esac +done +LOOP +chmod +x "$LOOP_SCRIPT" + +fm_backend_herdr_send_text_line "$SUPERVISOR_TARGET" "bash '$LOOP_SCRIPT' '$LOG_FILE'" \ + || fail "could not start the supervisor-loop script in the scratch herdr pane" +sleep 1 # let the loop start and settle + +# --- herdr shim: forwards to the real binary, optionally swallows one Enter -- +REAL_HERDR=$(command -v herdr) +HERDR_SHIM_DIR=$(mktemp -d "${TMPDIR:-/tmp}/fm-herdr-shim.XXXXXX") +cat > "$HERDR_SHIM_DIR/herdr" <"$STATE_DIR/daemon.out" 2>"$STATE_DIR/daemon.err" & + DAEMON_PID=$! + local i=0 + while [ "$i" -lt 30 ]; do + [ -f "$STATE_DIR/.supervise-daemon.pid" ] && break + sleep 0.2 + i=$((i + 1)) + done + [ -f "$STATE_DIR/.supervise-daemon.pid" ] || { + echo "daemon stderr:" >&2; cat "$STATE_DIR/daemon.err" >&2 + fail "daemon did not start (no pid file after 6s)" + } + grep -q 'backend=herdr' "$STATE_DIR/.supervise-daemon.log" 2>/dev/null \ + || fail "daemon startup log did not record backend=herdr: $(cat "$STATE_DIR/.supervise-daemon.log" 2>/dev/null)" +} + +stop_daemon() { + [ -n "${DAEMON_PID:-}" ] || return 0 + afk_exit "$STATE_DIR" 2>/dev/null || true + kill "$DAEMON_PID" 2>/dev/null || true + wait "$DAEMON_PID" 2>/dev/null || true + DAEMON_PID="" + sleep 1 +} + +reset_state() { + rm -f "$STATE_DIR"/*.status \ + "$STATE_DIR"/.subsuper-* \ + "$STATE_DIR"/.wake-queue* \ + "$STATE_DIR"/.watch.lock* \ + "$STATE_DIR"/.last-* \ + "$STATE_DIR"/.hash-* \ + "$STATE_DIR"/.count-* \ + "$STATE_DIR"/.stale-* \ + "$STATE_DIR"/.seen-* \ + "$STATE_DIR"/.heartbeat-streak \ + "$STATE_DIR"/.swallow-enter \ + 2>/dev/null || true + : > "$LOG_FILE" +} + +# --- pane_input_pending environment self-check ------------------------------ +# Verify pane_input_pending (dispatched through fm_backend_composer_state for +# backend=herdr) can detect typed text in THIS real herdr environment before +# trusting the scenarios below to prove anything. + +selfcheck_pane_input_pending() { + local check_text="selfcheck-marker-12345" + fm_backend_herdr_send_literal "$SUPERVISOR_TARGET" "$check_text" \ + || fail "selfcheck: could not send literal text to the scratch pane" + sleep 0.5 + if PATH="$HERDR_SHIM_DIR:$PATH" pane_input_pending "$SUPERVISOR_TARGET" herdr; then + fm_backend_herdr_send_key "$SUPERVISOR_TARGET" Enter + sleep 0.5 + return 0 + fi + echo "pane_input_pending cannot detect typed text in this real-herdr environment" >&2 + fm_backend_herdr_capture "$SUPERVISOR_TARGET" 10 | sed 's/^/ /' >&2 + fm_backend_herdr_send_key "$SUPERVISOR_TARGET" Enter + fail "pane_input_pending self-check failed against real herdr" +} + +selfcheck_pane_input_pending + +# --- Scenario A: human-partial-input ---------------------------------------- + +test_scenario_a() { + reset_state + afk_enter "$STATE_DIR" + start_daemon + + fm_backend_herdr_send_literal "$SUPERVISOR_TARGET" "human draft text" + sleep 0.5 + + echo "done: PR https://example.test/pr/100" > "$STATE_DIR/fake-c1.status" + + sleep 8 + + if grep -q 'Supervisor escalate' "$LOG_FILE"; then + fail "Scenario A: daemon injected while the herdr pane had pending input" + fi + if grep -q 'human draft text.*Supervisor escalate' "$LOG_FILE" 2>/dev/null || \ + grep -q 'Supervisor escalate.*human draft text' "$LOG_FILE" 2>/dev/null; then + fail "Scenario A: human text and digest were merged into one line" + fi + + fm_backend_herdr_send_key "$SUPERVISOR_TARGET" Enter + sleep 0.5 + + sleep 8 + + grep -q 'human draft text' "$LOG_FILE" \ + || fail "Scenario A: human text not in log after submit" + grep -q 'Supervisor escalate' "$LOG_FILE" \ + || fail "Scenario A: digest not injected after the pane went idle" + if grep -q 'human draft text.*Supervisor escalate' "$LOG_FILE" || \ + grep -q 'Supervisor escalate.*human draft text' "$LOG_FILE"; then + fail "Scenario A: human text and digest merged into one line (after idle)" + fi + + local human_line + human_line=$(grep 'human draft text' "$LOG_FILE" | head -1) + case "$human_line" in + *user) ;; + *) fail "Scenario A: human text misclassified (expected user): $human_line" ;; + esac + + local digest_line + digest_line=$(grep 'Supervisor escalate' "$LOG_FILE" | head -1) + case "$digest_line" in + *injection) ;; + *) fail "Scenario A: digest misclassified (expected injection): $digest_line" ;; + esac + + stop_daemon + pass "real herdr Scenario A: partial input defers injection; digest arrives clean after idle" +} + +# --- Scenario B: swallowed-Enter -------------------------------------------- + +test_scenario_b() { + reset_state + afk_enter "$STATE_DIR" + + touch "$STATE_DIR/.swallow-enter" + + start_daemon + + echo "done: PR https://example.test/pr/200" > "$STATE_DIR/fake-c1.status" + + sleep 10 + + local digest_count + digest_count=$(grep -c 'Supervisor escalate' "$LOG_FILE" || true) + [ "$digest_count" -eq 1 ] \ + || fail "Scenario B: expected exactly 1 digest, got $digest_count (duplicate or lost)" + + if grep -q "$(printf '\x1f').*$(printf '\x1f')" "$LOG_FILE"; then + fail "Scenario B: digest concatenated with itself (two sentinel markers in one line)" + fi + + local digest_line digest_hex + digest_line=$(grep 'Supervisor escalate' "$LOG_FILE" | head -1) + digest_hex=$(printf '%s' "$digest_line" | cut -f1) + case "$digest_hex" in + 1f*) ;; + *) fail "Scenario B: digest does not start with the sentinel marker (hex: $digest_hex)" ;; + esac + + local user_count + user_count=$(grep -c $'\tuser$' "$LOG_FILE" || true) + [ "$user_count" -eq 0 ] \ + || fail "Scenario B: expected 0 user lines, got $user_count (spurious Enter submitted an empty line?)" + + stop_daemon + pass "real herdr Scenario B: swallowed Enter (via the herdr shim) produces exactly one clean digest" +} + +# --- Scenario C: normal digest ----------------------------------------------- + +test_scenario_c() { + reset_state + afk_enter "$STATE_DIR" + start_daemon + + echo "done: PR https://example.test/pr/300" > "$STATE_DIR/fake-c1.status" + sleep 8 + + local digest_count + digest_count=$(grep -c 'Supervisor escalate' "$LOG_FILE" || true) + [ "$digest_count" -eq 1 ] \ + || fail "Scenario C: expected exactly 1 digest, got $digest_count" + + if grep -q "$(printf '\x1f').*$(printf '\x1f')" "$LOG_FILE"; then + fail "Scenario C: digest concatenated with itself (two sentinel markers in one line)" + fi + + local digest_line digest_hex + digest_line=$(grep 'Supervisor escalate' "$LOG_FILE" | head -1) + case "$digest_line" in + *injection) ;; + *) fail "Scenario C: digest misclassified (expected injection): $digest_line" ;; + esac + digest_hex=$(printf '%s' "$digest_line" | cut -f1) + case "$digest_hex" in + 1f*) ;; + *) fail "Scenario C: digest does not start with the sentinel marker (hex: $digest_hex)" ;; + esac + + local user_count + user_count=$(grep -c $'\tuser$' "$LOG_FILE" || true) + [ "$user_count" -eq 0 ] \ + || fail "Scenario C: expected 0 user lines, got $user_count (spurious submission?)" + + stop_daemon + pass "real herdr Scenario C: a normal captain status injects exactly one clean single-line sentinel digest" +} + +# --- Scenario D: max-defer alarm on a persistently non-clearing composer ----- +# A pending composer that NEVER clears (every Enter attempt leaves real text +# behind) must never be silently swallowed: the daemon must alarm (write +# state/.subsuper-inject-wedged) while preserving the buffered escalation, and +# must never crash or hot-loop. Exercises fm_backend_composer_state(herdr, ...) +# reporting "pending" indefinitely through the REAL structural border reader. + +test_scenario_d_max_defer() { + reset_state + afk_enter "$STATE_DIR" + # Persistent-pending composer: type real text and never submit it, so every + # composer read is genuinely "pending" against the real herdr binary. + fm_backend_herdr_send_literal "$SUPERVISOR_TARGET" "stuck-in-the-box" + sleep 0.5 + + PATH="$HERDR_SHIM_DIR:$PATH" \ + HERDR_SESSION="$SESSION" \ + FM_STATE_OVERRIDE="$STATE_DIR" \ + FM_SUPERVISOR_BACKEND=herdr \ + FM_SUPERVISOR_TARGET="$SUPERVISOR_TARGET" \ + FM_ESCALATE_BATCH_SECS=99999 \ + FM_HOUSEKEEPING_TICK=1 \ + FM_POLL=1 \ + FM_SIGNAL_GRACE=1 \ + FM_HEARTBEAT=999999 \ + FM_CHECK_INTERVAL=999999 \ + FM_MAX_DEFER_SECS=3 \ + FM_INJECT_CONFIRM_SLEEP=0.3 \ + FM_INJECT_CONFIRM_RETRIES=2 \ + FM_STALE_ESCALATE_SECS=999999 \ + nohup "$DAEMON" >"$STATE_DIR/daemon.out" 2>"$STATE_DIR/daemon.err" & + DAEMON_PID=$! + local i=0 + while [ "$i" -lt 30 ]; do + [ -f "$STATE_DIR/.supervise-daemon.pid" ] && break + sleep 0.2 + i=$((i + 1)) + done + [ -f "$STATE_DIR/.supervise-daemon.pid" ] || fail "Scenario D: daemon did not start" + + echo "needs-decision: pick A or B" > "$STATE_DIR/fake-c1.status" + + sleep 12 + + [ -s "$STATE_DIR/.subsuper-inject-wedged" ] \ + || fail "Scenario D: a persistently pending real herdr composer never raised the max-defer wedge alarm" + [ -s "$STATE_DIR/.subsuper-escalations" ] \ + || fail "Scenario D: the buffered escalation was lost instead of preserved during the wedge" + if grep -q 'Supervisor escalate' "$LOG_FILE" 2>/dev/null; then + fail "Scenario D: a digest was somehow logged as submitted despite the composer never clearing" + fi + kill -0 "$DAEMON_PID" 2>/dev/null || fail "Scenario D: the daemon process died instead of alarming and continuing" + grep -F 'stuck-in-the-box' "$STATE_DIR/daemon.err" >/dev/null 2>&1 && : # not fatal either way + + stop_daemon + # Clean up the stuck composer text for a tidy teardown (best-effort). + fm_backend_herdr_send_key "$SUPERVISOR_TARGET" C-c >/dev/null 2>&1 || true + pass "real herdr Scenario D: a persistently pending composer raises the max-defer wedge alarm, preserves the buffer, and never crashes the daemon" +} + +test_scenario_a +test_scenario_b +test_scenario_c +test_scenario_d_max_defer + +echo "all real-herdr afk injection e2e tests passed" + +fm_backend_herdr_kill "$SUPERVISOR_TARGET" 2>/dev/null || true +fm_backend_herdr_kill "$SESSION:$FAKE_CREW_PANE_ID" 2>/dev/null || true +cleanup_all +trap - EXIT diff --git a/tests/fm-backend-herdr.test.sh b/tests/fm-backend-herdr.test.sh index aa104e09ccd..84be0469811 100755 --- a/tests/fm-backend-herdr.test.sh +++ b/tests/fm-backend-herdr.test.sh @@ -984,6 +984,33 @@ test_dispatch_busy_state_unknown_for_tmux() { pass "fm_backend_busy_state: tmux (no native primitive) always reports unknown, preserving the P1 regex-only path" } +test_dispatch_composer_state_routes_by_backend() { + # fm_backend_composer_state (the generic per-backend composer/pending-input + # classifier the away-mode daemon dispatches through - bin/fm-supervise-daemon.sh's + # pane_input_pending) must route to each backend's OWN named classifier with + # the target passed through unchanged, fall back to unknown for a backend with + # no named classifier (zellij), and unknown for an unrecognized backend name. + # Sourced-guards are pre-set so fm_backend_source no-ops and these stubs are + # never clobbered by the real per-backend files trying (and failing) a live call. + ( + # shellcheck source=bin/fm-backend.sh + . "$ROOT/bin/fm-backend.sh" + _FM_BACKEND_TMUX_SOURCED=1 + _FM_BACKEND_HERDR_SOURCED=1 + _FM_BACKEND_ORCA_SOURCED=1 + _FM_BACKEND_ZELLIJ_SOURCED=1 + fm_tmux_composer_state() { [ "$1" = "sess:win" ] || fail "tmux composer_state got wrong target: $1"; printf 'pending'; } + fm_backend_herdr_composer_state() { [ "$1" = "default:w1:p2" ] || fail "herdr composer_state got wrong target: $1"; printf 'empty'; } + fm_backend_orca_composer_state() { [ "$1" = "term-1" ] || fail "orca composer_state got wrong target: $1"; printf 'empty'; } + [ "$(fm_backend_composer_state tmux sess:win)" = pending ] || fail "composer_state did not dispatch to the tmux classifier" + [ "$(fm_backend_composer_state herdr default:w1:p2)" = empty ] || fail "composer_state did not dispatch to the herdr classifier" + [ "$(fm_backend_composer_state orca term-1)" = empty ] || fail "composer_state did not dispatch to the orca classifier" + [ "$(fm_backend_composer_state zellij sess:win)" = unknown ] || fail "composer_state should report unknown for zellij (no named classifier yet)" + [ "$(fm_backend_composer_state bogus x)" = unknown ] || fail "composer_state should report unknown for an unrecognized backend" + ) || fail "composer_state dispatch subshell failed" + pass "fm_backend_composer_state dispatches tmux/herdr/orca to their named classifiers, unknown for zellij/unrecognized backends" +} + test_scripts_route_explicit_target_through_meta_backend() { local dir state log resp fb neutral out dir="$TMP_ROOT/script-explicit-target"; state="$dir/state"; mkdir -p "$state" "$dir/responses" @@ -1303,4 +1330,5 @@ test_send_text_submit_send_failed test_send_text_submit_unknown_on_capture_failure test_dispatch_routes_herdr_backend test_dispatch_busy_state_unknown_for_tmux +test_dispatch_composer_state_routes_by_backend test_scripts_route_explicit_target_through_meta_backend diff --git a/tests/fm-daemon.test.sh b/tests/fm-daemon.test.sh index 917421b8579..ef16647c5c8 100755 --- a/tests/fm-daemon.test.sh +++ b/tests/fm-daemon.test.sh @@ -769,6 +769,168 @@ test_fm_send_exits_nonzero_on_initial_send_failure() { pass "fm-send exits non-zero when initial text send fails" } +# --- herdr backend-awareness (fm-turnend-guard-h6-adjacent transport fix) ---- +# Discovery, busy/pending dispatch, and the full inject_msg guard chain must +# work through the herdr backend, not just tmux. Env-var prefix assignments +# (e.g. `TMUX_PANE= HERDR_ENV=1 ... discover_supervisor_target`) neutralize +# whatever ambient TMUX_PANE/HERDR_ENV the CURRENT dev/CI shell happens to carry +# for the duration of that one call only, so these tests are deterministic +# regardless of what runtime backend is running this test suite itself. + +test_discover_supervisor_backend_precedence() { + local out + out=$(FM_SUPERVISOR_BACKEND=herdr TMUX_PANE='%9' HERDR_ENV=1 HERDR_PANE_ID=w1:p1 discover_supervisor_backend) + [ "$out" = herdr ] || fail "explicit FM_SUPERVISOR_BACKEND override was not honored: $out" + + out=$(FM_SUPERVISOR_BACKEND='' TMUX_PANE='%9' HERDR_ENV=1 HERDR_PANE_ID=w1:p1 discover_supervisor_backend) + [ "$out" = tmux ] || fail "TMUX_PANE should win over HERDR_ENV (tmux nested in herdr resolves to tmux): $out" + + out=$(FM_SUPERVISOR_BACKEND='' TMUX_PANE='' HERDR_ENV=1 HERDR_PANE_ID=w1:p1 discover_supervisor_backend) + [ "$out" = herdr ] || fail "HERDR_ENV=1 with HERDR_PANE_ID present should resolve to herdr: $out" + + if out=$(FM_SUPERVISOR_BACKEND='' TMUX_PANE='' HERDR_ENV='' HERDR_PANE_ID='' discover_supervisor_backend); then + fail "bare fallback (no override, no TMUX_PANE, no HERDR_ENV) should return non-zero" + fi + [ "$out" = tmux ] || fail "bare fallback should still print tmux: $out" + + pass "discover_supervisor_backend: override > TMUX_PANE > HERDR_ENV+HERDR_PANE_ID > tmux fallback" +} + +test_discover_supervisor_target_herdr() { + local out + out=$(FM_SUPERVISOR_TARGET=explicit:target TMUX_PANE='' HERDR_ENV=1 HERDR_PANE_ID=w1:p9 discover_supervisor_target) + [ "$out" = "explicit:target" ] || fail "explicit FM_SUPERVISOR_TARGET override was not honored: $out" + + out=$(FM_SUPERVISOR_TARGET='' TMUX_PANE='%3' HERDR_ENV=1 HERDR_PANE_ID=w1:p9 discover_supervisor_target) + [ "$out" = '%3' ] || fail "TMUX_PANE should win over herdr markers: $out" + + out=$(FM_SUPERVISOR_TARGET='' TMUX_PANE='' HERDR_ENV=1 HERDR_PANE_ID=w1:p9 HERDR_SESSION='' discover_supervisor_target) + [ "$out" = "default:w1:p9" ] || fail "herdr target should default HERDR_SESSION to 'default': $out" + + out=$(FM_SUPERVISOR_TARGET='' TMUX_PANE='' HERDR_ENV=1 HERDR_PANE_ID=w1:p9 HERDR_SESSION=iso1 discover_supervisor_target) + [ "$out" = "iso1:w1:p9" ] || fail "herdr target should use an explicit HERDR_SESSION: $out" + + if out=$(FM_SUPERVISOR_TARGET='' TMUX_PANE='' HERDR_ENV='' HERDR_PANE_ID='' discover_supervisor_target); then + fail "bare fallback should return non-zero" + fi + [ "$out" = "firstmate:0" ] || fail "bare fallback should still print firstmate:0: $out" + + pass "discover_supervisor_target: override > TMUX_PANE > herdr ':' composition > firstmate:0 fallback" +} + +test_pane_is_busy_herdr_native_busy_state() { + ( + fm_backend_busy_state() { [ "$1" = herdr ] && [ "$2" = "default:w1:p2" ] || fail "unexpected busy_state args: $1 $2"; printf 'busy'; } + fm_backend_capture() { fail "capture should not be consulted when busy_state is conclusive"; } + pane_is_busy "default:w1:p2" herdr || fail "pane_is_busy should report busy from herdr's native busy_state" + ) || fail "herdr native-busy pane_is_busy subshell failed" + pass "pane_is_busy: herdr native busy_state='busy' short-circuits without a capture fallback" +} + +test_pane_is_busy_herdr_falls_back_to_capture_regex() { + ( + fm_backend_busy_state() { printf 'unknown'; } + fm_backend_capture() { [ "$1" = herdr ] && [ "$2" = "default:w1:p2" ] || fail "unexpected capture args: $1 $2"; printf 'esc to interrupt\n'; } + pane_is_busy "default:w1:p2" herdr || fail "pane_is_busy should fall back to the regex-over-capture reader when busy_state is unknown" + ) || fail "herdr capture-fallback pane_is_busy subshell failed" + pass "pane_is_busy: herdr falls back to the shared regex-over-capture reader when native busy_state is unknown" +} + +test_pane_is_busy_defaults_to_tmux_when_backend_omitted() { + local dir fakebin capture + dir=$(make_supercase busy-default-backend) + fakebin="$dir/fakebin"; capture="$dir/pane.txt" + printf 'esc to interrupt\n' > "$capture" + PATH="$fakebin:$PATH" FM_FAKE_TMUX_CAPTURE="$capture" pane_is_busy "fakepane" \ + || fail "pane_is_busy with no backend arg should still default to tmux" + pass "pane_is_busy: omitted backend arg defaults to tmux (pre-existing callers unaffected)" +} + +test_pane_input_pending_herdr_dispatch() { + ( + fm_backend_composer_state() { [ "$1" = herdr ] && [ "$2" = "default:w1:p2" ] || fail "unexpected composer_state args: $1 $2"; printf 'pending'; } + pane_input_pending "default:w1:p2" herdr || fail "pane_input_pending should report pending from herdr composer_state" + ) || fail "herdr pane_input_pending (pending case) subshell failed" + ( + fm_backend_composer_state() { printf 'empty'; } + if pane_input_pending "default:w1:p2" herdr; then + fail "pane_input_pending should report not-pending for an empty herdr composer" + fi + ) || fail "herdr pane_input_pending (empty case) subshell failed" + pass "pane_input_pending: dispatches through fm_backend_composer_state for backend=herdr" +} + +test_inject_msg_herdr_busy_guard_defers() { + local dir state + dir=$(make_supercase inject-herdr-busy) + state="$dir/state" + afk_enter "$state" + ( + fm_backend_target_exists() { [ "$1" = herdr ] && [ "$2" = "default:w1:p2" ] || fail "unexpected target_exists args: $1 $2"; return 0; } + fm_backend_busy_state() { printf 'busy'; } + fm_backend_capture() { fail "capture should not run when busy_state is conclusive"; } + fm_backend_composer_state() { fail "composer_state should not be consulted once the busy-guard already deferred"; } + fm_backend_send_text_submit() { fail "send_text_submit should not run when the busy-guard defers"; } + if FM_SUPERVISOR_BACKEND=herdr FM_SUPERVISOR_TARGET="default:w1:p2" inject_msg "hello" "$state"; then + fail "inject_msg should defer (return non-zero) when the herdr supervisor pane is busy" + fi + ) || fail "herdr busy-guard inject_msg subshell failed" + pass "inject_msg: herdr busy-guard defers before ever attempting a submit" +} + +test_inject_msg_herdr_composer_guard_defers() { + local dir state + dir=$(make_supercase inject-herdr-pending) + state="$dir/state" + afk_enter "$state" + ( + fm_backend_target_exists() { return 0; } + fm_backend_busy_state() { printf 'idle'; } + fm_backend_composer_state() { [ "$1" = herdr ] && [ "$2" = "default:w1:p2" ] || fail "unexpected composer_state args: $1 $2"; printf 'pending'; } + fm_backend_send_text_submit() { fail "send_text_submit should not run when the composer-guard defers"; } + if FM_SUPERVISOR_BACKEND=herdr FM_SUPERVISOR_TARGET="default:w1:p2" inject_msg "hello" "$state"; then + fail "inject_msg should defer when the herdr composer has pending input" + fi + ) || fail "herdr composer-guard inject_msg subshell failed" + pass "inject_msg: herdr composer-guard defers before ever attempting a submit" +} + +test_inject_msg_herdr_pane_gone_defers() { + local dir state + dir=$(make_supercase inject-herdr-gone) + state="$dir/state" + afk_enter "$state" + ( + fm_backend_target_exists() { return 1; } + fm_backend_busy_state() { fail "busy_state should not be consulted once the pane-exists check already failed"; } + fm_backend_send_text_submit() { fail "send_text_submit should not run when the pane does not exist"; } + if FM_SUPERVISOR_BACKEND=herdr FM_SUPERVISOR_TARGET="default:w1:gone" inject_msg "hello" "$state"; then + fail "inject_msg should defer when the herdr target does not exist" + fi + ) || fail "herdr pane-gone inject_msg subshell failed" + pass "inject_msg: herdr pane-gone check defers before any busy/composer/submit call" +} + +test_inject_msg_herdr_submits_through_backend_dispatch() { + local dir state + dir=$(make_supercase inject-herdr-submit) + state="$dir/state" + afk_enter "$state" + ( + fm_backend_target_exists() { return 0; } + fm_backend_busy_state() { printf 'idle'; } + fm_backend_composer_state() { printf 'empty'; } + fm_backend_send_text_submit() { + [ "$1" = herdr ] && [ "$2" = "default:w1:p2" ] || fail "unexpected send_text_submit args: $1 $2" + case "$3" in *"hello"*) : ;; *) fail "digest text missing from send_text_submit: $3" ;; esac + printf 'empty' + } + FM_SUPERVISOR_BACKEND=herdr FM_SUPERVISOR_TARGET="default:w1:p2" inject_msg "hello" "$state" \ + || fail "inject_msg should succeed when send_text_submit confirms empty" + ) || fail "herdr successful-submit inject_msg subshell failed" + pass "inject_msg: dispatches busy-guard/composer-guard/submit through the herdr backend and succeeds on a confirmed empty composer" +} + test_daemon_state_root_uses_fm_home test_classify_routine_signal_self test_classify_terminal_signal_escalates @@ -813,3 +975,13 @@ test_below_max_defer_does_nothing test_max_defer_afk_inactive_does_not_flush_or_alarm test_fm_send_exits_nonzero_on_confirmed_swallow test_fm_send_exits_nonzero_on_initial_send_failure +test_discover_supervisor_backend_precedence +test_discover_supervisor_target_herdr +test_pane_is_busy_herdr_native_busy_state +test_pane_is_busy_herdr_falls_back_to_capture_regex +test_pane_is_busy_defaults_to_tmux_when_backend_omitted +test_pane_input_pending_herdr_dispatch +test_inject_msg_herdr_busy_guard_defers +test_inject_msg_herdr_composer_guard_defers +test_inject_msg_herdr_pane_gone_defers +test_inject_msg_herdr_submits_through_backend_dispatch From 3deb9cb3b356da10edfc3fa5685fcd5243e522cc Mon Sep 17 00:00:00 2001 From: kunchenguid Date: Sat, 4 Jul 2026 12:14:18 -0700 Subject: [PATCH 2/5] no-mistakes(review): Corroborate Herdr idle busy state --- bin/fm-supervise-daemon.sh | 10 +++------- tests/fm-daemon.test.sh | 39 ++++++++++++++++++++++++++++++++++++++ 2 files changed, 42 insertions(+), 7 deletions(-) diff --git a/bin/fm-supervise-daemon.sh b/bin/fm-supervise-daemon.sh index 112a243dbe3..4e70786b2ba 100755 --- a/bin/fm-supervise-daemon.sh +++ b/bin/fm-supervise-daemon.sh @@ -483,7 +483,7 @@ mark_escalated_seen() { # # case statement here, mirroring the same fallback pattern # stale_window_is_busy already uses for per-task panes: try the backend's # native busy-state first, and fall back to the shared regex-over-capture -# reader when the backend reports "unknown" (tmux has no native busy-state +# reader whenever it does not report "busy" (tmux has no native busy-state # primitive, so it always takes this fallback path - byte-identical to the # pre-existing fm_pane_is_busy, since fm_backend_capture's tmux arm runs the # exact same `tmux capture-pane -p -t -S -40`). @@ -492,7 +492,6 @@ pane_is_busy() { # [backend] bs=$(fm_backend_busy_state "$backend" "$target" 2>/dev/null) case "$bs" in busy) return 0 ;; - idle) return 1 ;; esac tail40=$(fm_backend_capture "$backend" "$target" 40 2>/dev/null) || return 1 printf '%s' "$tail40" | grep -v '^[[:space:]]*$' | tail -6 \ @@ -522,12 +521,9 @@ stale_window_is_busy() { # bs=$(fm_backend_busy_state "$backend" "$win" 2>/dev/null) case "$bs" in busy) return 0 ;; - idle) return 1 ;; - *) - printf '%s' "$tail40" | grep -v '^[[:space:]]*$' | tail -6 \ - | grep -qiE "${FM_BUSY_REGEX:-$FM_TMUX_BUSY_REGEX_DEFAULT}" - ;; esac + printf '%s' "$tail40" | grep -v '^[[:space:]]*$' | tail -6 \ + | grep -qiE "${FM_BUSY_REGEX:-$FM_TMUX_BUSY_REGEX_DEFAULT}" } escalate_add() { # diff --git a/tests/fm-daemon.test.sh b/tests/fm-daemon.test.sh index ef16647c5c8..4d45523657c 100755 --- a/tests/fm-daemon.test.sh +++ b/tests/fm-daemon.test.sh @@ -164,6 +164,32 @@ test_housekeeping_herdr_persistent_stale_resolves_meta() { pass "persistent herdr stale resolves the target from metadata and escalates" } +test_housekeeping_herdr_idle_busy_footer_clears_stale() { + local dir state key + dir=$(make_supercase stale-herdr-idle-busy-footer) + state="$dir/state" + fm_write_meta "$state/herdr-footer.meta" "window=default:w1:p4" "backend=herdr" + printf 'working\n' > "$state/herdr-footer.status" + key=$(printf '%s' "herdr-footer" | tr ':/.' '___') + echo $(( $(date +%s) - 500 )) > "$state/.subsuper-stale-$key" + ( + fm_backend_capture() { + [ "$1" = herdr ] || fail "expected herdr capture backend, got $1" + [ "$2" = "default:w1:p4" ] || fail "expected herdr window target, got $2" + printf 'esc to interrupt\n' + } + fm_backend_busy_state() { + [ "$1" = herdr ] || fail "expected herdr busy backend, got $1" + [ "$2" = "default:w1:p4" ] || fail "expected herdr busy target, got $2" + printf 'idle' + } + FM_STATE_OVERRIDE="$state" FM_STALE_ESCALATE_SECS=240 housekeeping "$state" + ) || fail "herdr idle busy-footer housekeeping failed" + [ ! -e "$state/.subsuper-stale-$key" ] || fail "idle+busy-footer herdr stale marker was not cleared" + [ ! -s "$state/.subsuper-escalations" ] || fail "idle+busy-footer herdr stale was escalated" + pass "herdr idle busy-footer stale clears through capture corroboration" +} + test_housekeeping_herdr_resumed_stale_cleared() { local dir state key dir=$(make_supercase stale-herdr-resumed) @@ -836,6 +862,15 @@ test_pane_is_busy_herdr_falls_back_to_capture_regex() { pass "pane_is_busy: herdr falls back to the shared regex-over-capture reader when native busy_state is unknown" } +test_pane_is_busy_herdr_idle_falls_back_to_capture_regex() { + ( + fm_backend_busy_state() { printf 'idle'; } + fm_backend_capture() { [ "$1" = herdr ] && [ "$2" = "default:w1:p2" ] || fail "unexpected capture args: $1 $2"; printf 'esc to interrupt\n'; } + pane_is_busy "default:w1:p2" herdr || fail "pane_is_busy should fall back to the regex-over-capture reader when busy_state is idle" + ) || fail "herdr idle capture-fallback pane_is_busy subshell failed" + pass "pane_is_busy: herdr corroborates native idle with the shared regex-over-capture reader" +} + test_pane_is_busy_defaults_to_tmux_when_backend_omitted() { local dir fakebin capture dir=$(make_supercase busy-default-backend) @@ -886,6 +921,7 @@ test_inject_msg_herdr_composer_guard_defers() { ( fm_backend_target_exists() { return 0; } fm_backend_busy_state() { printf 'idle'; } + fm_backend_capture() { printf 'idle prompt\n'; } fm_backend_composer_state() { [ "$1" = herdr ] && [ "$2" = "default:w1:p2" ] || fail "unexpected composer_state args: $1 $2"; printf 'pending'; } fm_backend_send_text_submit() { fail "send_text_submit should not run when the composer-guard defers"; } if FM_SUPERVISOR_BACKEND=herdr FM_SUPERVISOR_TARGET="default:w1:p2" inject_msg "hello" "$state"; then @@ -919,6 +955,7 @@ test_inject_msg_herdr_submits_through_backend_dispatch() { ( fm_backend_target_exists() { return 0; } fm_backend_busy_state() { printf 'idle'; } + fm_backend_capture() { printf 'idle prompt\n'; } fm_backend_composer_state() { printf 'empty'; } fm_backend_send_text_submit() { [ "$1" = herdr ] && [ "$2" = "default:w1:p2" ] || fail "unexpected send_text_submit args: $1 $2" @@ -940,6 +977,7 @@ test_stale_terminal_escalates test_housekeeping_persistent_stale_escalates test_housekeeping_resumed_stale_cleared test_housekeeping_herdr_persistent_stale_resolves_meta +test_housekeeping_herdr_idle_busy_footer_clears_stale test_housekeeping_herdr_resumed_stale_cleared test_housekeeping_orca_persistent_stale_resolves_terminal test_escalate_batches_into_one_digest @@ -979,6 +1017,7 @@ test_discover_supervisor_backend_precedence test_discover_supervisor_target_herdr test_pane_is_busy_herdr_native_busy_state test_pane_is_busy_herdr_falls_back_to_capture_regex +test_pane_is_busy_herdr_idle_falls_back_to_capture_regex test_pane_is_busy_defaults_to_tmux_when_backend_omitted test_pane_input_pending_herdr_dispatch test_inject_msg_herdr_busy_guard_defers From 4920381efbd38ea54386e605eb77bf8ee8473237 Mon Sep 17 00:00:00 2001 From: kunchenguid Date: Sat, 4 Jul 2026 12:23:19 -0700 Subject: [PATCH 3/5] no-mistakes(review): Stabilize Herdr daemon startup wait --- tests/fm-afk-inject-herdr-e2e.test.sh | 45 ++++++++++++++++----------- 1 file changed, 26 insertions(+), 19 deletions(-) diff --git a/tests/fm-afk-inject-herdr-e2e.test.sh b/tests/fm-afk-inject-herdr-e2e.test.sh index 0e9ba8f8b60..aa41b3d76d7 100755 --- a/tests/fm-afk-inject-herdr-e2e.test.sh +++ b/tests/fm-afk-inject-herdr-e2e.test.sh @@ -184,7 +184,29 @@ exec "$REAL_HERDR" "\$@" SHIM chmod +x "$HERDR_SHIM_DIR/herdr" +wait_daemon_started() { + local label=${1:-daemon} start_line=${2:-0} i=0 new_log + while [ "$i" -lt 30 ]; do + new_log=$(tail -n +"$((start_line + 1))" "$STATE_DIR/.supervise-daemon.log" 2>/dev/null || true) + if printf '%s\n' "$new_log" | grep -q 'backend=herdr'; then + [ -f "$STATE_DIR/.supervise-daemon.pid" ] || fail "$label startup log recorded backend=herdr but no pid file was written" + kill -0 "$DAEMON_PID" 2>/dev/null || fail "$label exited after recording backend=herdr" + return 0 + fi + if ! kill -0 "$DAEMON_PID" 2>/dev/null; then + echo "daemon stderr:" >&2; cat "$STATE_DIR/daemon.err" >&2 + fail "$label exited before recording backend=herdr: $(cat "$STATE_DIR/.supervise-daemon.log" 2>/dev/null)" + fi + sleep 0.2 + i=$((i + 1)) + done + echo "daemon stderr:" >&2; cat "$STATE_DIR/daemon.err" >&2 + fail "$label did not record backend=herdr after 6s: $new_log" +} + start_daemon() { + local log_start=0 + [ ! -f "$STATE_DIR/.supervise-daemon.log" ] || log_start=$(wc -l < "$STATE_DIR/.supervise-daemon.log") PATH="$HERDR_SHIM_DIR:$PATH" \ HERDR_SESSION="$SESSION" \ FM_STATE_OVERRIDE="$STATE_DIR" \ @@ -201,18 +223,7 @@ start_daemon() { FM_STALE_ESCALATE_SECS=999999 \ nohup "$DAEMON" >"$STATE_DIR/daemon.out" 2>"$STATE_DIR/daemon.err" & DAEMON_PID=$! - local i=0 - while [ "$i" -lt 30 ]; do - [ -f "$STATE_DIR/.supervise-daemon.pid" ] && break - sleep 0.2 - i=$((i + 1)) - done - [ -f "$STATE_DIR/.supervise-daemon.pid" ] || { - echo "daemon stderr:" >&2; cat "$STATE_DIR/daemon.err" >&2 - fail "daemon did not start (no pid file after 6s)" - } - grep -q 'backend=herdr' "$STATE_DIR/.supervise-daemon.log" 2>/dev/null \ - || fail "daemon startup log did not record backend=herdr: $(cat "$STATE_DIR/.supervise-daemon.log" 2>/dev/null)" + wait_daemon_started daemon "$log_start" } stop_daemon() { @@ -407,6 +418,8 @@ test_scenario_c() { test_scenario_d_max_defer() { reset_state afk_enter "$STATE_DIR" + local log_start=0 + [ ! -f "$STATE_DIR/.supervise-daemon.log" ] || log_start=$(wc -l < "$STATE_DIR/.supervise-daemon.log") # Persistent-pending composer: type real text and never submit it, so every # composer read is genuinely "pending" against the real herdr binary. fm_backend_herdr_send_literal "$SUPERVISOR_TARGET" "stuck-in-the-box" @@ -429,13 +442,7 @@ test_scenario_d_max_defer() { FM_STALE_ESCALATE_SECS=999999 \ nohup "$DAEMON" >"$STATE_DIR/daemon.out" 2>"$STATE_DIR/daemon.err" & DAEMON_PID=$! - local i=0 - while [ "$i" -lt 30 ]; do - [ -f "$STATE_DIR/.supervise-daemon.pid" ] && break - sleep 0.2 - i=$((i + 1)) - done - [ -f "$STATE_DIR/.supervise-daemon.pid" ] || fail "Scenario D: daemon did not start" + wait_daemon_started "Scenario D daemon" "$log_start" echo "needs-decision: pick A or B" > "$STATE_DIR/fake-c1.status" From d3ff7ec8a54e43da2794538b8ee2905795ebe65c Mon Sep 17 00:00:00 2001 From: kunchenguid Date: Sat, 4 Jul 2026 12:32:39 -0700 Subject: [PATCH 4/5] no-mistakes(review): Captain, route cmux composer and update AFK docs --- .agents/skills/afk/SKILL.md | 5 +---- bin/fm-backend.sh | 11 ++++++----- docs/herdr-backend.md | 3 ++- tests/fm-backend-cmux.test.sh | 14 ++++++++++++++ 4 files changed, 23 insertions(+), 10 deletions(-) diff --git a/.agents/skills/afk/SKILL.md b/.agents/skills/afk/SKILL.md index b9c77c963e8..9f1bf0a5822 100644 --- a/.agents/skills/afk/SKILL.md +++ b/.agents/skills/afk/SKILL.md @@ -82,10 +82,7 @@ The daemon never injects into an in-use pane. Two checks run before every injection, dispatched through `bin/fm-backend.sh` for the supervisor's own backend (tmux or herdr; see "Auto-discovered supervisor pane" below): -- **`pane_is_busy`** - the harness shows a busy footer (agent mid-turn) on - tmux (shared with `fm-send.sh` via `bin/fm-tmux-lib.sh`); on herdr, tries the - native `agent.get`-backed busy state first and falls back to the same - regex-over-capture reader when that reports unknown. +- **`pane_is_busy`** - the harness shows a busy footer (agent mid-turn) on tmux (shared with `fm-send.sh` via `bin/fm-tmux-lib.sh`); on herdr, tries the native `agent.get`-backed busy state first, trusts only `busy` outright, and corroborates every non-`busy` verdict with the same regex-over-capture reader. - **`pane_input_pending`** - the composer holds real unsubmitted text (a human's half-typed line, or a previous injection whose Enter was swallowed). On tmux, the cursor-line detector **strips the harness's composer box diff --git a/bin/fm-backend.sh b/bin/fm-backend.sh index e74cbdf1665..000e8a804a5 100644 --- a/bin/fm-backend.sh +++ b/bin/fm-backend.sh @@ -424,11 +424,11 @@ fm_backend_busy_state() { # # pending-input guard, bin/fm-supervise-daemon.sh) can ask the same question # without duplicating per-backend composer-reading logic. tmux and herdr both # expose a named classifier already (fm_tmux_composer_state, -# fm_backend_herdr_composer_state), as does orca -# (fm_backend_orca_composer_state); zellij's submit path uses an internal -# content-diff approach with no separately named classifier, so it reports -# unknown here - callers fall back to their own policy, exactly as an unknown -# fm_backend_busy_state already does. +# fm_backend_herdr_composer_state), as do orca and cmux +# (fm_backend_orca_composer_state, fm_backend_cmux_composer_state); zellij's +# submit path uses an internal content-diff approach with no separately named +# classifier, so it reports unknown here - callers fall back to their own +# policy, exactly as an unknown fm_backend_busy_state already does. fm_backend_composer_state() { # -> empty|pending|unknown local backend=$1 shift @@ -437,6 +437,7 @@ fm_backend_composer_state() { # -> empty|pending|unknown tmux) fm_tmux_composer_state "$@" ;; herdr) fm_backend_herdr_composer_state "$@" ;; orca) fm_backend_orca_composer_state "$@" ;; + cmux) fm_backend_cmux_composer_state "$@" ;; *) printf 'unknown' ;; esac } diff --git a/docs/herdr-backend.md b/docs/herdr-backend.md index c8a096d778a..62973bcb4f6 100644 --- a/docs/herdr-backend.md +++ b/docs/herdr-backend.md @@ -350,7 +350,8 @@ zellij and orca are not yet supported as a supervisor backend - the daemon refus **Injection dispatch.** `inject_msg`'s pane-exists probe, busy-guard (`pane_is_busy`), composer-guard (`pane_input_pending`), and verified submit all take an optional `` argument (defaulting to `tmux` when omitted, so every pre-existing caller/test is unaffected) and route through the generic dispatchers instead of calling `tmux` directly. For `backend=tmux` every dispatch resolves to the exact same underlying call as before (`fm_backend_capture`'s tmux arm runs the identical `tmux capture-pane -p -t -S -40`; `fm_backend_tmux_send_text_submit` re-exports `fm_tmux_submit_core` verbatim), so tmux behavior is unchanged byte-for-byte. -For `backend=herdr`, busy detection tries the native `agent.get`-backed `fm_backend_herdr_busy_state` first and falls back to the shared regex-over-capture reader on `unknown`, exactly mirroring the per-task stale-pane busy check `bin/fm-supervise-daemon.sh`'s `stale_window_is_busy` already used; composer/pending detection and the verified submit reuse `fm_backend_herdr_composer_state`/`fm_backend_herdr_send_text_submit` unchanged. +For `backend=herdr`, busy detection tries the native `agent.get`-backed `fm_backend_herdr_busy_state` first, trusts only `busy` outright, and corroborates every non-`busy` verdict with the shared regex-over-capture reader before treating the supervisor pane as not busy. +This mirrors the per-task stale-pane busy check `bin/fm-supervise-daemon.sh`'s `stale_window_is_busy` already used; composer/pending detection and the verified submit reuse `fm_backend_herdr_composer_state`/`fm_backend_herdr_send_text_submit` unchanged. The wedge alarm's supervisor-client status-line flash (`tmux display-message ...`) is tmux-only cosmetic UI with no herdr equivalent; it is skipped for non-tmux backends, while the ERROR log line and the durable `state/.subsuper-inject-wedged` marker (the actual signal) are backend-independent and unaffected. **A pre-existing bug this surfaced: `fm_backend_target_exists`'s herdr arm.** Before this task, that function's herdr case called `HERDR_SESSION="$session" herdr pane get "$pane"` directly, WITHOUT the `--session` flag. diff --git a/tests/fm-backend-cmux.test.sh b/tests/fm-backend-cmux.test.sh index c1642d50475..13621008662 100755 --- a/tests/fm-backend-cmux.test.sh +++ b/tests/fm-backend-cmux.test.sh @@ -310,6 +310,19 @@ test_dispatch_busy_state_unknown_for_cmux() { pass "fm_backend_busy_state: cmux (no native primitive) always reports unknown, same as tmux/zellij/orca" } +test_dispatch_composer_state_routes_cmux() { + local dir fb out target + dir="$TMP_ROOT/dispatch-composer"; mkdir -p "$dir/responses" + target="aaaaaaaa-0000-0000-0000-000000000000:bbbbbbbb-1111-1111-1111-111111111111" + cmux_panes_response "$dir" 1 "bbbbbbbb-1111-1111-1111-111111111111" + cmux_read_screen_response "$dir" 2 $' ╭────────────────────────╮\n │ ❯ hello captain │\n ╰──────── Composer ─────╯' + fb=$(make_cmux_fakebin "$dir") + out=$( PATH="$fb:$PATH" FM_CMUX_LOG="$dir/log" FM_CMUX_RESPONSES="$dir/responses" \ + bash -c '. "$0/bin/fm-backend.sh"; fm_backend_composer_state cmux "$1"' "$ROOT" "$target" ) + [ "$out" = pending ] || fail "fm_backend_composer_state should route cmux to its classifier, got '$out'" + pass "fm_backend_composer_state: routes cmux to the cmux composer classifier" +} + # --- ping_state / ensure_running --------------------------------------------- test_ping_state_ok() { @@ -884,6 +897,7 @@ test_scoped_title_uses_secondmate_home_label test_scoped_title_changes_with_root_path test_dispatch_routes_cmux_backend test_dispatch_busy_state_unknown_for_cmux +test_dispatch_composer_state_routes_cmux test_ping_state_ok test_ping_state_denied test_ping_state_unauth From 3206f82ea41c851c0e3477d03e5e75e2da7b2336 Mon Sep 17 00:00:00 2001 From: kunchenguid Date: Sat, 4 Jul 2026 12:58:40 -0700 Subject: [PATCH 5/5] no-mistakes(document): Document AFK supervisor backend support --- .agents/skills/afk/SKILL.md | 7 ++++--- bin/fm-supervise-daemon.sh | 25 +++++++++++++------------ docs/architecture.md | 5 ++++- docs/configuration.md | 13 ++++++++++++- docs/herdr-backend.md | 2 +- docs/scripts.md | 4 ++-- 6 files changed, 36 insertions(+), 20 deletions(-) diff --git a/.agents/skills/afk/SKILL.md b/.agents/skills/afk/SKILL.md index 9f1bf0a5822..787bc185948 100644 --- a/.agents/skills/afk/SKILL.md +++ b/.agents/skills/afk/SKILL.md @@ -208,9 +208,10 @@ the marker lets firstmate distinguish it from a real captain message. `":"` target), then `$TMUX_PANE`, then `"${HERDR_SESSION:-default}:${HERDR_PANE_ID}"` under herdr, then a `firstmate:0` fallback with a warning. Both resolution sources are logged at - startup so a wrong-but-resolving fallback is detectable. zellij and orca are - not yet supported as a supervisor backend; the daemon refuses loudly at - startup instead of misapplying tmux primitives to a pane that isn't one + startup so a wrong-but-resolving fallback is detectable. Other runtime + backends, including zellij, orca, and cmux, are not yet supported as + supervisor backends; the daemon refuses loudly at startup instead of + misapplying tmux primitives to a pane that isn't one (docs/herdr-backend.md "Away-mode daemon: herdr supervisor-pane support"). ## Reliability properties diff --git a/bin/fm-supervise-daemon.sh b/bin/fm-supervise-daemon.sh index 4e70786b2ba..c3c2da9c36c 100755 --- a/bin/fm-supervise-daemon.sh +++ b/bin/fm-supervise-daemon.sh @@ -70,10 +70,10 @@ # resolves the runtime firstmate itself is # executing inside - $TMUX_PANE selects tmux, # $HERDR_ENV=1 selects herdr - falling back to -# tmux). zellij and orca are not yet supported -# as a supervisor backend; the daemon refuses -# loudly at startup rather than trying tmux -# primitives against a non-tmux pane. +# tmux). zellij, orca, and cmux are not yet +# supported as supervisor backends; the daemon +# refuses loudly at startup rather than trying +# tmux primitives against a non-tmux pane. # FM_INJECT_SKIP |-prefixes force-self-handle bypassing # classification (default "heartbeat"); empty # disables. Use sparingly: it overrides the @@ -142,12 +142,13 @@ FM_SUPERVISOR_TARGET_DEFAULT="firstmate:0" # detected) assumes tmux - matching this daemon's pre-herdr-support behavior # byte-for-byte when run outside both tmux and herdr. FM_SUPERVISOR_BACKEND_DEFAULT="tmux" -# Supervisor backends this daemon knows how to inject into today. zellij and -# orca are real backends elsewhere in firstmate (bin/fm-backend.sh) but this +# Supervisor backends this daemon knows how to inject into today. zellij, orca, +# and cmux are real backends elsewhere in firstmate (bin/fm-backend.sh) but this # daemon has no verified composer/busy primitives wired up for them yet - see -# docs/herdr-backend.md and AGENTS.md section 4's harness-verification -# discipline. Selecting one refuses loudly at startup instead of silently -# running tmux primitives against a pane that is not a tmux pane. +# docs/herdr-backend.md and AGENTS.md section 4's +# harness-verification discipline. Selecting one refuses loudly at startup +# instead of silently running tmux primitives against a pane that is not a tmux +# pane. FM_SUPERVISOR_SUPPORTED_BACKENDS="tmux herdr" INJECT_SKIP_DEFAULT="heartbeat" STALE_ESCALATE_SECS_DEFAULT=240 @@ -903,10 +904,10 @@ fm_super_main() { local BACKEND="$FM_SUPERVISOR_BACKEND" # --- refuse an unsupported supervisor backend loudly, before ever trying a - # tmux/herdr-specific call against it (zellij/orca have no verified + # tmux/herdr-specific call against it (zellij, orca, and cmux have no verified # composer/busy primitives wired up for this daemon yet - AGENTS.md section 4 - # harness-verification discipline). This is the "clear refusal" the task - # calls for, instead of a confusing "does not resolve to a tmux pane" error. + # harness-verification discipline). This is the clear refusal the task calls + # for, instead of a confusing "does not resolve to a tmux pane" error. if ! fm_backend_list_contains "$FM_SUPERVISOR_SUPPORTED_BACKENDS" "$BACKEND"; then echo "error: away-mode daemon does not support supervisor backend '$BACKEND' yet (supported: $FM_SUPERVISOR_SUPPORTED_BACKENDS); set FM_SUPERVISOR_BACKEND=tmux|herdr and FM_SUPERVISOR_TARGET to run firstmate's own pane under a supported backend" >&2 log "startup failed: unsupported supervisor backend '$BACKEND' (source=$backend_source)" diff --git a/docs/architecture.md b/docs/architecture.md index ff85dd5cc45..b20fca192ba 100644 --- a/docs/architecture.md +++ b/docs/architecture.md @@ -33,7 +33,10 @@ A presence-gated sub-supervisor (`bin/fm-supervise-daemon.sh`) extends this for The watcher and daemon share `bin/fm-classify-lib.sh` for captain-relevant status verbs and status-scan primitives. The always-on watcher also uses that library's provably-working predicate on no-verb signals and first-sighting stale panes before status-log terminality is trusted, while the daemon keeps its away-mode stale recheck unchanged. The daemon escalates only captain-relevant events as one batched, single-line digest (prefixed with an in-band sentinel marker so firstmate can tell daemon injections apart from real messages). -Its tmux supervisor injection path shares the same submit core used by the tmux send backend, so dim-ghost-aware and border-aware composer detection plus verified submit retry stay consistent; stalled escalation delivery raises `state/.subsuper-inject-wedged` after `FM_MAX_DEFER_SECS` instead of silently deferring forever. +Its supervisor injection path supports tmux and herdr panes, with `FM_SUPERVISOR_BACKEND` and `FM_SUPERVISOR_TARGET` resolved independently from the task-spawn backend. +Pane existence, busy checks, composer checks, capture, and verified submit route through `bin/fm-backend.sh`: tmux keeps the same submit core used by the tmux send backend, while herdr reuses its native busy state and structural composer classifier. +Unsupported supervisor backends refuse at daemon startup. +Stalled escalation delivery raises `state/.subsuper-inject-wedged` after `FM_MAX_DEFER_SECS` instead of silently deferring forever. `fm-send.sh` selects a pre-Enter popup-settle for slash commands and for codex `$...` skill invocations using the target's recorded `harness=` meta, then adds its own `FM_SEND_SETTLE` pause after successful text sends so immediate peeks catch the receiving turn starting; the sub-supervisor uses only the shared submit core and does not pay that post-submit pause. ## Runtime session backends diff --git a/docs/configuration.md b/docs/configuration.md index 278b4bfae98..0d780e2a39c 100644 --- a/docs/configuration.md +++ b/docs/configuration.md @@ -48,6 +48,16 @@ The caller-facing label remains `fm-`, but the actual cmux workspace title i Test cleanup must use the guarded path described in [`docs/cmux-backend.md`](cmux-backend.md)'s "Test safety" section, never enumerate-and-close every workspace. The `config/backend` file is not inherited by secondmate homes. +## Away-mode supervisor backend (FM_SUPERVISOR_BACKEND / FM_SUPERVISOR_TARGET) + +The `/afk` sub-supervisor injects escalation digests into firstmate's own pane independently of where new task endpoints are spawned. +It currently supports only `tmux` and `herdr` supervisor panes. +Set `FM_SUPERVISOR_BACKEND=tmux|herdr` and `FM_SUPERVISOR_TARGET=` to override both axes explicitly; for herdr the target is `":"`. +Without overrides, backend detection uses `$TMUX_PANE` first, then `HERDR_ENV=1` with `HERDR_PANE_ID`, then falls back to `tmux`. +That keeps a tmux pane nested inside herdr on the tmux transport, matching the runtime backend's innermost-first rule. +Target detection uses `FM_SUPERVISOR_TARGET`, then `$TMUX_PANE`, then `"${HERDR_SESSION:-default}:${HERDR_PANE_ID}"` under herdr, then the legacy `firstmate:0` tmux fallback with a warning. +Selecting any other supervisor backend, including `zellij`, `orca`, or `cmux`, refuses at daemon startup instead of trying tmux injection primitives against a non-tmux pane. + ## Gate defaults (.no-mistakes.yaml) The tracked `.no-mistakes.yaml` keeps test evidence outside the repo and defines `commands.test` so no-mistakes runs firstmate's bash behavior suite directly. @@ -251,7 +261,8 @@ FM_SEND_RETRIES=3 # fm-send Enter-retry attempts after typing the line onc FM_SEND_SLEEP=0.4 # seconds between fm-send submit checks FM_SEND_SETTLE=1 # seconds fm-send waits after a successful text submit; 0 disables # sub-supervisor (bin/fm-supervise-daemon.sh); presence-gated via /afk -FM_SUPERVISOR_TARGET=firstmate:0 # supervisor tmux target (override; auto-discovers from $TMUX_PANE) +FM_SUPERVISOR_BACKEND= # optional supervisor pane backend override; tmux/herdr only, otherwise detects $TMUX_PANE then HERDR_ENV/HERDR_PANE_ID before tmux fallback +FM_SUPERVISOR_TARGET= # optional supervisor pane target override; tmux target or herdr :, otherwise auto-detected FM_INJECT_SKIP=heartbeat # |-prefixes force-self-handled bypassing classification; empty disables FM_ESCALATE_BATCH_SECS=90 # buffer window for batched escalation digests; 0 = flush immediately FM_MAX_DEFER_SECS=300 # max buffered escalation age before retry plus wedge alarm; 0 disables diff --git a/docs/herdr-backend.md b/docs/herdr-backend.md index 62973bcb4f6..a78c37d323d 100644 --- a/docs/herdr-backend.md +++ b/docs/herdr-backend.md @@ -346,7 +346,7 @@ Classification policy, batching, the max-defer escape, the `FM_INJECT_MARK` sent **Discovery.** `FM_SUPERVISOR_TARGET` remains the explicit override, now accepting either a tmux target or a herdr `":"` target. A new `FM_SUPERVISOR_BACKEND` override (`tmux`|`herdr`) resolves independently, mirroring `bin/fm-backend.sh`'s own `fm_backend_detect`: `$TMUX_PANE` set selects tmux (even nested inside herdr, matching the innermost-first rule); `$HERDR_ENV=1` with `$HERDR_PANE_ID` present selects herdr, composing the target as `"${HERDR_SESSION:-default}:${HERDR_PANE_ID}"`; absent both, the daemon falls back to tmux/`firstmate:0`, byte-identical to its pre-herdr-support behavior. -zellij and orca are not yet supported as a supervisor backend - the daemon refuses loudly at startup (`FM_SUPERVISOR_SUPPORTED_BACKENDS="tmux herdr"`) rather than misapplying tmux primitives to a pane that isn't a tmux pane. +Other runtime backends, including zellij, orca, and cmux, are not yet supported as supervisor backends - the daemon refuses loudly at startup (`FM_SUPERVISOR_SUPPORTED_BACKENDS="tmux herdr"`) rather than misapplying tmux primitives to a pane that isn't a tmux pane. **Injection dispatch.** `inject_msg`'s pane-exists probe, busy-guard (`pane_is_busy`), composer-guard (`pane_input_pending`), and verified submit all take an optional `` argument (defaulting to `tmux` when omitted, so every pre-existing caller/test is unaffected) and route through the generic dispatchers instead of calling `tmux` directly. For `backend=tmux` every dispatch resolves to the exact same underlying call as before (`fm_backend_capture`'s tmux arm runs the identical `tmux capture-pane -p -t -S -40`; `fm_backend_tmux_send_text_submit` re-exports `fm_tmux_submit_core` verbatim), so tmux behavior is unchanged byte-for-byte. diff --git a/docs/scripts.md b/docs/scripts.md index 57f19048569..e21135dcc39 100644 --- a/docs/scripts.md +++ b/docs/scripts.md @@ -15,7 +15,7 @@ Each file also starts with a short header comment. | `fm-guard.sh` | Warn when the primary checkout is tangled, when queued wakes are pending, or when a stale or missing watcher needs a prominent banner; `FM_GUARD_READ_ONLY=1` keeps the alarms but suppresses drain, arm, and checkout repair commands | | `fm-home-seed.sh` | Lease/provision a secondmate home transactionally, clone projects, initialize gates, and maintain `data/secondmates.md` | | `fm-spawn.sh` | Spawn one task, several `id=repo` pairs, or a persistent secondmate with `--secondmate`; accepts concrete `--harness`, `--model`, `--effort`, and `--backend` axes; ship/scout spawns require an explicit resolved harness when dispatch profiles are active and an isolated worktree, install per-harness turn-end signaling, and secondmate spawns resolve the secondmate harness plus optional `config/secondmate-harness` model/effort tokens, locally sync the home, propagate declared inheritable config, land herdr tabs in the target home's workspace, land zellij tabs in the selected shared zellij session, land cmux workspaces in the shared cmux app, or create Orca worktrees/terminals before launch | -| `fm-backend.sh` | Runtime session-provider backend selector with explicit/env/config/runtime auto-detection precedence, meta helper, selector resolver, spawn-capability validation, operation dispatcher, and shell-portable backend-name membership for bash-sourced scripts or zsh-sourced diagnostics; defaults absent `backend=` meta to `tmux`; `fm_backend_target_exists` is a cheap read-only alive/dead endpoint check that never starts a server or session | +| `fm-backend.sh` | Runtime session-provider backend selector with explicit/env/config/runtime auto-detection precedence, meta helper, selector resolver, spawn-capability validation, operation dispatcher, and shell-portable backend-name membership for bash-sourced scripts or zsh-sourced diagnostics; defaults absent `backend=` meta to `tmux`; `fm_backend_target_exists` is a cheap read-only alive/dead endpoint check that never starts a server or session; `fm_backend_composer_state` exposes backend composer checks for guarded submit paths | | `backends/tmux.sh` | Verified tmux session-provider adapter used by `fm-backend.sh`; owns create, send, capture, current-path, live-window, and kill primitives | | `backends/herdr.sh` | Experimental herdr session-provider adapter used by `fm-backend.sh`; owns version/tool gating, per-home workspace/tab creation, created-vs-adopted default-tab prune safety, restored-layout husk respawn replacement, session-scoped CLI calls, send with structural composer-state verification, capture, native busy-state, current-path, label-based live discovery, and kill primitives | | `backends/zellij.sh` | Experimental zellij session-provider adapter used by `fm-backend.sh`; owns version/tool gating, one-session/tab-per-task creation, session-scoped CLI calls, send, capture, active current-path probing, label-checked target safety, label-based live discovery, and tab cleanup primitives | @@ -28,7 +28,7 @@ Each file also starts with a short header comment. | `fm-marker-lib.sh` | Shared from-firstmate request marker and detector sourced by `fm-send.sh`, `fm-brief.sh`, and tests | | `fm-watch-arm.sh` | Verified per-home watcher re-arm; reports `started`, `healthy`, or `FAILED`; `--restart` relaunches only this home's watcher | | `fm-watch.sh` | Singleton-safe always-on watcher and default pull-based event source; uses backend-native busy state when available before the shared regex fallback, absorbs no-verb signal and stale wakes only when the crew is provably working, checks that evidence before trusting stale status-log terminality, queues and exits for actionable wakes, and reverts to daemon-owned one-shot behavior while `state/.afk` exists | -| `fm-supervise-daemon.sh` | Presence-gated sub-supervisor for walk-away (`/afk`) supervision: wraps `fm-watch.sh`, uses the shared wake classifier and backend-aware stale rechecks, self-handles routine wakes in bash, and escalates only captain-relevant events as one verified, batched, single-line digest prefixed with a sentinel marker | +| `fm-supervise-daemon.sh` | Presence-gated sub-supervisor for walk-away (`/afk`) supervision: wraps `fm-watch.sh`, uses the shared wake classifier, backend-aware stale rechecks, and tmux/herdr supervisor injection, self-handles routine wakes in bash, and escalates only captain-relevant events as one verified, batched, single-line digest prefixed with a sentinel marker | | `fm-crew-state.sh` | Print one stable current-state line for a crew by reconciling its matching no-mistakes run-step, including coarse cross-branch attribution from `no-mistakes runs`, even when the pane has closed, with backend-aware pane fallback that corroborates native idle/unknown verdicts before the status log | | `fm-tangle-lib.sh` | Shared default-branch resolution and primary-checkout tangle classification sourced by bootstrap and guard | | `fm-ff-lib.sh` | Shared guarded fast-forward helper for `/updatefirstmate` origin pulls and no-fetch local secondmate syncs |