Skip to content
3 changes: 2 additions & 1 deletion .agents/skills/bootstrap-diagnostics/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,8 @@ When any diagnostic needs captain attention, report the plain consequence and re
- `MISSING: <tool> (install: <command>)` - list the missing tools to the captain with a one-line purpose each plus the printed install commands, wait for consent (one approval may cover the list), then run `bin/fm-bootstrap.sh install <approved tools...>`.
For `treehouse`, this also covers an installed version whose `treehouse get` lacks `--lease`; treat it as an upgrade request.
For `no-mistakes`, this also covers an installed version older than 1.31.2, because crewmate validation briefs delegate gate mechanics to no-mistakes' version-matched guidance.
For `tasks-axi`, this also covers an installed build that fails the compatibility probe (`docs/configuration.md` "Backlog backend" owns the definition); `config/backlog-backend=manual` only suppresses the verbose `BOOTSTRAP_INFO: tasks-axi available` fact, not this missing-tool report.
For `gh-axi`, this also covers an installed version below the bootstrap-owned floor; treat it as an upgrade request so non-interactive PR merges keep a working bare `--squash` shorthand.
For `tasks-axi`, this also covers an installed build that fails the compatibility probe (`bin/fm-tasks-axi-lib.sh` owns the definition); `config/backlog-backend=manual` only suppresses the verbose `BOOTSTRAP_INFO: tasks-axi available` fact, not this missing-tool report.
For `quota-axi`, bootstrap requires it because firstmate reads its current output directly before resolving every crew-dispatch profile array; without it, report the missing requirement and do not choose around an unexamined candidate.
- `MISSING_MANUAL: <tool> (instructions: <url>)` - tell the captain why the tool is required and give them the printed instructions URL, but do not pass the tool to `bin/fm-bootstrap.sh install`; wait for the captain to complete the manual installation, then rerun session start to confirm the dependency is present.
- `BACKEND_INVALID: <name> (known: <names>)` - the resolved runtime backend has no verified dependency or lifecycle contract, so do not dispatch work until the invalid `FM_BACKEND` or `config/backend` value is corrected to one of the listed backends.
Expand Down
2 changes: 1 addition & 1 deletion .agents/skills/secondmate-provisioning/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -34,7 +34,7 @@ Each registry entry stays concise and single-line: the summary is one sentence n
Natural-language summary and `scope:` text may contain parentheses and semicolons; keep the generated `(home: ...; scope: ...; projects: ...; added ...)` suffix intact so operational consumers resolve its explicit field markers.
The `home:` path points to the seeded home containing `data/charter.md`; no extra registry pointer field is needed.
For a remote route, `host:` is an OpenSSH config alias and `root:` is that host's separate tracked Firstmate code root.
Host placement is independent from the remote home's ordinary local runtime backend.
A remote second-mate agent always runs on the Herdr backend and every seed, launch, and liveness relaunch first gates its host on `bin/fm-remote-doctor.sh` readiness, so an unready host refuses with that doctor's own gap text rather than half-creating a route; the workers that second mate supervises keep the home's ordinary backend selection.
This release places whole secondmate homes remotely and never individual workers.
[`docs/remote-secondmates.md`](../../../docs/remote-secondmates.md) owns current operator setup and transport behavior.
The home-seeded `data/charter.md` is the sole owner of boilerplate idle-by-default behavior, the normal delegation lifecycle, and standard escalation contracts, so point to that charter rather than restating those contracts in the registry entry.
Expand Down
2 changes: 1 addition & 1 deletion AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -91,7 +91,7 @@ state/ volatile runtime signals; gitignored
<id>.turn-ended touched by turn-end hooks
<id>.grok-turnend-token firstmate-owned grok hook registry token for the task; removed by teardown
<id>.kimi-turnend-token firstmate-owned Kimi hook registry token for the task; removed by teardown
<id>.meta written by fm-spawn: window=, endpoint_task_id=, worktree=, project=, harness=, model=, effort=, kind=, mode=, yolo=, tasktmp=; an optional traceparent= only when trace context is enabled (docs/configuration.md "Trace context propagation"); kind=secondmate also records home= and projects=, plus remote_host=/remote_root=/remote_backend=/remote_target= for a remote route; a non-default runtime backend records further backend-specific fields (docs/configuration.md "Runtime backend"; bin/fm-backend.sh, section 8); fm-pr-check, including through fm-pr-merge, records one canonical pr= and the forge's pr_head= when available (GitHub pull requests and GitLab merge requests; docs/gitlab-merge-watch.md); fm-x-link appends x_request=, x_request_ts=, x_followups=, and optional x_platform=/x_reply_max_chars= for an X-mode-originated task (section 14)
<id>.meta written by fm-spawn: window=, endpoint_task_id=, worktree=, project=, harness=, model=, effort=, kind=, mode=, yolo=, tasktmp=; an optional traceparent= only when trace context is enabled (docs/configuration.md "Trace context propagation"); kind=secondmate also records home= and projects=, plus remote_host=/remote_root=/remote_backend=/remote_herdr_session=/remote_target= for a remote route; a non-default runtime backend records further backend-specific fields (docs/configuration.md "Runtime backend"; bin/fm-backend.sh, section 8); fm-pr-check, including through fm-pr-merge, records one canonical pr= and the forge's pr_head= when available (GitHub pull requests and GitLab merge requests; docs/gitlab-merge-watch.md); fm-x-link appends x_request=, x_request_ts=, x_followups=, and optional x_platform=/x_reply_max_chars= for an X-mode-originated task (section 14)
<id>.herdr-presentation quarantinable attempt and restart-binding journal for Herdr's optional visual projection; never task or endpoint authority; see docs/herdr-backend.md "Optional presentation spaces"
<id>.check.sh authenticated slow poll; the watcher dispatches validated PR data and the byte-identified X shim through trusted repository scripts, runs registered custom checks from hash-validated private snapshots, and rejects every other state check without execution
<id>.check-trust private content binding created by fm-check-register.sh for an intentional custom check
Expand Down
2 changes: 1 addition & 1 deletion bin/fm-backlog-handoff.sh
Original file line number Diff line number Diff line change
Expand Up @@ -309,7 +309,7 @@ remote_deliver_outbox() { # <secondmate-id> <outbox-path>
fi
rm -f -- "$snapshot"
if ! receive_out=$("$SCRIPT_DIR/fm-on.sh" "$id" fm-backlog-receive.sh \
"$remote_rel" "$bytes" "$hash" "$generation" 2>&1); then
"$remote_rel" "$bytes" "$hash" "$generation" < /dev/null 2>&1); then
[ -z "$receive_out" ] || printf '%s\n' "$receive_out" >&2
echo "error: handoff receipt by $id was unavailable or completion is unknown; outbox preserved at $outbox" >&2
return 1
Expand Down
49 changes: 45 additions & 4 deletions bin/fm-bootstrap.sh
Original file line number Diff line number Diff line change
Expand Up @@ -52,8 +52,11 @@
# "treehouse get --lease" support.
# no-mistakes is also MISSING when its installed version is older than
# 1.31.2.
# gh-axi is also MISSING when its installed version is older than
# 0.1.29, the first release whose bare --squash shorthand works for
# firstmate's non-interactive PR merge path.
# tasks-axi and quota-axi are required bootstrap tools (same class as
# lavish-axi). tasks-axi is also version and feature gated (0.1.1+
# lavish-axi). tasks-axi is also version and feature gated (0.2.2+
# with update --archive-body and mv [<id>...]); an installed but
# incompatible build reports MISSING like no-mistakes. A compatible
# tasks-axi default backend is silent. quota-axi is required for the
Expand Down Expand Up @@ -120,6 +123,8 @@ DATA="${FM_DATA_OVERRIDE:-$FM_HOME/data}"
. "$SCRIPT_DIR/fm-x-lib.sh"
# shellcheck source=bin/fm-backend.sh disable=SC1091
. "$SCRIPT_DIR/fm-backend.sh"
# shellcheck source=bin/fm-remote-readiness-lib.sh disable=SC1091
. "$SCRIPT_DIR/fm-remote-readiness-lib.sh"

fleet_sync_origin_backed_project_count() {
local count proj
Expand Down Expand Up @@ -457,7 +462,7 @@ secondmate_sync() {
fi
nudge_needed=0
converged=1
if sync_out=$("$SCRIPT_DIR/fm-on.sh" "$id" fm-remote-secondmate-control.sh sync "$id" 2>&1); then
if sync_out=$("$SCRIPT_DIR/fm-on.sh" "$id" fm-remote-secondmate-control.sh sync "$id" < /dev/null 2>&1); then
case "$sync_out" in synced:*) nudge_needed=1 ;; esac
else
echo "SECONDMATE_SYNC: secondmate $id: skipped: remote tracked-file sync failed on $remote_host: $(first_line "$sync_out")"
Expand Down Expand Up @@ -500,7 +505,7 @@ secondmate_liveness_sweep() {
# primary-only no-op there. Mid-session liveness remains explicitly out of
# scope and requires a separate periodic signal.
[ -d "$STATE" ] || return 0
local meta id window harness backend target agent_state out cause remote_host remote_rc
local meta id window harness backend target agent_state out cause remote_host remote_rc readiness_reason route_out remote_backend
SECONDMATE_RESPAWNED_IDS=""
for meta in "$STATE"/*.meta; do
[ -f "$meta" ] || continue
Expand All @@ -511,7 +516,21 @@ secondmate_liveness_sweep() {
harness=$(fm_meta_get "$meta" harness)
remote_host=$(fm_meta_get "$meta" remote_host)
if [ -n "$remote_host" ]; then
if out=$("$SCRIPT_DIR/fm-on.sh" "$id" fm-remote-secondmate-control.sh state "$id" 2>/dev/null); then
remote_rc=0
fm_remote_readiness_ensure "$SCRIPT_DIR" "$id" || remote_rc=$?
if [ "$remote_rc" -eq 255 ]; then
echo "SECONDMATE_LIVENESS: secondmate $id: skipped: remote host unavailable or endpoint state unknown; route preserved on $remote_host"
continue
fi
if [ "$remote_rc" -ne 0 ]; then
readiness_reason=$(printf '%s\n' "$FM_REMOTE_READINESS_OUT" \
| awk '/^check [^=]+=(fixable|human):|^action:|^error:/ { print; exit }')
[ -n "$readiness_reason" ] || readiness_reason=$(first_line "$FM_REMOTE_READINESS_OUT")
[ -n "$readiness_reason" ] || readiness_reason="unknown readiness failure"
echo "SECONDMATE_LIVENESS: secondmate $id: skipped: remote readiness failed on $remote_host: $readiness_reason"
continue
fi
if out=$("$SCRIPT_DIR/fm-on.sh" "$id" fm-remote-secondmate-control.sh state "$id" < /dev/null 2>/dev/null); then
remote_rc=0
else
remote_rc=$?
Expand All @@ -527,6 +546,24 @@ secondmate_liveness_sweep() {
agent_state=$(printf '%s\n' "$out" | tail -1)
case "$agent_state" in
alive)
if route_out=$("$SCRIPT_DIR/fm-on.sh" "$id" fm-remote-secondmate-control.sh route "$id" < /dev/null 2>/dev/null); then
remote_rc=0
else
remote_rc=$?
fi
if [ "$remote_rc" -eq 255 ]; then
echo "SECONDMATE_LIVENESS: secondmate $id: skipped: remote host unavailable or endpoint route unknown; route preserved on $remote_host"
continue
fi
if [ "$remote_rc" -ne 0 ]; then
echo "SECONDMATE_LIVENESS: secondmate $id: skipped: alive remote endpoint route is unreadable on $remote_host; inspect and migrate or retire it explicitly"
continue
fi
remote_backend=$(printf '%s\n' "$route_out" | sed -n 's/^backend=//p' | tail -1)
if [ "$remote_backend" != herdr ]; then
echo "SECONDMATE_LIVENESS: secondmate $id: skipped: alive remote endpoint is recorded on backend '${remote_backend:-missing}'; migrate or retire it explicitly"
continue
fi
[ "${FM_BOOTSTRAP_VERBOSE_FACTS:-0}" != 1 ] || echo "BOOTSTRAP_INFO: remote secondmate $id already live (host=$remote_host)"
;;
dead|missing)
Expand Down Expand Up @@ -656,6 +693,7 @@ if ! BACKEND_TOOLS=$(fm_backend_required_tools "$BACKEND"); then
fi
TOOLS="$BACKEND_TOOLS $COMMON_TOOLS"
NO_MISTAKES_MIN=1.31.2
GH_AXI_MIN=0.1.29

treehouse_supports_lease() {
treehouse get --help 2>&1 | grep -Eq '(^|[^[:alnum:]_-])--lease([^[:alnum:]_-]|$)'
Expand Down Expand Up @@ -995,6 +1033,9 @@ fi
if command -v no-mistakes >/dev/null 2>&1 && ! tool_version_at_least no-mistakes "$NO_MISTAKES_MIN"; then
echo "MISSING: no-mistakes (install: $(install_cmd no-mistakes))"
fi
if command -v gh-axi >/dev/null 2>&1 && ! tool_version_at_least gh-axi "$GH_AXI_MIN"; then
echo "MISSING: gh-axi (install: $(install_cmd gh-axi))"
fi
if command -v quota-axi >/dev/null 2>&1 && ! fm_quota_axi_compatible; then
echo "MISSING: quota-axi (install: $(install_cmd quota-axi))"
fi
Expand Down
4 changes: 2 additions & 2 deletions bin/fm-fleet-snapshot.sh
Original file line number Diff line number Diff line change
Expand Up @@ -481,7 +481,7 @@ task_json_lines() {
agent_alive=not_checked
if [ -n "$remote_host" ]; then
if remote_state=$(run_timed "$FM_SNAPSHOT_SECONDMATE_TIMEOUT" \
"$SCRIPT_DIR/fm-on.sh" "$id" fm-remote-secondmate-control.sh state "$id" 2>/dev/null); then
"$SCRIPT_DIR/fm-on.sh" "$id" fm-remote-secondmate-control.sh state "$id" < /dev/null 2>/dev/null); then
remote_rc=0
else
remote_rc=$?
Expand Down Expand Up @@ -1200,7 +1200,7 @@ secondmate_current_json() { # <parent-tasks-json>
if [ -z "$reason" ]; then
if [ "$remote" = true ]; then
summary=$(run_timed "$FM_SNAPSHOT_SECONDMATE_TIMEOUT" \
"$SCRIPT_DIR/fm-on.sh" "$id" fm-fleet-snapshot.sh --secondmate-home-summary 2>/dev/null)
"$SCRIPT_DIR/fm-on.sh" "$id" fm-fleet-snapshot.sh --secondmate-home-summary < /dev/null 2>/dev/null)
summary_rc=$?
else
summary=$(run_timed "$FM_SNAPSHOT_SECONDMATE_TIMEOUT" env \
Expand Down
16 changes: 16 additions & 0 deletions bin/fm-on.sh
Original file line number Diff line number Diff line change
Expand Up @@ -21,6 +21,14 @@
# This command explicitly disables agent forwarding, forwarding setup, and
# configured SendEnv patterns. The remote entrypoint executes the selected
# command under an empty environment with only its fixed runtime values.
#
# ServerAliveInterval/ServerAliveCountMax arm dead-peer detection so a vanished
# peer (a reboot, a dropped link) becomes a bounded ssh failure (exit 255)
# instead of an indefinite hang on a half-open TCP connection. The remote
# sshd answers keepalive probes independently of whatever the remote command
# is doing, so a legitimately long-but-alive remote command is never falsely
# killed. FM_SSH_ALIVE_INTERVAL and FM_SSH_ALIVE_COUNT_MAX override the
# defaults; the worst-case detection window is roughly interval * count.
set -eu

SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
Expand Down Expand Up @@ -88,9 +96,17 @@ ROOT_B64=$(printf '%s' "$ROOT" | encode_base64)
HOME_B64=$(printf '%s' "$HOME_PATH" | encode_base64)
ARGV_B64=$(printf '%s\0' "$COMMAND" "$@" | encode_base64)
SSH_BIN=${FM_SSH_BIN:-ssh}
ALIVE_INTERVAL=${FM_SSH_ALIVE_INTERVAL:-15}
ALIVE_COUNT_MAX=${FM_SSH_ALIVE_COUNT_MAX:-3}
case "$ALIVE_INTERVAL" in ''|*[!0-9]*) die "FM_SSH_ALIVE_INTERVAL must be a positive integer: $ALIVE_INTERVAL" ;; esac
case "$ALIVE_COUNT_MAX" in ''|*[!0-9]*) die "FM_SSH_ALIVE_COUNT_MAX must be a positive integer: $ALIVE_COUNT_MAX" ;; esac
[ "$ALIVE_INTERVAL" -gt 0 ] || die "FM_SSH_ALIVE_INTERVAL must be a positive integer: $ALIVE_INTERVAL"
[ "$ALIVE_COUNT_MAX" -gt 0 ] || die "FM_SSH_ALIVE_COUNT_MAX must be a positive integer: $ALIVE_COUNT_MAX"

"$SSH_BIN" \
-o ForwardAgent=no \
-o ClearAllForwardings=yes \
-o 'SendEnv=-*' \
-o "ServerAliveInterval=$ALIVE_INTERVAL" \
-o "ServerAliveCountMax=$ALIVE_COUNT_MAX" \
-- "$HOST" fm-remote-entrypoint.sh "$PROTOCOL" "$ROOT_B64" "$HOME_B64" "$ARGV_B64"
2 changes: 1 addition & 1 deletion bin/fm-pending-reply-lib.sh
Original file line number Diff line number Diff line change
Expand Up @@ -1041,7 +1041,7 @@ fm_pending_reply_tick() { # <state-dir>
if [ "$found" = 0 ]; then
if [ -n "$remote_host" ]; then
observation=$("$_FM_PENDING_REPLY_LIB_DIR/fm-on.sh" "$task_id" \
fm-remote-secondmate-control.sh observe "$task_id" 2>/dev/null || printf 'unknown')
fm-remote-secondmate-control.sh observe "$task_id" < /dev/null 2>/dev/null || printf 'unknown')
case "$observation" in busy|idle|fallback-idle|unknown) ;; *) observation=unknown ;; esac
else
observation=$(fm_pending_reply_backend_observation "$backend" "$target" "$label" "$harness")
Expand Down
4 changes: 2 additions & 2 deletions bin/fm-procevent-remote-reply.sh
Original file line number Diff line number Diff line change
Expand Up @@ -194,7 +194,7 @@ cmd_source() {
validate_id "$id"
read_cursor "$id"
exec "$SCRIPT_DIR/fm-on.sh" "$id" fm-remote-delta-read.sh \
"$REMOTE_LOG" "$CURSOR_OFFSET" "$CURSOR_HASH" "$WAIT_SECONDS"
"$REMOTE_LOG" "$CURSOR_OFFSET" "$CURSOR_HASH" "$WAIT_SECONDS" < /dev/null
}

safe_doc_path() {
Expand All @@ -219,7 +219,7 @@ fetch_document() { # <id> <remote-relative> <result-var>
case "$parent_real" in "$base"|"$base"/*) ;; *) return 1 ;; esac
[ ! -L "$destination" ] || return 1
tmp=$(umask 077; mktemp "$parent/.remote-doc.XXXXXX") || return 1
if ! "$SCRIPT_DIR/fm-on.sh" "$id" fm-remote-file.sh get "$rel" "$MAX_DOC_BYTES" > "$tmp"; then
if ! "$SCRIPT_DIR/fm-on.sh" "$id" fm-remote-file.sh get "$rel" "$MAX_DOC_BYTES" < /dev/null > "$tmp"; then
rm -f -- "$tmp"
return 1
fi
Expand Down
4 changes: 4 additions & 0 deletions bin/fm-quota-axi-lib.sh
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,10 @@
# those fields a dispatch candidate cannot be checked against the authentication
# surface it actually uses, which is how one harness's expired CLI token used to
# produce a captain-facing sign-out claim for a candidate that never read it.
# 0.1.16 already emits schemaVersion 3 with per-model effectiveAvailability;
# 0.1.17 only adds optional runway under that same schema. quota-array-dispatch
# treats absent runway or pace as disclosed uncertainty, so the floor stays
# 0.1.16 rather than tracking the latest additive field.
#
# This file is the single owner of that version number. bin/fm-bootstrap.sh
# turns a failing check into the operator-facing MISSING diagnostic, which is
Expand Down
Loading