Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 3 additions & 1 deletion .agents/skills/bootstrap-diagnostics/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@
name: bootstrap-diagnostics
description: >-
Agent-only handling playbook for session-start bootstrap diagnostics.
Use whenever the session-start digest's bootstrap or network-checks section prints an actionable diagnostic line - MISSING, MISSING_MANUAL, PRESENTATION_UNAVAILABLE, BACKEND_INVALID, NEEDS_GH_AUTH, TANGLE, STARTUP_MEMORY_BUDGET, CREW_DISPATCH invalid, FLEET_SYNC, NETWORK_CHECKS, HOME_SUMMARY, BACKLOG_RECONCILE, SECONDMATE_SYNC, SECONDMATE_LIVENESS, SECONDMATE_HANDOFF, NUDGE_SECONDMATES, or FMX - or reports that an interrupted backlog cleanup may have left an endpoint or local copy, or when a standalone bin/fm-bootstrap.sh or bin/fm-startup-network.sh run prints one of those lines.
Use whenever the session-start digest's bootstrap or network-checks section prints an actionable diagnostic line - MISSING, MISSING_MANUAL, PRESENTATION_UNAVAILABLE, BACKEND_INVALID, NEEDS_GH_AUTH, TANGLE, STARTUP_MEMORY_BUDGET, CREW_DISPATCH invalid, FLEET_SYNC, NETWORK_CHECKS, HOME_SUMMARY, BACKLOG_RECONCILE, SECONDMATE_SYNC, SECONDMATE_LIVENESS, SECONDMATE_HANDOFF, NUDGE_SECONDMATES, LOAD_GUARD, or FMX - or reports that an interrupted backlog cleanup may have left an endpoint or local copy, or when a standalone bin/fm-bootstrap.sh or bin/fm-startup-network.sh run prints one of those lines.
A silent bootstrap section, or any other BOOTSTRAP_INFO fact, means no skill load.
user-invocable: false
metadata:
Expand Down Expand Up @@ -73,5 +73,7 @@ When any diagnostic needs captain attention, report the plain consequence and re
An unsafe-outbox variant requires path and file-type inspection before any retry.
- `NUDGE_SECONDMATES: secondmate <id>: send failed: <reason>` - secondmate convergence changed a running home's loaded instructions or inherited config, but the deterministic `fm-send.sh fm-<id>` re-read nudge failed.
Inspect the reason, keep the pending marker under `state/.secondmate-nudge-pending/` intact, and rerun session start after the endpoint or metadata issue is fixed so bootstrap can retry the exact same marked send on the same local or remote route.
- `LOAD_GUARD: memory and CPU monitoring could not be armed - <reason>` - this home is not watching the machine's memory and CPU, so nothing will warn before the fleet exhausts it.
Fix the named reason - usually an unusable `state/` directory or a symlink at `state/load-guard.check.sh` - then run `bin/fm-load-guard.sh arm` or rerun session start; never hand-write the shim, and opt out only through `config/load-guard` (`docs/configuration.md` "Load guard").
- `FMX: X mode on ...` / `FMX: X mode off ...` - bootstrap confirmed or removed the local Relay poll artifacts (`docs/configuration.md` "Relay (.env)"); the emitted line still carries Relay's former `X mode` wording.
Only when a running watcher needs the cadence transition applied immediately, restart the home-scoped watcher through the emitted harness supervision protocol; bootstrap deliberately never restarts the watcher itself.
19 changes: 18 additions & 1 deletion .agents/skills/housekeeping/SKILL.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
---
name: housekeeping
description: Reclaim disk taken by finished crew work - stale worktrees, and the containers, volumes, build cache, and images left behind by isolated stacks. Use when the captain invokes /housekeeping, asks to reclaim disk or clean up after finished work, or reports that the machine is low on space.
description: Reclaim disk taken by finished crew work - stale worktrees, and the containers, volumes, build cache, and images left behind by isolated stacks - and shed fleet load when this machine runs short of memory or CPU. Use when the captain invokes /housekeeping, asks to reclaim disk or clean up after finished work, reports that the machine is low on space, or on a `check:` wake whose line starts `load-guard:`.
user-invocable: true
metadata:
internal: true
Expand Down Expand Up @@ -115,3 +115,20 @@ If verification reports that a protected container is no longer running, say so

Report in the captain's terms, per `AGENTS.md` section 9: space reclaimed, what was kept and why, worktrees left alone because they hold unlanded work, and anything the sweep could not attribute.
Container IDs, image digests, volume hashes, and raw prune output are evidence you read, not the report you send.

## Load-guard wakes

A `check:` line starting `load-guard:` comes from `bin/fm-load-guard.sh`, which owns the measurement, thresholds, and repeat suppression (`docs/configuration.md` "Load guard").
It means this machine is short of memory or CPU right now, so respond before doing anything that adds load.

1. Identify whose load it is.
Compare this home's live workers, from the structured fleet view, with the machine's top memory and CPU consumers, for example from `ps -axo pid,ppid,rss,%cpu,comm`.
A worker counts as this home's only when its recorded endpoint or process tree ties it to a task this home owns.
2. While the condition persists, throttle this home's own workers: tell each to run one job at a time and to start no helper agents or parallel subprocesses, and hold every new spawn.
`bin/fm-spawn.sh` already warns over a threshold and refuses new work below the memory floor, while recovery of a task that already exists only warns; an override reason is for a launch the captain explicitly wants anyway, never a way around the hold.
3. Never kill or signal a process this home does not own, and never kill by name or pattern.
Stopping one of this home's own workers goes through `bin/fm-control.sh`, never a direct kill, and never discards unlanded work.
4. When the load is not the fleet's, tell the captain plainly which processes are using the memory or CPU, and that the fleet is holding back rather than adding to it.
5. Lift the throttle and release held spawns once the guard stops reporting and `bin/fm-load-guard.sh status` prints `verdict=ok`.

A wake that names `config/load-guard` as invalid is a configuration problem rather than load: report the named error so the file can be corrected.
5 changes: 4 additions & 1 deletion AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -92,6 +92,7 @@ config/wedge-defer-parked-gate optional presence flag opting this home into the
config/cmux-socket-password optional cmux control-socket password; LOCAL, gitignored; read fresh on every cmux CLI call and passed through without ever overriding an operator's own ambient CMUX_SOCKET_PASSWORD when absent (docs/cmux-backend.md "Setup")
config/wedge-alarm optional away-mode wedge-alarm active-alert directives; LOCAL, gitignored; absent means auto (macOS Notification Center when available); see docs/wedge-alarm.md
config/watched-tools.json optional list of the tools this home depends on, read by the update check armed with bin/fm-tool-update-check.sh; LOCAL, gitignored, firstmate-maintained but human-editable, and NOT inherited by secondmate homes; see docs/configuration.md "Watched tool updates"
config/load-guard optional memory and CPU load-guard thresholds or an "off" opt-out; LOCAL, gitignored, and not inherited; see docs/configuration.md "Load guard"
config/x-mode.env generated Relay watcher cadence; LOCAL, gitignored; source before arming watcher when present
data/ personal fleet records; LOCAL, gitignored as a whole
backlog.md task queue, dependencies, history
Expand Down Expand Up @@ -135,6 +136,7 @@ state/ runtime records and signals; gitignored
x-watch.check.sh generated Relay poll shim; present only when opted in (section 14)
tool-updates.check.sh generated watched-tool update poll shim and its .check-trust binding; present only after bin/fm-tool-update-check.sh arm; its report record .tool-updates is what keeps one pending update from being reported on every poll
mail.check.sh generated received-mail poll shim and its .check-trust binding; present only after bin/fm-mail-check.sh arm; report record .mail-check (mail schema: docs/configuration.md "Mail plane")
load-guard.check.sh generated memory and CPU load-guard shim and its .check-trust binding, armed by every locked session start unless config/load-guard is off; episode record .load-guard keeps a sustained condition from waking every poll
.mail-seen .mail-woken .mail-retry .mail-retry-pos .mail-turn .mail-seen.lock mail-plane poll cursor, emission journal, transient-fetch retry set, retry-scan position, contended-slot turn flag, and overlapping-poll lock; written only by bin/fm-mail.sh (mail schema: docs/configuration.md "Mail plane")
pending-replies/ parent-owned secondmate pending-reply records (correlation id, delivery vs reply, recovery, escalation); fm-pending-reply-lib.sh
procevent/ registered process-to-event sources, one private record per canonical source id; written only by bin/fm-procevent.sh, and their presence alone keeps supervision required (section 13)
Expand Down Expand Up @@ -461,6 +463,7 @@ Handle actionable wakes as follows:
3. For `check:`, act on the named poll result, including merges, contribution signals, Relay events, process-to-event source results, and captain inbox notes; a handled inbox note is also acknowledged with `bin/fm-inbox.sh drain --ack <id>`, or it stays counted as still waiting for firstmate.
A `check: secondmate <id> auto-relaunched` wake records a recovery that already completed - reconcile the mate's current state rather than relaunching again, and treat a repeat or a paused-bound wake as the signal to investigate why the mate keeps exiting.
When the note needs a durable answer the submitter can read, publish it with `bin/fm-inbox.sh reply <id>` (the script header owns the reply contract) rather than leaving the answer only in this transcript.
A `check:` line starting `load-guard:` means this machine is short of memory or CPU: load `housekeeping` and follow its load-guard section before spawning more work.
4. For `heartbeat:`, review the whole fleet from the structured fleet view, reconcile suspicious tasks and PR state, update the backlog, and never report an unchanged fleet as progress.

Load `bearings` on a contributions check wake or when filing work linked to an upstream issue; its contribution-follow-up section owns triage and exact signal acknowledgement.
Expand Down Expand Up @@ -602,7 +605,7 @@ It owns the omp update, the fast-forward, the merge into the omp adapter branch

These skills are not captain-invocable; load them only at their precise triggers.

- `bootstrap-diagnostics` - load whenever the session-start digest's bootstrap or network-checks section prints an actionable diagnostic line (`MISSING:`, `MISSING_MANUAL:`, `PRESENTATION_UNAVAILABLE:`, `BACKEND_INVALID:`, `NEEDS_GH_AUTH`, `TANGLE:`, `STARTUP_MEMORY_BUDGET:`, `CREW_DISPATCH: invalid`, `FLEET_SYNC:`, `NETWORK_CHECKS:`, `HOME_SUMMARY:`, `BACKLOG_RECONCILE:`, `SECONDMATE_SYNC:`, `SECONDMATE_LIVENESS:`, `SECONDMATE_HANDOFF:`, `NUDGE_SECONDMATES:`, or `FMX:`), or when `BOOTSTRAP_INFO:` says an interrupted backlog cleanup may have left an endpoint or local copy; silence and other `BOOTSTRAP_INFO:` facts need no load.
- `bootstrap-diagnostics` - load whenever the session-start digest's bootstrap or network-checks section prints an actionable diagnostic line (`MISSING:`, `MISSING_MANUAL:`, `PRESENTATION_UNAVAILABLE:`, `BACKEND_INVALID:`, `NEEDS_GH_AUTH`, `TANGLE:`, `STARTUP_MEMORY_BUDGET:`, `CREW_DISPATCH: invalid`, `FLEET_SYNC:`, `NETWORK_CHECKS:`, `HOME_SUMMARY:`, `BACKLOG_RECONCILE:`, `SECONDMATE_SYNC:`, `SECONDMATE_LIVENESS:`, `SECONDMATE_HANDOFF:`, `NUDGE_SECONDMATES:`, `LOAD_GUARD:`, or `FMX:`), or when `BOOTSTRAP_INFO:` says an interrupted backlog cleanup may have left an endpoint or local copy; silence and other `BOOTSTRAP_INFO:` facts need no load.
- `diagnostic-reasoning` - load before scoping a reported bug and before acting on a diagnostic report.
- `ask-user-authority` - load before deciding any ask-user finding.
- `quota-array-dispatch` - load before choosing among a matched crew-dispatch profile array from current quota-axi default TOON.
Expand Down
14 changes: 14 additions & 0 deletions bin/fm-bootstrap.sh
Original file line number Diff line number Diff line change
Expand Up @@ -22,7 +22,11 @@
# "BOOTSTRAP_INFO: nudged fm-<id> with '<message>'",
# "SECONDMATE_LIVENESS: secondmate <id>: skipped: <reason>|respawn failed after <cause>: <reason>",
# "SECONDMATE_HANDOFF: secondmate <id>: pending delivery: <n> item(s)",
# "LOAD_GUARD: memory and CPU monitoring could not be armed - <reason>",
# "FMX: X mode on ..." or "FMX: X mode off ...".
# On every locked run, bin/fm-load-guard.sh arm converges this home to
# an armed memory and CPU load guard, or to disarmed when
# config/load-guard is off; an already-armed home is not rewritten.
# When a RUNNING secondmate home is fast-forwarded, its target is
# firstmate's own current default-branch commit. A local worktree uses
# a purely local fast-forward with no origin fetch; a remote route hands
Expand Down Expand Up @@ -1583,6 +1587,16 @@ if [ "${FM_BOOTSTRAP_DETECT_ONLY:-0}" != 1 ]; then
"$SCRIPT_DIR/fm-contributions.sh" arm --if-owned >/dev/null \
|| echo "MISSING: contribution observation could not be armed; coverage is unconfirmed"
fi
# Converge every home to an armed memory and CPU load guard, or to disarmed
# when config/load-guard says off. Idempotent: an already-armed home is not
# rewritten. Local only, and locked like every other mutating sweep here.
if local_phase && [ -x "$SCRIPT_DIR/fm-load-guard.sh" ]; then
if ! load_guard_err=$(FM_HOME="$FM_HOME" FM_STATE_OVERRIDE="$STATE" FM_CONFIG_OVERRIDE="$CONFIG" \
"$SCRIPT_DIR/fm-load-guard.sh" arm 2>&1 >/dev/null); then
load_guard_err=${load_guard_err//$'\n'/ }
echo "LOAD_GUARD: memory and CPU monitoring could not be armed - ${load_guard_err:-no reason given}"
fi
fi
if [ -n "$fleet_sync_pid" ]; then
wait "$fleet_sync_pid" || true
cat "$fleet_sync_out"
Expand Down
Loading