Skip to content

feat(bin): add Prime Agent as a verified crewmate and scout harness - #2

Merged
karanmrn merged 6 commits into
mainfrom
fm/prime-agent-harness-verify
Sep 15, 2026
Merged

karanmrn merged 6 commits into
mainfrom
fm/prime-agent-harness-verify

Conversation

@karanmrn

Copy link
Copy Markdown
Owner

Intent

Captain (13 Sep 2026, from data/harness-inventory-13sep/report.md part B): verify Prime Agent as a firstmate worker harness under the harness-adapters verify plan. Two known entry blockers: the PI_CODING_AGENT marker collision with the pi adapter, and the 366k-token skills prompt from ~/.agents/skills. Also clear the two idle Prime Agent sessions the 13 Sep probes left open.

What Changed

  • Adds prime (Prime Agent) as a verified crewmate and scout harness. bin/fm-harness.sh checks for PRIME_AGENT_KERNEL_OWNER_PID, PRIME_AGENT_INTERNAL_DAEMON_WORKER, or the prime-agent process title before it checks PI_CODING_AGENT=true. This stops a Prime worker from showing as pi. fm-spawn.sh starts it with --no-skills --no-session, which keeps the startup prompt small and stops the worker when its TUI exits. fm-spawn.sh also maps shared effort levels to --thinking. Bootstrap validation, quota provider mapping, and teardown now include prime.
  • Adds a per-task state/<id>.prime-ext.ts extension with the prime-ext busy source. It sets busy on agent_start. After agent_end, it polls ctx.isIdle() with no time limit and sets idle when that check is true. It also sets idle on session_shutdown and touches the turn-end marker on turn_end. In fm-control-lib.sh, Prime uses one C-c to interrupt and /quit to exit.
  • Prime runs only on the herdr backend and never as a secondmate. fm-spawn.sh refuses other backends before it creates an endpoint. fm-control.sh refuses a relaunch onto prime on another backend before it stops the running agent. New docs: the references/harness/prime.md skill reference and docs/verification/prime.md evidence. New tests: tests/fm-prime-harness.test.sh, the opt-in tests/fm-prime-herdr-live-e2e.test.sh, and Prime cases in the busy-wiring, control, and relaunch tests.

🤖 Generated with Claude Code

Risk Assessment

✅ Low: The change adds an opt-in, herdr-only crewmate adapter. Prime markers are checked before the Pi marker, and --no-skills bounds the startup prompt. A shared capability check in fm-control-lib.sh refuses Prime on other backends, in both spawn and relaunch, before anything stops. The idle poll has no cap, and a newer run or shutdown supersedes it. Tests cover the new behavior, and no existing adapter path changes behavior.

Testing

I drove the real prime-agent inside an isolated Herdr lab session through the exact launch that fm-spawn composes. Native detection, Prime-over-Pi harness detection, extension idle settling, the single Ctrl+C interrupt, and /quit worker cleanup all passed. A live token measurement confirmed that --no-skills cuts the prompt from 60,533 to 18,148 tokens. The real fm-spawn refuses Prime on tmux and zellij and records no task state. The 13 Sep probe sessions are archived. The relaunch refusal before stop and the uncapped idle poll were not driven live. Only the faked-tmux relaunch test and the virtual-time busy wiring test cover them. The Prime harness, busy wiring, control, and relaunch suites pass. The earlier baseline failure in fm-lint.test.sh is fixed. When I reported, the baseline rerun had finished 27 files with no failures and was still running.

  • Live validation: ✅ go - 8 of 10 scenarios driven live against the product
Scenario Result Live Evidence
Operator spawns a Prime crewmate on herdr: Herdr detects the pane as prime-agent ✅ pass live prime-herdr-live-e2e.txt
A tool inside a real Prime session, where PI_CODING_AGENT=true is set, detects harness prime instead of pi ✅ pass live prime-herdr-live-e2e.txt
Prime launch with --no-skills avoids the huge skills prompt (60,533 to 18,148 tokens with 480 skills) ✅ pass live prime-skills-prompt-budget.txt
Prime turn end: the generated extension settles busy state to idle and touches the turn-ended marker ✅ pass live prime-herdr-live-e2e.txt
One Ctrl+C cancels a running Prime turn, keeps the agent alive, and settles idle ✅ pass live prime-herdr-live-e2e.txt
/quit ends the client-owned Prime worker and its Python kernel, so no resident session is left ✅ pass live prime-herdr-live-e2e.txt
Adversarial: spawning Prime with --backend tmux or zellij is refused with a one-line reason, and no task state is written ✅ pass live prime-spawn-non-herdr-refusal.txt
Adversarial: relaunching a running tmux claude task onto prime refuses before stopping the current agent ⏸️ untested no The prior payload did not establish a live result. tmux is not installed on this host, so only the faked-tmux test tests/fm-control-relaunch.test.sh covered it. Run the relaunch on a host with tmux…
Prime idle poll after agent_end has no cap and never leaves busy stuck ⏸️ untested no The prior payload did not establish a live result. A live Prime turn that stays non-idle long after agent_end (for example auto-compaction) cannot be triggered on demand. Only the virtual-time test `t…
The two idle Prime Agent sessions from the 13 Sep probes are no longer open ✅ pass live prime-13sep-sessions.txt (both archived, no worker; 0 live)
Evidence: Live Prime Agent Herdr guard transcript

Source: Live Prime Agent Herdr guard transcript

# prime-agent 0.9.4 # herdr 0.9.0 # model openrouter/moonshotai/kimi-k2.6 ok - real herdr: detects the Prime pane natively as agent prime-agent ok - real prime-agent: a tool beside PI_CODING_AGENT=true detects harness prime ok - real prime-agent: the generated extension settles the turn idle and touches the turn-end marker ok - real prime-agent: one Ctrl+C cancels the run, keeps the agent, and settles idle ok - real prime-agent: /quit ends the client-owned worker and its Python kernel

# prime-agent 0.9.4 at ~/.local/bin/prime-agent
# herdr 0.9.0
# model openrouter/moonshotai/kimi-k2.6
ok - real herdr: detects the Prime pane natively as agent prime-agent
ok - real prime-agent: a tool beside PI_CODING_AGENT=true detects harness prime
ok - real prime-agent: the generated extension settles the turn idle and touches the turn-end marker
ok - real prime-agent: one Ctrl+C cancels the run, keeps the agent, and settles idle
ok - real prime-agent: /quit ends the client-owned worker and its Python kernel
exit=
Evidence: Skills prompt budget, live

Source: Skills prompt budget, live

480 skills; default: prompt_tokens 60533; --no-skills: prompt_tokens 18148

# skills in ~/.agents/skills: 480
$ prime-agent -p --mode json --no-session  --model openrouter/moonshotai/kimi-k2.6 'Reply with the single word OK.'  (cwd: firstmate worktree)
{"prompt_tokens":60533,"stop":"stop","text":" OK"}
$ prime-agent -p --mode json --no-session --no-skills --model openrouter/moonshotai/kimi-k2.6 'Reply with the single word OK.'  (cwd: firstmate worktree)
{"prompt_tokens":18148,"stop":"stop","text":" OK, captain."}
Evidence: Real fm-spawn refuses Prime on non-herdr backends

Source: Real fm-spawn refuses Prime on non-herdr backends

error: prime is verified on the herdr backend only; backend 'tmux' is unverified for Prime. Select --backend herdr or a different verified harness. exit=1 # state dir after refusal: .last-watcher-beat

# host: tmux=absent prime-agent=~/.local/bin/prime-agent herdr=/opt/homebrew/bin/herdr
$ bin/fm-spawn.sh prime-refuse-x1 <project> prime --mode no-mistakes --yolo off --backend tmux
error: prime is verified on the herdr backend only; backend 'tmux' is unverified for Prime. Select --backend herdr or a different verified harness.
exit=1
# state dir after refusal: .last-watcher-beat 
$ bin/fm-spawn.sh prime-refuse-x1 <project> prime --mode no-mistakes --yolo off --backend zellij
error: prime is verified on the herdr backend only; backend 'zellij' is unverified for Prime. Select --backend herdr or a different verified harness.
exit=1
# state dir after refusal: .last-watcher-beat 
Evidence: 13 Sep Prime probe sessions state

Source: 13 Sep Prime probe sessions state

01a09ad4-... archived idle 2026-09-13T12:53 no-worker 01a09ad3-... archived idle 2026-09-13T12:52 no-worker live sessions created 2026-09-13: 0

$ prime-agent list --all --json | jq (sessions created 2026-09-13)
01a09ad4-c574-71dc-bbe5-16f9458cd3fd	archived	idle	2026-09-13T12:53:50.580Z	~	no-worker
01a09ad3-d76c-774c-be70-54ac472284b4	archived	idle	2026-09-13T12:52:49.644Z	~	no-worker

$ count of live sessions created 2026-09-13
0
Evidence: Prime harness behavior tests

Source: Prime harness behavior tests

ok - either Prime marker outranks the Pi marker Prime sets for its own tools
ok - pi and pi-signed detection is unchanged when no Prime marker is set
ok - claude and cursor markers keep precedence over Prime markers
ok - prime is detected through an exact prime-agent ancestor only
ok - prime spawn on herdr launches without skills or a resident session and loads its extension
ok - prime spawn refuses a non-herdr backend before creating an endpoint
ok - prime launch clears foreign markers so a Prime tool detects prime
ok - prime maps the shared effort vocabulary to --thinking and keeps the provider-qualified model
ok - prime spawn refuses before creating an endpoint when prime-agent is absent
ok - prime is refused as a secondmate harness
Evidence: Prime busy-state wiring tests

Source: Prime busy-state wiring tests

ok - prime extension reports agent_start busy, settles idle only after ctx.isIdle(), and keeps turn_end a notification
ok - prime extension events from a superseded incarnation are rejected as stale
- Outcome: ⚠️ 1 error across 2 runs (3h44m38s)

Pipeline

Updates from git push no-mistakes

✅ **intent** - passed

✅ No issues found.

✅ **Rebase** - passed

✅ No issues found.

🔧 **Review** - 2 issues found → auto-fixed (2) ✅
  • ⚠️ bin/fm-spawn.sh:2801 - The Prime extension stops polling 30 seconds after agent_end. If ctx.isIdle() has not read true by then, nothing publishes idle, and the busy record stays busy prime-ext until the next agent_start. Example sequence: a long turn ends with agent_end, then Prime keeps the session non-idle for more than 30s (for example threshold auto-compaction over a large context) and settles without a new run. The poll gave up at IDLE_POLL_LIMIT_MS, so the crewmate classifies busy with no end. Consumers that wait for idle (inbox delivery, supervision) never see the turn finish. docs/verification/prime.md lists this case as not verified, and the comment says leaving busy is deliberate. That is why this is ask-user. Smallest remedy: keep polling with backoff until a newer agent_start or session_shutdown supersedes it, instead of a hard 30s give-up.
  • ⚠️ bin/backends/tmux.sh:172 - Prime is now dispatchable on every backend, including default tmux, but only Herdr was verified. fm_backend_tmux_classify_process_name has no prime-agent entry, and fm_harness_path_name has no prime-agent name. On tmux, a running Prime pane therefore classifies other, and fm_backend_agent_state reads ambiguous. As a result, bin/fm-control.sh &lt;id&gt; interrupt|exit refuses the task. The verification doc confirms this: 'the tmux control fake reads a Prime pane as ambiguous and refuses lifecycle verbs'. So an operator on the default backend can spawn a Prime crewmate that the control plane cannot interrupt or stop. The verify plan requires tmux liveness only for secondmate-capable adapters, so this does not contradict the plan. It is a product decision: either refuse Prime spawns on non-Herdr backends until tmux is verified, or verify and add prime-agent to the tmux classifier.

🔧 Fix applied.
1 warning still open:

  • ⚠️ bin/fm-spawn.sh:1398 - The fix round added a herdr-only refusal for Prime, but only bin/fm-spawn.sh checks it. fm-control.sh reaches that check after it has already stopped the running agent. Reachable path: a crewmate runs on the default tmux backend with harness=claude. The operator runs bin/fm-control.sh &lt;id&gt; relaunch --harness prime --note .... resolve_relaunch_profile accepts it, because fm_control_harness_supported prime and fm_control_harness_supports_kind prime ship both pass. Then do_relaunch runs safe_checkpoint and do_exit, which stops the claude worker. Next, fm-spawn.sh --relaunch takes BACKEND=tmux from meta and exits with "prime is verified on the herdr backend only". fm-control then dies with "the replacement agent ... could not be launched on prime". The task has no running worker. This breaks the invariant stated at bin/fm-control.sh:658-663: a launch that must be refused is refused before anything stops. tests/fm-control-relaunch.test.sh:741 already covers this for the task-kind boundary. Fix: move the backend boundary into the shared capability table in bin/fm-control-lib.sh, for example fm_control_harness_supports_backend &lt;harness&gt; &lt;backend&gt;. Call it from resolve_relaunch_profile before the stop, using the recorded $BACKEND, and from the bin/fm-spawn.sh refusal. Add a relaunch test like test_secondmate_relaunch_onto_a_crewmate_only_adapter_refuses_before_stop that proves the tmux agent is still running after the refusal.

🔧 Fix applied.
✅ Re-checked - no issues remain.

⚠️ **Test** - 1 error
  • 🚨 tests failed with exit code 1
  • ⚠️ tests/fm-lint.test.sh - The baseline command exits 1 because tests/fm-lint.test.sh reports 'not ok - changed-mode lint run failed'. It is a lint failure on the changed files, not a Prime behavior failure. This test step may not run linters, so the lint phase or CI must find and fix the specific lint error before merge.
  • Live validation: ✅ go - 7 of 10 scenarios driven live against the product
Scenario Result Live Evidence
Herdr detects a Prime crewmate pane launched with fm-spawn's command as agent prime-agent ✅ pass live prime-herdr-live-e2e.log: 'ok - real herdr: detects the Prime pane natively as agent prime-agent'
Marker collision: a tool inside a real Prime session, with PI_CODING_AGENT=true set, runs fm-harness.sh and gets prime, not pi ✅ pass live prime-herdr-live-e2e.log: 'ok - real prime-agent: a tool beside PI_CODING_AGENT=true detects harness prime'
The generated busy extension marks the turn idle and touches the turn-end marker after a real turn ✅ pass live prime-herdr-live-e2e.log: 'ok - real prime-agent: the generated extension settles the turn idle and touches the turn-end marker'
One Ctrl+C (the control-plane interrupt key) cancels a running 90s tool call without killing the agent, and the turn goes idle ✅ pass live prime-herdr-live-e2e.log: 'ok - real prime-agent: one Ctrl+C cancels the run, keeps the agent, and settles idle'
/quit (the control-plane exit command) ends the --no-session worker and its Python kernel, with no resident session left ✅ pass live prime-herdr-live-e2e.log: 'ok - real prime-agent: /quit ends the client-owned worker and its Python kernel'; pgrep shows no kernel-venv process afterwards
Skills prompt blocker: a --no-skills launch keeps the startup prompt small with ~/.agents/skills populated ✅ pass live prime-skills-prompt-budget.txt: default 41,433 tokens vs --no-skills 2,604 tokens from $HOME; tests/fm-prime-harness.test.sh checks that fm-spawn's launch includes --no-skills
Cleanup: the two idle 13 Sep Prime sessions (01a09ad3, 01a09ad4) are no longer open ✅ pass live prime-sessions-after.txt: both archived, isSessionActive=false; prime-agent list --json reports 0 active sessions
Adversarial: spawning harness prime on the tmux backend is refused before any endpoint, meta, or extension is created, and the error names the backend ⏸️ untested no The prior payload did not establish a live result. tmux is not installed on this host, so the real tmux backend cannot run. Only tests/fm-prime-harness.test.sh covers it, with the real fm-spawn.sh aga…
Adversarial: relaunching a running claude task on tmux onto harness prime is refused while the claude agent is still running ⏸️ untested no The prior payload did not establish a live result. tmux is not installed here, and a live run needs a real running tmux crewmate. Only tests/fm-control-relaunch.test.sh covers it, with the real fm-con…
Adversarial: prime is refused as a secondmate harness ⏸️ untested no The prior payload did not establish a live result. A live run would dispatch a real secondmate onto the fleet, which needs operator approval. Only tests/fm-prime-harness.test.sh covers it, with the re…
  • bin/fm-test-run.sh --changed --exclude-family real-herdr-gated
  • FM_PRIME_HERDR_LIVE=1 HERDR_LAB_HELPER=bin/fm-herdr-lab.sh tests/fm-prime-herdr-live-e2e.test.sh (real prime-agent 0.9.4, real herdr 0.9.0, openrouter/moonshotai/kimi-k2.6)
  • prime-agent -p --mode json --no-session [--no-skills] --model openrouter/moonshotai/kimi-k2.6 &#39;Reply with the single word OK.&#39; from $HOME, comparing prompt tokens
  • prime-agent list --json and prime-agent list --all --json to check the 13 Sep probe sessions
  • tests/fm-prime-harness.test.sh
  • tests/fm-control-relaunch.test.sh (includes test_relaunch_onto_prime_on_a_non_herdr_backend_refuses_before_stop)
  • bin/fm-test-run.sh --changed --exclude-family real-herdr-gated rerun, which identified the tests/fm-lint.test.sh failure
  • herdr session list and pgrep -fl &#39;prime-agent|kernel-venv&#39; to confirm lab teardown and no leftover workers

🔧 Fix applied.
1 error still open:

  • 🚨 tests failed with exit code 1
  • Live validation: ✅ go - 8 of 10 scenarios driven live against the product
Scenario Result Live Evidence
Operator spawns a Prime crewmate on herdr: Herdr detects the pane as prime-agent ✅ pass live prime-herdr-live-e2e.txt
A tool inside a real Prime session, where PI_CODING_AGENT=true is set, detects harness prime instead of pi ✅ pass live prime-herdr-live-e2e.txt
Prime launch with --no-skills avoids the huge skills prompt (60,533 to 18,148 tokens with 480 skills) ✅ pass live prime-skills-prompt-budget.txt
Prime turn end: the generated extension settles busy state to idle and touches the turn-ended marker ✅ pass live prime-herdr-live-e2e.txt
One Ctrl+C cancels a running Prime turn, keeps the agent alive, and settles idle ✅ pass live prime-herdr-live-e2e.txt
/quit ends the client-owned Prime worker and its Python kernel, so no resident session is left ✅ pass live prime-herdr-live-e2e.txt
Adversarial: spawning Prime with --backend tmux or zellij is refused with a one-line reason, and no task state is written ✅ pass live prime-spawn-non-herdr-refusal.txt
Adversarial: relaunching a running tmux claude task onto prime refuses before stopping the current agent ⏸️ untested no The prior payload did not establish a live result. tmux is not installed on this host, so only the faked-tmux test tests/fm-control-relaunch.test.sh covered it. Run the relaunch on a host with tmux…
Prime idle poll after agent_end has no cap and never leaves busy stuck ⏸️ untested no The prior payload did not establish a live result. A live Prime turn that stays non-idle long after agent_end (for example auto-compaction) cannot be triggered on demand. Only the virtual-time test `t…
The two idle Prime Agent sessions from the 13 Sep probes are no longer open ✅ pass live prime-13sep-sessions.txt (both archived, no worker; 0 live)
  • bin/fm-test-run.sh --changed --exclude-family real-herdr-gated
  • FM_PRIME_HERDR_LIVE=1 HERDR_LAB_HELPER=bin/fm-herdr-lab.sh tests/fm-prime-herdr-live-e2e.test.sh (real prime-agent, real herdr lab session, openrouter/moonshotai/kimi-k2.6)
  • prime-agent -p --mode json --no-session [--no-skills] --model openrouter/moonshotai/kimi-k2.6 &#39;Reply with the single word OK.&#39; from the firstmate worktree with 480 skills in ~/.agents/skills
  • Real bin/fm-spawn.sh &lt;id&gt; &lt;project&gt; prime --mode no-mistakes --yolo off --backend tmux|zellij with isolated FM_HOME on a host without tmux
  • prime-agent list --all --json | jq filtered to sessions created 2026-09-13
  • tests/fm-prime-harness.test.sh
  • tests/fm-control-relaunch.test.sh (test_relaunch_onto_prime_on_a_non_herdr_backend_refuses_before_stop)
  • tests/fm-busy-adapter-wiring.test.sh (Prime extension cases)
  • tests/fm-control.test.sh (Prime control table)
  • Rerun of baseline bin/fm-test-run.sh --changed --exclude-family real-herdr-gated (partial: 27 files finished, all exit 0 including tests/fm-lint.test.sh)
✅ **Document** - passed

✅ No issues found.

⚠️ **Lint** - 1 warning
  • ⚠️ linter found issues (exit code 1)

🔧 Fix applied.
1 warning still open:

  • ⚠️ linter found issues (exit code 1)
✅ **Push** - passed

✅ No issues found.

Karan Manoharan and others added 6 commits September 14, 2026 19:26
Detect Prime Agent through its own kernel and daemon-worker markers, before the Pi marker that Prime sets for every tool.
Launch Prime workers with --no-skills and --no-session, and load a per-task extension that reports busy state and turn ends.
Map effort to --thinking, interrupt with one Ctrl+C, and exit with /quit.
Refuse Prime for secondmates, because no Prime primary integration exists.
Record the live Herdr evidence in docs/verification/prime.md and add an opt-in live guard.
@karanmrn
karanmrn merged commit a60bd14 into main Sep 15, 2026
0 of 14 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant