test(orca): stub claude in Orca spawn fixtures so Linux CI clears preflight by Amplify-Logic · Pull Request #800 · kunchenguid/firstmate

Amplify-Logic · 2026-07-21T08:16:42Z

Intent

Unblock the firstmate board's critical path by fixing the single Linux CI behavior-test failure that broke PR Amplify-Logic#12 after Actions was enabled on the fork for the first time (2026-07-21).

CI failed: pathless worktree failure should explain the missing path (missing: 'orca worktree create did not return a path'). Diagnosis: production error path in bin/backends/orca.sh already emits that message (unit helper test passed on CI); the spawn integration test never reached it on Linux because fm-spawn harness preflight refuses when claude is missing, and Linux CI does not install claude, while macOS local does. Fix is test-only: stub claude in make_orca_fakebin via fm_fake_exit0 so Orca spawn fixtures clear preflight and exercise the real pathless-error path. Deliberately did not touch production orca.sh, and deliberately did not fix three separate macOS full-suite flakes (cmux send_text_submit absent-target, secondmate relaunch missing new-window, Journey Herdr treehouse timeout) - those are a follow-up, not this CI failure.

Prior validation run 01KY1NJT65FVMHP2N7PGVQPSRP failed mid-test on a connection drop (claude exited), not on the change; restarting without redoing the implementation. Rebase ask-user for unpushed main was already confirmed a stale-ref false positive (origin/main == local main == ff1015f; branch is only 0fbd62f ahead).

What Changed

Added a claude stub (via fm_fake_exit0) to make_orca_fakebin in tests/fm-backend-orca.test.sh, so Orca spawn fixtures pass the fm-spawn harness preflight on machines without a real claude binary (e.g. Linux CI) and actually reach the pathless-worktree error path under test.
Test-only change: production bin/backends/orca.sh is untouched. Validation confirmed the counterfactual — with the 4-line stub reverse-applied and claude absent, the exact Linux CI failure (pathless worktree failure should explain the missing path) reproduces; with the stub, all 50 tests pass.

Risk Assessment

✅ Low: The branch adds a single test-only stub line (plus comment) using an existing helper, exactly matching the stated intent with no production code touched.

Testing

Beyond the already-passing baseline full suite, I reproduced the Linux CI environment locally by stripping claude from PATH: the pre-fix test file fails exactly as CI did (spawn preflight refuses before the Orca worktree call), while the fixed fixture passes all 50 Orca tests and exercises the real production pathless error message — demonstrating the fix end-to-end without touching production code.

Evidence: Before/after summary: CI failure reproduced without stub, passes with stub

== Simulated Linux CI (claude absent from PATH) == -- BASE (without stub): reproduces CI failure -- not ok - pathless worktree failure should explain the missing path (missing: 'orca worktree create did not return a path') error: harness 'claude' launch binary 'claude' was not found (install: npm install -g @anthropic-ai/claude-code); refusing before creating a task endpoint -- TARGET (with fm_fake_exit0 claude stub): passes -- ok - fm_backend_orca_worktree_create: removes created worktree when path is missing ok - fm-spawn.sh --backend orca: preserves metadata when pathless cleanup fails

== Simulated Linux CI (claude absent from PATH) ==
-- BASE (without stub): reproduces CI failure --
ok - Orca lifecycle helpers: register repo, create worktree, create terminal, parse stable ids
ok - fm_backend_orca_worktree_create: removes created worktree when path is missing
not ok - pathless worktree failure should explain the missing path (missing: 'orca worktree create did not return a path')
--- output ---
error: harness 'claude' launch binary 'claude' was not found (install: npm install -g @anthropic-ai/claude-code); refusing before creating a task endpoint

-- TARGET (with fm_fake_exit0 claude stub): passes --
ok - fm_backend_orca_worktree_create: removes created worktree when path is missing
ok - fm-spawn.sh --backend orca: preserves metadata when pathless cleanup fails

Evidence: Full Orca suite log, target commit, claude absent (50 ok / 0 not ok)

ok - fm_backend_orca_capture: parses result.terminal.tail and calls terminal read
ok - fm_backend_orca_capture: falls back to result text fields
ok - fm_backend_orca_capture: fails closed on Orca read error JSON
ok - fm_backend_orca_runtime_check: accepts reachable ready runtime
ok - fm_backend_orca_runtime_check: fails closed when runtime is not ready
ok - fm_backend_orca_send_text_submit: verifies empty composer after Enter
ok - fm_backend_orca_send_text_submit: preserves current tail when limited reads fetch older cursor text
ok - fm_backend_orca_send_text_submit: retries Enter while composer remains pending
ok - fm_backend_orca_composer_state: a slash-command popup's argument-hint placeholder still reads pending
ok - fm_backend_orca_composer_state: a bare dead-shell prompt reads unknown (unsafe-for-injection), never empty
ok - fm_backend_orca_send_text_submit: a slash-command popup's placeholder fill on Enter #1 does not short-circuit as submitted; Enter #2 is retried and lands it
ok - fm_backend_orca_send_literal: sends text without submitting
ok - fm_backend_orca_send_text_submit: reports send-failed when Orca send fails
ok - Orca send helpers: fail closed on ok:false JSON
ok - fm_backend_orca_send_key: Enter maps to empty enter, C-c maps to interrupt
ok - fm_backend_orca_send_key: refuses unsupported keys loudly
ok - fm_backend_orca_send_key: refuses Escape instead of mapping it to interrupt
ok - fm_backend_orca_kill: calls terminal close and stays best-effort
ok - fm_backend_orca_remove_worktree: refuses empty worktree ids
ok - fm_backend_orca_remove_worktree: fails closed on ok:false JSON
ok - fm_backend_orca_worktree_path: resolves an Orca worktree id to its path
ok - fm-backend dispatcher: accepts orca and routes capture through bin/backends/orca.sh
ok - fm_backend_orca_json_get: ignores undocumented terminal id shapes
ok - Orca lifecycle helpers: register repo, create worktree, create terminal, parse stable ids
ok - fm_backend_orca_worktree_create: removes created worktree when path is missing
ok - fm-spawn.sh --backend orca: preserves metadata when pathless cleanup fails
ok - fm-spawn.sh --backend orca: reuses implicit terminal, records metadata, launches harness
ok - fm-spawn.sh --backend orca --secondmate: refuses before secondmate-home mutation
ok - fm-spawn.sh --backend orca: refuses before mutation when Orca runtime is not ready
ok - fm-spawn.sh --backend orca: refuses non-isolated worktrees and closes implicit terminals
ok - fm-spawn.sh --backend orca: removes worktree when terminal creation fails
ok - fm-spawn.sh --backend orca: preserves metadata when abort cleanup fails
ok - fm-spawn.sh --backend orca: releases terminal and worktree on later aborts
ok - fm-peek/fm-send/fm-crew-state route through backend=orca metadata
ok - fm-peek/fm-crew-state: Orca read error JSON fails closed
ok - fm_backend_target_exists: Orca ok:false read JSON is not live
ok - fm-teardown.sh backend=orca: scout report gate then helper-backed worktree removal
ok - fm-teardown.sh backend=orca: scout teardown refuses id/path mismatches
ok - fm-teardown.sh backend=orca: releases terminal/worktree when path is absent
ok - fm-teardown.sh backend=orca: preserves metadata on remove ok:false JSON
ok - fm-teardown.sh backend=orca: scout report gate precedes pathless helper cleanup
ok - fm-teardown.sh backend=orca: ship teardown fails closed when worktree path is missing
ok - fm-teardown.sh backend=orca: ship teardown requires a matching Orca id path
ok - fm-teardown.sh backend=orca: ship teardown fails closed when id resolution fails
ok - fm-teardown.sh backend=orca: ship teardown refuses id/path mismatches
ok - fm-teardown.sh backend=orca: refuses missing worktree ids before cleanup
ok - fm-teardown.sh backend=orca: removes partial worktree-only metadata
ok - fm-teardown.sh --force: removes Orca secondmate children through Orca
ok - fm-teardown.sh --force: refuses Orca child id/path mismatches
ok - fm-teardown.sh --force: removes partial Orca secondmate children

Evidence: Full Orca suite log, base fixture (stub reverted), claude absent — CI failure reproduced

ok - fm_backend_orca_capture: parses result.terminal.tail and calls terminal read
ok - fm_backend_orca_capture: falls back to result text fields
ok - fm_backend_orca_capture: fails closed on Orca read error JSON
ok - fm_backend_orca_runtime_check: accepts reachable ready runtime
ok - fm_backend_orca_runtime_check: fails closed when runtime is not ready
ok - fm_backend_orca_send_text_submit: verifies empty composer after Enter
ok - fm_backend_orca_send_text_submit: preserves current tail when limited reads fetch older cursor text
ok - fm_backend_orca_send_text_submit: retries Enter while composer remains pending
ok - fm_backend_orca_composer_state: a slash-command popup's argument-hint placeholder still reads pending
ok - fm_backend_orca_composer_state: a bare dead-shell prompt reads unknown (unsafe-for-injection), never empty
ok - fm_backend_orca_send_text_submit: a slash-command popup's placeholder fill on Enter #1 does not short-circuit as submitted; Enter #2 is retried and lands it
ok - fm_backend_orca_send_literal: sends text without submitting
ok - fm_backend_orca_send_text_submit: reports send-failed when Orca send fails
ok - Orca send helpers: fail closed on ok:false JSON
ok - fm_backend_orca_send_key: Enter maps to empty enter, C-c maps to interrupt
ok - fm_backend_orca_send_key: refuses unsupported keys loudly
ok - fm_backend_orca_send_key: refuses Escape instead of mapping it to interrupt
ok - fm_backend_orca_kill: calls terminal close and stays best-effort
ok - fm_backend_orca_remove_worktree: refuses empty worktree ids
ok - fm_backend_orca_remove_worktree: fails closed on ok:false JSON
ok - fm_backend_orca_worktree_path: resolves an Orca worktree id to its path
ok - fm-backend dispatcher: accepts orca and routes capture through bin/backends/orca.sh
ok - fm_backend_orca_json_get: ignores undocumented terminal id shapes
ok - Orca lifecycle helpers: register repo, create worktree, create terminal, parse stable ids
ok - fm_backend_orca_worktree_create: removes created worktree when path is missing
not ok - pathless worktree failure should explain the missing path (missing: 'orca worktree create did not return a path')
--- output ---
error: harness 'claude' launch binary 'claude' was not found (install: npm install -g @anthropic-ai/claude-code); refusing before creating a task endpoint

Pipeline

Updates from git push no-mistakes

✅ **intent** - passed

✅ No issues found.

⚠️ **Rebase** - 1 warning

⚠️ .agents/skills/afk/SKILL.md - branch carries 11 commit(s) that exist on your local main branch but were never pushed to origin/main; rebasing would bundle this unrelated work (127 file(s)) into the PR:
ff1015f fix(herdr): corroborate busy cursor panes before waiting-on-human escalation (docs: require full PR URLs in captain reports #11)
1bcc667 fix(bootstrap): recognize verified Cursor workers (fix: keep idle watcher supervision silent #10)
3b00d42 fix(test): stabilize watcher healthy-peer restart fixture (docs: clarify live supervision constraints #9)
d3d0274 feat(primary): unify orchestrator status bar (feat(bin): guard watcher liveness across supervision scripts #8)
248a6ae fix(spawn): refuse missing harness binaries before launch (feat: add scout task lifecycle #7)
e07b7cc feat(harness): add verified Cursor worker adapter (Cursor CLI + Grok 4.5) (fix: reduce crewmate status wake noise #6)
5053c3c Fix primary launcher shim root resolution (ci: enforce contributor guardrails #5)
6cf9e61 Add human-readable Herdr worker presentation (fix(bin): coalesce watcher signals into one wake #4)
86ddc8c chore: no-mistakes(document): add Kimi protocol and fm-primary launcher to doc indexes (docs: align tmux and harness guidance #3)
dde673b test(fm-brief): align secondmate charter role assertion with fix: accept secondmate house vocabulary #685 vocabulary (docs: polish README banner and repo housekeeping #2)
4b884f3 fix(herdr-lab): place --session before the child-argv delimiter and refuse ambiguous delimiter shapes (chore: initialize no-mistakes gate #1)

Push main to origin, or rebase your branch onto origin/main, before gating.

✅ **Review** - passed

✅ No issues found.

✅ **Test** - passed

✅ No issues found.

command -v tmux >/dev/null || { echo "tmux is required for e2e tests" >&2; exit 1; }; tmux -V; rc=0; for t in tests/*.test.sh; do echo "== $t =="; bash "$t" || rc=1; done; exit "$rc"
Baseline configured command (full tests/*.test.sh suite via tmux runner) already ran successfully before this validation
env PATH=<sanitized, no claude> bash tests/fm-backend-orca.test.sh at target commit — claude confirmed absent, all 50 tests pass (exit 0), including pathless worktree failure should explain the missing path
Counterfactual: reverse-applied the 4-line stub (git apply -R), reran with claude absent — reproduced the exact Linux CI failure (not ok - pathless worktree failure should explain the missing path, preflight error launch binary 'claude' was not found), then restored the file (worktree clean)
Confirmed git show 0fbd62f touches only tests/fm-backend-orca.test.sh (no production bin/backends/orca.sh change)

✅ **Document** - passed

✅ No issues found.

✅ **Lint** - passed

✅ No issues found.

✅ **Push** - passed

✅ No issues found.

…efuse ambiguous delimiter shapes (#1) * fix(herdr-lab): place --session before the child-argv delimiter `agent start ... -- <child argv...>` always got a trailing --session appended after --, so Herdr never consumed it and the child process received it instead - meaning Herdr could fall through to the live default session for agent-start calls. fm_herdr_lab_raw now places exactly one --session immediately before -- for that shape and still appends it for every ordinary call. fm_herdr_lab_cli rejects multiple delimiters, a delimiter on any command other than agent start, and agent start without a delimiter plus non-empty child command, before any Herdr call is made. Updated the brief scaffold and backend doc wording that claimed the selector is always trailing. * no-mistakes(document): note run's delimiter-shape refusals in herdr-backend doc * no-mistakes(review): refuse option before child-argv delimiter; collapse raw invocation * no-mistakes(review): only treat first -- as delimiter; child argv untouched

…uid#685 vocabulary (#2) Commit 1182883 (kunchenguid#685) intentionally changed the secondmate charter scaffold's role declaration from 'a persistent domain supervisor' to 'a persistent second mate' and updated the captain-translation-contract test, but missed the assertion in tests/fm-brief.test.sh. The stale expectation left the secondmate-charter case failing. Point the role-declaration assertion at the current wording so it verifies the charter still declares its role without weakening the check or reverting the intended vocabulary.

…er to doc indexes (#3) * feat(primary): add guarded runtime profiles and Kimi K3 support * no-mistakes(review): make Kimi registry merge atomic and clean temp files * no-mistakes(document): add Kimi protocol and fm-primary launcher to doc indexes

* feat: add human-readable Herdr worker presentation * no-mistakes(review): capability-gate herdr presentation and skip secondmate worker rename

…4.5) (#6) * feat(harness): add verified cursor worker adapter (Cursor CLI + Grok 4.5) Adds `cursor` (Cursor CLI `agent` 2026.07.16-899851b) as a verified WORKER adapter, after empirical verification on 2026-07-19. Worker only: primary support was not verified and is not claimed, the mirror of the Kimi primary-only boundary. Kimi stays primary-only; no Kimi worker support here. Verification found two defects in shared composer monitoring that would have made an adapter monitoring cannot trust, both fixed in the shared owners: - Reverse-video cursor cell. cursor draws the terminal cursor as SGR 7 over the idle placeholder's FIRST character. Reverse video is neither dim/faint nor a dark foreground, so fm_composer_strip_ghost kept it and an idle composer reduced to a lone "A" -> `pending`, deferring every away-mode escalation forever. A plain FM_COMPOSER_IDLE_RE does not fix this, because the idle regex was only matched against ghost-stripped content; it is now matched against the plain row too, which is styling-independent. - #{cursor_y} does not point at cursor's composer (it parks the cursor in the bottom status area, observed 5 rows off). Reading that row found an empty status line, so a composer holding real unsubmitted text classified as `empty` - a FALSE-EMPTY, the dangerous direction, since the away-mode injector picks targets by emptiness and would type over pending input. The tmux path now locates the composer structurally, matching herdr's existing scan. Other integration surfaces: - fm-spawn: launch template (--yolo autonomy, --workspace pinned to the task worktree, and deliberately no -w so cursor never allocates a second worktree), plus a per-task gitignored .cursor/hooks.json `stop` hook verified to fire once per turn with no extra hook-trust grant. - Effort axis: cursor has no effort flag - effort is a SUFFIX on the model id - so cursor_model_with_effort folds it in. xhigh/max cap at high, an explicit tiered model is never retiered, and -fast variants are never implicit. - fm-harness: CURSOR_AGENT=1 detection, tested BEFORE CLAUDECODE because cursor does not clear an inherited CLAUDECODE=1 and a cursor worker under a claude-hosted firstmate would otherwise be steered with claude's vocabulary. - Busy signature `ctrl+c to stop` in both owners. The spinner VERB is deliberately not matched: it flips Working -> Running mid-turn, so matching it reads a tool-executing pane as idle. - tmux liveness: cursor's wrapper execs node, but the versioned cursor-agent bundle path survives exec -a in argv, so a `node` COMM resolves to `alive`. Any other bare node stays `unknown`, never dead; pi's gap is unchanged. Backends reviewed rather than assumed: tmux verified; herdr composer-safe by construction (structural scan + shared idle default) with liveness unverified; zellij/orca/cmux not exercised with cursor. Recorded in docs/cursor-harness.md. Also documents that cursor executes claude-format hooks (.claude/settings*.json, Stop -> stop), so cursor must never be launched from the firstmate primary checkout, and that --model mutates the account-global default. A live fm-spawn dispatch was NOT run: it allocates a real pooled worktree and writes live fleet state outside the task worktree. Every underlying behaviour was verified by raw launches and the integrated path (launch, turn-end hook, composer, liveness) was confirmed together on a real pane; the first supervised fm-spawn --harness cursor run remains firstmate's gate. Evidence: docs/cursor-harness.md. Regressions: tests/fm-cursor-adapter.test.sh (20 assertions). bin/fm-lint.sh clean; composer/tmux/herdr/orca/cmux suites pass. * no-mistakes(review): scope cursor composer-row scan to identified cursor panes * no-mistakes(document): document cursor worker adapter in config, architecture, and indexes

* fix(spawn): refuse missing harness binaries before launch Resolve and probe verified launch executables before creating task endpoints, preventing absent CLIs from leaving dead worker shells. Co-authored-by: Cursor <cursoragent@cursor.com> * no-mistakes(review): add stdin-closed timeout-bounded version probe with override knob * no-mistakes(review): escalate timed-out version probe to SIGKILL after grace * no-mistakes(review): kill probe process group so timed-out wrapper descendants die --------- Co-authored-by: Cursor <cursoragent@cursor.com>

Define one fleet-status contract for guarded Claude, Pi, and Kimi surfaces while preserving Cursor's worker-only boundary.

* test(watcher): wait for healthy peer readiness Wait for the fixture's SIGTERM handler before exercising restart so startup scheduling cannot turn the healthy-peer case into the child-owned path. Co-authored-by: Cursor <cursoragent@cursor.com> * no-mistakes(review): clean up fixture processes on healthy-peer assertion failures --------- Co-authored-by: Cursor <cursoragent@cursor.com>

Keep startup dispatch validation and secondmate liveness aligned with the verified worker adapter set, including Cursor's model-suffixed effort range. Co-authored-by: Cursor <cursoragent@cursor.com>

…alation (#11) * fix(herdr): do not escalate busy cursor panes as waiting-on-human Herdr can report agent_status=blocked while a cursor worker is mid-turn (ctrl+c to stop footer). Corroborate blocked/idle against that busy signature before immediate stale escalation or poll-path idle. Co-authored-by: Cursor <cursoragent@cursor.com> * no-mistakes(test): fix jq-1.6 and preflight fixture gaps in failing tests * no-mistakes(docs): corroborate herdr idle/blocked in architecture summary Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * no-mistakes(lint): silence SC2329 on indirectly-invoked test stub --------- Co-authored-by: Cursor <cursoragent@cursor.com> Co-authored-by: Claude Fable 5 <noreply@anthropic.com>

Spawn preflight refuses before any Orca worktree call when the harness binary is missing. Linux CI has no claude, so the pathless-cleanup spawn test aborted early and never exercised the production error message. Co-authored-by: Cursor <cursoragent@cursor.com>

chatgpt-codex-connector · 2026-07-21T08:16:47Z

You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard.
To continue using code reviews, you can upgrade your account or add credits to your account and enable them for code reviews in your settings.

Amplify-Logic · 2026-07-21T08:23:57Z

Opened against the wrong repository by mistake — this was not an intended contribution to upstream. Closing; apologies for the noise.

Amplify-Logic and others added 12 commits July 19, 2026 01:19

Add human-readable Herdr worker presentation (#4)

6cf9e61

* feat: add human-readable Herdr worker presentation * no-mistakes(review): capability-gate herdr presentation and skip secondmate worker rename

Fix primary launcher shim root resolution (#5)

5053c3c

feat(primary): unify orchestrator status bar (#8)

d3d0274

Define one fleet-status contract for guarded Claude, Pi, and Kimi surfaces while preserving Cursor's worker-only boundary.

fix(bootstrap): recognize verified Cursor workers (#10)

1bcc667

Keep startup dispatch validation and secondmate liveness aligned with the verified worker adapter set, including Cursor's model-suffixed effort range. Co-authored-by: Cursor <cursoragent@cursor.com>

Amplify-Logic closed this Jul 21, 2026

This was referenced Jul 24, 2026

fm-teardown: content_in_default fallback reaps an unmerged ship task's state when a sibling PR's content is already in the default branch #986

Open

fix(fm-teardown): open own-PR vetoes the content-in-default landed check #988

Closed

Navigation Menu

test(orca): stub claude in Orca spawn fixtures so Linux CI clears preflight - #800

test(orca): stub claude in Orca spawn fixtures so Linux CI clears preflight#800
Amplify-Logic wants to merge 12 commits into
kunchenguid:mainfrom
Amplify-Logic:fm/firstmate-ci-orca-pathless-fail-c9

Amplify-Logic commented Jul 21, 2026

Uh oh!

chatgpt-codex-connector Bot commented Jul 21, 2026

Uh oh!

Amplify-Logic commented Jul 21, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Conversation

Amplify-Logic commented Jul 21, 2026

Intent

What Changed

Risk Assessment

Testing

Pipeline

Uh oh!

chatgpt-codex-connector Bot commented Jul 21, 2026

Uh oh!

Amplify-Logic commented Jul 21, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant