fix(tests): scope Relay cadence prefix to launching adapters and close local/CI env parity gap - #41
Merged
zeeshaanahmad merged 2 commits intoSep 3, 2026
Conversation
…adapters
The guard and turn-end repair lines prepended "source <config/x-mode.env>
first, then " for every primary harness. That instruction is only actionable
where the model itself runs the command that launches the watcher: under the
Claude Stop hook, the Cursor park, and the Pi extension the launcher sources
the cadence config in its own path (bin/fm-claude-stop-autoarm.sh,
bin/fm-turnend-guard-cursor.sh, .pi/extensions/fm-primary-pi-watch.ts), and
the model is handed no command to source into. Emit the prefix for codex,
grok, opencode, and unknown only, matching what
docs/supervision-protocols/{codex,grok}.md already tell the model.
Also close the reason tests/fm-watcher-lock.test.sh's guard-xmode case was red
locally and green in CI: the fixture set FM_ROOT_OVERRIDE but not FM_HOME, and
bin/fm-guard.sh resolves CONFIG from FM_HOME first, so the ambient FM_HOME that
every firstmate-spawned shell exports answered the x-mode probe from a real
home. tests/lib.sh now scrubs the same fleet-home overrides bin/fm-test-run.sh
already scrubbed around its lanes, so a direct `bash tests/<x>.test.sh` run
gets the lane's environment; bin/fm-test-env-lib.sh is the single owner of that
list. This also fixes a second, pre-existing local-only failure in
tests/fm-turnend-guard.test.sh.
Coverage for the cadence config is asserted per adapter in both directions, at
the renderer and end to end through the guard.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Intent
Fix the red
guard-xmodecase in tests/fm-watcher-lock.test.sh on firstmate main, and make CI catch this class of failure.Original task: on main,
bash tests/fm-watcher-lock.test.shfails with 'not ok - guard repair line did not source the X-mode cadence config'. The case runs bin/fm-guard.sh with CLAUDECODE=1 and an X-mode config present, expecting the repair line to carry "source '/config/x-mode.env' first". It fails in a home whose Claude Stop hook is registered but CI is green, so the task required establishing the exact environment difference (the case sets CLAUDECODE=1 explicitly, so the harness marker was not the variable). The task required deciding which side was wrong by reading the contract - docs/supervision-protocols/, bin/fm-supervision-instructions.sh, bin/fm-guard.sh, docs/turnend-guard.md, docs/configuration.md 'X mode'/Relay cadence - and named two acceptable outcomes: (a) the guard's Claude-adapter repair line must still name the cadence config, so fix the guard and keep the assertion; or (b) the Stop-owned path legitimately has no command to source into, in which case the test case must assert the Claude-adapter contract AND a separate case must cover the adapter that does emit a command, so cadence-config coverage is not silently lost. Deleting the assertion was explicitly forbidden. The task also required finding why CI is green, naming the exact environment difference, and closing it so a red guard repair line fails CI (or explaining concretely why it cannot and where the coverage lives instead).What I found and the decisions I made:
Contract decision - outcome (b). The Relay cadence config is inherited by whatever LAUNCHES the watcher process. docs/supervision-protocols/codex.md and grok.md tell the model to source it because under those adapters the model itself runs the launching command. Under the hook-, park-, and extension-owned adapters the launcher sources it in its own code (bin/fm-claude-stop-autoarm.sh:184, bin/fm-turnend-guard-cursor.sh:273, .pi/extensions/fm-primary-pi-watch.ts:447, .opencode/plugins/fm-primary-watch-arm.js:345) and the model is handed no command to source into, so the prefix was an instruction it could not act on. bin/fm-supervision-instructions.sh now emits the 'source first, then ' repair-line prefix only for codex, grok, opencode, and unknown. opencode is deliberately IN that set because its repair line still offers bin/fm-watch-arm.sh as a manual recovery probe the model would run itself. The independent queue-pending prefix is unchanged on both sides.
The prior diagnosis of this bug (in data/upstream-batch-4-deflake-collision/completion-report.md section 7a) was wrong about the mechanism: it claimed the Claude adapter emitted no command to prepend the source to, so the prefix was absent. In fact the prefix was harness-independent and WAS emitted for claude; the case failed for an unrelated reason. The real environment difference is an ambient FM_HOME. bin/fm-guard.sh resolves FM_HOME=${FM_HOME:-${FM_ROOT_OVERRIDE:-...}} and then CONFIG=${FM_CONFIG_OVERRIDE:-$FM_HOME/config}, so a fixture that sets only FM_ROOT_OVERRIDE has its config probes answered by whatever real home the invoking shell exports - and every firstmate-spawned worker shell exports FM_HOME. A bare CI runner exports none, and bin/fm-test-run.sh already scrubbed the same overrides around every script it schedules, which is why CI was green: a red guard repair line ALREADY failed CI. What did not hold was local/CI parity for a direct 'bash tests/.test.sh' run.
Closing that class. bin/fm-test-env-lib.sh is a new single owner of the never-inherit fleet-home override list; bin/fm-test-run.sh now sources it instead of re-spelling the list, and tests/lib.sh scrubs the same list at source time so a direct single-test run gets the lane's environment. This also fixed a second, pre-existing local-only failure of the same class in tests/fm-turnend-guard.test.sh ('healthy no-supervision-needed native stop must allow: expected exit 0, got 2'), which is red on main in any firstmate-spawned shell and green under 'env -u FM_HOME'. Because bin/fm-test-run.sh now has one library dependency, tests/fm-test-run.test.sh's three fixture repos install it via a new install_runner helper, and the runner refuses with a named error if that library is missing rather than producing zero test markers.
Coverage, deliberately kept in both directions rather than deleted. tests/fm-watcher-lock.test.sh's guard case is now two cases pinning FM_HOME to the fixture: the Claude adapter must carry the Stop-owned recovery contract and must NOT name the cadence config, and a new guard-xmode-arm case (grok, pinned via GROK_AGENT=1) proves the guard still wires x-mode detection into a repair line that does launch a watcher. tests/fm-supervision-instructions.test.sh gains test_x_mode_cadence_rides_only_launching_repair_lines, asserting all four launching adapters name the config, all four non-launching adapters do not, and the queue-pending prefix survives in the right order on both sides. tests/fm-turnend-guard.test.sh's test_hook_x_mode_reason_sources_cadence became test_hook_x_mode_reason_keeps_stop_owned_contract for the same contract (that hook only ever runs under the Claude adapter) and points at where the launching-adapter coverage lives.
No documentation change was needed: docs/configuration.md already delegates with 'The active primary-harness supervision protocol owns how that sourced cadence reaches the watcher process.'
Verified before starting this run: tests/fm-watcher-lock.test.sh exit 0; tests/fm-supervision-instructions.test.sh exit 0 both with and without an ambient FM_HOME; tests/fm-turnend-guard.test.sh exit 0; tests/fm-test-run.test.sh exit 0; lane portable-parallel-1 exit 0 (253 ok); lane portable-serial-3of4 exit 0 (549 ok, 32 scripts); lane portable-serial-4of4 has fm-watcher-lock.test.sh exit=0 but exits 1 on two failures that are NOT from this change and reproduce identically on main under 'env -u FM_HOME' (tests/fm-pending-reply.test.sh concurrent-resolver case, and tests/fm-backend-herdr-focus-flash-e2e.test.sh against locally installed Herdr 0.7.5 where CI pins 0.7.4); bin/fm-test-run.sh --check-coverage ok; bin/fm-lint.sh clean; /bin/bash -n (3.2.57) clean on every changed script. Mutation-tested the new coverage in both directions in a clean-environment copy: removing grok from the launching set and adding claude to it each produce named failures at the renderer and end to end through the guard.
This is firstmate's own tracked material, so the firstmate-coding-guidelines skill applies: one sentence per line in tracked Markdown, plain dash never an em dash, no agent co-author trailer, bin/*.sh shellcheck-clean via bin/fm-lint.sh, tests colocated in tests/ extending existing scripts, and tests must exercise behavior through an executable interface rather than asserting implementation-source bytes.
What Changed
bin/fm-supervision-instructions.sh: the guard repair line'ssource <x-mode config> first, thenprefix is now emitted only for thecodex,grok,opencode, andunknownadapters (the ones whose repair line hands the model a command that launches the watcher); the Claude/Stop-owned and other launcher-owned adapters no longer get an instruction the model has no command to act on.bin/fm-test-env-lib.shas the single owner of the fleet-home override list (FM_HOME,FM_STATE_OVERRIDE,FM_DATA_OVERRIDE,FM_ROOT_OVERRIDE,FM_PROJECTS_OVERRIDE,FM_CONFIG_OVERRIDE,FM_BACKEND) that a test must never inherit from its invoking shell;bin/fm-test-run.shnow sources it (and dies with a named error if it's missing) instead of re-declaring the list inline, andtests/lib.shscrubs the same list at source time so a directbash tests/<x>.test.shrun gets the same environment as the runner's lanes.docs/scripts.md's toolbelt table.tests/fm-watcher-lock.test.sh'sguard-xmodecase is split in two: the Claude adapter case now pinsFM_HOMEand asserts the Stop-owned repair line does NOT name the cadence config, plus a newguard-xmode-armcase (viaGROK_AGENT=1) asserts the guard still wires x-mode detection into a repair line that does source the config.tests/fm-supervision-instructions.test.shgainstest_x_mode_cadence_rides_only_launching_repair_lines, asserting all four launching adapters name the cadence config and all four non-launching adapters don't, with the queue-pending prefix ordering preserved on both sides.tests/fm-turnend-guard.test.sh'stest_hook_x_mode_reason_sources_cadenceis renamed totest_hook_x_mode_reason_keeps_stop_owned_contractand updated for the same contract, plus other cases now pinFM_HOMEto avoid ambient-environment leakage.tests/fm-test-run.test.shfixtures installbin/fm-test-env-lib.shvia a newinstall_runnerhelper so the runner's new library dependency is satisfied in isolated fixture repos.Risk Assessment
✅ Low: The change correctly root-causes the failure to ambient FM_HOME inheritance (verified: fm-guard.sh and fm-supervision-instructions.sh both resolve CONFIG from FM_HOME before falling back to FM_ROOT_OVERRIDE, and bin/fm-watch-arm.sh does not self-source x-mode.env while the claude/cursor/pi launch paths do), scopes the cadence-prefix fix to exactly the adapters whose repair line hands the model a command it must run itself (verified against bin/fm-claude-stop-autoarm.sh:184, bin/fm-turnend-guard-cursor.sh:273, .pi/extensions/fm-primary-pi-watch.ts, and .opencode/plugins/fm-primary-watch-arm.js, which all self-source the config in their launch paths, versus codex/grok/opencode-fallback/unknown where the model runs bin/fm-watch-arm.sh or fm-watch-checkpoint.sh directly), and preserves cadence-config coverage in both directions exactly as the intent's outcome (b) required (guard-xmode + new guard-xmode-arm cases, plus test_x_mode_cadence_rides_only_launching_repair_lines). The env-scrub fix (bin/fm-test-env-lib.sh as single owner, sourced by both fm-test-run.sh and tests/lib.sh) is idempotent, applies before any test file sets its own overrides (confirmed no test file sets these vars pre-source), and is consistent with the pre-existing scrub already present in fm-test-run.sh's serial/parallel lanes, which explains why CI was already green. No forbidden behavior (the assertion was not deleted) and no functional or security issues found.
Testing
Directly reproduced the original CI/local-parity bug on the pre-fix commit (ambient FM_HOME caused the guard-xmode case to fail with the exact reported message) and confirmed the target commit fixes it under identical conditions; the full guard-xmode / guard-xmode-arm split, the adapter-scoped cadence-config renderer test, and the Stop-owned turnend-guard contract test all pass consistently with and without an ambient FM_HOME, and fm-test-run.test.sh's fleet-home-scrub coverage passes too — no failures found, worktree left clean.
Pipeline
Updates from git push no-mistakes
✅ **intent** - passed
✅ No issues found.
✅ **Rebase** - passed
✅ No issues found.
✅ **Review** - passed
✅ No issues found.
✅ **Test** - passed
✅ No issues found.
FM_HOME=/tmp/fake-ambient-fm-home bash tests/fm-watcher-lock.test.shon base commit 8884885 in a temporary detached worktree — reproduced exact reported failurenot ok - guard repair line did not source the X-mode cadence config, exit 1FM_HOME=/tmp/fake-ambient-fm-home bash tests/fm-watcher-lock.test.shon target commit d21b15b — full pass, exit 0, including new guard-xmode (Stop-owned contract) and guard-xmode-arm (Grok cadence-config) casesFM_HOME=/tmp/fake-ambient-fm-home bash tests/fm-supervision-instructions.test.shandenv -u FM_HOME bash tests/fm-supervision-instructions.test.shon target commit — both exit 0, test_x_mode_cadence_rides_only_launching_repair_lines passes both waysFM_HOME=/tmp/fake-ambient-fm-home bash tests/fm-turnend-guard.test.shandenv -u FM_HOME bash tests/fm-turnend-guard.test.shon target commit — both exit 0, test_hook_x_mode_reason_keeps_stop_owned_contract passes both waysFM_HOME=/tmp/fake-ambient-fm-home bash tests/fm-test-run.test.shon target commit — exit 0, including test_fleet_home_overrides_are_scrubbed_in_both_lanesManual check: copied bin/fm-test-run.sh alone (no bin/fm-test-env-lib.sh) into a fresh fixture with one dummy test script and ranbash bin/fm-test-run.sh --all— confirmed it refuses with the named errormissing bin/fm-test-env-lib.sh beside this runner...and exit 2, rather than silently producing zero test markers✅ **Document** - passed
✅ No issues found.
✅ **Lint** - passed
✅ No issues found.
✅ **Push** - passed
✅ No issues found.