Skip to content

test(tests): provide fake treehouse for secondmate harness spawns - #7

Merged
elixlabssolutions merged 1 commit into
mainfrom
fm/secondmate-harness-fake-treehouse
Aug 7, 2026
Merged

elixlabssolutions merged 1 commit into
mainfrom
fm/secondmate-harness-fake-treehouse

Conversation

@elixlabssolutions

@elixlabssolutions elixlabssolutions commented Aug 7, 2026 •

Copy link
Copy Markdown

Intent

Carve-out delivery, by explicit captain decision (decision key herdr-lane-red-round-2): land the CI-proven half of the fix-forward for red main now, while the unsolved Herdr half is redone by a separately equipped task. Context that must be plain in the PR body: PR #5 (fix(spawn): record task worktrees from allocator leases, squash-merged as 569dae6) was merged before its CI had ever completed, during the 2026-08-06 GitHub Actions outage, and when Actions recovered, main's run at the merge commit (run 31130627813, red on both attempts) showed two failing jobs: "Behavior portable serial 4" and "Behavior tests (Herdr)". This branch fixes the first of those two.

The single commit here (fix(tests): give the secondmate-harness crew spawn the lease-capable fake treehouse) fixes "Behavior portable serial 4": not ok - crew-unaffected: expected an ordinary ship task in tests/fm-secondmate-harness.test.sh. Causation was established by counterfactual, not from the diff: green at 84de72c (the commit before the merge), red at 569dae6 (the merge commit), reproduced locally in clean detached clones, deterministic on both CI attempts, and the fix was already proven green in CI by the "Behavior portable serial 4" check of PR #6 before that PR was split. Root cause: fm-spawn.sh now hard-requires treehouse get --lease; PR #5 migrated 17 test files to the new shared fake treehouse in tests/lib.sh but missed this file, whose crew-spawn block resolves commands against a bare system PATH, so the spawn died on the missing allocator before writing its task record. The fix is test-only: install the shared fake treehouse in that test's spawn stub set (make_launch_capturing_tmux); the fake leases FM_FAKE_PANE_PATH so the recorded worktree and the pane witness agree and the confirmation logic is exercised, not bypassed. PR #5's protections (allocator-lease worktree recording, independent pane confirmation) are untouched.

Known and expected on this PR's CI, to be stated plainly in the PR body: the "Behavior tests (Herdr)" lane will remain RED here, because that second regression lives in this branch's BASE (main at 569dae6) and this carve-out deliberately does not touch it. Evidence it is the base's own failure and not this commit's: main itself is red on that lane at 569dae6 on two consecutive run attempts with the identical assertion (projected teardown changed active workspace/tab from w4/w4:t1 to w9/w9:t2), and this commit changes only tests/fm-secondmate-harness.test.sh, which that lane does not run. The Herdr fix is deliberately NOT attempted here: two prior fixes were written blind against a herdr version the developer machine does not have (the outage-era "apply CI fixes" in PR #5 and the fix in PR #6) and both failed in CI, so by captain decision there will be no third blind attempt; a dedicated follow-up task with a guarded isolated herdr lab will reproduce the failure against the CI-pinned herdr 0.7.4 before committing anything. Do not attempt to fix, skip around, or auto-repair the red Herdr lane in this run; it is out of scope by explicit decision.

PR body requirements, all mandatory: (a) the outage fix-forward context above, so the record shows why a red main existed; (b) the per-commit causation method above (counterfactual around the merge commit, local reproduction in detached clones, deterministic CI failures, and prior CI proof on PR #6's serial-4 check), so a future reader knows the regression was introduced by PR #5 rather than pre-existing; (c) the split record: this PR carries only the secondmate-harness test fix, while the Herdr focus fix and the test-fixture pipe repair remain unmerged on branch fm/fm-ci-red-after-worktree-fix (PR #6, closed unmerged) pending the lab-equipped redo, with PR #6's closing note carrying the same cross-reference; (d) the expected-red statement for the "Behavior tests (Herdr)" lane with the base-failure evidence above; and (e) the false-green record, at the captain's direction: during PR #6's validation the no-mistakes CI monitor reported all checks passed while the behavior lanes had not yet registered on the head commit (real state at that moment: 4 of 14 checks completed), a success report issued before its evidence existed - the fifth control today to report success it had not established.

GitHub Actions may still be degraded from the outage, so CI may queue longer than usual; that is expected.

What Changed

  • Add the shared lease-capable fake treehouse to the secondmate harness launch-capturing stub set.
  • Let crew and scout spawn coverage acquire a leased worktree and confirm it through the fake pane path.

Risk Assessment

✅ Low: a single test-only commit with no production code. The fix is CI-proven (PR #6's "Behavior portable serial 4" check before the split, and this PR's own run), and the mandatory delivery record the reviewer flagged as absent now exists: it is this description, written before this PR was created - the review and test notes below record that pre-PR state as run history. The "Behavior tests (Herdr)" lane remains red here from the base (main's own unfixed regression, per the expected-red statement above); that redness is inherited, not introduced or risked by this change.

Testing

The base counterfactual fails at the cited crew-spawn assertion; the target’s complete affected behavior suite passes, and direct end-to-end evidence shows the lease path is both recorded and independently confirmed. The Herdr lane was not run, as explicitly out of scope; this shell-only change has no rendered UI surface.

Evidence: Base regression transcript

Base commit reproduces the exact historical crew-unaffected: expected an ordinary ship task failure.

FM_TEST_BEGIN 2026-08-07T02:03:56Z tests/fm-secondmate-harness.test.sh family=secondmate expected_gate_skip=none
ok - A1 fm-harness.sh secondmate resolves the fallback chain; crew mode unchanged
ok - C1 fm-harness.sh secondmate-model/secondmate-effort resolve the optional tokens; bare harness stays empty (backward-compat)
ok - pi-signed identity: authoritative launch selection distinguishes shared wrapper ancestry
ok - harness identity: dash-leading ps command names are basename operands, not options
ok - B1 propagate_inheritable_config: copy, idempotence, convergence, absence-mirror, exclusion, no-op, skip diagnostics
ok - B2 spawn: secondmate runs the secondmate harness; its home inherits declared config
ok - B3 spawn: an absent secondmate-harness falls back to the crew harness (backward-compat)
ok - B4 spawn: no config at all -> own harness and no propagation side effects
ok - B5 spawn: an explicit per-spawn harness arg overrides config/secondmate-harness
ok - B6 spawn: an unverified resolved secondmate harness is refused (guard intact)
ok - B5b spawn: FM_BACKEND wins over inherited config/backend
ok - B5c spawn: explicit --backend wins over FM_BACKEND and inherited config/backend
ok - C2 spawn: a bare harness-only secondmate-harness file launches with no model/effort flag (backward-compat)
ok - C3 spawn: config/secondmate-harness's model token threads --model into the launch and meta
ok - C4 spawn: config/secondmate-harness's model+effort tokens thread into the launch and meta
ok - C5 spawn: an explicit --model overrides config/secondmate-harness's model token; the file's effort token still applies
ok - C6 spawn: an explicit --effort overrides config/secondmate-harness's effort token; the file's model token still applies
ok - C7 spawn: an explicit --harness starts with clean model/effort defaults
ok - C8 spawn: an explicit --harness still honors explicit model/effort flags
ok - C9 spawn: secondmate launch pins supervision to its own harness
not ok - crew-unaffected: expected an ordinary ship task
FM_TEST_END 2026-08-07T02:04:22Z tests/fm-secondmate-harness.test.sh exit=1 duration_ms=25525 gate_skip=false
FM_TEST_SUMMARY total=1 failed=1 skipped_gate=0 duration_ms=25570
FM_TEST_SUMMARY_FAMILY family=secondmate count=1 duration_ms=25525 failed=1
FM_TEST_SLOWEST rank=1 script=tests/fm-secondmate-harness.test.sh duration_ms=25525
fm-test-run: wrote timing artifact: /var/folders/_4/q0rlygb56rs6y6xp9sbql5240000gn/T/no-mistakes-evidence/01KZCZ63K31AKPQ8VQDR58SYM3/base-affected-timing.json
base_test_exit=1
Evidence: Target behavior transcript

Target commit completes the affected behavior suite successfully.

FM_TEST_BEGIN 2026-08-07T02:04:29Z tests/fm-secondmate-harness.test.sh family=secondmate expected_gate_skip=none
ok - A1 fm-harness.sh secondmate resolves the fallback chain; crew mode unchanged
ok - C1 fm-harness.sh secondmate-model/secondmate-effort resolve the optional tokens; bare harness stays empty (backward-compat)
ok - pi-signed identity: authoritative launch selection distinguishes shared wrapper ancestry
ok - harness identity: dash-leading ps command names are basename operands, not options
ok - B1 propagate_inheritable_config: copy, idempotence, convergence, absence-mirror, exclusion, no-op, skip diagnostics
ok - B2 spawn: secondmate runs the secondmate harness; its home inherits declared config
ok - B3 spawn: an absent secondmate-harness falls back to the crew harness (backward-compat)
ok - B4 spawn: no config at all -> own harness and no propagation side effects
ok - B5 spawn: an explicit per-spawn harness arg overrides config/secondmate-harness
ok - B6 spawn: an unverified resolved secondmate harness is refused (guard intact)
ok - B5b spawn: FM_BACKEND wins over inherited config/backend
ok - B5c spawn: explicit --backend wins over FM_BACKEND and inherited config/backend
ok - C2 spawn: a bare harness-only secondmate-harness file launches with no model/effort flag (backward-compat)
ok - C3 spawn: config/secondmate-harness's model token threads --model into the launch and meta
ok - C4 spawn: config/secondmate-harness's model+effort tokens thread into the launch and meta
ok - C5 spawn: an explicit --model overrides config/secondmate-harness's model token; the file's effort token still applies
ok - C6 spawn: an explicit --effort overrides config/secondmate-harness's effort token; the file's model token still applies
ok - C7 spawn: an explicit --harness starts with clean model/effort defaults
ok - C8 spawn: an explicit --harness still honors explicit model/effort flags
ok - C9 spawn: secondmate launch pins supervision to its own harness
ok - C9 spawn: the harness fallback chain still resolves with no tokens; crew/scout launches are unaffected by this feature
ok - B7 bootstrap sweep pushes, re-converges, and mirrors absence; never inherits secondmate-harness
ok - B8 bootstrap sweep propagates config even when the home's tracked files are already current
ok - B9 bootstrap sweep defers new inherited config until the home ignores it
ok - B10 bootstrap sweep materializes and inherits the startup-memory default while fast-forwarding
ok - B12b backend inheritance: present values and primary absence converge exactly
ok - B12c presentation inheritance: the primary default converges on, and only an explicit opt-out propagates off
ok - B11 bootstrap sweep surfaces config propagation failures
ok - B11 bootstrap rereads completed config writes after partial propagation
ok - B12 config-push propagates via shared live discovery, reports items, rereads on change only, and does not fast-forward
ok - B13 config-push reports dirty, non-allowing, and invalid homes without failing warnings-only runs
ok - B14 config-push exits nonzero on real propagation errors
ok - B14 config-push rereads completed config writes after partial propagation
ok - B15 config reread is per-home, exact-byte, ordered, and pointer-only
ok - B16 config reread isolation, ABSENT, generation safety, send failure, and retry
ok - B20 config reread publication failures retain exact generations for retry
ok - B21 config reread instruction-write failures retain exact retry generations
ok - B21 config reread preserves exact bytes when temporary adoption also fails
ok - B21 config reread serializes concurrent propagation and delivery
ok - B22 full config reread retry queues drain before new publication
ok - B23 mixed config reread delivery failures still bound sent history
ok - B26 config reread delivery stops after the oldest failed generation
ok - B17 config reread skips unchanged homes and reads destination post-write bytes
ok - B18 bootstrap config reread path works; spawn flexibility remains defaults-only
ok - B19 bootstrap respawns before inherited-config reread
ok - B25 spawn quarantines stale rereads without blocking relaunch
ok - B24 bootstrap detect-only mode remains filesystem read-only
# all fm-secondmate-harness tests passed
FM_TEST_END 2026-08-07T02:07:03Z tests/fm-secondmate-harness.test.sh exit=0 duration_ms=154119 gate_skip=false
FM_TEST_SUMMARY total=1 failed=0 skipped_gate=0 duration_ms=154164
FM_TEST_SUMMARY_FAMILY family=secondmate count=1 duration_ms=154119 failed=0
FM_TEST_SLOWEST rank=1 script=tests/fm-secondmate-harness.test.sh duration_ms=154119
fm-test-run: wrote timing artifact: /var/folders/_4/q0rlygb56rs6y6xp9sbql5240000gn/T/no-mistakes-evidence/01KZCZ63K31AKPQ8VQDR58SYM3/target-affected-timing.json
Evidence: Leased crew-spawn evidence

End-to-end crew spawn calls get --lease, records the same leased worktree as kind=ship, and matches the independent pane witness.

END-TO-END LEASED CREW SPAWN EVIDENCE
allocator calls: get --lease --lease-holder fm-crew-lease-e2e 
recorded task metadata:
worktree=/var/folders/_4/q0rlygb56rs6y6xp9sbql5240000gn/T/no-mistakes-evidence/01KZCZ63K31AKPQ8VQDR58SYM3/leased-crew-spawn-world/leased-worktree
harness=codex
kind=ship
model=default
effort=default
independent pane witness: /var/folders/_4/q0rlygb56rs6y6xp9sbql5240000gn/T/no-mistakes-evidence/01KZCZ63K31AKPQ8VQDR58SYM3/leased-crew-spawn-world/leased-worktree
launch flags preserved: no model or effort flags
- Outcome: ⚠️ 1 warning across 1 run (5m37s)

Pipeline

Updates from git push no-mistakes

✅ **intent** - passed

✅ No issues found.

✅ **Rebase** - passed

✅ No issues found.

⚠️ **Review** - 1 error
⚠️ **Test** - 1 warning
  • ⚠️ The mandatory PR-body record cannot be verified: no pull request exists for fm/secondmate-harness-fake-treehouse, so the required outage, causation, carve-out, expected-Herdr-red, and false-green statements are absent from a reviewable PR. Create or point me to the intended PR before merge.
  • git archive 569dae62b67dac7364916d9c361bcde9b2268d94 followed by bin/fm-test-run.sh tests/fm-secondmate-harness.test.sh - reproduced the exact historical failure.
  • bin/fm-test-run.sh tests/fm-secondmate-harness.test.sh --json /var/folders/_4/q0rlygb56rs6y6xp9sbql5240000gn/T/no-mistakes-evidence/01KZCZ63K31AKPQ8VQDR58SYM3/target-affected-timing.json - target affected suite passed.
  • Isolated target-source snapshot plus verify-leased-crew-spawn.sh - verified allocator call, persisted kind=ship metadata, matching pane witness, and no injected model/effort flags.
  • bin/fm-test-run.sh --list --lane portable-serial-4of4 - confirmed the affected script belongs to the named CI shard.
  • gh-axi pr list --state all --head fm/secondmate-harness-fake-treehouse - found no matching pull request.
✅ **Document** - passed

✅ No issues found.

✅ **Lint** - passed

✅ No issues found.

✅ **Push** - passed

✅ No issues found.


View with [code]smith Autofix with [code]smith
Need help on this PR? Tag @codesmith-bot with what you need. Autofix is disabled.

…fake treehouse

The crew-unaffected block spawns an ordinary ship task through
fm-spawn.sh, which since the allocator-lease change (#5) takes the task
worktree from 'treehouse get --lease' before confirming it against the
pane probe. That block's fakebin carried no treehouse at all and the
test's BASE_PATH is the bare system directories, so the spawn died on a
missing allocator before writing its meta record and the crew-unaffected
assertion failed deterministically, in CI and locally. The lease change
migrated every other spawn-driving test to tests/lib.sh's shared fake
treehouse but missed this file.

Install the shared fake in make_launch_capturing_tmux, whose stub set
exists exactly to serve the crew/scout spawn path. The fake leases
FM_FAKE_PANE_PATH, so the recorded worktree and the pane witness agree
and the confirmation step is exercised rather than bypassed. Secondmate
spawns never consult the allocator and are unaffected.
@elixlabssolutions
elixlabssolutions merged commit 7b26778 into main Aug 7, 2026
15 of 16 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant