Skip to content

fix(bin): keep remote home seeding portable on Bash 3.2 and fail closed on missing backend adapters - #72

Merged
withally merged 4 commits into
mainfrom
fm/fm-remote-lifecycle-e2e-seed-fail-f1
Aug 24, 2026
Merged

withally merged 4 commits into
mainfrom
fm/fm-remote-lifecycle-e2e-seed-fail-f1

Conversation

@withally

Copy link
Copy Markdown
Owner

Intent

Fix the two pre-existing baseline test failures found on 2026-08-24 in the firstmate repository's own suite on clean current main using bin/fm-test-run.sh: reproduce and properly root-cause the remote-lifecycle E2E seed failure and its sibling baseline failure, fixing defects rather than weakening assertions. If a failure is environment-dependent, make the test deterministic or properly gated and disclose that in the PR body. Scope was expanded after clean-main proof to include the fm-pending-reply and fm-wake-queue regressions. The fm-tool-update-check regression already had a fix in flight on fm/fm-lock-steal-recursion-fix-x2, so do not duplicate that fix. The resulting PR must make the relevant baseline suite green together with the x2 branch. Preserve Firstmate's shared tracked-material coding guidelines and do not broaden into unrelated baseline or environment failures.

What Changed

  • bin/fm-remote-home-seed.sh now walks PROJECT_NAMES by index instead of expanding "${PROJECT_NAMES[@]}", so a --no-projects seed no longer aborts under set -u on Bash 3.2 (the baseline /bin/bash on macOS).
  • bin/fm-backend.sh's fm_backend_source prechecks that the adapter file exists and is readable, printing error: backend adapter is unavailable: <path> and returning 1, instead of letting a dot-source of an absent file abort the whole non-interactive shell before the per-adapter || return 1 can run.
  • tests/fm-remote-secondmate-lifecycle-e2e.test.sh replaces the cp -R + diff -ru unchanged-tree checks with a shared tree_snapshot helper that records each entry by its own type (symlink by literal target, regular file by digest), removing the platform-dependent handling of symlinks and non-regular entries; the helper fails closed when the directory is missing so a deleted tree can no longer read as unchanged, and every call site fails with a distinct message. Failure output in this test and in tests/fm-teardown.test.sh now includes the captured stderr/stdout of the failing step.

Risk Assessment

✅ Low: Both source fixes are narrow, root-caused corrections I reproduced directly (bash 3.2 empty-array expansion under set -u, and . on a missing file aborting an errexit shell), the test changes strengthen rather than weaken the residue assertions, and the round-2 fail-closed fix genuinely closes the empty-snapshot hole at every call site.

Testing

Reproduced both reported baseline failures by reverting only the two bin/ source files to the base commit and re-running the affected scripts under bin/fm-test-run.sh, then confirmed the branch turns them green (5/5 across the remote-lifecycle E2E, teardown, backend-dispatch, pending-reply and wake-queue scripts). Root-caused each to a real defect — an empty-array expansion under set -u on macOS bash 3.2, and a failing adapter source that set -e plus an EXIT trap silently converts into a successful teardown that skips its safety preflight — and captured before/after CLI transcripts at both user-facing surfaces plus the suite summaries as evidence. No visual artifact applies: this change is entirely shell CLI behavior with no rendered surface. fm-pending-reply and fm-wake-queue, named in the expanded scope, pass with base sources too and never reproduced here; worktree left clean with all temporary checkouts removed.

Evidence: fm-remote-home-seed.sh --no-projects: before/after CLI transcript

Source: fm-remote-home-seed.sh --no-projects: before/after CLI transcript

--- BEFORE (base bac5c84) --- $ FM_HOME=<home> bin/fm-remote-home-seed.sh demo remote-mac <repo> <remote-home> --no-projects <repo>/bin/fm-remote-home-seed.sh: line 190: PROJECT_NAMES[@]: unbound variable exit=0 --- AFTER (branch 30f2a9b) --- $ FM_HOME=<home> bin/fm-remote-home-seed.sh demo remote-mac <repo> <remote-home> --no-projects error: remote command is not tracked by this Firstmate checkout: fm-remote-doctor.sh error: remote runtime preflight failed; nothing was provisioned. Close the gaps listed above, or update the remote code root if it predates the current fm-remote-doctor.sh exit=1

=== USER-FACING CLI: seed a remote secondmate home with no projects (macOS bash 3.2) ===
Command is identical in both runs; only the checkout differs.

--- BEFORE (base bac5c84): crashes in the project-assembly loop ---
$ FM_HOME=<home> bin/fm-remote-home-seed.sh demo remote-mac <repo> <remote-home> --no-projects   # (before)
<repo>/bin/fm-remote-home-seed.sh: line 190: PROJECT_NAMES[@]: unbound variable
exit=0

--- AFTER (this branch 30f2a9b): loop is safe on an empty project list;
    the run advances to the remote-runtime preflight and refuses there for
    an unrelated, correctly-reported reason (this demo points the remote
    code root at the local checkout, which has no fm-remote-doctor.sh) ---
$ FM_HOME=<home> bin/fm-remote-home-seed.sh demo remote-mac <repo> <remote-home> --no-projects   # (after)
error: remote command is not tracked by this Firstmate checkout: fm-remote-doctor.sh
error: remote runtime preflight failed; nothing was provisioned. Close the gaps listed above, or update the remote code root if it predates the current fm-remote-doctor.sh
exit=1
Evidence: fm-teardown herdr preflight with a missing adapter: before/after

Source: fm-teardown herdr preflight with a missing adapter: before/after

--- BEFORE (base bac5c84) --- <checkout>/bin/fm-backend.sh: line 609: <checkout>/bin/backends/herdr.sh: No such file or directory teardown exit status: 0 --- AFTER (branch 30f2a9b) --- error: backend adapter is unavailable: <checkout>/bin/backends/herdr.sh error: herdr teardown prerequisites are unavailable for task-x1; nothing was changed - restore the adapter and rerun teardown teardown exit status: 1

=== fm-teardown.sh herdr preflight when bin/backends/herdr.sh is missing ===

fm-teardown.sh runs under 'set -eu' (line 136) with an EXIT trap (line 207).
The driver below reproduces exactly that shell contract around the dispatcher
call at fm-teardown.sh:2072 (teardown_herdr_require_prerequisites).

BEFORE (base bac5c84): the failing '. adapter' aborts the shell under set -e,
the EXIT trap succeeds, and teardown reports SUCCESS having silently skipped
its required safety preflight -- the baseline failure
'herdr-preflight-missing-adapter: teardown continued without its required preflight'.
--- BEFORE (base bac5c84) ---
<checkout>/bin/fm-backend.sh: line 609: <checkout>/bin/backends/herdr.sh: No such file or directory
teardown exit status: 0

AFTER (this branch 30f2a9b): the dispatcher checks the adapter is present and
readable and returns 1 without ever running a failing '.', so teardown reaches
its own refusal path and exits non-zero, changing nothing.
--- AFTER (branch 30f2a9b) ---
error: backend adapter is unavailable: <checkout>/bin/backends/herdr.sh
error: herdr teardown prerequisites are unavailable for task-x1; nothing was changed - restore the adapter and rerun teardown
teardown exit status: 1
Evidence: bin/fm-test-run.sh before/after summaries

Source: bin/fm-test-run.sh before/after summaries

BEFORE (branch tests + base bin/ sources): not ok - failing seed exited before remote provisioning bin/fm-remote-home-seed.sh: line 190: PROJECT_NAMES[@]: unbound variable not ok - herdr-preflight-missing-adapter: teardown continued without its required preflight AFTER (branch 30f2a9b): FM_TEST_SUMMARY total=5 failed=0 skipped_gate=0 duration_ms=514956

=== bin/fm-test-run.sh on the affected baseline scripts (macOS, GNU bash 3.2.57) ===

--- BEFORE: branch tests + base (bac5c84) bin/ sources ---
$ bin/fm-test-run.sh tests/fm-remote-secondmate-lifecycle-e2e.test.sh tests/fm-pending-reply.test.sh tests/fm-wake-queue.test.sh
not ok - failing seed exited before remote provisioning
FM_TEST_SUMMARY total=3 failed=1 skipped_gate=0 duration_ms=49466
/Users/ivan/.no-mistakes/worktrees/37852af5566c/01M0SSQ9W7VEMZYXANR8PTV8RX/bin/fm-remote-home-seed.sh: line 190: PROJECT_NAMES[@]: unbound variable
$ bin/fm-test-run.sh tests/fm-teardown.test.sh
not ok - herdr-preflight-missing-adapter: teardown continued without its required preflight
FM_TEST_SUMMARY total=1 failed=1 skipped_gate=0 duration_ms=39080

--- AFTER: this branch at 30f2a9b ---
$ bin/fm-test-run.sh tests/fm-remote-secondmate-lifecycle-e2e.test.sh tests/fm-teardown.test.sh tests/fm-pending-reply.test.sh tests/fm-wake-queue.test.sh tests/fm-backend.test.sh
FM_TEST_SUMMARY total=5 failed=0 skipped_gate=0 duration_ms=514956
FM_TEST_SUMMARY_FAMILY family=backend-dispatch count=1 duration_ms=32329 failed=0
FM_TEST_SUMMARY_FAMILY family=pr-forge count=1 duration_ms=159065 failed=0
FM_TEST_SUMMARY_FAMILY family=secondmate count=1 duration_ms=266146 failed=0
FM_TEST_SUMMARY_FAMILY family=unclassified count=1 duration_ms=18198 failed=0
FM_TEST_SUMMARY_FAMILY family=watcher-wake-lock count=1 duration_ms=39006 failed=0
- Outcome: ⚠️ 1 info across 1 run (14m39s)

Pipeline

Updates from git push no-mistakes

✅ **intent** - passed

✅ No issues found.

✅ **Rebase** - passed

✅ No issues found.

⚠️ **Review** - 1 info
  • ⚠️ tests/fm-remote-secondmate-lifecycle-e2e.test.sh:1273 - tree_snapshot returns empty output for a non-existent directory (cd &#34;$dir&#34; 2&gt;/dev/null || exit 0 at line 447), so assert_retirement_refused now compares "" to "" and passes silently if $REMOTE_HOME is absent. The replaced cp -R + diff -ru pair failed loudly in that case (cp errored, diff on two missing paths returned non-zero -> "refusal changed the remote directory"). Concrete trace: if a preceding case leaves $REMOTE_HOME deleted, home_before="", teardown refuses, second snapshot is "", the equality holds and the assertion reports success without ever inspecting a remote home. Add [ -d &#34;$REMOTE_HOME&#34; ] || fail &#34;$label refusal ran without a remote home&#34; before capturing home_before (assert_retirement_completed already distinguishes this via its existed flag).
  • ⚠️ tests/fm-remote-secondmate-lifecycle-e2e.test.sh:1 - The intent states "Scope was expanded after clean-main proof to include the fm-pending-reply and fm-wake-queue regressions" and "The resulting PR must make the relevant baseline suite green together with the x2 branch". The branch diff (bac5c84..d68565a) contains nothing that touches those paths: neither tests/fm-pending-reply.test.sh nor tests/fm-wake-queue.test.sh is modified, neither contains the empty-array pattern fixed in fm-remote-home-seed.sh, and fm-wake-queue.test.sh does not reference the backend adapter path fixed in fm-backend.sh. Either those two regressions are already resolved by the base commit bac5c84 (the merged non-recursive steal-mutex lock fix, which both tests exercise heavily) - in which case say so and disclose it - or the expanded scope is unmet. This needs the author's confirmation rather than a reviewer guess.
  • ℹ️ bin/fm-backend.sh:598 - The new guard closes the missing-adapter case, but the same bash 3.2 hazard (. on an unopenable file terminates the shell, defeating || return 1) still exists one level down: bin/backends/tmux.sh:22-26 sources fm-tmux-lib.sh / fm-session-lock-lib.sh / fm-cursor-lib.sh unguarded, and bin/fm-teardown.sh:2088 sources fm-wake-lib.sh unguarded. Not reachable from any current test mode (the missing-parser and missing-explicit-close-helper cases keep the adapter file present), so this is not a blocker - noting it as the residual reach of the same class if a partial bin/ deployment ever occurs.

🔧 Fix: make tree_snapshot fail closed on missing directories
1 info still open:

  • ℹ️ bin/fm-backend.sh:598 - The adapter-existence guard is evaluated before the _FM_BACKEND_&lt;NAME&gt;_SOURCED cache check, so a second fm_backend_source herdr in a process that already sourced the adapter now returns 1 if the file disappeared in the meantime, where it previously succeeded as a no-op from in-memory functions. Concrete path: a long-lived bin/fm-watch.sh sources herdr, a non-atomic bin/ redeploy or branch checkout removes bin/backends/herdr.sh, and the next teardown_herdr_require_prerequisites (bin/fm-teardown.sh:2072) refuses with "restore the adapter and rerun teardown" despite every prerequisite function still being defined. Failing closed here is defensible (the process is running against a torn tree), so this is a note on the behavior change rather than a defect; moving the guard inside each if [ -z &#34;${_FM_BACKEND_*_SOURCED:-}&#34; ] block would preserve the old cached-success semantics if that is preferred.
⚠️ **Test** - 1 info
  • ℹ️ tests/fm-pending-reply.test.sh - fm-pending-reply and fm-wake-queue were named in the expanded intent scope, but they pass on this machine both with base (bac5c84) bin/ sources and with the branch — the regressions did not reproduce here, and nothing on this branch changes their code paths. Worth disclosing in the PR body so reviewers do not read them as fixed by this change.
  • bin/fm-test-run.sh tests/fm-remote-secondmate-lifecycle-e2e.test.sh tests/fm-teardown.test.sh tests/fm-pending-reply.test.sh tests/fm-wake-queue.test.sh tests/fm-backend.test.sh on the branch — 5/5 pass
  • Regression proof: temporarily restored bin/fm-remote-home-seed.sh and bin/fm-backend.sh to bac5c84 and re-ran the same scripts — reproduced not ok - failing seed exited before remote provisioning (PROJECT_NAMES[@]: unbound variable) and not ok - herdr-preflight-missing-adapter: teardown continued without its required preflight; source files restored, worktree verified clean
  • Manual CLI transcript: bin/fm-remote-home-seed.sh demo remote-mac &lt;root&gt; &lt;home&gt; --no-projects run against base and branch checkouts (git archive into /tmp) — base crashes and exits 0, branch advances past project assembly to the remote-runtime preflight and exits 1
  • Manual CLI transcript: fm_backend_source herdr with bin/backends/herdr.sh removed, driven under fm-teardown.sh's exact shell contract (set -eu + EXIT trap) — base exits 0 with only raw shell noise, branch prints error: backend adapter is unavailable: ... plus teardown's nothing was changed refusal and exits 1
  • grep -n &#39;^set -&#39; bin/fm-teardown.sh and grep -n &#39;trap .* EXIT&#39; bin/fm-teardown.sh to confirm the set -e / EXIT-trap masking mechanism
✅ **Document** - passed

✅ No issues found.

✅ **Lint** - passed

✅ No issues found.

✅ **Push** - passed

✅ No issues found.

@withally
withally merged commit 228119e into main Aug 24, 2026
13 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant