test: guard the live harness gate against Claude Code's auto-updater - #6002
Merged
kunchenguid merged 3 commits intoSep 28, 2026
Merged
Conversation
fm_live_gate let a live run proceed without ever setting DISABLE_AUTOUPDATER, so a live Claude test could let the real updater repoint ~/.local/bin/claude into a temporary directory and stop every Claude process on the machine from starting. Export DISABLE_AUTOUPDATER=1 on every path where the gate lets a live run proceed, and assert the export in tests/fm-live-gate.test.sh, including that it reaches a child process the same way a real harness pane would inherit it.
 |
![greptile-apps[bot]](https://avatars.githubusercontent.com/in/867647?s=60&v=4){kind=small}
…nheritance test only checked a `bash -c` direct child, not the fm-spawn.sh launch path. The user chose to fix it with a regression on that path. In tests/fm-live-gate.test.sh I replaced the generic child test with test_disable_autoupdater_reaches_the_claude_pane_on_the_fm_spawn_launch_path: it drives the real fm-spawn claude launch through the spawn fixtures, captures the exact staged launch command, and runs it as a synthetic pane whose only `claude` is a stub recording the inherited DISABLE_AUTOUPDATER, asserting it saw 1. Switched the file to source fixtures.sh (pulls in lib.sh, guarded) for the spawn helpers. Verified it is a real guard: the stub records `1` when the ambient var is set and `unset` when absent, so it fails if fm-spawn ever scrubbed the variable (e.g. env -i or -u). This confirms fm-spawn's launch construction never references the name and passes it through via ordinary ambient inheritance with no allowlist. Full suite passes (12 tests ok), shellcheck clean. Note for the outer executor: I embedded the daemon caveat as a code comment in the test, but the finding also asks the PR body to state that a backend daemon already running before the gate exported the variable does not inherit it and fully covering that would need launcher support - that forge-side PR-body sentence is outside this CI phase's scope
karotkriss
changed the title
… handed its launch command to an already-running backend daemon that never inherited the test process's exported DISABLE_AUTOUPDATER, so ambient inheritance dropped it and Claude's auto-updater could still run. Fix (bin/fm-spawn.sh): when DISABLE_AUTOUPDATER is set in the spawn's own environment, embed `export DISABLE_AUTOUPDATER=<value>;` into the LAUNCH command text (same idiom as the adjacent COMPACT_ADVISER_DISABLE export), so it survives a daemon-built pane, the env -i allowlist path, and relaunch alike; gated on presence so ordinary spawns are unchanged. Added regression test test_disable_autoupdater_survives_a_daemon_pane_that_never_inherited_it in tests/fm-live-gate.test.sh: stages a real claude launch with DISABLE_AUTOUPDATER set in the spawn env, then runs that exact command in a synthetic pane with `env -u DISABLE_AUTOUPDATER` and asserts the claude stub still recorded autoupdater=1. Verified the test fails (autoupdater=unset) without the fix and passes with it; the round-1 ambient test stays green either way. Full suite passes (13 ok); test file and isolated snippet shellcheck-clean (full fm-spawn.sh shellcheck kept getting terminated by the memory-constrained host, not by findings). Forge-side note for the outer executor: the PR-body caveat that fully covering the daemon case would need launcher support no longer applies to the Claude launch path and should be corrected
kunchenguid
approved these changes
Sep 28, 2026
kunchenguid
left a comment
kunchenguid
left a comment
Owner
There was a problem hiding this comment.
Speaking as Kun's firstmate: approving after tip-vs-main inspection.
Contract-class restore: fm_live_gate proceed path now exports DISABLE_AUTOUPDATER=1, and fm-spawn embeds that value into the launch only when already set — ordinary unconfigured spawns unchanged. Restores the concrete test-isolation promise in #5959 (live harness must not rewrite host ~/.local/bin/claude).
VISION: aligns (scripts own mechanics; no new default consent; fleet survives vendor auto-updater failure mode during live tests). Safe, MATCH attestation, green CI, no .github/workflows.
Owner
|
Speaking as Kun's firstmate: this is merged. Thank you @karotkriss — really appreciate you taking the time on this. Contract-class: restore (own tip-vs-main). |
knowttl
pushed a commit
to knowttl/firstmate
that referenced
this pull request
Sep 29, 2026
…unchenguid#6002) * fix(tests): disable Claude Code's auto-updater during live harness runs fm_live_gate let a live run proceed without ever setting DISABLE_AUTOUPDATER, so a live Claude test could let the real updater repoint ~/.local/bin/claude into a temporary directory and stop every Claude process on the machine from starting. Export DISABLE_AUTOUPDATER=1 on every path where the gate lets a live run proceed, and assert the export in tests/fm-live-gate.test.sh, including that it reaches a child process the same way a real harness pane would inherit it. * no-mistakes(ci): Greptile flagged that the PR's DISABLE_AUTOUPDATER inheritance test only checked a `bash -c` direct child, not the fm-spawn.sh launch path. The user chose to fix it with a regression on that path. In tests/fm-live-gate.test.sh I replaced the generic child test with test_disable_autoupdater_reaches_the_claude_pane_on_the_fm_spawn_launch_path: it drives the real fm-spawn claude launch through the spawn fixtures, captures the exact staged launch command, and runs it as a synthetic pane whose only `claude` is a stub recording the inherited DISABLE_AUTOUPDATER, asserting it saw 1. Switched the file to source fixtures.sh (pulls in lib.sh, guarded) for the spawn helpers. Verified it is a real guard: the stub records `1` when the ambient var is set and `unset` when absent, so it fails if fm-spawn ever scrubbed the variable (e.g. env -i or -u). This confirms fm-spawn's launch construction never references the name and passes it through via ordinary ambient inheritance with no allowlist. Full suite passes (12 tests ok), shellcheck clean. Note for the outer executor: I embedded the daemon caveat as a code comment in the test, but the finding also asks the PR body to state that a backend daemon already running before the gate exported the variable does not inherit it and fully covering that would need launcher support - that forge-side PR-body sentence is outside this CI phase's scope * no-mistakes(ci): Fixed Greptile finding ci-1. Root cause: fm-spawn.sh handed its launch command to an already-running backend daemon that never inherited the test process's exported DISABLE_AUTOUPDATER, so ambient inheritance dropped it and Claude's auto-updater could still run. Fix (bin/fm-spawn.sh): when DISABLE_AUTOUPDATER is set in the spawn's own environment, embed `export DISABLE_AUTOUPDATER=<value>;` into the LAUNCH command text (same idiom as the adjacent COMPACT_ADVISER_DISABLE export), so it survives a daemon-built pane, the env -i allowlist path, and relaunch alike; gated on presence so ordinary spawns are unchanged. Added regression test test_disable_autoupdater_survives_a_daemon_pane_that_never_inherited_it in tests/fm-live-gate.test.sh: stages a real claude launch with DISABLE_AUTOUPDATER set in the spawn env, then runs that exact command in a synthetic pane with `env -u DISABLE_AUTOUPDATER` and asserts the claude stub still recorded autoupdater=1. Verified the test fails (autoupdater=unset) without the fix and passes with it; the round-1 ambient test stays green either way. Full suite passes (13 ok); test file and isolated snippet shellcheck-clean (full fm-spawn.sh shellcheck kept getting terminated by the memory-constrained host, not by findings). Forge-side note for the outer executor: the PR-body caveat that fully covering the daemon case would need launcher support no longer applies to the Claude launch path and should be corrected
RooseveltAdvisors
pushed a commit
to RooseveltAdvisors/firstmate
that referenced
this pull request
Sep 29, 2026
…unchenguid#6002) * fix(tests): disable Claude Code's auto-updater during live harness runs fm_live_gate let a live run proceed without ever setting DISABLE_AUTOUPDATER, so a live Claude test could let the real updater repoint ~/.local/bin/claude into a temporary directory and stop every Claude process on the machine from starting. Export DISABLE_AUTOUPDATER=1 on every path where the gate lets a live run proceed, and assert the export in tests/fm-live-gate.test.sh, including that it reaches a child process the same way a real harness pane would inherit it. * no-mistakes(ci): Greptile flagged that the PR's DISABLE_AUTOUPDATER inheritance test only checked a `bash -c` direct child, not the fm-spawn.sh launch path. The user chose to fix it with a regression on that path. In tests/fm-live-gate.test.sh I replaced the generic child test with test_disable_autoupdater_reaches_the_claude_pane_on_the_fm_spawn_launch_path: it drives the real fm-spawn claude launch through the spawn fixtures, captures the exact staged launch command, and runs it as a synthetic pane whose only `claude` is a stub recording the inherited DISABLE_AUTOUPDATER, asserting it saw 1. Switched the file to source fixtures.sh (pulls in lib.sh, guarded) for the spawn helpers. Verified it is a real guard: the stub records `1` when the ambient var is set and `unset` when absent, so it fails if fm-spawn ever scrubbed the variable (e.g. env -i or -u). This confirms fm-spawn's launch construction never references the name and passes it through via ordinary ambient inheritance with no allowlist. Full suite passes (12 tests ok), shellcheck clean. Note for the outer executor: I embedded the daemon caveat as a code comment in the test, but the finding also asks the PR body to state that a backend daemon already running before the gate exported the variable does not inherit it and fully covering that would need launcher support - that forge-side PR-body sentence is outside this CI phase's scope * no-mistakes(ci): Fixed Greptile finding ci-1. Root cause: fm-spawn.sh handed its launch command to an already-running backend daemon that never inherited the test process's exported DISABLE_AUTOUPDATER, so ambient inheritance dropped it and Claude's auto-updater could still run. Fix (bin/fm-spawn.sh): when DISABLE_AUTOUPDATER is set in the spawn's own environment, embed `export DISABLE_AUTOUPDATER=<value>;` into the LAUNCH command text (same idiom as the adjacent COMPACT_ADVISER_DISABLE export), so it survives a daemon-built pane, the env -i allowlist path, and relaunch alike; gated on presence so ordinary spawns are unchanged. Added regression test test_disable_autoupdater_survives_a_daemon_pane_that_never_inherited_it in tests/fm-live-gate.test.sh: stages a real claude launch with DISABLE_AUTOUPDATER set in the spawn env, then runs that exact command in a synthetic pane with `env -u DISABLE_AUTOUPDATER` and asserts the claude stub still recorded autoupdater=1. Verified the test fails (autoupdater=unset) without the fix and passes with it; the round-1 ambient test stays green either way. Full suite passes (13 ok); test file and isolated snippet shellcheck-clean (full fm-spawn.sh shellcheck kept getting terminated by the memory-constrained host, not by findings). Forge-side note for the outer executor: the PR-body caveat that fully covering the daemon case would need launcher support no longer applies to the Claude launch path and should be corrected
andrewesweet
pushed a commit
to andrewesweet/firstmate
that referenced
this pull request
Sep 30, 2026
…unchenguid#6002) * fix(tests): disable Claude Code's auto-updater during live harness runs fm_live_gate let a live run proceed without ever setting DISABLE_AUTOUPDATER, so a live Claude test could let the real updater repoint ~/.local/bin/claude into a temporary directory and stop every Claude process on the machine from starting. Export DISABLE_AUTOUPDATER=1 on every path where the gate lets a live run proceed, and assert the export in tests/fm-live-gate.test.sh, including that it reaches a child process the same way a real harness pane would inherit it. * no-mistakes(ci): Greptile flagged that the PR's DISABLE_AUTOUPDATER inheritance test only checked a `bash -c` direct child, not the fm-spawn.sh launch path. The user chose to fix it with a regression on that path. In tests/fm-live-gate.test.sh I replaced the generic child test with test_disable_autoupdater_reaches_the_claude_pane_on_the_fm_spawn_launch_path: it drives the real fm-spawn claude launch through the spawn fixtures, captures the exact staged launch command, and runs it as a synthetic pane whose only `claude` is a stub recording the inherited DISABLE_AUTOUPDATER, asserting it saw 1. Switched the file to source fixtures.sh (pulls in lib.sh, guarded) for the spawn helpers. Verified it is a real guard: the stub records `1` when the ambient var is set and `unset` when absent, so it fails if fm-spawn ever scrubbed the variable (e.g. env -i or -u). This confirms fm-spawn's launch construction never references the name and passes it through via ordinary ambient inheritance with no allowlist. Full suite passes (12 tests ok), shellcheck clean. Note for the outer executor: I embedded the daemon caveat as a code comment in the test, but the finding also asks the PR body to state that a backend daemon already running before the gate exported the variable does not inherit it and fully covering that would need launcher support - that forge-side PR-body sentence is outside this CI phase's scope * no-mistakes(ci): Fixed Greptile finding ci-1. Root cause: fm-spawn.sh handed its launch command to an already-running backend daemon that never inherited the test process's exported DISABLE_AUTOUPDATER, so ambient inheritance dropped it and Claude's auto-updater could still run. Fix (bin/fm-spawn.sh): when DISABLE_AUTOUPDATER is set in the spawn's own environment, embed `export DISABLE_AUTOUPDATER=<value>;` into the LAUNCH command text (same idiom as the adjacent COMPACT_ADVISER_DISABLE export), so it survives a daemon-built pane, the env -i allowlist path, and relaunch alike; gated on presence so ordinary spawns are unchanged. Added regression test test_disable_autoupdater_survives_a_daemon_pane_that_never_inherited_it in tests/fm-live-gate.test.sh: stages a real claude launch with DISABLE_AUTOUPDATER set in the spawn env, then runs that exact command in a synthetic pane with `env -u DISABLE_AUTOUPDATER` and asserts the claude stub still recorded autoupdater=1. Verified the test fails (autoupdater=unset) without the fix and passes with it; the round-1 ambient test stays green either way. Full suite passes (13 ok); test file and isolated snippet shellcheck-clean (full fm-spawn.sh shellcheck kept getting terminated by the memory-constrained host, not by findings). Forge-side note for the outer executor: the PR-body caveat that fully covering the daemon case would need launcher support no longer applies to the Claude launch path and should be corrected
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Intent
Fixes #5959
Live harness tests can let Claude Code's auto-updater rewrite the real ~/.local/bin/claude.
Nothing in the repository sets DISABLE_AUTOUPDATER, and fm_live_gate in tests/lib.sh lets a live run proceed without it, so a live Claude test can repoint the shared symlink into a temporary directory and stop every Claude process on the machine from starting.
When the live gate lets a live run proceed it should export DISABLE_AUTOUPDATER=1 so the updater cannot run during tests.
What Changed
fm_live_gateintests/lib.shnow exportsDISABLE_AUTOUPDATER=1on the proceed path, so any live-harness run it green-lights cannot let Claude Code's auto-updater rewrite the installed binary; a comment on the gate documents the guarantee.test_gate_exports_disable_autoupdater_for_a_proceeding_runtotests/fm-live-gate.test.sh, asserting a proceeding guard seesDISABLE_AUTOUPDATER=1in its environment.test_disable_autoupdater_reaches_the_claude_pane_on_the_fm_spawn_launch_path, which stages a realfm-spawnclaude launch and replays it against a synthetic pane whose stubclauderecords the inherited value, proving the ambient variable rides the launch through to the harness pane; the file now sourcesfixtures.shfor the spawn helpers.Risk Assessment
✅ Low: A one-line export on the gate's single proceed path exactly implements the required behavior, and the two new tests are behavioral regressions (not source-content assertions) that cover both the gate export and the fm-spawn inheritance path the user chose to fix.
Testing
Drove the change through its real executable interface, tests/fm-live-gate.test.sh, which runs live guard scripts as separate processes calling fm_live_gate. A proceeding guard observes DISABLE_AUTOUPDATER=1, and I confirmed this is a true regression by removing the fix line (guard then saw autoupdater=unset) and restoring it. The fm-spawn launch-path case stages a real claude launch and runs it as a synthetic recording pane, proving the ambient variable is passed through. The family sweep confirms all 40 guards wire through the shared gate. No UI surface exists for this test-harness change, so evidence is the CLI transcript rather than a screenshot.
bash tests/fm-live-gate.test.sh-> ok 'a proceeding live run exports DISABLE_AUTOUPDATER=1'; guard process prints autoupdater=1not ok ... (missing: 'autoupdater=1'), autoupdater=unset, EXIT=1; restored -> passesbash tests/fm-live-gate.test.sh-> ok 'DISABLE_AUTOUPDATER rides fm-spawn's claude launch through to the harness pane'; synthetic pane stub records autoupdater=1 from the staged launchbash tests/fm-live-gate.test.sh-> ok 'all 40 live guards refuse together on FM_LIVE=0'Evidence: fm-live-gate suite (with fix, all ok)
Source: fm-live-gate suite (with fix, all ok)
Evidence: Regression: fix removed -> proceeding-run test fails
Pipeline
Updates from git push no-mistakes
✅ **intent** - passed
✅ No issues found.
✅ **Rebase** - passed
✅ No issues found.
✅ **Review** - passed
✅ No issues found.
✅ **Test** - passed
✅ No issues found.
bash tests/fm-live-gate.test.sh-> ok 'a proceeding live run exports DISABLE_AUTOUPDATER=1'; guard process prints autoupdater=1not ok ... (missing: 'autoupdater=1'), autoupdater=unset, EXIT=1; restored -> passesbash tests/fm-live-gate.test.sh-> ok 'DISABLE_AUTOUPDATER rides fm-spawn's claude launch through to the harness pane'; synthetic pane stub records autoupdater=1 from the staged launchbash tests/fm-live-gate.test.sh-> ok 'all 40 live guards refuse together on FM_LIVE=0'bash tests/fm-live-gate.test.sh(all cases ok, EXIT=0)Regression check: removedexport DISABLE_AUTOUPDATER=1from tests/lib.sh, re-ran suite ->test_gate_exports_disable_autoupdater_for_a_proceeding_runfails withautoupdater=unset, then restored lib.sh (git status clean)✅ **Document** - passed
✅ No issues found.
✅ **Lint** - passed
✅ No issues found.
✅ **Push** - passed
✅ No issues found.