Skip to content

glaeda-cmux-runner-hook: give test-e2e's build and test the gui token - #1259

Merged
teamleaderleo merged 3 commits into
mainfrom
hook-test-e2e-gui
Sep 25, 2026
Merged

teamleaderleo merged 3 commits into
mainfrom
hook-test-e2e-gui

Conversation

@teamleaderleo

@teamleaderleo teamleaderleo commented Sep 25, 2026 •

Copy link
Copy Markdown
Owner

Why

The hook did not know manaflow-ai/cmux test-e2e.yml's build and test, so it treated both as a compile pinned to root 1 (persistent-dd + root). Neither job took the gui token. The cmux PR "ci: run E2E tests on the runner that built them" (branch ci/e2e-single-runner) moves E2E test execution into build, through the e2e-run-tests composite action. That action runs cmuxTests through run-in-console-session.sh, and cmuxUITests in the console user's GUI bootstrap. test stays only as a fallback. As a result, an E2E build on an owned mini ran tests in the shared console session while app-host-unit-tests, tests-build-and-lag or a rerun held the gui token. The old test job had the same gap.

What

  • ("test-e2e.yml", "build") gets a new class, compile-gui: 2 units, tokens gui + root.
    • No persistent-dd. The job never runs owned_build_state.py, never sets CMUX_OWNED_STATE_ROOT or CMUX_SEED_LOCAL_CACHE, and adopts seeds from R2. The only DerivedData and CAS it wipes are <root>/derived-data-compile-admission and <root>/compile-admission-cas at the root it holds. The kept admission state lives in the store and is cloned in, so it is not touched.
    • Producer, not a root consumer. It takes the root its token picks, and every root path in the job uses ${CMUX_CI_CANONICAL_ROOT:-/private/tmp/cmux-ci}, so it is no longer pinned to root 1. Its own restore step re-takes that root through glaeda-canonical-root take, which is a no-op.
    • Root 1 first. Unlike compile admission, it skips the own-root preference: it tries root 1, then any free root. Main publishes seeds and caches keyed to root 1, since compile-app-host-test-product.sh adds root= to the fingerprint for any other root, and the E2E build has no per-root state that would favor another root.
    • Tokens are taken in the order gui, then root. When the gui token is taken, the refusal is the one GUI_WAIT_S (60 s) waits for.
  • ("test-e2e.yml", "test") becomes gui, like app-host-test-rerun's rerun. With more than one root it is a consumer: its restore step takes the producer's root, which restore-app-host-test-product.sh reads from the receipt.
  • Comments and docs/CMUX_MINI_RUNNER.md no longer say that bare build/test name test-e2e's jobs. The stale doc line listing app-host-test-rerun as an unknown job is fixed.

Trade-off

compile-gui takes the gui token at job start and holds it through the compile. The hook cannot see the later step where the tests start. So while an E2E build runs on a mini, other GUI jobs there wait 60 s and are then refused, and the owned-pool rescue moves them (REFUSAL_SECONDS 120 covers test-e2e too). Releasing the token during the compile would need a cmux change, such as a take-gui helper run before the test step, and this PR does not change cmux.

Tests

  • New test_capacity_e2e_jobs_hold_the_gui_token (two roots). It checks:
    • build holds gui+root-1 without persistent-dd and exports the root.
    • The build's re-take of its own root is a no-op.
    • A compile admission still runs in root 2.
    • app-host-unit-tests, E2E test and a second E2E build each wait for the gui token, then are refused.
    • E2E test takes no root at job start and then takes the producer's root.
  • test_job_class_keys_on_workflow_file_and_job_id is extended, including a check that every class named in the tables has a cost.
  • scripts/test-glaeda-cmux-runner.py: 152 OK. scripts/test-glaeda-fleet-status.py: 65 OK.

Review

An independent read-only review of the first head found no correctness problems (persistent-dd, root paths, deadlock and stranding all checked). It found one performance problem: a build on runner instance 1 would compile cold at root 2. That is fixed in 22a2241, which also clarifies the docs for the one-root case. Full suite on 22a2241: 152 OK.

Rollout

After merge: scripts/glaeda-cmux-runner-fleet --apply --hosts <canary>, then the rest. job-started runs the hook fresh for each job, so no restart is needed.

🤖 Generated with Claude Code

teamleaderleo and others added 2 commits September 25, 2026 13:49
test-e2e.yml's `build` and `test` were unknown job ids, so both were costed
as a pinned compile (persistent-dd + root 1) and neither took the gui token.
cmux's "ci: run E2E tests on the runner that built them" moves the E2E tests
into `build` (cmuxTests through run-in-console-session.sh, cmuxUITests in the
console user's GUI bootstrap), so on an owned mini it ran GUI tests next to
app-host shards holding the one console session. The old `test` had the same
gap.

- ("test-e2e.yml", "build") is a new class, compile-gui: 2 units, gui + root,
  no persistent-dd (it never runs owned_build_state.py or sets
  CMUX_SEED_LOCAL_CACHE). It is a producer, so it takes the root its token
  picks and reads it from CMUX_CI_CANONICAL_ROOT, as every path in the job
  already does; its own restore's take-root is a no-op.
- ("test-e2e.yml", "test") is gui, like app-host-test-rerun's rerun: a root
  consumer that takes the producer's root from its restore step.
- Both wait GUI_WAIT_S for the gui token before refusing, like other GUI jobs.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A second root has its own seed and cache keys (compile-app-host-test-product.sh
fingerprint adds root=), and the E2E build keeps no per-root state, so the
own-root preference only made a build on runner instance 1 compile cold at
root 2 while root 1 was free. compile-gui now tries root 1 first and falls
back to any free root. Docs note the unpinning applies with canonicalRoots > 1.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
cmux#14692 starts owned E2E builds from compile admission's kept state for
their root (clone only, never keep/save/record). The root token already
guards that store, so compile-gui still takes no persistent-dd; the
comment and docs said the job never touched it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@teamleaderleo
teamleaderleo merged commit 1e19776 into main Sep 25, 2026
4 checks passed
@teamleaderleo
teamleaderleo deleted the hook-test-e2e-gui branch September 25, 2026 19:46
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant