ci: UI test runs adopt compile admission's product, skip the re-upload, and report progress

[teamleaderleo]

Why

A pull request that changes a UI test class runs it in ui-tests, which dispatches test-e2e.yml and waits. On #15160 (run 36404376226) that looked stuck and compiled the app again although compile admission had just built it. The comparable dispatch, run 36403079789, spent:

• 376 s compiling the app-host and UI test product,
• 156 s running the selected tests,
• 92 s uploading the compiled product after the tests, before the verdict.

Cause of the recompile

test-e2e.yml already adopts compile admission's product and runs test-without-building when their product contracts match. They did not match. I compared the receipts sealed in the two artifacts (admission af95251f, run 36401440165, and the UI run's f807552c): they differ only in environment.SDKROOT. Hashing the UI run's contract with SDKROOT unset gives af95251f, admission's key.

Some owned Macs' runner services export SDKROOT=<Xcode>/.../MacOSX.sdk, and others export nothing (5 of 12 checked hosts export it). Both build against the same SDK, and the SDK build is already in the contract as sdk. The UI run in question landed on a Mac of one kind and admission on the other, so it missed with no_matching_contract_artifact and compiled. A pair on two Macs of the same kind (runs 36402498206 and 36404799894) adopted the product.

What

1. Reuse. reuse_app_host_products.contract() records SDKROOT as empty when it resolves to the selected Xcode's default macOS SDK (xcrun --show-sdk-path). Any other SDK still hashes. Products built with SDKROOT unset keep their keys, so nothing already published changes name.
2. Upload off the critical path. When the build job adopted a product and ran the tests itself, it no longer uploads the same archive again after the tests. The run it adopted already publishes that product, and later dispatches adopt it from there. The upload still runs when this job compiled, or when it leaves the tests to the test job, which reads it. e2e_sibling_build.py stops waiting on a sibling whose compile step was skipped, because the product it adopted is already there.
3. Progress. "Wait for the UI test run" prints one line a minute. It shows compile admission's state (queued for which label and for how long, or compiling on which runner, at which step) while the dispatch waits for its product. Once the test run exists, it shows its URL, whether build is queued (label, time) or running (runner, time), the current step, compile time or "adopted the compiled product, no build", and "testing N selected classes for T". It reads on every third verdict poll (a third more REST calls on the job token), and a failed read of any kind only skips a line: the test-e2e job list once the run is known, and before that, alternately, the listing that finds it and this attempt's jobs. A failed read skips a line and never changes the verdict. The webhook feed and glaeda-gh are not reachable from this GitHub-hosted runner, so this uses the job token.

Measured before (CI history)

• ci-timing.json, PR ui-tests job, last 24 h: run p50 1333 s, p90 1535 s, n=13.
• test-e2e.yml build jobs on owned Macs, 2026-09-27 23:29Z to 09-28 09:53Z (300 runs):
  • adopted: n=24, job p50 364 s. The late re-upload of the adopted product took p50 162 s, p90 245 s (n=18).
  • compiled: n=115, job p50 846 s. Compile p50 513 s, late upload p50 109 s.
• The UI runs dispatched by ci-ui-tests.yml in that window:
  • 36403079789: compiled 376 s beside admission's product (the SDKROOT mismatch).
  • 36401530356: compiled 652 s, then cancelled (same mismatch pattern: the admission host exports SDKROOT, the test host does not).
  • 36404799894: same kind of Mac as admission, so it adopted: 0 s compile, 228 s tests, then 153 s re-uploading.

Measured after (this PR's CI)

• Progress output, ui-tests in run 36412727305 (job 108897404233). One line a minute, for example:

  11:01:17Z Waiting for compile admission's product: admission is queued for glaeda-root-std-xcode-26.6 for 3m25s.
  11:11:37Z Waiting for compile admission's product: compiling on <mac> for 1m36s, at 'Restore SwiftPM manifest cache'.
  11:15:44Z Compile admission ended success; waiting for the dispatcher to start a UI test run on its product.
  11:16:46Z UI test run started: https://github.com/manaflow-ai/cmux/actions/runs/36414601363
  11:21:51Z UI test run: build on <mac> for 5m36s, compiling the app and UI tests for 29s.
  11:27:56Z UI test run: build on <mac> for 11m41s, testing 1 selected class for 12s; compiled in 5m56s.
  11:31:00Z UI test run finished (build success, test skipped); waiting for its verdict.
  11:32:00Z UI tests passed
  

  Of that job's 32 minutes, 12 were compile admission queued for an owned Mac and 5 compiling it.
• Contracts now agree across both kinds of Mac. Run 36414601363 (a Mac without SDKROOT) and run 36422710055 (a Mac that exports it) sealed the same product key, ce56a748. Both found admission's product by contract.
• Adopted, no re-upload. Run 36421840320 moved to the root where admission compiled and adopted the product: reuse 84 s, no compile, package 23 s, and Upload the compiled test product after the tests skipped. Its selector matched no test at that commit (my selector choice), so the step failed; the product path is what this run shows. Before, an adopting run spent p50 162 s uploading after its tests.
• Remaining cause, not fixed here. Runs 36414601363 and 36422710055 still compiled (356 s and 201 s): the product was compiled at canonical root 1, and on the test Mac root 1 stayed busy past the 120 s switch wait (root_unavailable). That is the next fix.

Testing

• python3 tests/test_reuse_app_host_products.py (118 OK, including the new SDKROOT case), tests/test_reuse_release_product.py, tests/test_e2e_sibling_build.py, tests/test_ci_ui_tests_dispatch.py (new ProgressTests), tests/test_ci_e2e_compilation_cache.py (the new late-upload condition passes; E2ECapturePreflight times out locally on main too).
• actionlint and a YAML parse of ci.yml and test-e2e.yml.
• In CI: a temporary commit touching HelpMenuUITests ran this PR's ui-tests lane (the measurements above). It has been dropped.

Note for #15327

Whichever of this and #15327 lands second: await_verdict there follows a replacement dispatch run (watched). Recreate Progress, or reset progress.e2e, each time watched changes, and call progress.report() inside that loop's check().

Changelog

none

🤖 Generated with Claude Code

[coderabbitai]

Warning

Review limit reached

Next included review available in 17 minutes.

Check out review usage here.

View limit details

Limit details: You’ve used all 10 included reviews currently available.

You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository.

Learn how review limits work.

Review configuration:

⚙️ Run configuration

Configuration used: Repository: manaflow-ai/cmux/.coderabbit.yaml

Review profile: ASSERTIVE

Plan: Advanced

Run ID: 98998ee8-4d98-4b37-8874-6f16a8063741

📥 Commits

Reviewing files that changed from the base of the PR and between 56ec600 and 0ef7309.

📒 Files selected for processing (11)

• .github/workflows/ci.yml
• .github/workflows/test-e2e.yml
• scripts/ci/app_host_test_rerun.py
• scripts/ci/e2e_sibling_build.py
• scripts/ci/reuse_app_host_products.py
• scripts/ci/ui_tests_dispatch.py
• tests/test_app_host_test_rerun.py
• tests/test_ci_e2e_compilation_cache.py
• tests/test_ci_ui_tests_dispatch.py
• tests/test_e2e_sibling_build.py
• tests/test_reuse_app_host_products.py

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}

[github-actions]

All contributors have signed the CLA ✍️ ✅
_{Posted by the CLA Assistant Lite bot.}

[github-actions]

Dogfood build of 0ef73091006730f03f656212213306aa79de5f41

cmux DEV pr-15331-0ef73091.app

The link opens this exact commit in the cmux dev menu bar app. The build starts on each push and the page waits until it is ready; a newer push replaces it. It signs in against production, so Cloud or backend changes still need a tagged build with a development backend.

[github-actions]

CI failure attribution

CI passes on 0ef7309100 (run 36428038892 attempt 1).

Written by scripts/ci/classify_failures.py (ci-failure-attribution.yml); signatures are its SIGNATURES table. A machine verdict is the runner's fault, not this PR's.

[cursor]

Bugbot is paused — on-demand spend limit reached

Bugbot uses usage-based billing for this team and has hit its on-demand spend limit.

A team admin can raise the spend limit in the Cursor dashboard, or wait for the next billing cycle to continue.

[github-actions]

Merge receipt for 0ef7309100: every check was green at merge (22 verified; 21 skipped by policy). Full suite runs on main after merge.