Repository navigation
ci: advisory fleet test-compile check for the test targets a change reaches - #23423
Conversation
…can break (red) fleet/compile-check compiles only the app executable, so #23360 (a throwing call in #expect without try, CmuxNextActionsTests) and #23163 (a FakeClient that missed #21258's requirement, CmuxNextAppTests) passed it and broke feat-cmux-next's test build. This test pins the plan the check must compile and the new package-test-lane.sh compile-tests argument check; neither exists yet. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Agent-Session: https://cmux.com/deeplink/agent?harness=claude&session=01a15bf1-fdac-78a0-ac2e-a3aef81402c7&host=cosmic-gray
…ches fleet/compile-check compiled only the app executable, which links no test target, and merge_compile_guard accepts it in place of a queued Release compile. #23360 (CmuxNextActionsTests) and #23163 (CmuxNextAppTests) passed it while every Mac job that builds tests sat queued, and broke the base's test build. The workflow now reads the head's merge base with feat-cmux-next from the compare API and runs `package-test-lane.sh compile-tests MERGE_BASE` on a fleet Mac beside the Linux app compile; success needs both. The plan (scripts/ci/compile_test_plan.py) is the CmuxNext test targets that depend on a changed target in the existing SwiftPM target graph (the routing cmux_next_route.py gives the swift test tier), plus the macOS package lane's packages whose sources or path dependencies changed. Each CmuxNext test target builds with `swift build --target`, so nothing links or runs. The Linux pool cannot do it: its SDK has no Testing module. A head whose package-test-lane.sh predates the phase gets no verdict (error), with "merge feat-cmux-next and dispatch again". Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Agent-Session: https://cmux.com/deeplink/agent?harness=claude&session=01a15bf1-fdac-78a0-ac2e-a3aef81402c7&host=cosmic-gray
|
Owned by ci-speed on cosmic-gray · session |
|
cmux status (one comment, edited in place; each line is updated by the job that owns it) Review App of this head (app.zip, kept 1 day)
web and size checks: green |
|
Found 7 test failures on Blacksmith runners: Failures
|
|
Bead |
… is the app compile again coord-deputy: an ~11 min Mac queue on every PR's merge gate costs too much. fleet-compile-check.yml and merge_compile_guard.py are back to feat-cmux-next's (the Linux app compile, same status semantics). fleet-test-compile.yml runs on pull requests into feat-cmux-next that touch Packages/ or vendor/: a Linux job plans with compile_test_plan.py over the merge commit's diff; an empty plan passes there in seconds and queues no Mac; a plan runs `package-test-lane.sh compile-tests BASE` as a light fleet step, and the check `fleet/test-compile (advisory)` goes red only when a planned test target does not compile. It is a bridge until the Linux pool image carries cmuxterm-hq #2750 (Testing search paths). Tests: the guard ignores the advisory check, red or pending, and it never stands in for the Release compile; CI and app-host changes plan nothing. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Agent-Session: https://cmux.com/deeplink/agent?harness=claude&session=01a15bf1-fdac-78a0-ac2e-a3aef81402c7&host=cosmic-gray
|
|
…llback per package; advisory verdicts Red on c150a5c: a target failure followed by a later package's setup failure exits before the failure marker, so the check only warns; a refused --target falls back to --build-tests once per target; the job's 45 min timeout is shorter than the client's queue plus run timeouts, so a slow fleet ends red; exit 128 after the marker maps to a warning. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Agent-Session: https://cmux.com/deeplink/agent?harness=claude&session=01a15bf1-fdac-78a0-ac2e-a3aef81402c7&host=cosmic-gray
…er does - package-test-lane.sh compile-tests prints the failure marker as soon as a target fails, so a later package's setup failure cannot hide it. - A refused --target falls back to --build-tests once per package, in place of its remaining targets. - fleet-test-compile.yml: red on the marker whatever the exit; the job timeout (55 min) outlasts the client's queue plus run timeouts (45 min), so a timeout is a warning, never red. - cmux-next.yml runs tests/test_ci_compile_test_plan.py with the merge guard tests. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Agent-Session: https://cmux.com/deeplink/agent?harness=claude&session=01a15bf1-fdac-78a0-ac2e-a3aef81402c7&host=cosmic-gray
|
Review (independent subagent, correctness first): SHIP at b96188a |
…age test job package-test-lane.sh compile-tests runs it, so test_ci_select_package_tests.py requires it in GLOBAL_INPUTS (red on b96188a: "scripts the package test job runs are not global inputs"). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Agent-Session: https://cmux.com/deeplink/agent?harness=claude&session=01a15bf1-fdac-78a0-ac2e-a3aef81402c7&host=cosmic-gray
|
Review (independent subagent, correctness first): SHIP at 3a2851b. The SHIP at b96188a still stands: the only change since then adds scripts/ci/compile_test_plan.py to GLOBAL_INPUTS in select_package_tests.py, which fixes this PR's own 'Validate package test selection' red. The defaults-snapshot check-menu-actions failure (FileExplorerSectionView.swift) is red on base 68a3517 too. |
|
Shipped in:
|
…id, reload shim test, fleet-test-compile runners) (#23553) * fix(cmux-next media): a display step's shot passes the app's pid to the screen route Screen.use_display takes the app's pid since #22046 (it finds the window server id by owner pid), and run_tour passes it; display_capture still called use_display() with no argument, so every `display` tour step died with a TypeError. tests/test_cmux_next_tour.py's display-step cases were red on feat-cmux-next (preflight guard group). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Agent-Session: https://cmux.com/deeplink/agent?harness=claude&session=01a15c31-89b3-7d62-8e8c-b116a23ae693&host=cosmic-gray * fix(ci): the reload DerivedData test loads the shim writer from scripts/lib/cmux-dev-shim-install reload.sh now sources write_dev_cli_shim and reload_socket_is_live from scripts/lib/cmux-dev-shim-install (the one source of the managed shim), so the test's awk extraction from reload.sh found nothing and failed with "write_dev_cli_shim not found in reload.sh" on feat-cmux-next (preflight guard group). The test sources the library as reload.sh does, and checks reload.sh still sources it. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Agent-Session: https://cmux.com/deeplink/agent?harness=claude&session=01a15c31-89b3-7d62-8e8c-b116a23ae693&host=cosmic-gray * fix(ci): fleet-test-compile.yml runs on Blacksmith, not a GitHub-hosted label #23423's plan job and its empty-plan compile leg selected ubuntu-24.04, which tests/test_ci_self_hosted_guard.sh refuses (a GitHub billing block must not stop CI): "Validate macOS runner guards" was red on feat-cmux-next (preflight guard group). Both take the standard selector: the fork branch, then vars.LINUX_RUNNER with the Blacksmith fallback; a needed plan still takes vars.CI_COMPILE_CHECK_GATEWAY. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Agent-Session: https://cmux.com/deeplink/agent?harness=claude&session=01a15c31-89b3-7d62-8e8c-b116a23ae693&host=cosmic-gray * fix(ci): fleet-test-compile.yml sends fork pull requests to Blacksmith before any runner variable The previous commit put vars.LINUX_RUNNER (and the compile gateway) right after the owner branch, so tests/test_ci_fork_runner_routing.py flagged both lines: a fork pull request could reach an owned runner label. Both selectors now use the standard form (agent-brands.yml, backend.yml): the fork pull-request branch first, then the gateway and LINUX_RUNNER. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Agent-Session: https://cmux.com/deeplink/agent?harness=claude&session=01a15c31-89b3-7d62-8e8c-b116a23ae693&host=cosmic-gray --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Summary
fleet/compile-checkcompiles only the app executable, which links no test target, andmerge_compile_guard.pyaccepts its pass in place of a queued Release compile. Two PRs passed it today and then broke feat-cmux-next's test build: #23360 (a throwing call inside#expectwithouttry,CmuxNextActionsTests) and #23163 (aFakeClientmissing the requirement #21258 added,CmuxNextAppTests). On #23360 every Mac job that builds tests was still queued when it merged.This adds an advisory check as a bridge, and leaves the merge gate alone:
fleet/compile-checkis unchanged: still the Linux app compile, same timing, same github-actions[bot] status.fleet-compile-check.ymlandmerge_compile_guard.pyare byte-identical to feat-cmux-next.fleet-test-compile.ymlruns on pull requests into feat-cmux-next that touchPackages/orvendor/. A Linux job plans withscripts/ci/compile_test_plan.pyover the merge commit's diff: the CmuxNext test targets that depend on a changed target in the existing SwiftPM target graph (cmux_next_route.route, the swift test tier's routing), every one for a manifest or graph change, plus the macOS package lane's packages whose sources or path dependencies changed. An empty plan passes on that Linux runner in seconds and queues no Mac. A plan runspackage-test-lane.sh compile-tests BASEas a light fleet step, which builds each planned CmuxNext test target withswift build --target(no link, no run) and a lane package with--build-tests. The check,fleet/test-compile (advisory), goes red only when a planned test target fails to compile. The lane prints its failure marker as soon as a target fails, so a later package's setup failure can't hide it. Anything else (queue or run timeout, setup, the client) is a warning, and the job's 55-minute timeout outlasts the client's 20-minute queue plus 25-minute run timeouts, so a slow fleet never reads red. If SwiftPM refuses--target, the package falls back to one--build-testsin place of its remaining targets.fleet/compile-check;gh-merge-greenis main-only and this workflow runs only on feat PRs.It's a bridge until the Linux pool compiles test targets. The pool's darwin SDK can't find
Testingtoday (job1382fbdb06c0fc7c023643d3at df5e2eb:no such module 'Testing'); cmuxterm-hq #2750 adds Xcode's platform search paths and reaches the pool with the next image bake. The per-target pool argument is manaflow-ai/cmuxterm-hq#2759.Testing
Red/green on a fleet Mac with the build the new phase runs (the reviewed
package-test-lane.sh suitephase, which runsswift build --build-testsover CmuxNext before one suite):4f867f5e57c2d943ac1f4e9743d42cc89730e683PaletteMenuVocabularyTests.swift:74:106and:79:101, "call can throw, but it is not marked with 'try'"ed72e6e6f72d215befe9bb9b(aws-m4pro-8)df5e2ebb1c03db048843083530e86feeda2c1b2aBuild complete! (89.11s), then CmuxNextActionsTests (136 tests) passed54d316aebe3d339a9a2126eb(aws-m4pro-7)cmux-ci run --class light --script scripts/ci/package-test-lane.sh --ref <sha> --arg=suite --arg=Packages/macOS/CmuxNext --arg=CmuxNextActionsTests;cmux-ci log <step>on the tailnet prints each log. Each ran about 1.5 to 2 minutes after 11 minutes in the light-class queue, which is why this is advisory and off the merge gate.tests/test_ci_compile_test_plan.py(linux-guard) was committed red first (ec4b51c) and passes: #23360's diff selectsCmuxNextActionsTestsand its reverse dependents and nothing that doesn't reachCmuxNextActions; a newCmuxNextAppTestsfile selects only that target; the manifest selects every target; aCMUXAuthCorechange selects it, its lane dependents and the CmuxNext targets that use it; iOS-only, docs, CI and app-host changes plan nothing; the phase refuses a bad merge base. The same file runs thecompile-testsphase with a fakeswift(committed red first in a9201f8): a target that doesn't compile exits 1 with the marker; an earlier target failure stays red when a later package's setup fails; a setup failure alone has no marker; a refused--targetfalls back to--build-testsonce per package; an empty plan builds nothing. It also runs the workflow step with a fake client to check how exits map to red, warning or green, and that the job timeout outlasts the client's. cmux-next.yml now runs it with the merge guard tests.tests/test_merge_compile_guard.pyadds that a red or pending advisory check never holds a merge or stands in for the Release compile, and that the advisory workflow is not the fleet check the guard trusts. Passing on Linux (huge-blue): those two,test_ci_cmux_ci_client_guarded,test_ci_workflow_guards_are_wired,test_supplychain_security,test_runner_label_policy,test_ci_pr_runner_pool,test_ci_fork_runner_routing,test_ci_merge_queue_required_checks,test_ci_workflow_path_filter_parity,test_ci_workflow_run_sources, the reusable-workflow and cache guards,package-test-lane-suite-args.test.sh;actionlint;prepush-gate.py.Observed / Not run
no such module 'Testing'), the tests listed above,compile_test_plan.pyon cmux-next palette: titles end in "…" exactly when the action asks first, outside Phone #23360's diff (12 CmuxNext test targets includingCmuxNextActionsTestsandCmuxNextAppTests).compile-testsphase andfleet-test-compile.ymlon the fleet. A changed allowlisted script runs on the fleet only after it lands (hqstep_reviewed.go), so the first real run is the first feat PR touchingPackages/after merge. Unverified there:swift build --target <test target>; if SwiftPM refuses it, the phase falls back to--build-testsfor that package.... (advisory)) lands. Expected red here: this PR's own swift-package-tests fleet step is refused untilpackage-test-lane.shlands, as for every allowlisted-script change.Changelog
none
Agent session: cosmic-gray
🤖 Generated with Claude Code