Skip to content

ci: move the 20 legacy tests with a Linux execution path onto linux-guard - #13726

Merged
teamleaderleo merged 1 commit into
mainfrom
promote-legacy-lanes-v2
Sep 22, 2026
Merged

teamleaderleo merged 1 commit into
mainfrom
promote-legacy-lanes-v2

Conversation

@teamleaderleo

@teamleaderleo teamleaderleo commented Sep 22, 2026 •

Copy link
Copy Markdown
Collaborator

tests/test-execution.toml documents its own legacy lane as a holding pen:

legacy is migration inventory only: existing files may remain legacy while we model their current execution path, but newly added tests may not enter as legacy.

Twenty of the 81 legacy entries already have that path modelled. A workflow runs them, on a job whose runs-on resolves to Linux. They can stop being inventory.

How each one was resolved

Not by name-matching. For every workflow in .github/workflows/, each job was parsed, the test path matched inside the job body, and that job's runs-on read:

workflow :: job tests
ci-artifact-transport.yml :: transport test_app_host_layer_transport, test_app_host_layered_products, test_ci_r2_artifact, test_ci_r2_canary, test_node_product_cache
ci-guards.yml :: workflow-guard-tests test_check_ghostty_zig_workflows, test_ci_release_helper_archs, test_triage_radar, test_tui_focused_filter_guard
remote-daemon.yml :: remote-daemon-tests test_remote_daemon_release_assets, test_remote_daemon_release_verification
cmux-skill-contract.yml :: browser-skill test_cmux_browser_skill, test_cmux_settings_supported_paths
ci-artifact-canary.yml :: canary test_ci_r2_canary
ci-cache-receipts.yml :: receipt-contract test_ci_cache_restore_receipt
ci.yml :: static-preflight test_normalize_pbxproj
cmux-tui-sdks.yml :: contract test_tui_publish_workflow_security
indexnow-tests.yml :: notification-tests test_indexnow_notification
repair-nightly-appcast-content-types.yml :: test test_repair_nightly_appcast_content_types
terminal-hang-diagnostics.yml :: release-gate test_terminal_hang_release_gate
web-validation.yml :: changes test_web_validation

Four of these are guards that #13696 wired earlier today — test_check_ghostty_zig_workflows, test_ci_release_helper_archs, test_triage_radar, test_tui_focused_filter_guard. Their entries said legacy because at the time it was true; it no longer is.

What deliberately stays legacy

Eleven other legacy entries are also run by a workflow, and all eleven stay. Every one executes on a macOS job — MACOS_RUNNER_PR, MACOS_RUNNER_15, MACOS_RUNNER_26 — and the macOS lanes in this manifest (macos-shell, macos-cli-no-socket, …) are run_python_test_lane.py lanes that do not run these tests. Marking them linux-guard would make the manifest assert something false about where they execute, which is worse than leaving them in the pen:

test_cli_broken_pipe_writes, test_cli_config_doctor, test_cli_contract_help,
test_cli_creation_initial_command, test_cli_glaeda_execution,
test_cli_version_memory_guard, test_cli_vm_create_name,
test_cloud_task_local_lifecycle, test_issue_11257_prompt_command_export,
test_plain_text_paste_worker, test_workspace_churn_up_arrow_lag

Giving them an honest lane means adding a macOS lane that matches how they are actually invoked — a real change to run_python_test_lane.py's model, not a lane rename, and not this pull request.

No entry resolved to a mix of Linux and macOS jobs, so the split needed no tie-breaking. The remaining 50 legacy entries resolve to no job at all, which is exactly what the lane is for.

Validation

Python test execution registry valid: 218 tests
(legacy=61, linux-guard=96, macos-cli-no-socket=48,
 macos-cli-no-socket-post-fish=11, macos-shell=1, macos-shell-fish=1)

legacy 81 → 61, linux-guard 76 → 96. Every changed line is a lane = value — 20 insertions, 20 deletions, nothing added, removed or reordered — and tomllib parses all 218 entries.

Stacking

Based directly on main. An earlier attempt (#13714) was stacked on #13710's branch; squash-merging that base orphaned the stack, and GitHub marked #13714 merged while none of its 20 lane changes reached main — verified by re-running the validator on main afterwards and finding legacy=81 with test_triage_radar still legacy. This is the same commit, cherry-picked onto main.

🤖 Generated with Claude Code


Summary by cubic

Moves 20 tests from the legacy lane to linux-guard in tests/test-execution.toml, because these tests already run on Linux jobs in existing workflows. The legacy lane is meant only for tests whose execution path hasn't been modeled yet, and these have been.

Eleven other tests that also run in workflows stay legacy because they run on macOS jobs that don't have a corresponding lane in this manifest; moving them would misrepresent where they execute.

Written for commit 4a1de47. Summary will update on new commits.

Review in cubic

…uard

The `legacy` lane is documented as migration inventory: "existing files may
remain legacy while we model their current execution path". Twenty of the 81
entries already have that path modelled -- a workflow runs them, on a job whose
runs-on resolves to Linux -- so they can stop being inventory.

Resolved per entry by walking every workflow's jobs, matching the test path
inside the job body, and reading that job's runs-on:

  ci-artifact-transport.yml :: transport
      test_app_host_layer_transport, test_app_host_layered_products,
      test_ci_r2_artifact, test_ci_r2_canary, test_node_product_cache
  ci-guards.yml :: workflow-guard-tests
      test_check_ghostty_zig_workflows, test_ci_release_helper_archs,
      test_triage_radar, test_tui_focused_filter_guard
  remote-daemon.yml :: remote-daemon-tests
      test_remote_daemon_release_assets,
      test_remote_daemon_release_verification
  cmux-skill-contract.yml :: browser-skill
      test_cmux_browser_skill, test_cmux_settings_supported_paths
  ci-artifact-canary.yml :: canary                test_ci_r2_canary
  ci-cache-receipts.yml :: receipt-contract       test_ci_cache_restore_receipt
  ci.yml :: static-preflight                      test_normalize_pbxproj
  cmux-tui-sdks.yml :: contract                   test_tui_publish_workflow_security
  indexnow-tests.yml :: notification-tests        test_indexnow_notification
  repair-nightly-appcast-content-types.yml :: test
      test_repair_nightly_appcast_content_types
  terminal-hang-diagnostics.yml :: release-gate   test_terminal_hang_release_gate
  web-validation.yml :: changes                   test_web_validation

Eleven other legacy entries are also run by a workflow and deliberately stay
legacy: every one of them executes on a macOS job (MACOS_RUNNER_PR,
MACOS_RUNNER_15, MACOS_RUNNER_26), and the macOS lanes in this manifest are
run_python_test_lane.py lanes that do not run these. Calling them linux-guard
would make the manifest assert something false. No entry resolved to a mix of
Linux and macOS jobs, so the split needed no tie-breaking.

The remaining 50 legacy entries resolve to no job at all, which is what the
lane is for.

legacy 81 -> 61, linux-guard 76 -> 96. Every changed line is a lane value;
nothing is added, removed or reordered.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@coderabbitai

coderabbitai Bot commented Sep 22, 2026

Copy link
Copy Markdown

Warning

Review limit reached

Next included review available in 26 minutes.

Check out review usage here.

View limit details

Limit details: You’ve used all 10 included reviews currently available.

You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository.

Learn how review limits work.

Review configuration:

⚙️ Run configuration

Configuration used: Repository: manaflow-ai/cmux/.coderabbit.yaml

Review profile: ASSERTIVE

Plan: Advanced

Run ID: df7ccef4-85ae-4c2b-bbff-e364ca4d63dd

📥 Commits

Reviewing files that changed from the base of the PR and between e0b52a8 and 4a1de47.

📒 Files selected for processing (1)
  • tests/test-execution.toml

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@github-actions

Copy link
Copy Markdown
Contributor

All contributors have signed the CLA ✍️ ✅
Posted by the CLA Assistant Lite bot.

@teamleaderleo
teamleaderleo merged commit 60e52da into main Sep 22, 2026
32 of 33 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant