Skip to content

ci: derive whether a scripts/ci helper reaches a routed lane - #14063

Merged
teamleaderleo merged 2 commits into
mainfrom
ci/derive-ci-helper-routing
Sep 24, 2026
Merged

teamleaderleo merged 2 commits into
mainfrom
ci/derive-ci-helper-routing

Conversation

@teamleaderleo

@teamleaderleo teamleaderleo commented Sep 24, 2026 •

Copy link
Copy Markdown
Collaborator

A pull request that edits a scripts/ci/*.py helper the router has no list entry for runs the macOS, web and Release lanes, even when no job in ci.yml's tree can execute that helper. #14051 queued macOS compile admission for a script only the dispatch-only E2E workflow runs. Hand-maintained lists (CI_CONTROL_PLANE_ONLY, CI_PUBLISHING_ONLY, #14061's CI_DISPATCH_ONLY) fix one file at a time, and every new helper starts on the expensive side.

The router now derives the answer. Areas only gate ci.yml and the local workflows it calls, so the question is whether any job in that tree can execute the helper. ci_helper_reaches_routed_lane walks back from the helper through every tracked file that names it as a whole name (git grep, then a boundary match, so tests/test_foo.py does not count as naming foo):

  • A script or composite action that names it is followed in turn.
  • A workflow outside ci.yml's call tree (dispatch, schedule, nightly, release) is a dead end.
  • A test that only Linux guard jobs run is a dead end, using the existing is_guard_only_test rule.
  • Documentation is ignored.
  • Anything else answers "reachable" and keeps today's fail-open routing: a routed workflow, app sources, the Xcode project, a helper absent from the tree, or a failed git grep.

It runs inside the trusted base router and reads the pull request's tree only as data, through the existing CMUX_CI_HEAD_TEST_REFERENCE_ROOT. Listed helpers keep their explicit routing.

e2e_warm_derived_data.py also leaves PRODUCT_CI_INPUTS, which #14016's review added. Keeping it there made every edit change the compile-admission identity, and routing followed the literal name into the macOS lane. Instead, the E2E identity (e2e_recipe) now hashes every scripts/ci/ file the E2E build job names. An edit still invalidates E2E products, and nothing else.

Review follow-up: calls in ci.yml may be quoted, and the routed call tree is the union of the base's and the head's, so a pull request cannot unroute a workflow by editing ci.yml.

Effect. Replaying the 600 most recent pull requests merged since 2026-09-16 through main's router and this one, against today's tree: 104 touched an unowned helper. 11 of those would have skipped the macOS lane (#13972, #13970, #13901, #13900, #13899, #13875, #13867, #13695, #13694, #13680, #12934), and 4 more would have skipped only web or Release. All 11 edit dispatch, nightly, release or offline tooling. Of the 46 unowned helpers today, 14 now derive as unrouted and 32 still fail open. test_execution_registry.py, cache_restore_receipt.py, xcodebuild_noninteractive.py and run_python_test_lane.py stay routed.

Tradeoffs. Name matching over-approximates: a comment or a shared stem keeps a helper routed, which only costs CI. An indirect execution path that never names the helper (a computed path, for example) would be missed; the walk covers scripts, composite actions, workflows and tests, which is how helpers are invoked here today. A new helper nothing names yet still fails open.

Validation. New router tests on synthetic repos cover: a dispatch-only run; a routed workflow running the helper directly, through a script and through a composite action; a product-source referrer; a test on a native lane; an unreferenced helper. They also assert that e2e_warm_derived_data.py is unrouted and test_execution_registry.py is routed in the real tree. All 139 linux-guard tests pass locally. Because this edits the router, it runs every area once. Supersedes #14061.

— Dulcinea g1 🎐

🤖 Generated with Claude Code

An unknown scripts/ci helper used to fail open to every area. The router now
walks back from the helper through every tracked file that names it: scripts
and composite actions are followed, workflows outside ci.yml's call tree and
Linux-only guard tests are dead ends, and anything else (a routed workflow,
product sources, the Xcode project, an absent file, a read failure) keeps the
fail-open answer. e2e_warm_derived_data.py leaves product identity: it only
warms the dispatch-only E2E lane and never changes ci-macos products.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@cursor

cursor Bot commented Sep 24, 2026

Copy link
Copy Markdown

Bugbot is paused — on-demand spend limit reached

Bugbot uses usage-based billing for this team and has hit its on-demand spend limit.

A team admin can raise the spend limit in the Cursor dashboard, or wait for the next billing cycle to continue.

@coderabbitai

coderabbitai Bot commented Sep 24, 2026 •

Copy link
Copy Markdown

Warning

Review limit reached

Next included review available in 3 minutes.

Check out review usage here.

View limit details

Limit details: You’ve used all 10 included reviews currently available.

You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository.

Learn how review limits work.

Review configuration:

⚙️ Run configuration

Configuration used: Repository: manaflow-ai/cmux/.coderabbit.yaml

Review profile: ASSERTIVE

Plan: Advanced

Run ID: bc771dc6-3755-4254-ba80-0bcfce3b5f63

📥 Commits

Reviewing files that changed from the base of the PR and between 1ba90a1 and a3de373.

📒 Files selected for processing (4)
  • scripts/ci/detect_ci_change_areas.py
  • scripts/ci/product_input_identity.py
  • tests/test_ci_change_areas.py
  • tests/test_reuse_app_host_products.py

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@github-actions

Copy link
Copy Markdown
Contributor

All contributors have signed the CLA ✍️ ✅
Posted by the CLA Assistant Lite bot.

Review of the derived helper routing found the call tree was read only from
the pull request's ci.yml, so a quoted or removed `uses:` line could unroute
a workflow and its helpers. Calls may now be quoted, and the routed set is
the union of the base's call tree and the head's.

E2E products no longer take e2e_warm_derived_data.py through
PRODUCT_CI_INPUTS. Instead the E2E identity hashes every scripts/ci file its
build job names, so an edit to one still invalidates E2E products without
touching the compile-admission identity or naming the helper where routing
would follow it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@teamleaderleo

Copy link
Copy Markdown
Collaborator Author

Independent review (a separate agent in its own scratch worktree). No blockers. For each of the 14 helpers that now derive as unrouted, it grepped every referrer. Each one is run only from nightly, release, test-e2e, test-ios, persistent-macos-compile, Linux guard tests or docs.

# Finding Disposition in HEAD
1 Should-fix. _LOCAL_WORKFLOW_CALL_RE missed quoted uses: lines, and the call tree came only from the PR's ci.yml. A PR could quote or drop a call and unroute that workflow's helpers. Fixed. Quoted forms match. The routed set is the union of the base's call tree (the trusted router's working directory) and the head's, so a PR can add routed workflows but not remove them. Test: test_a_pull_request_cannot_unroute_a_workflow_by_editing_ci_yml.
2 Nit. A guard-only test is a dead end even if a macOS-run test imports it. Kept. No test in today's tree does this, and the existing is_guard_only_test rule already has the same limit.
3 Dropping the helper from PRODUCT_CI_INPUTS let a bad helper edit seal stale E2E products under an unchanged key. Fixed. e2e_recipe now also hashes the blob of every scripts/ci/ file the E2E build job names. An edit invalidates E2E products only, and the compile-admission identity is unchanged. Nothing in the identity code names the helper, so routing does not follow it into the macOS lane. Test: test_e2e_identity_binds_the_helpers_its_build_job_runs.

Checked and sound: composite-action tokens, the workload-profile chain, hyphenated names, the transitive call-tree walk, read-only git grep -F -e on the PR tree, and cost (about 0.2 s per helper). All 139 linux-guard tests pass locally on HEAD.

— Dulcinea g1 🎐

@cursor

cursor Bot commented Sep 24, 2026

Copy link
Copy Markdown

Bugbot is paused — on-demand spend limit reached

Bugbot uses usage-based billing for this team and has hit its on-demand spend limit.

A team admin can raise the spend limit in the Cursor dashboard, or wait for the next billing cycle to continue.

@teamleaderleo
teamleaderleo enabled auto-merge (squash) September 24, 2026 00:23
@teamleaderleo
teamleaderleo merged commit 99dc5a3 into main Sep 24, 2026
46 of 47 checks passed
rustybret pushed a commit to rustybret/bmux that referenced this pull request Sep 24, 2026
a3b7014 ci: skip the CLI lane for project.pbxproj edits outside the CLI targets (manaflow-ai#14071)
789edd3 ci: bootstrap the app-host known-failure catalog from main's census (manaflow-ai#14074)
2c2314e ci: route focused tests to one front door, and half of them to 12 vCPU (manaflow-ai#14067)
d7409fc ci(e2e): read adopted DerivedData over parallel ranges (manaflow-ai#14051)
99dc5a3 ci: derive whether a scripts/ci helper reaches a routed lane (manaflow-ai#14063)
cf8b073 ci: stop routing the Claude wrapper lane on test-registry edits (manaflow-ai#14065)

# Conflicts:
#	.github/workflows/ci.yml
#	.github/workflows/perf-activation.yml
#	.github/workflows/test-e2e.yml
#	.github/workflows/test-macos-suite.yml
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant