Repository navigation
ci: split tests/test-execution.toml into one registry file per area - #36348
Merged
Merged
Conversation
tests/test-execution.toml had 186 commits in 14 days, and every pull request that touched it conflicted with the next. It is now tests/test-execution.d/, one file per area (the test name's first word: ci.toml, nightly.toml, ..., other.toml for the rest), each sorted by path. Every reader loads the same merged view: the old single file while present, then each fragment by name. Leo, 2026-11-05: "i meant can you do all of htem" Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…fragment Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Agent-Session: https://cmux.com/deeplink/agent?id=aid-a345f4d82727e281eabe
…test-registry # Conflicts: # tests/test-execution.toml
Contributor
|
cmux status (one comment, edited in place; each line is updated by the job that owns it) Review App of this head (app.zip, kept 1 day)
Shipped in:
|
Review follow-ups at adca52a: the change-area detector compared the base's and head's native registry entries as ordered lists, so the split's re-ordering (by area, then path) read as a native change. It compares by path now. `test_execution_registry.py fix` kept one of two entries for the same path and dropped the other's fields; it refuses a duplicate now. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Agent-Session: https://cmux.com/deeplink/agent?id=aid-a345f4d82727e281eabe
…registry Rebuilt, not hand-resolved: tests/test-execution.toml taken from origin/feat-cmux-next, the fragments deleted, then python3 scripts/ci/test_execution_registry.py split. Agent-Session: https://cmux.com/deeplink/agent?id=aid-a345f4d82727e281eabe
…the registry The car guards listed every tests/test-execution.d/*.toml as unmapped: no test names a fragment by its own path. focused_tests.py now maps a fragment to the tests that read the registry (they name tests/test-execution.toml or tests/test-execution.d), as the single file was mapped. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Agent-Session: https://cmux.com/deeplink/agent?id=aid-a345f4d82727e281eabe
teamleaderleo
temporarily deployed
to
ci-routing-agent
November 5, 2026 09:47 — with
GitHub Actions
Inactive
Collaborator
Author
|
Review (codex gpt-6-astra high): SHIP at 67c0c52 router-review of 42abbd2..67c0c52 (gpt-6.1-sol gave no reply in 1491 s; gpt-6-astra at high gave the verdict). Visual verdict: no images supplied (range review); no UI in this diff.
Follow-ups:
|
This was referenced Nov 5, 2026
teamleaderleo
added a commit
that referenced
this pull request
Nov 5, 2026
…gain (#36451) test_registry_change_is_linux_only gave up on any registry holding a serial entry or a linux-guard entry with requirements, and the registry has both, so every registry edit (every new test) queued the macOS and Release jobs. It accepts serial now (compared as part of a native entry) and counts a linux-guard entry with requirements as native, instead of giving up on the whole registry. Already so with the single tests/test-execution.toml before #36348; the review of #36348 at 67c0c52 found it (F1). Leo, 2026-11-05: "i meant can you do all of htem" Agent-Session: https://cmux.com/deeplink/agent?id=aid-a345f4d82727e281eabe Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
This was referenced Nov 5, 2026
This branch was previously deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
tests/test-execution.tomlhad 186 commits in 14 days, and nearly every pull request that added a test conflicted with the next one, because every new entry went in at the top of the same file. The registry is nowtests/test-execution.d/, one file per area. A new test's entry goes only in its area's file, at its sorted place, so pull requests in different areas never touch the same file, and two in the same area insert at different lines.Leo, 2026-11-05, on the hot-files list (
schemas/defaults/cmux-next.jsonandtests/test-execution.toml): "i meant can you do all of htem"How it's split
tests/test_<word>_….pygoes in<word>.tomlwhen that file exists, otherwise inother.toml. Nothing lists the areas, so there's no shared index everyone appends to. The split made an area file for each first word with at least 5 tests:ci,cmux,nightly,ios,issue,shell,pr,web,release,swift,check,tui,gh,cloud,app,dev,ghostty,lint, plusother.[[test]]entries. The registry'sversion = 1is implied.scripts/ci/test_execution_registry.py(load_tree,load_tree_atfor a commit).detect_ci_change_areas.pyandfocused_tests.pyread the fragments inline, because their trusted copies run without that module.python3 scripts/ci/test_execution_registry.py fixrepairs both.validate_test_execution_registry.py --writeand the pre-commit hook already put new entries in the right place.Readers changed
run_python_test_lane.py,focused_tests.py(a fragment maps to the tests that read the registry, so the car guards don't list fragments as unmapped),validate_test_execution_registry.py(reads, base reads through git,--write, messages),cmux_next_route.py,detect_ci_change_areas.py(head registry,--test-registry-basenow also takes a directory, native entries compared by path, not by order; referrer checks),ci.yml(extracts the base's registry files for that flag),delta_since_green.py,workflow_guard_groups.py,post_edit_checks.py,guard_attribution.py(message),scripts/git-hooks/pre-commit, plus the docs that name the file (CLAUDE.md, the dev-workflow skill,docs/ci/derived-not-declared.md). No Swift reads the registry.Rebasing an open PR that edits the old file
Merging or rebasing onto this leaves a modify/delete conflict on
tests/test-execution.toml. Run:It reads what your branch changed in the old file (against the commit your branch started from), puts added and changed entries in their area files, removes deleted ones, deletes the old file and stages the result. Then
git merge --continue/git rebase --continue.This PR's own merges of the base aren't hand-resolved either. Each one is rebuilt: take the base's
tests/test-execution.toml, deletetests/test-execution.d/, runpython3 scripts/ci/test_execution_registry.py split, commit. The split is deterministic, so the result is the base's registry, entry for entry. 4 open teammate PRs edit the old file (#27910, #23811, #22857, #18426), out of 203 open teammate PRs into feat-cmux-next.The defaults JSON stays one file
schemas/defaults/cmux-next.jsonis a generated snapshot (defaults_guard.py snapshot). The base autofix writes it after each merge, and PRs aren't supposed to commit it (#21946): 164 of its 205 commits in 14 days are the bot's. It's generated from the Swift sources and the settings and action exports, and those are already spread across per-area files. Splitting the output wouldn't take conflicts off any pull request, so this PR leaves it alone.Verification
RegistrySplitTests.test_the_real_pre_split_registry_splits_into_the_same_entriesre-runs this from history.run_python_test_lane.py --listfor each lane, and each--shard K/3, the validator,focused_tests.lanes,cmux_next_route.linux_guard_testsand the detector's inert-lane answer for every test. Order within a lane is now by area and path instead of insertion order. The lane members and shards are the same.tests/test_ci_test_execution_registry.py(split round-trip, tomllib merged text, placement and sort warnings plusfix,--writeplacement, reading from git in both layouts,absorbon a real merge and a real rebase,fixrefusing a duplicate instead of dropping it),tests/test_ci_change_areas.py(a fragment edit routes through the normal and trusted router, and re-ordered entries are no native change),tests/test_install_git_hooks.py(the hook registers into a fragment),tests/test_car_guards.py(a fragment is mapped) andtests/test_cmux_next_route.py.fixrefuses a duplicate path instead of keeping one entry).Observed / Not run
Ran on the lane host, at the base and at this head: test_ci_test_execution_registry, test_ci_change_areas, test_cmux_next_route, test_cmux_next_route_replay, test_install_git_hooks, test_ci_delta_since_green, test_feat_ci_guard_group, test_car_guards, test_ci_parallel_artifact_transport, test_ci_workflow_guards_are_wired, test_risk_tier, test_web_validation, test_cmux_next_prepush_gate, test_preflight_trust, test_post_edit_checks and test_pr_lane_coverage. All pass except
test_validator_exit_code_tracks_added_tests_in_a_git_fixture. On this host it fails identically at the base (42abbd2) and at this head (67c0c52), before any registry code runs: the fixture'sgit push -u origin mainto a local bare repo is refused by the host's brokered git wrapper. Base:Ran 40 tests ... FAILED (errors=1), head:Ran 52 tests ... FAILED (errors=1), both with onlyERROR: test_validator_exit_code_tracks_added_tests_in_a_git_fixture ... CalledProcessError: Command '['git', ..., 'push', '-u', 'origin', 'main']' returned non-zero exit status 2.CI runs it. The car's linux guards (car_guards.py run --where linux) at 67c0c52 against 42abbd2: 40 pass, nothing unmapped, and two host-only base reds (that test, andtest_web_lint_prepush.py, whose web build doesn't run here). Not run: theci.ymlrouting step itself, which runs on this PR's CI.Changelog
none
Agent session: agent session
🤖 Generated with Claude Code