ci: register the settings JSONC and test-wiring regressions by teamleaderleo · Pull Request #13739 · manaflow-ai/cmux

teamleaderleo · 2026-09-22T17:56:45Z

Problem

guards / workflow-guard-tests / preflight is red on main for every pull request:

Python test execution registry validation failed:
  - tests/test_cmux_settings_jsonc.py: test exists but has no execution registry entry
  - tests/test_sync_test_wiring.py: test exists but has no execution registry entry

Seen on #13724, #13727, #13730, #13733, #13735, #13737. The two tests landed today with #13218 and #13216 and never got registry entries. This is the same failure #13710 and #13731 fixed for other tests; the check runs on every event, so an unregistered test reddens main immediately.

Resulting behavior

Both tests are registered in tests/test-execution.toml on the linux-guard lane, in their sorted positions. tests/test_cmux_settings_jsonc.py runs from cmux-skill-contract.yml; tests/test_sync_test_wiring.py runs from tests/test_ci_pbxproj_test_wiring.sh, which ci.yml's Fast static checks invokes. The registry resolves a linux-guard entry by looking for the test's path in workflow text, so the pbxproj lint step now carries a comment naming the test it executes — the path is documented where the execution actually happens.

Validation and remaining gap

python3 scripts/ci/validate_test_execution_registry.py now passes: 223 tests (legacy=61, linux-guard=101, ...)
RUNNER_TEMP=... python3 tests/test_ci_change_areas.py passes
actionlint .github/workflows/ci.yml is clean
Remaining gap, as ci: register the R2 cache census in the test execution registry #13731 noted: the registry is hand-maintained beside the workflows, so the next test that lands without an entry reddens main again. Generating it from the workflows is still open.

🤖 Generated with Claude Code

^{Need help on this PR? Tag @codesmith-bot with what you need. Autofix is disabled.}

Summary by cubic

Fixes the guards / workflow-guard-tests / preflight check red on main by registering two tests that landed without execution registry entries.

Adds tests/test_cmux_settings_jsonc.py and tests/test_sync_test_wiring.py to tests/test-execution.toml on the linux-guard lane.
Documents in ci.yml that the pbxproj lint step also runs one of the registered tests, since the registry matches tests to workflow text.

^{Written for commit 2786e0b. Summary will update on new commits.}

Summary by CodeRabbit

Documentation
- Added clarification to the automated validation workflow, making test execution checks easier to understand and maintain.
Tests
- Registered two additional regression checks in the Linux validation lane.
- Improved coverage for configuration synchronization and settings-file validation.
- No changes to application functionality or end-user behavior.

tests/test_cmux_settings_jsonc.py (#13218) and tests/test_sync_test_wiring.py (#13216) landed without execution registry entries, so the unconditional "test exists but has no execution registry entry" check fails and guards / workflow-guard-tests / preflight is red on main for every pull request, whatever it touched. Both run on Linux: the settings regression from cmux-skill-contract.yml, the wiring regression from tests/test_ci_pbxproj_test_wiring.sh in ci.yml's Fast static checks. Both take the linux-guard lane. The registry decides whether a linux-guard test really runs by looking for its path in workflow text, so the pbxproj lint step now names the test it calls. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

coderabbitai · 2026-09-22T17:57:50Z

Navigate logical layers of code changes, visualize relationships, and explore their blast radius.

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: Repository: manaflow-ai/cmux/.coderabbit.yaml

Review profile: ASSERTIVE

Plan: Advanced

Run ID: 2865db18-920f-44c3-a622-9c30860d05ce

📥 Commits

Reviewing files that changed from the base of the PR and between 6c14405 and 2786e0b.

📒 Files selected for processing (2)

.github/workflows/ci.yml
tests/test-execution.toml

Included review availability: Your plan provides up to 10 included reviews per hour; 0 remain after this review.

📝 Walkthrough

Walkthrough

Changes

Test execution registry

Layer / File(s)	Summary
Registry and workflow documentation `.github/workflows/ci.yml`, `tests/test-execution.toml`	The workflow comment documents that the test-wiring script runs `tests/test_sync_test_wiring.py`. The registry adds `tests/test_sync_test_wiring.py` and `tests/test_cmux_settings_jsonc.py` to the `linux-guard` lane.

Priority: ⬇️ Low

Estimated code review effort: 1 (Trivial) | ~5 minutes

Change: Bug fix

Merge Risk: ⚪ Minimal · up to 2786e

The registry updates and workflow documentation introduce no identified merge-blocking risk.

🚥 Pre-merge checks | ✅ 25

✅ Passed checks (25 passed)

Check name	Status	Explanation
Title check	✅ Passed	The title clearly summarizes the main change: registering the settings JSONC and test-wiring regressions in CI.
Description check	✅ Passed	The description explains the problem, resulting behavior, validation performed, and remaining limitation. It is complete and relevant for this CI-only change, although it omits the template checklist …
Docstring Coverage	✅ Passed	No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check. Docstring coverage is scoped to functions touched by this diff. Analyzed 0 functions across 0…
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Cmux Cloud Persistent Session And Early Input	✅ Passed	The pull request changes only CI workflow documentation and two test-execution registry entries. It does not change Cloud terminal creation, cmux-tui transport, manual renderers, input routing, authen…
Cmux Swift Actor Isolation	✅ Passed	The pull request changes only `.github/workflows/ci.yml` and `tests/test-execution.toml`. The diff adds a workflow comment and two test-registry entries. It introduces no Swift code, so the cmux Swift…
Cmux Swift Blocking Runtime	✅ Passed	The pull request changes only `.github/workflows/ci.yml` and `tests/test-execution.toml`. The authoritative diff contains no Swift files or Swift runtime code, and it introduces no semaphores, blockin…
Cmux Browser Automation Off-Main	✅ Passed	PASS: The pull request changes only `.github/workflows/ci.yml` and `tests/test-execution.toml`. The diff adds a CI comment and two `linux-guard` test registry entries. It does not modify `Sources/Term…
Cmux Expensive Synchronous Load	✅ Passed	The pull request changes only `.github/workflows/ci.yml` and `tests/test-execution.toml`. The diff contains no Swift changes and adds no synchronous agent-history load, interactive-path load, or cache…
Cmux Cache Substitution Correctness	✅ Passed	PASS. The reviewed range changes only `.github/workflows/ci.yml` and `tests/test-execution.toml`. It adds CI documentation and test-registry entries. It does not change production Swift, TypeScript, o…
Cmux No Hacky Sleeps	✅ Passed	PASS. The PR changes only `.github/workflows/ci.yml` and `tests/test-execution.toml`. It adds a workflow comment and two registry entries. The authoritative diff introduces no `sleep`, timer, polling,…
Cmux Algorithmic Complexity	✅ Passed	PASS: The pull request changes only `.github/workflows/ci.yml` and `tests/test-execution.toml`. The workflow change adds a comment, and the registry changes add two test entries. No production Swift, …
Cmux Swift Concurrency	✅ Passed	PASS. The pull request changes only `.github/workflows/ci.yml` and `tests/test-execution.toml`. The reviewed diff contains no Swift paths and no added or removed Swift concurrency patterns. The custom…
Cmux Swift `@Concurrent`	✅ Passed	The pull request changes only `.github/workflows/ci.yml` and `tests/test-execution.toml`. The authoritative diff contains no Swift files or Swift code, so the `@concurrent` check is not applicable.
Cmux Swift Package Boundaries	✅ Passed	PASS: The pull request changes only `.github/workflows/ci.yml` and `tests/test-execution.toml`. The authoritative diff contains no Swift or Swift package changes, so the cmux Swift package-boundary ch…
Cmux Swiftpm Lockfiles	✅ Passed	PASS. The PR changes only a CI workflow comment and two test-registry entries. The diff contains no Package.swift, Package.resolved, .gitignore, or Xcode project package-reference changes. Therefore, …
Cmux Swift Logging	✅ Passed	PASS: The authoritative pull-request diff changes only `.github/workflows/ci.yml` and `tests/test-execution.toml`. It adds workflow documentation and test-registry entries. It adds or materially chang…
Cmux User-Facing Error Privacy	✅ Passed	PASS: The pull request changes only `.github/workflows/ci.yml` and `tests/test-execution.toml`. The workflow change is a developer-only CI comment, and the registry changes are test metadata. No chang…
Cmux Full Internationalization	✅ Passed	The pull request changes only `.github/workflows/ci.yml` and `tests/test-execution.toml`. The workflow change is a developer-only comment, and the registry change adds test execution entries. No user-…
Cmux Swiftui State Layout	✅ Passed	The pull request changes only `.github/workflows/ci.yml` and `tests/test-execution.toml`. The diff contains no Swift or SwiftUI source changes, so the SwiftUI state-layout rules do not apply.
Cmux Architecture Rethink	✅ Passed	PASS. The pull request changes only `.github/workflows/ci.yml` and `tests/test-execution.toml`. It adds a CI comment and two test-execution registry entries. It introduces no Swift code or Swift archi…
Cmux Swift Auxiliary Window Close Shortcuts	✅ Passed	The pull request changes only `.github/workflows/ci.yml` and `tests/test-execution.toml`. The authoritative diff contains no Swift changes and no window code. The auxiliary-window close-shortcut check…
Cmux Source Artifacts	✅ Passed	The PR changes only `.github/workflows/ci.yml` and `tests/test-execution.toml`. The changes add a workflow comment and two test-execution registry entries, which are intentional CI and test-system con…
Cmux No Test Or Debug Seam In Production Source	✅ Passed	The pull request changes only `.github/workflows/ci.yml` and `tests/test-execution.toml`. The authoritative diff contains no Swift file under a production `Sources/` path, so it cannot introduce the p…

✨ Finishing Touches

🧪 Generate unit tests (beta)

Commit to this branch
Create a new PR

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

_{Comment @coderabbitai help to get the list of available commands.}

github-actions · 2026-09-22T17:58:31Z

All contributors have signed the CLA ✍️ ✅
_{Posted by the CLA Assistant Lite bot.}

…equests `tests/test-execution.toml` requires an entry for every `tests/*.py`, and the preflight validator fails on any unregistered file no matter which pull request is being checked. A test that lands on main without an entry therefore turns every open pull request red until somebody registers it. That happened three times today: #13615's landing left 9 unregistered tests (#13710), then `test_ci_r2_cache_census.py` (#13731), then `test_cmux_settings_jsonc.py` and `test_sync_test_wiring.py` (#13739). This commit adds the regression only, so CI shows it red before the fix. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

#13738 and #13739 each added a `[[test]]` block for `tests/test_sync_test_wiring.py`. The two merged cleanly and left main with the test registered twice, which the registry validator rejects, so every open pull request is currently red on `guards / workflow-guard-tests / preflight`. Both blocks were identical; this removes the second one. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* test: prove the execution registry validator reddens unrelated pull requests `tests/test-execution.toml` requires an entry for every `tests/*.py`, and the preflight validator fails on any unregistered file no matter which pull request is being checked. A test that lands on main without an entry therefore turns every open pull request red until somebody registers it. That happened three times today: #13615's landing left 9 unregistered tests (#13710), then `test_ci_r2_cache_census.py` (#13731), then `test_cmux_settings_jsonc.py` and `test_sync_test_wiring.py` (#13739). This commit adds the regression only, so CI shows it red before the fix. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * ci: fail only the pull request that adds an unregistered test The validator now compares `tests/` against the merge base with the pull request's base commit. A test file with no registry entry is a hard failure only when this branch added it; one that was already unregistered on the base branch becomes a warning, printed as a GitHub annotation and a step summary note, so an unrelated pull request stays green. Everything a branch can only break by editing the registry itself stays a hard failure: entries pointing at missing tests, malformed or duplicated entries, unsupported requirements, manual entries without a reason, and lanes no workflow invokes. The failure now prints the exact TOML block to paste. When a workflow already runs the file directly, the block names `lane = "linux-guard"` and cites the workflow it derived that from; otherwise it lists the live runner lanes. CI never writes the registry itself. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * ci: drop the duplicate test_sync_test_wiring registration #13738 and #13739 each added a `[[test]]` block for `tests/test_sync_test_wiring.py`. The two merged cleanly and left main with the test registered twice, which the registry validator rejects, so every open pull request is currently red on `guards / workflow-guard-tests / preflight`. Both blocks were identical; this removes the second one. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * ci: warn instead of fail on a duplicate registration the base branch already has A duplicate entry was treated as something only the pull request at fault could produce. Today showed otherwise: two pull requests each registering the same test merge cleanly into a duplicate neither one wrote, and it reddens every open pull request exactly like an unregistered test does. The validator now reads the registry at the merge base. A path already duplicated there warns; a duplicate this branch introduces still fails. Parsing moved into `parse_registry(text, label)` so the base revision can be read through `git show` without a file on disk. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5 <noreply@anthropic.com>

teamleaderleo requested a review from lawrencecchen as a code owner September 22, 2026 17:56

teamleaderleo enabled auto-merge (squash) September 22, 2026 17:57

teamleaderleo merged commit e6b3d6b into main Sep 22, 2026
47 of 49 checks passed

teamleaderleo mentioned this pull request Sep 22, 2026

ci: fail only the pull request that adds an unregistered test #13745

Merged

teamleaderleo mentioned this pull request Sep 22, 2026

docs: inventory the CI lists that must track something else #13778

Merged

Navigation Menu

Repository navigation

ci: register the settings JSONC and test-wiring regressions - #13739

ci: register the settings JSONC and test-wiring regressions#13739
teamleaderleo merged 1 commit into
mainfrom
ci/register-two-unregistered-tests

teamleaderleo commented Sep 22, 2026 •

edited by coderabbitai Bot

Loading

Uh oh!

coderabbitai Bot commented Sep 22, 2026 •

edited

Loading

Walkthrough

Changes

Uh oh!

github-actions Bot commented Sep 22, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Conversation

teamleaderleo commented Sep 22, 2026 • edited by coderabbitai Bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Problem

Resulting behavior

Validation and remaining gap

Summary by cubic

Summary by CodeRabbit

Uh oh!

coderabbitai Bot commented Sep 22, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Walkthrough

Changes

Uh oh!

github-actions Bot commented Sep 22, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

teamleaderleo commented Sep 22, 2026 •

edited by coderabbitai Bot

Loading

coderabbitai Bot commented Sep 22, 2026 •

edited

Loading