Skip to content

test: derive the review-fabric contract group instead of naming it - #13785

Closed
teamleaderleo wants to merge 1 commit into
mainfrom
ci/review-fabric-routing-contract
Closed

teamleaderleo wants to merge 1 commit into
mainfrom
ci/review-fabric-routing-contract

Conversation

@teamleaderleo

@teamleaderleo teamleaderleo commented Sep 22, 2026 •

Copy link
Copy Markdown
Collaborator

#13788 landed first and is the same fix. This is now the follow-up that removes the one behaviour the two versions did not share.

#13788 asserts "preflight" in groups_for_path(path). That pins today's group name into the assertion. Move the Test review fabric contracts step to another group and move PATH_OWNERS with it — routing still correct end to end — and it fails:

AssertionError: 'preflight' not found in ('ci',) : editing .github/review-fabric-policy.json
must select the preflight guard group, which is where tests/test_review_fabric.py runs

The message asserts a fact it never checks.

This reads the owning group out of ci-guards.yml via direct_path_owners() and requires the routed groups to intersect it, keeping #13788's subTest loop and its classify() assertion that the workflow-guard-tests lane actually runs.

Verified against the same base, swapping only tests/test_review_fabric.py:

Scenario #13788 this
drop the review-fabric PATH_OWNERS entries fails fails
replace the run: python3 tests/test_review_fabric.py line fails fails
reroute a path to the wrong group fails fails
move the contract step preflight → ci, ownership with it fails (false positive) passes

Full file: 16 tests, OK.

🤖 Generated with Claude Code

…es them

#13775 moved the guard routing declarations out of
detect_linux_guard_changes.py and into workflow_guard_groups.PATH_OWNERS.
test_ci_executes_review_fabric_contracts still greps the detector for the four
review-fabric paths, so it now finds none of them and main is red for every
pull request in the repo.

The contract the test wants is behavioural: an edit to a review-fabric input
must route the guard group that runs the contracts. Ask groups_for_path() and
direct_path_owners() that question directly, so the next move of the
declarations cannot break the test while the routing still works — and a
genuinely dropped route still fails, now naming the path and the groups.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@github-actions

Copy link
Copy Markdown
Contributor

All contributors have signed the CLA ✍️ ✅
Posted by the CLA Assistant Lite bot.

@coderabbitai

coderabbitai Bot commented Sep 22, 2026 •

Copy link
Copy Markdown

Review in Change Stack →

Navigate logical layers of code changes, visualize relationships, and explore their blast radius.

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Repository: manaflow-ai/cmux/.coderabbit.yaml

Review profile: ASSERTIVE

Plan: Advanced

Run ID: f406a65f-8be7-4c74-9115-35d8d8b4919d

📥 Commits

Reviewing files that changed from the base of the PR and between 95e843a and abfc9c3.

📒 Files selected for processing (1)
  • tests/test_review_fabric.py

Included review availability: Your plan provides up to 10 included reviews per hour; 0 remain after this review.


📝 Walkthrough

Walkthrough

The CI contract test now uses workflow_guard_groups to verify that each review fabric contract path routes to a group-conditioned step that runs the contracts.

Changes

CI contract routing

Layer / File(s) Summary
Verify contract path routing
tests/test_review_fabric.py
The test imports workflow_guard_groups and checks that the test file and each contract path map to the group-conditioned step that runs the contracts.

Priority: ⬇️ Low

Estimated code review effort: 2 (Simple) | ~10 minutes

Change: Other

Merge Risk: ⚪ Minimal · up to abfc9

The CI contract-routing test remains aligned with the workflow routing model; no merge-blocking risk is identified.

🚥 Pre-merge checks | ✅ 24 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 1 functions across 1 files. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (24 passed)
Check name Status Explanation
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
Cmux Cloud Persistent Session And Early Input ✅ Passed The check is not applicable to this pull request. The reviewed diff changes only tests/test_review_fabric.py, which updates a CI routing test. It changes no Cloud terminal creation, transport, sessi…
Cmux Swift Actor Isolation ✅ Passed The pull request changes only tests/test_review_fabric.py, a Python test file. It introduces no production Swift changes, so the Swift actor-isolation check does not apply.
Cmux Swift Blocking Runtime ✅ Passed The reviewed diff changes only tests/test_review_fabric.py, a Python test file. It contains no production Swift changes and does not introduce or expand any of the specified Swift blocking or timing…
Cmux Browser Automation Off-Main ✅ Passed The check is not applicable to this pull request. The authoritative diff changes only tests/test_review_fabric.py; it updates CI routing assertions and does not change browser socket automation, wor…
Cmux Expensive Synchronous Load ✅ Passed The check applies to production Swift changes. The reviewed diff changes only tests/test_review_fabric.py, a Python test file, and adds no Swift changes or synchronous agent-history loads.
Cmux Cache Substitution Correctness ✅ Passed The pull request changes only tests/test_review_fabric.py, a Python file. It does not change production Swift, TypeScript, or JavaScript, so the cache-substitution check does not apply.
Cmux No Hacky Sleeps ✅ Passed The pull request changes only tests/test_review_fabric.py. It adds Python test routing assertions and introduces no production TypeScript, JavaScript, shell, or build/runtime changes. The check does…
Cmux Algorithmic Complexity ✅ Passed The diff changes only tests/test_review_fabric.py, a Python test file. The check applies to production Swift, TypeScript, JavaScript, shell, and runtime code, and explicitly passes tests. No applica…
Cmux Swift Concurrency ✅ Passed The reviewed diff changes only tests/test_review_fabric.py. It adds Python routing assertions and removes a source-text check. It introduces no cmux-owned Swift code or Swift concurrency patterns co…
Cmux Swift @Concurrent ✅ Passed The reviewed diff changes only tests/test_review_fabric.py, a Python file. It introduces no Swift changes or Swift async work covered by this check.
Cmux Swift Package Boundaries ✅ Passed The check applies to production Swift changes. The reviewed diff changes only tests/test_review_fabric.py, and the changed-path inventory contains no Swift files. Therefore, the Swift package-bounda…
Cmux Swiftpm Lockfiles ✅ Passed The reviewed diff changes only tests/test_review_fabric.py. It does not change a SwiftPM package, Xcode project, .gitignore, workflow, or dependency file, so the SwiftPM lockfile check is not appl…
Cmux Swift Logging ✅ Passed The pull request changes only tests/test_review_fabric.py. It contains no production Swift changes or logging changes covered by this check.
Cmux User-Facing Error Privacy ✅ Passed The only changed file is tests/test_review_fabric.py. The diff changes test assertions and routing checks, not production user-facing errors or output. The policy explicitly allows tests.
Cmux Full Internationalization ✅ Passed The pull request changes only tests/test_review_fabric.py. The diff updates a CI routing test and adds no user-facing text, production UI, metadata, API response, rendered content, or locale catalog…
Cmux Swiftui State Layout ✅ Passed The pull request changes only tests/test_review_fabric.py. Its diff updates Python test imports and routing assertions. It adds no SwiftUI state, layout measurement, list-row, or render-time mutatio…
Cmux Architecture Rethink ✅ Passed The pull request changes only tests/test_review_fabric.py. It contains no Swift source changes and does not modify .github/review-bot-rules/swift-architectural-rethink.md. The Swift architecture c…
Cmux Swift Auxiliary Window Close Shortcuts ✅ Passed The reviewed diff changes only tests/test_review_fabric.py. It adds Python test routing checks and changes no Swift code, so it does not add or materially change a standalone cmux-owned window. The …
Cmux Source Artifacts ✅ Passed The only changed path is tests/test_review_fabric.py. The diff adds test logic that checks CI routing for review-fabric inputs. This is an intentional test source file, which the source-control arti…
Cmux No Test Or Debug Seam In Production Source ✅ Passed The check applies only to changed Swift files under production Sources/ paths. The review-scoped diff changes only tests/test_review_fabric.py, a Python test file. It introduces no production Swif…
Title check ✅ Passed The title clearly describes the main change: the test now uses routing behavior to determine where review-fabric edits land instead of searching file names.
Description check ✅ Passed The description explains the failure, cause, fix, and verification results. It includes the required Summary and Testing information. The Demo Video, review-trigger block, and checklist sections are o…
  • Fix all pre-merge checks with AI
✨ Finishing Touches 💡 1
📝 Generate docstrings 💡
  • Commit to this branch
  • Create a new PR
🧪 Generate unit tests (beta)
  • Commit to this branch
  • Create a new PR

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@teamleaderleo

Copy link
Copy Markdown
Collaborator Author

Independently verified this on a clean checkout of the branch, since the value of the change is that the test still fails when coverage genuinely breaks.

python3 tests/test_review_fabric.py — 16 tests, OK.

Two mutations, each reverted after:

mutation result
drop .github/review-fabric-policy.json from PATH_OWNERS fails: .github/review-fabric-policy.json routes to no guard group
move the Test review fabric contracts step to matrix.group == 'quality-determinism' fails: routed=['preflight'] contracts=['quality-determinism']

The second one is the case the old assertion could not catch: the four literal paths stay in place while the contract stops running. Using direct_path_owners(workflow) rather than matching the matrix.group condition by hand is also what makes the multi-group case fall out for free.

I had written the same fix before finding this; this one is better, so I dropped mine unpushed.

@teamleaderleo

Copy link
Copy Markdown
Collaborator Author

Superseded: #13788 landed the same fix at 19:40 and tests/test_review_fabric.py now passes on a clean main (16 tests, OK). This branch conflicts with it.

#13788 is the better of the two — it asserts both that the path routes linux_guard_tests and that groups_for_path explicitly owns it with preflight, and it says why the second assertion cannot use classify_test_groups: that function falls open to every group for an unknown path, so it would keep passing if ownership were dropped. That is a weakness my version and this one both had.

Three of us independently wrote this fix within about an hour. Leaving the PR open rather than closing it, per standing instruction.

@teamleaderleo

Copy link
Copy Markdown
Collaborator Author

Heads up: #13785 and #13788 are the same fix. Both change only tests/test_review_fabric.py, and both rewrite test_ci_executes_review_fabric_contracts — the assertion #13775 invalidated by making guard routes derived instead of literal. Whichever lands second will conflict.

I ran both against the same base (95e843a26) in a worktree, swapping only the test file.

Real regressions — both catch all three, identically:

Scenario #13785 #13788
R1 drop the review-fabric PATH_OWNERS entries caught caught
R2 stop running python3 tests/test_review_fabric.py in ci-guards.yml caught caught
R3 reroute a path to the wrong group caught caught

Legitimate refactor — they differ. Move the contract step from preflight to ci and move PATH_OWNERS with it, so routing stays correct end to end:

result
#13785 PASS — correct
#13788 FAIL — false positive

#13788 fails with:

AssertionError: 'preflight' not found in ('ci',) : editing .github/review-fabric-policy.json
must select the preflight guard group, which is where tests/test_review_fabric.py runs

That message asserts a fact it never checks. #13788 hardcodes "preflight"; #13785 derives the owning group from direct_path_owners(workflow), so it keeps testing the real invariant — the paths route to whichever group actually runs the contracts — instead of pinning today's group name.

Same detection power, one fewer false positive: #13785 is the one to land. No objection to taking #13788's subTest loop or its explanatory comment on top of it.

@teamleaderleo

Copy link
Copy Markdown
Collaborator Author

Superseded by #13788, which landed the same behavioural fix and additionally guards the fall-open case in classify_test_groups that my version would have missed. Nothing unique here to carry over. Closing.

@teamleaderleo teamleaderleo changed the title ci: ask the router where review-fabric edits land, not which file names them test: derive the review-fabric contract group instead of naming it Sep 22, 2026
@teamleaderleo

Copy link
Copy Markdown
Collaborator Author

Reopened as #13801 — this branch (ci/review-fabric-routing-contract) is now rebased onto main on top of #13788, so it no longer conflicts. GitHub pinned this PR to the pre-rebase head when it was closed and refused to reopen it, hence the new number.

It keeps #13788's subTest loop and classify() lane assertion and only replaces the hardcoded "preflight" with the group derived from ci-guards.yml.

teamleaderleo added a commit that referenced this pull request Sep 23, 2026
* docs: tell agent sessions how not to duplicate each other

Several agent sessions work this repo at once and cannot see each other.
Nothing in CLAUDE.md says so, and the resulting waste is now measurable.

On 2026-09-22 a shared observable -- main going red on
test_ci_executes_review_fabric_contracts -- reached every session at once.
Each diagnosed it independently and opened a PR: #13785, #13788, #13800,
#13801 and #13802, five PRs on one test function in twenty-one minutes, two
of them five seconds apart. One landed. The reviewer attention spent on the
other four is the cost this section exists to avoid.

Two failures showed up repeatedly and are written down here because neither
is guessable:

Sessions share one GitHub account, so `author` and `mergedBy` name the
account and never the actor. Three separate claims about which session did
what were made from those fields today, all wrong, and two were relayed to
the user before being retracted.

GitHub keeps serving `mergeable` and `mergeStateStatus` on closed and merged
pull requests, where they are stale. Reading CONFLICTING off an already
merged PR sent a session to resolve a conflict that did not exist, twice.

The last paragraph guards the opposite error. #13754 and #13797 changed
exactly the same two files, fixed different bugs, and both merged, so an
overlap scan keyed on file paths would have proposed closing a good PR.
Composing them locally and running the shared test is what distinguishes
the cases.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* docs: give sessions a callsign to sign their work with

The section above tells sessions how not to collide. It does not give them a
way to say who they were, and that gap produced its own failures today: three
claims about which session opened, merged or reviewed something, every one of
them read off `author` or `mergedBy`, every one wrong, two relayed to the user
before being retracted.

Those fields name the shared push account. Nothing in the repository answers
"which session did this", so sessions inferred it from timing and were wrong.
A callsign in a commit trailer answers it directly.

Stated as attribution and not authority, deliberately. The Stensibly product
model is explicit that callsigns, names, branches and prior activity never
substitute for current authority evidence, and a self-assigned name two
sessions can pick independently is exactly the kind of identity that must not
gate an action. It records who acted. It grants nothing.

This commit signs itself, which is the whole convention.

Callsign: Teakettle 🫖
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* docs: correct the callsign section against the live registry

The previous commit invented a convention. There is already a working one, and
checking it showed the invented version wrong in three ways.

`teamleaderleo/stensibly` #454 is a live registrar: a `github-actions[bot]`
workflow that accepts `/callsign reserve`, answers in seconds with a
`callsign-receipt/v0` carrying an accepted generation and a 24h lease, and
releases on request. Its worker quickstart is
`docs/callsign-registry-dogfood.md` in that repo. This section now points there
instead of describing a parallel scheme.

I reserved through it rather than trusting the document, and each correction
below is something the receipt disproved:

The sigil is derived from the callsign by the registrar, not chosen by the
worker. Reserving `Teakettle` returned `💾`, not the emoji the previous commit
had picked for itself and put in its own trailer.

Names are leased. Collision keys are compared without case or separators, so
`Rook`, `rook` and `r-o_o k` are one name. The previous commit said collisions
were expected and tolerable, which is true of the derived sigil and false of
the name.

A generation may be shown only from an accepted receipt, with `pending` or
`unregistered` as the honest fallback. The previous commit had no notion of a
generation at all.

The sign-off format follows the registry's: `— <Callsign> g<generation>
<sigil>`, not a bare name and emoji.

Attribution and not authority is unchanged and now cites its owner:
`teamleaderleo/quarry` #1103 tracks the defect that a callsign in comment text
is marker text rather than an authenticated principal.

Callsign: Teakettle g1 💾
Run: run_cmux_ci_delineation_20260922_01
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* docs: check local worktrees and recent remote branches

---------

Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant