Skip to content

tests: consolidate the 17 duplicate-name cmuxTests suites - #13786

Merged
teamleaderleo merged 2 commits into
manaflow-ai:mainfrom
teamleaderleo:dupe-suite-consolidation
Sep 23, 2026
Merged

teamleaderleo merged 2 commits into
manaflow-ai:mainfrom
teamleaderleo:dupe-suite-consolidation

Conversation

@teamleaderleo

@teamleaderleo teamleaderleo commented Sep 22, 2026 •

Copy link
Copy Markdown
Collaborator

What

RFC #13519 item 2: the 17 cmuxTests suites that share a name with an existing package test suite (227 app-host tests). I diffed every pair assertion by assertion — same name turned out not to mean same coverage in five of the seventeen — and consolidated only where the app-host copy was genuinely redundant.

Net: 80 tests leave the app-host bundle; 44 tests arrive in package targets; no assertion is lost.

Deleting a suite because a same-named package suite exists would have dropped ~50 real assertions here. The table below is explicit about which ones and why.

Disposition of all 17

app tests suite disposition
38 SocketControlSettingsTests consolidated — 33 tests ported to CmuxSettings as SocketControlSettingsChannelTests; 5 already covered by the existing package suite; app copy deleted
9 ShortcutWhenClauseTests deleted — all 9 are a strict subset of the 26-test package suite
9 RemoteLoopbackHTTPRequestRewriterTests consolidated — 3 uncovered cases ported to CmuxRemoteWorkspace (Origin/Referer on alias subdomains, response subdomain rewrite, leading-dot subdomain cookie domain); app copy deleted
6 RovoDevHookConfigTests consolidated — 5 uncovered tests ported to CMUXAgentLaunch; app copy deleted
4 SocketControlPasswordStoreTests consolidated — 2 uncovered tests ported (externally-written password file; verify with nothing configured); app copy deleted
4 QuitConfirmationPolicyTests consolidated — 1 uncovered assertion ported (the dev-build bypass under dirtyOnly, not just the default mode); app copy deleted
3 CommandPaletteEmojiTitleSearchTests deleted — exact subset; the package suite has the same 3 plus 3 nucleo-FFI variants
3 RecentlyClosedBrowserStackTests deleted — all 3 covered; the app copy only substituted the concrete snapshot type for the package stub, which the compiler already proves
3 DiffViewerPickerCommandRunnerTests partially consolidated — 2 duplicated tests deleted (the concurrency-ceiling assertion they carried is ported into the package fixture); the third is kept: it drives CmuxDiffViewerURLSchemeHandler (app target) through a real WKURLSchemeTask, which is genuinely host-dependent
2 WorkspaceRemoteDaemonManifestTests deleted — Info-dictionary decode covered by CmuxCoreTests/…decodesFromInfoDictionary; the versioned cache path covered by CmuxRemoteWorkspaceTests/…cachePathShape with identical inputs
28 NotificationFeedHistoryTests left alone — not a duplicate. Same filename and suite name, zero shared types. The app copy tests NotificationFeedHistoryStore/…Persistence (app target: retention trimming, snapshot quarantine/recovery, revision monotonicity, mobile-host RPC). The package copy tests NotificationFeedProjection in CmuxMobileShellUI (iOS list rows, disclosure groups, grouping). 0 of 28 covered.
18 SidebarDropPlannerTests left alone — not a duplicate. The package file of that name holds SidebarDropPlannerPackageTests, 9 of whose 11 tests exercise SidebarWorkspaceReorderDropResolver. crossWindowInsertion and targetIndex are called nowhere in any package test. 0 of 18 covered. It is class-(a) movable, so it belongs to the suite-relocation stream, not here.
13 SettingsSearchIndexTests left alone — name collision, not duplication. Sources/SettingsSearchIndex.swift declares an app-owned enum SettingsSearchIndex with a static API and its own alias table; CmuxSettingsUI declares an unrelated public struct SettingsSearchIndex with an instance API. Two production types, same name.
1 GhosttyCopyActionResolverTests left alone — deliberate. Its doc comment says it exists so the app test target proves it can consume the package's public API "instead of silently executing zero tests". That is a linkage guard the package cannot make.
65 CommandPaletteSearchEngineTests skipped — in-flight (#13759 / #13752 / #13736 / #13744 touch cmuxTests/CommandPaletteSearchEngineTests.swift)
19 CommandPaletteNucleoFFITests skipped — in-flight (same set touches cmuxTests/CommandPaletteNucleoFFITests.swift)
2 SidebarWorkspaceAuxiliaryDetailVisibilityTests skipped — in-flight (lives in cmuxTests/WorkspaceUnitTests.swift)

Why the big one was worth porting rather than deleting

SocketControlSettingsTests is the clearest case of "same name, complementary coverage". The package suite covers mode resolution, managed-device forcing, and the XCTest socket fallback. The app-host copy covered a disjoint area: per-channel default paths (stable/nightly/staging/tagged-debug), refusal to hijack the stable socket through a user-scoped alias, a case variant, a /private/tmp alias, a leaf symlink or a 64-deep symlink chain, and the pre-listener startup probe. Every symbol it touches is CmuxSettings or Foundation — the app host bought it nothing.

Verification

  • swift test on each touched package (CmuxSettings, CMUXAgentLaunch, CmuxRemoteWorkspace, CmuxBrowser).
  • python3 scripts/normalize-pbxproj.py after unwiring the four deleted files.
  • scripts/lint-pbxproj-test-wiring.sh — ok (1047 test files).
  • tests/test_ci_test_execution_registry.py, tests/test_normalize_pbxproj.py, tests/test_sync_test_wiring.py — all pass. (tests/test-execution.toml registers Python/shell regressions only, so no registry change was needed.)

🤖 Generated with Claude Code


View with [code]smith Autofix with [code]smith
Need help on this PR? Tag @codesmith-bot with what you need. Autofix is disabled.


Summary by cubic

Consolidates the 17 app-host cmuxTests suites that share a name with package test suites, diffing each pair assertion-by-assertion first: five "same-name" pairs turned out not to be duplicates, so only twelve were consolidated. Net: 80 tests leave the app-host bundle, 44 land in package targets, and no assertion is lost.

Disposition

  • Deleted outright where the package suite fully covered the app copy: ShortcutWhenClauseTests, CommandPaletteEmojiTitleSearchTests, RecentlyClosedBrowserStackTests, WorkspaceRemoteDaemonManifestTests.
  • Ported the uncovered cases into CMUXAgentLaunch (RovoDevHookConfigTests), CmuxRemoteWorkspace (RemoteLoopbackHTTPRequestRewriterTests), and CmuxSettings (SocketControlPasswordStoreTests, QuitConfirmationPolicyTests), then deleted the app copies.
  • Moved 33 disjoint SocketControlSettingsTests into CmuxSettings as SocketControlSettingsChannelTests (per-channel defaults, socket hijack refusal, pre-listener startup probe); the other 5 were already covered.
  • Kept only the WKURLSchemeTask teardown test for DiffViewerPickerCommandRunnerTests; it drives the app-target CmuxDiffViewerURLSchemeHandler, and the runner tests moved to CmuxBrowser (with the concurrency-ceiling and cancellation assertions ported into the package fixture).

Left alone

  • NotificationFeedHistoryTests and SettingsSearchIndexTests are name collisions across module boundaries, not duplicates: the paired tests exercise unrelated production types, so 0 of their 41 assertions are covered elsewhere.
  • SidebarDropPlannerTests tests app-side types (crossWindowInsertion, targetIndex) that no package test exercises.
  • GhosttyCopyActionResolverTests is a deliberate app-target linkage guard.
  • CommandPaletteSearchEngineTests, CommandPaletteNucleoFFITests, and SidebarWorkspaceAuxiliaryDetailVisibilityTests sit in files owned by in-flight PRs and are deferred.

Written for commit 760c943. Summary will update on new commits.

Review in cubic

@coderabbitai

coderabbitai Bot commented Sep 22, 2026 •

Copy link
Copy Markdown

Warning

Review limit reached

Next included review available in 22 minutes.

Check out review usage here.

View limit details

Limit details: You’ve used all 10 included reviews currently available.

You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository.

Learn how review limits work.

Review configuration:

⚙️ Run configuration

Configuration used: Repository: manaflow-ai/cmux/.coderabbit.yaml

Review profile: ASSERTIVE

Plan: Advanced

Run ID: 2b5fde39-ce54-4838-83dc-d4280b3bac9d

📥 Commits

Reviewing files that changed from the base of the PR and between d7f6648 and 760c943.

📒 Files selected for processing (14)
  • Packages/macOS/CMUXAgentLaunch/Tests/CMUXAgentLaunchTests/RovoDevHookConfigTests.swift
  • Packages/macOS/CmuxBrowser/Tests/CmuxBrowserTests/DiffViewerPickerCommandRunnerTests.swift
  • Packages/macOS/CmuxRemoteWorkspace/Tests/CmuxRemoteWorkspaceTests/RemoteLoopbackHTTPRewriterTests.swift
  • Packages/macOS/CmuxSettings/Tests/CmuxSettingsTests/DomainSettingsStoreTests.swift
  • Packages/macOS/CmuxSettings/Tests/CmuxSettingsTests/SocketControl/SocketControlPasswordStoreTests.swift
  • Packages/macOS/CmuxSettings/Tests/CmuxSettingsTests/SocketControl/SocketControlSettingsChannelTests.swift
  • cmux.xcodeproj/project.pbxproj
  • cmuxTests/CommandPaletteEmojiTitleSearchTests.swift
  • cmuxTests/DiffViewerPickerCommandRunnerTests.swift
  • cmuxTests/GhosttyConfigTests.swift
  • cmuxTests/RovoDevHookConfigTests.swift
  • cmuxTests/ShortcutAndCommandPaletteTests.swift
  • cmuxTests/ShortcutWhenClauseTests.swift
  • cmuxTests/SocketControlPasswordStoreTests.swift

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@github-actions

Copy link
Copy Markdown
Contributor

All contributors have signed the CLA ✍️ ✅
Posted by the CLA Assistant Lite bot.

RFC manaflow-ai#13519 item 2. Seventeen cmuxTests suites share a name with an
existing package test suite (227 app-host tests). Diffing them assertion
by assertion shows five are not duplicates at all, so this consolidates
only the twelve that are.

Removed from the app-host bundle (80 tests), after porting every
assertion the package copy did not already make:

- SocketControlSettingsTests (38): the package suite covers mode
  resolution and the XCTest fallback; the app copy covered per-channel
  default paths and stable-socket hijack refusal (user-scoped alias,
  case variant, /private/tmp alias, leaf symlink, 64-deep chain) plus
  the pre-listener startup probe. 33 tests move to CmuxSettings as
  SocketControlSettingsChannelTests; 5 were already covered.
- ShortcutWhenClauseTests (9), CommandPaletteEmojiTitleSearchTests (3),
  RecentlyClosedBrowserStackTests (3), WorkspaceRemoteDaemonManifestTests
  (2): strict subsets of the package suites, deleted outright.
- RemoteLoopbackHTTPRequestRewriterTests (9), RovoDevHookConfigTests (6),
  SocketControlPasswordStoreTests (4), QuitConfirmationPolicyTests (4):
  the uncovered cases move into the package suite first.
- DiffViewerPickerCommandRunnerTests: the two runner tests go; the
  WKURLSchemeTask teardown test stays, since it drives the app-target
  CmuxDiffViewerURLSchemeHandler.

Left in place, with reasons:

- NotificationFeedHistoryTests and SettingsSearchIndexTests are name
  collisions across module boundaries, not duplicates — the app copies
  test app-target types that merely share a name with unrelated package
  types. 0 of 41 assertions are covered elsewhere.
- SidebarDropPlannerTests: the same-named package file mostly tests
  SidebarWorkspaceReorderDropResolver; crossWindowInsertion and
  targetIndex are asserted nowhere in Packages.
- GhosttyCopyActionResolverTests is a deliberate app-target linkage
  guard.
- CommandPaletteSearchEngineTests, CommandPaletteNucleoFFITests and
  SidebarWorkspaceAuxiliaryDetailVisibilityTests sit in files owned by
  open PRs and are left for a follow-up.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@teamleaderleo
teamleaderleo force-pushed the dupe-suite-consolidation branch from 0ef3945 to e8e9277 Compare September 22, 2026 23:29
@teamleaderleo
teamleaderleo enabled auto-merge (squash) September 22, 2026 23:59
@teamleaderleo
teamleaderleo merged commit 83a6294 into manaflow-ai:main Sep 23, 2026
48 of 49 checks passed
teamleaderleo added a commit that referenced this pull request Sep 23, 2026
* ci: run package tests on the pull requests that change packages

`macos / swift-package-tests` is gated on `inputs.full_suite == 'true'`,
and `full_suite` is false for every pull request under
`CI_PULL_REQUEST_SUITE=compile-only`. The job is therefore skipped on
every PR. The only macOS signal a PR gets is `macOS compile admission`,
which builds package *library* targets and never their test targets, so
a change to a package's tests currently lands with zero CI execution of
those assertions. #13786 and #13790 move ~150 of them into package test
targets.

Route the lane from changed paths, the way the `cli` lane selects itself
(#13760). A new `swift_packages` change area asks
`scripts/ci/select_package_tests.py` — the same selector the job already
uses to narrow its package list — whether a diff can affect any package
the job runs. Only paths that feed a package are offered to it:

  * The selector fails open, so the lane's own workflow and scripts
    select all 33 packages. On main that is right; on a pull request it
    is the 30-minute sweep under another name. Over the last 200 merged
    PRs, routing those would have queued 34 full runs.
  * A package outside the job's list selects nothing, so the lane would
    start, check out submodules and test zero packages.

Measured over the last 200 merged pull requests: 28 would run the lane
(0 do today), testing a median of 2 packages and never more than 15,
never all 33 and never zero.

The full suite is untouched: `app-host-unit-tests`, `tests-build-and-lag`
and the Release jobs stay gated on `full_suite` alone, and a test in this
change pins that set so a new lane cannot silently join it. The Release
Ghostty CLI helper steps inside the job now also require `full_suite`,
since only the `full_suite`-gated Release jobs consume that artifact.

Also collect the operating system's crash report when a package test
runner dies on a signal. `main` is currently red from a SIGBUS in
CmuxSettingsUI, and all the job records is SwiftPM's one-line
"Exited with unexpected signal code 10" — no faulting address, no
frames. Replaying the retry guard against that job's captured output
shows widening its signal list would not have retried it anyway: the
runner had already emitted 401 lines of test output, which the guard's
third clause correctly refuses to retry. The signal list is unchanged.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* test: catch full package sweeps on mixed routed changes

* ci: keep targeted package selection aligned with routing

---------

Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
@teamleaderleo
teamleaderleo deleted the dupe-suite-consolidation branch September 23, 2026 11:32
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant