Skip to content

test: respect build defaults in stable Cloud override assertions - #13838

Merged
teamleaderleo merged 2 commits into
mainfrom
codex/cloud-override-build-default-test
Sep 23, 2026
Merged

teamleaderleo merged 2 commits into
mainfrom
codex/cloud-override-build-default-test

Conversation

@teamleaderleo

@teamleaderleo teamleaderleo commented Sep 23, 2026 •

Copy link
Copy Markdown
Collaborator

The stable-channel Cloud override test assumes the unavailable remote flag always defaults to false. Debug builds intentionally default Cloud to true, so the nil-remote case fails twice in the app-host suite (hosted failure). A stable override capability disables local overrides; it does not change the host's compiled fallback.

Use explicit Debug/Release expectations and store the opposite override value so every remote case proves the override is ignored. Keep the existing rejected-write and clear assertions. No app behavior changes.

Validation: an executable probe using the production resolution type and extracted compiled fallback reproduces the old Debug nil-remote failure, then passes all three remote cases in Debug and Release. The complete test file parses in both configurations; strict determinism scan has zero findings. The probe does not execute the full app-host persistence/write path; no full app-host run is claimed.


Summary by cubic

Fixes the stable-channel Cloud override test so it passes in Debug builds by respecting the compiled fallback default instead of assuming it is always false.

  • Uses explicit Debug/Release defaults for the unavailable remote case and stores the opposite override value so every remote case proves the override is ignored.
  • Keeps the existing rejected-write and clear assertions.

No app behavior changes.

Written for commit 72ad3ee. Summary will update on new commits.

Review in cubic

@coderabbitai

coderabbitai Bot commented Sep 23, 2026 •

Copy link
Copy Markdown

Warning

Review limit reached

Next included review available in 59 seconds.

Check out review usage here.

View limit details

Limit details: You’ve used all 10 included reviews currently available.

You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository.

Learn how review limits work.

Review configuration:

⚙️ Run configuration

Configuration used: Repository: manaflow-ai/cmux/.coderabbit.yaml

Review profile: ASSERTIVE

Plan: Advanced

Run ID: f90252c8-6b5e-4288-9aed-b3d89a9a3ec4

📥 Commits

Reviewing files that changed from the base of the PR and between cd4d84e and 72ad3ee.

📒 Files selected for processing (1)
  • cmuxTests/CloudNightlyOverrideTests.swift

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@github-actions

Copy link
Copy Markdown
Contributor

All contributors have signed the CLA ✍️ ✅
Posted by the CLA Assistant Lite bot.

@teamleaderleo teamleaderleo added the full-ci EXPENSIVE: full macOS tests/builds; overrides selective PR routing. Not needed for normal checks. label Sep 23, 2026
@teamleaderleo

teamleaderleo commented Sep 23, 2026 •

Copy link
Copy Markdown
Collaborator Author

Added full-ci to run the changed Cloud test through the required macOS suite. The prior suite-coverage failure was the missing label, not a source/test failure. Fresh run: https://github.com/manaflow-ai/cmux/actions/runs/35807113831 on unchanged head b6e3acc.

The older run 35806309630 was blocking it in workflow concurrency. Immediately before cancellation its only unfinished job was queued compile admission with no assigned runner; no active native build or compiled product was interrupted. Requested cancellation of that superseded run so the required full-suite run can proceed.

— TestLantern g1 ✨
Run: run_cmux_ci_landing_20260923_01

@teamleaderleo

Copy link
Copy Markdown
Collaborator Author

Refreshed onto main cd4d84e in 72ad3ee, bringing in the landed Swift Testing selector fix (#13831), package execution evidence (#13824), broken-pipe reporting fix (#13737), and Linux manifest routing (#13821). This does not claim to fix the separate CmuxSettingsUI signal-10 crash.

The PR still changes only CloudNightlyOverrideTests.swift relative to main. Its test source is identical to the previously validated patch; DEBUG and Release syntax checks and diff check pass. Full-ci remains enabled for actual app-host validation.

Cancelled the preceding run35807113831 before either native job acquired a runner, then pushed this single refresh to avoid spending the suite on obsolete CI. No active native build was interrupted.

— TestLantern g2 ✨
Run: run_cmux_ci_landing_20260923_02

@teamleaderleo
teamleaderleo enabled auto-merge (squash) September 23, 2026 03:12
@teamleaderleo

Copy link
Copy Markdown
Collaborator Author

These failures are not this PR's, and this PR cannot be made green by editing it.

Every failing test reproduces on main with byte-identical assertion text at identical line numbers. Baseline: run 35814962578 on main 432b1103 — dispatched 03:35Z, newer than this PR's base cd4d84eb — where all six app-host shards fail (35/18/31/26/1/1 distinct failures). Older run 35788803553 on main f3d204a4 shows the same.

  • Shard 6 testFiveTabRendererFootprintReturnsToOneRendererTargetAcrossHideRevealCycles: PR job 107035659768 vs main job 107040546303 fail at the same six lines (TerminalAndGhosttyTests.swift:4225/4226/4273/4282/4283/4289) with the same messages. Note this test used to be XCTSkipped behind CMUX_RENDERER_MEMORY_REGRESSION; 52da854e4e (ci: forward the renderer-memory gate across the console-session hop #13776) made the gate actually reach the runner, so it was unmasked, not newly broken. It looks untracked.
  • Shard 4 Cloud adoption/projection/placement: PR job 107035659818 vs main job 107040546305 — identical set and lines across CloudMachineWorkspaceAdoptionTests, CloudTerminalPlacementTests, CloudWorkspaceLiveProjectionTests, same destinationNotFound failures. Only shard assignment differs between runs.
  • Shard 5 workspaceRevealKeepsTerminalSizeUntilAnUnchangedGeometryPass(): PR job 107035659821 and main jobs 107040546271/106959865880 fail at GhosttyTerminalViewVisibilityPolicyTests.swift:501-502 with the same text.

macos / tests-build-and-lag (107035659806) passed. Shards 1-3 were still running at triage time; main has 35/18/31 failures in those shards at the same baseline, so they are expected red for the same reason.

This PR changes one file, cmuxTests/CloudNightlyOverrideTests.swift, and neither of its tests appears in any failing shard log — that suite lands in shards 1-3. The earlier shard 5/6 failure cited in the description was a different branch's run with different sharding.

Tracking: #13879 (main full-suite red, self-closes on next green main). Broader history: #12232, #9555, #12180.

Nothing here is flaky — the failures are deterministic on main. Holding this PR to a green full suite means holding it until main is green.

— Rockall g1 🪙
Run: run_cmux_land_ready_prs_20260923_A

@teamleaderleo

Copy link
Copy Markdown
Collaborator Author

Independent review of the diff. It holds up.

stableIgnoresPersistedOverrideAndRejectsWrites asserted the Cloud flag's unavailable fallback is always false, but 9c2ba78be4 made cloudMachinesDefault true under #if DEBUG (Sources/FeatureFlags.swift:56-60). That commit added an unavailableDefault ladder to CloudFeatureFlagTests and missed this file, so the remote == nil case has been red in the Debug app-host suite ever since. This applies the same, already-precedented ladder to the file that was missed.

The split matches the production constant in both configurations, and SWIFT_ACTIVE_COMPILATION_CONDITIONS carries DEBUG only in name = Debug configs for both the app and the cmuxTests target, so the test's #if DEBUG and the flag's cannot diverge. CI runs this suite with -configuration Debug (ci-macos.yml:1150), which is exactly the condition that was failing.

Storing !expectedValue is a real strengthening rather than a restatement. Previously the remote == true case persisted true against an expected true and passed whether or not the override was honored — vacuous. It now persists false and genuinely proves the stable identity discards the override. All three parameterizations are non-vacuous after the change. The rejected-write assertion still bites because storedOverrideValue reads defaults.object(forKey:) and so distinguishes a stored false from an absent key. I could not construct a regression in the stable-channel override path that the new version misses and the old one caught; the reverse exists.

No wiring change needed — the file is modified, not added, and ./scripts/sync-test-wiring --check reports ok (checked 1051 test files).

One nit, not blocking: this is now the fourth hardcoded copy of the Debug/Release default. let unavailableDefault = cloud.defaultWhenUnavailable would be drift-free and still non-vacuous, since the default is independently pinned at CloudFeatureFlagTests.swift:48-51. Worth doing before the flag's reviewBy: 2026-10-01.

Reviewed by source inspection only — I did not build or run the suite, which is macOS-only.

— Rockall g1 🪙
Run: run_cmux_land_ready_prs_20260923_A

@teamleaderleo
teamleaderleo merged commit cd3ce57 into main Sep 23, 2026
37 of 44 checks passed
@teamleaderleo
teamleaderleo deleted the codex/cloud-override-build-default-test branch September 23, 2026 05:33
@blacksmith-sh

blacksmith-sh Bot commented Sep 23, 2026

Copy link
Copy Markdown

Found 8 test failures on Blacksmith runners:

Failures

Test View Logs
cmux_ssh_attach_retry_reason='"'"'connection interrupted'"'"'; cmux_ssh_attach_no_progr
ess_retry=0; : ;;
View Logs
cmux_ssh_attach_retry_reason='"'"'connection interrupted'"'"'; cmux_ssh_attach_no_progr
ess_retry=0; cmux_ssh_attach_reconnect_delay="$cmux_ssh_attach_reconnect_initial_delay"
;;
View Logs
cmux_ssh_attach_retry_reason='"'"'host unreachable'"'"'; cmux_ssh_attach_no_progress_re
try=0; if [ "$cmux_ssh_attach_auth_succeeded" -eq 0 ]; then :; fi ;;
View Logs
cmux_ssh_attach_retry_reason='"'"'remote service is starting'"'"'; cmux_ssh_attach_no_p
rogress_retry=0 ;;
View Logs
cmux_ssh_attach_retry_reason='"'"'remote service is starting'"'"'; cmux_ssh_attach_no_p
rogress_retry=0; if [ "$cmux_ssh_attach_auth_succeeded" -eq 0 ]; then :; fi ;;
View Logs
cmux_ssh_attach_retry_reason='"'"'remote service made no progress'"'"'; cmux_ssh_attach
no_progress_retry=$((cmux_ssh_attach_no_progress_retry + 1)); cmux_ssh_attach_reconnec
t_delay="$cmux_ssh_attach_reconnect_initial_delay"; if [ "$cmux_ssh_attach_no_progress

retry" -ge "$cmux_ssh_attach_no_progress_limit" ]; then if [ "$cmux_ssh_attach_no_progr
ess_limit" -eq 1 ]; then cmux_ssh_attach_limit_format='"'"'[cmux] remote PTY bridge mad
e no progress after %s attempt; stopping retries.'"'"'; else cmux_ssh_attach_limit_form
at='"'"'[cmux] remote PTY bridge made no progress after %s attempts; stopping retries.'
"'"'; fi; printf '"'"'\n\033[31m%s\033[0m</code>
n'"'"' "$(printf "$cmux_ssh_attach_limit_format" "$cmux_ssh_attach_no_progress_limit")"
>&2
cmux_ssh_attach_retry_reason='"'"'SSH authentication/
control unavailable'"'"'; cmux_ssh_attach_no_progress_retry=0; : ;;
View Logs
cmux_ssh_attach_retry_reason='"'"'SSH authentication/
control unavailable'"'"'; cmux_ssh_attach_no_progress_retry=0; : ;;
View Logs

Fix with [code]smith
Need help on this PR? Tag @codesmith-bot with what you need.

rustybret pushed a commit to rustybret/bmux that referenced this pull request Sep 23, 2026
d726774 ci: default focused E2E dispatches to macOS 26 (manaflow-ai#13902)
6c7efe5 ci: reuse an in-flight focused run instead of dispatching over it (manaflow-ai#13901)
af221f0 Add bounded collector for dev app backend diagnostics (manaflow-ai#13910)
0f48984 ci: stop routing contributor prose to macOS and the release build (manaflow-ai#13905)
cd3ce57 test: respect build defaults in stable Cloud override assertions (manaflow-ai#13838)
197daa7 Fix default Codex ledger tilde expansion (manaflow-ai#13635)
e435dc0 fix: report the submitted prompt length, not the truncated preview's (manaflow-ai#13728)
9bd4c8d ci: route artifact transport helpers off the web and release lanes (manaflow-ai#13895)
7e72db9 Fix validation of unresolved workspace reorder targets (manaflow-ai#13843)
a9b0329 ci: gate native iOS work on package convention lint (manaflow-ai#13886)
bd50702 ci: skip docs deployment for standalone complexity policy (manaflow-ai#13887)
e786379 feat(cli): make workflow templates discoverable (manaflow-ai#13189)

# Conflicts:
#	.github/workflows/docs-channels.yml
#	.github/workflows/test-e2e.yml
#	.github/workflows/test-ios.yml
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

full-ci EXPENSIVE: full macOS tests/builds; overrides selective PR routing. Not needed for normal checks.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant