Skip to content

test(cmux-next): the XCTest-phase regression test also starts xctest without -XCTest (follow-up of #33013) - #33276

Merged
lawrencecchen merged 1 commit into
feat-cmux-nextfrom
fix/xctest-phase-test-no-flag
Nov 3, 2026
Merged

lawrencecchen merged 1 commit into
feat-cmux-nextfrom
fix/xctest-phase-test-no-flag

Conversation

@lawrencecchen

@lawrencecchen lawrencecchen commented Nov 3, 2026 •

Copy link
Copy Markdown
Contributor

Summary

Follow-up of #33013 (the xctest signal 6 base red). XCTestPhaseTests.anXCTestWaitInTheXCTestPhaseReturns now starts its child xctest in both of the ways SwiftPM starts the XCTest phase, not only with -XCTest.

Where SwiftPM decides (swift-package-manager release/6.3, commit f096e728bff0, Sources/Commands/SwiftTestCommand.swift):

  • With no --filter (a whole-package swift test), xctestArgs returns no arguments: return ([], nil) at line 402. args(forTestAt:) then starts [xctestPath] (line 1008), then the additional arguments, which are empty here (line 1013), then the bundle path (line 1014). The result is plain xctest <bundle>.
  • With a filter, xctestArguments adds ["-XCTest", <tests>] (line 938).

#33023 stopped the guard's hold only when the command line has -XCTest, so it did not cover the whole-package start. #33013 limits the guard to swiftpm-testing-helper, which covers both. This test now shows that difference.

XCTestPhaseRunLoopProbe is the only XCTestCase in the bundle, so the child without -XCTest also runs exactly one test. A comment says that it must stay the only one.

Testing

Focused command (fleet): cmux-ci run --class light --script scripts/ci/package-test-lane.sh --arg suite --arg Packages/macOS/CmuxNext --arg XCTestPhaseTests --ref SHA --target-worker WORKER

commit worker step result
red f54406d7500b (feat 145c5a35b809: has #33023, not #33013; plus this test without the one unit test that needs #33013's C function; never merge) aws-m4pro-6 10ce589a the case selection → [] fails: child killed at 120 s, its only output is the guard's "holding the main run loop open" line; the -XCTest case passes; exit 1
red f54406d7500b aws-m4pro-8 95f2953d same; exit 1
green e21feea (this head) aws-m4pro-6 a72579af both cases pass (0.171 s); TestProcessExitTests passes; exit 0
green e21feea cmux-austin-mini-0 (pinned to aws-m4pro-8; the controller moved it, because aws-m4pro-8 was paused) 50761b09 still running when this was written; not counted

Observed / Not run

Observed: the steps above. prepush-gate.py: no new violations. I read the SwiftPM source at the lines above.

Not run: a real whole-package swift test on a slow host. The child test runs the same xctest <bundle> start.

Changelog

none

Proof

No capture: test code only.

…Test

A whole-package swift test starts its XCTest phase as plain
'xctest <bundle>' (swift-package-manager release/6.3,
Sources/Commands/SwiftTestCommand.swift: xctestArgs returns no
arguments with no filter, line 402; args(forTestAt:), lines
1008-1014); -XCTest is added only for a filter (line 938). #33023's
argv check did not cover that start, so the regression test now runs
its child both ways.

Red at feat 145c5a3 (#33023, without #33013) plus this test
(f54406d): the no -XCTest case fails at 120 s on aws-m4pro-6
(step 10ce589a) and aws-m4pro-8 (step 95f2953d).
@lawrencecchen lawrencecchen added the risk:low Risk tier (docs/ci/risk-tiers.md) label Nov 3, 2026
@cursor

cursor Bot commented Nov 3, 2026

Copy link
Copy Markdown

Bugbot needs on-demand usage enabled

Bugbot uses usage-based billing for this team and requires on-demand usage to be enabled.

A team admin can enable on-demand usage in the Cursor dashboard.

@github-actions

github-actions Bot commented Nov 3, 2026 •

Copy link
Copy Markdown
Contributor

cmux status (one comment, edited in place; each line is updated by the job that owns it)

Review e21feea: HQ feed card · gallery before/after runs · checks

App of this head (app.zip, kept 1 day)

gh workflow run reload-build.yml -R manaflow-ai/cmux --ref feat-cmux-next -f tag=pr33276 -f ref=fix/xctest-phase-test-no-flag -f platform=macos -f backend_mode=local

Shipped in:

@blacksmith-sh

blacksmith-sh Bot commented Nov 3, 2026

Copy link
Copy Markdown

Found 1 test failure on Blacksmith runners:

Failure

Test View Logs
build_only must be a conjunctive exclusion on the helper, signing, and publication jobs
, and must not gate the unsigned app build/
build_only must be a conjunctive exclusion on the helper, signing, and publication jobs
, and must not gate the unsigned app build
View Logs

Fix with [code]smith
Need help on this PR? Tag @codesmith-bot with what you need.

@lawrencecchen

Copy link
Copy Markdown
Contributor Author

Review (independent subagent): SHIP at e21feea

  • The parameterized test (arguments: [["-XCTest", "CmuxNextAppTests.XCTestPhaseRunLoopProbe"], []]) has valid syntax. Green step a72579af built it at this head and ran both cases. .minutes(5) is enough for 60 s of xcrun --find plus 120 s for the child. The two parallel children write to separate log files.
  • The child started without -XCTest is safe. XCTestPhaseRunLoopProbe (XCTestPhaseTests.swift:124) is the only XCTestCase in CmuxNext. SWIFT_TESTING_ENABLED=0 keeps the Swift Testing tests out of the child, so "Executed 1 test, with 0 failures" holds.
  • The SwiftPM citations are right at release/6.3 f096e728bff0: line 402 return ([], nil), line 938 return ["-XCTest", testSpecifier], lines 1008/1013/1014 build xctest + extra arguments + bundle. Nit: line 402 applies only when there is no filter and no --skip.
  • The red is valid. f54406d is feat 145c5a3 (test(next): the run loop guard never holds the XCTest leg's main loop (signal-6 abort on some fleet Macs) #33023, without base red: the test run-loop guard holds only the Swift Testing helper, never xctest (xctest signal 6 after every CmuxNext suite) #33013) plus the test file only. Only the [] case fails, at 120 s with the guard's "holding" line, on aws-m4pro-6 (10ce589a, exit 1) and aws-m4pro-8 (95f2953d, exit 1). The -XCTest case passes there.
  • The green is valid on aws-m4pro-6 (a72579af): both cases pass, exit 0. The second green step (50761b09) was moved to cmux-austin-mini-0 and has no result yet. One green host is enough for risk:low.
  • Not blocking: the fixed "Executed 1 test" count breaks if someone adds a second XCTestCase. A comment says the probe must stay the only one.

@lawrencecchen lawrencecchen added the ready-to-land Reviewed at this head; next-autoland merges it into feat-cmux-next when its only reds are base reds label Nov 3, 2026
@lawrencecchen
lawrencecchen merged commit 4a0b593 into feat-cmux-next Nov 3, 2026
227 of 252 checks passed
@lawrencecchen
lawrencecchen deleted the fix/xctest-phase-test-no-flag branch November 3, 2026 08:44

This branch was previously deployed

1 inactive deployment
ci-routing-agent — e21feea8 Deployed Nov 3, 2026 by lawrencecchen via wait for same-key tree publisher #104654
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ready-to-land Reviewed at this head; next-autoland merges it into feat-cmux-next when its only reds are base reds risk:low Risk tier (docs/ci/risk-tiers.md)

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant