Repository navigation
base red: the test run-loop guard holds only the Swift Testing helper, never xctest (xctest signal 6 after every CmuxNext suite) - #33013
Merged
Conversation
swift test runs the bundle a second time in xctest after the Swift Testing tests. From 2026-11-02 that phase hung for about 10 minutes and died on signal 6 on CmuxNext suite steps. The hung stack (aws-m4pro-6, feat 305434b): XCTestDriver waits on its start-up future with a turn of the main run loop, and the run-loop guard's block, which never returns, runs in that turn. This runs an XCTest probe that waits the same way in a child xctest of this bundle and needs it to pass and exit 0 within 120 s. Red on the base on every host; the fix follows.
… helper, never xctest The guard (#25861) queues a block on the main run loop that never returns. That is right only in swiftpm-testing-helper, whose main is swift_task_asyncMainDrainQueue. swift test's XCTest phase loads the same bundle in xctest, where XCTestDriver waits on its start-up future with a turn of the main run loop: the guard's block ran in that turn, XCTest never ran its suite, and the process died on signal 6 about 10 minutes later (stack from aws-m4pro-6 at feat 305434b). Every CmuxNext suite step was red after its Swift Testing tests passed. The guard now installs nothing in a process that is not swiftpm-testing-helper. Its own regression test (TestProcessExitTests.aStopOfTheOutermostRunLoopDoesNotEndTheTestRun) runs in the helper and is unchanged.
lawrencecchen
temporarily deployed
to
ci-routing-agent
November 3, 2026 04:29 — with
GitHub Actions
Inactive
Contributor
|
cmux status (one comment, edited in place; each line is updated by the job that owns it) Review App of this head (app.zip, kept 1 day)
web and size checks: green Shipped in:
|
|
Found 1 test failure on Blacksmith runners: Failure
|
…phase Review of #33013: the child ran through the xcrun shim (SIP-protected, so the DYLD_* paths SwiftPM gives the phase were dropped) and without SWIFT_TESTING_ENABLED=0 (so XCTest could run Swift Testing tests in it, this test included). It now finds xctest with xcrun --find, runs it directly with the helper's environment and SWIFT_TESTING_ENABLED=0, as run_swift_test_bundle.py does. Comments in the header and Package.swift say the guard is in swiftpm-testing-helper only.
lawrencecchen
had a problem deploying
to
ci-routing-agent
November 3, 2026 04:42 — with
GitHub Actions
Error
lawrencecchen
temporarily deployed
to
ci-routing-agent
November 3, 2026 04:42 — with
GitHub Actions
Inactive
…onisolated This target defaults to the main actor; an XCTestCase subclass there does not compile (its initializers override nonisolated ones), and the test's file reads need not run on the main thread.
lawrencecchen
temporarily deployed
to
ci-routing-agent
November 3, 2026 04:44 — with
GitHub Actions
Inactive
lawrencecchen
added a commit
that referenced
this pull request
Nov 3, 2026
lawrencecchen
temporarily deployed
to
ci-routing-agent
November 3, 2026 04:44 — with
GitHub Actions
Inactive
Contributor
Author
|
Non-green checks at bc800c7, classified: all are base findings in files this PR does not touch.
|
Contributor
Author
|
Review (independent subagent): SHIP at bc800c7
|
Conflict in CmuxNextTestRunLoopGuard.c with #33023 (e512575), which stops the hold in a process whose argv has -XCTest. Resolved to one gate: the guard installs only in swiftpm-testing-helper. That covers #33023's xctest leg and every other host; runs_xctest_leg and its crt_externs include are removed, and its note is kept in the comment.
lawrencecchen
temporarily deployed
to
ci-routing-agent
November 3, 2026 06:22 — with
GitHub Actions
Inactive
lawrencecchen
temporarily deployed
to
ci-routing-agent
November 3, 2026 06:22 — with
GitHub Actions
Inactive
Contributor
Author
|
Review (independent subagent): SHIP at 11884af
|
This was referenced Nov 3, 2026
Merged
lawrencecchen
added a commit
that referenced
this pull request
Nov 3, 2026
…Test (#33276) A whole-package swift test starts its XCTest phase as plain 'xctest <bundle>' (swift-package-manager release/6.3, Sources/Commands/SwiftTestCommand.swift: xctestArgs returns no arguments with no filter, line 402; args(forTestAt:), lines 1008-1014); -XCTest is added only for a filter (line 938). #33023's argv check did not cover that start, so the regression test now runs its child both ways. Red at feat 145c5a3 (#33023, without #33013) plus this test (f54406d): the no -XCTest case fails at 120 s on aws-m4pro-6 (step 10ce589a) and aws-m4pro-8 (step 95f2953d).
This was referenced Nov 3, 2026
This branch was previously deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
![Fix with [code]smith](https://pr-comments-assets.blacksmith.sh/codesmith/fix-with-codesmith-light-3x.png)
Summary
Takes back #33023: its
-XCTestargv check in CmuxNextTestRunLoopGuard.c is replaced by a stricter gate (the guard installs only in swiftpm-testing-helper), which covers the same xctest leg and also the whole-packageswift testXCTest phase that startsxctest <bundle>with no-XCTest(SwiftPM release/6.3 SwiftTestCommand.swift lines 402, 1008-1014); nothing of #33023's behavior is lost.Base red on
feat-cmux-next(STATUS 8187): CmuxNext Swift test steps pass every Swift Testing test, then the XCTest phase ofswift test(xctest -XCTest '' …/CmuxNextPackageTests.xctest) hangs for about 10 minutes and ends with "unexpected signal code 6". Seen on aws-m4pro-1, -3, -4, -6, -9 and on Blacksmith macOS 26.Cause, from the stack of the hung
xctest(aws-m4pro-6, feat305434be55a1, step 3bd61ca1):The run-loop guard in
Tests/Support/CmuxNextTestExitTrace(#25861) queues a block on the main run loop that never returns. That is correct only inswiftpm-testing-helper, whosemainisswift_task_asyncMainDrainQueue.xctestloads the same bundle. When XCTest's start-up work is not done at once, XCTest turns the main run loop to wait for it. The guard's block runs in that turn and keeps the main thread, so XCTest never runs its suite. The host-dependent part is only how slow XCTest start-up is. On some hosts a Contacts store lookup inxctestfails for 22 s first: those are the CoreData 134020 / NSXPCConnection lines. On aws-m4pro-1 the same hang came with no CoreData line. No cmux code uses Contacts, AddressBook or CoreData. The CoreData lines are a symptom of a slow start-up, not the abort.Fix: the guard installs nothing (no observer, no source, no block) in a process that is not
swiftpm-testing-helper. In the helper, it works the same as before. Its own regression test (TestProcessExitTests.aStopOfTheOutermostRunLoopDoesNotEndTheTestRun) is unchanged.Testing
Regression test
XCTestPhaseTests.anXCTestWaitInTheXCTestPhaseReturns(red commit 980a603, before the fix): it runs a childxctestof this same bundle with an XCTest probe that waits on an expectation, the same way XCTest's start-up waits. It needs the probe to pass andxctestto exit 0 within 120 s. On the base, the guard keeps the child's main thread on every host, so the test does not depend on a slow start-up.Focused command (fleet):
cmux-ci run --class light --script scripts/ci/package-test-lane.sh --arg suite --arg Packages/macOS/CmuxNext --arg XCTestPhaseTests --arg EmptyLabelMeasureTests --ref SHA --target-worker WORKERRed:
fix/base-red-xctest-red-proofat69e5a3b7c330(base362be7b7d362plus the head'sXCTestPhaseTests.swift, without the one test that calls the new C function; never merge). Green: PR headbc800c77fbe0.aws-m4pro-6 failed on the base before (steps 3554f3fe, 3bd61ca1), and it fails at the red commit and passes at the head.
Observed / Not run
Observed: the hung xctest stack above (step 3bd61ca1, from the lane watchdog with a 240 s limit); the four steps in the table;
scripts/cmux-next/prepush-gate.pyon each push: no new violations. The base passed on aws-m4pro-7 and -9 in some runs (steps 7048809b, 65f92591): the base red needs a slow XCTest start-up, but the new test is red at the red commit on every host, because the probe's ownwait(for:)turns the run loop.Not run: the whole-package
swift test(the broad pass); Blacksmith macOS. I did not find what in xctest does the Contacts store lookup (a system framework; no cmux code uses Contacts, AddressBook or CoreData). With the fix, that lookup can only make the XCTest phase slower; it cannot hold it.Changelog
none
Proof
No capture: test-support code only; nothing in the app changes.