Skip to content

cmux-next: swift test log goes in the job's RUNNER_TEMP, not the shared /tmp - #23143

Merged
teamleaderleo merged 2 commits into
feat-cmux-nextfrom
chief-sidebar-2/swift-test-log-runner-temp
Oct 21, 2026
Merged

teamleaderleo merged 2 commits into
feat-cmux-nextfrom
chief-sidebar-2/swift-test-log-runner-temp

Conversation

@teamleaderleo

@teamleaderleo teamleaderleo commented Oct 21, 2026 •

Copy link
Copy Markdown
Collaborator

swift-test-summary.test.sh failed once on #21185's CI (cmux-next checks, job 119061489846), in the "crashed run" case:

a crashed run does not name crashesHere()
awk: fatal: cannot open file `/tmp/swift-test-log.8ynCVu' for reading: No such file or directory
sed: can't read /tmp/swift-test-log.8ynCVu: No such file or directory

The wrapper's temp log in /tmp disappeared before the wrapper reported, so the report could not name the unfinished test. The wrapper only deletes its own log, on EXIT, and no script in the repo sweeps /tmp/swift-test-log.*, so the delete probably came from outside the run, most likely one of the ~32 script tests the checks job runs at once (run-checks-concurrently.sh). I did not find the deleter. This is a flake, not a steady base red: the six later feat-cmux-next runs of cmux-next checks all print swift-test-summary.test.sh: ok.

Fix:

  • The wrapper keeps its log in $RUNNER_TEMP (the job's own temp dir) when it's set, falling back to $TMPDIR or /tmp as before. The wrapper's crash reports already go to $RUNNER_TEMP. This helps if the deleter targets /tmp, which is likely but unproven.
  • Found in review: if mktemp failed, $log was empty and the EXIT trap ran rm -f "" .*, deleting every dotfile in the working directory. The wrapper now stops with an ::error instead.

This is separate from #23129, which fixes a different flake in the same test (the crash-report mtime in the signal case).

Observed (macOS):

  • bash scripts/cmux-next/tests/swift-test-summary.test.sh passes with RUNNER_TEMP unset and with it set to a fresh directory.
  • With RUNNER_TEMP=/nonexistent/x, the wrapper exits 1 with the ::error and leaves the working directory's dotfiles alone.

The cmux-next CI guard group red fails the same set of workflow-guard tests on base; it is not this PR's.

🤖 Generated with Claude Code

Agent session: cosmic-gray

…ed /tmp

swift-test-summary.test.sh failed once on #21185's CI: the wrapper's
/tmp/swift-test-log.* vanished mid-run, so the report could not name the
unfinished test. Six later feat-cmux-next runs pass it. The crash reports
already go to $RUNNER_TEMP; the log now does too.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Agent-Session: https://cmux.com/deeplink/agent?harness=claude&session=01a15d18-9c97-76c3-88d7-6d0422d2fadc&host=cosmic-gray
@github-actions

Copy link
Copy Markdown
Contributor

cmux status (one comment, edited in place; each line is updated by the job that owns it)

Review 52b1cdf: HQ feed card · gallery before/after runs · checks

App of this head (app.zip, kept 1 day)

gh workflow run reload-build.yml -R manaflow-ai/cmux --ref feat-cmux-next -f tag=pr23143 -f ref=chief-sidebar-2/swift-test-log-runner-temp -f platform=macos -f backend_mode=local

@teamleaderleo

teamleaderleo commented Oct 21, 2026 •

Copy link
Copy Markdown
Collaborator Author

Owned by projects-ee on cosmic-gray · session

@blacksmith-sh

blacksmith-sh Bot commented Oct 21, 2026

Copy link
Copy Markdown

Found 27 test failures on Blacksmith runners:

Failures

Test View Logs
.github/workflows/
nightly-next-tick.yml: triggers on the display names ['cmux-next', 'cmux-tui artifacts'
] and declares no SOURCE_WORKFLOW_PATHS, so renaming any of those workflows stops this
one from ever triggering and nothing reports it
View Logs
main.IOSWiring.test_test_ios_macos_jobs_take_the_runner_jobs_pool/
test_test_ios_macos_jobs_take_the_runner_jobs_pool
View Logs
main.SideLanes.test_a_completed_failed_blacksmith_retry_is_not_watched_again/
test_a_completed_failed_blacksmith_retry_is_not_watched_again
View Logs
main.SideLanes.test_a_real_failure_on_cmux_next_attempt_two_is_not_rescued/
test_a_real_failure_on_cmux_next_attempt_two_is_not_rescued
View Logs
cmux-next.yml job swift-canary step 'Report the build fleet compile verdict' uses the t
oolchain (swift) before scripts/select-ci-xcode.sh runs
View Logs
debug info {'CMUX_SWIFT_SANITIZE': 'address', 'CMUX_SWIFT_TEST_DEBUG_INFO': 'none'}: 's
wift --version' lacks '-debug-info-format none'/
debug info {'CMUX_SWIFT_SANITIZE': 'address', 'CMUX_SWIFT_TEST_DEBUG_INFO': 'none'}: 's
wift --version' lacks '-debug-info-format none'
View Logs
debug info {'CMUX_SWIFT_SANITIZE': 'address', 'CMUX_SWIFT_TEST_DEBUG_INFO': 'none'}: 's
wift package resolve --package-path Packages/macOS/Pkg --scratch-path Packages/macOS/
Pkg/.build-sanitize-address' lacks '-debug-info-format none'
View Logs
debug info {'CMUX_SWIFT_SANITIZE': 'address', 'CMUX_SWIFT_TEST_DEBUG_INFO': 'none'}: wa
nt one build and two suites: ['swift package resolve --package-path Packages/macOS/
Pkg --scratch-path Packages/macOS/Pkg/
.build-sanitize-address', 'swift --version', 'swift build --build-tests -c debug -debug
-info-format none --sanitize=address --scratch-path Packages/macOS/Pkg/
.build-sanitize-address -Xlinker -no_compact_unwind --package-path Packages/macOS/
Pkg', 'swift test -c debug -debug-info-format none --sanitize=address --scratch-path Pa
ckages/macOS/Pkg/
.build-sanitize-address -Xlinker -no_compact_unwind --package-path Packages/macOS/
Pkg --skip-build --filter ATests', 'swift test -c debug -debug-info-format none --sanit
ize=address --scratch-path Packages/macOS/Pkg/
.build-sanitize-address -Xlinker -no_compact_unwind --package-path Packages/macOS/
Pkg --skip-build --filter BTests']
View Logs
debug info {'CMUX_SWIFT_SANITIZE': 'address'}: want one build and two suites: ['swift p
ackage resolve --package-path Packages/macOS/Pkg --scratch-path Packages/macOS/Pkg/
.build-sanitize-address', 'swift --version', 'swift build --build-tests -c debug --sani
tize=address --scratch-path Packages/macOS/Pkg/
.build-sanitize-address -Xlinker -no_compact_unwind --package-path Packages/macOS/
Pkg', 'swift test -c debug --sanitize=address --scratch-path Packages/macOS/Pkg/
.build-sanitize-address -Xlinker -no_compact_unwind --package-path Packages/macOS/
Pkg --skip-build --filter ATests', 'swift test -c debug --sanitize=address --scratch-pa
th Packages/macOS/Pkg/
.build-sanitize-address -Xlinker -no_compact_unwind --package-path Packages/macOS/
Pkg --skip-build --filter BTests']
View Logs
debug info {'CMUX_SWIFT_TEST_DEBUG_INFO': 'dwarf'}: want one build and two suites: ['sw
ift package resolve --package-path Packages/macOS/
Pkg', 'swift --version', 'swift build --build-tests -c debug --package-path Packages/
macOS/Pkg', 'swift test -c debug --package-path Packages/macOS/
Pkg --skip-build --filter ATests', 'swift test -c debug --package-path Packages/macOS/
Pkg --skip-build --filter BTests']
View Logs

...and 17 more test failures. View all on Blacksmith

Fix with [code]smith
Need help on this PR? Tag @codesmith-bot with what you need.

An empty $log made the EXIT trap rm -f "" .* (every dotfile in the working
directory). Found by the review of this PR.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Agent-Session: https://cmux.com/deeplink/agent?harness=claude&session=01a15d18-9c97-76c3-88d7-6d0422d2fadc&host=cosmic-gray
@cursor

cursor Bot commented Oct 21, 2026

Copy link
Copy Markdown

Bugbot needs on-demand usage enabled

Bugbot uses usage-based billing for this team and requires on-demand usage to be enabled.

A team admin can enable on-demand usage in the Cursor dashboard.

@teamleaderleo

Copy link
Copy Markdown
Collaborator Author

Bead cx-piva0 created for this PR by the cmux coordinator.

@teamleaderleo

Copy link
Copy Markdown
Collaborator Author

merge-audit: #23143 merged at 024a53b without no SHIP review receipt on the head; no risk tier label. #23292 reverts it. To keep this change, add the missing evidence for 024a53b here; the next audit pass then closes #23292. Otherwise review and land #23292.

@github-actions

Copy link
Copy Markdown
Contributor

Shipped in:

teamleaderleo added a commit that referenced this pull request Oct 21, 2026
#23290)

The watcher subshell inherited the EXIT trap. Killed before its first line ran
(a run that ended at once on a loaded host), it ran the trap and deleted the log
before the report read it: swift-test-summary.test.sh failed about 1 in 10 runs
at 32 checks at once (reproduced on huge-blue: 9/96, 17/192; with this change
0/192). #23143 moved the log out of /tmp on a wrong guess; its mktemp guard stays.


Agent-Session: https://cmux.com/deeplink/agent?harness=claude&session=01a15d18-9c97-76c3-88d7-6d0422d2fadc&host=cosmic-gray

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
@teamleaderleo

Copy link
Copy Markdown
Collaborator Author

Review (subagent, correctness first): SHIP at 024a53b

This is a post-hoc independent review of the merged change (merge commit cc347e0). It was read-only and nothing was run. risk:low (CI wrapper script only).

Nothing blocks. Notes:

The merge-audit revert of this PR is handled separately. See leo-workspace briefs/2026-10-20-gui-push/revert-triage.md.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant