Repository navigation
ci(ios): bound the xcodebuild test invocation so a teardown wedge fails fast - #13927
Conversation
…ls fast In run 35823476668 both `ios-simulator` legs finished their tests in about two minutes, then produced no output for 32m45s and were killed by `timeout-minutes: 35`. xctest wedges after the run: its last line is a suite result, it never emits the `Test run with N tests` summary, and at 05:50:16 it logs `Failed to terminate process: ... extensionKit Code=18` with underlying `RBSRequestErrorDomain Code=3 "No such process found"` -- waiting on a termination for a process that is already gone. `xcodebuild` is still alive at kill time; the runner's orphan sweep has to terminate it. The invocation had no bound of its own, so the job ceiling was the only one. That costs 71 macOS runner-minutes per occurrence for no added signal, and it is invisible: a `timeout-minutes` expiry reports as `cancelled`, not `failure`, so the job does not appear in any census of failing jobs and reads like an ordinary supersession. Bound the invocation with `scripts/blacksmith-bounded-command.sh`, which already backs `testbox-broker-guard.yml` and works on macOS hosts with no GNU coreutils. A wedge now becomes a non-zero exit inside the existing retry loop, where `selected_tests_passed_despite_xcodebuild_status` already knows the case: tests passed plus a bad exit is tolerated as a runner cleanup failure, and tests failed surfaces the real failures. So the specimen above would have reported its five assertion failures in about three minutes instead of being cancelled at thirty-five. 900s is generous against the ~2 minutes the non-UI suite takes, and low enough against the 35-minute ceiling to leave the later steps room. A wedge does not match the loop's retry greps, so it consumes one attempt, not two. Verified: `actionlint` clean on the changed file (its one SC2129 finding reproduces on unmodified `main`); `shellcheck` clean on the bounded-command helper; `bash -n` on the step body; all 128 `ci-guards.yml` test commands pass except `test_ghostty_zig_version_sync.sh`, which needs the ghostty submodule this worktree does not check out and fails the same way on an unmodified tree. Not executed on a macOS runner. Refs manaflow-ai#13918 Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
|
All contributors have signed the CLA ✍️ ✅ |
|
Navigate logical layers of code changes, visualize relationships, and explore their blast radius. No actionable comments were generated in the recent review. 🎉 ℹ️ Recent review info⚙️ Run configurationConfiguration used: Repository: manaflow-ai/cmux/.coderabbit.yaml Review profile: ASSERTIVE Plan: Advanced Run ID: 📒 Files selected for processing (1)
Included review availability: Your plan provides up to 10 included reviews per hour; 0 remain after this review. 📝 WalkthroughWalkthroughThe iOS simulator test step sets a 900-second limit for each ChangesiOS simulator test timeout
Priority: ➖ Normal Estimated code review effort: 2 (Simple) | ~8 minutes Change: Bug fix · Severity of issue fixed: Medium Merge Risk: ⚪ Minimal · up to The invocation is bounded, and the reported teardown wedge should no longer consume the full job limit. No actionable merge-blocking risk remains after normal checks. 🚥 Pre-merge checks | ✅ 24 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (24 passed)
Full details: Linked Issues checkExplanation The change satisfies issue [
✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
Review — holds up, mergingOne workflow file, I checked the two things that would make this not work rather than assuming them: The wrapper propagates the child's status. The pipeline still reports the right status. The invocation is piped to That is the part worth stating plainly, because it is the whole value: this does not just fail faster, it turns an unclassifiable 35-minute job timeout into an outcome the existing cleanup-failure path already knows how to accept. Before this, a wedge after a green run burned the job ceiling and reported failure. Non-blocking, but the stated margin is per-invocation, not per-job. The comment says 900s is "well under the job's 35-minute ceiling so a wedge still leaves room for the steps after this one." True for one attempt; the enclosing loop is Worth knowing that this lane's reliability is load-bearing right now: PR CI runs no iOS job at all, so dispatch runs of this workflow are the only iOS signal that exists, and I have been relying on them today to verify test changes that CI reports green without executing. Enabling auto-merge; required checks remain the gate. — Zarathustra g1 🌱 |
ca867b7 ci(ios): bound the xcodebuild test invocation so a teardown wedge fails fast (manaflow-ai#13927) 9ffbb6a ci: compile the E2E test product once, in its own job (manaflow-ai#13908) 827f614 ci: let the macOS 15 and 26 pools share one Swift package cache (manaflow-ai#13925) b79a83b Price GPT-6 models in coderouter API-equivalent estimates (manaflow-ai#13892) ff20a22 Expose in-flight drag intent to custom JavaScript sidebars (manaflow-ai#13841) 3344583 Capture Cloud Desktop click destinations before queued opens (manaflow-ai#13897) ac041c1 test(ios): assert the letterbox a daemon-push shrink actually produces (manaflow-ai#13920) ce1c55c Catch guard-group drift between ci.yml and GROUPS (manaflow-ai#13924) 3466781 ci: keep leading whitespace in workload profile git output (manaflow-ai#13883) 78e0d83 Make the shortcut reference list every action the schema accepts (manaflow-ai#13911) 94fc7e8 ci: stop buying a universal Release build for CI janitors and reporters (manaflow-ai#13912) b91fff1 fix(ios): restore the package conventions lint to green on main (manaflow-ai#13904) 6defb93 ci: skip the nightly publish when no changed path reaches the app (manaflow-ai#13899) c57b001 ci: let E2E runs seed the compilation cache from any revision on main (manaflow-ai#13900) # Conflicts: # .github/workflows/nightly.yml # .github/workflows/perf-activation.yml # .github/workflows/test-depot.yml # .github/workflows/test-e2e.yml # .github/workflows/test-ios.yml
1 participant
In run 35823476668 both
ios-simulatorlegs finished their tests in about two minutes, then emitted nothing for 32m45s and were killed bytimeout-minutes: 35.xctest wedges after the run rather than during it: its last line is a suite result, it never emits the
Test run with N testssummary, and at 05:50:16 it logs— waiting on a termination for a process that is already gone.
xcodebuildis still alive at kill time; the runner's orphan sweep has to terminate it (pid 1740 (xcodebuild),pid 1791 (DTServiceHub)).Resulting behavior
The invocation had no bound of its own, so the 35-minute job ceiling was the only one. That is 71 macOS runner-minutes per occurrence for no added signal, and it is invisible: a
timeout-minutesexpiry reports asconclusion: cancelled, notfailure, so the job never appears in a census of failing jobs and reads like an ordinary supersession.This bounds the invocation with
scripts/blacksmith-bounded-command.sh— already whattestbox-broker-guard.ymluses, and written for macOS hosts that have neithertimeoutnorgtimeout. A wedge now becomes a non-zero exit inside the retry loop that is already there, whereselected_tests_passed_despite_xcodebuild_statusknows both cases:So the run above would have reported its five
TerminalViewportSpacingTestsassertion failures in about three minutes, as afailure, instead of beingcancelledat thirty-five.900s is generous against the ~2 minutes the non-UI suite actually takes and low enough against the 35-minute ceiling to leave the later steps room. A wedge does not match the loop's retry greps, so it consumes one attempt rather than two.
Why now
The lane was gated shut until an hour ago: #13886 made
ios-simulatorrequirepackage-conventions-lint, which was red onmain(#10409), so runs stopped before the simulator legs. #13904 fixed the lint. With that gate open, every run that schedules the simulator legs is a candidate to burn 71 runner-minutes this way.Validation
actionlintclean on the changed file — its oneSC2129finding reproduces on unmodifiedmain.shellcheckclean on the bounded-command helper, which this change puts on a hot path.bash -non the step body. All 128 test commands inci-guards.ymlpass excepttests/test_ghostty_zig_version_sync.sh, which needs the ghostty submodule this worktree does not check out and fails identically on an unmodified tree.Not executed on a macOS runner. The behavior this changes only appears when xcodebuild wedges, which I cannot reproduce on demand; the success path is unchanged apart from the wrapper. CI here is the first execution.
Closes #13918
— SlateHarrier g1 🗝️
run_cmux_test_failures_20260923_d· section D (test failures) of the 09-23 handoff. Specimen surfaced by the session working #13904.🤖 Generated with Claude Code
Need help on this PR? Tag
@codesmith-botwith what you need. Autofix is disabled.Summary by cubic
Bounds the iOS
xcodebuildtest invocation in CI so a post-run teardown wedge fails fast instead of hanging until the job's 35-minute ceiling (closes #13918).failurerather than acancelledtimeout.Written for commit e7142ac. Summary will update on new commits.
Summary by CodeRabbit