test: fail fast instead of relaunching the app host on gated Cloud waits - #13664
Conversation
Every app-host restart in the 09-21/09-22 shard logs I sampled was a Swift Testing time-limit hit in one of two suites, and each hit relaunches the test host: - SurfaceCatalogTests: when catalog.project threw before reaching the provider (destinationNotFound, fixed by 3dc91b2), each gated test parked in MaterializeGate.waitUntilEntered() until the 300 s limit. Five tests restarted the host one after another, about 25 minutes per shard. - CloudDisplayCatalogTests: when the fake no longer recognized the create command (fixed by 70eaf44), create() failed before the exec started and `await started.result` parked until the 60 s limit. The waits now also end when the caller's task finishes, so the next such setup failure is an ordinary failed #require. The cancellation test also waits on its own cancellation signal instead of a 60 s Task.sleep. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
|
Important Review skippedAuto reviews are disabled on base/target branches other than the default branch. Please check the settings in the CodeRabbit UI or the ⚙️ Run configurationConfiguration used: Repository: manaflow-ai/cmux/.coderabbit.yaml Review profile: ASSERTIVE Plan: Advanced Run ID: You can disable this status message by setting the Use the checkbox below for a quick retry:
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
2f6adcc
into
manaflow-ai:fix/app-host-green
Stops two suites from relaunching the app-host test process. Both hangs had a real cause that is already fixed on
fix/app-host-green. This PR makes the next setup failure of the same kind fail right away instead of hanging.I sampled 14 app-host shard logs (09-20 to 09-22). Every "Restarting after unexpected exit, crash, or test timeout" in them came right after a Swift Testing
Time limit was exceededin one of the suites below. The tests named in the restart census (menuTrackingReconciliation…,commandTimeoutLeavesRecoverable…,cancellationDiscardsAProjection…, and so on) were bystanders: they happened to be the last tests started before the relaunch.SurfaceCatalogTests (restarts 5 times per shard, about 25 minutes)
SurfaceCatalogTests.swift:561/594/618/662/688: Time limit was exceeded: 300.000 seconds. The tests hit it one after another: "Concurrent reuse…", "An adopted projection…", "Cancelling the last project caller…", "Cancellation at provider completion…", "A removed local resource…".catalog.projectthrewdestinationNotFoundbefore it reached the fake provider (fixed by 3dc91b2). The test then waited inMaterializeGate.waitUntilEntered(), which only the provider could resume.waitUntilEntered(orEndOf:)also returns when the caller's task finishes. Each call site now usestry #require(...).CloudDisplayCatalogTests (restarts 2 times per shard)
CloudDisplayCatalogTests.swift:41and:99: Time limit was exceeded: 60.000 seconds.create()failed before the exec started (fixed by 70eaf44).await started.resultthen waited forever.guestExecStarted(_:before:)resolvesfalsewhen the operation ends first. The cancellation test now waits for its exec to be cancelled instead of running a 60 sTask.sleep.Checks:
swiftc -parse,scripts/check-test-determinism.py,tests/test_ci_pbxproj_test_wiring.sh. No pbxproj changes.🤖 Generated with Claude Code
Need help on this PR? Tag
@codesmith-botwith what you need. Autofix is disabled.Summary by cubic
Prevents two test suites from hanging and forcing the app host to restart when a gated Cloud setup fails. The waits now end when the caller's task finishes, so the next such failure is an ordinary failed
#requireinstead of a suite-level time limit.SurfaceCatalogTests:waitUntilEnterednow also resolves when the caller's task ends; each call site usestry #require.CloudDisplayCatalogTests:guestExecStartedreturnsfalsewhen the operation finishes first; the cancellation test waits for its exec to be cancelled instead of sleeping 60 seconds.The root causes of the hangs were already fixed in
fix/app-host-green; these changes make those tests fail fast if the same setup regression returns.Written for commit 4df563e. Summary will update on new commits.