Skip to content

test: fail fast instead of relaunching the app host on gated Cloud waits - #13664

Merged
teamleaderleo merged 1 commit into
manaflow-ai:fix/app-host-greenfrom
teamleaderleo:fix/app-host-restart-waits
Sep 22, 2026
Merged

teamleaderleo merged 1 commit into
manaflow-ai:fix/app-host-greenfrom
teamleaderleo:fix/app-host-restart-waits

Conversation

@teamleaderleo

@teamleaderleo teamleaderleo commented Sep 22, 2026 •

Copy link
Copy Markdown
Collaborator

Stops two suites from relaunching the app-host test process. Both hangs had a real cause that is already fixed on fix/app-host-green. This PR makes the next setup failure of the same kind fail right away instead of hanging.

I sampled 14 app-host shard logs (09-20 to 09-22). Every "Restarting after unexpected exit, crash, or test timeout" in them came right after a Swift Testing Time limit was exceeded in one of the suites below. The tests named in the restart census (menuTrackingReconciliation…, commandTimeoutLeavesRecoverable…, cancellationDiscardsAProjection…, and so on) were bystanders: they happened to be the last tests started before the relaunch.

SurfaceCatalogTests (restarts 5 times per shard, about 25 minutes)

  • Signature: SurfaceCatalogTests.swift:561/594/618/662/688: Time limit was exceeded: 300.000 seconds. The tests hit it one after another: "Concurrent reuse…", "An adopted projection…", "Cancelling the last project caller…", "Cancellation at provider completion…", "A removed local resource…".
  • Root cause: catalog.project threw destinationNotFound before it reached the fake provider (fixed by 3dc91b2). The test then waited in MaterializeGate.waitUntilEntered(), which only the provider could resume.
  • Fix: waitUntilEntered(orEndOf:) also returns when the caller's task finishes. Each call site now uses try #require(...).

CloudDisplayCatalogTests (restarts 2 times per shard)

  • Signature: CloudDisplayCatalogTests.swift:41 and :99: Time limit was exceeded: 60.000 seconds.
  • Root cause: the fake no longer recognized the create command, so create() failed before the exec started (fixed by 70eaf44). await started.result then waited forever.
  • Fix: guestExecStarted(_:before:) resolves false when the operation ends first. The cancellation test now waits for its exec to be cancelled instead of running a 60 s Task.sleep.

Checks: swiftc -parse, scripts/check-test-determinism.py, tests/test_ci_pbxproj_test_wiring.sh. No pbxproj changes.

🤖 Generated with Claude Code


View with [code]smith Autofix with [code]smith
Need help on this PR? Tag @codesmith-bot with what you need. Autofix is disabled.


Summary by cubic

Prevents two test suites from hanging and forcing the app host to restart when a gated Cloud setup fails. The waits now end when the caller's task finishes, so the next such failure is an ordinary failed #require instead of a suite-level time limit.

  • SurfaceCatalogTests: waitUntilEntered now also resolves when the caller's task ends; each call site uses try #require.
  • CloudDisplayCatalogTests: guestExecStarted returns false when the operation finishes first; the cancellation test waits for its exec to be cancelled instead of sleeping 60 seconds.

The root causes of the hangs were already fixed in fix/app-host-green; these changes make those tests fail fast if the same setup regression returns.

Written for commit 4df563e. Summary will update on new commits.

Review in cubic

Every app-host restart in the 09-21/09-22 shard logs I sampled was a Swift
Testing time-limit hit in one of two suites, and each hit relaunches the
test host:

- SurfaceCatalogTests: when catalog.project threw before reaching the
  provider (destinationNotFound, fixed by 3dc91b2), each gated test
  parked in MaterializeGate.waitUntilEntered() until the 300 s limit.
  Five tests restarted the host one after another, about 25 minutes per shard.
- CloudDisplayCatalogTests: when the fake no longer recognized the create
  command (fixed by 70eaf44), create() failed before the exec started
  and `await started.result` parked until the 60 s limit.

The waits now also end when the caller's task finishes, so the next such
setup failure is an ordinary failed #require. The cancellation test also
waits on its own cancellation signal instead of a 60 s Task.sleep.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@coderabbitai

coderabbitai Bot commented Sep 22, 2026

Copy link
Copy Markdown

Important

Review skipped

Auto reviews are disabled on base/target branches other than the default branch.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Repository: manaflow-ai/cmux/.coderabbit.yaml

Review profile: ASSERTIVE

Plan: Advanced

Run ID: 8f8325a4-3be0-4186-a124-a74712799d38

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • 🔍 Trigger review

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@teamleaderleo
teamleaderleo merged commit 2f6adcc into manaflow-ai:fix/app-host-green Sep 22, 2026
26 checks passed
@teamleaderleo
teamleaderleo deleted the fix/app-host-restart-waits branch September 23, 2026 11:35
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant