Skip to content

feat(code reviewer) better workspace capacity handling - #4775

Merged
St0rmz1 merged 5 commits into
mainfrom
feat/code-review-workspace-capacity-handling
Jul 26, 2026
Merged

feat(code reviewer) better workspace capacity handling#4775
St0rmz1 merged 5 commits into
mainfrom
feat/code-review-workspace-capacity-handling

Conversation

@St0rmz1

@St0rmz1 St0rmz1 commented Jul 26, 2026

Copy link
Copy Markdown
Contributor

Summary

When a code review starts on a sandbox whose disk is full, the workspace admission check rejects it. That rejection was reaching the pending message queue as an unclassified error, so it got a single retry about two seconds later and then failed, and it was recorded with no distinct terminal reason. This classifies the rejection as a workspace_setup_failed / sandbox_storage_full failure, gives it a longer backoff budget, and reports it under its own category.

The admission check and stale workspace cleanup logic are unchanged. This only changes what happens after a rejection.

Changes

  • orchestrator.ts: translate WorkspaceCapacityAdmissionRejectedError into a retryable ExecutionError.workspaceSetupFailed with subtype sandbox_storage_full, so it flows through the existing subtype classification instead of being wrapped as a generic wrapper start failure.
  • pending-messages.ts: add WORKSPACE_CAPACITY_RETRY_DELAYS_MS of 10s, 30s, 60s for capacity failures, replacing the single 2s retry they previously got. Each retry re-enters sandbox setup, which runs another stale workspace cleanup pass.
  • pending-messages.ts: extract isResetEligibleFailure and reset the flush attempt counter only when a message enters a reset-eligible mode from one that is not. Without this, failures alternating between sandbox connect and sandbox capacity would reset the counter each time and retry indefinitely.
  • Add workspace_capacity to CODE_REVIEW_TERMINAL_REASONS (packages/db), CloudAgentTerminalReason (packages/worker-utils), and InternalStatusTerminalReasonSchema (services/code-review-infra), with a comment on the last noting that a value missing there is silently coerced to undefined by .catch(undefined).
  • code-review-status/[reviewId]/route.ts: add isWorkspaceCapacityFailure, which prefers the structured failure code and subtype and falls back to matching the safe error message for payloads from workers that predate this change.
  • admin-code-reviews-router.ts: add Sandbox Capacity and Sandbox Connection categories to both the review and attempt error category expressions, plus Delivery Failure and an additional Action Required clause. These previously fell into Other or Unknown Error.

Verification

  • [ ]

Visual Changes

No component changes. The admin code reviews dashboard will show the new error categories in place of Other / Unknown Error for these failures.

Reviewer Notes

  • The retry budget is bounded at roughly 100 seconds across three attempts, after which the message is marked terminalization-pending and terminalizes with workspace_capacity.
  • There is no dispatch side concurrency limit. services/code-review-infra/src/index.ts creates a Durable Object per review and calls start() immediately, so a burst of reviews against one sandbox still arrives at once. This makes that failure recoverable in some cases and legible in reporting, but does not prevent it.
  • The message text fallback in isWorkspaceCapacityFailure and the LIKE clauses in the admin router overlap on purpose, to cover reviews recorded before the structured subtype was available.
  • Tests added for the orchestrator translation, the capacity retry budget, a fresh budget on entry, bounded retries when modes alternate, the status route detection, and the admin category bucketing.

@St0rmz1
St0rmz1 merged commit ba89e81 into main Jul 26, 2026
65 checks passed
@St0rmz1
St0rmz1 deleted the feat/code-review-workspace-capacity-handling branch July 26, 2026 05:09
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants