Repository navigation
fix(sse): wait out in-flight duplicate turns instead of failing them with 409 - #18717
hongkongkiwi wants to merge 2 commits into
Conversation
|
Superseded by #18778. This PR's wait is kept in #18778 for genuinely concurrent foreign duplicates. What it missed: the blocked caller in the #18716 incident is usually the same client request re-entering chatCore (same-account retry after a pre-output failure, account rotation, combo legs), and waiting does not save it when the original outlives the 60s bound. Production run on 2026-11-02: the original held 118s, the same-account retry spun 116 acquire attempts, and the 409 landed 3s after the original released. #18778 adds a per-request ownership scope so those re-entries reuse the slot immediately, and keeps this PR's wait-and-reacquire for foreign duplicates. Closing this in favour of that one; the wait tests moved over with two added ownership tests. |
Summary
acquireTurnExecutionnow has a wait-and-reacquire wrapper,acquireTurnExecutionWithWait: when the turn slot for an effective idempotency key is held by an in-flight turn, the caller waits (bounded, default 60s, 500ms poll) for the original to release and then acquires. After the bound it returns the same blocked result as before, so the existingturn_in_progress409 path is unchanged as the fallback.handleChatCoreuses the wrapper for the turn guard. The duplicate-blocked branch, its log line, and the 409 body are untouched.stream_error(and the falseall accounts rate limited (reset after Ns)cooldown cascade driven by the 409) into a short serialized wait. Rationale and production logs in fix(resilience): Turn guard 409s internal retries against a stuck in-flight turn, producing a fatal client-facing stream error and false account cooldowns #18716.Related Issues
Target Branch
release/v3.8.xbranch (release/v3.8.53)Validation
npx tsx --test tests/unit/chatcore/*.test.ts(35/35 pass, including 5 new tests)npx eslintclean on the three touched filesrelease/v3.8.53today; the guard call site is unchanged upstream since 3.8.52)Tests Added Or Updated
tests/unit/chatcore/turn-execution-guard-wait.test.ts(new): immediate acquire, null-key no-op, waits then acquires after release, bounded fallback to the blocked result, abort stops the wait.Coverage Notes
open-sse/handlers/chatCore/turnExecutionGuard.tsand the guard call site inopen-sse/handlers/chatCore.tsare covered by the new test file. The deployed instance of this patch has been running since 2026-11-01; the duplicated-request path now waits ("duplicate waiting" log) instead of returning 409.Reviewer Notes
clientRawRequestin chatCore carries no abort signal, so the call site passes no signal; a disconnected caller's wait is bounded by the same cap.retryCount(it only feeds theX-OmniRoute-Turn-Retryheader on the fallback 409).