Repository navigation
fix(sse): treat antigravity empty completions with a normal stop as valid 200s (#14160) - #14243
Conversation
…alid 200s (cheaper-inference#14160) An empty completion from antigravity's Gemini can be a real answer: some prompts legitimately produce no text, and the upstream reports a normal terminal finish reason (STOP -> "stop"). The fake-success guard in isEmptyContentResponse flagged these regardless, so the non-streaming leg rewrote them into synthetic 502s that fed model lockout — a few hundred such "failures" a day kept most of the reporter's 22-connection pool excluded and starved unrelated clients. The guard exists for free-tier/scraping providers whose failure mode is an empty 200 shell (cheaper-inference#13461), so scope the exemption the same way the repo scopes classifyFakeSuccessBody: a trusted-provider allowlist. On antigravity, an empty completion with a normal stop reason (openai "stop", claude "end_turn") now passes through as a valid 200; every other provider keeps the existing guard, and antigravity responses with no terminal stop reason are still flagged.
|
This is a well-scoped fix — reusing the same trusted-provider-allowlist shape as classifyFakeSuccessBody rather than inventing a new mechanism, and keeping every other provider's guard untouched. The 5-case test (both wire shapes, an untrusted-provider control, and the no-terminal-reason control) covers exactly the boundary that matters. Good root-cause tie-back to #13461's original intent too. Optional follow-up question: |
|
I checked the two providers separately.
One nuance worth confirming: the allowlist matches on the provider id, not the wire format, so adding "gemini" does not loosen the guard for the other format: "gemini" entries (hcnsec, vertex, vertex/partner). Those stay guarded, which is what we want for third-party gateways. If that matches your intent, I will add "gemini" to TRUSTED_EMPTY_STOP_PROVIDERS and mirror the test: a gemini empty-stop case asserting a valid 200, plus an hcnsec control (same wire format, untrusted) asserting it still gets flagged. Otherwise I am fine keeping this antigravity-only and widening on a confirmed gemini report. |
|
Thanks @xiechimon — merging via the release merge-train. Validated in local merge-train (mt-train10c) on the devbox @ train tip 4d841aa1c740bbaa03868dc0a403c62099a99a42 with the 72 sibling PRs of this batch: typecheck:core, file-size, complexity, cognitive-complexity, changelog-integrity green; changed-area node:test 831/831 (0 failing) and vitest 480/482 — the two reds are |
c45884c
into
cheaper-inference:release/v3.8.51
What
An empty (but successful) completion from antigravity's Gemini is treated as a provider failure: the client gets
502 Provider returned empty contentand the model is locked out on that account. The reporter sees ~400 of these a day from a single observer prompt the model legitimately answers with no text, which keeps most of a 22-connection pool excluded and starves unrelated clients (their words: a cheap harmless request was starving the expensive ones).The fake-success guard exists for free-tier/scraping providers whose failure mode is an empty 200 shell (#13461). Antigravity is Google's first-party Gemini API: an empty candidate with
finishReason: STOPis a real result, and retrying the same prompt on the next account returns the same empty answer anyway.Changes
isEmptyContentResponse()gains an optionalproviderand aTRUSTED_EMPTY_STOP_PROVIDERSallowlist (same pattern as the provider-allowlistedclassifyFakeSuccessBodyfrom [BUG] Gateway returning bad response #13461). On antigravity, an empty completion with a normal terminal stop reason (openaistop, claudeend_turn) is no longer flagged.providerthrough, so these completions flow to the client as a valid 200 with empty content instead of a synthetic 502. No failure accounting, no model lockout, and no retrying the same prompt on the next account to get the same empty answer.Unchanged: every other provider keeps the guard, including on empty stop completions; antigravity responses with no terminal stop reason are still flagged; the existing legit-empty carve-outs (
length,tool_calls,content_filter,max_tokens,tool_use) are untouched.Verification
tests/unit/14160-antigravity-empty-stop-completion.test.ts, 5 cases: antigravity empty+stop and empty+end_turn pass through; untrusted providers and the no-provider call keep flagging them; an antigravity empty shell with no finish reason is still flagged; existing carve-outs unchanged.combo-empty-content-failover-5085,diagnostics-fake-success-13461,copilot-m365-web-silent-empty-7858,zai-web-silent-empty-repro).prettier --check,eslint,typecheck:coreclean on all touched files.Out of scope: the streaming variant (#13600) is a different code path (
streamEmptyChoices.ts) and is not touched here. If you'd rather have the passthrough behind a settings toggle, happy to add one.Fixes #14160