Skip to content

fix(antigravity): forward the trailing OpenAI usage-only chunk as usageMetadata - #18643

Open
AndresDiagoM wants to merge 3 commits into
cheaper-inference:release/v3.8.53from
AndresDiagoM:fix/antigravity-usage-only-chunk
Open

AndresDiagoM wants to merge 3 commits into
cheaper-inference:release/v3.8.53from
AndresDiagoM:fix/antigravity-usage-only-chunk

Conversation

@AndresDiagoM

Copy link
Copy Markdown
Contributor

⚠️ base-red inherited: #17219

Summary

  • Problem. OpenAI-compatible upstreams called with stream_options.include_usage send usage in a separate final chunk with choices: [], after the finish chunk (fix(sse): estimate usage in passthrough stream even with include_usage #12151). open-sse/translator/response/openai-to-antigravity.ts handles a chunk without choices[0] by stashing state._usage and returning null (L27-33 on the tip). The stash is only read when a later chunk with a choice builds a response, and after the finish chunk none comes. The stream-end flush does not rescue it either: stream.ts (L3174, L3218) calls translateResponse(..., null, state), and the translator returns null for a null chunk. So the usage never reaches the client: no usageMetadata, no prompt/candidate/total counts, no thoughts or cached tokens.

  • Scope. The same function is registered for OPENAI → GEMINI too (openai-to-gemini.ts:14), so both Antigravity and Gemini client formats lose usage this way. fix(gemini): forward OpenAI usage-only chunks and cached/thought token counts to /v1beta clients #17842 fixed the same gap in the /v1beta SSE converter (openai-to-gemini-sse.ts); this is the sibling path.

  • Fix (same approach as fix(gemini): forward OpenAI usage-only chunks and cached/thought token counts to /v1beta clients #17842). A small helper, usageOnlyChunkResponse, handles the no-choices case:

    • Usage before the finish chunk: stashed for the finish chunk, as before.
    • Usage after the finish chunk: emitted as its own chunk (empty text part, no finishReason, usageMetadata, and the stream's modelVersion / responseId).
    • Usage already sent: dropped, so it is never counted twice.

    The inline usage mapping is replaced by the shared buildGeminiUsageMetadata from geminiUsageMetadata.ts (fix(gemini): forward OpenAI usage-only chunks and cached/thought token counts to /v1beta clients #17842), which produces the same fields.

  • Not changed: tool-call accumulation, how the finish chunk carries usage, the stash-before-finish behavior, the non-streaming path. A trailing chunk with neither choices nor usage is still dropped.

Proof Of Fix

  • TDD in tests/unit/translator-resp-openai-to-antigravity.test.ts, 5 new tests:
    1. A trailing usage-only chunk after the finish chunk is forwarded with all five counts, no finishReason, and the stream's modelVersion / responseId.
    2. Usage already sent on the finish chunk is not repeated.
    3. Usage stashed before the finish chunk is not repeated by a second usage-only chunk.
    4. A trailing chunk without usage stays dropped.
    5. End to end through translateResponse(OPENAI, ANTIGRAVITY): text, finish, usage-only chunk, then the flush null. Exactly one chunk carries usageMetadata, and it is the last one.
# tip (b2a64106c9), new tests against the unfixed translator
ℹ tests 10  pass 8  fail 2   (tests 1 and 5: usage-only chunk after finish → null)
# with the fix (plus translator-resp-openai-to-gemini.test.ts)
ℹ tests 12  pass 12  fail 0

Validation

  • check:open-sse-typecheck (0 errors), check:complexity-ratchets --base-ref origin/release/v3.8.53 (complexity violations in the touched file 2 → 1), check:file-size: OK
  • prettier + eslint (with config/quality/eslint-suppressions.json) clean; commit hooks incl. check:ai-attribution pass
  • Not verified against a live Antigravity / Gemini client; the end-to-end test goes through the real translateResponse registry, not through stream.ts's transform stream.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants