Repository navigation
fix(sse): defer OpenAI-to-Claude finish emission until real usage arrives - #11933
Merged
diegosouzapw merged 1 commit intoAug 28, 2026
Merged
Conversation
diegosouzapw
added a commit
that referenced
this pull request
Aug 28, 2026
…#11876 shipped on release/v3.8.51 (#11944) Eleven PRs landed on release/v3.8.51 while the branch carried fifteen base reds, and nine more red tests hid among them. None is a defect in the shipped code; each test still encoded the contract that the merged PR deliberately replaced: - openai-to-claude finish deferral (dd35750, #11933): a finish chunk that carries no usage is now held until the end-of-stream flush that production performs (open-sse/utils/stream.ts flush -> translateResponse(..., null, state)). The drivers in stream-markdown-token-boundary, translator-tool-call-shim and gemini-malformed-function-call-finish-reason-2462 fed the finish chunk and asserted the terminal events immediately; they now mirror the flush. Assertions unchanged. - authoritative live catalog (3d2832b, #11919 fixes #11829): a synced catalog replaces the static registry, so model-lifecycle-integration no longer expects the static-only gpt-5.6-sol row to survive a sync. The #8627 contract the file guards (stale chat rows suppressed, typed media retained) is untouched. - provider asset provenance (#11876): the unit shards check out with depth 1. The fixture pinned a historical commit as auditedCommit (absent on a shallow clone), the "binds auditedCommit" case relied on the repository root commit (the grafted HEAD on a shallow clone, which matches the physical snapshot), and the real-manifest case needs the audited commit fetched. The fixture now audits HEAD, the mismatch case builds a dangling empty-tree commit (no ref written), and the real-manifest case skips only on a shallow checkout that lacks the commit - the gate itself keeps running on both fetch-depth-0 rails, which the next test asserts. All five files pass locally (30, 11, 38, 3 and 18 tests); lint with the frozen suppressions is clean.
muhamadgalihsaputra
pushed a commit
to niyatna/NiyatnaRoute
that referenced
this pull request
Sep 27, 2026
…ives (cheaper-inference#11915 follow-up on cheaper-inference#11883) (cheaper-inference#11933) Merges cheaper-inference#11883's already-merged usage-harvesting extraction with cheaper-inference#11915's finish-deferral mechanism, verified to fix a real remaining bug: the client-visible message_delta carried stale/zero usage when finish_reason arrived before the trailing usage chunk. 86/86 tests passing across 16 translator regression files.
muhamadgalihsaputra
pushed a commit
to niyatna/NiyatnaRoute
that referenced
this pull request
Sep 27, 2026
…#11933, cheaper-inference#11919 and cheaper-inference#11876 shipped on release/v3.8.51 (cheaper-inference#11944) Eleven PRs landed on release/v3.8.51 while the branch carried fifteen base reds, and nine more red tests hid among them. None is a defect in the shipped code; each test still encoded the contract that the merged PR deliberately replaced: - openai-to-claude finish deferral (590c0cf, cheaper-inference#11933): a finish chunk that carries no usage is now held until the end-of-stream flush that production performs (open-sse/utils/stream.ts flush -> translateResponse(..., null, state)). The drivers in stream-markdown-token-boundary, translator-tool-call-shim and gemini-malformed-function-call-finish-reason-2462 fed the finish chunk and asserted the terminal events immediately; they now mirror the flush. Assertions unchanged. - authoritative live catalog (c2cf93c, cheaper-inference#11919 fixes cheaper-inference#11829): a synced catalog replaces the static registry, so model-lifecycle-integration no longer expects the static-only gpt-5.6-sol row to survive a sync. The cheaper-inference#8627 contract the file guards (stale chat rows suppressed, typed media retained) is untouched. - provider asset provenance (cheaper-inference#11876): the unit shards check out with depth 1. The fixture pinned a historical commit as auditedCommit (absent on a shallow clone), the "binds auditedCommit" case relied on the repository root commit (the grafted HEAD on a shallow clone, which matches the physical snapshot), and the real-manifest case needs the audited commit fetched. The fixture now audits HEAD, the mismatch case builds a dangling empty-tree commit (no ref written), and the real-manifest case skips only on a shallow checkout that lacks the commit - the gate itself keeps running on both fetch-depth-0 rails, which the next test asserts. All five files pass locally (30, 11, 38, 3 and 18 tests); lint with the frozen suppressions is clean.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
#11883 (merged) fixed the OpenAI→Claude stream translator dropping usage carried on a
trailing
{"choices":[],"usage":{...}}chunk — but only fixed it forstate.usageinternally. When the
finish_reasonchunk arrives before the trailing usage chunk (thenormal order for Fireworks/vLLM/Together and other
stream_options.include_usageupstreams), the terminal
message_delta/message_stopevents were still emittedimmediately on
finish_reason, carrying whateverstate.usageheld at that moment(stale/zero) — the trailing chunk's correction arrived too late to matter, since nothing
re-emits after the finish event already went out to the client. Verified directly against
the merged #11883 code: the actual emitted
message_deltastill carried{"input_tokens":0,"output_tokens":0}in this exact sequence.This ports #11915's fix (HouMinXi): the finish emission now defers via
state.pendingClaudeFinishChoiceuntil either the real usage has arrived or a genuineend-of-stream flush (
chunk === null, already an established mechanism inopen-sse/translator/index.tsfor formats like openai-responses) forces it through withwhatever was available.
This is a hand-merged combination of #11883's already-merged
trackUsageFromChunk()extraction with #11915's deferral mechanism (the two conflicted on the same lines), not a
straight cherry-pick — full credit for the deferral design and its regression coverage
goes to #11915.
Verification (on release/v3.8.51 tip)
message_deltawith{"input_tokens":0,"output_tokens":0}; after the fix:null(deferred), then thetrailing chunk emits the correct
message_deltawith real usage.tests/unit/openai-to-claude-trailing-usage-11817.test.ts(fix(translator): keep upstream usage from trailing empty-choices chunks #11883's suite),
tests/unit/translator/openai-to-claude-trailing-usage.test.ts(fix(translator): preserve usage metrics from trailing choices-empty chunks (#11817) #11915's suite, ported), and 14 other openai-to-claude translator regression files
(including the fix(translator): prevent doubled tool args in OpenAI-to-Claude #5828 and fix(backend): internal reasoning-replay placeholder leaks into assistant message content #8081/fix: strip internal reasoning placeholder from user-visible content (#8081) #8162 regression guards, and the existing
"two finish_reason chunks emit finish events exactly once" duplicate-finish guard).
npm run typecheck:core— cleannode scripts/check/check-file-size.mjs— passRelated to #11817, ports #11915 (HouMinXi).