Skip to content

fix(translator): preserve usage metrics from trailing choices-empty chunks (#11817) - #11915

Closed
HouMinXi wants to merge 1 commit into
diegosouzapw:release/v3.8.51from
HouMinXi:fix/11817-openai-claude-usage-trailing-chunk
Closed

HouMinXi wants to merge 1 commit into
diegosouzapw:release/v3.8.51from
HouMinXi:fix/11817-openai-claude-usage-trailing-chunk

Conversation

@HouMinXi

Copy link
Copy Markdown
Contributor

Fixes #11817. Preserves usage (including cache tokens) from trailing OpenAI chunks with empty choices array in the OpenAI→Claude stream translator. Includes TDD test coverage. ⚠️ base-red inherited: #11449

@HouMinXi
HouMinXi requested a review from diegosouzapw as a code owner August 28, 2026 16:18
diegosouzapw added a commit that referenced this pull request Aug 28, 2026
…ives (#11915 follow-up on #11883) (#11933)

Merges #11883's already-merged usage-harvesting extraction with #11915's finish-deferral mechanism, verified to fix a real remaining bug: the client-visible message_delta carried stale/zero usage when finish_reason arrived before the trailing usage chunk. 86/86 tests passing across 16 translator regression files.
@diegosouzapw

Copy link
Copy Markdown
Owner

Thanks for this — and genuinely good catch. #11883 landed first with a narrower fix
(harvesting usage from the trailing chunk into state.usage), which conflicted with your
branch by the time I got to it. But your PR caught something #11883 actually missed: I
verified directly against #11883's own merged code that when finish_reason arrives
before the trailing usage-only chunk (the normal order for these upstreams), the
client-visible message_delta still went out with {"input_tokens":0,"output_tokens":0}
— state.usage got corrected internally, but too late to matter for the event already
sent. Your pendingClaudeFinishChoice deferral is what actually closes that gap.

I hand-merged your deferral mechanism onto #11883's already-merged trackUsageFromChunk()
extraction (they touched the same lines) as #11933, with your test suite carried over
verbatim. 86/86 tests passing including the existing #5828/#8081/#8162 regression guards.
Full credit for the design and the regression coverage — thank you for tracking this all
the way to the actual emitted event instead of stopping at state.usage. Closing this one
now that it's landed.

Raudbjorn pushed a commit to Raudbjorn/OmniRoute that referenced this pull request Sep 9, 2026
@HouMinXi
HouMinXi deleted the fix/11817-openai-claude-usage-trailing-chunk branch September 16, 2026 13:46
muhamadgalihsaputra pushed a commit to niyatna/NiyatnaRoute that referenced this pull request Sep 27, 2026
…ives (diegosouzapw#11915 follow-up on diegosouzapw#11883) (diegosouzapw#11933)

Merges diegosouzapw#11883's already-merged usage-harvesting extraction with diegosouzapw#11915's finish-deferral mechanism, verified to fix a real remaining bug: the client-visible message_delta carried stale/zero usage when finish_reason arrived before the trailing usage chunk. 86/86 tests passing across 16 translator regression files.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[BUG] Streaming OpenAI→Claude translator drops usage (incl. cache tokens) on trailing choices:[] chunk

2 participants