Skip to content

fix(sse): defer OpenAI-to-Claude finish emission until real usage arrives - #11933

Merged
diegosouzapw merged 1 commit into
release/v3.8.51from
fix/11915-openai-claude-usage-ordering-port
Aug 28, 2026
Merged

diegosouzapw merged 1 commit into
release/v3.8.51from
fix/11915-openai-claude-usage-ordering-port

Conversation

@diegosouzapw

Copy link
Copy Markdown
Collaborator

Summary

#11883 (merged) fixed the OpenAI→Claude stream translator dropping usage carried on a
trailing {"choices":[],"usage":{...}} chunk — but only fixed it for state.usage
internally. When the finish_reason chunk arrives before the trailing usage chunk (the
normal order for Fireworks/vLLM/Together and other stream_options.include_usage
upstreams), the terminal message_delta/message_stop events were still emitted
immediately on finish_reason, carrying whatever state.usage held at that moment
(stale/zero) — the trailing chunk's correction arrived too late to matter, since nothing
re-emits after the finish event already went out to the client. Verified directly against
the merged #11883 code: the actual emitted message_delta still carried
{"input_tokens":0,"output_tokens":0} in this exact sequence.

This ports #11915's fix (HouMinXi): the finish emission now defers via
state.pendingClaudeFinishChoice until either the real usage has arrived or a genuine
end-of-stream flush (chunk === null, already an established mechanism in
open-sse/translator/index.ts for formats like openai-responses) forces it through with
whatever was available.

This is a hand-merged combination of #11883's already-merged trackUsageFromChunk()
extraction with #11915's deferral mechanism (the two conflicted on the same lines), not a
straight cherry-pick — full credit for the deferral design and its regression coverage
goes to #11915.

Verification (on release/v3.8.51 tip)

Related to #11817, ports #11915 (HouMinXi).

@diegosouzapw
diegosouzapw merged commit dd35750 into release/v3.8.51 Aug 28, 2026
8 of 12 checks passed
diegosouzapw added a commit that referenced this pull request Aug 28, 2026
…#11876 shipped on release/v3.8.51 (#11944)

Eleven PRs landed on release/v3.8.51 while the branch carried fifteen base reds, and
nine more red tests hid among them. None is a defect in the shipped code; each test
still encoded the contract that the merged PR deliberately replaced:

- openai-to-claude finish deferral (dd35750, #11933): a finish chunk that carries no
  usage is now held until the end-of-stream flush that production performs
  (open-sse/utils/stream.ts flush -> translateResponse(..., null, state)). The drivers in
  stream-markdown-token-boundary, translator-tool-call-shim and
  gemini-malformed-function-call-finish-reason-2462 fed the finish chunk and asserted
  the terminal events immediately; they now mirror the flush. Assertions unchanged.
- authoritative live catalog (3d2832b, #11919 fixes #11829): a synced catalog replaces
  the static registry, so model-lifecycle-integration no longer expects the static-only
  gpt-5.6-sol row to survive a sync. The #8627 contract the file guards (stale chat rows
  suppressed, typed media retained) is untouched.
- provider asset provenance (#11876): the unit shards check out with depth 1. The fixture
  pinned a historical commit as auditedCommit (absent on a shallow clone), the
  "binds auditedCommit" case relied on the repository root commit (the grafted HEAD on
  a shallow clone, which matches the physical snapshot), and the real-manifest case
  needs the audited commit fetched. The fixture now audits HEAD, the mismatch case
  builds a dangling empty-tree commit (no ref written), and the real-manifest case
  skips only on a shallow checkout that lacks the commit - the gate itself keeps
  running on both fetch-depth-0 rails, which the next test asserts.

All five files pass locally (30, 11, 38, 3 and 18 tests); lint with the frozen
suppressions is clean.
muhamadgalihsaputra pushed a commit to niyatna/NiyatnaRoute that referenced this pull request Sep 27, 2026
…ives (cheaper-inference#11915 follow-up on cheaper-inference#11883) (cheaper-inference#11933)

Merges cheaper-inference#11883's already-merged usage-harvesting extraction with cheaper-inference#11915's finish-deferral mechanism, verified to fix a real remaining bug: the client-visible message_delta carried stale/zero usage when finish_reason arrived before the trailing usage chunk. 86/86 tests passing across 16 translator regression files.
muhamadgalihsaputra pushed a commit to niyatna/NiyatnaRoute that referenced this pull request Sep 27, 2026
…#11933, cheaper-inference#11919 and cheaper-inference#11876 shipped on release/v3.8.51 (cheaper-inference#11944)

Eleven PRs landed on release/v3.8.51 while the branch carried fifteen base reds, and
nine more red tests hid among them. None is a defect in the shipped code; each test
still encoded the contract that the merged PR deliberately replaced:

- openai-to-claude finish deferral (590c0cf, cheaper-inference#11933): a finish chunk that carries no
  usage is now held until the end-of-stream flush that production performs
  (open-sse/utils/stream.ts flush -> translateResponse(..., null, state)). The drivers in
  stream-markdown-token-boundary, translator-tool-call-shim and
  gemini-malformed-function-call-finish-reason-2462 fed the finish chunk and asserted
  the terminal events immediately; they now mirror the flush. Assertions unchanged.
- authoritative live catalog (c2cf93c, cheaper-inference#11919 fixes cheaper-inference#11829): a synced catalog replaces
  the static registry, so model-lifecycle-integration no longer expects the static-only
  gpt-5.6-sol row to survive a sync. The cheaper-inference#8627 contract the file guards (stale chat rows
  suppressed, typed media retained) is untouched.
- provider asset provenance (cheaper-inference#11876): the unit shards check out with depth 1. The fixture
  pinned a historical commit as auditedCommit (absent on a shallow clone), the
  "binds auditedCommit" case relied on the repository root commit (the grafted HEAD on
  a shallow clone, which matches the physical snapshot), and the real-manifest case
  needs the audited commit fetched. The fixture now audits HEAD, the mismatch case
  builds a dangling empty-tree commit (no ref written), and the real-manifest case
  skips only on a shallow checkout that lacks the commit - the gate itself keeps
  running on both fetch-depth-0 rails, which the next test asserts.

All five files pass locally (30, 11, 38, 3 and 18 tests); lint with the frozen
suppressions is clean.
@diegosouzapw
diegosouzapw deleted the fix/11915-openai-claude-usage-ordering-port branch October 18, 2026 23:48
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants