fix(sse): emit trailing usage estimate in translate streams when upstream stays silent - #12828
Merged
diegosouzapw merged 1 commit intoSep 10, 2026
Conversation
maxmad64bis
force-pushed
the
fix/translate-usage-estimate-order
branch
from
September 5, 2026 22:31
53a6abc to
6006eb6
Compare
…ream stays silent A chat client talking to a Responses upstream gets no token counts at all when that upstream never reports usage. The estimate is computed, but it only reaches the server logs: the passthrough flush emits it as a canonical trailing usage-only chunk before [DONE], and the translate flush has no equivalent. This adds that chunk on the translate path, gated the same way passthrough gates its own. Some translators already merge usage into the finish chunk, so a translateForwardedUsage latch mirrors passthroughForwardedUsage and prevents a double emission. A real upstream usage block still wins and is forwarded verbatim. The estimate is now computed before [DONE] rather than after, which is what makes it available to the chunk at all.
maxmad64bis
force-pushed
the
fix/translate-usage-estimate-order
branch
from
September 10, 2026 11:50
6006eb6 to
e61c81a
Compare
diegosouzapw
merged commit Sep 10, 2026
91990d6
into
diegosouzapw:release/v3.8.51
9 of 16 checks passed
muhamadgalihsaputra
pushed a commit
to niyatna/NiyatnaRoute
that referenced
this pull request
Sep 27, 2026
…ream stays silent (diegosouzapw#12828) Validado numa worktree combinada com a onda de streaming desta leva sobre `release/v3.8.51`: typecheck:core limpo, check-api-typecheck OK (289), check-file-size OK após rebaseline, 88/88 nos testes focados. O diegosouzapw#12151 cobriu só metade: passthrough emitia o chunk final de usage, translate calculava a estimativa **depois** de fechar o stream, então o número só chegava ao log do servidor e nunca ao cliente. Fechar essa metade é o que faz a feature existir de fato. Não emitir segundo chunk quando o upstream já mandou usage real é o detalhe que impede a correção de virar contagem dobrada.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Chat clients talking to a Responses upstream never got token counts. #12151 covered only one of the two stream paths: it emits the fallback estimate as a trailing usage-only chunk in passthrough mode, but translate mode computed the estimate after closing the stream, so it only ever reached the server log. This adds the missing trailing chunk in the translate flush. Real upstream usage still arrives inside its own finish chunk and stays untouched — no second chunk in that case.
Related Issues
Validation
npm run lintFocused suites, all green on
upstream/release/v3.8.51after the change:stream-translate-usage-trailing(3/3, new),stream-utils(52/52),stream-passthrough-usage-estimation+stream-numeric-ids+empty-choices-no-inject+responses-usage-trailing-6906+responses-translation-fixes(64/64), plus the commentary/control-lines/usage companions (18/18) and the Responses commentary set (67/67 with the numeric-ids group).typecheck:coreandtypecheck:noimplicit:coreclean.npm run lintexits 2 on this base, and it does so without reporting a single rule violation — the only output isThere are suppressions left that do not occur anymore, i.e. stale entries inconfig/quality/eslint-suppressions.jsonelsewhere in the tree. Lintingopen-sse/utils/stream.tson its own returns exactly the 2no-unused-varserrors already frozen for that file, so this change adds none. I left the box unchecked rather than tick it on a run that exits non-zero. No VPS gate applies — it's local stream assembly, no upstream round-trip involved.Tests Added Or Updated
tests/unit/stream-translate-usage-trailing.test.ts(new): silent Responses upstream in translate mode asserts exactly onechoices: []chunk carryingusagebeforedata: [DONE]with comments disabled; realresponse.completedusage asserts verbatim single delivery with noestimatedmarker; passthrough silent-upstream behavior pinned unchanged.Coverage Notes
open-sse/utils/stream.ts(translate flush + one latch in the translated-item emitter) plusopen-sse/utils/usageOnlyChunk.ts, the chunk builder now shared with the passthrough flush. The three new cases execute the added branch both ways (silent upstream emits, real usage suppresses) plus the untouched passthrough path.Reviewer Notes
open-sse/utils/stream.ts: atranslateForwardedUsagelatch mirrors the existing passthrough guard, the estimate moved ahead of the[DONE]emission, and the trailing chunk reusesfilterUsageForFormat+timing.withTpslike passthrough. It's gated to OpenAI-chat-shaped clients (sourceFormat === FORMATS.OPENAIwith the[DONE]terminator): Responses/Claude clients terminate on their own protocol events. One subtlety found while verifying: when a translator already merged an estimate into the finish chunk (Claude upstream), the latch is set there so the flush doesn't append a second chunk.stream.tsis a frozen file: the chunk builder moved out, and the remaining flush wiring (+15 lines over the base) is rebaselined inconfig/quality/file-size-baseline.jsonwith a note, like the existing_rebaseline_*entries.