Skip to content

fix(sse): emit trailing usage estimate in translate streams when upstream stays silent - #12828

Merged
diegosouzapw merged 1 commit into
diegosouzapw:release/v3.8.51from
maxmad64bis:fix/translate-usage-estimate-order
Sep 10, 2026
Merged

diegosouzapw merged 1 commit into
diegosouzapw:release/v3.8.51from
maxmad64bis:fix/translate-usage-estimate-order

Conversation

@maxmad64bis

@maxmad64bis maxmad64bis commented Sep 5, 2026 •

Copy link
Copy Markdown
Contributor

⚠️ base-red inherited: #12732

Summary

Chat clients talking to a Responses upstream never got token counts. #12151 covered only one of the two stream paths: it emits the fallback estimate as a trailing usage-only chunk in passthrough mode, but translate mode computed the estimate after closing the stream, so it only ever reached the server log. This adds the missing trailing chunk in the translate flush. Real upstream usage still arrives inside its own finish chunk and stays untouched — no second chunk in that case.

Related Issues

Validation

  • Change type: other
  • Focused tests and category gates from the golden path
  • npm run lint
  • Reconciled with the current active release base; focused checks rerun afterward
  • Production-code changes include a new or updated automated test in this PR

Focused suites, all green on upstream/release/v3.8.51 after the change: stream-translate-usage-trailing (3/3, new), stream-utils (52/52), stream-passthrough-usage-estimation + stream-numeric-ids + empty-choices-no-inject + responses-usage-trailing-6906 + responses-translation-fixes (64/64), plus the commentary/control-lines/usage companions (18/18) and the Responses commentary set (67/67 with the numeric-ids group). typecheck:core and typecheck:noimplicit:core clean. npm run lint exits 2 on this base, and it does so without reporting a single rule violation — the only output is There are suppressions left that do not occur anymore, i.e. stale entries in config/quality/eslint-suppressions.json elsewhere in the tree. Linting open-sse/utils/stream.ts on its own returns exactly the 2 no-unused-vars errors already frozen for that file, so this change adds none. I left the box unchecked rather than tick it on a run that exits non-zero. No VPS gate applies — it's local stream assembly, no upstream round-trip involved.

Tests Added Or Updated

  • tests/unit/stream-translate-usage-trailing.test.ts (new): silent Responses upstream in translate mode asserts exactly one choices: [] chunk carrying usage before data: [DONE] with comments disabled; real response.completed usage asserts verbatim single delivery with no estimated marker; passthrough silent-upstream behavior pinned unchanged.

Coverage Notes

  • Change is in open-sse/utils/stream.ts (translate flush + one latch in the translated-item emitter) plus open-sse/utils/usageOnlyChunk.ts, the chunk builder now shared with the passthrough flush. The three new cases execute the added branch both ways (silent upstream emits, real usage suppresses) plus the untouched passthrough path.

Reviewer Notes

  • open-sse/utils/stream.ts: a translateForwardedUsage latch mirrors the existing passthrough guard, the estimate moved ahead of the [DONE] emission, and the trailing chunk reuses filterUsageForFormat + timing.withTps like passthrough. It's gated to OpenAI-chat-shaped clients (sourceFormat === FORMATS.OPENAI with the [DONE] terminator): Responses/Claude clients terminate on their own protocol events. One subtlety found while verifying: when a translator already merged an estimate into the finish chunk (Claude upstream), the latch is set there so the flush doesn't append a second chunk.
  • stream.ts is a frozen file: the chunk builder moved out, and the remaining flush wiring (+15 lines over the base) is rebaselined in config/quality/file-size-baseline.json with a note, like the existing _rebaseline_* entries.

@maxmad64bis
maxmad64bis force-pushed the fix/translate-usage-estimate-order branch from 53a6abc to 6006eb6 Compare September 5, 2026 22:31
…ream stays silent

A chat client talking to a Responses upstream gets no token counts at all when
that upstream never reports usage. The estimate is computed, but it only reaches
the server logs: the passthrough flush emits it as a canonical trailing
usage-only chunk before [DONE], and the translate flush has no equivalent.

This adds that chunk on the translate path, gated the same way passthrough gates
its own. Some translators already merge usage into the finish chunk, so a
translateForwardedUsage latch mirrors passthroughForwardedUsage and prevents a
double emission. A real upstream usage block still wins and is forwarded
verbatim.

The estimate is now computed before [DONE] rather than after, which is what makes
it available to the chunk at all.
@maxmad64bis
maxmad64bis force-pushed the fix/translate-usage-estimate-order branch from 6006eb6 to e61c81a Compare September 10, 2026 11:50
@diegosouzapw
diegosouzapw merged commit 91990d6 into diegosouzapw:release/v3.8.51 Sep 10, 2026
9 of 16 checks passed
@maxmad64bis
maxmad64bis deleted the fix/translate-usage-estimate-order branch September 24, 2026 21:16
muhamadgalihsaputra pushed a commit to niyatna/NiyatnaRoute that referenced this pull request Sep 27, 2026
…ream stays silent (diegosouzapw#12828)

Validado numa worktree combinada com a onda de streaming desta leva sobre `release/v3.8.51`: typecheck:core limpo, check-api-typecheck OK (289), check-file-size OK após rebaseline, 88/88 nos testes focados.

O diegosouzapw#12151 cobriu só metade: passthrough emitia o chunk final de usage, translate calculava a estimativa **depois** de fechar o stream, então o número só chegava ao log do servidor e nunca ao cliente. Fechar essa metade é o que faz a feature existir de fato.

Não emitir segundo chunk quando o upstream já mandou usage real é o detalhe que impede a correção de virar contagem dobrada.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants