Repository navigation
fix(logging): raise the SSE payload collector's default cap - #12461
Merged
diegosouzapw merged 1 commit intoSep 3, 2026
Merged
diegosouzapw merged 1 commit into
diegosouzapw merged 1 commit into
Conversation
Follow-up to cheaper-inference#12460 (fail closed on a collector-truncated, empty output array) -- that fix stops a truncated response from breaking the NEXT turn's continuation, but doesn't stop the truncation itself. Root cause: createStructuredSSECollector's defaults were 200 events / 48KB. A reasoning-heavy response streams reasoning token-by-token as hundreds to thousands of tiny SSE deltas BEFORE the real output/tool_calls ever arrive -- the cap was routinely exhausted during the reasoning phase alone, dropping the completion event entirely. For a caller with no `format` (no live per-format reducer -- see the existing CollectorOptions.format doc comment, itself describing an analogous prior fix for providerPayloadCollector), the logged summary is reconstructed from getEvents() (open-sse/utils/stream.ts) -- the CAPPED, possibly-truncated array. clientPayloadCollector (the one that matters for previous_response_id continuation) is exactly this kind of caller. Measured live: ~22% of a sample of recent successful Ping responses hit this. Raised maxEvents 200 -> 2000 and maxBytes 48KB -> 512KB. This doesn't eliminate the class of bug for an arbitrarily long stream -- the format-driven live reducer (already used by providerPayloadCollector) is the cap-independent fix and remains the deeper follow-up for a caller that wants build()'s summary correct beyond any fixed cap -- but it removes truncation as a routine, everyday failure at realistic reasoning-heavy volumes. Regression tests added (no prior test file existed for this function): proves a realistic ~1600-event reasoning burst now survives intact (anonymized real-incident shape, proven RED against the old defaults, GREEN after), and proves the cap still protects against a genuinely pathological stream at the new, higher threshold. Full stream-adjacent suite: 986/986 passing, no regressions.
diegosouzapw
merged commit Sep 3, 2026
8729909
into
cheaper-inference:release/v3.8.51
15 of 16 checks passed
muhamadgalihsaputra
pushed a commit
to niyatna/NiyatnaRoute
that referenced
this pull request
Sep 27, 2026
…inference#12461) Validado em lote numa worktree combinada com os 9 PRs desta leva sobre o tip de `release/v3.8.51` (já com a leva anterior dentro): os nove boardaram **sem um único conflito**, `typecheck:core` limpo e **80/80** nos 6 arquivos de teste que os PRs trazem. O crescimento de arquivo próprio da leva foi rebaselinado num registro datado (`_rebaseline_2026_09_03_hartmark_batch`): `combos/page.tsx` 5012→5018 (cheaper-inference#12355, tratar o estado degradado quando o bundling de tiktoken de um provider sem relação falha) e `open-sse/services/combo.ts` 4023→4036 (cheaper-inference#12338, os fixes do universal-handoff). As violações restantes (`codex.ts`, `stream.ts`) foram medidas também no tip puro e são drift da base, não desta leva. Obrigado, @hartmark.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What Problem This Solves
Follow-up to #12460 (fail closed on a collector-truncated, empty output array) — that fix stops a truncated response from breaking the next turn's continuation, but doesn't stop the truncation itself.
Why This Change Was Made
createStructuredSSECollector's defaults were 200 events / 48KB. A reasoning-heavy response streams reasoning token-by-token as hundreds to thousands of tiny SSE deltas before the real output/tool_calls ever arrive — the cap was routinely exhausted during the reasoning phase alone, dropping the completion event entirely.For a caller with no
format(no live per-format reducer — see the existingCollectorOptions.formatdoc comment, itself describing an analogous prior fix forproviderPayloadCollector), the logged summary is reconstructed fromgetEvents()— the capped, possibly-truncated array.clientPayloadCollector(the one that matters forprevious_response_idcontinuation) is exactly this kind of caller. Measured live: ~22% of a sample of recent successful Ping responses hit this.Raised
maxEvents200 → 2000 andmaxBytes48KB → 512KB. This doesn't eliminate the class of bug for an arbitrarily long stream — the format-driven live reducer (already used byproviderPayloadCollector) is the cap-independent fix and remains the deeper follow-up for a caller that wantsbuild()'s summary correct beyond any fixed cap — but it removes truncation as a routine, everyday failure at realistic reasoning-heavy volumes.User Impact
Fewer collector-truncated call-log summaries, fewer resulting
previous_response_idcontinuation misses (#12460's fail-closed path exists for when this still happens, but should now fire much less often).Evidence
tests/unit/stream-payload-collector-cap.test.ts.🤖 Generated with Claude Code