Skip to content

fix(sse): provider-response summary format bugs (dashboard Provider Response panel) - #10037

Merged
diegosouzapw merged 3 commits into
cheaper-inference:release/v3.8.50from
hartmark:fix/provider-response-summary-format
Aug 13, 2026
Merged

diegosouzapw merged 3 commits into
cheaper-inference:release/v3.8.50from
hartmark:fix/provider-response-summary-format

Conversation

@hartmark

Copy link
Copy Markdown
Contributor

Summary

Extracted from PR #9439 (agentic conversation tracking) — these are general SSE/dashboard bugs affecting the "Provider Response" panel for any streamed request, not specific to conversation tracking. Splitting them out for independent, faster review.

Three related bugs, fixed in sequence:

  1. Stale/truncated summary reconstruction (fix(backend): dashboard log viewer shows stale provider response for long streamed responses #9315). open-sse/utils/stream.ts reconstructed the dashboard's "Provider Response" summary from buildStreamSummaryFromEvents(providerPayloadCollector.getEvents(), ...) — but getEvents() only returns whatever survived the collector's maxEvents/maxBytes cap. Once a stream exceeded it (easy with a reasoning + tool-calling model), everything after the cutoff (final finish_reason, tool_calls, rest of reasoning_content, usage) was silently dropped from the reconstruction, even though the client actually received the correct, complete response.

    Fix: streamPayloadCollector.ts's per-format summary builders are now also available as incremental reducers (ingest one chunk at a time, finalize at the end). createStructuredSSECollector accepts a format/fallbackModel and feeds the reducer on every push() — including chunks later dropped from the retained event array — via a new getSummary() method.

  2. Client format used instead of provider format. providerPayloadCollector was keyed on sourceFormat (the CLIENT's wire format) instead of targetFormat (the PROVIDER's). Whenever a request translates between two different formats — e.g. a Responses-API client routed to a plain-OpenAI-chat-completions upstream (the common OpenClaw/opencode-zen combo shape) — the reducer picked for sourceFormat could never recognize the provider's actual raw event shape, so it never left its empty initial state. The dashboard's "Provider Response" panel permanently showed output: [] while "Client Response" correctly showed full content, reading as if the two panels simply disagreed about the same request.

    Confirmed live via a wire-level pcap capture cross-referenced against a dashboard log entry: the actual response was complete and correct — this was purely a logging/summary bug, never a wire-format bug.

  3. Missing object marker on the fallback summary. The responseBody-as-summary fallback path (used when the format doesn't need the events-derived summary) never set object: "chat.completion" at all, caught by this PR's own new regression test.

⚠️ base-red inherited: #9985

Test plan

  • TDD: regression tests in tests/unit/stream-payload-collector.test.ts and tests/unit/stream-utils.test.ts — confirmed failing against the pre-fix code, passing after.
  • tests/unit/stream-utils.test.ts + tests/unit/stream-payload-collector.test.ts + tests/unit/stream-payload-collector-9315-truncated-provider-response.test.ts — 70/71 passing (the one remaining failure, "passthrough splits mixed reasoning and content deltas", is a pre-existing v3.6.6-era test confirmed failing identically on a pristine upstream/release/v3.8.50 checkout — unrelated to this change).
  • npx tsc --noEmit — clean for all changed files (pre-existing unrelated errors elsewhere confirmed identical to upstream, zero diff at those lines).
  • npm run lint — clean.

hartmark and others added 3 commits August 12, 2026 21:23
The dashboard's "Provider Response" panel showed a stale, incomplete
snapshot for long streamed responses. Root cause: open-sse/utils/stream.ts
reconstructed the summary from
buildStreamSummaryFromEvents(providerPayloadCollector.getEvents(), ...)
-- but getEvents() only returns whatever survived the collector's
maxEvents/maxBytes cap, so once a stream exceeded it (easy with a
reasoning + tool-calling model), everything after the cutoff (final
finish_reason, tool_calls, rest of reasoning_content, usage) was
silently dropped from the reconstruction, even though the client
actually received the correct, complete response.

Fix: streamPayloadCollector.ts's per-format summary builders
(buildOpenAISummary/buildResponsesSummary/buildClaudeSummary/
buildGeminiSummary) are now also available as incremental reducers
(createXReducer: ingest one chunk at a time, finalize at the end).
createStructuredSSECollector accepts a format + fallbackModel and feeds
the reducer on every push() -- including chunks that get dropped from
the retained event array once the cap is hit -- via a new getSummary()
method. stream.ts's error-path call site now uses
collector.getSummary() instead of reconstructing from the (possibly
truncated) getEvents().

Extracted from a squashed commit (originally authored alongside a
conversation-tracking continuation fix in the same commit) -- only the
files relevant to this SSE-summary bug are included here
(stream.ts/streamPayloadCollector.ts + their test); the unrelated
conversationTracker.ts continuation fix stays with the conversation-
tracking PR it belongs to.

Test plan:
- New TDD regression tests in tests/unit/stream-payload-collector.test.ts,
  confirmed failing before the fix and passing after.
… provider's

providerPayloadCollector (dashboard "Provider Response" panel) was keyed on
sourceFormat (the CLIENT's wire format) instead of targetFormat (the
PROVIDER's — see createSSEStream's own @PARAM doc: "targetFormat - Provider
format", "sourceFormat - Client format"). Whenever a request translates
between two different formats — e.g. a Responses-API client routed to a
plain-OpenAI-chat-completions upstream, the common OpenClaw/opencode-zen
shape — the reducer picked for sourceFormat could never recognize the
provider's actual raw event shape, so it stayed stuck at its empty initial
state. The dashboard's "Provider Response" panel showed a permanently empty
`output: []` while "Client Response" (built from separately-accumulated
state, unaffected by this bug) correctly showed full content — reading as
if the two panels simply disagreed about the same request.

Confirmed live via a wire-level pcap capture (scripts/sre/tcp-close-
analyzer.py) cross-referenced against the dashboard log
(1786032832181-1c6275): the actual response was complete and correct: this
was purely a logging/summary bug, never a wire-format bug.

Fix is mode-aware: TRANSLATE mode uses targetFormat (the provider's true
format); PASSTHROUGH mode keeps sourceFormat, since passthrough has no
separate provider/client format split — nothing gets translated there, and
real passthrough callers (createPassthroughStreamWithLogger) don't even
pass targetFormat.

New regression test reproduces the exact live scenario (Responses-API
source, OpenAI target, real chat.completion.chunk deltas) and asserts the
provider summary reflects them — confirmed it fails with the old
`sourceFormat`-keyed code (reproducing the live `output: []`-style
symptom) and passes with the fix.

Co-authored-by: Markus Hartung <markus.hartung@gmail.com>
createSSEStream's providerPayloadCollector.build() falls back to the
synthesized responseBody as the "Provider Response" dashboard summary
whenever sourceFormat/targetFormat isn't OPENAI_RESPONSES (in both the
passthrough and translate branches) -- but responseBody is built purely
for the client and never carries an `object` field at all, so the
summary ended up with `object: undefined` instead of the expected
"chat.completion", even though everything else (choices, usage) was
correct.

Caught by this PR's own new regression test ("createSSEStream translate
mode: providerPayload summary reflects the PROVIDER's format, not the
client's") -- the code itself was unchanged by the rebase (applied
cleanly from the original commit), so this was a latent gap in the
original fix, not a rebase regression.

Fix: stamp `object: "chat.completion"` on a shallow copy used only for
the provider summary in both branches; responseBody itself (sent to the
client elsewhere) stays untouched.

Verified: tests/unit/stream-utils.test.ts 51/52 passing (the one
remaining failure is an unrelated, pre-existing v3.6.6-era test,
confirmed present and failing identically on a pristine
upstream/release/v3.8.50 checkout -- base-red inherited: cheaper-inference#9985).
typecheck/lint clean (pre-existing unrelated errors elsewhere in the
file, confirmed identical to upstream).
@hartmark
hartmark force-pushed the fix/provider-response-summary-format branch from 121ae54 to 47b250f Compare August 12, 2026 19:24
@diegosouzapw

Copy link
Copy Markdown
Collaborator

Reviewed via TDD probe against a fresh worktree off release/v3.8.50 (merge-base 1 commit behind tip, no base-drift). All three claimed bugs verified independently: reverting the prod diff makes the corresponding new regression test fail with the exact symptom described (getSummary is not a function for bug 1; providerPayload.object undefined instead of 'chat.completion' for bugs 2+3 combined). Full suite green hermetically: stream-utils.test.ts 52/52, stream-payload-collector*.test.ts 19/19, plus the untouched sibling request-log-payloads.test.ts (still exercises getEvents()) unaffected at 9/9. The apparent -366 line deletions are reformatting noise, not real removals -- confirmed buildOpenAISummary/buildResponsesSummary/buildClaudeSummary/buildGeminiSummary all still exist, still internal-only, no lost callers. Raw eslint shows 17 hits but they match config/quality/eslint-suppressions.json's frozen counts exactly (1 pre-existing no-restricted-syntax + 16 pre-existing no-explicit-any) -- zero new violations, npm run lint is genuinely clean. tsc --noEmit full run: zero errors touching the changed files. Nothing mandatory before merge. Optional follow-up: open-sse/utils/streamEmptyChoices.ts:98 (rejectEmptyChoicesStream) still reconstructs its provider summary via the old getEvents()-based path rather than the new getSummary() reducer -- low impact since that path only fires when no valuable chunks were forwarded, but worth a small consistency pass in a later PR.


Note: the base-red that was failing this PR's CI (#9985) was drained today — base-reds PR #10213 just merged into release/v3.8.50. A rebase/sync onto the current tip should bring your checks green.

@diegosouzapw
diegosouzapw merged commit 4bda225 into cheaper-inference:release/v3.8.50 Aug 13, 2026
5 of 6 checks passed
@diegosouzapw

Copy link
Copy Markdown
Collaborator

Merged — thanks @hartmark! All three Provider Response summary bugs verified independently; the live per-format reducer is a real improvement over the capped getEvents() reconstruction.

muhamadgalihsaputra pushed a commit to niyatna/NiyatnaRoute that referenced this pull request Sep 27, 2026
…esponse panel) (cheaper-inference#10037)

* fix(sse): provider-response summary reconstructed from truncated events

The dashboard's "Provider Response" panel showed a stale, incomplete
snapshot for long streamed responses. Root cause: open-sse/utils/stream.ts
reconstructed the summary from
buildStreamSummaryFromEvents(providerPayloadCollector.getEvents(), ...)
-- but getEvents() only returns whatever survived the collector's
maxEvents/maxBytes cap, so once a stream exceeded it (easy with a
reasoning + tool-calling model), everything after the cutoff (final
finish_reason, tool_calls, rest of reasoning_content, usage) was
silently dropped from the reconstruction, even though the client
actually received the correct, complete response.

Fix: streamPayloadCollector.ts's per-format summary builders
(buildOpenAISummary/buildResponsesSummary/buildClaudeSummary/
buildGeminiSummary) are now also available as incremental reducers
(createXReducer: ingest one chunk at a time, finalize at the end).
createStructuredSSECollector accepts a format + fallbackModel and feeds
the reducer on every push() -- including chunks that get dropped from
the retained event array once the cap is hit -- via a new getSummary()
method. stream.ts's error-path call site now uses
collector.getSummary() instead of reconstructing from the (possibly
truncated) getEvents().

Extracted from a squashed commit (originally authored alongside a
conversation-tracking continuation fix in the same commit) -- only the
files relevant to this SSE-summary bug are included here
(stream.ts/streamPayloadCollector.ts + their test); the unrelated
conversationTracker.ts continuation fix stays with the conversation-
tracking PR it belongs to.

Test plan:
- New TDD regression tests in tests/unit/stream-payload-collector.test.ts,
  confirmed failing before the fix and passing after.

* fix(sse): provider-response summary used the client's format, not the provider's

providerPayloadCollector (dashboard "Provider Response" panel) was keyed on
sourceFormat (the CLIENT's wire format) instead of targetFormat (the
PROVIDER's — see createSSEStream's own @PARAM doc: "targetFormat - Provider
format", "sourceFormat - Client format"). Whenever a request translates
between two different formats — e.g. a Responses-API client routed to a
plain-OpenAI-chat-completions upstream, the common OpenClaw/opencode-zen
shape — the reducer picked for sourceFormat could never recognize the
provider's actual raw event shape, so it stayed stuck at its empty initial
state. The dashboard's "Provider Response" panel showed a permanently empty
`output: []` while "Client Response" (built from separately-accumulated
state, unaffected by this bug) correctly showed full content — reading as
if the two panels simply disagreed about the same request.

Confirmed live via a wire-level pcap capture (scripts/sre/tcp-close-
analyzer.py) cross-referenced against the dashboard log
(1786032832181-1c6275): the actual response was complete and correct: this
was purely a logging/summary bug, never a wire-format bug.

Fix is mode-aware: TRANSLATE mode uses targetFormat (the provider's true
format); PASSTHROUGH mode keeps sourceFormat, since passthrough has no
separate provider/client format split — nothing gets translated there, and
real passthrough callers (createPassthroughStreamWithLogger) don't even
pass targetFormat.

New regression test reproduces the exact live scenario (Responses-API
source, OpenAI target, real chat.completion.chunk deltas) and asserts the
provider summary reflects them — confirmed it fails with the old
`sourceFormat`-keyed code (reproducing the live `output: []`-style
symptom) and passes with the fix.

Co-authored-by: Markus Hartung <markus.hartung@gmail.com>

* fix(sse): stamp object: chat.completion on the provider-summary fallback

createSSEStream's providerPayloadCollector.build() falls back to the
synthesized responseBody as the "Provider Response" dashboard summary
whenever sourceFormat/targetFormat isn't OPENAI_RESPONSES (in both the
passthrough and translate branches) -- but responseBody is built purely
for the client and never carries an `object` field at all, so the
summary ended up with `object: undefined` instead of the expected
"chat.completion", even though everything else (choices, usage) was
correct.

Caught by this PR's own new regression test ("createSSEStream translate
mode: providerPayload summary reflects the PROVIDER's format, not the
client's") -- the code itself was unchanged by the rebase (applied
cleanly from the original commit), so this was a latent gap in the
original fix, not a rebase regression.

Fix: stamp `object: "chat.completion"` on a shallow copy used only for
the provider summary in both branches; responseBody itself (sent to the
client elsewhere) stays untouched.

Verified: tests/unit/stream-utils.test.ts 51/52 passing (the one
remaining failure is an unrelated, pre-existing v3.6.6-era test,
confirmed present and failing identically on a pristine
upstream/release/v3.8.50 checkout -- base-red inherited: cheaper-inference#9985).
typecheck/lint clean (pre-existing unrelated errors elsewhere in the
file, confirmed identical to upstream).

---------

Co-authored-by: Markus Hartung <markus.hartung@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants