Skip to content

fix(gateway): reasoning display is silently lost on every streamed turn - #57693

Open
CocaKova wants to merge 1 commit into
NousResearch:mainfrom
CocaKova:fix/streamed-reasoning-display
Open

fix(gateway): reasoning display is silently lost on every streamed turn#57693
CocaKova wants to merge 1 commit into
NousResearch:mainfrom
CocaKova:fix/streamed-reasoning-display

Conversation

@CocaKova

@CocaKova CocaKova commented Jul 3, 2026

Copy link
Copy Markdown
Contributor

Problem

With streaming.enabled: true, the stream consumer commits the final message and _run_agent_inner suppresses the normal send (already_sent=True). But the normal send path in _handle_message_with_agent is the only place the 💭 reasoning block is prepended (show_reasoning / reasoning_style). Net effect: enabling streaming silently turns off reasoning display for every model and every platform — the two features look independent but are mutually exclusive today.

Repro: any brain that emits reasoning_content, display.show_reasoning: true, streaming.enabled: true → the streamed final message arrives without the reasoning block; flip streaming off and it comes back.

Fix

  • Extract the block formatting into _format_reasoning_block() (per-platform show_reasoning resolution incl. the Mattermost explicit-override rule, 15-line collapse, reasoning_style rendering). The normal send path now calls it — behavior there is unchanged, this is a pure refactor.
  • In the suppression branch, fold the block into the already-streamed message with one edit_message(..., finalize=True) — the exact pattern the plugin-transform branch directly below already uses for the same "content changed after streaming" situation.
  • Best-effort: a failed edit only loses the reasoning display, never the answer, and never un-suppresses the send (no duplicate-message risk).

Testing

  • tests/test_gateway_streaming_nested_config.py: 3 passed.
  • Live-verified on my install (Matrix platform, vLLM brains with --reasoning-parser): streamed turns now show the collapsed reasoning block again, identical to non-streamed turns. I've been running this fix in production since 2026-07-02 as a local patch.

Contact for CLA/CI: jonathan.kovacs999@gmail.com

🤖 Generated with Claude Code

@alt-glitch alt-glitch added type/bug Something isn't working comp/gateway Gateway runner, session dispatch, delivery P2 Medium — degraded but workaround exists sweeper:risk-message-delivery Sweeper risk: may drop, duplicate, misroute, or suppress messages labels Jul 3, 2026

@teknium1 teknium1 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for tracing the streamed-final suppression path; the premise is confirmed on current main. gateway/run.py:11919-11953 formats reasoning only for the normal response, while gateway/run.py:20339-20347 marks a confirmed stream as already sent.

Problems

  • The new direct final edit in gateway/run.py:19307 omits stream metadata. gateway/stream_consumer.py:301-311 forwards that metadata for edits, and Slack uses it to select the workspace client (plugins/platforms/slack/adapter.py:1525, :1545-1547). A non-default Slack workspace can therefore fail this best-effort edit and still lose reasoning.
  • The diff changes only gateway/run.py; the cited tests/test_gateway_streaming_nested_config.py only covers parsing nested streaming configuration, not this delivery path.

Suggested changes

  • Reuse the stream consumer's metadata-aware edit path for the reasoning fold.
  • Add a streamed reasoning regression asserting the final edit, already_sent suppression, and metadata preservation.

This is an automated hermes-sweeper review.

Comment thread gateway/run.py Outdated
source, response.get("last_reasoning")
)
_sc_msg_id = _sc.message_id if _sc else None
if _reasoning_block and _sc_msg_id and _final:

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please preserve the stream consumer's routing metadata here. GatewayStreamConsumer._edit_message() forwards metadata when supported, and Slack uses it to select the workspace client; this direct call can silently fail for a non-default workspace and leave reasoning absent.

…a-aware edit

With streaming enabled the stream consumer commits the final message and the
gateway suppresses the normal send (already_sent=True). But the normal send
path is the only place the 💭 reasoning block is prepended (show_reasoning /
reasoning_style), so turning streaming on silently disabled reasoning display
for every model and platform.

Fix: extract the block formatting into _format_reasoning_block() (shared by the
normal path — behavior unchanged) and, in the suppression branch, fold the
block into the already-streamed message with one final edit. The fold routes
through the stream consumer's metadata-aware _edit_message() rather than a raw
adapter.edit_message(): the routing metadata Slack uses to select the workspace
client (and Telegram uses for topic/thread routing) is preserved, so a
non-default Slack workspace no longer fails the edit and loses reasoning.
Best-effort — a failed fold only loses the reasoning display, never the answer,
and never un-suppresses the send.

tests/gateway/test_streamed_reasoning_fold.py: metadata forwarding on the edit
path, the fold's content/message-id/finalize/metadata, best-effort no-ops
(no reasoning / no consumer / uncommitted stream / edit failure), and the
already_sent suppression gate. 9 passed; existing suppression + stream-consumer
suites (184) still green.
@CocaKova
CocaKova force-pushed the fix/streamed-reasoning-display branch from 3338f45 to b1128f4 Compare July 15, 2026 15:19
@CocaKova

Copy link
Copy Markdown
Contributor Author

Thanks — both points addressed, rebased onto current main (b1128f40a).

Metadata-aware edit path. The reasoning fold no longer calls adapter.edit_message directly. It now routes through the stream consumer's GatewayStreamConsumer._edit_message(), which forwards self.metadata to adapters that accept it (Slack uses it to select the workspace client; Telegram uses it for topic/thread routing). So a non-default Slack workspace keeps its routing on the fold edit instead of silently failing it. The fold is factored into _fold_reasoning_into_streamed_message() and stays best-effort — a failed edit only loses the reasoning display, never the answer, and never un-suppresses the send.

Real delivery-path regression. New tests/gateway/test_streamed_reasoning_fold.py exercises the actual code (not config parsing):

  • _edit_message forwards routing metadata when the adapter supports it, and omits it (without raising) when it does not;
  • the fold edits the streamed message with {reasoning_block}\n\n{answer}, the correct message_id, finalize=True, and preserved metadata;
  • best-effort no-ops: no reasoning, no stream consumer, an uncommitted stream (no message_id), and an adapter edit that raises;
  • the already_sent suppression gate still fires on confirmed streamed delivery and is unaffected by the fold outcome (transformed responses still take the sibling edit branch).

9 new tests pass; the existing suppression + stream-consumer suites (184) stay green. The formatting extraction into _format_reasoning_block() is behavior-preserving for the normal send path.

@teknium1 teknium1 added sweeper:risk-compatibility Sweeper risk: may break existing users, config, migrations, defaults, or upgrades sweeper:blast-moderate Sweeper blast radius: moderate — a subsystem or single platform area/streaming Streaming responses: gateway delivery, provider wire labels Jul 15, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area/streaming Streaming responses: gateway delivery, provider wire comp/gateway Gateway runner, session dispatch, delivery P2 Medium — degraded but workaround exists sweeper:blast-moderate Sweeper blast radius: moderate — a subsystem or single platform sweeper:risk-compatibility Sweeper risk: may break existing users, config, migrations, defaults, or upgrades sweeper:risk-message-delivery Sweeper risk: may drop, duplicate, misroute, or suppress messages type/bug Something isn't working

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants