Skip to content

fix(gateway): finalize oversize-split stream chunk so it gets final platform markup - #38971

Closed
GodsBoy wants to merge 1 commit into
NousResearch:mainfrom
GodsBoy:salvage/media-stream-finalize
Closed

fix(gateway): finalize oversize-split stream chunk so it gets final platform markup#38971
GodsBoy wants to merge 1 commit into
NousResearch:mainfrom
GodsBoy:salvage/media-stream-finalize

Conversation

@GodsBoy

@GodsBoy GodsBoy commented Jun 4, 2026

Copy link
Copy Markdown
Contributor

Summary

When a streamed message grows past the platform safe limit mid-stream, the consumer splits it: the first chunk is sent and then _message_id is reset to None so the remainder starts a fresh message. That first chunk was sent with the default finalize=False, so it never received the adapter's final rich-text markup (raw MarkdownV2 on Telegram) and could never be edited again.

This sends the split chunk with finalize=True so it gets its final platform formatting before it is surrendered, and with is_turn_final=False so it is not mistaken for the turn-final answer.

Why is_turn_final=False matters

The split chunk enters the _message_id is not None edit branch, where finalize=True can trigger _try_fresh_final. That path is enabled by default on Telegram (fresh_final_after_seconds=60). If it fired with the default is_turn_final=True, it would set _final_response_sent=True on a mid-stream chunk. A cancel or timeout before got_done would then let the gateway suppress the real final send and strand the continuation (the remainder is still the real answer). The sibling segment-break send already passes is_turn_final=got_done for the same reason (#29346).

Changes

  • gateway/stream_consumer.py: oversize-split first chunk now sent via _send_or_edit(chunk, finalize=True, is_turn_final=False).
  • tests/gateway/test_stream_consumer_fresh_final.py: two regression tests.
    • A unit check that _send_or_edit(finalize=True, is_turn_final=False) runs fresh-final without setting _final_response_sent.
    • An end-to-end test that drives the real run loop into the overflow branch and asserts the call site passes is_turn_final=False. It fails if the call site is reverted to a bare finalize=True.

Test plan

scripts/run_tests.sh tests/gateway/test_stream_consumer.py \
  tests/gateway/test_stream_consumer_fresh_final.py \
  tests/gateway/test_stream_consumer_draft.py \
  tests/gateway/test_stream_consumer_thread_routing.py

Result: 133 passed, 0 failed.

Context

This is the one salvageable change observed in #32609, rebased onto current main and hardened with the is_turn_final fix and regression coverage. The other changes in that PR are already on main (draft MarkdownV2 parity via #37250, sanitized skip-path logging) or broaden the media-delivery allowlist in a way that bypasses the credential denylist, so they are intentionally not carried here.

… turn-final

When a mid-stream message exceeds the platform safe limit, the first half is
sent and then _message_id is reset to None, so that chunk can never be edited
again. It was sent with the default finalize=False, leaving it stuck without
the adapter's final rich-text markup (raw MarkdownV2 on Telegram).

Pass finalize=True so the chunk receives its final platform formatting before
it is surrendered, and is_turn_final=False so _try_fresh_final does not set
_final_response_sent on this mid-stream chunk. Marking it as the turn-final
answer would let a cancel/timeout before got_done suppress the real final send
and strand the continuation (the remainder is still the real answer).

Adds two regression tests: a unit check that the _send_or_edit contract keeps
_final_response_sent False under is_turn_final=False, and an end-to-end run-loop
test that drives the overflow branch and guards the call site argument.
@alt-glitch alt-glitch added type/bug Something isn't working P2 Medium — degraded but workaround exists comp/gateway Gateway runner, session dispatch, delivery platform/telegram Telegram bot adapter labels Jun 4, 2026
@teknium1

Copy link
Copy Markdown
Contributor

Thanks for isolating the overflow-finalization behavior and preserving the non-turn-final flag.

Automated hermes-sweeper review found this exact behavior is already on main:

  • gateway/stream_consumer.py:727-729 calls _send_or_edit(chunk, finalize=True, is_turn_final=False) in the existing-message overflow path.
  • gateway/stream_consumer.py:1585-1587 only sets _final_response_sent when is_turn_final is true.
  • Commit 454d6cbe5250964e1e98cf44ae7d06d71a895b33 (fix(telegram): finalize sealed overflow chunk so split streamed replies render formatting) introduced the same change; it shipped in v2026.6.5.

Closing as already implemented on main.

@teknium1 teknium1 closed this Jul 14, 2026
@teknium1 teknium1 added the sweeper:implemented-on-main Sweeper: behavior already present on current main label Jul 14, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

comp/gateway Gateway runner, session dispatch, delivery P2 Medium — degraded but workaround exists platform/telegram Telegram bot adapter sweeper:implemented-on-main Sweeper: behavior already present on current main type/bug Something isn't working

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants