Skip to content

fix(stream-consumer): preserve accumulated text on chunk send failure - #8116

Open
chinadbo wants to merge 2 commits into
NousResearch:mainfrom
chinadbo:fix/stream-consumer-accumulated-text-loss
Open

chinadbo wants to merge 2 commits into
NousResearch:mainfrom
chinadbo:fix/stream-consumer-accumulated-text-loss

Conversation

@chinadbo

Copy link
Copy Markdown

Summary

  • When _send_new_chunk fails during the chunk-splitting path (network error, send returns success=False), _accumulated was cleared unconditionally at line 193, causing silent text loss
  • The fix tracks how many bytes of _accumulated were successfully delivered and only trims that portion; unsent text is preserved for retry
  • Consistent with the existing _message_id is not None overflow path (lines 207-226) which already checks _send_or_edit return before advancing _accumulated

Test plan

  • test_chunk_send_failure_preserves_accumulated — all chunk sends fail → _accumulated retains full text
  • test_partial_chunk_failure_preserves_unsent_text — first chunk succeeds, second fails → unsent portion preserved
  • All 28 existing test_stream_consumer.py tests still pass

@alt-glitch alt-glitch added type/bug Something isn't working P2 Medium — degraded but workaround exists comp/gateway Gateway runner, session dispatch, delivery labels Apr 27, 2026
@chinadbo
chinadbo force-pushed the fix/stream-consumer-accumulated-text-loss branch from da8cae0 to 6e8a261 Compare April 27, 2026 13:18
When the chunk-splitting path fails to deliver a chunk (network error,
send returns success=False), _accumulated was cleared unconditionally,
causing silent text loss. Now only the successfully-sent portion is
trimmed from _accumulated; unsent text is preserved for retry.
… chunk failure

- Call _send_fallback_final when unsent text remains after chunk loop
- Avoid setting _final_response_sent=True when text was not fully delivered
- Add test verifying gateway fallback is not suppressed on partial failure
@chinadbo
chinadbo force-pushed the fix/stream-consumer-accumulated-text-loss branch from 6e8a261 to ae4abbb Compare April 28, 2026 02:04

@teknium1 teknium1 left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for tracing the overflow failure path. The current-head defect is real: gateway/stream_consumer.py:668-676 clears the buffer after chunk attempts even when a send fails, and a non-final iteration continues at line 692.

Problems

  • gateway/stream_consumer.py:354 counts len(chunk) as source progress. BasePlatformAdapter.truncate_message() appends (i/N) to multi-message chunks (gateway/platforms/base.py:5620-5625), so this can over-trim _accumulated after a successful chunk.
  • gateway/stream_consumer.py:357 retains the suffix while leaving _message_id on the successful chunk. A later non-final flush follows the edit path at current gateway/stream_consumer.py:694-748, which can replace that delivered chunk with the suffix instead of sending a continuation.

Suggested changes

  • Track consumed source boundaries independently of decorated transport chunks, and introduce a continuation state for partial initial-overflow delivery.
  • Add a regression where failure occurs before _DONE, later deltas arrive, and the final output contains the full text exactly once.

Automated hermes-sweeper review.

sent_length += len(chunk)
continue
reply_id = self._message_id
new_id = await self._send_new_chunk(chunk, reply_id)

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

truncate_message() decorates multi-message chunks with (i/N) (gateway/platforms/base.py:5620-5625), so len(chunk) is not the number of source characters consumed. This slice can skip source text after a successful decorated chunk; track raw source boundaries instead.

new_id = await self._send_new_chunk(chunk, reply_id)
if new_id is not None and new_id != reply_id:
sent_length += len(chunk)
else:

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

After a partial success, _message_id still identifies the delivered chunk. A later non-final flush will enter _send_or_edit and edit that message with this retained suffix, replacing visible content rather than creating a continuation. The partial-failure state needs an explicit continuation transition.

@teknium1 teknium1 added sweeper:risk-message-delivery Sweeper risk: may drop, duplicate, misroute, or suppress messages sweeper:blast-broad Sweeper blast radius: broad — a core path most sessions hit area/streaming Streaming responses: gateway delivery, provider wire labels Jul 12, 2026

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area/streaming Streaming responses: gateway delivery, provider wire comp/gateway Gateway runner, session dispatch, delivery P2 Medium — degraded but workaround exists sweeper:blast-broad Sweeper blast radius: broad — a core path most sessions hit sweeper:risk-message-delivery Sweeper risk: may drop, duplicate, misroute, or suppress messages type/bug Something isn't working

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants