Conversation
…inal lane _seal_overflow_heads clears the edit target mid-iteration, so the overflow gate evaluated at the top of run() is already stale when the leftover is pushed. The seal loop exits after ONE successful seal (its condition includes _message_id is not None), so a buffer far past the limit leaves most of itself behind. That leftover then reaches _push_update -> _first_send with no message to edit, on a NON-final tick, and adapter.send() chunks and caps it itself, numbering the pieces (i/n). The turn-final lane later publishes the same text again with its own denominator, because the two lanes use different budgets and different payload shapes. Observed in production on Discord: one inbound message, one API call, no tool turns, a 36642-char answer, and two interleaved sequences on screen - (i/10) and (i/9) - whose first chunks were byte-identical once the indicator was stripped and whose concatenations shared a prefix. The gateway's normal final send was correctly suppressed (streamed=True, content_delivered=True), which places the duplicate inside the consumer rather than in the gateway's delivery ledger. Fix: re-check the overflow gate AFTER the seal and hand a still-oversized leftover to _split_first_send, which is the consumer's own cap-aware, ledger-aware splitter and already owns sealing. This removes the same-iteration invalidation rather than compensating for it downstream, and it does not add a fourth delivery flag: the existing flags behaved correctly here. Negative control on this branch: with the change reverted, both new tests fail, including the one asserting that no line is published twice. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
|
Thanks @dasgltd. Current If you see a remaining gap on current |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
_seal_overflow_headsclears the edit target mid-iteration, so the overflow gate evaluated at the top ofrun()is already stale by the time the leftover is pushed:The seal loop exits after one successful seal (
while self._overflows() and self._message_id is not None ...sets_message_id = None), so a buffer far past the limit leaves most of itself behind. That leftover reaches_push_update->_first_sendwith no message to edit, on a non-final tick, andadapter.send()chunks and caps it itself, numbering the pieces(i/n). The turn-final lane then publishes the same text again with its own denominator - the two lanes do not share a budget (MAX_MESSAGE_LENGTHvs the consumer's_safe_limit) nor the same payload shape (anotify=Truesend strips the rich tail before chunking).Evidence
Measured in production on Discord, from the platform API rather than from a screenshot:
tool_turns=0, a 36642-char answer(i/10)and(i/9)(i/n)indicator stripped), and the concatenations share a prefix - one answer, split twice, not two answersstreamed=True previewed=False content_delivered=True), which places the duplicate inside the consumer, not in the gateway's delivery ledgerLikely related: #25349.
Fix
Re-check the overflow gate after the seal and hand a still-oversized leftover to
_split_first_send, the consumer's own cap-aware, ledger-aware splitter, which already owns sealing. This removes the same-iteration invalidation rather than compensating for it downstream.Deliberately NOT done: no per-turn message cap and no new "already published" flag. The existing flag family (
_final_response_sent/_final_content_delivered/_turn_split_delivery) behaved correctly in this incident - a fourth state would be more surface, not less.Tests
New file
tests/gateway/test_stream_consumer_oversized_leftover.py, two cases:The toy adapter uses a 2000-char limit on purpose: the consumer floors its own budget at 500, so a smaller limit makes it legitimately emit chunks above the limit - a measurement artifact, not a leak.
Verification on this branch
2 passedwith the change567 passed, 4 skippedacross the streaming/split/overflow/orphan tests intests/gateway/uvx ruff check .->All checks passed!Searched open and merged PRs for an existing fix to this lane before opening; the nearby ones (#103160, #107643, #111296) tune the duplicate-send warning, not the seal/leftover path.
🤖 Generated with Claude Code