fix(gateway): finish the split-delivery bug class behind the swallowed-finals fix (salvages #78558)

[kshitijk4poor]

Summary

Salvages #78558 (@HexLab98) onto current main and finishes the bug class it opened.

The original fix is correct about the cause: on a multi-message split delivery the stream consumer set final_content_delivered=True without recording a payload, delivered_final_matches() returned None, and gateway/run.py treated None as "trust the flag" — so the completed reply was suppressed and never sent. That's #78541: 165 suppressed events with 100–3000 char replies lost over 48h across three Telegram groups.

But six code paths set _turn_split_delivery, and only one was taught to record a payload. The other five inherited the new "payload-less split ⇒ mismatch" verdict without being able to satisfy it, which swapped the swallow for the opposite defect. This PR fixes the producers rather than only distrusting them at the boundary.

Context — what actually changed for a user

Before (on main): a long Telegram group reply that overflows into several messages, where the stream ends before the tail arrives. Log shows the reply is ready, and it is never sent:

INFO gateway.run: Suppressing normal final send for session agent:main:telegram:group:-1004492624436:1: final delivery already confirmed (streamed=True previewed=False content_delivered=True).
INFO gateway.run: response ready: platform=telegram chat=-1004492624436 time=55.0s api_calls=2 response=1939 chars

The 1939-char answer never reaches the chat. Reproduced with a probe driving the real run() loop:

BEFORE (main)                      AFTER (this PR)
user saw the tail? : False         user saw the tail? : False
matcher verdict    : None          matcher verdict    : False
=> SUPPRESSES?  True               => SUPPRESSES?  False
   reply swallowed (#78541)           complete reply is delivered

What #78558 alone would have shipped. Same probe harness, three shapes it does not cover:

shape	main	#78558 alone	this PR
complete overflow split, everything delivered	suppress ✅	suppress ✅	suppress ✅
split + flood-controlled cosmetic final edit	suppress ✅	re-send ❌ duplicate	suppress ✅
split + fallback final send (tail-only record)	suppress ✅	re-send ❌ duplicate	suppress ✅
split on Telegram's fresh-final route	swallow ❌	swallow ❌	delivered ✅

The duplicate on row 2 is the regression that matters most: it re-opens the double-answer #45517 fixed (#36965 / #25349). Row 4 is why the reported symptom would likely have persisted — _try_fresh_final is Telegram's default finalize route.

Changes

Kept from #78558 (@HexLab98's two commits, authorship preserved): _stream_ledger (an un-truncated mirror of _accumulated, since the overflow paths truncate _accumulated to the tail once heads are sealed), the False-on-payload-less-split verdict, and the tri-state reordering. Confirmed the ledger is genuinely load-bearing — nothing else records the union of a multi-message delivery: _delivered_segment_texts is only appended in _reset_segment_state (which the split paths never reach; they clear _last_sent_text first), and has_delivered_text compares whole-string equality per entry, so it structurally cannot match chunk1 + chunk2.

Added on top:

• _send_or_edit failed-final-edit branch — record the visible payload on split turns too. It deliberately skipped recording, which now reads as a mismatch and re-sends an answer already on screen.
• _send_fallback_final (×2) + _send_empty_fallback_final — route through _record_turn_final_payload instead of assigning _delivered_final_text directly. On a split turn their final_text is only the trailing chunk, so a fully delivered heads+tail reply recorded a tail-only payload and was re-sent in full.
• _try_fresh_final — refuse the fresh-final route once a head chunk is sealed. It replaces every tracked preview with one message, which only holds the whole answer on a single-message turn; after a split it deleted the sealed heads while sending just the tail.
• Set _turn_split_delivery at seal time rather than after the tail send, so the tail's own finalize sees the split state. The sibling overflow path already did this — that divergence is what let fresh-final delete the heads.
• run.py stale-finalize reconciliation — skip the in-place edit on a split delivery. message_id is only the last chunk there, so editing it with the complete response repeated every sealed head's text inside the tail message. Falls through to the normal final send.
• Dropped a dead or "".join(chunks) fallback (all growth funnels through _append_accumulated, so the ledger is never empty at that call site; and joined chunks carry injected fence markers that could never equal final_response), and documented that _record_turn_final_payload intentionally ignores its argument on split turns.

Validation

	result
tests/gateway/test_stale_finalize_suppression.py	16 passed
tests/gateway/ -k 'stream or consumer or final or deliver or suppress or overflow or split or telegram'	1062 passed, 1 skipped (pre-existing mautrix mock skip)
ruff + py_compile on all three files	clean
probes vs main, real run() loop	7/7 shapes correct (see table above)

Mutation-checked — reverting any individual fix turns its own test red, so none of the new tests are vacuous:

mutation	result
revert _try_fresh_final split guard	test_split_delivery_keeps_sealed_heads_on_fresh_final fails
revert failed-final-edit recorder	test_failed_final_edit_after_split_records_visible_payload fails
revert delivered_final_matches to pre-PR	7 tests fail

Four new tests drive the real overflow-split loop rather than hand-setting private flags: complete split still suppresses (no duplicate), split missing a tail does not suppress, fresh-final keeps sealed heads, and a flood-controlled final edit after a split stays suppressed.

One pre-existing assertion relaxed: the gateway-boundary test asserted the recovery route (that _run_agent performs the reconcile edit) rather than the guarantee. Since a split delivery now deliberately skips that edit, it asserts the real contract — either _run_agent puts the complete text on the wire, or it declines to claim delivery so the caller's normal final send does. Verified the latter holds: already_sent comes back falsy.

Credit

@HexLab98 diagnosed the root cause and wrote the ledger mechanism; both commits are preserved with authorship intact.

Also closes #78556 (@686f6c61), which fixed the same issue independently. Its approach records "".join(chunks[:-1]) + _accumulated, but the splitter strips the newline separators (.lstrip("\n")) and injects fence markers at boundaries, so the join can never equal final_response — verified it returns False on every complete overflow split, i.e. duplicate sends on all long replies. Credited for independent diagnosis.

Closes #78541