Strip lone surrogates from chat-surface replies at the gateway delivery boundary

[MaxFreedomPollard]

Rebased onto current main and extended per the sweeper review:

• Final-response boundary (original scope): _sanitize_gateway_final_response surrogate-sanitizes the delivered copy; the stored history copy was already sanitized in build_assistant_message. The interrupt-sentinel fast path added on main (Bug: Telegram users can receive internal system messages like home-channel setup and interrupt status text #7921) is preserved ahead of the sanitize, as requested.
• Streaming path (review ask): raw deltas used to reach GatewayStreamConsumer unchanged, so Telegram's strict UTF-16 length/edit paths could raise UnicodeEncodeError mid-stream before the boundary sanitizer ran. Deltas are now sanitized at the consumer's on_delta entry — the single chokepoint every delta source (gateway stream callback and the proxy SSE path) funnels through, which also makes the behavior directly unit-testable.

Regressions: chat-surface sanitize + raw-programmatic-surface passthrough at the boundary, and a streaming test asserting the queued delta content is UTF-16 encodable.

Repro/report: #55309.

[tonydwb]

Code Review Summary

Verdict: LGTM

Small fix for lone surrogates crashing Telegram utf16_len encoding.

Looks Good

• Reuses existing _sanitize_surrogates from message_sanitization module
• Only affects chat surfaces (raw text passthrough preserved)
• Test verifies both chat and raw surfaces
• Complements existing surrogate handling in build_assistant_message

[teknium1]

Thanks for tracing the raw final-response path; the non-streaming premise is verified on current main.

Problems

• With streaming enabled, gateway/run.py:18123-18125 forwards raw deltas into GatewayStreamConsumer.on_delta, which queues them unchanged at gateway/stream_consumer.py:370-379. The consumer later calls the adapter edit path (gateway/stream_consumer.py:1771-1774), while Telegram measures content through strict UTF-16 encoding (plugins/platforms/telegram/adapter.py:3980, gateway/platforms/base.py:145). A lone surrogate can therefore still fail before _sanitize_gateway_final_response() is reached.

Suggested changes

• Retain the final-response boundary fix, and sanitize stream deltas before they enter GatewayStreamConsumer; add a streaming regression that verifies the Telegram-facing content is UTF-16 encodable.
• Current gateway/run.py:429-432 adds an interrupt-sentinel fast path inside this hunk; preserve it when resolving the current conflict.

Automated hermes-sweeper review.

[swissly]

📌 Overlap note: opened #77395 (fix(agent): close unclosed JSON tool-call args in LIFO order). Touches agent/message_sanitization.py — unrelated concern (repair-correctness vs surrogate stripping), but same file; flagging for coordination.