fix(gateway): respect utf16 budget in _split_text_chunks no-newline fallback

[Frowtek]

What does this PR do?

GatewayStreamConsumer._split_text_chunks splits the fallback final
message into platform-sized chunks. Its no-newline fallback hard-split at
limit, but limit is measured in len_fn units (UTF-16 code units for
Telegram) while it was used as a codepoint slice index. Non-BMP text
(emoji = 2 UTF-16 units each) produced chunks up to 2x the budget,
exceeding the platform cap — so the final fallback message was rejected
and lost.

The function already computes _cp_budget (the largest codepoint offset
within the unit budget) and uses it to bound the newline search; the fix
also uses it for the hard split. For the default len_fn=len,
_cp_budget == limit, so plain-text behavior is unchanged.

Related Issue

N/A

Type of Change

• Bug fix (non-breaking change that fixes an issue)

Changes Made

• gateway/stream_consumer.py — in _split_text_chunks, hard-split at
  _cp_budget instead of limit so chunks respect the len_fn budget.
• tests/gateway/test_stream_consumer.py — add regression tests.

How to Test

Run:

pytest tests/gateway/test_stream_consumer.py -q

Added tests (all pass):

• emoji without newlines: every chunk utf16_len(chunk) <= limit
• emoji with newlines: stays within budget
• no-newline run is delivered losslessly across chunks
• CJK-BMP stays within budget (root-cause boundary)
• regression: default len_fn behavior unchanged

Before the fix, an emoji-only chunk reached utf16_len = 8000 for a
4000-unit limit; after, all chunks are <= limit.

Checklist

• I've read the Contributing Guide
• My commit messages follow Conventional Commits
• I searched for existing PRs to make sure this isn't a duplicate
• My PR contains only changes related to this fix
• I've added tests for my changes
• I've updated relevant documentation — N/A (internal helper)

[liuhao1024]

Positive verification: The root-cause fix is correct — _cp_budget (codepoints that fit the len_fn budget) replaces limit (measured in len_fn units, not codepoints) in the no-newline fallback split. Tests cover the exact failure mode (emoji without newlines exceeding UTF-16 budget), plus BMP CJK and plain-len regression guards. Clean one-line fix with excellent coverage.

[teknium1]

Thanks for isolating the fallback UTF-16 boundary defect. Current main still has the exact unsafe fallback at gateway/stream_consumer.py:975-980: _cp_budget is calculated, then limit is used as a Python slice index. The proposed replacement correctly fixes that path, and Telegram supplies utf16_len through plugins/platforms/telegram/adapter.py:509-511.

Problems

• The equivalent existing-message overflow path remains unsafe at gateway/stream_consumer.py:710-716: it calculates _cp_budget but hard-splits with _safe_limit, which is likewise measured in adapter units. The new direct _split_text_chunks tests do not exercise that path.

Suggested changes

• Apply the codepoint-safe boundary to the overflow loop at gateway/stream_consumer.py:715 and add a UTF-16 regression through the streaming path.

Automated hermes-sweeper review.