feat(gateway): opt-in SSE side-channel for live token streaming to external chat clients

[hashbender]

What

An opt-in, transient Server-Sent-Events side-channel that lets an external chat client render assistant tokens live without platform edit-streaming. Built for (and shipping with) Keryx, an Android Matrix client that acts as a command interface for a Hermes agent — but the mechanism is platform-agnostic.

• gateway/keryx_stream.py (new) — in-process pub/sub hub keyed by (platform, chat_id) + the aiohttp handler for GET /keryx/stream?platform=…&chat_id=… on the existing API server (same Bearer auth via API_SERVER_KEY, 20 s keepalive pings, bounded per-subscriber queues that drop on overflow rather than ever blocking the agent's worker thread).
• gateway/stream_consumer.py — mirrors delta / segment / stop events to the hub (thread-safe: call_soon_threadsafe onto each subscriber's loop), and while a subscriber is attached, suppresses interval/threshold platform edits for that chat: the side-channel carries the tokens and the platform receives only the single final committed message.
• gateway/platforms/api_server.py — one route registration, wrapped in try/except so the API server is unaffected if the module is absent.

Why

Matrix is the motivating case: m.replace edit-streaming bloats homeserver databases (every partial is a persisted event) and heavy edit-streams can corrupt client timelines (we hit Trixnity's "loop in timeline generation" in the wild). With this side-channel a client gets full live token rendering while the room's event history stays exactly one message per turn. Clients subscribe right before sending a command and the switch is evaluated per flush, so attachment mid-turn behaves correctly.

Default behaviour is unchanged

With no subscriber attached, every path flows exactly as today. The only behavioural knob is opt-in: KERYX_STREAM_FALLBACK_EDITS=1 lets Matrix fall back to throttled m.replace edits (driven by the standard streaming.edit_interval / buffer_threshold config) when the side-channel client is offline — off by default.

Wire protocol

event: delta    data: {"text": "…incremental tokens…"}
event: segment  data: {}     # text → tool → text boundary
event: stop     data: {}     # turn complete; server closes the channel
event: ping     data: {}     # 20s keepalive

Testing

Running in production on my gateway (Matrix + Keryx client): verified 200/text/event-stream with valid key, 401 without, live delta mirroring during agent turns, single final Matrix commit per turn with a subscriber attached, and no change to /health, /v1/models, or existing chat-completions routes. All three touched files py_compile clean.

Happy to rename the module/route to something more generic (e.g. client_stream) if you'd prefer — kept the shipping name so the released client and this PR match.

🤖 Generated with Claude Code

---

Mirror-of: NousResearch#57091
NousResearch#57091