feat(gateway): opt-in SSE side-channel for live token streaming to external chat clients

[CocaKova]

What

An opt-in, transient Server-Sent-Events side-channel that lets an external chat client render assistant tokens live without platform edit-streaming. Built for (and shipping with) Keryx, an Android Matrix client that acts as a command interface for a Hermes agent — but the mechanism is platform-agnostic.

• gateway/keryx_stream.py (new) — in-process pub/sub hub keyed by (platform, chat_id) + the aiohttp handler for GET /keryx/stream?platform=…&chat_id=… on the existing API server (same Bearer auth via API_SERVER_KEY, 20 s keepalive pings, bounded per-subscriber queues that drop on overflow rather than ever blocking the agent's worker thread).
• gateway/stream_consumer.py — mirrors delta / segment / stop events to the hub (thread-safe: call_soon_threadsafe onto each subscriber's loop), and while a subscriber is attached, suppresses interval/threshold platform edits for that chat: the side-channel carries the tokens and the platform receives only the single final committed message.
• gateway/platforms/api_server.py — one route registration, wrapped in try/except so the API server is unaffected if the module is absent.

Why

Matrix is the motivating case: m.replace edit-streaming bloats homeserver databases (every partial is a persisted event) and heavy edit-streams can corrupt client timelines (we hit Trixnity's "loop in timeline generation" in the wild). With this side-channel a client gets full live token rendering while the room's event history stays exactly one message per turn. Clients subscribe right before sending a command and the switch is evaluated per flush, so attachment mid-turn behaves correctly.

Default behaviour is unchanged

With no subscriber attached, every path flows exactly as today. The only behavioural knob is opt-in: KERYX_STREAM_FALLBACK_EDITS=1 lets Matrix fall back to throttled m.replace edits (driven by the standard streaming.edit_interval / buffer_threshold config) when the side-channel client is offline — off by default.

Wire protocol

event: delta    data: {"text": "…incremental tokens…"}
event: segment  data: {}     # text → tool → text boundary
event: stop     data: {}     # turn complete; server closes the channel
event: ping     data: {}     # 20s keepalive

Testing

Running in production on my gateway (Matrix + Keryx client): verified 200/text/event-stream with valid key, 401 without, live delta mirroring during agent turns, single final Matrix commit per turn with a subscriber attached, and no change to /health, /v1/models, or existing chat-completions routes. All three touched files py_compile clean.

Automated regression test (tests/gateway/test_keryx_stream_side_channel.py, 3 cases, all green in the Hermes venv): asserts events mirror in order to an attached subscriber, suppress_protocol_edits gates on subscriber presence, (platform, chat_id) keying isolates rooms, the Matrix FALLBACK_EDITS tier returns the throttled path with no subscriber, and the bounded per-subscriber queue drops on overflow instead of blocking the worker thread.

Update — 2026-07-03: verified end-to-end from a real client. With the companion reasoning-display fix (#57693) applied, I've exercised the full path live from the Keryx Android client against a production gateway across many turns: tokens render live off the side-channel, the Matrix room commits exactly one final message per turn (no m.replace history), reasoning blocks display correctly, and there are zero duplicate or stuck bubbles. The KERYX_STREAM_FALLBACK_EDITS=1 path still degrades cleanly to throttled edits when no subscriber is attached.

Happy to rename the module/route to something more generic (e.g. client_stream) if you'd prefer — kept the shipping name so the released client and this PR match.

🤖 Generated with Claude Code

[CocaKova]

Currently working on several fixes for the message delivery bugs the sweeper picked up.

[teknium1]

Thanks for the concrete implementation and the focused queue/ordering coverage.

Automated hermes-sweeper review: this is a Keryx-specific integration embedded in the gateway core (gateway/keryx_stream.py, /keryx/stream, and KERYX_STREAM_FALLBACK_EDITS). The standing policy for third-party product integrations is to ship them as standalone plugins rather than carry their maintenance coupling in this repository.

• The repository policy explicitly directs third-party integrations to standalone plugins installed under ~/.hermes/plugins/ or through pip entry points (AGENTS.md:126-135).
• Current main already has an authenticated generic run-event SSE surface at gateway/platforms/api_server.py:4702-4751 (introduced by 57da1e1cee940b9bdb7372972c44a37ab0407872).
• Please publish the Keryx bridge as a standalone plugin repository, where it can evolve with the released Android client and be promoted in #plugins-skills-and-skins.

This is an automated hermes-sweeper review.

---

Closed as not-planned per standing maintainer policy (in-tree-provider-integration). This is a design-direction decision, not a code-quality judgment — see the Contribution Rubric in AGENTS.md for what the project is looking for. If you believe this policy was misapplied to your change, comment here and a maintainer will take a look.

[CocaKova]

Agreed — the policy applies here; no misapplication. The Keryx bridge is a product-specific integration and doesn't belong in the gateway core, and main already exposes an authenticated generic surface (GET /v1/runs/{run_id}/events) that a client bridge can consume without any in-tree coupling.

We'll republish it as a standalone plugin installed into ~/.hermes/plugins/ (registering through the existing discovery path, consuming that run-events SSE rather than a bespoke /keryx/stream), and promote it in #plugins-skills-and-skins. Thanks for the clear pointer to the right surface.