Skip to content

feat(serve): SSE replay sizing + slow_client_warning backpressure (#4175 Wave 2.5 PR 10) #4237

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
wenshao merged 7 commits into from
May 17, 2026
Merged

feat(serve): SSE replay sizing + slow_client_warning backpressure (#4175 Wave 2.5 PR 10) #4237

Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
7 commits
Select commit Hold shift + click to select a range
033f1ca
feat(serve): SSE replay sizing + slow_client_warning backpressure
doudouOUC May 17, 2026
a6ccde5
refactor(serve): adopt PR #4237 review feedback (eventBus polish)
doudouOUC May 17, 2026
bae42c8
fix(serve): correct queue tagging + plumb maxQueued through SDK
doudouOUC May 17, 2026
7a0224a
refactor(serve): adopt PR #4237 copilot review feedback
doudouOUC May 17, 2026
b51e04f
refactor(serve): adopt PR #4237 wenshao round-2 review feedback
doudouOUC May 17, 2026
d785a94
merge: resolve main conflicts for PR #4237
doudouOUC May 17, 2026
9cdcb70
merge: resolve latest main conflicts for PR #4237
doudouOUC May 17, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
41 changes: 26 additions & 15 deletions docs/developers/qwen-serve-protocol.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -91,13 +91,16 @@ registry. Clients **must** gate UI off `features`, not off `mode` (per design
['health', 'capabilities', 'session_create', 'session_scope_override',
'session_load', 'unstable_session_resume',
'session_list', 'session_prompt', 'session_cancel', 'session_events',
'session_set_model', 'permission_vote']
'slow_client_warning',
'session_set_model', 'client_identity', 'permission_vote']
```

`session_scope_override` is the negotiation handle for the per-request `sessionScope` field on `POST /session` (see below). Older daemons silently ignore the field, so SDK clients should pre-flight `caps.features` for this tag before sending it.

`session_load` and `unstable_session_resume` advertise the explicit-restore routes (`POST /session/:id/load` and `POST /session/:id/resume`). Older daemons return `404` for these paths, so SDK clients should pre-flight `caps.features` before calling. The `unstable_` prefix on `unstable_session_resume` mirrors the underlying ACP method (`connection.unstable_resumeSession`) — the daemon's wire shape is committed for v1, but the ACP method name itself may change before ACP marks resume stable.

`slow_client_warning` covers two co-released SSE backpressure knobs introduced in #4175 Wave 2.5 PR 10: (a) the daemon emits a `slow_client_warning` synthetic event-stream frame when a subscriber's queue crosses 75% full, once per overflow episode (rearmed after the queue drains below 37.5%); (b) `GET /session/:id/events` accepts a `?maxQueued=N` query param (range `[16, 2048]`) to pre-size the per-subscriber backlog for cold reconnects against a large replay ring. The daemon-wide ring size is controlled by `--event-ring-size` (default **8000**, per #3803 §02). Old daemons silently lack both — pre-flight this tag before opting in.

## Routes

> **Stage 1 limitation — no `DELETE /session/:id`.** Sessions live until
Expand Down Expand Up @@ -341,6 +344,12 @@ Accept: text/event-stream
Last-Event-ID: 42 ← optional, replays from after id 42
```

Query params:

| Param | Required | Notes |
| ----------- | -------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `maxQueued` | no | Per-subscriber live-backlog cap. Range `[16, 2048]`, default 256. Bump for cold reconnects with `Last-Event-ID: 0` against a large replay ring so the force-pushed replay frames don't immediately trip eviction. Out-of-range / non-decimal values return `400 invalid_max_queued` before the SSE handshake opens. Pre-flight `caps.features.slow_client_warning` — old daemons silently ignore the param. |
Comment thread
doudouOUC marked this conversation as resolved.
Outdated

Frame format. The `data:` line is the **full event envelope**, JSON-stringified on a single line — `{id?, v, type, data, originatorClientId?}`. The ACP-specific payload (`sessionUpdate`, `requestPermission` arguments, etc.) sits under the envelope's `data` field; the envelope's own `type` matches the SSE `event:` line.

```
Expand All @@ -360,28 +369,30 @@ data: {"v":1,"type":"client_evicted","data":{"reason":"queue_overflow","droppedA

The SSE-level `id:` / `event:` lines duplicate `envelope.id` / `envelope.type` for EventSource compatibility. Raw-`fetch` consumers (the SDK's `parseSseStream`) read everything off the JSON envelope and ignore the SSE preamble lines.

| Event type | Trigger |
| --------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `session_update` | Any ACP `sessionUpdate` notification (LLM chunks, tool calls, usage) |
| `permission_request` | Agent asked for tool approval |
| `permission_resolved` | Some client voted on a permission via `POST /permission/:requestId` |
| `model_switched` | `POST /session/:id/model` succeeded |
| `model_switch_failed` | `POST /session/:id/model` rejected |
| `session_died` | Agent child crashed unexpectedly. **Terminal: SSE stream closes after this frame; the session is gone from `byId`.** Subscribers should reconnect via `POST /session` to spawn a fresh one. |
| `client_evicted` | Subscriber-local: queue overflow. **Terminal: SSE stream closes after this frame** (no `id` — synthetic). Other subscribers on the same session continue. |
| `stream_error` | Daemon-side error during fan-out. **Terminal: SSE stream closes after this frame** (no `id` — synthetic). |
| Event type | Trigger |
| --------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
| `session_update` | Any ACP `sessionUpdate` notification (LLM chunks, tool calls, usage) |
| `permission_request` | Agent asked for tool approval |
| `permission_resolved` | Some client voted on a permission via `POST /permission/:requestId` |
| `model_switched` | `POST /session/:id/model` succeeded |
| `model_switch_failed` | `POST /session/:id/model` rejected |
| `session_died` | Agent child crashed unexpectedly. **Terminal: SSE stream closes after this frame; the session is gone from `byId`.** Subscribers should reconnect via `POST /session` to spawn a fresh one. |
| `slow_client_warning` | Subscriber-local: queue ≥ 75% full. **Non-terminal** — the stream continues; the warning is a heads-up before eviction. Carries `{queueSize, maxQueued, lastEventId}`. Fires ONCE per overflow episode; re-arms after the queue drains below 37.5%. No `id` (synthetic). Pre-flight `caps.features.slow_client_warning`. |
| `client_evicted` | Subscriber-local: queue overflow. **Terminal: SSE stream closes after this frame** (no `id` — synthetic). Other subscribers on the same session continue. |
| `stream_error` | Daemon-side error during fan-out. **Terminal: SSE stream closes after this frame** (no `id` — synthetic). |

Reconnect semantics:

- Send `Last-Event-ID: <n>` to replay events with `id > n` from the per-session ring (default depth 4000)
- Send `Last-Event-ID: <n>` to replay events with `id > n` from the per-session ring (default depth **8000**, tunable via `qwen serve --event-ring-size <n>`)
- **Gap detection (client-side):** if `<n>` predates the oldest event still in the ring (e.g. you reconnect with `Last-Event-ID: 50` but the ring now holds 200–1199), the daemon replays from the oldest available event without raising. Compare the first replayed event's `id` against `n + 1`; any difference is the size of the lost window. Stage 2 will inject an explicit `stream_gap` synthetic frame on the daemon side; in Stage 1 detection is the client's responsibility.
- IDs are monotonic per session, starting at 1
- Synthetic terminal frames (`client_evicted`, `stream_error`) intentionally omit `id` so they don't burn a sequence slot for other subscribers
- Synthetic frames (`client_evicted`, `slow_client_warning`, `stream_error`) intentionally omit `id` so they don't burn a sequence slot for other subscribers

Backpressure:

- Per-subscriber queue defaults to `maxQueued: 256` live items (replay frames during reconnect bypass the cap)
- On overflow the bus emits the `client_evicted` terminal frame and closes the subscription
- Per-subscriber queue defaults to `maxQueued: 256` live items (replay frames during reconnect bypass the cap). Override via `?maxQueued=N` (range `[16, 2048]`) on the SSE request.
- When a subscriber's queue crosses 75% full the bus force-pushes a `slow_client_warning` synthetic frame to that subscriber (once per overflow episode; re-armed after drain below 37.5%). The stream stays open — the warning is a heads-up so the client can drain faster or detach + reconnect cleanly.
- If the queue actually overflows the warning, the bus emits the `client_evicted` terminal frame and closes the subscription.

### `POST /permission/:requestId`

Expand Down
Loading
Loading