Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
73 changes: 73 additions & 0 deletions docs/design/2026-07-07-bounded-replay-snapshot-window.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,73 @@
# Bounded Replay Snapshot Window

## Problem

Live daemon sessions currently retain replay history in memory so `POST /session/:id/load` can inject replay for clients that attach after the session already exists. That replay retention must be bounded independently from the SSE ring: response-mode restore can seed large historical updates in bulk, and completed live turns can accumulate indefinitely in long-running sessions.

Disk session history remains the authoritative full transcript source. PR-1 only bounds the daemon's live in-memory replay window; it does not add a full-transcript endpoint.

## Goals

- Cap retained replay events by serialized bytes per live session, defaulting to 4 MiB and rejecting invalid configuration at boot.
- Apply the cap to both completed live-turn replay segments and response-mode or stream-mode restored historical replay.
- Preserve the existing snapshot wire shape: `compactedReplay`, `liveJournal`, and `lastEventId`.
- Keep at least one real replay event or one completed live-turn segment even when that single unit exceeds the cap.
- Surface truncation with an id-less `history_truncated` marker at the start of `compactedReplay`.
- Treat `history_truncated` as status only. It must not trigger `state_resync_required`, reload loops, or persistence back into the replay window.

## Non-Goals

- No cap on a single in-flight live turn in PR-1; `liveJournal` continues to hold the active turn until a boundary.
- No turn-count cap. Turn counts are diagnostic only when the engine can count dropped completed turn segments exactly.
- No `/capabilities` feature tag for this additive event. The resolved limit is exposed in daemon status.
- No complete transcript endpoint. PR-2 must design paginated or streaming transcript reads and must not expose a one-shot full array response.

## Design

`TurnBoundaryCompactionEngine` stores retained replay as ordered segments instead of an unbounded flat array. A completed live turn is one segment. Restore/bulk seed replay is stored as event-level segments so the oldest restore events can be discarded independently when the byte cap is exceeded.

Sizing reuses the EventBus safe JSON sizing semantics. Sizing failure logs diagnostics and counts that event as zero bytes so publish and seed paths keep their never-throws contract.

When `replayBytes > maxReplayBytes`, the engine drops oldest segments while more than one segment remains. It increments `truncatedEvents`, and increments `truncatedTurns` only for dropped live-turn segments. `snapshot()` flattens retained segments and prepends:

```json
{
"type": "history_truncated",
"data": {
"reason": "replay_window_exceeded",
"truncatedEvents": 12,
"retainedEvents": 8,
"maxBytes": 4194304,
"truncatedTurns": 3,
"fullTranscriptAvailable": false
}
}
```

The marker is synthetic and id-less. It is excluded from byte accounting and from transient replay retention. `ingest()`, `seed(snapshot)`, and `seedReplayEvents()` all filter it out so loading a bounded snapshot cannot compound markers.

`EventBus.seedReplayEvents()` assigns ids and timestamps to restore replay events, calls the compaction engine's dedicated seed method, and clears the SSE ring as before. This prevents bulk restore replay from being appended to `liveJournal`.

The CLI wiring passes one resolved cap through yargs, the fast-path parser, `ServeOptions`, server wiring, `BridgeOptions`, bridge status, and daemon status rendering. Invalid values (`0`, negative, non-integer, `NaN`, `Infinity`, or values above 256 MiB) fail closed.

SDK and WebUI know `history_truncated`, validate its payload, project it to view-state counters and transcript status, and render a terminal status line. The event is not an unknown/debug event and is not part of resync gating.

## Audit Notes

Round 1: A cap only on completed live turns is insufficient because response-mode restore can seed large historical replay without live boundaries. The design therefore adds `seedReplayEvents()` and event-level historical segments.

Round 2: Reusing `state_resync_required` for truncation would create reload loops because `/load` would keep returning the same bounded window. The design uses a separate status marker that never sets `awaitingResync`.

Round 3: A turn-count cap does not bound memory when one turn contains large tool output. PR-1 uses byte-only enforcement and leaves active-turn capping out of scope.

Round 4: Returning the full transcript as an array would recreate the same peak memory problem at request time. PR-2 is explicitly constrained to pagination or streaming.

Round 5: Empty replay after truncation would make clients lose all visible state. The engine preserves the newest segment even when oversized.

## Verification Plan

- Unit-test live turn trimming, restore seed trimming, marker placement, transient marker filtering, oversized latest retention, safe sizing failure, and EventBus never-throws behavior.
- Unit-test bridge response-mode restore and live-session load behavior with the bounded window.
- Unit-test CLI parsing, fast-path parsing, runQwenServe validation, server bridge wiring, and daemon status limits.
- Unit-test SDK known-event validation, reducer state, UI normalizer, transcript status, terminal rendering, and WebUI replay injection.
- Keep final verification on `npm run build`, `npm run typecheck`, and `npm run lint`.
4 changes: 2 additions & 2 deletions docs/developers/daemon/01-architecture.md
Original file line number Diff line number Diff line change
Expand Up @@ -196,7 +196,7 @@ sequenceDiagram
Note over EB,SR: If subscriber queue >= maxQueued,<br/>EventBus emits client_evicted terminal frame<br/>and closes subscriber.
```

The ring buffer is bounded (`eventRingSize`, default 8000). A reconnecting client whose `Last-Event-ID` is older than the ring's head receives a synthetic catch-up signal and must call `loadSession` / `resumeSession` to rebuild deeper state. Slow clients trigger `slow_client_warning` at 75% queue fill and `client_evicted` at the cap.
The ring buffer is bounded (`eventRingSize`, default 8000). A reconnecting client whose `Last-Event-ID` is older than the ring's head receives `state_resync_required` and must rebuild from `loadSession`'s bounded replay snapshot window or use `resumeSession` when it already has local history. Slow clients trigger `slow_client_warning` at 75% queue fill and `client_evicted` at the cap.

## Workflow 3: Multi-client permission mediation

Expand Down Expand Up @@ -327,7 +327,7 @@ The two-phase shutdown matters because in-flight HTTP requests, in-flight SSE su

| Concern | File |
| -------------------- | ----------------------------------------------------------- |
| Bootstrap | `packages/cli/src/serve/run-qwen-serve.ts` |
| Bootstrap | `packages/cli/src/serve/run-qwen-serve.ts` |
| Express app | `packages/cli/src/serve/server.ts` |
| Capability registry | `packages/cli/src/serve/capabilities.ts` |
| Auth middleware | `packages/cli/src/serve/auth.ts` |
Expand Down
10 changes: 7 additions & 3 deletions docs/developers/daemon/03-acp-bridge.md
Original file line number Diff line number Diff line change
Expand Up @@ -242,9 +242,13 @@ In addition to the core `spawnOrAttach`, `sendPrompt`, `cancelSession`,

`BridgeSpawnRequest.sessionScope` was renamed from `'per-client'` to
`'thread'`. `BridgeRestoredSession` now carries `compactedReplay`,
`liveJournal`, and `lastEventId`. `BridgeClientRequestContext` is the request
context threaded through bridge calls; it carries `clientId`,
`fromLoopback: boolean`, and `promptId`.
`liveJournal`, and `lastEventId`. Those replay fields are a bounded in-memory
window for live sessions, capped by `BridgeOptions.compactedReplayMaxBytes`
(default 4 MiB, hard ceiling 256 MiB). If older retained replay was dropped,
`compactedReplay[0]` is the id-less `history_truncated` marker. The full
persisted transcript remains on disk and is not exposed by this bridge response.
`BridgeClientRequestContext` is the request context threaded through bridge
calls; it carries `clientId`, `fromLoopback: boolean`, and `promptId`.

## Caveats & Known Limits

Expand Down
14 changes: 11 additions & 3 deletions docs/developers/daemon/08-session-lifecycle.md
Original file line number Diff line number Diff line change
Expand Up @@ -100,7 +100,7 @@ sequenceDiagram

### Load / resume

`POST /session/:id/load` — replays full ACP history (`session/load` notifications fire before the response returns).
`POST /session/:id/load` — restores a persisted session and returns the current bounded replay snapshot window (`session/load` notifications or response-mode replay are seeded before the response returns).
`POST /session/:id/resume` — restores without replay (`connection.unstable_resumeSession`, exposed under the stable `session_resume` daemon capability; `unstable_session_resume` remains a deprecated alias).

Both:
Expand Down Expand Up @@ -259,11 +259,19 @@ not as a transport error.

`POST /session/:id/load` now returns a `BridgeRestoredSession` that can include
`compactedReplay?: BridgeEvent[]`, `liveJournal?: BridgeEvent[]`, and
`lastEventId?: number`. `compactedReplay` is produced by
`lastEventId?: number`. These fields are the daemon's bounded in-memory replay
window for a live session, not a full transcript API. The default window cap is
4 MiB per live session (`--compacted-replay-max-bytes`), and boot rejects
invalid caps; the hard ceiling is 256 MiB. `compactedReplay` is produced by
`TurnBoundaryCompactionEngine`: at turn boundaries it folds consecutive text /
thought blocks, collapses tool-call sequences to their final state, discards
transient signals, and produces O(turns) replay logs instead of O(tokens) logs
(typically a 25-30x reduction).
(typically a 25-30x reduction). When older replay entries have been dropped
from that byte window, `compactedReplay[0]` is a synthetic id-less
`history_truncated` marker with `{reason: 'replay_window_exceeded',
truncatedEvents, retainedEvents, maxBytes, truncatedTurns?,
fullTranscriptAvailable: false}`. Clients should render it as status and apply
the retained replay normally; it must not trigger a resync loop.

### ACP Child Preheat

Expand Down
5 changes: 3 additions & 2 deletions docs/developers/daemon/09-event-schema.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@

## Overview

Every SSE frame emitted by the daemon on `GET /session/:id/events` has the shape `{ id, v, type, data, originatorClientId?, _meta? }`. `v: 1` is the current `EVENT_SCHEMA_VERSION`. `type` comes from the closed, version-pinned `DAEMON_KNOWN_EVENT_TYPE_VALUES` set in `packages/sdk-typescript/src/daemon/events.ts`; the current set has 47 known event types. The envelope `_meta` field is stamped at the SSE write boundary by `formatSseFrame()` in `packages/cli/src/serve/routes/sse-events.ts`; see [Envelope-level metadata](#envelope-level-metadata).
Every SSE frame emitted by the daemon on `GET /session/:id/events` has the shape `{ id, v, type, data, originatorClientId?, _meta? }`. `v: 1` is the current `EVENT_SCHEMA_VERSION`. `type` comes from the closed, version-pinned `DAEMON_KNOWN_EVENT_TYPE_VALUES` set in `packages/sdk-typescript/src/daemon/events.ts`. The envelope `_meta` field is stamped at the SSE write boundary by `formatSseFrame()` in `packages/cli/src/serve/routes/sse-events.ts`; see [Envelope-level metadata](#envelope-level-metadata).

The SDK exposes `asKnownDaemonEvent(evt)`. It returns a discriminated `KnownDaemonEvent` for known event types and `undefined` for other types. SDK consumers can therefore handle forward compatibility without requiring a lockstep SDK upgrade when a newer daemon adds an event type; the session reducer records those as `unrecognizedKnownEventCount`.

Expand All @@ -15,7 +15,7 @@ The wire format lives in [`../qwen-serve-protocol.md`](../qwen-serve-protocol.md
- Provide pure reducers (`reduceDaemonSessionEvent`, `reduceDaemonAuthEvent`) that project an event stream into SDK view state.
- Broadcast the `typed_event_schema` capability tag as an informational signal. If the tag is absent, `asKnownDaemonEvent` still falls back to `unknown`.

## Event vocabulary (47 known types)
## Event vocabulary

Grouped by domain.

Expand All @@ -37,6 +37,7 @@ Grouped by domain.
| `slow_client_warning` | Live frame backlog or live serialized-byte backlog >= 75%; force-pushed and **has no `id`** | `queueSize, maxQueued, lastEventId, queuedBytes?, maxQueuedBytes?, threshold?: 'frames' \| 'bytes' \| 'frames_and_bytes'`; re-armed after both frame and byte measurements drop below 37.5%. |
| `stream_error` | `SubscriberLimitExceededError` or another route stream error | `error: string`; terminal for the subscription. |
| `state_resync_required` | `subscribe({lastEventId})` detects that the daemon ring no longer holds `[lastEventId+1, earliestInRing-1]`, or the client cursor is from a previous bus epoch. Force-pushed **before** remaining replay frames and **has no `id`**. | `reason: 'ring_evicted' \| 'epoch_reset' \| string`, `lastDeliveredId: number`, `earliestAvailableId: number`. This is a recovery signal, not terminal: the SSE stream stays open and replay + live frames continue. The SDK reducer sets `awaitingResync = true` and skips deltas until the caller resets with `loadSession`. |
| `history_truncated` | `POST /session/:id/load` returns a bounded replay snapshot after older in-memory replay entries were dropped. Prepended to `compactedReplay` and **has no `id`**. | `reason: 'replay_window_exceeded'`, `truncatedEvents: number`, `retainedEvents: number`, `maxBytes: number`, `truncatedTurns?: number`, `fullTranscriptAvailable: false`. This is a status marker, not a resync request; clients render it and continue applying retained replay. |
| `replay_complete` | Id-less sentinel emitted after the `Last-Event-ID` replay loop finishes, for both clean replay and ring-evicted paths, even when `data.replayedCount === 0`. **No `id`** | `replayedCount: number`; lets consumers remove catch-up UI deterministically without a timeout. |

### Permissions (F3 + base)
Expand Down
14 changes: 7 additions & 7 deletions docs/developers/daemon/10-event-bus.md
Original file line number Diff line number Diff line change
Expand Up @@ -239,13 +239,13 @@ The daemon's `EventBus` replays all events from the ring buffer whose `id > Last

### Replay Behavior

| Scenario | Behavior |
| -------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `Last-Event-ID` absent | Live-only stream; no replay. Backward-compatible with pre-resume clients. |
| `Last-Event-ID: 0` | Replay entire ring buffer from the beginning (bounded by `--event-ring-size`, default 8000). |
| `Last-Event-ID: N` where `ring[0].id <= N+1` | Contiguous replay of events `id > N`, then live. |
| `Last-Event-ID: N` where `ring[0].id > N+1` | Gap detected — `state_resync_required` (`reason: 'ring_evicted'`) emitted before replay of surviving suffix. SDK must call `loadSession` to recover full state. |
| `Last-Event-ID: N` where `N >= nextId` | Epoch reset (daemon restart) — `state_resync_required` (`reason: 'epoch_reset'`) emitted, then full ring replay. |
| Scenario | Behavior |
| -------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `Last-Event-ID` absent | Live-only stream; no replay. Backward-compatible with pre-resume clients. |
| `Last-Event-ID: 0` | Replay entire ring buffer from the beginning (bounded by `--event-ring-size`, default 8000). |
| `Last-Event-ID: N` where `ring[0].id <= N+1` | Contiguous replay of events `id > N`, then live. |
| `Last-Event-ID: N` where `ring[0].id > N+1` | Gap detected — `state_resync_required` (`reason: 'ring_evicted'`) emitted before replay of surviving suffix. SDK must call `loadSession` to recover a bounded replay snapshot window; the returned `compactedReplay` may begin with `history_truncated` if older in-memory replay entries were dropped. |
| `Last-Event-ID: N` where `N >= nextId` | Epoch reset (daemon restart) — `state_resync_required` (`reason: 'epoch_reset'`) emitted, then full ring replay. |

### Validation Rules

Expand Down
8 changes: 6 additions & 2 deletions docs/developers/daemon/13-sdk-daemon-client.md
Original file line number Diff line number Diff line change
Expand Up @@ -302,10 +302,14 @@ async function* subscribe(sessionId: string, signal: AbortSignal) {
}
// Handle ring-eviction gap.
if (event.type === 'state_resync_required') {
// State is stale — reload full session state.
// State is stale — reload the daemon's bounded replay snapshot window.
await client.loadSession(sessionId);
continue;
}
if (event.type === 'history_truncated') {
// Informational only. Render a status notice, then continue applying
// the retained replay events; do not trigger another reload.
}
yield event;
}
}
Expand Down Expand Up @@ -336,7 +340,7 @@ async function resilientSubscribe(session: DaemonSessionClient) {
}
```

On reconnect the daemon replays events with `id > lastSeenEventId` from its bounded ring (default 8000 events). If the gap exceeds the ring, a `state_resync_required` frame signals the client to call `loadSession` for a full state rebuild.
On reconnect the daemon replays events with `id > lastSeenEventId` from its bounded ring (default 8000 events). If the gap exceeds the ring, a `state_resync_required` frame signals the client to call `loadSession` and rebuild from the current bounded replay snapshot window. That snapshot may begin with `history_truncated`; treat it as an operator-visible status marker, not as another resync request.

### Seeding `lastEventId` at Construction

Expand Down
Loading
Loading