feat(kap-server): add flat entity message protocol (v3 WS + history API) - #3532
Conversation
|
commit: |
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: e28327a85d
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
| for (const item of materializer.materialize(op)) { | ||
| target.send(this.buildV3Envelope(state, v3ItemFrame(item, seq))); |
There was a problem hiding this comment.
When one transcript op batch materializes into multiple items, this sends each as a separate frame carrying the same seq. If the connection drops after the client receives and persists that sequence from an early frame, reconnecting with transcript_since skips the entire batch, permanently losing the remaining messages; the client also has no marker indicating which frame is last. Send the materialized batch atomically or add an item index/count or completion marker so the watermark can be committed safely.
AGENTS.md reference: AGENTS.md:L28-L28
Useful? React with 👍 / 👎.
| id: `remove.${this.removeCounter}`, | ||
| session_id: env.sessionId, | ||
| agent_id: env.agentId, | ||
| ids: [...op.ids], |
There was a problem hiding this comment.
When context.undone removes a turn that has already streamed steps and frames, onContextUndone puts only top-level item IDs and anchored interaction IDs in op.ids, and this forwards that list unchanged. Because v3 exposes each step and frame as an independent message, applying this removal deletes the turn message but leaves its step_*, text, thinking, and tool messages orphaned, diverging from the REST snapshot where the whole turn subtree is gone. Include every flattened descendant ID in the removal.
Useful? React with 👍 / 👎.
ed38e86 to
319e6e4
Compare
sailist
changed the title
ff5795b to
704b4eb
Compare
704b4eb to
b552e27
Compare
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: b552e27182
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
| readonly sinceSeq: number; | ||
| /** Injectable for tests. */ | ||
| readonly fetchImpl?: typeof fetch; | ||
| return { messages: parsed.data.messages, inFlight: parsed.data.in_flight }; |
There was a problem hiding this comment.
The history response includes has_more, but this adapter discards it, while ChatChannel and fetchFullHistory infer continuation by comparing the returned flat-message count with the requested size of 500. The server caps page_size to 200 turn anchors for default/before_turn pages, so a history with additional older turns normally returns a count other than exactly 500 even when has_more is true; lazy chat paging and full-history plan lookup then stop early and silently omit older history. Return and consume parsed.data.has_more instead of deriving it from message count.
AGENTS.md reference: apps/kimi-inspect/AGENTS.md:L38-L38
Useful? React with 👍 / 👎.
| const end = Math.min(messages.length, index + 1 + pageSize); | ||
| return { messages: messages.slice(index + 1, end), hasMore: end < messages.length }; |
There was a problem hiding this comment.
When more than pageSize messages were persisted after the requested step, this arbitrary message slice can end before any later terminal step or partway through one. The API exposes only an after_step cursor: repeating the old cursor returns the same page, while advancing to a terminal step included in the page skips that step's remaining entities because the server searches for its last occurrence. A reconnect catch-up can therefore never retrieve the complete suffix; page at complete step boundaries or return a message-level continuation cursor.
Useful? React with 👍 / 👎.
| this.state = { ...this.state, entries: next }; | ||
| this.applyTruncations(messages); |
There was a problem hiding this comment.
When reconnect recovery falls back to a replace refresh after an undo or clear, this assignment replaces only entries and retains the old interaction/task/todo maps. If a pending interaction and its tool call were removed while disconnected, the rebuilt page contains neither entity; applyTruncations also cannot discover the removed tool because it is already absent from next, so the obsolete pending interaction remains rendered and subsequent attempts to answer it target a nonexistent request. Rebuild the state maps from the replacement page while preserving only demonstrably newer live entities.
AGENTS.md reference: apps/kimi-inspect/AGENTS.md:L40-L40
Useful? React with 👍 / 👎.
… and history has_more
…d when no task records exist
… filter empty thinking
b552e27 to
7f8bcbd
Compare
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 7f8bcbd048
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
| default: | ||
| return []; |
There was a problem hiding this comment.
When /sessions/:id:delete publishes event.session.deleted, this switch falls through to default, so connected v3 clients never receive the session message with subtype deleted that the protocol schema supports. Any client maintaining a session list from global v3 events therefore retains the deleted session until it performs a separate full refresh.
Useful? React with 👍 / 👎.
| applyLive(message: ServerMessage): void { | ||
| switch (message.type) { | ||
| case 'assistant.delta': { | ||
| this.patchText(`assistant:${message.message_id}`, message.text); |
There was a problem hiding this comment.
When a WS delta lands while a REST refresh is in flight, this appends the text but leaves the entity's timestamp at its initial value. A REST snapshot produced before the latest delta can consequently appear newer in preferHeld, overwrite the accumulated text, and make later deltas append after a missing chunk until another authoritative entity frame arrives; propagate the delta timestamp into the patched entity (and do the same for the analogous thinking/tool patches).
AGENTS.md reference: apps/kimi-inspect/AGENTS.md:L40-L40
Useful? React with 👍 / 👎.
| for (const [toolCallId, tool] of tools) { | ||
| if (tool.turnId === id) removedKeys.add(`tool:${toolCallId}`); |
There was a problem hiding this comment.
When an undone or cleared turn contains a tool with an approval or question, markRemoved removes the tool order key but not interaction records whose toolCallId points to it. The cold history response therefore still contains an orphan interaction, which a fresh inspector renders as an unanchored interaction at the bottom even though its turn no longer exists; cascade those interaction keys while the removed tool IDs are still available.
Useful? React with 👍 / 👎.
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 6c387a79fd
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
| const key = timelineKeyOf(message); | ||
| const index = this.state.entries.findIndex((entry) => entry.key === key); | ||
| if (index < 0) { | ||
| this.state = { ...this.state, entries: [...this.state.entries, { key, message }] }; |
There was a problem hiding this comment.
When reconnecting after missing a completed turn while a newer turn is in flight, the server sends recovery immediately after the ack, while the ack starts an asynchronous REST catch-up. The newer recovery turn therefore usually reaches the store first, and this append path subsequently places the missed, older catch-up turn after it; groupTimeline preserves that first-encounter order, so the chat and audit views show turns out of chronology. Tail pages need to be merged before already-newer recovery entities or recovery traffic must be held until catch-up completes.
AGENTS.md reference: apps/kimi-inspect/AGENTS.md:L40-L40
Useful? React with 👍 / 👎.
| info.path === undefined && revisionPaths.length > 0 | ||
| ? { ...info, path: revisionPaths.at(-1) } |
There was a problem hiding this comment.
When history contains multiple plan cycles and an older ExitPlanMode call lacks a path in its interaction, display, and output, this assigns revisionPaths.at(-1), even when that revision belongs to a later plan and occurs after the call. Querying the older call—or listing all plans—then displays the later plan's document path; associate revisions by their payload identity or timeline position instead of applying the globally newest path to every pathless plan.
AGENTS.md reference: apps/kimi-inspect/AGENTS.md:L17-L17
Useful? React with 👍 / 👎.
2 participants
Related Issue
N/A — internal protocol redesign per the "New API draft" design doc (flat, self-contained message union to replace the dual-track event/transcript streams).
Problem
kap-server currently ships two divergent delivery pipelines: 51 agent frame types + 19
event.*frame types (legacy lane) and transcript reset/ops (v2 lane). The same fact is projected twice, kept consistent by a hand-maintained suppression table; transcript's store/ops/granularity concepts force every consumer through an apply/store middle layer before WS data becomes usable; approvals, tasks, busy and other state domains have 2–3 competing sources of truth; and the declared schema has systematically drifted from what the server actually emits.What changed
Implements the next-generation protocol as a pure addition — v1/v2 lanes, legacy REST, and the transcript package are untouched, so existing clients keep working:
packages/kap-server/src/protocol/messages/): the zod single source for a flat union of 22 entity messages + 4 control messages (snake_case, no envelope, no seq/epoch/volatile/offset). Browser-safe; exported via the./protocolsubpath for clients to import directly.services/projection/): a single direct projector mapping agent-core-v2 event-bus events + service emitters + queryable services to entity messages, with zero dependency on the transcript package. Holds in-flight turn/step accumulation and current state entities; heals against wire.jsonl at turn end; validates every outbound message against the schema./api/v3/ws:hello → subscribe → ack → recovery payload → live. Recovery in three sentences: persisted state comes from REST, in-flight steps replay, state entities resend in full — no cursors, no journal, no seq/epoch. Per-session ordered sequence (recovery and live queued atomically); bounded outbound queue with a dedicatedWS_SLOW_CONSUMERdisconnect; protocol-level ping/pong.GET /api/v1/sessions/{id}/history: cold rebuild from wire.jsonl into the same flat entity messages (same schema, same id rules as the live projection), withbefore_turn/after_stepcursors and anin_flightmarker.system(undo,clear)truncation); dependency on@moonshot-ai/transcriptremoved.Zero changes to
agent-core-v2, to the v1/v2 lanes, to legacy REST, or to the transcript package.Verification: kap-server/kimi-inspect typecheck green; 67 contract + 24 projection + 20 WS + 27 history + 116 kimi-inspect tests green; 39/39 end-to-end smoke checks against a real server (handshake, live streaming, approvals, todo, subagent modes A/B/C, undo/clear, all three recovery scenarios, REST↔WS convergence); full repo suite green except pre-existing agent-core-v2 zip environment failures.
Checklist
gen-changesetsskill, or this PR needs no changeset. (No changeset: internal server protocol surface + kimi-inspect; not user-perceivable.)gen-docsskill, or this PR needs no doc update.