feat(kap-server): add flat entity message protocol (v3 WS + history API)

apps/kimi-inspect/AGENTS.md

@@ -14,7 +14,7 @@ A left icon rail (`src/components/NavRail.tsx`) switches top-level views:
 
 The **Agent scope** stays in the Chat view's right dock (`src/components/RightPanel.tsx` — Audit / Agent / State / Session tabs) across two of them:
 
-- `Agent` tab — `Inspector`: agent switcher + a Plan lookup card (`PlanCard` in `src/components/Inspector.tsx` — querying `GET /sessions/{id}/transcript/plan` (one tool_call_id, or every plan of the agent) via `src/transcript/api.ts`'s `fetchTranscriptPlan`) plus the agent Service panels.
+- `Agent` tab — `Inspector`: agent switcher + a Plan lookup card (`PlanCard` in `src/components/Inspector.tsx` — deriving the reviewed plan of one ExitPlanMode tool call, or every plan of the agent, from the message stream: a full `GET /sessions/{id}/history` read via `src/transcript/api.ts`'s `fetchFullHistory` + client-side `projectPlans` in `src/transcript/plan.ts`) plus the agent Service panels.
 - `State` tab — every key an Agent Service registered into the agent-state container, polled live via `IAgentStateService.snapshot()` — the same live diff-tree view as the session State tab, sharing `StateCard` from `src/components/StateCard.tsx`.
 
 The **Session scope** lives in the same right dock as the `Session` tab (`src/components/SessionPane.tsx`, embedded by `RightPanel`) with two sub-tabs: Services (the pending-interactions card — `src/components/InteractionsCard.tsx`, which lists and answers approvals/questions over the public REST endpoints `/api/v1/sessions/{id}/approvals|questions` via `src/interactions/api.ts`, since the interaction kernel is a process-global singleton with no debug channel — plus the session Service panels) and State (every key a Session Service registered into the session-state container, read on demand via `ISessionStateService.snapshot()`).
@@ -33,12 +33,12 @@ The Vite dev server proxies `/api` to a running kap-server (`KIMI_SERVER_URL`, d
 
 ## Chat view
 
-The per-session chat (`src/components/ChatView.tsx`) renders turn-granularly from the **transcript** surface instead of context memory and carries an in-chat search bar (`src/components/ChatSearchBar.tsx`): it searches the current session via `POST /api/v1/search` with `container: { session_id }` (usually served by the live route, since selecting a session resumes it), and a result click funnels through the app shell's `openSearchHit` — the same agent-switch + `ChatJump` (page-back, scroll, flash) path the global search view uses.
+The per-session chat (`src/components/ChatView.tsx`) renders turn-granularly from the **message protocol v3** surface and carries an in-chat search bar (`src/components/ChatSearchBar.tsx`): it searches the current session via `POST /api/v1/search` with `container: { session_id }` (usually served by the live route, since selecting a session resumes it), and a result click funnels through the app shell's `openSearchHit` — the same agent-switch + `ChatJump` (page-back, scroll, flash) path the global search view uses.
 
-Full state is read from `GET /api/v1/sessions/{id}/transcript` (initial load = newest page, refreshes re-read from the tail backwards), older history auto-pages with `before_turn` via an IntersectionObserver sentinel at the top of the scroll view, and each timeline item is wrapped in `content-visibility: auto` + `contain-intrinsic-size` so the browser virtualizes off-screen rendering natively (no windowing library).
+Persisted state comes from `GET /api/v1/sessions/{id}/history` only (client in `src/transcript/api.ts`): the initial load reads the newest page (default 500 messages, replace mode), older history auto-pages with `before_turn` via an IntersectionObserver sentinel at the top of the scroll view (prepend mode; a short or empty page ends paging — the response deliberately carries no has-more flag), and each timeline item is wrapped in `content-visibility: auto` + `contain-intrinsic-size` so the browser virtualizes off-screen rendering natively (no windowing library).
 
-`/api/v1/ws` is an incremental channel (`transcript.ops`, grade `block` — the cheapest grade that still carries whole-state frame upserts, dropping per-token `append` frames; `transcript.reset` is ignored by the store, surfaced only to the audit recorder via the optional `onReset` handler). The channel tracks the op-batch watermark: a dedicated `subscribe_v2` control frame carries the per-agent grades and the `transcript_since` cursor, a seq gap / reconnect / `resync_required` / append gap triggers a point-to-point catch-up (`fetchTranscriptOps` → `GET .../transcript/ops?since_seq=`), and any legacy/incomplete answer falls back to the full REST refresh. Convergence reuses `@moonshot-ai/transcript`'s L2 reducer (`src/transcript/`: REST/WS clients + store; the data model and reducer come from the package, nothing is re-implemented locally).
+`/api/v3/ws` (client in `src/transcript/ws.ts`) is the live channel: server `hello` → `subscribe {id, session_id, agent_ids: [agent]}` → `ack` → recovery payload (in-flight entities + pending interactions + running tasks + todo + `session.state`) → live traffic, heartbeat at the WS protocol level. The store (`src/transcript/store.ts`) applies recovery and live messages through one idempotent path — entity messages upsert by (type, own id) with content fields authoritative, the delta family (`assistant.delta` / `thinking.delta` / `tool_call.delta`) appends by id, `tool.progress` patches the entity, `system(undo/clear)` truncates the timeline by `payload.removed_ids` (subtree included, linked interactions cascaded), and `interaction` / `task` / `todo` / `session.state` upsert their own single-source maps; an upsert older than the held entity's `timestamp` is skipped. Notifications are throttled trailing-edge so the per-token delta stream does not re-render per token. Every subscribe ack (initial and every reconnect) triggers an `after_step` catch-up from the newest terminal step; an empty catch-up whose anchor vanished (undo/clear while away) falls back to a full refresh. All of this is orchestrated by `ChatChannel` (`src/transcript/channel.ts`) — no buffering, no cursors beyond the two REST page cursors, no reset frames.
 
 ## Transcript audit panel
 
-The Transcript audit panel (`src/components/audit/`, the `Audit` tab of the chat view's right dock — `src/components/RightPanel.tsx`, fed the trail by `ChatView`'s `onTrailChange`) replays how the visible store was built: an `AuditTrail` (`src/audit/`) records every step — each REST page (request + replace/prepend), every WS frame (`transcript.ops` live/buffered/flushed/catchup, `transcript.reset`), loss signals, and prompt/cancel actions — with the resulting immutable `AgentState` per entry; the panel offers a draggable timeline plus a Diff tab (structural diff vs the previous entry: added/modified/removed colored, long strings tail-truncated, all fields kept), a full State view, and the raw Event payload.
+The Transcript audit panel (`src/components/audit/`, the `Audit` tab of the chat view's right dock — `src/components/RightPanel.tsx`, fed the trail by `ChatView`'s `onTrailChange`) replays how the visible store was built: an `AuditTrail` (`src/audit/`) records every step — each REST history page (request + replace/prepend/tail mode), every WS message (entity/delta/state as applied), channel events (subscribe ack, reconnect, catch-up fallback, protocol errors), and prompt/cancel actions — with the resulting immutable `ChatState` per entry; the panel offers a draggable timeline plus a Diff tab (structural diff vs the previous entry: added/modified/removed colored, long strings tail-truncated, all fields kept), a full State view (the flat entity timeline plus the interaction/task/todo/session.state entities), and the raw Event payload.

apps/kimi-inspect/package.json

@@ -23,7 +23,7 @@
   },
   "dependencies": {
     "@moonshot-ai/agent-core-v2": "workspace:^",
-    "@moonshot-ai/transcript": "workspace:^",
+    "@moonshot-ai/kap-server": "workspace:^",
     "@tanstack/react-query": "^5.74.4",
     "react": "^19.1.0",
     "react-dom": "^19.1.0"

apps/kimi-inspect/src/audit/audit.test.ts

@@ -3,27 +3,57 @@
  * and tail-preserving truncation used by the chat view's audit panel.
  */
 
-import { EMPTY_AGENT_STATE, type AgentState, type TranscriptTurn } from '@moonshot-ai/transcript';
+import type { StepMessage, TurnMessage } from '@moonshot-ai/kap-server/protocol';
 import { describe, expect, it } from 'vitest';
 
+import { EMPTY_CHAT_STATE, type ChatState } from '../transcript/store';
 import { diffValue, type DiffNode } from './diff';
 import { serializeState } from './serialize';
 import { AuditTrail, AUDIT_TRAIL_MAX_ENTRIES } from './trail';
 import { tailTrunc } from './truncate';
 
-function turnItem(n: number): TranscriptTurn {
+const T0 = Date.parse('2026-01-01T00:00:00.000Z');
+let tick = 0;
+
+function ts(): number {
+  tick += 1;
+  return T0 + tick * 1000;
+}
+
+function turnMsg(n: number, status: 'running' | 'completed' = 'completed'): TurnMessage {
   return {
-    kind: 'turn',
-    turnId: `t${n}`,
+    type: 'turn',
+    session_id: 's1',
+    agent_id: 'main',
+    timestamp: ts(),
+    turn_id: `t${n}`,
     ordinal: n,
-    state: 'completed',
+    status,
     origin: { kind: 'user' },
-    steps: [],
   };
 }
 
-function stateWith(items: readonly TranscriptTurn[]): AgentState {
-  return { ...EMPTY_AGENT_STATE, items };
+function stepMsg(stepId: string, status: 'running' | 'completed'): StepMessage {
+  return {
+    type: 'step',
+    session_id: 's1',
+    agent_id: 'main',
+    timestamp: ts(),
+    step_id: stepId,
+    turn_id: stepId.split('.')[0] ?? 't1',
+    ordinal: Number(stepId.split('.')[1] ?? '1'),
+    status,
+  };
+}
+
+function stateWithTimeline(items: readonly (TurnMessage | StepMessage)[]): ChatState {
+  return {
+    ...EMPTY_CHAT_STATE,
+    entries: items.map((message) => ({
+      key: message.type === 'turn' ? `turn:${message.turn_id}` : `step:${message.step_id}`,
+      message,
+    })),
+  };
 }
 
 // ---------------------------------------------------------------- diff
@@ -53,39 +83,35 @@ describe('diffValue', () => {
   });
 
   it('matches entity arrays by id instead of index', () => {
-    const prev = [turnItem(1), turnItem(2)];
-    const next = [turnItem(1), { ...turnItem(2), state: 'running' as const }, turnItem(3)];
+    const t1 = turnMsg(1);
+    const t2 = turnMsg(2);
+    const prev = [t1, t2];
+    const next = [t1, { ...t2, status: 'running' as const }, turnMsg(3)];
     const node = diffValue(prev, next);
     expect(node.children?.get('t1')?.status).toBe('unchanged');
     expect(node.children?.get('t2')?.status).toBe('modified');
-    expect(node.children?.get('t2')?.children?.get('state')).toMatchObject({
+    expect(node.children?.get('t2')?.children?.get('status')).toMatchObject({
       status: 'modified',
       prev: 'completed',
       value: 'running',
     });
     expect(node.children?.get('t3')?.status).toBe('added');
   });
 
-  it('keys steps by stepId (not their shared turnId) so siblings never collide', () => {
-    const step = (id: string, state: 'running' | 'completed') => ({
-      kind: 'step' as const,
-      stepId: id,
-      turnId: 't1',
-      ordinal: 1,
-      state,
-      frames: [],
-    });
+  it('keys steps by step_id (not their shared turn_id) so siblings never collide', () => {
+    const done = stepMsg('t1.1', 'completed');
     const node = diffValue(
-      [step('t1.1', 'completed'), step('t1.2', 'completed')],
-      [step('t1.1', 'completed'), step('t1.2', 'running')],
+      [done, stepMsg('t1.2', 'completed')],
+      [done, stepMsg('t1.2', 'running')],
     );
     expect([...(node.children?.keys() ?? [])]).toEqual(['t1.1', 't1.2']);
     expect(node.children?.get('t1.1')?.status).toBe('unchanged');
     expect(node.children?.get('t1.2')?.status).toBe('modified');
   });
 
   it('marks removed array elements by id', () => {
-    const node = diffValue([turnItem(1), turnItem(2)], [turnItem(2)]);
+    const t2 = turnMsg(2);
+    const node = diffValue([turnMsg(1), t2], [t2]);
     expect(node.children?.get('t1')).toMatchObject({ status: 'removed' });
     expect(node.children?.get('t2')?.status).toBe('unchanged');
   });
@@ -105,48 +131,69 @@ describe('diffValue', () => {
     expect(diffValue([1], { 0: 1 }).status).toBe('modified');
   });
 
-  it('diffs two serialized states with meta changes visible (goal/plan fields)', () => {
-    const prev = serializeState(stateWith([turnItem(1)]));
-    const nextState: AgentState = {
-      ...stateWith([turnItem(1)]),
-      meta: {
+  it('diffs two serialized states with session.state changes visible', () => {
+    const base = stateWithTimeline([turnMsg(1)]);
+    const prev = serializeState(base);
+    const nextState: ChatState = {
+      ...base,
+      sessionState: {
+        type: 'session.state',
+        session_id: 's1',
+        timestamp: ts(),
+        status: 'running',
         goal: { objective: 'ship it', status: 'active' },
-        modes: { plan: { reviewPath: '/tmp/plan.md' } },
+        modes: { plan: { review_path: '/tmp/plan.md' } },
       },
     };
     const node: DiffNode = diffValue(prev, serializeState(nextState));
-    expect(node.children?.get('items')?.status).toBe('unchanged');
-    const meta = node.children?.get('meta');
-    expect(meta?.status).toBe('modified');
-    expect(meta?.children?.get('goal')?.status).toBe('added');
-    // Whole-subtree add: `modes` was absent before, so the block (plan
-    // included) is marked added without descending into children.
-    expect(meta?.children?.get('modes')?.status).toBe('added');
-    expect(meta?.children?.get('modes')?.children).toBeUndefined();
+    expect(node.children?.get('timeline')?.status).toBe('unchanged');
+    const sessionState = node.children?.get('sessionState');
+    expect(sessionState?.status).toBe('added');
+    expect(sessionState?.children).toBeUndefined();
   });
 });
 
 // ---------------------------------------------------------------- serialize
 
 describe('serializeState', () => {
-  it('turns maps into sorted plain objects and sets into arrays', () => {
-    const state: AgentState = {
-      ...EMPTY_AGENT_STATE,
+  it('turns maps into sorted plain objects and flattens the timeline', () => {
+    const state: ChatState = {
+      ...EMPTY_CHAT_STATE,
+      entries: stateWithTimeline([turnMsg(1)]).entries,
       tasks: new Map([
         [
           'b-task',
-          { taskId: 'b-task', kind: 'shell', state: 'running', detached: false, outputTail: '' },
+          {
+            type: 'task',
+            session_id: 's1',
+            agent_id: 'main',
+            timestamp: ts(),
+            task_id: 'b-task',
+            kind: 'shell',
+            status: 'running',
+            detached: false,
+            output_tail: '',
+          },
         ],
         [
           'a-task',
-          { taskId: 'a-task', kind: 'tool', state: 'completed', detached: false, outputTail: '' },
+          {
+            type: 'task',
+            session_id: 's1',
+            agent_id: 'main',
+            timestamp: ts(),
+            task_id: 'a-task',
+            kind: 'tool',
+            status: 'completed',
+            detached: false,
+            output_tail: '',
+          },
         ],
       ]),
-      pendingInteractions: new Set(['z', 'a']),
     };
     const out = serializeState(state);
-    expect(Object.keys(out.tasks as Record<string, unknown>)).toEqual(['a-task', 'b-task']);
-    expect(out.pendingInteractions).toEqual(['a', 'z']);
+    expect(Object.keys(out.tasks)).toEqual(['a-task', 'b-task']);
+    expect(out.timeline.map((m) => (m.type === 'turn' ? m.turn_id : ''))).toEqual(['t1']);
     expect(out.hasMoreOlder).toBe(false);
   });
 });
@@ -171,37 +218,20 @@ describe('tailTrunc', () => {
 // ---------------------------------------------------------------- trail
 
 describe('AuditTrail', () => {
-  const page = {
-    items: [turnItem(1)],
-    hasMoreOlder: false,
-    tasks: [],
-    interactions: [],
-    attachments: [],
-    todos: [],
-    meta: {},
-    pendingInteractions: [],
-  };
-
   it('records entries with increasing indices, timestamps, and state references', () => {
     const trail = new AuditTrail();
-    const s1 = stateWith([turnItem(1)]);
-    const s2 = stateWith([turnItem(1), turnItem(2)]);
-    trail.recordRest({ pageSize: 30 }, 'replace', page, s1);
-    trail.recordOps([{ op: 'turn.upsert', turn: turnItem(2) }], 'live', '2026-01-01T00:00:00Z', s2);
+    const s1 = stateWithTimeline([turnMsg(1)]);
+    const s2 = stateWithTimeline([turnMsg(1), turnMsg(2)]);
+    trail.recordRest({ pageSize: 500 }, 'replace', 1, { turn_id: 't1', step_id: 't1.1' }, s1);
+    trail.recordWs(turnMsg(2, 'running'), s2);
     trail.recordEvent('prompt', 'hello', s2);
-    trail.recordReset(
-      { items: [], tasks: [], interactions: [], attachments: [], todos: [], prompts: [], meta: {} },
-      false,
-      undefined,
-      s2,
-    );
 
     const entries = trail.getEntries();
-    expect(entries.map((entry) => entry.kind)).toEqual(['rest', 'ops', 'event', 'reset']);
-    expect(entries.map((entry) => entry.index)).toEqual([0, 1, 2, 3]);
+    expect(entries.map((entry) => entry.kind)).toEqual(['rest', 'ws', 'event']);
+    expect(entries.map((entry) => entry.index)).toEqual([0, 1, 2]);
     expect(entries[0]!.state).toBe(s1);
     expect(entries[1]!.state).toBe(s2);
-    expect(entries[1]).toMatchObject({ delivery: 'live', envelopeAt: '2026-01-01T00:00:00Z' });
+    expect(entries[0]).toMatchObject({ mode: 'replace', messageCount: 1 });
     expect(entries[2]).toMatchObject({ event: 'prompt', detail: 'hello' });
     expect(entries.every((entry) => typeof entry.at === 'string' && entry.at.length > 0)).toBe(
       true,
@@ -215,18 +245,18 @@ describe('AuditTrail', () => {
     const unsubscribe = trail.subscribe(() => {
       notified += 1;
     });
-    trail.recordEvent('cancel', undefined, EMPTY_AGENT_STATE);
-    trail.recordEvent('gap', undefined, EMPTY_AGENT_STATE);
+    trail.recordEvent('cancel', undefined, EMPTY_CHAT_STATE);
+    trail.recordEvent('ack', undefined, EMPTY_CHAT_STATE);
     expect(notified).toBe(2);
     unsubscribe();
-    trail.recordEvent('resync', undefined, EMPTY_AGENT_STATE);
+    trail.recordEvent('reconnect', undefined, EMPTY_CHAT_STATE);
     expect(notified).toBe(2);
   });
 
   it('drops the oldest entries beyond the cap while indices keep increasing', () => {
     const trail = new AuditTrail();
     for (let i = 0; i < AUDIT_TRAIL_MAX_ENTRIES + 10; i += 1) {
-      trail.recordEvent('prompt', `p${i}`, EMPTY_AGENT_STATE);
+      trail.recordEvent('prompt', `p${i}`, EMPTY_CHAT_STATE);
     }
     const entries = trail.getEntries();
     expect(entries).toHaveLength(AUDIT_TRAIL_MAX_ENTRIES);