Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
64 changes: 64 additions & 0 deletions docs/design/web-shell-thinking-stream-performance.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,64 @@
# Web Shell thinking stream performance

## Problem

Streaming thought deltas update the collapsed "Thinking" row without showing
the thought body, but each visible transcript tick still projects the complete
transcript. MessageList also limits its existing tail-only cache to assistant
messages, so thinking streams repeat compact-message merging and display-item
derivation. Both costs grow with retained history and compete with the
paint-bound thinking shimmer on the browser main thread.

## Design

First, extend MessageList's committed tail cache to thinking messages. In
non-compact mode the new thinking message replaces the previous tail directly.
In compact mode the streaming thought is represented by the final synthetic
tool summary, so update only that summary and its final thought while preserving
all earlier message, tool, and display-item identities. Any dependency or
structural change uses the complete derivation path.

Second, expose an optional transcript block change summary from the SDK store.
The summary identifies its source store and advances a barrier for every change
except a validated append to the active top-level assistant or thought block.
Web Shell carries the summary with the throttled block snapshot. Equal barriers
from the same source prove that skipped revisions were pure tail appends, so the
message hook can append only the new text to its committed tail message instead
of invoking the complete projector.

Reconciliation-derived keys and resolved background-agent history use the same
barrier, connection session, and resolution snapshot identities. They are
reused only for a proven tail append; tool, permission, notification, history,
reset, session, metadata, and terminal changes rebuild through the existing
paths.

Third, top-level assistant and thought deltas share reducer side indexes that
they cannot mutate, including historical tool, permission, parent, and progress
indexes. The normal cloning path remains in place for nested deltas, mixed
batches, and any update that can cross the effective transcript block limit.

Finally, a virtualized MessageList drives bottom-follow and overflow reporting
from its measured total height and item count instead of message identity.
Content-only updates with unchanged geometry therefore perform no scroll
layout reads or writes. Non-virtual transcripts keep the existing per-message
follow behavior because their row height is not tracked by the virtualizer.

## Compatibility

The store method is optional. Older store implementations retain the existing
reference-scan fallback. There are no daemon protocol, persistence, route, or
animation changes.

## Verification

- Compare incremental compact and non-compact thinking results with complete
derivation.
- Prove pure assistant and thought appends preserve the store barrier while
mixed, structural, terminal, reset, and bounded-text changes advance it.
- Prove pure tail ticks skip the complete projector and reconciliation scans.
- Prove top-level text deltas reuse populated side indexes without sharing
across nested or transcript-trimming paths.
- Prove content-only virtual transcript updates perform no scroll geometry
reads or writes while the streamed tail still updates.
- Extend the deterministic browser performance scenario to stream thought
events and record animation-frame gaps.
Original file line number Diff line number Diff line change
Expand Up @@ -173,9 +173,9 @@ describe('slash completion during commandContext churn (#9494)', () => {
});

it('argument completion still receives the latest commandContext', async () => {
const completionSpy = vi.fn<
(context: CommandContext, argString: string) => Promise<string[]>
>().mockResolvedValue(['arg-a', 'arg-b']);
const completionSpy = vi
.fn<(context: CommandContext, argString: string) => Promise<string[]>>()
.mockResolvedValue(['arg-a', 'arg-b']);
const markerOf = (callIndex: number): string =>
(
completionSpy.mock.calls[callIndex]![0] as unknown as {
Expand Down
4 changes: 3 additions & 1 deletion packages/sdk-typescript/scripts/build.js
Original file line number Diff line number Diff line change
Expand Up @@ -113,7 +113,9 @@ const rootDir = join(__dirname, '..');
// its own delta in isolation, and the combined feature sets land here. The
// attachment read/remove + binary hydration feature that separately bumped
// main to 199KB merges within this headroom, so no further bump is needed.
const MAX_DAEMON_BROWSER_BUNDLE_BYTES = 206 * 1024;
// Bumped from 206KB to 208KB for transcript block change summaries used to
// avoid complete Web Shell projection on every streamed text update.
const MAX_DAEMON_BROWSER_BUNDLE_BYTES = 208 * 1024;
// The opt-in `daemon/transports` browser bundle legitimately ships the concrete
// ACP transports (AcpHttpTransport/AcpWsTransport/AutoReconnect + negotiate), so
// it's larger than the default barrel — but still budgeted so a future PR can't
Expand Down
1 change: 1 addition & 0 deletions packages/sdk-typescript/src/daemon/index.ts
Original file line number Diff line number Diff line change
Expand Up @@ -155,6 +155,7 @@ export type {
DaemonToolPreview,
DaemonToolTranscriptBlock,
DaemonTranscriptBlock,
DaemonTranscriptBlockChangeSummary,
DaemonTranscriptBlockKind,
DaemonTranscriptQuestion,
DaemonTranscriptQuestionOption,
Expand Down
1 change: 1 addition & 0 deletions packages/sdk-typescript/src/daemon/ui/index.ts
Original file line number Diff line number Diff line change
Expand Up @@ -84,6 +84,7 @@ export type {
DaemonToolPreview,
DaemonToolTranscriptBlock,
DaemonTranscriptBlock,
DaemonTranscriptBlockChangeSummary,
DaemonTranscriptBlockKind,
DaemonTranscriptQuestion,
DaemonTranscriptQuestionOption,
Expand Down
136 changes: 134 additions & 2 deletions packages/sdk-typescript/src/daemon/ui/store.ts
Original file line number Diff line number Diff line change
Expand Up @@ -5,9 +5,11 @@
*/

import type {
DaemonTextDeltaMeta,
DaemonTranscriptBlock,
DaemonTranscriptBlockChangeSummary,
DaemonTranscriptReducerOptions,
DaemonTranscriptState,
DaemonTextDeltaMeta,
DaemonTranscriptStore,
DaemonUiEvent,
} from './types.js';
Expand All @@ -30,6 +32,12 @@ export function createDaemonTranscriptStore(
? { onTruncation }
: {};
let state = createState(stateSeed);
const blockChangeSource = {};
let blockChangeSummary: DaemonTranscriptBlockChangeSummary = {
source: blockChangeSource,
revision: 0,
tailAppendBarrierRevision: 0,
};
const listeners = new Set<() => void>();
let notifyScheduled = false;

Expand All @@ -55,6 +63,9 @@ export function createDaemonTranscriptStore(
getSnapshot() {
return state;
},
getBlockChangeSummary() {
return blockChangeSummary;
},
subscribe(listener: () => void) {
listeners.add(listener);
return () => {
Expand All @@ -64,7 +75,14 @@ export function createDaemonTranscriptStore(
dispatch(event: DaemonUiEvent | DaemonUiEvent[]) {
const events = Array.isArray(event) ? event : [event];
if (events.length === 0) return;
state = reduceDaemonTranscriptEvents(state, events, reducerOptions);
const previous = state;
state = reduceDaemonTranscriptEvents(previous, events, reducerOptions);
blockChangeSummary = nextBlockChangeSummary(
blockChangeSummary,
previous,
state,
events,
);
scheduleNotify();
},
appendLocalUserMessage(
Expand All @@ -85,6 +103,7 @@ export function createDaemonTranscriptStore(
files,
...reducerOptions,
});
blockChangeSummary = invalidateTailAppend(blockChangeSummary);
scheduleNotify();
},
reset(nextSeed: Partial<DaemonTranscriptState> = {}) {
Expand All @@ -95,6 +114,7 @@ export function createDaemonTranscriptStore(
nextSeed.retainSubagentBlocks ?? state.retainSubagentBlocks,
...nextSeed,
});
blockChangeSummary = invalidateTailAppend(blockChangeSummary);
scheduleNotify();
},
// wenshao R4-R6 (qwen3.7-max): explicit recovery from the
Expand Down Expand Up @@ -135,6 +155,118 @@ export function createDaemonTranscriptStore(
};
}

function nextBlockChangeSummary(
current: DaemonTranscriptBlockChangeSummary,
previous: DaemonTranscriptState,
next: DaemonTranscriptState,
events: readonly DaemonUiEvent[],
): DaemonTranscriptBlockChangeSummary {
if (previous.blocks === next.blocks) return current;
const revision = current.revision + 1;
const tailBlockId = streamingTailAppendBlockId(previous, next, events);
return tailBlockId
? {
source: current.source,
revision,
tailAppendBarrierRevision: current.tailAppendBarrierRevision,
tailBlockId,
}
: {
source: current.source,
revision,
tailAppendBarrierRevision: revision,
};
}

function invalidateTailAppend(
current: DaemonTranscriptBlockChangeSummary,
): DaemonTranscriptBlockChangeSummary {
const revision = current.revision + 1;
return {
source: current.source,
revision,
tailAppendBarrierRevision: revision,
};
}

function streamingTailAppendBlockId(
previous: DaemonTranscriptState,
next: DaemonTranscriptState,
events: readonly DaemonUiEvent[],
): string | undefined {
const first = events[0];
if (
!first ||
(first.type !== 'assistant.text.delta' &&
first.type !== 'thought.text.delta') ||
first.parentToolCallId !== undefined ||
events.some(
(event) =>
event.type !== first.type ||
('parentToolCallId' in event && event.parentToolCallId !== undefined),
) ||
previous.blocks.length === 0 ||
previous.blocks.length !== next.blocks.length ||
previous.blockIndexById !== next.blockIndexById
) {
return undefined;
}

const blockId =
first.type === 'assistant.text.delta'
? previous.activeAssistantBlockId
: previous.activeThoughtBlockId;
const nextBlockId =
first.type === 'assistant.text.delta'
? next.activeAssistantBlockId
: next.activeThoughtBlockId;
const before = previous.blocks[previous.blocks.length - 1];
const after = next.blocks[next.blocks.length - 1];
const appendedTextLength = events.reduce(
(length, event) =>
length +
(event.type === 'assistant.text.delta' ||
event.type === 'thought.text.delta'
? event.text.length
: 0),
0,
);
if (
!blockId ||
nextBlockId !== blockId ||
before?.id !== blockId ||
after?.id !== blockId ||
before.kind !== after.kind ||
!isTextBlock(before) ||
!isTextBlock(after) ||
after.streaming !== true ||
before.parentToolCallId !== after.parentToolCallId ||
before.meta !== after.meta ||
before.usage !== after.usage ||
before.branchRecordId !== after.branchRecordId ||
before.clientReceivedAt !== after.clientReceivedAt ||
before.promptId !== after.promptId ||
before.sourceRecordIds !== after.sourceRecordIds ||
after.text.length !== before.text.length + appendedTextLength
) {
return undefined;
}
return blockId;
}

function isTextBlock(
block: DaemonTranscriptBlock,
): block is Extract<
DaemonTranscriptBlock,
{ kind: 'assistant' | 'thought' | 'user' }
> {
return (
block.kind === 'assistant' ||
block.kind === 'thought' ||
block.kind === 'user'
);
}

function reportListenerError(error: unknown): void {
const reporter = (
globalThis as typeof globalThis & {
Expand Down
Loading
Loading