Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
19 changes: 19 additions & 0 deletions docs/design/web-shell-mid-turn-file-references.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,19 @@
# Web Shell mid-turn file attachments

## Problem

Web Shell turns an `@` file selection into prompt text plus a file input annotation. Annotated prompts currently wait for the next turn, while images can be uploaded and inserted into a running turn. File insertion must use the same durable attachment and rendering path as an ordinary prompt with an attached file.

## Design

When the daemon advertises `session_attachments`, Web Shell uploads both composer file attachments and files resolved from annotations to the current session attachment store. Annotated files are read through the selected trusted workspace with the existing bounded workspace-file reader. The returned attachment references travel with the existing mid-turn `content` payload alongside image references. Prompts containing non-file annotations, unavailable workspace ownership, unreadable files, or oversized files continue through the ordinary pending queue or restore to the editor before daemon admission.

The inserted display text omits annotated `@` tokens because the referenced files are rendered as attachment rows. Pending file attachments appear beside image previews and open in the existing attachment preview panel. Reconciliation and injection echoes recover file rows from the same `resource` attachment references used by an ordinary prompt with files.

Deleting a queued mid-turn message removes its referenced file attachments after the daemon confirms the message was removed. Failed removals leave the attachments intact because the queued or running message may still need them.

No new daemon protocol or attachment type is introduced.

## Compatibility

Older daemons without `session_attachments` keep annotated prompts on the ordinary queue. Existing image-only mid-turn messages and text insertion are unchanged.
26 changes: 26 additions & 0 deletions packages/web-shell/client/App.module.css
Original file line number Diff line number Diff line change
Expand Up @@ -1086,6 +1086,32 @@
opacity: 0.85;
}

.queuedPromptFile {
display: inline-flex;
min-width: 0;
max-width: 120px;
height: 22px;
align-items: center;
gap: 4px;
padding: 0 6px;
border: 1px solid var(--border);
border-radius: 4px;
background: var(--muted);
color: var(--foreground);
cursor: pointer;
}

.queuedPromptFile:disabled {
cursor: default;
opacity: 0.7;
}

.queuedPromptFile span {
overflow: hidden;
text-overflow: ellipsis;
white-space: nowrap;
}

.queuedPromptState {
display: inline-flex;
align-items: center;
Expand Down
2 changes: 2 additions & 0 deletions packages/web-shell/client/App.tsx
Original file line number Diff line number Diff line change
Expand Up @@ -6289,6 +6289,7 @@ export function App({
canMutateMidTurn,
canQueryMidTurn,
canInjectMidTurnMedia,
workspaceFileActions: artifactWorkspaceActions,
streamingState,
sessionActions,
store,
Expand Down Expand Up @@ -12632,6 +12633,7 @@ export function App({
onDelete={removeQueuedPrompt}
onEdit={editQueuedPrompt}
onImagePreview={openImagePanel}
onAttachmentPreview={openAttachmentPanel}
/>
{CustomComposerHeader && (
<div className={styles.composerHeader}>
Expand Down
5 changes: 5 additions & 0 deletions packages/web-shell/client/adapters/messageTypes.ts
Original file line number Diff line number Diff line change
Expand Up @@ -158,6 +158,11 @@ export interface DaemonSystemMessage extends DaemonMessageMeta {
source?: string;
data?: unknown;
images?: Array<{ data: string; mimeType: string }>;
files?: Array<{
name: string;
mimeType: string;
attachmentId?: string;
}>;
}

export interface DaemonUserShellMessage extends DaemonMessageMeta {
Expand Down
1 change: 1 addition & 0 deletions packages/web-shell/client/adapters/promptTypes.ts
Original file line number Diff line number Diff line change
Expand Up @@ -9,4 +9,5 @@ export interface PromptFile {
data?: Blob;
text?: string;
size?: number;
attachmentId?: string;
}
39 changes: 39 additions & 0 deletions packages/web-shell/client/adapters/transcriptToMessages.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -640,6 +640,45 @@ describe('transcriptBlocksToDaemonMessages', () => {
]);
});

it('extracts file attachments from mid-turn injected message items', () => {
const messages = transcriptBlocksToDaemonMessages([
statusBlock('mid-1', 'explain this', 1, {
source: 'mid_turn_message_injected',
data: {
sessionId: 's1',
messages: ['explain this'],
items: [
{
content: [
{
type: 'resource',
attachmentId: 'notes.txt',
mimeType: 'text/plain',
size: 5,
},
],
},
],
},
}),
]);

expect(messages).toEqual([
expect.objectContaining({
role: 'system',
content: 'explain this',
source: 'mid_turn_message_injected',
files: [
{
name: 'notes.txt',
attachmentId: 'notes.txt',
mimeType: 'text/plain',
},
],
}),
]);
});

it('shows the degraded-media notice when the echo text is empty', () => {
// When the stored media is gone at drain, the daemon echoes an empty
// messages array whose items carry only the placeholder text block; the
Expand Down
53 changes: 44 additions & 9 deletions packages/web-shell/client/adapters/transcriptToMessages.ts
Original file line number Diff line number Diff line change
Expand Up @@ -281,6 +281,37 @@ function getMidTurnInjectedImages(
return images.length > 0 ? images : undefined;
}

function getMidTurnInjectedFiles(
data: unknown,
): Array<{ name: string; mimeType: string; attachmentId: string }> | undefined {
if (!data || typeof data !== 'object') return undefined;
const items = (data as { items?: unknown }).items;
if (!Array.isArray(items)) return undefined;
const files = items.flatMap((item) => {
if (!item || typeof item !== 'object') return [];
const content = (item as { content?: unknown }).content;
if (!Array.isArray(content)) return [];
return content.flatMap((block) => {
if (!block || typeof block !== 'object') return [];
const record = block as Record<string, unknown>;
return record['type'] === 'resource' &&
typeof record['attachmentId'] === 'string'
? [
{
name: record['attachmentId'],
attachmentId: record['attachmentId'],
mimeType:
typeof record['mimeType'] === 'string'
? record['mimeType']
: 'application/octet-stream',
},
]
: [];
});
});
return files.length > 0 ? files : undefined;
}

/**
* Collect text content blocks from mid-turn injected message items. The
* degraded-media drain echo ships an empty `messages` array whose items carry
Expand Down Expand Up @@ -437,6 +468,13 @@ export function transcriptBlocksToDaemonMessages(
data: img.data,
mimeType: img.mimeType || 'image/*',
}));
const files = textBlock.files?.map((file) => ({
name: file.name,
mimeType: file.mimeType || 'text/plain',
...(file.data !== undefined ? { data: file.data } : {}),
...(file.text !== undefined ? { text: file.text } : {}),
...(file.attachmentId ? { attachmentId: file.attachmentId } : {}),
}));
if (source === 'mid_turn_message_injected') {
messages.push({
id: block.id,
Expand All @@ -446,6 +484,7 @@ export function transcriptBlocksToDaemonMessages(
source,
timestamp: blockTime,
...(images && images.length > 0 ? { images } : {}),
...(files && files.length > 0 ? { files } : {}),
});
needsNewContentMessage = true;
break;
Expand All @@ -462,15 +501,7 @@ export function transcriptBlocksToDaemonMessages(
if (images && images.length > 0) {
msg.images = images;
}
if (textBlock.files && textBlock.files.length > 0) {
msg.files = textBlock.files.map((file) => ({
name: file.name,
mimeType: file.mimeType || 'text/plain',
...(file.data !== undefined ? { data: file.data } : {}),
...(file.text !== undefined ? { text: file.text } : {}),
...(file.attachmentId ? { attachmentId: file.attachmentId } : {}),
}));
}
if (files && files.length > 0) msg.files = files;
messages.push(msg);
break;
}
Expand Down Expand Up @@ -838,6 +869,9 @@ export function transcriptBlocksToDaemonMessages(
const midTurnInjectedImages = getMidTurnInjectedImages(
statusBlock.data,
);
const midTurnInjectedFiles = getMidTurnInjectedFiles(
statusBlock.data,
);
messages.push({
id: block.id,
role: 'system',
Expand All @@ -853,6 +887,7 @@ export function transcriptBlocksToDaemonMessages(
? { data: statusBlock.data }
: {}),
...(midTurnInjectedImages ? { images: midTurnInjectedImages } : {}),
...(midTurnInjectedFiles ? { files: midTurnInjectedFiles } : {}),
});
needsNewContentMessage = true;
break;
Expand Down
6 changes: 6 additions & 0 deletions packages/web-shell/client/components/ChatPane.tsx
Original file line number Diff line number Diff line change
Expand Up @@ -90,6 +90,7 @@ import type {
TurnOutputOpenRequest,
} from './artifacts/TurnOutputs';
import { TURN_OUTPUT_KINDS } from './artifacts/TurnOutputs';
import { useArtifactWorkspaceTarget } from './artifacts/useArtifactWorkspaceTarget';
import {
getArtifactsByTurn,
getFileChangesByTurn,
Expand Down Expand Up @@ -241,6 +242,9 @@ export function ChatPane({
const connection = useConnection();
const actions = useActions();
const workspace = useWorkspace();
const attachmentWorkspaceTarget = useArtifactWorkspaceTarget(
connection.workspaceCwd,
);
const sessionCatalogController = useSessionCatalogController(
workspace.client,
);
Expand Down Expand Up @@ -545,6 +549,7 @@ export function ChatPane({
canMutateMidTurn,
canQueryMidTurn,
canInjectMidTurnMedia,
workspaceFileActions: attachmentWorkspaceTarget?.actions,
streamingState,
sessionActions: actions,
store,
Expand Down Expand Up @@ -1118,6 +1123,7 @@ export function ChatPane({
onDelete={removeQueuedPrompt}
onEdit={editQueuedPrompt}
onImagePreview={handleImagePreview}
onAttachmentPreview={handleAttachmentPreview}
/>
{unknownPromptAdmission && (
<div
Expand Down
2 changes: 2 additions & 0 deletions packages/web-shell/client/components/MessageItem.tsx
Original file line number Diff line number Diff line change
Expand Up @@ -147,8 +147,10 @@ export const MessageItem = memo(function MessageItem({
source={message.source}
data={message.data}
images={message.images}
files={message.files}
onShowContextDetail={onShowContextDetail}
onImagePreview={onImagePreview}
onAttachmentPreview={onAttachmentPreview}
isLatest={isLatest}
showRetryHint={showRetryHint && message.retryable === true}
onRetryClick={onRetryClick}
Expand Down
34 changes: 34 additions & 0 deletions packages/web-shell/client/components/QueuedPromptDisplay.test.tsx
Original file line number Diff line number Diff line change
Expand Up @@ -97,6 +97,40 @@ describe('QueuedPromptDisplay', () => {
expect(position & Node.DOCUMENT_POSITION_FOLLOWING).toBeTruthy();
});

it('renders attached files after the text and opens their preview', () => {
const onAttachmentPreview = vi.fn();
const file = {
name: 'notes.txt',
media_type: 'text/plain',
attachmentId: 'attachment-1',
};
const { container } = setup({
prompts: [
{
id: 1,
text: '带附件消息',
files: [file],
midTurnState: 'queued',
midTurnMessageId: 'mid-1',
},
],
onAttachmentPreview,
});

const text = container.querySelector('[class*="queuedPromptText"]');
const fileButton = container.querySelector<HTMLButtonElement>(
'[class*="queuedPromptFile"]',
);
expect(text?.nextElementSibling).toContain(fileButton);
expect(fileButton?.textContent).toContain('notes.txt');
act(() => fileButton?.click());
expect(onAttachmentPreview).toHaveBeenCalledWith({
name: 'notes.txt',
mimeType: 'text/plain',
attachmentId: 'attachment-1',
});
});

it('does not render unsafe image data URIs', () => {
const { container } = setup({
prompts: [
Expand Down
Loading
Loading