Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
12 changes: 6 additions & 6 deletions AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@ MCP Apps SDK (`@modelcontextprotocol/ext-apps`) enables MCP servers to display i

Key abstractions:

- **Guest** - UI running in an iframe, uses `App` class with `PostMessageTransport` to communicate with host
- **View** - UI running in an iframe, uses `App` class with `PostMessageTransport` to communicate with host
- **Host** - Chat client embedding the iframe, uses `AppBridge` class to proxy MCP requests
- **Server** - MCP server that registers tools/resources with UI metadata

Expand Down Expand Up @@ -67,14 +67,14 @@ rm -fR package-lock.json node_modules && \
### Protocol Flow

```
Guest UI (App) <--PostMessageTransport--> Host (AppBridge) <--MCP Client--> MCP Server
View (App) <--PostMessageTransport--> Host (AppBridge) <--MCP Client--> MCP Server
```

1. Host creates iframe with Guest UI HTML
2. Guest UI creates `App` instance and calls `connect()` with `PostMessageTransport`
3. App sends `ui/initialize` request, receives host capabilities and context
1. Host creates iframe with view HTML
2. View creates `App` instance and calls `connect()` with `PostMessageTransport`
3. View sends `ui/initialize` request, receives host capabilities and context
4. Host sends `sendToolInput()` with tool arguments after initialization
5. Guest UI can call server tools via `app.callServerTool()` or send messages via `app.sendMessage()`
5. View can call server tools via `app.callServerTool()` or send messages via `app.sendMessage()`
6. Host sends `sendToolResult()` when tool execution completes
7. Host calls `teardownResource()` before unmounting iframe

Expand Down
16 changes: 8 additions & 8 deletions docs/migrate_from_openai_apps.md
Original file line number Diff line number Diff line change
Expand Up @@ -72,7 +72,7 @@ function createServer() {
inputSchema: { userId: z.string() },
annotations: { readOnlyHint: true },
_meta: {
"openai/outputTemplate": "ui://widget/cart.html",
"openai/outputTemplate": "ui://view/cart.html",
"openai/toolInvocation/invoking": "Loading cart...",
"openai/toolInvocation/invoked": "Cart ready",
"openai/widgetAccessible": true,
Expand All @@ -89,13 +89,13 @@ function createServer() {

// Register UI resource
server.registerResource(
"Cart Widget",
"ui://widget/cart.html",
"Cart View",
"ui://view/cart.html",
{ mimeType: "text/html+skybridge" },
async () => ({
contents: [
{
uri: "ui://widget/cart.html",
uri: "ui://view/cart.html",
mimeType: "text/html+skybridge",
text: getCartHtml(),
_meta: {
Expand Down Expand Up @@ -137,7 +137,7 @@ function createServer() {
description: "Display the user's shopping cart",
inputSchema: { userId: z.string() },
annotations: { readOnlyHint: true },
_meta: { ui: { resourceUri: "ui://widget/cart.html" } },
_meta: { ui: { resourceUri: "ui://view/cart.html" } },
},
async (args) => {
const cart = await getCart(args.userId);
Expand All @@ -151,13 +151,13 @@ function createServer() {
// Register UI resource
registerAppResource(
server,
"Cart Widget",
"ui://widget/cart.html",
"Cart View",
"ui://view/cart.html",
{ description: "Shopping cart UI" },
async () => ({
contents: [
{
uri: "ui://widget/cart.html",
uri: "ui://view/cart.html",
mimeType: RESOURCE_MIME_TYPE,
text: getCartHtml(),
_meta: {
Expand Down
10 changes: 5 additions & 5 deletions docs/patterns.md
Original file line number Diff line number Diff line change
Expand Up @@ -82,11 +82,11 @@ When you need to send more data than fits in a message, use {@link app!App.updat

_See [`examples/transcript-server/`](https://github.com/modelcontextprotocol/ext-apps/tree/main/examples/transcript-server) for a full implementation of this pattern._

## Persisting widget state
## Persisting view state

To persist widget state across conversation reloads (e.g., current page in a PDF viewer, camera position in a map), use [`localStorage`](https://developer.mozilla.org/en-US/docs/Web/API/Window/localStorage) with a stable identifier provided by the server.
To persist view state across conversation reloads (e.g., current page in a PDF viewer, camera position in a map), use [`localStorage`](https://developer.mozilla.org/en-US/docs/Web/API/Window/localStorage) with a stable identifier provided by the server.

**Server-side**: Tool handler generates a unique `widgetUUID` and returns it in `CallToolResult._meta.widgetUUID`:
**Server-side**: Tool handler generates a unique `viewUUID` and returns it in `CallToolResult._meta.viewUUID`:

{@includeCode ./patterns.tsx#persistDataServer}

Expand All @@ -96,9 +96,9 @@ To persist widget state across conversation reloads (e.g., current page in a PDF

_See [`examples/map-server/`](https://github.com/modelcontextprotocol/ext-apps/tree/main/examples/map-server) for a full implementation of this pattern._

## Pausing computation-heavy widgets when out of view
## Pausing computation-heavy views when out of view

Widgets with animations, WebGL rendering, or polling can consume significant CPU/GPU even when scrolled out of view. Use [`IntersectionObserver`](https://developer.mozilla.org/en-US/docs/Web/API/Intersection_Observer_API) to pause expensive operations when the widget isn't visible:
Views with animations, WebGL rendering, or polling can consume significant CPU/GPU even when scrolled out of view. Use [`IntersectionObserver`](https://developer.mozilla.org/en-US/docs/Web/API/Intersection_Observer_API) to pause expensive operations when the view isn't visible:

{@includeCode ./patterns.tsx#visibilityBasedPause}

Expand Down
50 changes: 23 additions & 27 deletions docs/patterns.tsx
Original file line number Diff line number Diff line change
Expand Up @@ -206,61 +206,57 @@ function hostStylingReact() {
}

/**
* Example: Persisting widget state (server-side)
* Example: Persisting view state (server-side)
*/
function persistWidgetStateServer(
url: string,
title: string,
pageCount: number,
) {
function persistViewStateServer(url: string, title: string, pageCount: number) {
function toolCallback(): CallToolResult {
//#region persistDataServer
// In your tool callback, include widgetUUID in the result metadata.
// In your tool callback, include viewUUID in the result metadata.
return {
content: [{ type: "text", text: `Displaying PDF viewer for "${title}"` }],
structuredContent: { url, title, pageCount, initialPage: 1 },
_meta: {
widgetUUID: randomUUID(),
viewUUID: randomUUID(),
},
};
//#endregion persistDataServer
}
}

/**
* Example: Persisting widget state (client-side)
* Example: Persisting view state (client-side)
*/
function persistWidgetState(app: App) {
function persistViewState(app: App) {
//#region persistData
// Store the widgetUUID received from the server
let widgetUUID: string | undefined;
// Store the viewUUID received from the server
let viewUUID: string | undefined;

// Helper to save state to localStorage
function saveState<T>(state: T): void {
if (!widgetUUID) return;
if (!viewUUID) return;
try {
localStorage.setItem(widgetUUID, JSON.stringify(state));
localStorage.setItem(viewUUID, JSON.stringify(state));
} catch (err) {
console.error("Failed to save widget state:", err);
console.error("Failed to save view state:", err);
}
}

// Helper to load state from localStorage
function loadState<T>(): T | null {
if (!widgetUUID) return null;
if (!viewUUID) return null;
try {
const saved = localStorage.getItem(widgetUUID);
const saved = localStorage.getItem(viewUUID);
return saved ? (JSON.parse(saved) as T) : null;
} catch (err) {
console.error("Failed to load widget state:", err);
console.error("Failed to load view state:", err);
return null;
}
}

// Receive widgetUUID from the tool result
// Receive viewUUID from the tool result
app.ontoolresult = (result) => {
widgetUUID = result._meta?.widgetUUID
? String(result._meta.widgetUUID)
viewUUID = result._meta?.viewUUID
? String(result._meta.viewUUID)
: undefined;

// Restore any previously saved state
Expand All @@ -270,21 +266,21 @@ function persistWidgetState(app: App) {
}
};

// Call saveState() whenever your widget state changes
// Call saveState() whenever your view state changes
// e.g., saveState({ currentPage: 5 });
//#endregion persistData
}

/**
* Example: Pausing computation-heavy widgets when out of view
* Example: Pausing computation-heavy views when out of view
*/
function visibilityBasedPause(
app: App,
container: HTMLElement,
animation: { play: () => void; pause: () => void },
) {
//#region visibilityBasedPause
// Use IntersectionObserver to pause when widget scrolls out of view
// Use IntersectionObserver to pause when view scrolls out of view
const observer = new IntersectionObserver((entries) => {
entries.forEach((entry) => {
if (entry.isIntersecting) {
Expand All @@ -296,7 +292,7 @@ function visibilityBasedPause(
});
observer.observe(container);

// Clean up when the host tears down the widget
// Clean up when the host tears down the view
app.onteardown = async () => {
observer.disconnect();
animation.pause();
Expand All @@ -310,6 +306,6 @@ void chunkedDataServer;
void chunkedDataClient;
void hostStylingVanillaJs;
void hostStylingReact;
void persistWidgetStateServer;
void persistWidgetState;
void persistViewStateServer;
void persistViewState;
void visibilityBasedPause;
2 changes: 1 addition & 1 deletion examples/basic-host/src/implementation.ts
Original file line number Diff line number Diff line change
Expand Up @@ -277,7 +277,7 @@ export function newAppBridge(
},
});

// Register all handlers before calling connect(). The Guest UI can start
// Register all handlers before calling connect(). The view can start
// sending requests immediately after the initialization handshake, so any
// handlers registered after connect() might miss early requests.

Expand Down
6 changes: 3 additions & 3 deletions examples/basic-host/src/sandbox.ts
Original file line number Diff line number Diff line change
Expand Up @@ -56,15 +56,15 @@ const PROXY_READY_NOTIFICATION: McpUiSandboxProxyReadyNotification["method"] =
// Message relay: This Sandbox (outer iframe) acts as a bidirectional bridge,
// forwarding messages between:
//
// Host (parent window) ↔ Sandbox (outer frame) ↔ Guest UI (inner iframe)
// Host (parent window) ↔ Sandbox (outer frame) ↔ View (inner iframe)
//
// Reason: the parent window and inner iframe have different origins and can't
// communicate directly, so the outer iframe forwards messages in both
// directions to connect them.
//
// Special case: The "ui/notifications/sandbox-proxy-ready" message is
// intercepted here (not relayed) because the Sandbox uses it to configure and
// load the inner iframe with the Guest UI HTML content.
// load the inner iframe with the view HTML content.
//
// Security: CSP is enforced via HTTP headers on sandbox.html (set by serve.ts
// based on ?csp= query param). This is tamper-proof unlike meta tags.
Expand Down Expand Up @@ -128,7 +128,7 @@ window.addEventListener("message", async (event) => {
}
});

// Notify the Host that the Sandbox is ready to receive Guest UI HTML.
// Notify the Host that the Sandbox is ready to receive view HTML.
// Use specific origin instead of "*" to ensure only the expected host receives this.
window.parent.postMessage({
jsonrpc: "2.0",
Expand Down
2 changes: 1 addition & 1 deletion examples/map-server/server.ts
Original file line number Diff line number Diff line change
Expand Up @@ -185,7 +185,7 @@ export function createServer(): McpServer {
},
],
_meta: {
widgetUUID: randomUUID(),
viewUUID: randomUUID(),
},
}),
);
Expand Down
24 changes: 11 additions & 13 deletions examples/map-server/src/mcp-app.ts
Original file line number Diff line number Diff line change
Expand Up @@ -71,7 +71,7 @@ let persistViewTimer: ReturnType<typeof setTimeout> | null = null;
// Track whether tool input has been received (to know if we should restore persisted state)
let hasReceivedToolInput = false;

let widgetUUID: string | undefined = undefined;
let viewUUID: string | undefined = undefined;

/**
* Persisted camera state for localStorage
Expand Down Expand Up @@ -122,7 +122,7 @@ function schedulePersistViewState(cesiumViewer: any): void {
* Persist current view state to localStorage
*/
function persistViewState(cesiumViewer: any): void {
if (!widgetUUID) {
if (!viewUUID) {
log.info("No storage key available, skipping view persistence");
return;
}
Expand All @@ -132,8 +132,8 @@ function persistViewState(cesiumViewer: any): void {

try {
const value = JSON.stringify(state);
localStorage.setItem(widgetUUID, value);
log.info("Persisted view state:", widgetUUID, value);
localStorage.setItem(viewUUID, value);
log.info("Persisted view state:", viewUUID, value);
} catch (e) {
log.warn("Failed to persist view state:", e);
}
Expand All @@ -143,10 +143,10 @@ function persistViewState(cesiumViewer: any): void {
* Load persisted view state from localStorage
*/
function loadPersistedViewState(): PersistedCameraState | null {
if (!widgetUUID) return null;
if (!viewUUID) return null;

try {
const stored = localStorage.getItem(widgetUUID);
const stored = localStorage.getItem(viewUUID);
if (!stored) {
console.info("No persisted view state found");
return null;
Expand Down Expand Up @@ -938,16 +938,14 @@ app.ontoolinput = async (params) => {
// },
// );

// Handle tool result - extract widgetUUID and restore persisted view if available
// Handle tool result - extract viewUUID and restore persisted view if available
app.ontoolresult = async (result) => {
widgetUUID = result._meta?.widgetUUID
? String(result._meta.widgetUUID)
: undefined;
log.info("Tool result received, widgetUUID:", widgetUUID);
viewUUID = result._meta?.viewUUID ? String(result._meta.viewUUID) : undefined;
log.info("Tool result received, viewUUID:", viewUUID);

// Now that we have widgetUUID, try to restore persisted view
// Now that we have viewUUID, try to restore persisted view
// This overrides the tool input position if a saved state exists
if (viewer && widgetUUID) {
if (viewer && viewUUID) {
const restored = restorePersistedView(viewer);
if (restored) {
log.info("Restored persisted view from tool result handler");
Expand Down
2 changes: 1 addition & 1 deletion examples/pdf-server/server.ts
Original file line number Diff line number Diff line change
Expand Up @@ -189,7 +189,7 @@ The viewer supports zoom, navigation, text selection, and fullscreen mode.`,
],
structuredContent: result,
_meta: {
widgetUUID: randomUUID(),
viewUUID: randomUUID(),
},
};
},
Expand Down
18 changes: 8 additions & 10 deletions examples/pdf-server/src/mcp-app.ts
Original file line number Diff line number Diff line change
Expand Up @@ -35,7 +35,7 @@ let totalPages = 0;
let scale = 1.0;
let pdfUrl = "";
let pdfTitle: string | undefined;
let widgetUUID: string | undefined;
let viewUUID: string | undefined;
let currentRenderTask: { cancel: () => void } | null = null;

// DOM Elements
Expand Down Expand Up @@ -404,10 +404,10 @@ async function renderPage() {
}

function saveCurrentPage() {
log.info("saveCurrentPage: key=", widgetUUID, "page=", currentPage);
if (widgetUUID) {
log.info("saveCurrentPage: key=", viewUUID, "page=", currentPage);
if (viewUUID) {
try {
localStorage.setItem(widgetUUID, String(currentPage));
localStorage.setItem(viewUUID, String(currentPage));
log.info("saveCurrentPage: saved successfully");
} catch (err) {
log.error("saveCurrentPage: error", err);
Expand All @@ -416,10 +416,10 @@ function saveCurrentPage() {
}

function loadSavedPage(): number | null {
log.info("loadSavedPage: key=", widgetUUID);
if (!widgetUUID) return null;
log.info("loadSavedPage: key=", viewUUID);
if (!viewUUID) return null;
try {
const saved = localStorage.getItem(widgetUUID);
const saved = localStorage.getItem(viewUUID);
log.info("loadSavedPage: saved value=", saved);
if (saved) {
const page = parseInt(saved, 10);
Expand Down Expand Up @@ -706,9 +706,7 @@ app.ontoolresult = async (result) => {
pdfUrl = parsed.url;
pdfTitle = parsed.title;
totalPages = parsed.pageCount;
widgetUUID = result._meta?.widgetUUID
? String(result._meta.widgetUUID)
: undefined;
viewUUID = result._meta?.viewUUID ? String(result._meta.viewUUID) : undefined;

// Restore saved page or use initial page
const savedPage = loadSavedPage();
Expand Down
Loading
Loading