Skip to content

Conversation

@aharvard
Copy link
Collaborator

@aharvard aharvard commented Aug 18, 2025

This PR enhances MCP-UI functionality in Goose.

Chore: MCP-UI Upgrade

I bumped @mcp-ui/client to version (5.9.0), this lets us experiment with the following:

  1. tailoring htmlProps.sandboxPermissions
  2. use new isUIResource(content) helper for convenience

Feat: Allow forms for iframe sandbox

We got a request to allow-forms from our MCP-UI iframe: #4117. So, this PR allows us to experiment with setting htmlProps.sandboxPermissions to allow-forms — for more info refer to docs https://mcpui.dev/guide/client/resource-renderer#props-details.

Important

I'm unsure what attack vectors this introduces but would like to ask if we can be flexible and quick to make updates to htmlProps.sandboxPermissions over time as we learn more.

Feat: Embedded UI Message Support

For more info on what MCP-UIs can message to Goose, refer to: https://mcpui.dev/guide/embeddable-ui#message-types

goose-mcp-ui-message-handling.mov
Message Handled in Goose? Goose Response
intent: the user has interacted with the UI and expressed an intent, and the host should act on it ❌ Acknowledge by goose with a meaningful response, but does not fire any host capability. It needs an implementation plan to become functional. Returns an error status with additional information about what the MCP UI requested
notify: the iframe already acted upon the user interaction, and is notifying the host to trigger any side effects ✅ Acknowledge by goose with a meaningful response and fires a toast notification. This could evolve into passing a message into the chat, but would need an implementation plan. Returns meaningful data if successful or an error message if it fails.
prompt: the iframe asks the host to run a prompt ✅ Acknowledge by goose with a meaningful response, and the message is passed directly to the chat engine. Returns meaningful data if successful or an error message if it fails.
tool: the iframe asks the host to run a tool call ❌ Acknowledge by goose with a meaningful response, but does not fire any host capability. It needs an implementation plan to become functional. Returns an error status with additional information about what the MCP UI requested.
link: the iframe asks the host to navigate to a link ✅ Acknowledge by goose with a meaningful response, and calls electron openExternal() method. Returns meaningful data if successful or an error message if it fails

Fix: UI Action callback response

In our first iteration, onUIAction={handleUIAction} did not return a meaningful response (refer to this issue raised over on mcp-ui). Now it does.

@aharvard
Copy link
Collaborator Author

.bundle

@github-actions
Copy link
Contributor

macOS ARM64 Desktop App (Apple Silicon)

📱 Download macOS Desktop App (arm64, unsigned)

Instructions:
After downloading, unzip the file and drag the Goose.app to a location you prefer. The app is unsigned, so to run it run xattr -r -d com.apple.quarantine '/path/to/Goose.app' and then open the app

@block block deleted a comment from github-actions bot Aug 18, 2025
@michaelneale
Copy link
Collaborator

@aharvard nice - I think if you update to main and fix conflicts, you may run into error with react versions that I had - 2e2369f is one way to fix it

…ror management

- Introduced specific result types for tool calls, prompts, links, notifications, and intents.
- Added error handling with strongly typed error codes for better clarity and debugging.
- Implemented separate handlers for each action type to improve type safety and maintainability.
- Updated the main action handler to support new action types and provide exhaustive type checking.
- Implemented a new IPC handler in the main process to open external URLs securely.
- Updated the preload script to expose the openExternal function for invoking the new handler from the renderer process.
…sage handling

- Added a global event listener for scroll-to-bottom requests in BaseChat to improve user experience.
- Enhanced MCPUIResourceRenderer and ToolCallWithResponse components to support an append function for message handling.
- Updated prompt action handling to utilize the append function, providing fallback options for message delivery.
… and enhancing error handling

- Removed optional callbacks for tool calls, prompts, navigation, and intents to streamline the component.
- Improved error handling for unsupported actions in tool calls, prompts, and intents.
- Ensured that the append function is utilized for prompt actions and added custom event dispatching for chat scrolling.
…message handling

- Renamed the append prop to appendPromptToChat for clarity in MCPUIResourceRenderer.
- Updated ToolCallWithResponse to pass the new appendPromptToChat prop to MCPUIResourceRenderer.
- Revised TODO comments for better clarity on future enhancements regarding message handling.
…dling

- Expanded action handling in MCPUIResourceRenderer to include specific cases for tools, prompts, links, notifications, and intents.
- Improved error handling with consistent status codes and messages for unsupported actions.
- Streamlined the main action handler for better readability and maintainability, ensuring exhaustive type checking.
- Removed unused callback functions to simplify the component structure.
…nous

- Converted action handler functions in MCPUIResourceRenderer to asynchronous to support promise-based operations.
- Ensured that the main action handler awaits results from tool, prompt, notify, and intent cases for improved error handling and response management.
Comment on lines 66 to 272
const handleUIAction = async (actionEvent: UIActionResult): Promise<UIActionHandlerResult> => {
console.log('[MCP-UI] Action received:', actionEvent);

case 'notify': {
// TODO: Implement notify handling
handleAction(result);
break;
let result: UIActionHandlerResult;

const handleToolCase = async (
actionEvent: UIActionResultToolCall
): Promise<UIActionHandlerResult> => {
const { toolName, params } = actionEvent.payload;
return {
status: 'error' as const,
error: {
code: UIActionErrorCode.UNSUPPORTED_ACTION,
message: 'Tool calls are not yet implemented',
details: { toolName, params },
},
};
};

const handlePromptCase = async (
actionEvent: UIActionResultPrompt
): Promise<UIActionHandlerResult> => {
const { prompt } = actionEvent.payload;

if (appendPromptToChat) {
try {
appendPromptToChat(prompt);
window.dispatchEvent(new CustomEvent('scroll-chat-to-bottom'));
return {
status: 'success' as const,
message: 'Prompt sent to chat successfully',
};
} catch (error) {
return {
status: 'error' as const,
error: {
code: UIActionErrorCode.PROMPT_FAILED,
message: 'Failed to send prompt to chat',
details: error instanceof Error ? error.message : error,
},
};
}
}

case 'prompt': {
// TODO: Implement prompt handling
handleAction(result);
break;
return {
status: 'error' as const,
error: {
code: UIActionErrorCode.UNSUPPORTED_ACTION,
message: 'Prompt handling is not implemented - append prop is required',
details: { prompt },
},
};
};

const handleLinkCase = async (actionEvent: UIActionResultLink) => {
const { url } = actionEvent.payload;

try {
const urlObj = new URL(url);
if (!['http:', 'https:'].includes(urlObj.protocol)) {
return {
status: 'error' as const,
error: {
code: UIActionErrorCode.NAVIGATION_FAILED,
message: `Blocked potentially unsafe URL protocol: ${urlObj.protocol}`,
details: { url, protocol: urlObj.protocol },
},
};
}

await window.electron.openExternal(url);
return {
status: 'success' as const,
message: `Opened ${url} in default browser`,
};
} catch (error) {
if (error instanceof TypeError && error.message.includes('Invalid URL')) {
return {
status: 'error' as const,
error: {
code: UIActionErrorCode.INVALID_PARAMS,
message: `Invalid URL format: ${url}`,
details: { url, error: error.message },
},
};
} else if (error instanceof Error && error.message.includes('Failed to open')) {
return {
status: 'error' as const,
error: {
code: UIActionErrorCode.NAVIGATION_FAILED,
message: `Failed to open URL in default browser`,
details: { url, error: error.message },
},
};
} else {
return {
status: 'error' as const,
error: {
code: UIActionErrorCode.NAVIGATION_FAILED,
message: `Unexpected error opening URL: ${url}`,
details: error instanceof Error ? error.message : error,
},
};
}
}
};

case 'tool': {
// TODO: Implement tool call handling
handleAction(result);
break;
const handleNotifyCase = async (
actionEvent: UIActionResultNotification
): Promise<UIActionHandlerResult> => {
const { message } = actionEvent.payload;

try {
const notificationId = `notify-${Date.now()}`;
toast.info(message);
return {
status: 'success' as const,
data: {
notificationId,
displayedAt: new Date().toISOString(),
message,
},
};
} catch (error) {
return {
status: 'error' as const,
error: {
code: UIActionErrorCode.UNKNOWN_ACTION,
message: 'Failed to display notification',
details: error instanceof Error ? error.message : error,
},
};
}
};

const handleIntentCase = async (
actionEvent: UIActionResultIntent
): Promise<UIActionHandlerResult> => {
const { intent, params } = actionEvent.payload;

return {
status: 'error' as const,
error: {
code: UIActionErrorCode.UNSUPPORTED_ACTION,
message: 'Intent handling is not yet implemented',
details: { intent, params },
},
};
};

try {
switch (actionEvent.type) {
case 'tool':
result = await handleToolCase(actionEvent);
break;

case 'prompt':
result = await handlePromptCase(actionEvent);
break;

case 'link':
result = await handleLinkCase(actionEvent);
break;

default: {
console.warn('unsupported message sent from MCP-UI:', result);
break;
case 'notify':
result = await handleNotifyCase(actionEvent);
break;

case 'intent':
result = await handleIntentCase(actionEvent);
break;

default: {
// TypeScript exhaustiveness check
const _exhaustiveCheck: never = actionEvent;
console.error('Unhandled action type:', _exhaustiveCheck);
result = {
status: 'error',
error: {
code: UIActionErrorCode.UNKNOWN_ACTION,
message: `Unknown action type`,
details: actionEvent,
},
};
}
}
} catch (error) {
console.error('[MCP-UI] Unexpected error:', error);
result = {
status: 'error',
error: {
code: UIActionErrorCode.UNKNOWN_ACTION,
message: 'An unexpected error occurred',
details: error instanceof Error ? error.stack : error,
},
};
}

// Log result with appropriate level
if (result.status === 'error') {
console.error('[MCP-UI] Action failed:', result);
} else {
console.log('[MCP-UI] Action succeeded:', result);
}
}, []);

return result;
};
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hey @idosal, thanks again for taking a look at this.
I took another pass at handling UI actions here. Now, I'm returning a lot more info. Is this more in line with your suggestion?

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @aharvard, it looks great. It captures the data that could be relevant to the UI.

@block block deleted a comment from github-actions bot Aug 20, 2025
This dependency was accidentally included through a merge conflict resolution and should not be there.
@aharvard aharvard changed the title MCP-UI Enhancements: allow-forms and fix missing message response feat: upgrade @mcp-ui/client package and start UI message implementation Aug 20, 2025
@aharvard aharvard changed the title feat: upgrade @mcp-ui/client package and start UI message implementation feat: upgrade @mcp-ui/client package and improve UI message handling Aug 20, 2025
@aharvard
Copy link
Collaborator Author

.bundle

@github-actions
Copy link
Contributor

macOS ARM64 Desktop App (Apple Silicon)

📱 Download macOS Desktop App (arm64, unsigned)

Instructions:
After downloading, unzip the file and drag the Goose.app to a location you prefer. The app is unsigned, so to run it run xattr -r -d com.apple.quarantine '/path/to/Goose.app' and then open the app

toolResponse={toolResponsesMap.get(toolRequest.id)}
notifications={toolCallNotifications.get(toolRequest.id)}
isStreamingMessage={isStreaming}
append={append}
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

what does this do?

Copy link
Collaborator Author

@aharvard aharvard Aug 21, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

append is an existing convention in this file, and I had to add it to the <ToolCallWithResponse /> component to prop drill. TBH, this feels like a hack.

Inside of <ToolCallWithResponse />, append is passed to <MCPUIResourceRenderer content={content} appendPromptToChat={append} /> — where, I think, appendPromptToChat is a bit clearer.

The goal is to

  1. Catch the prompt message from an MCP UI
  2. Pass that message to the chat engine
  3. Let goose take the wheel

@zane, do you know if there is a more elegant way to pass a prompt string to the chat engine w/out prop drilling?

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think prop drilling is ok for 2 levels like this for now but we don't have any other global state mechanism currently other than React context. So you could use context if needed. We plan on adding a global state library soon that can make things like this easier.

});

// Handle external URL opening
ipcMain.handle('open-external', async (_event, url: string) => {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

what functionality does this add that wasn't before - is it for cases where the type is links and it will open it in external browser? (if so - a hyperlink won't do that?)

Copy link
Collaborator Author

@aharvard aharvard Aug 21, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since MCP-UI hyperlinks live inside of an iframe, clicking on a typical <a href="" /> link will refresh the contents inside of the iframe. That might be perfect for some MCP-UIs but not others.

Also, I think that Electron brings in some form of protection that may prevent hyperlinks from performing as authored, to protect the app user. (I have a lack of knowledge in this area, I could be wrong)

For example, in a typical web browser that has an iframe on the page, the hyperlink author can add an attribute to force a navigation at the parent level. In our Electron app, these attributes do nothing and clicking on a link fails to do what a user might expect it to.

The MCP-UI SDK gives us the link message type for, I believe, better security hygiene. So adding open-external was my attempt at finding the safest execution route.

@michaelneale
Copy link
Collaborator

I think a good upgrade - main Q was what to test for expecially adding messages, and also - the new open rpc - not sure what that does that a hyperlink can't do?

…and toast notifications

- Introduced new action types for size changes, iframe readiness, and data requests to improve message handling.
- Implemented a ToastComponent for displaying notifications with support for implemented and unimplemented message types.
- Updated the main action handler to accommodate new action cases, ensuring comprehensive handling of messages from the iframe.
- Improved theme management for toast notifications to enhance user experience.
@aharvard
Copy link
Collaborator Author

FYI, just push some last bits of feature refinement. Ready for re-review @zanesq, @michaelneale, @JHKennedy4, and others. TY!

@zanesq
Copy link
Collaborator

zanesq commented Aug 21, 2025

.bundle

@aharvard aharvard requested a review from zanesq August 21, 2025 16:28
Comment on lines +142 to +145
"overrides": {
"react": "^19.1.1",
"react-dom": "^19.1.1"
},
Copy link
Collaborator Author

@aharvard aharvard Aug 21, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

had to do this, but we can remove when this issue is completely resolved MCP-UI-Org/mcp-ui#90

@github-actions
Copy link
Contributor

⚠️ Cannot proceed with operation

  • reviewDecision: skip_reviews
  • commitStatus: FAILURE

Reviews are not required for this operation but CI checks must be passing in order to continue

@github-actions
Copy link
Contributor

macOS ARM64 Desktop App (Apple Silicon)

📱 Download macOS Desktop App (arm64, unsigned)

Instructions:
After downloading, unzip the file and drag the Goose.app to a location you prefer. The app is unsigned, so to run it run xattr -r -d com.apple.quarantine '/path/to/Goose.app' and then open the app

Copy link
Collaborator

@zanesq zanesq left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM after react version issue fixed

@aharvard aharvard merged commit db8518f into main Aug 21, 2025
15 of 17 checks passed
@aharvard aharvard deleted the feat/enhance-mcp-ui branch August 21, 2025 20:21
lifeizhou-ap added a commit that referenced this pull request Aug 22, 2025
* main: (108 commits)
  Remove unused game (#4226)
  fix issue where app redirects to home after initialization but user has already started a chat (#4260)
  Feat: Let providers configure a fast model for summarization (#4228)
  docs: update tool selection strategy (#4258)
  feat: upgrade `@mcp-ui/client` package and improve UI message handling (#4164)
  stop replacing chat window when changing working directory (#4200)
  Only fetch session tokens when chat state is idle to avoid resetting during streaming (#4104)
  bump timeouts for e2e tests (#4251)
  docs: custom context files improvements (#4096)
  chore: upgrade rmcp to 0.6.0 (#4243)
  doc: uvx not npx (#4240)
  Add PKCE support for Tetrate Agent Router Service (#4165)
  Read AGENTS.md by default (#4232)
  docs: configure provider and model (#4235)
  docs: add figma tutorial (#4231)
  Add Nix flake for reproducible builds (#4213)
  Enhanced onboarding page visual design (#4156)
  feat: adds mtls to all providers (#2794) (#2799)
  Don't show a confirm dialog for quitting (#4225)
  Fix: Missing smart_approve in CLI /mode help text and error message (#4132)
  ...
michaelneale added a commit that referenced this pull request Aug 22, 2025
* main:
  docs: update View/Edit Recipe menu item name (#4267)
  Remove unused game (#4226)
  fix issue where app redirects to home after initialization but user has already started a chat (#4260)
  Feat: Let providers configure a fast model for summarization (#4228)
  docs: update tool selection strategy (#4258)
  feat: upgrade `@mcp-ui/client` package and improve UI message handling (#4164)
  stop replacing chat window when changing working directory (#4200)
  Only fetch session tokens when chat state is idle to avoid resetting during streaming (#4104)
  bump timeouts for e2e tests (#4251)
  docs: custom context files improvements (#4096)
  chore: upgrade rmcp to 0.6.0 (#4243)
  doc: uvx not npx (#4240)
  Add PKCE support for Tetrate Agent Router Service (#4165)
  Read AGENTS.md by default (#4232)
  docs: configure provider and model (#4235)
michaelneale added a commit that referenced this pull request Aug 22, 2025
* main: (42 commits)
  feat: Add message queue system with interruption handling (#4179)
  Start extensions concurrently  (#4234)
  Add X-Title and referer headers on exchange to tetrate (#4250)
  docs: update View/Edit Recipe menu item name (#4267)
  Remove unused game (#4226)
  fix issue where app redirects to home after initialization but user has already started a chat (#4260)
  Feat: Let providers configure a fast model for summarization (#4228)
  docs: update tool selection strategy (#4258)
  feat: upgrade `@mcp-ui/client` package and improve UI message handling (#4164)
  stop replacing chat window when changing working directory (#4200)
  Only fetch session tokens when chat state is idle to avoid resetting during streaming (#4104)
  bump timeouts for e2e tests (#4251)
  docs: custom context files improvements (#4096)
  chore: upgrade rmcp to 0.6.0 (#4243)
  doc: uvx not npx (#4240)
  Add PKCE support for Tetrate Agent Router Service (#4165)
  Read AGENTS.md by default (#4232)
  docs: configure provider and model (#4235)
  docs: add figma tutorial (#4231)
  Add Nix flake for reproducible builds (#4213)
  ...
@alexhancock alexhancock mentioned this pull request Aug 22, 2025
shellz-n-stuff pushed a commit to shellz-n-stuff/goose that referenced this pull request Aug 27, 2025
dorien-koelemeijer pushed a commit to dorien-koelemeijer/goose that referenced this pull request Sep 2, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

6 participants