-
Notifications
You must be signed in to change notification settings - Fork 2.4k
feat: upgrade @mcp-ui/client package and improve UI message handling
#4164
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
|
.bundle |
macOS ARM64 Desktop App (Apple Silicon)📱 Download macOS Desktop App (arm64, unsigned) Instructions: |
…nd package-lock.json
…o use isUIResource utility
…ror management - Introduced specific result types for tool calls, prompts, links, notifications, and intents. - Added error handling with strongly typed error codes for better clarity and debugging. - Implemented separate handlers for each action type to improve type safety and maintainability. - Updated the main action handler to support new action types and provide exhaustive type checking.
- Implemented a new IPC handler in the main process to open external URLs securely. - Updated the preload script to expose the openExternal function for invoking the new handler from the renderer process.
…sage handling - Added a global event listener for scroll-to-bottom requests in BaseChat to improve user experience. - Enhanced MCPUIResourceRenderer and ToolCallWithResponse components to support an append function for message handling. - Updated prompt action handling to utilize the append function, providing fallback options for message delivery.
… and enhancing error handling - Removed optional callbacks for tool calls, prompts, navigation, and intents to streamline the component. - Improved error handling for unsupported actions in tool calls, prompts, and intents. - Ensured that the append function is utilized for prompt actions and added custom event dispatching for chat scrolling.
…message handling - Renamed the append prop to appendPromptToChat for clarity in MCPUIResourceRenderer. - Updated ToolCallWithResponse to pass the new appendPromptToChat prop to MCPUIResourceRenderer. - Revised TODO comments for better clarity on future enhancements regarding message handling.
…dling - Expanded action handling in MCPUIResourceRenderer to include specific cases for tools, prompts, links, notifications, and intents. - Improved error handling with consistent status codes and messages for unsupported actions. - Streamlined the main action handler for better readability and maintainability, ensuring exhaustive type checking. - Removed unused callback functions to simplify the component structure.
…nous - Converted action handler functions in MCPUIResourceRenderer to asynchronous to support promise-based operations. - Ensured that the main action handler awaits results from tool, prompt, notify, and intent cases for improved error handling and response management.
| const handleUIAction = async (actionEvent: UIActionResult): Promise<UIActionHandlerResult> => { | ||
| console.log('[MCP-UI] Action received:', actionEvent); | ||
|
|
||
| case 'notify': { | ||
| // TODO: Implement notify handling | ||
| handleAction(result); | ||
| break; | ||
| let result: UIActionHandlerResult; | ||
|
|
||
| const handleToolCase = async ( | ||
| actionEvent: UIActionResultToolCall | ||
| ): Promise<UIActionHandlerResult> => { | ||
| const { toolName, params } = actionEvent.payload; | ||
| return { | ||
| status: 'error' as const, | ||
| error: { | ||
| code: UIActionErrorCode.UNSUPPORTED_ACTION, | ||
| message: 'Tool calls are not yet implemented', | ||
| details: { toolName, params }, | ||
| }, | ||
| }; | ||
| }; | ||
|
|
||
| const handlePromptCase = async ( | ||
| actionEvent: UIActionResultPrompt | ||
| ): Promise<UIActionHandlerResult> => { | ||
| const { prompt } = actionEvent.payload; | ||
|
|
||
| if (appendPromptToChat) { | ||
| try { | ||
| appendPromptToChat(prompt); | ||
| window.dispatchEvent(new CustomEvent('scroll-chat-to-bottom')); | ||
| return { | ||
| status: 'success' as const, | ||
| message: 'Prompt sent to chat successfully', | ||
| }; | ||
| } catch (error) { | ||
| return { | ||
| status: 'error' as const, | ||
| error: { | ||
| code: UIActionErrorCode.PROMPT_FAILED, | ||
| message: 'Failed to send prompt to chat', | ||
| details: error instanceof Error ? error.message : error, | ||
| }, | ||
| }; | ||
| } | ||
| } | ||
|
|
||
| case 'prompt': { | ||
| // TODO: Implement prompt handling | ||
| handleAction(result); | ||
| break; | ||
| return { | ||
| status: 'error' as const, | ||
| error: { | ||
| code: UIActionErrorCode.UNSUPPORTED_ACTION, | ||
| message: 'Prompt handling is not implemented - append prop is required', | ||
| details: { prompt }, | ||
| }, | ||
| }; | ||
| }; | ||
|
|
||
| const handleLinkCase = async (actionEvent: UIActionResultLink) => { | ||
| const { url } = actionEvent.payload; | ||
|
|
||
| try { | ||
| const urlObj = new URL(url); | ||
| if (!['http:', 'https:'].includes(urlObj.protocol)) { | ||
| return { | ||
| status: 'error' as const, | ||
| error: { | ||
| code: UIActionErrorCode.NAVIGATION_FAILED, | ||
| message: `Blocked potentially unsafe URL protocol: ${urlObj.protocol}`, | ||
| details: { url, protocol: urlObj.protocol }, | ||
| }, | ||
| }; | ||
| } | ||
|
|
||
| await window.electron.openExternal(url); | ||
| return { | ||
| status: 'success' as const, | ||
| message: `Opened ${url} in default browser`, | ||
| }; | ||
| } catch (error) { | ||
| if (error instanceof TypeError && error.message.includes('Invalid URL')) { | ||
| return { | ||
| status: 'error' as const, | ||
| error: { | ||
| code: UIActionErrorCode.INVALID_PARAMS, | ||
| message: `Invalid URL format: ${url}`, | ||
| details: { url, error: error.message }, | ||
| }, | ||
| }; | ||
| } else if (error instanceof Error && error.message.includes('Failed to open')) { | ||
| return { | ||
| status: 'error' as const, | ||
| error: { | ||
| code: UIActionErrorCode.NAVIGATION_FAILED, | ||
| message: `Failed to open URL in default browser`, | ||
| details: { url, error: error.message }, | ||
| }, | ||
| }; | ||
| } else { | ||
| return { | ||
| status: 'error' as const, | ||
| error: { | ||
| code: UIActionErrorCode.NAVIGATION_FAILED, | ||
| message: `Unexpected error opening URL: ${url}`, | ||
| details: error instanceof Error ? error.message : error, | ||
| }, | ||
| }; | ||
| } | ||
| } | ||
| }; | ||
|
|
||
| case 'tool': { | ||
| // TODO: Implement tool call handling | ||
| handleAction(result); | ||
| break; | ||
| const handleNotifyCase = async ( | ||
| actionEvent: UIActionResultNotification | ||
| ): Promise<UIActionHandlerResult> => { | ||
| const { message } = actionEvent.payload; | ||
|
|
||
| try { | ||
| const notificationId = `notify-${Date.now()}`; | ||
| toast.info(message); | ||
| return { | ||
| status: 'success' as const, | ||
| data: { | ||
| notificationId, | ||
| displayedAt: new Date().toISOString(), | ||
| message, | ||
| }, | ||
| }; | ||
| } catch (error) { | ||
| return { | ||
| status: 'error' as const, | ||
| error: { | ||
| code: UIActionErrorCode.UNKNOWN_ACTION, | ||
| message: 'Failed to display notification', | ||
| details: error instanceof Error ? error.message : error, | ||
| }, | ||
| }; | ||
| } | ||
| }; | ||
|
|
||
| const handleIntentCase = async ( | ||
| actionEvent: UIActionResultIntent | ||
| ): Promise<UIActionHandlerResult> => { | ||
| const { intent, params } = actionEvent.payload; | ||
|
|
||
| return { | ||
| status: 'error' as const, | ||
| error: { | ||
| code: UIActionErrorCode.UNSUPPORTED_ACTION, | ||
| message: 'Intent handling is not yet implemented', | ||
| details: { intent, params }, | ||
| }, | ||
| }; | ||
| }; | ||
|
|
||
| try { | ||
| switch (actionEvent.type) { | ||
| case 'tool': | ||
| result = await handleToolCase(actionEvent); | ||
| break; | ||
|
|
||
| case 'prompt': | ||
| result = await handlePromptCase(actionEvent); | ||
| break; | ||
|
|
||
| case 'link': | ||
| result = await handleLinkCase(actionEvent); | ||
| break; | ||
|
|
||
| default: { | ||
| console.warn('unsupported message sent from MCP-UI:', result); | ||
| break; | ||
| case 'notify': | ||
| result = await handleNotifyCase(actionEvent); | ||
| break; | ||
|
|
||
| case 'intent': | ||
| result = await handleIntentCase(actionEvent); | ||
| break; | ||
|
|
||
| default: { | ||
| // TypeScript exhaustiveness check | ||
| const _exhaustiveCheck: never = actionEvent; | ||
| console.error('Unhandled action type:', _exhaustiveCheck); | ||
| result = { | ||
| status: 'error', | ||
| error: { | ||
| code: UIActionErrorCode.UNKNOWN_ACTION, | ||
| message: `Unknown action type`, | ||
| details: actionEvent, | ||
| }, | ||
| }; | ||
| } | ||
| } | ||
| } catch (error) { | ||
| console.error('[MCP-UI] Unexpected error:', error); | ||
| result = { | ||
| status: 'error', | ||
| error: { | ||
| code: UIActionErrorCode.UNKNOWN_ACTION, | ||
| message: 'An unexpected error occurred', | ||
| details: error instanceof Error ? error.stack : error, | ||
| }, | ||
| }; | ||
| } | ||
|
|
||
| // Log result with appropriate level | ||
| if (result.status === 'error') { | ||
| console.error('[MCP-UI] Action failed:', result); | ||
| } else { | ||
| console.log('[MCP-UI] Action succeeded:', result); | ||
| } | ||
| }, []); | ||
|
|
||
| return result; | ||
| }; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Hey @idosal, thanks again for taking a look at this.
I took another pass at handling UI actions here. Now, I'm returning a lot more info. Is this more in line with your suggestion?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks @aharvard, it looks great. It captures the data that could be relevant to the UI.
This dependency was accidentally included through a merge conflict resolution and should not be there.
@mcp-ui/client package and start UI message implementation
@mcp-ui/client package and start UI message implementation @mcp-ui/client package and improve UI message handling
|
.bundle |
macOS ARM64 Desktop App (Apple Silicon)📱 Download macOS Desktop App (arm64, unsigned) Instructions: |
| toolResponse={toolResponsesMap.get(toolRequest.id)} | ||
| notifications={toolCallNotifications.get(toolRequest.id)} | ||
| isStreamingMessage={isStreaming} | ||
| append={append} |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
what does this do?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
append is an existing convention in this file, and I had to add it to the <ToolCallWithResponse /> component to prop drill. TBH, this feels like a hack.
Inside of <ToolCallWithResponse />, append is passed to <MCPUIResourceRenderer content={content} appendPromptToChat={append} /> — where, I think, appendPromptToChat is a bit clearer.
The goal is to
- Catch the prompt message from an MCP UI
- Pass that message to the chat engine
- Let goose take the wheel
@zane, do you know if there is a more elegant way to pass a prompt string to the chat engine w/out prop drilling?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think prop drilling is ok for 2 levels like this for now but we don't have any other global state mechanism currently other than React context. So you could use context if needed. We plan on adding a global state library soon that can make things like this easier.
| }); | ||
|
|
||
| // Handle external URL opening | ||
| ipcMain.handle('open-external', async (_event, url: string) => { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
what functionality does this add that wasn't before - is it for cases where the type is links and it will open it in external browser? (if so - a hyperlink won't do that?)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Since MCP-UI hyperlinks live inside of an iframe, clicking on a typical <a href="" /> link will refresh the contents inside of the iframe. That might be perfect for some MCP-UIs but not others.
Also, I think that Electron brings in some form of protection that may prevent hyperlinks from performing as authored, to protect the app user. (I have a lack of knowledge in this area, I could be wrong)
For example, in a typical web browser that has an iframe on the page, the hyperlink author can add an attribute to force a navigation at the parent level. In our Electron app, these attributes do nothing and clicking on a link fails to do what a user might expect it to.
The MCP-UI SDK gives us the link message type for, I believe, better security hygiene. So adding open-external was my attempt at finding the safest execution route.
|
I think a good upgrade - main Q was what to test for expecially adding messages, and also - the new open rpc - not sure what that does that a hyperlink can't do? |
…and toast notifications - Introduced new action types for size changes, iframe readiness, and data requests to improve message handling. - Implemented a ToastComponent for displaying notifications with support for implemented and unimplemented message types. - Updated the main action handler to accommodate new action cases, ensuring comprehensive handling of messages from the iframe. - Improved theme management for toast notifications to enhance user experience.
|
FYI, just push some last bits of feature refinement. Ready for re-review @zanesq, @michaelneale, @JHKennedy4, and others. TY! |
|
.bundle |
| "overrides": { | ||
| "react": "^19.1.1", | ||
| "react-dom": "^19.1.1" | ||
| }, |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
had to do this, but we can remove when this issue is completely resolved MCP-UI-Org/mcp-ui#90
|
macOS ARM64 Desktop App (Apple Silicon)📱 Download macOS Desktop App (arm64, unsigned) Instructions: |
zanesq
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM after react version issue fixed
* main: (108 commits) Remove unused game (#4226) fix issue where app redirects to home after initialization but user has already started a chat (#4260) Feat: Let providers configure a fast model for summarization (#4228) docs: update tool selection strategy (#4258) feat: upgrade `@mcp-ui/client` package and improve UI message handling (#4164) stop replacing chat window when changing working directory (#4200) Only fetch session tokens when chat state is idle to avoid resetting during streaming (#4104) bump timeouts for e2e tests (#4251) docs: custom context files improvements (#4096) chore: upgrade rmcp to 0.6.0 (#4243) doc: uvx not npx (#4240) Add PKCE support for Tetrate Agent Router Service (#4165) Read AGENTS.md by default (#4232) docs: configure provider and model (#4235) docs: add figma tutorial (#4231) Add Nix flake for reproducible builds (#4213) Enhanced onboarding page visual design (#4156) feat: adds mtls to all providers (#2794) (#2799) Don't show a confirm dialog for quitting (#4225) Fix: Missing smart_approve in CLI /mode help text and error message (#4132) ...
* main: docs: update View/Edit Recipe menu item name (#4267) Remove unused game (#4226) fix issue where app redirects to home after initialization but user has already started a chat (#4260) Feat: Let providers configure a fast model for summarization (#4228) docs: update tool selection strategy (#4258) feat: upgrade `@mcp-ui/client` package and improve UI message handling (#4164) stop replacing chat window when changing working directory (#4200) Only fetch session tokens when chat state is idle to avoid resetting during streaming (#4104) bump timeouts for e2e tests (#4251) docs: custom context files improvements (#4096) chore: upgrade rmcp to 0.6.0 (#4243) doc: uvx not npx (#4240) Add PKCE support for Tetrate Agent Router Service (#4165) Read AGENTS.md by default (#4232) docs: configure provider and model (#4235)
* main: (42 commits) feat: Add message queue system with interruption handling (#4179) Start extensions concurrently (#4234) Add X-Title and referer headers on exchange to tetrate (#4250) docs: update View/Edit Recipe menu item name (#4267) Remove unused game (#4226) fix issue where app redirects to home after initialization but user has already started a chat (#4260) Feat: Let providers configure a fast model for summarization (#4228) docs: update tool selection strategy (#4258) feat: upgrade `@mcp-ui/client` package and improve UI message handling (#4164) stop replacing chat window when changing working directory (#4200) Only fetch session tokens when chat state is idle to avoid resetting during streaming (#4104) bump timeouts for e2e tests (#4251) docs: custom context files improvements (#4096) chore: upgrade rmcp to 0.6.0 (#4243) doc: uvx not npx (#4240) Add PKCE support for Tetrate Agent Router Service (#4165) Read AGENTS.md by default (#4232) docs: configure provider and model (#4235) docs: add figma tutorial (#4231) Add Nix flake for reproducible builds (#4213) ...
block#4164) Signed-off-by: Alex Rosenzweig <[email protected]>
block#4164) Signed-off-by: Dorien Koelemeijer <[email protected]>
This PR enhances MCP-UI functionality in Goose.
Chore: MCP-UI Upgrade
I bumped @mcp-ui/client to version (5.9.0), this lets us experiment with the following:
htmlProps.sandboxPermissionsisUIResource(content)helper for convenienceFeat: Allow forms for iframe sandbox
We got a request to
allow-formsfrom our MCP-UI iframe: #4117. So, this PR allows us to experiment with settinghtmlProps.sandboxPermissionstoallow-forms— for more info refer to docs https://mcpui.dev/guide/client/resource-renderer#props-details.Important
I'm unsure what attack vectors this introduces but would like to ask if we can be flexible and quick to make updates to
htmlProps.sandboxPermissionsover time as we learn more.Feat: Embedded UI Message Support
For more info on what MCP-UIs can message to Goose, refer to: https://mcpui.dev/guide/embeddable-ui#message-types
goose-mcp-ui-message-handling.mov
openExternal()method.Fix: UI Action callback response
In our first iteration,
onUIAction={handleUIAction}did not return a meaningful response (refer to this issue raised over on mcp-ui). Now it does.