✂️ refactor: Artifacts and Tool Callbacks to Pass UI Resources

[danny-avila]

Originally #9472

---

Summary

By @samuelpath

I recently shipped the 2 followings PRs to enable a basic integration with MCP-UI:

• 🎨 feat: MCP UI basic integration #9299
• 🚦 feat: Simplify MCP UI integration and add unit tests #9418

The approach I took to pass the UI resources received from the MCP Server tool response to the front-end was to use a content item of type text and passing the UI resources array in a new metadata property.

This worked but the issue is that these content items of type text are passed as such the LLMs. We have been lucky so far since all the LLM providers we've tested silently ignored this unknown metadata property. But there is no guarantee that all providers do, nor that existing providers could enforce stricter payload checks.

In order to work around this, we proposed the following PR in the Agents package: danny-avila/agents#16. It would remove the metadata property just before the call to the LLM would be made. However, this was rightly rejected by @danny-avila who explained that there was a better way, using callbacks.

This PR now uses @danny-avila's recommended approach with the callbacks.

Here's the idea:

• Create a new artifact type for the UI resources Tools.ui_resources
• Add the UI Resources as artifacts and not in the LLM content
• The callback is called after the LLM is being called, in which we can parse the artifacts as LLM content attachment
• We pass the attachments from ToolCall to ToolCallInfo
• We parse the attachments of type UI Resources in ToolCallInfo to render them

Change Type

• Bug fix (non-breaking change which fixes an issue)

Testing

See the 2 PRs mentioned above. The behaviour tested is the same.

Test Configuration:

Checklist

Please delete any irrelevant options.

• My code adheres to this project's style guidelines
• I have performed a self-review of my own code
• I have commented in any complex areas of my code
• My changes do not introduce new warnings
• I have written tests demonstrating that my changes are effective or that my feature works
• Local unit tests pass with my changes
• Any changes dependent on mine have been merged and published in downstream modules.

[samuelpath]

Thanks @danny-avila, I find the PR cleaner with your follow-up changes!

Also, thank you for 2 things:

• For cherry-picking my commit and keeping my authorship
• For adding follow-up commits in an atomic way with a clear separation of concerns

I appreciate how you've been taking some our suggestions into account here for smoother collaboration, this means a lot. Thanks!