-
Notifications
You must be signed in to change notification settings - Fork 2.4k
added validation and debug for invalid call tool result #6368
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
added validation and debug for invalid call tool result #6368
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Pull request overview
This PR addresses an issue where invalid CallToolResult objects with empty content were being saved to sessions, causing deserialization failures when loading. The fix adds validation to convert invalid tool results into error responses and improves debugging with additional logging.
Key changes:
- Added
validate()function to detect invalidCallToolResultobjects and convert them to errors before saving - Enhanced deserialization error handling with debug logging that includes the original data
- Applied validation in the agent tool execution flow to prevent invalid results from being stored
Reviewed changes
Copilot reviewed 3 out of 3 changed files in this pull request and generated 2 comments.
| File | Description |
|---|---|
crates/goose/src/conversation/tool_result_serde.rs |
Added validation function and debug logging for deserialization failures; includes comprehensive test coverage |
crates/goose/src/conversation/mod.rs |
Changed module visibility from private to public to expose validation function |
crates/goose/src/agents/agent.rs |
Integrated validation into tool result processing pipeline |
| Ok(call_tool_result) => match serde_json::to_string(call_tool_result) { | ||
| Ok(json_str) => match serde_json::from_str::<CallToolResult>(&json_str) { | ||
| Ok(_) => result, | ||
| Err(e) => { | ||
| tracing::error!("CallToolResult failed validation by deserialization: {}. Original data: {}", e, json_str); | ||
| Err(ErrorData { | ||
| code: ErrorCode::INTERNAL_ERROR, | ||
| message: Cow::from(format!("Tool result validation failed: {}", e)), | ||
| data: None, | ||
| }) | ||
| } | ||
| }, | ||
| Err(e) => { | ||
| tracing::error!("CallToolResult failed serialization: {}", e); | ||
| Err(ErrorData { | ||
| code: ErrorCode::INTERNAL_ERROR, | ||
| message: Cow::from(format!("Tool result serialization failed: {}", e)), | ||
| data: None, | ||
| }) | ||
| } | ||
| }, |
Copilot
AI
Jan 7, 2026
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This validation performs an unnecessary serialize->deserialize round-trip. Since CallToolResult is from the rmcp crate which performs validation during deserialization, you can validate directly by checking if content is empty and structured_content is None, matching the MCP standard constraint mentioned in the PR description.
| Ok(call_tool_result) => match serde_json::to_string(call_tool_result) { | |
| Ok(json_str) => match serde_json::from_str::<CallToolResult>(&json_str) { | |
| Ok(_) => result, | |
| Err(e) => { | |
| tracing::error!("CallToolResult failed validation by deserialization: {}. Original data: {}", e, json_str); | |
| Err(ErrorData { | |
| code: ErrorCode::INTERNAL_ERROR, | |
| message: Cow::from(format!("Tool result validation failed: {}", e)), | |
| data: None, | |
| }) | |
| } | |
| }, | |
| Err(e) => { | |
| tracing::error!("CallToolResult failed serialization: {}", e); | |
| Err(ErrorData { | |
| code: ErrorCode::INTERNAL_ERROR, | |
| message: Cow::from(format!("Tool result serialization failed: {}", e)), | |
| data: None, | |
| }) | |
| } | |
| }, | |
| Ok(call_tool_result) => { | |
| let has_empty_content = call_tool_result.content.is_empty(); | |
| let has_no_structured_content = call_tool_result.structured_content.is_none(); | |
| if has_empty_content && has_no_structured_content { | |
| tracing::error!( | |
| "CallToolResult failed validation: content must not be empty when structured_content is None" | |
| ); | |
| Err(ErrorData { | |
| code: ErrorCode::INTERNAL_ERROR, | |
| message: Cow::from( | |
| "Tool result validation failed: content must not be empty when structured_content is None", | |
| ), | |
| data: None, | |
| }) | |
| } else { | |
| result | |
| } | |
| } |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
we could, but it introduced duplicated logic in goose and rmcp. So it would be good to reuse the deserialization function in rmcp to use it as source of truth
| ); | ||
| } | ||
| #[test] | ||
| fn test_validate_return_error_for_invalid_calltoolresult() { |
Copilot
AI
Jan 7, 2026
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Function name has grammatical error: should be "returns_error" not "return_error".
| fn test_validate_return_error_for_invalid_calltoolresult() { | |
| fn test_validate_returns_error_for_invalid_calltoolresult() { |
michaelneale
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
nice one - would be great if you could make this a branch and then it runs through the test_providers.sh end to end etc with live providers
Thanks for the review! The empty content is only returned by router_llm_search tool and most of the tools return valid result . It is hard to create the test data for smoke test. Therefore we use unit test to test this scenario |
* main: (31 commits) added validation and debug for invalid call tool result (#6368) Update MCP apps tutorial: fix _meta structure and version prereq (#6404) Fixed fonts (#6389) Update confidence levels prompt injection detection to reduce false positive rates (#6390) Add ML-based prompt injection detection (#5623) docs: update custom extensions tutorial (#6388) fix ResultsFormat error when loading old sessions (#6385) docs: add MCP Apps tutorial and documentation updates (#6384) changed z-index to make sure the search highlighter does not appear on modal overlay (#6386) Handling special claude model response in github copilot provider (#6369) fix: prevent duplicate rendering when tool returns both mcp-ui and mcp-apps resources (#6378) fix: update MCP Apps _meta.ui.resourceUri to use nested format (SEP-1865) (#6372) feat(providers): add streaming support for Google Gemini provider (#6191) Blog: edit links in mcp apps post (#6371) fix: prevent infinite loop of tool-input notifications in MCP Apps (#6374) fix: Show platform-specific keyboard shortcuts in UI (#6323) fix: we load extensions when agent starts so don't do it up front (#6350) docs: credit HumanLayer in RPI tutorial (#6365) Blog: Goose Lands MCP Apps (#6172) Claude 3.7 is out. we had some harcoded stuff (#6197) ...
* main: (89 commits) fix(google): treat signed text as regular content in streaming (#6400) Add frameDomains and baseUriDomains CSP support for MCP Apps (#6399) fix(ci): add missing dependencies to openapi-schema-check job (#6367) feat: http proxy support Add support for changing working dir and extensions in same window/session (#6057) Sort keys in canonical models (#6403) added validation and debug for invalid call tool result (#6368) Update MCP apps tutorial: fix _meta structure and version prereq (#6404) Fixed fonts (#6389) Update confidence levels prompt injection detection to reduce false positive rates (#6390) Add ML-based prompt injection detection (#5623) docs: update custom extensions tutorial (#6388) fix ResultsFormat error when loading old sessions (#6385) docs: add MCP Apps tutorial and documentation updates (#6384) changed z-index to make sure the search highlighter does not appear on modal overlay (#6386) Handling special claude model response in github copilot provider (#6369) fix: prevent duplicate rendering when tool returns both mcp-ui and mcp-apps resources (#6378) fix: update MCP Apps _meta.ui.resourceUri to use nested format (SEP-1865) (#6372) feat(providers): add streaming support for Google Gemini provider (#6191) Blog: edit links in mcp apps post (#6371) ...
* main: (89 commits) fix(google): treat signed text as regular content in streaming (#6400) Add frameDomains and baseUriDomains CSP support for MCP Apps (#6399) fix(ci): add missing dependencies to openapi-schema-check job (#6367) feat: http proxy support Add support for changing working dir and extensions in same window/session (#6057) Sort keys in canonical models (#6403) added validation and debug for invalid call tool result (#6368) Update MCP apps tutorial: fix _meta structure and version prereq (#6404) Fixed fonts (#6389) Update confidence levels prompt injection detection to reduce false positive rates (#6390) Add ML-based prompt injection detection (#5623) docs: update custom extensions tutorial (#6388) fix ResultsFormat error when loading old sessions (#6385) docs: add MCP Apps tutorial and documentation updates (#6384) changed z-index to make sure the search highlighter does not appear on modal overlay (#6386) Handling special claude model response in github copilot provider (#6369) fix: prevent duplicate rendering when tool returns both mcp-ui and mcp-apps resources (#6378) fix: update MCP Apps _meta.ui.resourceUri to use nested format (SEP-1865) (#6372) feat(providers): add streaming support for Google Gemini provider (#6191) Blog: edit links in mcp apps post (#6371) ...
|
Hi @alexhancock, For the fix with the validation of CallToolResult, do you think it is worthwhile to fix in the rmcp? Either to check during CallToolResult serialization or in the constructor of CallToolResult. I feel check during serialization might be safer |
Summary
Root Cause
When the route llm_search tool was called, it return the empty content in the call_tool_result payload, and saved into session.
payload:
When loading the session and deserialized call_tool_result via rmcp, it returns error as the call_tool_result because of
result.content.is_empty() && result.structured_content.is_none(). This is based on the mcp standard that it should either have at least one content or structured_content.Fix
Type of Change
AI Assistance
Testing
Unit test and manual testing
Related Issues
#6345