Skip to content

fix: sanitize tool error results before llm injection - #1635

Closed
serrrfirat wants to merge 3 commits into
feat/multi-tenant-isolation-phases-2-4from
fix/tool-error-sanitization
Closed

serrrfirat wants to merge 3 commits into
feat/multi-tenant-isolation-phases-2-4from
fix/tool-error-sanitization

Conversation

@serrrfirat

Copy link
Copy Markdown
Collaborator

Summary

  • route failed tool results through the same sanitize-and-wrap safety path as successful tool outputs
  • switch chat dispatch to use the shared process_tool_result() helper instead of formatting raw tool errors into LLM context
  • add regressions covering wrapped error output, boundary-injection neutralization, and dispatcher-level tool-name preservation

Verification

  • cargo test test_process_tool_result_error --lib
  • cargo test test_tool_error_format_includes_tool_name --lib
  • cargo check --lib

Ultraworked with [Sisyphus](https://github.com/code-yeongyu/oh-my-openagent)

Co-authored-by: Sisyphus <clio-agent@sisyphuslabs.ai>
@github-actions github-actions Bot added scope: agent Agent core (agent loop, router, scheduler) size: M 50-199 changed lines risk: medium Business logic, config, or moderate-risk modules contributor: experienced 6-19 merged PRs labels Mar 25, 2026
@gemini-code-assist

Copy link
Copy Markdown
Contributor

Summary of Changes

Hello, I'm Gemini Code Assist1! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request enhances the safety and consistency of how tool execution results, particularly errors, are handled before being fed into the Large Language Model (LLM). By routing all tool outputs through a unified sanitization and wrapping process, it mitigates potential injection vulnerabilities and provides the LLM with a more structured and secure representation of tool interactions, improving its ability to interpret and respond to both successful and failed tool calls.

Highlights

  • Consistent Tool Result Handling: Ensured all tool results, including errors, are sanitized and wrapped before being sent to the LLM, preventing potential injection vulnerabilities.
  • Centralized Processing: Refactored the ChatDelegate in dispatcher.rs to consistently use the shared process_tool_result helper for all tool outputs, streamlining logic.
  • Enhanced Testing: Added new regression tests to cover wrapped error output, boundary injection neutralization, and preservation of tool names in error messages.
Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point by creating a comment using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands on the current page.

Feature Command Description
Code Review /gemini review Performs a code review for the current pull request in its current state.
Pull Request Summary /gemini summary Provides a summary of the current pull request in its current state.
Comment @gemini-code-assist Responds in comments when explicitly tagged, both in pull request comments and review comments.
Help /gemini help Displays a list of available commands.

Customization

To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for GitHub and other Google products, sign up here.

Footnotes

  1. Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution. ↩

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request refactors the handling of tool results by centralizing the sanitization and wrapping logic within the process_tool_result function in src/tools/execute.rs. This ensures that both successful tool outputs and error messages are consistently formatted, XML-wrapped, and protected against injection attempts before being passed to the LLM. The src/agent/dispatcher.rs file was updated to utilize this new centralized function, and comprehensive tests were added or modified to verify the correct behavior, including the neutralization of potential injection vulnerabilities in error messages. The review feedback indicates that the documentation for process_tool_result should be updated to clearly state that both success and error paths are now sanitized and wrapped. Additionally, a high-severity concern was raised regarding compliance with Rule 4, which dictates that sanitization should only be applied to data paths sent to external services, not trusted local endpoints; clarification is requested on whether this tool's output is intended for an external service or if the sanitization needs reconsideration.

Comment thread src/tools/execute.rs
@@ -118,7 +118,6 @@ pub async fn execute_tool_with_safety(
/// Process a tool result into a `ChatMessage::tool_result` with safety sanitization.
///
/// On success: sanitize → wrap → ChatMessage::tool_result.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

high

The documentation for this function could be improved to reflect the new behavior for handling errors. While the old line about error handling was correctly removed, the current docstring only mentions the success path. It's important to clarify that both success and error results are now sanitized and wrapped. Additionally, please ensure this sanitization aligns with Rule 4: 'Tool output previews sent to trusted local endpoints (e.g., TUI, web gateway) should not be sanitized. Sanitization should only be applied to data paths sent to external services, such as an LLM.' If this tool's output is for a trusted local endpoint, the sanitization should be reconsidered. If it's for an external service, the documentation should explicitly state this context.

Suggested change
/// On success: sanitize → wrap → ChatMessage::tool_result.
/// On success or error: sanitize → wrap → ChatMessage::tool_result.
References
  1. Tool output previews sent to trusted local endpoints (e.g., TUI, web gateway) should not be sanitized. Sanitization should only be applied to data paths sent to external services, such as an LLM.

serrrfirat and others added 2 commits March 25, 2026 09:26
Ultraworked with [Sisyphus](https://github.com/code-yeongyu/oh-my-openagent)

Co-authored-by: Sisyphus <clio-agent@sisyphuslabs.ai>
@serrrfirat

Copy link
Copy Markdown
Collaborator Author

Closing this because it was based on feat/multi-tenant-isolation-phases-2-4 instead of staging. Replacement PR: #1639

@serrrfirat serrrfirat closed this Mar 25, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

contributor: experienced 6-19 merged PRs risk: medium Business logic, config, or moderate-risk modules scope: agent Agent core (agent loop, router, scheduler) size: M 50-199 changed lines

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant