fix: keep large MCP response files off tmpfs

[bzqzheng]

Summary

When an MCP tool response exceeds the large-response threshold, goose writes the full text to a spill file and returns the file path to the model. That path previously lived under the process temp directory (/tmp/goose_mcp_responses on Unix-like systems). In Docker, /tmp is often tmpfs-backed, so large responses can still consume container memory even after goose has moved them out of the model context.

This keeps the existing large-response behavior, but moves the spill files to goose's data directory and keeps the path private:

• write large MCP response files under Paths::in_data_dir("mcp_responses") instead of the OS temp directory
• ensure the spill directory is a real directory with 0700 permissions on Unix
• reject a symlink or non-directory at that path before writing
• create persisted response files via tempfile, which gives the file a randomized name and 0600 permissions
• avoid counting the same large text twice while deciding whether to spill it

Testing

• cargo fmt --package goose
• cargo test -p goose agents::large_response_handler::tests -- --nocapture
• cargo clippy -p goose --all-targets -- -D warnings
• git diff --check

Related Issues

Related to #9153. This addresses the tmpfs-backed /tmp spill path described there by moving large MCP response files into goose's data directory. It does not add lifecycle cleanup for old spill files.

Screenshots/Demos (for UX changes)

Before: N/A

After: N/A

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: aec12d6012

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

[chatgpt-codex-connector]

Serialize all tests that read GOOSE_PATH_ROOT

When Rust runs these module tests in parallel, this guard mutates the process-wide GOOSE_PATH_ROOT while sibling large-response tests in the same file also call process_tool_response without taking env_lock. Those calls can spill into this temporary root and then try to read the generated file after this test drops temp_root, making the test suite flaky; please either take the same env lock around every test that reaches Paths::in_data_dir here or avoid changing global env for this assertion.

Useful? React with 👍 / 👎.

[DOsinga]

Thanks for the PR — the Docker tmpfs issue in #9153 is real, but we're not sure moving spill files to data_dir is the right fix.

The current code writes to /tmp, which on real Linux systems gets cleaned up periodically by systemd-tmpfiles / tmpwatch (typically files older than 10 days), and on macOS gets cleaned on reboot. These spill files are temporary by nature — they're written once, read by the agent, and never needed again.

Moving them to ~/.local/share/goose/mcp_responses/ means nothing ever cleans them up. With tool calls that can generate 3MB+ per spill file, this will grow without bound in a user data directory. That trades the Docker RAM issue for unbounded disk growth everywhere.

Could you think about an approach that keeps the temporary nature of these files? For example:

• Clean up spill files after they've been consumed
• Clean up old spill files on startup
• Keep /tmp but document that Docker users should mount a real filesystem there

We'd want to see a cleanup story before moving these into a persistent directory.

[DOsinga]

Going to close this since no answer to the concern. it's a genuine issue though, so if you come up with a better idea, let me know.