Skip to content

Fix/vision sandbox path resolution - #14990

Closed
Sihui wants to merge 3 commits into
NousResearch:mainfrom
Sihui:fix/vision-sandbox-path-resolution
Closed

Fix/vision sandbox path resolution#14990
Sihui wants to merge 3 commits into
NousResearch:mainfrom
Sihui:fix/vision-sandbox-path-resolution

Conversation

@Sihui

@Sihui Sihui commented Apr 24, 2026

Copy link
Copy Markdown

What does this PR do?

When terminal.backend: docker, file-discovery tools (search_files, terminal) return container-local paths like ./screenshot.png or /workspace/screenshot.png. The vision_analyze tool runs on the host where these paths do not exist, causing "Invalid image source" errors.

This adds _resolve_sandbox_path_to_host() which translates sandbox paths to host paths before the is_file() check, using the volume mappings already defined in config.yaml. No impact on non-Docker setups since unmatched paths pass through unchanged.

Related Issue

Type of Change

  • 🐛 Bug fix (non-breaking change that fixes an issue)

Changes Made

  • tools/vision_tools.py: Add _resolve_sandbox_path_to_host() function that translates container paths to host paths using three strategies:
    1. Relative paths (./foo, ../foo, bare filenames) resolved against TERMINAL_CWD
    2. Absolute container paths matched against docker_volumes from config.yaml
    3. /workspace/... mapped to TERMINAL_CWD when docker_mount_cwd_to_workspace is enabled
  • tools/vision_tools.py: Call _resolve_sandbox_path_to_host() in vision_analyze_tool() before the local file check

How to Test

  1. Set terminal.backend: docker in config.yaml with docker_mount_cwd_to_workspace: true
  2. Open the WebUI, start a session with a workspace that is mounted as a docker volume
  3. Paste an image in the chat
  4. Ask the agent to analyze the image with vision
  5. Before fix: agent discovers the file via search_files at ./screenshot-xxx.png, passes it to vision_analyze, which rejects it with "Invalid image source. Provide an HTTP/HTTPS URL or a valid local file path."
  6. After fix: vision_analyze resolves ./screenshot-xxx.png to the host path and analysis succeeds

Checklist

Code

  • I've read the Contributing Guide
  • My commit messages follow Conventional Commits (fix(scope):, feat(scope):, etc.)
  • I searched for existing PRs to make sure this isn't a duplicate
  • My PR contains only changes related to this fix/feature (no unrelated commits)
  • I've run pytest tests/ -q and all tests pass
  • I've added tests for my changes (required for bug fixes, strongly encouraged for features)
  • I've tested on my platform: macOS 15 (Darwin 25.3.0)

Documentation & Housekeeping

  • I've updated relevant documentation (README, docs/, docstrings) — or N/A
  • I've updated cli-config.yaml.example if I added/changed config keys — or N/A
  • I've updated CONTRIBUTING.md or AGENTS.md if I changed architecture or workflows — or N/A
  • I've considered cross-platform impact (Windows, macOS) per the compatibility guide — or N/A
    • Note: vol.split(":") may mis-split Windows volume paths like C:\Users\...:/container. Docker on Windows is not tested.
  • I've updated tool descriptions/schemas if I changed tool behavior — or N/A

Screenshots / Logs

Before fix - agent finds file via search_files but vision rejects the sandbox path:

vision_analyze  Error analyzing image: Invalid image source.
  image_url: ./screenshot-1777815648778.png
  Error analyzing image: Invalid image source. Provide an HTTP/HTTPS URL or a valid local file path.

After fix - same flow, vision resolves the path and succeeds:

vision_analyze  {"success": true, "analysis": "This screenshot is very clear and readable..."}
  image_url: ./screenshot-1777017504283.png

Sihui added 2 commits April 24, 2026 01:05
When terminal.backend is docker, file-discovery tools (search_files,
terminal) return container-local paths like ./screenshot.png or
/workspace/screenshot.png. The vision tool runs on the host where
these paths do not exist, causing "Invalid image source" errors.

Add _resolve_sandbox_path_to_host() which translates sandbox paths
to host paths using:
- TERMINAL_CWD for relative paths
- docker_volumes from config.yaml for absolute container paths
- /workspace -> TERMINAL_CWD when docker_mount_cwd_to_workspace is set
Cover all path translation strategies in _resolve_sandbox_path_to_host:
- relative paths (./foo, ../foo, bare filenames) via TERMINAL_CWD
- docker_volumes mapping (container prefix -> host prefix)
- /workspace mapping via docker_mount_cwd_to_workspace
- edge cases: missing config, no TERMINAL_CWD, false prefix matches,
  spaces in paths
@alt-glitch alt-glitch added type/bug Something isn't working P2 Medium — degraded but workaround exists tool/vision Vision analysis and image generation backend/docker Docker container execution labels Apr 24, 2026
@teknium1

Copy link
Copy Markdown
Contributor

Thanks for the Docker vision-path investigation. This is now implemented on main by a stronger superseding fix.

  • Automated hermes-sweeper review verified tools/vision_tools.py:970 routes both vision paths through tools.image_source.resolve_image_source(...) with the active task ID.
  • tools/image_source.py:142-145 and :273-316 read non-local paths inside the active sandbox, covering /workspace/... and container-only images without turning container paths into host filesystem reads.
  • Commit 316e77517e82d8f4af8936e036661ca0836990c2 (fix(vision): unified image-source resolver + terminal-backend confinement), merged via PR fix(vision): unified image-source resolver + terminal-backend confinement #57890, explicitly supersedes Fix/vision sandbox path resolution #14990 and adds Docker-resolution coverage in tests/integration/test_vision_docker_resolve.py:103.
  • The commit is contained in release tag v2026.7.7.

@teknium1 teknium1 closed this Jul 12, 2026
@teknium1 teknium1 added the sweeper:implemented-on-main Sweeper: behavior already present on current main label Jul 12, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

backend/docker Docker container execution P2 Medium — degraded but workaround exists sweeper:implemented-on-main Sweeper: behavior already present on current main tool/vision Vision analysis and image generation type/bug Something isn't working

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants