fix(browser): persist remote browser_vision screenshots locally - #11752
fix(browser): persist remote browser_vision screenshots locally#11752sgaofen wants to merge 1 commit into
Conversation
|
CI note: the failing |
teknium1
left a comment
There was a problem hiding this comment.
Thanks for isolating the remote-screenshot failure and adding cloud/local regression coverage.
Problems
- The new
cloud_provider is Nonecheck attools/browser_tool.py:1963does not cover direct CDP sessions. Current main acceptsBROWSER_CDP_URLandbrowser.cdp_urlas direct, non-local browser endpoints (tools/browser_tool.py:451-460,781-785), but those have no cloud provider and would still receive the Hermes-host output path. - The patch targets the pre-Lightpanda
browser_visionlayout. Current main has a separate Chrome pre-route and cache-copy path attools/browser_tool.py:3953-4016; the salvage needs to preserve that flow.
Suggested changes
- Rework the capture-path decision against current
browser_vision, covering configured cloud providers and direct CDP overrides, then copy the returned local temporary file into the managed screenshot cache. - Add a direct-CDP regression alongside the provider-cloud and local cases.
Automated hermes-sweeper review.
| @@ -1961,30 +1962,48 @@ def browser_vision(question: str, annotate: bool = False, task_id: Optional[str] | |||
| if annotate: | |||
| screenshot_args.append("--annotate") | |||
There was a problem hiding this comment.
_get_cloud_provider() is None does not mean this session is local: current main supports direct BROWSER_CDP_URL / browser.cdp_url sessions, which are non-local but have no cloud provider. This branch would still pass the Hermes-host cache path to that remote CDP browser; cover the CDP override path too.
Summary
Root Cause
browser_vision()always passed a local~/.hermes/cache/screenshots/...path into the screenshot command and then immediately expected that same host path to exist. That works for local Chromium, but remote Browserbase/CDP sessions cannot write into WSL or other host-local filesystem paths.Testing
python3 -m py_compile /Users/stephenyu/Documents/hermes-agent-wt-11729/tools/browser_tool.py /Users/stephenyu/Documents/hermes-agent-wt-11729/tests/tools/test_browser_console.pyuv run --directory /Users/stephenyu/Documents/hermes-agent-wt-11729 --extra dev pytest -o addopts='' /Users/stephenyu/Documents/hermes-agent-wt-11729/tests/tools/test_browser_console.py -qPlatform Tested
Contribution Guide Notes
browser_vision()and its regression tests