Skip to content

fix(desktop): expose preview tools for remote gateways - #80597

Closed
BearHuddleston wants to merge 1 commit into
NousResearch:mainfrom
BearHuddleston:fix/desktop-remote-preview-tools
Closed

fix(desktop): expose preview tools for remote gateways#80597
BearHuddleston wants to merge 1 commit into
NousResearch:mainfrom
BearHuddleston:fix/desktop-remote-preview-tools

Conversation

@BearHuddleston

@BearHuddleston BearHuddleston commented Aug 6, 2026

Copy link
Copy Markdown
Contributor

What does this PR do?

Hermes Desktop identifies remote-gateway sessions as platform="desktop", but the preview tools were gated only by the gateway process's HERMES_DESKTOP environment variable. A separately launched gateway does not inherit that variable, so open_preview and read_preview were omitted from the agent's tool schemas even though the existing gateway transport can route their events to Desktop.

This change evaluates preview-tool availability in a context-local session-platform scope. It keeps the existing HERMES_DESKTOP behavior for locally Desktop-managed gateways while making the tools available to remote Desktop sessions. Both the tool-definition cache and requirement-check cache include the session platform so concurrent Desktop and CLI sessions cannot influence each other's tool surfaces.

Related Issue

Fixes #79585

Related: #76008 addressed rendering remote HTML files over SSH. This PR covers the separate case where the model never receives the in-app browser tools when Desktop connects to an independently launched gateway.

Also distinct from #48760, which proposes a separate desktop_browser_* surface. This patch fixes the existing open_preview / read_preview tools on current main and does not add a new browser implementation.

Type of Change

  • 🐛 Bug fix (non-breaking change that fixes an issue)
  • ✨ New feature (non-breaking change that adds functionality)
  • 🔒 Security fix
  • 📝 Documentation update
  • ✅ Tests (adding or improving test coverage)
  • ♻️ Refactor (no behavior change)
  • 🎯 New skill (bundled or hub)

Changes Made

  • Add a context-local platform scope for tool-availability checks in tools/registry.py.
  • Include the scoped platform in the inner requirement-check cache and outer tool-definition cache keys.
  • Build initial agent tool definitions using the agent's resolved platform.
  • Preserve the agent platform when MCP/tool refreshes rebuild definitions.
  • Allow open_preview and read_preview for Desktop-platform sessions while retaining the existing environment-based behavior.
  • Update module/toolset comments to describe both local environment and remote Desktop-session availability.
  • Add regressions for remote Desktop availability, Desktop/CLI cache isolation, concurrent definition builds, and MCP refreshes.

How to Test

  1. Unset HERMES_DESKTOP and construct an AIAgent with platform="desktop"; verify open_preview and read_preview remain in valid_tool_names.
  2. Build Desktop and CLI tool definitions sequentially and concurrently; verify only Desktop receives the preview tools and the caches remain isolated.
  3. Refresh a remote Desktop agent's MCP/tool definitions; verify both preview tools remain available.
  4. Run the focused suite:
    scripts/run_tests.sh tests/agent/test_desktop_tool_availability.py tests/tools/test_open_preview_tool.py tests/tools/test_read_preview_tool.py tests/tools/test_refresh_agent_mcp_tools.py tests/tools/test_registry.py tests/test_model_tools.py tests/test_get_tool_definitions_cache_isolation.py tests/test_toolsets.py tests/tui_gateway/test_compute_host.py tests/tui_gateway/test_compute_host_phase1.py

Checklist

Code

  • I've read the Contributing Guide
  • My commit messages follow Conventional Commits (fix(scope):, feat(scope):, etc.)
  • I searched for existing PRs to make sure this isn't a duplicate
  • My PR contains only changes related to this fix/feature (no unrelated commits)
  • I've run pytest tests/ -q and all tests pass
  • I've added tests for my changes (required for bug fixes, strongly encouraged for features)
  • I've tested on my platform: CachyOS Linux, Python 3.11.15

Documentation & Housekeeping

  • I've updated relevant documentation (README, docs/, docstrings) — or N/A
  • I've updated cli-config.yaml.example if I added/changed config keys — or N/A
  • I've updated CONTRIBUTING.md or AGENTS.md if I changed architecture or workflows — or N/A
  • I've considered cross-platform impact (Windows, macOS) per the compatibility guide — or N/A
  • I've updated tool descriptions/schemas if I changed tool behavior — or N/A

Security / Cross-platform Impact

This does not change authentication, gateway routing, renderer permissions, or the preview event protocol. It only makes an existing Desktop-only tool surface depend on immutable per-session context rather than mutable process-global state. ContextVar scoping prevents concurrent sessions from leaking availability state. The implementation uses only cross-platform Python primitives, and scripts/check-windows-footguns.py reports no new issues.

Limitations

This intentionally covers only the Desktop preview/browser tools involved in #79585. It does not broaden remote availability for unrelated Desktop terminal or pane-control tools.

Screenshots / Logs

Final focused canonical suite: 110 tests passed across 10 files. The full canonical suite completed before the comment-only review follow-up with 26,143 passed and 1 failed. The failure was the unrelated tests/agent/test_credential_pool_routing.py::TestFailureAttribution::test_unmatched_key_does_not_retry_only_pool_entry; it reproduces with the same assertion on a disposable worktree at the exact upstream/main base commit. The runner also retried two unrelated flaky files (test_config_read_guard.py and test_wake_word.py), both of which passed on retry.

The split-process compute-host path also emitted a real preview.open JSON-RPC event to the owning session, confirming that current gateway event transport is already present once the tool is exposed.

Use session-scoped platform context when filtering tool definitions so independently launched gateways retain open_preview and read_preview for Desktop sessions. Keep both cache layers platform-isolated and preserve the platform across MCP refreshes.
@BearHuddleston
BearHuddleston force-pushed the fix/desktop-remote-preview-tools branch from 3f38c92 to 80033af Compare August 6, 2026 21:12
@alt-glitch alt-glitch added type/bug Something isn't working comp/agent Core agent runtime: loop, agent_init, prompt builder, context-compression, responses endpoint comp/tools Tool registry, model_tools, toolsets comp/desktop Electron desktop app (apps/desktop/*) tool/browser Browser automation (CDP, Playwright) P2 Medium — degraded but workaround exists labels Aug 6, 2026
@BearHuddleston

Copy link
Copy Markdown
Contributor Author

Closing as superseded by #80699, which landed the same remote/cloud Desktop tool-availability fix on main using the dedicated desktop_ui toolset and session-source scoping. That broader implementation also covers the terminal, pane-focus, reaction, and late-MCP-refresh paths.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

comp/agent Core agent runtime: loop, agent_init, prompt builder, context-compression, responses endpoint comp/desktop Electron desktop app (apps/desktop/*) comp/tools Tool registry, model_tools, toolsets P2 Medium — degraded but workaround exists tool/browser Browser automation (CDP, Playwright) type/bug Something isn't working

Projects

None yet

Development

Successfully merging this pull request may close these issues.

feat(desktop): route preview.open / desktop_ui events through gateway WebSocket for remote-backend deployments

2 participants