Add platform-aware skill/tool filtering - #25735
Conversation
Detect the runtime platform as windows, macos, linux, or wsl and filter skills/tools before exposing them to the model. This prevents Linux-only instructions from appearing in native Windows prompts, which previously caused Hermes to suggest Linux commands or fall back to Python snippets for basic file operations like writing or copying files. WSL is treated as a distinct platform, missing platforms remain universal, and prompt/tool caches now include the runtime platform.
teknium1
left a comment
There was a problem hiding this comment.
Thanks for the cross-platform work. The WSL distinction is a potentially useful direction, but this branch needs a re-scope and redesign against current code.
Problems
agent/skill_utils.py:163-185on current main already filtersplatforms: [linux]againstsys.platform; native Windows (win32) does not match Linux. The reported Linux-skill leak therefore is not demonstrated by the current implementation.tools/registry.py:366-373adds filtering, but this PR changes no concrete built-in tool registration to passplatforms=. The branch therefore leaves all built-in tools universal and does not deliver the claimed tool filtering.hermes_constants.py:258-259uses the host runtime, whileagent/prompt_builder.py:1065-1070distinguishes remote execution backends. Host-based filtering can hide Linux-capable tools/instructions when Hermes runs on Windows but executes in Docker or SSH.
Suggested changes
- Define platform compatibility from the actual execution environment, then apply it consistently to skills and schemas.
- Add concrete tool declarations and end-to-end coverage after that contract is settled.
Automated hermes-sweeper review.
| entry = entries_by_name.get(name) | ||
| if not entry: | ||
| continue | ||
| if entry.platforms and runtime_platform not in entry.platforms: |
There was a problem hiding this comment.
This filter has no effect for built-in tools in this PR: no concrete tools/*.py registration is changed to pass platforms=, so every existing entry normalizes to universal. Add declarations for the tools that genuinely require a platform and exercise them through get_tool_definitions().
| return "windows" | ||
| if sys.platform == "darwin": | ||
| return "macos" | ||
| if sys.platform.startswith("linux"): |
There was a problem hiding this comment.
This is the Hermes host platform, but remote terminal backends execute tool operations in a separate environment. A Windows host with a Linux Docker or SSH backend would incorrectly lose Linux-tagged tools/skills. Resolve compatibility from the execution backend, or explicitly limit this feature to local backends.
|
Thanks, this makes sense. The main issue is that the current implementation treats the host platform as the execution platform, which is incorrect for Docker and SSH backends. I’ll re-scope the branch around an execution-environment platform contract, apply it consistently to skills and tool schemas, and add concrete platform declarations with end-to-end coverage. I’ll also verify whether the reported Windows skill leak is still reproducible on current main before keeping that claim in the PR description. |
|
I think it makes sense to close this PR. I’ll start again from the current main in a new branch and open a replacement PR with the necessary redesign and updated implementation |
GottZ
left a comment
There was a problem hiding this comment.
This was generated by AI during triage.
Summary
Two PRs address the platform-compatibility complex. #21561 established native Windows support and broad platform metadata, while #25735 attempts to add WSL-aware skill/tool filtering but bases compatibility on the Hermes host rather than the actual execution backend and supplies no concrete built-in tool restrictions.
Related pull requests
- #21561 [merged]
related— (+7675/-625) — merged reference implementation: established native Windows support across installers, subprocess and process handling, UTF-8 I/O, gateway lifecycle, tools, tests, documentation, and initial skill platform metadata. It remains relevant because #25735 builds its filtering model on the platform metadata and execution-environment distinctions introduced by this merged baseline. - #25735
related— (+491/-248) — close and redesign: adds exact windows/macos/linux/wsl matching, platform-scoped caches, registry metadata, and widespread WSL declarations, but the diff derives compatibility from the host runtime, does not add platform declarations to concrete built-in tools, and does not demonstrate the claimed current-main Linux-skill leak. Despite the keep_open review on #25735, closure is warranted because the contributor review identifies an execution-backend contract flaw and low salvageability, and the author explicitly agreed to replace the branch from current main.
Suggested consolidation
Do not merge #25735; close it in favor of a replacement PR based on current main that derives compatibility from the actual execution environment, applies that contract consistently to skills and tool schemas, adds concrete built-in tool declarations with end-to-end coverage, and first reproduces the reported Windows skill leak. Keep merged #21561 as the reference implementation and baseline rather than treating it as a duplicate; this recommendation departs from the keep_open review on #25735 because the diff confirms the review's host-versus-backend mismatch and incomplete built-in tool filtering, and the author has accepted a restart.
Cross-PR triage: Reviewed 2 pull requests and 0 issues in this complex. Each diff was read against this issue; Assessment working set: 711 kB of PR diffs, 17 kB of issue/PR text, 7 kB of discussion (4 comments), 0 verify verdicts. verdicts reflect diff content, not PR titles. Part of an automated triage batch.
Detect the runtime platform as
windows,macos,linux, orwsland filter skills/tools before exposing them to the model. This prevents Linux-only instructions from appearing in native Windows prompts, which previously caused Hermes to suggest Linux commands or fall back to Python snippets for basic file operations like writing or copying files.WSL is treated as a distinct platform, missing platforms remain universal, and prompt/tool caches now include the runtime platform.
What does this PR do?
This PR adds runtime platform-aware filtering for Hermes skills and tools.
Hermes can now detect the current runtime platform as one of:
windowsmacoslinuxwslSkills and tools can declare compatible platforms. Hermes filters them before exposing them to the model, so platform-specific instructions and tool schemas are only included when they match the current runtime environment.
This solves a real prompt-quality issue on native Windows. Before this change, Linux-specific skills could still be included in the system prompt, which made Hermes sometimes suggest Linux commands first or work around simple file operations by generating Python code instead of using the more appropriate Windows-aware path.
This approach keeps compatibility simple and explicit:
platformsmeans universal compatibility;linuxmeans native Linux only;wslis treated as a first-class platform;platforms: [linux, wsl].The implementation also includes the runtime platform in prompt/tool cache keys to avoid cross-platform cache leakage.
Related Issue
N/A — no linked issue.
Type of Change
Changes Made
hermes_constants.py.wslsupport.agent/skill_utils.pyto use exact normalized platform keys.agent/prompt_builder.pyso runtime platform participates in filtering/cache behavior.platformsmetadata to registered tools intools/registry.py.model_tools.pyto filter tool definitions by runtime platform before exposing them to the model.platformsmetadata inhermes_cli/plugins.py.wsl.windows,macos,linux, andwslplatform keys.How to Test
Run lint checks for the modified files:
uv run ruff check hermes_constants.py agent/skill_utils.py agent/prompt_builder.py agent/skill_commands.py tools/skills_tool.py tools/registry.py model_tools.py hermes_cli/plugins.py tests/test_hermes_constants.py tests/tools/test_skills_tool.py tests/agent/test_prompt_builder.py tests/agent/test_skill_commands.py tests/tools/test_registry.py tests/test_get_tool_definitions_cache_isolation.py tests/hermes_cli/test_plugins.py ```Run the focused test suite:
uv run pytest -o addopts='' tests/test_hermes_constants.py tests/tools/test_skills_tool.py tests/agent/test_prompt_builder.py tests/agent/test_skill_commands.py tests/tools/test_registry.py tests/test_get_tool_definitions_cache_isolation.py tests/hermes_cli/test_plugins.pyCheck whitespace issues:
Optional manual smoke test on native Windows 11:
windows.Checklist
Code
Documentation & Housekeeping
Screenshots / Logs
Lint and focused tests passed:
git diff --checkis clean.