Skip to content

Add platform-aware skill/tool filtering - #25735

Open
AlexFoxD wants to merge 1 commit into
NousResearch:mainfrom
AlexFoxD:feature/dynamic-skill-filtering
Open

Add platform-aware skill/tool filtering#25735
AlexFoxD wants to merge 1 commit into
NousResearch:mainfrom
AlexFoxD:feature/dynamic-skill-filtering

Conversation

@AlexFoxD

Copy link
Copy Markdown

Detect the runtime platform as windows, macos, linux, or wsl and filter skills/tools before exposing them to the model. This prevents Linux-only instructions from appearing in native Windows prompts, which previously caused Hermes to suggest Linux commands or fall back to Python snippets for basic file operations like writing or copying files.

WSL is treated as a distinct platform, missing platforms remain universal, and prompt/tool caches now include the runtime platform.

What does this PR do?

This PR adds runtime platform-aware filtering for Hermes skills and tools.

Hermes can now detect the current runtime platform as one of:

  • windows
  • macos
  • linux
  • wsl

Skills and tools can declare compatible platforms. Hermes filters them before exposing them to the model, so platform-specific instructions and tool schemas are only included when they match the current runtime environment.

This solves a real prompt-quality issue on native Windows. Before this change, Linux-specific skills could still be included in the system prompt, which made Hermes sometimes suggest Linux commands first or work around simple file operations by generating Python code instead of using the more appropriate Windows-aware path.

This approach keeps compatibility simple and explicit:

  • missing or empty platforms means universal compatibility;
  • linux means native Linux only;
  • wsl is treated as a first-class platform;
  • skills/tools that support both native Linux and WSL should declare platforms: [linux, wsl].

The implementation also includes the runtime platform in prompt/tool cache keys to avoid cross-platform cache leakage.

Related Issue

N/A — no linked issue.

Type of Change

  • 🐛 Bug fix (non-breaking change that fixes an issue)
  • ✨ New feature (non-breaking change that adds functionality)
  • 🔒 Security fix
  • 📝 Documentation update
  • ✅ Tests (adding or improving test coverage)
  • ♻️ Refactor (no behavior change)
  • 🎯 New skill (bundled or hub)

Changes Made

  • Added canonical runtime platform detection in hermes_constants.py.
  • Added first-class wsl support.
  • Updated skill platform matching in agent/skill_utils.py to use exact normalized platform keys.
  • Updated skill prompt construction in agent/prompt_builder.py so runtime platform participates in filtering/cache behavior.
  • Added optional platforms metadata to registered tools in tools/registry.py.
  • Updated model_tools.py to filter tool definitions by runtime platform before exposing them to the model.
  • Added plugin passthrough support for optional tool platforms metadata in hermes_cli/plugins.py.
  • Added debug logging for platform-excluded skills and tools.
  • Updated bundled/optional skill frontmatter so Linux skills that also support WSL explicitly include wsl.
  • Updated tests for platform detection, skill filtering, tool registry behavior, tool definition cache isolation, and plugin compatibility.
  • Updated docs for the windows, macos, linux, and wsl platform keys.

How to Test

  1. Run lint checks for the modified files:

    uv run ruff check hermes_constants.py agent/skill_utils.py agent/prompt_builder.py agent/skill_commands.py tools/skills_tool.py tools/registry.py model_tools.py hermes_cli/plugins.py tests/test_hermes_constants.py tests/tools/test_skills_tool.py tests/agent/test_prompt_builder.py tests/agent/test_skill_commands.py tests/tools/test_registry.py tests/test_get_tool_definitions_cache_isolation.py tests/hermes_cli/test_plugins.py ```
    
  2. Run the focused test suite:

    uv run pytest -o addopts='' tests/test_hermes_constants.py tests/tools/test_skills_tool.py tests/agent/test_prompt_builder.py tests/agent/test_skill_commands.py tests/tools/test_registry.py tests/test_get_tool_definitions_cache_isolation.py tests/hermes_cli/test_plugins.py
  3. Check whitespace issues:

    git diff --check
  4. Optional manual smoke test on native Windows 11:

    • Confirm detected runtime platform is windows.
    • Confirm Linux-only and WSL-only skills are excluded from the system prompt.
    • Confirm universal and Windows-compatible tools are still exposed.
    • Confirm excluded tools do not run availability checks.

Checklist

Code

Documentation & Housekeeping

Screenshots / Logs

Lint and focused tests passed:

388 passed, 1 skipped

git diff --check is clean.

Detect the runtime platform as windows, macos, linux, or wsl and filter
skills/tools before exposing them to the model. This prevents Linux-only
instructions from appearing in native Windows prompts, which previously
caused Hermes to suggest Linux commands or fall back to Python snippets for
basic file operations like writing or copying files.

WSL is treated as a distinct platform, missing platforms remain universal,
and prompt/tool caches now include the runtime platform.
@alt-glitch alt-glitch added type/feature New feature or request P2 Medium — degraded but workaround exists comp/agent Core agent runtime: loop, agent_init, prompt builder, context-compression, responses endpoint comp/tools Tool registry, model_tools, toolsets tool/skills Skills system (list, view, manage) labels May 14, 2026

@teknium1 teknium1 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the cross-platform work. The WSL distinction is a potentially useful direction, but this branch needs a re-scope and redesign against current code.

Problems

  • agent/skill_utils.py:163-185 on current main already filters platforms: [linux] against sys.platform; native Windows (win32) does not match Linux. The reported Linux-skill leak therefore is not demonstrated by the current implementation.
  • tools/registry.py:366-373 adds filtering, but this PR changes no concrete built-in tool registration to pass platforms=. The branch therefore leaves all built-in tools universal and does not deliver the claimed tool filtering.
  • hermes_constants.py:258-259 uses the host runtime, while agent/prompt_builder.py:1065-1070 distinguishes remote execution backends. Host-based filtering can hide Linux-capable tools/instructions when Hermes runs on Windows but executes in Docker or SSH.

Suggested changes

  • Define platform compatibility from the actual execution environment, then apply it consistently to skills and schemas.
  • Add concrete tool declarations and end-to-end coverage after that contract is settled.

Automated hermes-sweeper review.

Comment thread tools/registry.py
entry = entries_by_name.get(name)
if not entry:
continue
if entry.platforms and runtime_platform not in entry.platforms:

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This filter has no effect for built-in tools in this PR: no concrete tools/*.py registration is changed to pass platforms=, so every existing entry normalizes to universal. Add declarations for the tools that genuinely require a platform and exercise them through get_tool_definitions().

Comment thread hermes_constants.py
return "windows"
if sys.platform == "darwin":
return "macos"
if sys.platform.startswith("linux"):

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is the Hermes host platform, but remote terminal backends execute tool operations in a separate environment. A Windows host with a Linux Docker or SSH backend would incorrectly lose Linux-tagged tools/skills. Resolve compatibility from the execution backend, or explicitly limit this feature to local backends.

@teknium1 teknium1 added sweeper:risk-compatibility Sweeper risk: may break existing users, config, migrations, defaults, or upgrades sweeper:risk-caching Sweeper risk: may break/degrade prompt caching or cache-key stability (invariant) sweeper:risk-platform-windows Sweeper risk: may break or behave differently on native Windows sweeper:blast-broad Sweeper blast radius: broad — a core path most sessions hit labels Jul 13, 2026
@AlexFoxD

Copy link
Copy Markdown
Author

Thanks, this makes sense. The main issue is that the current implementation treats the host platform as the execution platform, which is incorrect for Docker and SSH backends.

I’ll re-scope the branch around an execution-environment platform contract, apply it consistently to skills and tool schemas, and add concrete platform declarations with end-to-end coverage. I’ll also verify whether the reported Windows skill leak is still reproducible on current main before keeping that claim in the PR description.

@AlexFoxD

Copy link
Copy Markdown
Author

I think it makes sense to close this PR. I’ll start again from the current main in a new branch and open a replacement PR with the necessary redesign and updated implementation

@GottZ GottZ left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This was generated by AI during triage.

Summary

Two PRs address the platform-compatibility complex. #21561 established native Windows support and broad platform metadata, while #25735 attempts to add WSL-aware skill/tool filtering but bases compatibility on the Hermes host rather than the actual execution backend and supplies no concrete built-in tool restrictions.

Related pull requests

  • #21561 [merged] related — (+7675/-625) — merged reference implementation: established native Windows support across installers, subprocess and process handling, UTF-8 I/O, gateway lifecycle, tools, tests, documentation, and initial skill platform metadata. It remains relevant because #25735 builds its filtering model on the platform metadata and execution-environment distinctions introduced by this merged baseline.
  • #25735 related — (+491/-248) — close and redesign: adds exact windows/macos/linux/wsl matching, platform-scoped caches, registry metadata, and widespread WSL declarations, but the diff derives compatibility from the host runtime, does not add platform declarations to concrete built-in tools, and does not demonstrate the claimed current-main Linux-skill leak. Despite the keep_open review on #25735, closure is warranted because the contributor review identifies an execution-backend contract flaw and low salvageability, and the author explicitly agreed to replace the branch from current main.

Suggested consolidation

Do not merge #25735; close it in favor of a replacement PR based on current main that derives compatibility from the actual execution environment, applies that contract consistently to skills and tool schemas, adds concrete built-in tool declarations with end-to-end coverage, and first reproduces the reported Windows skill leak. Keep merged #21561 as the reference implementation and baseline rather than treating it as a duplicate; this recommendation departs from the keep_open review on #25735 because the diff confirms the review's host-versus-backend mismatch and incomplete built-in tool filtering, and the author has accepted a restart.

Cross-PR triage: Reviewed 2 pull requests and 0 issues in this complex. Each diff was read against this issue; Assessment working set: 711 kB of PR diffs, 17 kB of issue/PR text, 7 kB of discussion (4 comments), 0 verify verdicts. verdicts reflect diff content, not PR titles. Part of an automated triage batch.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

comp/agent Core agent runtime: loop, agent_init, prompt builder, context-compression, responses endpoint comp/tools Tool registry, model_tools, toolsets P2 Medium — degraded but workaround exists sweeper:blast-broad Sweeper blast radius: broad — a core path most sessions hit sweeper:risk-caching Sweeper risk: may break/degrade prompt caching or cache-key stability (invariant) sweeper:risk-compatibility Sweeper risk: may break existing users, config, migrations, defaults, or upgrades sweeper:risk-platform-windows Sweeper risk: may break or behave differently on native Windows tool/skills Skills system (list, view, manage) type/feature New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants