Skip to content

fix(picker): keep tokenizer Router models in the OpenRouter list - #126030

Open
djsanchezsuarez wants to merge 2 commits into
NousResearch:mainfrom
djsanchezsuarez:fix/openrouter-router-models
Open

djsanchezsuarez wants to merge 2 commits into
NousResearch:mainfrom
djsanchezsuarez:fix/openrouter-router-models

Conversation

@djsanchezsuarez

Copy link
Copy Markdown

Symptom

Models whose OpenRouter architecture.tokenizer is "Router" (e.g. typesafe/jev-router) never appear in the model picker, although https://openrouter.ai/api/v1/models lists them and their endpoints work.

Exact line

hermes_cli/models.py::_openrouter_model_supports_tools (the Kilo #9068 port):

return "tools" in params if isinstance(params, list) else True

A router SKU hands each request to a tool-capable backend instead of calling tools itself, so OpenRouter reports "supported_parameters": [] for it. An explicit empty list is the one shape this helper treats as "no tools" — intentional, an image-only model advertising no tools must stay hidden — so router models are dropped from the curated ∩ live intersection and stay invisible in every picker surface (hermes model, /model, the desktop Models page).

Change

Exempt only architecture.tokenizer == "Router". Every other tokenizer keeps the existing rule.

  • new contract test: router tokenizer + empty list → kept
  • new contract test: non-router tokenizer + empty list → still dropped
  • the pre-existing test_empty_supported_parameters_list_drops_model still passes

Second commit (droppable)

The curated manifest is still the outer gate: the picker intersects the manifest with the live catalog, so a model missing from the manifest can never surface no matter what the filter does. The second commit adds typesafe/jev-router to models_catalog_static.py and website/static/api/model-catalog.json. Drop it if you would rather not curate that SKU — the code fix stands alone.

Verification

pytest tests/hermes_cli/test_models.py -q -k "ToolSupport or OpenRouter"
# 7 passed

pytest tests/hermes_cli/test_models_catalog_late_plugin_provider.py tests/hermes_cli/test_list_picker_providers.py \
       tests/hermes_cli/test_provider_groups.py tests/hermes_cli/test_api_key_providers.py \
       tests/hermes_cli/test_plugin_provider_picker_admission.py -q
# 71 passed, 1 failed: TestDeepInfraProviderProfile::test_profile_registered_with_alias_and_aux
# (KeyError 'DEEPINFRA_API_KEY') — reproduced identically at 68fa7e9f84 with these commits absent,
# so it is pre-existing and unrelated.

End-to-end on a live install: with a catalog that lists the model, the picker payload's openrouter row contains it (build_model_options_payload(load_picker_context())); without the change it does not.

_openrouter_model_supports_tools dropped every model whose supported_parameters
was an empty list. Router models (architecture.tokenizer == "Router", e.g.
typesafe/jev-router) hand each request to a tool-capable backend instead of
calling tools themselves, so they legitimately advertise no tools and were
silently invisible in the model picker even though the live catalog lists them.

Only the router tokenizer is exempted; a non-router model without "tools" is
still dropped.
Adds the router SKU to the in-repo fallback list and the shipped manifest so it
survives a manifest that omits it; without a catalog entry the picker intersects
the curated list with the live /v1/models response and the model never appears.
@alt-glitch alt-glitch added type/bug Something isn't working P3 Low — cosmetic, nice to have comp/cli CLI entry point, hermes_cli/, setup wizard provider/openrouter OpenRouter aggregator labels Sep 28, 2026
@teknium1

Copy link
Copy Markdown
Collaborator

Cross-link from the weekly Kilo Code PR scout: Kilo landed the general form of this fix as Kilo-Org/kilocode#14592 — a missing or empty supported_parameters is treated as tool-capable ([] carries no capability information, exactly like an absent field), rather than a tokenizer-scoped exemption.

Live evidence gathered today against https://openrouter.ai/api/v1/models: 460 models, 4 with supported_parameters: [] — typesafe/jev-router, openrouter/fusion, openrouter/pareto-code, openrouter/bodybuilder — and 0 with the field missing. An N=1 chat/completions probe with a ping function tool against pareto-code, jev-router and fusion returned a real tool_calls entry from each (routed to a tool-capable backend). So on today's catalog the two rules select the same set; the general rule additionally covers a router whose architecture.tokenizer is not literally "Router" (e.g. a future first-party router), and it also affects hermes_cli/models_reasoning_caps.py::parse_openrouter_reasoning_capabilities, which currently reads [] as {"supports_reasoning": False} instead of "unknown → static prefix table".

Also related: #82767 (Aug) implements the same empty-list-permissive rule and replaces test_empty_supported_parameters_list_drops_model; issue #46064 tracks the symptom. Leaving the design choice (tokenizer-scoped vs. general) to review here rather than opening a competing PR.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

comp/cli CLI entry point, hermes_cli/, setup wizard P3 Low — cosmetic, nice to have provider/openrouter OpenRouter aggregator type/bug Something isn't working

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants