Skip to content

fix(models): validate OpenRouter presets structurally on custom endpoints (#97907) - #97951

Open
Tranquil-Flow wants to merge 1 commit into
NousResearch:mainfrom
Tranquil-Flow:fix/97907-preset-custom-endpoint
Open

Tranquil-Flow wants to merge 1 commit into
NousResearch:mainfrom
Tranquil-Flow:fix/97907-preset-custom-endpoint

Conversation

@Tranquil-Flow

Copy link
Copy Markdown

What

OpenRouter dashboard presets (@preset/<slug>) are account-scoped server-side objects — they never appear in a /models listing. On OpenRouter-compatible custom providers, validate_requested_model()'s custom-endpoint branch probed the catalog, always missed, and soft-accepted with a false "not found in this custom endpoint's model listing" warning embedding the raw catalog URL. Telegram's link preview fetched that URL and rendered the large JSON response as a models.json document on every preset switch.

Note the premise: validate_requested_model() reclassifies openrouter + non-openrouter.ai base_url to custom (models.py:6508-6509), so the custom branch is exactly the path this issue reports.

Fix

Handle preset references structurally in the custom branch instead of catalog-probing them:

  • Bare @preset/<slug> — accepted without probing (OpenRouter validates the slug at request time), with a URL-free message. Nothing for a link preview to fetch.
  • <model>@preset/<slug> — the BASE model is validated against the live listing; the preset suffix is preserved verbatim on the persisted/corrected id (same rule as the existing :nitro routing-variant handling, which the fuzzy corrector would otherwise strip).
  • Malformed slug (empty / invalid charset) — rejected with the slug contract ([A-Za-z0-9._~-]+), not a misleading catalog warning.
  • Multiple @preset/ markers — not a defined form; keeps existing behavior.
  • Non-preset identifiers keep the existing custom-branch behavior untouched.

No gateway/Telegram adapter change: the document trigger is removed at the source (no URL in the message), and normal link previews remain enabled per the issue's Expected section.

Verification

  • tests/hermes_cli/test_openrouter_preset_custom_validation.py — 13 new tests covering all four layers (bare preset, combined form, malformed slug, non-preset unchanged).
  • Fail-without-fix proven: on clean upstream/main the suite runs 8 failed / 5 passed (the 5 passing pin unchanged non-preset behavior); with the fix 13/13 pass. Re-verified independently by the reviewer on upstream 0d02f0d1dc.
  • Nearby suites (test_model_validation.py, test_model_switch_custom_providers.py, test_custom_provider_model_switch.py, test_user_providers_model_switch.py, test_openai_codex_model_validation_fallback.py): 140 passed, 2 failed — both failures reproduce identically on clean upstream/main (pre-existing, unrelated).
  • Branch is exactly one commit ahead of upstream/main (git rev-list --left-right --count → 0 1).

Competitor analysis

Open PR #89129 "fix(models): support OpenRouter preset references" handles @preset/ only in the normalized == "openrouter" branch — correct for native OpenRouter, but the custom-provider scenario of this issue never satisfies that gate (it is reclassified to custom upstream of the branch). Scopes are complementary with no gated-code-path overlap; the slug-charset rule mirrors #89129's for cross-branch consistency.

Fixes #97907.

Auto-published by Moonsong via Path B automated pipeline.

…ints (NousResearch#97907)

OpenRouter dashboard presets (@preset/<slug>) are server-side objects that
never appear in /models listings. On OpenRouter-compatible custom providers
the custom-endpoint validation branch probed the catalog, missed, and
soft-accepted with a false "not found" warning embedding the raw catalog
URL — which Telegram's link preview fetched and rendered as a models.json
document on every preset switch.

Validate preset references structurally instead: a bare @preset/<slug> is
accepted without probing (OpenRouter validates the slug at request time)
with a URL-free message; <model>@preset/<slug> validates the BASE model
against the listing and preserves the suffix through auto-correction (same
rule as the :nitro routing variants); malformed slugs are rejected with the
slug contract. Non-preset identifiers keep the existing behavior.
@alt-glitch alt-glitch added type/bug Something isn't working P2 Medium — degraded but workaround exists comp/cli CLI entry point, hermes_cli/, setup wizard provider/openrouter OpenRouter aggregator platform/telegram Telegram bot adapter labels Aug 29, 2026
@Enough1122

Copy link
Copy Markdown

AI code review — automated review for reference; please use your judgment.

PR #97951 — fix(models): validate OpenRouter presets structurally on custom endpoints

Summary: OpenRouter @preset/<slug> identifiers are server-side objects never present in /models. Adds _split_openrouter_preset and _PRESET_SLUG_RE (letters/digits/._~-) and structural handling in validate_requested_model for custom providers: slug charset is validated, bare @preset/... is accepted without a probe (no URL in message to trigger Telegram link previews), combined <model>@preset/... validates the base against the listing and preserves suffix through auto-correction (mirrors :nitro handling). Multiple markers (a@preset/x@preset/y) fall back to ordinary catalog validation.

Non-blocking observations:

  • _split_openrouter_preset (hermes_cli/models.py:35) counts exactly one @preset/ marker; empty slug @preset/ still returns a suffix and is then rejected by the charset check — correct per contract.
  • Suggestion generation now uses _lookup (base) not original id, so close-match hints are for the base — appropriate.
  • Leading/trailing whitespace in requested is not stripped before marker count; upstream validate_requested_model trims elsewhere, but if a raw " @preset/foo" slipped through it would probe incorrectly. Low risk.

Verdict: Well-tested fix for #97907 (four layers: bare, combined, malformed, non-preset unchanged). No blocking issues.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

comp/cli CLI entry point, hermes_cli/, setup wizard P2 Medium — degraded but workaround exists platform/telegram Telegram bot adapter provider/openrouter OpenRouter aggregator type/bug Something isn't working

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Bug]: OpenRouter preset validation causes models.json preview in Telegram

3 participants