fix(models): drop curated OpenRouter ids that are retired or no longer tool-capable - #77203
fix(models): drop curated OpenRouter ids that are retired or no longer tool-capable#77203jbbottoms wants to merge 2 commits into
Conversation
…r tool-capable Five of the 39 entries in OPENROUTER_MODELS no longer work, checked against OpenRouter's live /api/v1/models: openrouter/elephant-alpha retired, no successor poolside/laguna-m.1:free retired; family moved to laguna-s-2.1 tencent/hy3:free free tier withdrawn inclusionai/ring-2.6-1t:free free tier withdrawn openrouter/pareto-code served, but supported_parameters is [] The first four fail the moment a user picks them. The fifth is the worse one: it is still served, so it selects and runs fine, and only fails when the agent tries to call a tool. In an agent framework that is a silent trap rather than an error. Nothing caught this because every existing model test asserts against the curated list itself, which makes the list self-consistent and says nothing about whether OpenRouter still serves what is in it. The two ":free" ids have paid equivalents still live (tencent/hy3, inclusionai/ring-2.6-1t). They are removed rather than re-pointed: moving a user's free selection onto a paid id is a curation decision for maintainers, not something a drift fix should do quietly. tencent/hy3 is already curated separately and is unaffected. Adds tests/hermes_cli/test_openrouter_catalog_drift_live.py so this cannot recur silently. Opt-in and live, in the same shape as the other *_live.py tests here (HERMES_LIVE_TESTS=1 plus a provider key), because it asserts against a third-party catalogue that changes without us and must never redden CI on someone else's deploy. Verified: - skips clean with no opt-in: 2 skipped - passes live after this change: 2 passed - non-vacuous: against the unfixed list it fails and names all five - scripts/build_model_catalog.py regenerated: openrouter 39 -> 34 models - pytest tests/hermes_cli/test_model_catalog.py test_model_search.py test_ai_gateway_models.py -> 30 passed - ruff check -> passed
CI slice failed because the TokenHub catalog pin still asserted the free tier id after OPENROUTER_MODELS dropped it. Keep paid tencent/hy3; assert :free is absent.
SummarySeven PRs address or materially reference this catalog complex: #3756/#3803 add the original Gemini 3.1 entries, #34581/#73479 update later curated generations, #77159 expands the Gemini subset, #77203 removes retired or non-tool-capable entries and adds a live drift check, and #77254 provides an opt-in full tool-capable catalog for BYOK users. Related pull requests
Duplicates#3756 and #3803 contain the same OpenRouter Gemini 3.1 additions; #3756 was closed after being incorporated into the more complete, merged #3803. The remaining PRs represent distinct catalog revisions, drift repair, or opt-in expansion rather than equivalent diffs. Suggested consolidationKeep #77203 open with a salvage path focused on the five removals and the opt-in live drift test, and keep #77254 open with its distinct salvage path as the recorded best fix for #76732's BYOK discovery problem. Leave #77159 closed rather than reopening it over the contributor's curation objection and the author's acceptance of that objection; #3756 is already superseded by merged #3803, while #34581 and #73479 remain merged historical baselines rather than closure candidates. Cross-PR triage: Reviewed 7 pull requests and 2 issues in this complex. Each diff was read against this issue; Assessment working set: 28 kB of PR diffs, 14 kB of issue/PR text, 5 kB of discussion (8 comments), 12 verify verdicts. verdicts reflect diff content, not PR titles. Part of an automated triage batch. |
Summary
Five of the 39 entries in
OPENROUTER_MODELSno longer work. Checked against OpenRouter's live/api/v1/models(337 models):openrouter/elephant-alphapoolside/laguna-m.1:freelaguna-s-2.1tencent/hy3:freeinclusionai/ring-2.6-1t:freeopenrouter/pareto-codesupported_parametersis[]The first four fail the moment a user picks them.
The fifth is the worse one.
openrouter/pareto-codeis still served, so it selects and runs fine, and only fails when the agent tries to call a tool. In an agent framework that is a silent trap rather than an error.Nothing caught this because every existing model test asserts against the curated list itself. That makes the list self-consistent, and says nothing about whether OpenRouter still serves what is in it.
The two
:freeremovalstencent/hy3andinclusionai/ring-2.6-1tstill exist as paid ids. I removed the:freeentries rather than re-pointing them, because moving a user's free selection onto a paid id is a curation decision for you, not something a drift fix should do quietly.tencent/hy3is already curated separately and is untouched.Same reasoning for
poolside/laguna-m.1:free:laguna-s-2.1andlaguna-xs-2.1are live, but adding them is a new curation call rather than a repair.Preventing recurrence
Adds
tests/hermes_cli/test_openrouter_catalog_drift_live.py, opt-in and live in the same shape as the other*_live.pytests here:Live and opt-in on purpose: it asserts against a third-party catalogue that changes without us, so it must never be able to redden CI on someone else's deploy. It is a maintenance check run deliberately.
It checks the two failure modes separately, because they are not the same problem — one fails on selection, the other on first tool call.
Verification
scripts/build_model_catalog.pyregenerated — openrouter 39 → 34 modelspytest tests/hermes_cli/test_model_catalog.py test_model_search.py test_ai_gateway_models.py→ 30 passedruff check→ passed