fix(cli): wire Anthropic model picker to live /v1/models discovery - #62986
fix(cli): wire Anthropic model picker to live /v1/models discovery#62986libreleee wants to merge 4 commits into
Conversation
_model_flow_anthropic() read the static _PROVIDER_MODELS["anthropic"]
list directly instead of calling provider_model_ids("anthropic"),
so new Claude releases never appeared in `hermes model` / `/model
--provider anthropic` without a Hermes code change.
provider_model_ids("anthropic") already implements everything
needed: live /v1/models fetch via _fetch_anthropic_models(),
curated-first merge so newly-routed aliases stay visible, and a
fallback to the same static _PROVIDER_MODELS["anthropic"] list
when the live fetch fails. This is a two-line change that wires
the picker flow to that existing path — behavior is identical to
today whenever the live fetch is unavailable.
Complements NousResearch#55915 and NousResearch#55853, which add claude-sonnet-5 to the
static fallback lists. NousResearch#56617 (merged) illustrates the failure
mode of manual list updates: it added anthropic/claude-sonnet-5
to the OpenRouter curated list but the native anthropic list was
missed, so the picker still omits it.
Fixes NousResearch#55846 (the recurrence path — this PR fixes the underlying
cause so future Claude releases don't need a manual list update).
teknium1
left a comment
teknium1
left a comment
There was a problem hiding this comment.
Thanks for isolating this to the existing catalog-resolution path. The current-main premise is confirmed: hermes_cli/model_setup_flows.py:2950 bypasses provider_model_ids("anthropic"), while hermes_cli/models.py:2352-2381 already implements the live fetch, curated-first merge, and static fallback.
Problems
- The changed integration has no direct regression coverage.
tests/hermes_cli/test_anthropic_picker_curated.py:19-57validatesprovider_model_ids()only, andtests/cli/test_cli_provider_resolution.py:438-471does not assert the list passed by_model_flow_anthropic().
Suggested changes
- Add a focused flow test that patches
provider_model_ids("anthropic")to return a sentinel live-only model and verifies_prompt_model_selection()receives it.
Automated hermes-sweeper review.
alt-glitch
added
type/bug
Related: this is the root-cause fix for the recurring "new Claude model missing from the picker" class — it points |
teknium1
added
sweeper:risk-compatibility
…y wiring Review feedback on NousResearch#62986 (teknium1/hermes-sweeper) noted that the provider_model_ids("anthropic") wiring in _model_flow_anthropic() had no direct regression coverage: test_anthropic_picker_curated.py only validates the helper itself, and the existing model_setup_flows tests never assert what list reaches _prompt_model_selection(). Add test_model_flow_anthropic_passes_live_discovery_list_to_picker, which patches provider_model_ids("anthropic") with a live-only sentinel absent from the static catalog and asserts _prompt_model_selection() receives exactly that list. Verified this test fails against the pre-fix code (reading _PROVIDER_MODELS directly) and passes against the fix.
|
Added |
config.yaml written by save_config() contains non-ASCII template comments (box-drawing characters), so a bare read_text() decodes with the platform locale and can fail on Windows (e.g. cp949). Pin encoding="utf-8" so the final config assertion is portable.
|
#55846 has been resolved by 07f39cf through the static catalog update. This PR addresses a separate recurrence path by wiring the Anthropic native picker to the existing live-discovery mechanism while preserving the static fallback. I’ve also added the requested flow-level regression test and the Windows UTF-8 portability fix. The CI workflow is currently awaiting approval. Could a maintainer approve the run when convenient? |
|
Tested this fix in a current downstream integration. The picker wiring is correct, but current To keep the Anthropic picker aligned with the other current model-switch paths, I recommend importing and calling Downstream verification after that substitution:
|
|
Can confirm the premise on today's main (0157180): the picker behind |
Switch _model_flow_anthropic() from provider_model_ids("anthropic") to
cached_provider_model_ids("anthropic") so the picker matches the other
model-switch paths (hermes_cli/model_switch.py) instead of issuing a
live catalog request on every invocation.
cached_provider_model_ids() wraps the same resolver, so authenticated
/v1/models discovery and the static fallback are preserved; it adds a
credential-fingerprinted TTL cache that `hermes model --refresh`
invalidates explicitly. This matters because `hermes fallback add`
reuses this flow, which would otherwise re-fetch the catalog each time.
Suggested by @s-a-s-k-i-a in review.
|
@s-a-s-k-i-a Thanks — adopted in e039c02. The picker now calls @AIalliAI Thanks for the independent verification, and you're right that the framing had gone stale: Not rebasing onto current main: the PR is still |
5 participants
What does this PR do?
Wires the Anthropic model picker (
hermes model/hermes fallback add) to the shared model resolver, so newly released Claude models appear without a Hermes release._model_flow_anthropic()inhermes_cli/model_setup_flows.pyreads the static_PROVIDER_MODELS["anthropic"]list directly, bypassing the resolver that every other model-switch path already uses.The fix is small because everything needed already exists in
cached_provider_model_ids("anthropic")→provider_model_ids("anthropic")(hermes_cli/models.py):/v1/modelsfetch via_fetch_anthropic_models()_PROVIDER_MODELS["anthropic"]list when the live fetch fails or no credentials are configuredhermes model --refreshinvalidates explicitlyWhen the live fetch is unavailable, behavior is identical to today — so there is no regression surface.
Why this keeps mattering
Manual static-list updates keep falling behind.
07f39cfaddedclaude-sonnet-5to the curated Anthropic list, which closed #55846 — but the live Anthropic catalog now servesclaude-opus-5, which the static list again omits. Every new Claude release repeats this cycle until the picker resolves the catalog rather than a hardcoded list.Related Issue
Related to #55846 (now closed by the static-list update in
07f39cf). This PR addresses the recurrence path rather than the individual model, so future releases don't need a manual list edit.Type of Change
Changes Made
hermes_cli/model_setup_flows.py— inside_model_flow_anthropic():_PROVIDER_MODELS→cached_provider_model_ids_PROVIDER_MODELS.get("anthropic", [])→cached_provider_model_ids("anthropic")tests/cli/test_cli_provider_resolution.py— new flow-level regression testtest_model_flow_anthropic_passes_live_discovery_list_to_pickerPer review feedback, the picker calls
cached_provider_model_ids()(notprovider_model_ids()directly) to matchhermes_cli/model_switch.pyand avoid a live catalog request each timehermes fallback addreuses this flow.How to Test
python -m pytest tests/cli/test_cli_provider_resolution.py -q— 24 passedtest_model_flow_anthropic_passes_live_discovery_list_to_pickeris the sole failure, confirming it pins the wiringhermes model→ Anthropic lists the live catalog (13 models locally, includingclaude-opus-5), curated entries firstChecklist
Code
tests/cli/test_cli_provider_resolution.py(24), plustests/hermes_cli/test_anthropic_picker_curated.py,test_anthropic_model_flow_stale_oauth.py,test_anthropic_provider_persistence.py,test_model_provider_persistence.pyDocumentation & Housekeeping
cli-config.yaml.example— N/ACONTRIBUTING.md/AGENTS.md— N/Aencoding="utf-8"on its config read for Windows locales