Skip to content

fix(models): list claude-opus-5 in the curated Anthropic catalog - #91678

Closed
justinjohnson25600 wants to merge 1 commit into
NousResearch:mainfrom
justinjohnson25600:fix/anthropic-opus-5-curated
Closed

justinjohnson25600 wants to merge 1 commit into
NousResearch:mainfrom
justinjohnson25600:fix/anthropic-opus-5-curated

Conversation

@justinjohnson25600

Copy link
Copy Markdown
Contributor

Summary

  • claude-opus-5 was missing from the curated _PROVIDER_MODELS["anthropic"] list in hermes_cli/models.py — it is present in OPENROUTER_MODELS, the nous catalog, opencode-zen, and reasoning_timeouts.py, but absent from the one list native-Anthropic pickers render.
  • Anthropic's /v1/models endpoint 401s subscription OAuth tokens (Claude Pro/Max), so for subscription users the picker always falls back to the curated list — hiding the current flagship entirely. (The model itself works fine when addressed directly.)
  • Adds official-docs pricing entries for claude-opus-5, claude-opus-5-fast (2x), and the claude-opus-5-20260723 dated snapshot — same $5/$25 line as 4.6-4.8, per the OpenRouter model metadata.
  • Adds a regression test pinning the curated-list fallback path for OAuth subscriptions (extends tests/hermes_cli/test_anthropic_picker_curated.py).

Root cause chain

  1. Claude Max OAuth setup succeeds (hermes auth add anthropic --type oauth) and inference works — sessions run on claude-opus-5.
  2. The model picker calls provider_model_ids("anthropic"), which tries the live /v1/models fetch first.
  3. The platform-API endpoint rejects subscription tokens with HTTP 401 → _fetch_anthropic_models() returns None.
  4. Fallback returns the curated list verbatim — which never added Opus 5.

Test plan

  • pytest tests/hermes_cli/test_anthropic_picker_curated.py — 4 passed (3 existing + 1 new)
  • pytest tests/agent/test_usage_pricing.py tests/agent/test_anthropic_thinking_disable.py tests/agent/test_anthropic_billing_guidance.py — 78 passed
  • Broader sweep (adapter, request-client-reuse, aux client, error classifier, reasoning floor, model_normalize): 541 passed, 9 pre-existing env failures (identical on clean base with changes stashed — scratch-venv dependency/mock quirks, not from this change)

🤖 This PR was created with Hermes Agent

Anthropic's /v1/models endpoint 401s subscription OAuth tokens (Claude
Pro/Max), so the model picker always falls back to the curated
_PROVIDER_MODELS["anthropic"] list for subscription users. That list
skipped claude-opus-5 — present in OPENROUTER_MODELS, the nous catalog,
opencode-zen, and reasoning_timeouts, but absent from the one list
subscription pickers actually see. The model itself is fully supported
(the anthropic adapter treats it as modern-contract by default; tests
already exercise thinking-disable against it).

Also add official-docs pricing for claude-opus-5, -fast (2x), and the
20260723 dated snapshot — same /5 line as 4.6-4.8, per the
OpenRouter model metadata — and a regression test pinning the
curated-list fallback for OAuth subscriptions.

Fixes the symptom: Claude Max OAuth users don't see Opus 5 in /model.
@alt-glitch alt-glitch added type/bug Something isn't working comp/agent Core agent runtime: loop, agent_init, prompt builder, context-compression, responses endpoint comp/cli CLI entry point, hermes_cli/, setup wizard provider/anthropic Anthropic native Messages API P3 Low — cosmetic, nice to have labels Aug 21, 2026
@Enough1122

Copy link
Copy Markdown
Contributor

AI code review — automated review for reference; please use your judgment.

Straightforward catalog addition, and the OAuth-fallback rationale in the new test is a nice touch. Minor items:

  • agent/usage_pricing.py:216 — issue — the claude-opus-5-fast entry declares source="official_docs_snapshot" but its source_url is openrouter.ai, unlike the two base entries that cite platform.claude.com — why it matters — provenance fields exist so future audits can tell verified-official from aggregator-derived numbers; a mislabeled source silently erodes that — suggestion — either point source_url at the official pricing page (if it documents the fast lane) or set source="openrouter" to match reality.

  • agent/usage_pricing.py:222 — issue — the base model gets a dated-snapshot alias (claude-opus-5-20260723) but the fast variant does not (claude-opus-5-fast-20260723 absent) — why it matters — if the Anthropic API echoes dated IDs for the fast lane the way it does for base Opus, those responses will miss the exact-match pricing entry and fall back to whatever estimate path exists — suggestion — confirm how cost lookup normalizes dated suffixes and add the dated fast alias (or a documented normalization) so both lanes behave identically.

  • tests/hermes_cli/test_anthropic_picker_curated.py:36 — issue (coverage) — the new test only exercises the picker side; nothing asserts the three new model IDs actually resolve in usage_pricing's table — why it matters — these two catalogs have drifted apart before (that's the failure mode the curated-list docstring itself describes); a missing pricing entry surfaces later as wrong cost reporting — suggestion — add a small parametrized test asserting each curated Anthropic ID (at minimum the three added here) has a PricingEntry with positive input/output costs.

No blocking issues found — all three items are consistency/hygiene rather than functional regressions.

— reviewer-b (automated review)

@AlexEfimov

Copy link
Copy Markdown

Independent confirmation of the root cause on a real Claude Max OAuth install (macOS, Hermes v0.20.5, 6 profiles).

Symptom reproduces exactly as described. 4 of my 6 profiles run claude-opus-5 and inference works fine — but the model is absent from the curated list, so the picker can only show it if the live fetch succeeds. It doesn't: with subscription OAuth (ANTHROPIC_API_KEY unset, hermes auth add anthropic --type oauth), _fetch_anthropic_models() returns None and provider_model_ids("anthropic") falls back to _PROVIDER_MODELS["anthropic"] verbatim.

Reading hermes_cli/models.py at _fetch_anthropic_models() confirms step 3 of your chain is structural, not incidental: the only HTTPError that gets a retry is the 400 "long context beta is not yet available" case — every other status, 401 included, is re-raised into the outer handler and becomes None. So for subscription tokens the curated list isn't a fallback, it's the only path.

One extra data point that may be useful for reviewers evaluating impact: the fallback result is written to provider_models_cache.json, so the cache ends up byte-identical to the curated list. I initially mistook that cached copy for a live API response and concluded the model resolved on its own — it doesn't. Anyone verifying this by inspecting the cache rather than the curated list will reach the same wrong conclusion, which may be part of why this went unnoticed.

I'd been carrying the one-line curated addition as a local patch against my checkout since before finding this PR; it's a strict subset of what you have here (this PR additionally covers pricing entries and the OAuth-subscription regression test). Happy to see it fixed upstream instead — commenting rather than opening a competing PR, per CONTRIBUTING's duplicate guidance.

Verified against origin/main at f14059f (2026-08-24): claude-opus-5 is still present in OPENROUTER_MODELS, the nous catalog and reasoning_timeouts.py, but still missing from _PROVIDER_MODELS["anthropic"].

@AlexEfimov

Copy link
Copy Markdown

Correction to my comment above — one claim was too broad, and the nuance turns out to be diagnostically useful.

I wrote that for subscription tokens the curated list is the only path. That holds for the default configuration, but not when model.base_url is set explicitly. The two cases split cleanly on this install:

profile model.base_url cached catalog identical to curated?
hermes-ops https://api.anthropic.com 10 models, opus-first ordering no — live API response
knowledge-curator https://api.anthropic.com 10 models, opus-first ordering no — live API response
chatterbox '' (unset) 12 models yes — byte-identical to curated
hermes-admin (absent) 12 models yes — byte-identical to curated

With an explicit base_url, provider_model_ids() takes the custom-endpoint branch (models.py ~4058: api_mode = "anthropic_messages" if _base_url_looks_like_anthropic_messages(base_url), then fetch_api_models(...) and an early return live), which returns the live catalog without the curated merge. Those profiles get a real /v1/models answer — 10 IDs sorted opus-first, clearly not our list — and claude-opus-5 is among them.

So the same OAuth credentials do reach /v1/models successfully. What differs is the code path, not the token.

Two things this doesn't change:

  1. The bug is still real for the default config. Profiles without an explicit base_url fall back to curated and lose the alias — that's the majority case and what most users hit.
  2. The cache trap I described still applies, and this table is a clean way to detect it: if provider_models_cache.json is byte-identical to the curated list, you're looking at the fallback, not an API answer. Sorted-by-family ordering vs. curated ordering is the giveaway.

If it helps triage: the workaround for anyone blocked today is setting model.base_url: https://api.anthropic.com explicitly, which routes around the curated list entirely. That's a workaround, not a fix — the curated list should still carry the current flagship.

Apologies for the imprecision in my earlier comment; I verified the split only after posting it.

@kshitijk4poor

Copy link
Copy Markdown
Contributor

The picker-catalog half of this landed via #106636 (merged as e306374): claude-opus-5 (and claude-fable-5.1) are now in the native Anthropic curated list. The usage-pricing half (Opus 5 / fast / snapshot pricing entries in agent/usage_pricing.py) is NOT yet on main — that part is real and still useful. Closing this PR as fixed for the catalog half; @justinjohnson25600, a slimmed PR carrying only the usage_pricing.py entries would be welcome (or we'll pick it up).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

comp/agent Core agent runtime: loop, agent_init, prompt builder, context-compression, responses endpoint comp/cli CLI entry point, hermes_cli/, setup wizard P3 Low — cosmetic, nice to have provider/anthropic Anthropic native Messages API type/bug Something isn't working

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants