Skip to content

fix(reasoning): add GLM and Claude family-before-version names to heuristic family list - #3413

Closed
Carry00 wants to merge 1 commit into
nesquena:masterfrom
Carry00:fix/reasoning-glm-claude-family-names
Closed

Carry00 wants to merge 1 commit into
nesquena:masterfrom
Carry00:fix/reasoning-glm-claude-family-names

Conversation

@Carry00

@Carry00 Carry00 commented Jun 2, 2026

Copy link
Copy Markdown
Contributor

What

PR #3327 added dot-separator normalization so custom-provider model IDs like deepseek.v3.2 correctly surface the reasoning-effort selector. However, two model families were left out of _candidate_supports_reasoning and remain broken after that fix.

GLM (ZAI / Zhipu AI) — zai.glm-5, zai.glm-4.7, glm-5, etc.
All modern GLM generations (4.5+) have reasoning: true in models.dev, but glm was never added to the heuristic family list. The dot-stripping from #3327 correctly surfaces glm-5 as a candidate — it just has nowhere to match.

Claude family-before-version naming — claude-opus-4-8, claude-haiku-4-5, anthropic.claude-opus-4-8
The existing check is startswith("claude-3", "claude-4"), which matches the old claude-3-5-sonnet-... format. The newer naming convention puts the family before the version (claude-opus-4-8), so it starts with claude-opus, not claude-4, and silently misses.

Changes

  • api/config.py — two additions to _candidate_supports_reasoning:
    • glm- prefix for the GLM family
    • claude- + {3, 4} token intersection for the new Claude naming convention
  • tests/test_custom_provider_bare_model_reasoning.py — 11 new parametrized cases (28 total, all pass)
  • CHANGELOG.md — user-visible entry

Verification

28 passed in 3.74s

Models now correctly detected (were all [] before):

Model ID Before After
zai.glm-5 [] full efforts
zai.glm-4.7 [] full efforts
anthropic.claude-opus-4-8 [] full efforts
anthropic.claude-haiku-4-5 [] full efforts
deepseek.v3.2 ✓ (from #3327) ✓

Non-reasoning models unchanged (meta.llama-3.1-70b, thinkinghub.* → []).


Note on the underlying design

This is a targeted fix but also a band-aid. The heuristic family list (_candidate_supports_reasoning) will need a new entry every time a provider uses an unfamiliar naming convention — GLM and Claude are the two that surfaced this week, but there will be more.

A more durable solution would be to allow per-model capability overrides in the custom provider config, so users (or aggregator presets) can declare reasoning: true without requiring a code change. Happy to prototype that if maintainers think it's worth pursuing — just want to flag the root cause before closing it with another patch.

… family list

PR #3327 added dot-separator normalization for custom-provider model IDs but
left two model families out of _candidate_supports_reasoning:

- GLM (ZAI / Zhipu AI): zai.glm-5, zai.glm-4.7, glm-5, etc. all have
  reasoning=true in models.dev but the family was never added to the heuristic.
- Claude family-before-version naming: claude-opus-4-8, claude-haiku-4-5,
  anthropic.claude-opus-4-8 use the pattern <family>-<version> rather than
  claude-3-... / claude-4-..., so the existing startswith("claude-3","claude-4")
  check misses them.

Adds:
- glm- prefix check for GLM models
- claude- + {3,4} token intersection for the new Claude naming convention
- 11 new parametrized test cases (28 total, all pass)
@nesquena-hermes

Copy link
Copy Markdown
Collaborator

Closing as superseded by #3379, shipped in v0.51.211 (Release GE). Thanks @Carry00 — your additions (GLM models + Claude family-before-version naming like claude-opus-4-8/claude-haiku-4-5) are both fully covered by #3379's broader family-based rewrite of _candidate_supports_reasoning. I verified your exact target cases (zai.glm-4.7, glm-5, anthropic.claude-opus-4-8, claude-haiku-4-5) all resolve correctly with the merged change. Appreciate the fix — it pointed at the same gap.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants