Skip to content

fix(model-switch): resolve bare model names via custom_providers catalog - #13764

Closed
rzbdz wants to merge 1 commit into
NousResearch:mainfrom
rzbdz:fix/custom-provider-model-switch-v2
Closed

fix(model-switch): resolve bare model names via custom_providers catalog#13764
rzbdz wants to merge 1 commit into
NousResearch:mainfrom
rzbdz:fix/custom-provider-model-switch-v2

Conversation

@rzbdz

@rzbdz rzbdz commented Apr 22, 2026

Copy link
Copy Markdown

Problem

When a user types /model claude-sonnet-4.6 while on a custom provider (e.g. smt-claude), the switch fails with:

✗ Note: `claude-sonnet-4.6` was not found in this custom endpoint's model listing (https://openrouter.ai/api/v1/models).
  Similar models: `anthropic/claude-sonnet-4.6`, ...

Three root causes:

  1. cli.py: _handle_model_switch only loaded custom_providers in the no-args picker path. When a model name was given, switch_model was called with custom_providers=None.

  2. model_switch.py: No step searched the user's custom_providers models catalog before falling through to a live API probe, which could hit the wrong URL.

  3. model_switch.py / models.py: custom:smt-claude slugs weren't recognized as custom in the is_custom guard or in validate_requested_model.

Fix

  • New _find_model_in_custom_providers() helper scans every custom_providers entry's models dict/list. Matching is case-insensitive and normalises dots↔hyphens (claude-sonnet-4.6 matches claude-sonnet-4-6).
  • Inserted as step c2 in PATH B of switch_model() — before any live API probe.
  • cli.py loads custom_provs unconditionally before calling switch_model().
  • is_custom guard extended to cover custom:* slugs.
  • validate_requested_model normalises custom:*"custom".

Tests

5 new tests in tests/hermes_cli/test_model_switch_custom_providers.py. All 2440 existing tests pass.

When a user types /model <name> while on a custom provider, the pipeline
previously fell through to a live API probe against the endpoint's /models
listing. If the endpoint returned model IDs in a different format (or the
probe hit OpenRouter's URL due to stale state), validation failed and the
switch was rejected — leaving the user stuck on the default model.

Three fixes:

1. model_switch.py: Add _find_model_in_custom_providers() helper that
   scans every custom_providers entry's models dict/list for a matching
   model name (case-insensitive, dots-vs-hyphens normalised). Insert it
   as step c2 in PATH B so bare names like 'claude-sonnet-4.6' resolve
   to the correct custom:smt-claude provider before hitting live probing.

2. model_switch.py: Extend the is_custom guard (step e) to cover
   custom:* slugs, preventing detect_provider_for_model() from
   incorrectly routing custom provider models to OpenRouter/Anthropic.

3. cli.py: Load custom_provs before calling switch_model() in
   _handle_model_switch(), not just in the picker path. Previously the
   switch call always received custom_providers=None, so step c2 never
   had data to search.

4. models.py: Treat custom:* slugs identically to 'custom' in
   validate_requested_model() so named custom providers probe their own
   endpoint rather than falling through to the generic error path.
@alt-glitch alt-glitch added type/bug Something isn't working P2 Medium — degraded but workaround exists comp/cli CLI entry point, hermes_cli/, setup wizard labels Apr 22, 2026
@alt-glitch

Copy link
Copy Markdown
Collaborator

Note: #13760 was a prior closed attempt at the same fix; this is the replacement PR.

1 similar comment
@alt-glitch

Copy link
Copy Markdown
Collaborator

Note: #13760 was a prior closed attempt at the same fix; this is the replacement PR.

@trevorgordon981

Copy link
Copy Markdown
Contributor

I have tested this solution locally. The model-switch logic now correctly resolves bare model names via the custom_providers catalog. This ensures that users can switch to custom providers using simple model names without needing fully qualified strings. The fix is stable and ready for merge.\n\nTested and confirmed. ✅

1 similar comment
@trevorgordon981

Copy link
Copy Markdown
Contributor

I have tested this solution locally. The model-switch logic now correctly resolves bare model names via the custom_providers catalog. This ensures that users can switch to custom providers using simple model names without needing fully qualified strings. The fix is stable and ready for merge.\n\nTested and confirmed. ✅

@zons-zhaozhy

Copy link
Copy Markdown

This fix covers custom_providers path, but providers: dict has same problem.

Reproduce:

# config.yaml
model:
  default: sensenova-6.7-flash-lite
  provider: sensenova
providers:
  sensenova:
    base_url: https://token.sensenova.cn/v1
    api_key: sk-xxx

Run /model sensenova-6.7-flash-lite under sensenova provider, model name got sensenova/ prefix added, provider silently switched to OpenRouter.

Root cause is same — detect_provider_for_model() matched model in OpenRouter catalog, but is_custom guard not cover non-built-in provider defined in providers: dict.

I did a general check using get_provider(current_provider) is None, both paths covered: zons-zhaozhy@917afabc6

Tests: zons-zhaozhy@c64056e30

@teknium1

Copy link
Copy Markdown
Contributor

Closing as superseded by #28908.

Triage notes (medium confidence):
Merged PR #28908 'fix(model): match bare custom provider by active base URL' (plus #17891, #18185) added custom_providers resolution before live API probe in hermes_cli/model_switch.py (see custom_providers and isinstance(custom_providers, list) block ~line 948).

Thanks for the contribution — the underlying problem this PR addresses has been resolved by the linked PR on current main. If you believe this was closed in error, please comment and we'll reopen.

(Bulk-closed during a CLI PR triage sweep.)

@teknium1 teknium1 closed this May 24, 2026
@alt-glitch alt-glitch added area/config Config system, migrations, profiles sweeper:risk-compatibility Sweeper risk: may break existing users, config, migrations, defaults, or upgrades labels Aug 8, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area/config Config system, migrations, profiles comp/cli CLI entry point, hermes_cli/, setup wizard P2 Medium — degraded but workaround exists sweeper:risk-compatibility Sweeper risk: may break existing users, config, migrations, defaults, or upgrades type/bug Something isn't working

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants