Skip to content

feat(model): honor discover_models in terminal hermes model named-custom flow (closes #18726) - #40093

Merged
kshitijk4poor merged 1 commit into
NousResearch:mainfrom
kshitijk4poor:feat/named-custom-discover-models-18726
Jun 5, 2026
Merged

feat(model): honor discover_models in terminal hermes model named-custom flow (closes #18726)#40093
kshitijk4poor merged 1 commit into
NousResearch:mainfrom
kshitijk4poor:feat/named-custom-discover-models-18726

Conversation

@kshitijk4poor

Copy link
Copy Markdown
Collaborator

Summary

Fixes #18726 — the terminal hermes model wizard (_model_flow_named_custom) always live-probed a custom provider's /models endpoint and ignored the configured models: list. For plans whose endpoint exposes a large catalog this floods the picker with models the user can't use (the issue's example: Baidu Qianfan Coding Plan returns 100+ models — ERNIE, embeddings, image models — for a 2–3 model plan).

Root cause

Two gaps in the terminal flow (the slash-command /model picker in model_switch.py §3/§4 already handles this):

  1. _named_custom_provider_map() built the provider_info dict but dropped both models and discover_models, so the flow never received them.
  2. _model_flow_named_custom() called fetch_api_models() unconditionally.

Fix (Option B — explicit opt-out, consistent with §3/§4 and #25810/#40080)

  • Wire models + discover_models through _named_custom_provider_map into the flow.
  • Honor discover_models: false: use the configured models: list verbatim and skip the live probe. String values ("false"/"no"/"0") normalise to False, matching the slash picker.
  • Default stays True — no behaviour change for existing configs. A models: list alone does not implicitly whitelist (that would silently break the documented "configure models: for context-length overrides while still live-fetching" use case).
  • If discovery is on but the probe returns empty, fall back to the configured models: list instead of forcing manual entry.

Config (now works)

providers:
  baidu:
    base_url: https://qianfan.baidubce.com/v2/coding
    key_env: BAIDU_CODING_API_KEY
    discover_models: false
    models: [kimi-k2.5, glm-5]

hermes model → Baidu → shows exactly 2 models, no probe.

Why not the implicit-whitelist default

The issue's literal ask was "no flag needed". I went with the explicit discover_models: false opt-out instead because: (1) it's one consistent mental model across every picker path (discover_models: false = "trust my list" everywhere — §3, §4, and now the terminal flow); (2) flipping the default would silently break configs that list models: purely for context-length overrides while relying on live discovery. The Baidu user adds one line and gets exactly the requested result.

Tests

tests/hermes_cli/test_custom_provider_model_switch.py — new TestCustomProviderDiscoverModels (5 tests):

  • discover_models: false + configured models → no probe, list used
  • selection from the configured list persists
  • default (flag unset) still probes even with a models: list (Option B guard)
  • probe-empty falls back to the configured list
  • string "false" normalises to disable

All pass. Verified end-to-end with a live call that raises if fetch_api_models is touched under discover_models: false — probe is skipped, "Using configured models" shown, exactly the 2 configured models listed.

Note: 3 unrelated tests in test_model_switch_custom_providers.py (groups_same_endpoint, distinct_endpoints_stay_separate, total_models_reflects_grouped_count) fail identically on bare origin/main — they don't mock fetch_api_models and probe a live local Ollama. Not touched by this PR.

…tom flow

The terminal `hermes model` wizard (_model_flow_named_custom) always
live-probed a custom provider's /models endpoint, ignoring the configured
`models:` list. For plans whose endpoint exposes a large catalog (e.g. Baidu
Qianfan Coding Plan returns 100+ models for a 2-3 model plan) the picker
flooded with models the user can't use.

This wires `discover_models` (and the `models:` list) through
_named_custom_provider_map into the flow and honors `discover_models: false`
the same way the slash-command picker (model_switch.py sections 3 & 4) does:
- Default stays True — live probe, no behaviour change.
- discover_models: false → use the configured `models:` list verbatim,
  skip the probe (string 'false'/'no'/'0' normalised to False).
- If the probe is on but returns empty, fall back to the configured list
  instead of forcing manual entry.

Closes NousResearch#18726
@kshitijk4poor
kshitijk4poor enabled auto-merge June 5, 2026 20:05
@kshitijk4poor
kshitijk4poor merged commit e6f7e21 into NousResearch:main Jun 5, 2026
22 checks passed
alt-glitch pushed a commit that referenced this pull request Jun 14, 2026
…er-models-18726

feat(model): honor discover_models in terminal hermes model named-custom flow (closes #18726)
T02200059 pushed a commit to T02200059/hermes-agent that referenced this pull request Jun 18, 2026
…ustom-discover-models-18726

feat(model): honor discover_models in terminal hermes model named-custom flow (closes NousResearch#18726)
waefrebeorn pushed a commit to waefrebeorn/slermes that referenced this pull request Jul 2, 2026
…ustom-discover-models-18726

feat(model): honor discover_models in terminal hermes model named-custom flow (closes NousResearch#18726)
santhreal pushed a commit to santhreal/hermes-agent that referenced this pull request Jul 13, 2026
…ustom-discover-models-18726

feat(model): honor discover_models in terminal hermes model named-custom flow (closes NousResearch#18726)
Gravezzz pushed a commit to Gravezzz/hermes-agent that referenced this pull request Jul 21, 2026
…ustom-discover-models-18726

feat(model): honor discover_models in terminal hermes model named-custom flow (closes NousResearch#18726)
@kshitijk4poor
kshitijk4poor deleted the feat/named-custom-discover-models-18726 branch August 5, 2026 07:07
leewenjie pushed a commit to leewenjie/hermes-agent that referenced this pull request Aug 7, 2026
…ustom-discover-models-18726

feat(model): honor discover_models in terminal hermes model named-custom flow (closes NousResearch#18726)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

feat: use provider models config as whitelist in /model picker for custom providers

1 participant