Skip to content

fix(cli): probe /v1/models live when custom provider has no api_key - #40554

Closed
Morad37 wants to merge 1 commit into
NousResearch:mainfrom
Morad37:fix/probe-models-no-apikey-40542
Closed

fix(cli): probe /v1/models live when custom provider has no api_key#40554
Morad37 wants to merge 1 commit into
NousResearch:mainfrom
Morad37:fix/probe-models-no-apikey-40542

Conversation

@Morad37

@Morad37 Morad37 commented Jun 6, 2026

Copy link
Copy Markdown
Contributor

The should_probe condition in list_authenticated_providers() Section 4 used not grp[models] to decide whether to skip live discovery for endpoints without an api_key. But grp[models] collects models from both the explicit models: dict (user intentionally narrowed) AND the singular model: field (just the active selection).

For custom providers without api_key, the single model: value made the list non-empty, causing should_probe = False and live discovery skipped. The /model picker would show only 1 model instead of the full catalog.

Fix: Use len(grp[models]) <= 1 instead of not grp[models] so that a single entry from the model: field is not mistaken for an explicit narrowing. When a provider has no api_key and only the singular model: field, live /v1/models probing still occurs -- matching the CLI's hermes model behaviour.

Also updated the surrounding comments to accurately describe the three distinct cases: (1) explicit models: list (2+ entries) -- skip probe, (2) singular model: field (1 entry) -- probe, (3) no models -- probe.

Closes #40542

The should_probe condition in list_authenticated_providers() Section 4
used "not grp["models"]" to decide whether to skip live discovery for
endpoints without an api_key. But grp["models"] collects models from
both the explicit "models:" dict (user intentionally narrowed) AND
the singular "model:" field (just the active selection). For custom
providers without api_key, the single "model:" value made the list
non-empty, causing should_probe = False and live discovery to be
skipped. The /model picker would then only show 1 model instead of
the full catalog.

Fix: use "len(grp["models"]) <= 1" instead of "not grp["models"]" so
that a single entry from the "model:" field is not mistaken for an
explicit narrowing and live /v1/models probing still occurs.

Closes NousResearch#40542

@alpindiay alpindiay left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

PR Review: #40554 -- fix(cli): probe /v1/models live when custom provider has no api_key

Summary

This PR fixes model discovery for custom providers that use only the singular model: config field (1 entry). Previously, not grp["models"] only probed live /v1/models when the models list was empty -- but a single-entry list from the singular model: field was treated as "explicitly narrowed" and skipped live discovery. This change adjusts the condition to len(grp["models"]) <= 1, so single-model entries (which likely represent a bare endpoint, not an intentional narrow-down) also trigger live probing.

Analysis

Security: [PASS] No issues. No secrets, no injection vectors, no path traversal.

Bugs: [PASS] Fixes a real bug where local Ollama/llama.cpp servers with only a model: field would show a single model instead of the full catalog.

Logic: [PASS] The change is sound. The key insight -- that a single model: entry is more likely to be a user specifying a default model on a bare endpoint rather than intentionally narrowing -- is well-reasoned and matches CLI behavior.

Style: [PASS] Clean, minimal diff. Comments updated to explain the new behavior clearly.

Missing tests: [NOTE] No new tests added. While the change is small, a unit test covering the len(grp["models"]) <= 1 branch would be valuable to prevent regression.

Verdict: LGTM. The fix is correct, focused, and well-documented.

@tonydwb tonydwb left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review Summary

Verdict: Approved

Looks Good

  • Single line change: the probe threshold treats a one-entry model: list same as an empty list.
  • Fix targets the exact scenario the bug report describes (local Ollama/llama.cpp with a single model entry).
  • Comment block updated to match the new behavior.

Reviewed by Hermes Agent

@alt-glitch alt-glitch added type/bug Something isn't working P3 Low — cosmetic, nice to have comp/cli CLI entry point, hermes_cli/, setup wizard labels Jun 6, 2026
@kshitijk4poor

Copy link
Copy Markdown
Collaborator

Merged via #61928 using #55158's declaration-tracking implementation. Your PR was the first submitted fix for #40542 and correctly identified the singular-model probe gap; we credited that contribution in the salvage PR. The merged implementation avoids the one-item explicit-catalog regression in the len(models) <= 1 heuristic. Thank you!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

comp/cli CLI entry point, hermes_cli/, setup wizard P3 Low — cosmetic, nice to have type/bug Something isn't working

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Bug]: /model picker only shows 1 model for local custom providers without api_key (while hermes model CLI shows all)

5 participants