Skip to content

fix: skip canonical providers when user has config-defined models - #21983

Open
rocktear wants to merge 1 commit into
NousResearch:mainfrom
rocktear:rocktear-patch-1
Open

fix: skip canonical providers when user has config-defined models#21983
rocktear wants to merge 1 commit into
NousResearch:mainfrom
rocktear:rocktear-patch-1

Conversation

@rocktear

@rocktear rocktear commented May 8, 2026

Copy link
Copy Markdown

fix: skip canonical providers when user has config-defined models

What does this PR do?

Problem

Plugin-registered providers (e.g. free-nvidia, free-router) auto-enter CANONICAL_PROVIDERS via the auto-extension at line 826–841 of models.py. When Section 2b of list_authenticated_providers() processes them:

  1. It looks for curated models in _PROVIDER_MODELS — finds none (plugin providers have no curated entry)
  2. Sets models = [] and adds slug to seen_slugs
  3. Section 3 (user-config from providers: in config.yaml) skips because slug is already seen
  4. Result: the provider appears in /model with 0 models

Fix

Before emitting a canonical row in Section 2b, check if the slug is defined in user_providers (the providers: section of config.yaml). If so, skip and let Section 3 handle it — the user's explicit base_url / api_key / models config always wins.

Related Issue

Fixes a regression introduced by the CANONICAL_PROVIDERS auto-extension feature.

Type of Change

  • 🐛 Bug fix (non-breaking change that fixes an issue)

Changes Made

  • hermes_cli/model_switch.py: In list_authenticated_providers(), Section 2b — added a guard before emitting the results entry: if the slug exists in user_providers, skip and let Section 3 provide the models list from config.

How to Test

  1. Create a model-provider plugin: ~/.hermes/plugins/model-providers/free-nvidia/ with a ProviderProfile
  2. Add free-nvidia: under providers: in config.yaml with 3 models + discover_models: false
  3. Before fix: /model shows free-nvidia with 0 models
  4. After fix: /model shows free-nvidia with 3 models

Checklist

Code

  • I've read the Contributing Guide
  • My commit messages follow Conventional Commits (fix(scope):, feat(scope):, etc.)
  • I searched for existing PRs to make sure this isn't a duplicate
  • My PR contains only changes related to this fix/feature (no unrelated commits)
  • I've run pytest tests/ -q and all tests pass
  • I've added tests for my changes (required for bug fixes, strongly encouraged for features)
  • I've tested on my platform: macOS

Documentation & Housekeeping

  • I've updated relevant documentation (README, docs/, docstrings) — N/A
  • I've updated cli-config.yaml.example if I added/changed config keys — N/A
  • I've updated CONTRIBUTING.md or AGENTS.md if I changed architecture or workflows — N/A
  • I've considered cross-platform impact (Windows, macOS) per the compatibility guide — N/A
  • I've updated tool descriptions/schemas if I changed tool behavior — N/A

fix: skip canonical providers when user has config-defined models
@rocktear

rocktear commented May 8, 2026

Copy link
Copy Markdown
Author

Just wanted to add a bit of context on why this fix matters beyond my specific setup:

The core tension here is between auto-discovery (CANONICAL_PROVIDERS auto-extending from plugins) and user control (explicit providers: config). Right now, the auto-extension wins unconditionally — even when the user has explicitly defined which models they want to see.

This affects a few scenarios:

  • Anyone building a third-party plugin — your provider shows up in /model with 0 models until you file a PR to add entries to _PROVIDER_MODELS
  • Enterprise deployments — teams running internal inference endpoints as plugins can't control their model list from config without this fix
  • Users who just want a clean /model picker — discover_models: false is great for trimming 300+ models down to the 3 you actually use, but it only works if Section 3 gets a chance to run

Without this fix, the user's explicit providers: config is silently overridden — which feels like a regression from the pre-plugin days when custom_providers at least respected what you wrote.

Happy to adjust anything if the maintainers have suggestions!

@rocktear

rocktear commented May 9, 2026

Copy link
Copy Markdown
Author

One more thought as a future direction — right now the plugin system does the heavy lifting of provider registration (name, base_url, env_vars), but model discovery still depends on either _PROVIDER_MODELS or the user's providers: config. The plugin's own fallback_models are never consulted during picker construction.

Ideally, a plugin should be fully self-contained:

ProviderProfile(
name="free-nvidia",
 fallback_models=("deepseek-ai/deepseek-v4-pro", "minimaxai/minimax-m2.7", "z-ai/glm4.7"),
 base_url="https://integrate.api.nvidia.com/v1",
)

Drop it in, restart, and /model shows 3 models — no config.yaml changes needed.

This would mean: if Section 2b finds a canonical provider with no _PROVIDER_MODELS entry, it falls back to profile.fallback_models from the registered plugin. Then discover_models: false could also live in the plugin itself rather than requiring config duplication.

Totally understand if this is out of scope for this PR — just wanted to flag it as a natural next step for the plugin system. Happy to open a separate feature request if that's preferred!

@alt-glitch alt-glitch added type/bug Something isn't working P2 Medium — degraded but workaround exists comp/cli CLI entry point, hermes_cli/, setup wizard labels May 11, 2026
@teknium1

Copy link
Copy Markdown
Contributor

Thanks for isolating the Section 2b/Section 3 precedence collision. The underlying config-loss path remains on current main: hermes_cli/model_switch.py:1946-2007 emits a remaining canonical row and records its slug, while Section 3 skips the matching providers: entry at hermes_cli/model_switch.py:2017-2024.

Problems

  • This needs a Section 2b-specific regression test. Current coverage added by 2b5d4ae916a805829e0c789a6dffe05f48e08a07 tests a models.dev-mapped DeepSeek row, not a canonical-only plugin row.
  • On current main, cached_provider_model_ids(_cp.slug) runs at hermes_cli/model_switch.py:1991 before the proposed skip point. If the goal includes respecting discover_models: false, apply the precedence decision before that lookup during salvage.

Suggested changes

  • Add a test for a canonical-only provider plus matching providers: config, including configured model declarations and discover_models: false.
  • Reapply the guard in current Section 2b, with the final behavior aligned deliberately against the Section 1 configured-model merge introduced in 2b5d4ae916a805829e0c789a6dffe05f48e08a07.

Automated hermes-sweeper review.

@teknium1 teknium1 added sweeper:risk-compatibility Sweeper risk: may break existing users, config, migrations, defaults, or upgrades sweeper:blast-moderate Sweeper blast radius: moderate — a subsystem or single platform labels Jul 13, 2026
pierrenode added a commit to pierrenode/hermes-agent that referenced this pull request Aug 11, 2026
… picker rows

list_authenticated_providers() merges a providers.<slug>.models config
block into a built-in (section 1, PROVIDER_TO_MODELS_DEV) row's discovered
catalog, so a model declared there shows up in the /model picker. Section 2
(HERMES_OVERLAYS — nous, openai-codex, copilot, opencode-go, etc.) builds
its model_ids the same way but never applies this merge: a model declared
under providers.nous.models (or providers.opencode-go.models, etc.) is
already typeable via /model <name> (_configured_provider_matches() scans
user_providers generically, by design) but never appears in the picker
list — inconsistent with the identical config surface working correctly
for built-in providers.

Fix: apply the same merge (configured models first, deduplicated against
the discovered list) right before model_ids is capped to max_models, using
hermes_slug as the config key — the same slug the row itself reports and
the same namespace _configured_provider_matches() already scans generically
(providers.nous, providers.openai-codex are documented config keys, see
hermes_cli/auth.py's own comments).

Scoped to section 2 only. Section 2b (CANONICAL_PROVIDERS) has the same
"providers.<slug>.models is invisible to the picker" symptom for a
different subset of providers, but two other open PRs (NousResearch#21983, NousResearch#60656)
are already actively reworking that exact code region with a different
approach (skip-and-defer to section 3 vs. merge) — not touching it here
to avoid overlapping scope on the same lines.

Adds a mapped-slug regression case: HERMES_OVERLAYS key "github-copilot"
(a models.dev ID) that PROVIDER_TO_MODELS_DEV maps back to the Hermes/
config slug "copilot", with providers.copilot.models — proving the
providers.<hermes_slug>.models lookup keys off the RESOLVED slug, not the
raw overlay/pid key, and exercising the
hermes_slug in {"openai-codex", "copilot", "copilot-acp"} special
live-discovery branch specifically (the existing overlay tests only
exercised the generic fallback branch via a synthetic 1:1 slug).
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

comp/cli CLI entry point, hermes_cli/, setup wizard P2 Medium — degraded but workaround exists sweeper:blast-moderate Sweeper blast radius: moderate — a subsystem or single platform sweeper:risk-compatibility Sweeper risk: may break existing users, config, migrations, defaults, or upgrades type/bug Something isn't working

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants