feat(model_switch): honor discover_models in custom_providers section 4 (salvage #29810) - #40080
Merged
kshitijk4poor merged 2 commits intoJun 5, 2026
Conversation
Section 3 (user `providers:`) already honors `discover_models: false` to
skip live /models discovery and keep the explicit `models:` list. Section 4
(`custom_providers:` list) did not — `should_probe` ignored the field, so any
grouped custom provider with an api_key always had its configured subset
replaced by the full live /models catalog.
This adds the same `discover_models` support to section 4:
- Default True — no behaviour change for existing configs.
- `discover_models: false` keeps the explicit `models:` list even when an
api_key is present.
- String values ("false"/"no"/"0") are normalised to False, matching
section 3.
- If any entry in a grouped endpoint opts out, the whole group opts out.
Use case: endpoints that expose a full aggregator catalog via /models but
only serve a configured subset.
Salvaged from NousResearch#29810 — rebased onto current main. The PR's other change
(`key_env` resolution in section 4) landed independently in commit aa283d1
(custom provider picker credential isolation), so only the discover_models
portion is carried here.
Co-authored-by: ohMyJason <42903577+ohMyJason@users.noreply.github.com>
alt-glitch
pushed a commit
that referenced
this pull request
Jun 14, 2026
…section4-29810 feat(model_switch): honor discover_models in custom_providers section 4 (salvage #29810)
T02200059
pushed a commit
to T02200059/hermes-agent
that referenced
this pull request
Jun 18, 2026
…over-models-section4-29810 feat(model_switch): honor discover_models in custom_providers section 4 (salvage NousResearch#29810)
waefrebeorn
pushed a commit
to waefrebeorn/slermes
that referenced
this pull request
Jul 2, 2026
…over-models-section4-29810 feat(model_switch): honor discover_models in custom_providers section 4 (salvage NousResearch#29810)
santhreal
pushed a commit
to santhreal/hermes-agent
that referenced
this pull request
Jul 13, 2026
…over-models-section4-29810 feat(model_switch): honor discover_models in custom_providers section 4 (salvage NousResearch#29810)
Gravezzz
pushed a commit
to Gravezzz/hermes-agent
that referenced
this pull request
Jul 21, 2026
…over-models-section4-29810 feat(model_switch): honor discover_models in custom_providers section 4 (salvage NousResearch#29810)
leewenjie
pushed a commit
to leewenjie/hermes-agent
that referenced
this pull request
Aug 7, 2026
…over-models-section4-29810 feat(model_switch): honor discover_models in custom_providers section 4 (salvage NousResearch#29810)
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Salvages the still-relevant part of #29810 (which is 1,568 commits behind main and no longer applies). That PR bundled two changes:
key_envresolution in section 4 — already landed independently inaa283d1e4(custom-provider picker credential isolation), which replaced the old 2-tuple group key with(api_url, credential_identity, api_mode)and added thekey_envfallback. Not carried here (redundant).discover_modelsin section 4 — still missing on main. This PR.What this does
Section 3 (user
providers:) already honorsdiscover_models: falseto skip live/modelsdiscovery and keep the explicitmodels:list. Section 4 (custom_providers:list) did not —should_probeignored the field, so any grouped custom provider with anapi_keyalways had its configured subset replaced by the full live/modelscatalog.This adds the same
discover_modelssupport to section 4:True— no behaviour change for existing configs.discover_models: falsekeeps the explicitmodels:list even when anapi_keyis present."false"/"no"/"0") are normalised toFalse, matching section 3.Use case
Endpoints that expose a full aggregator catalog via
/modelsbut only serve a configured subset:Tests
Added two regression tests in
tests/hermes_cli/test_model_switch_custom_providers.py:discover_models: false+api_keypreserves the explicit subset and never callsfetch_api_models."false"is normalised to disable discovery.Both pass. Three pre-existing failures in that file (
test_..._groups_same_endpoint,test_..._distinct_endpoints_stay_separate,test_..._total_models_reflects_grouped_count) fail identically on bareorigin/main— they probe a live local Ollama server because they don't mockfetch_api_models; unrelated to this change.Credit to @ohMyJason (original author of #29810).
Closes #29810