Skip to content

fix: expose custom provider models in ACP picker - #70696

Closed
HrRobert wants to merge 3 commits into
NousResearch:mainfrom
HrRobert:fix/custom-provider-model-discovery
Closed

fix: expose custom provider models in ACP picker#70696
HrRobert wants to merge 3 commits into
NousResearch:mainfrom
HrRobert:fix/custom-provider-model-discovery

Conversation

@HrRobert

Copy link
Copy Markdown

Problem

When using custom providers (liteLLM, vLLM, etc.) with ACP clients like VSCode, only the default model appears in the model picker instead of all available models.

Root Cause

The ACP adapter calls curated_models_for_provider() which eventually calls provider_model_ids(). This function had no handler for custom providers, so it returned an empty list. Additionally, _get_custom_base_url() only checked model.base_url in config, not the custom_providers[] list where the actual base_url and api_key are stored.

Solution

  • Modified _get_custom_base_url() to fall back to custom_providers[0].base_url when model.base_url is empty
  • Modified the custom handler in provider_model_ids() to also check custom_providers[0].api_key when no API key is found in model config or environment
  • Added a fallback handler that matches custom provider names (e.g., 'litellm') against the custom_providers list and queries their /v1/models endpoint

Testing

Verified with a liteLLM instance serving 48 models. Before fix: ACP picker showed 1 model. After fix: ACP picker shows all 48 models.

Impact

  • Fixes ACP model picker for all custom OpenAI-compatible providers
  • Also improves /model slash command to show custom provider models
  • No breaking changes — existing behavior preserved when custom_providers is not configured

Custom providers (e.g., liteLLM, vLLM) now query /v1/models endpoint
to populate the model picker in ACP clients like VSCode and Zed.

- Enhanced _get_custom_base_url() to fall back to custom_providers config
- Enhanced API key resolution to check custom_providers when model config lacks it
- Added custom provider model discovery in provider_model_ids()

Fixes issue where ACP clients only showed the default model instead of
all available models from the custom provider.
@isak-ialogics

Copy link
Copy Markdown
Contributor

One concrete multi-provider regression remains: both new bare-custom fallbacks select custom_providers[0] for the URL/key. Main already has a regression documenting that “first saved provider” is not necessarily active (tests/hermes_cli/test_custom_provider_model_switch.py::test_bare_custom_current_provider_matches_env_base_url_before_first_fallback; e.g. Cerebras first, NeuralWatt active). ACP makes this path likely because SessionManager._make_agent() stores the resolved runtime provider (custom) on the agent, so _build_model_state() calls curated_models_for_provider("custom") and the named-provider loop cannot recover the active name. With model.base_url absent, this patch can therefore query/expose the first provider’s catalog (and credential) instead of the active second provider. Suggested next action: resolve the active custom entry from model.provider / the agent base URL (the existing canonical_custom_identity / find_custom_provider_identity helpers encode this rule), and add an ACP test with two custom providers where the active one is second.

@alt-glitch alt-glitch added type/bug Something isn't working comp/cli CLI entry point, hermes_cli/, setup wizard comp/acp Agent Communication Protocol adapter area/config Config system, migrations, profiles P4 Best-effort: we will get to it when we get to it (no commitment) needs-decision Awaiting maintainer decision before any implementation labels Jul 24, 2026
Robert added 2 commits July 24, 2026 18:28
When using custom providers (liteLLM, vLLM, etc.) via ACP clients
like VSCode, switching models would send the request to the wrong
endpoint (e.g., OpenRouter) instead of the custom provider.

Root causes:
1. ACP encode used canonicalized provider 'custom' instead of the
   original requested_provider name (e.g., 'litellm'), so the
   model switch round-trip lost the provider identity.
2. set_session_model passed provider='custom' to resolve_runtime_provider,
   which has no config entry for 'custom' and fell through to OpenRouter.
3. parse_model_input does not split on colon for custom provider names
   since they are not in _KNOWN_PROVIDER_NAMES.

Fixes:
- _build_model_state: use requested_provider for encoding model choices
- set_session_model: use requested_provider for resolution
- _resolve_model_selection: manually split provider:model for custom
  provider names that parse_model_input does not recognize
- _get_custom_base_url: fall back to custom_providers list
- provider_model_ids: query custom provider /v1/models endpoint
Without this, model switches in ACP sessions could not find the
correct custom_providers entry because agent.requested_provider
fell back to the canonicalized "custom" instead of the original
provider name (e.g., "litellm").
@HrRobert

Copy link
Copy Markdown
Author

Thanks for the review! You are right that both bare-custom fallbacks currently use custom_providers[0], which would pick the wrong provider in a multi-custom-provider setup (e.g., Cerebras first, NeuralWatt active).

Our current setup has only one custom provider (liteLLM), so [0] is safe for us. But I agree this should use canonical_custom_identity / find_custom_provider_identity to match the active entry properly. I will address that in a follow-up PR — the approach would be to resolve the active custom entry from either the agent base URL or model.provider config, rather than blindly indexing [0].

For this PR, the critical path is making ACP model switching work at all for single custom-provider users, which was completely broken before (models would route to OpenRouter instead). The multi-provider refinement is additive — it makes an already-working path correct for edge cases without changing the single-provider behavior.

@teknium1

Copy link
Copy Markdown
Contributor

Thanks @HrRobert — this was a real gap and your diagnosis of the root cause (ACP's _build_model_state leaning on curated_models_for_provider(), which had no custom-provider handler) was correct.

Closing as superseded by two PRs that just landed and rebuilt this path:

  • feat(acp): expose authenticated cross-provider model choices #70404 replaced ACP's enumeration source entirely: the selector now consumes the shared authenticated-provider inventory (load_picker_context / build_models_payload) — the same substrate as hermes model, the TUI, and the dashboard — which already probes configured custom endpoints. The curated_models_for_provider() call this PR patches no longer exists in the adapter.
  • feat(acp): list named custom providers (declared-models catalogs) in the ACP model selector #70082 adds named custom-provider (declared-models) catalogs to the ACP selector: each providers: / custom_providers: entry appears as its own group, with declared models first and live /models discovery layered on top when a credential resolves — which covers the liteLLM/vLLM case directly. Selection ids use the custom:<name>:<model> shape and round-trip through parse_model_input / resolve_runtime_provider, so switching to a custom-provider model re-resolves the correct entry — the same problem your requested_provider plumbing addressed.

If you still see a custom endpoint missing from the ACP picker on current main after these two, please open a fresh issue with your config shape — the custom_providers[0]-fallback corners in hermes_cli/models.py would be worth revisiting against the new substrate rather than the old path. Appreciate the work here.

@teknium1 teknium1 closed this Jul 24, 2026
@HrRobert

Copy link
Copy Markdown
Author

Just upgraded to clean upstream main — confirmed the new inventory system picks up our liteLLM custom provider with all 48 models in the ACP picker. No local patches needed anymore. Thanks for shipping this!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area/config Config system, migrations, profiles comp/acp Agent Communication Protocol adapter comp/cli CLI entry point, hermes_cli/, setup wizard needs-decision Awaiting maintainer decision before any implementation P4 Best-effort: we will get to it when we get to it (no commitment) type/bug Something isn't working

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants