fix: expose custom provider models in ACP picker - #70696
Conversation
Custom providers (e.g., liteLLM, vLLM) now query /v1/models endpoint to populate the model picker in ACP clients like VSCode and Zed. - Enhanced _get_custom_base_url() to fall back to custom_providers config - Enhanced API key resolution to check custom_providers when model config lacks it - Added custom provider model discovery in provider_model_ids() Fixes issue where ACP clients only showed the default model instead of all available models from the custom provider.
|
One concrete multi-provider regression remains: both new bare- |
When using custom providers (liteLLM, vLLM, etc.) via ACP clients like VSCode, switching models would send the request to the wrong endpoint (e.g., OpenRouter) instead of the custom provider. Root causes: 1. ACP encode used canonicalized provider 'custom' instead of the original requested_provider name (e.g., 'litellm'), so the model switch round-trip lost the provider identity. 2. set_session_model passed provider='custom' to resolve_runtime_provider, which has no config entry for 'custom' and fell through to OpenRouter. 3. parse_model_input does not split on colon for custom provider names since they are not in _KNOWN_PROVIDER_NAMES. Fixes: - _build_model_state: use requested_provider for encoding model choices - set_session_model: use requested_provider for resolution - _resolve_model_selection: manually split provider:model for custom provider names that parse_model_input does not recognize - _get_custom_base_url: fall back to custom_providers list - provider_model_ids: query custom provider /v1/models endpoint
Without this, model switches in ACP sessions could not find the correct custom_providers entry because agent.requested_provider fell back to the canonicalized "custom" instead of the original provider name (e.g., "litellm").
|
Thanks for the review! You are right that both bare- Our current setup has only one custom provider (liteLLM), so For this PR, the critical path is making ACP model switching work at all for single custom-provider users, which was completely broken before (models would route to OpenRouter instead). The multi-provider refinement is additive — it makes an already-working path correct for edge cases without changing the single-provider behavior. |
|
Thanks @HrRobert — this was a real gap and your diagnosis of the root cause (ACP's Closing as superseded by two PRs that just landed and rebuilt this path:
If you still see a custom endpoint missing from the ACP picker on current |
|
Just upgraded to clean upstream main — confirmed the new inventory system picks up our liteLLM custom provider with all 48 models in the ACP picker. No local patches needed anymore. Thanks for shipping this! |
Problem
When using custom providers (liteLLM, vLLM, etc.) with ACP clients like VSCode, only the default model appears in the model picker instead of all available models.
Root Cause
The ACP adapter calls
curated_models_for_provider()which eventually callsprovider_model_ids(). This function had no handler for custom providers, so it returned an empty list. Additionally,_get_custom_base_url()only checkedmodel.base_urlin config, not thecustom_providers[]list where the actual base_url and api_key are stored.Solution
_get_custom_base_url()to fall back tocustom_providers[0].base_urlwhenmodel.base_urlis emptycustomhandler inprovider_model_ids()to also checkcustom_providers[0].api_keywhen no API key is found in model config or environmentcustom_providerslist and queries their/v1/modelsendpointTesting
Verified with a liteLLM instance serving 48 models. Before fix: ACP picker showed 1 model. After fix: ACP picker shows all 48 models.
Impact
/modelslash command to show custom provider models