Skip to content

fix(routing): keep keyless custom-compatible connections in the auto/* pool - #11198

Merged
diegosouzapw merged 1 commit into
cheaper-inference:release/v3.8.50from
pacocartones:fix/11180-keyless-custom-provider-auto-pool
Aug 23, 2026
Merged

diegosouzapw merged 1 commit into
cheaper-inference:release/v3.8.50from
pacocartones:fix/11180-keyless-custom-provider-auto-pool

Conversation

@pacocartones

Copy link
Copy Markdown
Contributor

A custom OpenAI-compatible connection pointing at a keyless local backend is silently dropped from every auto/* candidate pool. It never appears as a routing candidate, so no strategy, weight or priority setting can select it, while direct <provider>/<model> calls keep working.

Root cause

open-sse/services/autoCombo/virtualFactory.ts:489 filters the pool through hasUsableConnectionCredential, which requires one of three things:

const hasApiKey = typeof conn.apiKey === "string" && conn.apiKey.trim().length > 0;
return hasApiKey || hasUsableOAuthToken(conn) || hasProviderSpecificSessionData(conn);

A keyless local endpoint has none of them: api_key is NULL, there is no OAuth token, and there is no web session data. The connection is removed before the pool is built.

This is one gate later than #5873. That fix made the registry-absent path fall back to defaultModel, and it still works; the connection's models do get enumerated. It is then dropped by the credential filter.

Fix

Treat the four generated compatible-provider ID shapes as credential-optional, reusing the existing isCompatibleProviderConnectionId helper rather than a new inline regex (the lesson of #8326, where an inline pattern matched only 1 of the 4 shapes).

Deliberately narrow. A first-party provider with an empty key really is unconfigured and stays filtered out, and AUTO_COMBO_NOAUTH_ALLOWLIST is untouched, so this does not widen the no-auth registry policy.

Red / green

Before, on a clean checkout at 62ab93d78:

✖ keyless custom openai-compatible connection enters the auto pool (#11180)
  AssertionError: a keyless custom-compatible connection must not be dropped by the credential gate
  actual: undefined, expected: true
✔ a keyless FIRST-PARTY provider connection stays out of the pool (#11180)
ℹ pass 1  ℹ fail 1

After:

✔ keyless custom openai-compatible connection enters the auto pool (#11180)
✔ a keyless FIRST-PARTY provider connection stays out of the pool (#11180)
✔ createVirtualAutoCombo includes custom openai-compatible providers via defaultModel (#5873)
✔ fresh install: the allowlisted no-auth providers are present in the auto-combo pool
✔ fresh install: non-allowlisted no-auth providers are EXCLUDED from the auto-combo pool
✔ fresh install: EVERY synthetic-noauth candidate belongs to an allowlisted provider
ℹ pass 6  ℹ fail 0

The second test is a negative control: it fails if the relaxation is ever widened to first-party providers. noauth-autocombo-allowlist, -exclude-7622, -hidden-7620 and -lockout-7623 were also run and stay green.

@diegosouzapw one note on CI: release/v3.8.50 is red at the base right now (stale ESLint suppressions, unit shards, Fast Quality Gates), so those reds should appear on other PRs too and are not from this change.

Closes #11180

…* pool

A custom openai-compatible-* / anthropic-compatible-* connection pointing at a keyless self-hosted backend (llama.cpp, Ollama, vLLM started without an API key) has no apiKey, no OAuth token and no provider-specific session data, so hasUsableConnectionCredential dropped it from validConnections before the auto/* candidate pool was built.

For those generated IDs 'no credential' is the normal configuration rather than an unconfigured connection. The relaxation is scoped to the four compatible-provider ID shapes via the existing isCompatibleProviderConnectionId helper, so an unconfigured first-party connection stays excluded and the no-auth registry allowlist is untouched.

Closes cheaper-inference#11180
@pacocartones
pacocartones force-pushed the fix/11180-keyless-custom-provider-auto-pool branch from 51a7c33 to aa1814c Compare August 23, 2026 02:57
@diegosouzapw
diegosouzapw merged commit a966c75 into cheaper-inference:release/v3.8.50 Aug 23, 2026
10 of 16 checks passed
muhamadgalihsaputra pushed a commit to niyatna/NiyatnaRoute that referenced this pull request Sep 27, 2026
…* pool (cheaper-inference#11198)

Validated on the combined batch board over tip 429b730: static gates clean (changelog, file-size 158 frozen, complexity 2626<=2774, cognitive 1183<=1223, dead-code 409<=416), typecheck:core clean, 70 focused tests green (PR suites 49/49 + auth/combo neighbors 21/21).

Keyless custom-compatible connections stay in auto/* pools — the credential filter now recognizes a registry-free keyless endpoint instead of dropping the connection before pool construction. Fixes cheaper-inference#11180. Thank you @pacocartones!
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[BUG] Keyless local providers (llama.cpp) silently excluded from auto/* candidate pool by credential gate

2 participants