fix(web): judge model variants on the providers that serve them - #6053
Merged
Conversation
The providers-by-model snapshot collapses variants such as
nvidia/nemotron-3.5-lightning and nvidia/nemotron-3.5-lightning:free
onto one entry, so an organization that allows the model but only the
paid variant's providers still saw the free variant in the model list
and as a kilo-auto/free candidate, even though every request would fail.
Narrow the snapshot provider set per exact model id using the stored
OpenRouter /models/{id}/endpoints metadata, whose endpoint tags share
the provider slug namespace, and pass the unsuffixed-or-not model id
through to the provider lookup. Ids without endpoint metadata keep the
collapsed provider set. The per-caller models-by-provider catalog now
evaluates the variant each provider actually serves.
Contributor
Code Review SummaryStatus: No Issues Found | Recommendation: Merge Executive SummaryVariant-aware provider narrowing is consistent across lookup, access decisions, and the per-caller catalog, with conservative fallbacks and coverage for the paid-vs-free split. Files Reviewed (7 files)
Reviewed by grok-4.6 · Input: 253.3K · Output: 17.1K · Cached: 807.6K Review guidance: REVIEW.md from base branch |
johnnyeric
approved these changes
Sep 10, 2026
This was referenced Sep 11, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
The models-and-providers access screen collapses model variants such as
nvidia/nemotron-3.5-lightningandnvidia/nemotron-3.5-lightning:freeinto one entry, andbuildModelIdToProviderSlugsIndexdoes the same server-side: both variants map to the union of their providers.The variants' provider sets need not overlap (live data today: paid →
deepinfra,coreweave; free →nvidia). So an enterprise org that allows the model and only the paid variant's providers still saw…:freein the model list and as akilo-auto/freecandidate, even though every request for it would fail with no eligible route (and vice versa for the paid variant).Approach
Use metadata we already store at sync time.
models_by_provider.openrouterholds/models/{id}/endpointsper exact model id (including:free), and each endpointtagis<provider-slug>[/<quantization>], i.e. the same slug namespace as the provider allow lists andprovider.only. (Verified against the live API: across ~100 sampled models every tag prefix matched anall-providersslug, exceptsambanova-turbo, which is already a known inference provider id.)getProviderSlugsForModel(modelId)now accepts variant-suffixed ids: it looks up the collapsed snapshot providers by normalized id, then narrows them to the providers that have an endpoint for the exact id. Ids without endpoint metadata (kilo-exclusive/injected models, routing hints like:nitro, stale metadata) or whose endpoints match no snapshot provider keep the collapsed set, so behaviour degrades to today's rather than hiding something that may work.getEffectiveModelDecisionand the legacycreateAllowPredicateFromProviderAllowListpass the requested model id through toProviderLookupunchanged instead of pre-normalizing it. Deny/allow-list matching is still normalized./api/openrouter/models-by-provider(per-caller catalog) evaluates the variant each provider entry actually serves (getSnapshotModelVariantId:<slug>:<variant>unlessstandard), so a provider that only offers the free variant is judged on that variant's routes rather than being dropped when the paid routes are the allowed ones.Not covered (by design, non-watertight): the Providers & Models admin UI still shows one row per collapsed model; its "no enabled provider" warning is still computed on the union.
Testing
models-by-provider-index.server.test.tscovering variant narrowing, fallbacks, and the loader.effective-model-access.server.test.ts(paid allowed, free denied withorganization_provider).tsgo --noEmit(web), oxlint on changed files, and jest formodels-by-provider-index.server,effective-model-access.server,model-allow.server,supported-fim-models,legacy-model-restrictions-parity,organization-models,defaults/route,openrouter/models/route,auto-model/resolution,organization-settings-router,organization-modes-router,model-preferences-router. Full suite left to CI.