Skip to content

fix(web): judge model variants on the providers that serve them - #6053

Merged
chrarnoldus merged 3 commits into
mainfrom
cosmic-handle
Sep 10, 2026
Merged

fix(web): judge model variants on the providers that serve them#6053
chrarnoldus merged 3 commits into
mainfrom
cosmic-handle

Conversation

@chrarnoldus

Copy link
Copy Markdown
Contributor

Problem

The models-and-providers access screen collapses model variants such as nvidia/nemotron-3.5-lightning and nvidia/nemotron-3.5-lightning:free into one entry, and buildModelIdToProviderSlugsIndex does the same server-side: both variants map to the union of their providers.

The variants' provider sets need not overlap (live data today: paid → deepinfra, coreweave; free → nvidia). So an enterprise org that allows the model and only the paid variant's providers still saw …:free in the model list and as a kilo-auto/free candidate, even though every request for it would fail with no eligible route (and vice versa for the paid variant).

Approach

Use metadata we already store at sync time. models_by_provider.openrouter holds /models/{id}/endpoints per exact model id (including :free), and each endpoint tag is <provider-slug>[/<quantization>], i.e. the same slug namespace as the provider allow lists and provider.only. (Verified against the live API: across ~100 sampled models every tag prefix matched an all-providers slug, except sambanova-turbo, which is already a known inference provider id.)

  • getProviderSlugsForModel(modelId) now accepts variant-suffixed ids: it looks up the collapsed snapshot providers by normalized id, then narrows them to the providers that have an endpoint for the exact id. Ids without endpoint metadata (kilo-exclusive/injected models, routing hints like :nitro, stale metadata) or whose endpoints match no snapshot provider keep the collapsed set, so behaviour degrades to today's rather than hiding something that may work.
  • getEffectiveModelDecision and the legacy createAllowPredicateFromProviderAllowList pass the requested model id through to ProviderLookup unchanged instead of pre-normalizing it. Deny/allow-list matching is still normalized.
  • /api/openrouter/models-by-provider (per-caller catalog) evaluates the variant each provider entry actually serves (getSnapshotModelVariantId: <slug>:<variant> unless standard), so a provider that only offers the free variant is judged on that variant's routes rather than being dropped when the paid routes are the allowed ones.

Not covered (by design, non-watertight): the Providers & Models admin UI still shows one row per collapsed model; its "no enabled provider" warning is still computed on the union.

Testing

  • New models-by-provider-index.server.test.ts covering variant narrowing, fallbacks, and the loader.
  • New decision-level test in effective-model-access.server.test.ts (paid allowed, free denied with organization_provider).
  • Ran locally: tsgo --noEmit (web), oxlint on changed files, and jest for models-by-provider-index.server, effective-model-access.server, model-allow.server, supported-fim-models, legacy-model-restrictions-parity, organization-models, defaults/route, openrouter/models/route, auto-model/resolution, organization-settings-router, organization-modes-router, model-preferences-router. Full suite left to CI.

The providers-by-model snapshot collapses variants such as
nvidia/nemotron-3.5-lightning and nvidia/nemotron-3.5-lightning:free
onto one entry, so an organization that allows the model but only the
paid variant's providers still saw the free variant in the model list
and as a kilo-auto/free candidate, even though every request would fail.

Narrow the snapshot provider set per exact model id using the stored
OpenRouter /models/{id}/endpoints metadata, whose endpoint tags share
the provider slug namespace, and pass the unsuffixed-or-not model id
through to the provider lookup. Ids without endpoint metadata keep the
collapsed provider set. The per-caller models-by-provider catalog now
evaluates the variant each provider actually serves.
@chrarnoldus chrarnoldus self-assigned this Sep 10, 2026
@kilo-code-bot

kilo-code-bot Bot commented Sep 10, 2026

Copy link
Copy Markdown
Contributor

Code Review Summary

Status: No Issues Found | Recommendation: Merge

Executive Summary

Variant-aware provider narrowing is consistent across lookup, access decisions, and the per-caller catalog, with conservative fallbacks and coverage for the paid-vs-free split.

Files Reviewed (7 files)
  • apps/web/src/app/api/openrouter/models-by-provider/route.ts
  • apps/web/src/lib/ai-gateway/providers/openrouter/models-by-provider-index.server.ts
  • apps/web/src/lib/ai-gateway/providers/openrouter/models-by-provider-index.server.test.ts
  • apps/web/src/lib/model-allow.server.ts
  • apps/web/src/lib/organizations/effective-model-access.server.test.ts
  • apps/web/src/lib/organizations/group-policies/model-access/model-access.server.ts
  • apps/web/src/routers/organizations/organization-settings-router.test.ts

Reviewed by grok-4.6 · Input: 253.3K · Output: 17.1K · Cached: 807.6K

Review guidance: REVIEW.md from base branch main

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants