Repository navigation
fix(capabilities): resolve synced vision metadata for Mistral -latest aliases (#4073) - #4161
Conversation
… aliases (#4073) models.dev catalogs Pixtral 12B under the short id `pixtral-12b` (attachment: true, image modality), while requests use the Mistral API alias `pixtral-12b-latest`. getSyncedCapabilityForResolved tried only exact / raw / static-spec-canonical ids, all of which miss the short form, so vision fell through to the #4071 model-id heuristic and `attachment` stayed null. Add a last-resort fallback that retries the synced lookup with a trailing `-latest` stripped, so synced metadata wins for these aliases. Models whose `-latest` id is stored verbatim (e.g. pixtral-large-latest) keep resolving directly. TDD: tests/unit/model-capabilities-mistral-vision-sync-4073.test.ts (RED 2/4 -> GREEN 4/4) seeds a synced fixture and asserts the verdict comes via the synced path (attachment === true/false), not the heuristic. The models.dev sync remains manual-only (no scheduled refresh) — documented as a separate follow-up.
|
You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard. |
There was a problem hiding this comment.
Code Review
This pull request resolves issue #4073 by ensuring that models.dev synced vision metadata can be correctly resolved for Mistral -latest aliases (e.g., mapping pixtral-12b-latest to pixtral-12b). It introduces a stripLatestAlias helper and updates getSyncedCapabilityForResolved to fallback to stripped aliases when exact matches are not found. Comprehensive unit tests have been added to verify this behavior. The feedback suggests optimizing the candidate loop in getSyncedCapabilityForResolved to prevent redundant lookups and duplicate iterations.
Important
The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.
| for (const candidate of [model, rawModel]) { | ||
| const base = stripLatestAlias(candidate); | ||
| if (base && base !== model && base !== rawModel) { | ||
| const byAlias = getSyncedCapability(provider, base); | ||
| if (byAlias) return byAlias; | ||
| } | ||
| } |
There was a problem hiding this comment.
To avoid redundant database lookups and duplicate iterations, we can optimize the candidate loop:
- Skip iterating over
nullvalues. - Avoid duplicate lookups when
model === rawModel. - Avoid looking up
baseif it is equal tocanonical(which was already looked up and missed on lines 222-225).
const candidates = [];
if (model) candidates.push(model);
if (rawModel && rawModel !== model) candidates.push(rawModel);
for (const candidate of candidates) {
const base = stripLatestAlias(candidate);
if (base && base !== model && base !== rawModel && base !== canonical) {
const byAlias = getSyncedCapability(provider, base);
if (byAlias) return byAlias;
}
}… aliases (cheaper-inference#4073) (cheaper-inference#4161) models.dev catalogs Pixtral 12B under the short id `pixtral-12b` (attachment: true, image modality), while requests use the Mistral API alias `pixtral-12b-latest`. getSyncedCapabilityForResolved tried only exact / raw / static-spec-canonical ids, all of which miss the short form, so vision fell through to the cheaper-inference#4071 model-id heuristic and `attachment` stayed null. Add a last-resort fallback that retries the synced lookup with a trailing `-latest` stripped, so synced metadata wins for these aliases. Models whose `-latest` id is stored verbatim (e.g. pixtral-large-latest) keep resolving directly. TDD: tests/unit/model-capabilities-mistral-vision-sync-4073.test.ts (RED 2/4 -> GREEN 4/4) seeds a synced fixture and asserts the verdict comes via the synced path (attachment === true/false), not the heuristic. The models.dev sync remains manual-only (no scheduled refresh) — documented as a separate follow-up.
Closes #4073
Root cause (confirmed against the live models.dev API)
getResolvedModelCapabilities("mistral/pixtral-12b-latest").supportsVisionresolvednull— vision came only from the #4071 model-id heuristic, withattachmentstillnull— even though Pixtral is multimodal.models.dev does expose the data; it is not an upstream gap. It catalogs Pixtral 12B under the short id
pixtral-12b:Requests use the Mistral API alias
pixtral-12b-latest.getSyncedCapabilityForResolvedtried the exact id, the raw id and the static-spec canonical id — all of which miss the short form — so it fell through to the heuristic (steps 1–4 ofresolveVisionCapabilityall missed). Models whose-latestid is stored verbatim (pixtral-large-latest,mistral-medium-latest) already resolved correctly; only the short-formedpixtral-12bwas broken.Fix
getSyncedCapabilityForResolvedgains a last-resort fallback: after exact / raw / canonical all miss, retry the synced lookup with a trailing-lateststripped (pixtral-12b-latest→pixtral-12b). Synced metadata (attachment/ image modalities) then wins over the heuristic. Because it only fires after the exact lookups fail, verbatim-latestids keep resolving directly (no behavior change for them).Test (TDD, Hard Rule #18)
tests/unit/model-capabilities-mistral-vision-sync-4073.test.tsseeds a synced fixture and asserts the verdict comes via the synced path — the discriminator isattachment(synced sets ittrue/false; the heuristic leaves itnull):mistral/pixtral-12b-latest→attachment === true,supportsVision === true(RED before:null)mistral/ministral-8b-latest→attachment === false,supportsVision === false(syncedfalsewins; RED before:null)pixtral-large-latest/ barepixtral-12bstill resolve directly-latestid never fabricates a match (staysnull)RED 2/4 → GREEN 4/4. Related suites: 89/89 pass (
model-capabilities-registry,models-catalog-route,combo-vision-aware-routing,modelsDevSync,vision-detection-consistency#4072). typecheck:core, eslint, check:file-size all clean.Out of scope (documented follow-up)
The models.dev sync is currently manual-only (Settings → models.dev) with no scheduled refresh, so a fresh instance still relies on the #4071 heuristic until that sync runs. A periodic-refresh cadence is left as a separate follow-up rather than added here.
Follow-up to #4071; sibling of #4072.