Repository navigation
fix(models): keep OpenRouter :batch variants out of chat routing - #13622
Conversation
ModelSync imported OpenRouter's Batch-API-only variants into the chat
catalogue. A chat completion against one is rejected upstream with
404 This model is only available through the Batch API.
Use the /api/beta/batches endpoint instead.
which cheaper-inference#13596 measured 91 times in 41 hours, third by volume, plus the
`model not found - locking mode` failover churn behind it.
OpenRouter's /models carries no endpoint metadata that separates a batch
variant from a chat one, so `classifyExplicitEndpoints` cannot decide it
and the rule belongs in `modelEndpointPolicy`, beside the OpenAI
image/video policy and for the same reason: the file exists so discovery,
import and catalog projection agree on one answer.
Matched as the exact `:batch` suffix, not "has a variant suffix" --
`:free`, `:nitro`, `:floor`, `:online`, `:extended` and `:thinking` are
routing hints on the same chat model, and excluding them would silently
shrink the routable catalogue. Applied unconditionally for this provider:
there is no "batch" endpoint name an upstream could declare next to a chat
one, and the already-stored rows carry the synthetic `["chat"]` default
that re-imported them in the first place.
Closes cheaper-inference#13596
|
Clean root cause and fix — confirmed |
|
Thanks @L4XB — merging via the release merge-train. Validated in local merge-train (.claude/worktrees/merge-train-20260918-111718-suite.log) on the devbox @ train tip 7bb373fba5e241964c0ffb17bd03a804700839bb, boarded with 55 sibling PRs: typecheck:core, file-size, complexity, cognitive-complexity, changelog-integrity green; 747/747 changed-area node:test cases + 476/476 vitest green (fast parity mode — the full suite ran today on the release tip via the base-red train and runs again on the 3b train). Merged --admin per merge-gates §7. |
ce56098
into
cheaper-inference:release/v3.8.51
…aper-inference#13622) * fix(models): keep OpenRouter :batch variants out of chat routing ModelSync imported OpenRouter's Batch-API-only variants into the chat catalogue. A chat completion against one is rejected upstream with 404 This model is only available through the Batch API. Use the /api/beta/batches endpoint instead. which cheaper-inference#13596 measured 91 times in 41 hours, third by volume, plus the `model not found - locking mode` failover churn behind it. OpenRouter's /models carries no endpoint metadata that separates a batch variant from a chat one, so `classifyExplicitEndpoints` cannot decide it and the rule belongs in `modelEndpointPolicy`, beside the OpenAI image/video policy and for the same reason: the file exists so discovery, import and catalog projection agree on one answer. Matched as the exact `:batch` suffix, not "has a variant suffix" -- `:free`, `:nitro`, `:floor`, `:online`, `:extended` and `:thinking` are routing hints on the same chat model, and excluding them would silently shrink the routable catalogue. Applied unconditionally for this provider: there is no "batch" endpoint name an upstream could declare next to a chat one, and the already-stored rows carry the synthetic `["chat"]` default that re-imported them in the first place. Closes cheaper-inference#13596 * docs(changelog): add fragment for the OpenRouter batch-variant fix
Closes #13596.
What was wrong
ModelSyncimported OpenRouter's Batch-API-only variants into the chat catalogue, so a chat request that landed on one was rejected upstream:91 of those in the reporter's 41-hour window (third error class by volume), plus the
[provider] Node <uuid> model not found (N) for google/gemini-*:batch - locking modefailover churn behind them. Deleting the 77*:batchkeys locally only holds until the next sync cycle.Why here
open-sse/services/modelEndpointPolicy.tsalready exists for exactly this: "Upstream/modelsresponses often omit endpoint/modality metadata… Keep the exceptional provider knowledge here so discovery, import, and catalog projection agree." OpenRouter's/modelscarries nothing that separates a batch variant from a chat one, soclassifyExplicitEndpointsreturnsnulland the model falls through to{ kind: "unknown", chatSelectable: true }— which is what imports it.Putting the rule here means the one filter (
filterChatSelectableModels) thatsync-modelsalready goes through — it sendsx-omniroute-model-surface: chat— and thatmanagedModelImportalso calls, both pick it up. No new filtering path.Two deliberate choices
Exact
:batch, not "has a variant suffix".:free,:nitro,:floor,:online,:extendedand:thinkingare routing hints on the same chat model. Excluding them would silently shrink the routable catalogue — a worse failure than the one being fixed, because it is invisible.Applied unconditionally for this provider, unlike the OpenAI branch below it, which lets an explicit multi-endpoint declaration win. There is no "batch" endpoint name an upstream could declare next to a chat one, and the rows already stored for these variants carry the synthetic
["chat"]default — letting that win would keep importing exactly what this excludes.Tests
Five, in
tests/unit/model-endpoint-policy.test.ts::batchvariants classify asnon-chat/ not selectable:free,:thinking,:nitro,:onlineand the bare base model stay selectablesupportedEndpoints: ["chat"]does not re-admit a:batchvariantcustom-providerand anullprovider are unaffected)filterChatSelectableModelsdrops the batch variant and keeps its base modelEach earns its place — every mutation is caught by a different one:
:batchclassification, the synthetic-default case, the filter case:variant instead of:batchif (true))Not in scope
The issue also asks for auto-quarantine of models the upstream rejects as "not supported" (the 34 GitHub Copilot
400s) and a routable-vs-catalog count in the dashboard. Both are policy decisions with their own state, and neither is needed to stop the:batchrejections — happy to follow up on either if you want them.