Skip to content

fix(models): keep OpenRouter :batch variants out of chat routing - #13622

Merged
diegosouzapw merged 2 commits into
cheaper-inference:release/v3.8.51from
L4XB:fix/13596-openrouter-batch-variants-are-not-chat
Sep 18, 2026
Merged

diegosouzapw merged 2 commits into
cheaper-inference:release/v3.8.51from
L4XB:fix/13596-openrouter-batch-variants-are-not-chat

Conversation

@L4XB

@L4XB L4XB commented Sep 14, 2026

Copy link
Copy Markdown
Contributor

Closes #13596.

What was wrong

ModelSync imported OpenRouter's Batch-API-only variants into the chat catalogue, so a chat request that landed on one was rejected upstream:

404 This model is only available through the Batch API. Use the /api/beta/batches endpoint instead.

91 of those in the reporter's 41-hour window (third error class by volume), plus the [provider] Node <uuid> model not found (N) for google/gemini-*:batch - locking mode failover churn behind them. Deleting the 77 *:batch keys locally only holds until the next sync cycle.

Why here

open-sse/services/modelEndpointPolicy.ts already exists for exactly this: "Upstream /models responses often omit endpoint/modality metadata… Keep the exceptional provider knowledge here so discovery, import, and catalog projection agree." OpenRouter's /models carries nothing that separates a batch variant from a chat one, so classifyExplicitEndpoints returns null and the model falls through to { kind: "unknown", chatSelectable: true } — which is what imports it.

Putting the rule here means the one filter (filterChatSelectableModels) that sync-models already goes through — it sends x-omniroute-model-surface: chat — and that managedModelImport also calls, both pick it up. No new filtering path.

Two deliberate choices

Exact :batch, not "has a variant suffix". :free, :nitro, :floor, :online, :extended and :thinking are routing hints on the same chat model. Excluding them would silently shrink the routable catalogue — a worse failure than the one being fixed, because it is invisible.

Applied unconditionally for this provider, unlike the OpenAI branch below it, which lets an explicit multi-endpoint declaration win. There is no "batch" endpoint name an upstream could declare next to a chat one, and the rows already stored for these variants carry the synthetic ["chat"] default — letting that win would keep importing exactly what this excludes.

Tests

Five, in tests/unit/model-endpoint-policy.test.ts:

  • :batch variants classify as non-chat / not selectable
  • :free, :thinking, :nitro, :online and the bare base model stay selectable
  • a stored supportedEndpoints: ["chat"] does not re-admit a :batch variant
  • the rule is scoped to OpenRouter (custom-provider and a null provider are unaffected)
  • filterChatSelectableModels drops the batch variant and keeps its base model

Each earns its place — every mutation is caught by a different one:

mutation tests that fail
remove the OpenRouter branch entirely the :batch classification, the synthetic-default case, the filter case
match any : variant instead of :batch the other-suffixes case, the filter case
drop the provider scope (if (true)) the scoping case
node --test tests/unit/model-endpoint-policy.test.ts   → 12 pass, 0 fail
prettier --check (both files)                          → clean
tsc --noEmit --strict (the changed module)             → clean

Not in scope

The issue also asks for auto-quarantine of models the upstream rejects as "not supported" (the 34 GitHub Copilot 400s) and a routable-vs-catalog count in the dashboard. Both are policy decisions with their own state, and neither is needed to stop the :batch rejections — happy to follow up on either if you want them.

ModelSync imported OpenRouter's Batch-API-only variants into the chat
catalogue. A chat completion against one is rejected upstream with

  404 This model is only available through the Batch API.
      Use the /api/beta/batches endpoint instead.

which cheaper-inference#13596 measured 91 times in 41 hours, third by volume, plus the
`model not found - locking mode` failover churn behind it.

OpenRouter's /models carries no endpoint metadata that separates a batch
variant from a chat one, so `classifyExplicitEndpoints` cannot decide it
and the rule belongs in `modelEndpointPolicy`, beside the OpenAI
image/video policy and for the same reason: the file exists so discovery,
import and catalog projection agree on one answer.

Matched as the exact `:batch` suffix, not "has a variant suffix" --
`:free`, `:nitro`, `:floor`, `:online`, `:extended` and `:thinking` are
routing hints on the same chat model, and excluding them would silently
shrink the routable catalogue. Applied unconditionally for this provider:
there is no "batch" endpoint name an upstream could declare next to a chat
one, and the already-stored rows carry the synthetic `["chat"]` default
that re-imported them in the first place.

Closes cheaper-inference#13596
@L4XB
L4XB requested a review from diegosouzapw as a code owner September 14, 2026 03:52
@diegosouzapw

Copy link
Copy Markdown
Collaborator

Clean root cause and fix — confirmed modelEndpointPolicy.ts has zero batch-variant handling on
the current tip, matching your 91-occurrence production evidence from #13596. Tests pass clean
(12/12). This looks merge-ready as-is.

@diegosouzapw

Copy link
Copy Markdown
Collaborator

Thanks @L4XB — merging via the release merge-train. Validated in local merge-train (.claude/worktrees/merge-train-20260918-111718-suite.log) on the devbox @ train tip 7bb373fba5e241964c0ffb17bd03a804700839bb, boarded with 55 sibling PRs: typecheck:core, file-size, complexity, cognitive-complexity, changelog-integrity green; 747/747 changed-area node:test cases + 476/476 vitest green (fast parity mode — the full suite ran today on the release tip via the base-red train and runs again on the 3b train). Merged --admin per merge-gates §7.

@diegosouzapw
diegosouzapw merged commit ce56098 into cheaper-inference:release/v3.8.51 Sep 18, 2026
3 checks passed
muhamadgalihsaputra pushed a commit to niyatna/NiyatnaRoute that referenced this pull request Sep 27, 2026
…aper-inference#13622)

* fix(models): keep OpenRouter :batch variants out of chat routing

ModelSync imported OpenRouter's Batch-API-only variants into the chat
catalogue. A chat completion against one is rejected upstream with

  404 This model is only available through the Batch API.
      Use the /api/beta/batches endpoint instead.

which cheaper-inference#13596 measured 91 times in 41 hours, third by volume, plus the
`model not found - locking mode` failover churn behind it.

OpenRouter's /models carries no endpoint metadata that separates a batch
variant from a chat one, so `classifyExplicitEndpoints` cannot decide it
and the rule belongs in `modelEndpointPolicy`, beside the OpenAI
image/video policy and for the same reason: the file exists so discovery,
import and catalog projection agree on one answer.

Matched as the exact `:batch` suffix, not "has a variant suffix" --
`:free`, `:nitro`, `:floor`, `:online`, `:extended` and `:thinking` are
routing hints on the same chat model, and excluding them would silently
shrink the routable catalogue. Applied unconditionally for this provider:
there is no "batch" endpoint name an upstream could declare next to a chat
one, and the already-stored rows carry the synthetic `["chat"]` default
that re-imported them in the first place.

Closes cheaper-inference#13596

* docs(changelog): add fragment for the OpenRouter batch-variant fix
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

fix(providers): ModelSync imports batch-only variants (*:batch) into chat routing — 91 Batch API rejections

2 participants