Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,8 @@ _In development — bullets added per PR; finalized at release._
- **fix(capabilities): unify vision model-id detection into one shared source** — three code paths kept independent, drifting vision-model lists, so the same model id could get up to three different verdicts. Two concrete bugs: lite compression's gate was missing pixtral / llava / qwen-vl / glm-4v / kimi-vl / mistral-medium-3, so it **stripped images for those real vision models and blinded them** (same class as #4071 / #4012); and the `/v1/models` list was too broad, flagging text models (`gemma`, bare `kimi` like `kimi-k2`) as vision. All three (`modelCapabilities` routing fallback, `/v1/models` listing, lite image-strip gate) now delegate to a single conservative source `src/shared/constants/visionModels.ts`, which also restores `glm-4v` / `gemini-3` coverage and keeps the #3328 MiniMax M3 carve-out. ([#4072](https://github.com/diegosouzapw/OmniRoute/issues/4072) — thanks @diego-anselmo)
- **fix(sse): surface mid-stream Gemini errors instead of returning a truncated 200** — when an upstream Gemini SSE stream emitted some partial content and then a JSON error object (`{"error":{"code":503,"message":"…high demand…","status":"UNAVAILABLE"}}`) instead of a `candidates` payload, OmniRoute silently dropped it: the gemini→openai translator's no-candidate branch only handled `promptFeedback` (content-filter) and returned `null` for anything else, so the stream simply ended and the client got HTTP 200 with a truncated body and `finish_reason: "stop"` — masking the failure and skipping combo fallback. `geminiToOpenAIResponse` now detects an `error` object (optionally wrapped in `response`), records it as `state.upstreamError` (preserving the real status — 503/`UNAVAILABLE`, or 429 for `RESOURCE_EXHAUSTED`), and lets `stream.ts` error the stream out through the existing `onFailure`/`buildErrorBody`/`controller.error` path — the same mechanism the openai-responses translator already uses. ([#4177](https://github.com/diegosouzapw/OmniRoute/issues/4177) — thanks @hartmark)
- **fix(capabilities): resolve models.dev-synced vision metadata for Mistral `-latest` aliases** — root cause behind the #4071 heuristic: `getResolvedModelCapabilities("mistral/pixtral-12b-latest").supportsVision` resolved `null` (vision came only from the #4071 model-id heuristic, with `attachment` still `null`) even though models.dev exposes the model as multimodal. Confirmed against the live models.dev API: it catalogs Pixtral 12B under the **short** id `pixtral-12b` (with `attachment: true`, `modalities.input: ["text","image"]`), while requests use the Mistral API alias `pixtral-12b-latest`. The synced lookup tried the exact / raw / static-spec-canonical ids — all of which miss the short form — so it fell through to the heuristic. `getSyncedCapabilityForResolved` now adds a last-resort fallback that retries with a trailing `-latest` stripped, so synced metadata (`attachment` / image modalities) wins for these aliases; models whose `-latest` id is stored verbatim (e.g. `pixtral-large-latest`) keep resolving directly. Note: the models.dev sync is currently manual-only (Settings → models.dev) with no scheduled refresh, so a fresh instance still relies on the #4071 heuristic until that sync runs — a periodic-refresh cadence is left as a separate follow-up. ([#4073](https://github.com/diegosouzapw/OmniRoute/issues/4073) — thanks @diego-anselmo)
- **fix(sse): map Xiaomi MiMo reasoning control to its native `thinking:{type}` shape** — MiMo (`api.xiaomimimo.com`) controls chain-of-thought **only** via top-level `thinking:{type:"enabled"|"disabled"}` and does not understand OpenAI's `reasoning_effort`/`reasoning`, while its request validator is strict (`400 Param Incorrect`). OmniRoute's OpenAI path carried reasoning intent as `reasoning_effort`, and the claude→openai translator can leave a Claude-shaped `thinking:{type, budget_tokens}` — so the client's on/off choice was silently dropped and `budget_tokens`/`reasoning_effort` rode along as extra params the validator can reject. New `open-sse/services/mimoThinking.ts::normalizeMimoThinking` (wired in `chatCore` for `provider==="xiaomi-mimo"`) reduces any thinking object to just `{type}` (`disabled` stays; `enabled`/`adaptive`/other → `enabled`) and drops `reasoning_effort`/`reasoning`. It deliberately does **not** synthesize thinking from a bare `reasoning_effort` — `mimo-v2-omni` is non-thinking, so that could turn a silently-ignored param into a hard error. ([#4224](https://github.com/diegosouzapw/OmniRoute/pull/4224))
- **fix(capabilities): Xiaomi MiMo `*-pro` chat models are text-only (no vision)** — only `mimo-v2.5` and `mimo-v2-omni` accept images per Xiaomi's docs; `mimo-v2.5-pro`/`mimo-v2-pro` are text-only, but `modelSpecs` marked them vision-capable and models.dev mislabels them ([hermes-agent#18884](https://github.com/NousResearch/hermes-agent/issues/18884)). Since `resolveVisionCapability` lets a synced `attachment:true` win first, an image request could be routed to a blind model (the #4071 failure mode). Corrected the specs **and** added a hard override in `resolveVisionCapability` (checked before the synced branch, anchored so `mimo-v2.5-pro` never matches the multimodal `mimo-v2.5`) that beats the wrong synced attachment. Also registered the missing native `mimo-v2-pro` chat model and the missing `mimo-v2-tts` speech model. ([#4224](https://github.com/diegosouzapw/OmniRoute/pull/4224))

### 🧪 Tests

Expand Down
2 changes: 1 addition & 1 deletion config/quality/file-size-baseline.json
Original file line number Diff line number Diff line change
Expand Up @@ -69,7 +69,7 @@
"open-sse/executors/muse-spark-web.ts": 1284,
"open-sse/executors/perplexity-web.ts": 1013,
"open-sse/handlers/audioSpeech.ts": 965,
"open-sse/handlers/chatCore.ts": 5086,
"open-sse/handlers/chatCore.ts": 5095,
"open-sse/handlers/imageGeneration.ts": 3777,
"open-sse/handlers/responseSanitizer.ts": 1103,
"open-sse/handlers/search.ts": 1546,
Expand Down
1 change: 1 addition & 0 deletions open-sse/config/audioRegistry.ts
Original file line number Diff line number Diff line change
Expand Up @@ -390,6 +390,7 @@ export const AUDIO_SPEECH_PROVIDERS: Record<string, AudioProvider> = {
{ id: "mimo-v2.5-tts", name: "MiMo V2.5 TTS" },
{ id: "mimo-v2.5-tts-voicedesign", name: "MiMo V2.5 Voice Design" },
{ id: "mimo-v2.5-tts-voiceclone", name: "MiMo V2.5 Voice Clone" },
{ id: "mimo-v2-tts", name: "MiMo V2 TTS" },
],
},
};
Expand Down
1 change: 1 addition & 0 deletions open-sse/config/providers/shared.ts
Original file line number Diff line number Diff line change
Expand Up @@ -376,6 +376,7 @@ export const CHAT_OPENAI_COMPAT_MODELS: Record<string, RegistryModel[]> = {
"xiaomi-mimo": [
{ id: "mimo-v2.5-pro", name: "MiMo-V2.5-Pro", contextLength: 1048576, maxOutputTokens: 131072 },
{ id: "mimo-v2.5", name: "MiMo-V2.5", contextLength: 1048576, maxOutputTokens: 131072 },
{ id: "mimo-v2-pro", name: "MiMo-V2-Pro", contextLength: 262144, maxOutputTokens: 131072 },
{ id: "mimo-v2-omni", name: "MiMo-V2-Omni", contextLength: 262144, maxOutputTokens: 131072 },
{ id: "mimo-v2-flash", name: "MiMo-V2-Flash", contextLength: 262144, maxOutputTokens: 65536 },
],
Expand Down
9 changes: 9 additions & 0 deletions open-sse/handlers/chatCore.ts
Original file line number Diff line number Diff line change
Expand Up @@ -79,6 +79,7 @@ import {
stripIncompatibleMessageContent,
} from "../services/modelStrip.ts";
import { resolveModelAlias } from "../services/modelDeprecation.ts";
import { normalizeMimoThinking } from "../services/mimoThinking.ts";
import { getUnsupportedParams } from "../config/providerRegistry.ts";
import { supportsMaxTokens } from "@/lib/modelCapabilities.ts";
import { normalizeThinkingForModel } from "@/shared/constants/modelSpecs.ts";
Expand Down Expand Up @@ -2536,6 +2537,14 @@ export async function handleChatCore({
translatedBody = normalizeThinkingForModel(translatedBody, finalModelToUpstream);
}

// Xiaomi MiMo controls reasoning ONLY via `thinking:{type:"enabled"|"disabled"}` and
// rejects unknown/extra params with a strict "400 Param Incorrect". Map OmniRoute's
// OpenAI reasoning signals onto that native shape: reduce any thinking object to
// `{type}` and drop `reasoning_effort`/`reasoning`. See services/mimoThinking.ts.
if (provider === "xiaomi-mimo") {
translatedBody = normalizeMimoThinking(translatedBody);
}

const previousResponseIdPolicy = applyResponsesPreviousResponseIdPolicy(translatedBody, {
mode: settings.responsesPreviousResponseIdMode,
sourceFormat,
Expand Down
54 changes: 54 additions & 0 deletions open-sse/services/mimoThinking.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,54 @@
type JsonRecord = Record<string, unknown>;

function asRecord(value: unknown): JsonRecord | null {
return value && typeof value === "object" && !Array.isArray(value) ? (value as JsonRecord) : null;
}

/**
* Normalize chain-of-thought control to Xiaomi MiMo's native request shape.
*
* MiMo controls thinking ONLY via a top-level `thinking: { type: "enabled" | "disabled" }`
* object (mimo.mi.com .../usage-guide/text-generation/deep-thinking). It does NOT
* understand OpenAI's `reasoning_effort` / `reasoning`, and its request validator is
* strict ("400 Param Incorrect"). OmniRoute's OpenAI path otherwise carries reasoning
* intent as `reasoning_effort`, and the claude→openai translator may leave a
* Claude-shaped `thinking:{type, budget_tokens}` — forwarding either verbatim means the
* client's on/off choice is silently dropped AND unknown keys (`budget_tokens`,
* `reasoning_effort`) ride along as extra params the validator can reject.
*
* This returns a NEW object only when it changes the body:
* - an existing `thinking` object → reduced to just `{type}`; `"disabled"` stays
* disabled, anything else (`enabled` / `adaptive` / unknown) becomes `"enabled"`.
* This strips `budget_tokens` / `keep` / … that MiMo does not accept.
* - `reasoning_effort` / `reasoning` → removed (MiMo's unknown params).
*
* It deliberately does NOT synthesize `thinking:{type:"enabled"}` from a bare
* `reasoning_effort`: `mimo-v2-omni` is documented as non-thinking, so forcing thinking
* on from an effort hint could turn a silently-ignored param into a hard error. Clients
* enable thinking by sending MiMo's `thinking:{type:"enabled"}` (or relying on the
* per-model default) and disable it with `thinking:{type:"disabled"}` — both now reach
* MiMo cleanly. When neither a thinking object nor `reasoning_effort` / `reasoning` is
* present, the body is returned untouched so MiMo keeps its documented per-model default.
*
* Scope: the native `xiaomi-mimo` provider only (callers gate on provider id). This is a
* separate concern from the OpenRouter/native reasoning_effort tier mapping in
* `sanitizeReasoningEffortForProvider` (base.ts), which runs later and becomes a no-op
* here once `reasoning_effort` has been removed.
*/
export function normalizeMimoThinking<T extends Record<string, unknown>>(body: T): T {
const record = asRecord(body);
if (!record) return body;

const thinking = asRecord(record.thinking);
const hasReasoningEffort = record.reasoning_effort !== undefined;
const hasReasoning = record.reasoning !== undefined;
if (!thinking && !hasReasoningEffort && !hasReasoning) return body;

const next: JsonRecord = { ...record };
if (thinking) {
next.thinking = { type: thinking.type === "disabled" ? "disabled" : "enabled" };
}
delete next.reasoning_effort;
delete next.reasoning;
return next as T;
}
27 changes: 27 additions & 0 deletions src/lib/modelCapabilities.ts
Original file line number Diff line number Diff line change
Expand Up @@ -255,6 +255,28 @@ export function modelIdLikelyVision(modelId: string | null | undefined): boolean
return isVisionModelId(modelId);
}

/**
* Models that upstream catalogs (notably models.dev) mislabel as vision-capable but
* are TEXT-ONLY per the vendor's own docs. Listed here so a wrong synced
* `attachment:true` cannot route an image request to a blind model (the #4071 failure
* mode). Keep this list tiny and doc-backed.
*
* Xiaomi MiMo: only `mimo-v2.5` and `mimo-v2-omni` accept images; the `*-pro` chat
* models are text-only (mimo.mi.com .../image-understanding; hermes-agent#18884).
* Anchored to the full id (`$`) and tolerant of a `provider/` prefix so `mimo-v2.5-pro`
* never matches the multimodal `mimo-v2.5`, and `mimo-v2-pro` never matches `mimo-v2-omni`.
*/
const KNOWN_TEXT_ONLY_DESPITE_SYNC: readonly RegExp[] = [
/(?:^|\/)mimo-v2\.5-pro$/i,
/(?:^|\/)mimo-v2-pro$/i,
];

function isKnownTextOnlyDespiteSync(modelId: string | null | undefined): boolean {
if (!modelId) return false;
const id = String(modelId);
return KNOWN_TEXT_ONLY_DESPITE_SYNC.some((pattern) => pattern.test(id));
}

function resolveVisionCapability(
spec: ModelSpec | undefined,
registryModel: { supportsVision?: boolean } | null,
Expand All @@ -267,6 +289,11 @@ function resolveVisionCapability(
String(entry).toLowerCase()
);

// Hard override FIRST: a wrong synced `attachment:true` (or image modality) must not
// win for models the vendor documents as text-only. Beats every branch below so an
// image request can never be routed to a blind model (#4071).
if (isKnownTextOnlyDespiteSync(modelId)) return false;

if (typeof synced?.attachment === "boolean") {
return synced.attachment;
}
Expand Down
8 changes: 6 additions & 2 deletions src/shared/constants/modelSpecs.ts
Original file line number Diff line number Diff line change
Expand Up @@ -329,11 +329,15 @@ export const MODEL_SPECS: Record<string, ModelSpec> = {
},

// ── Xiaomi MiMo V2.5 (1M context, consensus across 7+ sync sources) ──
// Vision: ONLY mimo-v2.5 and mimo-v2-omni accept images per Xiaomi's docs
// (mimo.mi.com .../image-understanding). The *-pro chat models are TEXT-ONLY;
// models.dev mislabels them (hermes-agent#18884) — a hard override in
// src/lib/modelCapabilities.ts also beats that wrong synced attachment.
"mimo-v2.5-pro": {
maxOutputTokens: 131072,
contextWindow: 1048576,
supportsTools: true,
supportsVision: true,
supportsVision: false,
},
"mimo-v2.5": {
maxOutputTokens: 131072,
Expand All @@ -345,7 +349,7 @@ export const MODEL_SPECS: Record<string, ModelSpec> = {
maxOutputTokens: 131072,
contextWindow: 262144,
supportsTools: true,
supportsVision: true,
supportsVision: false,
},
"mimo-v2-omni": {
maxOutputTokens: 131072,
Expand Down
89 changes: 89 additions & 0 deletions tests/unit/mimo-thinking-normalize.test.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,89 @@
/**
* Xiaomi MiMo thinking normalization — `normalizeMimoThinking`.
*
* MiMo controls reasoning ONLY via `thinking:{type:"enabled"|"disabled"}` and rejects
* extra/unknown request params with a strict "400 Param Incorrect". These tests pin the
* normalization that maps OmniRoute's internal reasoning signals onto MiMo's native shape:
* reduce any thinking object to `{type}`, and drop `reasoning_effort` / `reasoning`.
*/
import test from "node:test";
import assert from "node:assert/strict";
import { normalizeMimoThinking } from "../../open-sse/services/mimoThinking.ts";

test("thinking:{type:'disabled'} is preserved (lets a client turn thinking OFF)", () => {
const body = { model: "mimo-v2.5-pro", messages: [], thinking: { type: "disabled" } };
const result = normalizeMimoThinking(body);
assert.deepEqual(result.thinking, { type: "disabled" });
});

test("Claude-shaped thinking:{type:'enabled', budget_tokens} → {type:'enabled'} (extras stripped)", () => {
const body = {
model: "mimo-v2.5-pro",
messages: [],
thinking: { type: "enabled", budget_tokens: 2048, keep: null },
};
const result = normalizeMimoThinking(body);
// MiMo only accepts `type`; budget_tokens/keep would be rejected as extra params.
assert.deepEqual(result.thinking, { type: "enabled" });
});

test("thinking:{type:'adaptive'} collapses to MiMo's binary 'enabled'", () => {
const body = { model: "mimo-v2.5", messages: [], thinking: { type: "adaptive" } };
const result = normalizeMimoThinking(body);
assert.deepEqual(result.thinking, { type: "enabled" });
});

test("reasoning_effort is removed (MiMo does not understand it) and no thinking is synthesized", () => {
const body = { model: "mimo-v2-flash", messages: [], reasoning_effort: "high" };
const result = normalizeMimoThinking(body) as Record<string, unknown>;
assert.equal(result.reasoning_effort, undefined, "reasoning_effort must be dropped");
// Deliberately NOT synthesized — mimo-v2-omni is non-thinking; forcing it on could 400.
assert.equal(result.thinking, undefined, "no thinking object is invented from a bare effort hint");
});

test("nested `reasoning` object is removed", () => {
const body = { model: "mimo-v2.5", messages: [], reasoning: { effort: "high", summary: "auto" } };
const result = normalizeMimoThinking(body) as Record<string, unknown>;
assert.equal(result.reasoning, undefined, "reasoning must be dropped");
});

test("reasoning_effort alongside an explicit thinking:{type:'disabled'} → thinking wins, effort dropped", () => {
const body = {
model: "mimo-v2.5-pro",
messages: [],
reasoning_effort: "xhigh",
thinking: { type: "disabled", budget_tokens: 1000 },
};
const result = normalizeMimoThinking(body) as Record<string, unknown>;
assert.deepEqual(result.thinking, { type: "disabled" });
assert.equal(result.reasoning_effort, undefined);
});

test("body with neither thinking nor reasoning is returned UNTOUCHED (same reference)", () => {
const body = { model: "mimo-v2.5-pro", messages: [{ role: "user", content: "hi" }] };
const result = normalizeMimoThinking(body);
assert.equal(result, body, "no-op must not allocate a new object");
});

test("non-object body is returned unchanged", () => {
const body = null as unknown as Record<string, unknown>;
assert.equal(normalizeMimoThinking(body), body);
});

test("unrelated fields are preserved", () => {
const body = {
model: "mimo-v2.5-pro",
messages: [{ role: "user", content: "hi" }],
temperature: 0.7,
max_tokens: 1024,
thinking: { type: "enabled", budget_tokens: 4096 },
reasoning_effort: "high",
};
const result = normalizeMimoThinking(body) as Record<string, unknown>;
assert.equal(result.model, "mimo-v2.5-pro");
assert.equal(result.temperature, 0.7);
assert.equal(result.max_tokens, 1024);
assert.deepEqual(result.messages, [{ role: "user", content: "hi" }]);
assert.deepEqual(result.thinking, { type: "enabled" });
assert.equal(result.reasoning_effort, undefined);
});
Loading
Loading