Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions changelog.d/features/15234-muse-two-account-rotation.md
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
- **feat(muse):** session-level two-account rotation with caller-bound reasoning ownership — fresh sessions round-robin across native OAuth accounts while continuations and tool-result turns stay pinned to owner connection, account identity, and credential generation (hashes only), failing closed on unknown owners or generation changes instead of silent cross-account replay; OAuth-narrowed (API-key operation preserved) with Muse semantic-cache bypass; empty Muse responses retry up to twice on the same owner without locking it out; a single OAuth account keeps working without a session ID, and sessions idle for 30 days are pruned so ownership rows stay bounded ([#15234](https://github.com/diegosouzapw/OmniRoute/pull/15234))
1 change: 1 addition & 0 deletions changelog.d/features/15521-muse-code-consolidated.md
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
- **feat(muse-code):** consolidate open Muse Code PRs — opencode-go sends `reasoning.effort=max` verbatim for muse-spark-1.3+ with one `xhigh` retry on the upstream unsupported-effort 400 ([#12687](https://github.com/diegosouzapw/OmniRoute/pull/12687)); `MuseCodeExecutor` restores the 24h `prompt_cache_retention` window the generic sanitizer strips ([#13634](https://github.com/diegosouzapw/OmniRoute/pull/13634)); device-flow hardening: origin-pinned verification URLs, 24h expiry cap, allowlisted poll errors with fixed messages, bounded auth requests ([#14267](https://github.com/diegosouzapw/OmniRoute/pull/14267)); mint-failure contract: 401/403 forces re-login, 429/transient keeps the stored key ([#14138](https://github.com/diegosouzapw/OmniRoute/pull/14138)) — thanks @Chewji9875 @KSonny4 @caniko @SIGTERM-015
1 change: 1 addition & 0 deletions changelog.d/fixes/15234-muse-code-min-output-tokens.md
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
- **fix(muse):** floor tiny caller output budgets for `muse-code` (e.g. health probes sending `max_output_tokens: 16`) to `MUSE_SPARK_MIN_OUTPUT_TOKENS = 512`. Muse spends the budget on hidden reasoning, so these requests ended `response.incomplete` + `response.failed` "Provider returned empty content", which OmniRoute turned into a `upstream_empty_response` 502 plus a model lockout ([#15234](https://github.com/diegosouzapw/OmniRoute/pull/15234))
18 changes: 16 additions & 2 deletions docs/routing/REASONING_REPLAY.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,8 +4,6 @@ version: 3.8.40
lastUpdated: 2026-06-28
---

# Reasoning Replay Cache

> **Source of truth:** `src/lib/db/reasoningCache.ts`, `open-sse/services/reasoningCache.ts`
> **Last updated:** 2026-06-28 — v3.8.40

Expand Down Expand Up @@ -163,6 +161,22 @@ The cache exposes two endpoints under `src/app/api/cache/reasoning/route.ts`. Bo
- **Write is gated too:** both call sites in `chatCore.ts` (non-streaming and streaming) only call `cacheReasoningFromAssistantMessage()` when `requiresReasoningReplay(provider, model)` is `true` — the same predicate the read side checks. Installs that never touch a replay provider stop paying for the write, the index update, and the try/catch on every reasoning-bearing response.
- **Non-strict providers:** When `requiresReasoningReplay` is `false` and the target format is OpenAI, the translator **strips** any `reasoning_content` field from outgoing messages — OpenAI Chat Completions does not accept it.

## Muse Opaque-Reasoning Ownership

Muse (`muse-code`) returns caller-bound opaque reasoning (`encrypted_content`). Replaying it under a different account or reminted credential fails upstream, so native OAuth requests use session-level ownership instead of per-request rotation (`src/sse/services/museSessionOwnership.ts`, enforced in `src/sse/handlers/chat.ts`). Explicitly selected API-key connections and API-key-only configurations retain their existing routing without this ownership mode.

- Supply a stable session ID on every turn: `x-omniroute-session-id`, `x-omniroute-session`, `x-session-id`, or `x-codex-session-id`; alternatively use `prompt_cache_key`, `session_id`, `conversation_id`, or `metadata.session_id` in the body (in that precedence order). Missing or blank IDs return 400 when two or more native OAuth accounts are active; with a single OAuth account there is nothing to rotate to, so such a request keeps unpinned routing. The scope includes the caller's OmniRoute API-key ID.
- Fresh independent sessions round-robin across active native OAuth accounts permitted by the API-key connection policy, ordered by connection ID. Round-robin skips accounts in connection cooldown, with a terminal status (`banned`, `expired`, `credits_exhausted`), or model-locked for the requested model; Muse upstream failures record cooldown and breaker state like other providers. An explicitly forced connection bypasses round-robin for a new session but cannot override an existing owner.
- Until the owner serves a successful response, a session whose owner has become unavailable is re-claimed onto a healthy account. After the first successful response, the session stays pinned to that owner until it has been idle for 30 days (`MUSE_SESSION_IDLE_TTL_MS`).
- Every continuation and tool-result turn stays pinned to the session owner: connection, account identity, and inference-credential generation, checked again before upstream attempts, including credential-refresh retries. Responses call IDs/item references and Chat Completions tool-call IDs must have been recorded for that session and generation.
- SQLite `key_value` records in the `muse_session_ownership` namespace (`src/lib/db/museSessionOwnership.ts`) persist ownership and the rotation cursor across restarts. Each claim and served response refreshes the session's activity stamp; a session idle longer than `MUSE_SESSION_IDLE_TTL_MS` is pruned (owner pin, served flag and recorded opaque/reference hashes), at most once every 10 minutes and in bounded batches, so a later replay of its history fails closed as an unknown owner. Records contain connection IDs and hashes of account identity, credentials, opaque content, and continuation references, not secrets or conversation history. Clients must retain and send their full history and opaque reasoning.
- Unknown or foreign opaque reasoning, a missing owner, an unavailable owner account, or a generation change with recorded history fails closed with an explicit 4xx/503 — never silent cross-account failover and never dropped reasoning.
- A reminted same-account key is adopted only when the session recorded no replayable history; recorded sessions keep failing closed until started fresh.
- An empty upstream response (`upstream_empty_response` / `empty_response`, nothing emitted to the caller) is retried at most twice on the same owner, after 0.5 s then 1.5 s (`MUSE_EMPTY_RESPONSE_RETRY_DELAYS_MS`). If all attempts are empty, the original 502 is returned without cooldown or breaker marking, so a flaky reply never locks out the owner.
- All `muse-code` requests bypass semantic-cache reads and writes, including streaming writes, so cached opaque output cannot acquire a different session owner.

Regression guards: `tests/unit/muse-session-ownership.test.ts`, `tests/unit/muse-session-ownership-retention.test.ts`, `tests/unit/muse-empty-response-retry.test.ts`, and `tests/unit/chatcore-semantic-cache.test.ts`.

## See Also

- [RESILIENCE_GUIDE.md](../architecture/RESILIENCE_GUIDE.md) — circuit breakers, cooldowns, model lockouts
Expand Down
8 changes: 7 additions & 1 deletion open-sse/config/providerModels.ts
Original file line number Diff line number Diff line change
Expand Up @@ -245,7 +245,13 @@ export function findModelName(aliasOrId: string, modelId: string): string {
// OpenCode's Muse Spark family is Responses-only. Keep this rule provider-scoped
// and version-agnostic so a newly published Muse Spark model is routed correctly
// before the static catalog is refreshed.
const OPENCODE_MUSE_SPARK_ALIASES = new Set(["opencode-zen", "opencode-go"]);
const OPENCODE_MUSE_SPARK_ALIASES = new Set([
"opencode-zen",
"opencode-go",
// Short aliases that do not normalize via PROVIDER_ID_TO_ALIAS (#12687).
"ocg",
"ocz",
]);
const MUSE_SPARK_MODEL_PATTERN = /^muse-spark(?:-|$)/i;

const OPENCODE_MODEL_PREFIXES = ["opencode/", "opencode-zen/", "opencode-go/"] as const;
Expand Down
15 changes: 14 additions & 1 deletion open-sse/config/providers/registry/opencode/go/index.ts
Original file line number Diff line number Diff line change
Expand Up @@ -283,7 +283,9 @@ export const opencode_goProvider: RegistryEntry = {
},
// #12674: Muse Spark 1.3 Contributor — base + effort-tier aliases from the
// OpenCode Go registry (`opencode models opencode-go --refresh --verbose`;
// exact suffix set: minimal/low/medium/high/xhigh, no max — same as 1.2).
// exact suffix set: minimal/low/medium/high/xhigh — same as 1.2). #12687 adds
// a `-max` variant: the executor sends `max` verbatim and retries once with
// `xhigh` only if upstream rejects it (dispatchWithMuseSparkMaxFallback).
// Upstream serves Muse Spark only on the Responses API; without
// targetFormat:"openai-responses" these fall through to /chat/completions
// and the upstream returns 500 (same class as #12196).
Expand Down Expand Up @@ -354,6 +356,17 @@ export const opencode_goProvider: RegistryEntry = {
supportsVideo: true,
targetFormat: "openai-responses",
},
{
id: "muse-spark-1.3-contributor-max",
name: "Muse Spark 1.3 Contributor (max effort)",
contextLength: 1048576,
maxOutputTokens: 131072,
supportsReasoning: true,
supportsVision: true,
supportsAudio: true,
supportsVideo: true,
targetFormat: "openai-responses",
},
// #8353: Grok 4.5 + effort tiers from the OpenCode Go registry.
// The Go endpoint table (opencode.ai/docs/go) serves the Grok family on
// /responses only; without targetFormat:"openai-responses" these rows fall
Expand Down
30 changes: 25 additions & 5 deletions open-sse/executors/base/reasoningEffort.ts
Original file line number Diff line number Diff line change
Expand Up @@ -64,6 +64,7 @@ export const MAX_TIER_REASONING_MODEL_PATTERN =
export const O1_O3_REASONING_MODELS_PATTERN = /(?:^|\/|\b)(?:o1-mini|o1|o3-mini|o3-pro|o3)(?:$|-)/i;
export const O1_PREVIEW_PATTERN = /(?:^|\/|\b)o1-preview(?:$|-)/i;
export const MUSE_SPARK_PATTERN = /(?:^|\/|\b)muse-spark/i;
export const MUSE_SPARK_12_PATTERN = /(?:^|\/|\b)muse-spark-1\.2(?:$|-|\b)/i;
export const MINIMAX_REASONING_PATTERN = /(?:^|\/|\b)minimax(?:-m3|-m2)/i;
export const GROK_45_PATTERN = /(?:^|\/|\b)grok-4\.5/i;
export const GROK_46_PATTERN = /(?:^|\/|\b)grok-4\.6/i;
Expand Down Expand Up @@ -414,14 +415,16 @@ export function sanitizeReasoningEffortForProvider(
return body;
}

// ── Muse Spark models (muse-spark-1.2, etc.) ─────────────────────────────
// Accepts minimal|low|medium|high|xhigh. Rejects none (400) and max.
// max/ultra → xhigh; none → minimal.
if (MUSE_SPARK_PATTERN.test(modelStr)) {
// ── Muse Spark models (muse-spark-1.2 vs 1.3+ / future versions) ───────
// 1.2: Accepts minimal|low|medium|high|xhigh (no max).
// max/ultra → xhigh; none → minimal.
// 1.3+ / future versions: Accepts minimal|low|medium|high|xhigh|max.
// ultra → max; none → minimal.
if (MUSE_SPARK_12_PATTERN.test(modelStr)) {
if (effortStr === "max" || effortStr === "ultra") {
log?.info?.(
"REASONING_SANITIZE",
`${provider}/${modelStr}: clamped reasoning_effort ${effortStr} → xhigh (Muse Spark ceiling)`
`${provider}/${modelStr}: clamped reasoning_effort ${effortStr} → xhigh (Muse Spark 1.2 ceiling)`
);
return writeEffortValue(b, "xhigh", c);
}
Expand All @@ -434,6 +437,23 @@ export function sanitizeReasoningEffortForProvider(
}
return body;
}
if (MUSE_SPARK_PATTERN.test(modelStr)) {
if (effortStr === "ultra") {
log?.info?.(
"REASONING_SANITIZE",
`${provider}/${modelStr}: clamped reasoning_effort ultra → max (Muse Spark ceiling)`
);
return writeEffortValue(b, "max", c);
}
if (effortStr === "none") {
log?.info?.(
"REASONING_SANITIZE",
`${provider}/${modelStr}: clamped reasoning_effort none → minimal (Muse Spark floor)`
);
return writeEffortValue(b, "minimal", c);
}
return body;
}

// ── OpenAI o1 / o3-mini models ───────────────────────────────────────────
// Accepts only low|medium|high. Clamp xhigh/max/ultra → high.
Expand Down
23 changes: 3 additions & 20 deletions open-sse/executors/default.ts
Original file line number Diff line number Diff line change
Expand Up @@ -60,6 +60,8 @@ import {
normalizeGigachatChatUrl,
} from "@/lib/providers/validation/urlHelpers";
import { forwardOpencodeClientHeaders } from "../utils/opencodeHeaders.ts";
import { floorMuseOutputTokens } from "./default/museCode.ts";
import { defaultPerplexityAgentMaxOutputTokens } from "./default/perplexityAgentDefaults.ts";
import { resolveZaiUrl } from "./default/zaiFormatOverride.ts";
import { normalizePoolConfig, rejectStrictPool } from "./default/poolConfig.ts";
import { isForceStreamProvider } from "./default/forceStreamBody.ts";
Expand All @@ -72,26 +74,6 @@ import { applyRegistryBodyRepairs } from "../utils/strictChatHistory.ts";
import { withWorkbuddyHeaders, sanitizeWorkbuddyBody } from "../utils/workbuddyGateway.ts";

const NVIDIA_TOOL_CALL_ID_PATTERN = /^[A-Za-z0-9]{9}$/;
const PERPLEXITY_AGENT_DEFAULT_MAX_OUTPUT_TOKENS = 4096;

function defaultPerplexityAgentMaxOutputTokens<T>(body: T): T {
if (!body || typeof body !== "object" || Array.isArray(body)) return body;

const record = body as Record<string, unknown>;
if (
record.max_output_tokens !== undefined ||
record.max_completion_tokens !== undefined ||
record.max_tokens !== undefined
) {
return body;
}

return {
...record,
max_output_tokens: PERPLEXITY_AGENT_DEFAULT_MAX_OUTPUT_TOKENS,
} as T;
}

const ZAI_GLM_53_OPENAI_MODEL_PATTERN = /^glm-5\.3(?:-flash)?$/i;
const ZAI_GLM_53_EFFORT_MODEL_PATTERN = /^(glm-5\.3(?:-flash)?)-(low|high|max)$/i;

Expand Down Expand Up @@ -835,6 +817,7 @@ export class DefaultExecutor extends BaseExecutor {
if (this.provider === "perplexity-agent") {
withDefaults = defaultPerplexityAgentMaxOutputTokens(withDefaults);
}
if (this.provider === "muse-code") withDefaults = floorMuseOutputTokens(withDefaults);

if (this.provider === "nvidia") normalizeNvidiaToolCallIds(withDefaults);
if (this.provider === "openai") withDefaults = normalizeOpenAIToolCallIds(withDefaults);
Expand Down
24 changes: 24 additions & 0 deletions open-sse/executors/default/museCode.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,24 @@
/**
* Muse Code request tweaks for DefaultExecutor (extracted from default.ts). The Responses
* URL for muse-code is owned by the `case "muse-code"` branch of buildUrl (#15576).
*/

import { MUSE_SPARK_MIN_OUTPUT_TOKENS } from "../opencodeMuseSpark.ts";

/**
* Muse spends the whole budget on hidden reasoning before any text: a tiny caller budget
* (health probes send 16–32) ends `response.incomplete` + `response.failed` "Provider returned
* empty content" → 502 and a model lockout. Same floor as the opencode muse-spark path; larger
* budgets and unset budgets are untouched.
*/
export function floorMuseOutputTokens<T>(body: T): T {
if (!body || typeof body !== "object" || Array.isArray(body)) return body;
const record = { ...(body as Record<string, unknown>) };
for (const field of ["max_output_tokens", "max_tokens"] as const) {
const value = record[field];
if (typeof value === "number" && value < MUSE_SPARK_MIN_OUTPUT_TOKENS) {
record[field] = MUSE_SPARK_MIN_OUTPUT_TOKENS;
}
}
return record as T;
}
19 changes: 19 additions & 0 deletions open-sse/executors/default/perplexityAgentDefaults.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,19 @@
const PERPLEXITY_AGENT_DEFAULT_MAX_OUTPUT_TOKENS = 4096;

export function defaultPerplexityAgentMaxOutputTokens<T>(body: T): T {
if (!body || typeof body !== "object" || Array.isArray(body)) return body;

const record = body as Record<string, unknown>;
if (
record.max_output_tokens !== undefined ||
record.max_completion_tokens !== undefined ||
record.max_tokens !== undefined
) {
return body;
}

return {
...record,
max_output_tokens: PERPLEXITY_AGENT_DEFAULT_MAX_OUTPUT_TOKENS,
} as T;
}
3 changes: 2 additions & 1 deletion open-sse/executors/defaultResolver.ts
Original file line number Diff line number Diff line change
@@ -1,12 +1,13 @@
import { DefaultExecutor } from "./default.ts";
import { MuseCodeExecutor } from "./muse-code.ts";

const defaultExecutorCache = new Map<string, DefaultExecutor>();

/** Resolve the shared fallback executor without initializing the specialized executor registry. */
export function getDefaultExecutor(provider: string): DefaultExecutor {
let executor = defaultExecutorCache.get(provider);
if (!executor) {
executor = new DefaultExecutor(provider);
executor = provider === "muse-code" ? new MuseCodeExecutor() : new DefaultExecutor(provider);
defaultExecutorCache.set(provider, executor);
}
return executor;
Expand Down
49 changes: 49 additions & 0 deletions open-sse/executors/muse-code.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,49 @@
import { DefaultExecutor } from "./default.ts";

type JsonRecord = Record<string, unknown>;

function asRecord(value: unknown): JsonRecord | null {
return value && typeof value === "object" && !Array.isArray(value)
? (value as JsonRecord)
: null;
}

/**
* Muse Code uses the normal OpenAI-compatible executor for inference, plus one
* provider-specific behaviour: Muse's Responses API supports a 24h
* prompt-cache retention window. The generic executor intentionally strips this
* field for strict upstreams, so restore it only on the Muse Code route after
* generic sanitisation (#13634).
*
* Credential remint stays on the centralized path: DefaultExecutor
* refreshCredentials → getAccessToken → refreshMuseCodeToken (remint from the
* durable `dca:` token, with race dedup). No override here on purpose.
*/
export class MuseCodeExecutor extends DefaultExecutor {
constructor() {
super("muse-code");
}

transformRequest(
model: string,
body: unknown,
stream: boolean,
credentials: Parameters<DefaultExecutor["transformRequest"]>[3]
): unknown {
const original = asRecord(body);
const requestedRetention =
typeof original?.prompt_cache_retention === "string"
? original.prompt_cache_retention
: "24h";
const cleaned = super.transformRequest(model, body, stream, credentials);
const record = asRecord(cleaned);
if (!record) return cleaned;

return {
...record,
prompt_cache_retention: requestedRetention,
};
}
}

export default MuseCodeExecutor;
Loading
Loading