fix(combo): shuffle strict-random fallback remainder to spread load (#3959)

CHANGELOG.md

@@ -23,6 +23,7 @@
 
 ### 🐛 Fixed
 
+- **fix(combo): strict-random spreads fallbacks across healthy peers instead of retrying a failing model** — with the `strict-random` strategy, a model that kept failing was retried on essentially every request and traffic concentrated on a few models. The strategy shuffled only the deck-selected slot 0 and left the fallback remainder in **fixed priority order**, so after any failing deck pick the dispatch chain always fell through to the same top-priority model next. The fallback remainder is now shuffled (like the `random` strategy), so the fallback load — and recovery from a persistently-failing target — spreads evenly across the healthy peers. (Note: when the client always sends `tools` (e.g. OpenCode), the combo still correctly routes only to the tool-capable models in the combo — that capability filtering is by design.) ([#3959](https://github.com/diegosouzapw/OmniRoute/issues/3959) — thanks @KeNJiKunG)
 - **fix(dashboard): Logs auto-refresh now works even when the tab loads in the background** — the Logs page never auto-refreshed (only the manual Refresh button worked). The auto-refresh interval gated each tick on a visibility ref seeded once at mount and updated only by a `visibilitychange` event; when the tab mounted while the document reported `hidden` (background load, bfcache restore, embedded/Docker-proxied webviews) and no `visibilitychange` ever fired, the ref stayed `false` forever, so the interval ticked but never fetched. The tick now reads the live `document.visibilityState`, so polling self-heals as soon as the tab is visible — while still pausing when genuinely hidden. ([#3972](https://github.com/diegosouzapw/OmniRoute/issues/3972) — thanks @tjengbudi)
 - **fix(providers): LLM7 (and BytePlus) now fetch the live `/models` catalog instead of a stale hardcoded list** — importing an LLM7 key surfaced only a small, outdated model list even though `GET https://api.llm7.io/v1/models` returns the full pro/standard catalog. Both providers carried a correct `modelsUrl` in the registry, but neither was classified by any live-fetch branch of the model-import route (not `openai-compatible-*`, not self-hosted, not in `NAMED_OPENAI_STYLE_PROVIDERS`), so the route skipped the upstream probe and served the registry's 4 hardcoded entries (`source: "local_catalog"`). Added `llm7` and `byteplus` to `NAMED_OPENAI_STYLE_PROVIDERS` so the route probes `<baseUrl>/models` with the key and serves the live catalog, falling back to the local catalog only when the upstream fetch fails (so key import never breaks). ([#3976](https://github.com/diegosouzapw/OmniRoute/issues/3976) — thanks @FerLuisxd)
 - **fix(resilience): respect connection cooldown stored as a numeric epoch (router kept hammering 429 accounts)** — the router kept dispatching to connections still inside their rate-limit cooldown, causing client timeouts and "connection cooldown isn't respected" reports. Root cause: `rate_limited_until` is a `TEXT` column, but the Antigravity full-quota path (`setConnectionRateLimitUntil`) persists a raw epoch **number**, which SQLite coerces to a numeric string like `"1781696905131.0"`. The account-selection predicate then did `new Date("1781696905131.0")` → `Invalid Date` → `NaN`, so `NaN > Date.now()` was false and the cooling connection was never skipped. The cooldown read predicates (`isAccountUnavailable`, `getEarliestRateLimitedUntil`, `filterAvailableAccounts`, `parseFutureDateMs`) now normalize numeric-epoch strings as well as ISO strings/Date/number via a shared `cooldownUntilMs()` helper — ISO behavior is unchanged. ([#3954](https://github.com/diegosouzapw/OmniRoute/issues/3954))

config/quality/file-size-baseline.json

@@ -15,6 +15,7 @@
   "_rebaseline_2026_06_15_3929_vertex_media": "PR #3929 own growth: audioSpeech.ts 952->965 (+13) and videoGeneration.ts 1026->1078 (+52) = vertex/* media branches (Gemini TTS, Veo predictLongRunning poll) wired into the speech/video handlers; new logic lives in open-sse/executors/vertexMedia.ts (341, under cap). Cohesive media-provider feature.",
   "_rebaseline_2026_06_15_3879_redact_thinking": "PR #3879 + #3921 reconcile: AddApiKeyModal.tsx 843->845 (+2 = merging #3879's CcCompatibleRequestDefaultsFields (context1m + opt-in redact-thinking toggle) into #3921's preset-input block in the cc-compatible settings group). Cohesive UI; not extractable.",
   "_rebaseline_2026_06_15_3890_cache_preserve": "Issue #3890 own growth: chatCore.ts 5815->5823 (+8 = wire resolveCacheAwareConfig() into the compression apply step so the system prompt is never compressed in a caching context — honors the cache-aware skipSystemPrompt flag that selectCompressionStrategy could not carry). Cohesive cache-preservation guard at the existing compression chokepoint; not extractable.",
+  "_rebaseline_2026_06_16_3959_strict_random_shuffle": "Issue #3959 own growth: combo.ts 5277->5283 (+6 = shuffle the strict-random fallback remainder via fisherYatesShuffle, plus a 4-line comment, so a failing deck pick no longer always falls through to the same fixed top-priority model). Cohesive one-spot routing fix; structural shrink of combo.ts tracked in #3501.",
   "_rebaseline_2026_06_16_3972_logs_autorefresh": "Issue #3972 own growth: RequestLoggerV2.tsx 1282->1287 (+5 = the auto-refresh interval now reads the live document.visibilityState each tick instead of a stale mount-time ref, plus a 3-line comment explaining the hidden-tab trap). Cohesive one-spot fix; structural shrink of this component tracked in #3501.",
   "_rebaseline_2026_06_16_3976_llm7_byteplus_models": "Issue #3976 own growth: models/route.ts 2489->2494 (+5 = add llm7 + byteplus to NAMED_OPENAI_STYLE_PROVIDERS with an explanatory comment so the import route does a live <baseUrl>/models fetch instead of serving the stale hardcoded registry catalog). Structural shrink of this route tracked in #3789.",
   "_rebaseline_2026_06_16_3954_cooldown_epoch": "Issue #3954 own growth: accountFallback.ts 1708->1727 (+19 = a shared cooldownUntilMs() normalizer + its use in isAccountUnavailable/getEarliestRateLimitedUntil/filterAvailableAccounts so a rate_limited_until persisted as a numeric-epoch string is honored, not parsed to NaN) and auth.ts 2216->2219 (+3 = parseFutureDateMs reuses cooldownUntilMs). Cohesive cooldown read-path hardening at the existing chokepoints; one helper, not extractable.",
@@ -47,7 +48,7 @@
     "open-sse/services/batchProcessor.ts": 828,
     "open-sse/services/browserBackedChat.ts": 850,
     "open-sse/services/claudeCodeCompatible.ts": 1202,
-    "open-sse/services/combo.ts": 5277,
+    "open-sse/services/combo.ts": 5283,
     "open-sse/services/rateLimitManager.ts": 1017,
     "open-sse/services/tokenRefresh.ts": 1997,
     "open-sse/services/usage.ts": 3408,

open-sse/services/combo.ts

@@ -3608,7 +3608,13 @@ export async function handleComboChat({
     );
     const selectedTarget =
       orderedTargets.find((target) => target.executionKey === selectedExecutionKey) || null;
-    const rest = orderedTargets.filter((target) => target.executionKey !== selectedExecutionKey);
+    // #3959: shuffle the fallback remainder too. Previously `rest` kept fixed
+    // priority order, so after a failing deck pick the chain always fell through
+    // to the same top-priority model — a persistently-failing model was retried
+    // on essentially every request and fallback load never spread across peers.
+    const rest = fisherYatesShuffle(
+      orderedTargets.filter((target) => target.executionKey !== selectedExecutionKey)
+    );
     orderedTargets = [selectedTarget, ...rest].filter(
       (target): target is ResolvedComboTarget => target !== null
     );

tests/unit/combo-strict-random-distribution-3959.test.ts

@@ -0,0 +1,149 @@
+/**
+ * TDD regression for #3959: strict-random combo "routes to a model that never
+ * succeeds" and concentrates traffic on a few models.
+ *
+ * Root cause: the strict-random branch shuffled only slot 0 (the deck pick) and
+ * left the fallback remainder (`rest`) in FIXED priority order. So whenever the
+ * deck-selected target failed, the dispatch chain always fell through to the same
+ * top-priority model next — a persistently-failing model was retried on
+ * essentially every request, and the fallback load never spread across peers.
+ * (`random` does not have this defect because it shuffles the whole list.)
+ *
+ * Fix: shuffle `rest` too, so the fallback chain is randomized like `random`.
+ */
+import test from "node:test";
+import assert from "node:assert/strict";
+import fs from "node:fs";
+import os from "node:os";
+import path from "node:path";
+
+const TEST_DATA_DIR = fs.mkdtempSync(path.join(os.tmpdir(), "omniroute-3959-"));
+process.env.DATA_DIR = TEST_DATA_DIR;
+
+const { handleComboChat } = await import("../../open-sse/services/combo.ts");
+const core = await import("../../src/lib/db/core.ts");
+const { resetAllComboMetrics } = await import("../../open-sse/services/comboMetrics.ts");
+const { resetAllCircuitBreakers } = await import("../../src/shared/utils/circuitBreaker.ts");
+const { resetAll: resetAllSemaphores } = await import(
+  "../../open-sse/services/rateLimitSemaphore.ts"
+);
+const { _resetAllDecks } = await import("../../src/shared/utils/shuffleDeck.ts");
+
+function createLog() {
+  return {
+    info: () => {},
+    warn: () => {},
+    error: () => {},
+    debug: () => {},
+  };
+}
+
+function okResponse() {
+  return new Response(JSON.stringify({ choices: [{ message: { content: "ok" } }] }), {
+    status: 200,
+    headers: { "content-type": "application/json" },
+  });
+}
+
+function errorResponse(status: number) {
+  return new Response(JSON.stringify({ error: { message: `Error ${status}` } }), {
+    status,
+    headers: { "content-type": "application/json" },
+  });
+}
+
+test.beforeEach(() => {
+  resetAllComboMetrics();
+  resetAllCircuitBreakers();
+  resetAllSemaphores();
+  _resetAllDecks();
+});
+
+test.after(() => {
+  core.resetDbInstance();
+  fs.rmSync(TEST_DATA_DIR, { recursive: true, force: true });
+});
+
+test("#3959 strict-random spreads the fallback across healthy peers, not a fixed model", async () => {
+  const FAILING = "openai/gpt-4o-mini";
+  const HEALTHY = ["claude/sonnet", "gemini/gemini-2.5-flash", "groq/llama-3.3-70b"];
+  const models = [FAILING, ...HEALTHY];
+
+  const combo = {
+    name: "strict-random-fallback-spread-3959",
+    strategy: "strict-random",
+    models,
+    config: { maxRetries: 0, retryDelayMs: 0, fallbackDelayMs: 0 },
+  };
+
+  // Collect the immediate fallback target chosen whenever the deck picked the
+  // always-failing model first.
+  const fallbackAfterFailure = new Set<string>();
+  let failingWasPickedFirst = 0;
+
+  for (let i = 0; i < 80; i += 1) {
+    const calls: string[] = [];
+    const result = await handleComboChat({
+      body: {},
+      combo,
+      handleSingleModel: async (_body: unknown, modelStr: string) => {
+        calls.push(modelStr);
+        return modelStr === FAILING ? errorResponse(500) : okResponse();
+      },
+      isModelAvailable: async () => true,
+      log: createLog(),
+      settings: null,
+      allCombos: null,
+    });
+
+    assert.equal(result.ok, true, "a healthy peer must always serve the request");
+    if (calls[0] === FAILING) {
+      failingWasPickedFirst += 1;
+      assert.ok(calls[1], "must fall through to a peer after the failing deck pick");
+      fallbackAfterFailure.add(calls[1]);
+    }
+  }
+
+  assert.ok(
+    failingWasPickedFirst >= 5,
+    `deck should pick the failing model first several times (saw ${failingWasPickedFirst})`
+  );
+  // Pre-fix: `rest` is fixed priority order, so the fallback after the failing
+  // pick is ALWAYS the same single model → set size 1 (RED).
+  // Post-fix: `rest` is shuffled → the fallback spreads across the healthy peers.
+  assert.ok(
+    fallbackAfterFailure.size >= 2,
+    `fallback after a failing deck pick must spread across peers, got only ${[
+      ...fallbackAfterFailure,
+    ].join(", ")}`
+  );
+});
+
+test("#3959 strict-random still reaches the one healthy target when others fail", async () => {
+  const models = ["openai/a", "claude/b", "gemini/c", "groq/d"];
+  const HEALTHY = "gemini/c";
+  const combo = {
+    name: "strict-random-single-healthy-3959",
+    strategy: "strict-random",
+    models,
+    config: { maxRetries: 0, retryDelayMs: 0, fallbackDelayMs: 0 },
+  };
+
+  for (let i = 0; i < 12; i += 1) {
+    const calls: string[] = [];
+    const result = await handleComboChat({
+      body: {},
+      combo,
+      handleSingleModel: async (_body: unknown, modelStr: string) => {
+        calls.push(modelStr);
+        return modelStr === HEALTHY ? okResponse() : errorResponse(503);
+      },
+      isModelAvailable: async () => true,
+      log: createLog(),
+      settings: null,
+      allCombos: null,
+    });
+    assert.equal(result.ok, true);
+    assert.equal(calls[calls.length - 1], HEALTHY, "must fall through until the healthy target");
+  }
+});