fix(providers): fetch live /models for LLM7 and BytePlus (#3976)

CHANGELOG.md

@@ -23,6 +23,7 @@
 
 ### 🐛 Fixed
 
+- **fix(providers): LLM7 (and BytePlus) now fetch the live `/models` catalog instead of a stale hardcoded list** — importing an LLM7 key surfaced only a small, outdated model list even though `GET https://api.llm7.io/v1/models` returns the full pro/standard catalog. Both providers carried a correct `modelsUrl` in the registry, but neither was classified by any live-fetch branch of the model-import route (not `openai-compatible-*`, not self-hosted, not in `NAMED_OPENAI_STYLE_PROVIDERS`), so the route skipped the upstream probe and served the registry's 4 hardcoded entries (`source: "local_catalog"`). Added `llm7` and `byteplus` to `NAMED_OPENAI_STYLE_PROVIDERS` so the route probes `<baseUrl>/models` with the key and serves the live catalog, falling back to the local catalog only when the upstream fetch fails (so key import never breaks). ([#3976](https://github.com/diegosouzapw/OmniRoute/issues/3976) — thanks @FerLuisxd)
 - **fix(resilience): respect connection cooldown stored as a numeric epoch (router kept hammering 429 accounts)** — the router kept dispatching to connections still inside their rate-limit cooldown, causing client timeouts and "connection cooldown isn't respected" reports. Root cause: `rate_limited_until` is a `TEXT` column, but the Antigravity full-quota path (`setConnectionRateLimitUntil`) persists a raw epoch **number**, which SQLite coerces to a numeric string like `"1781696905131.0"`. The account-selection predicate then did `new Date("1781696905131.0")` → `Invalid Date` → `NaN`, so `NaN > Date.now()` was false and the cooling connection was never skipped. The cooldown read predicates (`isAccountUnavailable`, `getEarliestRateLimitedUntil`, `filterAvailableAccounts`, `parseFutureDateMs`) now normalize numeric-epoch strings as well as ISO strings/Date/number via a shared `cooldownUntilMs()` helper — ISO behavior is unchanged. ([#3954](https://github.com/diegosouzapw/OmniRoute/issues/3954))
 - **fix(compression/memory): stop memory + compression from poisoning the upstream prompt cache** — with compression and/or memory enabled, requests to caching providers (Anthropic-family) missed the prompt cache on every turn, multiplying cost. Two root causes: (1) memory injection prepended the retrieved memories — which **vary per user query** — at index 0 of the message array, shifting the entire cacheable prefix every turn; memory is now inserted just before the last user message when the request carries `cache_control` breakpoints, keeping the cacheable prefix (system prompt + prior turns) byte-stable. (2) the cache-aware `skipSystemPrompt` flag computed by `getCacheAwareStrategy()` was dropped by `selectCompressionStrategy()` (which can only return a mode), so the system prompt could still be compressed under caching; a new `resolveCacheAwareConfig()` now forces `preserveSystemPrompt` on for caching requests. ([#3936](https://github.com/diegosouzapw/OmniRoute/pull/3936), closes [#3890](https://github.com/diegosouzapw/OmniRoute/issues/3890) — thanks @xenstar / @diegosouzapw)
 - **fix(providers): register BytePlus ModelArk so its API key can be added** — adding a BytePlus (`ark-…`) key reported "invalid". `byteplus` was present in the provider catalog (`APIKEY_PROVIDERS`) but **never registered in the routing registry**, so key validation fell through to `{ unsupported: true }` → HTTP 400 → the UI rendered every key as invalid (and the provider was unusable for inference). Added a registry entry modeled on the existing Volcengine Ark provider: OpenAI-compatible format, base `https://ark.ap-southeast.bytepluses.com/api/v3` (region `ap-southeast-1`), `Authorization: Bearer` auth, seeded with the catalog's advertised models (Seed 2.0, Kimi K2 Thinking, GLM 4.7, GPT-OSS-120B). ([#3935](https://github.com/diegosouzapw/OmniRoute/pull/3935), closes [#3877](https://github.com/diegosouzapw/OmniRoute/issues/3877) — thanks @nikohd12 / @diegosouzapw)

config/quality/file-size-baseline.json

@@ -15,6 +15,7 @@
   "_rebaseline_2026_06_15_3929_vertex_media": "PR #3929 own growth: audioSpeech.ts 952->965 (+13) and videoGeneration.ts 1026->1078 (+52) = vertex/* media branches (Gemini TTS, Veo predictLongRunning poll) wired into the speech/video handlers; new logic lives in open-sse/executors/vertexMedia.ts (341, under cap). Cohesive media-provider feature.",
   "_rebaseline_2026_06_15_3879_redact_thinking": "PR #3879 + #3921 reconcile: AddApiKeyModal.tsx 843->845 (+2 = merging #3879's CcCompatibleRequestDefaultsFields (context1m + opt-in redact-thinking toggle) into #3921's preset-input block in the cc-compatible settings group). Cohesive UI; not extractable.",
   "_rebaseline_2026_06_15_3890_cache_preserve": "Issue #3890 own growth: chatCore.ts 5815->5823 (+8 = wire resolveCacheAwareConfig() into the compression apply step so the system prompt is never compressed in a caching context — honors the cache-aware skipSystemPrompt flag that selectCompressionStrategy could not carry). Cohesive cache-preservation guard at the existing compression chokepoint; not extractable.",
+  "_rebaseline_2026_06_16_3976_llm7_byteplus_models": "Issue #3976 own growth: models/route.ts 2489->2494 (+5 = add llm7 + byteplus to NAMED_OPENAI_STYLE_PROVIDERS with an explanatory comment so the import route does a live <baseUrl>/models fetch instead of serving the stale hardcoded registry catalog). Structural shrink of this route tracked in #3789.",
   "_rebaseline_2026_06_16_3954_cooldown_epoch": "Issue #3954 own growth: accountFallback.ts 1708->1727 (+19 = a shared cooldownUntilMs() normalizer + its use in isAccountUnavailable/getEarliestRateLimitedUntil/filterAvailableAccounts so a rate_limited_until persisted as a numeric-epoch string is honored, not parsed to NaN) and auth.ts 2216->2219 (+3 = parseFutureDateMs reuses cooldownUntilMs). Cohesive cooldown read-path hardening at the existing chokepoints; one helper, not extractable.",
   "_rebaseline_2026_06_15_3938_perplexity_v218": "PR #3938 own growth: perplexity-web.ts 868->939 (+71 = rebuild buildPplxRequestBody to mirror the current www.perplexity.ai schematized request body — version 2.18, use_schematized_api + the full supported_block_use_cases list, dsl_query, shared requestId for frontend_uuid/client_search_results_cache_key, last_backend_uuid only on follow-ups — plus the x-perplexity-request-* / x-request-id headers replacing the stale X-App-ApiVersion pair that triggered HTTP 400). Cohesive upstream-schema sync in a single executor; not extractable.",
   "cap": 800,
@@ -89,7 +90,7 @@
     "src/app/(dashboard)/dashboard/usage/components/EvalsTab.tsx": 2148,
     "src/app/(dashboard)/dashboard/usage/components/ProviderLimits/index.tsx": 1069,
     "src/app/api/oauth/[provider]/[action]/route.ts": 918,
-    "src/app/api/providers/[id]/models/route.ts": 2489,
+    "src/app/api/providers/[id]/models/route.ts": 2494,
     "src/app/api/providers/[id]/test/route.ts": 842,
     "src/app/api/usage/analytics/route.ts": 941,
     "src/app/api/v1/models/catalog.ts": 1435,

src/app/api/providers/[id]/models/route.ts

@@ -141,6 +141,11 @@ const NAMED_OPENAI_STYLE_PROVIDERS = new Set([
   "nous-research",
   "poe",
   "siliconflow",
+  // #3976: these carry a real modelsUrl but were not classified by any live-fetch
+  // branch, so their hardcoded registry catalog was served instead of the live
+  // `<baseUrl>/models` list. Live fetch falls back to the local catalog on error.
+  "llm7",
+  "byteplus",
 ]);
 
 function isNamedOpenAIStyleProvider(provider: string): boolean {

tests/unit/llm7-byteplus-models-fetch-3976.test.ts

@@ -0,0 +1,155 @@
+/**
+ * TDD regression for #3976: LLM7 (and BytePlus) `GET /models` returned a stale
+ * hardcoded list instead of the live catalog.
+ *
+ * Root cause: `llm7`/`byteplus` carry a correct `modelsUrl` in the registry, but
+ * neither is classified by any live-fetch branch of the import route — not
+ * `openai-compatible-*`, not self-hosted, and not in NAMED_OPENAI_STYLE_PROVIDERS.
+ * So the route never probes the upstream `/models` and falls through to the
+ * registry's hardcoded `models[]` (4 entries), reported as `source:"local_catalog"`.
+ *
+ * Fix: add `llm7` and `byteplus` to NAMED_OPENAI_STYLE_PROVIDERS so the route
+ * does a live `<baseUrl>/models` fetch (falling back to the local catalog only
+ * when the upstream fetch fails, so import never breaks).
+ */
+import test from "node:test";
+import assert from "node:assert/strict";
+import fs from "node:fs";
+import os from "node:os";
+import path from "node:path";
+
+const TEST_DATA_DIR = fs.mkdtempSync(path.join(os.tmpdir(), "omniroute-3976-"));
+process.env.DATA_DIR = TEST_DATA_DIR;
+
+const core = await import("../../src/lib/db/core.ts");
+const providersDb = await import("../../src/lib/db/providers.ts");
+const modelsRoute = await import("../../src/app/api/providers/[id]/models/route.ts");
+
+async function resetStorage() {
+  core.resetDbInstance();
+  fs.rmSync(TEST_DATA_DIR, { recursive: true, force: true });
+  fs.mkdirSync(TEST_DATA_DIR, { recursive: true });
+}
+
+test.after(() => {
+  core.resetDbInstance();
+  fs.rmSync(TEST_DATA_DIR, { recursive: true, force: true });
+});
+
+interface ModelsBody {
+  provider: string;
+  connectionId: string;
+  models: Array<{ id: string }>;
+  source?: string;
+}
+
+test("#3976 LLM7 import fetches the live /v1/models catalog (not the 4 hardcoded models)", async () => {
+  await resetStorage();
+  const connection = await providersDb.createProviderConnection({
+    provider: "llm7",
+    authType: "apikey",
+    name: "llm7-live",
+    apiKey: "llm7-key",
+  });
+
+  let fetched = false;
+  const originalFetch = globalThis.fetch;
+  globalThis.fetch = async (url) => {
+    if (String(url) === "https://api.llm7.io/v1/models") {
+      fetched = true;
+      return Response.json({
+        object: "list",
+        data: [
+          { id: "gpt-5.1-nano-pro" },
+          { id: "deepseek-v4-standard" },
+          { id: "qwen3.6-coder-pro" },
+        ],
+      });
+    }
+    // Bogus probe variants (…/v1/v1/models, …/chat/completions/models) → 404
+    return new Response("not found", { status: 404 });
+  };
+
+  try {
+    const response = await modelsRoute.GET(
+      new Request(`http://localhost/api/providers/${connection.id}/models?refresh=true`),
+      { params: { id: connection.id } }
+    );
+    assert.equal(response.status, 200);
+    const body = (await response.json()) as ModelsBody;
+    assert.equal(body.provider, "llm7");
+    assert.equal(body.source, "api", "should serve the live upstream catalog, not local_catalog");
+    assert.ok(fetched, "should have probed https://api.llm7.io/v1/models");
+    const ids = body.models.map((m) => m.id);
+    assert.ok(ids.includes("gpt-5.1-nano-pro"), `live ids missing: ${ids.join(",")}`);
+    // The stale hardcoded entries must not be what we serve.
+    assert.ok(!ids.includes("gpt-4o-mini-2024-07-18"), "served stale hardcoded catalog");
+  } finally {
+    globalThis.fetch = originalFetch;
+  }
+});
+
+test("#3976 LLM7 import falls back to the local catalog when the live fetch fails", async () => {
+  await resetStorage();
+  const connection = await providersDb.createProviderConnection({
+    provider: "llm7",
+    authType: "apikey",
+    name: "llm7-fallback",
+    apiKey: "llm7-key-2",
+  });
+
+  const originalFetch = globalThis.fetch;
+  globalThis.fetch = async () => new Response("bad gateway", { status: 502 });
+
+  try {
+    const response = await modelsRoute.GET(
+      new Request(`http://localhost/api/providers/${connection.id}/models?refresh=true`),
+      { params: { id: connection.id } }
+    );
+    assert.equal(response.status, 200);
+    const body = (await response.json()) as ModelsBody;
+    assert.equal(body.provider, "llm7");
+    assert.equal(body.source, "local_catalog", "import must not break when upstream is down");
+    assert.ok(body.models.length > 0, "fallback catalog should be non-empty");
+  } finally {
+    globalThis.fetch = originalFetch;
+  }
+});
+
+test("#3976 BytePlus import fetches the live /api/v3/models catalog", async () => {
+  await resetStorage();
+  const connection = await providersDb.createProviderConnection({
+    provider: "byteplus",
+    authType: "apikey",
+    name: "byteplus-live",
+    apiKey: "ark-key",
+  });
+
+  let fetched = false;
+  const originalFetch = globalThis.fetch;
+  globalThis.fetch = async (url) => {
+    if (String(url) === "https://ark.ap-southeast.bytepluses.com/api/v3/models") {
+      fetched = true;
+      return Response.json({
+        object: "list",
+        data: [{ id: "seed-2.5-live" }, { id: "kimi-k2.5-live" }],
+      });
+    }
+    return new Response("not found", { status: 404 });
+  };
+
+  try {
+    const response = await modelsRoute.GET(
+      new Request(`http://localhost/api/providers/${connection.id}/models?refresh=true`),
+      { params: { id: connection.id } }
+    );
+    assert.equal(response.status, 200);
+    const body = (await response.json()) as ModelsBody;
+    assert.equal(body.provider, "byteplus");
+    assert.equal(body.source, "api");
+    assert.ok(fetched, "should have probed https://ark.ap-southeast.bytepluses.com/api/v3/models");
+    assert.ok(body.models.map((m) => m.id).includes("seed-2.5-live"));
+  } finally {
+    globalThis.fetch = originalFetch;
+  }
+});