Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
15 changes: 8 additions & 7 deletions open-sse/config/providers/registry/theoldllm/index.ts
Original file line number Diff line number Diff line change
Expand Up @@ -10,15 +10,16 @@ export const theoldllmProvider: RegistryEntry = {
baseUrls: ["https://theoldllm.vercel.app/api/chatgpt"],
authType: "none",
authHeader: "none",
defaultContextLength: 200000,

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

high

Setting defaultContextLength to 200000 as a provider-wide fallback is unsafe. Standard models like GPT_4o (which has a maximum context window of 128k/131,072 tokens) will inherit this 200k limit, leading to potential context overflow errors when downstream clients attempt to send payloads up to 200k.\n\nAdditionally, because passthroughModels is enabled, any unknown models requested through this provider will also default to 200k, which exceeds the limits of many common models.\n\nIt is highly recommended to set defaultContextLength to 128000 (128k) as a safe fallback, and explicitly define the larger context windows on the specific models that support them.

Suggested change
defaultContextLength: 200000,
defaultContextLength: 128000,

models: [
{ id: "GPT_5_4", name: "GPT-5.4 (The Old LLM 🆓)" },
{ id: "GPT_5_4", name: "GPT-5.4 (The Old LLM 🆓)", contextLength: 400000 },

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

According to the Repository Style Guide (Rule 9), changing production code under open-sse/ requires including tests. Please ensure that appropriate unit or integration tests are added or updated in the tests/ directory to cover these new context length configurations.

References
  1. Always include tests when changing production code (src/, open-sse/, electron/, bin/).

{ id: "GPT_4o", name: "GPT-4o (The Old LLM 🆓)" },
{ id: "claude_opus_4", name: "Claude Opus 4 (The Old LLM 🆓)" },
{ id: "claude_sonnet_4", name: "Claude Sonnet 4 (The Old LLM 🆓)" },
{ id: "claude_haiku_3_5", name: "Claude Haiku 3.5 (The Old LLM 🆓)" },
{ id: "deepseek_v4", name: "DeepSeek V4 (The Old LLM 🆓)" },
{ id: "gemini_3_flash", name: "Gemini 3 Flash (The Old LLM 🆓)" },
{ id: "gemini_3_pro", name: "Gemini 3 Pro (The Old LLM 🆓)" },
{ id: "claude_opus_4", name: "Claude Opus 4 (The Old LLM 🆓)", contextLength: 200000 },
{ id: "claude_sonnet_4", name: "Claude Sonnet 4 (The Old LLM 🆓)", contextLength: 200000 },
{ id: "claude_haiku_3_5", name: "Claude Haiku 3.5 (The Old LLM 🆓)", contextLength: 200000 },
{ id: "deepseek_v4", name: "DeepSeek V4 (The Old LLM 🆓)", contextLength: 200000 },
{ id: "gemini_3_flash", name: "Gemini 3 Flash (The Old LLM 🆓)", contextLength: 1000000 },
{ id: "gemini_3_pro", name: "Gemini 3 Pro (The Old LLM 🆓)", contextLength: 1000000 },
],
passthroughModels: true,
};
57 changes: 57 additions & 0 deletions tests/unit/theoldllm-context-length-4184.test.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,57 @@
import test from "node:test";
import assert from "node:assert/strict";

// Regression guard for #4184.
//
// The theoldllm provider (free OpenAI-compatible upstream) listed its models
// with NO contextLength, so getResolvedModelCapabilities resolved their context
// window to `null` and the dashboard/catalog reported no usable window. #4184
// adds an entry-level `defaultContextLength` plus per-model `contextLength`
// overrides reflecting each upstream model's real window. This test asserts both
// the registry data (source of truth) and the resolved context window for the
// models that carry an explicit override — the latter would resolve to `null`
// on the pre-#4184 registry, so it fails without the fix.
const { getRegistryEntry } = await import("../../open-sse/config/providerRegistry.ts");
const { getResolvedModelCapabilities } = await import("../../src/lib/modelCapabilities.ts");

function model(id: string) {
const entry = getRegistryEntry("theoldllm");
assert.ok(entry, "theoldllm registry entry must exist");
return (entry.models ?? []).find((m) => m.id === id);
}

test("#4184 theoldllm entry declares a 200000 defaultContextLength", () => {
const entry = getRegistryEntry("theoldllm");
assert.ok(entry, "theoldllm registry entry must exist");
assert.equal(entry.defaultContextLength, 200000);
});

test("#4184 per-model contextLength overrides match each upstream window", () => {
assert.equal(model("GPT_5_4")?.contextLength, 400000, "GPT-5.4 window is 400K");
assert.equal(model("gemini_3_flash")?.contextLength, 1000000, "Gemini 3 Flash window is 1M");
assert.equal(model("gemini_3_pro")?.contextLength, 1000000, "Gemini 3 Pro window is 1M");
for (const id of ["claude_opus_4", "claude_sonnet_4", "claude_haiku_3_5", "deepseek_v4"]) {
assert.equal(model(id)?.contextLength, 200000, `${id} window is 200K`);
}
});

test("#4184 GPT_4o carries no explicit contextLength (relies on defaultContextLength)", () => {
// Intentionally left to the entry default — documents the fallback contract so a
// later edit that removes defaultContextLength is caught by the assertion above.
assert.equal(model("GPT_4o")?.contextLength, undefined);
});

test("#4184 resolved context window reflects the override (null before the fix)", () => {
assert.equal(
getResolvedModelCapabilities({ provider: "theoldllm", model: "GPT_5_4" }).contextWindow,
400000
);
assert.equal(
getResolvedModelCapabilities({ provider: "theoldllm", model: "gemini_3_pro" }).contextWindow,
1000000
);
assert.equal(
getResolvedModelCapabilities({ provider: "theoldllm", model: "claude_opus_4" }).contextWindow,
200000
);
});
Loading