Repository navigation
fix(provider-registry): add correct contextLength to a since-removed keyless provider's models #4184
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
diegosouzapw
merged 2 commits into
cheaper-inference:release/v3.8.29
from
herjarsa:fix/theoldllm-context-length
Jun 18, 2026
Merged
fix(provider-registry): add correct contextLength to a since-removed keyless provider's models #4184
Changes from all commits
Commits
File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -10,15 +10,16 @@ export const theoldllmProvider: RegistryEntry = { | |
| baseUrls: ["https://theoldllm.vercel.app/api/chatgpt"], | ||
| authType: "none", | ||
| authHeader: "none", | ||
| defaultContextLength: 200000, | ||
| models: [ | ||
| { id: "GPT_5_4", name: "GPT-5.4 (The Old LLM 🆓)" }, | ||
| { id: "GPT_5_4", name: "GPT-5.4 (The Old LLM 🆓)", contextLength: 400000 }, | ||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. According to the Repository Style Guide (Rule 9), changing production code under References
|
||
| { id: "GPT_4o", name: "GPT-4o (The Old LLM 🆓)" }, | ||
| { id: "claude_opus_4", name: "Claude Opus 4 (The Old LLM 🆓)" }, | ||
| { id: "claude_sonnet_4", name: "Claude Sonnet 4 (The Old LLM 🆓)" }, | ||
| { id: "claude_haiku_3_5", name: "Claude Haiku 3.5 (The Old LLM 🆓)" }, | ||
| { id: "deepseek_v4", name: "DeepSeek V4 (The Old LLM 🆓)" }, | ||
| { id: "gemini_3_flash", name: "Gemini 3 Flash (The Old LLM 🆓)" }, | ||
| { id: "gemini_3_pro", name: "Gemini 3 Pro (The Old LLM 🆓)" }, | ||
| { id: "claude_opus_4", name: "Claude Opus 4 (The Old LLM 🆓)", contextLength: 200000 }, | ||
| { id: "claude_sonnet_4", name: "Claude Sonnet 4 (The Old LLM 🆓)", contextLength: 200000 }, | ||
| { id: "claude_haiku_3_5", name: "Claude Haiku 3.5 (The Old LLM 🆓)", contextLength: 200000 }, | ||
| { id: "deepseek_v4", name: "DeepSeek V4 (The Old LLM 🆓)", contextLength: 200000 }, | ||
| { id: "gemini_3_flash", name: "Gemini 3 Flash (The Old LLM 🆓)", contextLength: 1000000 }, | ||
| { id: "gemini_3_pro", name: "Gemini 3 Pro (The Old LLM 🆓)", contextLength: 1000000 }, | ||
| ], | ||
| passthroughModels: true, | ||
| }; | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,57 @@ | ||
| import test from "node:test"; | ||
| import assert from "node:assert/strict"; | ||
|
|
||
| // Regression guard for #4184. | ||
| // | ||
| // The theoldllm provider (free OpenAI-compatible upstream) listed its models | ||
| // with NO contextLength, so getResolvedModelCapabilities resolved their context | ||
| // window to `null` and the dashboard/catalog reported no usable window. #4184 | ||
| // adds an entry-level `defaultContextLength` plus per-model `contextLength` | ||
| // overrides reflecting each upstream model's real window. This test asserts both | ||
| // the registry data (source of truth) and the resolved context window for the | ||
| // models that carry an explicit override — the latter would resolve to `null` | ||
| // on the pre-#4184 registry, so it fails without the fix. | ||
| const { getRegistryEntry } = await import("../../open-sse/config/providerRegistry.ts"); | ||
| const { getResolvedModelCapabilities } = await import("../../src/lib/modelCapabilities.ts"); | ||
|
|
||
| function model(id: string) { | ||
| const entry = getRegistryEntry("theoldllm"); | ||
| assert.ok(entry, "theoldllm registry entry must exist"); | ||
| return (entry.models ?? []).find((m) => m.id === id); | ||
| } | ||
|
|
||
| test("#4184 theoldllm entry declares a 200000 defaultContextLength", () => { | ||
| const entry = getRegistryEntry("theoldllm"); | ||
| assert.ok(entry, "theoldllm registry entry must exist"); | ||
| assert.equal(entry.defaultContextLength, 200000); | ||
| }); | ||
|
|
||
| test("#4184 per-model contextLength overrides match each upstream window", () => { | ||
| assert.equal(model("GPT_5_4")?.contextLength, 400000, "GPT-5.4 window is 400K"); | ||
| assert.equal(model("gemini_3_flash")?.contextLength, 1000000, "Gemini 3 Flash window is 1M"); | ||
| assert.equal(model("gemini_3_pro")?.contextLength, 1000000, "Gemini 3 Pro window is 1M"); | ||
| for (const id of ["claude_opus_4", "claude_sonnet_4", "claude_haiku_3_5", "deepseek_v4"]) { | ||
| assert.equal(model(id)?.contextLength, 200000, `${id} window is 200K`); | ||
| } | ||
| }); | ||
|
|
||
| test("#4184 GPT_4o carries no explicit contextLength (relies on defaultContextLength)", () => { | ||
| // Intentionally left to the entry default — documents the fallback contract so a | ||
| // later edit that removes defaultContextLength is caught by the assertion above. | ||
| assert.equal(model("GPT_4o")?.contextLength, undefined); | ||
| }); | ||
|
|
||
| test("#4184 resolved context window reflects the override (null before the fix)", () => { | ||
| assert.equal( | ||
| getResolvedModelCapabilities({ provider: "theoldllm", model: "GPT_5_4" }).contextWindow, | ||
| 400000 | ||
| ); | ||
| assert.equal( | ||
| getResolvedModelCapabilities({ provider: "theoldllm", model: "gemini_3_pro" }).contextWindow, | ||
| 1000000 | ||
| ); | ||
| assert.equal( | ||
| getResolvedModelCapabilities({ provider: "theoldllm", model: "claude_opus_4" }).contextWindow, | ||
| 200000 | ||
| ); | ||
| }); |
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Setting
defaultContextLengthto200000as a provider-wide fallback is unsafe. Standard models likeGPT_4o(which has a maximum context window of 128k/131,072 tokens) will inherit this 200k limit, leading to potential context overflow errors when downstream clients attempt to send payloads up to 200k.\n\nAdditionally, becausepassthroughModelsis enabled, any unknown models requested through this provider will also default to 200k, which exceeds the limits of many common models.\n\nIt is highly recommended to setdefaultContextLengthto128000(128k) as a safe fallback, and explicitly define the larger context windows on the specific models that support them.