-
Notifications
You must be signed in to change notification settings - Fork 5
V4 context window: state the 1M limit instead of inheriting a 200K fallback (TASK-393) #391
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Merged
Changes from all commits
Commits
File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,38 @@ | ||
| import { describe, expect, it, vi } from "vitest"; | ||
|
|
||
| // The ClawBox AI catalog is hardcoded rather than fetched, so nothing upstream | ||
| // will ever correct it: whatever these entries say is what every model picker | ||
| // on the device shows. They used to claim 128K — a number that was never V4's | ||
| // limit — and text+image, which the text-only proxy rejects. This pins them to | ||
| // the same values the provider definition writes into openclaw.json, so the | ||
| // picker and the gateway can't drift apart again. | ||
|
|
||
| vi.mock("child_process", () => ({ spawn: vi.fn() })); | ||
| vi.mock("@/lib/openclaw-config", () => ({ | ||
| findOpenclawBin: () => "openclaw", | ||
| openclawIsAbsent: () => false, | ||
| })); | ||
| vi.mock("@/lib/config-store", () => ({ DATA_DIR: "/tmp/clawbox-catalog-clawai-test" })); | ||
|
|
||
| import { CLAWAI_STATIC_MODELS } from "@/app/setup-api/ai-models/catalog/route"; | ||
|
|
||
| describe("ClawBox AI static catalog", () => { | ||
| it("offers exactly the two subscription tiers", () => { | ||
| expect(CLAWAI_STATIC_MODELS.map((m) => m.id)).toEqual([ | ||
| "deepseek-v4-flash", | ||
| "deepseek-v4-pro", | ||
| ]); | ||
| }); | ||
|
|
||
| it("reports V4's real 1M context window on both tiers", () => { | ||
| for (const model of CLAWAI_STATIC_MODELS) { | ||
| expect(model.contextWindow).toBe(1_000_000); | ||
| } | ||
| }); | ||
|
|
||
| it("declares text-only input, matching the proxy", () => { | ||
| for (const model of CLAWAI_STATIC_MODELS) { | ||
| expect(model.input).toBe("text"); | ||
| } | ||
| }); | ||
| }); |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,138 @@ | ||
| import { describe, it, expect, beforeEach, afterEach } from "vitest"; | ||
| import { mkdtempSync, rmSync, writeFileSync, readFileSync } from "node:fs"; | ||
| import { execFileSync, spawnSync } from "node:child_process"; | ||
| import { tmpdir } from "node:os"; | ||
| import path from "node:path"; | ||
|
|
||
| // A configured provider entry in openclaw.json overrides OpenClaw's bundled | ||
| // model catalog outright. A V4 model that omits contextWindow therefore does | ||
| // NOT inherit the canonical 1M window — it resolves to the generic 200,000 | ||
| // default. Confirmed on a real device on OpenClaw 2026.7.1 (2026-08-17): | ||
| // `openclaw models list` reported 200K with the field absent and 1M once it | ||
| // was written. Devices in the field are in one of three states (absent, an old | ||
| // explicit 128000, or a 200000 written back by an earlier run) and the boot | ||
| // migration has to fix all three without touching a cap someone set on purpose. | ||
| // | ||
| // These run the migration block out of the shipped .sh, not a copy of it, so | ||
| // the test fails if the real script drifts. | ||
|
|
||
| const SCRIPT = path.resolve(process.cwd(), "scripts/gateway-pre-start.sh"); | ||
| const hasPython3 = spawnSync("python3", ["--version"], { stdio: "ignore" }).status === 0; | ||
|
|
||
| /** Pull the DeepSeek V4 model-normalisation block out of the .sh verbatim. */ | ||
| function extractPolicy(): string { | ||
| const src = readFileSync(SCRIPT, "utf-8"); | ||
| const start = src.indexOf("if isinstance(ds_models, list):"); | ||
| const end = src.indexOf("if changed:", start); | ||
| if (start < 0 || end < 0) throw new Error("deepseek V4 model block not found"); | ||
| return src.slice(start, end); | ||
| } | ||
|
|
||
| const POLICY = hasPython3 ? extractPolicy() : ""; | ||
|
|
||
| let dir: string; | ||
| beforeEach(() => { dir = mkdtempSync(path.join(tmpdir(), "v4-context-")); }); | ||
| afterEach(() => { rmSync(dir, { recursive: true, force: true }); }); | ||
|
|
||
| type MigratedModel = { | ||
| id: string; | ||
| contextWindow?: number; | ||
| maxTokens?: number; | ||
| input?: string[]; | ||
| [key: string]: unknown; | ||
| }; | ||
|
|
||
| /** Run the extracted block over a model list and return the migrated models. */ | ||
| function migrate(models: Record<string, unknown>[]): MigratedModel[] { | ||
| const file = path.join(dir, "models.json"); | ||
| writeFileSync(file, JSON.stringify(models)); | ||
| const program = [ | ||
| "import json, sys", | ||
| "ds_models = json.load(open(sys.argv[1]))", | ||
| "changed = False", | ||
| POLICY, | ||
| "print(json.dumps({'models': ds_models, 'changed': changed}))", | ||
| ].join("\n"); | ||
| const out = JSON.parse(execFileSync("python3", ["-c", program, file], { encoding: "utf-8" }).trim()); | ||
| return out.models; | ||
| } | ||
|
|
||
| /** Same, but reporting whether the script decided a rewrite was needed. */ | ||
| function migrateChanged(models: Record<string, unknown>[]): boolean { | ||
| const file = path.join(dir, "models.json"); | ||
| writeFileSync(file, JSON.stringify(models)); | ||
| const program = [ | ||
| "import json, sys", | ||
| "ds_models = json.load(open(sys.argv[1]))", | ||
| "changed = False", | ||
| POLICY, | ||
| "print(json.dumps({'changed': changed}))", | ||
| ].join("\n"); | ||
| return JSON.parse(execFileSync("python3", ["-c", program, file], { encoding: "utf-8" }).trim()).changed; | ||
| } | ||
|
|
||
| const V4_IDS = ["deepseek-v4-flash", "deepseek-v4-pro"] as const; | ||
|
|
||
| describe.skipIf(!hasPython3)("gateway-pre-start.sh V4 context migration", () => { | ||
| it.each(V4_IDS)("fills an absent contextWindow with 1M on %s", (id) => { | ||
| const [m] = migrate([{ id, name: "ClawBox AI" }]); | ||
| expect(m.contextWindow).toBe(1_000_000); | ||
| expect(m.maxTokens).toBe(384_000); | ||
| expect(m.input).toEqual(["text"]); | ||
| }); | ||
|
|
||
| it.each(V4_IDS)("replaces the old explicit 128K on %s", (id) => { | ||
| const [m] = migrate([{ id, contextWindow: 128000 }]); | ||
| expect(m.contextWindow).toBe(1_000_000); | ||
| }); | ||
|
|
||
| it.each(V4_IDS)("replaces the 200K fallback written back by an earlier run on %s", (id) => { | ||
| const [m] = migrate([{ id, contextWindow: 200000 }]); | ||
| expect(m.contextWindow).toBe(1_000_000); | ||
| }); | ||
|
|
||
| it("migrates Flash and Pro in the same pass", () => { | ||
| const models = migrate([ | ||
| { id: "deepseek-v4-flash", contextWindow: 128000 }, | ||
| { id: "deepseek-v4-pro" }, | ||
| ]); | ||
| expect(models.map((m) => m.contextWindow)).toEqual([1_000_000, 1_000_000]); | ||
| }); | ||
|
|
||
| it("is idempotent — a second run reports no change", () => { | ||
| const once = migrate([{ id: "deepseek-v4-flash", contextWindow: 128000 }]); | ||
| expect(migrateChanged(once)).toBe(false); | ||
| }); | ||
|
|
||
| it("leaves a deliberate non-standard cap alone", () => { | ||
| // 32K is not a value we ever shipped, so someone chose it — probably to fit | ||
| // a smaller box. Overwriting it would be the migration overruling its owner. | ||
| const [m] = migrate([{ id: "deepseek-v4-flash", contextWindow: 32000, maxTokens: 8192 }]); | ||
| expect(m.contextWindow).toBe(32000); | ||
| expect(m.maxTokens).toBe(8192); | ||
| }); | ||
|
|
||
| it("does not touch models from other providers or other deepseek ids", () => { | ||
| const models = migrate([{ id: "deepseek-chat", contextWindow: 65536 }]); | ||
| expect(models[0].contextWindow).toBe(65536); | ||
| expect(models[0].maxTokens).toBeUndefined(); | ||
| expect(models[0].input).toBeUndefined(); | ||
| }); | ||
|
|
||
| it("preserves unrelated fields it did not come to change", () => { | ||
| const [m] = migrate([{ | ||
| id: "deepseek-v4-pro", | ||
| name: "ClawBox AI Pro", | ||
| reasoning: true, | ||
| cost: { input: 0, output: 0, cacheRead: 0, cacheWrite: 0 }, | ||
| }]); | ||
| expect(m.name).toBe("ClawBox AI Pro"); | ||
| expect(m.reasoning).toBe(true); | ||
| expect(m.cost).toEqual({ input: 0, output: 0, cacheRead: 0, cacheWrite: 0 }); | ||
| }); | ||
|
|
||
| it("replaces an input that is present but not a usable list", () => { | ||
| const [m] = migrate([{ id: "deepseek-v4-flash", input: [] }]); | ||
| expect(m.input).toEqual(["text"]); | ||
| }); | ||
| }); |
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
🗄️ Data Integrity & Integration | 🟠 Major | ⚡ Quick win
Preserve a
131072context-window value.The migration contract corrects only absent,
128000, and200000values. This predicate also rewrites131072. An operator who set a 131072-token cap will have that configuration changed to 1,000,000 on gateway startup. Remove131072from this migration set and add a regression test that preserves it.Proposed fix
📝 Committable suggestion
🤖 Prompt for AI Agents