Repository navigation
fix(catalog): make combos auto-compute context_length for any provider id form #3417
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
diegosouzapw
merged 6 commits into
cheaper-inference:release/v3.8.17
from
herjarsa:pr/backport-opencode-context-window
Jun 8, 2026
Merged
Changes from all commits
Commits
Show all changes
6 commits
Select commit
Hold shift + click to select a range
e6426ec
fix(opencode): emit limit.context AND limit.output for every catalog …
herjarsa d01a113
fix(catalog): make combos auto-compute context for any provider id form
herjarsa 60750ca
docs(opencode): add operator runbook for the context-window fix
herjarsa f31eb55
test: e2e runtime test for opencode-zen alias fallback in catalog chain
03001a5
style(catalog): address 5 medium-priority gemini-code-assist review c…
herjarsa 65df402
docs(env): document OMNIROUTE_URL/KEY and OPENCODE_API_KEY for regen-…
herjarsa File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,108 @@ | ||
| # Fix: OpenCode combo context window detection | ||
|
|
||
| ## Bug | ||
|
|
||
| OpenCode v1's `opencode.json` requires an explicit `limit.context` (and | ||
| `limit.output`) for every model. Without these fields, OpenCode's heuristic | ||
| kicks in and reports a wrong context window. | ||
|
|
||
| The OmniRoute `/v1/models` catalog is the single source of truth for context | ||
| windows. The bug was that some combos were published to the catalog WITHOUT | ||
| a computed `context_length`, so any OpenCode client pulling the catalog got | ||
| no answer for them. | ||
|
|
||
| ## Root cause | ||
|
|
||
| The "Opencode FREE Omni" combo references four `opencode/<model>` targets | ||
| (`big-pickle`, `deepseek-v4-flash-free`, `minimax-m3-free`, | ||
| `nemotron-3-super-free`). The catalog's `buildComboCatalogMetadata` computes | ||
| the combo's `context_length` as the **minimum of its targets' known | ||
| contexts**, but every target's lookup was returning `null`. | ||
|
|
||
| The lookup chain in `getCanonicalModelMetadata`: | ||
|
|
||
| 1. `getSyncedCapability("opencode", "big-pickle")` → returns `null` because | ||
| the DB row is stored under `provider = "opencode-zen"`, not `"opencode"`. | ||
| 2. `getRegistryModel("opencode", "big-pickle")` → returns `null` because | ||
| `PROVIDER_MODELS["opencode"]` (and `["oc"]`) don't have static entries | ||
| for these models. | ||
| 3. `getModelSpec("big-pickle")` → returns `null` (no static spec). | ||
|
|
||
| All three lookups fail → metadata is `null` → combo has no context. | ||
|
|
||
| Why was the DB row under `"opencode-zen"` and not `"opencode"`? | ||
| `MODELS_DEV_PROVIDER_MAP["opencode"]` was `["opencode-zen"]` only — a | ||
| historical one-way mapping. New syncs continued to write under the alias | ||
| side of the pair, while the catalog & combo targets use the canonical id | ||
| side. | ||
|
|
||
| ## Fix (3 layers, in this order) | ||
|
|
||
| ### 1. `src/lib/modelsDevSync.ts` — symmetric `mapProviderId` mapping | ||
|
|
||
| ```ts | ||
| // Before: | ||
| opencode: ["opencode-zen"], | ||
| "opencode-go": ["opencode-go"], | ||
|
|
||
| // After: | ||
| opencode: ["opencode", "opencode-zen"], | ||
| "opencode-go": ["opencode-go", "opencode-zen"], | ||
| ``` | ||
|
|
||
| Now models.dev data lands under BOTH the canonical id and the historical | ||
| alias, so any future sync keeps the lookup paths in sync. | ||
|
|
||
| ### 2. `src/lib/modelsDevSync.ts` — alias-aware fallback in `getSyncedCapability` | ||
|
|
||
| The runtime fix that takes effect **immediately**, without waiting for a | ||
| re-sync. Existing DB rows under `"opencode-zen"` are now found when | ||
| callers pass `"opencode"` (or vice-versa). | ||
|
|
||
| ```ts | ||
| const SYNCED_CAPABILITY_FALLBACK_ALIASES: Record<string, string[]> = { | ||
| opencode: ["opencode-zen"], | ||
| "opencode-zen": ["opencode"], | ||
| "opencode-go": ["opencode-zen"], | ||
| }; | ||
| ``` | ||
|
|
||
| ### 3. `src/lib/cli-helper/config-generator/opencode.ts` — drop the hardcoded 128K fallback | ||
|
|
||
| The previous band-aid hardcoded `FALLBACK_CONTEXT_LENGTH = 128_000` for | ||
| models whose context was unknown. That's wrong: combos like "Opencode | ||
| FREE Omni" should report **200K** (the min of their 200K targets), not | ||
| the universal 128K default. | ||
|
|
||
| The generator now: | ||
|
|
||
| - Uses the catalog as the **single source of truth** for `limit.context`. | ||
| - Emits the model **without** `limit.context` if the catalog has no entry | ||
| — OpenCode's own heuristic applies and the user can fix the upstream. | ||
| - **Throws** if the catalog fetch fails outright — the CLI catches and | ||
| surfaces the error. We never silently write a stale opencode.json. | ||
|
|
||
| ## Deployment steps (for the operator) | ||
|
|
||
| 1. Pull the latest from the branch: | ||
| `git fetch && git checkout fix/opencode-context-window` | ||
| 2. Rebuild OmniRoute: `npm run build` | ||
| 3. Restart the OmniRoute server (kill the running process and re-run). | ||
| 4. Trigger a models.dev sync from the Settings → Models.dev panel | ||
| (or POST `/api/settings/models-dev` with `{"action": "sync"}`). | ||
| 5. Re-run the opencode.json generator (the CLI command or the | ||
| `scripts/regen-opencode-config.ts` script). | ||
|
|
||
| After step 5, `Opencode FREE Omni`'s `limit.context` will be **200000** | ||
| and every other combo will reflect its targets' min context. | ||
|
|
||
| ## Verification | ||
|
|
||
| After the rebuild, hit `GET /v1/models` and inspect the response: | ||
|
|
||
| ```bash | ||
| curl -s http://localhost:20128/v1/models \ | ||
| -H "Authorization: Bearer $API_KEY" \ | ||
| | jq '.data[] | select(.id == "Opencode FREE Omni") | .context_length' | ||
| # → 200000 | ||
| ``` |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,23 @@ | ||
| /** | ||
| * One-shot regen of ~/.config/opencode/opencode.json from the live | ||
| * OmniRoute /v1/models catalog. Run after a catalog change to refresh | ||
| * the opencode client. | ||
| * | ||
| * Usage: bun run scripts/regen-opencode-config.ts | ||
| * or npx tsx scripts/regen-opencode-config.ts | ||
| */ | ||
| import { generateOpencodeConfig } from "../src/lib/cli-helper/config-generator/opencode.ts"; | ||
|
|
||
| const baseURL = process.env.OMNIROUTE_URL ?? "http://localhost:20128"; | ||
| const apiKey = process.env.OMNIROUTE_KEY ?? process.env.OPENCODE_API_KEY ?? ""; | ||
|
|
||
| if (!apiKey) { | ||
| console.error( | ||
| "OMNIROUTE_KEY (or OPENCODE_API_KEY) env var is required. " + | ||
| "Find it in OmniRoute dashboard → Settings → API Keys." | ||
| ); | ||
| process.exit(1); | ||
| } | ||
|
|
||
| const out = await generateOpencodeConfig({ baseUrl: baseURL, apiKey }); | ||
| console.log(out); | ||
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
According to the Repository Style Guide (Section 1: File Placement & Organization), all maintenance, debugging, generation, or experimental scripts must be placed strictly inside one of the
scripts/subfolders (such asscripts/ad-hoc/orscripts/dev/). Loose scripts must never be dumped in the top-levelscripts/folder. Please move this script toscripts/ad-hoc/regen-opencode-config.ts.References