feat(auto-combo): add auto-updating model intelligence scoring - #3660
Merged
diegosouzapw merged 6 commits intoJun 12, 2026
Merged
diegosouzapw merged 6 commits into
diegosouzapw merged 6 commits into
Conversation
Add a sync pipeline that fetches Arena AI leaderboard ELO scores and derives model intelligence tiers from models.dev capabilities, replacing the hardcoded-only taskFitness lookup with a 5-layer resolution chain: 1. User override (DB) — manual per-model overrides 2. Arena ELO (DB) — auto-synced from wulong.dev leaderboard API 3. Models.dev tier — derived from capability data (reasoning, tool_call) 4. Static FITNESS_TABLE — existing hardcoded lookup (preserved) 5. Wildcard boosts — pattern matching (preserved) New files: - src/lib/db/migrations/097_model_intelligence.sql — DB table - src/lib/db/modelIntelligence.ts — domain module (CRUD, resolution) - src/lib/arenaEloSync.ts — periodic sync from Arena leaderboard API - src/app/api/intelligence/sync/route.ts — POST/GET/DELETE API routes - tests/unit/model-intelligence-db.test.ts — DB module tests - tests/unit/arena-elo-sync.test.ts — sync module tests Modified: - taskFitness.ts — rewritten with resolution chain + in-memory cache - autoCombo.test.ts — added resolution chain test block - schemas.ts — added intelligenceSyncRequestSchema - server-init.ts — wired Arena ELO sync init - localDb.ts — added re-exports Opt-in via ARENA_ELO_SYNC_ENABLED=true env var.
![gemini-code-assist[bot]](https://avatars.githubusercontent.com/in/956858?s=60&v=4){kind=small}
Contributor
There was a problem hiding this comment.
Code Review
This pull request introduces a model intelligence and task-fitness resolution system, including a new database table, a sync engine to fetch and normalize Arena AI leaderboard ELO scores, and an API route for manual synchronization. Feedback on these changes highlights critical issues with SQLite datetime comparisons due to ISO8601 string mismatches, transient database errors being permanently cached, and potential API crashes when handling empty POST request bodies. Additionally, the reviewer recommended refactoring duplicated resolution logic and introducing concurrency guards to prevent overlapping sync operations.
Important
The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.
| `SELECT * FROM model_intelligence | ||
| WHERE model = ? AND category = ? | ||
| AND source IN ('user_override', 'arena_elo', 'models_dev_tier') | ||
| AND (expires_at IS NULL OR expires_at > datetime('now')) |
Contributor
There was a problem hiding this comment.
Comparing expires_at (stored as an ISO8601 string with T and Z like 2025-10-27T15:30:00.000Z) directly against SQLite's datetime('now') (which returns a space-separated string like 2025-10-27 15:30:00) is broken. In SQLite, string comparison is done character-by-character. Since 'T' (ASCII 84) is greater than ' ' (ASCII 32), any ISO8601 string will evaluate as greater than the space-separated datetime string for the same date, meaning expired entries will incorrectly be treated as active. Wrapping expires_at in datetime() normalizes it to the space-separated format, making the comparison correct.
Suggested change
| AND (expires_at IS NULL OR expires_at > datetime('now')) | |
| AND (expires_at IS NULL OR datetime(expires_at) > datetime('now')) |
| .prepare( | ||
| `SELECT * FROM model_intelligence | ||
| WHERE model = ? AND source = ? AND category = ? | ||
| AND (expires_at IS NULL OR expires_at > datetime('now'))` |
Contributor
There was a problem hiding this comment.
Comparing expires_at (stored as an ISO8601 string with T and Z like 2025-10-27T15:30:00.000Z) directly against SQLite's datetime('now') (which returns a space-separated string like 2025-10-27 15:30:00) is broken. In SQLite, string comparison is done character-by-character. Since 'T' (ASCII 84) is greater than ' ' (ASCII 32), any ISO8601 string will evaluate as greater than the space-separated datetime string for the same date, meaning expired entries will incorrectly be treated as active. Wrapping expires_at in datetime() normalizes it to the space-separated format, making the comparison correct.
AND (expires_at IS NULL OR datetime(expires_at) > datetime('now'))|
|
||
| export function deleteExpiredIntelligence(source?: string): number { | ||
| const db = getDbInstance(); | ||
| const conditions = ["expires_at IS NOT NULL", "expires_at < datetime('now')"]; |
Contributor
There was a problem hiding this comment.
Because of the string format mismatch between ISO8601 and SQLite's space-separated datetime, expires_at < datetime('now') will evaluate to false for expired entries, meaning deleteExpiredIntelligence will never actually delete any expired entries. Wrapping expires_at in datetime() normalizes the format and fixes this issue.
Suggested change
| const conditions = ["expires_at IS NOT NULL", "expires_at < datetime('now')"]; | |
| const conditions = ["expires_at IS NOT NULL", "datetime(expires_at) < datetime('now')"]; |
Comment on lines
+212
to
+220
| try { | ||
| const entry = getModelIntelligenceBySource(model, source, category); | ||
| const score = entry ? entry.score : null; | ||
| _intelligenceCache.set(cacheKey, score); | ||
| return score; | ||
| } catch { | ||
| _intelligenceCache.set(cacheKey, null); | ||
| return null; | ||
| } |
Contributor
There was a problem hiding this comment.
In the catch block of queryModelIntelligence, if getModelIntelligenceBySource throws an error (e.g., due to a transient database lock or initialization delay), the function caches null for that key. This permanently prevents the system from querying the database again for that model/category/source combination until the cache is invalidated, turning a transient error into a permanent failure. It is safer to avoid caching the result when an exception is thrown.
try {
const entry = getModelIntelligenceBySource(model, source, category);
const score = entry ? entry.score : null;
_intelligenceCache.set(cacheKey, score);
return score;
} catch {
return null;
}
Comment on lines
+331
to
+359
| export function getTaskFitness(model: string, taskType: string): number { | ||
| const normalizedModel = model.toLowerCase(); | ||
| const normalizedTask = taskType.toLowerCase(); | ||
|
|
||
| const userOverride = queryModelIntelligence( | ||
| normalizedModel, | ||
| normalizedTask, | ||
| "user_override", | ||
| ); | ||
| if (userOverride !== null) return userOverride; | ||
|
|
||
| const arenaElo = queryModelIntelligence( | ||
| normalizedModel, | ||
| normalizedTask, | ||
| "arena_elo", | ||
| ); | ||
| if (arenaElo !== null) return arenaElo; | ||
|
|
||
| const tierScore = getModelsDevTierFitness(normalizedModel, normalizedTask); | ||
| if (tierScore !== null) return tierScore; | ||
|
|
||
| const staticScore = lookupStaticFitnessTable( | ||
| normalizedModel, | ||
| normalizedTask, | ||
| ); | ||
| if (staticScore !== null) return staticScore; | ||
|
|
||
| return lookupWildcardBoosts(normalizedModel, normalizedTask); | ||
| } |
Contributor
There was a problem hiding this comment.
getTaskFitness and getTaskFitnessWithSource share the exact same resolution chain logic. To improve maintainability and ensure the resolution chain remains consistent, getTaskFitness should be refactored to delegate directly to getTaskFitnessWithSource.
export function getTaskFitness(model: string, taskType: string): number {
return getTaskFitnessWithSource(model, taskType).score;
}
Comment on lines
+19
to
+32
| let rawBody: unknown; | ||
| try { | ||
| rawBody = await request.json(); | ||
| } catch { | ||
| return NextResponse.json( | ||
| { | ||
| error: { | ||
| message: "Invalid request", | ||
| details: [{ field: "body", message: "Invalid JSON body" }], | ||
| }, | ||
| }, | ||
| { status: 400 } | ||
| ); | ||
| } |
Contributor
There was a problem hiding this comment.
Calling request.json() directly will throw an error and return a 400 Bad Request if the client sends a POST request with an empty body or no body at all. Since all fields in intelligenceSyncRequestSchema are optional, the API should gracefully handle empty or missing bodies by defaulting to an empty object.
let rawBody: unknown = {};
const text = await request.text();
if (text) {
try {
rawBody = JSON.parse(text);
} catch {
return NextResponse.json(
{
error: {
message: "Invalid request",
details: [{ field: "body", message: "Invalid JSON body" }],
},
},
{ status: 400 }
);
}
}
Comment on lines
+355
to
+420
| export async function syncArenaElo(dryRun = false): Promise<SyncResult> { | ||
| try { | ||
| // Backup DB before first sync (same pattern as pricingSync) | ||
| if (!firstSyncDone && !dryRun) { | ||
| backupDbFile("pre-arena-elo-sync"); | ||
| firstSyncDone = true; | ||
| } | ||
|
|
||
| // Clean up stale entries before writing new ones | ||
| if (!dryRun) { | ||
| try { | ||
| deleteExpiredIntelligence(); | ||
| } catch (err) { | ||
| const message = err instanceof Error ? err.message : String(err); | ||
| console.warn( | ||
| `[ARENA_ELO_SYNC] Failed to delete expired intelligence: ${message}` | ||
| ); | ||
| } | ||
| } | ||
|
|
||
| const leaderboards = await fetchArenaLeaderboards(); | ||
| const entries = transformToModelIntelligence(leaderboards); | ||
|
|
||
| if (!dryRun && entries.length > 0) { | ||
| try { | ||
| bulkUpsertModelIntelligence(entries); | ||
| } catch (err) { | ||
| const message = err instanceof Error ? err.message : String(err); | ||
| console.warn( | ||
| `[ARENA_ELO_SYNC] Failed to bulk upsert intelligence: ${message}` | ||
| ); | ||
| return { | ||
| success: false, | ||
| modelCount: 0, | ||
| source: "arena_elo", | ||
| error: message, | ||
| }; | ||
| } | ||
| } | ||
|
|
||
| if (!dryRun) { | ||
| lastSyncTime = new Date().toISOString(); | ||
| lastSyncModelCount = entries.length; | ||
| } | ||
|
|
||
| const countLabel = dryRun ? "would sync" : "synced"; | ||
| console.log( | ||
| `[ARENA_ELO_SYNC] ${countLabel} ${entries.length} model intelligence entries from Arena leaderboards` | ||
| ); | ||
|
|
||
| return { | ||
| success: true, | ||
| modelCount: entries.length, | ||
| source: "arena_elo", | ||
| }; | ||
| } catch (err) { | ||
| const message = err instanceof Error ? err.message : String(err); | ||
| console.warn("[ARENA_ELO_SYNC] Sync failed:", message); | ||
| return { | ||
| success: false, | ||
| modelCount: 0, | ||
| source: "arena_elo", | ||
| error: message, | ||
| }; | ||
| } | ||
| } |
Contributor
There was a problem hiding this comment.
syncArenaElo can be triggered concurrently (e.g., via manual API calls overlapping with the periodic sync interval). Running multiple sync operations simultaneously can lead to database write conflicts, SQLite lock errors, and redundant API requests. Introducing a simple isSyncing boolean flag will prevent concurrent executions.
let isSyncing = false;
export async function syncArenaElo(dryRun = false): Promise<SyncResult> {
if (isSyncing) {
return {
success: false,
modelCount: 0,
source: "arena_elo",
error: "Sync already in progress",
};
}
isSyncing = true;
try {
// Backup DB before first sync (same pattern as pricingSync)
if (!firstSyncDone && !dryRun) {
backupDbFile("pre-arena-elo-sync");
firstSyncDone = true;
}
// Clean up stale entries before writing new ones
if (!dryRun) {
try {
deleteExpiredIntelligence();
} catch (err) {
const message = err instanceof Error ? err.message : String(err);
console.warn(
`[ARENA_ELO_SYNC] Failed to delete expired intelligence: ${message}`
);
}
}
const leaderboards = await fetchArenaLeaderboards();
const entries = transformToModelIntelligence(leaderboards);
if (!dryRun && entries.length > 0) {
try {
bulkUpsertModelIntelligence(entries);
} catch (err) {
const message = err instanceof Error ? err.message : String(err);
console.warn(
`[ARENA_ELO_SYNC] Failed to bulk upsert intelligence: ${message}`
);
return {
success: false,
modelCount: 0,
source: "arena_elo",
error: message,
};
}
}
if (!dryRun) {
lastSyncTime = new Date().toISOString();
lastSyncModelCount = entries.length;
}
const countLabel = dryRun ? "would sync" : "synced";
console.log(
`[ARENA_ELO_SYNC] ${countLabel} ${entries.length} model intelligence entries from Arena leaderboards`
);
return {
success: true,
modelCount: entries.length,
source: "arena_elo",
};
} catch (err) {
const message = err instanceof Error ? err.message : String(err);
console.warn("[ARENA_ELO_SYNC] Sync failed:", message);
return {
success: false,
modelCount: 0,
source: "arena_elo",
error: message,
};
} finally {
isSyncing = false;
}
}- Fix SQLite datetime comparisons: use datetime() for ISO8601 normalization - Don't cache transient DB errors permanently (remove null cache on error) - Add sync concurrency guard (syncInProgress flag) to prevent overlapping syncs - Add env vars to .env.example and ENVIRONMENT.md - Update file-size-baseline.json for schemas.ts growth
![kilo-code-bot[bot]](https://avatars.githubusercontent.com/in/2193792?s=60&v=4){kind=small}
| let activeSyncIntervalMs = SYNC_INTERVAL_MS; | ||
| let firstSyncDone = false; | ||
| let syncInProgress = false; | ||
| let syncInProgress = false; |
There was a problem hiding this comment.
CRITICAL: Duplicate syncInProgress declaration will fail TypeScript compilation
This module declares let syncInProgress = false; multiple times in the same scope. Keep a single declaration; otherwise the PR cannot compile.
| }; | ||
| // ─── DB access helpers ────────────────────────────────────────────────── | ||
|
|
||
| const _intelligenceCache = new Map<string, number | null>(); |
There was a problem hiding this comment.
WARNING: Intelligence cache misses are never invalidated after Arena sync writes
queryModelIntelligence() caches null for DB misses. If a model/category/source is looked up before syncArenaElo() bulk-upserts Arena data, the cached miss will keep routing on stale static/wildcard fitness until process restart. Add invalidation after bulk upserts, use a TTL, or avoid caching negative DB results.
| const taskCategories = CATEGORY_TASK_MAP[category]; | ||
| if (!taskCategories) continue; | ||
|
|
||
| const models = leaderboard.models; |
There was a problem hiding this comment.
WARNING: Arena API payload shape is trusted after JSON parse
leaderboard.models is used as an array without validating that the parsed response contains it. A malformed API response will throw during transform and abort the sync. Validate the shape before iterating, or make models fallback to [] only for explicitly invalid data.
| * since the DB module provides per-key deletion. This is used by the | ||
| * DELETE /api/intelligence/sync endpoint. | ||
| */ | ||
| export function clearSyncedIntelligence(): void { |
There was a problem hiding this comment.
WARNING: DELETE behavior does not match route/docs promise
The route comment says DELETE clears arena_elo + models_dev_tier, but this only deletes arena_elo. Either delete both sources here or update the route comment/PR docs so stale models_dev_tier rows are not left behind.
Code Review SummaryStatus: 5 Issues Found | Recommendation: Address before merge Overview
Issue Details (click to expand)CRITICAL
WARNING
Other Observations (not in diff)Previously reported issues resolved in the latest commits:
No additional unchanged-code issues found. Files Reviewed (14 files)
Fix these issues in Kilo Cloud Reviewed by nex-n2-pro:free · 4,278,814 tokens |
- Fix import paths: ../../../src/ (not ../../src/) for autoCombo/ depth - Refactor getTaskFitness to delegate to getTaskFitnessWithSource (DRY) - Add payload validation for Arena API response (Array.isArray guard) - Fix route comment to match actual DELETE scope (arena_elo only)
- Only cache non-null DB results (positive cache) — stale nulls from pre-sync lookups don't block post-sync data from being found - This fixes: 'cache misses never invalidated after sync writes'
…intelligence-scoring # Conflicts: # src/lib/localDb.ts
Co-authored-by: diegosouzapw <diegosouza.pw@gmail.com>
Owner
|
Merged into |
Merged
diegosouzapw
added a commit
that referenced
this pull request
Jun 12, 2026
- CHANGELOG: complete v3.8.23 section (28 bullets, 27 commits) - fix(webdav): resolve promise on writeStream finish, not req end — eliminates intermittent 500 on PUT update (writeStream may not have flushed at rename time) - test(autoCombo): stub DB calls from PR #3660 in tieredRotation.test.ts to prevent 5s timeout in vitest (getModelIntelligenceBySource DB init path) - chore(env-sync): add XDG_DATA_HOME + OMNIROUTE_OPENCODE_PLUGIN_DIR to IGNORE_FROM_CODE allowlist (introduced by PR #3726 setup-open-code.mjs, not OmniRoute config vars) - chore(cli): regenerated bin/cli/api-commands/*.mjs (7 new, 27 updated)
diegosouzapw
added a commit
that referenced
this pull request
Jun 13, 2026
* chore(release): open v3.8.23 development cycle * fix(anthropic): strip top_p when temperature is set to avoid 400 (#3691) Integrated into release/v3.8.23 * fix(vertex): support Vertex AI Express-mode API keys (#3690) Integrated into release/v3.8.23 * fix(stream): error on empty Claude SSE instead of synthetic success (#3689) Integrated into release/v3.8.23 * fix(oauth): stop token-refresh invalidation loop + harden proxy resolution (#3692) Integrated into release/v3.8.23 * docs: add FUNDING.yml and Support section to README (#3698) Integrated into release/v3.8.23 * feat: gemini - handle known ratelimits (#3686) Integrated into release/v3.8.23 * fix: stream combo fails over on empty content-filtered response (#3685) (#3702) Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(antigravity): preserve gemini-3.1-pro high/low budget tiers (#3696) (#3703) Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com> * feat(auto-combo): add auto-updating model intelligence scoring (#3660) Integrated into release/v3.8.23 * fix(gemini): context-mode fallback for signatureless tool calls (#3688) (#3704) * chore(quality-gate): reconcile file-size baseline (27 files + providerLimits.ts) (#3705) * feat(vertex): dynamic model discovery via Generative Language models API (#3712) Integrated into release/v3.8.23. Vertex dynamic model discovery — surfaces image models (imagen-*, gemini-*-image), embeddings and audio from the live Generative Language catalog, with cached→static fallback and the shared parseGeminiModelsList helper. Validated: parser test 5/5, typecheck:core clean. * fix(combo): gate reasoning token buffer (#3700) Integrated into release/v3.8.23. Makes the #3588 reasoning token buffer safe and configurable: only inflates max_tokens when the model has a known, non-default output cap and the buffered value fits inside it; otherwise preserves/clamps the client limit. Adds the reasoningTokenBufferEnabled kill switch (default ON). Validated: combo-routing-engine 81/81, combo-config 25/25, combo-quality-validator-reasoning 12/12, phase1f 10/10, typecheck:core clean. * refactor(#3501): god-component Phase 1g-1j — client 4062→3408 LOC (-654) (#3717) Phase 1g-1j of #3501: client 4062→3408 LOC. Pure extraction (ProviderPlaygroundPanel, useCommandCodeAuth, useExternalLinkFlow+ExternalLinkModal, useAuthFileHandlers) + loadConnProxies ReferenceError fix + phase1f test path fix. Co-authored-by: oyi77 <14921983+oyi77@users.noreply.github.com> * refactor(#3501): god-component Phase 1k-1m — client 3408→2553 LOC (-855) (#3721) Phase 1k-1m of #3501: client 3408→2553 LOC. Pure extraction (useModelImportHandlers+ImportProgressModal, useModelVisibilityHandlers, ProviderModelsSection). Co-authored-by: oyi77 <14921983+oyi77@users.noreply.github.com> * docs(changelog): restore #3590 bullet lost on the v3.8.20 release branch The fix itself reached main pre-tag via cherry-pick #3591, but its changelog bullet (commit e33fdd4) only ever existed on release/v3.8.20 after the squash-merge. Restored under [3.8.20] per the 2026-06-12 release-branch leftover audit (_tasks/release-audit/release-leftovers-audit-2026-06-12.md). * fix(kiro): resolve quota for IAM Identity Center accounts missing a profileArn (#3722) Integrated into release/v3.8.23 * refactor(#3501): god-component Phase 1n-1s — client 2553→1376 LOC (-1177) (#3725) Phase 1n-1s of #3501: client 2553→1376 LOC. Pure extraction (ConnectionsListPanel, ConnectionsHeaderToolbar, ZedImportCard, BatchTestResultsModal, AdaptaTutorialModal, useApiKeySave + helpers). Co-authored-by: oyi77 <14921983+oyi77@users.noreply.github.com> * feat(model-lockout): settings UI, backend integration, error classification, and success-decay recovery (#3629) Integrated into release/v3.8.23 * refactor(#3501): god-component Phase 1t — client 1376→781 LOC (≤800 TARGET REACHED ✅) (#3727) Phase 1t of #3501: client 1376→781 LOC (≤800 reached). Original god-component 12,882→781 (−94%). Co-authored-by: oyi77 <14921983+oyi77@users.noreply.github.com> * fix: bundle @omniroute/opencode-plugin inside omniroute + add 'setup opencode' CLI command (#3726) Integrated into release/v3.8.23 * feat(vertex): self-tracked USD spend since account added (#3724) Integrated into release/v3.8.23 * fix(qwen-web): migrate to v2 chat API with full cookie-jar replay (#3288) (#3723) Integrated into release/v3.8.23 * fix(sse): make safeLogEvents async — 'await' in a sync function broke every chatHelpers import #3692 added a lazy 'await import(proxyEgress)' for egress-IP visibility inside safeLogEvents, which is a sync function — an ES syntax error. It went unnoticed because typecheck:core does not cover src/sse and no test in the merge gates loaded chatHelpers via tsx; any consumer that did (chat-context-relay and chat-route-coverage suites, integration harnesses) failed at module load with 'await can only be used inside an async function'. safeLogEvents is fire-and-forget logging with an outer try/catch, so making it async (and 'void'-ing the single chat.ts call site) preserves behavior exactly. Validation: tests/unit/chat-context-relay.test.ts + chat-route-coverage.test.ts went from failing-at-load to green (+14 tests destravados). * fix(sse): remove cross-provider credential leak in emergency fallback + combo/proxy audit fixes (#3699) Integrated into release/v3.8.23 * fix(executors): inject MiMoCode anti-abuse marker so free endpoint stops 403ing (#3728) Integrated into release/v3.8.23 * fix(dashboard): repair "Test all models" — toast crash, status icons, auto-hide (#3729) Integrated into release/v3.8.23 * chore(deps): bump actions/upload-artifact from 4 to 7 (#3735) Integrated into release/v3.8.23 — aligns upload-artifact to v7 (already used across ci.yml). * chore(deps): bump actions/cache from 4 to 5 (#3734) Integrated into release/v3.8.23 — actions/cache v4→v5. * chore(deps): bump actions/download-artifact from 4 to 8 (#3733) Integrated into release/v3.8.23 — download-artifact v4→v8. * feat(fallback): add OMNIROUTE_EMERGENCY_FALLBACK env switch (#3741) Adds an OMNIROUTE_EMERGENCY_FALLBACK env switch to disable the emergency budget-exhaustion fallback (reroute to free nvidia/gpt-oss-120b). Default unchanged (enabled). Closes #3739, related #2879. Integrated into release/v3.8.23. * i18n: comprehensive zh-CN translation improvements (#3736) Aligns zh-CN to en (hundreds of entries), translates batch-action labels + settings sidebar menu, adds categoryConfig/endpointTokenSaver keys, resolves __MISSING__ stubs. Sidebar/SidebarTab hardcoded strings replaced with t(). en.json purely additive (8 new sidebar.* keys, 0 removed); cli-i18n gate green. Integrated into release/v3.8.23. * chore(release): v3.8.23 — 2026-06-12 - CHANGELOG: complete v3.8.23 section (28 bullets, 27 commits) - fix(webdav): resolve promise on writeStream finish, not req end — eliminates intermittent 500 on PUT update (writeStream may not have flushed at rename time) - test(autoCombo): stub DB calls from PR #3660 in tieredRotation.test.ts to prevent 5s timeout in vitest (getModelIntelligenceBySource DB init path) - chore(env-sync): add XDG_DATA_HOME + OMNIROUTE_OPENCODE_PLUGIN_DIR to IGNORE_FROM_CODE allowlist (introduced by PR #3726 setup-open-code.mjs, not OmniRoute config vars) - chore(cli): regenerated bin/cli/api-commands/*.mjs (7 new, 27 updated) * fix(model-family): fallback lookup also tries bare model name with dots getNextFamilyFallback normalized dots-to-hyphens ("gemini-3.1-pro-high" → "gemini-3-1-pro-high") but MODEL_FAMILIES keys use the literal dot form. The lookup always missed, returning null for any model whose dots are part of the name rather than a version separator. Fallback: try MODEL_FAMILIES[lookupKey] ?? MODEL_FAMILIES[bareModel] so both naming conventions are covered. Fixes T30 test (pre-existing since v3.8.22). * feat: expose API key cost drilldown + quota % used (#3742) Adds all-time USD cost per API key in the API Key Manager, a per-key deep-link into the Cost Explorer (filtered + grouped by model), URL-param hydration of range/groupBy/apiKeyIds, and a '% used' quota display. Review adjustments: extracted URL-param parsers to a tested module (Rule #18), i18n'd the new strings (en + zh-CN), dropped the redundant webdav-handler entry already on release. Integrated into release/v3.8.23. * feat: add provider display modes — All / Configured / Compact (#3743) Replaces the Providers page configured-only toggle with All/Configured/Compact display modes (Compact = flat deduped grid, no-auth last). Persists the preference and migrates the legacy localStorage key. Rebased onto release/v3.8.23. Integrated into release/v3.8.23. * fix(cache): scope semantic-cache signature to API key (#3740) Adds the api_key_id dimension to generateSignature's SHA-256 hash so two callers with different API keys never receive each other's cached responses. Threads apiKeyId through checkSemanticCache + both write sites; migration 098 clears pre-existing key-less entries; unauthenticated requests stay isolated from keyed ones. 3 TDD tests. Integrated into release/v3.8.23. * fix(responses): apply OpenAI Responses API stream=false spec default (#3708) resolveStreamFlag now applies the stream=false-when-omitted default for sourceFormat=openai-responses (same as the existing claude path), so spec-compliant /v1/responses upstreams that return JSON no longer fall through to the wildcard-Accept heuristic and trigger STREAM_EARLY_EOF / 502. Codex CLI (stream:true) and explicit text/event-stream clients unaffected. Integrated into release/v3.8.23. * chore(release): reconcile CI gates for v3.8.23 - file-size baseline: re-freeze 8 files grown by PRs #3742/#3743/#3740 (cost drilldown, provider display modes, cache key isolation) - ARCHITECTURE.md: update executor count 55→60 (check:docs-counts drift) - .env.example: add OMNIROUTE_EMERGENCY_FALLBACK (#3741, env-doc-sync) - CHANGELOG: add formatted bullets for #3742, #3743, #3708, #3740, model-family-fallback fix; remove duplicate raw ### Fixed section * test: restore assert count to satisfy check:test-masking gate Three test files had net assertion removals after behavior-changing PRs: - chatcore-translation-paths: emergency fallback moved to routing layer (#3699) — add body error assertion + model-name guard - executor-vertex-extended: non-JSON is now Express API key (#3690) — add projects/-path guard to the express-key URL test - stream-utils: empty streams now emit error (#3685) — add code/message/ status/completePayload guards to both passthrough and translate variants All new assertions are meaningful (code enum value, 5xx range, non-empty message, onComplete must-not-fire contract). * fix(ci): move rtl-logical-classes test to ui/ so vitest:ui runner collects it --------- Co-authored-by: Felipe Almeman <4226997+zhiru@users.noreply.github.com> Co-authored-by: NOXX - Commiter <artur1992123@mail.ru> Co-authored-by: Nick Sullivan <142708+TechNickAI@users.noreply.github.com> Co-authored-by: Markus Hartung <mail@hartmark.se> Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Co-authored-by: PizzaV <103120356+pizzav-xyz@users.noreply.github.com> Co-authored-by: Randi <55005611+rdself@users.noreply.github.com> Co-authored-by: oyi77 <14921983+oyi77@users.noreply.github.com> Co-authored-by: Chewji <126886556+Chewji9875@users.noreply.github.com> Co-authored-by: Hernan Javier Ardila Sanchez <hjasgr@gmail.com> Co-authored-by: Felipe Sartori <felipesartori.ti@gmail.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Zois Pagoulatos <zpagoulatos@hotmail.com> Co-authored-by: sdfsdfw2 <167810361+sdfsdfw2@users.noreply.github.com> Co-authored-by: Witroch4 <witalo_rocha@hotmail.com>
HouMinXi
pushed a commit
to HouMinXi/OmniRoute
that referenced
this pull request
Aug 2, 2026
* chore(release): open v3.8.23 development cycle * fix(anthropic): strip top_p when temperature is set to avoid 400 (diegosouzapw#3691) Integrated into release/v3.8.23 * fix(vertex): support Vertex AI Express-mode API keys (diegosouzapw#3690) Integrated into release/v3.8.23 * fix(stream): error on empty Claude SSE instead of synthetic success (diegosouzapw#3689) Integrated into release/v3.8.23 * fix(oauth): stop token-refresh invalidation loop + harden proxy resolution (diegosouzapw#3692) Integrated into release/v3.8.23 * docs: add FUNDING.yml and Support section to README (diegosouzapw#3698) Integrated into release/v3.8.23 * feat: gemini - handle known ratelimits (diegosouzapw#3686) Integrated into release/v3.8.23 * fix: stream combo fails over on empty content-filtered response (diegosouzapw#3685) (diegosouzapw#3702) Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(antigravity): preserve gemini-3.1-pro high/low budget tiers (diegosouzapw#3696) (diegosouzapw#3703) Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com> * feat(auto-combo): add auto-updating model intelligence scoring (diegosouzapw#3660) Integrated into release/v3.8.23 * fix(gemini): context-mode fallback for signatureless tool calls (diegosouzapw#3688) (diegosouzapw#3704) * chore(quality-gate): reconcile file-size baseline (27 files + providerLimits.ts) (diegosouzapw#3705) * feat(vertex): dynamic model discovery via Generative Language models API (diegosouzapw#3712) Integrated into release/v3.8.23. Vertex dynamic model discovery — surfaces image models (imagen-*, gemini-*-image), embeddings and audio from the live Generative Language catalog, with cached→static fallback and the shared parseGeminiModelsList helper. Validated: parser test 5/5, typecheck:core clean. * fix(combo): gate reasoning token buffer (diegosouzapw#3700) Integrated into release/v3.8.23. Makes the diegosouzapw#3588 reasoning token buffer safe and configurable: only inflates max_tokens when the model has a known, non-default output cap and the buffered value fits inside it; otherwise preserves/clamps the client limit. Adds the reasoningTokenBufferEnabled kill switch (default ON). Validated: combo-routing-engine 81/81, combo-config 25/25, combo-quality-validator-reasoning 12/12, phase1f 10/10, typecheck:core clean. * refactor(diegosouzapw#3501): god-component Phase 1g-1j — client 4062→3408 LOC (-654) (diegosouzapw#3717) Phase 1g-1j of diegosouzapw#3501: client 4062→3408 LOC. Pure extraction (ProviderPlaygroundPanel, useCommandCodeAuth, useExternalLinkFlow+ExternalLinkModal, useAuthFileHandlers) + loadConnProxies ReferenceError fix + phase1f test path fix. Co-authored-by: oyi77 <14921983+oyi77@users.noreply.github.com> * refactor(diegosouzapw#3501): god-component Phase 1k-1m — client 3408→2553 LOC (-855) (diegosouzapw#3721) Phase 1k-1m of diegosouzapw#3501: client 3408→2553 LOC. Pure extraction (useModelImportHandlers+ImportProgressModal, useModelVisibilityHandlers, ProviderModelsSection). Co-authored-by: oyi77 <14921983+oyi77@users.noreply.github.com> * docs(changelog): restore diegosouzapw#3590 bullet lost on the v3.8.20 release branch The fix itself reached main pre-tag via cherry-pick diegosouzapw#3591, but its changelog bullet (commit db04ef2) only ever existed on release/v3.8.20 after the squash-merge. Restored under [3.8.20] per the 2026-06-12 release-branch leftover audit (_tasks/release-audit/release-leftovers-audit-2026-06-12.md). * fix(kiro): resolve quota for IAM Identity Center accounts missing a profileArn (diegosouzapw#3722) Integrated into release/v3.8.23 * refactor(diegosouzapw#3501): god-component Phase 1n-1s — client 2553→1376 LOC (-1177) (diegosouzapw#3725) Phase 1n-1s of diegosouzapw#3501: client 2553→1376 LOC. Pure extraction (ConnectionsListPanel, ConnectionsHeaderToolbar, ZedImportCard, BatchTestResultsModal, AdaptaTutorialModal, useApiKeySave + helpers). Co-authored-by: oyi77 <14921983+oyi77@users.noreply.github.com> * feat(model-lockout): settings UI, backend integration, error classification, and success-decay recovery (diegosouzapw#3629) Integrated into release/v3.8.23 * refactor(diegosouzapw#3501): god-component Phase 1t — client 1376→781 LOC (≤800 TARGET REACHED ✅) (diegosouzapw#3727) Phase 1t of diegosouzapw#3501: client 1376→781 LOC (≤800 reached). Original god-component 12,882→781 (−94%). Co-authored-by: oyi77 <14921983+oyi77@users.noreply.github.com> * fix: bundle @omniroute/opencode-plugin inside omniroute + add 'setup opencode' CLI command (diegosouzapw#3726) Integrated into release/v3.8.23 * feat(vertex): self-tracked USD spend since account added (diegosouzapw#3724) Integrated into release/v3.8.23 * fix(qwen-web): migrate to v2 chat API with full cookie-jar replay (diegosouzapw#3288) (diegosouzapw#3723) Integrated into release/v3.8.23 * fix(sse): make safeLogEvents async — 'await' in a sync function broke every chatHelpers import diegosouzapw#3692 added a lazy 'await import(proxyEgress)' for egress-IP visibility inside safeLogEvents, which is a sync function — an ES syntax error. It went unnoticed because typecheck:core does not cover src/sse and no test in the merge gates loaded chatHelpers via tsx; any consumer that did (chat-context-relay and chat-route-coverage suites, integration harnesses) failed at module load with 'await can only be used inside an async function'. safeLogEvents is fire-and-forget logging with an outer try/catch, so making it async (and 'void'-ing the single chat.ts call site) preserves behavior exactly. Validation: tests/unit/chat-context-relay.test.ts + chat-route-coverage.test.ts went from failing-at-load to green (+14 tests destravados). * fix(sse): remove cross-provider credential leak in emergency fallback + combo/proxy audit fixes (diegosouzapw#3699) Integrated into release/v3.8.23 * fix(executors): inject MiMoCode anti-abuse marker so free endpoint stops 403ing (diegosouzapw#3728) Integrated into release/v3.8.23 * fix(dashboard): repair "Test all models" — toast crash, status icons, auto-hide (diegosouzapw#3729) Integrated into release/v3.8.23 * chore(deps): bump actions/upload-artifact from 4 to 7 (diegosouzapw#3735) Integrated into release/v3.8.23 — aligns upload-artifact to v7 (already used across ci.yml). * chore(deps): bump actions/cache from 4 to 5 (diegosouzapw#3734) Integrated into release/v3.8.23 — actions/cache v4→v5. * chore(deps): bump actions/download-artifact from 4 to 8 (diegosouzapw#3733) Integrated into release/v3.8.23 — download-artifact v4→v8. * feat(fallback): add OMNIROUTE_EMERGENCY_FALLBACK env switch (diegosouzapw#3741) Adds an OMNIROUTE_EMERGENCY_FALLBACK env switch to disable the emergency budget-exhaustion fallback (reroute to free nvidia/gpt-oss-120b). Default unchanged (enabled). Closes diegosouzapw#3739, related diegosouzapw#2879. Integrated into release/v3.8.23. * i18n: comprehensive zh-CN translation improvements (diegosouzapw#3736) Aligns zh-CN to en (hundreds of entries), translates batch-action labels + settings sidebar menu, adds categoryConfig/endpointTokenSaver keys, resolves __MISSING__ stubs. Sidebar/SidebarTab hardcoded strings replaced with t(). en.json purely additive (8 new sidebar.* keys, 0 removed); cli-i18n gate green. Integrated into release/v3.8.23. * chore(release): v3.8.23 — 2026-06-12 - CHANGELOG: complete v3.8.23 section (28 bullets, 27 commits) - fix(webdav): resolve promise on writeStream finish, not req end — eliminates intermittent 500 on PUT update (writeStream may not have flushed at rename time) - test(autoCombo): stub DB calls from PR diegosouzapw#3660 in tieredRotation.test.ts to prevent 5s timeout in vitest (getModelIntelligenceBySource DB init path) - chore(env-sync): add XDG_DATA_HOME + OMNIROUTE_OPENCODE_PLUGIN_DIR to IGNORE_FROM_CODE allowlist (introduced by PR diegosouzapw#3726 setup-open-code.mjs, not OmniRoute config vars) - chore(cli): regenerated bin/cli/api-commands/*.mjs (7 new, 27 updated) * fix(model-family): fallback lookup also tries bare model name with dots getNextFamilyFallback normalized dots-to-hyphens ("gemini-3.1-pro-high" → "gemini-3-1-pro-high") but MODEL_FAMILIES keys use the literal dot form. The lookup always missed, returning null for any model whose dots are part of the name rather than a version separator. Fallback: try MODEL_FAMILIES[lookupKey] ?? MODEL_FAMILIES[bareModel] so both naming conventions are covered. Fixes T30 test (pre-existing since v3.8.22). * feat: expose API key cost drilldown + quota % used (diegosouzapw#3742) Adds all-time USD cost per API key in the API Key Manager, a per-key deep-link into the Cost Explorer (filtered + grouped by model), URL-param hydration of range/groupBy/apiKeyIds, and a '% used' quota display. Review adjustments: extracted URL-param parsers to a tested module (Rule diegosouzapw#18), i18n'd the new strings (en + zh-CN), dropped the redundant webdav-handler entry already on release. Integrated into release/v3.8.23. * feat: add provider display modes — All / Configured / Compact (diegosouzapw#3743) Replaces the Providers page configured-only toggle with All/Configured/Compact display modes (Compact = flat deduped grid, no-auth last). Persists the preference and migrates the legacy localStorage key. Rebased onto release/v3.8.23. Integrated into release/v3.8.23. * fix(cache): scope semantic-cache signature to API key (diegosouzapw#3740) Adds the api_key_id dimension to generateSignature's SHA-256 hash so two callers with different API keys never receive each other's cached responses. Threads apiKeyId through checkSemanticCache + both write sites; migration 098 clears pre-existing key-less entries; unauthenticated requests stay isolated from keyed ones. 3 TDD tests. Integrated into release/v3.8.23. * fix(responses): apply OpenAI Responses API stream=false spec default (diegosouzapw#3708) resolveStreamFlag now applies the stream=false-when-omitted default for sourceFormat=openai-responses (same as the existing claude path), so spec-compliant /v1/responses upstreams that return JSON no longer fall through to the wildcard-Accept heuristic and trigger STREAM_EARLY_EOF / 502. Codex CLI (stream:true) and explicit text/event-stream clients unaffected. Integrated into release/v3.8.23. * chore(release): reconcile CI gates for v3.8.23 - file-size baseline: re-freeze 8 files grown by PRs diegosouzapw#3742/diegosouzapw#3743/diegosouzapw#3740 (cost drilldown, provider display modes, cache key isolation) - ARCHITECTURE.md: update executor count 55→60 (check:docs-counts drift) - .env.example: add OMNIROUTE_EMERGENCY_FALLBACK (diegosouzapw#3741, env-doc-sync) - CHANGELOG: add formatted bullets for diegosouzapw#3742, diegosouzapw#3743, diegosouzapw#3708, diegosouzapw#3740, model-family-fallback fix; remove duplicate raw ### Fixed section * test: restore assert count to satisfy check:test-masking gate Three test files had net assertion removals after behavior-changing PRs: - chatcore-translation-paths: emergency fallback moved to routing layer (diegosouzapw#3699) — add body error assertion + model-name guard - executor-vertex-extended: non-JSON is now Express API key (diegosouzapw#3690) — add projects/-path guard to the express-key URL test - stream-utils: empty streams now emit error (diegosouzapw#3685) — add code/message/ status/completePayload guards to both passthrough and translate variants All new assertions are meaningful (code enum value, 5xx range, non-empty message, onComplete must-not-fire contract). * fix(ci): move rtl-logical-classes test to ui/ so vitest:ui runner collects it --------- Co-authored-by: Felipe Almeman <4226997+zhiru@users.noreply.github.com> Co-authored-by: NOXX - Commiter <artur1992123@mail.ru> Co-authored-by: Nick Sullivan <142708+TechNickAI@users.noreply.github.com> Co-authored-by: Markus Hartung <mail@hartmark.se> Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Co-authored-by: PizzaV <103120356+pizzav-xyz@users.noreply.github.com> Co-authored-by: Randi <55005611+rdself@users.noreply.github.com> Co-authored-by: oyi77 <14921983+oyi77@users.noreply.github.com> Co-authored-by: Chewji <126886556+Chewji9875@users.noreply.github.com> Co-authored-by: Hernan Javier Ardila Sanchez <hjasgr@gmail.com> Co-authored-by: Felipe Sartori <felipesartori.ti@gmail.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Zois Pagoulatos <zpagoulatos@hotmail.com> Co-authored-by: sdfsdfw2 <167810361+sdfsdfw2@users.noreply.github.com> Co-authored-by: Witroch4 <witalo_rocha@hotmail.com>
Poid-ZA
pushed a commit
to Poid-ZA/OmniRoute
that referenced
this pull request
Aug 5, 2026
* chore(release): open v3.8.23 development cycle * fix(anthropic): strip top_p when temperature is set to avoid 400 (diegosouzapw#3691) Integrated into release/v3.8.23 * fix(vertex): support Vertex AI Express-mode API keys (diegosouzapw#3690) Integrated into release/v3.8.23 * fix(stream): error on empty Claude SSE instead of synthetic success (diegosouzapw#3689) Integrated into release/v3.8.23 * fix(oauth): stop token-refresh invalidation loop + harden proxy resolution (diegosouzapw#3692) Integrated into release/v3.8.23 * docs: add FUNDING.yml and Support section to README (diegosouzapw#3698) Integrated into release/v3.8.23 * feat: gemini - handle known ratelimits (diegosouzapw#3686) Integrated into release/v3.8.23 * fix: stream combo fails over on empty content-filtered response (diegosouzapw#3685) (diegosouzapw#3702) Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(antigravity): preserve gemini-3.1-pro high/low budget tiers (diegosouzapw#3696) (diegosouzapw#3703) Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com> * feat(auto-combo): add auto-updating model intelligence scoring (diegosouzapw#3660) Integrated into release/v3.8.23 * fix(gemini): context-mode fallback for signatureless tool calls (diegosouzapw#3688) (diegosouzapw#3704) * chore(quality-gate): reconcile file-size baseline (27 files + providerLimits.ts) (diegosouzapw#3705) * feat(vertex): dynamic model discovery via Generative Language models API (diegosouzapw#3712) Integrated into release/v3.8.23. Vertex dynamic model discovery — surfaces image models (imagen-*, gemini-*-image), embeddings and audio from the live Generative Language catalog, with cached→static fallback and the shared parseGeminiModelsList helper. Validated: parser test 5/5, typecheck:core clean. * fix(combo): gate reasoning token buffer (diegosouzapw#3700) Integrated into release/v3.8.23. Makes the diegosouzapw#3588 reasoning token buffer safe and configurable: only inflates max_tokens when the model has a known, non-default output cap and the buffered value fits inside it; otherwise preserves/clamps the client limit. Adds the reasoningTokenBufferEnabled kill switch (default ON). Validated: combo-routing-engine 81/81, combo-config 25/25, combo-quality-validator-reasoning 12/12, phase1f 10/10, typecheck:core clean. * refactor(diegosouzapw#3501): god-component Phase 1g-1j — client 4062→3408 LOC (-654) (diegosouzapw#3717) Phase 1g-1j of diegosouzapw#3501: client 4062→3408 LOC. Pure extraction (ProviderPlaygroundPanel, useCommandCodeAuth, useExternalLinkFlow+ExternalLinkModal, useAuthFileHandlers) + loadConnProxies ReferenceError fix + phase1f test path fix. Co-authored-by: oyi77 <14921983+oyi77@users.noreply.github.com> * refactor(diegosouzapw#3501): god-component Phase 1k-1m — client 3408→2553 LOC (-855) (diegosouzapw#3721) Phase 1k-1m of diegosouzapw#3501: client 3408→2553 LOC. Pure extraction (useModelImportHandlers+ImportProgressModal, useModelVisibilityHandlers, ProviderModelsSection). Co-authored-by: oyi77 <14921983+oyi77@users.noreply.github.com> * docs(changelog): restore diegosouzapw#3590 bullet lost on the v3.8.20 release branch The fix itself reached main pre-tag via cherry-pick diegosouzapw#3591, but its changelog bullet (commit e33fdd4) only ever existed on release/v3.8.20 after the squash-merge. Restored under [3.8.20] per the 2026-06-12 release-branch leftover audit (_tasks/release-audit/release-leftovers-audit-2026-06-12.md). * fix(kiro): resolve quota for IAM Identity Center accounts missing a profileArn (diegosouzapw#3722) Integrated into release/v3.8.23 * refactor(diegosouzapw#3501): god-component Phase 1n-1s — client 2553→1376 LOC (-1177) (diegosouzapw#3725) Phase 1n-1s of diegosouzapw#3501: client 2553→1376 LOC. Pure extraction (ConnectionsListPanel, ConnectionsHeaderToolbar, ZedImportCard, BatchTestResultsModal, AdaptaTutorialModal, useApiKeySave + helpers). Co-authored-by: oyi77 <14921983+oyi77@users.noreply.github.com> * feat(model-lockout): settings UI, backend integration, error classification, and success-decay recovery (diegosouzapw#3629) Integrated into release/v3.8.23 * refactor(diegosouzapw#3501): god-component Phase 1t — client 1376→781 LOC (≤800 TARGET REACHED ✅) (diegosouzapw#3727) Phase 1t of diegosouzapw#3501: client 1376→781 LOC (≤800 reached). Original god-component 12,882→781 (−94%). Co-authored-by: oyi77 <14921983+oyi77@users.noreply.github.com> * fix: bundle @omniroute/opencode-plugin inside omniroute + add 'setup opencode' CLI command (diegosouzapw#3726) Integrated into release/v3.8.23 * feat(vertex): self-tracked USD spend since account added (diegosouzapw#3724) Integrated into release/v3.8.23 * fix(qwen-web): migrate to v2 chat API with full cookie-jar replay (diegosouzapw#3288) (diegosouzapw#3723) Integrated into release/v3.8.23 * fix(sse): make safeLogEvents async — 'await' in a sync function broke every chatHelpers import diegosouzapw#3692 added a lazy 'await import(proxyEgress)' for egress-IP visibility inside safeLogEvents, which is a sync function — an ES syntax error. It went unnoticed because typecheck:core does not cover src/sse and no test in the merge gates loaded chatHelpers via tsx; any consumer that did (chat-context-relay and chat-route-coverage suites, integration harnesses) failed at module load with 'await can only be used inside an async function'. safeLogEvents is fire-and-forget logging with an outer try/catch, so making it async (and 'void'-ing the single chat.ts call site) preserves behavior exactly. Validation: tests/unit/chat-context-relay.test.ts + chat-route-coverage.test.ts went from failing-at-load to green (+14 tests destravados). * fix(sse): remove cross-provider credential leak in emergency fallback + combo/proxy audit fixes (diegosouzapw#3699) Integrated into release/v3.8.23 * fix(executors): inject MiMoCode anti-abuse marker so free endpoint stops 403ing (diegosouzapw#3728) Integrated into release/v3.8.23 * fix(dashboard): repair "Test all models" — toast crash, status icons, auto-hide (diegosouzapw#3729) Integrated into release/v3.8.23 * chore(deps): bump actions/upload-artifact from 4 to 7 (diegosouzapw#3735) Integrated into release/v3.8.23 — aligns upload-artifact to v7 (already used across ci.yml). * chore(deps): bump actions/cache from 4 to 5 (diegosouzapw#3734) Integrated into release/v3.8.23 — actions/cache v4→v5. * chore(deps): bump actions/download-artifact from 4 to 8 (diegosouzapw#3733) Integrated into release/v3.8.23 — download-artifact v4→v8. * feat(fallback): add OMNIROUTE_EMERGENCY_FALLBACK env switch (diegosouzapw#3741) Adds an OMNIROUTE_EMERGENCY_FALLBACK env switch to disable the emergency budget-exhaustion fallback (reroute to free nvidia/gpt-oss-120b). Default unchanged (enabled). Closes diegosouzapw#3739, related diegosouzapw#2879. Integrated into release/v3.8.23. * i18n: comprehensive zh-CN translation improvements (diegosouzapw#3736) Aligns zh-CN to en (hundreds of entries), translates batch-action labels + settings sidebar menu, adds categoryConfig/endpointTokenSaver keys, resolves __MISSING__ stubs. Sidebar/SidebarTab hardcoded strings replaced with t(). en.json purely additive (8 new sidebar.* keys, 0 removed); cli-i18n gate green. Integrated into release/v3.8.23. * chore(release): v3.8.23 — 2026-06-12 - CHANGELOG: complete v3.8.23 section (28 bullets, 27 commits) - fix(webdav): resolve promise on writeStream finish, not req end — eliminates intermittent 500 on PUT update (writeStream may not have flushed at rename time) - test(autoCombo): stub DB calls from PR diegosouzapw#3660 in tieredRotation.test.ts to prevent 5s timeout in vitest (getModelIntelligenceBySource DB init path) - chore(env-sync): add XDG_DATA_HOME + OMNIROUTE_OPENCODE_PLUGIN_DIR to IGNORE_FROM_CODE allowlist (introduced by PR diegosouzapw#3726 setup-open-code.mjs, not OmniRoute config vars) - chore(cli): regenerated bin/cli/api-commands/*.mjs (7 new, 27 updated) * fix(model-family): fallback lookup also tries bare model name with dots getNextFamilyFallback normalized dots-to-hyphens ("gemini-3.1-pro-high" → "gemini-3-1-pro-high") but MODEL_FAMILIES keys use the literal dot form. The lookup always missed, returning null for any model whose dots are part of the name rather than a version separator. Fallback: try MODEL_FAMILIES[lookupKey] ?? MODEL_FAMILIES[bareModel] so both naming conventions are covered. Fixes T30 test (pre-existing since v3.8.22). * feat: expose API key cost drilldown + quota % used (diegosouzapw#3742) Adds all-time USD cost per API key in the API Key Manager, a per-key deep-link into the Cost Explorer (filtered + grouped by model), URL-param hydration of range/groupBy/apiKeyIds, and a '% used' quota display. Review adjustments: extracted URL-param parsers to a tested module (Rule diegosouzapw#18), i18n'd the new strings (en + zh-CN), dropped the redundant webdav-handler entry already on release. Integrated into release/v3.8.23. * feat: add provider display modes — All / Configured / Compact (diegosouzapw#3743) Replaces the Providers page configured-only toggle with All/Configured/Compact display modes (Compact = flat deduped grid, no-auth last). Persists the preference and migrates the legacy localStorage key. Rebased onto release/v3.8.23. Integrated into release/v3.8.23. * fix(cache): scope semantic-cache signature to API key (diegosouzapw#3740) Adds the api_key_id dimension to generateSignature's SHA-256 hash so two callers with different API keys never receive each other's cached responses. Threads apiKeyId through checkSemanticCache + both write sites; migration 098 clears pre-existing key-less entries; unauthenticated requests stay isolated from keyed ones. 3 TDD tests. Integrated into release/v3.8.23. * fix(responses): apply OpenAI Responses API stream=false spec default (diegosouzapw#3708) resolveStreamFlag now applies the stream=false-when-omitted default for sourceFormat=openai-responses (same as the existing claude path), so spec-compliant /v1/responses upstreams that return JSON no longer fall through to the wildcard-Accept heuristic and trigger STREAM_EARLY_EOF / 502. Codex CLI (stream:true) and explicit text/event-stream clients unaffected. Integrated into release/v3.8.23. * chore(release): reconcile CI gates for v3.8.23 - file-size baseline: re-freeze 8 files grown by PRs diegosouzapw#3742/diegosouzapw#3743/diegosouzapw#3740 (cost drilldown, provider display modes, cache key isolation) - ARCHITECTURE.md: update executor count 55→60 (check:docs-counts drift) - .env.example: add OMNIROUTE_EMERGENCY_FALLBACK (diegosouzapw#3741, env-doc-sync) - CHANGELOG: add formatted bullets for diegosouzapw#3742, diegosouzapw#3743, diegosouzapw#3708, diegosouzapw#3740, model-family-fallback fix; remove duplicate raw ### Fixed section * test: restore assert count to satisfy check:test-masking gate Three test files had net assertion removals after behavior-changing PRs: - chatcore-translation-paths: emergency fallback moved to routing layer (diegosouzapw#3699) — add body error assertion + model-name guard - executor-vertex-extended: non-JSON is now Express API key (diegosouzapw#3690) — add projects/-path guard to the express-key URL test - stream-utils: empty streams now emit error (diegosouzapw#3685) — add code/message/ status/completePayload guards to both passthrough and translate variants All new assertions are meaningful (code enum value, 5xx range, non-empty message, onComplete must-not-fire contract). * fix(ci): move rtl-logical-classes test to ui/ so vitest:ui runner collects it --------- Co-authored-by: Felipe Almeman <4226997+zhiru@users.noreply.github.com> Co-authored-by: NOXX - Commiter <artur1992123@mail.ru> Co-authored-by: Nick Sullivan <142708+TechNickAI@users.noreply.github.com> Co-authored-by: Markus Hartung <mail@hartmark.se> Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Co-authored-by: PizzaV <103120356+pizzav-xyz@users.noreply.github.com> Co-authored-by: Randi <55005611+rdself@users.noreply.github.com> Co-authored-by: oyi77 <14921983+oyi77@users.noreply.github.com> Co-authored-by: Chewji <126886556+Chewji9875@users.noreply.github.com> Co-authored-by: Hernan Javier Ardila Sanchez <hjasgr@gmail.com> Co-authored-by: Felipe Sartori <felipesartori.ti@gmail.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Zois Pagoulatos <zpagoulatos@hotmail.com> Co-authored-by: sdfsdfw2 <167810361+sdfsdfw2@users.noreply.github.com> Co-authored-by: Witroch4 <witalo_rocha@hotmail.com>
|
هلا |
tkgo11
pushed a commit
to tkgo11/OmniRoute
that referenced
this pull request
Sep 23, 2026
…souzapw#3660) Integrated into release/v3.8.23
tkgo11
pushed a commit
to tkgo11/OmniRoute
that referenced
this pull request
Sep 23, 2026
- CHANGELOG: complete v3.8.23 section (28 bullets, 27 commits) - fix(webdav): resolve promise on writeStream finish, not req end — eliminates intermittent 500 on PUT update (writeStream may not have flushed at rename time) - test(autoCombo): stub DB calls from PR diegosouzapw#3660 in tieredRotation.test.ts to prevent 5s timeout in vitest (getModelIntelligenceBySource DB init path) - chore(env-sync): add XDG_DATA_HOME + OMNIROUTE_OPENCODE_PLUGIN_DIR to IGNORE_FROM_CODE allowlist (introduced by PR diegosouzapw#3726 setup-open-code.mjs, not OmniRoute config vars) - chore(cli): regenerated bin/cli/api-commands/*.mjs (7 new, 27 updated)
muhamadgalihsaputra
pushed a commit
to niyatna/NiyatnaRoute
that referenced
this pull request
Sep 27, 2026
* chore(release): open v3.8.23 development cycle * fix(anthropic): strip top_p when temperature is set to avoid 400 (diegosouzapw#3691) Integrated into release/v3.8.23 * fix(vertex): support Vertex AI Express-mode API keys (diegosouzapw#3690) Integrated into release/v3.8.23 * fix(stream): error on empty Claude SSE instead of synthetic success (diegosouzapw#3689) Integrated into release/v3.8.23 * fix(oauth): stop token-refresh invalidation loop + harden proxy resolution (diegosouzapw#3692) Integrated into release/v3.8.23 * docs: add FUNDING.yml and Support section to README (diegosouzapw#3698) Integrated into release/v3.8.23 * feat: gemini - handle known ratelimits (diegosouzapw#3686) Integrated into release/v3.8.23 * fix: stream combo fails over on empty content-filtered response (diegosouzapw#3685) (diegosouzapw#3702) Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(antigravity): preserve gemini-3.1-pro high/low budget tiers (diegosouzapw#3696) (diegosouzapw#3703) Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com> * feat(auto-combo): add auto-updating model intelligence scoring (diegosouzapw#3660) Integrated into release/v3.8.23 * fix(gemini): context-mode fallback for signatureless tool calls (diegosouzapw#3688) (diegosouzapw#3704) * chore(quality-gate): reconcile file-size baseline (27 files + providerLimits.ts) (diegosouzapw#3705) * feat(vertex): dynamic model discovery via Generative Language models API (diegosouzapw#3712) Integrated into release/v3.8.23. Vertex dynamic model discovery — surfaces image models (imagen-*, gemini-*-image), embeddings and audio from the live Generative Language catalog, with cached→static fallback and the shared parseGeminiModelsList helper. Validated: parser test 5/5, typecheck:core clean. * fix(combo): gate reasoning token buffer (diegosouzapw#3700) Integrated into release/v3.8.23. Makes the diegosouzapw#3588 reasoning token buffer safe and configurable: only inflates max_tokens when the model has a known, non-default output cap and the buffered value fits inside it; otherwise preserves/clamps the client limit. Adds the reasoningTokenBufferEnabled kill switch (default ON). Validated: combo-routing-engine 81/81, combo-config 25/25, combo-quality-validator-reasoning 12/12, phase1f 10/10, typecheck:core clean. * refactor(diegosouzapw#3501): god-component Phase 1g-1j — client 4062→3408 LOC (-654) (diegosouzapw#3717) Phase 1g-1j of diegosouzapw#3501: client 4062→3408 LOC. Pure extraction (ProviderPlaygroundPanel, useCommandCodeAuth, useExternalLinkFlow+ExternalLinkModal, useAuthFileHandlers) + loadConnProxies ReferenceError fix + phase1f test path fix. Co-authored-by: oyi77 <14921983+oyi77@users.noreply.github.com> * refactor(diegosouzapw#3501): god-component Phase 1k-1m — client 3408→2553 LOC (-855) (diegosouzapw#3721) Phase 1k-1m of diegosouzapw#3501: client 3408→2553 LOC. Pure extraction (useModelImportHandlers+ImportProgressModal, useModelVisibilityHandlers, ProviderModelsSection). Co-authored-by: oyi77 <14921983+oyi77@users.noreply.github.com> * docs(changelog): restore diegosouzapw#3590 bullet lost on the v3.8.20 release branch The fix itself reached main pre-tag via cherry-pick diegosouzapw#3591, but its changelog bullet (commit a6b99843f) only ever existed on release/v3.8.20 after the squash-merge. Restored under [3.8.20] per the 2026-06-12 release-branch leftover audit (_tasks/release-audit/release-leftovers-audit-2026-06-12.md). * fix(kiro): resolve quota for IAM Identity Center accounts missing a profileArn (diegosouzapw#3722) Integrated into release/v3.8.23 * refactor(diegosouzapw#3501): god-component Phase 1n-1s — client 2553→1376 LOC (-1177) (diegosouzapw#3725) Phase 1n-1s of diegosouzapw#3501: client 2553→1376 LOC. Pure extraction (ConnectionsListPanel, ConnectionsHeaderToolbar, ZedImportCard, BatchTestResultsModal, AdaptaTutorialModal, useApiKeySave + helpers). Co-authored-by: oyi77 <14921983+oyi77@users.noreply.github.com> * feat(model-lockout): settings UI, backend integration, error classification, and success-decay recovery (diegosouzapw#3629) Integrated into release/v3.8.23 * refactor(diegosouzapw#3501): god-component Phase 1t — client 1376→781 LOC (≤800 TARGET REACHED ✅) (diegosouzapw#3727) Phase 1t of diegosouzapw#3501: client 1376→781 LOC (≤800 reached). Original god-component 12,882→781 (−94%). Co-authored-by: oyi77 <14921983+oyi77@users.noreply.github.com> * fix: bundle @omniroute/opencode-plugin inside omniroute + add 'setup opencode' CLI command (diegosouzapw#3726) Integrated into release/v3.8.23 * feat(vertex): self-tracked USD spend since account added (diegosouzapw#3724) Integrated into release/v3.8.23 * fix(qwen-web): migrate to v2 chat API with full cookie-jar replay (diegosouzapw#3288) (diegosouzapw#3723) Integrated into release/v3.8.23 * fix(sse): make safeLogEvents async — 'await' in a sync function broke every chatHelpers import diegosouzapw#3692 added a lazy 'await import(proxyEgress)' for egress-IP visibility inside safeLogEvents, which is a sync function — an ES syntax error. It went unnoticed because typecheck:core does not cover src/sse and no test in the merge gates loaded chatHelpers via tsx; any consumer that did (chat-context-relay and chat-route-coverage suites, integration harnesses) failed at module load with 'await can only be used inside an async function'. safeLogEvents is fire-and-forget logging with an outer try/catch, so making it async (and 'void'-ing the single chat.ts call site) preserves behavior exactly. Validation: tests/unit/chat-context-relay.test.ts + chat-route-coverage.test.ts went from failing-at-load to green (+14 tests destravados). * fix(sse): remove cross-provider credential leak in emergency fallback + combo/proxy audit fixes (diegosouzapw#3699) Integrated into release/v3.8.23 * fix(executors): inject MiMoCode anti-abuse marker so free endpoint stops 403ing (diegosouzapw#3728) Integrated into release/v3.8.23 * fix(dashboard): repair "Test all models" — toast crash, status icons, auto-hide (diegosouzapw#3729) Integrated into release/v3.8.23 * chore(deps): bump actions/upload-artifact from 4 to 7 (diegosouzapw#3735) Integrated into release/v3.8.23 — aligns upload-artifact to v7 (already used across ci.yml). * chore(deps): bump actions/cache from 4 to 5 (diegosouzapw#3734) Integrated into release/v3.8.23 — actions/cache v4→v5. * chore(deps): bump actions/download-artifact from 4 to 8 (diegosouzapw#3733) Integrated into release/v3.8.23 — download-artifact v4→v8. * feat(fallback): add OMNIROUTE_EMERGENCY_FALLBACK env switch (diegosouzapw#3741) Adds an OMNIROUTE_EMERGENCY_FALLBACK env switch to disable the emergency budget-exhaustion fallback (reroute to free nvidia/gpt-oss-120b). Default unchanged (enabled). Closes diegosouzapw#3739, related diegosouzapw#2879. Integrated into release/v3.8.23. * i18n: comprehensive zh-CN translation improvements (diegosouzapw#3736) Aligns zh-CN to en (hundreds of entries), translates batch-action labels + settings sidebar menu, adds categoryConfig/endpointTokenSaver keys, resolves __MISSING__ stubs. Sidebar/SidebarTab hardcoded strings replaced with t(). en.json purely additive (8 new sidebar.* keys, 0 removed); cli-i18n gate green. Integrated into release/v3.8.23. * chore(release): v3.8.23 — 2026-06-12 - CHANGELOG: complete v3.8.23 section (28 bullets, 27 commits) - fix(webdav): resolve promise on writeStream finish, not req end — eliminates intermittent 500 on PUT update (writeStream may not have flushed at rename time) - test(autoCombo): stub DB calls from PR diegosouzapw#3660 in tieredRotation.test.ts to prevent 5s timeout in vitest (getModelIntelligenceBySource DB init path) - chore(env-sync): add XDG_DATA_HOME + OMNIROUTE_OPENCODE_PLUGIN_DIR to IGNORE_FROM_CODE allowlist (introduced by PR diegosouzapw#3726 setup-open-code.mjs, not OmniRoute config vars) - chore(cli): regenerated bin/cli/api-commands/*.mjs (7 new, 27 updated) * fix(model-family): fallback lookup also tries bare model name with dots getNextFamilyFallback normalized dots-to-hyphens ("gemini-3.1-pro-high" → "gemini-3-1-pro-high") but MODEL_FAMILIES keys use the literal dot form. The lookup always missed, returning null for any model whose dots are part of the name rather than a version separator. Fallback: try MODEL_FAMILIES[lookupKey] ?? MODEL_FAMILIES[bareModel] so both naming conventions are covered. Fixes T30 test (pre-existing since v3.8.22). * feat: expose API key cost drilldown + quota % used (diegosouzapw#3742) Adds all-time USD cost per API key in the API Key Manager, a per-key deep-link into the Cost Explorer (filtered + grouped by model), URL-param hydration of range/groupBy/apiKeyIds, and a '% used' quota display. Review adjustments: extracted URL-param parsers to a tested module (Rule diegosouzapw#18), i18n'd the new strings (en + zh-CN), dropped the redundant webdav-handler entry already on release. Integrated into release/v3.8.23. * feat: add provider display modes — All / Configured / Compact (diegosouzapw#3743) Replaces the Providers page configured-only toggle with All/Configured/Compact display modes (Compact = flat deduped grid, no-auth last). Persists the preference and migrates the legacy localStorage key. Rebased onto release/v3.8.23. Integrated into release/v3.8.23. * fix(cache): scope semantic-cache signature to API key (diegosouzapw#3740) Adds the api_key_id dimension to generateSignature's SHA-256 hash so two callers with different API keys never receive each other's cached responses. Threads apiKeyId through checkSemanticCache + both write sites; migration 098 clears pre-existing key-less entries; unauthenticated requests stay isolated from keyed ones. 3 TDD tests. Integrated into release/v3.8.23. * fix(responses): apply OpenAI Responses API stream=false spec default (diegosouzapw#3708) resolveStreamFlag now applies the stream=false-when-omitted default for sourceFormat=openai-responses (same as the existing claude path), so spec-compliant /v1/responses upstreams that return JSON no longer fall through to the wildcard-Accept heuristic and trigger STREAM_EARLY_EOF / 502. Codex CLI (stream:true) and explicit text/event-stream clients unaffected. Integrated into release/v3.8.23. * chore(release): reconcile CI gates for v3.8.23 - file-size baseline: re-freeze 8 files grown by PRs diegosouzapw#3742/diegosouzapw#3743/diegosouzapw#3740 (cost drilldown, provider display modes, cache key isolation) - ARCHITECTURE.md: update executor count 55→60 (check:docs-counts drift) - .env.example: add OMNIROUTE_EMERGENCY_FALLBACK (diegosouzapw#3741, env-doc-sync) - CHANGELOG: add formatted bullets for diegosouzapw#3742, diegosouzapw#3743, diegosouzapw#3708, diegosouzapw#3740, model-family-fallback fix; remove duplicate raw ### Fixed section * test: restore assert count to satisfy check:test-masking gate Three test files had net assertion removals after behavior-changing PRs: - chatcore-translation-paths: emergency fallback moved to routing layer (diegosouzapw#3699) — add body error assertion + model-name guard - executor-vertex-extended: non-JSON is now Express API key (diegosouzapw#3690) — add projects/-path guard to the express-key URL test - stream-utils: empty streams now emit error (diegosouzapw#3685) — add code/message/ status/completePayload guards to both passthrough and translate variants All new assertions are meaningful (code enum value, 5xx range, non-empty message, onComplete must-not-fire contract). * fix(ci): move rtl-logical-classes test to ui/ so vitest:ui runner collects it --------- Co-authored-by: Felipe Almeman <4226997+zhiru@users.noreply.github.com> Co-authored-by: NOXX - Commiter <artur1992123@mail.ru> Co-authored-by: Nick Sullivan <142708+TechNickAI@users.noreply.github.com> Co-authored-by: Markus Hartung <mail@hartmark.se> Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Co-authored-by: PizzaV <103120356+pizzav-xyz@users.noreply.github.com> Co-authored-by: Randi <55005611+rdself@users.noreply.github.com> Co-authored-by: oyi77 <14921983+oyi77@users.noreply.github.com> Co-authored-by: Chewji <126886556+Chewji9875@users.noreply.github.com> Co-authored-by: Hernan Javier Ardila Sanchez <hjasgr@gmail.com> Co-authored-by: Felipe Sartori <felipesartori.ti@gmail.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Zois Pagoulatos <zpagoulatos@hotmail.com> Co-authored-by: sdfsdfw2 <167810361+sdfsdfw2@users.noreply.github.com> Co-authored-by: Witroch4 <witalo_rocha@hotmail.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Add a sync pipeline that fetches Arena AI leaderboard ELO scores and derives model intelligence tiers from models.dev capabilities, replacing the hardcoded-only lookup with a 5-layer resolution chain.
Resolution Chain (highest → lowest priority)
New Files
src/lib/db/migrations/097_model_intelligence.sqlsrc/lib/db/modelIntelligence.tssrc/lib/arenaEloSync.tssrc/app/api/intelligence/sync/route.tstests/unit/model-intelligence-db.test.tstests/unit/arena-elo-sync.test.tsModified Files
taskFitness.tsautoCombo.test.tsschemas.tsintelligenceSyncRequestSchemaserver-init.tslocalDb.tsHow to Enable
Key Design Decisions
getTaskFitness()signature unchanged, FITNESS_TABLE preserved as lowest fallback_intelligenceCache) prevents 60+ DB queries per routing decisionReview Notes
getModelIntelligenceBySourcenow includes expiry check in SQL WHERE clauseSOURCE_PRIORITY,_dbAvailable,require()calls)Tests