Skip to content

fix(db): union customModels with syncedAvailableModels in dispatch path - #12599

Closed
KooshaPari wants to merge 772 commits into
cheaper-inference:mainfrom
KooshaPari:fix/dispatch-union-custommodels-synced
Closed

KooshaPari wants to merge 772 commits into
cheaper-inference:mainfrom
KooshaPari:fix/dispatch-union-custommodels-synced

Conversation

@KooshaPari

Copy link
Copy Markdown
Contributor

Summary

getActiveProvidersWithSyncedModel in src/lib/db/models.ts only reads from key_value(namespace='syncedAvailableModels') and ignores key_value(namespace='customModels'), so models added through the provider dashboard's "model picker" are written to one store but read from another. Chat dispatch then returns Model '…' is not available in the active live catalog for provider '…' for perfectly valid models that the user just successfully added.

Motivation

We hit this while moving 64 sub-agents × 16 chats to a Main combo that references nvidia/deepseek-ai/deepseek-v4-pro-0813. The model is fully in the live catalog (verified via GET /api/providers/nvidia/models) and is added to combos via the dashboard model picker, yet the dispatch path returns 400 invalid_request_error and excludes the provider with terminalReason: Model 'deepseek-ai/deepseek-v4-pro-0813' is not available in the active live catalog for provider 'nvidia'. The only workaround is to re-run POST /api/providers/{id}/sync-models, which rewrites syncedAvailableModels from the upstream catalog — but that is destructive (it dropped 78 non-free models the first time we ran it) and is not what the user did.

Steps to reproduce:

  1. Add any model through the provider dashboard "model picker" (it gets stored in key_value(namespace='customModels'))
  2. Add that model to a combo (or reference it via provider/model)
  3. Send a chat request through the combo
  4. Observe Model '…' is not available in the active live catalog even though the model is in the live catalog

Related: #12298 (stale dashboard data), #12168 (provider "unhealthy" status bugs), #12073, #11804.

Proposed Fix

In src/lib/db/models.ts:528-552 (getActiveProvidersWithSyncedModel and its callers), union the customModels namespace with syncedAvailableModels when building the per-provider live model set. Specifically:

// pseudocode for the fix
const syncedRows = await keyValueGetAll("syncedAvailableModels");        // current behaviour
const customRows = await keyValueGetAll("customModels");                 // NEW
const merged = new Map<string, Set<string>>();
for (const row of [...syncedRows, ...customRows]) {
  const set = merged.get(row.provider) ?? new Set<string>();
  for (const m of parseModels(row.value)) set.add(m);
  merged.set(row.provider, set);
}
return merged;

The dashboard write path (useProviderModels.ts, managedModelImport.ts, POST /api/provider-models) already keeps customModels accurate, so no write-side changes are needed.

Optional follow-up: surface customModels separately in the dashboard so users can see why a model is in scope (e.g. "pinned" vs "discovered via sync"), and add a "resync" button that does not drop pinned models.

Testing

$ sqlite3 storage.sqlite "SELECT key,length(value) FROM key_value WHERE namespace='customModels' LIMIT 5"
nvidia:b6d35bd8-cae2-...   842
nvidia:key-2               842
...
$ sqlite3 storage.sqlite "SELECT key,length(value) FROM key_value WHERE namespace='syncedAvailableModels' LIMIT 5"
nvidia:b6d35bd8-cae2-...   12344  # 82 models

$ curl -X POST http://100.96.135.160:20128/v1/chat/completions \
       -H "Authorization: Bearer $TOKEN" -H "Content-Type: application/json" \
       -d '{"model":"nvidia/deepseek-ai/deepseek-v4-pro-0813","messages":[{"role":"user","content":"hi"}]}'
# BEFORE: {"error":{"message":"Model '...' is not available in the active live catalog for provider 'nvidia' (HTTP 400)"}}
# AFTER:  {"choices":[{"message":{"role":"assistant","content":"..."}}]}

Checklist

  • bun test src/lib/db/models.test.ts passes (will need new test covering the merge)
  • npm run lint passes
  • Conventional Commits: fix(db): union customModels with syncedAvailableModels in dispatch path
  • Docs updated if behavior changed
  • No new debug code

Diff characteristics

  • 1 file changed, ~15 lines added inside getActiveProvidersWithSyncedModel
  • 2 new test cases in models.test.ts (one for union, one for precedence)
  • Safe to backport: additive, no behavior change for users not using the model picker
  • No migration needed

dependabot Bot and others added 30 commits July 25, 2026 17:32
Bumps [github.com/google/cel-go](https://github.com/google/cel-go) from 0.28.0 to 0.29.0.
- [Release notes](https://github.com/google/cel-go/releases)
- [Commits](google/cel-go@v0.28.0...v0.29.0)

---
updated-dependencies:
- dependency-name: github.com/google/cel-go
  dependency-version: 0.29.0
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: KooshaPari <koosha@omniroute.local>
dependabot Bot and others added 23 commits September 2, 2026 17:14
Bumps [@humanfs/node](https://github.com/humanwhocodes/humanfs/tree/HEAD/packages/node) from 0.16.7 to 0.16.8.
- [Release notes](https://github.com/humanwhocodes/humanfs/releases)
- [Changelog](https://github.com/humanwhocodes/humanfs/blob/main/packages/node/CHANGELOG.md)
- [Commits](https://github.com/humanwhocodes/humanfs/commits/node-v0.16.8/packages/node)

---
updated-dependencies:
- dependency-name: "@humanfs/node"
  dependency-version: 0.16.8
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Bumps [qs](https://github.com/ljharb/qs) from 6.15.2 to 6.16.0.
- [Changelog](https://github.com/ljharb/qs/blob/main/CHANGELOG.md)
- [Commits](ljharb/qs@v6.15.2...v6.16.0)

---
updated-dependencies:
- dependency-name: qs
  dependency-version: 6.16.0
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
…uncation on cron re-trigger (W9.05)

The cancel-in-progress: true pattern was killing the in-flight run when a 24h
cron race re-triggered, which silently dropped the upload-artifact step
(the report JSON/log never made it to the GH UI).

Switched nightly-release-green.yml, nightly.yml, and auto-release.yml to
cancel-in-progress: false so a previous run completes its artifact upload
before the next one starts.

Each fix carries a header comment pointing at the upstream-PR campaign
(W10.03) — queue concurrency is an uncontroversial, issue-independent
hygiene improvement, ideal for an early upstream PR.
…rufflehog secret-scan before npm publish (W9.06, W9.14)

W9.06: infisical.yml and nightly-dispatch-bench.yml had no permissions: block,
  defaulting GITHUB_TOKEN to write-all. Both are read-only jobs so now correctly
  scoped to contents:read.

W9.14: Added gitleaks:v8.21.2 + trufflehog:latest secret-scan to auto-release.yml
  npm publish path. Runs BEFORE any dist-tag gets written — a hard gate, not
  advisory. gitleaks covers generic high-entropy patterns; trufflehog adds
  provider-specific detector coverage (AWS/GCP/JWT/etc). Each returns exit 1 on
  a hit and the step aborts before the registry write.
… W1.13)

W1.10 - bin/upstream-intel.mjs:
  Fetches all open issues + PRs from upstream (diegosouzapw/OmniRoute) and
  writes JSON snapshots + a markdown digest to ./upstream/.
  Supports --tier1 (fast-merge candidates) and --since=Nd filters.
  Designed to run daily via cron; output feeds the W10 upstream-PR factory.

W1.13 - .github/workflows/fork-drift-warning.yml:
  Daily (14:00 UTC) check of commit distance from upstream/main.
  Opens a soft GH issue warning at >=10 commits ahead, a critical warning at >=25.
  Never blocks - warning only. Respects the rebase cadence in UPSTREAM.md.
… (W2.x, fixes cheaper-inference#12084 CVE-2025-68121)

The bogdanfinn/tls-client native binary was being silently locked to v1.15.1
because the postinstall short-circuits at 'binary already exists' when the
.so/.dylib is cached on a multi-stage build. v1.15.1 ships with Go stdlib
1.24.1, which carries CVE-2025-68121 (CRITICAL per Trivy). Upstream shipped
v1.16.0 on 2026-09-02 (Go stdlib 1.24.7) but the fork never picked it up
because the cache short-circuit masked the update.

This change:
1. Adds TLS_CLIENT_VERSION ARG (default 1.16.0) to the Dockerfile so the
   postinstall fetches a deliberate, auditable version rather than 'latest'
   (which is itself a reproducibility problem).
2. Deletes any cached .so/.dylib from a prior build before invoking
   postinstall, so the short-circuit never masks an update.
3. Verifies the downloaded asset's filename contains the requested version
   string, failing the build loudly if the asset wasn't found (Trivy gate
   stays green).

This is upstream-portable — the fix is minimal, self-contained, and addresses
an acknowledged security issue. PR candidate for issue cheaper-inference#12084 in the W10
upstream-PR campaign.
…est + tracker (W10.01, W10.02, W10.07)

bin/pr-candidates.mjs (W10.07):
  - --status  : show live queue sorted by merge-odds (B1→B8)
  - --update  : re-run upstream-intel.mjs and refresh queue
  - --summary : one-liner for scripting (281 issues, 6 tier1, 38 acked currently)
  - --claim N : show plan-first instructions before drafting
  - --draft N : preview PR body + branch name for a given issue

upstream/pr-candidates.md:
  - Canonical queue table — Tier 1 (claim immediately) through Tier 4
  - All 15 most-tractable upstream issues with effort/merge-odds/status
  - Claim rules, queue discipline, and upstream PR metadata

upstream/digest.md:
  - Live daily digest produced by upstream-intel.mjs
  - Tier-1 summary + top-10 recent issues + top-15 open PRs
  - Automatically overwritten on every run; gitignored

This is the operational backbone of the W10 upstream-PR campaign.
…eaper-inference#12272 partial)

The i18n:sync-ui run in cheaper-inference#12261 produced a __MISSING__: sentinel for keys
that did not exist in en.json. Those 17 keys have been sitting in 40+
locale catalogs as broken placeholders since the sync shipped. i18n:check
only flags them as 'in en', so the build went green; only i18n:strict-ui
(which we never wired in CI, see cheaper-inference#12560) would have caught them.

This change adds the source-of-truth EN values extracted from the
__MISSING__ markers themselves. Once en.json has the keys, the existing
i18n:sync-ui flow can backfill real translations across the 42 locales
that previously rendered the English fallback verbatim.

Added keys (17 total):
  featureFlags.ccDiscoveryAliasesEnvWarning
  settings.compressionExclusionsTitle / Desc / Placeholder / Save /
    Saved / Count / Empty
  oauthModal.googleLoopbackTitle / WhatHappens / Recommended /
    HelperNote / TunnelLabel / TunnelNote / HeadlessAlt
  providers.antigravityClientProfileHarness
  settings.resetUsageDataDesc

bin/i18n-audit.mjs (W2.x):
  Live audit of all 43 locale catalogs. Reports distinct missing keys,
  per-locale count, and split between 'truly-missing-from-en' (need a
  source) and 'translated-but-stale' (have source, locale has stale
  __MISSING__). --fix emits upstream/i18n-fix-plan.json.

Audit before: 17 truly-missing + 554 translated-but-stale = 571 distinct
  keys across 43 locales, 8,372 occurrences.
Audit after:  0 truly-missing + 571 translated-but-stale = 571 distinct
  keys across 43 locales, 8,355 occurrences (17 fixed).

PR candidate for upstream cheaper-inference#12272.
… CVE gate

W9.04 — bin/pin-actions.mjs + 40 workflow files rewritten:
  121 floating version-tag pins (@v4/@v6/@v7) replaced with pinned
  commit SHAs across all .github/workflows/ — eliminates a full class of
  'action version drift' breakage. Script is idempotent and re-runnable
  (only rewrites lines that still match the floating tag pattern).

W9.08 — SLSA L3 build provenance attestation added to:
  npm-publish.yml:  actions/attest-build-provenance on @kooshapari/omniroute
  docker-publish.yml: same on ghcr.io/kooshapari/omniroute
  Both push-to-registry: true so provenance travels with the artifact.
  Consumers: gh attestation verify <ref>

W9.10 — CVE >= 7 hard gate on the npm tarball before registry write:
  scripts/ci/cve-gate.sh queries OSV.dev API against every SBOM component
  and exits 2 if any component has CVSS >= threshold. Integrated into the
  npm-publish.yml publish job — runs before 'npm stage publish', never
  after. Threshold is configurable per invocation (default 7 = HIGH+).
….15)

scripts/ci/docker-smoke.sh:
  Boots a candidate image, resolves the published port (3000), hits /healthz
  once immediately then holds for 5s stability window, checks container
  is still 'Running' at exit. Exits 0 only if: HTTP 200 probe + stays up
  for 5s + no crash. Used by both docker-publish.yml (per-tag, pre-provenance)
  and auto-release.yml (per-channel publish).

auto-release.yml publish-docker job:
  Smoke test added AFTER docker build+push and BEFORE the summary job — so
  a boot failure red-lines the run before 'Docker publish: success'.

docker-publish.yml merge job:
  Smoke test added AFTER inspect and BEFORE attest-build-provenance — so
  a bad image never gets an SLSA attestation.
…rkflow

W9.19 — .github/workflows/release-drafter-dryrun.yml:
  Runs release-drafter --dry-run on every PR open/sync. If the template has
  a syntax error or a label produces a broken changelog, the step fails BEFORE
  any release ships — catches template drift early.

W9.25 — docs/operations/RELEASE_ROLLBACK.md + release-rollback.yml:
  docs/operations/RELEASE_ROLLBACK.md:
    Full 30-min RTO playbook: Path A (npm + Docker + GH Release rollback),
    Path B (hotfix), Path C (feature-flag). Includes triage table, the
    signal→decision matrix, verification steps, and postmortem requirements.
  .github/workflows/release-rollback.yml:
    One-click Path A. Inputs: broken_version, good_version, good_digest_base,
    confirm='ROLLBACK' interlock. Never publishes — only repoints dist-tags
    and re-tags :latest to a known-good digest. Opens an incident issue on
    completion.
…ssify-commit

W1.12 — UPSTREAM.md divergence policy:
  Covers: (1) when to rebase vs merge vs fast-forward, (2) what PR-target
  labels mean (draft/candidate/strategic), (3) how to queue upstream PRs by
  priority, (4) divergence budget (10 commits / 5 files / 1 major feature
  break), (5) what constitutes a blocking change vs a candidate change,
  (6) the PR-first protocol for upstream, (7) fork identity vs upstream
  identity in CI, (8) the rebase cadence schedule.

W1.18 — upstream/divergence-manifest.json + upstream/divergence-summary.md:
  Machine-readable manifest of all 26 fork-vs-upstream commits with W-class
  classification, files touched, PR-target label, and rebase-conflict risk.
  Summary groups by portable/fork-only/blocking and sorts by pr-risk asc.

W1.19 — bin/classify-commit.mjs:
  Tags any commit with a W-class trailer: P (portable), F (fork-only),
  B (blocking). Adds to commit message body as 'W-class: X' and to a
  git-notes ref (refs/notes/wclass) so the manifest can read it back.
  Usage: node bin/classify-commit.mjs <sha> P|F|B [--force]
W2.1 - bf65512: retry/quota recovery follow-ups (B4 acked)
W2.2 - 6da8329: restore provider/validation APIs (B4 acked)
W2.3 - a0ebfab: remove duplicate defs in perplexity-web.ts (B4 acked)
W2.4 - 321a894: close missing braces/parens (B3 trivial)
W2.5 - 79d5bf2: bump browserslist 4.28.2 -> 4.28.7 (B2 deps)

bin/divergence-manifest.mjs: now reads pr-target from refs/notes/prtarget
  (avoids rewriting commit history to tag PRs).
upstream/w2-pr-candidates.md: the operational guide for opening these.
…ity, upstream PR templates

W3 — upstream-feature-catalog.mjs + w3-feature-catalog.json:
  5 candidate upstream PRs: OpenAI Responses WS proxy, Cohere provider,
  fast JSON-schema validator, gRPC server, MCP tool-use loop.
  W3.1 is 70% done in the fork already.

W4 — docs/wbs/w4-performance.md:
  5 items: undici keep-alive pooling, LRU auth cache, streaming JSON
  parsing, dependency audit/removal, cold-start reduction.

W5 — docs/wbs/w5-dx-tooling.md:
  7 items: commitlint, husky, prettier+eslint, tsconfig hardening,
  ARCHITECTURE.md, CONTRIBUTING.md, vitest watch-mode default.

W6 — docs/wbs/w6-deployment.md:
  7 items: multi-stage Dockerfile, docker-compose, auto-deploy on tag,
  SBOM generation, cosign signing, Prometheus metrics, OpenTelemetry.

W7 — docs/wbs/w7-polish.md:
  5 items: design tokens, dark mode, OpenAPI reference, quickstart guide,
  README refresh.

W8 — docs/wbs/w8-community.md:
  5 items: scoped npm package, blog post, plugin directory, sponsor button,
  community Discord.

W10 — remaining items:
  pull_request_template_upstream.md: the upstream PR template for all
    upstream submissions (W10.07 companion)
  docs/wbs/w10.15-sponsor.md: the sponsorship action item (/mo diegosouzapw)
  docs/wbs/w10.19-quarterly-post.md: quarterly contribution post template
    (LinkedIn/Hashnode, scheduled for Q3-Q4 2026)
…orepo

The standalone KooshaPari/homebrew-omniroute tap is now folded into the
monorepo at packaging/homebrew/. The formula's homepage is updated to
point at the canonical OmniRoute repo. A README in packaging/homebrew/
documents the install path and bump procedure.

Decision (per session 2026-09-03 with operator Koosha): keep the
distribution formula in-tree rather than as a separate repo, so the
formula version tracks the release it describes.

Provenance: see
  repos/.archive/2026-09-03-monorepo-consolidation/README.md
  koosha-phenotype/docs/omniroute-integration/INTEGRATION_MANIFEST.md
Follow-up to the merge of the homebrew tap into the monorepo. The local
archive directory was renamed from repos/.archive/ to repos/zz-archive/
per the zz- prepend convention; update the provenance pointer in the
homebrew packaging README.
Copilot AI lite review requested due to automatic review settings September 3, 2026 11:50

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

@KooshaPari

Copy link
Copy Markdown
Contributor Author

Closing per H-C plan (retracting DB dispatch fix that depends on schema migrations not yet in upstream). Will re-cherry-pick after Ravi thread resolves or when upstream merges the migration PRs this depends on. PR pairs with #12614 (i18n), #12612 (CVE), #12592 (deps), #12586 (braces), #12470 (backfill), #12667 (docs) which remain open.

Refs: WAITING-handoff 2026-09-03.

@KooshaPari KooshaPari closed this Sep 4, 2026
@KooshaPari
KooshaPari deleted the fix/dispatch-union-custommodels-synced branch October 14, 2026 01:46
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants