Skip to content

feat(vision-bridge): honor configured vision model + support custom model input & all active providers (#10808, #10809) - #10859

Merged
diegosouzapw merged 1 commit into
diegosouzapw:release/v3.8.50from
Stazyu:feat/vision-bridge-consolidated
Aug 21, 2026
Merged

diegosouzapw merged 1 commit into
diegosouzapw:release/v3.8.50from
Stazyu:feat/vision-bridge-consolidated

Conversation

@Stazyu

@Stazyu Stazyu commented Aug 20, 2026

Copy link
Copy Markdown
Contributor

Summary

  • Bug ([BUG] : Vision Bridge ignores configured Vision Model (eg. oc/mimo-v2.5-free) and routes/fallbacks to claude-opus-4-7 #10808) — the Vision Bridge ignored the operator-configured vision
    model (e.g. oc/mimo-v2.5-free) and fell back to command-code/claude-opus-4-7,
    producing repeated 403 MODEL_NOT_IN_PLAN failures. Root cause: the bridge
    credential gate queried provider_connections with the public alias (oc)
    instead of the canonical registry id (opencode), returning zero rows and
    excluding every candidate; no-auth providers were also judged by the
    "must have a stored API key" bar even though they authenticate through the
    synthetic noauth connection.
  • Feature ([Feature] Support Custom Model Input & All Active Providers in Vision Bridge #10809) — the Vision Bridge model picker was limited to a
    hardcoded available subset. It now reads the unified catalog (all
    configured providers + user-added custom models, matching the Audio tab),
    derives supportsVision from catalog capability/modality metadata, and adds
    an editable custom-input field so operators can type self-hosted/unlisted
    vision model ids (Ollama/vLLM, …).
  • Supporting fixes in the same commit: reasoning field fallback for
    opencode-routed gateways (SSE + non-stream parsing, empty-choices
    misclassification), Command Code wire-model normalization for bare ids
    (mimo-v2.5 → xiaomi/mimo-v2.5), stable Vision Bridge describe cache key
    (base prompt, not task-composed prompt), and cmd/gpt-5.3-codex* marked
    text-only in capability resolution.

Related Issues

Validation

Choose the change type and focused loop from the
Contribution Golden Path. The full unit suite,
Vitest, the 60% coverage gate, and the production build all run in CI on this PR (#8329):

  • Change type: provider / routing / UI / i18n / CLI / DB / build-deploy / other — provider + routing + UI
  • Focused tests and category gates from the golden path:
    • node --import tsx/esm --test tests/unit/guardrails/visionBridgeCredentials.test.ts
    • node --import tsx/esm --test tests/unit/guardrails/vision-bridge-cache-key.test.ts
    • node --import tsx/esm --test tests/unit/model-select-field-catalog-vision-10809.test.ts
    • node --import tsx/esm --test tests/unit/model-capabilities-command-code-codex-textonly-10703.test.ts
    • node --import tsx/esm --test tests/unit/command-code-vision.test.ts
    • node --import tsx/esm --test tests/unit/command-code-registry-vision.test.ts
    • node --import tsx/esm --test tests/unit/guardrails/vision-bridge-sse-and-reasoning.test.ts
    • node --import tsx/esm --test tests/unit/issue-6623-opencode-mimo-reasoning-details-nonstream.test.ts
    • node --import tsx/esm --test tests/unit/vision-bridge-cc-no-reroute.test.ts
  • npm run lint
  • Reconciled with the current active release base; focused checks rerun afterward
  • Production-code changes include a new or updated automated test in this PR
  • SonarQube is temporarily opt-in while the private project has no quota; it is not a PR gate.

Tests Added Or Updated

  • tests/unit/guardrails/visionBridgeCredentials.test.ts (new) — [BUG] Vision Bridge fails with "No vision-capable provider connected" despite active settings #10702
    regression: alias resolution + no-auth provider handling against the real
    DB-backed path (isolated DATA_DIR, resetDbInstance() cleanup).
  • tests/unit/guardrails/vision-bridge-cache-key.test.ts (new) — describe
    cache reuse across full-transcript turns; new images still force a fresh call.
  • tests/unit/model-select-field-catalog-vision-10809.test.ts (new) —
    readCatalogModels vision verdict derivation (capabilities / input
    modalities / conservative heuristic for custom models only).
  • tests/unit/model-capabilities-command-code-codex-textonly-10703.test.ts
    (new) — cmd/gpt-5.3-codex text-only override.
  • tests/unit/command-code-vision.test.ts — wire-model normalization
    (command-code/mimo-v2.5 → xiaomi/mimo-v2.5, aliased cmd/, prefixed
    passthrough); conservative vision-family lock kept for gpt-5.4-mini.
  • tests/unit/command-code-registry-vision.test.ts — registry verdict moves
    gpt-5.3-codex from vision → text-only list.
  • tests/unit/guardrails/vision-bridge-sse-and-reasoning.test.ts — plain
    reasoning field fallback (opencode gateway) for non-stream responses.
  • tests/unit/issue-6623-opencode-mimo-reasoning-details-nonstream.test.ts —
    reasoning-only non-stream completions no longer flagged empty.
  • tests/unit/ui/modality-bridge-vision-tab.test.tsx — Vision tab with
    catalog model source + custom input.
  • tests/unit/vision-bridge-cc-no-reroute.test.ts — no-reroute expectations
    for native-vision CC models.

Coverage Notes

  • Production changes span src/lib/guardrails/ (vision bridge), src/lib/ modelCapabilities.ts, src/shared/components/ModelSelectField.tsx,
    src/app/(dashboard)/dashboard/settings/components/modalityBridge/, and
    open-sse/ (commandCode executor, errorClassifier, diagnostics).
  • Coverage: new tests directly exercise the changed code paths — credential
    resolution (real DB-backed), cache keying, catalog vision derivation, wire
    model normalization, and reasoning-field parsing. UI change (Vision tab
    props) is covered by the updated modality-bridge-vision-tab.test.tsx.
  • No coverage regression expected in touched files; the new test files add
    statement/branch coverage to previously untested branches (no-auth provider
    path in visionBridgeCredentials.ts, reasoning fallback branches).

Reviewer Notes

  • No feature flags or DB migrations — all changes are in-memory/resolution
    logic; no schema change, no new env vars.
  • Intentional behavior change: hasUsableCredentialsForModel now treats an
    empty active connection set as usable for no-auth providers (oc/opencode,
    ddgw/duckduckgo-web, …). Terminal statuses (disabled/banned/expired)
    still block. This is what lets oc/mimo-v2.5-free pass the credential gate.
  • Cache key change (visionBridge.ts): keying switched from the
    task-aware composed prompt to the base prompt. Intentional and stable — the
    composed prompt appends the last user text and changes every turn, defeating
    the cache for full-transcript clients. A genuinely new image has a different
    contentRef and still misses.
  • Wire-model normalization (commandCode.ts) is a minimal, doc-backed
    allowlist (mimo-v2.5, mimo-v2.5-pro); ids containing / pass through
    untouched. Verify against Command Code's /alpha/generate when testing.
  • reasoning field fallback relies on OpenAI-compatible upstreams naming
    chain-of-thought reasoning (opencode gateway). reasoning_content remains
    the primary fallback.
  • Manual validation worth doing: configure oc/mimo-v2.5-free as the Vision
    Bridge model with a text-only target, send an image payload, and confirm the
    describe call goes to oc/mimo-v2.5-free (no fallback to claude-opus-4-7).

@Stazyu
Stazyu requested a review from diegosouzapw as a code owner August 20, 2026 15:32
@diegosouzapw
diegosouzapw merged commit 3805494 into diegosouzapw:release/v3.8.50 Aug 21, 2026
5 checks passed
backryun added a commit to backryun/OmniRoute that referenced this pull request Aug 21, 2026
…chable

diegosouzapw#10859 (3805494) taught the reroute heuristic to consider
not-combo text-only models, so the three describe-path assertions in
vision-bridge-maxchars.test.ts were silently hijacked by a live
reroute (meta.rerouted=true) before reaching the describe branch.
Inject hasUsableCredentials: async () => false — the definitive
exclusion signal per visionBridgeRouter's fail-open contract — so the
describe path under test actually runs. 5/5 green.
backryun added a commit to backryun/OmniRoute that referenced this pull request Aug 21, 2026
…chable

diegosouzapw#10859 (3805494) taught the reroute heuristic to consider
not-combo text-only models, so the three describe-path assertions in
vision-bridge-maxchars.test.ts were silently hijacked by a live
reroute (meta.rerouted=true) before reaching the describe branch.
Inject hasUsableCredentials: async () => false — the definitive
exclusion signal per visionBridgeRouter's fail-open contract — so the
describe path under test actually runs. 5/5 green.
backryun added a commit to backryun/OmniRoute that referenced this pull request Aug 21, 2026
…chable

diegosouzapw#10859 (3805494) taught the reroute heuristic to consider
not-combo text-only models, so the three describe-path assertions in
vision-bridge-maxchars.test.ts were silently hijacked by a live
reroute (meta.rerouted=true) before reaching the describe branch.
Inject hasUsableCredentials: async () => false — the definitive
exclusion signal per visionBridgeRouter's fail-open contract — so the
describe path under test actually runs. 5/5 green.
backryun added a commit to backryun/OmniRoute that referenced this pull request Aug 22, 2026
…chable

diegosouzapw#10859 (3805494) taught the reroute heuristic to consider
not-combo text-only models, so the three describe-path assertions in
vision-bridge-maxchars.test.ts were silently hijacked by a live
reroute (meta.rerouted=true) before reaching the describe branch.
Inject hasUsableCredentials: async () => false — the definitive
exclusion signal per visionBridgeRouter's fail-open contract — so the
describe path under test actually runs. 5/5 green.
muhamadgalihsaputra pushed a commit to niyatna/NiyatnaRoute that referenced this pull request Sep 27, 2026
…, creds, cache key, CC vision) (diegosouzapw#10859)

Validado no worktree combinado do lote: typecheck:core, lint, gates de qualidade (file-size rebaselineado com justificativa — crescimento legítimo em modelCapabilities.ts/commandCode.ts) e os 97+9 testes focados (vision-bridge, command-code vision, model-select-field-catalog-vision) todos verdes. Duas correções reais (diegosouzapw#10808/diegosouzapw#10809) bem documentadas. CI vermelho neste PR é o base-red já rastreado em diegosouzapw#9985. Obrigado!
muhamadgalihsaputra pushed a commit to niyatna/NiyatnaRoute that referenced this pull request Sep 27, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

2 participants