Skip to content

fix(providers): custom OpenAI-compatible node models keep synced vision capability (#14081) - #14428

Merged
diegosouzapw merged 1 commit into
release/v3.8.51from
fix/14081-custom-node-modalities
Sep 24, 2026
Merged

diegosouzapw merged 1 commit into
release/v3.8.51from
fix/14081-custom-node-modalities

Conversation

@diegosouzapw

Copy link
Copy Markdown
Collaborator

Closes #14081

Root cause

A custom OpenAI-compatible node's synced model persists a per-connection supportsVision boolean on its syncedAvailableModels row (detectVisionInput in src/lib/providerModels/modelDiscovery.ts). /v1/models correctly reads that field through buildSyncedCapabilities (src/app/api/v1/models/syncedCapabilities.ts) and reports capabilities.vision: true.

The Vision Bridge guardrail (src/lib/guardrails/visionBridge.ts) gates on getResolvedModelCapabilities(model).supportsVision, whose resolveVisionCapability() (src/lib/modelCapabilities.ts) had no code path back to syncedAvailableModels at all — confirmed by ModelCapabilityResolutionSnapshot, which bulk-loads models.dev sync data and several override maps but never this per-connection table. The guardrail disagreed with the catalog it feeds and rerouted an image-capable custom-node model as text-only, causing the fallback model to 400 on the image request.

Fix

Added a new positive-only vision source, syncedAvailableModelVision (src/lib/db/models/syncedAvailableModelVision.ts):

  • Bulk-loaded into ModelCapabilityResolutionSnapshot (mirrors customVisionOverrides) so catalog/build-local generation stays one SQLite read instead of N+1 queries.
  • Consulted on-demand in getResolvedModelCapabilities() when no snapshot is supplied (the Vision Bridge guardrail's per-request path).
  • Threaded into resolveVisionCapability() as a new parameter, checked after the isKnownTextOnlyDespiteSync hard override (so the fix(sse): route image requests only to confirmed-vision combo targets #4071 safety net is never bypassed) and before the last-resort model-id heuristic.
  • Only ever returns true or null — never false — so it can add vision capability but never downgrade another source's verdict.

Regression test

tests/unit/vision-bridge-custom-node-modalities-14081.test.ts (2 cases):

  1. /v1/models catalog and getResolvedModelCapabilities agree on vision:true for a synced custom-node row (both the on-demand path and the snapshot-backed path).
  2. Positive-only contract: a synced row with supportsVision:false, or no row at all, is never upgraded to true by this source (fix(sse): route image requests only to confirmed-vision combo targets #4071 regression guard).

RED on the unfixed code (verified via git apply -R of the fix, no branch switch):

✖ issue #14081: /v1/models catalog and getResolvedModelCapabilities agree on vision:true for a synced custom-node row (3650.460797ms)
✔ issue #14081: positive-only contract -- a synced row without supportsVision:true is never upgraded (6223.709093ms)

GREEN after the fix:

✔ issue #14081: /v1/models catalog and getResolvedModelCapabilities agree on vision:true for a synced custom-node row (6032.852814ms)
✔ issue #14081: positive-only contract -- a synced row without supportsVision:true is never upgraded (4941.301785ms)
ℹ tests 2
ℹ pass 2
ℹ fail 0
EXIT:0

Gates run

  • node --import tsx/esm --test tests/unit/vision-bridge-custom-node-modalities-14081.test.ts → 2/2 pass
  • npm run typecheck:core → exit 0
  • npx eslint --suppressions-location config/quality/eslint-suppressions.json <changed files> → exit 0
  • node scripts/check/check-file-size.mjs → no violations on touched files
  • node scripts/check/check-complexity.mjs → OK — 2904 violations (baseline 3218)
  • node scripts/check/check-cognitive-complexity.mjs → OK — 1316 violations (baseline 1437)
  • node scripts/check/check-changelog-integrity.mjs → OK

⚠️ base-red inherited: #13866 (docs-sync-strict .env.example drift + open-sse/executors/codex.ts file-size drift) — not touched by this PR.

Existing tests aligned

None — ran the full touched-area vision/capability suite (142 tests across model-capability-resolution-snapshot-9199, model-capabilities-kimi-k3-vision-8250, model-capabilities-path-shaped-vision-8032, command-code-mimo-v2-5-safety, command-code-registry-vision, vision-bridge-cc-no-reroute, vision-bridge-custom-path-id-12758, guardrails/visionBridge*, combo-vision-aware-routing, combo-multimodal-capability-13847, fusion-vision-panel-3378, vision-compression-authoritative-capability-7237, api-models-hide-paid-6328, etc.) — all 142 passed unmodified, no #4071-class regression.

Plan-file: _tasks/pipeline/bugs/2-implementing/14081-fix-providers-custom-openai-compatible-node-models-lose-input-.plan.md

…on capability (#14081)

Vision Bridge gates on getResolvedModelCapabilities().supportsVision, whose
resolveVisionCapability() never consulted the per-connection
syncedAvailableModels row that already made /v1/models report
capabilities.vision:true for a custom node's synced model. Add a
positive-only syncedAvailableModelVision source (bulk-loaded in
ModelCapabilityResolutionSnapshot, on-demand DB helper otherwise) so the
guardrail agrees with the catalog it feeds instead of rerouting an
image-capable model as text-only.
@diegosouzapw
diegosouzapw merged commit 735ba13 into release/v3.8.51 Sep 24, 2026
21 checks passed
@diegosouzapw
diegosouzapw deleted the fix/14081-custom-node-modalities branch October 18, 2026 23:49
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

fix(providers): custom OpenAI-compatible node models lose input_modalities, so Vision Bridge reroutes image-capable models as text-only

1 participant