fix(backend): fail closed when capability filters empty the combo pool

changelog.d/fixes/8488-capability-filter-fail-closed.md

@@ -0,0 +1 @@
+- **fix(backend):** capability filters fail closed when every combo target is incompatible, with opt-in `compatFilterFailOpen` ([#8488](https://github.com/diegosouzapw/OmniRoute/issues/8488)) — thanks @Prudhvivuda

config/quality/eslint-suppressions.json

@@ -1152,7 +1152,7 @@
   },
   "tests/unit/combo-routing-engine.test.ts": {
     "@typescript-eslint/no-explicit-any": {
-      "count": 271
+      "count": 267
     }
   },
   "tests/unit/combo-same-provider-cascade.test.ts": {

config/quality/file-size-baseline.json

@@ -1,4 +1,5 @@
 {
+  "_rebaseline_2026_07_25_8494_capability_filter_fail_closed": "PR #8494 (fix/capability-filters-fail-closed, #8488) own growth: open-sse/services/combo.ts 3640->3693 (+53) adds a fail-closed guard after filterTargetsByRequestCompatibility() — when every eligible target is excluded by request-capability filtering (vision/tools/etc) instead of quota/health, the combo now returns an explicit `capability_mismatch` 400 (describeCapabilityFilterExhaustion, imported from combo/comboStructure.ts) rather than silently falling through to a generic no-targets error, plus a `compatFilterFailOpen` escape hatch (combo config OR settings) mirrored at both the main/auto and round-robin call sites for symmetry. combo/comboStructure.ts (previously under cap, un-frozen) grows 794->918 (+124) — new home for describeCapabilityFilterExhaustion + providerSupportsEmulatedToolCalling (#5240 emulated tool-calling exemption so fail-closed does not regress prompt-emulation-only combos like all-chatgpt-web). Irreducible orchestration wiring at the existing filter chokepoint (same precedent as #7301's universal-cooldown-retry generalization). Companion test tests/unit/combo-routing-engine.test.ts 3409->3449 (+40, fail-closed/fail-open coverage across both call sites) also rebaselined. Covered by tests/unit/8488-capability-filter-fail-closed.test.ts (new) + 95/95 passing across both files. Structural shrink of combo.ts tracked in #3501.",
   "_rebaseline_2026_07_25_8499_ts7_result_union_predicates": "PR #8499 (backryun, chore/ts7-types-executor-scattered) own growth: muse-spark-web.ts 1396->1405 (+9, irreducible). Under this workspace's `strictNullChecks: false`, the boolean-literal discriminant on `GraphqlResult` (`{ ok: true } | { ok: false; error: string }`) narrows the positive `.ok===true` branch but leaves `!result.ok` at the full union under TS7, making `.error` unreachable to the checker at the two call sites (warmup, mode-switch). Fixed by adding a single `isGraphqlFailure()` type-predicate helper (doc comment + 3-line body) reused at both call sites instead of duplicating the predicate inline — not extractable to a shared module without splitting a single-file executor's local narrowing helper out of its own file. Covered by the existing muse-spark-web executor test suite (no behavior change, pure narrowing fix).",
   "_rebaseline_2026_07_22_8131_windowshide_cloudflared_spawn": "PR #8167 (Dingding-leo, fix/windows-hide-child-process, #8131) own growth: src/lib/cloudflaredTunnel.ts 934->935 (+1, irreducible call-site wiring — the single `windowsHide: true` option added to the existing cloudflared spawn() options object so no transient conhost.exe/cmd console window flashes open on Windows). Covered by the pre-merge-fix regression test tests/unit/windows-hide-child-process-spawns-8131.test.ts (added for the two additional spawn() sites the PR missed: ServiceSupervisor.ts, versionManager/processManager.ts) plus the windowsHide assertion added to tests/unit/services/installers/runNpm-shell-5379.test.ts (installers/utils.ts buildNpmExecOptions).",
   "_rebaseline_2026_07_22_8006_adobe_firefly_media_provider": "PR #8006 (artickc, feat/adobe-firefly-media) own growth: adds Adobe Firefly as a media-only (image + video) provider — unofficial IMS/cookie-session bridge for firefly.adobe.com covering IMS cookie->access_token exchange, discovery-catalog fallback, credits/balance usage, and submit+poll dispatch for both image (nano-banana/gpt-image families) and video (Sora 2/Veo 3.1/Kling 3.0) generation, with 408-under-load retry handling. New leaf open-sse/services/adobeFireflyClient.ts frozen at 1958 (>>cap 800) — a single self-contained upstream client (mirrors the qoderCli.ts precedent for a new provider client that is legitimately large on day one: IMS auth, cookie/JWT normalization, payload builders for 2 media types x multiple model families, SSE-less submit/poll state machine, error sanitization); not extractable without scattering a single upstream integration across artificial module boundaries mid-PR. open-sse/config/imageRegistry.ts (existing, previously under cap) grows 800->821 (+21, the new adobe-firefly IMAGE_PROVIDERS entry + models list, additive registry data at the existing registry chokepoint). src/lib/usage/providerLimits.ts 1000->1003 (+3, adobe-firefly/firefly added to the existing apikey-usage-fetcher allowlist, irreducible call-site wiring mirroring the sibling #7994 PromptQL/HyperAgent entries in the same PR group). Covered by tests/unit/adobe-firefly.test.ts (35/35). Structural shrink tracked in #3501.",
@@ -208,7 +209,8 @@
     "_rebaseline_2026_06_24_task_aware_routing": "Task-aware routing strategy (port PR #2045, OmniRoute #4945): combo.ts 3190->3225 (+35) = one new `else if (strategy === \"task-aware\")` dispatch branch delegating 100% to selectTaskAwareTarget + its imports/log lines. All scoring/classification logic lives OUT of the god-file in the new leaf open-sse/services/taskAwareRouting.ts (553 LOC <cap). Only the dispatch wiring is irreducible at the existing combo strategy chokepoint (mirrors quota-share/headroom/reset-aware branches). ZERO existing strategy cases modified. Covered by tests/unit/combo-task-aware.test.ts (35 tests). Structural shrink of combo.ts tracked in #3501.",
     "_rebaseline_2026_07_22_8213_combo_cooldown_wait_recording": "PR #8213 (hartmark, fix/gemini-tpm-quota-cooldown-wait) own growth: open-sse/services/combo.ts 3548->3604 (+56, entirely this PR's diff — no other commit touched this file between the PR's merge-base and the release tip). Fixes combo cooldown-wait state recording so a bogus 503 is no longer crystallized when the cooldown-wait vars reset every setTry, adds an OpenAI-format SSE error frame path for combo-exhausted rejections (capturing request body + attempted models), and gives an abandoned per-target dispatch its own timeout instead of leaking a permanent 'pending' dashboard entry. Irreducible additions at the existing handleComboChat dispatch/retry chokepoint (mirrors the prior quota-share/headroom/task-aware strategy-branch precedents already frozen in this file). Covered by the PR's own combo-config + Gemini TPM-ceiling benchmark test additions.",
     "_rebaseline_2026_07_25_8476_combo_input_bound_homogeneous_scope": "PR #8476 (herjarsa, fix/8375-8459-combo-image-fixes, #8375) own growth: open-sse/services/combo.ts 3642->3679 (+37 net: +29 the PR's own isInputBoundFailure short-circuit for deterministic context_length_exceeded/context_window_exceeded failures, +8 a /green-prs pre-merge fix scoping that short-circuit to homogeneous remainders only — the shipped code fired unconditionally on ANY target, regressing the intentional heterogeneous-combo fallback #6637/isContextOverflow400 protects, exactly as flagged by this PR's own review evidence but never actually implemented in the branch). The fix compares orderedTargets[i+1..] modelStr against the failing target's modelStr at the existing executeTarget dispatch chokepoint (mirrors the sameProviderNext precedent a few lines below) — irreducible call-site wiring, not extractable without hiding the dispatch boundary. Covered by tests/unit/combo-input-bound-failure-8375.test.ts (homogeneous pool still short-circuits) and the new tests/unit/combo-input-bound-heterogeneous-8375.test.ts (heterogeneous combo now correctly falls through to the larger-context target).",
-    "open-sse/services/combo.ts": 3679,
+    "open-sse/services/combo.ts": 3642,
+    "open-sse/services/combo/comboStructure.ts": 917,
     "_rebaseline_2026_06_26_fidelity_gate_extraction": "Milestone-B fidelity-gate wiring residual: bodyToText+gateAdvance extracted to fidelityGateStep.ts (889->854, -35), but the StackOptions.fidelityGate field, the `const fidelityGate` reads at the two stacked-loop dispatch chokepoints, and the import of FidelityGateConfig are irreducible wiring that cannot leave strategySelector without an architectural refactor of the pre-existing stacked pipeline. Net: 889->854 (+6 vs the pre-Milestone-B frozen 848). Covered by tests/unit/compression/*.test.ts (940 pass).",
     "_rebaseline_2026_06_28_5243_risk_gate_prepass": "PR #5243 (compression risk-gate pre-pass) own growth: open-sse/services/compression/strategySelector.ts 854->899 (+45). The three exported entry points (applyCompression/applyStackedCompression/applyStackedCompressionAsync) become thin wrappers over pure-extracted private bodies (runCompression/runStackedCompression/runStackedCompressionAsync) so the risk-gate mask->run->restore wrapper sits strictly OUTSIDE the per-step loop — a single universal integration point. The wrapper logic itself (resolveRiskGate/withRiskGate) lives in the new riskGate/strategyWrap.ts (<cap); the residual growth is the duplicated thin-wrapper signatures + the extracted bodies' dispatch boundary, guarded by a byte-identical parity test (riskGateIntegration). Default off (DEFAULT_COMPRESSION_CONFIG unchanged). Not extractable without hiding the dispatch boundary, mirroring prior compression rebaselines. Structural shrink tracked in #3501.",
     "_rebaseline_2026_06_29_5286_memoization": "PR #5286 own growth: strategySelector.ts 899->960 (+61 = the opt-in result-memoization branches in applyCompression/applyCompressionAsync — principal+determinism gate, makeMemoKey lookup/store with model+supportsVision folded into the key, recompute-with-memo-off). Default off (memoizeCompressionResults), so zero behavior change. The memo helpers live in the leaf resultMemo.ts (<cap); the chokepoint wiring here is not extractable. Structural shrink of this hot-path file tracked in #3501.",

open-sse/services/combo.ts

@@ -196,6 +196,7 @@ import { attemptCompatRejectedFallback } from "./combo/comboCompatFallback.ts";
 import { applyContextRequirements } from "./combo/contextRequirements.ts";
 import {
   computeCompatRejectedTargets,
+  describeCapabilityFilterExhaustion,
   filterTargetsByRequestCompatibility,
   resolveComboRuntimeUnits,
   resolveComboTargets,
@@ -1246,7 +1247,33 @@ export async function handleComboChat({
   if (!cacheStrategyAffinityApplied) {
     orderedTargets = orderTargetsByEvalScores(orderedTargets, config.evalRouting, log);
   }
-  orderedTargets = filterTargetsByRequestCompatibility(orderedTargets, body, log);
+  const compatFilterFailOpen =
+    (config as { compatFilterFailOpen?: unknown }).compatFilterFailOpen === true ||
+    (settings as { compatFilterFailOpen?: unknown } | null | undefined)?.compatFilterFailOpen ===
+      true;
+  const preCompatTargets = orderedTargets;
+  orderedTargets = filterTargetsByRequestCompatibility(orderedTargets, body, log, undefined, {
+    failOpen: compatFilterFailOpen,
+  });
+  if (orderedTargets.length === 0 && preCompatTargets.length > 0) {
+    const exhaustion = describeCapabilityFilterExhaustion(preCompatTargets, body, combo.name);
+    if (exhaustion) {
+      recordComboFailure(effectiveSessionId, combo.name);
+      return errorResponseWithComboDiagnostics(
+        400,
+        exhaustion.message,
+        {
+          poolSize: preCompatTargets.length,
+          attempted: 0,
+          excluded: exhaustion.excluded,
+          attemptOrder: [],
+          terminalReason: exhaustion.terminalReason,
+          recovery: buildRecoveryHint("no_executable_targets"),
+        },
+        { code: "capability_mismatch", type: "invalid_request_error" }
+      );
+    }
+  }
   orderedTargets = applyContextRequirements(orderedTargets, config.contextRequirements, log);
 
   // Task-aware reordering: only active for strategies ["smart","task","task-aware","task_aware","auto"].
@@ -2859,11 +2886,17 @@ async function handleRoundRobinCombo({
       { code: "context_length_exceeded", type: "invalid_request_error" }
     );
   }
+  // Align with the main/auto paths: combo config OR top-level settings.
+  const rrCompatFailOpen =
+    (config as { compatFilterFailOpen?: unknown }).compatFilterFailOpen === true ||
+    (settings as { compatFilterFailOpen?: unknown } | null | undefined)?.compatFilterFailOpen ===
+      true;
   let filteredTargets = filterTargetsByRequestCompatibility(
     evalRankedTargets,
     body,
     log,
-    "Context-aware round-robin fallback"
+    "Context-aware round-robin fallback",
+    { failOpen: rrCompatFailOpen }
   );
   // #6238: keep the targets the compat pre-filter rejected so they can serve as a
   // last-resort fallback tier. The pre-filter drops request-incompatible targets
@@ -2877,6 +2910,25 @@ async function handleRoundRobinCombo({
   );
   let modelCount = filteredTargets.length;
   if (modelCount === 0) {
+    const exhaustion = describeCapabilityFilterExhaustion(
+      evalRankedTargets,
+      body,
+      rrExpandedCombo?.name || combo?.name
+    );
+    if (exhaustion) {
+      return errorResponseWithComboDiagnostics(
+        400,
+        exhaustion.message,
+        {
+          poolSize: evalRankedTargets.length,
+          attempted: 0,
+          excluded: exhaustion.excluded,
+          attemptOrder: [],
+          terminalReason: exhaustion.terminalReason,
+        },
+        { code: "capability_mismatch", type: "invalid_request_error" }
+      );
+    }
     return comboModelNotFoundResponse("Round-robin combo has no executable targets");
   }