Skip to content

fix(auto-combo): short-circuit expandAutoComboCandidatePool when models[] is non-empty (closes #8597) - #8598

Merged
diegosouzapw merged 1 commit into
cheaper-inference:release/v3.8.49from
mikeiagents:fix/auto-combo-explicit-models-guard
Jul 27, 2026
Merged

diegosouzapw merged 1 commit into
cheaper-inference:release/v3.8.49from
mikeiagents:fix/auto-combo-explicit-models-guard

Conversation

@mikeiagents

@mikeiagents mikeiagents commented Jul 25, 2026 •

Copy link
Copy Markdown
Contributor

Summary

Closes #8597

expandAutoComboCandidatePool in open-sse/services/combo/autoStrategy.ts
silently expands the candidate pool of an auto-combo to every model of
every active provider connection
whenever the operator has populated
models[] but left config.auto.candidatePool empty (the default for
combos built through the dashboard). This overrides the operator's
populated models[] and lets unintended models (in the wild,
gemini/gemini-3.1-flash-lite, opencode/opencode-minimax-m3-free,
etc.) win the auto-strategy scoring contest.

In omniroute@3.8.48 from npm only one guard exists before the
expansion loop (GUARD A on line 414: if (config.auto.candidatePool populated) return eligibleTargets). The upstream release branch
release/v3.8.49 has a partial second guard added by
#7301 for
combo-ref entries, but it does not cover the common pattern where
the operator populates with explicit kind: "model" entries (the default
for combos built through the dashboard multi-model editor). Both gaps
share the same root mechanism and the same fix.

What this PR does

Adds a single new guard between GUARD A and the expansion loop:

// When the operator has populated the combo's models[] (the
// common case for combos like "gpt-5.5" with strategy=auto), the
// explicit list IS the candidate pool — expansion to every active
// provider's catalog would silently override the operator's intent
// and inject models the operator never approved.
const explicitModels = (combo as Record<string, unknown> | null | undefined)?.models;
if (Array.isArray(explicitModels) && explicitModels.length > 0) return eligibleTargets;

The new guard short-circuits whenever models[] is a non-empty array,
covering both kind: "model" entries (the dashboard default) and
kind: "combo-ref" entries (which #7301 already handles). With this
guard in place, the existing combo-ref check becomes redundant; it is
left in place for the minimal-scope surgical fix and can be removed
in a follow-up cleanup.

Behaviour preservation matrix

Combo models[] candidatePool Before After Verdict
α 3 explicit upstreams empty 60 6 Bug fixed
β 3 explicit upstreams populated (3) 6 6 Preserved (GUARD A)
γ 1 combo-ref + 2 explicit empty 64 10 Bug fixed (new guard subsumes combo-ref)
δ empty [] (virtual auto) empty 57 57 Preserved (expansion is the intended behaviour here)

The 6 / 10 target count (rather than 3 / 5 set by the operator) is
buildAutoCandidates doing its job — each target becomes ~2 candidates
(primary + fallback). The same poolSize=6 shows up for β pre-patch,
confirming 6 is the desired count for a 3-populated-target combo.

Validation (end-to-end, isolated environment)

  • node --import tsx/esm --test tests/unit/combo-auto-candidate-expansion.test.ts → 6/6 pass (5 pre-existing + 1 new test for the explicit-models guard)
  • npx eslint open-sse/services/combo/autoStrategy.ts tests/unit/combo-auto-candidate-expansion.test.ts → exit 0 (clean)
  • Isolated Docker validation: 3 synthetic providers + 4 controlled combos
    • Pre-fix: α 3→60, β 3→6, γ 5→64, δ auto virtual 0→57
    • Post-fix: α 3→6, β 3→6, γ 5→10, δ auto virtual 0→57
    • This confirms the new guard eliminates the silent broadening for
      operator-populated combos while preserving the intentional expansion
      for virtual auto/* combos (which need the full catalog).

Test changes

tests/unit/combo-auto-candidate-expansion.test.ts has one test
renamed/extended in scope to cover the explicit-models case. The new
test is the expandAutoComboCandidatePool is a no-op when the operator has populated models[] (no candidatePool) case. All 5 pre-existing
tests pass unmodified.

Related

Test plan for reviewers

# Unit test for this specific change (6/6 pass)
node --import tsx/esm --test tests/unit/combo-auto-candidate-expansion.test.ts

# Full unit suite (CI runs this in shards)
npm run test:unit

# Coverage gate (must stay ≥ 60%)
npm run test:coverage

# Lint
npm run lint

# Build
npm run build

Notes

  • Branch: fix/auto-combo-explicit-models-guard based on
    release/v3.8.49 per CONTRIBUTING.md (PR base is the active
    release branch, not main).
  • Conventional-commits message per CONTRIBUTING.md.
  • Scope: minimal-scope surgical fix. The existing combo-ref guard is
    intentionally left in place rather than deleted in this PR, even
    though it becomes dead code — splitting the cleanup into a separate
    PR keeps the diff focused and reviewable. Happy to follow up with
    the deletion + test simplification in a sibling PR if preferred.
  • Reversibility: the guard is a single-line return eligibleTargets
    with no side effects; reverting this PR is a one-commit revert.
  • Production behaviour: until this lands in a 3.8.x npm release,
    operators can continue using the manual candidatePool mitigation
    described in fix(backend): auto-combo candidate pool silently expands to entire provider catalog when models[] is populated but candidatePool is empty #8597.

@mikeiagents
mikeiagents force-pushed the fix/auto-combo-explicit-models-guard branch 2 times, most recently from c816ca4 to 4649d93 Compare July 25, 2026 18:29
…ls[] is non-empty

When an auto-combo has models[] populated by the operator but
config.auto.candidatePool is empty (the default for combos created
via the dashboard), expandAutoComboCandidatePool silently expands
the candidate pool to every model of every active provider
connection. This overrides the explicit list in models[] and lets
unintended models (e.g. gemini-3.1-flash-lite) win the auto-strategy
scoring contest.

In omniroute@3.8.48 (npm) only one guard exists before the expansion
loop (GUARD A: if (config.auto.candidatePool populated) return
eligibleTargets). The upstream release branch release/v3.8.49 added
a second guard (combo-ref check, PR cheaper-inference#7301) but it does not cover
the common pattern where models[] holds explicit kind:"model"
entries. Both gaps share the same root mechanism and the same fix.

The new guard short-circuits whenever models[] is a non-empty array,
covering both kind:"model" entries (the dashboard default) and
kind:"combo-ref" entries (which cheaper-inference#7301 already handles). With this
guard in place, the existing combo-ref check becomes redundant; it
is left in place for the minimal-scope surgical fix, and can be
removed in a follow-up cleanup.

Validation (in isolated Docker, 3 providers + 4 controlled combos):
  - 3 explicit models, empty candidatePool: pool 60 -> 6
  - 1 combo-ref + 2 explicit, empty candidatePool: pool 64 -> 10
  - 3 explicit models, populated candidatePool (GUARD A path): 6 -> 6
  - empty models[] virtual auto: 57 -> 57 (expansion preserved)

Closes cheaper-inference#8597
@mikeiagents
mikeiagents force-pushed the fix/auto-combo-explicit-models-guard branch from 4649d93 to a366b8f Compare July 25, 2026 18:32
@diegosouzapw
diegosouzapw merged commit 39e7772 into cheaper-inference:release/v3.8.49 Jul 27, 2026
3 checks passed
@diegosouzapw diegosouzapw mentioned this pull request Jul 28, 2026
HouMinXi pushed a commit to HouMinXi/OmniRoute that referenced this pull request Aug 2, 2026
…ls[] is non-empty (cheaper-inference#8598)

When an auto-combo has models[] populated by the operator but
config.auto.candidatePool is empty (the default for combos created
via the dashboard), expandAutoComboCandidatePool silently expands
the candidate pool to every model of every active provider
connection. This overrides the explicit list in models[] and lets
unintended models (e.g. gemini-3.1-flash-lite) win the auto-strategy
scoring contest.

In omniroute@3.8.48 (npm) only one guard exists before the expansion
loop (GUARD A: if (config.auto.candidatePool populated) return
eligibleTargets). The upstream release branch release/v3.8.49 added
a second guard (combo-ref check, PR cheaper-inference#7301) but it does not cover
the common pattern where models[] holds explicit kind:"model"
entries. Both gaps share the same root mechanism and the same fix.

The new guard short-circuits whenever models[] is a non-empty array,
covering both kind:"model" entries (the dashboard default) and
kind:"combo-ref" entries (which cheaper-inference#7301 already handles). With this
guard in place, the existing combo-ref check becomes redundant; it
is left in place for the minimal-scope surgical fix, and can be
removed in a follow-up cleanup.

Validation (in isolated Docker, 3 providers + 4 controlled combos):
  - 3 explicit models, empty candidatePool: pool 60 -> 6
  - 1 combo-ref + 2 explicit, empty candidatePool: pool 64 -> 10
  - 3 explicit models, populated candidatePool (GUARD A path): 6 -> 6
  - empty models[] virtual auto: 57 -> 57 (expansion preserved)

Closes cheaper-inference#8597

Co-authored-by: Michael de Souza Marcos <michael.smarcos@hotmail.com>
muhamadgalihsaputra pushed a commit to niyatna/NiyatnaRoute that referenced this pull request Sep 27, 2026
…ls[] is non-empty (cheaper-inference#8598)

When an auto-combo has models[] populated by the operator but
config.auto.candidatePool is empty (the default for combos created
via the dashboard), expandAutoComboCandidatePool silently expands
the candidate pool to every model of every active provider
connection. This overrides the explicit list in models[] and lets
unintended models (e.g. gemini-3.1-flash-lite) win the auto-strategy
scoring contest.

In omniroute@3.8.48 (npm) only one guard exists before the expansion
loop (GUARD A: if (config.auto.candidatePool populated) return
eligibleTargets). The upstream release branch release/v3.8.49 added
a second guard (combo-ref check, PR cheaper-inference#7301) but it does not cover
the common pattern where models[] holds explicit kind:"model"
entries. Both gaps share the same root mechanism and the same fix.

The new guard short-circuits whenever models[] is a non-empty array,
covering both kind:"model" entries (the dashboard default) and
kind:"combo-ref" entries (which cheaper-inference#7301 already handles). With this
guard in place, the existing combo-ref check becomes redundant; it
is left in place for the minimal-scope surgical fix, and can be
removed in a follow-up cleanup.

Validation (in isolated Docker, 3 providers + 4 controlled combos):
  - 3 explicit models, empty candidatePool: pool 60 -> 6
  - 1 combo-ref + 2 explicit, empty candidatePool: pool 64 -> 10
  - 3 explicit models, populated candidatePool (GUARD A path): 6 -> 6
  - empty models[] virtual auto: 57 -> 57 (expansion preserved)

Closes cheaper-inference#8597

Co-authored-by: Michael de Souza Marcos <michael.smarcos@hotmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

fix(backend): auto-combo candidate pool silently expands to entire provider catalog when models[] is populated but candidatePool is empty

3 participants