Repository navigation
test(auto-combo): seed providers for #4235 tier weight-composition tests - #16151
Merged
diegosouzapw merged 1 commit intoOct 11, 2026
Merged
diegosouzapw merged 1 commit into
diegosouzapw merged 1 commit into
Conversation
…ht-composition tests Since cheaper-inference#15979, uncataloged models inherit the provider ToS verdict, so the keyless opencode pool (tos: avoid) is excluded by the default excludeTosAvoid=true. With no connections the auto/coding:fast and auto/coding:reliable pools are empty, and the empty-pool early return in virtualFactory yields DEFAULT_WEIGHTS before the tier overlay runs. Seed real provider connections in both tests and assert a non-empty pool so they keep guarding the tier -> mode-pack weight mapping. Refs cheaper-inference#15306
This was referenced Oct 10, 2026
Collaborator
|
Hi @shipsfromrio — closing this pull request to clear the open queue. The branch |
Collaborator
|
Hi @shipsfromrio — reopening this pull request. The close was a mistake, and it is back in the merge queue. The branch |
diegosouzapw
merged commit Oct 11, 2026
884f8fc
into
cheaper-inference:release/v3.8.52
42 of 89 checks passed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
Two cases in
tests/unit/auto-combos-suffixes-4235.test.tsfail deterministically onrelease/v3.8.52(base-reds in #15306):createBuiltinAutoComboforauto/coding:fastandauto/coding:reliablereturnsDEFAULT_WEIGHTSinstead of theship-fast/reliability-firstpacks.The cases seed no provider connections, so their only candidates were the keyless
opencodemodels. Since #15979 an uncataloged model inherits its provider's ToS rating;opencodeis"avoid"andexcludeTosAvoiddefaults to true, so the pool is now empty. On an empty poolvirtualFactoryreturnsDEFAULT_WEIGHTSearly, before the:fast/:reliableweight overlay runs. Production is correct: #15979 is an intentional ToS guard, and weights on an empty combo never affect routing.Fix (test only)
seedProviders()first, so the overlay runs on a real pool.combo.models.length > 0, so an empty pool fails loudly instead of silently comparing default weights again.deepEqualagainstMODE_PACKS["ship-fast"]/["reliability-first"]is unchanged.Tests
The file passes 9/9 (the two target cases failed before). The ToS suites (
tos-provider-alias-15059,first-run-catalog-15059,issue-15059-tos-avoid-auto-default) and the other*suffix*files pass 69/69; the othercreateBuiltinAutoCombosuites pass 26/26; vitestautoCombo/provider-family-combospasses 18/18.Refs #15306