Repository navigation
[codex] Keep mode-pack weights consistent in auto fallback ranking - #7008
diegosouzapw merged 2 commits into
Conversation
|
Warning You have reached your daily quota limit. Please wait up to 24 hours and I will start processing your requests again! |
|
Thanks for tracking this down — the inconsistency between One gap though: this doesn't fully close the loop. Could you extend the same fix to recompute |
…ights parseAutoConfig() already resolves `weights` from a combo's own STORED modePack, but resolveAutoStrategyOrder() also supports a per-request X-OmniRoute-Mode header override (cheaper-inference#6024/cheaper-inference#6025) that can select a DIFFERENT mode pack than the one stored on the combo, for that single request only. selectAutoProvider() (engine.ts) already re-derives weights internally from the modePack it receives, so it correctly reacts to the override -- but scoreAutoTargets(), which ranks the fallback tail, had no such re-derivation and only ever saw the stale pre-override weights from parseAutoConfig(). Net effect: a request overriding e.g. "quality-first" to "ship-fast" would select its primary target under ship-fast weights but rank every fallback under quality-first weights -- the identical "select under one policy, rank fallbacks under another" bug this module's original fix (honoring the combo's own stored modePack) set out to close. Recompute `weights` from the effective (post-override) `modePack` right after it's resolved, so both selectAutoProvider and scoreAutoTargets consume the same weight vector. Adds a regression test proving a request-level X-OmniRoute-Mode override produces IDENTICAL fallback-ranking weights to a combo natively configured with that same modePack. Co-authored-by: diegosouzapw <8016841+diegosouzapw@users.noreply.github.com>
2cbb47d
into
cheaper-inference:release/v3.8.49
|
Thanks @KooshaPari! Merged into release/v3.8.49 after combined merge-train validation. We resolved the stale .mergify.yml against the tip's queue config and decomposed the over-threshold helpers inside your branch — you stay author throughout. |
…heaper-inference#7008) * fix(routing): honor modePack weights in combo fallback ranking * fix(routing): make effective post-override modePack drive fallback weights parseAutoConfig() already resolves `weights` from a combo's own STORED modePack, but resolveAutoStrategyOrder() also supports a per-request X-OmniRoute-Mode header override (cheaper-inference#6024/cheaper-inference#6025) that can select a DIFFERENT mode pack than the one stored on the combo, for that single request only. selectAutoProvider() (engine.ts) already re-derives weights internally from the modePack it receives, so it correctly reacts to the override -- but scoreAutoTargets(), which ranks the fallback tail, had no such re-derivation and only ever saw the stale pre-override weights from parseAutoConfig(). Net effect: a request overriding e.g. "quality-first" to "ship-fast" would select its primary target under ship-fast weights but rank every fallback under quality-first weights -- the identical "select under one policy, rank fallbacks under another" bug this module's original fix (honoring the combo's own stored modePack) set out to close. Recompute `weights` from the effective (post-override) `modePack` right after it's resolved, so both selectAutoProvider and scoreAutoTargets consume the same weight vector. Adds a regression test proving a request-level X-OmniRoute-Mode override produces IDENTICAL fallback-ranking weights to a combo natively configured with that same modePack. Co-authored-by: diegosouzapw <8016841+diegosouzapw@users.noreply.github.com> --------- Co-authored-by: diegosouzapw <8016841+diegosouzapw@users.noreply.github.com>
…heaper-inference#7008) * fix(routing): honor modePack weights in combo fallback ranking * fix(routing): make effective post-override modePack drive fallback weights parseAutoConfig() already resolves `weights` from a combo's own STORED modePack, but resolveAutoStrategyOrder() also supports a per-request X-OmniRoute-Mode header override (cheaper-inference#6024/cheaper-inference#6025) that can select a DIFFERENT mode pack than the one stored on the combo, for that single request only. selectAutoProvider() (engine.ts) already re-derives weights internally from the modePack it receives, so it correctly reacts to the override -- but scoreAutoTargets(), which ranks the fallback tail, had no such re-derivation and only ever saw the stale pre-override weights from parseAutoConfig(). Net effect: a request overriding e.g. "quality-first" to "ship-fast" would select its primary target under ship-fast weights but rank every fallback under quality-first weights -- the identical "select under one policy, rank fallbacks under another" bug this module's original fix (honoring the combo's own stored modePack) set out to close. Recompute `weights` from the effective (post-override) `modePack` right after it's resolved, so both selectAutoProvider and scoreAutoTargets consume the same weight vector. Adds a regression test proving a request-level X-OmniRoute-Mode override produces IDENTICAL fallback-ranking weights to a combo natively configured with that same modePack. Co-authored-by: diegosouzapw <8016841+diegosouzapw@users.noreply.github.com> --------- Co-authored-by: diegosouzapw <8016841+diegosouzapw@users.noreply.github.com>
Summary
modePackvalues in the shared config parsership-fastWhy
selectAutoProvider()appliesmodePack, while fallback ranking previously received raw/default weights. That could select a provider under one policy and rank fallbacks under another.Validation
node --import tsx/esm --test tests/unit/combo-auto-config-split.test.tsnpx --yes --package prettier prettier --check open-sse/services/combo/autoConfig.ts tests/unit/combo-auto-config-split.test.ts