Repository navigation
fix(autoCombo,sse): drop vendor-retired ids from the auto-combo pool and skip leftover arena_elo (#11625) - #11626
Merged
diegosouzapw merged 2 commits intoAug 26, 2026
Conversation
…and skip leftover arena_elo (diegosouzapw#11625) Auto-combo still selected vendor-retired catalog ids after diegosouzapw#11507/diegosouzapw#11508 because buildAutoCandidates only filtered dashboard-hidden models, and getTaskFitnessWithSource returned leftover arena_elo / user_override rows before the layer-3 veto could run. A variant could also inherit a dead base's arena score via resolveScoresAs. One prefix-stripped predicate reads model-lifecycle.json (status=retired only; retiring-without-date is left alone). The candidate pool drops those ids, layers 1–3 miss so a leftover row cannot short-circuit, and getModelLifecycleDecision consults the same snapshot so aggregator traffic is not untracked for ids the JSON already knows. Dated OpenAI rows stay provider-scoped. Fixes diegosouzapw#11625
Merged
4 of 5 tasks
# Conflicts: # open-sse/services/autoCombo/taskFitness.ts
diegosouzapw
merged commit Aug 26, 2026
00d385b
into
diegosouzapw:release/v3.8.51
4 of 7 checks passed
5 tasks done
This was referenced Aug 26, 2026
muhamadgalihsaputra
pushed a commit
to niyatna/NiyatnaRoute
that referenced
this pull request
Sep 27, 2026
…and skip leftover arena_elo (diegosouzapw#11625) (diegosouzapw#11626) Merged via /merge-batch (lote 2026-08-26 batch 2, v3.8.51). Conflito real em taskFitness.ts: seu código pula corretamente as camadas 1-3 para ids vendor-retired (a versão anterior fazia lookup redundante de arena_elo/inherited/tier mesmo quando já sabia que o id era retired). Resolvido mantendo o skip antecipado. Validado: 23/23 testes passando. Obrigado pela contribuição.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #11625
Not a replacement for #11598. That PR's layer-3 veto (prefix / inherited-base) stays on #11508. This is the candidate pool + layers 1–2, which still select a retired model even after a layer-3 veto.
Problem
getTaskFitnessWithSource()isuser_override → arena_elo → inherited 1–2 → models_dev_tier → FITNESS_TABLE → wildcard.#11507 stopped layer 4 from ranking retired ids at 0.98. #11508 / #11598 veto layer 3. Neither stops auto-combo from picking a retired model:
buildAutoCandidates()only drops dashboard-hidden models.filterSelectableModels()exists but readsMODEL_LIFECYCLE_RECORDS(OpenAI-only hardcoded). An Anthropic/Google retired id isuntracked→allow.arena_elo/user_overriderow returns at layer 1–2, so the layer-3 veto never runs.lookupInheritedFitnessalso copies a dead base's arena score onto an effort variant.openrouter+gpt-5.2-codex) isuntrackedbecause the dated table is provider-scoped.allowedRetiredInCatalog(6 ids) stays a catalog/CI burn-down. This PR does not delete catalog rows; it stops auto-combo from selecting them and stops leftover intelligence rows from scoring above the 0.5 neutral.Fix
One prefix-stripped predicate,
isVendorRetiredId(), readsconfig/quality/model-lifecycle.json(status === "retired"only —retiringwithretiredOn: nullis left alone).rejectRetiredAutoComboCandidates()runs after the hidden-model filter inbuildAutoCandidates.getTaskFitnessWithSourceskips layers 1–3 when the request id or itsresolveScoresAs()base is retired, and returns wildcard 0.5 with no*codex/codeboost.getModelLifecycleDecisionconsults the same snapshot after the dated OpenAI table misses, so aggregator traffic is notuntrackedfor ids the JSON already knows. Dated OpenAI warn-before-shutdown rows stay provider-scoped.Tests
TDD (Hard Rule #18):
tests/unit/retired-pool-11625.test.tswas written first and failed 9/11 on the pre-fix chain.isVendorRetiredId("openai/gpt-5.2-codex")and the bare formrejectRetiredAutoComboCandidateskeepsgpt-5.6-sol, drops snapshot-retired idsisModelSelectable("anthropic", "claude-3-7-sonnet-20250219") === falsearena_eloongpt-5.2-codex/openai/gpt-5.2-codexis not returnedclaude-3-7-sonnet-20250219-highdoes not inherit the dead base's arena rowgpt-5.6-solarena row still winsExisting
model-lifecycle.test.tsupdated: dated OpenAI deprecations stay provider-scoped; snapshot-retired ids are rejected on every provider.#11489inherit tests still pass.Verification
npm run typecheck:corereports the pre-existingomniglyphpackage-export mismatches on files this PR does not touch (same baseline as #11492 / #11507). Touched files are under the 1000-line cap.