fix(tests): retire claude-3-5-sonnet-20241022 from the combo integration suites

[doramirdor]

Summary

Sibling of #12670, which retired dead model ids from chat-pipeline.test.ts. The same id, claude-3-5-sonnet-20241022, is still the claude (and, in one suite, anthropic) target in eight other integration suites. The vendor lifecycle registry marks it retired (shut down 2025-10-28, replacement claude-sonnet-4-6), the pipeline correctly refuses to auto-route it, and every combo that lists it as a target loses that target:

❌ claude [410]: Model "claude/claude-3-5-sonnet-20241022" was shut down on 2025-10-28 and cannot be routed automatically. Use "anthropic/claude-sonnet-4-6" instead.

That single refusal produces 23 failures on the untouched release/v3.8.51 tip (ba597b6). Two more failures sit in the same suites with unrelated causes; they are listed separately below and are not touched by this PR.

suite	failing	cases
combo-matrix/ordered.test.ts	6	priority fallback (502 !== 200), round-robin, least-used, random, strict-random, p2c: claude is never reached
combo-matrix/quota-aware.test.ts	3	reset-aware demotion, headroom, lkgp
combo-matrix/weighted.test.ts	1	70/30 split; burns ~187 s in per-target retries before failing
combo-matrix/fusion.test.ts	1	expected 3 panel + 1 judge, got 3
combo-provider-exhaustion.test.ts	6	every #1731 fast-skip scenario: the anthropic/claude targets answer 410 instead of the scripted 429
combo-routing-e2e.test.ts	3	round-robin over three providers, secondary-model fallback, strategy update
combo-failover-e2e.test.ts	2	3-target chain 500 → 503 → success, per-target timeout failover
proxy-context-passthrough.test.ts	1	combo targets under their own connection's proxy

Not caused by the retired id, failing identically before and after this change, left for their own PRs:

suite	case	assertion
combo-matrix/quota-share.test.ts	DRR fairness: 2 equal-weight connections alternate across 6 requests	gemini must be selected at least 2 times out of 6; got 0. Full sequence: [openai ×6] (fixture is openai + gemini, no claude)
combo-routing-e2e.test.ts	priority combo can repeat the same provider/model with different fixed accounts	expected combo call logs to be persisted (actual: null)

The change

One substitution, claude-3-5-sonnet-20241022 → claude-sonnet-4-6, in the eight suites above. That is the successor the lifecycle record names, the id claude-sonnet-4-6 already used by chat-pipeline.test.ts since #12670, and it is live in the claude provider registry. The _comboRoutingHarness recording fetch keys on the host (api.anthropic.com → claude), not on the model id, so no harness change is needed.

No assertion is changed, removed or relaxed. Left alone on purpose: _chatPipelineHarness.ts's buildClaudeResponse() default model string (a response echo, inert for these suites, and #12670 left it too) and playground-improve-prompt.test.ts (sends the id as a direct request model, not auto-routed, passes today).

One observation for a maintainer, not changed here: the anthropic registry lists this model as claude-sonnet-4.6 (dotted) while the lifecycle replacement and the claude registry say claude-sonnet-4-6 (dashed). The fixtures follow the lifecycle spelling.

Related Issues

• Related to fix(tests): retire dead model ids from the chat-pipeline integration suite (base-red #12581) #12670, 🔴 Release branch not green: release/v3.8.51 #12581 (base-red), feat(auto-combo): add nadir router strategy for prompt-aware model choice #13051 (where these failures were first written up)

Validation

Change type: routing tests only (tests/integration/**), no production code.

• Change type: provider / routing / UI / i18n / CLI / DB / build-deploy / other
• Focused tests and category gates from the golden path
• npm run lint (eslint on the eight files, zero findings). Deliberately not Prettier-formatted: 7 of the 8 files already fail prettier --check at the pinned 3.9.6 on the untouched ba597b6, so running the formatter here would add unrelated churn to a fixture-only change.
• Reconciled with the current active release base (release/v3.8.51, tip ba597b6)
• Production-code changes include a new or updated automated test in this PR (no production code changed)
• SonarQube is temporarily opt-in while the private project has no quota; it is not a PR gate.

Run with the CI invocation (setupPolyfill + isolateDataDir + --test-force-exit --test-concurrency=1):

npm run test:combo:matrix
# before: tests 27 | pass 15 | fail 12
# after:  tests 27 | pass 26 | fail 1   (the DRR fairness case above, unchanged)

node --import tsx/esm --import ./open-sse/utils/setupPolyfill.ts --import ./tests/_setup/isolateDataDir.ts --test --test-force-exit --test-concurrency=1 \
  tests/integration/combo-routing-e2e.test.ts tests/integration/combo-failover-e2e.test.ts \
  tests/integration/combo-provider-exhaustion.test.ts tests/integration/proxy-context-passthrough.test.ts
# before: tests 22 | pass 9 | fail 13
# after:  tests 22 | pass 21 | fail 1   (the call-log persistence case above, unchanged)

Same caveat as #12670: without --import ./tests/_setup/isolateDataDir.ts these suites hit the developer's real ~/.omniroute database.

Tests Added Or Updated

• tests/integration/combo-matrix/ordered.test.ts
• tests/integration/combo-matrix/weighted.test.ts
• tests/integration/combo-matrix/fusion.test.ts
• tests/integration/combo-matrix/quota-aware.test.ts
• tests/integration/combo-routing-e2e.test.ts
• tests/integration/combo-failover-e2e.test.ts
• tests/integration/combo-provider-exhaustion.test.ts
• tests/integration/proxy-context-passthrough.test.ts

Fixture model id only; no production code changed.

Coverage Notes

No src/, open-sse/, electron/ or bin/ file changed. Coverage can only go up: 23 integration cases that were failing now exercise their intended paths again.

Reviewer Notes

• The weighted matrix case alone spent ~187 s retrying the dead target before failing; that is a plausible contributor to the integration gate's 2400 s ceiling deaths mentioned in fix(tests): retire dead model ids from the chat-pipeline integration suite (base-red #12581) #12670. That retry loop disappears with the fix.
• If you would rather keep a retired id in one suite as a deliberate "refuses dead models" fixture, say which and I will restore it with an explicit 410 assertion.
• The two leftover failures are real and separate: the DRR case routes six requests to openai and never to gemini even with two equal-weight connections, and the fixed-accounts case never sees its combo call-log row. Both reproduce on the untouched base with the same assertion text, so neither is masked or introduced here. I did not investigate them beyond that.
• The diff is the substitution and nothing else: 30 changed lines, every one carrying the model id, plus the changelog fragment. An earlier revision of this branch also carried incidental Prettier reflow on lines that have no model id; that has been reverted (see the force-push note in the comments).

[diegosouzapw]

Solid follow-up to #12670 — verified ordered.test.ts + fusion.test.ts pass at your head
(10/10). The retirement rationale (vendor lifecycle registry, #12670 precedent) checks out.
Merge-ready from my side.

[doramirdor]

Heads up before you merge: I force-pushed after your approval, so the head you verified (d893880) is gone. The change is formatting-only, and I would rather you hear that from me than notice the hash moved.

72474612 is the new head. What changed: the old revision carried incidental Prettier reflow on lines that have no model id in them — the dynamic await import(...) in quota-aware.test.ts, the harness destructure in proxy-context-passthrough.test.ts, a claude-free models: array plus a trailing blank line in combo-provider-exhaustion.test.ts, and assert wrapping in fusion / ordered / combo-failover-e2e. That was my formatter running over whole files, not anything the substitution required. All of it is reverted. The diff is now 31 additions / 30 deletions (was 72 / 53), and every one of the 30 changed test lines carries the model id.

Two things I checked before pushing, so you do not have to re-verify from scratch:

The code is identical to what you approved. I rebuilt both commits from ba597b6 with a single global claude-3-5-sonnet-20241022 → claude-sonnet-4-6 substitution and nothing else, then ran prettier@3.9.6 over the old head and the new one into separate trees and diffed the normalized output: byte-identical. Same program, different whitespace. ordered.test.ts and fusion.test.ts will still be the 10/10 you saw.

There is no Prettier gate to satisfy here anyway. 7 of the 8 files already fail prettier --check at the pinned 3.9.6 on the untouched ba597b6, so formatting them was never a requirement — it just made a fixture-only diff look like a refactor. The PR body's Validation and Reviewer Notes sections are updated to say this instead of the old claim about re-flowed models: arrays.

Commit message, authorship and the changelog fragment are unchanged; the force-push used --force-with-lease pinned to the old head.

[diegosouzapw]

Thanks @doramirdor — merging via the release merge-train. Validated in local merge-train (.claude/worktrees/merge-train-20260918-111718-suite.log) on the devbox @ train tip 7bb373fba5e241964c0ffb17bd03a804700839bb, boarded with 55 sibling PRs: typecheck:core, file-size, complexity, cognitive-complexity, changelog-integrity green; 747/747 changed-area node:test cases + 476/476 vitest green (fast parity mode — the full suite ran today on the release tip via the base-red train and runs again on the 3b train). Merged --admin per merge-gates §7.