Skip to content

fix(providers): filter unsupported family-fallback candidates against the provider catalog (#8134) - #8240

Merged
diegosouzapw merged 1 commit into
release/v3.8.49from
fix/8134-github-t5-fallback-filter
Jul 23, 2026
Merged

diegosouzapw merged 1 commit into
release/v3.8.49from
fix/8134-github-t5-fallback-filter

Conversation

@diegosouzapw

Copy link
Copy Markdown
Owner

Closes #8134

Root cause

open-sse/services/modelFamilyFallback.ts::getNextFamilyFallback() resolves a provider's supported model ids into supportedIds but only used that set to try dot/hyphen notation variants of a candidate — never to filter out a candidate that is provably absent from the provider's registered catalog under any notation. When neither the hyphen form nor a dot-variant matched, the loop fell through and returned the unresolved candidate anyway.

For the github (GitHub Copilot) registry entry, the Claude Opus tier list is 4.8 / 4.8-fast / 4.7 / 4.5 — there is no 4.6 tier. MODEL_FAMILIES['claude-opus-4-8'] unconditionally includes claude-opus-4-6, so the second fallback hop returned github/claude-opus-4-6 verbatim, and the upstream would 400 a third time before the family was exhausted — the wasted round-trip described in the issue.

Fix

When the provider registry is resolved, a family candidate is now skipped (continue) instead of returned unfiltered if it has no match in supportedIds under any notation — hyphen, dot-variant, or (new) a dated-snapshot id (claude-opus-4-5-20251101) with its trailing -YYYYMMDD suffix stripped before the dot-variant check. The date-stripping addition was needed to keep the kiro sibling regression test green (see landmine note below) — without it, dated Sonnet candidates that do have an undated match in a provider's catalog (claude-sonnet-4.5) would be wrongly skipped too.

No change to MODEL_FAMILIES itself or to the github/kiro registries — filtering happens purely against the already-resolved catalog.

TDD evidence (Hard Rule #18)

New permanent regression file tests/unit/8134-github-t5-fallback-filter.test.ts:

  • RED (unfixed code): second fallback "github/claude-opus-4-6" is not in github's registered model catalog: ... — 2 tests, both failing.
  • GREEN (fixed code): both tests pass; a 5-hop walk of the whole github Opus fallback chain asserts every returned candidate is a real member of getRegistryEntry('github').models.

Landmine — kiro sibling test

tests/unit/kiro-claude-sonnet-5-2267.test.ts (guards a different fallback contract, previously broken by a naive continue-based fix in a reverted PR) stays green, unmodified. The kiro registry only has claude-sonnet-4.5 (no 4-6 tier, no dated snapshots) — the new date-stripped dot-variant check lets claude-sonnet-4-5-20250929 resolve to kiro's real claude-sonnet-4.5 entry instead of being incorrectly skipped, which is what a naive "skip everything unmatched" fix would have done (and why #8143 reverted that approach).

Gates run (from the worktree)

  • node --import tsx/esm --test tests/unit/8134-github-t5-fallback-filter.test.ts — 2/2 pass (RED→GREEN)
  • node --import tsx/esm --test tests/unit/kiro-claude-sonnet-5-2267.test.ts tests/unit/t30-kiro-400-model-unavailable.test.ts tests/unit/repro-7268-401-model-not-supported-lockout.test.ts tests/unit/model-family-fallback-notation.test.ts — 17/17 pass (all sibling fallback tests green, including the landmine)
  • node scripts/check/check-file-size.mjs — OK
  • node scripts/check/check-complexity.mjs — reports a pre-existing repo-wide regression (2156 vs baseline 2130) unrelated to this diff: modelFamilyFallback.ts itself has 0 complexity violations, and the scan scope excludes tests/, so nothing in this PR contributes to it.
  • node scripts/check/check-cognitive-complexity.mjs — OK (944 ≤ baseline 950)
  • npm run typecheck:core — clean
  • npx eslint --suppressions-location config/quality/eslint-suppressions.json open-sse/services/modelFamilyFallback.ts tests/unit/8134-github-t5-fallback-filter.test.ts — clean
  • node scripts/check/check-changelog-integrity.mjs — OK
  • npm run test:vitest — 271/271 pass, 30/30 files
  • npm run test:unit — full suite run; 0 failures observed through the run

Scope

Diff is scoped strictly to open-sse/services/modelFamilyFallback.ts (new pure helper functions + the fix) plus the new regression test and a changelog fragment. No changes to MODEL_FAMILIES, the github registry, or the kiro registry.

@diegosouzapw

Copy link
Copy Markdown
Owner Author

Follow-up note on the full `npm run test:unit` local run: it surfaced 5 pre-existing failures unrelated to this PR's diff — `tests/unit/authz/spawn-capable-prefixes-client-safe.test.ts` (1) and `tests/unit/auto-combo-credentialed-model-pool.test.ts` (4). Confirmed base-red: `git diff origin/release/v3.8.49 -- <those test files + the modules they exercise>` is empty (no diff at all vs the target branch), and neither file imports `modelFamilyFallback.ts`. Re-ran both files in isolation — same deterministic failures, not a concurrency flake. Pre-existing on `release/v3.8.49`, out of scope for #8134.

@diegosouzapw
diegosouzapw merged commit 3f8280b into release/v3.8.49 Jul 23, 2026
10 checks passed
@diegosouzapw
diegosouzapw deleted the fix/8134-github-t5-fallback-filter branch July 23, 2026 03:47
HouMinXi pushed a commit to HouMinXi/OmniRoute that referenced this pull request Aug 2, 2026
… the provider catalog (diegosouzapw#8134) (diegosouzapw#8240)

Co-authored-by: Probe Test <probe@example.com>
muhamadgalihsaputra pushed a commit to niyatna/NiyatnaRoute that referenced this pull request Sep 27, 2026
… the provider catalog (diegosouzapw#8134) (diegosouzapw#8240)

Co-authored-by: Probe Test <probe@example.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

fix(providers): GitHub Models T5 family fallback tries unsupported models (claude-opus-4.8/4.7)

1 participant