Skip to content

fix(combo): lock GitHub models rejected as "not supported" for future requests - #11781

Merged
diegosouzapw merged 2 commits into
diegosouzapw:release/v3.8.51from
turbolego:fix/github-model-not-supported-lockout
Aug 29, 2026
Merged

diegosouzapw merged 2 commits into
diegosouzapw:release/v3.8.51from
turbolego:fix/github-model-not-supported-lockout

Conversation

@turbolego

Copy link
Copy Markdown
Contributor

Follow-up to #11762 / #11774 — same bug class, this time in combo's own model routing

Checking a fresh 1h log confirmed another instance of the "permanent failure never gets locked out, so it's retried forever" bug — this time not in checkFallbackError's classification, but in combo.ts's own model-lockout wiring.

The symptom

github rejects several models with a 400 that is permanent for this account's Copilot integration — it will never start working mid-session:

"provider": "github",
"model": "gpt-5.4",
"status": 400,
"error": "[400]: The requested model is not supported."
"provider": "github",
"model": "gpt-5.3-codex",
"status": 400,
"error": "[400]: The requested model is not available for integrator \"vscode-chat\".
          Available models: [gpt-4.1 claude-fable-5 ... gemini-3.6-flash ...].
          Verify the correct Copilot-Integration-Id header is being sent."

This isn't a one-off — the same models (gpt-5.3-codex, gpt-5.4, gpt-5.4-mini, gpt-5.5, gpt-5.6-luna, gpt-5.6-sol, gpt-5.6-terra, mai-code-1-flash) get rejected with this exact 400 on every single virtual-auto-default-*-github combo step, in every request, for the entire hour (dozens of times across the log). Each occurrence is a wasted upstream call.

Root cause

Combo's #5249 guard (isModelScoped400) correctly lets the combo advance to the next target within the same request — that's intentional and correct, since a different provider might serve the model. But nothing ever recorded a cross-request lockout for the rejected model, because the per-request "same-model retry" lockout path in combo.ts (recordModelLockoutFailure, gated by isTransient) only fires for [408, 429, 500, 502, 503, 504] — 400 is deliberately excluded there (a 400 always advances immediately, never retries the same model in-request). So a model that GitHub will reject forever for this account gets tried again from scratch on every new incoming request, indefinitely.

The fix

In open-sse/services/combo.ts, right after the existing #2101/#5249 stop-guard: when isModelScoped400(errorText) is true, call lockModelIfPerModelQuota(provider, connectionId, rawModel, "model_capacity", 1h).

  • github already has per-model-quota enabled (hasPerModelQuota("github") === true), so this locks only the rejected model, not the whole GitHub connection — sibling models keep working normally.
  • isModelLocked() is already checked before dispatch on every future request (combo.ts pre-check), so the lockout is honored automatically — no other wiring needed.
  • The in-request combo-advance behavior from fix(combo): advance to next model on 400 'model not supported' #5249 is completely unchanged; this only adds a side effect for future, separate requests.

Testing

Added tests/unit/github-model-not-supported-lockout.test.ts (3 tests, all passing):

  • isModelScoped400 matches both GitHub phrasings from the logs.
  • github confirmed as a per-model-quota provider.
  • lockModelIfPerModelQuota locks the rejected model while leaving a sibling model on the same connection eligible.

Also fixed tests/unit/combo-model-scoped-400-advance.test.ts: its sub-tests reuse the same model name (github/claude-fable-5) across several test() blocks without clearing lockout state — with the new persistent lockout, the first sub-test's 400 now (correctly) locks that model, causing later sub-tests in the same process to skip it. Added a clearAllModelLockouts() beforeEach to keep each sub-test isolated; no behavioral assertions changed.

Ran locally:

node --import tsx/esm --test tests/unit/github-model-not-supported-lockout.test.ts
# 3/3 pass

node --import tsx/esm --test tests/unit/combo-body-specific-400-stop-4279.test.ts \
  tests/unit/account-fallback-service.test.ts tests/unit/agentrouter-lock-scope-10334.test.ts
# 113/113 pass

node --import tsx/esm --test tests/unit/combo-model-scoped-400-advance.test.ts \
  tests/unit/combo-param-validation-fallback-4519.test.ts
# 12/12 pass (was 2 failing before the beforeEach fix — confirmed root cause, not a real regression)

node --import tsx/esm --test tests/unit/combo-quota-token-limit.test.ts \
  tests/unit/combo-lockout-quota-reset-6863.test.ts tests/unit/model-lockout-max-cooldown.test.ts \
  tests/unit/combo-terminal-status-policy-10501.test.ts tests/unit/combo-resource-404-health.test.ts \
  tests/unit/combo-input-bound-failure-8375.test.ts tests/unit/combo-input-bound-heterogeneous-8375.test.ts
# 12/12 pass, no regressions

Also registered the new test in stryker.conf.json's tap.testFiles (required for the mutation-test-coverage gate, per CI feedback on #11762).

Files changed

  • open-sse/services/combo.ts — new lockModelIfPerModelQuota import + call at the model-scoped-400 site.
  • tests/unit/github-model-not-supported-lockout.test.ts — new regression test.
  • tests/unit/combo-model-scoped-400-advance.test.ts — added lockout-state isolation between sub-tests.
  • stryker.conf.json — register the new test file.

… requests

Same bug class as diegosouzapw#11762/diegosouzapw#11774: a permanent-for-this-account 400 ("The
requested model is not supported." / "not available for integrator
'vscode-chat'") was never recorded as a model lockout, because combo's
per-request same-model retry lockout only fires for [408,429,500,502,503,504]
statuses — 400 is deliberately excluded there. The diegosouzapw#5249 guard correctly lets
combo advance to the next target within the SAME request, but nothing
persisted the failure across SEPARATE requests, so the identical dead GitHub
model got retried on every single future auto-combo request, forever
(observed in production logs: every request wasted several upstream 400
calls on gpt-5.4, gpt-5.5, gpt-5.6-luna/sol/terra, gpt-5.3-codex,
mai-code-1-flash — all day).

Add an explicit lockModelIfPerModelQuota() call when isModelScoped400(errorText)
is true, right after the existing diegosouzapw#2101 stop-guard. github already has
per-model-quota enabled, so this locks only the specific model (not the whole
connection) for 1h; isModelLocked() is already checked before dispatch on
every future request, so the lockout is honored automatically. The in-request
combo-advance behavior (diegosouzapw#5249) is unchanged.

Also fixes tests/unit/combo-model-scoped-400-advance.test.ts: its sub-tests
reused the same model name without clearing lockout streused the same model name without clearing lockout streused the same model nacross sub-tests within
the same process — added a clearAllModelLockouts() beforeEach.
@diegosouzapw
diegosouzapw merged commit d887937 into diegosouzapw:release/v3.8.51 Aug 29, 2026
16 checks passed
diegosouzapw pushed a commit that referenced this pull request Aug 29, 2026
…EADME (#11772)

Finishes the Freepik → Magnific rebrand from #10594 across 40 locale files and 3 README feature-list bullets (README.md, docs/i18n/it, docs/i18n/tr) — legacy `freepik` alias intentionally left in code/tests/redirects for backward compatibility, and historical CHANGELOG entries left untouched as documented history.

The README bullet had base-drifted since the PR branched (release tip's "What's New" changelog snippet had already dropped two providers mentioned nowhere else in the codebase, unrelated to this PR's scope) — resolved by keeping the tip's current bullet shape and applying only the Freepik→Magnific rename on top, in both the combined-worktree validation and the pushed branch.

Validated: all 40 edited locale JSON files parse; re-verified after resync onto the updated tip (post #11762/#11774/#11781).
muhamadgalihsaputra pushed a commit to niyatna/NiyatnaRoute that referenced this pull request Sep 27, 2026
… requests (diegosouzapw#11781)

Follow-up to diegosouzapw#11762/diegosouzapw#11774, same bug class in combo's own model-lockout wiring: GitHub rejects several models (gpt-5.4, gpt-5.3-codex, etc.) with a 400 that's permanently unavailable for this account's Copilot integration, but nothing recorded a cross-request lockout — combo's diegosouzapw#5249 in-request advance guard is correct but doesn't persist, so the same doomed model gets retried from scratch on every new request, indefinitely.

Fix: on a model-scoped 400 (`isModelScoped400`), call `lockModelIfPerModelQuota(provider, connectionId, rawModel, "model_capacity", 1h)`. GitHub already has per-model-quota enabled, so only the rejected model locks — siblings keep working. `isModelLocked()` is already checked pre-dispatch, so no other wiring needed.

Validated: 3/3 new tests + fixed a pre-existing test-isolation gap in combo-model-scoped-400-advance.test.ts (shared model name across sub-tests without clearing lockout state). Thanks!
muhamadgalihsaputra pushed a commit to niyatna/NiyatnaRoute that referenced this pull request Sep 27, 2026
…EADME (diegosouzapw#11772)

Finishes the Freepik → Magnific rebrand from diegosouzapw#10594 across 40 locale files and 3 README feature-list bullets (README.md, docs/i18n/it, docs/i18n/tr) — legacy `freepik` alias intentionally left in code/tests/redirects for backward compatibility, and historical CHANGELOG entries left untouched as documented history.

The README bullet had base-drifted since the PR branched (release tip's "What's New" changelog snippet had already dropped two providers mentioned nowhere else in the codebase, unrelated to this PR's scope) — resolved by keeping the tip's current bullet shape and applying only the Freepik→Magnific rename on top, in both the combined-worktree validation and the pushed branch.

Validated: all 40 edited locale JSON files parse; re-verified after resync onto the updated tip (post diegosouzapw#11762/diegosouzapw#11774/diegosouzapw#11781).
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants