Skip to content

fix(grok-cli): park a 402 on the empty Grok Build login, not grok-4.6 - #13061

Merged
diegosouzapw merged 1 commit into
diegosouzapw:release/v3.8.51from
HouMinXi:fix/grok-cli-402-connection-credits
Sep 11, 2026
Merged

diegosouzapw merged 1 commit into
diegosouzapw:release/v3.8.51from
HouMinXi:fix/grok-cli-402-connection-credits

Conversation

@HouMinXi

@HouMinXi HouMinXi commented Sep 8, 2026

Copy link
Copy Markdown
Contributor

Problem

space-grok is a reset-aware combo of five grok-cli/grok-4.6 steps, one
connection each. Grok Build bills Chat/Imagine/Voice/Build/API against a
single weekly percent pool. When one login answers 402 Grok Build usage balance exhausted, the registry still has passthroughModels: true, so
auth treated that 402 as a per-model billing miss (#12242). Combo then
locked grok-4.6 and skipped the remaining live logins as "model locked
by resilience". The client only saw the 402.

Live X500 (storage.sqlite + podman logs, 13:04-13:05Z): four Grok
Build logins still answering 200 with 32-93% weekly remaining; only
364432949@qq.com (1%) returned 402. Combo ordered that empty login
first, then skipped the other four.

Change

  • isSharedWalletCredits402("grok-cli", 402) is a connection-wide wallet
    signal. passthroughModels still stands for catalog/404.
  • markAccountUnavailable parks that connection as credits_exhausted
    instead of a model lock. disableCooling does not rescue it.
  • Combo marks only that provider:connectionId exhausted for the rest of
    the request. Sibling Grok Build logins stay eligible.
  • The body phrase usage balance exhausted is a credits-exhausted signal.

Tests

tests/unit/auth-grok-cli-402-shared-wallet.test.ts plus one case in
tests/unit/combo/combo-target-exhaustion.test.ts. Injected the old
model-lock path: tests failed (testStatus=active, sibling skip false).
After the change: 149 related tests pass, including the existing
passthrough 402 suite (ollama-cloud still model-locks) and
disableCooling suite.

Notes

Contributor only — please do not merge from this account.
Does not overlap PR #13006 (quota-weighted snapshot cache). That PR
stops the next draw; this one unblocks the same request.

@HouMinXi
HouMinXi force-pushed the fix/grok-cli-402-connection-credits branch from 5c05a17 to e64963c Compare September 8, 2026 16:27
@HouMinXi

HouMinXi commented Sep 8, 2026

Copy link
Copy Markdown
Contributor Author

Head is now e64963c5c (same branch). Review of the first SHA caught a redundant lowercase in the wallet matcher; the 402 predicate now also requires the Grok Build body phrase when one is present. LOCAL review of this SHA is in flight.

@HouMinXi
HouMinXi force-pushed the fix/grok-cli-402-connection-credits branch from e64963c to 19a1c91 Compare September 8, 2026 16:42
@HouMinXi

HouMinXi commented Sep 8, 2026

Copy link
Copy Markdown
Contributor Author

Head is now 19a1c9154. Same branch: grok-web and xai-oauth 402s now park the login the same way grok-cli does (shared weekly wallet). xai API-key connections were already connection-wide. ollama-cloud / kilo-gateway stay model-scoped.

@HouMinXi
HouMinXi force-pushed the fix/grok-cli-402-connection-credits branch from 19a1c91 to bca9d3f Compare September 8, 2026 17:15
@HouMinXi

HouMinXi commented Sep 8, 2026

Copy link
Copy Markdown
Contributor Author

Head bca9d3f6e01774ea6f0ad6549cbdfbe7c73e20d9. Empty-body grok-cli 402 now parks the login as credits_exhausted (AUTH + combo agree). Unrelated 402 bodies stay model-scoped.

@HouMinXi

HouMinXi commented Sep 9, 2026

Copy link
Copy Markdown
Contributor Author

R4 LOCAL on bca9d3f6e (6 files): HOLD, 0 CONFIRMED, 7 UNCERTAIN. No TTY so HOLD UX aborted. Product change stands.

Adjudication (source, not the summary field):

  1. isCreditsExhausted matching "usage balance exhausted" for every provider — DISMISS. The phrase is Grok's wallet string. Callers of isCreditsExhausted already treat it as empty wallet. Docstring in sharedWalletCredits.ts already says so.

  2. provider as string in markSharedWalletCreditsExhaustion — DISMISS. Same pattern as markAuthLevelExhaustion / markAgentrouterConnectionQuotaExhaustion. Guard isSharedWalletCredits402 already requires typeof provider === "string".

  3. Three AUTH sites — DISMISS. Each site owns a different lockout path (force creditsExhausted, skip fix(resilience): per-model 402 on a passthrough gateway terminalizes the whole connection (free models knocked out, never recovers) — 402 variant of #3027 #12242 lockout, skip persistUnavailableState lockModel). Collapsing them would grow auth.ts past the file-size freeze.

4+5. Claim that isSharedWalletCredits402 returns true for every grok-cli 402 body — DISPROVED. Non-empty unrelated body ("Add credits to continue, or switch to a free model") is tested and stays model-scoped: tests/unit/auth-grok-cli-402-shared-wallet.test.ts "a grok-cli 402 with an unrelated body does not park the connection".

  1. Empty/null body treated as wallet — KEEP as designed. Empty-body AUTH park is now tested. Live Grok 402s carry "Grok Build usage balance exhausted"; empty is the fail-closed case for a 402 with no body from these three providers.

  2. Combo branch before auth-level — DISMISS. 402 is not in AUTH_LEVEL_ERROR_STATUSES [401, 403]. Without the new branch the 402 falls through and does not mark the connection at all.

Not reopening LOCAL.

@HouMinXi
HouMinXi force-pushed the fix/grok-cli-402-connection-credits branch 2 times, most recently from 636cb91 to d30d3c9 Compare September 10, 2026 12:58
Grok Build bills Chat/Imagine/Voice/Build/API against one weekly pool, but
the registry still sets passthroughModels so a 402 was treated as a
per-model billing miss. Combo then locked grok-4.6 and skipped every other
login in the same set. Classify that 402 as connection-wide credits
exhaustion so the next Grok Build login can still run.

Signed-off-by: Minxi Hou <houminxi@gmail.com>
@HouMinXi
HouMinXi force-pushed the fix/grok-cli-402-connection-credits branch from d30d3c9 to 39e388a Compare September 11, 2026 08:22
@HouMinXi

Copy link
Copy Markdown
Contributor Author

CI red on this PR matches current release/v3.8.51, not the unique files in the diff.

Measured on tip af49d4972 (#12925) against the HouMinXi open set:

  • Docs Gates: live provider modules 358 vs docs/SVG still saying 356; cli-tunnel skill dry-run out of date.
  • Fast Quality: open-sse/services/autoCombo/__tests__/autoCombo.test.ts TS2739 vs frozen baseline 0; pack-policy follows that.
  • ESLint: tests/unit/volcengine-plan-binding-upsert.test.ts @typescript-eslint/no-explicit-any (file is on tip).
  • Unit shards: chat-rate-limit-body-lock, antigravity-missing-project-chat, chat-rejects-image-only-model, stream-handler-public-error-boundary (and the glm GLM_STREAM_BUFFER_BYTES assert, which fix(stream): accept the buffer size glm.ts has been passing since #12179 #12925 already landed on tip).

Unique diff does not touch those files. Contributor-only; not merging.

@diegosouzapw
diegosouzapw merged commit f0b83b8 into diegosouzapw:release/v3.8.51 Sep 11, 2026
8 of 16 checks passed
@HouMinXi
HouMinXi deleted the fix/grok-cli-402-connection-credits branch September 16, 2026 14:03
Githab-capibara added a commit to Githab-capibara/OmniRoute that referenced this pull request Sep 17, 2026
…diegosouzapw#13061)

The live evidence makes this: four Grok Build logins at 32-93% weekly remaining, skipped as "model locked by resilience" because the one login at 1% answered 402 and `passthroughModels: true` made that read as a per-model billing miss. A shared weekly wallet is a connection-wide signal, not a model verdict — `isSharedWalletCredits402` puts the scope where the biller put it.

---

Validated in one consolidated worktree cut from `release/v3.8.51`, boarded with the other 19 PRs of this batch. Two in-batch conflicts, both additive and resolved by keeping each side: the `ENVIRONMENT.md` table (diegosouzapw#13035 + diegosouzapw#13011) and the `chatHelpers.ts` import block (diegosouzapw#12975 on the tip + diegosouzapw#13017).

- `typecheck:core` clean; `check:dashboard-typecheck` OK (206 pre-existing, within baseline); `check:changelog-integrity` OK; `check:docs-counts` migrations ✓
- complexity 2816 / baseline 3218 and cognitive-complexity 1271 / baseline 1437 — both under baseline
- 531 of 532 focused assertions green across the batch's 46 test files
- `check-file-size` rebaselined for the batch's real growth (annotation `_rebaseline_2026_09_11_mergebatch_v3851_houminxi`, landed on diegosouzapw#13038), attributed per PR

The single red is **not this batch**: `tests/unit/combo/quota-weighted-strategy.test.ts` → "A/B isolation: 7 hard-empty + 2 at 0.5% + 1 at 40%, floor=1" asserts an order between two connections of identical weight and flakes on the pure tip too — 2 failures in 4 runs at `origin/release/v3.8.51` with nothing from this batch applied.

⚠️ base-red inherited: diegosouzapw#12732 — `Docs Gates`, `Merge integrity`, `No new ESLint warnings`, `Unit Tests fast-path` and `Fast Quality Gates` reproduce on the pure tip (provider count 356 vs the 358 the modules define, SKILL.md drift, and `open-sse/utils/stream.ts` at 3115 > frozen 3098, untouched here).

Thanks @HouMinXi — the live evidence on these (X500 logs, `storage.sqlite` state, real `/v1/models` probes, the 36-minute outage write-up) is what let a 20-PR batch be reviewed as a unit.
muhamadgalihsaputra pushed a commit to niyatna/NiyatnaRoute that referenced this pull request Sep 27, 2026
…diegosouzapw#13061)

The live evidence makes this: four Grok Build logins at 32-93% weekly remaining, skipped as "model locked by resilience" because the one login at 1% answered 402 and `passthroughModels: true` made that read as a per-model billing miss. A shared weekly wallet is a connection-wide signal, not a model verdict — `isSharedWalletCredits402` puts the scope where the biller put it.

---

Validated in one consolidated worktree cut from `release/v3.8.51`, boarded with the other 19 PRs of this batch. Two in-batch conflicts, both additive and resolved by keeping each side: the `ENVIRONMENT.md` table (diegosouzapw#13035 + diegosouzapw#13011) and the `chatHelpers.ts` import block (diegosouzapw#12975 on the tip + diegosouzapw#13017).

- `typecheck:core` clean; `check:dashboard-typecheck` OK (206 pre-existing, within baseline); `check:changelog-integrity` OK; `check:docs-counts` migrations ✓
- complexity 2816 / baseline 3218 and cognitive-complexity 1271 / baseline 1437 — both under baseline
- 531 of 532 focused assertions green across the batch's 46 test files
- `check-file-size` rebaselined for the batch's real growth (annotation `_rebaseline_2026_09_11_mergebatch_v3851_houminxi`, landed on diegosouzapw#13038), attributed per PR

The single red is **not this batch**: `tests/unit/combo/quota-weighted-strategy.test.ts` → "A/B isolation: 7 hard-empty + 2 at 0.5% + 1 at 40%, floor=1" asserts an order between two connections of identical weight and flakes on the pure tip too — 2 failures in 4 runs at `origin/release/v3.8.51` with nothing from this batch applied.

⚠️ base-red inherited: diegosouzapw#12732 — `Docs Gates`, `Merge integrity`, `No new ESLint warnings`, `Unit Tests fast-path` and `Fast Quality Gates` reproduce on the pure tip (provider count 356 vs the 358 the modules define, SKILL.md drift, and `open-sse/utils/stream.ts` at 3115 > frozen 3098, untouched here).

Thanks @HouMinXi — the live evidence on these (X500 logs, `storage.sqlite` state, real `/v1/models` probes, the 36-minute outage write-up) is what let a 20-PR batch be reviewed as a unit.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants