Repository navigation
fix(antigravity): skip credits retry on full_quota_exhausted, persist 24h cooldown to DB (#3707) - #3754
Conversation
… 24h cooldown to DB (#3707) Three bugs fixed together: 1. decide429() was called AFTER the Google One AI credits retry — a quota-exhausted account could hang ~41s on the credits HTTP call before the full_quota_exhausted verdict was ever computed. Now decide429() runs first; credits retry is skipped when kind === full_quota_exhausted. 2. setConnectionRateLimitUntil() was never called from the antigravity executor — the 24h cooldown state lived only in memory and was lost on restart, causing post-restart requests to re-learn exhaustion the hard way (7,412 upstream 429s documented in #3707). markConnectionQuotaExhausted() now persists it to the DB. 3. antigravity429Engine classify429() did not recognise the real Antigravity quota message ("Individual quota reached. Contact your administrator to enable overages.") — it fell through to the unknown category and never triggered the quota_exhausted path. Added "quota reached", "enable overages", "individual quota" to the keywords.
|
You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard. |
There was a problem hiding this comment.
Code Review
This pull request optimizes the Antigravity executor by skipping Google One AI credits retry on a full quota exhaustion verdict and persisting the 24-hour cooldown in the database. It also updates the 429 engine to recognize native Antigravity quota-exhausted error messages and adds unit tests. However, the reviewer identified a critical logic bug where calling decide429 before the credits retry check completely disables the credits retry mechanism, making it dead code. The reviewer provided actionable suggestions to refactor the logic and pre-compute whether a credits retry is possible before marking the connection as exhausted.
Important
The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.
| // 3. Decide final retry time BEFORE the credits retry so that | ||
| // full_quota_exhausted can skip the credits attempt entirely | ||
| // (avoids ~41s hold on an already-exhausted account) and | ||
| // persist the cooldown to DB for post-restart routing. | ||
| const decision: Decision = decide429(category, parsedRetryMs); | ||
| retryMs = decision.retryAfterMs; | ||
| log?.debug?.( | ||
| "AG_429", | ||
| `Category: ${category}, Decision: ${decision.kind} — ${decision.reason}` | ||
| ); | ||
|
|
||
| if (decision.kind === "full_quota_exhausted" && retryMs) { | ||
| markConnectionQuotaExhausted(accountId, retryMs); | ||
| } | ||
|
|
||
| const creditsAlreadyInjected = | ||
| (transformedBody as { enabledCreditTypes?: unknown }).enabledCreditTypes != null; |
There was a problem hiding this comment.
⚠️ Critical Logic Bug: Google One AI Credits Retry is completely disabled
In this PR, decide429() is called before the credits retry check. However, decide429(category, ...) always returns decision.kind === "full_quota_exhausted" when category is "quota_exhausted" (as verified by the unit test decide429: quota_exhausted category → full_quota_exhausted kind).
Because of this:
decision.kind !== "full_quota_exhausted"will always evaluate tofalsewhencategory === "quota_exhausted".- As a result, the
ifblock for retrying with credits (lines 1224-1229) is now dead code and will never be executed. - On the very first
quota_exhaustederror, the connection is immediately marked as fully exhausted in the DB for 24 hours, completely bypassing the Google One AI credits retry mechanism.
Solution
We should only mark the connection as full_quota_exhausted and skip the credits retry if we cannot retry with credits (i.e., credits are already injected, or credits mode is off/not supported). Let's define canRetryWithCredits first, and then use it to determine whether we should mark the connection as exhausted.
const creditsAlreadyInjected =
(transformedBody as { enabledCreditTypes?: unknown }).enabledCreditTypes != null;
const canRetryWithCredits =
category === "quota_exhausted" &&
!creditsAlreadyInjected &&
shouldRetryWithCredits(credentials?.accessToken || "", creditsMode !== "off");
// 3. Decide final retry time BEFORE the credits retry so that
// full_quota_exhausted can skip the credits attempt entirely
// (avoids ~41s hold on an already-exhausted account) and
// persist the cooldown to DB for post-restart routing.
const decision: Decision = decide429(category, parsedRetryMs);
retryMs = decision.retryAfterMs;
log?.debug?.(
"AG_429",
`Category: ${category}, Decision: ${decision.kind} — ${decision.reason}`
);
const isFullQuotaExhausted =
decision.kind === "full_quota_exhausted" && !canRetryWithCredits;
if (isFullQuotaExhausted && retryMs) {
markConnectionQuotaExhausted(accountId, retryMs);
}| if ( | ||
| category === "quota_exhausted" && | ||
| decision.kind !== "full_quota_exhausted" && | ||
| !creditsAlreadyInjected && | ||
| shouldRetryWithCredits(credentials?.accessToken || "", creditsMode !== "off") | ||
| ) { |
… 24h cooldown to DB (cheaper-inference#3707) (cheaper-inference#3754) Three bugs fixed together: 1. decide429() was called AFTER the Google One AI credits retry — a quota-exhausted account could hang ~41s on the credits HTTP call before the full_quota_exhausted verdict was ever computed. Now decide429() runs first; credits retry is skipped when kind === full_quota_exhausted. 2. setConnectionRateLimitUntil() was never called from the antigravity executor — the 24h cooldown state lived only in memory and was lost on restart, causing post-restart requests to re-learn exhaustion the hard way (7,412 upstream 429s documented in cheaper-inference#3707). markConnectionQuotaExhausted() now persists it to the DB. 3. antigravity429Engine classify429() did not recognise the real Antigravity quota message ("Individual quota reached. Contact your administrator to enable overages.") — it fell through to the unknown category and never triggered the quota_exhausted path. Added "quota reached", "enable overages", "individual quota" to the keywords.
Closes #3707
Root Causes
Three interconnected bugs, all in the antigravity 429 path:
Bug 1 — decide429() called too late (~41s hang)
The Google One AI credits retry HTTP call was attempted before
decide429()was ever consulted. For a quota-exhausted account, this meant a live upstream request was made even though the final verdict was always going to befull_quota_exhausted. On slow/loaded upstream, this caused ~41s request hangs before the account was skipped.Bug 2 — learned cooldown lost on restart
setConnectionRateLimitUntil()was exported but never called from the antigravity executor. The in-memoryproviderCooldownTrackercleared on restart, forcing every post-restart request to re-discover exhaustion via a live 429.Bug 3 — real AG message never classified as quota_exhausted
classify429()inantigravity429Engineonly checked for"quota_exhausted"(literal) and"quota exhausted". The actual Antigravity error message —"Individual quota reached. Contact your administrator to enable overages. Resets in Xh."— fell through tounknown, meaning thefull_quota_exhaustedpath never fired for real AG quota hits.Fix
decide429()call to before the credits retry checkdecision.kind === "full_quota_exhausted"→ skip credits retry entirely + callmarkConnectionQuotaExhausted(accountId, retryAfterMs)to persist the cooldown"quota reached","enable overages","individual quota"keywords toQUOTA_EXHAUSTED_KEYWORDSValidation
5 new TDD tests in
tests/unit/antigravity-429-quota-cooldown.test.ts, failing before fix → passing after:decide429("quota_exhausted")→full_quota_exhausted(engine contract)quota_exhausted(new keyword coverage)markConnectionQuotaExhaustedpersists 24h cooldown;isConnectionRateLimitedreturnstruetypecheck:core clean (0 errors). 44/44 tests green on affected files.