Skip to content

fix(antigravity): retry transient upstream failures - #2054

Closed
Jordannst wants to merge 4 commits into
decolua:masterfrom
Jordannst:fix/antigravity-auto-retry
Closed

Jordannst wants to merge 4 commits into
decolua:masterfrom
Jordannst:fix/antigravity-auto-retry

Conversation

@Jordannst

@Jordannst Jordannst commented Jun 24, 2026 •

Copy link
Copy Markdown
Contributor

Why

Antigravity upstream occasionally returns short-lived capacity/transient errors for Claude models, especially:

  • 503 MODEL_CAPACITY_EXHAUSTED / “No capacity available...”
  • high-traffic or terminated-agent style transient failures
  • intermittent 5xx/network-like failures before a response can complete

Before this change, these failures could immediately surface to clients even when a short retry or another Antigravity account could recover successfully.

While testing the retry path with Claude clients, I also hit a separate Antigravity request validation failure:

  • 400 invalid_request_error: tools: Tool names must be unique.

That can happen when client tool names collide after Antigravity name sanitization, e.g. distinct incoming names normalize into the same forwarded tool name.

What changed

  • Adds Antigravity-specific retry delay handling for transient upstream failures.
  • Enables the retry hook for Antigravity 500 responses in addition to existing 429/503 behavior.
  • Keeps bounded backoff behavior:
    • uses Retry-After / reset hints when present,
    • skips retry when the wait would be too long,
    • uses short exponential-style retry delays for transient 5xx/capacity failures.
  • Preserves non-transient 400 behavior: invalid requests are not retried.
  • Deduplicates sanitized Antigravity tool names before emitting the final single functionDeclarations group.
  • Adds tests for retry behavior, BaseExecutor retry-hook gating, and sanitized tool-name collisions.

Why this is safe

  • Retry remains config-gated by status code; adding 500 only enables the existing Antigravity retry hook for that status.
  • The hook can still veto retries, e.g. long Retry-After / reset windows.
  • Non-transient 400 errors remain non-retryable.
  • Tool dedupe only affects duplicate names after sanitization, preventing upstream rejection of the whole request.

Verification

  • npx vitest run tests/unit/antigravity-retry-hook.test.js tests/unit/base-executor-retry.test.js tests/translator/bugs-antigravity.test.js
    • 3 files passed, 20 tests passed, 1 expected fail.

Manual e2e via 9Router dev server on 127.0.0.1:20138 with Claude client using ag/claude-opus-4-6-thinking:

  • observed 503 retries with 2s → 4s → 8s backoff,
  • observed fallback to another Antigravity account,
  • observed successful streamed completion after retry/fallback,
  • confirmed the previous tools: Tool names must be unique error no longer occurred after the dedupe patch and fresh restart.

diegosouzapw pushed a commit to diegosouzapw/OmniRoute that referenced this pull request Jun 26, 2026
Port of upstream fix adding transient 5xx retry support and tool
declaration deduplication to the antigravity executor.

Three changes:
1. Add ANTIGRAVITY_TRANSIENT_ERROR_PATTERNS regex array,
   ANTIGRAVITY_TRANSIENT_STATUSES set (500/502/503/504), and
   ANTIGRAVITY_TRANSIENT_RETRY_MAX_MS (15s cap). Expose two new
   class methods — extractErrorMessage() and isTransientAntigravityError()
   — so the intra-request retry block can classify 5xx responses by body
   pattern as well as status code ("Agent execution terminated due to error",
   "high traffic", "capacity", etc.). Extends the existing 429 auto-retry
   block to also back off on matching transient 5xx, capped at 15s.

2. Equivalent of upstream per-status retry config: the inline
   MAX_AUTO_RETRIES=3 constant already covers 5xx now that the
   retry condition includes transient statuses.

3. Tool-name deduplication in buildGeminiTools (geminiToolsSanitizer.ts):
   add a seenToolNames Set so duplicate sanitized function names (same raw
   name repeated across groups, or two different names that collapse to the
   same sanitized form) are silently dropped — fixes Gemini API rejections
   caused by duplicate functionDeclaration names.

Co-authored-by: Sutarto Jordan Chrisfivo <Jordannst@users.noreply.github.com>
Inspired-by: decolua/9router#2054
@decolua

decolua commented Jun 26, 2026

Copy link
Copy Markdown
Owner

Thanks @Jordannst! Reviewed and merged into master. The transient 5xx/capacity retry with bounded backoff, tool-name dedupe, and Headroom phantom-savings diagnostics all look solid. 🙏

@decolua decolua closed this Jun 26, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants