Skip to content

fix(antigravity): classify geo-blocked egress, exclude account, real connection probe - #10420

Merged
diegosouzapw merged 19 commits into
cheaper-inference:release/v3.8.50from
rqzbeh:fix/antigravity-geoblock-resilience
Aug 16, 2026
Merged

diegosouzapw merged 19 commits into
cheaper-inference:release/v3.8.50from
rqzbeh:fix/antigravity-geoblock-resilience

Conversation

@rqzbeh

@rqzbeh rqzbeh commented Aug 14, 2026 •

Copy link
Copy Markdown
Contributor

Summary

Antigravity / Gemini Code Assist models (gemini-3.6-flash, etc.) fail from this
class of deployment with 400 FAILED_PRECONDITION "User location is not
supported for the API use."
— Google refuses the Cloud Code model API
(daily-cloudcode-pa.googleapis.com/v1internal:streamGenerateContent) from
unsupported egress locations. The dashboard "Test Connection" stayed green
because it only probed the OAuth userinfo endpoint
(googleapis.com/oauth2/v1/userinfo), which is NOT geo-restricted — so a green
tick never proved the model path worked.

This PR fixes three things:

  1. Classification — the geo refusal is now recognized as a distinct
    GEO_BLOCKED provider error (open-sse/services/errorClassifier.ts),
    instead of falling through as a generic 400.
  2. Account exclusion — on GEO_BLOCKED, the connection is marked and cached
    with a 24h rate_limited_until exclusion (open-sse/handlers/chatCore.ts),
    so routing moves to other accounts instead of re-selecting the broken one on
    every request. It is non-terminal (never banned/expired — src/sse/services/auth.ts),
    so the account recovers automatically once egress is fixed.
  3. Honest connection test — antigravity/agy "Test Connection" now probes the
    REAL model surface (streamGenerateContent with a 1-token body, mirroring the
    executor's URL/headers), so a green tick means the model path actually works
    and a geo-blocked egress shows red with an actionable diagnosis.

Error messages now explain the real cause (egress location vs account problem)
and the remedy (route antigravity/agy egress through a proxy in a supported
region, e.g. US/EU) instead of "Antigravity upstream error (400)".

Related Issues

Validation

  • Change type: routing
  • Focused tests and category gates from the golden path
  • npm run lint — no issues in changed files (repo-wide baseline has
    pre-existing errors in unrelated dashboard files; local install used
    --no-package-lock due to lockfile drift on the release branch)
  • Reconciled with the current active release base (branch from release/v3.8.50 tip)
  • Production-code changes include a new automated test in this PR
  • SonarQube PR analysis is green or any remaining issues are explicitly documented below

Focused tests run:

node --import tsx/esm --import ./open-sse/utils/setupPolyfill.ts --test \
  tests/unit/antigravity-geoblock-resilience.test.ts   # 10/10 pass
# + existing suites: errorclassifier-antigravity-403, classify429,
#   antigravity-429-quota-cooldown/tdd/switch-auth, antigravity-missing-project-chat
#   -> 68/68 total pass

Tests Added Or Updated

  • tests/unit/antigravity-geoblock-resilience.test.ts (new):
    • classifyProviderError: geo 400 → GEO_BLOCKED (JSON and raw-text bodies);
      generic 400 not classified as geo; 429 unaffected
    • isGeoBlockedError: Google wording variants matched; lookalikes rejected
    • classify429: RESOURCE_EXHAUSTED-per-minute stays rate_limited (guards the
      established repo behavior against regression)
    • buildAntigravityUpstreamError: geo body carries the actionable hint;
      non-geo body unchanged
    • connection-test probe: antigravity/agy buildProbe hits
      v1internal:streamGenerateContent with a minimal body (not userinfo)

Coverage Notes

  • Change spans open-sse/services/errorClassifier.ts,
    open-sse/handlers/chatCore.ts, open-sse/config/errorConfig.ts,
    open-sse/executors/antigravityUpstreamError.ts, src/sse/services/auth.ts,
    and the connection-test route (src/app/api/providers/[id]/test/{route,oauthTestConfig}.ts).
  • New tests cover the classifier, the error body, the 429 boundary, and the
    probe construction; the chatCore exclusion branch follows the existing
    PROJECT_ROUTE_ERROR pattern (connection state write is best-effort).

Reviewer Notes

  • The GEO_BLOCKED exclusion reuses rate_limited_until (24h via
    COOLDOWN_MS.geoBlocked) because the account-selection layer already honors
    it; the connection's lastErrorType/lastError/errorCode record the real
    reason for the dashboard.
  • buildProbe is a new optional OAuth-test-config hook; the codex-style
    acceptStatuses semantics are unchanged, and the 401/403 refresh-retry path
    now reuses the built probe when present.
  • This PR intentionally does NOT change 429 classification: the repo already
    treats Google's "Resource has been exhausted (e.g. queries per minute...)"
    as a per-minute rate limit (rate_limited), which is correct.
  • Operator action for the reported deployment: the configured egress proxy
    currently exits in Toronto, CA — Google does not offer this API from Canada.
    Point antigravity/agy egress at a proxy in a supported region (e.g. US/EU)
    and the accounts become usable without re-adding them.

Competitive system-prompt strip (follow-up commit)

Antigravity's server-side filter flags system prompts advertising competing
agents ("You are a Claude agent, built on Anthropic's Claude Agent SDK." —
Zed, Claude Code, etc.) and answers with 429 RESOURCE_EXHAUSTED.
sanitizeAntigravityGeminiRequest now strips known competitor identity
sentences from systemInstruction.parts before dispatch (port of
decolua/9router b566b20, generalized). Surrounding instruction text is
untouched; non-matching prompts pass through without allocation.

Tests: tests/unit/antigravity-competitive-prompt-strip.test.ts (6 cases) —
all green alongside the geo-block suite (50/50).

…connection probe

Google refuses the Cloud Code model API from unsupported egress locations
with 400 FAILED_PRECONDITION "User location is not supported for the API
use." Previously this surfaced as a cryptic "Antigravity upstream error
(400)", never excluded the account, and the dashboard connection test stayed
green because it only probed the (non-geo-restricted) OAuth userinfo endpoint.

- errorClassifier: new GEO_BLOCKED type + isGeoBlockedError detection
  (400/403 + location-not-supported wording); non-terminal classification.
- chatCore fallback: GEO_BLOCKED marks the connection and caches a 24h
  rate-limit-until exclusion so routing moves to other accounts instead of
  re-selecting the same one; never bans/expires the account.
- auth: GEO_BLOCKED joins the non-terminal group (no banned/expired state).
- antigravityUpstreamError: geo refusals carry an actionable message (egress
  location vs account problem, proxy-in-supported-region guidance).
- connection test: antigravity/agy now probe the REAL streamGenerateContent
  surface (buildProbe), so a green tick means the model path actually works
  and a geo-blocked egress shows red with a clear diagnosis.
@rqzbeh
rqzbeh requested a review from diegosouzapw as a code owner August 14, 2026 22:18
rqzbeh added 7 commits August 14, 2026 23:40
The earlier format pass (local prettier differs from the repo's pinned
version) rewrapped unrelated lines in chatCore.ts and the provider test
route. Restore the base formatting and re-apply only the GEO_BLOCKED
fallback branch and the buildProbe connection-test changes.
…EXHAUSTED)

Port decolua/9router b566b20, generalized: Antigravity flags system prompts
advertising competing agents ('You are a Claude agent, built on Anthropic's
Claude Agent SDK.' — Zed, Claude Code, etc.) and answers with a 429 quota
error. sanitizeAntigravityGeminiRequest now strips known competitor identity
sentences from systemInstruction.parts before dispatch; surrounding
instruction text is untouched and non-matching prompts pass through without
allocation.
… (#geoblock-resiliencefix/antigravity-geoblock-resilience)
@diegosouzapw

Copy link
Copy Markdown
Collaborator

Thanks for the deep dive here — the root-cause framing (green tick only proving the OAuth userinfo probe works, not the actual model path) is exactly right, and the fix is a clean, minimal extension of the existing non-terminal connection-cooldown pattern (mirrors PROJECT_ROUTE_ERROR almost 1:1 across errorClassifier.ts/chatCore.ts/auth.ts).

I checked this out locally against release/v3.8.50 tip and ran the suite:

  • All 16 new tests pass (antigravity-geoblock-resilience.test.ts 10/10, antigravity-competitive-prompt-strip.test.ts 6/6).
  • Re-ran the three existing suites this touches (errorclassifier-antigravity-403, antigravity-429-quota-cooldown, antigravity-missing-project-chat — 13 tests) to confirm the new GEO_BLOCKED check inserted ahead of the existing 403/400 branches in classifyProviderError doesn't shift PROJECT_ROUTE_ERROR/ACCOUNT_DEACTIVATED classification — all green.
  • CI is fully green on the PR already, and eslint/tsc on the touched files show nothing new (the one lint hit on route.ts's @/lib/localDb import predates this PR).

This is merge-ready as-is. Two things I'd love a quick take on, neither blocking:

  1. isGeoBlockedError() isn't scoped by provider, unlike the neighboring 403 PROJECT_ROUTE_ERROR check (which gates on isCloudCodeProvider). Since classifyProviderError is shared across every provider, any upstream that ever returns a 400/403 with wording like "not available in your region" would also get the 24h-cooldown-and-auto-retry treatment, even if that provider's block is actually permanent rather than egress-fixable. Worth scoping it, or just confirming that's an accepted tradeoff.
  2. The "competitive system-prompt strip" change is a second, unrelated fix bundled in as a follow-up commit — well tested and in the right file, but a different concern (silent request-content mutation to dodge a content filter) from the geo-block routing fix the title describes. A split into its own PR next time would make both easier to review/revert independently.

Nice work — appreciate the actionable error messages and the reviewer notes explaining the Toronto-proxy repro.

rqzbeh added 2 commits August 15, 2026 11:32
Address reviewer feedback: classifyProviderError is shared across every
provider, so a lookalike 'not available in your region' body from an
unrelated upstream must not receive the egress-fixable 24h exclusion
treatment. Gate GEO_BLOCKED behind isGeoBlockEligibleProvider, which
matches the surfaces that actually emit Google's regional-availability
refusal: Cloud Code / Gemini Code Assist (antigravity, agy, cloudcode*),
the Gemini Developer API (gemini, gemini-cli, vertex), plus a
registry-driven fallback on executor/format. Non-Google providers fall
through to their existing 400/403 classification (typically null for an
unclassified 400), so a permanent block still follows its own path.
@rqzbeh

rqzbeh commented Aug 15, 2026

Copy link
Copy Markdown
Contributor Author

Thanks for the close read — both points are fair, and point 1 is now fixed on the branch (bfd06e9).

  1. Provider scoping — done. GEO_BLOCKED is now gated behind a new isGeoBlockEligibleProvider() check in classifyProviderError. The gate matches only the upstream surfaces that emit Google's regional-availability refusal:

    • Cloud Code / Gemini Code Assist: antigravity, agy, and any cloudcode/cloud-code id (the same family the 403 PROJECT_ROUTE_ERROR branch already gates on).
    • Gemini Developer API (generativelanguage.googleapis.com): gemini, gemini-cli, vertex — included deliberately, because those upstreams return the exact same "User location is not supported" FAILED_PRECONDITION.
    • Registry-driven fallback: any provider whose registry entry uses the antigravity executor or the gemini format stays eligible, so a future sibling id is covered without editing the list.

    A lookalike body ("not available in your region", etc.) from any other provider now falls through to its existing 400/403 classification — an unclassified 400 still returns null, so no 24h exclusion is applied to a block that may be permanent. isGeoBlockedError() itself is unchanged; it stays a pure phrase matcher, still used by the antigravity-only probe path in route.ts.

    Tests: 4 new cases in antigravity-geoblock-resilience.test.ts (eligible: gemini/vertex/gemini-cli; not eligible: openai/anthropic/g4f-gemini/mistral → null; no provider → null; 403 → GEO_BLOCKED preserved). The full affected set — geoblock-resilience, competitive-prompt-strip, errorclassifier-antigravity-403, antigravity-429-quota-cooldown, antigravity-missing-project-chat — is 33/33 green, and eslint on the touched files is clean.

  2. Bundled competitive strip — agreed, it is a separate concern. It landed as a follow-up commit because both changes touch the same antigravity error surface and were validated in one pass; the changelog fragment documents them as two distinct fixes. Future unrelated fixes will go into their own PRs so each is independently reviewable and revertable.

Trigger a fresh CI run for the PR: the previous run's 'Vitest (fast-path)'
job failed in 'npm ci' because the onnxruntime-node postinstall could not
download its binary from the Microsoft CDN (connect ETIMEDOUT
150.171.109.118:443). No tests ran; no code changed in this commit.
@rqzbeh

rqzbeh commented Aug 15, 2026

Copy link
Copy Markdown
Contributor Author

Re-triggered CI with an empty commit (c727e77): the previous Quality Gates run's "Vitest (fast-path)" job failed during npm ci, not in any test — the onnxruntime-node postinstall could not download its binary from the Microsoft CDN (connect ETIMEDOUT 150.171.109.118:443). No code changed in this commit; it only restarts the checks.

rqzbeh added 7 commits August 15, 2026 12:40
…gate

isGeoBlockEligibleProvider passes the raw provider (string | null | undefined)
to getRegistryEntry(provider: string), failing typecheck:core and the
ts7-diagnostics ratchet (TS2345 at errorClassifier.ts:166). Add an explicit
null guard; runtime behavior is unchanged — a falsy provider already resolved
to !entry -> false.
@diegosouzapw
diegosouzapw merged commit e44a409 into cheaper-inference:release/v3.8.50 Aug 16, 2026
16 checks passed
@diegosouzapw

Copy link
Copy Markdown
Collaborator

Validated in local merge-train .claude/worktrees/merge-train-20260816-000002-suite.log on devbox-vm-06-dev002 @ 8be62df65f61cc4cf29cdc3e6bc38173ffafa4e3 (FAST gates green: static + changed tests + vitest; daily full-suite run still required). Merged under the batch merge-train protocol (merge-gates §7). ⚠️ base-red inherited: #10523 (ServiceSupervisor order-dependent test, not touched by this batch).

tkgo11 pushed a commit to tkgo11/OmniRoute that referenced this pull request Sep 23, 2026
…connection probe (cheaper-inference#10420)

* fix(antigravity): classify geo-blocked egress, exclude account, real connection probe

Google refuses the Cloud Code model API from unsupported egress locations
with 400 FAILED_PRECONDITION "User location is not supported for the API
use." Previously this surfaced as a cryptic "Antigravity upstream error
(400)", never excluded the account, and the dashboard connection test stayed
green because it only probed the (non-geo-restricted) OAuth userinfo endpoint.

- errorClassifier: new GEO_BLOCKED type + isGeoBlockedError detection
  (400/403 + location-not-supported wording); non-terminal classification.
- chatCore fallback: GEO_BLOCKED marks the connection and caches a 24h
  rate-limit-until exclusion so routing moves to other accounts instead of
  re-selecting the same one; never bans/expires the account.
- auth: GEO_BLOCKED joins the non-terminal group (no banned/expired state).
- antigravityUpstreamError: geo refusals carry an actionable message (egress
  location vs account problem, proxy-in-supported-region guidance).
- connection test: antigravity/agy now probe the REAL streamGenerateContent
  surface (buildProbe), so a green tick means the model path actually works
  and a geo-blocked egress shows red with a clear diagnosis.

* chore(changelog): fragment for cheaper-inference#10420 antigravity geo-block resilience

* chore(pr): drop prettier-version drift noise, keep only real hunks

The earlier format pass (local prettier differs from the repo's pinned
version) rewrapped unrelated lines in chatCore.ts and the provider test
route. Restore the base formatting and re-apply only the GEO_BLOCKED
fallback branch and the buildProbe connection-test changes.

* fix(antigravity): strip competing-agent system prompts (429 RESOURCE_EXHAUSTED)

Port decolua/9router b566b20, generalized: Antigravity flags system prompts
advertising competing agents ('You are a Claude agent, built on Anthropic's
Claude Agent SDK.' — Zed, Claude Code, etc.) and answers with a 429 quota
error. sanitizeAntigravityGeminiRequest now strips known competitor identity
sentences from systemInstruction.parts before dispatch; surrounding
instruction text is untouched and non-matching prompts pass through without
allocation.

* chore(changelog): cover competitive prompt strip in cheaper-inference#10420 fragment

* fix(antigravity): scope GEO_BLOCKED classification to Google AI surfaces

Address reviewer feedback: classifyProviderError is shared across every
provider, so a lookalike 'not available in your region' body from an
unrelated upstream must not receive the egress-fixable 24h exclusion
treatment. Gate GEO_BLOCKED behind isGeoBlockEligibleProvider, which
matches the surfaces that actually emit Google's regional-availability
refusal: Cloud Code / Gemini Code Assist (antigravity, agy, cloudcode*),
the Gemini Developer API (gemini, gemini-cli, vertex), plus a
registry-driven fallback on executor/format. Non-Google providers fall
through to their existing 400/403 classification (typically null for an
unclassified 400), so a permanent block still follows its own path.

* ci: re-run quality gates

Trigger a fresh CI run for the PR: the previous run's 'Vitest (fast-path)'
job failed in 'npm ci' because the onnxruntime-node postinstall could not
download its binary from the Microsoft CDN (connect ETIMEDOUT
150.171.109.118:443). No tests ran; no code changed in this commit.

* fix(antigravity): guard provider before registry lookup in geo-block gate

isGeoBlockEligibleProvider passes the raw provider (string | null | undefined)
to getRegistryEntry(provider: string), failing typecheck:core and the
ts7-diagnostics ratchet (TS2345 at errorClassifier.ts:166). Add an explicit
null guard; runtime behavior is unchanged — a falsy provider already resolved
to !entry -> false.

* ci: re-run quality gates (vitest npm ci onnxruntime CDN flake)

---------

Co-authored-by: Rouzbeh <rqzbeh@users.noreply.github.com>
muhamadgalihsaputra pushed a commit to niyatna/NiyatnaRoute that referenced this pull request Sep 27, 2026
…connection probe (cheaper-inference#10420)

* fix(antigravity): classify geo-blocked egress, exclude account, real connection probe

Google refuses the Cloud Code model API from unsupported egress locations
with 400 FAILED_PRECONDITION "User location is not supported for the API
use." Previously this surfaced as a cryptic "Antigravity upstream error
(400)", never excluded the account, and the dashboard connection test stayed
green because it only probed the (non-geo-restricted) OAuth userinfo endpoint.

- errorClassifier: new GEO_BLOCKED type + isGeoBlockedError detection
  (400/403 + location-not-supported wording); non-terminal classification.
- chatCore fallback: GEO_BLOCKED marks the connection and caches a 24h
  rate-limit-until exclusion so routing moves to other accounts instead of
  re-selecting the same one; never bans/expires the account.
- auth: GEO_BLOCKED joins the non-terminal group (no banned/expired state).
- antigravityUpstreamError: geo refusals carry an actionable message (egress
  location vs account problem, proxy-in-supported-region guidance).
- connection test: antigravity/agy now probe the REAL streamGenerateContent
  surface (buildProbe), so a green tick means the model path actually works
  and a geo-blocked egress shows red with a clear diagnosis.

* chore(changelog): fragment for cheaper-inference#10420 antigravity geo-block resilience

* chore(pr): drop prettier-version drift noise, keep only real hunks

The earlier format pass (local prettier differs from the repo's pinned
version) rewrapped unrelated lines in chatCore.ts and the provider test
route. Restore the base formatting and re-apply only the GEO_BLOCKED
fallback branch and the buildProbe connection-test changes.

* fix(antigravity): strip competing-agent system prompts (429 RESOURCE_EXHAUSTED)

Port decolua/9router b566b20, generalized: Antigravity flags system prompts
advertising competing agents ('You are a Claude agent, built on Anthropic's
Claude Agent SDK.' — Zed, Claude Code, etc.) and answers with a 429 quota
error. sanitizeAntigravityGeminiRequest now strips known competitor identity
sentences from systemInstruction.parts before dispatch; surrounding
instruction text is untouched and non-matching prompts pass through without
allocation.

* chore(changelog): cover competitive prompt strip in cheaper-inference#10420 fragment

* fix(antigravity): scope GEO_BLOCKED classification to Google AI surfaces

Address reviewer feedback: classifyProviderError is shared across every
provider, so a lookalike 'not available in your region' body from an
unrelated upstream must not receive the egress-fixable 24h exclusion
treatment. Gate GEO_BLOCKED behind isGeoBlockEligibleProvider, which
matches the surfaces that actually emit Google's regional-availability
refusal: Cloud Code / Gemini Code Assist (antigravity, agy, cloudcode*),
the Gemini Developer API (gemini, gemini-cli, vertex), plus a
registry-driven fallback on executor/format. Non-Google providers fall
through to their existing 400/403 classification (typically null for an
unclassified 400), so a permanent block still follows its own path.

* ci: re-run quality gates

Trigger a fresh CI run for the PR: the previous run's 'Vitest (fast-path)'
job failed in 'npm ci' because the onnxruntime-node postinstall could not
download its binary from the Microsoft CDN (connect ETIMEDOUT
150.171.109.118:443). No tests ran; no code changed in this commit.

* fix(antigravity): guard provider before registry lookup in geo-block gate

isGeoBlockEligibleProvider passes the raw provider (string | null | undefined)
to getRegistryEntry(provider: string), failing typecheck:core and the
ts7-diagnostics ratchet (TS2345 at errorClassifier.ts:166). Add an explicit
null guard; runtime behavior is unchanged — a falsy provider already resolved
to !entry -> false.

* ci: re-run quality gates (vitest npm ci onnxruntime CDN flake)

---------

Co-authored-by: Rouzbeh <rqzbeh@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants