[Ardem2025]

Related Issues & Background

Currently, when utilizing Gemini or customized Antigravity backends with complex routing combinations (such as combos incorporating fallback behaviors), the router struggles with two major issues:

1. Malformed/Unescaped Textual Tool Calls: Differences in how Gemini-compliant formats and OpenAI envelopes wrap tools sometimes leads to either tool definitions leaking into prose or malformed SSE JSON payload frames (especially when stream chunks split in the middle of pseudo-tool tags).
2. Repetitive 429 Rate-Limit Cascades: When a high-tier model's quota gets exhausted (e.g. Google's Resource has been exhausted or credits running out), the error was labeled as rate_limited rather than quota_exhausted. Because rate_limited defaults to a tiny cooldown (e.g. 5 seconds), every single incoming request repeatedly attempts to invoke the broken model, suffering slow network timeouts (4s to 12s per call) before eventually routing to a fallback low-tier model.

---

Key Changes In This Pull Request

1. Gemini / Antigravity Tool Call Normalization & Sanitization

• Pre-commit Filters: Added strict zero-width character stripping (stripZeroWidthText()) in the response sanitizer to guarantee correct payload parsing.
• Textual Marker Guard: Implemented robust tracking (containsTextualToolCallMarker() & parseTextualToolCallContent()) to transparently capture and convert inline textual tool calls to structured OpenAI tool_calls formats when missing, suppressing them from plaintext outputs so client UIs stay clean.
• Fragment Buffer in SSE Streams: Modified the stream transformer (applyTextualToolCallStreamingGuard()) to buffer fragmented JSON slices before emitting, preventing client-side parsers from choking on partial pseudo-tags.

2. Adaptive Model Lockout for Quota Exhaustion (429 classification)

• Custom Signature Mapping: Added regex pattern support to src/shared/utils/classify429.ts and open-sse/services/accountFallback.ts to seamlessly classify Google's Resource has been exhausted and Check quota warnings.
• Lockout Escalation: Modified src/sse/services/auth.ts to analyze 429 response bodies using looksLikeQuotaExhausted(). If a quota limit signature is found, the system marks the backend model as quota_exhausted.
• Instant Routing Skips: Once a model is registered with quota_exhausted status, the combo router dynamically skips the node altogether—returning fallback targets (like Flash) instantly (reducing latency from 13 seconds of network cascading down to < 2.5 seconds).

---

Verification & Testing

• Tested against our live self-hosted setups; successfully caught the Antigravity Opus fallback limits and instantaneously bypassed it to Flash.
• Automated unit tests updated and verified:
  • Node --test tests/unit/classify429.test.ts (14/14 tests passing)
  • All stream translator tests passing under typescript execution.

[gemini-code-assist]

Warning

You have reached your daily quota limit. Please wait up to 24 hours and I will start processing your requests again!

[kilo-code-bot]

Kilo Code Review could not run — your account is out of credits.

Add credits or switch to a free model to enable reviews on this change.