fix(agent): exponential backoff for rate-limit fallback cooldown (#30223 salvage) - #77607
Merged
kshitijk4poor merged 2 commits intoAug 3, 2026
Merged
Conversation
kshitijk4poor
enabled auto-merge (rebase)
August 3, 2026 11:40
alt-glitch
added
type/bug
|
Code Review: #77607 Verdict: Approve Trivial prompt detection: good optimization to skip memory provider on one-word prompts. LGTM - Reviewed diff. Changes are sound. |
Replace the fixed 60-second cooldown with exponential backoff: 30min → 1h → 2h → 4h cap. The counter is reset by restore_primary_runtime on successful primary-provider recovery, so the backoff is strictly for consecutive failures within a single degradation window. Closes NousResearch#29702
…ve rate-limits Review follow-up on the NousResearch#30223 salvage: the original changed the base cooldown from 60s to 1800s, benching the primary for 30 minutes on the FIRST 429 (30x regression in primary-restore latency) and breaking the existing test_rate_limit_exhaustion_keeps_60s_cooldown contract. Keep upstream's 60s base and escalate per consecutive rate-limit: 60s -> 2m -> 4m -> 8m -> ... capped at 4h. Counter still resets on successful primary restore (cicae's mechanism, unchanged). New tests: escalation doubling, 14400s cap, reset-on-restore. Existing 60s contract test passes UNCHANGED. Mutation-checked: escalation disabled -> 2 fail; reset disabled -> 1 fails.
kshitijk4poor
force-pushed
the
salvage/30223-ratelimit-backoff
branch
from
August 3, 2026 17:19
f79231a to
f37a149
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Salvage of #30223 by @cicav — commit cherry-picked to preserve authorship, plus one review follow-up commit.
Context — what this changes for users
When a provider rate-limits (429), Hermes fails over to a fallback and periodically tries to restore the primary. Upstream used a flat 60s cooldown: a provider in a sustained rate-limit storm gets hammered with restore attempts every minute, re-marshaling the whole conversation across providers each time. cicav's fix escalates the cooldown exponentially so persistent 429s back off.
Review follow-up folded
The original changed the BASE cooldown 60s -> 1800s: the FIRST 429 benched the primary for 30 minutes (a 30x regression in primary-restore latency for the common transient-429 case) and broke the existing test_rate_limit_exhaustion_keeps_60s_cooldown contract. Folded: keep upstream's 60s base, escalate per CONSECUTIVE rate-limit 60s -> 2m -> 4m -> ... capped at 4h. cicav's mechanism (counter + reset-on-successful-restore) is unchanged.
Verification
Closes #30223.