Skip to content

Conversation

@letanloc1998
Copy link
Contributor

Context

Fallback recovery: When no providers are available, reduce all cooldowns by 1 minute.
This allows the next adjustActiveHandler() call to potentially find an available provider without waiting for the full cooldown duration.

Implementation

When all providers are unavailable.
Reduce each provider's cooldown by 1 minute to progressively unblock them.
This creates a "soft recovery" mechanism where providers gradually become available rather than waiting for full cooldown expiry.

@changeset-bot
Copy link

changeset-bot bot commented Oct 25, 2025

⚠️ No Changeset found

Latest commit: e099045

Merging this PR will not cause a version bump for any packages. If these changes should not result in a new version, you're good to go. If these changes should result in a version bump, you need to add a changeset.

This PR includes no changesets

When changesets are added to this PR, you'll see the packages that this PR includes changesets for and the associated semver types

Click here to learn what changesets are, and how to add one.

Click here if you're a maintainer who wants to add a changeset to this PR

@letanloc1998
Copy link
Contributor Author

Current setCooldown Usage

setCooldown() is now only used in error handling (line 101-115):

  • Rate limit errors (429): 10min cooldown → switch provider → retry
  • Other errors: 10min cooldown → rethrow (not all errors are provider-related) => recovers via reduceCooldown when no handlers available

if (this.isRateLimitError(error) || this.isOverloadError(error)) {
// Set cooldown for the current provider
await this.usage.setCooldown(this.activeProfileId, 10 * 60 * 1000)
// Switch to a different provider
await this.adjustActiveHandler("Retryable Error")
// Retry the request with the new provider
yield* this.createMessage(systemPrompt, messages, metadata)
return
}
// For non-rate limit errors, set cooldown and rethrow
await this.usage.setCooldown(this.activeProfileId, 10 * 60 * 1000)
throw error

@letanloc1998 letanloc1998 changed the title Patch 7 Reduce all cooldowns by 1min to gradually unblock providers Oct 25, 2025
@letanloc1998 letanloc1998 changed the title Reduce all cooldowns by 1min to gradually unblock providers When no providers are available, reduce all cooldowns by 1 minute. Oct 25, 2025
@mcowger
Copy link
Contributor

mcowger commented Oct 29, 2025

This is a solid idea.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants