Repository navigation
feat: add ModelScope provider with circuit breaker and daily quota lock - #1430
diegosouzapw merged 11 commits into
Conversation
- Register ModelScope provider in provider registry - Add moonshotai/Kimi-K2.5, ZhipuAI/GLM-5, stepfun-ai/Step-3.5-Flash models - Set passthroughModels: true for per-model lockout behavior - Add ModelStatusBadge and ModelStatusContext UI components - Update provider details page with model status display Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Add provider-level failure tracking with circuit breaker behavior - PROVIDER_FAILURE_THRESHOLD: 5 failures in 20 minutes triggers cooldown - PROVIDER_COOLDOWN_MS: 10 minutes cooldown period - Add daily quota exhaustion detection (lock until tomorrow 00:00) - isDailyQuotaExhausted(): detect "today's quota" / "try again tomorrow" - getMsUntilTomorrow(): calculate ms until next day midnight - Add context overflow and malformed request pattern detection - Extend hasPerModelQuota() for openai-compatible-* providers - Add unit tests for new fallback functions Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Add proactive context compression when exceeding 85% of token limit - Improve Claude passthrough path (remove double translation) - Add daily quota exhaustion handling in chatCore - Enhance error classifier with new error patterns - Add tests for chat route coverage and error classification Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Add client headers support for Codex executor - Improve Antigravity executor with signature mode handling - Pass provider context for circuit breaker integration Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Add intelligent combo routing panel with enhanced UI - Update combo service with improved routing logic - Add unit tests for combo routing engine Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Update English and Chinese translations for combo routing features - Add new translation keys for intelligent panel and routing strategies Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Add logs directory to watch ignore list - Export provider cooldown helper functions from open-sse/index.ts - Wrap dashboard layout with ModelStatusProvider - Prettier formatting on guide-settings route - Clear provider failure state on successful request in chatHelpers - Check provider cooldown in auth service
- remove isContentModerationResponse import from accountFallback - remove content moderation validation from validateResponseQuality function
- Remove CONTENT_MODERATION_PATTERNS from checkFallbackError - Remove content moderation related exports from accountFallback - Remove content moderation tests from account-fallback-service.test - This reverts the Phase 8.5 content moderation detection feature
![gemini-code-assist[bot]](https://avatars.githubusercontent.com/in/956858?s=60&v=4){kind=small}
There was a problem hiding this comment.
Code Review
This pull request introduces a comprehensive provider-level circuit breaker and daily quota management system. Key additions include a new ModelStatusBadge component for real-time availability tracking, support for the ModelScope provider with per-model quota logic, and enhanced error classification for daily quota exhaustion. I have identified a few issues: a missing variable declaration in the combo handler that could lead to global leaks, a redundant map lookup in the provider registry, and a potential race condition in the model status polling logic where multiple concurrent registrations could trigger redundant API calls.
| let earliestRetryAfter = null; | ||
| let lastStatus = null; | ||
| const startTime = Date.now(); | ||
| globalAttempts = 0; |
There was a problem hiding this comment.
The variable globalAttempts is assigned here but appears to be missing a let or const declaration in this scope. This could lead to a global variable leak or reference errors in strict mode. Given that handleRoundRobinCombo uses a local let globalAttempts, this should be declared locally here as well to ensure thread-safety and avoid concurrency issues between different combo requests.
| globalAttempts = 0; | |
| let globalAttempts = 0; |
| // First check full model ID with provider prefix (e.g., "moonshotai/Kimi-K2.5") | ||
| const cachedWithPrefix = _unsupportedParamsMap.get(modelId); | ||
| if (cachedWithPrefix) return cachedWithPrefix; | ||
|
|
||
| // Fall back to bare ID (e.g., "Kimi-K2.5") |
There was a problem hiding this comment.
This lookup is redundant. The modelId has already been checked against _unsupportedParamsMap at the beginning of the function (line 2147). If it wasn't found there, it will not be found here either, as the value of modelId has not changed.
| // First check full model ID with provider prefix (e.g., "moonshotai/Kimi-K2.5") | |
| const cachedWithPrefix = _unsupportedParamsMap.get(modelId); | |
| if (cachedWithPrefix) return cachedWithPrefix; | |
| // Fall back to bare ID (e.g., "Kimi-K2.5") | |
| // Fall back to bare ID (e.g., "Kimi-K2.5") |
| return `${provider}/${model}`; | ||
| } | ||
|
|
||
| async function fetchModelStatus(): Promise<void> { |
There was a problem hiding this comment.
The fetchModelStatus function is called whenever a new model is registered. If multiple components register models simultaneously (e.g., on page load when multiple badges are rendered), this will trigger multiple concurrent fetch requests to the same endpoint. Consider adding a concurrency guard (e.g., an isFetching flag) to ensure only one request is in flight at a time, or debouncing the calls.
|
While testing 3.6.9 including this PR, I found that this PR does not honor the circuit breaker settings exposed in the Settings page. The new provider-level cooldown behavior is effectively hardcoded instead of following the existing resilience/profile configuration. That is particularly unfriendly for providers that have to compete with other users during peak hours, where operators may need to tune breaker behavior based on real traffic conditions. If the intention is to add this feature, it should be wired into the existing settings/config surface, and the related Settings page options should be updated accordingly. I do not agree with merging this into 3.6.9 in its current form. @diegosouzapw |
d2d013f to
3432dfd
Compare
d6cbdc1
into
cheaper-inference:release/v3.7.0
…ck (cheaper-inference#1430) Integrated into release/v3.7.0. Thanks @clousky2020 for this massive and important contribution! 🎉 We've translated the deprecation comments to English for consistency, and it is now officially merged into the release branch. Great work on the ModelScope integration and Circuit Breaker!
…ck (cheaper-inference#1430) Integrated into release/v3.7.0. Thanks @clousky2020 for this massive and important contribution! 🎉 We've translated the deprecation comments to English for consistency, and it is now officially merged into the release branch. Great work on the ModelScope integration and Circuit Breaker!
Summary
Test plan
npm run buildexit code 0)