Repository navigation
Conversation
…rializer (cheaper-inference#1438) Integrated into release/v3.7.0. Thanks @benzntech for this great contribution! 🎉 We've removed the unrelated sync-fork.yml file and it's now merged into the release branch.
…ck (cheaper-inference#1430) Integrated into release/v3.7.0. Thanks @clousky2020 for this massive and important contribution! 🎉 We've translated the deprecation comments to English for consistency, and it is now officially merged into the release branch. Great work on the ModelScope integration and Circuit Breaker!
Complete translation and terminology improvements for Ukrainian documentation: - docs/i18n/uk-UA/README.md: full Ukrainian translation - docs/i18n/uk-UA/SECURITY.md: full Ukrainian translation - docs/i18n/uk-UA/docs/A2A-SERVER.md: full Ukrainian translation - docs/i18n/uk-UA/docs/API_REFERENCE.md: full Ukrainian translation - docs/i18n/uk-UA/docs/AUTO-COMBO.md: full Ukrainian translation - docs/i18n/uk-UA/docs/USER_GUIDE.md: complete translation (966 lines) Changes: - Translated all English content to Ukrainian - Preserved all code examples, commands, and technical terms - Maintained proper Ukrainian orthography with diacritical marks
Address all 6 review comments from PR cheaper-inference#1457: - Fix typo: "drastично" → "драстично" (mixed Latin/Cyrillic) - Translate model table entries: "Unlimited" → "Необмежено" - Translate model table entries: "No reported cap" → "Немає повідомлень про ліміт" - Translate OpenRouter description to Ukrainian - Fix section header: "Documentation" → "Документація" - Fix section header: "License" → "Ліцензія" All terminology now uses proper Ukrainian orthography.
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
…nts' into docs/uk-ua-translation-improvements
Integrated into release/v3.7.0
…heaper-inference#1444) Integrated into release/v3.7.0
…ference#1456) Integrated into release/v3.7.0
… encrypted tokens to providers (cheaper-inference#1462) Integrated into release/v3.7.0
cheaper-inference#1455) Integrated into release/v3.7.0
Integrated into release/v3.7.0
Integrated into release/v3.7.0
- create createSettingsApiHarness function with temp directory setup - add beforeEach/afterEach hooks for storage reset between tests - add after hook for cleanup - use dynamic imports after env setup to ensure proper initialization
- add provider-level circuit breaker config to PROVIDER_PROFILES - remove hardcoded threshold constants in favor of profile-based config - use getProviderProfile() to read thresholds with fallback defaults - support different failure tolerance per provider type
The mergeProviderProfile function was missing the three new fields added to PROVIDER_PROFILES (providerFailureThreshold, providerFailureWindowMs, providerCooldownMs). This caused tests to fail because the profile returned by getRuntimeProviderProfile did not include these fields. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
1. Remove 429 from PROVIDER_FAILURE_ERROR_CODES - 429 (rate limit) is already handled by model-level and account-level locks - Including it in provider-wide circuit breaker causes premature cooldown 2. Fix reference counting in ModelStatusContext - Changed registeredModels from Set to Map<string, number> - Prevents polling stop when one component unmounts while others still track the model 3. Fix model ID parsing for providers with slashes in model names - Use indexOf/substring instead of split to handle models like "modelscope/moonshotai/Kimi-K2.5" Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Register LM Studio as an OpenAI-compatible local provider and map the new grok-4.3 thinking model for web executor requests. This update also hardens related platform behavior by switching backup archive creation to execFileSync, validating ACP agent ids, expanding shared CORS handling, and making prompt injection guard failures return an explicit 500 response. To preserve existing stored credentials, encryption now derives new keys from a secret-based salt while still falling back to the legacy static-salt key during decryption.
- 在访问 providerSpecificData 前对 connection 添加可选链操作符 - 防止 connection 为 null/undefined 时导致的运行时错误
feat(fallback): add provider-level circuit breaker with configurable thresholds
…ts (cheaper-inference#1485) Integrated into release/v3.7.0
… API (cheaper-inference#1483) Integrated into release/v3.7.0
… patch (cheaper-inference#1481) Integrated into release/v3.7.0
Integrated into release/v3.7.0
…-vision models (cheaper-inference#1476) * feat(vision-bridge): add automatic image description fallback for non-vision models Implements VisionBridgeGuardrail (priority 5) that intercepts image-bearing requests to non-vision models, extracts descriptions via a configurable vision model (default: gpt-4o-mini), and replaces images with text before forwarding. Fails open on any error. - Add VisionBridgeGuardrail class extending BaseGuardrail - Add visionBridgeHelpers: extractImageParts, callVisionModel, replaceImageParts, resolveImageAsDataUri - Add visionBridgeDefaults with configurable settings - Register VisionBridgeGuardrail in GuardrailRegistry at priority 5 - Add 51 unit tests covering all spec scenarios (VB-S01 through VB-S10) - Dependency injection for getSettings and callVisionModel (testable without SQLite) Closes cheaper-inference#1424 * fix(vision-bridge): resolve Anthropic API, parallel processing, provider keys, structuredClone
…ma state collisions
![gemini-code-assist[bot]](https://avatars.githubusercontent.com/in/956858?s=60&v=4){kind=small}
Contributor
There was a problem hiding this comment.
Code Review
This pull request bumps the version to 3.7.0 and introduces a comprehensive overhaul of the resilience system, including request queueing, connection-level cooldowns, and global provider circuit breakers. It adds support for the ModelScope and LM Studio providers, implements a new asynchronous Batch Processing API with a monitoring dashboard, and introduces a Vision Bridge guardrail for image description extraction. Feedback identifies a bug in the batch processor where streaming should be explicitly disabled to avoid parsing failures and recommends optimizing large file uploads in the API to prevent memory exhaustion.
| body: JSON.stringify({ | ||
| ...body, | ||
| // BATCH-SPECIFIC: Force stream: false — batches don't support SSE responses | ||
| stream: body.stream ?? false, |
Contributor
There was a problem hiding this comment.
Batch processing is asynchronous and does not support Server-Sent Events (SSE) responses. The current implementation allows stream: true if present in the input body, which will cause the processor to fail when it attempts to parse the SSE stream as a single JSON object. This should be forced to false regardless of the input.
Suggested change
| stream: body.stream ?? false, | |
| stream: false, |
| if (done) break; | ||
| chunks.push(value); | ||
| } | ||
| const content = Buffer.concat(chunks.map((c) => Buffer.from(c))); |
Contributor
There was a problem hiding this comment.
Reading the entire file into memory using Buffer.concat can lead to Out Of Memory (OOM) errors, especially since the maximum file size is set to 512MB. For a server that might run in constrained environments (like a small VPS), it is better to stream the data directly to its destination or process it in smaller chunks. Since the database stores this as a BLOB, consider using a temporary file or a more memory-efficient way to handle large uploads.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
My previous PR #1479 had one issue, it always routed to the chat endpoint, so for example bulking embeddings failed.
This PR fixes that issue.
Validation
npm run lintnpm run test:unitnpm run test:coverage>= 60%for statements, lines, functions, and branchesTests Added Or Updated
Added tests for affected code