fix: compression model should read context_length from custom_providers - #15397
Closed
bestnite wants to merge 1 commit into
Closed
fix: compression model should read context_length from custom_providers#15397bestnite wants to merge 1 commit into
bestnite wants to merge 1 commit into
Conversation
In _check_compression_model_feasibility, the compression model's context length resolution only checked auxiliary.compression.context_length in config.yaml, missing the per-model context_length configured in custom_providers[].models[].context_length. This mirrors the main model's custom_providers lookup logic (lines ~1689-1725) for the compression model path: 1. Store normalized custom_providers in __init__ alongside _aux_compression_context_length_config 2. In _check_compression_model_feasibility, if no explicit config was set, fall through to match aux_model + aux_base_url against custom_providers entries Fixes NousResearch#8550.
Collaborator
Author
|
Agreed — #13813 covers this more broadly (incl. providers). Closing this one. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
When
auxiliary.compression.context_lengthis not explicitly set in config.yaml, the compression model's context length falls through toDEFAULT_CONTEXT_LENGTHSor the 128K fallback, skippingcustom_providers[].models[].context_lengthentirely.This means users who correctly configure their model's context length in
custom_providers:...still see the false warning:
Fix
custom_providersonself._aux_compression_custom_providersaux_modelandaux_base_url$, if no explicitcontext_lengthwas configured, match the compression model + base_url againstcustom_providers` entries — mirroring the main model's logic.Fixes #8550