fix(compression): honor custom provider context lengths - #21953
Conversation
|
The test CI failure is pre-existing on |
|
The test CI failure is pre-existing on |
|
CI note: The failing checks ( |
Pass resolved custom provider metadata into the compressor context resolver so auxiliary auto-compression inherits per-model context_length values from config.yaml. Closes NousResearch#21947 Co-authored-by: Cursor <cursoragent@cursor.com>
Keep the new custom_providers passthrough compatible with dynamic test kwargs while preserving runtime behavior. Co-authored-by: Cursor <cursoragent@cursor.com>
Avoid widening ContextCompressor's public initializer; the auxiliary feasibility check is the only path that needs custom provider metadata. Co-authored-by: Cursor <cursoragent@cursor.com>
Keep the custom provider context fix line-stable in run_agent.py so pre-existing ty diagnostics do not appear as new diff warnings. Co-authored-by: Cursor <cursoragent@cursor.com>
e8c12fd to
1593ef5
Compare
|
Rebased onto current main. What I did:
Verified: Final diff: 2 files, +39 / -7 Closes #21947 (custom provider context length feasibility check). |
|
Closing to stay within contributor PR limit. Will resubmit with fresh rebase if the issue remains open in main. |
Problem
Compression models configured with
provider: autoand an empty model inherit the main model name, but the startup feasibility check did not inherit thecustom_providers[].models.<model>.context_lengthmetadata for that model. Users with custom OpenAI-compatible providers could see Hermes auto-lower the compression threshold to the 256K fallback even when config.yaml declared a larger context window.Closes #21947
Root cause
get_model_context_length()already supportscustom_providers, and the main compressor path already resolves custom-provider context intomodel.context_lengthbefore construction. The auxiliary compression feasibility probe, however, only passedauxiliary.compression.context_lengthand did not have access to the resolved custom provider list.Fix
Cache the resolved custom provider metadata on the agent and pass it into
_check_compression_model_feasibility()when resolving the inherited auxiliary compression model's context length.Testing
python3 -m py_compile run_agent.py tests/run_agent/test_compression_feasibility.py: cleanpython3 -m ruff check tests/run_agent/test_compression_feasibility.py --select F: cleanpython3 -m pytest -o addopts='' tests/run_agent/test_compression_feasibility.py -q: 17 passedNote: full-file
ruff --select E,W,Fonrun_agent.pystill reports pre-existing unrelated issues; the repo's CI uses the ruff diff workflow for PRs.