fix: use custom provider context length for compression model - #15210
fix: use custom provider context length for compression model#15210Hubedge wants to merge 1 commit into
Conversation
|
Thanks for flagging the relationship with #13052. This PR is complementary, not competing:
Even after #13052 merges, the compression feasibility check remains a blind spot for custom endpoints whose model IDs do not match Merge dependency: none. If #13052 lands first, I'm happy to rebase this to avoid trivial conflicts in Ready for review. |
|
Thanks for the focused report and regression test. This is an automated hermes-sweeper review. The requested behavior is already implemented on current main:
This also matches the later linked reports and competing implementation discussion around #19539 and #19564. |
Summary
context_lengthfor the auxiliary compression model during feasibility checks.get_model_context_length(..., config_context_length=...)so OpenAI-compatible custom endpoints that omit context metadata do not fall back to 128K when the config declares a larger window.custom_providers[].models[<model>].context_lengthwith noauxiliary.compression.context_lengthoverride.Tests
./venv/bin/python -m pytest tests/run_agent/test_compression_feasibility.py -q -o 'addopts='→ 17 passed./venv/bin/python -m py_compile run_agent.py tests/run_agent/test_compression_feasibility.py && git diff --check→ passedmain:tests/hermes_cli/test_custom_provider_model_switch.py::TestCustomProviderModelSwitch::test_saved_model_still_probes_endpointfails because the test expects noapi_modekwarg while current code callsfetch_api_models(..., api_mode=None). This failure is unrelated to this PR’s changed files.