fix(context): thread custom_providers to all context-length resolution call sites - #75738
Conversation
|
Thanks for tracing the missing propagation paths. The core premise is verified on current main: Problems
Suggested changes
This is an automated hermes-sweeper review. |
e5e1cc3 to
0298230
Compare
|
Thanks — both suggested changes are now implemented in
Regression coverage added in
Local validation on the rebased worktree: |
|
Coordination from the web_server.py god-file decomposition (epic #78647): the custom-endpoints family has been extracted into web_routers/custom_endpoints.py in #79127 (slice R3-C1). Verified LOW overlap with this PR — adjacent regions, disjoint code. Both compose; no conflict expected. Interlocked: #79127 and the full slice wave (#79123-#79129). |
ebc7caa to
5a8d5d5
Compare
f6be6c6 to
984166d
Compare
984166d to
ff77f14
Compare
7e4e674 to
c6b00a8
Compare
…n call sites custom_providers[].models.<id>.context_length overrides were only honored at agent startup (agent_init) and /model switch (model_switch). Several sibling call paths called get_model_context_length() without passing custom_providers, causing them to fall through to endpoint probing and the 256K/131K hardcoded defaults — even when the user had an explicit per-model override configured. Affected call sites: - ContextCompressor._resolve_context_length (deferred first-access probe) - auxiliary_client._candidate_context_window (fallback chain screening) - moa_loop._trim_messages_for_reference (MoA reference model trimming) - web_server.get_model_info (WebUI model info endpoint) Upstream added the custom_providers parameter to get_model_context_length (f981d47, NousResearch#15844); this PR threads it through the remaining call sites, which upstream has not covered yet (verified against current upstream/main: ContextCompressor construction, _candidate_context_window, moa_loop, and web_server.get_model_info all still omit it). Tests: 20 tests covering all four call sites, precedence rules, graceful degradation, and extended helper coverage.
c6b00a8 to
a06affd
Compare
|
Rebased onto current upstream/main (395c70d). The three red slices on the previous head (2/12, 6/12: test_session_api.py; 12/12: test_goal_continuation_drain.py + test_goal_max_turns_config.py + test_transcription_tools.py) were the Aug-18 gateway timing races and a transcription idle-timeout flake, de-flaked upstream in a75d1b5 / c70e152 / 5633764 — none reproduce on the rebased head; all 12 test slices and all required checks are green. The two sweeper review items remain implemented from the previous cycle: |
Problem
custom_providers[].models.<id>.context_lengthoverrides were only honored at agent startup (agent_init) and/modelswitch (model_switch). Several sibling call paths calledget_model_context_length()without passingcustom_providers, causing them to fall through to endpoint probing and the 256K/131K hardcoded defaults — even when the user had an explicit per-model override configured.Observed symptom (log spam on every session start with a custom provider model):
This fires twice per startup because both the built-in compressor and the context engine plugin resolve context length independently.
Affected call sites
ContextCompressor._resolve_context_lengthagent/context_compressor.py_candidate_context_windowagent/auxiliary_client.py_trim_messages_for_referenceagent/moa_loop.pyget_model_infohermes_cli/web_server.pyFix
Thread
custom_providersthrough each path:custom_providersconstructor parameter, passed fromagent_init(where_custom_providersis already resolved). Used in_resolve_context_length()._candidate_context_windowlazily loadscustom_providersfrom config viaload_config_readonly()+get_compatible_custom_providers()(matching the existing lazy-config pattern in the file)._load_custom_providers()helper with the same lazy pattern, called from_trim_messages_for_reference.get_model_infoextractscustom_providersfrom the config it already loads.All changes are backward-compatible:
custom_providersdefaults toNoneand every config load is wrapped intry/exceptso failures fall through to the existing probe chain.Tests
20 new tests in
tests/agent/test_custom_provider_context_threading.py:All 90 existing related tests pass without modification:
tests/hermes_cli/test_custom_provider_context_length.py(6)tests/agent/test_model_metadata.py(42)tests/agent/test_moa_context_max_tokens.py(2)tests/run_agent/test_switch_model_context.py(6)tests/run_agent/test_invalid_context_length_warning.py(3)tests/hermes_cli/test_model_switch_context_display.py(3)Closes the sibling-path gap left by #15779.