fix(context): honor custom_providers context_length on /model switch + bump probe tier to 256K - #15844
Merged
Conversation
…+ bump probe tier to 256K Fixes #15779. Custom-provider per-model context_length (`custom_providers[].models.<id>.context_length`) is now honored across every resolution path, not just agent startup. Also adds 256K as the top probe tier and default fallback. ## What changed New helper `hermes_cli.config.get_custom_provider_context_length()` — single source of truth for the per-model override lookup, with trailing-slash-insensitive base-url matching. `agent.model_metadata.get_model_context_length()` gains an optional `custom_providers=` kwarg (step 0b — runs after explicit `config_context_length` but before every other probe). Wired through five call sites that previously either duplicated the lookup or ignored it entirely: - `run_agent.py` startup — refactored to use the new helper (dedups legacy inline loop, keeps invalid-value warning) - `AIAgent.switch_model()` — re-reads custom_providers from live config on every /model switch - `hermes_cli.model_switch.resolve_display_context_length()` — new `custom_providers=` kwarg - `gateway/run.py` /model confirmation (picker callback + text path) - `gateway/run.py` `_format_session_info` (/info) ## Context probe tiers `CONTEXT_PROBE_TIERS = [256_000, 128_000, 64_000, 32_000, 16_000, 8_000]` — was `[128_000, ...]`. `DEFAULT_FALLBACK_CONTEXT` follows tier[0], so unknown models now default to 256K. The stale `128000` literal in the OpenRouter metadata-miss path is replaced with `DEFAULT_FALLBACK_CONTEXT` for consistency. ## Repro (from #15779) ```yaml custom_providers: - name: my-custom-endpoint base_url: https://example.invalid/v1 model: gpt-5.5 models: gpt-5.5: context_length: 1050000 ``` `/model gpt-5.5 --provider custom:my-custom-endpoint` → previously "Context: 128,000", now "Context: 1,050,000". ## Tests - `tests/hermes_cli/test_custom_provider_context_length.py` — new file, 19 tests covering the helper, step-0b integration, and the 256K tier invariants - `tests/hermes_cli/test_model_switch_context_display.py` — added regression tests for #15779 through the display resolver - `tests/gateway/test_session_info.py` — updated default-fallback assertion (128K → 256K) - `tests/agent/test_model_metadata.py` — updated tier assertions for the new top tier
Closed
1 task
alt-glitch
added
type/bug
This was referenced Apr 27, 2026
Closed
This was referenced Apr 27, 2026
ksmaze
added a commit
to ksmaze/hermes-agent
that referenced
this pull request
Apr 28, 2026
The _check_compression_model_feasibility call to get_model_context_length was missing the custom_providers= kwarg, so the aux compression model fell back to 256K instead of reading the per-model context_length from custom_providers config. This caused a spurious "Auto-lowered threshold" warning when both main and aux models are on the same custom endpoint. Also removes dead _ctx_kwargs code left over from the merge. Fixes the same gap that cbabf6a addressed via agent_config=, now using the upstream custom_providers= pattern from PR NousResearch#15844.
19 tasks
02356abc
pushed a commit
to 02356abc/hermes-agent
that referenced
this pull request
May 14, 2026
…+ bump probe tier to 256K (NousResearch#15844) Fixes NousResearch#15779. Custom-provider per-model context_length (`custom_providers[].models.<id>.context_length`) is now honored across every resolution path, not just agent startup. Also adds 256K as the top probe tier and default fallback. ## What changed New helper `hermes_cli.config.get_custom_provider_context_length()` — single source of truth for the per-model override lookup, with trailing-slash-insensitive base-url matching. `agent.model_metadata.get_model_context_length()` gains an optional `custom_providers=` kwarg (step 0b — runs after explicit `config_context_length` but before every other probe). Wired through five call sites that previously either duplicated the lookup or ignored it entirely: - `run_agent.py` startup — refactored to use the new helper (dedups legacy inline loop, keeps invalid-value warning) - `AIAgent.switch_model()` — re-reads custom_providers from live config on every /model switch - `hermes_cli.model_switch.resolve_display_context_length()` — new `custom_providers=` kwarg - `gateway/run.py` /model confirmation (picker callback + text path) - `gateway/run.py` `_format_session_info` (/info) ## Context probe tiers `CONTEXT_PROBE_TIERS = [256_000, 128_000, 64_000, 32_000, 16_000, 8_000]` — was `[128_000, ...]`. `DEFAULT_FALLBACK_CONTEXT` follows tier[0], so unknown models now default to 256K. The stale `128000` literal in the OpenRouter metadata-miss path is replaced with `DEFAULT_FALLBACK_CONTEXT` for consistency. ## Repro (from NousResearch#15779) ```yaml custom_providers: - name: my-custom-endpoint base_url: https://example.invalid/v1 model: gpt-5.5 models: gpt-5.5: context_length: 1050000 ``` `/model gpt-5.5 --provider custom:my-custom-endpoint` → previously "Context: 128,000", now "Context: 1,050,000". ## Tests - `tests/hermes_cli/test_custom_provider_context_length.py` — new file, 19 tests covering the helper, step-0b integration, and the 256K tier invariants - `tests/hermes_cli/test_model_switch_context_display.py` — added regression tests for NousResearch#15779 through the display resolver - `tests/gateway/test_session_info.py` — updated default-fallback assertion (128K → 256K) - `tests/agent/test_model_metadata.py` — updated tier assertions for the new top tier
dannyJ848
pushed a commit
to dannyJ848/hermes-agent
that referenced
this pull request
May 17, 2026
…+ bump probe tier to 256K (NousResearch#15844) Fixes NousResearch#15779. Custom-provider per-model context_length (`custom_providers[].models.<id>.context_length`) is now honored across every resolution path, not just agent startup. Also adds 256K as the top probe tier and default fallback. ## What changed New helper `hermes_cli.config.get_custom_provider_context_length()` — single source of truth for the per-model override lookup, with trailing-slash-insensitive base-url matching. `agent.model_metadata.get_model_context_length()` gains an optional `custom_providers=` kwarg (step 0b — runs after explicit `config_context_length` but before every other probe). Wired through five call sites that previously either duplicated the lookup or ignored it entirely: - `run_agent.py` startup — refactored to use the new helper (dedups legacy inline loop, keeps invalid-value warning) - `AIAgent.switch_model()` — re-reads custom_providers from live config on every /model switch - `hermes_cli.model_switch.resolve_display_context_length()` — new `custom_providers=` kwarg - `gateway/run.py` /model confirmation (picker callback + text path) - `gateway/run.py` `_format_session_info` (/info) ## Context probe tiers `CONTEXT_PROBE_TIERS = [256_000, 128_000, 64_000, 32_000, 16_000, 8_000]` — was `[128_000, ...]`. `DEFAULT_FALLBACK_CONTEXT` follows tier[0], so unknown models now default to 256K. The stale `128000` literal in the OpenRouter metadata-miss path is replaced with `DEFAULT_FALLBACK_CONTEXT` for consistency. ## Repro (from NousResearch#15779) ```yaml custom_providers: - name: my-custom-endpoint base_url: https://example.invalid/v1 model: gpt-5.5 models: gpt-5.5: context_length: 1050000 ``` `/model gpt-5.5 --provider custom:my-custom-endpoint` → previously "Context: 128,000", now "Context: 1,050,000". ## Tests - `tests/hermes_cli/test_custom_provider_context_length.py` — new file, 19 tests covering the helper, step-0b integration, and the 256K tier invariants - `tests/hermes_cli/test_model_switch_context_display.py` — added regression tests for NousResearch#15779 through the display resolver - `tests/gateway/test_session_info.py` — updated default-fallback assertion (128K → 256K) - `tests/agent/test_model_metadata.py` — updated tier assertions for the new top tier
gweeteve
pushed a commit
to gweeteve/hermes-agent
that referenced
this pull request
Jun 2, 2026
…+ bump probe tier to 256K (NousResearch#15844) Fixes NousResearch#15779. Custom-provider per-model context_length (`custom_providers[].models.<id>.context_length`) is now honored across every resolution path, not just agent startup. Also adds 256K as the top probe tier and default fallback. ## What changed New helper `hermes_cli.config.get_custom_provider_context_length()` — single source of truth for the per-model override lookup, with trailing-slash-insensitive base-url matching. `agent.model_metadata.get_model_context_length()` gains an optional `custom_providers=` kwarg (step 0b — runs after explicit `config_context_length` but before every other probe). Wired through five call sites that previously either duplicated the lookup or ignored it entirely: - `run_agent.py` startup — refactored to use the new helper (dedups legacy inline loop, keeps invalid-value warning) - `AIAgent.switch_model()` — re-reads custom_providers from live config on every /model switch - `hermes_cli.model_switch.resolve_display_context_length()` — new `custom_providers=` kwarg - `gateway/run.py` /model confirmation (picker callback + text path) - `gateway/run.py` `_format_session_info` (/info) ## Context probe tiers `CONTEXT_PROBE_TIERS = [256_000, 128_000, 64_000, 32_000, 16_000, 8_000]` — was `[128_000, ...]`. `DEFAULT_FALLBACK_CONTEXT` follows tier[0], so unknown models now default to 256K. The stale `128000` literal in the OpenRouter metadata-miss path is replaced with `DEFAULT_FALLBACK_CONTEXT` for consistency. ## Repro (from NousResearch#15779) ```yaml custom_providers: - name: my-custom-endpoint base_url: https://example.invalid/v1 model: gpt-5.5 models: gpt-5.5: context_length: 1050000 ``` `/model gpt-5.5 --provider custom:my-custom-endpoint` → previously "Context: 128,000", now "Context: 1,050,000". ## Tests - `tests/hermes_cli/test_custom_provider_context_length.py` — new file, 19 tests covering the helper, step-0b integration, and the 256K tier invariants - `tests/hermes_cli/test_model_switch_context_display.py` — added regression tests for NousResearch#15779 through the display resolver - `tests/gateway/test_session_info.py` — updated default-fallback assertion (128K → 256K) - `tests/agent/test_model_metadata.py` — updated tier assertions for the new top tier
T02200059
pushed a commit
to T02200059/hermes-agent
that referenced
this pull request
Jun 18, 2026
Mark a0d0068ec (probe fall-through + Step 7 list-only) and 4a60dd1ce6 (P41 complete: Step 7 dict format) as deprecated. Reason: P41 functionality is already provided in owner-v16 via the custom_providers mechanism (hermes_cli.config.get_custom_provider_context_length) called from agent/model_metadata.py Step 0b (before all probes). This came in through upstream PR NousResearch#15844 / commit bd5de5c. The probe fall-through behavior is also already in owner-v16 via Step 3b (consult DEFAULT_CONTEXT_LENGTHS catalog before 256K fallback), commit 43a2329. Note on 4a60dd1ce6: this commit is NOT in our-commits-inventory.md (generator skipped it; inventory has 339 lines but owner HEAD has 381 yangtb+tianbao.yang commits). Manually appended a row to preserve the deprecation decision. Inventory line number '199' collides with the existing row below; will be resolved by regenerating inventory later.
T02200059
pushed a commit
to T02200059/hermes-agent
that referenced
this pull request
Jun 18, 2026
Mark c7e5aaaee (probe fall-through + Step 7 list-only) and 4a60dd1ce6 (P41 complete: Step 7 dict format) as deprecated. Reason: P41 functionality is already provided in owner-v16 via the custom_providers mechanism (hermes_cli.config.get_custom_provider_context_length) called from agent/model_metadata.py Step 0b (before all probes). This came in through upstream PR NousResearch#15844 / commit 125de02. The probe fall-through behavior is also already in owner-v16 via Step 3b (consult DEFAULT_CONTEXT_LENGTHS catalog before 256K fallback), commit 2e61de0. Note on 4a60dd1ce6: this commit is NOT in our-commits-inventory.md (generator skipped it; inventory has 339 lines but owner HEAD has 381 yangtb+tianbao.yang commits). Manually appended a row to preserve the deprecation decision. Inventory line number '199' collides with the existing row below; will be resolved by regenerating inventory later.
waefrebeorn
pushed a commit
to waefrebeorn/slermes
that referenced
this pull request
Jul 2, 2026
…+ bump probe tier to 256K (NousResearch#15844) Fixes NousResearch#15779. Custom-provider per-model context_length (`custom_providers[].models.<id>.context_length`) is now honored across every resolution path, not just agent startup. Also adds 256K as the top probe tier and default fallback. ## What changed New helper `hermes_cli.config.get_custom_provider_context_length()` — single source of truth for the per-model override lookup, with trailing-slash-insensitive base-url matching. `agent.model_metadata.get_model_context_length()` gains an optional `custom_providers=` kwarg (step 0b — runs after explicit `config_context_length` but before every other probe). Wired through five call sites that previously either duplicated the lookup or ignored it entirely: - `run_agent.py` startup — refactored to use the new helper (dedups legacy inline loop, keeps invalid-value warning) - `AIAgent.switch_model()` — re-reads custom_providers from live config on every /model switch - `hermes_cli.model_switch.resolve_display_context_length()` — new `custom_providers=` kwarg - `gateway/run.py` /model confirmation (picker callback + text path) - `gateway/run.py` `_format_session_info` (/info) ## Context probe tiers `CONTEXT_PROBE_TIERS = [256_000, 128_000, 64_000, 32_000, 16_000, 8_000]` — was `[128_000, ...]`. `DEFAULT_FALLBACK_CONTEXT` follows tier[0], so unknown models now default to 256K. The stale `128000` literal in the OpenRouter metadata-miss path is replaced with `DEFAULT_FALLBACK_CONTEXT` for consistency. ## Repro (from NousResearch#15779) ```yaml custom_providers: - name: my-custom-endpoint base_url: https://example.invalid/v1 model: gpt-5.5 models: gpt-5.5: context_length: 1050000 ``` `/model gpt-5.5 --provider custom:my-custom-endpoint` → previously "Context: 128,000", now "Context: 1,050,000". ## Tests - `tests/hermes_cli/test_custom_provider_context_length.py` — new file, 19 tests covering the helper, step-0b integration, and the 256K tier invariants - `tests/hermes_cli/test_model_switch_context_display.py` — added regression tests for NousResearch#15779 through the display resolver - `tests/gateway/test_session_info.py` — updated default-fallback assertion (128K → 256K) - `tests/agent/test_model_metadata.py` — updated tier assertions for the new top tier
12 tasks
TurgutKural
added a commit
to TurgutKural/hermes-agent
that referenced
this pull request
Aug 19, 2026
…n call sites custom_providers[].models.<id>.context_length overrides were only honored at agent startup (agent_init) and /model switch (model_switch). Several sibling call paths called get_model_context_length() without passing custom_providers, causing them to fall through to endpoint probing and the 256K/131K hardcoded defaults — even when the user had an explicit per-model override configured. Affected call sites: - ContextCompressor._resolve_context_length (deferred first-access probe) - auxiliary_client._candidate_context_window (fallback chain screening) - moa_loop._trim_messages_for_reference (MoA reference model trimming) - web_server.get_model_info (WebUI model info endpoint) Upstream added the custom_providers parameter to get_model_context_length (f981d47, NousResearch#15844); this PR threads it through the remaining call sites, which upstream has not covered yet (verified against current upstream/main: ContextCompressor construction, _candidate_context_window, moa_loop, and web_server.get_model_info all still omit it). Tests: 20 tests covering all four call sites, precedence rules, graceful degradation, and extended helper coverage.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
/model <model> --provider custom:<name>now reports the context window the user configured incustom_providers[].models.<id>.context_lengthinstead of falling back to 128K. Closes #15779. Also bumps the top context-probe tier (and default fallback) from 128K to 256K.Root cause: the per-model override was read only in
AIAgent.__init__.switch_model(),resolve_display_context_length(), and_format_session_info()all ignored it and fell through to the probe-down fallback.Changes
hermes_cli/config.py: newget_custom_provider_context_length()helper — single source of truth for the per-model override lookup, trailing-slash-insensitiveagent/model_metadata.py:get_model_context_length()gainscustom_providers=kwarg (step 0b — after explicitconfig_context_length, before every probe);CONTEXT_PROBE_TIERSprefixed with 256K soDEFAULT_FALLBACK_CONTEXT= 256K; stale128000literal in OR metadata-miss path replaced withDEFAULT_FALLBACK_CONTEXTrun_agent.py: startup path refactored to use the helper (dedups inline loop, preserves invalid-value warning);AIAgent.switch_model()re-readscustom_providersfrom live config so mid-session switches resolve the overridehermes_cli/model_switch.py:resolve_display_context_length()accepts + forwardscustom_providersgateway/run.py:/modelconfirmation (picker callback + text path) and_format_session_infothreadcustom_providersthroughValidation
/model gpt-5.5 --provider custom:my-endpointwithcontext_length: 1050000CONTEXT_PROBE_TIERS[0]Targeted tests:
tests/agent/test_model_metadata.py tests/hermes_cli/test_custom_provider_context_length.py tests/hermes_cli/test_model_switch_context_display.py tests/hermes_cli/test_model_switch_custom_providers.py tests/hermes_cli/test_custom_provider_model_switch.py tests/run_agent/test_invalid_context_length_warning.py tests/run_agent/test_switch_model_context.py tests/agent/test_model_metadata_local_ctx.py tests/gateway/test_session_info.py— 171/171 pass. Broadertests/gateway/ tests/run_agent/delta: 47 failing on origin/main → 46 failing on branch (same pre-existing failures, none new).E2E via
execute_code: exact #15779 repro config + helper edge cases (trailing slash, wrong url/model, invalid value types, zero/negative,config_context_lengthprecedence) all green.New tests
tests/hermes_cli/test_custom_provider_context_length.py— 19 tests: helper unit + step 0b integration +CONTEXT_PROBE_TIERSinvariantstests/hermes_cli/test_model_switch_context_display.py— added regression tests for Bug: /model switch to named custom provider ignores custom_providers model context_length #15779 through the display resolverCloses #15779.