fix: honour HERMES_LLM_BASE_URL in runtime_provider.py api_key resolution path - #34023
Closed
OmarB97 wants to merge 19 commits into
Closed
fix: honour HERMES_LLM_BASE_URL in runtime_provider.py api_key resolution path#34023OmarB97 wants to merge 19 commits into
OmarB97 wants to merge 19 commits into
Conversation
Pre-filter fallback_providers when the agent starts up — entries whose provider has no credentials (no API key in env, no custom_providers definition, no base_url with env var set) are dropped silently from the chain and a startup diagnostic lists them with a hint naming the env var that would activate them. Before: runtime fallback loop discovered each failure one-by-one via 401/402/429 errors → wall of red logs, multiple seconds of latency, confusing UX (the #1 source of "why does my fallback keep 401ing"). After: chain is clean from the first failed API call; startup prints e.g. "Fallback chain: skipped 4 unconfigured: deepseek, minimax, zai, kimi-moonshot" with per-entry hints like "set DEEPSEEK_API_KEY in ~/.hermes/.env" or "`hermes auth add nous`".
- Add resolve_auto_skin() with 6-step terminal detection chain: HERMES_LIGHT/TUI_THEME/TUI_BACKGROUND env vars, COLORFGBG, OSC 11 query, default dark - Add _SKIN_VARIANT_PAIRS mapping dark/light skin pairs - Update load_skin(), set_active_skin(), init_skin_from_config() to handle 'auto' keyword - Update list_skins() to show 'auto' as virtual entry - Update /skin command in cli.py to display resolved auto skin - Works across all machines: dark terminal → dark variant, light terminal → light variant, no per-machine config needed Resolves: auto skin resolution for multi-device Hermes deployments
When config files (e.g. ~/.hermes/config.yaml) are symlinked to a different filesystem — common on WSL where dotfiles live under /mnt/c/ — os.replace() fails with EXDEV (errno 18: Invalid cross-device link). Fall back to shutil.copy2 + os.unlink when EXDEV is raised. copy2 preserves metadata (permissions, timestamps) so the result is equivalent. Non-EXDEV errors (EACCES, etc.) still propagate. This affects all atomic write paths (atomic_yaml_write, atomic_json_write, atomic_roundtrip_yaml_update) since they all route through atomic_replace. Added 4 regression tests: - Direct EXDEV fallback on plain file - EXDEV fallback preserving symlinks - Non-EXDEV errors are not swallowed - End-to-end atomic_yaml_write EXDEV recovery
…el rejected
When the user's main chat model is forwarded verbatim to a provider that
only supports a subset of its catalog for auxiliary (cheap / fast) tasks,
the operator currently sees a confusing HTTP 401 or HTTP 404 against a
model that clearly works for chat. Example: mimo-v2.5 on opencode-zen
returns HTTP 401: Model mimo-v2.5 is not supported for aux tasks,
while chat succeeds.
Root cause: call_llm() raises the error instead of trying the
provider's registered default aux model. The existing fallback chain
(payment / connection / rate-limit) correctly bypasses this case because
model-unsupported errors are neither quota-nor-connectivity-related.
Add is_model_unsupported_error() (agent/aux_unsupported_model.py) to
distinguish model rejection from credential / quota / connectivity
failures:
* 404 with a 'model' + 'not found/does not exist/unknown' marker
* 401 with explicit 'not supported / unsupported model' wording
(aggregators sometimes classify unsupported models as 401 instead of 404)
* 400/422 with 'not supported / unsupported model' wording
* Pure credential failures (401 without model wording) return False
so the existing auth-refresh chain still handles them.
Extend call_llm() with a 'Model-unsupported fallback' block inserted
between the existing payment/connection fallback and the teardown:
* For explicit (non-auto) providers: resolve the provider's default aux
model (ProviderProfile.default_aux_model or legacy fallback dict) and
retry the same request against it.
* Auto-users are skipped because their rejected model IS the aux model
— retrying would be a no-op.
* If the rejected model already equals the provider's default, no retry.
* If the retry also fails, re-raise the original error.
Operator impact: users on opencode-zen / Nous Portal / any provider whose
aux catalog differs from its chat catalog now get transparent recovery
for title generation, compression, session search, etc. — they no longer
need to manually pin auxiliary.<task>.provider/model in config.yaml.
When a user types `/model mimo-v2.5` on opencode-zen (or any aggregator),
the model switch pipeline falls through to `detect_provider_for_model()`
which finds the model in xiaomi's static catalog and switches providers —
requiring an API key the user doesn't have. The user gets a 401 auth error
instead of staying on their configured aggregator.
Root cause chain:
1. `/model mimo-v2.5` triggers `switch_model()` step d (aggregator catalog search)
2. `list_provider_models('opencode')` queries models.dev cache which has
`mimo-v2.5-free` (stale/free-tier slug) — no match for bare `mimo-v2.5`
3. Step d fails → `resolved_in_current_catalog` stays False
4. Step e `detect_provider_for_model('mimo-v2.5', 'opencode')` finds it in
xiaomi static catalog
5. Provider switches to xiaomi → needs `XIAOMI_API_KEY` → 401 → fallback
chain → abort
Changes:
- Aggregator catalog fallback (step d): After models.dev catalog search
fails to match, added two additional checks:
1. Check `custom_providers` model dicts from config.yaml — if the model
is listed there, it's authoritative for the user's configured aggregator
2. Query the live `/v1/models` endpoint via `fetch_endpoint_model_metadata()`
— the live API returns current model IDs (e.g. `mimo-v2.5`) not stale
models.dev slugs (e.g. `mimo-v2.5-free`)
- Validation override matching: Fixed the `custom_providers` override in
the validation rejection handler to also match `entry_name == target_provider`
and `entry_name == normalized_target` (e.g. both `opencode-zen` and
`opencode`), not just `custom:<name>`. Built-in provider aliases that
also appear in `custom_providers` were invisible to the old check.
- Import: Added `normalize_provider` to the imports from `hermes_cli.providers`.
Tests: 36 passed (1 pre-existing fixture failure unrelated to change).
…ast-fail
When Nous Portal returns HTTP 503 with 'upstream capacity limits', the
error classifier now marks it retryable=False, should_fallback=True
instead of retryable=True. This tells the conversation loop to
immediately fall back to the next provider in the chain (e.g., kimi-k2.6
via opencode-zen) instead of burning 2+ minutes on exponential backoff
retries that will all fail with the same capacity error.
Also added eager-fallback logic in conversation_loop.py (lines 2550-2563)
that explicitly checks for overload errors and triggers fallback,
mirroring the existing rate-limit eager-fallback path.
Changes:
- agent/error_classifier.py: classify 'upstream capacity limits' in
503/529 errors as non-retryable with should_fallback=True
- agent/conversation_loop.py: add eager fallback check for overload
errors (lines 2550-2563)
- tests/agent/test_error_classifier.py: 3 new tests validating the
capacity-pattern detection (153 total, all passing)
MeshBoard task: hermes-503-capacity-fast-fail
Nous Portal rejects requests with extra_body.tags from non-Portal aux clients (title generation, compression, vision). The tags field is not accepted by all auxiliary endpoints that the aux adapter falls back to. Returning empty list from nous_portal_tags() prevents the 400 errors while keeping the function signature intact for future conditional re-enablement. Fixes: 'Extra inputs are not permitted, field: tags' on aux title generation
When the agent has active tools, skip fallback providers marked with
supports_tools: false. This prevents silent failures when falling back
to local models or providers that reject tool schemas (e.g. kimi-k2.6
via opencode-zen returning 400, llama.cpp models that 'think about'
tools but never produce actual tool_calls).
Usage in config.yaml:
fallback_providers:
- provider: opencode-zen
model: kimi-k2.6
supports_tools: false
- provider: local-llama-swap
model: qwen3.6-27b
supports_tools: false
Fixes cascading agent death: Nous 503 → fallback to non-tool model
→ 400 or tool-less response → session hangs.
feat: filter fallback chain by tool capability (supports_tools flag)
The module-level `auxiliary_is_nous` flag is only reset inside _resolve_auto(), which is never called when a task has an explicit provider configured (e.g. auxiliary.title_generation.provider: opencode-zen). A stale True value from a previous call (e.g. compression via auto→Nous) leaks Nous Portal tags into requests for non-Nous providers, causing HTTP 400 'Extra inputs are not permitted, field: tags'. Reset the flag in both call_llm() and async_call_llm() when the resolved provider is not 'auto', so it accurately reflects whether the current call targets Nous Portal.
fix: reset auxiliary_is_nous for non-auto provider resolution paths
Add real-time macOS/Linux/Windows appearance change detection to the skin engine. When display.skin is set to 'auto', a background daemon thread polls the OS appearance every 2 seconds. When a change is detected (light->dark or dark->light), the skin is automatically switched without requiring user action or session restart. Changes: - Add macOS AppleInterfaceStyle detection to _detect_terminal_is_light() - Add AppearanceWatcher class with cross-platform polling (macOS/Linux/Windows) - Start watcher from TUI gateway entry point (emits skin.changed event) - Start watcher from CLI main loop (re-initializes skin) - Add get_current_os_appearance() standalone helper - Add 8 new tests (41 total pass)
feat: auto-detect OS appearance changes and switch skins mid-session
…oxes * fix: add tests for aggregator fallback + fix custom: prefix matching Address PR NousResearch#33716 review feedback: 1. Add tests for the new aggregator fallback code paths: - custom_providers model dict iteration resolves models not in models.dev - live fetch_endpoint_model_metadata fallback resolves missing models - live API exceptions are silently swallowed (except Exception: pass) - canonical model casing from live endpoint is preserved 2. Fix custom: prefix slug matching: - target_provider='Custom:OpenRouter' now matches entry_name='OpenRouter' - Added split(':',1)[-1].lower() suffix comparison as 5th condition - Normalise always lowercases so 'custom:openrouter' won't match 'OpenRouter' by name alone; the suffix strip handles arbitrary casing. Closes: NousResearch#33716 review feedback (liuhao1024) * fix: add HERMES_SKIP_PROFILE_OVERRIDE escape hatch for launcher sandboxes MeshBoard's stream-tap launcher creates a per-dispatch HERMES_HOME sandbox whose .env points at a local loopback proxy. Hermes' _apply_profile_override() was reading ~/.hermes/active_profile and clobbering the sandbox path, causing every dispatch to bypass the tap. Add an early return when HERMES_SKIP_PROFILE_OVERRIDE=1 is set in the environment. The MeshBoard launcher will set this alongside HERMES_HOME so the sandbox is honoured verbatim. Refs meshboard task: hermes-stream-tap-profile-override --------- Co-authored-by: Omar B <omar@kostudios.io>
…up (#19) Two systemic bugs in the fallback provider chain: 1. nous provider was misclassified as _AGGREGATOR_PROVIDERS, causing _prepend_vendor() to keep the nousresearch/ prefix when falling back to the Nous direct API (inference-api.nousresearch.com). The direct API expects bare model names like hermes-4-70b, not nousresearch/hermes-4-70b, resulting in HTTP 400 'Bad Request'. Fix: Move nous from _AGGREGATOR_PROVIDERS to _MATCHING_PREFIX_STRIP_PROVIDERS and add 'nousresearch' as a provider alias for 'nous' so _strip_matching_provider_prefix() correctly strips the nousresearch/ prefix. 2. _try_activate_fallback() only deduped entries matching the exact (provider, model) pair. When the primary provider (e.g. opencode-zen) was rate-limited, all fallback entries on the same provider would also hit the same 429, burning the retry budget pointlessly. Fix: When the trigger is a rate-limit (429) or billing error and an active cooldown is set, skip fallback entries on the same provider. Different providers are still tried normally. Files changed: - hermes_cli/model_normalize.py: Move nous from aggregator to matching-prefix-strip - hermes_cli/models.py: Add nousresearch → nous alias - agent/chat_completion_helpers.py: Add same-provider rate-limit dedup - tests/hermes_cli/test_model_normalize.py: Update nous normalization tests - tests/run_agent/test_provider_fallback.py: Add same-provider cooldown test Refs: fallback-chain-nous-normalization-same-provider-rate Co-authored-by: Omar Baradei <omar@kostudios.io>
…rl_env_var is absent MeshBoard's stream-tap launcher creates a per-dispatch sandbox whose .env sets HERMES_LLM_BASE_URL to a local proxy URL. Hermes was ignoring this because resolve_api_key_provider_credentials() only checked the provider's own base_url_env_var (e.g. OPENCODE_ZEN_BASE_URL), not the generic HERMES_LLM_BASE_URL fallback. When the provider-specific env var is unset, fall back to HERMES_LLM_BASE_URL so the stream-tap proxy is actually used. Refs meshboard task: stream-tap-empty-jsonl-diagnostic
The kimi worker added the fallback to hermes_cli/auth.py:resolve_api_key_provider_credentials() but the caller at runtime_provider.py:1163 already set base_url = pconfig.inference_base_url before creds were consulted — so the creds base_url was never used for non-kimi providers. Add the same HERMES_LLM_BASE_URL fallback at runtime_provider.py:1154 so env_url carries the proxy URL through to line 1163's base_url resolution. Refs meshboard task: stream-tap-runtime-provider-env-override
…tion The previous fix (auth.py + runtime_provider.py _resolve_explicit_runtime path) missed the pool-based code path in _resolve_runtime_from_pool_entry(). For providers like opencode-zen that resolve via the credential pool, the base_url was set from the pool entry's default without checking HERMES_LLM_BASE_URL. Add the override at line 306 (right after base_url is read from the pool entry) so the MeshBoard stream-tap proxy URL is honoured regardless of which resolution path is taken: explicit, named-custom, or pool. This is the actual code path exercised by MeshBoard dispatches, which use resolve_runtime_provider(requested='opencode-zen') → pool → _resolve_runtime_from_pool_entry. Refs meshboard task: stream-tap-runtime-provider-env-override
alt-glitch
added
type/bug
Collaborator
Contributor
Author
|
Per @alt-glitch's review, I've split the unique |
Closed
6 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What changed
Adds a
HERMES_LLM_BASE_URLenv var fallback inhermes_cli/runtime_provider.pyat theenv_urlresolution step (line 1155). When the provider-specific env var (e.g.OPENCODE_ZEN_BASE_URL) is unset, the code now reads the genericHERMES_LLM_BASE_URLbefore falling back to the hardcodedpconfig.inference_base_url.Why
MeshBoard-style launcher integrations create a per-dispatch sandbox whose
.envsetsHERMES_LLM_BASE_URLto a local loopback proxy URL (e.g.http://127.0.0.1:55176). Hermes was ignoring it because:HERMES_LLM_BASE_URLin the subprocess env and sandbox.envresolve_runtime_provider()never readsHERMES_LLM_BASE_URL— it only checks provider-specific env vars (e.g.OPENCODE_ZEN_BASE_URL) and then falls back to the hardcoded default URLThe existing fix in
hermes_cli/auth.py:resolve_api_key_provider_credentials()(commit a73ee55) added the fallback there — but the caller at runtime_provider.py line 1163 setsbase_url = env_url or pconfig.inference_base_urlBEFORE creds are consulted. For non-kimi providers, the credsbase_urlat line 1169 is skipped because base_url is already set.How to Review
api_keyauth_type providersexplicit_base_url(user-passed) still wins over the env var (it does — env_url is only consulted whenexplicit_base_urlis None at line 1157)Evidence
Verification
python3 -m pytest tests/hermes_cli/test_runtime_provider*.py -x --tb=shortExpected: all existing resolution tests pass (env var only activates when provider-specific var is absent).
Risks / Gaps
HERMES_SKIP_PROFILE_OVERRIDE=1to prevent_apply_profile_override()from clobbering sandboxHERMES_HOME— accepted, no follow-up needed: addressed by PR fix: add HERMES_SKIP_PROFILE_OVERRIDE escape hatch for launcher sandboxes #33986resolve_runtime_provider()paths for pool-based and custom providers don't benefit from this env var fallback — accepted, no follow-up needed: those paths already read env vars from their respectivebase_url_env_varfieldsCollaborators
Participants: Omar B (fork fix)
Process: Hermes CLI agent launch with overridden base URL
Task context: MeshBoard stream-tap integration; launcher sets HERMES_LLM_BASE_URL for proxy routing
Related work: PR #33986 (HERMES_SKIP_PROFILE_OVERRIDE), fork commit a73ee55 (auth.py fallback)