Skip to content

fix(runtime): honour HERMES_LLM_BASE_URL in pool-based and explicit runtime resolution - #34044

Closed
OmarB97 wants to merge 2 commits into
NousResearch:mainfrom
OmarB97:fix/runtime-provider-llm-base-url
Closed

fix(runtime): honour HERMES_LLM_BASE_URL in pool-based and explicit runtime resolution#34044
OmarB97 wants to merge 2 commits into
NousResearch:mainfrom
OmarB97:fix/runtime-provider-llm-base-url

Conversation

@OmarB97

@OmarB97 OmarB97 commented May 28, 2026

Copy link
Copy Markdown
Contributor

What does this PR do?

Two runtime-resolution paths ignore the HERMES_LLM_BASE_URL environment variable. When an operator points Hermes at a local proxy via that variable (e.g. a loopback proxy that captures/streams inference traffic), inference silently bypasses the proxy and goes straight to the provider's default upstream:

  • _resolve_runtime_from_pool_entry() — the pool-based path used for mid-session /model switches — never consulted HERMES_LLM_BASE_URL.
  • _resolve_explicit_runtime() — honoured a provider's own base_url_env_var but dropped HERMES_LLM_BASE_URL when a provider defined no such variable.

Other resolution paths already honour HERMES_LLM_BASE_URL; this brings the two remaining paths in line so the override is respected consistently. An explicit base_url (from config or the pool entry) still takes precedence.

Related Issue

N/A. Extracted as a focused, single-file change from #34023 at a reviewer's request — that PR shared a stacked branch and showed unrelated changes against main.

Type of Change

  • 🐛 Bug fix (non-breaking change that fixes an issue)

Changes Made

  • hermes_cli/runtime_provider.py
    • _resolve_runtime_from_pool_entry(): when HERMES_LLM_BASE_URL is set, use it as base_url.
    • _resolve_explicit_runtime(): fall back to HERMES_LLM_BASE_URL when the provider defines no base_url_env_var.

(+15 lines, 1 file.)

How to Test

# 1) Unset -> provider default
env -u HERMES_LLM_BASE_URL python -c "from hermes_cli.runtime_provider import resolve_runtime_provider as R; print(R(requested='opencode-zen')['base_url'])"
# -> https://opencode.ai/zen/v1

# 2) Set -> override honoured
HERMES_LLM_BASE_URL=http://127.0.0.1:55176 python -c "from hermes_cli.runtime_provider import resolve_runtime_provider as R; print(R(requested='opencode-zen')['base_url'])"
# -> http://127.0.0.1:55176

Both verified locally on this branch. An explicit per-model / pool-entry base_url still wins over the env var.

Checklist

Code

  • My commit messages follow Conventional Commits
  • I searched for existing PRs to make sure this isn't a duplicate
  • My PR contains only changes related to this fix (1 file, +15 lines)
  • I've run pytest tests/ -q and all tests pass
  • I've added tests for my changes

Omar Baradei added 2 commits May 28, 2026 11:26
The kimi worker added the fallback to hermes_cli/auth.py:resolve_api_key_provider_credentials()
but the caller at runtime_provider.py:1163 already set base_url = pconfig.inference_base_url
before creds were consulted — so the creds base_url was never used for non-kimi providers.

Add the same HERMES_LLM_BASE_URL fallback at runtime_provider.py:1154 so
env_url carries the proxy URL through to line 1163's base_url resolution.

Refs meshboard task: stream-tap-runtime-provider-env-override
…tion

The previous fix (auth.py + runtime_provider.py _resolve_explicit_runtime path)
missed the pool-based code path in _resolve_runtime_from_pool_entry().  For
providers like opencode-zen that resolve via the credential pool, the base_url
was set from the pool entry's default without checking HERMES_LLM_BASE_URL.

Add the override at line 306 (right after base_url is read from the pool entry)
so the MeshBoard stream-tap proxy URL is honoured regardless of which resolution
path is taken: explicit, named-custom, or pool.

This is the actual code path exercised by MeshBoard dispatches, which use
resolve_runtime_provider(requested='opencode-zen') → pool → _resolve_runtime_from_pool_entry.

Refs meshboard task: stream-tap-runtime-provider-env-override
@alt-glitch alt-glitch added type/bug Something isn't working P3 Low — cosmetic, nice to have comp/cli CLI entry point, hermes_cli/, setup wizard labels May 28, 2026
@OmarB97

OmarB97 commented May 31, 2026

Copy link
Copy Markdown
Contributor Author

Closing this older HERMES_LLM_BASE_URL runtime-provider PR in favor of the now-narrowed canonical PR #34332. #34332 is rebased to current main, one commit/two files, has the focused regression tests, and keeps the profile-override work separate in #35076.

@OmarB97 OmarB97 closed this May 31, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

comp/cli CLI entry point, hermes_cli/, setup wizard P3 Low — cosmetic, nice to have type/bug Something isn't working

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants