fix(config): resolve model.key_env in bare custom provider path (#67453) - #67554
Closed
JonthanaHanh wants to merge 1 commit into
Closed
JonthanaHanh wants to merge 1 commit into
JonthanaHanh wants to merge 1 commit into
Conversation
…Research#67453) The bare + path in resolve_runtime_provider() did not read key_env from the model config. It only checked explicit_api_key, OPENAI_API_KEY, OPENROUTER_API_KEY, and host-derived env vars. When a user configures: model: provider: custom key_env: SCW_LLM_API_KEY base_url: https://api.scaleway.ai/.../v1 The key was resolved via _host_derived_api_key on the first call (which may have found it through a different path), but subsequent sessions failed with 403 because the key_env was never consulted directly. Add model.key_env resolution to the api_key_candidates list so the key is re-resolved from the env var on every request. Fixes NousResearch#67453
alt-glitch
added
type/bug
teknium1
reviewed
Jul 19, 2026
teknium1
left a comment
teknium1
left a comment
Collaborator
There was a problem hiding this comment.
Thanks for tracing the missing key_env lookup. The current change does not reach the API-server path reported in #67453.
Problems
- The edited branch requires
explicit_base_url(hermes_cli/runtime_provider.py:940). API-server creation resolves credentials through_resolve_runtime_agent_kwargs()(gateway/platforms/api_server.py:1746), which callsresolve_runtime_provider()with no explicit arguments (gateway/run.py:2008). That path reaches_resolve_openrouter_runtime(), whose candidates athermes_cli/runtime_provider.py:1139-1149still omitmodel.key_env. - The PR changes no tests. A regression should cover plain
model.provider: custom+model.base_url+model.key_env; the linked issue's two-request API-server sequence also needs a targeted repro before claiming the repeat-request failure is fixed.
Suggested changes
- Apply the lookup in the configured-custom resolution path (or factor the candidate construction so both paths use it), then add the corresponding regression coverage.
Automated hermes-sweeper review.
| # on every request — not just the first session after boot (#67453). | ||
| _model_cfg_key_env = str( | ||
| (_get_model_config().get("key_env") or "").strip() | ||
| ) |
Collaborator
There was a problem hiding this comment.
This lookup is only in the explicit_base_url direct-alias branch. The API-server flow in #67453 resolves with no explicit arguments, then reaches _resolve_openrouter_runtime() where model.key_env is still absent from the candidate chain; please move/share this logic and add a regression test for the plain configured-custom path.
teknium1
added
sweeper:risk-security-boundary
Contributor
SummarySeventeen PRs reference this issue complex: four address model-alias dispatch, twelve address distinct custom-provider credential-resolution paths, and #67554 adds model.key_env only to the explicit-base-URL branch rather than the API-server path reported by #67453. The diffs therefore share configuration-routing themes but do not represent one interchangeable fix. Related pull requests
DuplicatesAlias cluster: closed #29198 and stale #30716 overlap with broader #19213, while #62534 is the focused current CLI implementation. Credential cluster: closed #44669, #44684, and #46023 duplicate #44667's runtime alias change; open #12443, #44868, and #62507 overlap there, while #43675 and #43704 are complementary halves substantially combined by #45873. Suggested consolidationKeep #67554 open with a salvage path, consistent with its contributor review: move or share the profile-scoped model.key_env/api_key_env lookup into the configured-custom _resolve_openrouter_runtime path and add both a public resolver regression and the reported two-request API-server test. Keep recorded best-fix PRs #19213, #45873, #62507, and #62534 open only for their documented current-head salvage work; #30716 can be closed as a duplicate of #62534 because its stale, testless constructor diff is subsumed, while the already-closed credential duplicates need no further state action. Complex graphflowchart LR
classDef open fill:#dbeafe,stroke:#1d4ed8,color:#1e3a8a
classDef merged fill:#dcfce7,stroke:#15803d,color:#14532d
classDef closed fill:#e5e7eb,stroke:#6b7280,color:#1f2937
classDef unverified fill:#f3f4f6,stroke:#9ca3af,color:#374151
classDef best stroke-width:3px,stroke:#b45309
classDef target stroke-width:3px,stroke:#4338ca
I67453(["issue #67453 (open)"])
P67554["PR #67554 (open)"]
P67554 -.->|partial| I67453
class I67453 open
class P67554 open
class P67554 target
click I67453 "https://github.com/NousResearch/hermes-agent/issues/67453"
click P67554 "https://github.com/NousResearch/hermes-agent/pull/67554"
Graph: solid arrow = fixes / best fix, dashed arrow = partial or unverified (see edge label); boxed group = PRs duplicating each other; amber border = best fix; indigo border = target; gray node = closed (state tag in the node label). Cross-PR triage: Reviewed 17 pull requests and 8 issues in this complex. Each diff was read against this issue; Assessment working set: 78 kB of PR diffs, 60 kB of issue/PR text, 49 kB of discussion (61 comments), 60 verify verdicts. verdicts reflect diff content, not PR titles. Part of an automated triage batch. |
13 tasks
jonpol01
added a commit
to jonpol01/hermes-agent
that referenced
this pull request
Sep 3, 2026
…able doctor's credential check runs only for providers in PROVIDER_REGISTRY with auth_type api_key. A named custom provider (a `providers:` entry or a legacy `custom_providers` item) is not in the registry, so the check skipped it and doctor reported the configuration healthy. At runtime the key resolves key_env → inline api_key → key_cmd (hermes_cli/runtime_provider.py); when key_env names a variable that is not set and the entry has no other source, every request fails with an auth error and the user is sent to inspect the provider instead of their .env — the tracker has a long tail of "custom provider key_env → 401" reports for exactly this. After the registry check, resolve the active model.provider to its custom entry through the same alias set the runtime uses and, when its key_env is declared but unset and no api_key / key_cmd is present, fail with the variable's name and where to set it. Only the active route is checked here; the bare `provider: custom` + model.key_env form is left alone because the runtime does not resolve that key yet (NousResearch#67554).
jonpol01
added a commit
to jonpol01/hermes-agent
that referenced
this pull request
Sep 6, 2026
…able doctor's credential check runs only for providers in PROVIDER_REGISTRY with auth_type api_key. A named custom provider (a `providers:` entry or a legacy `custom_providers` item) is not in the registry, so the check skipped it and doctor reported the configuration healthy. At runtime the key resolves key_env → inline api_key → key_cmd (hermes_cli/runtime_provider.py); when key_env names a variable that is not set and the entry has no other source, every request fails with an auth error and the user is sent to inspect the provider instead of their .env — the tracker has a long tail of "custom provider key_env → 401" reports for exactly this. After the registry check, resolve the active model.provider to its custom entry through the same alias set the runtime uses and, when its key_env is declared but unset and no api_key / key_cmd is present, fail with the variable's name and where to set it. Only the active route is checked here; the bare `provider: custom` + model.key_env form is left alone because the runtime does not resolve that key yet (NousResearch#67554).
jonpol01
added a commit
to jonpol01/hermes-agent
that referenced
this pull request
Sep 18, 2026
…able doctor's credential check runs only for providers in PROVIDER_REGISTRY with auth_type api_key. A named custom provider (a `providers:` entry or a legacy `custom_providers` item) is not in the registry, so the check skipped it and doctor reported the configuration healthy. At runtime the key resolves key_env → inline api_key → key_cmd (hermes_cli/runtime_provider.py); when key_env names a variable that is not set and the entry has no other source, every request fails with an auth error and the user is sent to inspect the provider instead of their .env — the tracker has a long tail of "custom provider key_env → 401" reports for exactly this. After the registry check, resolve the active model.provider to its custom entry through the same alias set the runtime uses and, when its key_env is declared but unset and no api_key / key_cmd is present, fail with the variable's name and where to set it. Only the active route is checked here; the bare `provider: custom` + model.key_env form is left alone because the runtime does not resolve that key yet (NousResearch#67554).
jonpol01
added a commit
to jonpol01/hermes-agent
that referenced
this pull request
Sep 18, 2026
…able doctor's credential check runs only for providers in PROVIDER_REGISTRY with auth_type api_key. A named custom provider (a `providers:` entry or a legacy `custom_providers` item) is not in the registry, so the check skipped it and doctor reported the configuration healthy. At runtime the key resolves key_env → inline api_key → key_cmd (hermes_cli/runtime_provider.py); when key_env names a variable that is not set and the entry has no other source, every request fails with an auth error and the user is sent to inspect the provider instead of their .env — the tracker has a long tail of "custom provider key_env → 401" reports for exactly this. After the registry check, resolve the active model.provider to its custom entry through the same alias set the runtime uses and, when its key_env is declared but unset and no api_key / key_cmd is present, fail with the variable's name and where to set it. Only the active route is checked here; the bare `provider: custom` + model.key_env form is left alone because the runtime does not resolve that key yet (NousResearch#67554).
pull Bot
pushed a commit
to dot-agi/hermes-agent
that referenced
this pull request
Sep 19, 2026
…et key_env is logged instead of laundered into no-key-required `model.provider: custom` + `model.base_url` + `model.key_env` (the wizard's bare-custom shape) resolved through `_resolve_openrouter_runtime`, whose candidate list only knew `model.api_key` and the host-gated OPENAI/OPENROUTER env keys — `model.key_env` was never read, so every request went out as `Bearer no-key-required` and the endpoint returned 401/403 in tens of milliseconds (NousResearch#67453). The same rung is what the API server platform resolves on each request, which is why the reporter saw it on every request. - `_model_cfg_key_env_for()` supplies the declared variable's value on the bare-custom and direct-alias rungs, only when the target base_url IS the configured `model.base_url` (a CUSTOM_BASE_URL or alias endpoint elsewhere never receives that key). - `_key_env_secret()` is the one key_env/api_key_env reader for custom blocks (model block and custom_providers entries): a declared variable that resolves to nothing is now WARNING-logged before the `no-key-required` substitution, so a misnamed/unexported var points at the Hermes config instead of the provider's IAM. Blocks with no key_env stay silent — that is the keyless local-server configuration. - `is_output_cap_error()` recognises "max_completion_tokens is limited to N" (Scaleway), so the budget step-down triggers instead of the compressor (second atom of NousResearch#67453). Supersedes NousResearch#67554 (@JonthanaHanh), which added the same lookup on the explicit-base_url branch only.
Collaborator
|
Thanks @JonthanaHanh — the work in this PR has landed on
Your contribution is credited there (cherry-picked authorship / co-author trailer or credit in the PR body; see the linked PR for what was kept and what was trimmed). Closing this one as landed / superseded so the backlog reflects reality. If something in your original diff is still missing on current |
jonpol01
added a commit
to jonpol01/hermes-agent
that referenced
this pull request
Sep 20, 2026
…able doctor's credential check runs only for providers in PROVIDER_REGISTRY with auth_type api_key. A named custom provider (a `providers:` entry or a legacy `custom_providers` item) is not in the registry, so the check skipped it and doctor reported the configuration healthy. At runtime the key resolves key_env → inline api_key → key_cmd (hermes_cli/runtime_provider.py); when key_env names a variable that is not set and the entry has no other source, every request fails with an auth error and the user is sent to inspect the provider instead of their .env — the tracker has a long tail of "custom provider key_env → 401" reports for exactly this. After the registry check, resolve the active model.provider to its custom entry through the same alias set the runtime uses and, when its key_env is declared but unset and no api_key / key_cmd is present, fail with the variable's name and where to set it. Only the active route is checked here; the bare `provider: custom` + model.key_env form is left alone because the runtime does not resolve that key yet (NousResearch#67554).
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
4 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
When
provider: customis configured withmodel.key_env(instead of an inlineapi_key), the API key is correctly resolved for the first session after boot but fails with 403 for every subsequent session.Root Cause
The bare
provider: custom+model.base_urlpath inresolve_runtime_provider()(lines 940-972) builds anapi_key_candidateslist but never readskey_envfrom the model config. It only checks:explicit_api_key(always None when called from_resolve_runtime_agent_kwargs)OPENAI_API_KEY(gated on OpenAI host)OPENROUTER_API_KEY(gated on OpenRouter host)_host_derived_api_key(derived from hostname)The
model.key_envfield (e.g.SCW_LLM_API_KEY) is completely ignored. The first session may succeed through a different resolution path (e.g._host_derived_api_keymatching), but subsequent sessions fail because the key is never re-resolved from the configured env var.Fix
Read
model.key_envfrom config and resolve the env var via_getenv(), adding it to theapi_key_candidateslist afterexplicit_api_keyand before host-gated fallbacks. This mirrors the named-custom-provider path at line 1013 which already readskey_env.Testing
tests/hermes_cli/test_runtime_provider*.pytests passpy_compilecleanFixes #67453