Skip to content

fix(auth): heal poisoned stored Nous inference URL on the no-refresh read path - #50265

Closed
r266-tech wants to merge 1 commit into
NousResearch:mainfrom
r266-tech:fix/auth-nous-url-heal-norefresh
Closed

fix(auth): heal poisoned stored Nous inference URL on the no-refresh read path#50265
r266-tech wants to merge 1 commit into
NousResearch:mainfrom
r266-tech:fix/auth-nous-url-heal-norefresh

Conversation

@r266-tech

Copy link
Copy Markdown
Contributor

Follow-up to merged #49735 (heal a poisoned Nous inference_base_url).

Problem

#49735 healed a poisoned stored URL only on the OAuth-refresh path (both refresh sites reset to the default when _validate_nous_inference_url_from_network() rejects the Portal-returned URL). But resolve_nous_runtime_credentials reads the stored inference_base_url via the unrestricted _optional_base_url (auth.py:5612), and the heal lives inside if force_refresh or invoke_jwt_status is not None:.

When the access token is still a usable invoke JWT — the common steady state — no refresh fires, the heal block is skipped, and the stored value flows straight to state["inference_base_url"] (persist) and the returned base_url. So a poisoned auth.json (e.g. a stg-inference-api... host persisted before the allowlist existed — the exact incident #49735 cites) keeps 401ing forever: a 401 from the dead inference endpoint does not trigger an OAuth refresh, so the refresh-path heal is never reached.

Fix

Validate the stored inference_base_url on read, before the or-chain, mirroring the refresh-site heal: reject → fall back to the default (which then persists, repairing auth.json on first read).

The stored value is network-provenance (it was persisted from a Portal refresh response), so validating it is consistent with the existing trust model. The user-provided NOUS_INFERENCE_BASE_URL env override stays unvalidated — it's the documented dev/staging escape hatch — by validating only the stored branch and leaving the os.getenv line untouched.

All three existing guard tests still hold:

  • test_no_unvalidated_inference_base_url_assignments_remain — only state.get(...) is read via _optional_base_url; the guarded refreshed.get/mint_payload.get patterns are untouched.
  • test_validator_wired_at_all_known_call_sites — the 2 refresh-site validator calls are unchanged (the new call uses stored_inference_base_url).
  • test_env_override_path_does_not_call_validator — the os.getenv("NOUS_INFERENCE_BASE_URL") line never gains the validator.

Tests

New TestNoRefreshReadPathHeals (3 cases, valid token so no refresh fires): poisoned stored URL → heals to default (fails on pre-fix code); allowlisted stored URL → preserved; poisoned stored + NOUS_INFERENCE_BASE_URL env set → env value used unvalidated (escape hatch intact).

Note / scope

refresh_nous_oauth_pure accepts inference_base_url as a caller-resolved parameter and likewise doesn't re-validate it on its own no-refresh path. I scoped this PR to the documented runtime resolver (resolve_nous_runtime_credentials, matching the #49735 incident); happy to fold a symmetric guard into refresh_nous_oauth_pure if you'd prefer.

Every symbol re-read against current main. If this has already been addressed via a separate patch, feel free to close.

…read path

NousResearch#49735 healed a poisoned inference_base_url only on the OAuth-refresh
path. On the no-refresh read path (valid invoke JWT, the common steady
state) resolve_nous_runtime_credentials read the stored value via
_optional_base_url and served it unvalidated, so a poisoned auth.json
401s forever (a 401 from the dead host never triggers a refresh).
Validate the stored (network-provenance) value on read, mirroring the
refresh-site heal; the user-provided NOUS_INFERENCE_BASE_URL env
override stays unvalidated.
@alt-glitch alt-glitch added type/security Security vulnerability or hardening P2 Medium — degraded but workaround exists comp/cli CLI entry point, hermes_cli/, setup wizard area/auth Authentication, OAuth, credential pools provider/nous Nous Research API (OAuth) labels Jun 21, 2026
@alt-glitch

Copy link
Copy Markdown
Collaborator

Related: #49735 (heals a poisoned inference_base_url on the OAuth refresh path) and the host-allowlist validation cluster #27612/#30611.

This PR fills the gap that #49735 does not cover: the no-refresh read path in resolve_nous_runtime_credentials(). When the access token is still a usable invoke JWT, no refresh fires, and main reads the stored inference_base_url via the unrestricted _optional_base_url() and serves it verbatim — a poisoned value (e.g. a staging host persisted before the allowlist existed) then 401s forever against a dead endpoint and never self-heals (a 401 from the inference host does not trigger an OAuth refresh). Confirmed live on main (the read path has no _validate_nous_inference_url_from_network() call). Not a duplicate of #49735 — same goal, different code path.

teknium1 pushed a commit that referenced this pull request Jun 25, 2026
…sessions (#52270)

The host-allowlist hardening (#30611) plus the refresh heal (#49735) left
the documented NOUS_INFERENCE_BASE_URL dev/staging escape hatch unreachable
for OAuth sessions, despite three code comments asserting it still works.

Root cause — resolution precedence in resolve_nous_runtime_credentials:

    inference_base_url = (
        _optional_base_url(state.get("inference_base_url"))  # stored — wins
        or os.getenv("NOUS_INFERENCE_BASE_URL")              # env — unreachable
        or DEFAULT_NOUS_INFERENCE_URL
    )

A staging OAuth login persists its inference_base_url, but the allowlist
rejects the staging host and the refresh heal rewrites the stored value to
the production default. The stored (now prod) value is then read BEFORE the
env var, so the override never takes effect — every request 401s against
prod or is pinned to prod, and setting the env var does nothing.

Fix: the user-set env override is the most-trusted source, so consult it
FIRST for the URL used to build the client / returned to callers — while
keeping the PERSISTED value the validated, network-provenance one (the
override is a runtime overlay, never written to auth.json, so unsetting it
cleanly reverts to prod). Applied at both chokepoints:

- resolve_nous_runtime_credentials (no-refresh read path AND refresh path)
- the nous_portal proxy adapter, which re-validates the resolver's returned
  base_url against the prod allowlist as defense-in-depth and would
  otherwise reject a legitimate staging override at the forward boundary.

New _nous_inference_env_override() / split of stored-vs-effective URL keep
the threat model intact: Portal-returned URLs are still allowlist-validated
at every network site, and the env path stays ungated (trusted OS user).

Also folds in the no-refresh read-path heal (supersedes the approach in
the open #50265): a poisoned stored staging host now heals to the prod
default on read even when no refresh fires.

Tests: TestEnvOverrideWins (env wins on read + refresh paths; override never
persisted; poisoned stored heals) and TestProxyAdapterEnvOverride. Verified
the 4 behavioral tests fail against pre-fix code and pass with the fix; full
inference-validation + nous-provider suites green (85 passed). E2E-validated
against a real temp HERMES_HOME exercising the real resolver + proxy adapter:
resolver→staging, persisted→prod, proxy→staging, unset→reverts to prod.
@teknium1

Copy link
Copy Markdown
Contributor

Superseded by #52270 (merged), which folds in the no-refresh-read-path heal you implemented here AND fixes the NOUS_INFERENCE_BASE_URL env-override precedence that this PR left unaddressed — your heal alone resolved a poisoned stored host to the production default, but an OAuth user with the staging env override set still couldn't reach staging because the stored (prod) value was read before the env var.

#52270 explicitly cites this PR as superseded and credits your read-path heal. Thanks for the fix — the core idea landed.

@teknium1 teknium1 closed this Jun 25, 2026
pai-scaffolde pushed a commit to pai-scaffolde/hermes-agent that referenced this pull request Jun 28, 2026
…sessions (NousResearch#52270)

The host-allowlist hardening (NousResearch#30611) plus the refresh heal (NousResearch#49735) left
the documented NOUS_INFERENCE_BASE_URL dev/staging escape hatch unreachable
for OAuth sessions, despite three code comments asserting it still works.

Root cause — resolution precedence in resolve_nous_runtime_credentials:

    inference_base_url = (
        _optional_base_url(state.get("inference_base_url"))  # stored — wins
        or os.getenv("NOUS_INFERENCE_BASE_URL")              # env — unreachable
        or DEFAULT_NOUS_INFERENCE_URL
    )

A staging OAuth login persists its inference_base_url, but the allowlist
rejects the staging host and the refresh heal rewrites the stored value to
the production default. The stored (now prod) value is then read BEFORE the
env var, so the override never takes effect — every request 401s against
prod or is pinned to prod, and setting the env var does nothing.

Fix: the user-set env override is the most-trusted source, so consult it
FIRST for the URL used to build the client / returned to callers — while
keeping the PERSISTED value the validated, network-provenance one (the
override is a runtime overlay, never written to auth.json, so unsetting it
cleanly reverts to prod). Applied at both chokepoints:

- resolve_nous_runtime_credentials (no-refresh read path AND refresh path)
- the nous_portal proxy adapter, which re-validates the resolver's returned
  base_url against the prod allowlist as defense-in-depth and would
  otherwise reject a legitimate staging override at the forward boundary.

New _nous_inference_env_override() / split of stored-vs-effective URL keep
the threat model intact: Portal-returned URLs are still allowlist-validated
at every network site, and the env path stays ungated (trusted OS user).

Also folds in the no-refresh read-path heal (supersedes the approach in
the open NousResearch#50265): a poisoned stored staging host now heals to the prod
default on read even when no refresh fires.

Tests: TestEnvOverrideWins (env wins on read + refresh paths; override never
persisted; poisoned stored heals) and TestProxyAdapterEnvOverride. Verified
the 4 behavioral tests fail against pre-fix code and pass with the fix; full
inference-validation + nous-provider suites green (85 passed). E2E-validated
against a real temp HERMES_HOME exercising the real resolver + proxy adapter:
resolver→staging, persisted→prod, proxy→staging, unset→reverts to prod.
waefrebeorn pushed a commit to waefrebeorn/slermes that referenced this pull request Jul 2, 2026
…sessions (NousResearch#52270)

The host-allowlist hardening (NousResearch#30611) plus the refresh heal (NousResearch#49735) left
the documented NOUS_INFERENCE_BASE_URL dev/staging escape hatch unreachable
for OAuth sessions, despite three code comments asserting it still works.

Root cause — resolution precedence in resolve_nous_runtime_credentials:

    inference_base_url = (
        _optional_base_url(state.get("inference_base_url"))  # stored — wins
        or os.getenv("NOUS_INFERENCE_BASE_URL")              # env — unreachable
        or DEFAULT_NOUS_INFERENCE_URL
    )

A staging OAuth login persists its inference_base_url, but the allowlist
rejects the staging host and the refresh heal rewrites the stored value to
the production default. The stored (now prod) value is then read BEFORE the
env var, so the override never takes effect — every request 401s against
prod or is pinned to prod, and setting the env var does nothing.

Fix: the user-set env override is the most-trusted source, so consult it
FIRST for the URL used to build the client / returned to callers — while
keeping the PERSISTED value the validated, network-provenance one (the
override is a runtime overlay, never written to auth.json, so unsetting it
cleanly reverts to prod). Applied at both chokepoints:

- resolve_nous_runtime_credentials (no-refresh read path AND refresh path)
- the nous_portal proxy adapter, which re-validates the resolver's returned
  base_url against the prod allowlist as defense-in-depth and would
  otherwise reject a legitimate staging override at the forward boundary.

New _nous_inference_env_override() / split of stored-vs-effective URL keep
the threat model intact: Portal-returned URLs are still allowlist-validated
at every network site, and the env path stays ungated (trusted OS user).

Also folds in the no-refresh read-path heal (supersedes the approach in
the open NousResearch#50265): a poisoned stored staging host now heals to the prod
default on read even when no refresh fires.

Tests: TestEnvOverrideWins (env wins on read + refresh paths; override never
persisted; poisoned stored heals) and TestProxyAdapterEnvOverride. Verified
the 4 behavioral tests fail against pre-fix code and pass with the fix; full
inference-validation + nous-provider suites green (85 passed). E2E-validated
against a real temp HERMES_HOME exercising the real resolver + proxy adapter:
resolver→staging, persisted→prod, proxy→staging, unset→reverts to prod.
habarmc1223-sudo pushed a commit to habarmc1223-sudo/hermes-agent-fluxmem that referenced this pull request Jul 8, 2026
…sessions (NousResearch#52270)

The host-allowlist hardening (NousResearch#30611) plus the refresh heal (NousResearch#49735) left
the documented NOUS_INFERENCE_BASE_URL dev/staging escape hatch unreachable
for OAuth sessions, despite three code comments asserting it still works.

Root cause — resolution precedence in resolve_nous_runtime_credentials:

    inference_base_url = (
        _optional_base_url(state.get("inference_base_url"))  # stored — wins
        or os.getenv("NOUS_INFERENCE_BASE_URL")              # env — unreachable
        or DEFAULT_NOUS_INFERENCE_URL
    )

A staging OAuth login persists its inference_base_url, but the allowlist
rejects the staging host and the refresh heal rewrites the stored value to
the production default. The stored (now prod) value is then read BEFORE the
env var, so the override never takes effect — every request 401s against
prod or is pinned to prod, and setting the env var does nothing.

Fix: the user-set env override is the most-trusted source, so consult it
FIRST for the URL used to build the client / returned to callers — while
keeping the PERSISTED value the validated, network-provenance one (the
override is a runtime overlay, never written to auth.json, so unsetting it
cleanly reverts to prod). Applied at both chokepoints:

- resolve_nous_runtime_credentials (no-refresh read path AND refresh path)
- the nous_portal proxy adapter, which re-validates the resolver's returned
  base_url against the prod allowlist as defense-in-depth and would
  otherwise reject a legitimate staging override at the forward boundary.

New _nous_inference_env_override() / split of stored-vs-effective URL keep
the threat model intact: Portal-returned URLs are still allowlist-validated
at every network site, and the env path stays ungated (trusted OS user).

Also folds in the no-refresh read-path heal (supersedes the approach in
the open NousResearch#50265): a poisoned stored staging host now heals to the prod
default on read even when no refresh fires.

Tests: TestEnvOverrideWins (env wins on read + refresh paths; override never
persisted; poisoned stored heals) and TestProxyAdapterEnvOverride. Verified
the 4 behavioral tests fail against pre-fix code and pass with the fix; full
inference-validation + nous-provider suites green (85 passed). E2E-validated
against a real temp HERMES_HOME exercising the real resolver + proxy adapter:
resolver→staging, persisted→prod, proxy→staging, unset→reverts to prod.
santhreal pushed a commit to santhreal/hermes-agent that referenced this pull request Jul 13, 2026
…sessions (NousResearch#52270)

The host-allowlist hardening (NousResearch#30611) plus the refresh heal (NousResearch#49735) left
the documented NOUS_INFERENCE_BASE_URL dev/staging escape hatch unreachable
for OAuth sessions, despite three code comments asserting it still works.

Root cause — resolution precedence in resolve_nous_runtime_credentials:

    inference_base_url = (
        _optional_base_url(state.get("inference_base_url"))  # stored — wins
        or os.getenv("NOUS_INFERENCE_BASE_URL")              # env — unreachable
        or DEFAULT_NOUS_INFERENCE_URL
    )

A staging OAuth login persists its inference_base_url, but the allowlist
rejects the staging host and the refresh heal rewrites the stored value to
the production default. The stored (now prod) value is then read BEFORE the
env var, so the override never takes effect — every request 401s against
prod or is pinned to prod, and setting the env var does nothing.

Fix: the user-set env override is the most-trusted source, so consult it
FIRST for the URL used to build the client / returned to callers — while
keeping the PERSISTED value the validated, network-provenance one (the
override is a runtime overlay, never written to auth.json, so unsetting it
cleanly reverts to prod). Applied at both chokepoints:

- resolve_nous_runtime_credentials (no-refresh read path AND refresh path)
- the nous_portal proxy adapter, which re-validates the resolver's returned
  base_url against the prod allowlist as defense-in-depth and would
  otherwise reject a legitimate staging override at the forward boundary.

New _nous_inference_env_override() / split of stored-vs-effective URL keep
the threat model intact: Portal-returned URLs are still allowlist-validated
at every network site, and the env path stays ungated (trusted OS user).

Also folds in the no-refresh read-path heal (supersedes the approach in
the open NousResearch#50265): a poisoned stored staging host now heals to the prod
default on read even when no refresh fires.

Tests: TestEnvOverrideWins (env wins on read + refresh paths; override never
persisted; poisoned stored heals) and TestProxyAdapterEnvOverride. Verified
the 4 behavioral tests fail against pre-fix code and pass with the fix; full
inference-validation + nous-provider suites green (85 passed). E2E-validated
against a real temp HERMES_HOME exercising the real resolver + proxy adapter:
resolver→staging, persisted→prod, proxy→staging, unset→reverts to prod.
Gravezzz pushed a commit to Gravezzz/hermes-agent that referenced this pull request Jul 21, 2026
…sessions (NousResearch#52270)

The host-allowlist hardening (NousResearch#30611) plus the refresh heal (NousResearch#49735) left
the documented NOUS_INFERENCE_BASE_URL dev/staging escape hatch unreachable
for OAuth sessions, despite three code comments asserting it still works.

Root cause — resolution precedence in resolve_nous_runtime_credentials:

    inference_base_url = (
        _optional_base_url(state.get("inference_base_url"))  # stored — wins
        or os.getenv("NOUS_INFERENCE_BASE_URL")              # env — unreachable
        or DEFAULT_NOUS_INFERENCE_URL
    )

A staging OAuth login persists its inference_base_url, but the allowlist
rejects the staging host and the refresh heal rewrites the stored value to
the production default. The stored (now prod) value is then read BEFORE the
env var, so the override never takes effect — every request 401s against
prod or is pinned to prod, and setting the env var does nothing.

Fix: the user-set env override is the most-trusted source, so consult it
FIRST for the URL used to build the client / returned to callers — while
keeping the PERSISTED value the validated, network-provenance one (the
override is a runtime overlay, never written to auth.json, so unsetting it
cleanly reverts to prod). Applied at both chokepoints:

- resolve_nous_runtime_credentials (no-refresh read path AND refresh path)
- the nous_portal proxy adapter, which re-validates the resolver's returned
  base_url against the prod allowlist as defense-in-depth and would
  otherwise reject a legitimate staging override at the forward boundary.

New _nous_inference_env_override() / split of stored-vs-effective URL keep
the threat model intact: Portal-returned URLs are still allowlist-validated
at every network site, and the env path stays ungated (trusted OS user).

Also folds in the no-refresh read-path heal (supersedes the approach in
the open NousResearch#50265): a poisoned stored staging host now heals to the prod
default on read even when no refresh fires.

Tests: TestEnvOverrideWins (env wins on read + refresh paths; override never
persisted; poisoned stored heals) and TestProxyAdapterEnvOverride. Verified
the 4 behavioral tests fail against pre-fix code and pass with the fix; full
inference-validation + nous-provider suites green (85 passed). E2E-validated
against a real temp HERMES_HOME exercising the real resolver + proxy adapter:
resolver→staging, persisted→prod, proxy→staging, unset→reverts to prod.
leewenjie pushed a commit to leewenjie/hermes-agent that referenced this pull request Aug 7, 2026
…sessions (NousResearch#52270)

The host-allowlist hardening (NousResearch#30611) plus the refresh heal (NousResearch#49735) left
the documented NOUS_INFERENCE_BASE_URL dev/staging escape hatch unreachable
for OAuth sessions, despite three code comments asserting it still works.

Root cause — resolution precedence in resolve_nous_runtime_credentials:

    inference_base_url = (
        _optional_base_url(state.get("inference_base_url"))  # stored — wins
        or os.getenv("NOUS_INFERENCE_BASE_URL")              # env — unreachable
        or DEFAULT_NOUS_INFERENCE_URL
    )

A staging OAuth login persists its inference_base_url, but the allowlist
rejects the staging host and the refresh heal rewrites the stored value to
the production default. The stored (now prod) value is then read BEFORE the
env var, so the override never takes effect — every request 401s against
prod or is pinned to prod, and setting the env var does nothing.

Fix: the user-set env override is the most-trusted source, so consult it
FIRST for the URL used to build the client / returned to callers — while
keeping the PERSISTED value the validated, network-provenance one (the
override is a runtime overlay, never written to auth.json, so unsetting it
cleanly reverts to prod). Applied at both chokepoints:

- resolve_nous_runtime_credentials (no-refresh read path AND refresh path)
- the nous_portal proxy adapter, which re-validates the resolver's returned
  base_url against the prod allowlist as defense-in-depth and would
  otherwise reject a legitimate staging override at the forward boundary.

New _nous_inference_env_override() / split of stored-vs-effective URL keep
the threat model intact: Portal-returned URLs are still allowlist-validated
at every network site, and the env path stays ungated (trusted OS user).

Also folds in the no-refresh read-path heal (supersedes the approach in
the open NousResearch#50265): a poisoned stored staging host now heals to the prod
default on read even when no refresh fires.

Tests: TestEnvOverrideWins (env wins on read + refresh paths; override never
persisted; poisoned stored heals) and TestProxyAdapterEnvOverride. Verified
the 4 behavioral tests fail against pre-fix code and pass with the fix; full
inference-validation + nous-provider suites green (85 passed). E2E-validated
against a real temp HERMES_HOME exercising the real resolver + proxy adapter:
resolver→staging, persisted→prod, proxy→staging, unset→reverts to prod.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area/auth Authentication, OAuth, credential pools comp/cli CLI entry point, hermes_cli/, setup wizard P2 Medium — degraded but workaround exists provider/nous Nous Research API (OAuth) type/security Security vulnerability or hardening

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants