fix(auth): restore secret-scope isolation for fallback API key resolution

[JoaoMarcos44]

Summary

• resolve_entry_api_key() (hermes_cli/fallback_config.py) resolved a fallback entry's key_env via a raw os.getenv(), ignoring the per-profile secret scope installed by the multiplexed gateway.
• agent/auxiliary_client.py had an independent, near-identical _fallback_entry_api_key() with the same raw-os.getenv() bug.
• Both now resolve through the existing scope-aware, fail-closed agent.secret_scope.get_secret() contract, and the auxiliary-client copy now delegates to the single centralized resolver instead of re-implementing it.

Root Cause

The gateway multiplexes several profiles from one process. Each profile's secrets are installed into a context-local scope (agent.secret_scope.set_secret_scope) precisely so one profile's credentials never leak into another profile's turn — process-global os.environ can hold whichever profile's .env was loaded most recently (or, under multiplexing, is not authoritative at all).

resolve_entry_api_key() bypassed that scope entirely:

key_env = str(entry.get("key_env") or entry.get("api_key_env") or "").strip()
if key_env:
    return os.getenv(key_env, "").strip() or None

Every caller of this function — hermes_cli/cli_agent_setup_mixin.py (CLI-side fallback), gateway/run.py (_try_resolve_fallback_provider), and cron/scheduler.py (scheduled-job fallback) — inherited the bug because none of them re-scope the key afterward; they trust resolve_entry_api_key() to already return the right value. When a fallback entry's key_env name happened to collide with (or simply read) a stale/other-profile value sitting in os.environ, a fallback request could run under the wrong profile's credential.

agent/auxiliary_client.py had _fallback_entry_api_key(), a separate implementation of the exact same inline-api_key-then-key_env logic, with the identical raw-os.getenv() flaw.

Fix

• resolve_entry_api_key() now resolves key_env via agent.secret_scope.get_secret():
  • When a per-profile secret scope is active, the scoped value wins.
  • When there's no active scope (single-profile / non-multiplexed deployments, the default), it falls back to os.environ exactly as before — no behavior change for existing non-gateway callers.
  • When multiplexing is active with no scope installed, get_secret() fails closed (UnscopedSecretError) instead of silently reading a stranger's credential. All three call sites already wrap this call in try/except and skip to the next fallback entry, so this fails safe rather than crashing.
• agent/auxiliary_client.py's _fallback_entry_api_key() now delegates to hermes_cli.fallback_config.resolve_entry_api_key() instead of re-implementing the same lookup, so there's one canonical, scope-aware implementation.
• The three downstream callers (cli_agent_setup_mixin.py, gateway/run.py, cron/scheduler.py) required no changes — they inherit the fix automatically and were verified to still import/compile cleanly.

Diff is surgical: no changes to unrelated fallback-chain merging or provider-resolution logic.

Test Plan

• Added test_key_env_resolves_from_active_secret_scope_not_raw_env to tests/hermes_cli/test_fallback_config.py: installs an active secret scope with fake-active-profile-key while os.environ holds a different fake-other-profile-key, asserts the scoped key wins.
• Added test_key_env_falls_back_to_env_when_no_active_scope: with no scope installed, asserts resolution still returns the env var (no regression for single-profile users).
• python -m pytest tests/hermes_cli/test_fallback_config.py -v — all 10 tests pass (8 pre-existing + 2 new).
• Verified hermes_cli.cli_agent_setup_mixin, gateway.run, cron.scheduler, and agent.auxiliary_client all import cleanly after the change (no circular-import regressions).

Closes #74311

[teknium1]

Thanks for tracing the shared and auxiliary fallback resolvers; the premise is verified on current main (hermes_cli/fallback_config.py:28-30, agent/auxiliary_client.py:4739-4741). The proposed get_secret() routing matches the isolation contract in agent/secret_scope.py:150-177 and preserves non-multiplexed environment fallback.

Problems

• The same fallback-entry shape is still resolved with raw os.getenv() in the init-time fallback at agent/agent_init.py:1224-1232 and in ordinary runtime failover at agent/chat_completion_helpers.py:1791-1807. Both pass that value as explicit_api_key, so the cross-profile credential bypass remains on active main-agent fallback paths.
• The added tests cover only the shared resolver, not either remaining main-agent path.

Suggested changes

• Reuse resolve_entry_api_key() in both remaining fallback paths and add a scoped-versus-process-environment regression test for main-agent failover.

This is an automated hermes-sweeper review.

[egilewski]

suggesting changes

The shared fallback-entry helper now resolves an entry-level key_env through the active profile secret scope, and the changed fallback callers pass that value as explicit_api_key. The named-custom-provider branch of resolve_provider_client, however, does not consult explicit_api_key. For a supported legacy custom_providers entry whose credential is named by key_env, _get_named_custom_provider returns the unresolved variable name and this branch reads it with os.getenv, so a named custom fallback can still attach a process-global credential belonging to another profile to the outbound client. This leaves the cross-profile isolation goal incomplete.

• [P1] Preserve the scoped key when resolving legacy named custom fallbacks (agent/auxiliary_client.py:5515)
  _resolve_fallback_entry passes the scoped fallback key to resolve_provider_client at lines 4743-4749, but the named-custom branch builds custom_key only from the custom-provider entry and then raw-reads its key_env at lines 5517-5520. Legacy list-style custom_providers entries are a supported resolver path and preserve key_env for this branch, so under multiplexing the raw read bypasses the active profile scope. Modern providers: entries are not evidence of this leak because hermes_cli.runtime_provider._get_named_custom_provider already resolves their key_env through the secret-scoped _getenv.
  Remediation: Give explicit_api_key precedence in the named-custom branch and resolve any remaining legacy custom_key_env through agent.secret_scope.get_secret. Add an end-to-end resolver test with multiplexing active, a legacy named custom fallback, a scoped key, and a different process-global key; assert that client construction receives only the scoped key. Also assert that a missing scoped key cannot fall through to the process environment.

Security evidence:

• trust boundary: A multiplexing gateway serves multiple profiles in one process. Per-profile credentials belong to the active ContextVar secret scope, while os.environ is process-global and cannot be trusted for profile credentials. Fallback and custom-provider configuration select the credential name, and outbound client construction is the sink where that value becomes authorization material.
• source/sink/invariant: In multiplex mode, every fallback credential must resolve from the active profile scope or fail closed, and a downstream provider branch must not discard an already scoped explicit_api_key and reread a profile secret from os.environ. The PR establishes this invariant in resolve_entry_api_key, but the legacy named-custom path from hermes_cli/runtime_provider.py:748-783 to agent/auxiliary_client.py:5515-5520 still violates it.
• current-main reproduction: A recorded probe against bound current main 4a8eeb5d1cd427d200ef2e9b55bc22e48bd4ebca installed scoped FB_KEY=active-profile-key and process-global FB_KEY=foreign-profile-key; current main returned the process-global value from resolve_entry_api_key, and source inspection confirms that its legacy named-custom branch also raw-reads the returned key_env before client construction.
• PR-head or patch-replay validation: The checkout was exactly 034870ff5766099a88307e2b6056a9723f7f231b. Focused probes reported that its shared helper returns the scoped key, returns None for an absent key in an installed multiplex scope, and raises UnscopedSecretError when multiplexing is active without a scope; source inspection confirms that the later legacy named-custom branch still ignores the explicit result and calls os.getenv. The security hunks also replayed against bound current main without conflict markers while preserving that sink.
• positive/negative cases: On PR head, focused checks showed scoped key_env overriding a different process-global value, inline keys retaining priority, the Ollama fallback using get_secret, and missing or unscoped keys failing closed, while the negative legacy named-custom integration case remains uncovered and source-reachable.
• residual bypass search: The fallback callers in agent_init.py, chat_completion_helpers.py, and auxiliary_client.py now use the shared resolver, but each reaches resolve_provider_client; the material residual path is fallback entry to scoped explicit_api_key to legacy named custom lookup to raw os.getenv at line 5520. The modern providers: lookup is already scope-aware, and unrelated raw environment reads are outside this finding.
• reviewer validation: The review inspected both PR commits and the exact three-dot diff, traced every changed fallback caller into client construction, compared the sink with bound current main, and ran git diff --check; the full pytest and ruff suites were unavailable because the checkout has no usable development virtualenv and system Python lacks pytest, httpx, and ruff.

Signed: GPT-5.6-sol-xhigh in Codex