Skip to content

fix(gateway): scope api_server secrets under multiplex profiles - #61283

Closed
giggling-ginger wants to merge 2 commits into
NousResearch:mainfrom
giggling-ginger:fix/61276-multiplex-profiles-secret-scope
Closed

fix(gateway): scope api_server secrets under multiplex profiles#61283
giggling-ginger wants to merge 2 commits into
NousResearch:mainfrom
giggling-ginger:fix/61276-multiplex-profiles-secret-scope

Conversation

@giggling-ginger

Copy link
Copy Markdown
Contributor

Summary

  • When gateway.multiplex_profiles is on, credential reads go through get_secret and fail closed unless a profile secret scope is installed.
  • Messaging platforms already wrap turns in _profile_runtime_scope; api_server (/v1/chat/completions SSE and /v1/runs) did not, so agent setup raised UnscopedSecretError on OPENROUTER_BASE_URL (and any other scoped credential).
  • Install the same profile secret scope on api_server agent entry (create + run), inside the executor thread where ContextVars actually apply. No-op when multiplexing is off.

Fixes #61276

Test plan

  • scripts/run_tests.sh tests/gateway/test_api_server_multiplex_secret_scope.py
  • Existing test_run_agent_binds_api_session_context_for_tool_env still passes
  • Optional live check: gateway.multiplex_profiles: true + api_server streaming chat completion with OpenRouter

API-server agent entry did not install a profile secret scope when
gateway.multiplex_profiles is on, so get_secret (e.g. OPENROUTER_BASE_URL)
failed closed during SSE chat completions. Mirror the cron/messenger
scope install on _run_agent and /v1/runs paths.

Fixes NousResearch#61276
@alt-glitch alt-glitch added type/bug Something isn't working P2 Medium — degraded but workaround exists comp/gateway Gateway runner, session dispatch, delivery area/auth Authentication, OAuth, credential pools sweeper:risk-security-boundary Sweeper risk: may affect sandboxing, auth, credentials, or sensitive data labels Jul 9, 2026
@alt-glitch

Copy link
Copy Markdown
Collaborator

This was generated by AI during triage.

Fix PR for #61276. Distinct call site from the same multiplex secret-scope family — #52399 (API_SERVER_* enable/key in config.py), #52307 (_apply_env_overrides) — this installs the profile secret scope on the api_server agent entry. Related, not duplicate.

@teknium1 teknium1 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for tracing the API-server executor boundary. The premise is confirmed on current main: gateway/platforms/api_server.py:4052-4096 runs _create_agent() and run_conversation() in a bare executor, while multiplex mode makes unscoped get_secret() calls fail closed (agent/secret_scope.py:149-157). The PR scopes that path and the independent /v1/runs construction/executor path.

Suggested changes

  • Add a /v1/runs regression covering scope visibility during both agent construction and run_conversation; the new test currently covers only _run_agent, whereas /v1/runs has separate regions at proposed gateway/platforms/api_server.py:4330 and :4379.

Automated hermes-sweeper review.

)
# Credential resolution during agent construction must run under
# the profile secret scope when multiplexing is on (#61276).
with _api_server_profile_secret_scope():

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please add a /v1/runs regression that verifies a scoped secret is available during agent construction and during the executor-run conversation. The new test currently covers _run_agent only; this path has an independent construction/executor split.

@giggling-ginger

Copy link
Copy Markdown
Contributor Author

Addressed the review feedback in commit 1952ff37c by adding a /v1/runs regression test that verifies profile secret-scope visibility during both agent construction and run_conversation in the executor thread.

@teknium1 teknium1 added the sweeper:blast-moderate Sweeper blast radius: moderate — a subsystem or single platform label Jul 11, 2026
teknium1 pushed a commit that referenced this pull request Jul 16, 2026
Rebuilt from PR #61283 onto the /p/<profile>/ routing world (7aa21e3):
_profile_scope(None) now enters the DEFAULT profile's runtime scope when
multiplexing is active instead of returning nullcontext(). api_server is
a port-binding platform living on the default profile, so plain requests
(no /p/ prefix) are the primary path — with fail-closed get_secret they
crashed with UnscopedSecretError on the first credential read (#61276).

All three wrapped call sites (chat-completions executor, /v1/runs agent
construction and _run_sync) inherit the fix through the one seam.
Single-profile gateways keep the no-op. Regression tests ported from
the original PR to the _profile_scope seam.

Fixes #61276
@teknium1

Copy link
Copy Markdown
Contributor

Merged via PR #65700 — rebuilt on current main with your authorship preserved in git log (rebase merge, commit credited to you). Your diagnosis of #61276 was correct and your PR was ahead of the codebase: the /p/<profile>/ routing rewrite (7aa21e3) landed after you opened this and superseded the mechanical diff, but left exactly the gap you identified — plain requests on the default listener still ran unscoped. The rebuild puts your fix at the new seam: _profile_scope(None) enters the default profile's runtime scope when multiplex is active, covering all three wrapped call sites at once. Your regression tests were ported to the new seam rather than discarded. Closes #61276. Thanks!

Note: #60726 (background-task UnscopedSecretError) is the same bug class at a different call site (_run_background_task in gateway/run.py) and still needs its own fix using the same pattern.

@teknium1 teknium1 closed this Jul 16, 2026
@teknium1 teknium1 added the area/profiles Multi-profile isolation, HERMES_HOME scoping label Jul 19, 2026
Gravezzz pushed a commit to Gravezzz/hermes-agent that referenced this pull request Jul 21, 2026
Rebuilt from PR NousResearch#61283 onto the /p/<profile>/ routing world (7aa21e3):
_profile_scope(None) now enters the DEFAULT profile's runtime scope when
multiplexing is active instead of returning nullcontext(). api_server is
a port-binding platform living on the default profile, so plain requests
(no /p/ prefix) are the primary path — with fail-closed get_secret they
crashed with UnscopedSecretError on the first credential read (NousResearch#61276).

All three wrapped call sites (chat-completions executor, /v1/runs agent
construction and _run_sync) inherit the fix through the one seam.
Single-profile gateways keep the no-op. Regression tests ported from
the original PR to the _profile_scope seam.

Fixes NousResearch#61276
randlee pushed a commit to randlee/hermes-agent that referenced this pull request Aug 11, 2026
Rebuilt from PR NousResearch#61283 onto the /p/<profile>/ routing world (e2d534a):
_profile_scope(None) now enters the DEFAULT profile's runtime scope when
multiplexing is active instead of returning nullcontext(). api_server is
a port-binding platform living on the default profile, so plain requests
(no /p/ prefix) are the primary path — with fail-closed get_secret they
crashed with UnscopedSecretError on the first credential read (NousResearch#61276).

All three wrapped call sites (chat-completions executor, /v1/runs agent
construction and _run_sync) inherit the fix through the one seam.
Single-profile gateways keep the no-op. Regression tests ported from
the original PR to the _profile_scope seam.

Fixes NousResearch#61276
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area/auth Authentication, OAuth, credential pools area/profiles Multi-profile isolation, HERMES_HOME scoping comp/gateway Gateway runner, session dispatch, delivery P2 Medium — degraded but workaround exists sweeper:blast-moderate Sweeper blast radius: moderate — a subsystem or single platform sweeper:risk-security-boundary Sweeper risk: may affect sandboxing, auth, credentials, or sensitive data type/bug Something isn't working

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Bug]: gateway.multiplex_profiles model error

3 participants