Skip to content

fix(secrets): hydrate cold multiplex sources locally - #75263

Closed
allenkaplan wants to merge 5 commits into
NousResearch:mainfrom
allenkaplan:fix/74317-profile-local-secret-hydration
Closed

fix(secrets): hydrate cold multiplex sources locally#75263
allenkaplan wants to merge 5 commits into
NousResearch:mainfrom
allenkaplan:fix/74317-profile-local-secret-hydration

Conversation

@allenkaplan

@allenkaplan allenkaplan commented Jul 31, 2026

Copy link
Copy Markdown

Summary

Fixes #74317 by hydrating a cold multiplex profile’s configured external secret sources into an isolated, per-profile mapping before its first runtime secret scope is constructed.

This preserves the existing profile-local .env / .op.env bootstrap behavior and closes the review-identified extension and metadata boundaries:

  • cold multiplex hydration is now an explicit isolated-environment boundary;
  • legacy plugin secret sources are skipped at that boundary unless they declare supports_isolated_environment = True and use get_source_environment() / its mapping for bootstrap and child-process inputs;
  • bundled Bitwarden, 1Password, and command sources declare and use that contract;
  • source provenance now follows the resolved home, just like the existing per-home external-secret value snapshots.

Why this route

  • Rejected: process-global source application from _profile_runtime_scope(), because it can expose a profile’s bootstrap or resolved credentials to siblings.
  • Rejected: treating the existing ContextVar alone as sufficient isolation. A third-party source could legally use os.environ under the prior public SecretSource.fetch(cfg, home_path) contract.
  • Chosen: an additive, fail-closed plugin contract. Normal startup remains compatible with existing sources; only cold multiplex hydration requires the explicit isolation opt-in. A legacy plugin is reported and skipped rather than receiving another profile’s process environment.

Behavior

  1. A first routed secondary profile seeds a private mapping from its own .env and non-overriding .op.env.
  2. The registry invokes only sources that declare isolated-environment support.
  3. Those sources resolve against the private mapping; no bootstrap or resolved value is written to os.environ.
  4. Resolved values and provenance are persisted only under the resolved profile home.
  5. A lookup with a resolved home returns that profile’s source label. The compatibility lookup without a home returns a label only when all matching loaded homes agree, never a misleading cross-profile label.

Regression coverage

  • Existing cold-secondary gateway-path isolation, .op.env bootstrap, same-home fetch sharing, concurrency, reset/retry, and no-global-env-leak coverage.
  • New plugin-style regression: a legacy source that directly reads os.environ is not invoked during isolated hydration and cannot observe a default-profile canary.
  • New provenance regression: two homes may resolve the same key from different sources without overwriting one another’s label; an ambiguous no-home lookup returns no label.
  • Existing built-in and MCP safe-environment coverage updated for resolved-home provenance.

Validation

/home/allenkaplan07_gmail_com/.hermes-venv-py31115-sqlite3534-fts5/bin/python -m pytest -q \
  tests/gateway/test_multiplex_credential_isolation.py \
  tests/test_env_loader_secret_sources.py \
  tests/test_env_loader_op_bootstrap.py \
  tests/secret_sources/test_profile_secrets.py \
  tests/test_command_secret_source.py \
  tests/tools/test_mcp_tool.py
135 passed

/home/allenkaplan07_gmail_com/.hermes-venv-py31115-sqlite3534-fts5/bin/python -m ruff check [changed Python files]
All checks passed

git diff --check

Prior art / related work

Prepared with AI assistance; implementation and validation were reviewed against the current PR feedback and current source.

@alt-glitch alt-glitch added type/bug Something isn't working P2 Medium — degraded but workaround exists comp/agent Core agent runtime: loop, agent_init, prompt builder, context-compression, responses endpoint comp/cli CLI entry point, hermes_cli/, setup wizard comp/gateway Gateway runner, session dispatch, delivery area/auth Authentication, OAuth, credential pools area/config Config system, migrations, profiles area/profiles Multi-profile isolation, HERMES_HOME scoping sweeper:risk-security-boundary Sweeper risk: may affect sandboxing, auth, credentials, or sensitive data sweeper:risk-compatibility Sweeper risk: may break existing users, config, migrations, defaults, or upgrades sweeper:risk-message-delivery Sweeper risk: may drop, duplicate, misroute, or suppress messages needs-decision Awaiting maintainer decision before any implementation labels Jul 31, 2026

@teknium1 teknium1 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for carrying forward the cold-profile fix and incorporating the .op.env bootstrap path.

Problems

  • Blocking: agent/secret_sources/registry.py:217 wraps source.fetch(cfg, home_path) in a ContextVar, but the public SecretSource API still passes no environment (agent/secret_sources/base.py:162-169). Third-party sources are supported through PluginContext.register_secret_source() (agent/secret_sources/registry.py:19-25); an existing source that reads os.environ will still receive process-global/default-profile credentials during cold multiplex hydration. The built-in source conversions do not protect that extension path.
  • hermes_cli/env_loader.py:175 writes provenance into global _SECRET_SOURCES; get_secret_source() has no home argument. Two profiles resolving the same key through different sources can produce an incorrect source label for the first profile.

Suggested changes

  • Establish an isolation-capable plugin-source contract (or skip legacy sources on multiplex hydration) and add a plugin-style regression proving direct process-environment reads cannot observe another profile.
  • Scope provenance by resolved home, matching the existing value snapshots.

Automated hermes-sweeper review.

)
try:
future = executor.submit(source.fetch, cfg, home_path)
def _fetch() -> FetchResult:

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This ContextVar only works for sources that opt in to get_source_environment(). The supported plugin SecretSource.fetch(cfg, home_path) contract still permits an installed source to read os.environ directly, so cold multiplex hydration can expose the process-global/default-profile credentials to that source. Please add an isolation-capable compatibility boundary (or skip legacy sources) and a plugin-style regression.

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Addressed in 2baa23f. Cold multiplex hydration now explicitly requests an isolated environment; legacy plugin sources are skipped unless they declare supports_isolated_environment = True. The bundled Bitwarden, 1Password, and command sources opt in via their existing get_source_environment() contract. Added a plugin-style regression whose legacy source reads os.environ; it is not invoked, so it cannot observe the default-profile canary. Normal startup remains compatible with existing sources.

Comment thread hermes_cli/env_loader.py Outdated
value = local_env.get(name)
if value is None:
continue
_SECRET_SOURCES[name] = applied.source

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

_SECRET_SOURCES remains process-global while the values are now per-home. If two profiles hydrate the same variable through different sources, this overwrites the first profile's provenance and get_secret_source() has no home argument to recover it. Please scope provenance by resolved home too.

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Addressed in 2baa23f. Provenance now uses the same resolved-home boundary as secret values. get_secret_source() and format_secret_source_suffix() accept an optional home, and their compatibility fallback returns a label only when every matching loaded home agrees. Added coverage for two homes resolving the same variable through different backends, including correct home-scoped labels and an ambiguous no-home lookup returning no label.

@teknium1 teknium1 added the sweeper:blast-moderate Sweeper blast radius: moderate — a subsystem or single platform label Jul 31, 2026
@allenkaplan

Copy link
Copy Markdown
Author

Thanks for carrying forward the cold-profile fix and incorporating the .op.env bootstrap path.

Problems

  • Blocking: agent/secret_sources/registry.py:217 wraps source.fetch(cfg, home_path) in a ContextVar, but the public SecretSource API still passes no environment (agent/secret_sources/base.py:162-169). Third-party sources are supported through PluginContext.register_secret_source() (agent/secret_sources/registry.py:19-25); an existing source that reads os.environ will still receive process-global/default-profile credentials during cold multiplex hydration. The built-in source conversions do not protect that extension path.
  • hermes_cli/env_loader.py:175 writes provenance into global _SECRET_SOURCES; get_secret_source() has no home argument. Two profiles resolving the same key through different sources can produce an incorrect source label for the first profile.

Suggested changes

  • Establish an isolation-capable plugin-source contract (or skip legacy sources on multiplex hydration) and add a plugin-style regression proving direct process-environment reads cannot observe another profile.
  • Scope provenance by resolved home, matching the existing value snapshots.

Automated hermes-sweeper review.

Updated pr and responded to review comments

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area/auth Authentication, OAuth, credential pools area/config Config system, migrations, profiles area/profiles Multi-profile isolation, HERMES_HOME scoping comp/agent Core agent runtime: loop, agent_init, prompt builder, context-compression, responses endpoint comp/cli CLI entry point, hermes_cli/, setup wizard comp/gateway Gateway runner, session dispatch, delivery needs-decision Awaiting maintainer decision before any implementation P2 Medium — degraded but workaround exists sweeper:blast-moderate Sweeper blast radius: moderate — a subsystem or single platform sweeper:risk-compatibility Sweeper risk: may break existing users, config, migrations, defaults, or upgrades sweeper:risk-message-delivery Sweeper risk: may drop, duplicate, misroute, or suppress messages sweeper:risk-security-boundary Sweeper risk: may affect sandboxing, auth, credentials, or sensitive data type/bug Something isn't working

Projects

None yet

Development

Successfully merging this pull request may close these issues.

bug(secrets): cold multiplex profiles do not resolve configured external sources into their secret scope

4 participants