Skip to content

fix(tools): isolate model tools by multiplex profile - #74023

Closed
tachyon-r wants to merge 2 commits into
NousResearch:mainfrom
tachyon-r:tachyon/fix-profile-scoped-model-tool-secrets-20260729
Closed

tachyon-r wants to merge 2 commits into
NousResearch:mainfrom
tachyon-r:tachyon/fix-profile-scoped-model-tool-secrets-20260729

Conversation

@tachyon-r

Copy link
Copy Markdown
Contributor

Summary

  • resolve Discord, Home Assistant, and Camofox credentials from the active profile secret scope
  • install the default profile's runtime scope before prompt rendering on both initial connection and reconnect
  • make Home Assistant auto-enable follow the active profile token
  • isolate both model-tool availability cache layers by profile, bypass unresolved multiplex scopes, and bound the profile-dimensional caches

Problem

Several model-callable integrations still read process-global credentials in multiplex gateways. A secondary profile could therefore authenticate as the launch/default profile's Discord bot, Home Assistant instance, or Camofox account.

After making those checks profile-dependent, two process-global availability caches could also advertise profile A's tools to profile B for their TTL/grace windows. Unscoped multiplex requests must not share fallback state, and profile-dimensional caches must remain bounded.

The primary adapter also rendered its prompt before entering profile scope, and its reconnect path installed an entirely unscoped handler. That made fail-closed credential reads incorrectly hide tools from the credentialed primary profile.

Validation

  • pre-fix credential-isolation regressions failed across the affected integrations
  • 416 credential, gateway, and Camofox tests passed
  • 275 registry, model-tool, tool-search, and tools-config tests passed
  • 31 secret-scope/profile-secret tests passed
  • profile A/B, unresolved-scope, reconnect, Home Assistant auto-enable, and 1,000-profile cache-bound regressions included
  • Ruff passed on all changed Python files
  • Windows-footgun scan passed (853 files)
  • git diff --check passed
  • independent final review: APPROVE, no important findings

Compatibility

Single-profile environment fallback is unchanged. Cache bypass occurs only when multiplex mode is active without a resolvable explicit profile scope; normal profile cache entries remain TTL-bound and are capped at 512.

@alt-glitch alt-glitch added type/security Security vulnerability or hardening comp/gateway Gateway runner, session dispatch, delivery comp/cli CLI entry point, hermes_cli/, setup wizard comp/tools Tool registry, model_tools, toolsets tool/browser Browser automation (CDP, Playwright) platform/discord Discord bot adapter area/auth Authentication, OAuth, credential pools area/profiles Multi-profile isolation, HERMES_HOME scoping P2 Medium — degraded but workaround exists sweeper:risk-message-delivery Sweeper risk: may drop, duplicate, misroute, or suppress messages sweeper:risk-security-boundary Sweeper risk: may affect sandboxing, auth, credentials, or sensitive data labels Jul 29, 2026
@tachyon-r
tachyon-r force-pushed the tachyon/fix-profile-scoped-model-tool-secrets-20260729 branch from 26e0b71 to d8a73a7 Compare July 29, 2026 09:48
@egilewski

Copy link
Copy Markdown
Contributor

not enough evidence

A mergeable current-main review tree is unavailable. The bound controller recorded deterministic patch replay failure at patch_replay_conflict, and a read-only three-tree merge confirms unresolved conflicts between current main a4973c3 and PR head d8a73a774fd6a11ffc9d15bce1b46270e9a90516, including import conflicts in model_tools.py and tools/discord_tool.py plus concurrent changes in gateway and tool-registry security paths. The PR-head patch is syntactically valid and whitespace-clean, but those checks cannot establish behavior after integration. The stale/conflicted submitted branch was not treated as a standalone blocker; the missing evidence is a coherent current-main integration candidate. A maintainer-resolved replay preserving both current-main changes and the PR's profile-scoping changes, followed by executable focused tests, is the exact missing setup evidence.

Security evidence:

  • trust boundary: The intended boundary is the active multiplex profile: process-wide environment secrets and cached tool availability from one profile must not authorize, configure, advertise, or authenticate tools for another profile. Inputs include profile identity, profile-scoped secret maps, Hermes-home overrides, platform events, and per-platform tool configuration; sensitive outputs include adapter authorization, prompt claims about available tools, tool schemas, and outbound Discord, Home Assistant, and Camofox credentials.
  • source/sink/invariant: The claimed invariant is that every credential read and every check_fn/tool-definition cache decision is keyed to the active profile, while an unresolved multiplex scope fails closed or bypasses caches. The patch routes selected credential reads through get_secret, scopes primary gateway message handling, and adds profile identity to availability caches. This invariant cannot be validated against current main because the conflicting current-main imports and registry changes have not been coherently combined with the patch.
  • current-main reproduction: The current-main commit object a4973c3 is locally present. The PR branches from 0157180. A read-only three-tree merge against current main exposes overlapping changes in gateway/run.py, gateway/session.py, hermes_cli/tools_config.py, model_tools.py, tools/discord_tool.py, tools/homeassistant_tool.py, and tools/registry.py. Without a resolved tree, the reported cross-profile behavior cannot be reproduced on the exact integration candidate.
  • PR-head or patch-replay validation: PR head d8a73a774fd6a11ffc9d15bce1b46270e9a90516 is checked out. Its changed production Python files parse successfully, and its patch passes git diff --check. The controller's deterministic replay failed with patch_replay_conflict. Independent read-only merge-tree output confirms unresolved conflict markers would be required in model_tools.py and tools/discord_tool.py and shows overlapping security-boundary edits elsewhere, so PR-head success would not predict integrated behavior.
  • positive/negative cases: The patch contains proposed positive cases for a scoped profile token enabling the corresponding tool and negative cases for a missing active-profile token refusing fallback to another profile's process environment. It also proposes cache separation, unscoped-cache bypass, bounded-cache, default-profile ingress-scope, and non-multiplex compatibility cases. These cases were inspected but could not be executed because no repository pytest environment is available in the leased checkout, and they cannot substitute for tests on a coherently resolved current-main replay.
  • residual bypass search: The patch surface was searched through the complete base-to-head file list and focused diffs for gateway ingress scoping, direct environment credential reads, tool check_fn caching, model tool-definition caching, platform configuration, and Discord, Home Assistant, and Camofox consumers. Concurrent current-main edits touch the same imports and registry/tool-error contracts. Consequently, residual bypass analysis is indeterminate until the conflicts are resolved and the combined call paths can be inspected and exercised.
  • reviewer validation: Local validation established commit identities and merge base, enumerated all 14 changed files, parsed all changed production Python files, checked patch whitespace, and used a non-writing three-tree merge to confirm the controller's replay failure. No source-backed defect is published because behavior of the required current-main integration candidate is unknown. Reviewer validation requires a deterministic conflict resolution plus focused positive and negative tests in an installed offline test environment.

Uncertainty: How maintainers intend to combine current main's tool_error and registry import changes with the PR's cache-scope imports is unknown.; Whether the profile-scoped credential and cache tests pass after a coherent conflict resolution is unknown.; Whether additional current-main call paths bypass the proposed active-profile scope cannot be determined from the stale PR-head tree.; Runtime behavior was not exercised because an offline pytest-capable environment was unavailable.

Signed: GPT-5.6-sol-xhigh in Codex

@tachyon-r
tachyon-r force-pushed the tachyon/fix-profile-scoped-model-tool-secrets-20260729 branch from d8a73a7 to 0b36168 Compare July 30, 2026 05:19

@teknium1 teknium1 left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for closing the verified primary-handler, credential, and availability-cache isolation gaps.

Problems

  • tools/browser_camofox.py:194 and :199 still read CAMOFOX_USER_ID and CAMOFOX_SESSION_KEY directly from os.environ. _get_session() consumes them at :368-375, and _ensure_tab() sends them as userId and listItemId at :405-412. The PR scopes the endpoint and bearer key, but a multiplexed secondary profile can still select a process-global externally managed Camofox browser identity or tab.

Suggested changes

  • Route those identity values through the active secret scope (with the existing profile-local config fallback) and add a scoped-profile regression covering both precedence and fail-closed behavior.

Automated hermes-sweeper review.

Comment thread tools/browser_camofox.py
@@ -82,15 +83,15 @@ def _get_command_timeout() -> int:

def _auth_headers() -> Dict[str, str]:
"""Return Authorization header when CAMOFOX_API_KEY is set."""
key = os.getenv("CAMOFOX_API_KEY", "").strip()
key = (get_secret("CAMOFOX_API_KEY", "") or "").strip()
if key:
return {"Authorization": f"Bearer {key}"}
return {}


def get_camofox_url() -> str:
"""Return the configured Camofox server URL, or empty string."""

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This scopes the endpoint and bearer key, but _camofox_identity_override() still reads CAMOFOX_USER_ID and CAMOFOX_SESSION_KEY from raw process environment at current lines 194 and 199. Those values are sent as userId and listItemId when creating/adopting tabs, so they need the same active-profile resolution to prevent a secondary profile from reusing another profile's externally managed browser session.

@tachyon-r
tachyon-r force-pushed the tachyon/fix-profile-scoped-model-tool-secrets-20260729 branch from 0b36168 to 47d2938 Compare July 30, 2026 16:45
@teknium1 teknium1 added sweeper:risk-caching Sweeper risk: may break/degrade prompt caching or cache-key stability (invariant) sweeper:blast-moderate Sweeper blast radius: moderate — a subsystem or single platform labels Jul 30, 2026
@egilewski

Copy link
Copy Markdown
Contributor

suggesting changes

Two in-scope defects remain in the profile-isolation mechanism:

  1. tools/browser_camofox.py keeps both _sessions and its VNC discovery state process-global without a profile dimension. _get_session() returns a cached entry before re-reading the active scope, so the same task ID under profile B reuses profile A's user_id, session_key, and tab_id; a direct PR-head probe observed exactly that. _vnc_url and _vnc_url_checked likewise caused profile B to receive profile A's VNC URL. Key session lookup, drop, cleanup, and VNC discovery by a stable profile scope plus task ID, and cover same-task IDs and cleanup across profiles.

  2. model_tools.py performs _tool_defs_cache lookup, eviction, insertion, and clearing without a lock, while GatewayRunner constructs agents through a 10-worker executor. Two concurrent profile misses can select the same oldest entry and race pop; a controlled PR-head interleaving produced KeyError, failing one tool-definition build. Guard the complete cache operation with a dedicated lock and add concurrent regression coverage.

Security evidence:

  • trust boundary: One gateway process serves independently configured profiles. A routed turn must not reuse credentials, browser state, model-visible schemas, or endpoint metadata from another profile, and concurrent turns must not corrupt shared availability state.
  • source/sink/invariant: _profile_runtime_scope correctly installs context-local Hermes-home and secret mappings, but every process-global cache reached inside that scope must include the same stable profile identity or be bypassed; shared cache mutations must also be serialized.
  • current-main reproduction: At bound current main c9de69c6d5ed602059f5e9c9950c150e07b89212, the recorded probe reproduced profile-blind check-function caching, and source inspection showed direct primary handlers plus process-environment credential reads in the reviewed Discord, Home Assistant, and Camofox sinks.
  • PR-head or patch-replay validation: At exact PR head 47d2938f3ab649fde9c0c51a30e7e0e2c425b5a7, profile-scoped registry keys and credential precedence work, but same-task Camofox sessions and VNC state still crossed profiles, and coordinated concurrent tool-definition eviction raised KeyError.
  • positive/negative cases: Existing focused probes confirmed distinct registry availability verdicts, scoped Discord, Home Assistant, and Camofox credential precedence, fail-closed missing secrets, unresolved-scope cache bypass, and bounded registry caches; negative PR-head probes reproduced Camofox session and VNC reuse plus the model-tool cache race.
  • residual bypass search: The changed credential sinks no longer use process-environment fallbacks, but tools/browser_camofox.py lines 52-53 and 316-428 retain profile-blind process-global state, while model_tools.py lines 287-372 retain unsynchronized shared cache operations.
  • reviewer validation: git diff --check and production-module compileall passed, and focused source probes reproduced current-main aliasing and the three PR-head residual failures; the focused pytest selection could not run because this checkout has no usable pytest environment.

Uncertainty: No live Discord, Home Assistant, or Camofox service was exercised; the blocking results use deterministic in-process source and cache probes.

Signed: GPT-5.6-sol-xhigh in Codex

@teknium1

teknium1 commented Aug 2, 2026

Copy link
Copy Markdown
Collaborator

Salvaged and merged in #76573 — both your commits are on main (76cf19f + 3d9a146) with your authorship. Review verdict: every claimed unscoped read verified real on main (Discord/HASS/Camofox + the profile-blind check_fn/tool-defs caches), the get_secret routing is canonical with no parallel machinery, the (fn, profile_home) cache keying with fail-closed bypass is exactly right, and the child-inheritance check passed — cache scope rides the same ContextVar copy_context propagates to subagents. The camofox _sessions/_vnc_url process-global caches you didn't touch are tracked in #76574. Thanks @tachyon-r!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area/auth Authentication, OAuth, credential pools area/profiles Multi-profile isolation, HERMES_HOME scoping comp/cli CLI entry point, hermes_cli/, setup wizard comp/gateway Gateway runner, session dispatch, delivery comp/tools Tool registry, model_tools, toolsets P2 Medium — degraded but workaround exists platform/discord Discord bot adapter sweeper:blast-moderate Sweeper blast radius: moderate — a subsystem or single platform sweeper:risk-caching Sweeper risk: may break/degrade prompt caching or cache-key stability (invariant) sweeper:risk-message-delivery Sweeper risk: may drop, duplicate, misroute, or suppress messages sweeper:risk-security-boundary Sweeper risk: may affect sandboxing, auth, credentials, or sensitive data tool/browser Browser automation (CDP, Playwright) type/security Security vulnerability or hardening

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants