Skip to content

fix(gateway): stop VIRTUAL_ENV leaking into agent subprocesses - #29980

Closed
TheoLong wants to merge 1 commit into
NousResearch:mainfrom
TheoLong:upstream/gateway-virtualenv-leak
Closed

fix(gateway): stop VIRTUAL_ENV leaking into agent subprocesses#29980
TheoLong wants to merge 1 commit into
NousResearch:mainfrom
TheoLong:upstream/gateway-virtualenv-leak

Conversation

@TheoLong

Copy link
Copy Markdown

Problem

When hermes-gateway is launched from a venv (very common: systemd user services, uv run, ad-hoc dev shells), VIRTUAL_ENV is inherited by every agent subprocess. That pins the agent's Python resolution to the gateway's venv instead of the user's project venv — tool resolution breaks, ModuleNotFoundErrors surface in tasks that work fine from the user's own shell.

Fix

  • Unset VIRTUAL_ENV (and PYTHONHOME, PYTHONPATH) in the gateway service launcher before spawning agents.
  • Extend _HERMES_PROVIDER_ENV_BLOCKLIST in tools/environments/local.py so terminal-tool subprocesses get the same treatment.

Test

tests/hermes_cli/test_gateway_service.py covers the unset behaviour.

Risk

Low. Pure subtractive — strips an env var that shouldn't be propagating in the first place. Anyone relying on inherited VIRTUAL_ENV in agent subprocesses (rare) would already be using the wrong venv.

@alt-glitch alt-glitch added type/bug Something isn't working P2 Medium — degraded but workaround exists comp/gateway Gateway runner, session dispatch, delivery backend/local Local shell execution labels May 21, 2026
Three independent layers shipped on main today silently corrupt a user's
Hermes venv: the systemd unit template sets VIRTUAL_ENV explicitly, a
gateway-service test asserts it must be present, and the subprocess env
blocklist in tools/environments/local.py has no entry for it. The result
is that any agent terminal call that runs 'uv sync', 'uv run --active',
'pip install', or 'poetry install' from a user project rebuilds
$VIRTUAL_ENV against that project's pyproject.toml, wiping every Hermes
runtime dependency.

Reproduced 2026-05-16: 'cd ~/Git/fb-operator && uv run --active fb obs'
removed ~/.hermes/hermes-agent/.venv (Python 3.11) and recreated it as
Python 3.12 with only fb-operator's 11 deps. Gateway restart then
ModuleNotFoundError: hermes_cli.

Fix (defense in depth):
1. hermes_cli/gateway.py: drop both 'Environment="VIRTUAL_ENV=..."' lines
   from the systemd unit templates. Gateway invokes python by absolute
   path; sys.prefix resolves from the interpreter location without
   VIRTUAL_ENV being set.
2. tests/hermes_cli/test_gateway_service.py: flip the assertion to
   'VIRTUAL_ENV not in unit' with an explanatory comment.
3. tools/environments/local.py: add VIRTUAL_ENV, VIRTUAL_ENV_PROMPT,
   UV_PROJECT_ENVIRONMENT, POETRY_ACTIVE, PIPENV_ACTIVE, CONDA_PREFIX,
   CONDA_DEFAULT_ENV to _HERMES_PROVIDER_ENV_BLOCKLIST so the case where
   a user invokes hermes interactively with VIRTUAL_ENV set in their
   shell is also covered.
@TheoLong
TheoLong force-pushed the upstream/gateway-virtualenv-leak branch from 9a99495 to 7e19b0d Compare June 8, 2026 06:51
@TheoLong

TheoLong commented Jun 8, 2026

Copy link
Copy Markdown
Author

Rebased on current main (1,824 commits absorbed). Carry still applies cleanly — clean replay, no conflicts.

Verified still needed: Environment="VIRTUAL_ENV={venv_dir}" lines are still in the systemd unit templates upstream (hermes_cli/config.py ~L2418, L2453), and the tools/environments/local.py blocklist still has no entry for VIRTUAL_ENV family vars. The leak path is unchanged. Bumping for review.

@TheoLong

Copy link
Copy Markdown
Author

Closing as resolved on main.

Since this PR was opened, upstream removed the VIRTUAL_ENV injection from the systemd unit templates (hermes_cli/config.py no longer emits Environment="VIRTUAL_ENV=…") and added a provider env-scoping system — _HERMES_PROVIDER_ENV_BLOCKLIST in tools/environments/local.py plus agent/secret_scope.py (security advisory GHSA-rhgp-j443-p4rf). The specific leak path this PR targeted no longer exists on current main.

My earlier "still needed" bump was accurate against a main ~1,800 commits behind current; it's since been superseded. Thanks for the review patience.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

backend/local Local shell execution comp/gateway Gateway runner, session dispatch, delivery P2 Medium — degraded but workaround exists type/bug Something isn't working

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants