Skip to content

fix(approval): use per-job ContextVar for cron-session flag instead of leaking env var (#56771) - #56796

Open
Tranquil-Flow wants to merge 2 commits into
NousResearch:mainfrom
Tranquil-Flow:fix/56771-cron-session-leak
Open

fix(approval): use per-job ContextVar for cron-session flag instead of leaking env var (#56771)#56796
Tranquil-Flow wants to merge 2 commits into
NousResearch:mainfrom
Tranquil-Flow:fix/56771-cron-session-leak

Conversation

@Tranquil-Flow

Copy link
Copy Markdown
Contributor

What

execute_code (and dangerous-command guards) were blocked in interactive gateway/CLI/TUI sessions whenever HERMES_CRON_SESSION=1 was present in the process environment — even though the user never ran a cron job in that session (#56771).

Root cause

cron/scheduler.py set os.environ["HERMES_CRON_SESSION"] = "1" process-wide at job start and never cleared it. The approval system then gates on env_var_enabled("HERMES_CRON_SESSION") at 4 sites. When the gateway and scheduler share a process (the normal architecture), the env var leaks via inheritance into every concurrent interactive session, so the approval system treats user chats as cron and blocks execute_code / dangerous commands.

This was the only place in the codebase that set the var, and approval.py was the only consumer — so replacing the process-global set has no other side effects.

Fix

Replace the process-global env var with a task-local ContextVar so the cron flag cannot leak into concurrent interactive sessions:

  • gateway/session_context.py — add _CRON_SESSION ContextVar (mirroring the existing _UNSET sentinel pattern of the session vars) plus set_cron_session() / clear_cron_session() / is_cron_session() helpers. is_cron_session() checks the ContextVar first and falls back to the env var only for backward compat with tests / CLI cron paths that set it directly (in production the scheduler no longer sets it, so interactive sessions fall through to False).
  • cron/scheduler.py — replace the os.environ set with set_cron_session(True); add clear_cron_session() to the run_job() finally block. The flag is set before the scheduler's existing copy_context() (line 2861) that the agent thread runs inside (_cron_context.run(agent.run_conversation)), so real cron jobs still see True and cron_mode: deny keeps working.
  • tools/approval.py — add _is_cron_session() (contextvar-aware, lazy-imported like the existing get_session_env calls) and replace all 4 env_var_enabled("HERMES_CRON_SESSION") checks with it.

Why ContextVar and not env-var reordering

The scheduler sets the cron flag before spawning the agent in a worker thread via copy_context().run(...). ContextVars propagate downward through ctx.run, so the agent thread sees True; concurrent interactive sessions run in their own tasks/contexts where the var is _UNSET, so they resolve to False. By contrast os.environ is shared across all threads/process-children, which is precisely the leak this fixes. Competing approaches that merely reorder the gateway/cron precedence checks (or gate on interactive indicators) leave the env var leaking into spawned subprocesses and reverse the original design invariant that cron takes absolute priority over gateway approval.

How verified

  • 11 new regression tests (tests/tools/test_cron_session_leak.py): contextvar resolution, the core thread-isolation mechanism (scheduler thread sets the flag, concurrent gateway thread does not see it), env-var fallback for backward compat, contextvar-overrides-leaked-env, and the full check_execute_code_guard + _is_gateway_approval_context matrix (cron blocks, interactive allowed, interactive-after-scheduler-ran still allowed).
  • 325 existing approval/cron tests pass.
  • 612 scheduler tests pass.
  • Branch is exactly one focused commit ahead of main (0 1).

Closes #56771.


Auto-published by Moonsong via Path B automated pipeline.

…f leaking env var (NousResearch#56771)

The scheduler set os.environ["HERMES_CRON_SESSION"]="1" process-wide at
job start and never cleared it. When the gateway and scheduler share a
process (the normal architecture), the env var leaked into concurrent
interactive sessions, causing execute_code and dangerous commands to be
blocked for users chatting via Telegram/CLI/TUI — even though they never
ran a cron job in their session.

Replace the process-global env var with a task-local ContextVar
(_CRON_SESSION) that is set per-job in run_job() and cleared in the
finally block. The approval system checks the ContextVar first (via
is_cron_session()), falling back to the env var only for backward compat
with tests and CLI cron paths that set it directly.

Changes:
- gateway/session_context.py: add _CRON_SESSION ContextVar + helpers
  (set_cron_session / clear_cron_session / is_cron_session)
- cron/scheduler.py: replace os.environ set with set_cron_session(True);
  add clear_cron_session() to the finally block
- tools/approval.py: replace all 4 env_var_enabled("HERMES_CRON_SESSION")
  checks with _is_cron_session() (contextvar-aware with env fallback)
- tests/tools/test_cron_session_leak.py: 11 regression tests covering
  thread isolation, contextvar override, backward compat, and the full
  execute_code guard matrix
@harjothkhara

Copy link
Copy Markdown
Contributor

Recommendation: this looks like the strongest lane for the #56771 cron-session leak cluster, including the newer corroborating reports like #57736 and #60350.

Why I would pick this over the sibling PRs:

Local proof on #56796:

scripts/run_tests.sh tests/tools/test_cron_session_leak.py tests/tools/test_cron_approval_mode.py tests/tools/test_execute_code_approval_cluster.py -- -q
# 58 tests passed

git diff --check origin/main...HEAD
# clean

.venv/bin/python scripts/check-windows-footguns.py cron/scheduler.py gateway/session_context.py tools/approval.py tests/tools/test_cron_session_leak.py tests/tools/test_cron_approval_mode.py tests/tools/test_execute_code_approval_cluster.py
# pass

Two-lens read: the runtime mechanism is covered because the new test demonstrates scheduler-thread cron state does not appear in a sibling gateway thread, while real cron contexts still hit cron_mode. Architecturally, this follows the existing contextvars.copy_context() path in cron/scheduler.py instead of trying to infer interactivity from a stale process environment.

@teknium1 teknium1 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for targeting the scheduler-owned process-global marker: current main still writes os.environ["HERMES_CRON_SESSION"] in cron/scheduler.py:2812, while the approval gates consume it in tools/approval.py:241, :2173, :2700, and :3121.

Problems

  • tools/approval.py:189 now delegates cron detection to gateway.session_context.is_cron_session(). The existing cron-mode tests in tests/tools/test_request_tool_approval.py:106-107 and :116-117 mock only approval.env_var_enabled, so that mock no longer controls the cron branch. Those tests need to set the environment marker or mock _is_cron_session.

Suggested changes

  • Update both affected tests and include tests/tools/test_request_tool_approval.py in the focused validation set.

Automated hermes-sweeper review.

Comment thread tools/approval.py
try:
from gateway.session_context import is_cron_session

return is_cron_session()

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This bypasses the approval.env_var_enabled mock used by tests/tools/test_request_tool_approval.py:106-107 and :116-117; those two cron-mode tests will no longer simulate cron in a clean environment. Please update them to set HERMES_CRON_SESSION or mock _is_cron_session.

@teknium1 teknium1 added sweeper:risk-compatibility Sweeper risk: may break existing users, config, migrations, defaults, or upgrades sweeper:blast-moderate Sweeper blast radius: moderate — a subsystem or single platform labels Jul 15, 2026
…arch#56796)

Sweeper feedback: update affected tests to test ContextVar-based
approach instead of old env-var approach.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area/sessions Session lifecycle, resume, persistence, history comp/agent Core agent runtime: loop, agent_init, prompt builder, context-compression, responses endpoint comp/cron Cron scheduler and job management comp/gateway Gateway runner, session dispatch, delivery P2 Medium — degraded but workaround exists sweeper:blast-moderate Sweeper blast radius: moderate — a subsystem or single platform sweeper:risk-compatibility Sweeper risk: may break existing users, config, migrations, defaults, or upgrades sweeper:risk-security-boundary Sweeper risk: may affect sandboxing, auth, credentials, or sensitive data type/bug Something isn't working

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Bug]: execute_code blocked in interactive Telegram gateway session because HERMES_CRON_SESSION env var leaks from cron scheduler into user's shell

4 participants