Skip to content

fix(cron): scrub Kanban worker env from non-dispatcher subprocess spawns - #87756

Open
QDung210 wants to merge 2 commits into
NousResearch:mainfrom
QDung210:fix/cron-kanban-env-subprocess-scrub
Open

fix(cron): scrub Kanban worker env from non-dispatcher subprocess spawns#87756
QDung210 wants to merge 2 commits into
NousResearch:mainfrom
QDung210:fix/cron-kanban-env-subprocess-scrub

Conversation

@QDung210

@QDung210 QDung210 commented Aug 16, 2026

Copy link
Copy Markdown
Contributor

What does this PR do?

Closes a Kanban-authority leak in cron's non-dispatcher isolation. Cron jobs can be fired in-process from inside a Kanban worker (cronjob(action="run") calls run_job() in the worker's own process, where HERMES_KANBAN_* is legitimately set). run_job() marks itself non-dispatcher-owned via enter_non_dispatcher_owned_context() / is_dispatcher_owned_worker_context() so it isn't misidentified as the worker in-process — but that identity fix never extended to subprocesses the cron job spawns.

Root cause: _scrub_delegated_child_kanban_env() in tools/environments/local.py (the sole chokepoint for build_subprocess_env() / hermes_subprocess_env()) only ever checked the older is_delegated_child_process_context() (for delegate_task children). It was never migrated to also check is_dispatcher_owned_worker_context() — "the single predicate every HERMES_KANBAN_* identity gate should use" per its own docstring — after that predicate was introduced. So any subprocess a non-dispatcher-owned cron job spawned (script jobs, terminal calls, browser/tts/lazy-deps helpers, etc.) still inherited the worker's HERMES_KANBAN_* env for its full duration.

On top of that, two specific spawn sites in cron/scheduler.py's run_job() — the no_agent script-launch path and the wake-gate prerun-script path — build their subprocess env before the existing enter_non_dispatcher_owned_context() call further down in the function, so even a corrected predicate wouldn't have covered them without also wrapping those two call sites explicitly.

Fix:

  1. tools/environments/local.py: _scrub_delegated_child_kanban_env() now also scrubs when not is_dispatcher_owned_worker_context(), alongside the existing delegated-child check.
  2. cron/scheduler.py: wrap the two early script-launch call sites with non_dispatcher_owned_context() so their subprocess env is built while the marker is active.

No behavior changes to any dispatcher-owned (normal worker) or delegated-child path — the new condition only fires when one of the two ContextVars is actually set.

Note for maintainers: open PR #81843 edits the same _scrub_delegated_child_kanban_env function (for a different, unrelated gap — terminal-spawned subprocess isolation from delegate_task children). The two patches don't touch the same lines but whichever lands second will need a small rebase.

Related Issue

Fixes #87725

Type of Change

  • 🐛 Bug fix (non-breaking change that fixes an issue)
  • ✨ New feature (non-breaking change that adds functionality)
  • 🔒 Security fix
  • 📝 Documentation update
  • ✅ Tests (adding or improving test coverage)
  • ♻️ Refactor (no behavior change)
  • 🎯 New skill (bundled or hub)

Changes Made

  • tools/environments/local.py: _scrub_delegated_child_kanban_env() now scrubs Kanban env when is_dispatcher_owned_worker_context() is False, not just for delegated children.
  • cron/scheduler.py: wrap the no_agent script-launch call and the wake-gate prerun-script call in non_dispatcher_owned_context().
  • tests/cron/test_cron_kanban_env_isolation.py: new TestSubprocessEnvScrub class — 6 tests covering the direct predicate gap (build_subprocess_env/hermes_subprocess_env inside non_dispatcher_owned_context()), the pre-existing delegated-child path (unaffected), and both run_job() script-launch sites via a fake _run_job_script that captures the env it was actually given.

How to Test

  1. Reproduction (pre-fix): inside non_dispatcher_owned_context(), call tools.environments.local.build_subprocess_env()HERMES_KANBAN_TASK/_RUN_ID/_CLAIM_LOCK/etc. are present in the returned env despite the context marking this execution as not dispatcher-owned.
  2. RED → GREEN: pytest tests/cron/test_cron_kanban_env_isolation.py -q -k TestSubprocessEnvScrub — 4 of the 6 new tests fail against the pre-fix code (verified by stashing the two production-file changes) and pass after the fix; the other 2 assert unchanged baseline behavior (dispatcher-owned and delegated-child paths still work as before).
  3. Full regression run: pytest tests/cron/test_cron_kanban_env_isolation.py -q → 24 passed.

Checklist

Code

  • I've read the Contributing Guide
  • My commit messages follow Conventional Commits (fix(scope):, feat(scope):, etc.)
  • I searched for existing PRs to make sure this isn't a duplicate
  • My PR contains only changes related to this fix/feature (no unrelated commits)
  • I've run pytest tests/cron/test_cron_kanban_env_isolation.py -q and all tests pass (24 passed). I also ran pytest tests/cron/ -q (690 passed, 20 skipped, 7 pre-existing failures unrelated to this change — Windows POSIX-permission assertions in test_file_permissions.py and a tilde-expansion assertion in test_cron_workdir.py, all reproduced identically on origin/main without this patch) and pytest tests/tools/test_delegate.py tests/tools/test_delegate_kanban_isolation.py tests/tools/test_delegate_cron_sync_fallback.py tests/test_delegate_cascade_49148.py tests/agent/test_subprocess_env_guard.py tests/cron/test_cron_script.py tests/tools/test_build_subprocess_env.py tests/tools/test_env_passthrough.py -q (133 passed, 1 skipped). I did not run the full tests/ suite (not attempted; only the modules above and directly-adjacent ones).
  • I've added tests for my changes (required for bug fixes, strongly encouraged for features)
  • I've tested on my platform: Windows 11

Documentation & Housekeeping

  • I've updated relevant documentation (README, docs/, docstrings) — or N/A (docstring on the changed function updated; no user-facing docs affected)
  • I've updated cli-config.yaml.example if I added/changed config keys — or N/A (no config keys changed)
  • I've updated CONTRIBUTING.md or AGENTS.md if I changed architecture or workflows — or N/A (no architecture/workflow change)
  • I've considered cross-platform impact (Windows, macOS) per the compatibility guide — ran scripts/check-windows-footguns.py against the changed files, no findings
  • I've updated tool descriptions/schemas if I changed tool behavior — or N/A (no tool schema changed)

Update (2026-08-16): addressed AI review feedback

  • Confirmed, no code change: non_dispatcher_owned_context() restores via ContextVar.reset(token) in a finally, and the two new wraps fully exit before the existing later token-based enter_non_dispatcher_owned_context() call — they're sequential, not nested, so there's no token-restore hazard. Added an explicit comment saying so at both wrap sites.
  • Confirmed, no code change: scrub_kanban_env() only ever pops the KANBAN_ENV_KEYS tuple and sets the delegated-child marker — nothing else. In a plain (non-cron, non-delegated) process is_dispatcher_owned_worker_context() defaults to True, so the new not is_dispatcher_owned_worker_context() condition is False and normal subprocess spawns take the same no-op path as before.
  • Added: a new assertion in both test_no_agent_script_subprocess_env_is_scrubbed and test_prerun_script_subprocess_env_is_scrubbed that is_dispatcher_owned_worker_context() is False during the script call itself (not just inferred from the scrubbed env), confirming the wrap covers the whole _run_job_script_with_claim_heartbeat call as requested.
  • Not changed: the "extract a single wrapper" suggestion — two with non_dispatcher_owned_context(): one-liners at genuinely different call sites didn't seem worth an extraction; happy to do it if a maintainer prefers.

Screenshots / Logs

N/A

`_scrub_delegated_child_kanban_env()` in tools/environments/local.py only
checked `is_delegated_child_process_context()`, never the unifying
`is_dispatcher_owned_worker_context()` predicate added for cron isolation.
A cron job fired in-process from inside a Kanban worker is marked
non-dispatcher-owned in-process, but every subprocess it spawned (via
build_subprocess_env / hermes_subprocess_env) still inherited the worker's
HERMES_KANBAN_* env, letting a cron subprocess act with the worker's
board/task authority.

Also wrap the two subprocess-launching call sites in cron/scheduler.py's
run_job() (the no_agent script path and the wake-gate prerun-script path)
with non_dispatcher_owned_context(), since both build their subprocess env
before the existing marker is entered later in the function.

Fixes NousResearch#87725
@alt-glitch alt-glitch added type/bug Something isn't working comp/cron Cron scheduler and job management backend/local Local shell execution P3 Low — cosmetic, nice to have labels Aug 16, 2026
@Enough1122

Copy link
Copy Markdown
Contributor

AI code review — automated review for reference, author can ignore or act on any point.

fix(cron): scrub Kanban worker env from non-dispatcher subprocess spawns

  1. cron/scheduler.py — the script-launch paths enter non_dispatcher_owned_context() explicitly, while run_job() also enters the same context further down for the agent path. Verify non_dispatcher_owned_context restores the previous value on exit (ContextVar token semantics) rather than unconditionally clearing — otherwise the outer agent-path marker could be lost after the inner script block exits in a nested entry.
  2. tools/environments/local.py — the scrub condition is now is_delegated_child_process_context() or not is_dispatcher_owned_worker_context(). In a plain (non-kanban) gateway/CLI process, is_dispatcher_owned_worker_context() is presumably False, so every subprocess env goes through scrub_kanban_env — confirm that function only strips HERMES_KANBAN_* keys and nothing else, so normal subprocesses see a no-op.
  3. The integration tests monkeypatch _run_job_script and assert the env is scrubbed before the marker is entered later in run_job — good coverage of the ordering issue. Consider one test asserting the _run_job_script_with_claim_heartbeat path itself runs inside the context.
  4. Minor: two call sites now wrap the same helper; extracting a single wrapper (or entering the context once at the top of run_job's script handling) would avoid future drift.

…clarify comments

Addresses AI review on NousResearch#87756:
- Add explicit assertions that is_dispatcher_owned_worker_context() is False
  DURING the no_agent/prerun script execution itself (not just inferred from
  the scrubbed env), confirming the wrap covers the whole
  _run_job_script_with_claim_heartbeat call.
- Clarify the wrap comments: the agent path's tool loop was already correctly
  covered by the existing later marker; only the two early script-launch call
  sites were the gap. Note explicitly that the two `with` blocks fully exit
  (ContextVar reset) before that later token-based entry, so they are
  sequential, not nested — no token-restore ordering hazard.

No production behavior change; scrub_kanban_env() strips only HERMES_KANBAN_*
keys (confirmed, unchanged) so dispatcher-owned subprocesses are unaffected.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

backend/local Local shell execution comp/cron Cron scheduler and job management P3 Low — cosmetic, nice to have type/bug Something isn't working

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Cron subprocesses inherit Kanban worker authority despite non-dispatcher isolation

3 participants