fix(cron): pop HERMES_CRON_SESSION env var after last cron job finishes - #69740
Closed
im47cn wants to merge 1 commit into
Closed
fix(cron): pop HERMES_CRON_SESSION env var after last cron job finishes#69740im47cn wants to merge 1 commit into
im47cn wants to merge 1 commit into
Conversation
The cron scheduler sets os.environ['HERMES_CRON_SESSION'] = '1' in run_job() to gate approval checks on cron_mode. This is process-global and never cleaned up — when the scheduler runs inside a gateway process (InProcessCronScheduler), the flag leaks into user-interactive sessions. Affected tools blocked by the stale flag: execute_code, web_request, and any tool gated on check_execute_code_guard or check_all_command_guards when cron_mode defaults to 'deny'. Fix: in run_job's finally block, use _running_job_ids as a reference counter — pop the env var only when no other cron jobs are still executing. This is safe under _running_lock which already serializes concurrent job tracking. Closes NousResearch#73195.
Author
|
Closing in favor of a root-cause fix. The reference-counting approach is a band-aid: the check-then-pop window still races under concurrent job execution. The root cause is that HERMES_CRON_SESSION uses process-global os.environ while the rest of the session state (HERMES_SESSION_PLATFORM etc.) already migrated to task-local ContextVars. Will submit a proper ContextVar migration. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
When the cron scheduler runs inside a gateway process (
InProcessCronScheduler, the default),os.environ["HERMES_CRON_SESSION"] = "1"is set inrun_job()and never cleaned up. Sinceos.environis process-global, the flag leaks into user-interactive sessions handled by the same gateway process.Result:
execute_code,web_request, and other tools gated oncheck_execute_code_guardorcheck_all_command_guardsare blocked with:Users replying to cron-delivered messages (e.g. Feishu/Telegram threads) get their session incorrectly classified as a cron session.
Root Cause
cron/scheduler.py:2920:Comment on line 2918: "This env var is process-wide and persists for the lifetime of the scheduler process — every job this process runs is a cron job."
The intent was for a dedicated scheduler process — but the gateway co-locates the scheduler. Once any cron job runs, every subsequent user session in the same gateway process inherits
HERMES_CRON_SESSION=1.Fix
In
run_job()'sfinallyblock, use_running_job_idsas a reference counter — popHERMES_CRON_SESSIONonly when no other cron jobs are still executing:This is safe under
_running_lockwhich already serialises concurrent job tracking.Tests
Added
tests/cron/test_cron_session_env_cleanup.py(4 tests):All 788 existing cron + approval tests pass without modification.
Evidence
Confirmed in production gateway process (2026-07-23 BJT):
User session (Feishu gateway, 08:45 BJT) blocked 3 consecutive
execute_codecalls with the cron-deny error.