Repository navigation
fix(send_message): HERMES_CRON_SESSION ContextVar isolation (stop in-process cron flag latch leaking interactive sends to home) - #88
Conversation
🔎 Lint report:
|
| Rule | Count |
|---|---|
unresolved-import |
2 |
invalid-assignment |
1 |
unresolved-attribute |
1 |
First entries
tests/run_agent/test_credits_notices_toggle.py:76: [invalid-assignment] invalid-assignment: Object of type `None` is not assignable to attribute `_credits_session_start_micros` of type `int`
tests/tools/test_cron_subagent_session.py:24: [unresolved-import] unresolved-import: Cannot resolve imported module `pytest`
tests/gateway/test_cron_session_contextvar.py:19: [unresolved-import] unresolved-import: Cannot resolve imported module `pytest`
tools/delegate_tool.py:1352: [unresolved-attribute] unresolved-attribute: Unresolved attribute `_is_cron_child` on type `AIAgent`
✅ Fixed issues (2):
| Rule | Count |
|---|---|
unresolved-attribute |
2 |
First entries
tests/run_agent/test_credits_notices_toggle.py:76: [unresolved-attribute] unresolved-attribute: Unresolved attribute `_credits_session_start_micros` on type `AIAgent`
run_agent.py:2940: [unresolved-attribute] unresolved-attribute: Object of type `Self@get_credits_spent_micros` has no attribute `_credits_session_start_micros`
Unchanged: 5936 pre-existing issues carried over.
Diagnostics are surfaced as warnings — this check never fails the build.
|
| Filename | Overview |
|---|---|
| gateway/session_context.py | Adds _CRON_SESSION ContextVar with correct _UNSET default, registered in _VAR_MAP, and exposes set/clear/is_cron_session() helpers; exception path in clear_cron_session correctly restores _UNSET (not "") to preserve os.environ fallback. |
| cron/scheduler.py | Replaces process-global os.environ write with set_cron_session() before copy_context() and clear_cron_session() in finally; follows the identical pre-existing pattern for _ctx_tokens. |
| tools/delegate_tool.py | Captures _is_cron_child at spawn with inner try/except, adds _bind_child_cron_session/_clear_child_cron_session helpers, and wires them into _run_with_thread_capture alongside the existing send-origin rebind. |
| tools/approval.py | Adds _is_cron_session() lazy-import wrapper with an error-level log on the fallback path; migrates all 4 env_var_enabled("HERMES_CRON_SESSION") call sites. |
| tools/send_message_tool.py | Migrates both env_var_enabled("HERMES_CRON_SESSION") calls in _has_messaging_origin() and _warn_unexpected_home_fallback() to the context-aware is_cron_session(). |
| tests/gateway/test_cron_session_contextvar.py | Covers thread isolation, copy_context inheritance, env fallback, bad-token exception path, nestability, _VAR_MAP registration gate, and the end-to-end send-target resolution after a cron cycle. |
| tests/tools/test_cron_subagent_session.py | RED-control test proves rebind is load-bearing; positive tests assert dangerous-cmd and execute_code stay BLOCKED inside a cron child; negative test confirms interactive-parent children are not over-marked. |
Sequence Diagram
%%{init: {'theme': 'neutral'}}%%
sequenceDiagram
participant GT as Gateway Process
participant CT as Cron Worker Thread
participant IT as Interactive Turn
participant DT as Delegate Child Thread
Note over GT: Before fix: os.environ["HERMES_CRON_SESSION"]="1" latched forever
rect rgb(200, 240, 200)
Note over CT: After fix — cron job lifecycle
CT->>CT: set_cron_session() → token (ContextVar only)
CT->>CT: copy_context() — main agent inherits marker
CT->>CT: "agent runs, tool calls — is_cron_session()=True"
CT->>DT: delegate_task() — bare ThreadPoolExecutor (no ContextVar inheritance)
Note over DT: _build_child_agent captures _is_cron_child=True
DT->>DT: _bind_child_cron_session(child) → set_cron_session()
DT->>DT: "check_dangerous_command → is_cron_session()=True → BLOCKED"
DT->>DT: finally: _clear_child_cron_session(token)
CT->>CT: finally: clear_cron_session(token) → ContextVar reset to _UNSET
end
rect rgb(200, 220, 255)
Note over IT: Concurrent interactive turn (same process, different context)
IT->>IT: "is_cron_session() → ContextVar=_UNSET → os.environ unset → False"
IT->>IT: "_has_messaging_origin() → cron=False → check session platform"
IT->>IT: _resolve_send_target() → ORIGIN (current channel)
end
Note over GT: is_cron_session() reads ContextVar first,<br/>falls back to os.environ for CLI/test back-compat
%%{init: {'theme': 'base', 'themeVariables': {"darkMode": true, "background": "#0d1117", "primaryColor": "#21262d", "primaryTextColor": "#e6edf3", "primaryBorderColor": "#8b949e", "lineColor": "#8b949e", "textColor": "#e6edf3", "edgeLabelBackground": "#161b22", "actorBkg": "#21262d", "actorBorder": "#8b949e", "actorTextColor": "#e6edf3", "actorLineColor": "#8b949e", "signalColor": "#8b949e", "signalTextColor": "#e6edf3", "noteBkgColor": "#373320", "noteBorderColor": "#d4a72c", "noteTextColor": "#f0e6c0", "labelBoxBkgColor": "#21262d", "labelBoxBorderColor": "#8b949e", "labelTextColor": "#e6edf3", "loopTextColor": "#e6edf3", "activationBkgColor": "#30363d", "activationBorderColor": "#8b949e"}}}%%
sequenceDiagram
participant GT as Gateway Process
participant CT as Cron Worker Thread
participant IT as Interactive Turn
participant DT as Delegate Child Thread
Note over GT: Before fix: os.environ["HERMES_CRON_SESSION"]="1" latched forever
rect rgb(200, 240, 200)
Note over CT: After fix — cron job lifecycle
CT->>CT: set_cron_session() → token (ContextVar only)
CT->>CT: copy_context() — main agent inherits marker
CT->>CT: "agent runs, tool calls — is_cron_session()=True"
CT->>DT: delegate_task() — bare ThreadPoolExecutor (no ContextVar inheritance)
Note over DT: _build_child_agent captures _is_cron_child=True
DT->>DT: _bind_child_cron_session(child) → set_cron_session()
DT->>DT: "check_dangerous_command → is_cron_session()=True → BLOCKED"
DT->>DT: finally: _clear_child_cron_session(token)
CT->>CT: finally: clear_cron_session(token) → ContextVar reset to _UNSET
end
rect rgb(200, 220, 255)
Note over IT: Concurrent interactive turn (same process, different context)
IT->>IT: "is_cron_session() → ContextVar=_UNSET → os.environ unset → False"
IT->>IT: "_has_messaging_origin() → cron=False → check session platform"
IT->>IT: _resolve_send_target() → ORIGIN (current channel)
end
Note over GT: is_cron_session() reads ContextVar first,<br/>falls back to os.environ for CLI/test back-compat
Reviews (2): Last reviewed commit: "fix(send_message): address Greptile P2s ..." | Re-trigger Greptile
…e in-process cron flag latch leaking interactive sends to home Root cause (PRD v3): the gateway ticks the cron scheduler IN-PROCESS. The first cron job set os.environ["HERMES_CRON_SESSION"]="1" process-globally and never cleared it, so every later interactive turn read cron=True -> send_message origin resolver took the cron-guard-first branch -> NOT_IN_TURN -> home channel. Caught live on a main turn (gateway PID 95240, v2-deployed); the v2 subagent fix was real but secondary. Fix: - gateway/session_context.py: HERMES_CRON_SESSION is now a task/thread-isolated ContextVar (registered in _VAR_MAP) with set_cron_session/clear_cron_session (token-based, nestable) and a context-aware is_cron_session() reader (ContextVar -> os.environ fallback for CLI/standalone/test back-compat). - cron/scheduler.py: run_job sets the marker via the ContextVar (BEFORE the per-job copy_context so the main agent inherits it) and resets it in finally. No more process-global os.environ write. - tools/approval.py + tools/send_message_tool.py: the 6 readers now use the context-aware is_cron_session() (approval keeps cron deny-gating; send resolver + tripwire no longer poisoned by a latched flag). - tools/delegate_tool.py (B1, Opus Pass-1 blocker): cron jobs spawn subagents in a bare ThreadPoolExecutor that does NOT inherit contextvars. Capture cron-ness at spawn (_is_cron_child) and re-bind it in the child-run wrapper (_bind_child_cron_session) so cron subagents KEEP their approval deny-gating — without this, contextvar-only would have flipped cron subagents to auto-approve dangerous commands/execute_code (a security regression introduced by the fix). Tests (RED-proven where load-bearing): tests/gateway/test_cron_session_contextvar.py (thread isolation, copy_context inheritance, env fallback, interactive-not-poisoned after a cron cycle, cron-guard-first preserved) + tests/tools/test_cron_subagent_session.py (cron child dangerous-cmd + execute_code STILL BLOCKED; RED control proves the rebind is load-bearing; interactive parent child not over-marked; reset after run). 64 tests green incl. all back-compat suites; grep gates clean (no prod os.environ writer, no prod raw reader). Pre-existing unrelated reds: test_hardline_blocklist (reboot detection) + TestDelegateHeartbeat (timing) both fail on clean fork/main. Spec: docs/specs/PRD-send-message-origin-leak-v3-cron-session-latch.md (Opus Pass-1 folded).
… is the dominant leak cause; Opus Pass-1 folded (B1 cron-subagent approval regression → Phase 5 rebind) (cherry picked from commit 1e1903e36f00599f899cc88a9f86a8935c330947)
345d2e0 to
d7730e1
Compare
…est cron-backstop docstring - session_context.clear_cron_session: on a cross-context reset() failure, restore _UNSET (not "") so get_session_env still falls through to os.environ — setting "" pinned a non-_UNSET value and silently defeated the I5 back-compat fallback. + test_clear_with_bad_token_restores_env_fallback. - approval._is_cron_session: correct the docstring — the os.environ except-fallback is NOT a reliable backstop for in-process gateway cron now that run_job no longer writes os.environ; the import failure is effectively impossible, so log loudly rather than claim cron gating is "never silently lost".
Closes the dominant, proven cause of the recurring send_message origin-leak (interactive turns landing in the home channel). v1 (#71) and v2 (#83) were real but secondary.
Root cause (proven live)
The gateway ticks the cron scheduler in-process (gateway/run.py:17161). The first cron job set
os.environ["HERMES_CRON_SESSION"]="1"process-globally (cron/scheduler.py:1586) and never cleared it (only TERMINAL_CWD + ContextVars are reset in the finally). So after the first cron tick, every interactive turn read cron=True -> send_message's_has_messaging_origin()cron-guard returned False first -> NOT_IN_TURN -> home channel.Caught live on a main turn (gateway PID 95240, already running v2): a terminal subprocess showed
HERMES_CRON_SESSION=1in the gateway process, and a bare send_message returned "Sent to discord home channel" while the user was in #fix-issues. The cron-session marker was the latch; this also explains the "main-turn seam" v2 could not prove.Fix
HERMES_CRON_SESSIONis now a task/thread-isolated ContextVar (registered in_VAR_MAP), with token-basedset_cron_session/clear_cron_session(nestable) and a context-awareis_cron_session()reader (ContextVar -> os.environ fallback for CLI/standalone-scheduler/test back-compat).is_cron_session(). Approval keeps cron deny-gating; the send resolver + tripwire are no longer poisoned by a latched flag._is_cron_child) and re-binding it in the child-run wrapper (_bind_child_cron_session), mirroring v2's send-origin rebind.Tests (RED-proven where load-bearing)
tests/gateway/test_cron_session_contextvar.py— thread isolation, copy_context inheritance, env fallback, interactive-not-poisoned after a cron cycle, cron-guard-first preserved.tests/tools/test_cron_subagent_session.py— cron child dangerous-cmd + execute_code STILL BLOCKED; a RED control proves the rebind is load-bearing (without it -> auto-approve); interactive parent's child not over-marked; marker reset after the child run.os.environ["HERMES_CRON_SESSION"]=writer, no prod rawenv_var_enabled("HERMES_CRON_SESSION")reader.Spec:
docs/specs/PRD-send-message-origin-leak-v3-cron-session-latch.md(Opus Pass-1 folded — B1 blocker -> Phase 5 rebind, B2 fake-green -> real subagent deny test, registration gate, ordering constraint).