Skip to content

fix(agent): catch cross-agent turn overlaps in the tripwire (#64934) - #67337

Closed
Hotragn wants to merge 1 commit into
NousResearch:mainfrom
Hotragn:fix/64934-tripwire-session-scoped
Closed

fix(agent): catch cross-agent turn overlaps in the tripwire (#64934)#67337
Hotragn wants to merge 1 commit into
NousResearch:mainfrom
Hotragn:fix/64934-tripwire-session-scoped

Conversation

@Hotragn

@Hotragn Hotragn commented Jul 19, 2026

Copy link
Copy Markdown
Contributor

What does this PR do?

Makes the #64934 turn-overlap tripwire able to see the overlap route it was merged to identify.

note_turn_start (merged in #65499) keeps its in-flight marker on the agent object. But the gateway caches agents per routing key (_agent_cache[session_key], gateway/run.py:19815), while the durable transcript is owned by session_id — and the key→id mapping is many-to-one on current main: SessionStore.switch_session() (gateway/session.py:2394) re-points a routing key at any target session with no check that another live key already routes there. Shipped callers that create this state: /resume of a named session from a second surface, CLI-continuity rebinding (gateway/run.py:8075-8079), async-delegation completion pinning (gateway/run.py:11800, #57498), and the Telegram topic-binding compression-tip walk (gateway/run.py:11828-11852).

Once two routing keys map to one session_id, every guard in the dispatch path (_active_sessions in the adapter, _running_agents in the runner — both keyed by routing key) admits a second concurrent turn on the same transcript, and the two turns run on two different agent objects — so the per-agent tripwire stays silent no matter how long production runs. The "wait for the tripwire to name the route" plan on #64934 cannot converge on this route without this change.

This PR adds a module-level, session_id-keyed in-flight registry alongside the per-agent marker. Same design contract as the original tripwire: log-only (no behavior change to dispatch or persistence), takes ownership of the slot on overlap, and under-reports rather than double-reports — a same-agent overlap still warns exactly once, and the first persist clears the slot unconditionally. The persist-time clear pops the session id stamped at turn start, so a mid-turn compression rotation of agent.session_id can't strand the slot or misfire on the rotated id.

Full route analysis with a runnable reproduction (real SessionStore against a temp HERMES_HOME) is in my comment on #64934.

Related Issue

Part of #64934 (diagnostic half — the issue stays open for the serialization fix). Extends #65499.

Type of Change

  • 🐛 Bug fix (non-breaking change that fixes an issue)

Changes Made

  • agent/agent_runtime_helpers.py — add _INFLIGHT_TURNS_BY_SESSION registry + lock; note_turn_start gains a cross-agent leg that warns when a different in-flight turn holds the same session_id under another agent object (dedup'd against the per-agent warning); note_turn_persisted clears the registry slot via the session id stamped at turn start.
  • tests/agent/test_turn_overlap_tripwire.py — 6 new behavior tests: cross-agent overlap warns with both turn ids; serial cross-agent turns are silent; distinct sessions never cross-warn; same-agent overlap does not double-report; mid-turn session rotation still releases the slot; crashed turn warns once then recovers. Plus an autouse fixture isolating the module registry between tests.

How to Test

  1. scripts/run_tests.sh tests/agent/test_turn_overlap_tripwire.py — 10 passed. Also ran tests/agent/ tests/run_agent/ tests/hermes_state/ (415 files) on native Windows and diffed the failing-test set against a pristine main worktree: identical (the failures are this machine's pre-existing native-Windows set — LSP tooling, shell hooks, sandbox-mirror tests; CI is Linux). Zero failures attributable to this change.
  2. Repro of the underlying route (shows why the per-agent tripwire can't fire): see the script in my Two turns can run concurrently on one gateway session: interleaved transcript flushes, permanent alternation wedge, repair_message_sequence fires on every request #64934 comment — with two routing keys switched onto one session_id, note_turn_start on two agent objects previously produced no warning; with this PR it names both turn ids and the shared session.

Checklist

Code

  • I've read the Contributing Guide
  • My commit messages follow Conventional Commits (fix(scope):, feat(scope):, etc.)
  • I searched for existing PRs to make sure this isn't a duplicate
  • My PR contains only changes related to this fix/feature (no unrelated commits)
  • I've run the test suite (scripts/run_tests.sh tests/agent/ tests/run_agent/ tests/hermes_state/) — failing set on native Windows is byte-identical to pristine main (pre-existing platform failures only; the changed file's suite is 10/10)
  • I've added tests for my changes (required for bug fixes, strongly encouraged for features)
  • I've tested on my platform: Windows 11 (native)

Documentation & Housekeeping

  • I've updated relevant documentation — N/A (log-only diagnostic; docstrings updated in place)
  • I've updated cli-config.yaml.example if I added/changed config keys — N/A
  • I've updated CONTRIBUTING.md or AGENTS.md if I changed architecture or workflows — N/A
  • I've considered cross-platform impact (Windows, macOS) — pure-Python threading/time, no platform surface; developed and tested on native Windows

…arch#64934)

note_turn_start kept its in-flight marker on the agent object, but the
gateway caches agents per routing key (_agent_cache) while transcripts
are owned by session_id — and switch_session (/resume from a second
surface, CLI-continuity rebinding, async-delegation pinning,
topic-binding tip-walks) maps multiple routing keys onto one session_id
without any cross-key check. Two keys mapped to one session run
concurrent turns on two different agent objects, so the per-agent
tripwire could never fire for exactly the dispatch route NousResearch#64934 is
waiting to identify.

Add a module-level session_id-keyed in-flight registry alongside the
per-agent marker. Same philosophy as the original tripwire: log-only,
takes ownership on overlap, under-reports rather than double-reports
(a same-agent overlap warns once, not twice). The persist-time clear
pops the session id stamped at turn start, so a mid-turn compression
rotation of agent.session_id cannot strand the slot.
@alt-glitch alt-glitch added type/feature New feature or request comp/agent Core agent runtime: loop, agent_init, prompt builder, context-compression, responses endpoint P3 Low — cosmetic, nice to have sweeper:risk-session-state Sweeper risk: may lose/corrupt/mis-associate session or context state labels Jul 19, 2026
@teknium1

Copy link
Copy Markdown
Contributor

Thanks for tracing this cross-key route. Current main still lets SessionStore.switch_session() repoint one routing key to an existing session ID without checking other keys (gateway/session.py:2394-2433), while the runner claims _running_agents by _quick_key (gateway/run.py:11177-11192) and caches agents by session_key (gateway/run.py:19815-19818).

The existing tripwire is per-agent state (agent/agent_runtime_helpers.py:376-390), although every turn reaches it through build_turn_context (agent/turn_context.py:330-337). The proposed session-ID registry therefore covers the verified blind spot without changing dispatch or persistence behavior. The stamped session ID and unconditional clear preserve the existing tripwire's intentional under-reporting behavior.

No correctness or design-fit issue was found in the two-file diff at cd9bc2af4a07d83cfdcc518eeca52f9f3234f313.

Automated hermes-sweeper review.

@teknium1 teknium1 added the sweeper:blast-broad Sweeper blast radius: broad — a core path most sessions hit label Jul 19, 2026
@teknium1

Copy link
Copy Markdown
Contributor

Merged via PR #67371 — your commit was cherry-picked onto current main with your authorship preserved in git log (8c6627638). Excellent route analysis on #64934; the runnable repro made verification straightforward.

One follow-up we added on top: background-review forks share the live parent's session_id (for prompt-cache warmth) while being _persist_disabled, so they would have tripped a false cross-agent warning against the parent's real turn and popped the parent's registry slot at their own persist. Both registry legs now skip persist-disabled agents symmetrically.

Thanks for the contribution!

@Hotragn
Hotragn deleted the fix/64934-tripwire-session-scoped branch July 19, 2026 10:20
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

comp/agent Core agent runtime: loop, agent_init, prompt builder, context-compression, responses endpoint P3 Low — cosmetic, nice to have sweeper:blast-broad Sweeper blast radius: broad — a core path most sessions hit sweeper:risk-session-state Sweeper risk: may lose/corrupt/mis-associate session or context state type/feature New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants