Fix #2323: --cursor-file on wait/wait-loop closes wait→process→wait race

docs/reference/agent-wait-patterns.md

@@ -79,6 +79,18 @@ calls together into a "block forever server-side" behaviour so the agent
 can issue one command and **do nothing else** until the orchestrator
 SIGTERMs it or a terminal event arrives.
 
+### Cursor threading is automatic across re-entered waits
+
+Reviewer `POLL` (waiting for proposals from each producer in turn) and
+post-ACK `STAY ALIVE` (waiting for the next BRC event after handling
+one) both re-enter `wait-loop` multiple times in a row. The CLI
+auto-persists the response cursor under
+`/tmp/egg-wait-cursor-${EGG_PIPELINE_ID}-${EGG_AGENT_ROLE}-<hash>`
+between invocations, so any event that lands in the gap between
+calls is still delivered on the next call (issue #2323). No flag
+needed — see the "Auto cursor threading" subsection below for the
+contract.
+
 A transient inner-call error (exit 2 — HTTP 5xx, ECONNRESET, etc.) makes
 the wrapper back off (≤ 2 s in test mode, exponential in production) and
 retry. A permanent error (exit 3 — 4xx, bad pipeline id, argparse misuse)
@@ -371,10 +383,16 @@ The `cursor` is opaque from the caller's perspective:
 iterations, so a cursor-less call that rides through several timeouts
 before matching does not reopen the race. However, `wait-loop` does
 **not** expose `--json` (its human-readable stdout has no cursor field),
-so an outer shell loop that wants strict cursor threading across
-successive invocations should use `wait --json` instead — the outer
-`while` loop covers the same multi-ACK consumption shape that
-`wait-loop` plays in the single-call case.
+so cursor threading across successive `wait-loop` CLI invocations
+needs a side-channel. The CLI provides this automatically when
+`EGG_AGENT_ROLE` is set (the production case), persisting the cursor
+to a per-(role, for-types) file on disk — see "Auto cursor
+threading" below.
+
+For shell pipelines that need to inspect the messages themselves,
+`wait --json` with `--since`-threaded `.data.cursor` remains the
+lower-level alternative — it gives you the cursor in-band and lets
+you branch on `$rc` per §3.
 
 **Recommended pattern (BRC producer loop):**
 
@@ -409,6 +427,76 @@ Callers that omit `--since` keep their pre-#1995 behaviour — still
 correct for the common "wait once, exit" shape, still vulnerable to
 the wait→wait race for multi-ACK loops.
 
+### Auto cursor threading (issue #2323)
+
+`egg-orch message wait` and `wait-loop` automatically persist the
+response cursor to a per-(pipeline, role, for-types, from-role) file
+under `/tmp` so successive CLI invocations thread their position
+without callers having to opt in. The mechanism:
+
+1. **Path derivation.** When `EGG_AGENT_ROLE` is set, the CLI uses
+   `${EGG_WAIT_CURSOR_DIR:-/tmp}/egg-wait-cursor-<pipeline_id>-<role>-<hash>`,
+   where `<hash>` is an MD5 of the **sorted** `--for` types together
+   with the `--from` filter (if any). Same type set + same `--from` →
+   same file (regardless of `--for` arg order); different type sets,
+   different `--from` filters, or different pipelines → different
+   files. POLL (`--for CONSENSUS_PROPOSE`) and STAY ALIVE
+   (`--for ... 4 types ...`) hash to distinct files automatically;
+   two pipelines sharing a `/tmp` mount (debug shells, integration
+   test reuse) cannot leak cursors into each other.
+2. **Read on entry.** If the file exists and is non-empty, its
+   contents become the default for `--since`. An explicit `--since`
+   still wins. A corrupt file (non-UTF-8, unreadable) is treated as
+   empty rather than failing the wait.
+3. **Write on success.** Every successful round-trip (match OR
+   timeout) atomically writes the response cursor back, but only
+   when the response carries a non-empty cursor. Match → ID of the
+   last delivered message. Timeout → current stream tip, so the
+   next call resumes strictly after what this one would have seen.
+   Safety cap → handler-advanced cursor. A `cursor=null` response
+   (empty stream, pathological safety cap with no observed events)
+   is preserved as a no-op so a previously-stored cursor never
+   moves backward.
+4. **Untouched on errors.** rc=2 (transient) and rc=3 (permanent)
+   leave the file alone — the wait did not advance, so the cursor
+   must not move.
+
+Debug shells without `EGG_AGENT_ROLE` set get the legacy from-tip
+behavior with no file-system side effects.
+
+**Operator debugging.** A stuck reviewer's cursor lives at
+`/tmp/egg-wait-cursor-${EGG_PIPELINE_ID}-${EGG_AGENT_ROLE}-*`.
+`cat` it to see what position the next wait will resume from; `rm`
+it to force a fresh from-tip start.
+
+If you only know the role (e.g., the pipeline id contains hyphens
+that make a hand-typed glob error-prone), use:
+
+```bash
+ls /tmp/egg-wait-cursor-*-${EGG_AGENT_ROLE}-*
+```
+
+If you only know the pipeline id, use:
+
+```bash
+ls /tmp/egg-wait-cursor-${EGG_PIPELINE_ID}-*
+```
+
+**Concurrency caveat.** Two processes writing the same cursor file
+race (last writer wins). For the LLM-agent use case this is not an
+issue — agents run waits sequentially per role.
+
+**Cross-type drift caveat.** A `wait-loop --for X` whose response
+cursor advances past an event of type `Y` (because the inner read
+saw `Y` but the type filter dropped it) means a follow-up call
+restarting from that cursor will not see the dropped `Y`. Mitigate
+by including all relevant types in the `--for` list — different
+type sets get different cursor files automatically, so a drifted
+POLL cursor can never affect a STAY ALIVE wait. The same isolation
+holds for `--from` filters: a wait scoped to one sender can drop
+messages its filter rejected, but a sibling wait with a different
+`--from` keeps its own cursor and sees them.
+
 ## 4. `HEARTBEAT` Message Type
 
 `HEARTBEAT` is a typed message agents emit on state transitions so the

orchestrator/routes/pipelines.py

@@ -8370,12 +8370,15 @@ def _build_brc_preamble(
                 "**Don't** wrap this in a shell `for i in 1..N` loop; "
                 "**don't** prefix it with `sleep N`.  The wait-loop "
                 "blocks server-side and returns the moment a NEW BRC "
-                "event arrives — events that predate the call (including "
-                "your own just-sent CONSENSUS_CONFIRMED) are skipped "
-                "(issue #1925).  Exit code 0 means act on the returned "
-                "message, 1 means the wrapper exhausted retries (surface "
-                "it).  If you need zero-drop semantics across a send→wait "
-                "boundary, capture your send's ID and pass `--since <id>`. "
+                "event arrives.  Exit code 0 means act on the returned "
+                "message, 1 means the wrapper exhausted retries "
+                "(surface it).  Cursor threading across re-entries is "
+                "automatic (issue #2323): the CLI persists the response "
+                "cursor under "
+                "/tmp/egg-wait-cursor-${EGG_PIPELINE_ID}-${EGG_AGENT_ROLE}-* so "
+                "events that land between your call returning and the next "
+                "call entering are still delivered, and the send→wait "
+                "race is closed without manual `--since` anchoring.  "
                 "See docs/reference/agent-wait-patterns.md.",
                 "7. **HANDLE RE-REVIEW**: If you receive a `CONSENSUS_RE_REVIEW` message "
                 "while staying alive, you MUST act on it — failure to do so will stall "
@@ -8403,13 +8406,21 @@ def _build_brc_preamble(
                 "`wait-loop` blocks server-side and returns exit 0 the moment "
                 "a proposal arrives (stdout has it); exit 1 means a permanent "
                 "error (surface it — do NOT retry).  It re-issues the inner "
-                "long-poll internally so timeouts never surface to you.  Do "
-                "NOT wrap this in a shell `for` loop, do NOT `sleep N`, and "
-                "do NOT use bare `egg-orch message wait` here — a bare `wait` "
-                "exits 1 on each timeout which the tool surface renders as an "
-                "error and invites a tight retry loop (issue #1943).  Finish "
-                "your preparation work from step 1 before entering the "
-                "wait-loop.",
+                "long-poll internally so timeouts never surface to you.  "
+                "**Re-enter the same command** after each ACK/NACK to wait "
+                "for the next producer's proposal — cursor threading across "
+                "these re-entries is automatic (issue #2323): the CLI "
+                "persists the response cursor under "
+                "/tmp/egg-wait-cursor-${EGG_PIPELINE_ID}-${EGG_AGENT_ROLE}-* "
+                "so a proposal "
+                "that lands in the gap between your previous wait returning "
+                "and the next one entering is still delivered.  Do NOT "
+                "wrap this in a shell `for` loop, do NOT `sleep N`, and "
+                "do NOT use bare `egg-orch message wait` here — a bare "
+                "`wait` exits 1 on each timeout which the tool surface "
+                "renders as an error and invites a tight retry loop "
+                "(issue #1943).  Finish your preparation work from "
+                "step 1 before entering the wait-loop.",
                 "3. **SYNC**: Before reviewing, sync your worktree so you have the "
                 "producer's commits: `git fetch origin && git merge "
                 + _resolve_origin_ref(branch or base_branch)
@@ -8489,14 +8500,16 @@ def _build_brc_preamble(
                 "orchestrator stops you. **Don't** wrap this in a "
                 "shell `for i in 1..N` loop; **don't** prefix it with "
                 "`sleep N`.  The wait-loop blocks server-side and "
-                "returns the moment a NEW BRC event arrives — events "
-                "that predate the call are skipped (issue #1925).  "
-                "Exit 0 means act on the returned event; exit 1 means "
-                "the wrapper exhausted retries (surface it).  If "
-                "you need zero-drop semantics across a send→wait "
-                "boundary, capture the ID of your most recent send and "
-                "pass `--since <id>`.  See "
-                "docs/reference/agent-wait-patterns.md.",
+                "returns the moment a NEW BRC event arrives.  Exit 0 "
+                "means act on the returned event; exit 1 means the "
+                "wrapper exhausted retries (surface it).  Cursor "
+                "threading across re-entries is automatic (issue "
+                "#2323): the CLI persists the response cursor under "
+                "/tmp/egg-wait-cursor-${EGG_PIPELINE_ID}-${EGG_AGENT_ROLE}-* so "
+                "events that land between your call returning and the next "
+                "call entering are still delivered, and the send→wait "
+                "race is closed without manual `--since` anchoring.  "
+                "See docs/reference/agent-wait-patterns.md.",
                 "8. **HANDLE RE-REVIEW**: If you receive a `CONSENSUS_RE_REVIEW` message "
                 "while staying alive, you MUST act on it — failure to do so will stall "
                 "the entire pipeline. Re-review the re-proposing producer's new proposal "

orchestrator/tests/test_pipeline_prompts.py

@@ -3580,6 +3580,57 @@ def test_reviewer_poll_uses_wait_loop_not_bare_wait(self):
         )
 
 
+class TestReviewerWaitLoopMentionsAutoCursor:
+    """Reviewer + producer wait-loop steps must explain auto cursor
+    threading (issue #2323).
+
+    Background: each ``wait-loop`` CLI invocation is a separate process,
+    and without cursor threading each new call starts at the stream
+    tip — skipping any event that arrived in the gap between the
+    previous wait-loop returning and the next one entering. On
+    multi-producer phases (plan: 3 producers) this stalled the phase
+    by 20-30 minutes per missed event. The fix is in the CLI itself:
+    ``wait`` and ``wait-loop`` auto-derive a per-(role, for_types)
+    cursor file under
+    ``/tmp/egg-wait-cursor-${EGG_PIPELINE_ID}-${EGG_AGENT_ROLE}-*``
+    with no flag needed. The prompts point at that path so operators
+    debugging a stuck reviewer know where to look.
+    """
+
+    def test_reviewer_poll_mentions_auto_cursor(self):
+        preamble = _build_brc_preamble("reviewer_code", "implement", branch="egg/issue-123")
+        poll_start = preamble.index("**POLL**")
+        poll_end = preamble.index("**SYNC**")
+        poll_block = preamble[poll_start:poll_end]
+        assert "automatic" in poll_block.lower(), (
+            "POLL must tell the reviewer that cursor threading "
+            "across re-entries is automatic (issue #2323)."
+        )
+        assert "/tmp/egg-wait-cursor-${EGG_PIPELINE_ID}-${EGG_AGENT_ROLE}-" in poll_block, (
+            "POLL must surface the cursor file path so operators "
+            "debugging a stuck reviewer can `cat` it."
+        )
+        assert "#2323" in poll_block
+
+    def test_reviewer_stay_alive_mentions_auto_cursor(self):
+        preamble = _build_brc_preamble("reviewer_code", "implement", branch="egg/issue-123")
+        sa_start = preamble.index("**STAY ALIVE**")
+        sa_end = preamble.index("**HANDLE RE-REVIEW**", sa_start)
+        sa_block = preamble[sa_start:sa_end]
+        assert "automatic" in sa_block.lower()
+        assert "/tmp/egg-wait-cursor-${EGG_PIPELINE_ID}-${EGG_AGENT_ROLE}-" in sa_block
+        assert "#2323" in sa_block
+
+    def test_producer_stay_alive_mentions_auto_cursor(self):
+        preamble = _build_brc_preamble("coder", "implement", branch="egg/issue-123")
+        sa_start = preamble.index("**STAY ALIVE**")
+        sa_end = preamble.index("**HANDLE RE-REVIEW**", sa_start)
+        sa_block = preamble[sa_start:sa_end]
+        assert "automatic" in sa_block.lower()
+        assert "/tmp/egg-wait-cursor-${EGG_PIPELINE_ID}-${EGG_AGENT_ROLE}-" in sa_block
+        assert "#2323" in sa_block
+
+
 class TestProducerOrientationSyncNote:
     """Tests for sync note in _build_producer_orientation (issue #1565)."""