Skip to content

fix(pi): recheck supervision outcomes before delivery - #2993

Closed
zachlandes wants to merge 7 commits into
kunchenguid:mainfrom
zachlandes:fm/pi-branch-summary-freshness
Closed

zachlandes wants to merge 7 commits into
kunchenguid:mainfrom
zachlandes:fm/pi-branch-summary-freshness

Conversation

@zachlandes

@zachlandes zachlandes commented Aug 25, 2026 •

Copy link
Copy Markdown
Contributor

What Changed

  • Add claim anchors to durable supervision outcomes and recheck them immediately before delivery and startup replay.
  • Hold outcomes produced during an active main turn until Pi reports an idle settled state; drop stale routine notes and surface stale captain notes as superseded with instructions to read current task state.
  • Document and test freshness handling, non-steering routine delivery, superseded replay, and Pi lifecycle behavior.

Risk Assessment

🚨 High: The durable freshness fix remains bypassable by the repository's supported same-ID relaunch transition, allowing a stale outcome from a superseded task incarnation to be presented as current.

Testing

Targeted store, extension, and real Pi SDK checks demonstrated that stale task claims are marked superseded or dropped, current claims still arrive, routine notes explicitly disable turn triggering, and captain outcomes retain their typed handoff; the strengthened extension regression also passed on rerun, with the complete CLI transcript captured as evidence.

Evidence: Targeted supervision freshness and real-SDK validation transcript

Source: Targeted supervision freshness and real-SDK validation transcript

$ bin/fm-test-run.sh tests/fm-branch-supervision.test.sh
FM_TEST_BEGIN 2026-08-31T18:48:55Z tests/fm-branch-supervision.test.sh family=unclassified expected_gate_skip=none
ok - branch prompt is byte-stable across homes, cwd, timezone, and time, above the cache floor
ok - outcome store is append-only and refuses sequence reuse after a torn tail
ok - startup replay skips silent outcomes and preserves visible and legacy rows
ok - a recorded outcome whose task claim moved on replays as superseded, and fleet and legacy rows do not
ok - lease exclusivity, same-actor refresh, release, staleness, and sweep hold
ok - fm-control and fm-teardown refuse the other actor's live lease and pass through otherwise
ok - PR merge, local landing, and new-task spawn refuse the branch actor and spare main
ok - a non-Pi home ignores stale Pi leases even when the recycled pid owns its lock
ok - lease liveness requires an exact valid session-lock pid
ok - concurrent stale-lease claims serialize so exactly one actor succeeds
ok - guard stale cleanup cannot race with or delete a newer lease claim
ok - lease guard excludes a concurrent actor for the complete mutation
ok - a claim naming the other actor fails loudly instead of silently impersonating it
ok - release commands authorize the caller and bulk release drops only that actor's leases
ok - the branch cannot force a teardown or bypass fm-control for a relaunch
FM_TEST_END 2026-08-31T18:49:03Z tests/fm-branch-supervision.test.sh exit=0 duration_ms=8353 gate_skip=false
FM_TEST_SUMMARY total=1 failed=0 skipped_gate=0 duration_ms=8410
FM_TEST_SUMMARY_FAMILY family=unclassified count=1 duration_ms=8353 failed=0
FM_TEST_SLOWEST rank=1 script=tests/fm-branch-supervision.test.sh duration_ms=8353

$ bin/fm-test-run.sh tests/fm-pi-branch-extension.test.sh
FM_TEST_BEGIN 2026-08-31T18:49:03Z tests/fm-pi-branch-extension.test.sh family=unclassified expected_gate_skip=none
ok - fm_branch_outcomes hides through ToolExecutionComponent while Calm-off and HTML export stay stock
ok - the installed Pi still bounds the picker's list and ranks its search
ok - branch owns accepted wakes with a stable prefix contract and verdict-driven merge delivery
ok - a captain outcome reaches main's model as typed, self-describing input while routine notes stay plain
ok - a queued summary whose task claim went stale is refreshed or dropped, never delivered as current
ok - requested and unsolicited healthy outcomes keep distinct delivery and event ownership
ok - a broken operational encoder still delivers one invisible instructed captain outcome as a follow-up
ok - scopeForUnreadWake excludes every main-only class without vetoing eligible task-local rows, and writes the eligible snapshot
ok - branch prompt_cache_key is stable per home across sessions and distinct between homes
ok - branch default-on eligibility (task-scoped, heartbeat, afk) binds and a broken branch falls back to main
ok - a heartbeat review survives a check row arriving before its drain
ok - pre-drain eligibility re-check excludes a newly main-owned row without deferring eligible work
ok - a settled branch turn releases an unacknowledged grant for main replay
ok - a stale main claim cannot silently suppress later wake delivery
ok - pre-drain eligibility re-check no-ops an already-drained wake
ok - dialog mirror filters tool and operational traffic, lands before wakes, and keeps a durable cursor
ok - branch session persists across process restarts through the recorded pointer
ok - the current pin state binds every branch create and reopen, and clearing it returns the branch to main's model
ok - unpinned branches follow main model changes live while pinned branches stay fixed
ok - supervision-model command persists the captain's pick and rebinds the live branch
ok - supervision-model opens a bounded searchable list, follow main first, and pins the branch alone
ok - branch model picker keeps follow main first and filters the eligible catalog
ok - the effort pin binds every branch create and reopen, and clearing it returns the branch to main's effort
ok - unpinned branches follow main effort changes live while pinned branches stay fixed
ok - supervision-model runs an effort picker after the model picker and persists both independently
ok - an unusable model pin falls back to main and an unparseable one is treated as no pin
ok - replacement activation cleans old branch leases and retries failed cleanup
ok - branch activates on a cold start once the lock is acquired, never before
ok - queued wakes and mirrors stop mutating branch state after lock ownership is lost
ok - stale reports, shells, mirrors, cursors, leases, and prompts perform no side effects
ok - a Pi session that does not own the lock accepts nothing and mutates no branch state
ok - an extension rebind re-mirrors undelivered dialog instead of dropping it
FM_TEST_END 2026-08-31T18:49:30Z tests/fm-pi-branch-extension.test.sh exit=0 duration_ms=27067 gate_skip=false
FM_TEST_SUMMARY total=1 failed=0 skipped_gate=0 duration_ms=27119
FM_TEST_SUMMARY_FAMILY family=unclassified count=1 duration_ms=27067 failed=0
FM_TEST_SLOWEST rank=1 script=tests/fm-pi-branch-extension.test.sh duration_ms=27067

$ FM_PI_BRANCH_LIVE_E2E=1 bin/fm-test-run.sh tests/fm-pi-branch-live-e2e.test.sh
FM_TEST_BEGIN 2026-08-31T18:49:38Z tests/fm-pi-branch-live-e2e.test.sh family=live-harness-optin expected_gate_skip=optin-env
ok - real Pi SDK 0.84.4 accepts the branch session construction, preserves an unpromptable wake, and emits no lifecycle event for a refused turn
ok - real Pi SDK 0.84.4 applies an explicit branch model on create and over a reopened session's recorded model
ok - real Pi SDK 0.84.4 reports its own supported effort levels and applies an explicit branch effort over a reopened session's recorded level
ok - real Pi SDK 0.84.4 delivers a custom message to the provider as user text carrying only content, so the captain outcome's typed envelope is what reaches the model
FM_TEST_END 2026-08-31T18:49:40Z tests/fm-pi-branch-live-e2e.test.sh exit=0 duration_ms=2329 gate_skip=false
FM_TEST_SUMMARY total=1 failed=0 skipped_gate=0 duration_ms=2381
FM_TEST_SUMMARY_FAMILY family=live-harness-optin count=1 duration_ms=2329 failed=0
FM_TEST_SLOWEST rank=1 script=tests/fm-pi-branch-live-e2e.test.sh duration_ms=2329

$ bin/fm-test-run.sh tests/fm-pi-branch-extension.test.sh  # rerun after strengthening routine handoff assertion
FM_TEST_BEGIN 2026-08-31T18:50:12Z tests/fm-pi-branch-extension.test.sh family=unclassified expected_gate_skip=none
ok - fm_branch_outcomes hides through ToolExecutionComponent while Calm-off and HTML export stay stock
ok - the installed Pi still bounds the picker's list and ranks its search
ok - branch owns accepted wakes with a stable prefix contract and verdict-driven merge delivery
ok - a captain outcome reaches main's model as typed, self-describing input while routine notes stay plain
ok - a queued summary whose task claim went stale is refreshed or dropped, never delivered as current
ok - requested and unsolicited healthy outcomes keep distinct delivery and event ownership
ok - a broken operational encoder still delivers one invisible instructed captain outcome as a follow-up
ok - scopeForUnreadWake excludes every main-only class without vetoing eligible task-local rows, and writes the eligible snapshot
ok - branch prompt_cache_key is stable per home across sessions and distinct between homes
ok - branch default-on eligibility (task-scoped, heartbeat, afk) binds and a broken branch falls back to main
ok - a heartbeat review survives a check row arriving before its drain
ok - pre-drain eligibility re-check excludes a newly main-owned row without deferring eligible work
ok - a settled branch turn releases an unacknowledged grant for main replay
ok - a stale main claim cannot silently suppress later wake delivery
ok - pre-drain eligibility re-check no-ops an already-drained wake
ok - dialog mirror filters tool and operational traffic, lands before wakes, and keeps a durable cursor
ok - branch session persists across process restarts through the recorded pointer
ok - the current pin state binds every branch create and reopen, and clearing it returns the branch to main's model
ok - unpinned branches follow main model changes live while pinned branches stay fixed
ok - supervision-model command persists the captain's pick and rebinds the live branch
ok - supervision-model opens a bounded searchable list, follow main first, and pins the branch alone
ok - branch model picker keeps follow main first and filters the eligible catalog
ok - the effort pin binds every branch create and reopen, and clearing it returns the branch to main's effort
ok - unpinned branches follow main effort changes live while pinned branches stay fixed
ok - supervision-model runs an effort picker after the model picker and persists both independently
ok - an unusable model pin falls back to main and an unparseable one is treated as no pin
ok - replacement activation cleans old branch leases and retries failed cleanup
ok - branch activates on a cold start once the lock is acquired, never before
ok - queued wakes and mirrors stop mutating branch state after lock ownership is lost
ok - stale reports, shells, mirrors, cursors, leases, and prompts perform no side effects
ok - a Pi session that does not own the lock accepts nothing and mutates no branch state
ok - an extension rebind re-mirrors undelivered dialog instead of dropping it
FM_TEST_END 2026-08-31T18:50:39Z tests/fm-pi-branch-extension.test.sh exit=0 duration_ms=27115 gate_skip=false
FM_TEST_SUMMARY total=1 failed=0 skipped_gate=0 duration_ms=27172
FM_TEST_SUMMARY_FAMILY family=unclassified count=1 duration_ms=27115 failed=0
FM_TEST_SLOWEST rank=1 script=tests/fm-pi-branch-extension.test.sh duration_ms=27115

Pipeline

Updates from git push no-mistakes

⏭️ **intent** - skipped

✅ No issues found.

✅ **Rebase** - passed

✅ No issues found.

⚠️ **Review** - 1 error
  • 🚨 bin/fm-branch-outcome.sh:138 - The claim anchor omits the task's durable spawn_gen incarnation. Concrete path: an outcome is held during a captain turn, fm-control relaunches the same task ID, and the replacement metadata retains the same pr= and poll presence; the anchor remains identical, so the old incarnation's outcome is delivered as current. Include the validated incarnation token in the anchor at this owning boundary, treating missing or ambiguous tokens as unverifiable for legacy records.
✅ **Test** - passed

✅ No issues found.

  • Inspected git diff 4ad8cbaeafc109a17c1af3911867b7fe9e04e801..78f90f2312ecee93c57fee4e7254c471f18e6e1f to identify the freshness and delivery behavior.
  • bin/fm-test-run.sh tests/fm-branch-supervision.test.sh
  • bin/fm-test-run.sh tests/fm-pi-branch-extension.test.sh
  • FM_PI_BRANCH_LIVE_E2E=1 bin/fm-test-run.sh tests/fm-pi-branch-live-e2e.test.sh
  • Strengthened tests/fm-pi-branch-extension.test.sh to require triggerTurn === false for immediate and held routine-note delivery, then reran bin/fm-test-run.sh tests/fm-pi-branch-extension.test.sh.
  • Verified the evidence transcript directly and ran git diff --check.
✅ **Document** - passed

✅ No issues found.

⚠️ **Lint** - 1 warning
  • ⚠️ linter found issues (exit code 1)
✅ **Push** - passed

✅ No issues found.

* A branch outcome was recorded the instant its claim was true and
  handed straight to Pi's own message queue, so a note reported inside
  the captain's running turn rendered after that turn saying whatever
  it had said at creation. A PR that merged in between still surfaced
  as review-ready, because nothing re-read the claim on the way out.
* Held a note reported mid-turn in the extension instead, carrying the
  task's claim anchor, and re-checked it at main's idle boundary. Once
  Pi owns a message it cannot be inspected again, so the hold is what
  makes any re-check possible at all.
* Dropped a stale routine note, which is noise by definition and stays
  readable in the durable store, and refreshed a stale captain one
  rather than suppressing it, so a real terminal outcome still opens
  its turn and reports the current truth.
* Applied the same anchor to the session-start replay, which carries
  the longer of the two delivery windows: those rows never reached a
  handoff at all and can be arbitrarily old.
* Anchored on the records that actually falsify a task-local claim,
  the task metadata and its armed merge poll, so ordinary progress on
  a task does not invalidate a note about it. An anchor that cannot be
  computed never reads as stale, so uncertainty never drops an outcome.
@greptile-apps

greptile-apps Bot commented Aug 25, 2026 •

Copy link
Copy Markdown

Confidence Score: 5/5

The PR appears safe to merge.

No blocking failure remains.

Reviews (6): Last reviewed commit: "no-mistakes(document): Correct Pi freshn..." | Re-trigger Greptile

@kunchenguid

Copy link
Copy Markdown
Owner

Speaking as Kun's firstmate: scheduled 7:10pm PT 8/24 pass (FM-FMOSS-CRON). First look on current main 038d0f7ec6ba7238a151722931434dcf06ff37c4 (#2942). VISION.md read in full. Thank you @zachlandes.

class=corrective. This is a bugfix on the already-default-on Pi supervision delivery path, not a new grant. Inspected the DIFF, not the title: .pi/extensions/fm-branch-supervision.ts (PendingNote, enqueueNote, releasePendingNotes, sendNote, claimAnchor, agent_settled+isIdle()), bin/fm-branch-outcome.sh (claim-anchor, annotate_superseded, append-only anchor), docs/pi-supervision-branch.md freshness section, tests/fm-pi-branch-extension.test.sh (test_stale_task_claim_is_rechecked_before_delivery), tests/fm-branch-supervision.test.sh superseded replay.

Does THIS DIFF fix mark-read-at-enqueue (#2984) vs a different path? It primarily fixes a different duplicate/stale-summary path: a review-ready note held across a captain turn was handed to Pi with the claim frozen at enqueue. The new hold queue re-checks the task claim-anchor (meta presence, pr=, .pr-poll) at the idle delivery boundary; stale routine notes are dropped; stale captain notes still open one turn with a SUPERSEDED marker. That is the race this PR actually closes.

On #2984 itself: enqueueNote does not call mark-read. Session shutdown leaves rows unread so startup-replay can recover a crash before the idle boundary. That narrows the old "queued behind a running turn + cursor already advanced" hole. sendNote still does pi.sendMessage then markRead immediately. The crash-inside-Pi-follow-up-turn window that #2984 named (11s between hidden inject and the turn) remains the documented residual. Author body is right that this does not resolve #2984.

#2977 (dup/displace captain outcomes): not covering. captainTurnPending stops later notes from steering the opened turn; it does not stop main from presenting outcome A and the branch then injecting A again, nor a later B turn repeating A. Distinct live-process presentation bug.

VISION.md (those files):

  • One captain, one interface: aligns. Stale "ready for review" after merge is a dishonest outcome. Supersession still interrupts rather than burying a captain result.
  • Authority is explicit: aligns. No new grant; default-on supervision already shipped in feat(pi): default branch supervision and route heartbeats #2939. Dropping a stale routine note is a presentation choice, not a new autonomy.
  • Scripts own the mechanics: aligns. Anchor compare is exact (meta/poll/pr). Unverifiable/empty/fleet anchors never read as stale.
  • A restart is a non-event: aligns for held-unread recovery. cannot tell as closed for Branch outcome is marked read at enqueue, so a crash before delivery leaves it durable but unannounced #2984's post-handoff crash: cursor still advances at sendMessage.
  • Delegation with a spine: aligns. Not a new task shape. Unlanded work is not torn down.
  • The fleet outlives any vendor: aligns. Pi-only extension; binds to agent_settled/isIdle() the SDK actually exposes (pinned 0.84.3).
  • Scope: aligns. Command-layer delivery freshness + regression coverage.

This HEAD: 0a63e72ea0a45a7b76441c7c9b2cb694e82ff3c8. MERGEABLE / UNSTABLE, ahead 8 / behind 0.
Attestation no-mistakes-pipeline-attestation:v1 head_sha 06264a5911ea9cd823cee51a259f152953605468 ≠ THIS HEAD (three later no-mistakes: apply CI fixes commits). Author/CI blocker; not escalated.
CI: Greptile SUCCESS only. action_required CI 32799185922 / Require no-mistakes 32799185929. Not first-time (zachlandes #370 had Behavior/Lint/Repo invariants SUCCESS); workflows not approved this pass. Greptile is not a merge gate.

Security: none. Overlap/holds: none of spawn-freshen, #2804, herdr pair, or lock PRs. Files are Pi supervision + outcome store.

Land-eligible: NO. Captain-flag NOW: no. waiting-on-author for a HEAD-matching attestation (and then green CI). Preferred later: this PR first among this batch once attestation+CI match, still not auto until those are green. Did not squash.

@kunchenguid

Copy link
Copy Markdown
Owner

Speaking as Kun's firstmate: first look on current main 038d0f7ec6ba (#2942). Closes ready-for-pr #2984. Never messaged the captain.

class=corrective. Enqueue no longer calls mark-read. Notes stay in pendingNotes with a claim anchor until releasePendingNotes re-reads claim-anchor and only then sendNote → mark-read. That is the #2984 defect (read cursor advanced at enqueue / send-before-delivery). Routine stale notes are dropped (durable row remains); stale captain notes still open one turn as SUPERSEDED so a terminal outcome cannot be buried.

Related #2977 (duplicate / displaced captain outcomes): partially covering — the delivery-boundary re-check and captainTurnPending serialize the stale-claim path. It does not add seq↔response correspondence or dedup of an already-presented main result. Leave #2977 open.

VISION.md (inspected .pi/extensions/fm-branch-supervision.ts enqueueNote/releasePendingNotes/sendNote/markRead, bin/fm-branch-outcome.sh claim-anchor/append --anchor, tests):

  • One captain, one interface: aligns (stale review-ready cannot render as current; SUPERSEDED is honest).
  • Authority is explicit: aligns (no new grant; freshness is a refusal to lie).
  • Scripts own the mechanics: aligns (anchor composition is exact; extension only compares).
  • A restart is a non-event: aligns (unread until delivered or deliberately dropped; startup-replay uses the same anchor).
  • Delegation with a spine: aligns (same primitive, fixed cursor).
  • The fleet outlives any vendor: aligns (Pi-only file; inert elsewhere).
  • Scope: aligns.

This HEAD: 0a63e72ea0a45a7b76441c7c9b2cb694e82ff3c8. MERGEABLE / UNSTABLE, ahead 8 / behind 0.
Attestation 06264a5911ea9cd823cee51a259f152953605468 ≠ THIS HEAD.
CI action_required: CI 32799185922, Require no-mistakes 32799185929. Not first-time (zachlandes has other firstmate PRs); workflows not approved this pass. Greptile is not a merge gate.

Security: none. Remaining documented non-atomic send-to-render window is the architecture's leftover, not a new hole. Overlap: none with spawn-freshen / pool / herdr / lock holds.

Land-eligible rec: NO. Captain-flag NOW: no (author/CI: matching attestation + green CI).

This is waiting-on-author. Not a captain-decision. Not a merge I will recommend.

@zachlandes
zachlandes force-pushed the fm/pi-branch-summary-freshness branch from 0a63e72 to 06264a5 Compare August 25, 2026 02:38
@kunchenguid

Copy link
Copy Markdown
Owner

Speaking as Kun's firstmate: correcting my earlier note on this thread.

#2993 does not close #2984. The author's body is right, and I over-read the eligible-fetch closing link. enqueueNote no longer marks read (crash-before-idle-delivery stays unread), but sendNote is still sendMessage then mark-read. That post-handoff crash window is #2984. Leave #2984 open.

This PR is the stale-claim / freshness path (queued review-ready note re-checked at idle delivery). #2977 stays open too (no A-then-B captain-outcome dedup).

class=corrective and waiting-on-author (HEAD-matching attestation) are unchanged. Not a merge I will recommend.

* Upstream moved 39 commits ahead while this reviewed change waited, and
  eight files conflicted; each was resolved as a union so both accepted
  intents survive rather than either side being dropped.
* The captain outcome's typed operational envelope from upstream now
  wraps the note body chosen at this branch's delivery boundary, so a
  superseded captain outcome is self-describing too. Upstream's own
  stated invariant is that an unwrapped captain note can be mistaken for
  main's earlier answer and lost, and a refreshed outcome is no more
  self-describing than the original it replaces.
* Two upstream tests asserted the mid-turn nextTurn handoff this change
  deliberately removed. Their assertions were kept intact and only the
  delivery timing moved to the idle boundary, because a note Pi already
  owns can no longer be re-checked before it is rendered.
* Re-ran the real-SDK live guard and the strict typecheck against the
  installed Pi 0.84.4 and recorded the dated result, because this change
  alters the guard's own recorded output line.
@zachlandes zachlandes changed the title fix(pi): re-check queued supervision summaries before delivery fix(pi): recheck supervision outcomes before delivery Aug 31, 2026
@zachlandes

Copy link
Copy Markdown
Contributor Author

Closing this one. Main rewrote the outcome delivery path after I opened it (mergeIntoMain is gone, and since #3312 delivery goes through reconcileUnreadOutcomes), so bringing this up to date would really be a rewrite: it conflicts in 9 of its 10 files.

The problem it was after is still there as far as I can tell. An outcome can still show up after the task it is about has moved on, for example a routine note queued while main is busy, or unread rows replayed after a crash. I have not proven those with a test yet. If I pick it back up it will be a fresh, narrower PR on the new path.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants