Skip to content

fix(pi): preserve queued wakes during successor recovery - #3440

Open
M00NLIG7 wants to merge 11 commits into
kunchenguid:mainfrom
M00NLIG7:fm/firstmate-pi-wake-coalescing-public-pr-v1-k9
Open

M00NLIG7 wants to merge 11 commits into
kunchenguid:mainfrom
M00NLIG7:fm/firstmate-pi-wake-coalescing-public-pr-v1-k9

Conversation

@M00NLIG7

@M00NLIG7 M00NLIG7 commented Sep 1, 2026 •

Copy link
Copy Markdown
Contributor

Intent

Continue the existing PR #3440 from its proven exact remote head 940beaf. The continuation branch was rooted directly at that fetched, object-verified head; do not use, reset to, replace from, cherry-pick from, or otherwise treat prior local commit 8d7d0264ac5b2fbf98973642d39cf3d3216270dc as an implementation source. Fix only the Pi supervision continuity defect where a live successor exists but is not ready and can indefinitely strand the queued failure and every later closure. Preserve exactly-once delivery, closure ordering, bounded retry behavior, recovered-continuity behavior, per-actor ownership, and all existing safety boundaries. Add the smallest executable regression proving delayed successor readiness eventually releases the queued failure and subsequent closures without loss, duplication, or indefinite delay, including relevant negative and ordering cases. Review every supported primary harness and runtime-backend integration surface required by the Firstmate coding guidelines, marking an axis not applicable only from inspected evidence. Keep the diff narrowly compatible with the current PR and add no unrelated cleanup, broad refactor, product coupling, or private fleet facts. Keep author and committer identity M00NLIG7 57321738+M00NLIG7@users.noreply.github.com. Run focused checks plus required lint, tests, documentation checks, and maintainer verification, then one fresh complete no-mistakes validation against the final exact head. Publish only by a normal fast-forward of the existing proven source branch fm/firstmate-pi-wake-coalescing-public-pr-v1-k9 from expected remote head 940beaf, updating only existing PR 3440. Never force-push, create or reopen another PR, alter the base, comment, approve, merge, or drop the verified archive ref; stop if the remote source branch advances. Merge remains unauthorized.

What Changed

  • Preserve ordered Pi wake delivery across successor recovery with durable row tickets, post-output presentation receipts, and bounded release of closes queued behind an unready successor.
  • Treat active no-mistakes approval or fix-review waits as current work, and use fresh structured validation activity to defer stale-worker escalation on its bounded cadence.
  • Document the updated watcher-continuity and runtime-backend contracts and expand Pi, wake-queue, crew-state, and watcher regression coverage.

Risk Assessment

🚨 High: The Pi queue-bound fix is bounded and regression-covered, but the branch also changes unrelated supervision and no-mistakes state behavior despite the explicit Pi-only scope constraint.

Testing

Reviewed the final Pi-only change, ran the focused process-level watcher suite, and captured its behavioral transcript: a live unready successor boundedly released the queued failure and later close in order and exactly once, without overlap or duplication after delayed readiness; Pi lifecycle/recovery cases and the inspected OpenCode integration cases also passed. No UI surface exists for this headless extension path, so the CLI behavioral transcript is the reviewer-visible evidence.

Evidence: Focused Pi watcher behavioral transcript

Source: Focused Pi watcher behavioral transcript

ok - Pi bounds an unready successor without losing or reordering queued closes ok - Pi surfaces a queued failure without disturbing its healthy successor ok - OpenCode pre-ready actionable close preserves its successor

ok - Pi extension reports external healthy watcher output
ok - Pi custom tool exposes repair-only metadata and returns automatic-continuation guidance
ok - Pi redundant tool call returns ownership guidance and spawns no second child
ok - Pi scheduled retry remains extension-owned after another tool call
ok - Pi actionable closes restore each successor before delivery and keep supervising while an earlier follow-up is blocked
ok - Pi revalidates delayed rows after either actor presents them, during queue mutation, after consumption, endpoint replacement, cleanup, and generation change while malformed evidence and current stale, declared-pause, merge, turn-end, and failure events still deliver
ok - Pi coalesces repeated same-task stale events to the newest durable identity and restores a successor for it
ok - Pi session replacement retires the old delayed delivery while the new session owns successor supervision
ok - Pi dispatcher branch offer owns accepted wakes and falls back to main
ok - Pi dispatcher flags a fleet-wide heartbeat offer as branch-eligible
ok - a co-present check row neither vetoes nor rides a heartbeat into main
ok - every main-only check class still reaches main, never the supervision branch
ok - heartbeat restoration failure stays on main
ok - watcher-failure repair stays with main even with a live, accepting branch listener
ok - Pi refused handling handshake is classified and not swallowed
ok - Pi hung successor falls back to one typed actionable wake
ok - Pi unretired successor falls back without an overlapping retry
ok - Pi late unretired closes resume classified supervision
ok - Pi clean empty close triggers a bounded continuity retry
ok - Pi retry resumes the actionable close queued behind a failed successor
ok - Pi retry lock loss resumes queued actionable closes without a successor
ok - Pi surfaces a queued failure without disturbing its healthy successor
ok - Pi bounds an unready successor without losing or reordering queued closes
ok - Pi established clean closes stop at the configured retry limit
ok - Pi close handler verifies session-lock ownership before successor launch
ok - Pi watcher arm distinguishes all session lock ownership states
ok - Pi session transitions use a generation owner across /new /resume /fork, stale callbacks, and quit
ok - Pi process-exit cleanup listener remains singular across session replacement
ok - Pi process-exit cleanup stops the attached arm child
ok - OpenCode plugins have an explicit ESM boundary even under a typeless parent package
ok - OpenCode watcher plugin uses the effective FM_HOME state
ok - OpenCode watcher plugin sources the effective config
ok - OpenCode watcher plugin requires session lock ownership
ok - OpenCode watcher coordinator respects primary scope
ok - OpenCode watcher plugin starts one successor before wake prompt delivery settles
ok - OpenCode pre-ready actionable close preserves its successor
ok - OpenCode hung successor falls back to one typed actionable wake
ok - OpenCode unretired successor falls back without an overlapping retry
ok - OpenCode late unretired closes resume classified supervision
ok - OpenCode clean empty close triggers a bounded continuity retry
ok - OpenCode established clean closes stop at the configured retry limit
ok - OpenCode close handler verifies session-lock ownership before successor launch
ok - OpenCode watcher plugin coordinates with the turn-end guard
ok - OpenCode healthy arm output does not suppress the turn-end guard

Pipeline

Updates from git push no-mistakes

✅ **intent** - passed

✅ No issues found.

⚠️ **Rebase** - 4 warnings
  • ⚠️ .pi/extensions/fm-primary-pi-watch.ts - merge conflict rebasing onto origin/main
  • ⚠️ docs/architecture.md - merge conflict rebasing onto origin/main
  • ⚠️ docs/scripts.md - merge conflict rebasing onto origin/main
  • ⚠️ docs/watcher-continuity.md - merge conflict rebasing onto origin/main
⚠️ **Review** - 2 errors
  • 🚨 bin/fm-watch.sh:833 - Intent conformance: “Fix only the Pi supervision continuity defect” and “add no unrelated cleanup” conflict with this generic watcher change. A non-Pi worker with a stale pane and advancing no-mistakes step log now resets its wedge timer rather than emitting the prior wedge wake. Retain only with explicit scope authorization; otherwise remove it.
  • 🚨 bin/fm-crew-state.sh:568 - Intent conformance: the Pi-only/narrow-diff requirement conflicts with changed global no-mistakes state semantics. A nonterminal run with stale outcome data plus fix-review/approval evidence is now classified as parked instead of by its outcome. This needs explicit authorization or removal.
✅ **Test** - passed

✅ No issues found.

  • Reviewed the target commit’s executable delayed-readiness regression and Pi-only implementation diff
  • bash tests/fm-pi-watch-extension.test.sh
  • git diff --check f42a6291d4335cc7e169660bd7114239c3830a08 354d61070f6b541895dd7380b6b13c3cc70fd2f6
✅ **Document** - passed

✅ No issues found.

✅ **Lint** - passed

✅ No issues found.

✅ **Push** - passed

✅ No issues found.

@greptile-apps

greptile-apps Bot commented Sep 1, 2026 •

Copy link
Copy Markdown

Confidence Score: 5/5

The PR appears safe to merge.

No blocking failure remains.

Reviews (8): Last reviewed commit: "fix(pi): bound queued-close readiness wa..." | Re-trigger Greptile

Comment thread .pi/extensions/fm-primary-pi-watch.ts
@kunchenguid

Copy link
Copy Markdown
Owner

Speaking as Kun's firstmate:

HEAD 28b437d770f6fd9e104c3425d22bb02060593e03. Attestation MATCH. MERGEABLE/UNSTABLE vs main a5f3cbeeb71768bca2ac54c6926d314b6d27b836. workflow-zero. Fork CI already ran this HEAD: CI 33489164438 SUCCESS, Require no-mistakes 33489164436 SUCCESS.

Contract-class: new-default. Unconfigured Pi watcher now coalesces delayed follow-ups, snapshots fm-crew-state.sh --activity-token, and changes watch-arm / wake-ticket / validation-activity defaults. A "fix" title does not make a new unconfigured supervision path restore. No auto-merge.

Greptile is not a required check on this repo (do not ping it). Its P1 is still a real defect to inspect: .pi/extensions/fm-primary-pi-watch.ts:742 — when an actionable close is queued behind a failure close, a retry starts a replacement arm without resuming those pending closes, so a healthy replacement can leave a required decision/failure/risk undelivered. That would hide a failure. Not auto-merge-ready until that path is closed or shown unreachable.

Overlaps #3441 on docs/watcher-continuity.md (3441 is turnend-guard recovery). Also touches bin/fm-crew-state.sh in the same area as #3381.

Waiting on the author, not the captain: address the stranded-pending-close retry, or show it cannot happen. After that it still needs a captain default-behavior decision.

VISION.md per-rule

  • One captain, one interface — cannot tell until the close-queue retry is resolved; stranding a failure/decision hides news.
  • Authority is explicit and never inferred — does not align as shipped (new default coalescing/activity path, not opt-in).
  • Scripts own the mechanics, agents own the judgment — aligns (watcher/scripts own coalescing).
  • A restart is a non-event — aligns as motive (generation-bound tickets); inconclusive on the stranded-close path.
  • Delegation with a spine — aligns (tests for tickets / re-arm).
  • The fleet outlives any vendor — aligns (Pi delivery tickets, not UI pixels).
  • Scope — aligns as command-layer supervision, resisted as a default-behavior widen.

Comment thread .pi/extensions/fm-primary-pi-watch.ts Outdated
Comment thread .pi/extensions/fm-primary-pi-watch.ts
@M00NLIG7
M00NLIG7 force-pushed the fm/firstmate-pi-wake-coalescing-public-pr-v1-k9 branch from afab8c9 to 79a6ea1 Compare September 1, 2026 18:01
@M00NLIG7 M00NLIG7 changed the title fix(pi): coalesce stale delayed wake notifications fix(pi): revalidate delayed wake delivery Sep 1, 2026
Comment thread .pi/extensions/fm-primary-pi-watch.ts
@kunchenguid

Copy link
Copy Markdown
Owner

Speaking as Kun's firstmate:

HEAD 12f1f4a60cb48d90d770511aa7a8e885381c3850 (moved past the prior 79a6ea stamp). Attestation MATCH (head_sha 12f1f4a). MERGEABLE/CLEAN vs main 355f46fe5528ccc9790481171bf9da48dee2e90d. workflow-zero. Fork CI already running this HEAD; no new approval this pass.

Contract-class: new-default. Re-verified on this HEAD; a "fix" title still does not make restore. Unconfigured Pi watcher coalesces delayed follow-ups; fm-watch.sh wedge timers now snapshot fm-crew-state.sh --activity-token and defer escalation on validation-log progress for every watcher, not Pi only; crew-state default classification lets current agent-wait outrank a stale terminal outcome; Pi delivery adds wake tickets / receipts behind FM_PI_WATCH_DELIVERY=1. No auto-merge.

Stranded-pending-close on this HEAD is fixed, not waiting on the author:

  • scheduleRetry always resumes the close queue after startArm (.pi/extensions/fm-primary-pi-watch.ts:727).
  • Failure rows are peeked, not shifted first (:739-753): an established successor surfaces the failure once without re-arming (:742-745); an unestablished child keeps the failure queued (:750) until it establishes or closes (finally :776-779 plus observeEstablishedArm :856).
  • Covering tests: test_pi_retry_resumes_actionable_close_queued_behind_failure, lock-loss resume, and test_pi_healthy_successor_surfaces_failure_queued_during_restore.

Current required no-mistakes check is the later edited-event run 33543542780 SUCCESS. Synchronize run 33543539328 FAILURE is the stale body-attestation race on the same SHA and is not the current required conclusion (GitHub merge state is CLEAN). CI 33543539426 SUCCESS (Lint/tests including Herdr and timing aggregate). Greptile SUCCESS 5/5 on 12f1f4a ("no blocking failure remains"); stale inline P1 threads from earlier SHAs are leftover mappings, not a live defect. Security: no workflows, no fake pins, no disguised gates; author M00NLIG7 is not blocked.

Overlaps #3441 on docs/watcher-continuity.md and #3381 on bin/fm-crew-state.sh (stale-outcome vs current gate).

This is a captain-decision hold, not waiting on the author. Default-behavior supervision widen still needs an explicit captain grant.

VISION.md per-rule

  • One captain, one interface — aligns now (queued failure surfaces once beside a healthy successor; delayed obsolete follow-ups are suppressed, not hidden decisions).
  • Authority is explicit and never inferred — does not align as shipped (new default coalescing / activity-token wedge / crew-state gate order, not opt-in).
  • Scripts own the mechanics, agents own the judgment — aligns (watcher/scripts own coalescing, tickets, activity tokens).
  • A restart is a non-event — aligns (generation-bound tickets, receipts, successor continuity).
  • Delegation with a spine — aligns (executable regressions for tickets / re-arm / queued close).
  • The fleet outlives any vendor — aligns (Pi delivery tickets, not UI pixels; other adapters keep one-line output).
  • Scope — aligns as command-layer supervision; resisted as a default-behavior widen.

…after a successful replacement arm, preventing actionable notifications behind a failure close from being stranded. Added an executable regression covering failure -> queued actionable close -> healthy retry delivery. Verified with `bash -n tests/fm-pi-watch-extension.test.sh`, `git diff --check`, and the focused Pi watcher test suite
…led retry clears its timer, pending closes now resume whether `startArm` succeeds or fails. Added an executable lock-loss regression proving a queued actionable wake is delivered with the continuity failure and no successor is launched. Verified: `bash tests/fm-pi-watch-extension.test.sh`, `bash -n tests/fm-pi-watch-extension.test.sh`, and `git diff --check` all pass
…ons/fm-primary-pi-watch.ts`: queued failures now remain queued while a successor establishes, resume once it establishes or closes, and cannot cause a retry-loop spin. Verified `git diff --check`, `node --check`, `bash -n`, and the focused Pi watcher suite (exit 0)
@M00NLIG7
M00NLIG7 force-pushed the fm/firstmate-pi-wake-coalescing-public-pr-v1-k9 branch from 12f1f4a to fd249cf Compare September 2, 2026 00:23
…secondmate-reconcile.test.sh. The failing assertion used a fixed 200 ms delay before checking a lock-contended notifier, so a loaded CI runner could fail before the child was scheduled. It now polls for prompt completion for up to 2 seconds, preserving failure for a genuinely blocked reconcile path. `bash -n tests/fm-secondmate-reconcile.test.sh` and `git diff --check` pass. The separate no-mistakes attestation failure is external to source: the PR body attests 12f1f4a… while the current head is fd249cf…, and must be refreshed by the outer pipeline push
Comment thread .pi/extensions/fm-primary-pi-watch.ts Outdated
@kunchenguid

Copy link
Copy Markdown
Owner

Speaking as Kun's firstmate:

HEAD 940beaf29c24e420a29c75c02e1a7cba849a65ac (moved past the prior waiting-captain stamp on 12f1f4a). Attestation MATCH (head_sha 940beaf). MERGEABLE/UNSTABLE vs main 7d4b5177b4ed999db46ca3570af2d776bc39b0ea (branch ahead 10 / behind 2). workflow-zero. No merge and no rebase — captain still holds merge authority.

What changed since waiting-captain 2026-09-01T18:45:22Z: Author/pipeline activity rewrote the PR tip. History was re-committed (committedDate 2026-09-02T00:13Z), then tip 940beaf (2026-09-02T00:51:24Z) — flake-only fix in tests/fm-secondmate-reconcile.test.sh (fixed 200 ms sleep → poll up to 2 s for lock-contended notifier). Product contract unchanged from the prior captain-hold surface. Overlaps #3441 (docs/watcher-continuity.md) and #3381 (bin/fm-crew-state.sh) remain.

Contract-class: new-default. Re-verified on this HEAD; a "fix" title still does not make restore. Unconfigured Pi watcher coalesces delayed follow-ups; fm-watch.sh wedge timers snapshot fm-crew-state.sh --activity-token and defer escalation on validation-log progress for every watcher; crew-state default classification lets current agent-wait outrank a stale terminal outcome; Pi delivery adds wake tickets / receipts behind FM_PI_WATCH_DELIVERY=1. No auto-merge.

Stranded-pending-close remains fixed on this HEAD (same design as prior stamp):

  • scheduleRetry always resumes the close queue after startArm (.pi/extensions/fm-primary-pi-watch.ts:708-730).
  • Failure rows are peeked, not shifted first (:742-753): established successor surfaces the failure once without re-arming; unestablished child keeps the failure queued (:750) until it establishes (observeEstablishedArm :849-857) or closes (finally :775-779).
  • Tip commit does not touch that path.

CI 33577027851 SUCCESS (Lint, Herdr, all portable shards, timing aggregate). Required no-mistakes is the later edited-event run 33577030673 SUCCESS; synchronize run 33577027872 FAILURE is the stale body-attestation race on the same SHA and is not the current required conclusion. Greptile FAILURE 4/5 on 940beaf with a new P1 at :750 ("Unready successor strands failures") — Greptile is not a required check; this restates the intentional keep-queued-until-establish-or-close design already accepted at the prior captain stamp (actionable hung-successor path still uses waitForReadiness + retireArm). Not flipping to waiting-author for that residual. Security: no workflows, no fake pins, no disguised gates; author M00NLIG7 is not blocked.

This remains a captain-decision hold, not waiting on the author. Default-behavior supervision widen still needs an explicit captain grant. Do not merge. Do not rebase while the captain decision is pending.

VISION.md per-rule

  • One captain, one interface — aligns (queued failure surfaces once beside a healthy successor; delayed obsolete follow-ups are suppressed, not hidden decisions).
  • Authority is explicit and never inferred — does not align as shipped (new default coalescing / activity-token wedge / crew-state gate order, not opt-in).
  • Scripts own the mechanics, agents own the judgment — aligns (watcher/scripts own coalescing, tickets, activity tokens).
  • A restart is a non-event — aligns (generation-bound tickets, receipts, successor continuity).
  • Delegation with a spine — aligns (executable regressions for tickets / re-arm / queued close).
  • The fleet outlives any vendor — aligns (Pi delivery tickets, not UI pixels).
  • Scope — aligns as command-layer supervision; resisted as a default-behavior widen.

@kunchenguid

Copy link
Copy Markdown
Owner

Speaking as Kun's firstmate: captain word on this hold — hold for now. Not waiting on the author.

HEAD still 940beaf29c24e420a29c75c02e1a7cba849a65ac. Class new-default unchanged. Do not merge. Do not rebase (branch is CONFLICTING/DIRTY vs main; leave it). CI left as-is.

Reason: Pi supervision is in active flux; the captain will revisit when it settles. Firstmate will not re-flag this unless the product surface changes or the captain asks.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants