Skip to content

fix(pi): reconcile the multi-brain Pi supervision architecture from upstream - #9

Closed
verbagem wants to merge 6 commits into
mainfrom
fm/firstmate-pi-multibrain-fork-reconcile-5834
Closed

verbagem wants to merge 6 commits into
mainfrom
fm/firstmate-pi-multibrain-fork-reconcile-5834

Conversation

@verbagem

Copy link
Copy Markdown
Owner

Status caveat: this is a direct-PR reconciliation task, so the repo's PR must be raised via no-mistakes check is expected to fail (precedent: PR #7, PR #8).

PR: https://github.com/verbagem/firstmate/pull/new/fm/firstmate-pi-multibrain-fork-reconcile-5834
Branch: fm/firstmate-pi-multibrain-fork-reconcile-5834
Base: verbagem/firstmate:main
Head: verbagem/firstmate:fm/firstmate-pi-multibrain-fork-reconcile-5834

What this brings in

Kun Chen's multi-brain Pi supervision architecture ("one agent, two branches
of attention, events are commits") was already implemented, reviewed, and
merged on kunchenguid/firstmate (this repo's upstream template) but absent
from verbagem/firstmate — ordinary fork lag: the last fork sync (PR #7,
merged 2026-08-21) predates the feature landing upstream (2026-08-21..24).
Captain decision: reconcile it in, using upstream only as read-only
evidence, never touching kunchenguid/firstmate.

Cherry-picked from kunchenguid/firstmate:main, in this order:

Plus one manual commit (489b7e9) reconstructing content that a partial
cherry-pick had silently dropped, caught by the strict Pi extension
typecheck (see "How this was verified").

Deliberate deviation from PR #7's technique

PR #7 merged origin/main wholesale. A wholesale merge of the current
origin/main (038d0f7) into verbagem/main was tried first and produces
106 conflicting files across unrelated subsystems (GitLab merge support,
durable steering inboxes, the spoken interface, decision-collapsing,
CI attestation, etc.) — verbagem/main's last sync was itself a squash
merge (PR #7), which discarded the shared-ancestry chain a 3-way merge needs,
so every file either side touched since is a spurious "add/add" conflict
even where content already converges. That merge was aborted; a scoped
cherry-pick of the 4 upstream commits that are the actual feature (plus one
verified prerequisite) was used instead, matching PR #8's precedent of
reconstructing accepted changes onto a clean verbagem/main base rather
than importing unrelated upstream drift.

3f03533 is not itself a multi-brain commit, but 7b88520 was developed on
a long-running branch and its squash-merge diff (against its immediate
parent) assumes 3f03533's confirmHandlingDelivery/WithRetry functions
and the owner.restoring re-entrancy guard already exist — they were
already on origin/main before 7b88520. A first pass without 3f03533
looked complete after manual conflict resolution (no error), but the strict
typecheck caught confirmHandlingDelivery referenced-and-undefined, and a
portable test (test_pi_actionable_close_starts_single_successor_before_delivery)
caught the stale confirm-after-send assertion ordering. 3f03533 was
cherry-picked to close both gaps rather than hand-patching around them.

Deliberately excluded (not "already-implemented upstream" — still open,
unreviewed PRs on kunchenguid): fm-supervision-unsplit-wakes-r1
(commits 7f44de6, f7324bf) and fm-branch-note-sailboat-r2
(93ba8c0). Neither is an ancestor of origin/main. A future reconciliation
should pick these up once they land upstream.

Requirements

Requirement Result Evidence
Do not open or modify any PR against kunchenguid/firstmate. pass Only PR created is this one, against verbagem/firstmate; only push was git push verbagem fm/firstmate-pi-multibrain-fork-reconcile-5834.
Use the existing guarded fork-reconciliation path. pass Followed PR #7/#8's own precedent (direct-PR to verbagem/firstmate, no-mistakes' PR gate knowingly failing because its push-target config points at origin=kunchenguid by design, per no-mistakes status).
Include the multi-brain implementation and its follow-up fixes/docs/tests. pass 4 upstream commits + 1 verified prerequisite; see commit graph.
Preserve fork-specific accepted changes. pass Scoped cherry-pick onto verbagem/main tip (3d8426c); no unrelated origin drift imported (e.g. bin/fm-send.sh's durable-inbox feature, out of scope, correctly absent).
Resolve conflicts semantically. pass Every conflict inspected individually (see commit 489b7e9 and PR description above); typecheck + full portable test sweep used to catch a silent drop, not just "accept a side."
Never merge, force-push main, or discard work. pass No merge command run; only a feature-branch push.
Run the complete required validation. pass See below.

How this was verified

  • PATH=".../typescript/bin:$PATH" bash tests/fm-pi-primary-types.test.sh — pass (strict no-emit typecheck against installed Pi 0.82.1); .pi/extensions/fm-primary-pi-watch.ts is byte-identical to origin/main's copy.
  • bash tests/fm-pi-branch-extension.test.sh — 13/13 pass.
  • bash tests/fm-branch-supervision.test.sh — 14/14 pass (includes the branch-cannot-merge/destructive-act authority tests).
  • bash tests/fm-pi-watch-extension.test.sh — 34/34 pass.
  • bash tests/fm-wake-queue.test.sh — 1 pre-existing failure (watcher did not exit for first signal), reproduced identically on a clean verbagem/main checkout before this change — not a regression.
  • bash tests/fm-session-start.test.sh — 1 pre-existing failure (the digest waited for the 12s unreachable-host probe...), reproduced identically on a clean verbagem/main checkout — not a regression.
  • bash tests/fm-watch-triage.test.sh — 14/14 pass.
  • bash tests/fm-gotmp.test.sh, fm-test-run.test.sh, fm-control.test.sh, fm-pr-merge.test.sh, fm-spawn-batch.test.sh, fm-send-strict.test.sh, fm-send-resolve-key.test.sh — all pass.
  • bash tests/fm-teardown.test.sh — 1 pre-existing failure (herdr-preflight-missing-adapter), reproduced identically on a clean verbagem/main checkout — not a regression.
  • CI=true bin/fm-lint.sh — pass (ShellCheck 0.11.0, actionlint 1.7.12, 3 workflow files valid).
  • bin/fm-doc-audience-check.sh — pass (surfaces=79 local_links=262).
  • bin/fm-ensure-agents-md.sh . — unchanged (AGENTS.md's auto-merge left the CLAUDE.md pointer intact).
  • git diff --check verbagem/main...HEAD and git diff --check origin/main...HEAD — both pass.

Commit graph

Changed files

43 files changed, 4830 insertions(+), 101 deletions(-) against verbagem/main.

Unverified or partial

  • CI had not completed at handoff time (only local bin/fm-lint.sh + focused/relevant bash tests/*.test.sh runs above; the full CI suite, including platform/Herdr/tmux/security lanes, was not run locally).
  • fm-supervision-unsplit-wakes-r1 / fm-branch-note-sailboat-r2 (open upstream PRs) intentionally not included — see "Deliberately excluded" above.

kunchenguid and others added 6 commits August 24, 2026 19:50
* wip: forked supervision on Pi (checkpoint before docs)

* fix(pi-branch): harden mirror delivery, fallback encoding, and session replacement

Peek-then-shift mirror flush so a failed append retries instead of dropping;
durable mirror cursor commits only after delivery into the branch;
the main fallback wake is operational-encoded like every watcher injection;
session_shutdown quiesces the generation and session_start re-arms, so /new
and /resume no longer kill the branch permanently. Registers the extension in
the strict typecheck, adds the dispatch handshake test, the branch extension
suite, the bash-level regression suite, the session-start replay test, and
the opt-in real-SDK live guard.

* test(fixtures): carry the branch-dispatch lib and lease lib into isolated fixtures

The watcher extension now imports lib/fm-branch-dispatch.ts and fm-teardown
sources fm-lease-lib.sh, so every fixture that copies or symlinks those
files in isolation gains the new sibling.

* no-mistakes(review): Prevent shutdown wake loss and serialize lease claims

* no-mistakes(review): Durably hand off wakes and retain portable leases

* no-mistakes(review): Require durable reports and clear disposed branch leases

* no-mistakes(review): Enforce per-wake outcomes and quiescent lease cleanup

* no-mistakes(review): Require wake acknowledgements and tighten branch lifecycle boundaries

* no-mistakes(review): Require complete acknowledgements and replay cleanup failures

* no-mistakes(review): Bind supervision to lock ownership and durable delivery

* no-mistakes(review): Activate branch lazily after session lock acquisition

* no-mistakes(review): Preserve undelivered mirror context across extension rebinds

* no-mistakes(review): Acknowledge startup replay only after main delivery

* no-mistakes(review): Isolate replay metadata from untrusted digest content

* no-mistakes(review): Reject duplicate reports for active wake sequences

* no-mistakes(review): Retain failed fallbacks and deduplicate outcome replay

* no-mistakes(review): Deduplicate durable outcomes and cache delivery receipts

* no-mistakes(review): Anchor wake sequence matching to outcome fields

* no-mistakes(document): Clarify Pi supervision durability contracts

* no-mistakes(lint): Fix ShellCheck issues in branch supervision scripts

* no-mistakes: apply CI fixes

* no-mistakes: apply CI fixes

* no-mistakes: apply CI fixes

* no-mistakes: apply CI fixes

* no-mistakes: apply CI fixes

* refactor(pi-branch): collapse to confused-agent-grade guards per captain decision

Captain decision A: the lease/actor guards target the CONFUSED-AGENT threat
model bin/fm-gate-refuse-lib.sh already documents; adversarial-grade
separation is impossible in the shared-process design and is filed as
separate follow-up work. Rip out the machinery that chased it: the
generation fence and shell-provenance markers, the wrapper-tagged ancestry
walks, guard auto-claim with per-script release traps, the pending-wake
files and ack-receipt correlation (the durable wake queue already
re-presents anything unacknowledged), the delivery-receipt store with
contiguous cursor advancement, the session-start replay-metadata channel,
and the branch tool quiescence counters.

Keep the behaviors the board requires, each on its simplest implementation:
lazy per-action session-lock ownership (cold start activates after the lock
lands; a secondary session stays inert), mirror durability across extension
rebinds via the durable cursor, replay-exactly-once from the one read
cursor, the awaited operational-encoded fallback, per-generation stray-lease
cleanup, session-lock-bound lease liveness (a recycled pid or a non-Pi home
never honors a leftover lease), the loud accidental-override guards
(readonly actor prelude, cross-actor claim refusal), and the role-partition
refinements (no forced teardown, no direct relaunch for the branch).
Default-on-for-Pi is unchanged.

* no-mistakes(review): Enforce lock ownership and serialize lease mutations

* no-mistakes(review): Synchronize guard cleanup and bind leases to lock owner

* no-mistakes(review): Report outcomes before acknowledging durable wakes

* no-mistakes(review): Restrict leases to Pi and instruct main claims

* no-mistakes(review): Reject malformed lease locks and torn outcome tails

* no-mistakes(review): Validate complete outcome tails before appending

* no-mistakes(review): Guard branch side effects across session replacements

* no-mistakes(document): Update Pi supervision durability and lease documentation

* no-mistakes(lint): Suppress intentional nested-shell expansion warning

* no-mistakes: apply CI fixes

* fix(pi-branch): authorize lease releases by caller

* fix(lint): break redundant source-analysis path in fm-lease-lib.sh

fm-lease-lib.sh's lazy fallback source of fm-wake-lib.sh gave ShellCheck's
--external-sources traversal a second path into an already 1540-line file
that fm-send.sh and fm-teardown.sh also source directly, blowing up the
recursive analysis past CI's lint timeout. Mark it a source=/dev/null
analysis boundary, matching the existing fm-task-inbox-lib.sh convention.

Also restores bin/fm-lint.sh and tests/fm-lint.test.sh to the shared
serial-lint definition (dropping an unrelated parallel-sharding change
that was itself hanging and masked this root cause).

* no-mistakes(document): Correct lease caller-authorization documentation

* no-mistakes: apply CI fixes

* no-mistakes: apply CI fixes

* no-mistakes: apply CI fixes

(cherry picked from commit 7b88520)
… icon (kunchenguid#2934)

* style(pi): restyle supervision merge notes with a sailboat and matching pad

Secondary-session notes were flush against the TUI edge and fully tinted.
Use the sailboat prefix, Pi's default outputPad, boat-only color, and dim remainder so they sit like real messages.

* style(pi): distinguish routine and captain merge notes by icon only

Visible notes now lead with a sailboat or anchor, then only the dim outcome.
Drop the branch-merged wording and verdict brackets so the icon is the only kind signal.

(cherry picked from commit 52ff62e)
…id#2938)

The markdown contract stays the owner; the still is only the visual of the idea.

(cherry picked from commit d55e00a)
kunchenguid#2858)

Cherry-picking 7b88520/8fa0505 alone silently dropped their prerequisite
content: confirmHandlingDelivery()/WithRetry() and the owner.restoring
re-entrancy guard were introduced by 3f03533 (fix: bound recovery
announcements and preserve supervision, kunchenguid#2733), which landed on
kunchenguid/firstmate main before 7b88520 and was therefore absent from
each later commit's own diff. tsc caught the missing confirmHandlingDelivery
reference; manually reconstructed to match origin/main's current file
(now byte-identical for .pi/extensions/fm-primary-pi-watch.ts).

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016KZ15GDC21fsLKVtsXnkLJ
…id#2733)

* fix(watch): announce recovery once per generation and keep successors supervising

A lost Pi/OpenCode handling handshake re-announced the same recovery
generation on every cycle and spent the successor's first ~55s blind, so
a real crew event could be ignored and then dropped. Record the
announcement in the durable marker, confirm the handshake before the
follow-up without swallowing failure, and enter the poll loop immediately.

* no-mistakes(review): Tighten recovery event timing regression

* no-mistakes(document): Document recovery-loop supervision guarantees

(cherry picked from commit 3f03533)
verbagem added a commit that referenced this pull request Aug 27, 2026
* feat: add persistent Pi supervision branch (kunchenguid#2858)

* wip: forked supervision on Pi (checkpoint before docs)

* fix(pi-branch): harden mirror delivery, fallback encoding, and session replacement

Peek-then-shift mirror flush so a failed append retries instead of dropping;
durable mirror cursor commits only after delivery into the branch;
the main fallback wake is operational-encoded like every watcher injection;
session_shutdown quiesces the generation and session_start re-arms, so /new
and /resume no longer kill the branch permanently. Registers the extension in
the strict typecheck, adds the dispatch handshake test, the branch extension
suite, the bash-level regression suite, the session-start replay test, and
the opt-in real-SDK live guard.

* test(fixtures): carry the branch-dispatch lib and lease lib into isolated fixtures

The watcher extension now imports lib/fm-branch-dispatch.ts and fm-teardown
sources fm-lease-lib.sh, so every fixture that copies or symlinks those
files in isolation gains the new sibling.

* no-mistakes(review): Prevent shutdown wake loss and serialize lease claims

* no-mistakes(review): Durably hand off wakes and retain portable leases

* no-mistakes(review): Require durable reports and clear disposed branch leases

* no-mistakes(review): Enforce per-wake outcomes and quiescent lease cleanup

* no-mistakes(review): Require wake acknowledgements and tighten branch lifecycle boundaries

* no-mistakes(review): Require complete acknowledgements and replay cleanup failures

* no-mistakes(review): Bind supervision to lock ownership and durable delivery

* no-mistakes(review): Activate branch lazily after session lock acquisition

* no-mistakes(review): Preserve undelivered mirror context across extension rebinds

* no-mistakes(review): Acknowledge startup replay only after main delivery

* no-mistakes(review): Isolate replay metadata from untrusted digest content

* no-mistakes(review): Reject duplicate reports for active wake sequences

* no-mistakes(review): Retain failed fallbacks and deduplicate outcome replay

* no-mistakes(review): Deduplicate durable outcomes and cache delivery receipts

* no-mistakes(review): Anchor wake sequence matching to outcome fields

* no-mistakes(document): Clarify Pi supervision durability contracts

* no-mistakes(lint): Fix ShellCheck issues in branch supervision scripts

* no-mistakes: apply CI fixes

* no-mistakes: apply CI fixes

* no-mistakes: apply CI fixes

* no-mistakes: apply CI fixes

* no-mistakes: apply CI fixes

* refactor(pi-branch): collapse to confused-agent-grade guards per captain decision

Captain decision A: the lease/actor guards target the CONFUSED-AGENT threat
model bin/fm-gate-refuse-lib.sh already documents; adversarial-grade
separation is impossible in the shared-process design and is filed as
separate follow-up work. Rip out the machinery that chased it: the
generation fence and shell-provenance markers, the wrapper-tagged ancestry
walks, guard auto-claim with per-script release traps, the pending-wake
files and ack-receipt correlation (the durable wake queue already
re-presents anything unacknowledged), the delivery-receipt store with
contiguous cursor advancement, the session-start replay-metadata channel,
and the branch tool quiescence counters.

Keep the behaviors the board requires, each on its simplest implementation:
lazy per-action session-lock ownership (cold start activates after the lock
lands; a secondary session stays inert), mirror durability across extension
rebinds via the durable cursor, replay-exactly-once from the one read
cursor, the awaited operational-encoded fallback, per-generation stray-lease
cleanup, session-lock-bound lease liveness (a recycled pid or a non-Pi home
never honors a leftover lease), the loud accidental-override guards
(readonly actor prelude, cross-actor claim refusal), and the role-partition
refinements (no forced teardown, no direct relaunch for the branch).
Default-on-for-Pi is unchanged.

* no-mistakes(review): Enforce lock ownership and serialize lease mutations

* no-mistakes(review): Synchronize guard cleanup and bind leases to lock owner

* no-mistakes(review): Report outcomes before acknowledging durable wakes

* no-mistakes(review): Restrict leases to Pi and instruct main claims

* no-mistakes(review): Reject malformed lease locks and torn outcome tails

* no-mistakes(review): Validate complete outcome tails before appending

* no-mistakes(review): Guard branch side effects across session replacements

* no-mistakes(document): Update Pi supervision durability and lease documentation

* no-mistakes(lint): Suppress intentional nested-shell expansion warning

* no-mistakes: apply CI fixes

* fix(pi-branch): authorize lease releases by caller

* fix(lint): break redundant source-analysis path in fm-lease-lib.sh

fm-lease-lib.sh's lazy fallback source of fm-wake-lib.sh gave ShellCheck's
--external-sources traversal a second path into an already 1540-line file
that fm-send.sh and fm-teardown.sh also source directly, blowing up the
recursive analysis past CI's lint timeout. Mark it a source=/dev/null
analysis boundary, matching the existing fm-task-inbox-lib.sh convention.

Also restores bin/fm-lint.sh and tests/fm-lint.test.sh to the shared
serial-lint definition (dropping an unrelated parallel-sharding change
that was itself hanging and masked this root cause).

* no-mistakes(document): Correct lease caller-authorization documentation

* no-mistakes: apply CI fixes

* no-mistakes: apply CI fixes

* no-mistakes: apply CI fixes

(cherry picked from commit 7b88520)

* style(pi): distinguish routine and captain supervision merge notes by icon (kunchenguid#2934)

* style(pi): restyle supervision merge notes with a sailboat and matching pad

Secondary-session notes were flush against the TUI edge and fully tinted.
Use the sailboat prefix, Pi's default outputPad, boat-only color, and dim remainder so they sit like real messages.

* style(pi): distinguish routine and captain merge notes by icon only

Visible notes now lead with a sailboat or anchor, then only the dim outcome.
Drop the branch-merged wording and verdict brackets so the icon is the only kind signal.

(cherry picked from commit 52ff62e)

* docs(pi): add the approved multi-brain architecture poster (kunchenguid#2938)

The markdown contract stays the owner; the still is only the visual of the idea.

(cherry picked from commit d55e00a)

* feat(pi): default branch supervision and route heartbeats (kunchenguid#2939)

(cherry picked from commit 8fa0505)

* fix(pi-watch): restore confirm-before-send handling delivery (prereq of kunchenguid#2858)

Cherry-picking 7b88520/8fa0505 alone silently dropped their prerequisite
content: confirmHandlingDelivery()/WithRetry() and the owner.restoring
re-entrancy guard were introduced by 3f03533 (fix: bound recovery
announcements and preserve supervision, kunchenguid#2733), which landed on
kunchenguid/firstmate main before 7b88520 and was therefore absent from
each later commit's own diff. tsc caught the missing confirmHandlingDelivery
reference; manually reconstructed to match origin/main's current file
(now byte-identical for .pi/extensions/fm-primary-pi-watch.ts).

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016KZ15GDC21fsLKVtsXnkLJ

* fix: bound recovery announcements and preserve supervision (kunchenguid#2733)

* fix(watch): announce recovery once per generation and keep successors supervising

A lost Pi/OpenCode handling handshake re-announced the same recovery
generation on every cycle and spent the successor's first ~55s blind, so
a real crew event could be ignored and then dropped. Record the
announcement in the durable marker, confirm the handshake before the
follow-up without swallowing failure, and enter the poll loop immediately.

* no-mistakes(review): Tighten recovery event timing regression

* no-mistakes(document): Document recovery-loop supervision guarantees

(cherry picked from commit 3f03533)

* fix(pi): correct the reconciled multi-brain supervision branch

Corrections found during PR #9's recovery onto current main:

- Fix the real CI failure. The multi-brain feature roughly doubled the
  portable-serial lane's total weight (2.19M ms vs the 1.14M ms baseline
  the sharding was designed around) without a matching shard-count
  increase, silently compressing the CI job timeout's intended ~3x
  hang-tripwire margin to under 2x. "Behavior portable serial 4" was
  cancelled at almost exactly the 15-minute job boundary on both PR #9
  CI attempts, mid-script, not inside any single hanging test - a margin
  failure, not a hang. Raise PORTABLE_SERIAL_SHARDS 4 -> 8, restoring
  each shard to ~273s (comfortably >3x under the unchanged 900s cap);
  docs/fm-test-portable-shards.md and the CI workflow comment carry the
  corrected numbers.

- Fix the outcome-cursor defect. bin/fm-branch-outcome.sh's read cursor
  is one monotonic sequence number, not a per-row acknowledgement, so a
  bare "mark-read --through N" could silently bury a lower-seq row still
  unread (a prior merge that lost lock ownership after appending but
  before sending, or after sending but before the cursor advanced).
  mergeIntoMain now redelivers every such stale row, in order, before
  delivering its own outcome and only then advances the cursor past
  everything it actually delivered.

- Fix misleading merge-failure reporting. A merge that sent its note but
  then lost lock ownership before the cursor advance used to report
  "merge refused" to the calling agent - false, since the note had
  already reached main - which could prompt a duplicate re-report.
  mergeIntoMain now returns delivered/cursorAdvanced separately so the
  fm_branch_report tool only ever reports true refusal when nothing was
  sent.

Regression coverage: two new tests in tests/fm-pi-branch-extension.test.sh
(stale-unread redelivery, cursor-advance-failure reporting).

Preserves PR #9's 6 cherry-picked commits (persistent Pi supervision
branch, its follow-up fixes, docs, and tests) reconstructed onto current
verbagem/main rather than the stale 3d8426c base, with the tests/fm-pi-
watch-extension.test.sh and .pi/extensions/fm-primary-pi-watch.ts merge
conflicts resolved by keeping both sides' independently-added tests and
functions rather than picking one.

Verified: strict Pi extension typecheck, fm-pi-branch-extension.test.sh
(15/15 incl. new), fm-branch-supervision.test.sh (14/14),
fm-pi-watch-extension.test.sh (36/36), fm-test-run.test.sh (portable
shard coverage/partition), fm-shared-captain-inheritance.test.sh,
fm-calm-pi-extension.test.sh, fm-watcher-lock.test.sh,
fm-session-start.test.sh, fm-spawn-*.test.sh, fm-send-*.test.sh,
fm-control-relaunch.test.sh, fm-wake-queue.test.sh, shellcheck + actionlint.

Superseded: #9

* no-mistakes(review): keep outcome cursor unadvanced when redelivery unproven

* no-mistakes(document): clarify outcome-cursor delivery proof in owner docs

---------

Co-authored-by: Kun Chen <3233006+kunchenguid@users.noreply.github.com>
Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
@verbagem

Copy link
Copy Markdown
Owner Author

Superseded by the no-mistakes-certified replacement: #13

@verbagem verbagem closed this Aug 27, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants