feat(bin): add captain-visible no-mistakes observers

[eyevanovich]

Intent

Implement Firstmate's default captain-visible no-mistakes observer experience: after the worker starts validation and an authoritative branch-matched run ID exists, open at most one separate non-focused observer terminal running 'no-mistakes attach --run ' without transferring any axi run/respond, cancellation, CI, ask-user, or merge ownership away from the worker. Support tmux and Herdr with trusted task/run/backend/worker/observer identity, idempotent retries, durable q/exit detach tombstones, explicit reopen only, exact observer-only cleanup integrated into task teardown, and bounded failure that preserves validation while printing the exact manual attach command. Keep zellij, Orca, and cmux on safe manual fallback until separately proven. Keep owning instructions in the lifecycle script with only concise trigger/safety pointers in shared docs. Cover harness invocation, authoritative discovery, separation, mismatch/malformed/unreadable state, interrupted create reconciliation, detach/reopen, fallback, and cleanup deterministically, plus an isolated tmux smoke test. Per captain decision, do not add live Herdr proof; document a concise captain-run manual verification checklist and keep the approved scope minimal.

What Changed

• Add a validation entrypoint that discovers authoritative branch runs and opens one non-focused, captain-visible no-mistakes observer in tmux or Herdr while preserving worker ownership.
• Persist trusted observer lifecycle state for idempotent retries, detach tombstones, explicit reopen, bounded manual fallback, and exact observer-only teardown cleanup.
• Document configuration and Herdr verification, with deterministic lifecycle coverage and an isolated tmux smoke test.

Risk Assessment

✅ Low: The change now consistently enforces token-bound identity, idempotent observer creation and detach handling, exact cleanup, safe fallback, and narrowly reconciled interrupted states without a remaining substantiated defect.

Testing

The supplied full baseline was green; focused lifecycle, live isolated-tmux, and teardown tests also passed, and the captured transcript demonstrates one non-focused sibling observer, durable q-detach with exact manual fallback, explicit-only reopen, and observer-only cleanup preserving the worker. Herdr behavior remains deterministic fake-CLI coverage with the intentionally documented captain-run checklist; no live Herdr proof was added per scope. No screenshot was captured because the smoke runs on a detached isolated tmux socket without a rendered GUI surface, so a direct terminal lifecycle transcript was used.

Evidence: Captain-visible tmux observer lifecycle transcript

observer: opened task task run run-live at @1 (tmux, no focus)
EVIDENCE after-open
window=@0 name=fm-task active=1 pane=%0
window=@1 name=nm-observer-task-8c1e568b active=0 pane=%1
task=task
run=run-live
worker_backend=tmux
worker_target=crew:fm-task
observer_backend=tmux
observer_target=@1
status=attached
EVIDENCE after-q status=detached
observer: the observer is detached and will not be respawned automatically
observer: attach manually with: cd '/var/folders/j_/g4b3__rs5j95fq6xf9fb17sc0000gn/T//fm-no-mistakes-observer-tmux.GZSX7b/repo-wt' && no-mistakes attach --run 'run-live'
EVIDENCE explicit-reopen
observer: opened task task run run-live at @2 (tmux, no focus)
EVIDENCE cleanup record_exists=no
EVIDENCE surviving-worker
window=@0 name=fm-task active=1 pane=%0
ok - live tmux observer uses one no-focus sibling, q detaches, reopen is explicit, and cleanup is exact

Pipeline

Updates from git push no-mistakes

✅ **intent** - passed

✅ No issues found.

✅ **Rebase** - passed

✅ No issues found.

🔧 **Review** - 2 issues found → auto-fixed (8) ✅

• 🚨 bin/fm-no-mistakes-observer.sh:804 - The required “idempotent retries” behavior is contradicted here: start adopts an existing branch run only when its top-level status is exactly running. Other active states already recognized elsewhere in this repository, such as awaiting_approval and fix_review, fall through and send the validation invocation again, potentially starting duplicate validation. Treat every authoritative nonterminal branch run as existing before invoking the worker.
• 🚨 bin/fm-no-mistakes-observer.sh:648 - The required “interrupted create reconciliation” and “idempotent retries” are incomplete for Herdr. The record becomes ready before the separate send-text and Enter operations; interruption after text is sent leaves the command in the composer, and retry sends it again before Enter, potentially executing a concatenated invalid command instead of attaching. Persist an intermediate launch phase or safely verify/clear the composer before retrying.

🔧 Fix: Harden observer retry idempotence
1 error still open:

• 🚨 bin/fm-no-mistakes-observer.sh:664 - The required “idempotent retries” behavior still has a Herdr race. After Enter succeeds, this resets the record to ready while the _session process cannot mark it attached until the caller releases the lifecycle lock. A retry that acquires the lock first sees ready and sends the full attach command again into the observer pane. Preserve a distinct submitted/launching state that _session can claim but ordinary retries refuse, so successful submission cannot reopen the text-sending path.

🔧 Fix: Prevent Herdr observer resubmission race
1 error still open:

• 🚨 bin/fm-no-mistakes-observer.sh:905 - The required “durable q/exit detach tombstones” behavior is still incomplete. If _session cannot claim the submitted record within its bounded retry loop—for example after the launcher dies while leaving its lock directory—it exits here without changing the record. Ordinary retries and explicit reopen then preserve submitted forever as “awaiting attachment,” although the observer process has exited. On claim timeout, safely tombstone the same task/run/token record as detached or failed without resending text or Enter.

🔧 Fix: Tombstone unclaimed Herdr observer sessions
2 errors still open:

• 🚨 bin/fm-no-mistakes-observer.sh:909 - The required bounded exit reconciliation is still incomplete. After the fixed claim wait, _session tombstones only if it can immediately acquire the lifecycle lock, and then only for submitted. A launcher or concurrent status operation can legitimately hold the lock past this point, leaving Herdr permanently submitted; tmux can similarly remain ready after its wrapper exits. Subsequent retries can therefore preserve or report an observer that has no attach process. Reconcile the same-token launch state after lock release within a bounded path for both backends.
• 🚨 bin/fm-no-mistakes-observer.sh:883 - The required trusted identity and idempotent lifecycle are contradicted because claiming an observer is not atomic with the lifecycle lock. _session checks that the lock directory is absent, then loads and rewrites the record without acquiring it; cleanup can acquire the lock and remove the record between those operations, after which _session resurrects stale state as attached and runs the old attach command. Acquire and hold the task lock while validating and transitioning the same task/run/token record, then release it before the foreground attach.

🔧 Fix: Lock observer session lifecycle transitions
1 error still open:

• 🚨 bin/fm-no-mistakes-observer.sh:933 - The required durable exit tombstone and bounded-failure behavior can still be lost to normal lock contention. Claim-timeout and attach-exit reconciliation each make one bounded transition attempt and silently ignore exhaustion, while another lifecycle operation may hold the same lock across several independently bounded status/terminal calls. The observer can therefore exit on q while remaining attached (or die while ready/submitted), blocking explicit reopen and printing no manual attach fallback. Make the exit/timed-out transition durable beyond transient lock ownership without resurrecting removed records.

🔧 Fix: Persist observer detach across lock contention
3 errors still open:

• 🚨 bin/fm-no-mistakes-observer.sh:324 - The required trusted token identity and durable exit handling can be lost because every generation writes the same task-wide pending path outside the lifecycle lock. A delayed old observer can overwrite a newer token’s pending detach after explicit reopen; consumption then discards the stale token mismatch and loses the newer exit evidence, potentially leaving its dead record attached. Use token-specific or otherwise non-clobbering pending transitions.
• 🚨 bin/fm-no-mistakes-observer.sh:944 - The required exact observer cleanup can race with late pending publication. Cleanup removes the record and current sidecar under the lock, but a contended _session may publish .observer.pending after cleanup releases it. Teardown only invokes observer cleanup when .observer exists and does not otherwise remove this late sidecar, leaving durable task/run/token state after teardown. Add a cleanup-safe publication handshake or unconditional race-safe pending cleanup.
• 🚨 bin/fm-no-mistakes-observer.sh:782 - The required explicit reopen behavior takes two commands when an attached observer endpoint has already exited. This branch tombstones it as detached and returns manual fallback even when the current action is reopen; only a second reopen creates the replacement. When allow_reopen=1, continue safely from the newly established detach tombstone in the same invocation.

🔧 Fix: Scope observer handoffs to generation tokens
1 error still open:

• 🚨 bin/fm-no-mistakes-observer.sh:514 - The required interrupted-create reconciliation and exact teardown cleanup still have a publication gap. Initialization creates the token generation directory before atomically writing the authoritative observer record; interruption between those operations leaves an unreferenced .<task>.observer-<token> directory. Later starts use another token, and teardown skips observer cleanup when .observer is absent, so the orphan survives task removal. Publish both transactionally or safely reconcile recordless task-owned generations.

🔧 Fix: Publish observer records before generation state
1 error still open:

• 🚨 bin/fm-no-mistakes-observer.sh:526 - The required interrupted-create reconciliation and teardown cleanup remain incomplete after the record-first change. Interruption after this record write but before generation creation leaves a valid creating record with no backend session or endpoint. Direct cleanup then treats the missing provisional identity as ambiguous and refuses teardown, although no observer could have been created. Recognize this same-token pristine pre-endpoint state as safely unlaunched and remove its record/generation without targeting a terminal.

🔧 Fix: Clean pristine unlaunched observer records safely
✅ Re-checked - no issues remain.

✅ **Test** - passed

✅ No issues found.

• command -v tmux >/dev/null || { echo "tmux is required for e2e tests" >&2; exit 1; }; tmux -V; rc=0; for t in tests/*.test.sh; do echo "== $t =="; bash "$t" || rc=1; done; exit "$rc"
• Baseline supplied by the gate: command -v tmux >/dev/null || { echo "tmux is required for e2e tests" >&2; exit 1; }; tmux -V; rc=0; for t in tests/*.test.sh; do echo "== $t =="; bash "$t" || rc=1; done; exit "$rc" (reported successful)
• bash tests/fm-no-mistakes-observer.test.sh
• bash tests/fm-no-mistakes-observer-tmux-smoke.test.sh
• bash tests/fm-teardown.test.sh
• Instrumented and ran the isolated tmux smoke flow to record actual window focus/identity, trusted observer state, q-detach tombstone, manual attach fallback, explicit reopen, exact cleanup, and surviving worker terminal in tmux-observer-lifecycle.txt

✅ **Document** - passed

✅ No issues found.

🔧 **Lint** - 1 issue found → auto-fixed ✅

• ⚠️ linter found issues (exit code 1)

🔧 Fix: Fix observer pending reset assignments, captain
✅ Re-checked - no issues remain.

✅ **Push** - passed

✅ No issues found.