Skip to content

fix(watcher): make check wakes lossless via watcher-side suppression - #34

Closed
e-jung wants to merge 5 commits into
kunchenguid:mainfrom
e-jung:fix/check-suppress-enqueue-before-suppress
Closed

e-jung wants to merge 5 commits into
kunchenguid:mainfrom
e-jung:fix/check-suppress-enqueue-before-suppress

Conversation

@e-jung

@e-jung e-jung commented Jun 21, 2026 •

Copy link
Copy Markdown
Contributor

What Changed

  • Moved check-wake suppression from opaque check scripts into the watcher: checks now print their current state idempotently, and the watcher dedups against .seen-check-<name> and enqueues to the durable queue before advancing that marker, so a crashed watcher or lost stdout can no longer swallow a check wake.
  • Added a catch-all backstop that scans .babysit-*.seen sidecars and force-escalates a swallowed terminal (MERGED/CLOSED) transition within one sweep, deduped via .escalated-<sidecar>, as a safety net for legacy edge-triggered checks.
  • Hardened fm-fleet-snapshot.sh: bounded secondmate-home validation with run_timed (a hung NFS/automount mount can no longer block the parent snapshot) and switched the seen-homes dedup from space-substring to newline-exact matching.

Risk Assessment

✅ Low: The incremental fix commit correctly resolves both prior warnings — validate_secondmate_home is now bounded by run_timed (with correct globals-via-stdout and rc 124/non-zero/zero handling, safe under set -u, no injection) and the seen_homes dedup now uses exact newline-delimited line matching (verified empirically: distinct homes and the space-prefix case are added, only genuine duplicates are caught) — and it introduces no new issues; the only remaining branch item is the acknowledged no-op watcher catch-all tradeoff from the prior round, requiring no action.

Testing

Completed 1 recorded test check.

  • Outcome: ⚠️ 1 error across 1 run (19m21s)

Pipeline

Updates from git push no-mistakes

⏭️ **intent** - skipped

✅ No issues found.

⚠️ **Rebase** - 1 warning

Push main to origin, or rebase your branch onto origin/main, before gating.

🔧 **Review** - 3 issues found → auto-fixed ✅
  • ⚠️ bin/fm-fleet-snapshot.sh:1016 - validate_secondmate_home is called directly (line 1016) without a run_timed wrapper, unlike every other cross-home read in this same commit (registry line ~430, parent-activity ~517, terminal ~546, summary ~1027). validate_secondmate_home (fm-ff-lib.sh:122-189) does ~8 stat/cd+pwd/cat syscalls against the home and its data/state/config/projects subdirs. A registered secondmate home on a stale NFS/automount path or hung mount blocks the parent snapshot for the mount timeout (60s+) with no recourse, directly violating the file's stated invariant at lines 62-63 ('one broken or unexpectedly large home cannot hang the parent snapshot'). Fix: route through run_timed and treat 124 as a validation failure; note validate_secondmate_home sets globals (VALIDATED_HOME/VALIDATION_ERROR) the caller reads, so the wrap must capture the result rather than run in a bare subshell.
  • ⚠️ bin/fm-fleet-snapshot.sh:1020 - The seen_homes dedup uses a space-delimited substring match: case &#34; $seen_homes &#34; in *&#34; $home &#34;*). If one validated secondmate home's canonical path is a space-prefixed prefix of another's (e.g. /a/foo registered before /a/foo bar, verified empirically: ' /a/foo ' matches inside ' /a/foo bar '), the second home is wrongly flagged 'invalid home: duplicate resolved home route' and its structured state is silently dropped from the snapshot. Unlikely for canonical secondmate paths but legal, and the fix is mechanical: use a newline-delimited set with exact per-line equality instead of substring matching.
  • ℹ️ bin/fm-watch.sh:685 - The new catch-all backstop (lines 685-700) dedups only against its own .escalated-<sidecar> marker, which is decoupled from the regular check path's .seen-check-<name> marker. For a check that both prints a terminal state idempotently AND carries a .babysit-*.seen sidecar (a hybrid/transitional check, or a migrated check with a leftover stale sidecar), the regular path wakes on sweep N and the catch-all emits one additional 'check: catch-all' wake on sweep N+1 for the same transition; the two records carry different wake keys (script path vs sidecar path) so drain does not collapse them. Bounded to once per terminal value (.escalated is then set), self-healing, and consistent with the stated lossless/belt-and-suspenders intent, but worth knowing since either path can change independently.

🔧 Fix: bound secondmate home validation and dedup exactly
✅ Re-checked - no issues remain.

⚠️ **Test** - 1 error
  • 🚨 tests failed with exit code 1
  • command -v tmux >/dev/null || { echo "tmux is required for e2e tests" >&2; exit 1; }; tmux -V; rc=0; for t in tests/*.test.sh; do echo "== $t =="; bash "$t" || rc=1; done; exit "$rc"
⚠️ **Document** - 1 info
  • ℹ️ AGENTS.md:108 - The state/ inventory line (.hash-* .count-* .stale-* ... .seen-* .hb-surfaced-* .last-* .heartbeat-streak watcher internals; never touch) does not enumerate the new .escalated-* watcher-internal marker this change introduced (written by the catch-all backstop). .seen-check-* is already covered by the existing .seen-* glob, but .escalated-* matches no glob on that line. This is not a falsehood — the line is an illustrative category label that already omits the pre-existing .babysit-*.seen marker, and the change authoritatively documents .escalated-* as a suppression marker in AGENTS.md:557. Adding only .escalated-* (without .babysit-*.seen) would make the inventory inconsistent; both-or-neither is the consistent choice, and .babysit-*.seen is out of scope for this change. Left as-is.
🔧 **Lint** - 1 issue found → auto-fixed ✅
  • ⚠️ linter found issues (exit code 1)

🔧 Fix: suppress SC2016 on intentional bash -c eval string
✅ Re-checked - no issues remain.

✅ **Push** - passed

✅ No issues found.

@kunchenguid

Copy link
Copy Markdown
Owner

Thanks for the PR. Wheelhouse found that this branch currently has a merge conflict with the base branch, so it is stepping out of the maintainer queue for now.

Please rebase on or merge the base branch into your branch, resolve the conflict, and push the result. Once GitHub reports the PR as mergeable again, Wheelhouse will resurface it for maintainer review.

Conflict noted for firstmate#34 at ae2d59d2.

e-jung added 5 commits July 14, 2026 22:38
Checks were the only wake source whose suppression lived inside an opaque
script: an edge-triggered check advanced its own .babysit-*.seen marker
before the print could become a wake, so a lost stdout (timeout / concurrent
run / crash) permanently swallowed the transition. This is the root cause of
the missed PR kunchenguid#3095 merge (see data/fm-git-events-s8/report.md): the
.cli-printing-press-3095.seen sidecar advanced to MERGED but no check wake
was ever emitted.

Port the kunchenguid#29 enqueue-before-suppress invariant to checks:

1. Watcher-side suppression (bin/fm-watch.sh). The check always prints its
   current state (idempotent); the watcher dedups against .seen-check-<name>
   and calls fm_wake_append (durable queue) BEFORE advancing the marker. A
   crash between detect and suppress leaves the wake in the queue (recovered
   next turn) and the marker un-advanced (re-detected next cycle). A lost
   check wake is now impossible - exactly the guarantee signals enjoy.

2. Backward-compatible with old edge-triggered checks: empty stdout never
   produces a wake, so they keep their quiet behavior.

3. Catch-all backstop: force-escalate any .babysit-*.seen sidecar showing a
   terminal state (MERGED/CLOSED) the watcher never delivered a wake for.
   Catches a swallowed transition within one sweep; deduped via
   .escalated-<sidecar> so each terminal state fires at most once.

Tests (tests/fm-wake-queue.test.sh):
- test_check_wake_survives_lost_delivery: the kunchenguid#3095 regression - a check wake
  survives a simulated crash between enqueue and suppress (queue recovery +
  re-detection).
- test_check_dedup_suppresses_repeats: identical repeated output wakes once.
- test_catch_all_escalates_swallowed_transition: a self-suppressed check whose
  sidecar shows MERGED is force-escalated within one sweep.

AGENTS.md: the check contract now documents the stateless "always print
current state" form as preferred, and .seen-check-*/.escalated-* join the
enqueue-before-suppress marker list.

shellcheck clean; all 14 wake-queue tests + 5 spawn-batch tests pass.
@e-jung
e-jung force-pushed the fix/check-suppress-enqueue-before-suppress branch from ae2d59d to a56af23 Compare July 15, 2026 01:21
@e-jung e-jung changed the title fix(watcher): lossless check wakes via watcher-side suppression + enqueue-before-suppress fix(watcher): make check wakes lossless via watcher-side suppression Jul 15, 2026
@kunchenguid

kunchenguid commented Jul 16, 2026 •

Copy link
Copy Markdown
Owner

Automated reminder: thanks for the PR! This branch currently has a merge conflict with the base branch.

When you get a chance, please rebase onto (or merge) the latest base branch, resolve the conflict, and push. After that, checks will re-run and the PR will get looked at again.

Noted for firstmate#34 at a56af233.

@e-jung

e-jung commented Jul 16, 2026

Copy link
Copy Markdown
Contributor Author

Closing as not planned because I have migrated my own orchestration workflow from FirstMate to Orca and will not keep this conflict-heavy branch current. The branch and discussion remain available as history. Thank you for the project and for considering the contribution.

@e-jung e-jung closed this Jul 16, 2026
eyevanovich added a commit to eyevanovich/firstmate that referenced this pull request Jul 23, 2026
## Intent

Fix the live GitLab note-verification regression blocking completion of
KissCut issue 146 after the guarded GitLab workflow changes. Preserve
idempotence so an existing note is reused without duplicates, and retain
trusted-project, positive-resource-ID, authenticated-author, exact-body,
non-system-note, regular-file, and noteable-resource guards. Use the
current GitLab work-item-backed issue note response shape, audit the
shared issue and merge-request note
creation/list/read-back/pagination/body-encoding identity path together,
and batch only directly implied note defects rather than broadening
scope. Add deterministic live-shape fixture coverage and validate
focused forge behavior, lint, and the portable suite. Do not mutate
KissCut issue 146 from this fix task.

## What Changed

- Accept GitLab work-item-backed issue notes while verifying the trusted
project, noteable resource ID and type, author, exact body, and
non-system status.
- Preserve idempotent issue and merge-request note creation by matching
existing notes against their underlying resource identity before
posting.
- Add a deterministic live-shape fixture and focused coverage for
work-item responses, body encoding, pagination, and duplicate
suppression.

## Risk Assessment

✅ Low: Captain, the change is well-bounded and satisfies the
authoritative note-verification intent; the only finding is a
non-functional simplification opportunity.

## Testing

The already-passing portable baseline was supplemented with focused
GitLab forge mutation testing and an end-to-end CLI trace;
work-item-shaped issue notes verify successfully, preserve exact encoded
content, reuse the existing note without duplicates, and retain the
shared issue/MR safety guards, with no live mutation of KissCut issue
146.

<details>
<summary>Evidence: Focused forge mutation test transcript</summary>

```text
ok - issue claim sends explicit JSON media type and preserves array encoding
ok - live work-item note shape verifies exact resource and suppresses duplicates
ok - issue creation validates labels, optional claim, identity, and canonical read-back
ok - issue claim, status, note, close, reopen, and release converge safely
ok - already-correct issue claims, statuses, and label deltas are no-ops
ok - issue release can return self-owned work to the ready queue without clobbering labels
ok - issue mutations cannot steal or bypass exact ownership
ok - missing, archived, and malformed label metadata blocks mutations
ok - malformed targets, labels, usernames, and non-regular note files are rejected
ok - issue API identity rejects foreign, mismatched, and malformed work-item URLs
ok - untrusted projects and API failures never report a successful mutation
ok - issue and note mutations require deterministic matching read-back
ok - merge-request claim, status, labels, notes, lifecycle, and release converge safely
ok - merge-request workflow labels require status and release commands
ok - merge-request mutations preserve project, author, branch, head, and API guards
ok - issue creation and comments surface API and verification failures
```
</details>
<details>
<summary>Evidence: End-to-end GitLab note CLI trace showing first
creation, exact body preservation, reuse on retry, pagination, and a
single POST</summary>

```text
+ 6: set -u
++ 9: dirname tests/fm-forge-mutations.test.sh
+ 9: . tests/lib.sh
++ 23: '[' -n '' ']'
++ 26: FM_TEST_LIB_SOURCED=1
++ 35: export FM_GATE_REFUSE_BYPASS=1
++ 35: FM_GATE_REFUSE_BYPASS=1
++++ 40: dirname tests/lib.sh
+++ 40: cd tests/..
+++ 40: pwd
++ 40: ROOT=/Users/ipiesh/.no-mistakes/worktrees/2814d4e34fac/01KY67HH2JGNYDV9VCR0WDWXX4
++ 45: unset GIT_CONFIG_COUNT GIT_CONFIG_PARAMETERS
++ 46: IFS=
++ 46: read -r variable
+++ 48: compgen -A variable GIT_CONFIG_KEY_
+++ 48: compgen -A variable GIT_CONFIG_VALUE_
++ 49: export GIT_CONFIG_GLOBAL=/Users/ipiesh/.no-mistakes/worktrees/2814d4e34fac/01KY67HH2JGNYDV9VCR0WDWXX4/tests/fixture.gitconfig
++ 49: GIT_CONFIG_GLOBAL=/Users/ipiesh/.no-mistakes/worktrees/2814d4e34fac/01KY67HH2JGNYDV9VCR0WDWXX4/tests/fixture.gitconfig
++ 69: FM_TEST_CLEANUP_DIRS=()
++ 11: fm_test_tmproot fm-forge-mutations-tests
++ 79: local prefix=fm-forge-mutations-tests root
+++ 80: mktemp -d /var/folders/j_/g4b3__rs5j95fq6xf9fb17sc0000gn/T//fm-forge-mutations-tests.XXXXXX
++ 80: root=/var/folders/j_/g4b3__rs5j95fq6xf9fb17sc0000gn/T//fm-forge-mutations-tests.eHx8KB
++ 81: '[' 0 -eq 0 ']'
++ 82: trap fm_test_cleanup EXIT
++ 84: FM_TEST_CLEANUP_DIRS+=("$root")
++ 85: printf '%s\n' /var/folders/j_/g4b3__rs5j95fq6xf9fb17sc0000gn/T//fm-forge-mutations-tests.eHx8KB
++ 1: fm_test_cleanup
++ 72: local d
++ 73: for d in "${FM_TEST_CLEANUP_DIRS[@]:-}"
++ 74: '[' -n /var/folders/j_/g4b3__rs5j95fq6xf9fb17sc0000gn/T//fm-forge-mutations-tests.eHx8KB ']'
++ 74: rm -rf /var/folders/j_/g4b3__rs5j95fq6xf9fb17sc0000gn/T//fm-forge-mutations-tests.eHx8KB
+ 11: TMP=/var/folders/j_/g4b3__rs5j95fq6xf9fb17sc0000gn/T//fm-forge-mutations-tests.eHx8KB
+ 12: ADAPTER=/Users/ipiesh/.no-mistakes/worktrees/2814d4e34fac/01KY67HH2JGNYDV9VCR0WDWXX4/bin/fm-forge.sh
+ 13: REPO=/var/folders/j_/g4b3__rs5j95fq6xf9fb17sc0000gn/T//fm-forge-mutations-tests.eHx8KB/repo
++ 14: fm_fakebin /var/folders/j_/g4b3__rs5j95fq6xf9fb17sc0000gn/T//fm-forge-mutations-tests.eHx8KB
++ 95: local dir=/var/folders/j_/g4b3__rs5j95fq6xf9fb17sc0000gn/T//fm-forge-mutations-tests.eHx8KB fakebin=/var/folders/j_/g4b3__rs5j95fq6xf9fb17sc0000gn/T//fm-forge-mutations-tests.eHx8KB/fakebin
++ 96: mkdir -p /var/folders/j_/g4b3__rs5j95fq6xf9fb17sc0000gn/T//fm-forge-mutations-tests.eHx8KB/fakebin
++ 97: printf '%s\n' /var/folders/j_/g4b3__rs5j95fq6xf9fb17sc0000gn/T//fm-forge-mutations-tests.eHx8KB/fakebin
+ 14: FAKEBIN=/var/folders/j_/g4b3__rs5j95fq6xf9fb17sc0000gn/T//fm-forge-mutations-tests.eHx8KB/fakebin
+ 15: LOG=/var/folders/j_/g4b3__rs5j95fq6xf9fb17sc0000gn/T//fm-forge-mutations-tests.eHx8KB/glab.log
+ 16: ISSUE_STATE=/var/folders/j_/g4b3__rs5j95fq6xf9fb17sc0000gn/T//fm-forge-mutations-tests.eHx8KB/issue.json
+ 17: MR_STATE=/var/folders/j_/g4b3__rs5j95fq6xf9fb17sc0000gn/T//fm-forge-mutations-tests.eHx8KB/mr.json
+ 18: ISSUE_NOTES=/var/folders/j_/g4b3__rs5j95fq6xf9fb17sc0000gn/T//fm-forge-mutations-tests.eHx8KB/issue-notes.json
+ 19: MR_NOTES=/var/folders/j_/g4b3__rs5j95fq6xf9fb17sc0000gn/T//fm-forge-mutations-tests.eHx8KB/mr-notes.json
+ 20: WORK_ITEM_NOTE_FIXTURE=/Users/ipiesh/.no-mistakes/worktrees/2814d4e34fac/01KY67HH2JGNYDV9VCR0WDWXX4/tests/fixtures/gitlab-work-item-note.json
+ 21: MUTATED=/var/folders/j_/g4b3__rs5j95fq6xf9fb17sc0000gn/T//fm-forge-mutations-tests.eHx8KB/mutated
+ 23: fm_git_init_commit /var/folders/j_/g4b3__rs5j95fq6xf9fb17sc0000gn/T//fm-forge-mutations-tests.eHx8KB/repo
+ 125: local dir=/var/folders/j_/g4b3__rs5j95fq6xf9fb17sc0000gn/T//fm-forge-mutations-tests.eHx8KB/repo
+ 126: mkdir -p /var/folders/j_/g4b3__rs5j95fq6xf9fb17sc0000gn/T//fm-forge-mutations-tests.eHx8KB/repo
+ 127: git -C /var/folders/j_/g4b3__rs5j95fq6xf9fb17sc0000gn/T//fm-forge-mutations-tests.eHx8KB/repo init -q
++ 128: basename /var/folders/j_/g4b3__rs5j95fq6xf9fb17sc0000gn/T//fm-forge-mutations-tests.eHx8KB/repo
+ 128: printf '# %s\n' repo
+ 129: git -C /var/folders/j_/g4b3__rs5j95fq6xf9fb17sc0000gn/T//fm-forge-mutations-tests.eHx8KB/repo add README.md
+ 130: git -C /var/folders/j_/g4b3__rs5j95fq6xf9fb17sc0000gn/T//fm-forge-mutations-tests.eHx8KB/repo -c 'user.name=Firstmate Tests' -c user.email=tests@example.invalid commit -qm initial
+ 24: git -C /var/folders/j_/g4b3__rs5j95fq6xf9fb17sc0000gn/T//fm-forge-mutations-tests.eHx8KB/repo remote add origin git@gitlab.com:kisscut-museum/kisscut-platform.git
+ 25: git -C /var/folders/j_/g4b3__rs5j95fq6xf9fb17sc0000gn/T//fm-forge-mutations-tests.eHx8KB/repo checkout -q -b fm/fix
+ 27: cat
+ 347: chmod +x /var/folders/j_/g4b3__rs5j95fq6xf9fb17sc0000gn/T//fm-forge-mutations-tests.eHx8KB/fakebin/glab
+ 797: test_live_415_regression_claim_uses_json_media_type
+ 375: local out
+ 376: reset_case
+ 365: :
+ 366: rm -f /var/folders/j_/g4b3__rs5j95fq6xf9fb17sc0000gn/T//fm-forge-mutations-tests.eHx8KB/issue.json /var/folders/j_/g4b3__rs5j95fq6xf9fb17sc0000gn/T//fm-forge-mutations-tests.eHx8KB/mr.json /var/folders/j_/g4b3__rs5j95fq6xf9fb17sc0000gn/T//fm-forge-mutations-tests.eHx8KB/issue-notes.json /var/folders/j_/g4b3__rs5j95fq6xf9fb17sc0000gn/T//fm-forge-mutations-tests.eHx8KB/mr-notes.json /var/folders/j_/g4b3__rs5j95fq6xf9fb17sc0000gn/T//fm-forge-mutations-tests.eHx8KB/mutated
++ 377: FM_FAKE_ISSUE_OWNER=none
++ 377: run_adapter issue-claim /var/folders/j_/g4b3__rs5j95fq6xf9fb17sc0000gn/T//fm-forge-mutations-tests.eHx8KB/repo 7
++ 350: PATH=/var/folders/j_/g4b3__rs5j95fq6xf9fb17sc0000gn/T//fm-forge-mutations-tests.eHx8KB/fakebin:/Users/ipiesh/.codex/tmp/arg0/codex-arg0wM5SxM:/nix/store/ygxqin6ydzjfawywqpp5pal8wv6sf5bh-python3-3.13.13/bin:/nix/store/yggg7hbh9bi0p4c44npy2mdyjbj5d37h-grc-1.13/bin:/Users/ipiesh/.local/bin:/Users/ipiesh/go/bin:/Users/ipiesh/.cargo/bin:/opt/homebrew/bin:/Users/ipiesh/.nix-profile/bin:/etc/profiles/per-user/ipiesh/bin:/run/current-system/sw/bin:/nix/var/nix/profiles/default/bin:/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin:/Users/ipiesh/bin:/opt/homebrew/sbin:/usr/local/sbin
++ 350: FM_FAKE_GLAB_LOG=/var/folders/j_/g4b3__rs5j95fq6xf9fb17sc0000gn/T//fm-forge-mutations-tests.eHx8KB/glab.log
++ 350: FM_FAKE_ISSUE_STATE_FILE=/var/folders/j_/g4b3__rs5j95fq6xf9fb17sc0000gn/T//fm-forge-mutations-tests.eHx8KB/issue.json
++ 350: FM_FAKE_MR_STATE_FILE=/var/folders/j_/g4b3__rs5j95fq6xf9fb17sc0000gn/T//fm-forge-mutations-tests.eHx8KB/mr.json
++ 350: FM_FAKE_ISSUE_NOTES_FILE=/var/folders/j_/g4b3__rs5j95fq6xf9fb17sc0000gn/T//fm-forge-mutations-tests.eHx8KB/issue-notes.json
++ 350: FM_FAKE_MR_NOTES_FILE=/var/folders/j_/g4b3__rs5j95fq6xf9fb17sc0000gn/T//fm-forge-mutations-tests.eHx8KB/mr-notes.json
++ 350: FM_FAKE_WORK_ITEM_NOTE_FIXTURE=/Users/ipiesh/.no-mistakes/worktrees/2814d4e34fac/01KY67HH2JGNYDV9VCR0WDWXX4/tests/fixtures/gitlab-work-item-note.json
++ 350: FM_FAKE_MUTATED_MARKER=/var/folders/j_/g4b3__rs5j95fq6xf9fb17sc0000gn/T//fm-forge-mutations-tests.eHx8KB/mutated
++ 350: FM_FORGE_HOSTS_FILE=
++ 350: GITLAB_TOKEN=AMBIENT
++ 350: GITLAB_ACCESS_TOKEN=AMBIENT
++ 350: OAUTH_TOKEN=AMBIENT
++ 350: GLAB_ENABLE_CI_AUTOLOGIN=true
++ 350: CI_JOB_TOKEN=AMBIENT
++ 350: /Users/ipiesh/.no-mistakes/worktrees/2814d4e34fac/01KY67HH2JGNYDV9VCR0WDWXX4/bin/fm-forge.sh issue-claim /var/folders/j_/g4b3__rs5j95fq6xf9fb17sc0000gn/T//fm-forge-mutations-tests.eHx8KB/repo 7
+ 377: out='{"forge":"gitlab","host":"gitlab.com","project":"kisscut-museum/kisscut-platform","issue":{"iid":7,"title":"Fix cutter","state":"opened","url":"https://gitlab.com/kisscut-museum/kisscut-platform/-/work_items/7","description":"Issue body","labels":["bug","status::in-progress"],"author":"ivan","assignees":["mate"],"updated_at":"2026-07-18T00:00:00Z"}}'
+ 379: jq -e '.issue.assignees == ["mate"]'
+ 381: assert_grep 'issues/7 --hostname gitlab.com --method PUT --input - --header Content-Type: application/json' /var/folders/j_/g4b3__rs5j95fq6xf9fb17sc0000gn/T//fm-forge-mutations-tests.eHx8KB/glab.log 'issue claim JSON media type'
+ 208: grep -F -- 'issues/7 --hostname gitlab.com --method PUT --input - --header Content-Type: application/json' /var/folders/j_/g4b3__rs5j95fq6xf9fb17sc0000gn/T//fm-forge-mutations-tests.eHx8KB/glab.log
+ 384: assert_grep 'input={"assignee_ids":[42]' /var/folders/j_/g4b3__rs5j95fq6xf9fb17sc0000gn/T//fm-forge-mutations-tests.eHx8KB/glab.log 'issue claim array

... [243589 bytes truncated] ...

S_FILE=\n++ 350: GITLAB_TOKEN=AMBIENT\n++ 350: GITLAB_ACCESS_TOKEN=AMBIENT\n++ 350: OAUTH_TOKEN=AMBIENT\n++ 350: GLAB_ENABLE_CI_AUTOLOGIN=true\n++ 350: CI_JOB_TOKEN=AMBIENT\n++ 350: /Users/ipiesh/.no-mistakes/worktrees/2814d4e34fac/01KY67HH2JGNYDV9VCR0WDWXX4/bin/fm-forge.sh issue-create /var/folders/j_/g4b3__rs5j95fq6xf9fb17sc0000gn/T//fm-forge-mutations-tests.eHx8KB/repo --title \'Create mismatch\' --body-file /var/folders/j_/g4b3__rs5j95fq6xf9fb17sc0000gn/T//fm-forge-mutations-tests.eHx8KB/repo/create-body.md --label bug\nerror: issue identity does not match trusted repository\nerror: created issue identity could not be verified'
+ 784: rc=1
+ 785: expect_code 1 1 'created issue identity mismatch'
+ 201: local expected=1 actual=1 'label=created issue identity mismatch'
+ 202: '[' 1 = 1 ']'
+ 786: assert_contains $'++ 350: PATH=/var/folders/j_/g4b3__rs5j95fq6xf9fb17sc0000gn/T//fm-forge-mutations-tests.eHx8KB/fakebin:/Users/ipiesh/.codex/tmp/arg0/codex-arg0wM5SxM:/nix/store/ygxqin6ydzjfawywqpp5pal8wv6sf5bh-python3-3.13.13/bin:/nix/store/yggg7hbh9bi0p4c44npy2mdyjbj5d37h-grc-1.13/bin:/Users/ipiesh/.local/bin:/Users/ipiesh/go/bin:/Users/ipiesh/.cargo/bin:/opt/homebrew/bin:/Users/ipiesh/.nix-profile/bin:/etc/profiles/per-user/ipiesh/bin:/run/current-system/sw/bin:/nix/var/nix/profiles/default/bin:/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin:/Users/ipiesh/bin:/opt/homebrew/sbin:/usr/local/sbin\n++ 350: FM_FAKE_GLAB_LOG=/var/folders/j_/g4b3__rs5j95fq6xf9fb17sc0000gn/T//fm-forge-mutations-tests.eHx8KB/glab.log\n++ 350: FM_FAKE_ISSUE_STATE_FILE=/var/folders/j_/g4b3__rs5j95fq6xf9fb17sc0000gn/T//fm-forge-mutations-tests.eHx8KB/issue.json\n++ 350: FM_FAKE_MR_STATE_FILE=/var/folders/j_/g4b3__rs5j95fq6xf9fb17sc0000gn/T//fm-forge-mutations-tests.eHx8KB/mr.json\n++ 350: FM_FAKE_ISSUE_NOTES_FILE=/var/folders/j_/g4b3__rs5j95fq6xf9fb17sc0000gn/T//fm-forge-mutations-tests.eHx8KB/issue-notes.json\n++ 350: FM_FAKE_MR_NOTES_FILE=/var/folders/j_/g4b3__rs5j95fq6xf9fb17sc0000gn/T//fm-forge-mutations-tests.eHx8KB/mr-notes.json\n++ 350: FM_FAKE_WORK_ITEM_NOTE_FIXTURE=/Users/ipiesh/.no-mistakes/worktrees/2814d4e34fac/01KY67HH2JGNYDV9VCR0WDWXX4/tests/fixtures/gitlab-work-item-note.json\n++ 350: FM_FAKE_MUTATED_MARKER=/var/folders/j_/g4b3__rs5j95fq6xf9fb17sc0000gn/T//fm-forge-mutations-tests.eHx8KB/mutated\n++ 350: FM_FORGE_HOSTS_FILE=\n++ 350: GITLAB_TOKEN=AMBIENT\n++ 350: GITLAB_ACCESS_TOKEN=AMBIENT\n++ 350: OAUTH_TOKEN=AMBIENT\n++ 350: GLAB_ENABLE_CI_AUTOLOGIN=true\n++ 350: CI_JOB_TOKEN=AMBIENT\n++ 350: /Users/ipiesh/.no-mistakes/worktrees/2814d4e34fac/01KY67HH2JGNYDV9VCR0WDWXX4/bin/fm-forge.sh issue-create /var/folders/j_/g4b3__rs5j95fq6xf9fb17sc0000gn/T//fm-forge-mutations-tests.eHx8KB/repo --title \'Create mismatch\' --body-file /var/folders/j_/g4b3__rs5j95fq6xf9fb17sc0000gn/T//fm-forge-mutations-tests.eHx8KB/repo/create-body.md --label bug\nerror: issue identity does not match trusted repository\nerror: created issue identity could not be verified' identity 'created issue verification refusal'
+ 185: case "$1" in
+ 186: :
+ 788: reset_case
+ 365: :
+ 366: rm -f /var/folders/j_/g4b3__rs5j95fq6xf9fb17sc0000gn/T//fm-forge-mutations-tests.eHx8KB/issue.json /var/folders/j_/g4b3__rs5j95fq6xf9fb17sc0000gn/T//fm-forge-mutations-tests.eHx8KB/mr.json /var/folders/j_/g4b3__rs5j95fq6xf9fb17sc0000gn/T//fm-forge-mutations-tests.eHx8KB/issue-notes.json /var/folders/j_/g4b3__rs5j95fq6xf9fb17sc0000gn/T//fm-forge-mutations-tests.eHx8KB/mr-notes.json /var/folders/j_/g4b3__rs5j95fq6xf9fb17sc0000gn/T//fm-forge-mutations-tests.eHx8KB/mutated
++ 790: FM_FAKE_API_FAIL=issue-create
++ 790: run_adapter issue-create /var/folders/j_/g4b3__rs5j95fq6xf9fb17sc0000gn/T//fm-forge-mutations-tests.eHx8KB/repo --title 'Create failure' --label bug
+ 790: out=$'++ 350: PATH=/var/folders/j_/g4b3__rs5j95fq6xf9fb17sc0000gn/T//fm-forge-mutations-tests.eHx8KB/fakebin:/Users/ipiesh/.codex/tmp/arg0/codex-arg0wM5SxM:/nix/store/ygxqin6ydzjfawywqpp5pal8wv6sf5bh-python3-3.13.13/bin:/nix/store/yggg7hbh9bi0p4c44npy2mdyjbj5d37h-grc-1.13/bin:/Users/ipiesh/.local/bin:/Users/ipiesh/go/bin:/Users/ipiesh/.cargo/bin:/opt/homebrew/bin:/Users/ipiesh/.nix-profile/bin:/etc/profiles/per-user/ipiesh/bin:/run/current-system/sw/bin:/nix/var/nix/profiles/default/bin:/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin:/Users/ipiesh/bin:/opt/homebrew/sbin:/usr/local/sbin\n++ 350: FM_FAKE_GLAB_LOG=/var/folders/j_/g4b3__rs5j95fq6xf9fb17sc0000gn/T//fm-forge-mutations-tests.eHx8KB/glab.log\n++ 350: FM_FAKE_ISSUE_STATE_FILE=/var/folders/j_/g4b3__rs5j95fq6xf9fb17sc0000gn/T//fm-forge-mutations-tests.eHx8KB/issue.json\n++ 350: FM_FAKE_MR_STATE_FILE=/var/folders/j_/g4b3__rs5j95fq6xf9fb17sc0000gn/T//fm-forge-mutations-tests.eHx8KB/mr.json\n++ 350: FM_FAKE_ISSUE_NOTES_FILE=/var/folders/j_/g4b3__rs5j95fq6xf9fb17sc0000gn/T//fm-forge-mutations-tests.eHx8KB/issue-notes.json\n++ 350: FM_FAKE_MR_NOTES_FILE=/var/folders/j_/g4b3__rs5j95fq6xf9fb17sc0000gn/T//fm-forge-mutations-tests.eHx8KB/mr-notes.json\n++ 350: FM_FAKE_WORK_ITEM_NOTE_FIXTURE=/Users/ipiesh/.no-mistakes/worktrees/2814d4e34fac/01KY67HH2JGNYDV9VCR0WDWXX4/tests/fixtures/gitlab-work-item-note.json\n++ 350: FM_FAKE_MUTATED_MARKER=/var/folders/j_/g4b3__rs5j95fq6xf9fb17sc0000gn/T//fm-forge-mutations-tests.eHx8KB/mutated\n++ 350: FM_FORGE_HOSTS_FILE=\n++ 350: GITLAB_TOKEN=AMBIENT\n++ 350: GITLAB_ACCESS_TOKEN=AMBIENT\n++ 350: OAUTH_TOKEN=AMBIENT\n++ 350: GLAB_ENABLE_CI_AUTOLOGIN=true\n++ 350: CI_JOB_TOKEN=AMBIENT\n++ 350: /Users/ipiesh/.no-mistakes/worktrees/2814d4e34fac/01KY67HH2JGNYDV9VCR0WDWXX4/bin/fm-forge.sh issue-create /var/folders/j_/g4b3__rs5j95fq6xf9fb17sc0000gn/T//fm-forge-mutations-tests.eHx8KB/repo --title \'Create failure\' --label bug\nerror: issue creation failed'
+ 791: rc=1
+ 792: expect_code 1 1 'issue creation API failure'
+ 201: local expected=1 actual=1 'label=issue creation API failure'
+ 202: '[' 1 = 1 ']'
+ 793: assert_contains $'++ 350: PATH=/var/folders/j_/g4b3__rs5j95fq6xf9fb17sc0000gn/T//fm-forge-mutations-tests.eHx8KB/fakebin:/Users/ipiesh/.codex/tmp/arg0/codex-arg0wM5SxM:/nix/store/ygxqin6ydzjfawywqpp5pal8wv6sf5bh-python3-3.13.13/bin:/nix/store/yggg7hbh9bi0p4c44npy2mdyjbj5d37h-grc-1.13/bin:/Users/ipiesh/.local/bin:/Users/ipiesh/go/bin:/Users/ipiesh/.cargo/bin:/opt/homebrew/bin:/Users/ipiesh/.nix-profile/bin:/etc/profiles/per-user/ipiesh/bin:/run/current-system/sw/bin:/nix/var/nix/profiles/default/bin:/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin:/Users/ipiesh/bin:/opt/homebrew/sbin:/usr/local/sbin\n++ 350: FM_FAKE_GLAB_LOG=/var/folders/j_/g4b3__rs5j95fq6xf9fb17sc0000gn/T//fm-forge-mutations-tests.eHx8KB/glab.log\n++ 350: FM_FAKE_ISSUE_STATE_FILE=/var/folders/j_/g4b3__rs5j95fq6xf9fb17sc0000gn/T//fm-forge-mutations-tests.eHx8KB/issue.json\n++ 350: FM_FAKE_MR_STATE_FILE=/var/folders/j_/g4b3__rs5j95fq6xf9fb17sc0000gn/T//fm-forge-mutations-tests.eHx8KB/mr.json\n++ 350: FM_FAKE_ISSUE_NOTES_FILE=/var/folders/j_/g4b3__rs5j95fq6xf9fb17sc0000gn/T//fm-forge-mutations-tests.eHx8KB/issue-notes.json\n++ 350: FM_FAKE_MR_NOTES_FILE=/var/folders/j_/g4b3__rs5j95fq6xf9fb17sc0000gn/T//fm-forge-mutations-tests.eHx8KB/mr-notes.json\n++ 350: FM_FAKE_WORK_ITEM_NOTE_FIXTURE=/Users/ipiesh/.no-mistakes/worktrees/2814d4e34fac/01KY67HH2JGNYDV9VCR0WDWXX4/tests/fixtures/gitlab-work-item-note.json\n++ 350: FM_FAKE_MUTATED_MARKER=/var/folders/j_/g4b3__rs5j95fq6xf9fb17sc0000gn/T//fm-forge-mutations-tests.eHx8KB/mutated\n++ 350: FM_FORGE_HOSTS_FILE=\n++ 350: GITLAB_TOKEN=AMBIENT\n++ 350: GITLAB_ACCESS_TOKEN=AMBIENT\n++ 350: OAUTH_TOKEN=AMBIENT\n++ 350: GLAB_ENABLE_CI_AUTOLOGIN=true\n++ 350: CI_JOB_TOKEN=AMBIENT\n++ 350: /Users/ipiesh/.no-mistakes/worktrees/2814d4e34fac/01KY67HH2JGNYDV9VCR0WDWXX4/bin/fm-forge.sh issue-create /var/folders/j_/g4b3__rs5j95fq6xf9fb17sc0000gn/T//fm-forge-mutations-tests.eHx8KB/repo --title \'Create failure\' --label bug\nerror: issue creation failed' 'issue creation failed' 'issue creation failure report'
+ 185: case "$1" in
+ 186: :
+ 794: pass 'issue creation and comments surface API and verification failures'
+ 59: printf 'ok - %s\n' 'issue creation and comments surface API and verification failures'
ok - issue creation and comments surface API and verification failures
```
</details>

## Pipeline

Updates from [git push
no-mistakes](https://github.com/kunchenguid/no-mistakes)

<details>
<summary>✅ **intent** - passed</summary>

✅ No issues found.

</details>

<details>
<summary>✅ **Rebase** - passed</summary>

✅ No issues found.

</details>

<details>
<summary>⚠️ **Review** - 1 info</summary>

- ℹ️ `bin/fm-forge.sh:586` - The existing-note selector duplicates the
full trusted note-identity predicate from `note_identity_valid()`.
Extract a shared jq predicate/filter so list-time idempotence matching
and read-back verification cannot drift as guards evolve.

</details>

<details>
<summary>✅ **Test** - passed</summary>

✅ No issues found.
- `command -v tmux >/dev/null || { echo "tmux is required for e2e tests"
>&2; exit 1; }; tmux -V; rc=0; for t in tests/*.test.sh; do echo "== $t
=="; bash "$t" || rc=1; done; exit "$rc"`
- <code>Reviewed `git diff --unified=100
f68d7c9..4bab937`
against the authoritative intent.</code>
- <code>Accepted the previously successful configured portable baseline:
`command -v tmux &gt;/dev/null || { echo &kunchenguid#34;tmux is required for e2e
tests&kunchenguid#34; &gt;&amp;2; exit 1; }; tmux -V; rc=0; for t in
tests/*.test.sh; do echo &kunchenguid#34;== $t ==&kunchenguid#34;; bash &kunchenguid#34;$t&kunchenguid#34; || rc=1;
done; exit &kunchenguid#34;$rc&kunchenguid#34;`.</code>
- <code>Ran `bash tests/fm-forge-mutations.test.sh` against the
deterministic local GitLab fixture.</code>
- <code>Ran `PS4=&kunchenguid#39;+ ${LINENO}: &kunchenguid#39; bash -x
tests/fm-forge-mutations.test.sh` to capture end-user CLI JSON and
API-call evidence.</code>
- <code>Verified `git status --short` remained empty and no
`.fm-forge-body.*` transient snapshots remained.</code>

</details>

<details>
<summary>✅ **Document** - passed</summary>

✅ No issues found.

</details>

<details>
<summary>✅ **Lint** - passed</summary>

✅ No issues found.

</details>

<details>
<summary>✅ **Push** - passed</summary>

✅ No issues found.

</details>

Co-authored-by: Ivan Miles Piesh <ivan@ivanpiesh.info>
eyevanovich added a commit to eyevanovich/firstmate that referenced this pull request Jul 23, 2026
## Intent

Implement Firstmate's default captain-visible no-mistakes observer
experience: after the worker starts validation and an authoritative
branch-matched run ID exists, open at most one separate non-focused
observer terminal running 'no-mistakes attach --run <id>' without
transferring any axi run/respond, cancellation, CI, ask-user, or merge
ownership away from the worker. Support tmux and Herdr with trusted
task/run/backend/worker/observer identity, idempotent retries, durable
q/exit detach tombstones, explicit reopen only, exact observer-only
cleanup integrated into task teardown, and bounded failure that
preserves validation while printing the exact manual attach command.
Keep zellij, Orca, and cmux on safe manual fallback until separately
proven. Keep owning instructions in the lifecycle script with only
concise trigger/safety pointers in shared docs. Cover harness
invocation, authoritative discovery, separation,
mismatch/malformed/unreadable state, interrupted create reconciliation,
detach/reopen, fallback, and cleanup deterministically, plus an isolated
tmux smoke test. Per captain decision, do not add live Herdr proof;
document a concise captain-run manual verification checklist and keep
the approved scope minimal.

## What Changed

- Add a validation entrypoint that discovers authoritative branch runs
and opens one non-focused, captain-visible no-mistakes observer in tmux
or Herdr while preserving worker ownership.
- Persist trusted observer lifecycle state for idempotent retries,
detach tombstones, explicit reopen, bounded manual fallback, and exact
observer-only teardown cleanup.
- Document configuration and Herdr verification, with deterministic
lifecycle coverage and an isolated tmux smoke test.

## Risk Assessment

✅ Low: The change now consistently enforces token-bound identity,
idempotent observer creation and detach handling, exact cleanup, safe
fallback, and narrowly reconciled interrupted states without a remaining
substantiated defect.

## Testing

The supplied full baseline was green; focused lifecycle, live
isolated-tmux, and teardown tests also passed, and the captured
transcript demonstrates one non-focused sibling observer, durable
q-detach with exact manual fallback, explicit-only reopen, and
observer-only cleanup preserving the worker. Herdr behavior remains
deterministic fake-CLI coverage with the intentionally documented
captain-run checklist; no live Herdr proof was added per scope. No
screenshot was captured because the smoke runs on a detached isolated
tmux socket without a rendered GUI surface, so a direct terminal
lifecycle transcript was used.

<details>
<summary>Evidence: Captain-visible tmux observer lifecycle
transcript</summary>

```text
observer: opened task task run run-live at @1 (tmux, no focus)
EVIDENCE after-open
window=@0 name=fm-task active=1 pane=%0
window=@1 name=nm-observer-task-8c1e568b active=0 pane=%1
task=task
run=run-live
worker_backend=tmux
worker_target=crew:fm-task
observer_backend=tmux
observer_target=@1
status=attached
EVIDENCE after-q status=detached
observer: the observer is detached and will not be respawned automatically
observer: attach manually with: cd '/var/folders/j_/g4b3__rs5j95fq6xf9fb17sc0000gn/T//fm-no-mistakes-observer-tmux.GZSX7b/repo-wt' && no-mistakes attach --run 'run-live'
EVIDENCE explicit-reopen
observer: opened task task run run-live at @2 (tmux, no focus)
EVIDENCE cleanup record_exists=no
EVIDENCE surviving-worker
window=@0 name=fm-task active=1 pane=%0
ok - live tmux observer uses one no-focus sibling, q detaches, reopen is explicit, and cleanup is exact
```
</details>

## Pipeline

Updates from [git push
no-mistakes](https://github.com/kunchenguid/no-mistakes)

<details>
<summary>✅ **intent** - passed</summary>

✅ No issues found.

</details>

<details>
<summary>✅ **Rebase** - passed</summary>

✅ No issues found.

</details>

<details>
<summary>🔧 **Review** - 2 issues found → auto-fixed (8) ✅</summary>

- 🚨 `bin/fm-no-mistakes-observer.sh:804` - The required “idempotent
retries” behavior is contradicted here: `start` adopts an existing
branch run only when its top-level status is exactly `running`. Other
active states already recognized elsewhere in this repository, such as
`awaiting_approval` and `fix_review`, fall through and send the
validation invocation again, potentially starting duplicate validation.
Treat every authoritative nonterminal branch run as existing before
invoking the worker.
- 🚨 `bin/fm-no-mistakes-observer.sh:648` - The required “interrupted
create reconciliation” and “idempotent retries” are incomplete for
Herdr. The record becomes `ready` before the separate `send-text` and
Enter operations; interruption after text is sent leaves the command in
the composer, and retry sends it again before Enter, potentially
executing a concatenated invalid command instead of attaching. Persist
an intermediate launch phase or safely verify/clear the composer before
retrying.

🔧 Fix: Harden observer retry idempotence
1 error still open:
- 🚨 `bin/fm-no-mistakes-observer.sh:664` - The required “idempotent
retries” behavior still has a Herdr race. After Enter succeeds, this
resets the record to `ready` while the `_session` process cannot mark it
`attached` until the caller releases the lifecycle lock. A retry that
acquires the lock first sees `ready` and sends the full attach command
again into the observer pane. Preserve a distinct submitted/launching
state that `_session` can claim but ordinary retries refuse, so
successful submission cannot reopen the text-sending path.

🔧 Fix: Prevent Herdr observer resubmission race
1 error still open:
- 🚨 `bin/fm-no-mistakes-observer.sh:905` - The required “durable q/exit
detach tombstones” behavior is still incomplete. If `_session` cannot
claim the `submitted` record within its bounded retry loop—for example
after the launcher dies while leaving its lock directory—it exits here
without changing the record. Ordinary retries and explicit `reopen` then
preserve `submitted` forever as “awaiting attachment,” although the
observer process has exited. On claim timeout, safely tombstone the same
task/run/token record as detached or failed without resending text or
Enter.

🔧 Fix: Tombstone unclaimed Herdr observer sessions
2 errors still open:
- 🚨 `bin/fm-no-mistakes-observer.sh:909` - The required bounded exit
reconciliation is still incomplete. After the fixed claim wait,
`_session` tombstones only if it can immediately acquire the lifecycle
lock, and then only for `submitted`. A launcher or concurrent status
operation can legitimately hold the lock past this point, leaving Herdr
permanently `submitted`; tmux can similarly remain `ready` after its
wrapper exits. Subsequent retries can therefore preserve or report an
observer that has no attach process. Reconcile the same-token launch
state after lock release within a bounded path for both backends.
- 🚨 `bin/fm-no-mistakes-observer.sh:883` - The required trusted identity
and idempotent lifecycle are contradicted because claiming an observer
is not atomic with the lifecycle lock. `_session` checks that the lock
directory is absent, then loads and rewrites the record without
acquiring it; cleanup can acquire the lock and remove the record between
those operations, after which `_session` resurrects stale state as
`attached` and runs the old attach command. Acquire and hold the task
lock while validating and transitioning the same task/run/token record,
then release it before the foreground attach.

🔧 Fix: Lock observer session lifecycle transitions
1 error still open:
- 🚨 `bin/fm-no-mistakes-observer.sh:933` - The required durable exit
tombstone and bounded-failure behavior can still be lost to normal lock
contention. Claim-timeout and attach-exit reconciliation each make one
bounded transition attempt and silently ignore exhaustion, while another
lifecycle operation may hold the same lock across several independently
bounded status/terminal calls. The observer can therefore exit on `q`
while remaining `attached` (or die while `ready`/`submitted`), blocking
explicit reopen and printing no manual attach fallback. Make the
exit/timed-out transition durable beyond transient lock ownership
without resurrecting removed records.

🔧 Fix: Persist observer detach across lock contention
3 errors still open:
- 🚨 `bin/fm-no-mistakes-observer.sh:324` - The required trusted token
identity and durable exit handling can be lost because every generation
writes the same task-wide pending path outside the lifecycle lock. A
delayed old observer can overwrite a newer token’s pending detach after
explicit reopen; consumption then discards the stale token mismatch and
loses the newer exit evidence, potentially leaving its dead record
`attached`. Use token-specific or otherwise non-clobbering pending
transitions.
- 🚨 `bin/fm-no-mistakes-observer.sh:944` - The required exact observer
cleanup can race with late pending publication. Cleanup removes the
record and current sidecar under the lock, but a contended `_session`
may publish `.observer.pending` after cleanup releases it. Teardown only
invokes observer cleanup when `.observer` exists and does not otherwise
remove this late sidecar, leaving durable task/run/token state after
teardown. Add a cleanup-safe publication handshake or unconditional
race-safe pending cleanup.
- 🚨 `bin/fm-no-mistakes-observer.sh:782` - The required explicit reopen
behavior takes two commands when an `attached` observer endpoint has
already exited. This branch tombstones it as `detached` and returns
manual fallback even when the current action is `reopen`; only a second
`reopen` creates the replacement. When `allow_reopen=1`, continue safely
from the newly established detach tombstone in the same invocation.

🔧 Fix: Scope observer handoffs to generation tokens
1 error still open:
- 🚨 `bin/fm-no-mistakes-observer.sh:514` - The required
interrupted-create reconciliation and exact teardown cleanup still have
a publication gap. Initialization creates the token generation directory
before atomically writing the authoritative observer record;
interruption between those operations leaves an unreferenced
`.&lt;task&gt;.observer-&lt;token&gt;` directory. Later starts use
another token, and teardown skips observer cleanup when `.observer` is
absent, so the orphan survives task removal. Publish both
transactionally or safely reconcile recordless task-owned generations.

🔧 Fix: Publish observer records before generation state
1 error still open:
- 🚨 `bin/fm-no-mistakes-observer.sh:526` - The required
interrupted-create reconciliation and teardown cleanup remain incomplete
after the record-first change. Interruption after this record write but
before generation creation leaves a valid `creating` record with no
backend session or endpoint. Direct cleanup then treats the missing
provisional identity as ambiguous and refuses teardown, although no
observer could have been created. Recognize this same-token pristine
pre-endpoint state as safely unlaunched and remove its record/generation
without targeting a terminal.

🔧 Fix: Clean pristine unlaunched observer records safely
✅ Re-checked - no issues remain.

</details>

<details>
<summary>✅ **Test** - passed</summary>

✅ No issues found.
- `command -v tmux >/dev/null || { echo "tmux is required for e2e tests"
>&2; exit 1; }; tmux -V; rc=0; for t in tests/*.test.sh; do echo "== $t
=="; bash "$t" || rc=1; done; exit "$rc"`
- <code>Baseline supplied by the gate: `command -v tmux &gt;/dev/null ||
{ echo &kunchenguid#34;tmux is required for e2e tests&kunchenguid#34; &gt;&amp;2; exit 1; };
tmux -V; rc=0; for t in tests/*.test.sh; do echo &kunchenguid#34;== $t ==&kunchenguid#34;;
bash &kunchenguid#34;$t&kunchenguid#34; || rc=1; done; exit &kunchenguid#34;$rc&kunchenguid#34;` (reported
successful)</code>
- `bash tests/fm-no-mistakes-observer.test.sh`
- `bash tests/fm-no-mistakes-observer-tmux-smoke.test.sh`
- `bash tests/fm-teardown.test.sh`
- <code>Instrumented and ran the isolated tmux smoke flow to record
actual window focus/identity, trusted observer state, q-detach
tombstone, manual attach fallback, explicit reopen, exact cleanup, and
surviving worker terminal in `tmux-observer-lifecycle.txt`</code>

</details>

<details>
<summary>✅ **Document** - passed</summary>

✅ No issues found.

</details>

<details>
<summary>🔧 **Lint** - 1 issue found → auto-fixed ✅</summary>

- ⚠️ linter found issues (exit code 1)

🔧 Fix: Fix observer pending reset assignments, captain
✅ Re-checked - no issues remain.

</details>

<details>
<summary>✅ **Push** - passed</summary>

✅ No issues found.

</details>

---------

Co-authored-by: Ivan Miles Piesh <ivan@ivanpiesh.info>
eyevanovich added a commit to eyevanovich/firstmate that referenced this pull request Jul 23, 2026
## Intent

Autofix the remaining live GitLab WorkItem completion bug after PR #7.
Issue 146 is a trusted self-owned WorkItem whose exact authenticated
completion note appears three times because WorkItem noteable_id is a
separate opaque positive identity from the legacy Issues API .id.
Through the trusted issue-notes endpoint, reuse any existing exact
authenticated non-system WorkItem note without comparing its opaque
noteable_id to the issue API ID and never POST another duplicate;
preserve strict comparable identity checks for legacy Issue and
MergeRequest notes, exact body and author matching, pagination,
malformed-ID rejection, and field-name-only mismatch diagnostics. Also
make issue-close converge both new and already-closed self-owned issues
to closed with all execution/workflow labels removed while preserving
unrelated labels and ownership, without repeating an already-landed
state transition; enforce the directly equivalent MR close invariant.
Keep all fixes and tests within WorkItem note identity/deduplication and
issue/MR close-state convergence, and do not broaden the public API.

## What Changed

- Reuse exact authenticated non-system GitLab WorkItem notes without
comparing their opaque `noteable_id` to the legacy issue ID, while
preserving strict Issue and MergeRequest identity checks and
malformed-ID rejection.
- Make issue and merge-request close operations remove workflow labels,
preserve unrelated metadata, and converge already-closed resources
without repeating the close transition.
- Expand mutation coverage for note deduplication, endpoint-scoped
identity validation, field-only mismatch diagnostics, malformed note
IDs, and idempotent close cleanup.

## Risk Assessment

✅ Low: Captain, the prior correctness findings are resolved and the only
remaining issue is an unused private helper.

## Testing

The previously completed full baseline passed, the focused GitLab
mutation suite passed twice, and its captured CLI/API trace directly
demonstrates WorkItem note reuse, strict legacy identity handling,
malformed-ID rejection, and idempotent issue/MR close-state convergence;
no UI surface was changed, so visual evidence was not applicable.

<details>
<summary>Evidence: GitLab end-user flow evidence</summary>

```text
137:+ out='{"resource":"issue","note":{"id":501,"body":"Live work-item note.\nSecond line: \"quoted\" & UTF-8 café.\n","author":"mate","already":true}}'
172:+ out='{"resource":"issue","note":{"id":501,"body":"Live work-item note.\nSecond line: \"quoted\" & UTF-8 café.\n","author":"mate","already":true}}'
202:+ out='{"resource":"issue","note":{"id":551,"body":"Legacy issue note.\n","author":"mate","already":true}}'
210:+ jq -cn --argjson exact '{"id":651,"body":"Legacy MR note.\n","author":{"id":42,"username":"mate"},"system":false,"noteable_id":8005,"noteable_type":"MergeRequest","project_id":314,"noteable_iid":5}' $'[\n    ($exact | .id=649 | .noteable_type="WorkItem" | .noteable_iid=null),\n    ($exact | .id=650 | .noteable_iid=6),\n    $exact\n  ]'
227:+ out='{"resource":"mr","note":{"id":651,"body":"Legacy MR note.\n","author":"mate","already":true}}'
257:+ jq -cn --rawfile body /var/folders/j_/g4b3__rs5j95fq6xf9fb17sc0000gn/T//fm-forge-mutations-tests.fstRVI/repo/malformed-existing-note.md $'\n    [{id:0,body:$body,author:{id:42,username:"mate"},system:false,\n      noteable_id:77146,noteable_type:"WorkItem",project_id:314,noteable_iid:null}]'
459:+ out='{"resource":"issue","note":{"id":501,"body":"Worker update.\n","author":"mate","already":true}}'
483:+ jq -e $'.issue.state == "closed" and .issue.labels == ["bug"]\n    and .issue.assignees == ["mate"]'
484:++ count_log '"state_event":"close"'
485:++ local 'pattern="state_event":"close"'
486:++ grep -c -- '"state_event":"close"' /var/folders/j_/g4b3__rs5j95fq6xf9fb17sc0000gn/T//fm-forge-mutations-tests.fstRVI/glab.log
504:++ count_log '"state_event":"close"'
505:++ local 'pattern="state_event":"close"'
506:++ grep -c -- '"state_event":"close"' /var/folders/j_/g4b3__rs5j95fq6xf9fb17sc0000gn/T//fm-forge-mutations-tests.fstRVI/glab.log
525:++ jq -r .issue.state
595:+ jq -e $'.issue.state == "closed" and .issue.labels == ["bug"]\n    and .issue.assignees == ["mate"]'
596:+ assert_no_grep '"state_event":"close"' /var/folders/j_/g4b3__rs5j95fq6xf9fb17sc0000gn/T//fm-forge-mutations-tests.fstRVI/glab.log 'issue close retry repeated an already-landed state transition'
597:+ grep -F -- '"state_event":"close"' /var/folders/j_/g4b3__rs5j95fq6xf9fb17sc0000gn/T//fm-forge-mutations-tests.fstRVI/glab.log
644:+ jq -e $'.mr.state == "closed" and .mr.labels == ["backend"]\n    and .mr.assignees == []'
645:+ assert_no_grep '"state_event":"close"' /var/folders/j_/g4b3__rs5j95fq6xf9fb17sc0000gn/T//fm-forge-mutations-tests.fstRVI/glab.log 'merge-request close retry repeated an already-landed state transition'
646:+ grep -F -- '"state_event":"close"' /var/folders/j_/g4b3__rs5j95fq6xf9fb17sc0000gn/T//fm-forge-mutations-tests.fstRVI/glab.log
757:++ count_log 'input={"add_labels":"docs","remove_labels":"bug"}'
758:++ local 'pattern=input={"add_labels":"docs","remove_labels":"bug"}'
759:++ grep -c -- 'input={"add_labels":"docs","remove_labels":"bug"}' /var/folders/j_/g4b3__rs5j95fq6xf9fb17sc0000gn/T//fm-forge-mutations-tests.fstRVI/glab.log
1297:+ out='{"resource":"mr","note":{"id":601,"body":"MR update.\n","author":"mate","already":true}}'
1323:+ jq -e '.mr.state == "closed" and .mr.labels == ["docs"]'
1341:++ jq -r .mr.state
```
</details>
<details>
<summary>Evidence: Targeted mutation-suite transcript</summary>

```text
ok - issue claim sends explicit JSON media type and preserves array encoding
ok - live work-item note shape verifies exact resource and suppresses duplicates
ok - note list matching and read-back share strict endpoint-scoped identity
ok - malformed created note IDs cannot pass read-back verification
ok - malformed matching note IDs fail closed without duplicate posts
ok - issue creation validates labels, optional claim, identity, and canonical read-back
ok - issue claim, status, note, close, reopen, and release converge safely
ok - close retries remove workflow labels without repeating landed transitions
ok - already-correct issue claims, statuses, and label deltas are no-ops
ok - issue release can return self-owned work to the ready queue without clobbering labels
ok - issue mutations cannot steal or bypass exact ownership
ok - missing, archived, and malformed label metadata blocks mutations
ok - malformed targets, labels, usernames, and non-regular note files are rejected
ok - issue API identity rejects foreign, mismatched, and malformed work-item URLs
ok - untrusted projects and API failures never report a successful mutation
ok - issue and note mutations require deterministic matching read-back
ok - merge-request claim, status, labels, notes, lifecycle, and release converge safely
ok - merge-request workflow labels require status and release commands
ok - merge-request mutations preserve project, author, branch, head, and API guards
ok - issue creation and comments surface API and verification failures
```
</details>

## Pipeline

Updates from [git push
no-mistakes](https://github.com/kunchenguid/no-mistakes)

<details>
<summary>✅ **intent** - passed</summary>

✅ No issues found.

</details>

<details>
<summary>✅ **Rebase** - passed</summary>

✅ No issues found.

</details>

<details>
<summary>⚠️ **Review** - 1 info</summary>

- 🚨 `bin/fm-forge.sh:591` - The intent requires “never POST another
duplicate” and preserving “malformed-ID rejection.” `find_matching_note`
silently skips an otherwise exact authenticated non-system note whose
`.id` is malformed; if no valid duplicate follows, `post_note` falls
through to POST. Reject this case with an `id`-only diagnostic instead
of posting.
- ⚠️ `bin/fm-forge.sh:1117` - The close path now changes workflow
labels, but the script header still states that workflow labels are
changed only by claim, status, and release commands. Update that safety
contract to include close cleanup.
- ℹ️ `bin/fm-forge.sh:1117` - Issue and MR close duplicate
workflow-label removal, expected-label calculation, idempotence, and
payload construction. Extract the shared close-state calculation so
these directly equivalent invariants cannot drift.

🔧 Fix: Fix note deduplication and close-state convergence
1 info still open:
- ℹ️ `bin/fm-forge.sh:575` - `note_identity_valid()` is now unused;
matching and required verification both call
`note_identity_mismatches()` directly. Remove this dead wrapper to
simplify the identity helper surface.

</details>

<details>
<summary>✅ **Test** - passed</summary>

✅ No issues found.
- `command -v tmux >/dev/null || { echo "tmux is required for e2e tests"
>&2; exit 1; }; tmux -V; rc=0; for t in tests/*.test.sh; do echo "== $t
=="; bash "$t" || rc=1; done; exit "$rc"`
- <code>Pre-run baseline: `command -v tmux &gt;/dev/null || { echo
&kunchenguid#34;tmux is required for e2e tests&kunchenguid#34; &gt;&amp;2; exit 1; }; tmux -V;
rc=0; for t in tests/*.test.sh; do echo &kunchenguid#34;== $t ==&kunchenguid#34;; bash
&kunchenguid#34;$t&kunchenguid#34; || rc=1; done; exit &kunchenguid#34;$rc&kunchenguid#34;`</code>
- `bash tests/fm-forge-mutations.test.sh`
- <code>`bash -x tests/fm-forge-mutations.test.sh` with filtered
CLI-response and mutation evidence captured to
`gitlab-user-flow-evidence.txt`</code>
- <code>Verified the testing run left the git worktree clean with `git
status --short`</code>

</details>

<details>
<summary>✅ **Document** - passed</summary>

✅ No issues found.

</details>

<details>
<summary>✅ **Lint** - passed</summary>

✅ No issues found.

</details>

<details>
<summary>✅ **Push** - passed</summary>

✅ No issues found.

</details>

---------

Co-authored-by: Ivan Miles Piesh <ivan@ivanpiesh.info>
eyevanovich added a commit to eyevanovich/firstmate that referenced this pull request Jul 23, 2026
## Intent

Autofix the remaining live GitLab merge-request creation verification
bug exposed by KissCut issue 146. Identify and handle the exact GitLab
API response semantics: create/update requests use remove_source_branch,
while read responses distinguish the MR-specific
should_remove_source_branch intent from force_remove_source_branch
project policy and remove_source_branch_after_merge project defaults.
Accept successful create or idempotent reuse only when title, exact
description intent (allowing only null/empty normalization), draft
state, source, target, head SHA, authenticated authorship, and requested
source-branch deletion semantics truly match. Preserve ambiguous-success
no-retry safety and never create a duplicate; conflicting existing state
must refuse unless an explicitly safe guarded convergence path exists.
Diagnostics must disclose only mismatched field names, never credentials
or full untrusted MR bodies. Keep all fixes focused on GitLab MR
create/reuse response semantics and field-only diagnostics, and add a
live-shape regression fixture. The captain explicitly authorized
per-task AUTOFIX via --yes for ordinary findings, but not destructive,
irreversible, credential, or security-sensitive choices.

## What Changed

- Verify GitLab merge-request creation and reuse against MR-specific
source-branch removal intent, exact description semantics, draft state,
head SHA, and authenticated authorship.
- Fail closed on missing or conflicting response fields without retrying
creation, and report only mismatched field names without exposing
response data or usernames.
- Expand GitLab live-response fixtures and regression coverage for
project policy/default distinctions, malformed descriptions, authorship
mismatches, and duplicate-prevention behavior.

## Risk Assessment

⚠️ Medium: Captain, the required verification behavior is now correct,
but the unused project-default parser adds avoidable coupling and
failure surface that is safe to remove mechanically.

## Testing

The previously successful full baseline was supplemented by the focused
GitLab forge suite and an end-to-end fake-API CLI run; exact live-shape
reuse avoided duplicate creation, conflicting MR deletion intent refused
with field-only diagnostics, and new draft creation preserved its body
and deletion request, with a clean worktree afterward.

<details>
<summary>Evidence: GitLab MR create/reuse end-to-end
transcript</summary>

```text
LIVE-SHAPE IDEMPOTENT REUSE (force=false, should=true, project-default=false)
/Users/ipiesh/.no-mistakes/worktrees/2814d4e34fac/01KY6J004VW1DFDW8RPC50CCPR/bin/fm-forge.sh mr-create <test-worktree>/repo --title 'Ship fix' --source fm/fix --remove-source-branch
out='{"forge":"gitlab","host":"gitlab.com","project":"kisscut-museum/kisscut-platform","mr":{"iid":5,"title":"Ship fix","state":"opened","url":"https://gitlab.com/kisscut-museum/kisscut-platform/-/merge_requests/5","source_branch":"fm/fix","target_branch":"main","draft":false,"merge_status":"can_be_merged","detailed_merge_status":"mergeable","sha":"e33d09c7d53e53dc466e7169c616c8c184001cf4","merge_commit_sha":null,"labels":["backend"],"author":"mate","assignees":[],"pipeline":{"id":9,"status":"success","sha":"aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa","url":"https://gitlab.com/p/9"},"already":true}}'
assert_no_grep '--method POST' <test-worktree>/glab.log 'live-shape reuse created another merge request'

CONFLICTING MR-SPECIFIC DELETION INTENT
run_adapter mr-create <test-worktree>/repo --title 'Ship fix' --source fm/fix --remove-source-branch
error: matching merge request does not match requested head and metadata mismatch (should_remove_source_branch)
assert_no_grep '--method POST' <test-worktree>/glab.log 'conflicting remove-source retry created another merge request'

NEW DRAFT MR CREATION WITH EXACT BODY AND DELETION REQUEST
/Users/ipiesh/.no-mistakes/worktrees/2814d4e34fac/01KY6J004VW1DFDW8RPC50CCPR/bin/fm-forge.sh mr-create <test-worktree>/repo --title 'Detailed fix' --source fm/fix --body-file <test-worktree>/repo/mr-create-body.md --draft --remove-source-branch
out='{"forge":"gitlab","host":"gitlab.com","project":"kisscut-museum/kisscut-platform","mr":{"iid":5,"title":"Draft: Detailed fix","state":"opened","url":"https://gitlab.com/kisscut-museum/kisscut-platform/-/merge_requests/5","source_branch":"fm/fix","target_branch":"main","draft":true,"merge_status":"can_be_merged","detailed_merge_status":"mergeable","sha":"e33d09c7d53e53dc466e7169c616c8c184001cf4","merge_commit_sha":null,"labels":["backend"],"author":"mate","assignees":[],"pipeline":{"id":9,"status":"success","sha":"aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa","url":"https://gitlab.com/p/9"},"already":false}}'
assert_grep '"description":"Requested body\n\n"' <test-worktree>/glab.log 'merge-request POST did not preserve trailing body newlines'
```
</details>

## Pipeline

Updates from [git push
no-mistakes](https://github.com/kunchenguid/no-mistakes)

<details>
<summary>✅ **intent** - passed</summary>

✅ No issues found.

</details>

<details>
<summary>✅ **Rebase** - passed</summary>

✅ No issues found.

</details>

<details>
<summary>⚠️ **Review** - 1 warning</summary>

- 🚨 `bin/fm-forge.sh:672` - The required distinction between MR intent
and project policy is violated: when `should_remove_source_branch` is
unavailable, this fallback infers MR intent by comparing
`force_remove_source_branch` with `remove_source_branch_after_merge`.
Those are independent project-policy/default fields, so their inequality
cannot prove the MR-specific choice and may falsely accept conflicting
state. Require a boolean `should_remove_source_branch`; otherwise fail
closed.
- 🚨 `bin/fm-forge.sh:680` - The required “exact description intent
(allowing only null/empty normalization)” is not enforced. jq&kunchenguid#39;s `//`
also converts boolean `false` and a missing member to `&kunchenguid#34;&kunchenguid#34;`,
allowing a malformed response to match an empty requested description.
Require `description` to be a string or explicit null, and normalize
only null.
- 🚨 `bin/fm-forge.sh:1350` - The forbidden diagnostic behavior remains:
authorship mismatch messages interpolate the authenticated username
instead of disclosing only mismatched field names. Report fields such as
`author.id,author.username` without including their values; the same
issue also affects created-MR verification at line 1382.

🔧 Fix: Harden GitLab merge-request response verification
1 warning still open:
- ⚠️ `bin/fm-forge.sh:167` - `FM_GITLAB_PROJECT_REMOVE_SOURCE_DEFAULT`
has no remaining consumer after verification switched exclusively to
`should_remove_source_branch`. Parsing it adds dead state and can make
every `load_trusted_project` caller fail on an irrelevant malformed
project-default field. Remove this parsing block and the global; the
fixture can retain the field as part of the live response shape.

</details>

<details>
<summary>✅ **Test** - passed</summary>

✅ No issues found.
- `command -v tmux >/dev/null || { echo "tmux is required for e2e tests"
>&2; exit 1; }; tmux -V; rc=0; for t in tests/*.test.sh; do echo "== $t
=="; bash "$t" || rc=1; done; exit "$rc"`
- <code>Baseline (already successful): `command -v tmux &gt;/dev/null ||
{ echo &kunchenguid#34;tmux is required for e2e tests&kunchenguid#34; &gt;&amp;2; exit 1; };
tmux -V; rc=0; for t in tests/*.test.sh; do echo &kunchenguid#34;== $t ==&kunchenguid#34;;
bash &kunchenguid#34;$t&kunchenguid#34; || rc=1; done; exit &kunchenguid#34;$rc&kunchenguid#34;`</code>
- <code>Focused regression suite: `bash tests/fm-forge.test.sh`</code>
- <code>Evidence run: `PS4=&kunchenguid#39;+${LINENO}: &kunchenguid#39; bash -x
tests/fm-forge.test.sh`</code>
- `Reviewed the evidence transcript for live-shape idempotent reuse,
conflicting deletion-intent refusal without POST retry, field-only
diagnostics, and successful draft MR creation preserving the exact body
and deletion request`
- <code>Verified the worktree remained clean with `git status
--short`</code>

</details>

<details>
<summary>✅ **Document** - passed</summary>

✅ No issues found.

</details>

<details>
<summary>🔧 **Lint** - 1 issue found → auto-fixed ✅</summary>

- ⚠️ linter found issues (exit code 1)

🔧 Fix: Remove unused GitLab project deletion default
✅ Re-checked - no issues remain.

</details>

<details>
<summary>✅ **Push** - passed</summary>

✅ No issues found.

</details>

---------

Co-authored-by: Ivan Miles Piesh <ivan@ivanpiesh.info>
eyevanovich added a commit to eyevanovich/firstmate that referenced this pull request Jul 25, 2026
## Intent

Diagnose and fix Firstmate's GitLab merge-request submission behavior so
future direct GitLab MRs reliably request and verify both Delete source
branch and Squash commits without false metadata failures. Treat
source-deletion verification and missing squash submission as distinct
faults until evidence joins them; compare creation and exact-MR reuse,
GitLab request payloads, response fields, project defaults, and
read-back behavior. Provide explicit idempotent mr-create options,
preserve ambiguous-success no-retry safety and exact
identity/head/authorship checks, keep omitted options delegated to
GitLab project defaults, and apply this captain's local
squash-and-delete preference through the existing captain-preference
configuration seam rather than imposing it on every Firstmate user.
Update exact help, configuration documentation, and regression coverage
for creation, reuse, project defaults, response variation, and true
mismatch behavior. Do not mutate existing live MRs or weaken guarded
verification.

## What Changed

- Add explicit `--squash` and `--remove-source-branch` options to GitLab
MR creation, while leaving omitted settings to project defaults.
- Verify requested squash and source-deletion behavior from exact-MR
read-back for both newly created and reused MRs, including supported
GitLab response variations.
- Apply affirmative captain preferences to generated direct-PR briefs,
document the accepted preference grammar, and expand regression coverage
for defaults, reuse, and mismatch handling.

## Risk Assessment

✅ Low: Captain, the change is well-bounded, the strict affirmative
preference grammar resolves the prior findings, and the guarded GitLab
MR creation and reuse invariants remain intact.

## Testing

The previously successful full baseline plus focused adapter,
generated-brief, help, and request/read-back checks demonstrated
explicit squash/delete creation, exact-MR reuse, project-default
omission, response-shape fallbacks, distinct mismatch failures, guarded
identity verification, and captain-scoped preference behavior; all
passed with reviewer-visible CLI evidence and no UI screenshot because
this change has no rendered UI surface.

<details>
<summary>Evidence: GitLab MR behavior transcript</summary>

```text
$ bin/fm-forge.sh --help | sed -n /mr-create/,/mr-note/p
  fm-forge.sh mr-create <repo> --title <text> --source <branch>
    [--target <branch>] [--body-file <file>] [--draft]
    [--remove-source-branch] [--squash]
  fm-forge.sh mr-view <repo> <iid|canonical-url> [--target <branch>]
  fm-forge.sh mr-find <repo> <source-branch> [--target <branch>]
  fm-forge.sh mr-claim <repo> <iid|canonical-url> [--target <branch>]
  fm-forge.sh mr-status <repo> <iid|canonical-url> [--target <branch>]
    --status in-progress|blocked|deferred
  fm-forge.sh mr-release <repo> <iid|canonical-url> [--target <branch>]
    --status blocked|deferred|ready
  fm-forge.sh mr-labels <repo> <iid|canonical-url> [--target <branch>]
    (--add <existing-label>|--remove <existing-label>)...
  fm-forge.sh mr-note <repo> <iid|canonical-url> [--target <branch>]

$ bash -x tests/fm-forge.test.sh 2>&1 | grep relevant request/read-back assertions
+ assert_grep merge_requests/5 /var/folders/j_/g4b3__rs5j95fq6xf9fb17sc0000gn/T//fm-forge-tests.MqNey8/glab.log 'exact reuse did not read back the exact merge request'
+ assert_no_grep '--method POST' /var/folders/j_/g4b3__rs5j95fq6xf9fb17sc0000gn/T//fm-forge-tests.MqNey8/glab.log 'project-default reuse created another merge request'
+ assert_no_grep '--method POST' /var/folders/j_/g4b3__rs5j95fq6xf9fb17sc0000gn/T//fm-forge-tests.MqNey8/glab.log 'reduced-list reuse created another merge request'
+ expect_code 1 1 'conflicting merge-request remove-source intent'
+ local expected=1 actual=1 'label=conflicting merge-request remove-source intent'
+ expect_code 1 1 'unproven merge-request remove-source intent'
```
</details>
<details>
<summary>Evidence: GitLab MR adapter validation</summary>

```text
$ bash tests/fm-forge.test.sh
ok - forge identity comes only from the origin remote
ok - GitLab authentication uses the trusted host and stored credential only
ok - public GitHub identity resolves without a credentialed call
ok - self-hosted GitLab requires an exact trusted registration
ok - unsupported and GitHub Enterprise-ambiguous hosts fail before credentials
ok - issue output uses a minimal schema and bounded body
ok - merge-request URLs cannot override the origin host or project
ok - issue and merge-request URLs share strict canonical validation
ok - merge-request body files cannot exfiltrate files outside the worktree
ok - merge-request lookup requires one exact source project and branch pair
ok - merge-request creation reuses only one exact trusted candidate
ok - every merge-request lifecycle action verifies trusted identity
ok - pipeline checks are aggregated and only actionable jobs are returned
ok - an empty pipeline list remains pending during pipeline creation
ok - only an independently disabled CI feature authorizes a no-CI merge
ok - a running GitLab pipeline blocks merge
ok - a failing GitLab pipeline blocks merge
ok - a passing pipeline for an older SHA cannot authorize merge
ok - GitLab merge pins the reviewed head and verifies the merged state
ok - GitLab merge requires and rechecks the reviewed head SHA
ok - GitLab checks remain pinned to the reviewed head SHA
ok - non-default targets remain explicit through every lifecycle action
ok - GitLab merge poll emits one line only after merge
ok - local and unsupported remote schemes never select a credentialed forge
ok - unsafe remote text is rejected and never evaluated
```
</details>
<details>
<summary>Evidence: Captain preference validation</summary>

```text
$ bash tests/fm-brief.test.sh
ok - fm-brief.sh: bash -n succeeds
ok - fm-brief.sh: --help renders the complete header
ok - fm-brief.sh: no-mistakes/direct-PR/local-only briefs generate cleanly
ok - fm-brief.sh: ship and scout work use named guarded GitLab mutation commands
ok - fm-brief.sh: GitLab direct-PR commands honor only configured captain defaults
ok - fm-brief.sh: faster paths use configured authority without stacked review
ok - fm-brief.sh: no-mistakes DOD wording and signing preflight stay intact
ok - fm-brief.sh: ship project-memory wording carries the AGENTS.md authoring bar
ok - fm-brief.sh: --herdr-lab emits the complete hard safety contract
ok - fm-brief.sh: --herdr-lab uses its quoted Firstmate-owned helper path
ok - fm-brief.sh: ship and scout scaffolds make omitted Herdr intent fail-visible
ok - fm-brief.sh: Herdr lab contract covers scouts and rejects secondmate misuse
ok - fm-brief.sh: --no-projects scaffolds a project-less charter and guards misuse
ok - fm-brief.sh: custom pause verb renders in every scaffold
ok - fm-brief.sh: investigation and visual-review completions load the shared decision policy
ok - fm-brief: scout and secondmate code paths still scaffold well-formed briefs
```
</details>

## Pipeline

Updates from [git push
no-mistakes](https://github.com/kunchenguid/no-mistakes)

<details>
<summary>✅ **intent** - passed</summary>

✅ No issues found.

</details>

<details>
<summary>✅ **Rebase** - passed</summary>

✅ No issues found.

</details>

<details>
<summary>🔧 **Review** - 1 issue found → auto-fixed (2) ✅</summary>

- ⚠️ `bin/fm-brief.sh:120` - Intent requires “affirmative MR defaults,”
but the parser treats any recognized line containing `squash` or
`delete/remove source branch` as affirmative. For example, `- GitLab MR
defaults: do not squash; do not delete source branch` enables both
flags, contradicting docs/configuration.md’s claim that negative prose
is ignored. Captain, either reject negated phrases or define a strict
positive-token grammar and add regression coverage.

🔧 Fix: Honor negated GitLab MR preferences
1 warning still open:
- ⚠️ `bin/fm-brief.sh:120` - Captain, the negation fix still treats
common negative forms such as `GitLab MR defaults: squash disabled` or
`squash: false` as affirmative because it recognizes only a fixed phrase
list. This still conflicts with the intent’s “affirmative MR defaults”
requirement and docs/configuration.md’s claim that negative prose is
ignored. Use a strict positive grammar or explicitly parse option values
instead of enumerating negation phrases.

🔧 Fix: Enforce affirmative GitLab MR preference grammar
✅ Re-checked - no issues remain.

</details>

<details>
<summary>✅ **Test** - passed</summary>

✅ No issues found.
- `command -v tmux >/dev/null || { echo "tmux is required for e2e tests"
>&2; exit 1; }; tmux -V; rc=0; for t in tests/*.test.sh; do echo "== $t
=="; bash "$t" || rc=1; done; exit "$rc"`
- <code>Confirmed the configured baseline had already passed: `command
-v tmux …; for t in tests/*.test.sh; do bash &kunchenguid#34;$t&kunchenguid#34;; done`</code>
- <code>Inspected the change from
`9845ea3e043deb43814e826f2f337ff09be8194d` to
`f7888c4b440e91fab17f8fda57afa40e21894aba`</code>
- `bash tests/fm-forge.test.sh`
- `bash tests/fm-brief.test.sh`
- `bin/fm-forge.sh --help | sed -n '/mr-create/,/mr-note/p'`
- `bash -x tests/fm-forge.test.sh 2>&1 | grep -E '<relevant
request/read-back assertions>'`
- `git status --short`

</details>

<details>
<summary>✅ **Document** - passed</summary>

✅ No issues found.

</details>

<details>
<summary>✅ **Lint** - passed</summary>

✅ No issues found.

</details>

<details>
<summary>✅ **Push** - passed</summary>

✅ No issues found.

</details>

---------

Co-authored-by: Ivan Miles Piesh <ivan@ivanpiesh.info>
eyevanovich added a commit to eyevanovich/firstmate that referenced this pull request Jul 28, 2026
## Intent

Port the five captain-approved upstream Firstmate safety fixes without
importing unrelated upstream work: require two identical non-primary
path reads before recording a spawned isolated copy; use Linux proc
starttime plus complete command-line bytes for watcher PID identity
while preserving the non-Linux locale-stable fallback; give
PR-description opened and edited events independent concurrency
identities while preserving the exact required check name, bot
exemptions, read-only pull_request security model, and fork workflow;
bound repeated notifications for exited workers parked on external waits
without hiding live decisions, unknown liveness, away-mode ownership, or
secondmate idle behavior; and add strictly bounded validated historical
status context only after durable queue consumption and lock release.
Preserve the fork's GitLab adapter, signing bridge, no-mistakes
observer, local-only delivery, secondmate isolation, away mode, X mode,
and all five runtime backends. Retain all fork tests and add
counterfactual coverage for transient/stable/timeout/symlink/Orca
worktree paths, Linux clock/PID/cmdline/malformed-proc/lock identity,
fixed-head PR edits, parked-worker liveness and cadence, and wake
annotation crash boundaries, unsafe files, lock release, and caps.

## What Changed

- Require two stable worktree-path reads after spawn, and identify Linux
watcher processes using proc start time plus complete command-line bytes
with a locale-stable fallback elsewhere.
- Give PR-body opened and edited events independent concurrency
identities, and bound repeated stale notifications for exited workers
parked on external waits while preserving live, unknown, away-mode, and
secondmate handling.
- Add validated, size-capped wake status context only after durable
queue consumption and lock release, rejecting unsafe annotation files
and covering the new safety boundaries with counterfactual tests.

## Risk Assessment

⚠️ Medium: The five requested safety fixes appear correctly implemented
and well covered by source tests, but the spawn-path change lacks the
repository-required empirical backend verification evidence.

## Testing

The supplied full baseline had already passed; five focused integration
suites and a direct workflow-contract inspection then demonstrated all
authoritative safety behaviors end-to-end, with reviewer-visible
transcripts captured and no failures or residual worktree artifacts.

<details>
<summary>Evidence: Focused safety validation transcript</summary>

```text
COMMAND: bash tests/fm-spawn-worktree-settle.test.sh
ok - a single transient stale pane_current_path read is not accepted as the worktree
ok - an already-settled pane confirms via the existing inter-poll sleep, not an extra full cycle
ok - an unsettled pane times out without recording worktree metadata
ok - a symlinked project root is compared by physical path before settling
ok - Orca remains excluded because it owns worktree creation directly
# all fm-spawn-worktree-settle tests passed
COMMAND: bash tests/fm-watcher-lock.test.sh
ok - simultaneous watcher starts leave exactly one live process
ok - fm_pid_identity is locale-invariant across LC_ALL/LC_TIME
ok - Linux process identity ignores wall-clock changes and detects PID reuse, command changes, and malformed proc data
ok - watcher lock ownership is bound to Linux starttime and complete command-line bytes
ok - killed watcher stale lock is reclaimed
ok - live watcher lock with stale heartbeat is actionable
ok - guard banner leads when down with pending wakes (re-arm-after-drain) and stays silent when fresh
ok - concurrent fm_lock_try_acquire yields exactly one winner
ok - dead-pid stale lock is reclaimed by a single acquirer
ok - concurrent stale-lock steal yields exactly one winner
ok - live steal mutex is not reclaimed
ok - live-held lock is not stolen
ok - empty mid-acquire lock keeps a minimum grace
ok - late original claimant cannot claim a recreated lock
ok - paused mid-acquire claimant backs off to active stealer
ok - watch restart refuses to signal a reused pid
ok - watch restart reports a healthy peer without attaching to it
ok - watcher self-evicts when the lock pid no longer names it
ok - arm attaches to a live fresh watcher and exits only when that cycle ends
ok - arm starts+confirms a fresh watcher on a clean lock and self-heals a dead-pid lock (never healthy off a dead pid)
ok - arm cleans child watcher and temp output on HUP
ok - arm propagates an immediate watcher wake before confirmation
ok - arm attaches to a peer watcher after child stands down and exits when peer dies
watcher: lock held by live pid 76413 but heartbeat is stale for 838509312s (>300s); inspect or stop that watcher before re-arming.
ok - arm reports FAILED and exits non-zero when no fresh watcher can be confirmed
COMMAND: bash tests/no-mistakes-required-workflow.test.sh
ok - fixed-head signed opened, unsigned edited, signed edited yields 0/1/0
ok - body event groups are distinct while head changes remain coalesced
ok - run names expose monotonic numbers and immutable IDs
ok - fork, permission, check-name, marker, and bot-exemption contracts are preserved
COMMAND: bash tests/fm-watch-triage.test.sh
ok - signal_reason_is_actionable: benign absorbed, captain verbs and coalesced batches surfaced
ok - stale_is_terminal: terminal status surfaces, non-terminal and no-status are benign
ok - scan_captain_relevant_statuses lists only captain-relevant statuses
ok - classifier primitives: keyed decisions and activity phases, captain relevance, window-to-task, and overrides
ok - crew_is_provably_working: only working+run-step/pane is provable; idle/finished/parked/failed/unknown surface
ok - status_is_paused: only the leading paused verb matches, and paused is not captain-relevant
ok - crew_absorb_class: working/paused/none from one read; crew_is_paused and crew_is_provably_working agree
ok - signal_crew_provably_working: benign only when every referenced crew is provably working
ok - a no-verb signal whose crew is provably working is absorbed (no exit, no queue, suppressor advanced, beacon present)
ok - a bare turn-end whose crew is provably working (busy pane) is absorbed
ok - a bare turn-end whose crew is not provably working is surfaced (the swallowed-finish fix)
ok - a no-verb working: note whose crew is idle with no running pipeline is surfaced
ok - captain-relevant signal is surfaced (queue + exit) and marked surfaced
ok - a stale pane sitting on a terminal status is surfaced (queue + exit)
ok - a stale terminal-looking status is overridden and absorbed while a run is actively working, then wedge-escalated
ok - provably-working non-terminal stale is absorbed on first sight, then wedge-escalated past the threshold
ok - consecutive wedge escalations on the same pane accumulate and demand deep inspection at the threshold
ok - a pane becoming active again resets the consecutive wedge-escalation counter
ok - a not-provably-working non-terminal stale is surfaced immediately (never left to wait out the timer)
ok - a declared pause is absorbed on first sight, then re-surfaced as a recheck past the threshold, never wedge-escalated
ok - exited declared-pause and captain-held panes use bounded pause cadence while a live decision gate still surfaces once
ok - unknown endpoint liveness remains visible instead of using dead-agent pause suppression
ok - a declared paused secondmate re-surfaces on the bounded normal-mode cadence
ok - a non-paused secondmate retains normal stale suppression
ok - a resumed secondmate clears pause and stale tracking before stale exemption
ok - unchanged stale hashes reclassify when a crew enters or leaves pause
ok - a declared pause is periodically rechecked against authoritative active-run state
ok - a paused status overridden by authoritative working preserves its wedge timer and escalates
ok - matching non-terminal stale suppressors repair missing or corrupt stale-since timers
ok - triage log capping handles wc byte counts with leading spaces
ok - a heartbeat with no captain-relevant change is absorbed and backs off the cadence
ok - heartbeat backstop fail-safe surfaces a captain-relevant status the per-wake path missed
ok - the liveness beacon stays fresh while the watcher absorbs benign wakes (fm-guard never false-alarms)
ok - with .afk present the watcher reverts to one-shot so the daemon owns triage (no double-triage)
ok - AFK changed paused panes hand off plain stale identities for daemon-owned pause triage
COMMAND: bash tests/fm-wake-queue.test.sh
ok - concurrent append plus drain preserves queue records
ok - signal written while no watcher runs is caught on next run
ok - stale wake is queued before suppressor state is advanced
ok - a not-provably-working stale wake is queued before its suppressor is advanced
ok - registered custom check output is queued before cadence suppression
ok - two atomic drains cannot consume the same records twice
ok - drain collapses obvious duplicate heartbeat and signal records
ok - drain asserts watcher liveness: warns on a lapse, stays silent right after a fire
ok - structural signal enrichment is separate, deduped, home-local, and ignores unsafe status files
ok - bounded reads and per-item/global caps fail open with explicit truncation and omission markers
ok - slow annotation releases the append lock and a deleted status file fails open
ok - interruptions restore before commitment and never replay after raw commitment
```
</details>
<details>
<summary>Evidence: Actual PR workflow security and concurrency
contract</summary>

```text
Actual deployed workflow contract (.github/workflows/no-mistakes-required.yml)
name: Require no-mistakes
run-name: "PR #${{ github.event.pull_request.number }} body compliance - ${{ github.event.action }} - event ${{ github.run_number }} (run ${{ github.run_id }})"

on:
  pull_request:
    types: [opened, edited, synchronize, reopened]
    branches:
      - main

permissions:
  contents: read

# GitHub concurrency groups retain at most one pending run, replacing older
# pending runs even when cancel-in-progress is false. Give body-bearing events
# an immutable per-event group so first-time-fork approvals can never collapse
# opened/edited checks. Keep synchronize/reopened coalescing as before.
concurrency:
  group: no-mistakes-required-${{ github.event.pull_request.number }}-${{ (github.event.action == 'opened' || github.event.action == 'edited') && github.run_id || 'head-change' }}
  cancel-in-progress: true

jobs:
  check:
    name: PR must be raised via no-mistakes
    runs-on: ubuntu-latest
    if: >-
      github.event.pull_request.user.login != 'github-actions[bot]' &&
      github.event.pull_request.user.login != 'dependabot[bot]'
    steps:
      - name: Verify no-mistakes signature in PR body
        env:
          PR_BODY: ${{ github.event.pull_request.body }}
          PR_AUTHOR: ${{ github.event.pull_request.user.login }}
          PR_NUMBER: ${{ github.event.pull_request.number }}
        run: |
          set -eu
          marker='Updates from [git push no-mistakes](https://github.com/kunchenguid/no-mistakes)'
          if printf '%s' "${PR_BODY:-}" | grep -qF -- "$marker"; then
            echo "Found no-mistakes signature in PR #${PR_NUMBER} body."
            exit 0
          fi
          {
            echo "::error::This PR was not raised through no-mistakes."
            echo
            echo "Contributions to this repository must be submitted via 'git push no-mistakes'."
            echo "That pipeline runs the required review/test/lint/CI steps and writes a"
            echo "deterministic '## Pipeline' section into the PR body containing:"
            echo
            echo "    $marker"
            echo
            echo "See CONTRIBUTING.md for setup and the full workflow."
            echo
            echo "PR author: ${PR_AUTHOR}"
          } >&2
          exit 1
```
</details>

## Pipeline

Updates from [git push
no-mistakes](https://github.com/kunchenguid/no-mistakes)

<details>
<summary>✅ **intent** - passed</summary>

✅ No issues found.

</details>

<details>
<summary>✅ **Rebase** - passed</summary>

✅ No issues found.

</details>

<details>
<summary>⚠️ **Review** - 1 warning</summary>

- ⚠️ `bin/fm-spawn.sh:858` - The new two-read settling logic changes
launch behavior shared by tmux, herdr, zellij, and cmux, but the range
adds only stubbed counterfactual tests and no real-adapter verification
notes. CONTRIBUTING.md requires empirical verification for `fm-spawn.sh`
launch mechanics and corresponding backend evidence. Confirm/document
that verification before merging, or explicitly approve this
backend-neutral change as exempt.

</details>

<details>
<summary>✅ **Test** - passed</summary>

✅ No issues found.
- `command -v tmux >/dev/null || { echo "tmux is required for e2e tests"
>&2; exit 1; }; tmux -V; rc=0; for t in tests/*.test.sh; do echo "== $t
=="; bash "$t" || rc=1; done; exit "$rc"`
- <code>Confirmed the supplied baseline had already passed: `command -v
tmux &gt;/dev/null || { echo &kunchenguid#34;tmux is required for e2e tests&kunchenguid#34;
&gt;&amp;2; exit 1; }; tmux -V; rc=0; for t in tests/*.test.sh; do echo
&kunchenguid#34;== $t ==&kunchenguid#34;; bash &kunchenguid#34;$t&kunchenguid#34; || rc=1; done; exit
&kunchenguid#34;$rc&kunchenguid#34;`</code>
- <code>Inspected the target diff with `git diff --stat
b75c898..b783156`
and mapped changed paths to the authoritative intent</code>
- <code>`bash tests/fm-spawn-worktree-settle.test.sh` —
transient/stable/timeout/symlink/Orca worktree behavior</code>
- <code>`bash tests/fm-watcher-lock.test.sh` — Linux starttime, full
cmdline bytes, malformed proc data, locale fallback, PID reuse, and lock
identity</code>
- <code>`bash tests/no-mistakes-required-workflow.test.sh` —
opened/edited concurrency identities, fixed-head edits, exact check
name, bot exemptions, permissions, and fork behavior</code>
- <code>`bash tests/fm-watch-triage.test.sh` — exited parked-worker
cadence, live decisions, unknown liveness, away-mode ownership, and
secondmate behavior</code>
- <code>`bash tests/fm-wake-queue.test.sh` — post-consumption
annotation, unsafe files, lock release, crash boundaries, and bounded
caps</code>
- <code>Captured the deployed workflow contract with `sed -n
&kunchenguid#39;1,80p&kunchenguid#39; .github/workflows/no-mistakes-required.yml`</code>
- <code>Confirmed cleanup with `git status --short`</code>

</details>

<details>
<summary>✅ **Document** - passed</summary>

✅ No issues found.

</details>

<details>
<summary>✅ **Lint** - passed</summary>

✅ No issues found.

</details>

<details>
<summary>✅ **Push** - passed</summary>

✅ No issues found.

</details>

---------

Co-authored-by: Kun Chen <3233006+kunchenguid@users.noreply.github.com>
Co-authored-by: Freudator86 <94322668+Freudator86@users.noreply.github.com>
Co-authored-by: Freudator86 <tim@allesknut.de>
Co-authored-by: vvizlan <steve.rule.wilson@gmail.com>
Co-authored-by: Ivan Miles Piesh <ivan@ivanpiesh.info>
eyevanovich added a commit to eyevanovich/firstmate that referenced this pull request Jul 28, 2026
## Intent

Port the two captain-approved upstream review-lifecycle correctness
fixes onto the Firstmate fork at PR #12's landed head. Review diffs must
prefer freshly fetched GitHub or GitLab review heads stored under
private refs, use reachable recorded pr_head only as an offline
fallback, refuse forge-origin identity mismatches, and warn before final
local-branch fallback. Merged-review polling must retire exactly once
only after its notification is durable, use provider-neutral
identity-bound crash-recovery receipts, preserve GitLab's
hash-registered custom-check route including trusted self-hosted forges,
retain task and persistent-secondmate lifecycle state, preserve X-mode
and non-executing migration security, and safely recover through
watcher, migration, rearm, replacement, tamper, and teardown races.
Preserve all fork-specific GitLab, direct signing, no-mistakes observer,
local-only, secondmate, away/X-mode, and five-backend behavior; add
comprehensive counterfactual tests; do not merge.

## What Changed

- Prefer freshly fetched GitHub or GitLab review heads in private refs,
with validated offline and warned local-branch fallbacks.
- Retire merged review polls through provider-neutral, identity-bound
recovery receipts that preserve replacement polls and lifecycle state
across races and restarts.
- Extend migration, teardown, security, and counterfactual coverage for
GitHub and GitLab review lifecycles; the full test suite and targeted
lifecycle tests pass.

## Risk Assessment

🚨 High: Captain, the change should not merge until the crash window that
can duplicate a merged-review notification is explicitly resolved or
approved.

## Testing

The full baseline was already green; focused end-to-end CLI tests then
exercised review-head selection, identity and fallback handling,
exactly-once durable retirement and recovery races, migration/X-mode
security, and GitLab lifecycle behavior, all successfully. A
reviewer-visible CLI transcript was captured; no screenshot was
applicable because this change has no rendered UI surface.

<details>
<summary>Evidence: Review lifecycle end-to-end transcript</summary>

```text
$ bash tests/fm-review-diff.test.sh
ok - fresh GitHub review heads defeat stale reachable recorded heads
ok - fresh GitLab review heads defeat stale reachable recorded heads
ok - trusted self-hosted GitLab identity can fetch the current review head
ok - remote unavailability falls back to a reachable recorded review head
ok - invalid and multiline recorded review heads are refused
ok - tasks without a recorded review retain the local branch diff
ok - the local branch is a warned final fallback when neither review head is usable
ok - remote review identity mismatches refuse rather than using recorded metadata
$ bash tests/fm-pr-check-security.test.sh
ok - raw-byte parser accepts canonical URLs and rejects the complete adversarial matrix
ok - merged GitHub polls notify once and retirement preserves lifecycle metadata
ok - nonterminal, failed, tampered, and replacement poll states retain the right authority
ok - retirement recovers across notification, migration, and teardown crash boundaries
ok - PR and teardown entrypoints reject invalid arguments before every side effect
ok - valid direct and merge flows record exact metadata and reject multiline head metadata
ok - rejected metacharacter bytes remain inert at generation and watcher time
ok - static poll is silent except for one merged line and remains watcher-bounded
ok - interrupted atomic preparation cleans private temporaries and publishes nothing
ok - concurrent watchers observe only complete private poll publications
ok - post-rename poll validation faults revoke both names and allow a clean retry
ok - migration creates and validates private state before watcher exclusion
ok - migration pauses older watchers and acquires exclusion before its first scan or marker
ok - poll, marker, diagnostic, and quarantine paths refuse symlinks and directories
ok - marker and diagnostic rename errors and signals fail closed and recover durably on retry
ok - post-rename marker, diagnostic, and obligation faults are revoked and reconstructed on retry
ok - quarantine type and mode faults fail closed and recover only when a retry can validate them
ok - canonical and ambiguous failure obligations block every retry until all task artifacts are repaired
●━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
●  WATCHER DOWN - SUPERVISION IS OFF
●  1 task(s) in flight, but no watcher has a fresh beacon (last beat: never, grace 300s).
●  Trust the emitted supervision protocol for this harness; do not use shell & for watcher repair.
●  This is a supervision warning only; the guarded operation WILL still run.
●  resume supervision according to the session-start block for this harness; do not use shell &.
●━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
ok - ambiguous migration recovery accepts an explicitly validated replacement poll
ok - ambiguous repair rejects copied, metadata- or task-mismatched, forged, and partial poll publications
ok - all live, marker, diagnostic, X, custom-check, obligation, and teardown boundaries require single-link files
ok - canonical publication failure remains incomplete until a later clean retry rebuilds the poll
ok - legacy reserved obligations and delimiter-bearing task IDs retry without ambiguity
ok - migration never executes legacy checks, preserves X mode, quarantines ambiguity, and is idempotent
ok - historical X shims migrate only from the exact single-link mode-0755 identity
ok - direct registration refreshes authenticated v1 X shims across marker states
ok - bootstrap runs the non-executing migration at the locked session boundary
ok - bootstrap isolates incomplete poll migration from unrelated recovery sweeps
ok - watcher signals promptly stop custom checks and clean private state
ok - returned custom check descendants are drained on installed and fallback timeout paths
ok - teardown removes safe poll artifacts and refuses quarantine-directory symlinks without traversal
$ bash tests/fm-gitlab-review-lifecycle.test.sh
ok - GitLab review metadata and authenticated merge poll are recorded
ok - GitLab merged polls retire once while trust tampering preserves evidence
ok - pushed GitLab tasks require canonical merge proof for cleanup
ok - GitLab cleanup proof covers task state and identity failures
ok - the guarded merge path delegates GitLab to the narrow adapter
ok - an explicit GitLab merge method is not overridden
ok - merged GitLab work is proven landed before cleanup
```
</details>
- Outcome: 🔧 1 issue found → auto-fixed (2) ✅ across 3 runs (1h12m32s)

## Pipeline

Updates from [git push
no-mistakes](https://github.com/kunchenguid/no-mistakes)

<details>
<summary>✅ **intent** - passed</summary>

✅ No issues found.

</details>

<details>
<summary>✅ **Rebase** - passed</summary>

✅ No issues found.

</details>

<details>
<summary>⚠️ **Review** - 1 error</summary>

- 🚨 `bin/fm-watch.sh:827` - The required criterion says, “Merged-review
polling must retire exactly once only after its notification is durable”
and must recover safely through watcher races. The wake is durably
appended here before the retirement receipt is published. A crash
between lines 827 and 829 leaves the poll armed with no recovery
receipt, so the next watcher can append a duplicate merged notification.
Couple the durable notification to an identity-bound retirement intent
that recovery can correlate before permitting another poll execution.

</details>

<details>
<summary>🔧 **Test** - 1 issue found → auto-fixed (2) ✅</summary>

- 🚨 tests failed with exit code 1
- `command -v tmux >/dev/null || { echo "tmux is required for e2e tests"
>&2; exit 1; }; tmux -V; rc=0; for t in tests/*.test.sh; do echo "== $t
=="; bash "$t" || rc=1; done; exit "$rc"`

🔧 Fix: Supply teardown fixtures’ new check-library dependency
1 error still open:
- 🚨 tests failed with exit code 1
- `command -v tmux >/dev/null || { echo "tmux is required for e2e tests"
>&2; exit 1; }; tmux -V; rc=0; for t in tests/*.test.sh; do echo "== $t
=="; bash "$t" || rc=1; done; exit "$rc"`

🔧 Fix: Supply legacy teardown fixture’s check-library dependency
✅ Re-checked - no issues remain.
- `command -v tmux >/dev/null || { echo "tmux is required for e2e tests"
>&2; exit 1; }; tmux -V; rc=0; for t in tests/*.test.sh; do echo "== $t
=="; bash "$t" || rc=1; done; exit "$rc"`
- <code>Baseline previously completed: `command -v tmux &gt;/dev/null ||
{ echo &kunchenguid#34;tmux is required for e2e tests&kunchenguid#34; &gt;&amp;2; exit 1; };
tmux -V; rc=0; for t in tests/*.test.sh; do echo &kunchenguid#34;== $t ==&kunchenguid#34;;
bash &kunchenguid#34;$t&kunchenguid#34; || rc=1; done; exit &kunchenguid#34;$rc&kunchenguid#34;`</code>
- `bash tests/fm-review-diff.test.sh`
- `bash tests/fm-pr-check-security.test.sh`
- `bash tests/fm-gitlab-review-lifecycle.test.sh`
- <code>Captured combined behavioral output with `tee
/var/folders/j_/g4b3__rs5j95fq6xf9fb17sc0000gn/T/no-mistakes-evidence/01KYMW7NF1APVGVDT8G2J1RRVJ/review-lifecycle-e2e-transcript.txt`</code>
- <code>Verified `git status --short` remained empty after
testing</code>

</details>

<details>
<summary>✅ **Document** - passed</summary>

✅ No issues found.

</details>

<details>
<summary>✅ **Lint** - passed</summary>

✅ No issues found.

</details>

<details>
<summary>✅ **Push** - passed</summary>

✅ No issues found.

</details>

---------

Co-authored-by: Ivan Miles Piesh <ivan@ivanpiesh.info>
eyevanovich added a commit to eyevanovich/firstmate that referenced this pull request Jul 31, 2026
## Intent

Port the approved upstream secondmate reliability batch onto the current
Firstmate fork in two sub-batches: first, confident missing endpoint
detection and startup recovery; second, correlated secondmate replies
plus generation-bound inherited-config rereads. Preserve fork-specific
safety, canonical FIRSTMATE_OP operational inputs, direct-report
ownership, all five harness/backend semantics, and the full inheritance
allowlist (crew-dispatch.json, crew-harness, backlog-backend,
forge-hosts, signing-agent, and captain-shared.md), while continuing to
exclude secondmate-harness, local captain.md, and learnings.md. Pending
replies must remain parent-owned, correlation-bound, observable without
reading secondmate chat, and resilient to malformed, symlinked,
hard-linked, wrong-device, wrong-home, or wrong-destination private
artifacts. Config rereads must use exact validated destination bytes,
generation-specific pointers, durable retry/quarantine handling, and
per-home serialization so older values cannot overtake newer ones. Add
extensive hermetic coverage, preserve direct captain input behavior,
validate every script/test/backend, and ship the committed branch for
review without merging it.

## What Changed

- Recover confidently missing or dead secondmate endpoints during
startup while preserving ambiguous and unreadable sessions.
- Add parent-owned, correlation-bound reply tracking with final-report
resolution, bounded recovery, and observable escalation.
- Deliver inherited-config rereads from immutable, generation-specific
snapshots with serialized coalescing, retry, and quarantine handling to
prevent stale values overtaking newer ones.

## Risk Assessment

✅ Low: Captain, the latest change makes stale generations logically
ineligible before best-effort cleanup, preserves merged immutable values
across partial updates, and leaves a crash-recoverable successor
generation without introducing a substantiated remaining defect.

## Testing

The previously successful full baseline plus focused end-to-end scripts
exercised confident endpoint recovery across backend semantics,
parent-owned correlation and escalation without chat scraping, hostile
private-artifact rejection, exact-byte generation-bound config rereads
with retry/quarantine/serialization, inheritance and direct-input
behavior, and session-start recovery; all passed, reviewer-visible CLI
transcripts were captured, and the worktree remained unchanged.

<details>
<summary>Evidence: Correlated pending-reply end-to-end
transcript</summary>

```text
ok - normal correlated reply resolves once (idempotent)
ok - correlated progress waits for the final answer
ok - completed turn with no report triggers exactly one recovery
ok - recovery attempts reconcile without reinjection
ok - recovery reply resolves the original expectation
ok - second missed turn escalates once and remains durable
ok - failed escalation publication remains retryable and publishes once
ok - transport success cannot masquerade as reply success
ok - undelivered records remain immutable across scan paths
ok - delivery confirmation fallback reconciles durably
ok - unrelated events and stale correlation ids cannot resolve
ok - restart preserves expectation and exact parent destination
ok - wrong-home reports are detected but do not silently acknowledge
ok - direct unmarked captain input creates no expectation
ok - fm-send marked secondmate path creates pending and embeds corr
ok - status-pointed document resolves the expectation
ok - optional helper report resolves without being required for correctness
ok - backend busy/idle observation covers Pi/Claude paths without conversation scrape
ok - tmux and zellij unknown states use bounded capture fallback
ok - tick skips terminal records and reuses target observations
ok - correlations are reused only for matching open task records
ok - tick end-to-end: miss -> one recovery -> escalate -> durable
ok - failed transport discards undelivered expectation only
ok - private pending records reject malformed and foreign artifacts
ok - unsafe delivery, parent-status, and wrong-home artifacts are never followed
ok - private pending directory symlinks are refused without side effects
ok - all pending-reply tests passed
```
</details>
<details>
<summary>Evidence: Secondmate endpoint detection and recovery
transcript</summary>

```text
ok - fm_backend_tmux_agent_state: separates live, dead, missing, ambiguous, and unreadable
ok - fm_backend_tmux_agent_state: rejects malformed targets before probing tmux
ok - fm_backend_herdr_agent_state: preserves missing/no-agent/live/unknown husk behavior
ok - fm_backend_agent_state: routes tmux/Herdr and preserves Zellij/Orca/cmux unverified semantics
ok - sweep: a confirmed-dead secondmate endpoint is killed and respawned
ok - sweep: an already-live secondmate is untouched and distinguishable in verbose diagnostics
ok - sweep: an authoritatively missing Pi secondmate window is relaunched
ok - sweep: an existing ambiguous Pi process prevents duplicate recovery
ok - sweep: transient target unreadability never licenses recovery
ok - sweep: failed relaunch diagnostics distinguish a confidently missing endpoint
ok - sweep: an unverified harness blocks recovery with a concrete diagnostic
ok - sweep: idempotent by construction - a live secondmate is never re-touched on a later run
ok - sweep: skipped entirely under FM_BOOTSTRAP_DETECT_ONLY=1, exactly like the other mutating sweeps
ok - sweep: a silent no-op with no kind=secondmate meta present (a secondmate home's own natural scoping)
# all fm-secondmate-liveness tests passed
```
</details>
<details>
<summary>Evidence: Inherited-config propagation and generation safety
transcript</summary>

```text
ok - A1 fm-harness.sh secondmate resolves the fallback chain; crew mode unchanged
ok - C1 fm-harness.sh secondmate-model/secondmate-effort resolve the optional tokens; bare harness stays empty (backward-compat)
ok - B1 propagate_inheritable_config: copy, idempotence, convergence, absence-mirror, exclusion, no-op, skip diagnostics
ok - B2 spawn: secondmate runs the secondmate harness; its home inherits declared config
ok - B3 spawn: an absent secondmate-harness falls back to the crew harness (backward-compat)
ok - B4 spawn: no config at all -> own harness and no propagation side effects
ok - B5 spawn: an explicit per-spawn harness arg overrides config/secondmate-harness
ok - B6 spawn: an unverified resolved secondmate harness is refused (guard intact)
ok - C2 spawn: a bare harness-only secondmate-harness file launches with no model/effort flag (backward-compat)
ok - C3 spawn: config/secondmate-harness's model token threads --model into the launch and meta
ok - C4 spawn: config/secondmate-harness's model+effort tokens thread into the launch and meta
ok - C5 spawn: an explicit --model overrides config/secondmate-harness's model token; the file's effort token still applies
ok - C6 spawn: an explicit --effort overrides config/secondmate-harness's effort token; the file's model token still applies
ok - C7 spawn: an explicit --harness starts with clean model/effort defaults
ok - C8 spawn: an explicit --harness still honors explicit model/effort flags
ok - C9 spawn: the harness fallback chain still resolves with no tokens; crew/scout launches are unaffected by this feature
ok - B7 bootstrap sweep pushes, re-converges, and mirrors absence; never inherits secondmate-harness
ok - B8 bootstrap sweep propagates config even when the home's tracked files are already current
ok - B9 bootstrap sweep defers new inherited config until the home ignores it
ok - B10 bootstrap sweep with no inherited config is a config no-op and still fast-forwards
ok - B11 bootstrap sweep surfaces config propagation failures
ok - B11 bootstrap rereads completed config writes after partial propagation
ok - B12 config-push propagates via shared live discovery, reports items, rereads on change only, and does not fast-forward
ok - B13 config-push reports dirty, non-allowing, and invalid homes without failing warnings-only runs
ok - B14 config-push exits nonzero on real propagation errors
ok - B14 config-push rereads completed config writes after partial propagation
ok - B15 config reread is per-home, exact-byte, ordered, and pointer-only
ok - B16 config reread isolation, ABSENT, generation safety, send failure, and retry
ok - B20 config reread publication failures retain exact generations for retry
ok - B21 config reread captures immutable bytes directly into reserved generations
ok - B21 config reread never falls back to mutable retry reports
ok - B21 config reread serializes concurrent propagation and delivery
ok - B22 full config reread retry queues coalesce before new publication
ok - B23 mixed config reread generations coalesce atomically
ok - B26 config reread supersedes older pending generations
ok - B27 legacy mutable retry reports are quarantined
ok - B28 config reread coalescing preserves prior validated items
ok - B29 cleanup failure cannot reactivate superseded config generations
ok - B17 config reread skips unchanged homes and reads destination post-write bytes
ok - B18 bootstrap config reread path works; spawn flexibility remains defaults-only
ok - B19 bootstrap respawns before inherited-config reread
ok - B25 spawn quarantines stale rereads without blocking relaunch
ok - B24 bootstrap detect-only mode remains filesystem read-only
# all fm-secondmate-harness tests passed
```
</details>
<details>
<summary>Evidence: Session-start recovery transcript</summary>

```text
ok - context digest distinguishes ABSENT, empty-but-present, and populated files
ok - a lock refusal prints a loud read-only banner, skips every mutating step, and still completes the digest
ok - digest sections are ordered diagnostics-first, bulk-context-last
ok - session start: configured and auto-detected Herdr homes never require tmux
ok - session start: an absent recorded tmux window relaunches its Pi secondmate exactly once
ok - session start: an existing ambiguous Pi process prevents duplicate recovery
ok - session start: transient tmux unreadability never licenses a relaunch
ok - session start: the proven bare-shell recovery path remains intact
ok - session start: a confirmed Herdr husk is closed and relaunched
ok - status tail is bounded to the configured line count, with the full log path always printed
ok - orphan status logs are printed once with bounded tails
ok - tmux endpoint liveness is reported per task: alive for a live window, dead for a gone one
ok - herdr endpoint liveness is reported per task: alive for a live pane, dead for a gone one
ok - fm-session-start.sh composes the real fm-lock.sh, fm-bootstrap.sh, and fm-wake-drain.sh output verbatim
ok - compatible tasks-axi backlog rendering is compact, bounded, and preserves recovery metadata
ok - manual backlog rendering prints only title lines with hold and blocker metadata
ok - unavailable or incompatible tasks-axi falls back to compact manual backlog rendering
ok - an empty fleet reports (none) for in-flight tasks and an absent AFK flag
ok - session start emits X-mode cadence guidance in the harness supervision block
ok - next step delegates watcher ownership to the AFK daemon
ok - session start emits exactly one detected harness block and reports Pi extension load state
ok - session start rejects stale Pi loaded markers
ok - session start accepts current Pi markers written before lock acquisition
ok - session start rejects Pi sessions missing the turn-end guard marker
ok - session start rejects Pi loaded markers from previous sessions
```
</details>

## Pipeline

Updates from [git push
no-mistakes](https://github.com/kunchenguid/no-mistakes)

<details>
<summary>✅ **intent** - passed</summary>

✅ No issues found.

</details>

<details>
<summary>✅ **Rebase** - passed</summary>

✅ No issues found.

</details>

<details>
<summary>🔧 **Review** - 3 issues found → auto-fixed (4) ✅</summary>

- 🚨 `bin/fm-config-inherit-lib.sh:1081` - The required invariant “older
values cannot overtake newer ones” remains reachable. If generation G1
is pending, a later push creates G2, then this code sorts and sends G1
before G2; if G1 succeeds but G2 fails, the live secondmate applies G1
after the destination already contains G2. Coalesce/supersede older
pending generations before delivery, or otherwise enforce
latest-generation application at this shared boundary.
- 🚨 `bin/fm-config-inherit-lib.sh:1005` - The required “exact validated
destination bytes” are not durably retained when initial instruction
creation fails before producing a temporary file. The code saves only
the change report, then a later retry rebuilds that generation by
rereading the mutable destination, which may already contain a newer
value. Capture immutable destination bytes during the originating locked
propagation, and never reconstruct an old generation from current
destination state.
- 🚨 `bin/fm-pending-reply-lib.sh:601` - A pending reply resolves on any
parent status line containing the correlation token, including a
`working` progress line. The secondmate brief permits material working
reports, so a correlated progress update can terminally resolve the
expectation before the actual answer arrives, defeating the required
observable pending-reply guarantee. Restrict resolution to
answer-bearing terminal/status-pointer forms or define an explicit
acknowledgement grammar.

🔧 Fix: Captain, enforce monotonic rereads and final replies
2 errors still open:
- 🚨 `bin/fm-config-inherit-lib.sh:991` - The required durable
retry/quarantine behavior is blocked by any `.report` artifact left by
the previous implementation: this branch returns before capturing or
sending the current immutable generation, and every later push repeats
the same failure. Quarantine unsupported mutable reports at this shared
boundary, then continue from a freshly captured immutable destination
snapshot.
- 🚨 `bin/fm-config-inherit-lib.sh:1020` - When coalescing backlog, the
synthetic report marks every allowlisted item as pushed regardless of
the current propagation report. A skipped or failed destination—such as
a symlink or non-gitignored path—is therefore rendered as `ABSENT` and
sent as authoritative, even though propagation deliberately left it
unchanged. Build the latest snapshot only from destination states
validated as copied or authoritatively absent; do not convert
unvalidated items into removals.

🔧 Fix: Captain, quarantine legacy retries and preserve validation
1 error still open:
- 🚨 `bin/fm-config-inherit-lib.sh:1085` - Coalescing a partial
propagation drops still-needed authoritative bytes from the prior
immutable generation. Example: G1 contains a successfully copied
`forge-hosts` change but its send is pending; G2 validates only
`crew-harness` because `forge-hosts` is now skipped or errors. The new
snapshot contains only `crew-harness`, then deletes G1, so the
secondmate never receives the validated G1 `forge-hosts` value. At this
boundary, merge current validated items with the latest immutable values
for currently unvalidated items before retiring the backlog.

🔧 Fix: Captain, preserve validated items across partial coalescing
1 error still open:
- 🚨 `bin/fm-config-inherit-lib.sh:1158` - The required monotonic
guarantee remains reachable when retiring merged backlog artifacts
fails. This loop ignores `rm` failures, publishes the merged generation,
and forgets the surviving old paths. On a later partial push, a
surviving old generation can be merged without the already-delivered
newer value and reapply older bytes. Require complete
retirement/quarantine before publication, or keep the newest merged
snapshot in the durable backlog until stale artifacts are conclusively
neutralized.

🔧 Fix: Captain, prevent superseded config generations from reactivating
✅ Re-checked - no issues remain.

</details>

<details>
<summary>✅ **Test** - passed</summary>

✅ No issues found.
- `command -v tmux >/dev/null || { echo "tmux is required for e2e tests"
>&2; exit 1; }; tmux -V; rc=0; for t in tests/*.test.sh; do echo "== $t
=="; bash "$t" || rc=1; done; exit "$rc"`
- <code>Configured baseline: `command -v tmux &gt;/dev/null || { echo
&kunchenguid#34;tmux is required for e2e tests&kunchenguid#34; &gt;&amp;2; exit 1; }; tmux -V;
rc=0; for t in tests/*.test.sh; do echo &kunchenguid#34;== $t ==&kunchenguid#34;; bash
&kunchenguid#34;$t&kunchenguid#34; || rc=1; done; exit &kunchenguid#34;$rc&kunchenguid#34;`</code>
- `bash tests/fm-pending-reply.test.sh`
- `bash tests/fm-secondmate-liveness.test.sh`
- `bash tests/fm-secondmate-harness.test.sh`
- `bash tests/fm-session-start.test.sh`
- <code>`git status --short` to verify testing introduced no worktree
artifacts</code>

</details>

<details>
<summary>✅ **Document** - passed</summary>

✅ No issues found.

</details>

<details>
<summary>🔧 **Lint** - 1 issue found → auto-fixed ✅</summary>

- ⚠️ linter found issues (exit code 1)

🔧 Fix: Fix ShellCheck temp handoff and unreachable assertions
✅ Re-checked - no issues remain.

</details>

<details>
<summary>✅ **Push** - passed</summary>

✅ No issues found.

</details>

---------

Co-authored-by: Ivan Miles Piesh <ivan@ivanpiesh.info>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants