Skip to content

fix(crew-state): report a PR merge only when the merge record proves it - #5

Merged
onyx-space merged 5 commits into
mainfrom
fm/crew-state-passed-mislabels-unmerged-pr
Sep 11, 2026
Merged

onyx-space merged 5 commits into
mainfrom
fm/crew-state-passed-mislabels-unmerged-pr

Conversation

@onyx-space

@onyx-space onyx-space commented Sep 11, 2026 •

Copy link
Copy Markdown
Owner

摘要

修的是机队读任务状态的工具 bin/fm-crew-state.sh 一处会误报的措辞:终端 passed 运行的状态行原先固定写 run passed: PR merged/closed,可是 no-mistakes 的 outcome: passed 只说明管线跑完,不等于 PR 已经合并——合并授权关闭时,合并仍等队长批准。实测这条:PR #4 当时在 gh pr view 报 OPEN / MERGEABLE,而状态行称它 merged;任何读这行的 agent(包括我)都会据此误报。

现在这条状态行只说被证实的 PR 状态:

  • 有证据(bin/fm-pr-lib.sh 的 merge-notified 记录,"由真实合并观察写入")→ run passed: PR merged: <url>,并把被证实的 PR 明确写出来,读者可当场核对;
  • 拿不到证据 → run passed: PR held for merge[: <url>],绝不出现 merged 字样;
  • RUN_STATE=done 分类不变(绿检查后的 held-for-merge 仍是 done),只改措辞的诚实度;
  • 证据身份以 run 自己记录的 pr: 为准,仅当 run 没记 pr: 时才回落到任务 state/<id>.meta 的 pr=——这样只会把结论从 merged 移向 held for merge,不会反向,即严格更保守。

bin/fm-crew-state.sh 文件头新增状态用词表(该脚本作为单一 owner):PR merged / PR held for merge / checks green: PR ready for review / parked at <gate> / validating (running|fixing) / ci running / run failed,并写死「未证实不得说 merged」这条铁律,后续加分支的人照抄即可。bin/fm-pr-lib.sh 与 bin/fm-merge-outcome-lib.sh 的对外契约未动。

验证

  • 真实数据端到端(live-verify,ALL-PASS):绑真实 no-mistakes axi status(cbm-axi 的 completed run,outcome: passed)+ 真实 forge 观察经 bin/fm-pr-poll.sh / fm_merge_outcome_report 写 merge-notified 记录。同一个 run:无记录读 PR held for merge: <url>;有记录读 PR merged: <url>;记录属于另一个 PR 时不算证据;记录存在但读不出(空文件)也不算证据。
  • 旧行为在同一批真实输入上复现:base aba68fc 读 run passed: PR merged/closed;中间版本 e40b4b4(按 meta 优先)对「旧 meta PR 已合并、本次 run 是另一个未合并 PR」会误报 merged——本 PR 已修成 run 优先。
  • tests/fm-crew-state.test.sh 89 项全绿,其中 5 条为本 PR 新增:无记录→held for merge;有记录→精确输出 PR merged;身份不符→不构成证据;run 的 pr 压过过时的 meta;完全没有 PR 身份→不声称合并也不编造 URL。
  • bin/fm-lint.sh(pinned ShellCheck 0.11.0 + actionlint 1.7.12)通过。

风险

低。只改一个状态报告器的措辞;state: / source: 两个 token 不变,仓库内没有消费者依赖被删掉的旧串;所有读不到证据的路径都倒向保守措辞,因此不存在「未证实却说 merged」的方向。

English

Intent

队长原话:「修」。

指的是这条实测缺陷(crew-state-passed-mislabels-unmerged-pr):机队读任务状态的工具会把「管线跑完」说成「PR 已合并」,可实际上 PR 还开着等你批。

现场:bin/fm-crew-state.sh 对 PR #4 输出 state: done · source: run-step · run passed: PR merged/closed,而同一时刻 gh pr view 4 是 OPEN / MERGEABLE、未合并(yolo=off,正等队长批准)。这行字会让人(和任何读它的 agent)以为已经合了 —— 队长刚差点被我据此误报。

队长要的是:状态行不能凭空声称一个没被证实的合并。

What Changed

  • bin/fm-crew-state.sh no longer prints "run passed: PR merged/closed" for every terminal passed run. A passed outcome now states only what is proven: run passed: PR merged: <url> when the merge-notified record owned by bin/fm-pr-lib.sh exists for this run's PR, otherwise run passed: PR held for merge[: <url>].
  • Added nm_pr_url / nm_pr_merge_proven, which resolve the attributed run's own pr field first (falling back to the task's state/<id>.meta pr=) and accept the merge-notified record as the only proof — an unknown, unparseable, or identity-mismatched URL is not proof, so a stale meta PR can no longer vouch for a different, still-open PR.
  • Documented the status vocabulary in the script header, including the hard rule that a no-mistakes terminal outcome: passed means the pipeline finished, never that the PR merged.
  • tests/fm-crew-state.test.sh adds five cases covering unmerged-passes-read-held-for-merge, proven-merge wording, an identity-mismatched merge record, run pr outranking stale meta, and a passed run with no PR identity (no merge claim, no invented URL).

Risk Assessment

✅ Low: The change is presentation-only (state/source words unchanged, detail wording gated by a durable proof), the four new tests discriminate the fixed behaviour from both the old unconditional claim and the stale-meta identity mix-up, and every unreadable/absent-proof path fails toward the hedged wording, so nothing unproven can be claimed as merged.

Testing

Drove the real bin/fm-crew-state.sh end-to-end against real run data: a real no-mistakes completed run with outcome passed (cbm-axi, branch docs/agent-pr-axi-usage, head == run head) plus the real merge-proof path (fm-pr-poll.sh reading the real forge with gh, fm_merge_outcome_report writing the merge-notified record), all with task meta and merge records in a throwaway home. Before the change the same real run read 'run passed: PR merged/closed' with no merge record present; the fixed script reads 'run passed: PR held for merge: <url>', upgrades to 'run passed: PR merged: <url>' only once a real merge observation is recorded, and refuses to be proved by a merge record for a different PR (the meta-first revision e40b4b4 claimed 'run passed: PR merged' on those same live inputs). An unreadable merge record was also driven live and stayed a hedge. The repo's targeted suite (tests/fm-crew-state.test.sh, 89 checks) passes, including the regression pair. Two shapes could not be bound live on this host: a passed run whose PR is still open on the forge (every bindable completed run references a merged PR, and all other candidates fail the branch+head attribution rule), and a passed run with no PR identity at all (the only such runs, cyber-mux and prime-agent, fail the head rule). The open-PR wording is covered by the repo suite's identity-matching case, and the no-PR-identity criterion is covered by a focused exact-line test added in this run; neither gap leaves doubt about the intent, since the product keys the claim on firstmate's own merge record rather than on the forge or on the pipeline outcome.

  • Live validation: ✅ go - 5 of 7 scenarios driven live against the product
Scenario Result Live Evidence
Reading crew state for a task whose run finished with outcome passed and whose merge has no firstmate merge record never prints a merged claim — it reads held-for-merge and names the PR ✅ pass live live-verify.transcript.txt case A: bound to the live completed run in ~/code/cbm-axi via real no-mistakes axi status (outcome: passed, pr .../cbm-axi/pull/2), state dir in a throwaway home…
The pre-change script really did mislabel that same passed run as merged (the reported defect reproduced on live data) ✅ pass live live-verify.transcript.txt case A, base revision aba68fc on the identical live inputs: state: done · source: run-step · run passed: PR merged/closed
Once the merge is actually observed on the forge and recorded through firstmate's own merge-notification path, the same run reads 'PR merged' and names the proved PR ✅ pass live live-verify.transcript.txt case B: bin/fm-pr-poll.sh --validated github ... github.com onyx-space/cbm-axi 2 returned merged from the real forge, fm_merge_outcome_report wrote the merge-notified…
Adversarial: a merge record for a different PR than the attributed run's own PR proves nothing — the line stays held-for-merge, names the run's own PR, and never names the stale merged PR ✅ pass live live-verify.transcript.txt case C: merge record published (from a real forge observation) for tasks-axi#1 while the bound run's own pr is cbm-axi#2 -> `run passed: PR held for merge: https://github.co…
Adversarial: a merge record that exists but cannot be read (present, empty) is not proof ✅ pass live live-verify.transcript.txt case D: empty live-d.pr-poll-merge-notified file -> held for merge, no merged claim
A passed run with no PR identity anywhere (neither the run nor the task meta records a PR) reads held-for-merge and prints no URL at all ⏸️ untested no Prior payload recorded result pass with live=false: the only evidence was a focused in-repo unit test (test_terminal_passed_without_pr_identity_names_no_url in tests/fm-crew-state.test.sh asserting th…
The originally reported instantiation — a passed run whose PR is still OPEN on the forge awaiting the captain's approval — reads held-for-merge, never merged ⏸️ untested no Missing capability: no live completed run on this host has a still-OPEN PR whose branch tip equals (or is descended by) the run head, so the open-PR state cannot be driven live. Provide one by keeping…
Evidence: Live verification driver (runs the real script against real no-mistakes/gh data; also runs base aba68fc and meta-first e40b4b4 on the same inputs)

Source: Live verification driver (runs the real script against real no-mistakes/gh data; also runs base aba68fc and meta-first e40b4b4 on the same inputs)

#!/usr/bin/env bash
# Live verification: crew-state-passed-mislabels-unmerged-pr
#
# Drives the REAL bin/fm-crew-state.sh against a REAL `no-mistakes axi status`
# answer read out of a REAL initialized checkout (default ~/code/cbm-axi,
# whose checked-out branch tip is exactly the head of a completed run with
# outcome: passed and pr: https://github.com/onyx-space/cbm-axi/pull/2), and
# against the REAL merge-notification path: bin/fm-pr-poll.sh observes the real
# forge with gh, and bin/fm-merge-outcome-lib.sh publishes the merge-notified
# record exactly as bin/fm-watch.sh does.
#
# Every task's meta + merge record lives in an isolated throwaway FM home under
# $TMPDIR, so nothing in the user's real firstmate state is touched.
#
# Usage: live-verify.sh <repo-root> [<second-live-repo-root>]
set -u

REPO=${1:?usage: live-verify.sh <repo-root>}
LIVE_REPO=${2:-/Users/onyx/code/cbm-axi}

WORK=$(mktemp -d "${TMPDIR:-/tmp}/fm-crew-state-live.XXXXXX")
STATE=$WORK/state
mkdir -p "$STATE"
FIXED=$REPO/bin/fm-crew-state.sh

# Two older revisions, for before/after on the same live inputs:
#   BASE  - pre-change, the released behaviour the defect report is against.
#   META1 - the first fix, which bound the merge proof to the task meta pr=
#           instead of the attributed run's own pr: (the reviewed defect).
BASE_REV=aba68fcd639320a06b80156f873dfe53aea74764
META1_REV=e40b4b4
for rev in "$BASE_REV" "$META1_REV"; do
  mkdir -p "$WORK/$rev"
  ( cd "$REPO" && git archive "$rev" bin ) | tar -x -C "$WORK/$rev" \
    || { echo "could not archive $rev"; exit 1; }
done
BASE=$WORK/$BASE_REV/bin/fm-crew-state.sh
META1=$WORK/$META1_REV/bin/fm-crew-state.sh

fail_count=0
ok()   { printf 'PASS  %s\n' "$1"; }
bad()  { printf 'FAIL  %s\n' "$1"; fail_count=$((fail_count + 1)); }
check_equals() { # <label> <expected> <actual>
  if [ "$2" = "$3" ]; then ok "$1"; else bad "$1
      expected: $2
      actual:   $3"; fi
}
check_not_contains() { # <label> <needle> <haystack>
  case "$3" in
    *"$2"*) bad "$1 (found '$2' in: $3)" ;;
    *) ok "$1" ;;
  esac
}
check_contains() { # <label> <needle> <haystack>
  case "$3" in
    *"$2"*) ok "$1" ;;
    *) bad "$1 (missing '$2' in: $3)" ;;
  esac
}

crew_state() { # <script> <id> -> one state line
  FM_HOME="$WORK" FM_STATE_OVERRIDE="$STATE" FM_ROOT_OVERRIDE="$REPO" \
    "$1" "$2"
}

write_meta() { # <id> <worktree> <pr>
  cat > "$STATE/$1.meta" <<EOF
window=fm:fm-$1
worktree=$2
kind=ship
pr=$3
EOF
}

# Publish a merge-notified record through the REAL firstmate path: the real poll
# program reads the real forge, and the outcome lib records what it observed.
record_real_merge() { # <id> <pr-url>
  local id=$1 url=$2 owner_path number observed rc
  owner_path=${url#https://github.com/}; owner_path=${owner_path%/pull/*}
  number=${url##*/}
  observed=$("$REPO/bin/fm-pr-poll.sh" --validated github "$url" github.com \
    "$owner_path" "$number" 2>/dev/null || true)
  if [ "$observed" != merged ]; then
    printf 'NOTE  %s: the real forge did not report %s as merged (poll said: %s)\n' \
      "$id" "$url" "${observed:-<nothing>}"
    return 1
  fi
  ( . "$REPO/bin/fm-merge-outcome-lib.sh"
    fm_merge_outcome_report "$WORK" "$STATE" "$id" "$url" poll ) || return 1
  rc=0
  [ -f "$STATE/$id.pr-poll-merge-notified" ] || rc=1
  # owner_path/number are used only to print what the record holds.
  printf 'NOTE  %s: merge observed on the real forge (%s/%s) and recorded via fm_merge_outcome_report\n' \
    "$id" "$owner_path" "$number"
  return "$rc"
}

printf '=== live inputs ===\n'
printf '$ no-mistakes axi status            (in %s)\n' "$LIVE_REPO"
( cd "$LIVE_REPO" && no-mistakes axi status 2>&1 | sed -n '1,8p;/^outcome:/p' )
printf '$ gh pr view %s --json state\n' "${LIVE_PR_NUMBER:-2}"
gh pr view 2 --repo onyx-space/cbm-axi --json state,url 2>&1 | head -2
printf 'branch: %s\nhead:   %s\n' \
  "$(git -C "$LIVE_REPO" branch --show-current)" "$(git -C "$LIVE_REPO" rev-parse --short HEAD)"
printf '\n'

PR_MAIN=https://github.com/onyx-space/cbm-axi/pull/2      # the live run's own PR
PR_STALE=https://github.com/onyx-space/tasks-axi/pull/1   # a different, real, merged PR
PR_STALE_REPO=~/code/tasks-axi

if [ -d "$PR_STALE_REPO" ]; then :; else PR_STALE_REPO=; fi

# ---------------------------------------------------------------------------
printf '=== A. terminal passed run, NO merge record: no merge may be claimed ===\n'
write_meta live-a "$LIVE_REPO" "$PR_MAIN"
printf '$ fm-crew-state.sh live-a   (fixed, this branch)\n'
a_new=$(crew_state "$FIXED" live-a); printf '%s\n' "$a_new"
printf '$ fm-crew-state.sh live-a   (base %s, before the change)\n' "${BASE_REV:0:7}"
a_old=$(crew_state "$BASE" live-a); printf '%s\n' "$a_old"
check_equals "A1 fixed: unproven merge reads held-for-merge and names the PR" \
  "state: done · source: run-step · run passed: PR held for merge: $PR_MAIN" "$a_new"
check_not_contains "A2 fixed: the line never claims a merge" "merged" "$a_new"
check_contains "A3 base revision really did mislabel a passed run (the reported defect)" \
  "run passed: PR merged/closed" "$a_old"

# ---------------------------------------------------------------------------
printf '\n=== B. same run, merge now PROVEN by the real poll -> merged may be claimed ===\n'
write_meta live-b "$LIVE_REPO" "$PR_MAIN"
if record_real_merge live-b "$PR_MAIN"; then
  printf '$ cat $STATE/live-b.pr-poll-merge-notified\n'
  sed 's/^/    /' "$STATE/live-b.pr-poll-merge-notified"
  printf '$ fm-crew-state.sh live-b   (fixed)\n'
  b_new=$(crew_state "$FIXED" live-b); printf '%s\n' "$b_new"
  check_equals "B1 a proven merge is reported as merged, naming the proved PR" \
    "state: done · source: run-step · run passed: PR merged: $PR_MAIN" "$b_new"
else
  bad "B0 could not record the real merge observation"
fi

# ---------------------------------------------------------------------------
printf '\n=== C. adversarial: task meta names another (merged) PR, the run works a different one ===\n'
write_meta live-c "$LIVE_REPO" "$PR_STALE"
if record_real_merge live-c "$PR_STALE"; then
  printf '$ fm-crew-state.sh live-c   (fixed)\n'
  c_new=$(crew_state "$FIXED" live-c); printf '%s\n' "$c_new"
  printf '$ fm-crew-state.sh live-c   (meta-first revision %s, the reviewed defect)\n' "$META1_REV"
  c_meta1=$(crew_state "$META1" live-c); printf '%s\n' "$c_meta1"
  check_equals "C1 fixed: only the run's own PR identity can be proved; a stale meta PR proves nothing" \
    "state: done · source: run-step · run passed: PR held for merge: $PR_MAIN" "$c_new"
  check_not_contains "C2 fixed: no merge claim when the run's own PR has no merge record" \
    "merged" "$c_new"
  check_not_contains "C3 fixed: the stale merged PR is never named" "tasks-axi" "$c_new"
  check_contains "C4 meta-first revision really did claim a merge for the stale meta PR (live repro of the reviewed finding)" \
    "run passed: PR merged" "$c_meta1"
else
  bad "C0 could not record the real merge observation for the stale meta PR"
fi

# ---------------------------------------------------------------------------
printf '\n=== D. adversarial: no merge record, unreadable/absent proof stays a hedge ===\n'
write_meta live-d "$LIVE_REPO" "$PR_MAIN"
: > "$STATE/live-d.pr-poll-merge-notified"      # present but empty: not proof
printf '$ fm-crew-state.sh live-d   (fixed, empty merge record)\n'
d_new=$(crew_state "$FIXED" live-d); printf '%s\n' "$d_new"
check_equals "D1 an unreadable merge record is not proof" \
  "state: done · source: run-step · run passed: PR held for merge: $PR_MAIN" "$d_new"

printf '\n=== result: %s ===\n' "$( [ "$fail_count" -eq 0 ] && echo ALL-PASS || echo "$fail_count-FAILED")"
printf 'workdir (removed): %s\n' "$WORK"
rm -rf "$WORK"
[ "$fail_count" -eq 0 ]
Evidence: Live verification transcript (ALL-PASS, reproducible)

Source: Live verification transcript (ALL-PASS, reproducible)

=== A. terminal passed run, NO merge record: no merge may be claimed === fm-crew-state.sh live-a (fixed): state: done · source: run-step · run passed: PR held for merge: https://github.com/onyx-space/cbm-axi/pull/2&#10;fm-crew-state.sh live-a (aba68fc): state: done · source: run-step · run passed: PR merged/closed === B. merge PROVEN by the real poll === fm-crew-state.sh live-b (fixed): state: done · source: run-step · run passed: PR merged: https://github.com/onyx-space/cbm-axi/pull/2&#10;=== C. stale meta names another merged PR, the run works a different one === fm-crew-state.sh live-c (fixed): state: done · source: run-step · run passed: PR held for merge: https://github.com/onyx-space/cbm-axi/pull/2&#10;fm-crew-state.sh live-c (e40b4b4): state: done · source: run-step · run passed: PR merged === result: ALL-PASS ===

=== live inputs ===
$ no-mistakes axi status            (in ~/code/cbm-axi)
run:
  id: "01M0T63R4GYMW1BVJG7HJ48SPY"
  branch: docs/agent-pr-axi-usage
  status: completed
  head: 8b3782b8
  head_sha: 8b3782b828cbe8100ffa140b1c018810cb48b107
  pr: "https://github.com/onyx-space/cbm-axi/pull/2"
  findings: none
outcome: passed
$ gh pr view 2 --json state
{"state":"MERGED","url":"https://github.com/onyx-space/cbm-axi/pull/2"}
branch: docs/agent-pr-axi-usage
head:   8b3782b

=== A. terminal passed run, NO merge record: no merge may be claimed ===
$ fm-crew-state.sh live-a   (fixed, this branch)
state: done · source: run-step · run passed: PR held for merge: https://github.com/onyx-space/cbm-axi/pull/2
$ fm-crew-state.sh live-a   (base aba68fc, before the change)
state: done · source: run-step · run passed: PR merged/closed
PASS  A1 fixed: unproven merge reads held-for-merge and names the PR
PASS  A2 fixed: the line never claims a merge
PASS  A3 base revision really did mislabel a passed run (the reported defect)

=== B. same run, merge now PROVEN by the real poll -> merged may be claimed ===
NOTE  live-b: merge observed on the real forge (onyx-space/cbm-axi/2) and recorded via fm_merge_outcome_report
$ cat $STATE/live-b.pr-poll-merge-notified
    fm-pr-poll-merge-notified-v1
    github
    github.com
    onyx-space/cbm-axi
    2
$ fm-crew-state.sh live-b   (fixed)
state: done · source: run-step · run passed: PR merged: https://github.com/onyx-space/cbm-axi/pull/2
PASS  B1 a proven merge is reported as merged, naming the proved PR

=== C. adversarial: task meta names another (merged) PR, the run works a different one ===
NOTE  live-c: merge observed on the real forge (onyx-space/tasks-axi/1) and recorded via fm_merge_outcome_report
$ fm-crew-state.sh live-c   (fixed)
state: done · source: run-step · run passed: PR held for merge: https://github.com/onyx-space/cbm-axi/pull/2
$ fm-crew-state.sh live-c   (meta-first revision e40b4b4, the reviewed defect)
state: done · source: run-step · run passed: PR merged
PASS  C1 fixed: only the run's own PR identity can be proved; a stale meta PR proves nothing
PASS  C2 fixed: no merge claim when the run's own PR has no merge record
PASS  C3 fixed: the stale merged PR is never named
PASS  C4 meta-first revision really did claim a merge for the stale meta PR (live repro of the reviewed finding)

=== D. adversarial: no merge record, unreadable/absent proof stays a hedge ===
$ fm-crew-state.sh live-d   (fixed, empty merge record)
state: done · source: run-step · run passed: PR held for merge: https://github.com/onyx-space/cbm-axi/pull/2
PASS  D1 an unreadable merge record is not proof

=== result: ALL-PASS ===
workdir (removed): /var/folders/jq/t9r90khx6kzcw250jx08lw1r0000gn/T//fm-crew-state-live.5q78A0
Evidence: Targeted repo suite log: tests/fm-crew-state.test.sh (89 checks) — includes the four new cases and the added no-PR-identity case

Source: Targeted repo suite log: tests/fm-crew-state.test.sh (89 checks) — includes the four new cases and the added no-PR-identity case

ok - terminal passed run with an open PR reads held-for-merge, never merged ok - terminal passed run with a proven merge record reads PR merged ok - an identity-mismatched merge record does not prove this PR merged ok - the attributed run's PR identity outranks a stale task meta PR ok - a passed run with no PR identity claims no merge and names no URL all fm-crew-state tests passed

ok - active run-step is authoritative
ok - stale needs-decision over active run is superseded
ok - stale blocked over active run is superseded
ok - daemon/timeout blocked claim over a live fixing run reads as run alive
ok - socket refusal or missing socket over a stale fixing run reports blocked
ok - socket refusal over a terminal attributed run reports blocked
ok - broken-pipe blocker over a live run keeps the plain superseded reading
ok - genuine daemon-down blocked line still reports blocked
ok - genuine parked run is not flagged superseded
ok - scalar gate parked run is not flagged superseded
ok - gate block parked run is not flagged superseded
ok - ci-ready status log beats monitoring run
ok - ci-monitoring run with checks already green surfaces done
ok - top-level ci status uses ci log green marker
ok - terminal no-checks ci-monitor marker surfaces done
ok - base-advance rearm after green stays working
ok - pending no-checks ci-monitor marker stays working
ok - ci-monitoring run with checks not yet green stays working
ok - a fresh issue after an earlier green reading is not masked
ok - stale checks-green status log does not mask CI relapse
ok - ci fixing is not overridden by an earlier green marker
ok - top-level fixing is not overridden by a stale ci running row
ok - top-level fixing is not overridden by a stale done log
ok - terminal passed run is authoritative
ok - terminal passed run with an open PR reads held-for-merge, never merged
ok - terminal passed run with a proven merge record reads PR merged
ok - an identity-mismatched merge record does not prove this PR merged
ok - the attributed run's PR identity outranks a stale task meta PR
ok - a passed run with no PR identity claims no merge and names no URL
ok - terminal failed run is authoritative
ok - orphaned ci monitor after green reads as held-for-merge done
ok - status-only failed orphaned ci monitor after green reads done
ok - genuinely failing CI keeps the failed verdict
ok - a second failed step disqualifies the orphaned-monitor reclassification
ok - cross-branch run is attributed via the real runs list
ok - socket refusal over a coarse active run reports blocked
ok - failed ledger record reads unknown only when the daemon is provably down
ok - cross-branch attribution picks the branch's most recent row
ok - a live run outranks a terminal run bound to the same worktree
ok - runs-list selection prefers a live row over a newer terminal one
ok - an unfetched live sibling outranks a terminal row at the worktree's exact commit
ok - two terminal rows keep the existing newest-first precedence
ok - an unclassifiable status row keeps the ledger's newest-first precedence
ok - a terminal run with no live sibling is unchanged
ok - coarse run does not probe another branch's ci log
ok - another branch's run is ignored, falls back
ok - no run + a busy semantic record reads working, attributed to its source
ok - a converted adapter never reads working from rendered footer text
ok - grok still reads working through its isolated rendered-tail fallback
ok - herdr's native busy verdict reads working with no record present
ok - a herdr CLI that fails to answer reads unknown/unreachable, never gone
ok - an alive endpoint whose scrollback read failed stays working
ok - a husk pane (agent gone) still reads gone for reclaim
ok - a mid-tool-call crew stays working because its record outranks herdr's generation state
ok - an idle record with idle agent_status stays not-busy (no regression for a human-blocked agent)
ok - no run + idle pane uses the status-log verb
ok - no run + idle pane parses keyed status syntax
ok - no run + idle pane on a paused: status reports state: paused with its reason
ok - no run + idle pane honors the configured paused verb
ok - a trailing resolved: event does not corrupt state render (idle stays idle)
ok - dead window ignores stale status log
ok - a tmux that fails to answer reads unknown/unreachable, never gone
ok - closed pane still reports a terminal run-step
ok - closed pane still reports an active run-step
ok - no timeout command uses perl bound
ok - scout skips the run lookup
ok - torn-down worktree is handled gracefully
ok - fm-crew-state remote: alive endpoint falls through to the routed status log
ok - fm-crew-state remote: an idle alive endpoint reads alive, never gone or dead
ok - fm-crew-state remote: an unreachable host reads unknown-remote, never gone or dead
ok - fm-crew-state remote: the remote host's own dead verdict is reported truthfully
ok - missing meta is handled gracefully
ok - crew_is_provably_working absorbs a validating crew found only via the runs-list fallback
ok - crew_is_provably_working still surfaces a genuinely stopped crew (safety property preserved)
ok - usage error exits 2
ok - historical same-branch rewritten head is not attributed as current
ok - active run with valid descendant fix head remains current
ok - local work advanced past run head invalidates attribution
ok - pipeline-owned active run binds without head equality and beats the failed row
ok - a genuinely failed run with no later run is not hidden
ok - coarse scan anchors the unresolvable active row instead of falling to an older one
ok - coarse scan with a mismatched anchor stays unknown and lets the pane answer
ok - the exemption requires branch_sync.state=pipeline_owned
ok - the exemption never applies to a terminal run
ok - missing run head falls back instead of matching by branch
ok - active fix round with an unfetched pipeline head reads working
ok - unanchored unverifiable active row is never attributed
ok - unresolvable terminal row never reads as current
ok - runs-list continuation attribution works when axi answers another branch
all fm-crew-state tests passed
Evidence: Evidence README: what was driven live, how to reproduce, and the live limits

Source: Evidence README: what was driven live, how to reproduce, and the live limits

# Evidence — crew-state-passed-mislabels-unmerged-pr

Intent: the status line must not claim a PR merged that has not been proven
(机队状态行不能凭空声称一个没被证实的合并).

## What was driven live

`live-verify.sh <worktree-root>` drives the **real** `bin/fm-crew-state.sh`
against **real** inputs:

- run status from a **real** `no-mistakes axi status` call inside
  `~/code/cbm-axi`, whose checked-out branch tip (`8b3782b8`) is
  exactly the head of a real completed run (`outcome: passed`,
  `pr: https://github.com/onyx-space/cbm-axi/pull/2`);
- the merge proof from the **real** firstmate path: `bin/fm-pr-poll.sh`
  observes the real forge with `gh`, and `bin/fm-merge-outcome-lib.sh`'s
  `fm_merge_outcome_report` publishes the `*.pr-poll-merge-notified` record
  exactly as `bin/fm-watch.sh` does;
- task `state/<id>.meta` files and merge records in a throwaway `$TMPDIR` home,
  so the user's real firstmate state is untouched.

Same driver also runs two older revisions against the same live inputs:
base `aba68fc` (pre-change) and `e40b4b4` (the reviewed meta-first fix), to
reproduce the reported behaviour before the change.

| case | live inputs | fixed (this branch) | older revision |
|---|---|---|---|
| A | passed run, no merge record | `run passed: PR held for merge: <url>` | base `aba68fc`: `run passed: PR merged/closed` |
| B | passed run, merge recorded by the real poll from a real forge observation | `run passed: PR merged: <url>` | — |
| C | stale meta names another real merged PR; the run's own PR has no record | held for merge, names the run's PR, never the stale one | `e40b4b4`: `run passed: PR merged` |
| D | merge record present but empty | held for merge | — |

Reproduce: `./live-verify.sh ~/.no-mistakes/worktrees/9573b29b9316/01M26XWAR71A7JAAGCGYNJ3F4F`
(needs `no-mistakes`, `gh`, and the initialized checkouts above).

`fm-crew-state.test.log` is the targeted repo suite run,
`tests/fm-crew-state.test.sh`, which drives the real script with stubbed
upstream tools (89 checks, all green).

## Limits of the live drive

- No bindable live run on this machine has an **open** PR: both bindable
  completed runs (`cbm-axi#2`, `tasks-axi#1`) reference PRs the forge already
  reports MERGED, and every other initialized repo's completed run fails the
  branch+head attribution rule (branch tip advanced past the run head). The
  open-PR instantiation of case A is therefore covered by the repo suite
  (`test_terminal_passed_without_merge_record_says_held_for_merge`), not live.
- A passed run with **no PR identity at all** cannot be bound live (the only
  such runs, in `cyber-mux` and `prime-agent`, fail the head rule), so that
  acceptance criterion is covered by the focused test
  `test_terminal_passed_without_pr_identity_names_no_url` added in this run.

Pipeline

Updates from git push no-mistakes

✅ **intent** - passed

✅ No issues found.

✅ **Rebase** - passed

✅ No issues found.

🔧 **Review** - 3 issues found → auto-fixed (3) ✅
  • ℹ️ bin/fm-crew-state.sh:41 - The new HARD RULE is stated absolutely ("no detail line may contain &fix(bin): split the silent-pane readout three ways and defer the focus-blocked teardown close #34;merged&fix(bin): split the silent-pane readout three ways and defer the focus-blocked teardown close #34; unless nm_pr_merge_proven confirms it"), but it is not true of this file. On the no-attributed-run fallback path (line 924) the helper relays a crew-authored status note verbatim (status_line_note returns the text after the verb, bin/fm-classify-lib.sh:381), so a crew line like done: PR merged (a shape this repo's own fixtures use, tests/fm-inactive-reconcile.test.sh:740) yields state: done · source: status-log · PR merged with no proof check. Because the new header declares itself the single owner of the vocabulary, a future maintainer enforcing the sentence literally would have to suppress the crew's claim. Scope the sentence to the detail lines this helper derives from the run outcome (source: run-step); the relayed line already carries source: status-log as its own provenance.
  • ℹ️ bin/fm-crew-state.sh:560 - The pre-existing nm_ci_checks_state comment still asserts the opposite of what this change adds in its header: 'for a repo where merge is left to the captain ... it only reaches outcome=passed once the PR is actually merged (or failed/cancelled if closed)'. The change's premise (PR fix(spawn): pre-answer Pi's project-trust gate on pi and pi-signed launches #4 read OPEN/MERGEABLE while the run-step line claimed a passed outcome, yolo=off) and the new header lines 39-44 say passed means the PIPELINE finished, without asserting a merge. The file now carries two contradictory statements of one invariant, and a reader trusting this comment would restore the old unconditional 'run passed: PR merged/closed' wording that the intent forbids. Reconcile the comment with the evidence-bounded reading (the ci-log heuristic it justifies stands on its own markers either way).
  • ℹ️ bin/fm-crew-state.sh:731 - Evidence-bounded ceiling of the new probe, not a defect. Concrete sequence: a ship task whose crew never ran bin/fm-pr-check.sh (no armed poll, so no merge-notified record can be written) is later approved by the captain in the forge UI; the run-step line then stays run passed: PR held for merge: &lt;url&gt; indefinitely although the merge landed, so a reader learns 'held for merge' means 'no proof of merge here', not 'the PR is still open'. The header documents this as the intended safe word whenever the record is absent or unreadable (lines 27-31), and the alternative (probing the forge on every read, in a path the watcher, fm-classify-lib.sh's absorb check, and the fleet snapshot call repeatedly under a timeout) is a larger change the intent does not ask for. No action; recorded so the threshold is explicit rather than implicit.

🔧 Fix applied.
1 warning still open:

🔧 Fix applied.
1 info still open:

  • ℹ️ tests/fm-crew-state.test.sh:994 - The three new tests call the real marker writer in-process: fm_pr_poll_merge_mark_notified at tests/fm-crew-state.test.sh:994, :1021 and :1042. That function runs umask 077 (bin/fm-pr-lib.sh:991) and deliberately never restores it, because its production callers (bin/fm-watch.sh, bin/fm-pr-merge.sh) are one-shot script processes. Here it is called from the sourced test shell, so umask 077 leaks for the remainder of the run — every test after this point (the file continues for ~1500 more lines of test invocations, including all the run-attribution, remote-secondmate and coarse-ledger cases) executes with umask 077 instead of the umask 022 that tests/lib.sh:34 pins on purpose for the state-root/process-event fixture contracts, and unlike the repo's own convention for scoped restrictive umasks (tests/fm-captain-hold-lifecycle.test.sh:203 does (umask 077; mkdir -p ...), tests/fm-procevent.test.sh:345 scopes it into a child process). The direction is currently harmless — a stricter umask only makes new files/dirs more private, and the only mode gate on that root (bin/fm-procevent-lib.sh:535) merely refuses group/world-writable state dirs, so no test fails today — so this is fixture hygiene, not a demonstrated breakage. Mechanical remedy: run the writer in a subshell, e.g. ( fm_pr_poll_merge_mark_notified &#34;$d/state&#34; &lt;id&gt; github github.com o/r 1 ) || fail ..., or re-umask 022 after each call, so the suite keeps running under the umask its fixtures were written against.

🔧 Fix applied.
✅ Re-checked - no issues remain.

✅ **Test** - passed

✅ No issues found.

  • Live validation: ✅ go - 5 of 7 scenarios driven live against the product
Scenario Result Live Evidence
Reading crew state for a task whose run finished with outcome passed and whose merge has no firstmate merge record never prints a merged claim — it reads held-for-merge and names the PR ✅ pass live live-verify.transcript.txt case A: bound to the live completed run in ~/code/cbm-axi via real no-mistakes axi status (outcome: passed, pr .../cbm-axi/pull/2), state dir in a throwaway home…
The pre-change script really did mislabel that same passed run as merged (the reported defect reproduced on live data) ✅ pass live live-verify.transcript.txt case A, base revision aba68fc on the identical live inputs: state: done · source: run-step · run passed: PR merged/closed
Once the merge is actually observed on the forge and recorded through firstmate's own merge-notification path, the same run reads 'PR merged' and names the proved PR ✅ pass live live-verify.transcript.txt case B: bin/fm-pr-poll.sh --validated github ... github.com onyx-space/cbm-axi 2 returned merged from the real forge, fm_merge_outcome_report wrote the merge-notified…
Adversarial: a merge record for a different PR than the attributed run's own PR proves nothing — the line stays held-for-merge, names the run's own PR, and never names the stale merged PR ✅ pass live live-verify.transcript.txt case C: merge record published (from a real forge observation) for tasks-axi#1 while the bound run's own pr is cbm-axi#2 -> `run passed: PR held for merge: https://github.co…
Adversarial: a merge record that exists but cannot be read (present, empty) is not proof ✅ pass live live-verify.transcript.txt case D: empty live-d.pr-poll-merge-notified file -> held for merge, no merged claim
A passed run with no PR identity anywhere (neither the run nor the task meta records a PR) reads held-for-merge and prints no URL at all ⏸️ untested no Prior payload recorded result pass with live=false: the only evidence was a focused in-repo unit test (test_terminal_passed_without_pr_identity_names_no_url in tests/fm-crew-state.test.sh asserting th…
The originally reported instantiation — a passed run whose PR is still OPEN on the forge awaiting the captain's approval — reads held-for-merge, never merged ⏸️ untested no Missing capability: no live completed run on this host has a still-OPEN PR whose branch tip equals (or is descended by) the run head, so the open-PR state cannot be driven live. Provide one by keeping…
  • tests/fm-crew-state.test.sh (targeted repo suite, real script under test, stubbed upstream tools): 89 checks, all green, including the four new cases and the regression pair
  • ~/.no-mistakes/evidence/01M26XWAR71A7JAAGCGYNJ3F4F/live-verify.sh &lt;worktree&gt; — live driver, re-run twice with identical ALL-PASS results
  • Live case A: real passed run (cbm-axi, outcome: passed, pr: .../pull/2), no merge record -> fixed reads run passed: PR held for merge: &lt;url&gt;; base aba68fc reads run passed: PR merged/closed
  • Live case B: real forge observation via bin/fm-pr-poll.sh --validated github &lt;url&gt; github.com onyx-space/cbm-axi 2 plus fm_merge_outcome_report -> fixed reads run passed: PR merged: &lt;url&gt;
  • Live case C (adversarial): merge record for a different real merged PR while the run's own PR has none -> fixed stays held-for-merge naming the run's PR and never names the stale PR; e40b4b4 (meta-first) claimed run passed: PR merged on the same inputs
  • Live case D (adversarial): an unreadable (empty) merge record is not proof -> held for merge
  • Added focused test test_terminal_passed_without_pr_identity_names_no_url for acceptance criterion 2 (no URL append when no PR identity exists anywhere)
  • Read-only scan of every initialized no-mistakes repo on this host (sqlite3 ~/.no-mistakes/state.sqlite, no-mistakes runs, git merge-base --is-ancestor) to establish that only cbm-axi and tasks-axi have a bindable live completed run, both with already-merged PRs
✅ **Document** - passed

✅ No issues found.

✅ **Lint** - passed

✅ No issues found.

✅ **Push** - passed

✅ No issues found.

onyx-space and others added 5 commits September 11, 2026 08:26
`outcome: passed` 只说明 no-mistakes 管线跑完,不代表 PR 已经合并——合并授权关闭时,合并仍等队长批准。此前 `bin/fm-crew-state.sh` 对每个 passed 运行都写死 `run passed: PR merged/closed`;实测 PR #4 在 `gh pr view 4` 报 OPEN/MERGEABLE 的同时,状态行却称它 merged,读它的 agent 会据此误报。

- passed 分支改为按证据说话:只有 merge-notified 记录(`bin/fm-pr-lib.sh` 的 `fm_pr_poll_merge_already_notified`,读写契约不动)证实了该任务规范 PR 身份的合并,才写 `run passed: PR merged`;否则写 `run passed: PR held for merge` 并在 `state/<id>.meta` 有 `pr=` 时附上 URL。`RUN_STATE=done` 的分类不变(绿检查后的 held-for-merge 仍是 done)。
- 文件头新增状态用词表(本脚本作为单一 owner),写明「未证实不得说 merged」,供后续加分支的人照抄。
- `tests/fm-crew-state.test.sh` 加三条用例:无合并记录 -> held for merge 且带 URL;有合并记录 -> 精确输出 `PR merged`;记录属于另一个 PR -> 不构成证据。三条在旧代码上均失败。
@onyx-space
onyx-space merged commit f6766dd into main Sep 11, 2026
27 of 28 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant