Skip to content
Open
Show file tree
Hide file tree
Changes from 3 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions .agents/skills/ship-landing/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,13 +9,13 @@ metadata:
# Ship landing

For PR-based ship tasks, the ready signal depends on mode: `no-mistakes` reports `done [at=<epoch>]: PR <url> checks green` after CI is green, while `direct-PR` reports `done [at=<epoch>]: PR <url>` after opening the PR, each only for a non-draft PR; a lane that deliberately holds a draft declares a wait instead, and `bin/fm-pr-check.sh` refuses to arm merge monitoring on a draft.
Run `bin/fm-pr-check.sh <id> <PR url>` with the URL copied from that ready signal or the resolved checks-green `fm-crew-state.sh` line - it records `pr=` and the forge's `pr_head=` when available in the task's meta and arms the watcher's merge poll.
Run `bin/fm-pr-check.sh <id> <PR url>` with the URL copied from that ready signal or the resolved done `fm-crew-state.sh` line - it records `pr=` and the forge's `pr_head=` when available in the task's meta and arms the watcher's merge poll.
`bin/fm-dod-lib.sh` owns the named-head gate on that ready signal: a ship `done:` whose named head exists only in the worker's disposable copy is not ready (`bin/fm-crew-state.sh` reports blocked, `bin/fm-pr-check.sh` refuses to register, and a secondmate does not publish that done upstream).
That blocked reading is the gate working, not a stuck worker, so steer the worker on the commit the refusal names rather than waiting.
A direct-PR worker pushes that commit to its PR branch, and a local-only worker commits it on its ship branch.
A no-mistakes worker re-validates it with /no-mistakes so the pipeline stays the one publisher; it never pushes from its copy.
In no-mistakes mode the earlier `done [at=<epoch>]: {summary}` is the pipeline handoff and is not gated.
Tell the captain the PR's full `https://...` URL copied from the worker's ready line, the resolved checks-green crew-state line, or the task's `pr=` metadata, a concise outcome summary, and the no-mistakes risk level when applicable.
Tell the captain the PR's full `https://...` URL copied from the worker's ready line, the resolved done crew-state line, or the task's `pr=` metadata, a concise outcome summary, and the no-mistakes risk level when applicable.
A captain instruction to merge is explicit authority; `yolo` is the only standing routine merge authority.
For any custom `state/<id>.check.sh` you write yourself, keep it an ordinary single-link mode-`0700` file, print one line only when firstmate should wake, print nothing otherwise, finish before `FM_CHECK_TIMEOUT`, then bind its current bytes with `bin/fm-check-register.sh <id>` before the watcher may execute it.
Retire a custom check only through `bin/fm-check-unregister.sh <id>` (or `bin/fm-teardown.sh` for a spawned task); never hand-compose an `rm` with `$STATE`/`$ID`.
Expand Down
347 changes: 233 additions & 114 deletions bin/fm-crew-state.sh

Large diffs are not rendered by default.

53 changes: 51 additions & 2 deletions bin/fm-nm-run-lib.sh
Original file line number Diff line number Diff line change
Expand Up @@ -112,6 +112,43 @@ fm_nm_head_matches_worktree() { # <worktree> <run_head>
git -C "$wt" merge-base --is-ancestor "$local_full" "$run_full" 2>/dev/null
}

# Task incarnation: code identity alone cannot tell this task's run from one an
# earlier task finished on the same branch at the same head, which is exactly
# what a refresh task reusing an existing PR branch sees. A run created before
# the task's start epoch is therefore never the task's result. no-mistakes run
# ids are ULIDs, whose first ten Crockford base32 characters encode the creation
# time in milliseconds (the same instant as the database's created_at).
# Prints the creation epoch in whole seconds; nonzero when $1 is not a ULID.
fm_nm_run_created_epoch() { # <run-id>
local id alphabet=0123456789ABCDEFGHJKMNPQRSTVWXYZ ms=0 i prefix
id=$(printf '%s' "${1:-}" | tr '[:lower:]' '[:upper:]')
[ "${#id}" -eq 26 ] || return 1
case "$id" in [0-7]*) ;; *) return 1 ;; esac
case "$id" in *[!0-9ABCDEFGHJKMNPQRSTVWXYZ]*) return 1 ;; esac
for ((i = 0; i < 10; i++)); do
prefix=${alphabet%%"${id:i:1}"*}
ms=$((ms * 32 + ${#prefix}))
done
printf '%s' $((ms / 1000))
}

# 0 when run $1 belongs to the task incarnation that started at epoch $2: no
# recorded start (no bound), or a creation time at or after it. A run whose
# creation time cannot be read cannot be proven to be the task's own and is
# refused whenever a start is recorded.
fm_nm_run_in_incarnation() { # <run-id> <task-start-epoch>
local created
[ -n "${2:-}" ] || return 0
created=$(fm_nm_run_created_epoch "$1") || return 1
[ "$created" -ge "$2" ]
}

# Local "YYYY-MM-DD HH:MM" of epoch $1, the resolution and timezone of the
# `no-mistakes runs` creation column (the CLI runs in this process's timezone).
fm_nm_epoch_ledger_minute() { # <epoch>
date -r "$1" '+%Y-%m-%d %H:%M' 2>/dev/null || date -d "@$1" '+%Y-%m-%d %H:%M' 2>/dev/null
}

# Liveness class of a recorded ledger status word.
# The coarse `no-mistakes runs` ledger emits database status words; an
# `axi status` run object reports its terminal result through its own outcome
Expand Down Expand Up @@ -440,13 +477,22 @@ fm_nm_run_is_executing() { # <toon-output>
# name matches. The one live bind is the EXECUTING record on the `axi status`
# route (fm_nm_run_is_executing above), which the caller pairs with its own
# liveness evidence.
# When optional task start epoch $5 is supplied, the answering row must have been
# created in or after that start's minute (fm_nm_run_in_incarnation above owns
# the rule): a newest row from before the task started is an earlier task's
# history, so it prints nothing. The anchor row is exempt; it proves code
# identity only.
# Read-only: git reads resolve objects in place; custody never changes.
fm_nm_runs_status_for_worktree() { # <worktree> <branch> <runs-list-output> [expected-head]
local wt=$1 branch=$2 list=$3 expected_head=${4:-}
fm_nm_runs_status_for_worktree() { # <worktree> <branch> <runs-list-output> [expected-head] [task-start-epoch]
local wt=$1 branch=$2 list=$3 expected_head=${4:-} task_start=${5:-} start_minute=''
local local_full row_full row st br sha day clock pr extra year_num month_num day_num max_day pending_st=''
local decided=''
local_full=$(git -C "$wt" rev-parse HEAD 2>/dev/null) || return 0
[ -n "$list" ] || return 0
if [ -n "$task_start" ]; then
start_minute=$(fm_nm_epoch_ledger_minute "$task_start") || return 0
[ -n "$start_minute" ] || return 0
fi
while IFS= read -r row; do
row=$(fm_nm_trim "$row")
[ -n "$row" ] || continue
Expand Down Expand Up @@ -486,6 +532,9 @@ fm_nm_runs_status_for_worktree() { # <worktree> <branch> <runs-list-output> [ex
fi
break
fi
if [ -n "$start_minute" ] && [[ "$day $clock" < "$start_minute" ]]; then
Comment thread
greptile-apps[bot] marked this conversation as resolved.
Resolved
break
fi
if [ -n "$expected_head" ]; then
case "$expected_head" in *[!A-Fa-f0-9]*|'') break ;; esac
[ "${#expected_head}" -ge 7 ] && [ "${#expected_head}" -le 40 ] || break
Expand Down
22 changes: 20 additions & 2 deletions bin/fm-spawn.sh
Original file line number Diff line number Diff line change
Expand Up @@ -451,6 +451,10 @@
# success line and state/<id>.meta omit them.
# Every fresh spawn or relaunch records a new spawn_gen= incarnation token so durable
# consumers can distinguish a replacement worker that reuses the same task id.
# A fresh spawn also records task_started=<epoch>, which a relaunch carries forward
# (derived from the prior spawn_gen= when an older record lacks it), so
# bin/fm-crew-state.sh never attributes a validation run created before the task
# started, such as an earlier task's run on a reused PR branch.
# When the home session's frozen trace-context decision is enabled (see
# docs/configuration.md and bin/fm-trace-context-lib.sh), the meta also records
# one W3C traceparent= carrier, the same value injected into the pane as
Expand Down Expand Up @@ -4842,7 +4846,20 @@ fi

META_WINDOW=$T
[ "$BACKEND" = orca ] && META_WINDOW=$W
SPAWN_GEN="s$(date +%s).${BASHPID:-$$}.$RANDOM"
SPAWN_EPOCH=$(date +%s)
SPAWN_GEN="s$SPAWN_EPOCH.${BASHPID:-$$}.$RANDOM"
TASK_STARTED=$SPAWN_EPOCH
if [ "$RELAUNCH" -eq 1 ]; then
RELAUNCH_TASK_STARTED=$(fm_meta_get "$RELAUNCH_META" task_started)
case "$RELAUNCH_TASK_STARTED" in
''|*[!0-9]*)
RELAUNCH_TASK_STARTED=$(fm_meta_get "$RELAUNCH_META" spawn_gen)
RELAUNCH_TASK_STARTED=${RELAUNCH_TASK_STARTED#s}
RELAUNCH_TASK_STARTED=${RELAUNCH_TASK_STARTED%%.*}
;;
esac
case "$RELAUNCH_TASK_STARTED" in ''|*[!0-9]*) ;; *) TASK_STARTED=$RELAUNCH_TASK_STARTED ;; esac
fi
SPAWN_META_PATH="$STATE/$ID.meta"
if [ "$SPAWN_META_LOCK_HELD" != 1 ]; then
SPAWN_META_LOCK=$(fm_meta_lock_path "$STATE/$ID.meta") || exit 1
Expand All @@ -4859,7 +4876,7 @@ SPAWN_META_PATH=$SPAWN_META_TMP
preserve_relaunch_meta() {
awk -F= '
BEGIN {
split("window endpoint_task_id worktree project harness kind mode yolo branch tasktmp model effort account account_provider busy_gen spawn_gen traceparent backend herdr_session herdr_workspace_id herdr_tab_id herdr_pane_id zellij_session zellij_tab_id zellij_pane_id orca_worktree_id terminal cmux_workspace_id cmux_surface_id home projects control_relaunch_tx", keys, " ")
split("window endpoint_task_id worktree project harness kind mode yolo branch tasktmp model effort account account_provider busy_gen spawn_gen task_started traceparent backend herdr_session herdr_workspace_id herdr_tab_id herdr_pane_id zellij_session zellij_tab_id zellij_pane_id orca_worktree_id terminal cmux_workspace_id cmux_surface_id home projects control_relaunch_tx", keys, " ")
for (i in keys) owned[keys[i]] = 1
}
!($1 in owned)
Expand All @@ -4884,6 +4901,7 @@ preserve_relaunch_meta() {
[ -z "$WORKER_ACCOUNT_PROVIDER" ] || echo "account_provider=$WORKER_ACCOUNT_PROVIDER"
[ -z "${BUSY_GEN:-}" ] || echo "busy_gen=$BUSY_GEN"
echo "spawn_gen=$SPAWN_GEN"
echo "task_started=$TASK_STARTED"
# Default-off writes no traceparent= line.
# backend= is written only for a non-default (non-tmux) backend, so the
# default path's meta stays byte-identical (absent backend= means tmux;
Expand Down
8 changes: 5 additions & 3 deletions docs/architecture.md
Original file line number Diff line number Diff line change
Expand Up @@ -123,11 +123,13 @@ The owned-append ledger only decides whether growth wakes this home; it never re
`bin/fm-crew-state.sh <id>` is the cheap current-state read for an actionable heartbeat review: it attributes an active or terminal no-mistakes run under the shared run-attribution contract, then keeps that run-step authoritative even if the pane has closed, except that a `blocked:` event reporting a refused or missing daemon socket outranks a potentially stale active run record only while that socket-down declaration is itself the log's latest recognized event, since any later event, including another `blocked:` one, means the crew moved on.
For other daemon, timeout, or unreachability claims, a running or fixing run with recent pipeline-reported activity supersedes the event and names reattachment as the recovery instead of surfacing a false block.
[`bin/fm-nm-run-lib.sh`](../bin/fm-nm-run-lib.sh) owns branch, head, and pipeline-custody attribution, plus complete same-branch run selection, optional inventory lookup, and ambiguity reporting.
Every route first binds a run to the task's own incarnation: a run created before the task's recorded start is never its result, so a refresh task that reuses an existing PR branch at an unchanged head cannot read the earlier task's finished run as its own.
A run executing on the crew's own branch is current regardless of head, because the pipeline rebases that branch and commits its fix rounds in its own checkout, so reading an older run that still matches the local head would report a working crew as failed; every other run, parked or terminal, still binds on head equality or ancestry, or on the pipeline's own custody attribution while it owns the branch, and that head-free live bind is withdrawn once an explicit `daemon status` probe answers that the daemon is down.
[`tests/fm-crew-state.test.sh`](../tests/fm-crew-state.test.sh) covers run selection; its [capture provenance and live-evidence limits](../tests/captures/no-mistakes-v1.70.1/README.md) distinguish recorded inputs from composed scenarios.
During no-mistakes' `ci` monitor phase, it also reads the full ci step log because `axi status` reports both "still waiting on checks" and "checks green, waiting on merge" as `ci,running`.
The most recent recognized ci log marker wins, so checks-green monitoring reports done while a later failed-check, checks-running, or issue marker returns the crew to working; a base-branch timeout re-arm is not a marker because it leaves readiness unchanged.
`bin/fm-crew-state.sh` owns the evidence guard that recognizes ended CI monitors after green checks, including cancelled runs and skipped rebase steps; a passed run alone never proves a forge merge.
During no-mistakes' `ci` monitor phase, it also reads the full ci step log because `axi status` reports both "still waiting on checks" and "checks passed, waiting on merge" as `ci,running`.
The most recent recognized ci log marker wins, so monitoring whose log reports checks passed reads done while a later failed-check, checks-running, or issue marker returns the crew to working; a log reporting no CI checks at all stays a wait, and a base-branch timeout re-arm is not a marker because it leaves readiness unchanged.
`bin/fm-crew-state.sh` owns the evidence guard that recognizes ended CI monitors after passed checks, including cancelled runs and skipped rebase steps.
A run record states only what that run did and is never presented as green; a merged claim comes only from the task's recorded `pr=` identity checked against the forge or a merge receipt, and a worker status line written after a finished run started is shown beside it with its source.
In the coarse runs-ledger fallback, which has no steps table and no ci log, a terminal failed record whose daemon an explicit `daemon status` probe proves down reports unknown as unverified instead: an instrument failure must never read as work failure.
The same instrument rule covers the ledger-anchored continuation of a selected run whose head this copy cannot resolve: once the probe answers down, that still-executing record reports unknown as unverified, while a run parked at a gate keeps its gate and findings because an open decision stays open when the instrument dies, and a `needs-decision` or `blocked` event the crew observed first hand stays open with the unverified record named as the reason rather than superseded by it.
Only when no matching run exists does it consult semantic busy state; exact busy reports working, exact idle permits fallback to the log's resolved current declaration - the newest decision the fold still holds open, otherwise a declared wait still standing after later resolved lines for other keys, otherwise the latest recognized event - when its verb maps to a recognized run-state, and unknown or a dead pane stays unknown instead of trusting a stale log.
Expand Down
2 changes: 1 addition & 1 deletion docs/gerrit-change-watch.md
Original file line number Diff line number Diff line change
Expand Up @@ -128,6 +128,6 @@ The server permitting self-approval is what makes this a policy boundary rather
- A no-mistakes done naming a Gerrit change is also refused unless the copy holds a passed pipeline's result: refused when the run's outcome is missing or not a pass, while the run reports `recover_custody` or `continue_active_run`, when HEAD's tree differs from the pipeline head's, or when the run cannot be read, and accepted once recovered even after the Change-Id stamp rewrote the branch's messages.
- A `published for review` done whose URL is not a canonical Gerrit change URL is refused, even when a remote-tracking ref holds HEAD.

`tests/fm-crew-state.test.sh` pins the crew-state read with no server either: a passed run whose change is open reports `PR open`, an abandoned one `PR closed`, a merged one `PR merged`, and an unreadable record or one naming another change reports an honest unknown rather than a merge.
`tests/fm-crew-state.test.sh` pins the crew-state read with no server either: a passed run whose recorded change is open reports `recorded PR open`, an abandoned one `recorded PR closed`, a merged one `recorded PR merged`, and an unreadable record or one naming another change reports an honest unknown rather than a merge.

Refresh this record by rerunning those suites, and rerun the transcripts above after a `gerrit-axi` upgrade.
21 changes: 21 additions & 0 deletions tests/fm-control-relaunch.test.sh
Original file line number Diff line number Diff line change
Expand Up @@ -490,6 +490,26 @@ test_relaunch_preserves_durable_task_metadata() {
pass "fm-control relaunch: durable task metadata survives replacement launch publication"
}

# A relaunch continues the same task, so its validation runs stay attributable:
# the task start carries forward, and a record older than task_started= derives
# it from the spawn generation it was first launched with.
test_relaunch_carries_the_task_start() {
local dir out rc
dir=$(new_case task-start rl43)
add_ship_task "$dir" rl43 claude
printf '%s\n' 'spawn_gen=s1700000000.1.1' >> "$dir/home/state/rl43.meta"
out=$(run_control "$dir" rl43 relaunch --note "continuing after a stall"); rc=$?
expect_code 0 "$rc" "a relaunch of a legacy record should succeed"$'\n'"$out"
[ "$(meta_field "$dir" rl43 task_started)" = 1700000000 ] \
|| fail "a legacy record must derive its task start from its prior spawn generation"
[ "$(meta_field "$dir" rl43 spawn_gen)" != s1700000000.1.1 ] || fail "a relaunch must mint a new spawn generation"
out=$(run_control "$dir" rl43 relaunch --note "continuing again"); rc=$?
expect_code 0 "$rc" "a second relaunch should succeed"$'\n'"$out"
[ "$(meta_field "$dir" rl43 task_started)" = 1700000000 ] || fail "the task start must survive every relaunch"
[ "$(grep -c '^task_started=' "$dir/home/state/rl43.meta")" = 1 ] || fail "the task start must be recorded once"
pass "fm-control relaunch: the task start carries across relaunches"
}

test_relaunch_serializes_concurrent_durable_metadata_publication() {
local dir control_pid link_pid rc i=0 traceparent prepare launch_release waiting ready release
dir=$(new_case metadata-race rl28)
Expand Down Expand Up @@ -2391,6 +2411,7 @@ test_relaunch_refuses_before_exit_when_the_composer_holds_pending_text
test_relaunch_refuses_before_exit_when_the_composer_state_is_unproven
test_relaunch_from_linked_home_preserves_recorded_worktree
test_relaunch_preserves_durable_task_metadata
test_relaunch_carries_the_task_start
test_relaunch_serializes_concurrent_durable_metadata_publication
test_disabled_relaunch_clears_prior_trace_context
test_relaunch_appends_the_progress_note_to_the_instructions
Expand Down
Loading