fix(bin): arm the teardown evidence gate only where a destination is configured by mattadams-dev · Pull Request #4 · mattadams-dev/firstmate

mattadams-dev · 2026-07-31T22:16:28Z

Intent

Repair the two teardown tests that are failing on the fork's default branch (tests/fm-gotmp.test.sh 'teardown exited non-zero with a valid tasktmp' and tests/fm-backend.test.sh 'old fm-teardown.sh (scout, report present) should succeed'), so the branch can go green and unblock fork PR #3, which fails only because it inherits this breakage through the shared suite.

Root cause, established by running and bisecting rather than reading: fork PR #2 (the checkpoint-commit discipline and opt-in evidence write-through, which merged with zero CI checks because the fork was diverged) wired bin/fm-evidence.sh into bin/fm-teardown.sh with an UNCONDITIONAL call whose every non-zero result was read as unpreserved evidence. That turned an opt-in, default-off feature into a hard precondition of every cleanup: any failure to RUN the helper - not a failure to preserve anything - refused cleanup, including in a home that never opted in. Both tests build a synthetic bin/ that predates the new sibling and so hit exactly that path.

Deliberate decisions the captain set, which a reviewer seeing only the diff would not know:

Fix the script, not the tests. The two failing assertions encode a contract teardown has always had and passed one commit earlier. Editing an assertion to turn a red suite green was explicitly the one thing this task must not produce. Neither failing assertion was touched.
Keep PR feat(bin): add checkpoint-commit discipline to briefs and an opt-in evidence write-through #2's evidence write-through. This is a regression fix, not a revert. Behavior where the feature IS enabled is unchanged: an unusable destination still refuses cleanup, an unreachable remote still does not block it, and --force still warns and proceeds.
The chosen fix arms the gate on the opt-in itself: 'if [ "$KIND" != secondmate ] && [ -f "$CONFIG/evidence-repo" ]'. This restores what both docs/configuration.md and bin/fm-evidence.sh's own header ALREADY promised - that with the config file absent, teardown behaves exactly as it does without the feature. The documented contract was correct; the wiring violated it. That is why no documentation change accompanies this fix.
The [ -f ] test is deliberately weaker than fm-evidence.sh's own enablement predicate, which additionally requires a non-empty value. That asymmetry is intentional and is explained in the code comment: any file the helper might accept still reaches it, so the helper alone judges whether a configured destination is usable, and teardown never decides that for it. Do not 'tidy' it into a content check.
Teardown's refusal path protects unlanded work, so it is guard-class code under the fleet mutation standard: a protection that is touched or added must be proven by the mutation that breaks exactly its own test. The prior test suite could not tell 'armed but harmless' from 'not armed', because every case ran against a complete bin/ where the helper is always present. The two added cases supply the one condition that separates them - a bin/ missing only that helper - and pin both directions. Each was verified against its own mutation: dropping the config/evidence-repo condition fails only the opted-out case, and adding [ -x ] so a non-runnable helper passes the armed gate fails only the opted-in case.
The added test fixture mirrors bin/ by symlinking the directory rather than listing sibling names, specifically so a future sibling cannot silently weaken it - the same failure mode that produced this regression.

Verified before validation: the two target suites plus tests/fm-teardown-evidence.test.sh and tests/fm-evidence.test.sh all pass; the portable-serial CI lane went from failed=2 to failed=1; bin/fm-lint.sh, bin/fm-test-run.sh --check-coverage, and bin/fm-doc-audience-check.sh are clean. The one remaining local failure is tests/fm-backend-herdr-focus-flash-e2e.test.sh, a real-Herdr live test that CI gate-skips (it accounts for the skipped_gate difference of 12 in CI versus 11 locally). It is unrelated to teardown and out of scope here.

Note for the PR description: the reason this reached the default branch at all - a diverged fork meant PR #2's checks never ran - is being prevented separately by a task filed as fm-fork-freshness-sweep. This task repairs the damage; that one stops the next. The PR description should mention the pairing so a later reader sees both halves rather than assuming this fix alone closed the hole.

What Changed

bin/fm-teardown.sh now runs the evidence write-through only when config/evidence-repo exists ([ "$KIND" != secondmate ] && [ -f "$CONFIG/evidence-repo" ]), instead of calling bin/fm-evidence.sh unconditionally and reading every non-zero result as unpreserved evidence. A home that never opted in is inert here again rather than acquiring a hard dependency on the helper, which is what made tests/fm-gotmp.test.sh and tests/fm-backend.test.sh fail on the default branch against their synthetic bin/ directories. Behavior where the feature is enabled is unchanged: an unusable destination still refuses cleanup, an unreachable remote still does not block it, and --force still warns and proceeds. The [ -f ] test is deliberately weaker than the helper's own non-empty-value predicate so the helper alone judges whether a configured destination is usable; the code comment records why.
tests/fm-teardown-evidence.test.sh gains two cases that pin the arming from both sides using the one condition the prior suite could not express - a bin/ complete except for fm-evidence.sh: an opted-out home cleans up without the helper, and an opted-in home refuses when the helper cannot run. The fixture mirrors bin/ by symlinking each entry rather than listing sibling names, so a future sibling cannot silently weaken it. Each case was verified against the mutation that breaks exactly it: dropping the config condition fails only the opted-out case, and adding [ -x ] to the gate fails only the opted-in case.
The teardown header comment and the gate's inline comment now state the opt-in condition the code enforces. No other documentation changed, because docs/configuration.md and bin/fm-evidence.sh's header already promised this behavior - the wiring, not the contract, was wrong.

The reason this reached the fork's default branch at all is that a diverged fork meant PR #2's checks never ran. This PR repairs the damage; the missing-checks hole itself is being closed separately under fm-fork-freshness-sweep.

Risk Assessment

✅ Low: A two-line, well-bounded regression fix that restores an already-documented opt-in contract, leaves all enabled-feature behavior and both failing assertions untouched, and adds mutation-separated coverage for both directions of the new arming condition.

Testing

Reproduced the red baseline by checking out the base commit and running the two named suites (both target assertions fail with the evidence REFUSED message), then confirmed both go green at the target commit, along with tests/fm-teardown-evidence.test.sh, tests/fm-evidence.test.sh, the runner's own --changed selection for this diff, and tests/fm-teardown-endpoint-safety.test.sh. Beyond the suites I drove bin/fm-teardown.sh directly against real fixture homes to capture an operator CLI transcript of the actual symptom and its repair, and of the opt-in paths (preserve-then-clean, refuse on an unusable destination, --force warn-and-proceed) staying unchanged. I also independently re-ran both guard mutations described in the intent and each one kills exactly its own new test case. No source or test files were left modified; the temporary script swap and mutations were restored and hash-verified. This change is CLI/script-level with no rendered UI surface, so the reviewer-visible evidence is a terminal transcript rather than a screenshot.

Evidence: Operator CLI transcript: regression and repair, plus opt-in paths unchanged


============================================================
1. OPTED-OUT HOME (no config/evidence-repo), bin/ without fm-evidence.sh
   -> BEFORE the fix (bin/fm-teardown.sh at a5044fd)
============================================================
$ fm-teardown.sh demo1
/tmp/no-mistakes-evidence/01KYX2VJD3RRQC5FZQQCBCRHSF/manual-run/optout-before-bin/fm-teardown.sh: line 1376: /tmp/no-mistakes-evidence/01KYX2VJD3RRQC5FZQQCBCRHSF/manual-run/optout-before-bin/fm-evidence.sh: No such file or directory
REFUSED: could not preserve task demo1's evidence before cleanup.
Cleanup destroys the worktree, so fix the evidence destination or clear config/evidence-repo first.
exit=1
  on disk: task state STILL PRESENT + worktree untouched -> cleanup refused

============================================================
2. SAME OPTED-OUT HOME -> AFTER the fix (bin/fm-teardown.sh at 40c60ca)
============================================================
$ fm-teardown.sh demo2
teardown demo2 complete (window firstmate:fm-demo2, worktree /tmp/no-mistakes-evidence/01KYX2VJD3RRQC5FZQQCBCRHSF/manual-run/optout-after/worktree)
Backlog: demo2 just finished. Run tasks-axi done demo2 --report data/demo2/report.md, then run tasks-axi ready for dependency-cleared candidates, check date gates, and dispatch only work whose blockers are gone and date is due.
exit=0
  on disk: task state cleared -> cleanup completed (worktree handed to treehouse return)

============================================================
3. OPTED-IN HOME (config/evidence-repo set), same bin/ without the helper
   -> AFTER the fix: the guard must still refuse
============================================================
$ cat config/evidence-repo
/tmp/no-mistakes-evidence/01KYX2VJD3RRQC5FZQQCBCRHSF/manual-run/optin-nohelper/evidence
$ fm-teardown.sh demo3
/tmp/no-mistakes-evidence/01KYX2VJD3RRQC5FZQQCBCRHSF/manual-run/optin-nohelper-bin/fm-teardown.sh: line 1386: /tmp/no-mistakes-evidence/01KYX2VJD3RRQC5FZQQCBCRHSF/manual-run/optin-nohelper-bin/fm-evidence.sh: No such file or directory
REFUSED: could not preserve task demo3's evidence before cleanup.
Cleanup destroys the worktree, so fix the evidence destination or clear config/evidence-repo first.
exit=1
  on disk: task state STILL PRESENT + worktree untouched -> cleanup refused

============================================================
4. OPTED-IN HOME with a usable destination and a complete bin/
   -> AFTER the fix: evidence is preserved AND cleanup completes
============================================================
$ fm-teardown.sh demo4
fm-evidence: committed 2 artifact(s) for demo4 in /tmp/no-mistakes-evidence/01KYX2VJD3RRQC5FZQQCBCRHSF/manual-run/optin-working/evidence
fm-evidence: evidence for demo4 is committed locally; skipped the push because the destination's visibility could not be verified
teardown demo4 complete (window firstmate:fm-demo4, worktree /tmp/no-mistakes-evidence/01KYX2VJD3RRQC5FZQQCBCRHSF/manual-run/optin-working/worktree)
Backlog: demo4 just finished. Run tasks-axi done demo4 --report data/demo4/report.md, then run tasks-axi ready for dependency-cleared candidates, check date gates, and dispatch only work whose blockers are gone and date is due.
exit=0
  on disk: task state cleared -> cleanup completed (worktree handed to treehouse return)
  evidence repo commits:
    1091466 evidence(firstmate-6ed6bab2/demo4): preserve 2 artifact(s)
  evidence repo files:
    firstmate-6ed6bab2/demo4/EVIDENCE-MANIFEST.txt
    firstmate-6ed6bab2/demo4/raw-measurement.txt
    firstmate-6ed6bab2/demo4/report.md

============================================================
5. OPTED-IN HOME whose destination cannot be written (unusable path)
   -> AFTER the fix: still refuses, worktree kept
============================================================
$ fm-teardown.sh demo5
fm-evidence: config/evidence-repo names no directory: /tmp/no-mistakes-evidence/01KYX2VJD3RRQC5FZQQCBCRHSF/manual-run/optin-unusable/not-a-repo
REFUSED: could not preserve task demo5's evidence before cleanup.
Cleanup destroys the worktree, so fix the evidence destination or clear config/evidence-repo first.
exit=1
  on disk: task state STILL PRESENT + worktree untouched -> cleanup refused

============================================================
6. SAME UNUSABLE DESTINATION, but --force
   -> AFTER the fix: warns and proceeds
============================================================
$ fm-teardown.sh demo6 --force
fm-evidence: config/evidence-repo names no directory: /tmp/no-mistakes-evidence/01KYX2VJD3RRQC5FZQQCBCRHSF/manual-run/optin-force/not-a-repo
WARNING: could not preserve task demo6's evidence; continuing because --force authorizes discard.
teardown demo6 complete (window firstmate:fm-demo6, worktree /tmp/no-mistakes-evidence/01KYX2VJD3RRQC5FZQQCBCRHSF/manual-run/optin-force/worktree)
Backlog: demo6 just finished. Run tasks-axi done demo6 --report data/demo6/report.md, then run tasks-axi ready for dependency-cleared candidates, check date gates, and dispatch only work whose blockers are gone and date is due.
exit=0
  on disk: task state cleared -> cleanup completed (worktree handed to treehouse return)

Evidence: Before/after excerpt (opted-out home, bin/ without fm-evidence.sh)

1. OPTED-OUT HOME (no config/evidence-repo), bin/ without fm-evidence.sh
-> BEFORE the fix (bin/fm-teardown.sh at a5044fd)
$ fm-teardown.sh demo1
.../optout-before-bin/fm-teardown.sh: line 1376: .../optout-before-bin/fm-evidence.sh: No such file or directory
REFUSED: could not preserve task demo1's evidence before cleanup.
Cleanup destroys the worktree, so fix the evidence destination or clear config/evidence-repo first.
exit=1
on disk: task state STILL PRESENT + worktree untouched -> cleanup refused

2. SAME OPTED-OUT HOME -> AFTER the fix (bin/fm-teardown.sh at 40c60ca)
$ fm-teardown.sh demo2
teardown demo2 complete (window firstmate:fm-demo2, worktree .../optout-after/worktree)
exit=0
on disk: task state cleared -> cleanup completed (worktree handed to treehouse return)

3. OPTED-IN HOME (config/evidence-repo set), same bin/ without the helper
-> AFTER the fix: the guard must still refuse
REFUSED: could not preserve task demo3's evidence before cleanup.
exit=1

4. OPTED-IN HOME with a usable destination and a complete bin/
fm-evidence: committed 2 artifact(s) for demo4 in .../optin-working/evidence
teardown demo4 complete
exit=0
evidence repo commits: a825ab6 evidence(firstmate-6ed6bab2/demo4): preserve 2 artifact(s)

Evidence: Baseline red at base commit a5044fd (both named assertions)

FM_TEST_BEGIN 2026-07-31T22:02:53Z tests/fm-gotmp.test.sh family=session-bootstrap expected_gate_skip=none
not ok - teardown exited non-zero with a valid tasktmp
FM_TEST_END 2026-07-31T22:02:53Z tests/fm-gotmp.test.sh exit=1 duration_ms=54 gate_skip=false
FM_TEST_BEGIN 2026-07-31T22:02:53Z tests/fm-backend.test.sh family=backend-dispatch expected_gate_skip=none
ok - fm_backend_name: FM_BACKEND env > config/backend > default tmux
ok - fm_backend_detect: no markers -> undetected, HERDR_ENV=1 -> herdr, $TMUX -> tmux, CMUX_WORKSPACE_ID -> cmux, nested combinations resolve innermost-first
ok - fm_backend_detect: falls back to __CFBundleIdentifier=com.cmuxterm.app when CMUX_WORKSPACE_ID is absent (signal bundle-id; foreign bundle ids rejected)
ok - fm_backend_detect: the cmux fallback signals are macOS-only (inert on a non-Darwin uname)
ok - fm_backend_detect: an inherited cmux bundle id never outranks $TMUX or HERDR_ENV (tmux/herdr-inside-cmux false positive absorbed)
ok - fm_backend_detect: ancestry fallback matches the lsappinfo-resolved (bundle-id) cmux app pid in the parent chain
ok - fm_backend_detect: ancestry fallback matches a bundle-shaped cmux comm path at any install location when lsappinfo cannot resolve a pid
ok - fm_backend_detect: ancestry fallback stops undetected at launchd (a reparented tmux server never reaches cmux)
ok - fm_backend_name: a fallback-detected cmux prints a NOTICE naming the fallback signal; the primary-marker notice is unchanged
ok - fm_backend_name: auto-detect selects herdr or cmux (loud notice) or tmux (silent, including nested tmux-in-herdr/tmux-in-cmux)
ok - fm_backend_name: an explicit FM_BACKEND or config/backend setting always wins over runtime auto-detection, including an ambient cmux marker
ok - fm_backend_validate: implemented adapters accepted, unknown and blocked codex-app backends refused loudly
ok - zsh: fm_backend_source recognizes known backends and rejects unknown ones
ok - bash: fm_backend_source recognizes known backends and rejects unknown ones
ok - fm_backend_validate_spawn: all implemented lifecycle backends are spawn-supported
ok - fm_meta_get / fm_backend_of_meta: read key=value, default backend to tmux
ok - fm_backend_resolve_selector: session:window literal, exact task id first, legacy fm-<id> label fallback, ad hoc bare name via tmux list-windows
ok - fm_backend_of_selector: exact task ids, legacy fm-<id> labels, and matching explicit targets inherit metadata backend
ok - fm-send.sh: explicit tmux targets are verified, while --key/plain/slash send command shape stays old-compatible
ok - fm-peek.sh: capture-pane invocation and output are byte-identical old vs new
ok - fm-spawn.sh: a project reached through a symlinked prefix (e.g. macOS /tmp -> /private/tmp) does not trip the isolation guard's false refusal
not ok - old fm-teardown.sh (scout, report present) should succeed
/tmp/fm-backend-tests.WnXki3/teardown-old/bin/fm-teardown.sh: line 1376: /tmp/fm-backend-tests.WnXki3/teardown-old/bin/fm-evidence.sh: No such file or directory
REFUSED: could not preserve task teardownconform1's evidence before cleanup.
Cleanup destroys the worktree, so fix the evidence destination or clear config/evidence-repo first.: expected exit 0, got 1
FM_TEST_END 2026-07-31T22:03:03Z tests/fm-backend.test.sh exit=1 duration_ms=9749 gate_skip=false
FM_TEST_SUMMARY total=2 failed=2 skipped_gate=0 duration_ms=9832
FM_TEST_SUMMARY_FAMILY family=backend-dispatch count=1 duration_ms=9749 failed=1
FM_TEST_SUMMARY_FAMILY family=session-bootstrap count=1 duration_ms=54 failed=1
FM_TEST_SLOWEST rank=1 script=tests/fm-backend.test.sh duration_ms=9749
FM_TEST_SLOWEST rank=2 script=tests/fm-gotmp.test.sh duration_ms=54

Evidence: Both suites green at target commit 40c60ca

FM_TEST_BEGIN 2026-07-31T22:01:35Z tests/fm-gotmp.test.sh family=session-bootstrap expected_gate_skip=none
ok - fm-teardown removes the dir pointed to by tasktmp= in meta
ok - fm-teardown skips gracefully when tasktmp= is absent (backward compat)
ok - fm-teardown skips gracefully when tasktmp= points to a nonexistent dir
FM_TEST_END 2026-07-31T22:01:35Z tests/fm-gotmp.test.sh exit=0 duration_ms=175 gate_skip=false
FM_TEST_BEGIN 2026-07-31T22:01:35Z tests/fm-backend.test.sh family=backend-dispatch expected_gate_skip=none
ok - fm_backend_name: FM_BACKEND env > config/backend > default tmux
ok - fm_backend_detect: no markers -> undetected, HERDR_ENV=1 -> herdr, $TMUX -> tmux, CMUX_WORKSPACE_ID -> cmux, nested combinations resolve innermost-first
ok - fm_backend_detect: falls back to __CFBundleIdentifier=com.cmuxterm.app when CMUX_WORKSPACE_ID is absent (signal bundle-id; foreign bundle ids rejected)
ok - fm_backend_detect: the cmux fallback signals are macOS-only (inert on a non-Darwin uname)
ok - fm_backend_detect: an inherited cmux bundle id never outranks $TMUX or HERDR_ENV (tmux/herdr-inside-cmux false positive absorbed)
ok - fm_backend_detect: ancestry fallback matches the lsappinfo-resolved (bundle-id) cmux app pid in the parent chain
ok - fm_backend_detect: ancestry fallback matches a bundle-shaped cmux comm path at any install location when lsappinfo cannot resolve a pid
ok - fm_backend_detect: ancestry fallback stops undetected at launchd (a reparented tmux server never reaches cmux)
ok - fm_backend_name: a fallback-detected cmux prints a NOTICE naming the fallback signal; the primary-marker notice is unchanged
ok - fm_backend_name: auto-detect selects herdr or cmux (loud notice) or tmux (silent, including nested tmux-in-herdr/tmux-in-cmux)
ok - fm_backend_name: an explicit FM_BACKEND or config/backend setting always wins over runtime auto-detection, including an ambient cmux marker
ok - fm_backend_validate: implemented adapters accepted, unknown and blocked codex-app backends refused loudly
ok - zsh: fm_backend_source recognizes known backends and rejects unknown ones
ok - bash: fm_backend_source recognizes known backends and rejects unknown ones
ok - fm_backend_validate_spawn: all implemented lifecycle backends are spawn-supported
ok - fm_meta_get / fm_backend_of_meta: read key=value, default backend to tmux
ok - fm_backend_resolve_selector: session:window literal, exact task id first, legacy fm-<id> label fallback, ad hoc bare name via tmux list-windows
ok - fm_backend_of_selector: exact task ids, legacy fm-<id> labels, and matching explicit targets inherit metadata backend
ok - fm-send.sh: explicit tmux targets are verified, while --key/plain/slash send command shape stays old-compatible
ok - fm-peek.sh: capture-pane invocation and output are byte-identical old vs new
ok - fm-spawn.sh: a project reached through a symlinked prefix (e.g. macOS /tmp -> /private/tmp) does not trip the isolation guard's false refusal
ok - fm-teardown.sh: treehouse return remains compatible while tmux cleanup uses exact selectors
ok - fm-spawn.sh --backend bogus is refused loudly
ok - fm-spawn.sh --backend codex-app is refused
ok - fm-spawn.sh honors FM_BACKEND and refuses an unimplemented value loudly
ok - fm-spawn.sh: an explicit --backend tmux resolves silently and writes no backend= (missing means tmux)
ok - fm-spawn.sh: explicit --backend tmux wins over an ambient HERDR_ENV=1 auto-detect marker
ok - fm-spawn.sh: auto-detect resolves nested tmux-in-herdr to tmux and stays silent end to end
FM_TEST_END 2026-07-31T22:01:50Z tests/fm-backend.test.sh exit=0 duration_ms=15177 gate_skip=false
FM_TEST_SUMMARY total=2 failed=0 skipped_gate=0 duration_ms=15384
FM_TEST_SUMMARY_FAMILY family=backend-dispatch count=1 duration_ms=15177 failed=0
FM_TEST_SUMMARY_FAMILY family=session-bootstrap count=1 duration_ms=175 failed=0
FM_TEST_SLOWEST rank=1 script=tests/fm-backend.test.sh duration_ms=15177
FM_TEST_SLOWEST rank=2 script=tests/fm-gotmp.test.sh duration_ms=175

Evidence: Guard-class mutation proof: each mutation kills exactly its own case

### MUTATION: A - drop the config/evidence-repo condition (the regression) not ok - an opted-out home was blocked by the evidence gate ### MUTATION: B - add [ -x ] so a non-runnable helper passes the armed gate ok - fm-teardown: an opted-out home cleans up even when the evidence helper is absent not ok - cleanup proceeded with a configured destination it could not write to

### MUTATION: A - drop the config/evidence-repo condition (unconditional call, i.e. the regression)
### gate is now: if [ "$KIND" != secondmate ]; then
ok - fm-teardown: unpreservable evidence refuses cleanup and keeps the worktree
ok - fm-teardown: an unreachable remote preserves evidence and still cleans up
ok - fm-teardown: a home without an evidence destination cleans up unchanged
not ok - an opted-out home was blocked by the evidence gate
### suite exit above; restoring

### MUTATION: B - add [ -x ] so a non-runnable helper passes the armed gate
### gate is now: if [ "$KIND" != secondmate ] && [ -f "$CONFIG/evidence-repo" ] && [ -x "$SCRIPT_DIR/fm-evidence.sh" ]; then
ok - fm-teardown: unpreservable evidence refuses cleanup and keeps the worktree
ok - fm-teardown: an unreachable remote preserves evidence and still cleans up
ok - fm-teardown: a home without an evidence destination cleans up unchanged
ok - fm-teardown: an opted-out home cleans up even when the evidence helper is absent
not ok - cleanup proceeded with a configured destination it could not write to
### suite exit above; restoring

Evidence: Reproducible manual demo script

#!/usr/bin/env bash
# Manual, operator-level demonstration of the teardown evidence-gate regression
# and its repair. Builds real Firstmate homes, invokes bin/fm-teardown.sh the way
# an operator does, and prints what the operator sees plus what survives on disk.
#
# usage: manual-teardown-demo.sh <repo-root> <workdir>
set -u

REPO=$1
WORK=$2
rm -rf "$WORK"
mkdir -p "$WORK"

# shellcheck source=/dev/null
. "$REPO/tests/lib.sh"
fm_git_identity

BASE_TEARDOWN="$WORK/pre-fix/fm-teardown.sh"   # a5044fd (fork default branch)
mkdir -p "$WORK/pre-fix"
git -C "$REPO" show a5044fd:bin/fm-teardown.sh > "$BASE_TEARDOWN"
chmod +x "$BASE_TEARDOWN"

TASKS_AXI_BIN=$(command -v tasks-axi || true)
[ -n "$TASKS_AXI_BIN" ] || { echo "tasks-axi not on PATH; cannot build a realistic home"; exit 2; }

# A completed scout task, past every gate that precedes the evidence gate.
make_home() {  # <name> <task-id>
  local dir="$WORK/$1" id=$2 fakebin
  mkdir -p "$dir/home/state" "$dir/home/data/$id" "$dir/home/config" \
    "$dir/worktree" "$dir/project" "$dir/evidence"
  cp "$REPO/.tasks.toml" "$dir/home/.tasks.toml"
  : > "$dir/worktree/sentinel"
  printf 'findings\n' > "$dir/home/data/$id/report.md"
  printf 'measurement\n' > "$dir/home/data/$id/raw-measurement.txt"
  git -C "$dir/evidence" init -q
  git -C "$dir/evidence" config user.name 'Firstmate Demo'
  git -C "$dir/evidence" config user.email 'demo@example.invalid'
  fakebin=$(fm_fakebin "$dir/home")
  fm_fake_exit0 "$fakebin" tmux treehouse no-mistakes gh gh-axi
  cat > "$dir/home/data/backlog.md" <<EOF
## In flight
- [ ] $id - Preserve sample measurements (repo: sample) (kind: scout) (since 2026-07-31)

## Queued

## Done
EOF
  fm_write_meta "$dir/home/state/$id.meta" \
    "window=firstmate:fm-$id" "endpoint_task_id=$id" \
    "worktree=$dir/worktree" "project=$dir/project" \
    "harness=codex" "kind=scout" "mode=scout"
  printf 'done: measurements captured\n' > "$dir/home/state/$id.status"
  PATH="$fakebin:$PATH" REAL_TASKS_AXI="$TASKS_AXI_BIN" FM_HOME="$dir/home" \
    FM_STATE_OVERRIDE="$dir/home/state" FM_DATA_OVERRIDE="$dir/home/data" \
    FM_CONFIG_OVERRIDE="$dir/home/config" \
    "$REPO/bin/fm-decision-hold.sh" complete "$id" --none >/dev/null \
    || { echo "fixture could not pass the completion gate"; exit 2; }
  printf '%s\n' "$dir"
}

# An installed bin/ from before bin/fm-evidence.sh existed as a sibling: every
# other script is the real one, only the new helper is missing.
make_bin_without_evidence() {  # <name>
  local shim="$WORK/$1-bin" entry
  mkdir -p "$shim"
  for entry in "$REPO"/bin/*; do ln -s "$entry" "$shim/$(basename "$entry")"; done
  rm -f "$shim/fm-evidence.sh"
  printf '%s\n' "$shim"
}

run_teardown() {  # <teardown-path> <case-dir> <task-id>
  local script=$1 dir=$2 id=$3
  PATH="$dir/home/fakebin:$PATH" FM_ROOT_OVERRIDE="$REPO" FM_HOME="$dir/home" \
    FM_STATE_OVERRIDE="$dir/home/state" FM_DATA_OVERRIDE="$dir/home/data" \
    FM_CONFIG_OVERRIDE="$dir/home/config" "$script" "$id" 2>&1
}

# Worktree removal is delegated to `treehouse return`, which is a stubbed no-op
# in these homes, so the sentinel surviving a SUCCESSFUL run is fixture, not
# product. The load-bearing on-disk signal is the task state: teardown clears it
# only when cleanup actually ran to completion.
survey() {  # <case-dir> <task-id>
  local dir=$1 id=$2
  if [ -e "$dir/home/state/$id.meta" ]; then
    echo "  on disk: task state STILL PRESENT + worktree untouched -> cleanup refused"
  else
    echo "  on disk: task state cleared -> cleanup completed (worktree handed to treehouse return)"
  fi
}

banner() { printf '\n============================================================\n%s\n============================================================\n' "$1"; }

banner "1. OPTED-OUT HOME (no config/evidence-repo), bin/ without fm-evidence.sh
   -> BEFORE the fix (bin/fm-teardown.sh at a5044fd)"
dir=$(make_home optout-before demo1)
shim=$(make_bin_without_evidence optout-before)
rm -f "$shim/fm-teardown.sh"   # drop the symlink first: never write through it into the repo
cp "$BASE_TEARDOWN" "$shim/fm-teardown.sh"
chmod +x "$shim/fm-teardown.sh"
echo "\$ fm-teardown.sh demo1"
run_teardown "$shim/fm-teardown.sh" "$dir" demo1; echo "exit=$?"
survey "$dir" demo1

banner "2. SAME OPTED-OUT HOME -> AFTER the fix (bin/fm-teardown.sh at 40c60ca)"
dir=$(make_home optout-after demo2)
shim=$(make_bin_without_evidence optout-after)
echo "\$ fm-teardown.sh demo2"
run_teardown "$shim/fm-teardown.sh" "$dir" demo2; echo "exit=$?"
survey "$dir" demo2

banner "3. OPTED-IN HOME (config/evidence-repo set), same bin/ without the helper
   -> AFTER the fix: the guard must still refuse"
dir=$(make_home optin-nohelper demo3)
shim=$(make_bin_without_evidence optin-nohelper)
printf '%s\n' "$dir/evidence" > "$dir/home/config/evidence-repo"
echo "\$ cat config/evidence-repo"; cat "$dir/home/config/evidence-repo"
echo "\$ fm-teardown.sh demo3"
run_teardown "$shim/fm-teardown.sh" "$dir" demo3; echo "exit=$?"
survey "$dir" demo3

banner "4. OPTED-IN HOME with a usable destination and a complete bin/
   -> AFTER the fix: evidence is preserved AND cleanup completes"
dir=$(make_home optin-working demo4)
printf '%s\n' "$dir/evidence" > "$dir/home/config/evidence-repo"
echo "\$ fm-teardown.sh demo4"
run_teardown "$REPO/bin/fm-teardown.sh" "$dir" demo4; echo "exit=$?"
survey "$dir" demo4
echo "  evidence repo commits:"
git -C "$dir/evidence" log --oneline | sed 's/^/    /'
echo "  evidence repo files:"
git -C "$dir/evidence" ls-files | sed 's/^/    /'

banner "5. OPTED-IN HOME whose destination cannot be written (unusable path)
   -> AFTER the fix: still refuses, worktree kept"
dir=$(make_home optin-unusable demo5)
printf '%s\n' "$dir/not-a-repo" > "$dir/home/config/evidence-repo"
echo "\$ fm-teardown.sh demo5"
run_teardown "$REPO/bin/fm-teardown.sh" "$dir" demo5; echo "exit=$?"
survey "$dir" demo5

banner "6. SAME UNUSABLE DESTINATION, but --force
   -> AFTER the fix: warns and proceeds"
dir=$(make_home optin-force demo6)
printf '%s\n' "$dir/not-a-repo" > "$dir/home/config/evidence-repo"
echo "\$ fm-teardown.sh demo6 --force"
PATH="$dir/home/fakebin:$PATH" FM_ROOT_OVERRIDE="$REPO" FM_HOME="$dir/home" \
  FM_STATE_OVERRIDE="$dir/home/state" FM_DATA_OVERRIDE="$dir/home/data" \
  FM_CONFIG_OVERRIDE="$dir/home/config" "$REPO/bin/fm-teardown.sh" demo6 --force 2>&1
echo "exit=$?"
survey "$dir" demo6

- Evidence: Runner's changed-file selection for this diff (6/6 pass) (local file: /tmp/no-mistakes-evidence/01KYX2VJD3RRQC5FZQQCBCRHSF/changed-selection.log)

Pipeline

Updates from git push no-mistakes

✅ **intent** - passed

✅ No issues found.

✅ **Rebase** - passed

✅ No issues found.

✅ **Review** - passed

✅ No issues found.

✅ **Test** - passed

✅ No issues found.

git checkout a5044fd && bin/fm-test-run.sh tests/fm-gotmp.test.sh tests/fm-backend.test.sh - baseline red, reproduces both named assertions failing, then restored HEAD to 40c60ca
bin/fm-test-run.sh tests/fm-gotmp.test.sh tests/fm-backend.test.sh at the target commit - both suites green
bin/fm-test-run.sh tests/fm-teardown-evidence.test.sh tests/fm-evidence.test.sh - all 6 + 20 cases pass, including the two added arming cases
bin/fm-test-run.sh --changed --base a5044fd - the runner's own changed-file selection (fm-pr-check-security, fm-pr-merge, fm-review-diff, fm-teardown, fm-x-mode, fm-teardown-evidence), 6/6 pass
bin/fm-test-run.sh tests/fm-teardown-endpoint-safety.test.sh - passes
Manual operator transcript: built real Firstmate homes past the completion gate and invoked bin/fm-teardown.sh <id> directly for six scenarios (opted-out home without the helper before/after the fix, opted-in home without the helper, opted-in home with a working evidence repo, opted-in home with an unusable destination, and the same with --force)
Mutation proof A: replaced the gate with if [ "$KIND" != secondmate ]; then and ran tests/fm-teardown-evidence.test.sh - fails only an opted-out home was blocked by the evidence gate
Mutation proof B: added && [ -x "$SCRIPT_DIR/fm-evidence.sh" ] to the gate and ran tests/fm-teardown-evidence.test.sh - opted-out case passes, fails only cleanup proceeded with a configured destination it could not write to
git status --porcelain and git hash-object bin/fm-teardown.sh after each temporary swap/mutation - worktree restored clean at 40c60ca

✅ **Document** - passed

✅ No issues found.

✅ **Lint** - passed

✅ No issues found.

✅ **Push** - passed

✅ No issues found.

…configured Cleanup called bin/fm-evidence.sh unconditionally and treated every non-zero result as unpreserved evidence. The write-through is opt-in and off by default, so that made an inert feature a hard precondition of every teardown: any failure to RUN the helper - not a failure to preserve anything - refused cleanup even in a home that never opted in. Gate the block on config/evidence-repo so an opted-out home is inert rather than merely quiet, which is what bin/fm-evidence.sh's own contract already promises. The [ -f ] test is deliberately weaker than the helper's enablement predicate, which additionally requires a non-empty value, so every file the helper might accept still reaches it and the helper alone judges a configured destination. Refusal behavior where the feature IS on is unchanged: an unusable destination still stops cleanup, an unreachable remote still does not, and --force still warns and proceeds.

The regression that reached main was an over-armed gate: an opted-out home inherited a hard dependency on bin/fm-evidence.sh. No test caught it, because every case ran against a complete bin/ where the helper is always present, so "armed but harmless" and "not armed" were indistinguishable. Add the one condition that separates them - a bin/ missing only that helper - and assert both directions through it: an opted-out home still cleans up, and an opted-in home refuses because unknown custody is not absent custody. Each case is proven by the mutation that breaks exactly it: dropping the config/evidence-repo condition fails only the opted-out case, and letting a non-runnable helper pass the armed gate fails only the opted-in one.

mattadams-dev added 3 commits July 31, 2026 14:15

no-mistakes(document): align teardown header with armed evidence gate

f3566d3

mattadams-dev merged commit 25faf40 into main Jul 31, 2026
10 checks passed

Navigation Menu

fix(bin): arm the teardown evidence gate only where a destination is configured - #4

fix(bin): arm the teardown evidence gate only where a destination is configured#4
mattadams-dev merged 3 commits into
mainfrom
fm/fm-teardown-regression-main-red

mattadams-dev commented Jul 31, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Conversation

mattadams-dev commented Jul 31, 2026

Intent

What Changed

Risk Assessment

Testing

Pipeline

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant