Skip to content

feat: require a before/after evidence pair in the ship definition of done - #4985

Open
keenvc wants to merge 1 commit into
kunchenguid:mainfrom
keenvc:fm/fm-evidence-before-state
Open

keenvc wants to merge 1 commit into
kunchenguid:mainfrom
keenvc:fm/fm-evidence-before-state

Conversation

@keenvc

@keenvc keenvc commented Sep 20, 2026

Copy link
Copy Markdown

What

The ship definition of done now requires a before/after evidence pair captured with one stated methodology, and the before is captured at reproduction time, before any fix, for any change with an observable surface.

The requirement is stated once, in its single owner bin/fm-dod-lib.sh, and rendered into every ship mode's Definition of done (no-mistakes, direct-PR, local-only), so every worker receives it in its brief and every promoted scout receives the same contract.

  • bin/fm-dod-lib.sh - owner: new fm_dod_evidence_pair block, emitted into all modes after the Delivery contract: line; header documents the new ownership; the mode-specific heredocs no longer duplicate the # Definition of done / Delivery contract: header lines.
  • AGENTS.md (section 7, Validate) - one-line cross-reference to the owner.
  • docs/architecture.md - the maintainer-architecture owner sentence now names the evidence-pair scope.

A change with no visible surface uses the same discipline with measured numbers or before/after output instead of screenshots; when nothing observable can move, the worker says so and names what was verified instead.

Why

An after-only check cannot catch a measurement that is wrong in both directions: the same wrong ruler applied twice shows no movement. A before/after pair taken with one stated methodology can, because a broken measurement becomes visible in the delta instead of passing in both directions.

This is the gap behind recent fleet failures where an after-check passed while the defect was still on screen - a notice that painted in the DOM but never composited, rows counted in the viewport instead of inside the grid container, and fixes that passed their own checks while the screen stayed broken.

Scope decisions

  • No transport adopted. The external skill this is learned from uploads evidence to a public paste host. This change keeps the fleet's own private evidence path and states plainly that evidence is never uploaded to a public host (phi-safe-evidence is the fleet's instance of that path). No skill name or host is hardcoded into firstmate's generic contract.
  • No enforcement code added. The requirement is instruction, owned by the DoD. A mechanical "does a before artifact exist" gate would need a project-specific artifact convention and path, which the direct path does not justify; the definition of done plus the existing browser-QA check is the lever.
  • No deterministic suite-walk was added to any no-mistakes config.

Evidence

This is a documentation/process change with no browser surface, so per the rule it states what was verified instead:

  • tests/fm-brief.test.sh - new test_ship_dod_requires_evidence_pair scaffolds all three ship modes and asserts the rendered DoD carries the requirement (one methodology, before-at-reproduction, non-UI measured/output pairs, never a public host). Full file: 26 ok, exit 0.
  • bin/fm-lint.sh - exit 0 (ShellCheck 0.11.0).
  • bin/fm-doc-audience-check.sh - ok surfaces=102 local_links=426.
  • bin/fm-ensure-agents-md.sh . - unchanged.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant