Skip to content

feat(bin): add maintenance agent grading loop - #1

Merged
mjskinner82 merged 14 commits into
mainfrom
fm/agent-grading-loop-build
Aug 7, 2026
Merged

mjskinner82 merged 14 commits into
mainfrom
fm/agent-grading-loop-build

Conversation

@mjskinner82

Copy link
Copy Markdown
Owner

Summary

  • Add an append-only private outcome ledger and a validated record helper for recurring maintenance agents.
  • Add a readable per-agent report scoreboard for merge, changed-on-merge, rejection, and false-positive rates.
  • Document the captain-reviewed brief refinement loop and explicitly prohibit unsupervised agent self-rewriting.
  • Add focused record and report verification, including duplicate-record and concurrent-write coverage.

Validation

  • No-mistakes run 01KZDNFJ6473ZCEAPBTGG4T0RB passed Review, Test, Document, and Lint at validated head 7b9394ad0f6183cc4638cac746355bd9e2ae799a.
  • Publication alone failed because the run targeted kunchenguid/firstmate, where mjskinner82 received HTTP 403.
  • Recovery preserved the validated head, which was then published unchanged to mjskinner82/firstmate.

Separated pre-existing failure

tests/fm-calm-pi-extension.test.sh fails because /export does not complete while Calm mode is on.
The same test blob fails identically with exit 1 on unchanged origin/main at 345de4e7aad1484f46bc5115c9e57640a0b060d9.
This PR does not change or attempt to repair that unrelated failure.

Base note

The fork's main was 11 commits behind kunchenguid/firstmate when the validated head was recovered.
This PR therefore also carries those already-upstream commits so that the exact no-mistakes-validated head can be reviewed on the fork.

Pipeline

Updates from git push no-mistakes

  • Run: 01KZDNFJ6473ZCEAPBTGG4T0RB
  • Validated head: 7b9394ad0f6183cc4638cac746355bd9e2ae799a

kunchenguid and others added 14 commits August 6, 2026 15:49
…kunchenguid#1842)

A captain decision opened by a keyed needs-decision:/blocked: status line
orphaned as permanently open whenever the answer kicked off work: the
worker's next event is working [key=<workstream>] in a different key
namespace, so no resolved [key=<decision>] ever landed and the OPEN
DECISIONS fold kept listing the answered decision forever.

Remove the writer-dependency at its source: the answering firstmate
already holds the decision key when it sends the answer, so fm-send's new
--resolve-key flag (repeatable) appends the closing resolved line to this
home's own state/<id>.status after the submit is confirmed. The close is
a local ledger append for crewmates, local secondmates, and remote
secondmates alike - a remote mate's escalations reach this ledger through
the parent-replies ingest, so only the answer message crosses the
transport.

Safety: each named key must currently be open per the authoritative
status_open_decisions fold or fm-send refuses before sending; a failed or
unconfirmed send never closes a key; an append failure after a delivered
answer exits nonzero with the manual close command so the decision
re-surfaces instead of silently vanishing; a send without the flag closes
nothing, and working:/done: still never clear a captain decision.

Complementary fixes: the wake-drain OPEN DECISIONS section prints the
answer-with-close command hint at the moment of use; brief scaffolds
separate resolved's two duties (keyed-phase end vs decision closure) and
state that a done:/working: line never closes a decision even when the
answer started that work, keeping worker self-close for blockers that
clear without a firstmate reply; AGENTS.md and docs/architecture.md carry
the one-line pointers to the fm-send contract.
…1836)

* feat(secondmate): seed a remote home from a supplied project origin

Remote seeding required a local projects/<name> clone purely to read
`git remote get-url origin` into the provisioning manifest, so setting up
a remote second mate forced disposable clones and no-mistakes inits in the
primary home for projects that home has no reason to hold.

Firstmate now resolves the origin itself and names it as <project>=<origin-url>.
The seed validates and transports what it is given, and the receiving host
re-validates it rather than trusting the sender; bin/fm-project-origin-lib.sh
is the single owner of which URLs are accepted, refusing executable remote-helper
transports, option-shaped values, and unusable spellings at both ends. A bare
<project> still reads an already-present clone's origin, so nothing that works
today has to change. Registry consistency is unchanged: an unregistered or
local-only project is still refused.

A remote seed therefore creates nothing in the primary home beyond the route,
the charter, and its launch record.

The lifecycle test now seeds a registered project the primary has never cloned
and asserts the primary project tree is byte-identical afterwards, alongside
refusals for a missing origin, an unsafe origin, a local-only project, and an
unregistered project.

* no-mistakes(review): Clarify project origin documentation ownership

* no-mistakes(document): Document supplied-origin remote seeding contract

* feat(secondmate): accept project origins from any host or forge

Firstmate is a shared template, so a project origin must be able to name any
host: GitHub Enterprise on a private domain, GitLab hosted or self-hosted,
Bitbucket, Gitea, Codeberg, sr.ht, a bare IP, an SSH config alias, or a plain
server nobody else has heard of. The validator already decided on structure
rather than on a forge allowlist, and this makes that guarantee explicit and
closes the two gaps that a host-agnostic rule exposed:

- a bracketed IPv6 literal in the scp-like form is now accepted, so a host
  reachable only by address is not excluded
- a "/../" traversal inside a local or file: origin is now refused, because
  that names a path on the cloning host's own filesystem

The library is the single owner of the accepted forms, and its header says
plainly that there is no host, domain, or forge allowlist and there must never
be one. The skill keeps its distinct agent-operating lines (the agent resolves
and supplies the origin; a remote seed creates nothing in the primary home
beyond the route, the charter, and its launch record) and points at the library
for URL acceptance and at the operator doc for the rest.

The lifecycle test now drives Bitbucket, a self-hosted enterprise domain, a
self-hosted GitLab over ssh with a port, and a bare scp-like custom host through
the real seed, manifest, transport, and remote provisioning path in one seed,
asserting each URL reaches git unchanged and each clone carries its own origin's
content. The unit matrix leads with non-GitHub hosts for the same reason.

* no-mistakes(review): Validate project origin authorities safely

* no-mistakes: apply CI fixes

* no-mistakes: apply CI fixes
…guid#1851)

* fix(tests): copy the whole bin/ tree into the old-vs-new conformance shim

main went red on tests/fm-backend.test.sh's "fm-send --key: old vs new
exit code" assertion, which reads as an fm-send fail-closed regression from

build_old_bin enumerated by hand the sibling scripts it copied into the
synthetic pre-refactor tree. kunchenguid#1842 made bin/fm-send.sh source
bin/fm-line-cap-lib.sh (added by kunchenguid#1798) and the list never learned about it,
so the pinned old fm-send.sh aborted at `. "$SCRIPT_DIR/fm-line-cap-lib.sh"`
under set -eu and exited 1 before parsing a single argument, while the
current one delivered the key and exited 0. The parity check compared a
crashed process against a working one and reported a behavior divergence
that never happened - the more so because BASE_REF collapses to HEAD on
main, where both sides run byte-identical source and a genuine divergence is
impossible. fm-send's --key exit path is unchanged and its fail-closed
contract is intact.

Copy the tree whole instead of enumerating it. An enumerated list has to be
extended by hand every time an entrypoint gains a dependency and is the only
thing that knows; it has been patched a dozen times for exactly that. A
whole-tree copy has nothing to forget. Extracting a refactored entrypoint the
baseline does not have now fails loudly instead of writing an empty file.

Only old-vs-new parity covered that exit contract, and parity is near-vacuous
on main. Pin it directly: tests/fm-send-strict.test.sh drives delivery both
ways from one stub and asserts an undelivered key exits nonzero naming the
key, so swallowing that error fails the suite.

* no-mistakes(review): Materialize historical fixture dependencies from baseline

* no-mistakes(document): Clarify fm-send key regression scope
* fix(bin): mirror the whole remote secondmate status stream

A remote secondmate's reply channel required corr=<16hex> on every line and
failed the entire delta when one line lacked it, so the cursor could never
advance past that line and the channel wedged permanently.

The charter tells a secondmate to report its own progress phases and to raise
new decisions with no correlation token, because correlation only answers a
marked parent request. Those lines were therefore unrepresentable on the remote
channel, while a local secondmate writes them straight into the parent's status
file.

Treat the channel as what it is: a mirror of the mate's status stream. A remote
mate now presents the same status and decision model as a local one, so a newly
raised needs-decision reaches the parent's open-decision fold identically, and
correlation goes back to being a per-line property that settles a pending
request rather than a gate on the stream.

Only what crossing a machine boundary genuinely adds stays behind: cursor
continuity, confined document fetch and rewrite, at-most-once append, and
control-byte normalization that rewrites bytes without ever dropping a line.
Line framing and size bounding already belong to fm-remote-delta-read.sh. A
document the remote reader refuses is named in one escalation instead of
stalling the stream, while an unavailable transport still leaves the delta for
the existing retry.

* refactor(bin): give the remote reply stream one append owner

Every line entering the parent status stream - a mirrored line, the continuity
escalation, and the undelivered-document escalation - now goes through one
at-most-once append, so the idempotence a replayed generation depends on is
stated once instead of copied at three call sites.

* no-mistakes(review): Keep local document transfer failures retryable

* no-mistakes(review): Isolate reply headers and normalize payload bytes

* no-mistakes(review): Correct remote reply mirror contract wording

* no-mistakes(review): Update remote reply script catalog description

* no-mistakes(document): Document remote status-stream mirroring
…chenguid#1847)

fm_secondmate_parent_record_parse read the .fm-secondmate-parent record
with bash's read, which drops NUL bytes - and different bash generations
disagree on the result: 3.2 truncates the value at the NUL while 5.x
splices the surrounding bytes together. A NUL-bearing parent_home could
therefore resolve to a home the record's bytes never name contiguously,
and which home fm-teardown.sh's promised-public-reply resolution read
(registration, registry, relay state) - or whether that protection
engaged at all - depended on which interpreter ran the cleanup.
Reproduced end to end: the same NUL-bearing record cleaned up under bash
5.x by resolving the spliced-together registered parent, while bash 3.2
refused it as unresolved, and a literal truncated path refused under
both.

Reject any NUL byte in the record before field parsing, putting corrupt
records in the same fail-closed bucket as duplicate fields, malformed
local bindings, unsupported routes, and symlinked records. The
regression test drives the real bin/fm-teardown.sh over the proven
clean-cleanup fixture with a NUL spliced mid-path into the recorded
parent_home, so before the fix it reproduced the wrong-home cleanup and
now it must refuse with the explicit binding refusal.
…unchenguid#1853)

* docs(secondmate-provisioning): require record intake for an inherited domain

A new mate seeded for an existing or inherited domain previously pulled in
charter, inherited config, captain-shared preferences, project clones, and
queued backlog rows with zero instruction about the domain's shipped history,
so it assumed a greenfield domain. A live backlog keeps only the configured
recent Done entries, so an inherited queue structurally over-represents plans
and under-represents deliveries, and already-delivered work resurfaced as open.

Add a record-intake step to the creation/seed path: classify greenfield versus
existing or inherited, and for the latter reconcile every inherited plan
against origin/main plus the live deployment, take only genuinely open work
and still-live durable knowledge, never carry a plan row for shipped work, and
record what could not be reconciled. Greenfield domains are untouched.

The skill owns the procedure; the backlog handoff section carries a one-line
reinforcement at the point where plan rows actually move.

* no-mistakes(document): Clarify secondmate record-intake scope
…guid#1860)

* perf(session-start): run every network check off the blocking path

The session-start digest runs on a session-open hook that blocks session
initialization, and every external-network call it made was individually
unbounded: `gh auth status`, secondmate liveness, secondmate convergence,
pending remote handoff delivery, and the fleet-sync fetch. One unreachable
remote secondmate could consume the whole FM_SESSION_START_TIMEOUT and
truncate the digest, so a slow network could cost the work queue itself.
Measured against a host hanging 25s per SSH connection, that startup took
1m18s.

The digest is now composed from local reads alone. bin/fm-startup-network.sh
runs the same checks concurrently in a bounded detached worker and the digest
harvests whatever finished, without ever waiting. Same fixture: 0.84s.

Nothing is dropped. fm-bootstrap.sh stays the single owner of every sweep and
still runs all of them, through a new FM_BOOTSTRAP_NETWORK phase split whose
`skip` and `only` halves are a partition of the unsplit run. Deferral is safe
because the sweeps are idempotent detectors, the result is durable and always
surfaces (inline, or as a `check: startup-network` wake), and the worker
re-verifies that the fleet lock still names the session that asked before it
mutates anything. While the worker is still running the digest names exactly
what is unconfirmed rather than implying it passed.

A relaunch performed by the deferred pass is now always reported, because the
digest that printed the superseded endpoint record is already out.

Also collapses the duplicate tasks-axi compatibility probe: the verdict is
computed once and handed to the bootstrap child for one process hop, then
consumed so it never reaches a spawned agent's environment. 10 tasks-axi
invocations per startup become 7.

Verified on Claude Code 2.1.222 that a worker detached by the session-open
hook survives the hook returning, the one vendor behavior this design needs
and no portable test can see.

Re-landed on current main, superseding PR kunchenguid#1845, which was cut from a
pre-kunchenguid#1842 base. The digest's section numbering in AGENTS.md section 3 now
states the emission order directly - supervision block and its read-once
contract, fleet state, network checks, then context - which keeps kunchenguid#1826's
fleet-state-before-context ordering. The old-bin test shim keeps main's
git-archive baseline from kunchenguid#1851, which already subsumes this branch's reason
for widening that shim.

* docs(verification): re-measure the deferred startup stage on the current base

Re-runs the unreachable-remote latency fixture against default-branch tip
8398d31 rather than the now-historical 345de4e, and records the sweep-result
comparison the deferral's safety argument rests on: the deferred worker's
published report is byte-identical to the three sweep lines the blocking
baseline printed, with the unreachable route preserved in both.

* no-mistakes(review): Fail deferred startup when report publication fails

* no-mistakes(document): Document deferred startup network behavior accurately
* fix(procevent): apply a captured adapter result in code, not by instruction

A remote secondmate's reply was captured and announced, but never applied.
Nothing dispatched the reply adapter's `handle` on a `procevent remote-reply`
wake, and the handling instruction named only the generic acknowledgement, so
the wake was retired while everything it carried was dropped: the reply never
reached the secondmate's local status mirror, the request it answered kept
escalating as a missed report, and the relay - whose registration each capture
retires, and which only that same handling re-arms - was left dead until the
next session start armed it again.

Applying such a result carries no judgement, so it belongs in code. After
publishing, the runner now calls
`bin/fm-procevent-<adapter>.sh autohandle <source-id> <sequence> <result-file>`
and lets the adapter apply and acknowledge its own result, through the same kind
of seam that already owns the terminal verdict. It runs strictly after terminal
retirement, because a handling adapter re-arms its own next source and retiring
afterwards would drop that fresh registration. An adapter with no such command,
or one whose pass does not complete, leaves the result unacknowledged and
therefore still announced, so a handler receives it exactly as before.

Resolving the request was not enough on its own either. An escalation opens a
durable keyed decision in the parent status log, and nothing ever closed it, so
a request the remote had answered kept surfacing in every later open-decisions
fold. The pending-reply library now owns both ends of that decision: it opens
one under a per-request key rather than the shared default key, and closes it
once the record resolves, appending the closing line only while that exact
decision is still open in the fold so it can neither double-close nor clear an
unrelated decision that has since taken the same key.

The handling instruction still routes a wake to its adapter, now as the
idempotent confirmation of what the runner already did rather than as the
guarantee.

Verified end to end in a throwaway isolated home driving the real armed source,
blocking delta reader, runner, and wake queue, with the handler doing only the
generic acknowledgement and no part of the ingest stubbed: before, seven failed
observations reproducing the incident; after, none. Each half is independently
load-bearing - without the runner change the reply never reaches the mirror,
without the escalation close the settled request still surfaces as an open
decision.

* no-mistakes(review): Prevent legacy reply closure from masking decisions

* no-mistakes(review): Serialize pending reply resolution and escalation closure

* no-mistakes(review): Serialize pending reply escalation with resolution

* no-mistakes(review): Clarify guarded legacy escalation closure behavior

* no-mistakes(review): Guard legacy closure and reserve pending reply keys

* no-mistakes(review): Match pending reply escalations by construction

* no-mistakes(document): Document automatic remote reply resolution

* no-mistakes(lint): Fix unused concurrent escalation loop variable

* no-mistakes(lint): Fix unused concurrent resolution loop binding

* no-mistakes(review): Version fold cache and gate autohandle on publication

* no-mistakes(document): Clarify remote reply relay documentation
…kill (kunchenguid#1841)

Bring the public installer-facing stow skill up to the internal skill's
current curation behavior while keeping it fully standalone:

- Replace the total-capture thesis with the compact-operating-map framing.
- Add read-the-destination-before-writing with the inspect-then-update
  triad (supersedes what, one-sentence rewrite, delete stale now).
- Add the concrete prune list together with its unique-fact guard, as an
  accuracy discipline with no size-budget machinery.
- Curate every memory file the pass has open, not only the routed one.
- Add the standing-decisions sweep category.
- Add the stronger-owner pointer-over-copy test before filing.
- Add tool-agnostic task-note discipline (inspect, classify, considered
  replacement body, never blind-append) and blocked-on recording.
- Give .stow-notes.md a closed set of three exits.
- Forbid storing, creating, or editing a skill as a stow destination.
- Report per-file action verbs in the completion receipt.
- Consolidate the repeated local-vs-external and .gitignore prose and fix
  the second-person voice slip, so the file does not grow (11334 -> 11276
  bytes).

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 7b9394ad0f

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

# raises itself - is at most once on exact bytes.
# Returns 0 appended, 1 already present, 2 the write itself failed.
append_status_once() { # <status-file> <line>
grep -Fqx -- "$2" "$1" 2>/dev/null && return 1

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Preserve repeated remote status events

Captain, a remote mate can legitimately emit the same keyed lifecycle line in separate work cycles—for example, blocked [key=deploy]: waiting after an earlier identical blocker was resolved—but this whole-file grep treats the later event as a replay and silently drops it. That can prevent a newly reopened decision from appearing in the parent's status fold; deduplication needs to be scoped to the captured generation rather than all historical lines.

AGENTS.md reference: AGENTS.md:L127-L127

Useful? React with 👍 / 👎.

Comment thread bin/fm-bootstrap.sh
if network_phase && network_sweep_authorized 'project clone refresh'; then fleet_sync; fi
fi
secondmate_handoff_detect
local_phase && secondmate_handoff_detect

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Report handoff status after deferred delivery

When a locked startup begins with a pending remote outbox, the local bootstrap phase runs this detection before the deferred worker attempts delivery, so it prints SECONDMATE_HANDOFF: ... pending delivery even if the worker immediately delivers and removes the outbox. The network-only phase never reruns the detector, leaving a stale actionable warning in an otherwise successful digest; detect the remaining outboxes after the deferred resume instead.

AGENTS.md reference: AGENTS.md:L146-L148

Useful? React with 👍 / 👎.

Comment thread bin/fm-send.sh
for k in $RESOLVE_KEYS; do
line="resolved [key=$k]: answered: $note"
fm_cap_line_var "$line"
if ! printf '%s\n' "$FM_LINE_CAP_LINE" >> "$RESOLVE_STATUS_FILE"; then

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Revalidate decision identity before closing it

If the worker resolves the original key and reopens that same stable key for a new blocker while backend submission is in flight, this unconditional append closes the new blocker even though the answer applied to the earlier decision; the only validation occurred before sending. Capture the original open record and confirm that the current key still refers to that exact record before appending the resolution, otherwise leave the new decision open and report the race.

AGENTS.md reference: AGENTS.md:L290-L291

Useful? React with 👍 / 👎.

@mjskinner82
mjskinner82 merged commit 561fcda into main Aug 7, 2026
mjskinner82 pushed a commit that referenced this pull request Sep 15, 2026
…nguid#3681)

* fix(bin): recognize active pipeline fix rounds with unfetched run heads

A no-mistakes fix round advances the run head beyond the submitted head,
and the pipeline commits in its own checkout, so the task copy never
receives the new commit object. fm-crew-state's strict head rule rejected
the active row, the coarse runs-list scan skipped it and matched the
older failed row at the submitted head, and an active validation read as
failed (observed on model-routing-benchmark-hardening: active head
ac61c64 vs task copy at fb47636d).

fm_nm_runs_status_for_worktree in bin/fm-nm-run-lib.sh now owns
runs-ledger attribution: the branch's newest row alone decides, and a
newest row whose head cannot resolve locally is recognized only as a
provable pipeline-owned continuation - active (running) and anchored by
the immediately older row for the same branch having ended at exactly
this worktree's HEAD. The reader keeps the axi TOON as full detail for
that proven same-branch run. Unanchored, ancestor-anchored, and terminal
unresolvable rows stay unattributed, so branch-name coincidence and other
tasks' runs never match, and fm_nm_head_matches_worktree keeps its exact
prior semantics for teardown (verified by the full teardown suite).

Tests: reproduction regression for the unfetched active fix head (reads
working via full run-step detail), coarse-path continuation when axi
answers another branch, and negative controls for the unanchored active
row and the unresolvable terminal row with the historical fallback
preserved.

Ported onto upstream/main f4d7875, where kunchenguid#3194 independently added the
branch_sync custody exemption on the full axi-status path: both mechanisms
now coexist, each owning one surface (TOON custody on the full path, the
runs ledger on the coarse path). The port deletes the superseded coarse
scan-and-skip (nm_runs_status_for_branch) and its now caller-less helpers
(fm_nm_head_resolvable, nm_coarse_head_matches_worktree), renames the
exemption comment's "the one exemption" phrasing now that a second
complementary exemption exists, and points the stale
FM_CREW_STATE_RUNS_LIMIT comment at fm_nm_runs_status_for_worktree
(judge follow-up #1). The parent coarse-guard test's fixture is the
ledger-anchored continuation shape, so its expectation flips to the fixed
behavior (working via run-step, never the older failed row); a new
mismatched-anchor coarse negative control preserves that guard's original
no-anchor protection (pane answers, never the older row).

* no-mistakes(document): Clarify pipeline attribution documentation
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants