Skip to content

feat(bin): relay trust hardening, usage ledger, and fleet ops tooling - #3533

Closed
derickdsouza wants to merge 11 commits into
kunchenguid:mainfrom
derickdsouza:fm/nm-agent-chain
Closed

derickdsouza wants to merge 11 commits into
kunchenguid:mainfrom
derickdsouza:fm/nm-agent-chain

Conversation

@derickdsouza

Copy link
Copy Markdown

Intent

Work the only open nivasritech/firstmate GitHub issue (#7): set the global no-mistakes reviewer chain to [pi, claude] so validation can launch in repos with AGENTS.md/CLAUDE.md.

Captain decisions already made on that issue:

  • Global chain is [pi, claude], not [pi, claude, grok, cursor]. Grok and cursor cannot neutralize agent-instruction files, so they cannot be fallbacks for fleet repos. Codex is the only other eligible fallback and is not included.
  • Do not add an agent: override to the tracked .no-mistakes.yaml.
  • Do not invent uncommented agent_config pins that are not already present on this home (pm-dev has no agent_config.pi/claude entries).
  • Document the fleet policy in docs/configuration.md (Gate defaults). data/learnings.md is gitignored and is not the owner. Bootstrap does not write ~/.no-mistakes/config.yaml; document that operators apply the chain after installing no-mistakes.

This change records that policy in docs/configuration.md, CONTRIBUTING.md (a pointer to Gate defaults), and a comment on .no-mistakes.yaml explaining why there is no repo-level agent override. The operator-local ~/.no-mistakes/config.yaml on this home was updated to agent: [pi, claude] separately and is not part of the tracked diff.

What Changed

  • Relay intake hardening: new bin/fm-untrusted-text-lib.sh sanitizes untrusted mention/thread text before the inbox stash (sanitized-to-empty mentions are claimed and dismissed at the relay instead of re-offered), fm-spawn.sh refuses to launch work while a stashed mention lacks its answered marker (recorded by fm-x-reply.sh on the initial answer), and the new bin/fm-x-watch-register.sh CLI plus unified fm-x-lib.sh validators let the watcher and bootstrap dispatch a hash-bound alternate poll shim (harbor telegram-hitl manifest or explicit state-local registration) alongside the stock generated shim.
  • Usage ledger and stuck-helper recovery: fm-spawn.sh binds each task's harness session log into meta (usage_source=/usage_log= for claude, codex, and pi) and fm-teardown.sh appends a best-effort fm-usage.v1 token line to data/usage.jsonl; new fm-secondmate-health-lib.sh classifiers let the bootstrap liveness sweep and the watcher's remote inbox tick replace alive-but-stuck remote helpers (unacked steering mail past the ladder, or a repeating Node missing-module capture) instead of reusing them.
  • Fleet ops, worker contract, and docs: new fm-fleet-watch-poller.sh keeps the when-watches armed and fm-agent-login-probe.sh probes pi/grok/cursor-agent/claude login health on mac and VPS; fm-dod-lib.sh gains a worker-facing rule-file layering line rendered into both ship briefs (fm-brief.sh) and promoted ship instructions (fm-promote.sh); docs/configuration.md "Gate defaults" records the global no-mistakes reviewer chain [pi, claude] with a CONTRIBUTING pointer and a .no-mistakes.yaml comment explaining why the tracked file sets no agent:; new test suites cover the sanitizer, usage ledger, poll-shim registration, health classifiers, and fleet poller.

Risk Assessment

⚠️ Medium: The run's own docs commit matches the stated intent exactly and adds no behavior, but the branch bundles ~3,400 lines of new fleet machinery whose probes and guards have one warning-level functional gap (claude login probe cannot detect logout) plus two acknowledged best-effort tradeoffs, all addressable as follow-ups.

Testing

Parsed the tracked .no-mistakes.yaml with PyYAML to prove the change is comments-only (semantic model identical to base, no agent: override, all existing pins intact), verified every intent-mandated policy element in the docs/configuration.md Gate defaults section plus the CONTRIBUTING.md pointer and resolvable cross-references, confirmed via source inspection that bootstrap never writes the global config, ran the adjacent fm-gate-refuse suite (7/7 pass), and confirmed the operator-local global chain is agent: [pi, claude] with this very pi session in an AGENTS.md/CLAUDE.md repo as end-to-end proof; all checks passed with the worktree left clean.

Evidence: Gate-defaults reviewer-chain evidence (rendered doc sections + verification transcript)

Source: Gate-defaults reviewer-chain evidence (rendered doc sections + verification transcript)

# Evidence: global no-mistakes reviewer chain documented as [pi, claude] (issue #7)

Commit under test: 034713f docs(config): record global no-mistakes reviewer chain as [pi, claude] (#7)
Tracked diff: .no-mistakes.yaml (+6 comment lines), CONTRIBUTING.md (+1), docs/configuration.md (+8) - docs/comments only.

## 1. docs/configuration.md - Gate defaults (.no-mistakes.yaml), final section

`` `markdown

The reviewer chain is global, in `~/.no-mistakes/config.yaml`, not a per-repo override.
Set `agent: [pi, claude]`.
No-mistakes refuses to launch a gate agent that cannot neutralize the target repo's `AGENTS.md` or `CLAUDE.md`; only pi, claude, and codex have a verified neutralization knob.
Grok and cursor do not, so they cannot sit in the chain for this repo or any other fleet checkout that carries those files.
Codex is the only other eligible fallback; the fleet default does not include it.
Do not add `agent:` to the tracked `.no-mistakes.yaml` unless a repo needs a different chain than that fleet default, and then record why.
A new firstmate home does not write that global file; apply the chain in `~/.no-mistakes/config.yaml` after installing no-mistakes, then confirm with `no-mistakes doctor` that gate validation names pi or claude rather than grok or cursor.
`` `

## 2. CONTRIBUTING.md pointer (line 73)

`` `markdown
The global reviewer chain is owned by [`docs/configuration.md`](docs/configuration.md) ("Gate defaults"); the tracked file does not set `agent:`.
`` `

## 3. .no-mistakes.yaml comment block (tracked file keeps NO agent: key)

`` `yaml
# Keep the chain global so harbor and other fleet repos share it. See
# docs/configuration.md "Gate defaults".

# Trusted documentation placement policy for the Document step.
# The audience inventory and coding guideline own the detail; keep this as a
# pointer so gate instructions cannot become a second prose policy.
`` `

## 4. Semantic verification of the tracked YAML (PyYAML parse)

`` `
PASS - tracked .no-mistakes.yaml parses into a mapping
PASS - no top-level "agent" key (intent forbids repo-level override)
PASS - no "agent_config" key invented anywhere
PASS - disable_project_settings still true (neutralization opt-out intact)
PASS - commands.lint pin intact
PASS - test.evidence.store_in_repo intact
PASS - semantic model identical to base commit => tracked diff is comments-only

current semantic model:
{"commands": {"lint": "bin/fm-lint.sh"},
 "disable_project_settings": true,
 "document": {"instructions": "Read docs/documentation-audiences.md ... branch diff again after every documentation or lint fix.\n"},
 "test": {"evidence": {"store_in_repo": true}}}
`` `

## 5. Operator-local global config (outside tracked diff, per intent)

`` `
path: ~/.no-mistakes/config.yaml exists: True
agent: ['pi', 'claude']
matches documented policy [pi, claude]: True
`` `

## 6. Adjacent automated suite (gate-agent refusal boundary around the tracked config)

`` `
ok - fm-gate-refuse-lib: refuses when NO_MISTAKES_GATE is set
ok - fm-gate-refuse-lib: refuses when NO_MISTAKES_GATE is set empty
ok - fm-gate-refuse-lib: refuses from a gate worktree via git-common-dir (marker unset)
ok - fm-gate-refuse-lib: no-op for a normal session (neither signal, set -eu clean)
ok - fm-spawn: refuses on marker and gate-worktree backstop; a normal crew spawn is unaffected
ok - fm-send: refuses on marker and gate-worktree backstop; a normal steer uses the inbox
ok - fm-teardown: refuses on marker and gate-worktree backstop; a normal teardown is unaffected
`` `

Pipeline

Updates from git push no-mistakes

✅ **intent** - passed

✅ No issues found.

⚠️ **Rebase** - 1 warning

Push main to origin, or rebase your branch onto origin/main, before gating.

⚠️ **Review** - 3 issues (1 warning, 2 infos)
  • ⚠️ bin/fm-agent-login-probe.sh:87 - check_mac_claude reports OK when ~/.claude/.credentials.json is absent but ~/.claude/plugins exists (line 87), or whenever claude --version succeeds (line 91) -- neither proves login, and --version works offline for a fully logged-out CLI. Concrete path: Claude Code session expires/credentials removed, plugins dir persists -> probe prints 'OK claude@mac' and exits 0, so the login-watch/fleet probe never flags claude for re-login unless the CLI is entirely uninstalled. The script's stated contract ('exit 1 when at least one target needs re-login') is silently unenforceable for one of its four targets. Remedy requires choosing a real login signal (credentials-only check, or a non-interactive auth-status command), which is a deliberate probe-design decision, so it needs user input rather than an auto-fix.
  • ℹ️ bin/fm-x-reply.sh:342 - A dry-run reply (FMX_DRY) writes the x-context/<id>.answered.json marker that disarms fmx_inbox_spawn_guard without any relay POST. This is documented deliberately (docs/configuration.md: 'records that marker on the initial answer (including dry-run)'), presumably so preview homes are not wedged. Residual tradeoff worth acknowledging: fmx_load_config lets an explicit FMX_DRY_RUN environment variable win over .env, so any process -- including the agent the fix: accept landed squash-merged PR heads #149 ack-before-long-work guard constrains -- can run FMX_DRY_RUN=1 fm-x-reply.sh &lt;id&gt; ... to mark the ack and pass the spawn refusal while the Telegram question remains unanswered at the relay. No change requested; noting the documented weakening and its reachable bypass.
  • ℹ️ bin/fm-usage-lib.sh:71 - The codex binding pins usage_log to the spawn-date directory (date +%Y/%m/%d), so a codex task whose session continues past local midnight has its later session files written into the next date directory and they are never matched at teardown -- a silent token undercount in data/usage.jsonl (fm_usage_collect_rows finds nothing new; no error, no usage_source=missing since the old-date files still match). The header documents the v1 reused-worktree overcount limitation but not this midnight-rollover undercount; a one-line contract note (or a follow-up date-range glob) would make the best-effort schema honest.
✅ **Test** - passed

✅ No issues found.

  • python3 PyYAML semantic verification: .no-mistakes.yaml parses, has no top-level agent key, no agent_config key, disable_project_settings: true, commands.lint/test.evidence.store_in_repo pins intact, and parsed model equals base commit 8988af2's model (comments-only tracked diff)
  • bash tests/fm-gate-refuse.test.sh (adjacent gate-agent refusal suite around the tracked no-mistakes config boundary) - 7/7 pass
  • Manual doc-acceptance check: sed -n &#39;201,218p&#39; docs/configuration.md and sed -n &#39;68,76p&#39; CONTRIBUTING.md confirm all intent-mandated policy elements and the Gate-defaults pointer; heading 'Gate defaults (.no-mistakes.yaml)' exists at docs/configuration.md:201
  • Manual factual-claim check: grep -rn &#39;no-mistakes/config\|config\.yaml&#39; bin/ confirms no firstmate script (including bin/fm-bootstrap.sh) writes ~/.no-mistakes/config.yaml
  • Read-only end-to-end check: parsed ~/.no-mistakes/config.yaml -> agent: [pi, claude], matching the documented policy (this pi-driven validation session in an AGENTS.md/CLAUDE.md repo is the live proof)
✅ **Document** - passed

✅ No issues found.

⚠️ **Lint** - 1 warning
  • ⚠️ linter found issues (exit code 1)
✅ **Push** - passed

✅ No issues found.

derickdsouza and others added 11 commits September 2, 2026 01:35
…guid#149) (#1)

Extend fmx_poll_shim_valid to honor config/harbor-telegram-hitl.poll-shim.registration.json
so harbor telegram-hitl can replace x-watch.check.sh without watcher rejection.

Enforce relay ack-before-long-work: fm-x-reply records x-context/<id>.answered.json
on the initial answer; fm-spawn refuses while any open x-inbox row lacks that marker.

Co-authored-by: Cursor <cursoragent@cursor.com>
Port of derickdsouza/firstmate PR #2 onto org main: the explicit
bin/fm-x-watch-register.sh binding command with colocated tests, the
stock/registered validator split, watcher exec-target dispatch with the
re-register hint, and bootstrap preservation of a still-matching binding.
Mainline harbor manifest validation is kept as fmx_poll_shim_manifest_valid
and chained into fmx_poll_shim_valid, so harbor-installed and
CLI-registered alternates authenticate through one dispatch-time check.
Port of derickdsouza/firstmate PR #3 in full: deterministic length-bound
fm-untrusted-text-lib.sh neutralizing injection role-markers, HTML comments,
and operational-prefix impersonation, wired into the relay mention stash
path with claim-and-dismiss for empty-after-sanitize mentions, batched
chain sanitizing bounded by string size rather than entry count, and
colocated coverage across locale, multibyte, fixed-point, and budget cases.
…ontract

Port of the parked fm/adopt-constitution-layering branch (7ba0d37):
ship and promoted-scout briefs carry the layering discipline - shared
worker-facing rule files are law and are never same-name overridden by a
project; project specialization lives in distinctly-namespaced local
files. Scaffold tests pin the contract on both brief paths.
Bring in 9 upstream commits (relay media, spawn/startup/wake fixes,
Bearings concurrency, pi supervision, CI shards) while preserving fleet
commits: HITL poll-shim trust, ack-before-spawn, sanitizer, constitution
layering.

Conflicts resolved in docs/configuration.md and tests/fm-x-mode.test.sh
(fleet untrusted-text sanitization kept).

Co-authored-by: Cursor <cursoragent@cursor.com>
…-09-02

merge(upstream): sync kunchenguid/firstmate main (9 commits)
Process-alive is not health. A remote helper with unacked steering-inbox
mail past the ladder, or a repeating Node missing-module capture after a
harness upgrade under a live process, is stuck.

launch now replaces stuck helpers instead of reusing them. Session-start
liveness respawns those stuck remotes. The parent watcher ticks the remote
inbox ladder on the host. Spawn/bootstrap reconcile the reply runner after
arming so a registered source is not left without a child.
…down (kunchenguid#4) (kunchenguid#5)

Co-authored-by: Derick D'Souza <derick@nivasritech.com>
* feat(bin): fleet when-watch poller with cron dedupe and single-flight lock

* feat(bin): agent login probe with login-shell PATH resolution

Cron hardening: default USER/LOGNAME when unset and prepend the standard
CLI install roots (~/.local/bin, nvm per-version bin, ~/.grok/bin,
Homebrew) to PATH so a minimal cron environment no longer misreports
installed CLIs as absent. Also fix the grok and cursor checks, whose
has_stdout ... | grep pipeline could never match because has_stdout
captures the output and returns only a status; they now grep the real
command output. Every expansion is set -u safe under env -i.
kunchenguid#7)

Grok and cursor cannot neutralize AGENTS.md/CLAUDE.md, so they cannot gate
fleet repos. Keep the chain in ~/.no-mistakes/config.yaml and out of the
tracked repo override.
@greptile-apps

greptile-apps Bot commented Sep 2, 2026

Copy link
Copy Markdown

Confidence Score: 4/5

The PR appears safe to merge, with a non-blocking relay-sanitizer hardening gap around zero-width joiners.

The new sanitizer leaves U+200D-containing role markers intact because it deliberately preserves the character while recognizing only contiguous ASCII role names; no blocking failure remains after accounting for the PR’s explicitly acknowledged tradeoffs.

Files Needing Attention: bin/fm-untrusted-text-lib.sh

Security Review

The sanitizer preserves U+200D while matching only contiguous role names, allowing visually disguised role markers to remain in agent-facing relay text. How this was verified: tracing a U+200D-containing role prefix through format stripping and contiguous role matching shows it reaches the inbox without replacement.

Reviews (1): Last reviewed commit: "docs(config): record global no-mistakes ..." | Re-trigger Greptile


fm_untrusted_strip_format_chars_var() {
local text=$1 c
# Strip Cf / bidi / BOM / soft-hyphen hides. Do not strip U+200D ZWJ.

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 security ZWJ bypasses role neutralization

A line-leading role marker containing U+200D, such as sys‍tem:, survives format stripping and fails the contiguous ASCII role-name match, leaving visually disguised role instructions in the agent-facing inbox and weakening the new prompt-injection boundary.

How this was verified: Tracing a U+200D-containing role prefix through format stripping and contiguous role matching shows it reaches the inbox without replacement.

@derickdsouza

Copy link
Copy Markdown
Author

Opened against the wrong parent. This home's no-mistakes gate is initialized to push derickdsouza/firstmate and PR kunchenguid/firstmate, so the run bundled ten already-merged nivasritech commits onto a stale fork main.

The actual #7 change is the one-commit docs PR against nivasritech/firstmate: https://github.com/nivasritech/firstmate/pull/8

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant