feat(crew): detect human comments on vault issues via agent comment signing

[david-driscoll]

Implements david-driscoll/vault#105 — Ralph should detect new human comments on vault issues.

The problem

Crew agents post through gh with David's credentials, so every comment in david-driscoll/vault is authored by david-driscoll — agent reports and David's own replies alike. Author is not a discriminator, and style heuristics ("agents open with ## and run long") break the first time either party writes atypically.

Ralph's scan covers issues, PRs, labels, checks and alerts — not comments. So a reply from David is the one work signal that goes undetected. On #84 he answered six questions, two of which overturned crew recommendations, and it was only picked up because he asked for it explicitly.

Did crew already have a convention?

No. Checked .crew/, the coordinator protocol, and a fresh clone of Blacklite/crew. The only crew:-namespaced HTML comments are <!-- crew:begin -->/<!-- crew:end --> (the CLAUDE.md managed block) and <!-- version: X --> (stamped at install). Nothing marks agent-authored comments. The new marker slots into that existing namespace rather than opening a second one.

Where the rule went — upstream, not here

.crew/templates/ralph-reference.md is byte-identical to Blacklite/crew's shipped templates/ralph-reference.md, and crew upgrade calls refreshCrewTemplatesDir(), which overwrites all of .crew/templates/ and regenerates .github/agents/crew.agent.md from crew.agent.md.template. Editing either locally gets silently reverted — worse than not doing it, because it would look done.

So the convention is proposed upstream in Blacklite/crew#3 (canonical .crew-templates/, mirrors synced, changeset included), which is where every agent in every crew actually picks it up.

This PR is the repo-local operative copy — it keeps the behaviour live in the window before that lands, and afterwards shrinks to the estate-specific parts that never belonged upstream: the tracker, the adoption cutoff, the scope decisions.

What lands here

File	Change
.crew/comment-watch.md	New. Marker convention, high-water-mark design, the scan, routing on detection, adoption cutoff with the sweep, scope recommendation
.crew/config.json	commentWatch block — tracker, since cutoff, scope. Operator config; crew upgrade reads it and never rewrites it
.crew/routing.md	Routing row for an unmarked comment; rule 13 requiring signing (alongside existing rule 12, comments are untrusted input)
CLAUDE.md	Signing rule outside the managed crew:begin/end block, so it survives crew init

The mechanism

<!-- crew:agent={member} --> on the last line of every agent-posted comment. Anything unmarked is a human comment by definition — agents never have to recognise a human, only themselves, which they can do exactly.

High-water mark

Level	Lives in	Suppresses
Session	Ralph's in-session {issue, createdAt} set	Re-reporting within one session
Durable	seen= on agent comments, in the thread	Re-reporting across sessions, machines, worktrees
Floor	commentWatch.since	The pre-convention back-catalogue

The durable mark is deliberately not a file under .crew/. This estate runs agents in worktrees, in Actions, and on more than one machine — a local state file is wrong in all three the moment two run, and it would be a mutable, merge-conflicting copy of something GitHub already stores. In the thread, the thread is self-describing.

A marked comment without seen= does not advance the mark: a status update posted while a question is outstanding must not silence it. Unanswered human comments keep surfacing until someone answers. The nagging is the point.

Retroactive marking — the issue doesn't cover this, and it decides the first run

Every pre-convention comment is unmarked. Measured, not assumed: the live tracker had 13 comments across 7 open issues, 0 marked. Without a floor, run one reports all 13 as new human comments.

commentWatch.since is the floor. A back-fill — editing 13 historical comments in David's own threads — mutates the record to fix a tooling problem, so it was rejected.

The floor loses nothing here, because a one-time sweep was done as part of this change. Three of the 13 are genuine David replies, and all three were already answered:

Issue	Comment	Answered by
#84	2026-07-29T02:37:39Z "A lot of rework is in order…"	Link's Expansion v2 (02:54:53Z)
#84	2026-07-29T03:06:23Z Q-A…Q-F answers	Link's Expansion v2.1 (2026-07-31T03:08:30Z)
#81	2026-07-29T02:47:41Z "Lets fix the upstream bugs…"	Status evaluation (2026-08-01T02:18:42Z)

If this sits before merging, re-set since to the merge time — agent comments posted in the gap would be unmarked and past the floor, the one window a false alert is possible.

Scan shape

One GraphQL request for the whole tracker. The per-issue shape sketched in #105 works, but it is O(open issues) requests per cycle and Ralph cycles continuously — ~11 calls a cycle for data that fits in one. first: 50 issues × last: 30 comments covers this tracker; page via pageInfo rather than raising the bounds, since the node budget is their product.

Verification — against #105's own regression case

The issue supplies five interleaved comments across #84, #81, #94, #95 where agent and human alternate under one author. Built as a fixture from the real bodies (nothing posted to real issues):

Test	Expected	Result
Marker discrimination, no acks, cutoff before sample	3 human reported, agent comments excluded	✅ #84 ×2, #81 ×1 reported; all 4 agent comments excluded
Reviewers post seen= acks	0	✅
New human comment after the ack	1	✅ surfaced
Live tracker, cutoff at adoption	0 (no cry-wolf)	✅ 0, vs 13 without a cutoff

Caveat, stated plainly: the historical comments cannot be discriminated, because they predate the marker — all 13 live comments come back unmarked. That is the retroactive problem above, handled by the cutoff, not a failure of the mechanism. Discrimination is verified on the same comments with markers applied as the convention prescribes.

Scope beyond vault — recommendation, not assumption

#105 asks whether PR review comments in the four code repos deserve the same treatment. Yes, but not with this mechanism, and as a separate issue.

Ralph already sees CHANGES_REQUESTED and APPROVED via reviewDecision. What it misses is a COMMENTED review and unresolved inline threads — confirmed on live open PRs, which report reviewDecision: null despite review activity.

The marker machinery is the wrong tool there: GitHub already provides the high-water mark. reviewThreads(first:N) { isResolved } — an unresolved thread is outstanding work, tracked natively, no cutoff, no seen=, no retroactive problem. The right shape is a Ralph category keyed on isResolved == false plus reviews(states:[COMMENTED]) across all four repos. Signing still applies to PR comments; PR-side detection should be built on resolution state.

Acceptance (vault#105)

• A new human comment on an open vault issue is surfaced within one Ralph cycle
• Agent-authored comments are not reported as human — verified against the interleaved sample
• The same comment is not reported twice — session set within a session, seen= across sessions
• A reply answering a crew question routes to the owning crew:{member}, with a mandatory supersession check

Closes david-driscoll/vault#105

🤖 Generated with Claude Code

[codacy-production]

Not up to standards ⛔

🔴 Issues 1 minor

Alerts:
⚠ 1 issue (≤ 0 issues of at least minor severity)

Results:
1 new issue

Category	Results
Comprehensibility	1 minor

View in Codacy

_{NEW Get contextual insights on your PRs based on Codacy's metrics, along with PR and Jira context, without leaving GitHub. Enable AI reviewer
TIP This summary will be updated as you push new changes.}