feat(templates): detect PR review feedback in Ralph's work-check cycle

[david-driscoll]

Follow-on to #3. That PR made human comments on issues detectable. This one closes the
other half of the same gap: review feedback on pull requests.

Parent issue: david-driscoll/vault#105 · Estate-side companion: david-driscoll/home-operations#627

The gap

Ralph's PR scan reads reviewDecision, which reports CHANGES_REQUESTED and APPROVED.
But a COMMENTED review leaves reviewDecision: null, and so does an inline review
thread. A reviewer can ask for a change on a specific line and Ralph reports the PR as
clean — so requested changes sit unread on work that is otherwise ready to merge.

What this does not reuse, and why

The seen= marker machinery from #3 exists because GitHub tracks nothing about whether a
comment was answered. Review threads carry isResolved, which is exactly that. So they
need no marker-based high-water mark, no commentWatch.since cutoff, and have no
retroactive adoption problem — a thread resolved before any of this existed is already
resolved, and one left open is genuinely still open. Adding seen= there would create a
second high-water mark that can disagree with GitHub's own.

seen= still applies to the surfaces with no resolution state: top-level COMMENTED
review bodies and PR conversation comments.

Surface	Resolution state	High-water mark
Inline review thread	✅ isResolved	GitHub's, natively
COMMENTED review body	❌	seen= on an agent reply
PR conversation comment	❌	seen= on an agent reply

The bot filter is asymmetric, and that part is measured

This is the correction that fell out of validating against live data. Keying the category on
reviews(states:[COMMENTED]) as originally sketched makes it fire on every PR forever:
quality/CI bots post a top-level review or PR comment unconditionally. Inline threads are
the opposite — rare and specific.

Sampled across the four code repos: a review bot had posted a top-level COMMENTED review
or PR comment on every PR sampled, but had opened an inline thread on one PR in sixty.

So:

• Inline threads — keep every author, bots included. Rare, anchored to a line, and they
  carry a native ack. A bot pointing at a real line is real feedback.
• Top-level reviews and PR comments — drop author.__typename == "Bot". Otherwise the
  category is permanently lit and becomes noise the operator learns to ignore.

On a live scan of six repos / eight open PRs, that filter is the difference between 12
reported items and 0 — all 12 bot boilerplate. Filtering on __typename rather than a
login allowlist needs no per-installation maintenance.

Agents do not resolve threads

The design decision that most affects whether this is trustworthy. Both directions have a
real argument, so compare the failure modes:

• Agent resolves a thread it only partly addressed → the feedback disappears from every
  view the reviewer uses. Fails silently, and nothing surfaces it later.
• Agent leaves an addressed thread open → it stays visible until someone confirms. Fails
  noisily, and self-corrects.

For a mechanism whose whole purpose is that feedback stops sitting unread, a silent-drop
failure defeats the thing being built. A noisy one merely annoys.

That asymmetry is only decisive because the queue-cleanliness argument has another answer:
a thread whose newest comment is a signed <!-- crew:agent= --> reply is demoted to
awaiting confirmation rather than re-reported as new work. The queue stays clean, the
reviewer keeps the resolve button, and a further reviewer reply returns the thread to the
queue automatically. This is the one place the marker earns its keep on the PR side — it is
what distinguishes "an agent has responded here" from "nobody has".

Explicit operator instruction to resolve remains an exception.

Config

Repos are configurable rather than hardcoded — review feedback spans the repos the crew
opens PRs in, not the single issue tracker:

"commentWatch": {
  "repo": "{owner}/{tracker}",
  "since": "...",
  "pullRequestRepos": ["{owner}/{repo-a}", "{owner}/{repo-b}"]
}

One GraphQL request covers all of them via aliases, matching #3's one-request-per-cycle
principle. Absent/empty pullRequestRepos is a no-op, so this is inert for existing crews
until an operator opts in.

Verification

Read-only throughout — no comments posted, no threads resolved on any real PR.

• Live, six repos in one request: correctly reports nothing (no unresolved threads on any
  open PR; all 11 PR comments are bot-authored and suppressed).
• Live, against a real unresolved thread (vault#40, read-only): fires with
  .mise.toml:39 and the gist.
• Fixture, 8 cases built from that real payload, all passing — including the load-bearing
  one, a resolved twin of the same thread, which is correctly suppressed:

Case	Expected	Result
Real unresolved inline thread	unresolved	✅
Same thread, isResolved: true	suppressed	✅
Human ask + newer signed agent reply	awaiting-confirm	✅
Agent reply, then human asks again	unresolved (reply is stale)	✅
Bot COMMENTED review	suppressed	✅
Human COMMENTED review, unsigned	review-commented	✅
Bot comment + signed agent comment + unsigned human comment	only the human one	✅
Unresolved and isOutdated	unresolved, flagged outdated	✅

isOutdated is reported, never filtered on: it means the diff hunk moved, not that the
point was addressed.

Only states:OPEN PRs are scanned — unresolved threads survive a merge, so dropping the
state filter would resurrect feedback on shipped work.

Files

.crew-templates/{ralph-reference,issue-lifecycle,crew.agent}.md, mirrored to the three
template targets and .github/agents/ via node scripts/sync-templates.mjs --sync.
Changeset included. Same 13-file footprint as #3.

[github-actions]

🟠 Impact Analysis — PR #4

Risk tier: 🟠 HIGH

📊 Summary

Metric	Count
Files changed	14
Files added	1
Files modified	13
Files deleted	0
Modules touched	5

🎯 Risk Factors

• 14 files changed (6-20 → MEDIUM)
• 5 modules touched (5-8 → HIGH)

📦 Modules Affected

ci-workflows (1 file)

• .github/agents/crew.agent.md

crew-cli (3 files)

• packages/crew-cli/templates/crew.agent.md.template
• packages/crew-cli/templates/issue-lifecycle.md
• packages/crew-cli/templates/ralph-reference.md

crew-sdk (3 files)

• packages/crew-sdk/templates/crew.agent.md.template
• packages/crew-sdk/templates/issue-lifecycle.md
• packages/crew-sdk/templates/ralph-reference.md

root (4 files)

• .changeset/pr-review-feedback-detection.md
• templates/crew.agent.md.template
• templates/issue-lifecycle.md
• templates/ralph-reference.md

templates (3 files)

• .crew-templates/crew.agent.md
• .crew-templates/issue-lifecycle.md
• .crew-templates/ralph-reference.md

---

This report is generated automatically for every PR. See #733 for details.

[github-actions]

🛫 PR Readiness Check

ℹ️ This comment updates on each push. Last checked: commit 9090bbc

PR Scope: 🔧 Infrastructure

⚠️ 4 item(s) to address before review

Status	Check	Details
❌	Single commit	2 commits — consider squashing before review
✅	Not in draft	Ready for review
❌	Branch up to date	dev is 1 commit(s) ahead — rebase recommended
❌	Copilot review	No Copilot review yet — it may still be processing
✅	Changeset present	Changeset file found
✅	Scope clean	No .crew/ or docs/proposals/ files
✅	No merge conflicts	Merge status unknown — GitHub is still computing
✅	Copilot threads resolved	No Copilot review threads
❌	CI passing	1 check(s) failing: Policy Gates

Files Changed (14 files, +1053 −26)

File	+/−
.changeset/pr-review-feedback-detection.md	+14 −0
.crew-templates/crew.agent.md	+23 −2
.crew-templates/issue-lifecycle.md	+47 −3
.crew-templates/ralph-reference.md	+184 −1
.github/agents/crew.agent.md	+23 −2
packages/crew-cli/templates/crew.agent.md.template	+23 −2
packages/crew-cli/templates/issue-lifecycle.md	+47 −3
packages/crew-cli/templates/ralph-reference.md	+184 −1
packages/crew-sdk/templates/crew.agent.md.template	+23 −2
packages/crew-sdk/templates/issue-lifecycle.md	+47 −3
packages/crew-sdk/templates/ralph-reference.md	+184 −1
templates/crew.agent.md.template	+23 −2
templates/issue-lifecycle.md	+47 −3
templates/ralph-reference.md	+184 −1

Total: +1053 −26

---

This check runs automatically on every push. Fix any ❌ items and push again.
See CONTRIBUTING.md and PR Requirements for details.

[david-driscoll]

Estate-side companion is up: david-driscoll/home-operations#627 (config + operative local spec). Parent issue: david-driscoll/vault#105.

Also pushed a second commit here, found while validating bradygaster#627: don't advance an adoption floor over an unanswered human comment. Applying the "reset commentWatch.since to merge time" guidance blindly on the live estate would have permanently silenced an unanswered operator instruction — the gap held four comments, all genuine replies and zero agent comments, because the crew was already signing by then. The guard protects against unmarked agent comments; enumerate the window before moving the floor.