Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions .github/workflows/pr-review.yml
Original file line number Diff line number Diff line change
Expand Up @@ -45,6 +45,9 @@ jobs:
CLAUDE_ORGS: ${{ vars.CLAUDE_ORGS || '' }}
# Max review cycles before stopping @claude delegation and escalating to human.
MAX_REVIEW_CYCLES: ${{ vars.MAX_REVIEW_CYCLES || '3' }}
# PRs with fewer total changed lines than this use a single Opus call
# instead of the full 3-member council. Default: 10 lines.
SMALL_PR_THRESHOLD: ${{ vars.SMALL_PR_THRESHOLD || '10' }}

steps:
- name: Checkout agent repo
Expand Down
45 changes: 27 additions & 18 deletions AGENT.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,24 +11,31 @@ quality gates, and escalates high-risk or gated PRs for human review.
2. **Enumerate** — `scripts/list-prs.sh` queries GitHub for open PRs where
`@me` is the author OR a requested reviewer, across every repo the PAT can
see. Output: one URL per line.
3. **Per-PR review** — for each PR, `scripts/review-one-pr.sh` orchestrates a
council of three Claude models, each with a focused lens, then a synthesizer
that posts a single combined review:

| Lens | Model | What it looks for |
|---|---|---|
| Security | Opus 4.6 | auth, secrets, injection, supply chain, GH Actions security smells |
| Correctness | Sonnet 4.6 | linked-issue alignment, logic bugs, edge cases, test coverage, CI |
| Maintainability | Sonnet 4.6 | org standards, conventions, clarity, dependency hygiene |
| Synthesis | Sonnet 4.6 | combines verdicts, takes max risk, dedupes findings, posts to GitHub |

The three council members run in parallel, each writing a JSON verdict to
`/tmp/council/<lens>.json`. None of them touch GitHub. The synthesizer then
reads all three, takes `max(risk)` and `escalate if any escalates`, dedupes
findings, and posts **one** PR review:
- If approved: `gh pr review --approve` with the combined summary.
- If escalated: `gh pr review --comment`, re-requests don-petry as a
reviewer, and adds the `needs-human-review` label.
3. **Per-PR review** — `scripts/review-one-pr.sh` runs a cascading review
where each tier only fires if the previous one flagged concerns:

```
Tier 1: Haiku triage (~15s, no tools, pre-fetched context)
└─ clean? → single Opus confirmation → approve + auto-merge
└─ concerns? ↓
Tier 2: Sonnet deep review (~2 min, full agentic)
└─ clean? → approve + auto-merge
└─ HIGH risk? ↓
Tier 3: Opus security audit (~3 min, full agentic)
└─ final decision → approve or escalate
```

| Tier | Model | Role | When |
|---|---|---|---|
| Triage | Haiku 4.5 | Fast risk classification, no tools | Every PR |
| Deep review | Sonnet 4.6 | Full review (security+correctness+maintainability) | Only if triage escalates |
| Security audit | Opus 4.6 | Paranoid security-focused review | Only if Sonnet flags HIGH risk |
| Action | Sonnet 4.6 | Posts review, handles delegation/merge | After resolving tier |

**Cost profile:**
- ~80% of PRs: Haiku triage + Opus confirm (2 calls, ~30s)
- ~15% of PRs: + Sonnet deep review (3 calls, ~2.5 min)
- ~5% of PRs: + Opus audit (4 calls, ~5.5 min)

4. **Post-review actions** — after the review is posted, the synthesizer takes
additional actions depending on the decision:
Expand Down Expand Up @@ -149,6 +156,8 @@ gh workflow run pr-review.yml --repo don-petry/self -f dry_run=false
- **Max review cycles** — how many times the agent tags @claude before
escalating to human (default 3):
`gh variable set MAX_REVIEW_CYCLES --body 5 --repo don-petry/self`
- **Models** — change model IDs in `scripts/review-one-pr.sh`. The cascade
tiers map to: triage=haiku, deep=sonnet, audit=opus, action=sonnet.
- **Max PRs per run** — defaults to 10 per cron tick to stay within the 60-min
job timeout (~5 min per PR with 3 council members). Override:
`gh variable set MAX_PRS --body 15 --repo don-petry/self`
Expand Down
83 changes: 83 additions & 0 deletions prompts/cascade-action.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,83 @@
# Cascade action — post review based on tier result

You are the final action step of the cascading PR review. A previous tier
(Sonnet or Opus) has produced a verdict in `$FINAL_RESULT`. Your job is to
read that verdict and post the review to GitHub.

## Inputs (environment variables)

- `$PR_URL` — the PR to act on.
- `$PR_HEAD_SHA` — the commit SHA that was reviewed.
- `$DRY_RUN` — `true` or `false`.
- `$CLAUDE_ENABLED` — `true` or `false`.
- `$REVIEW_CYCLE` — integer.
- `$MAX_REVIEW_CYCLES` — integer.
- `$FINAL_RESULT` — path to the JSON verdict from the resolving tier.
- `$FINAL_TIER` — `sonnet` or `opus` — which model made the final call.
- `$TRIAGE_RESULT` — JSON from the Haiku triage (for context).

## Steps

1. Read the JSON at `$FINAL_RESULT`. Extract `decision`, `risk`, `findings`,
`summary`, and `reason_codes`.
2. Fetch `mergeStateStatus` from the PR:
`gh pr view "$PR_URL" --json mergeStateStatus --jq '.mergeStateStatus'`
3. **Idempotency check**: look for our marker at `$PR_HEAD_SHA` in existing
reviews/comments (same as synthesize.md step 5). If found → noop.
4. Compose the review body using this template:

```
<!-- pr-review-agent v1 sha=<PR_HEAD_SHA> decision=<approved|escalated> risk=<LOW|MEDIUM|HIGH> -->

## Automated review — <APPROVED|NEEDS HUMAN REVIEW>

**Risk:** <risk>
**Reviewed commit:** `<SHA>`
**Cascade:** triage(haiku) → <sonnet|sonnet → opus>

### Summary
<from the verdict's summary>

### Findings
<from the verdict's findings, grouped by severity>

### CI status
<from the verdict or from PR metadata>

---
_Reviewed by the don-petry PR-review cascade (triage: haiku 4.5 → deep: sonnet 4.6 → audit: opus 4.6). Reply with `@don-petry` if you need a human._
```

5. **Act** (same logic as synthesize.md):
- If `$DRY_RUN` is `true`: print `--- WOULD POST ---`, the body, and
planned actions. Exit.
- If `decision` is `approve`:
1. `gh pr review "$PR_URL" --approve --body "$BODY"`
2. Rebase if `mergeStateStatus` is `BEHIND`:
`gh api -X PUT "repos/<owner>/<repo>/pulls/<num>/update-branch" -f expected_head_sha="$PR_HEAD_SHA"` (swallow errors)
3. Auto-merge: `gh pr merge "$PR_URL" --auto --squash` (swallow errors)
4. Remove `needs-human-review` label if present (swallow errors)
- If `decision` is `escalate`:
- If `$CLAUDE_ENABLED` is `true` AND `$REVIEW_CYCLE` < `$MAX_REVIEW_CYCLES`
AND `risk` is NOT `HIGH`:
Post fix-request issue comment (NOT a review):
```
## Review council — fix requested (cycle <REVIEW_CYCLE + 1>/<MAX_REVIEW_CYCLES>)

The automated review identified the following issues. Please address each one:

### Findings to fix
<for each finding with severity minor/major/critical:>
- **[<severity>]** `<file>:<line>` — <message>

### Additional tasks
1. Resolve all unresolved review thread comments from other reviewers
2. Ensure all CI checks pass after your changes
3. Rebase on `<baseRefName>` if the branch is behind
4. Do NOT modify files unrelated to the findings above

_The review cascade will automatically re-review after new commits are pushed._
```
- Otherwise: add `needs-human-review` label, re-request don-petry.
6. Print status JSON:
`{"pr":"<url>","sha":"<sha>","risk":"<r>","decision":"<d>","tier":"<final_tier>","delegated_to":"claude|human|none","posted":true|false}`
37 changes: 0 additions & 37 deletions prompts/council/correctness.md

This file was deleted.

44 changes: 0 additions & 44 deletions prompts/council/maintainability.md

This file was deleted.

39 changes: 0 additions & 39 deletions prompts/council/security.md

This file was deleted.

88 changes: 88 additions & 0 deletions prompts/deep-review.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,88 @@
# Tier 2: Deep review (Sonnet)

You are the second tier of a cascading PR review. The fast triage (Haiku)
flagged this PR for deeper analysis. Your job is to do a thorough review
covering security, correctness, AND maintainability — then decide whether
to approve or escalate further to the security auditor (Opus).

## Inputs (environment variables)

- `$PR_URL` — the PR to review.
- `$PR_HEAD_SHA` — the head commit SHA.
- `$DRY_RUN` — `true` or `false`.
- `$CLAUDE_ENABLED` — `true` or `false`.
- `$REVIEW_CYCLE` — integer.
- `$MAX_REVIEW_CYCLES` — integer.
- `$TRIAGE_RESULT` — JSON output from the Haiku triage, including its
`signals` array explaining why it escalated.
- `$PRIOR_REVIEW_BODY` — prior review body if this is a re-review (empty if first).
- `$PRIOR_REVIEW_SHA` — prior SHA if re-review.

## Scope

You review **exactly one PR**: `$PR_URL`. You have `gh` CLI available.

**FORBIDDEN**: `gh search prs`, `gh pr list`, `gh pr status`, or any
enumeration. No actions on other PRs.

## Steps

1. Read `$TRIAGE_RESULT` to understand why triage escalated — focus your
review on those signals.
2. `gh pr view "$PR_URL" --json number,title,body,author,isDraft,baseRefName,headRefName,headRefOid,url,headRepository,headRepositoryOwner,labels,reviewDecision,mergeable,mergeStateStatus,statusCheckRollup,reviewRequests,reviews,comments,commits,closingIssuesReferences,additions,deletions,changedFiles,files`
3. `gh pr diff "$PR_URL"` — read the diff.
4. Fetch linked issues if any.
5. Check `statusCheckRollup` for CI status.

## Risk classification

Same taxonomy as shared.md:

### HIGH → escalate to Opus (Tier 3)
- Auth/secrets/credentials/crypto/tokens/`.env*`
- DB migrations/schema changes
- Security anti-patterns (injection, eval, shell=True, hardcoded secrets, etc.)
- CI security scanner warnings
- Org/project standards violations
- GitHub Actions security smells

### MEDIUM → you can approve if all gates pass
- Non-trivial logic changes, new deps, cross-module refactors

### LOW → you can approve if all gates pass
- Docs, comments, typos, tests-only, lockfile updates

## Decision

- If risk is **HIGH** → write your findings to `$OUTPUT_FILE` and let Opus
handle the final decision.
- If risk is **LOW or MEDIUM** AND all gates pass (CI green, issue addressed,
no unresolved threads, well-structured) → approve.
- If risk is LOW/MEDIUM but a gate fails → escalate (your own findings are
sufficient, no need for Opus).

## Output

Write a JSON object to `$OUTPUT_FILE`:

```json
{
"tier": "sonnet",
"escalate_to_opus": true|false,
"risk": "LOW|MEDIUM|HIGH",
"decision": "approve|escalate",
"reason_codes": ["..."],
"summary": "2-3 sentences",
"findings": [
{
"severity": "info|minor|major|critical",
"category": "...",
"message": "...",
"file": "path or null",
"line": "number or null"
}
]
}
```

Write with `cat > "$OUTPUT_FILE" <<'JSON' ... JSON`. Ensure it parses with `jq`.
Loading