Skip to content

ci: opt-in Claude deep review via deep-review label (#273) #278

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
allxsmith merged 2 commits into from
Jul 11, 2026
Merged

ci: opt-in Claude deep review via deep-review label (#273) #278

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
f9a919b
ci: opt-in Claude deep review via deep-review label (#273)
claude Jul 10, 2026
528ec2d
docs: rewrap deep-review sentence in CLAUDE.md to paragraph width
claude Jul 11, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
92 changes: 73 additions & 19 deletions .github/workflows/claude-review.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,15 @@ name: Claude Deep Review
# claude-pr-loop.yml's verify mode and resolves only what is genuinely fixed
# (mirroring CodeRabbit's "✅ Addressed" verification).
#
# Second trigger path: a triage+ collaborator applies the `deep-review`
# label to ANY same-repo PR (any branch) to run the same one-shot review —
# e.g. Claude-Code-authored but human-opened PRs that never enter the loop.
# Security posture: same-repo PRs only (never forks), the labeler's live
# role is re-verified via the collaborators API (triage+ required, so a
# label applied by automation on behalf of an outsider is a silent no-op),
# and the dedupe marker still caps usage at one review per PR ever —
# re-labeling never re-fires it.
#
# Deliberately does NOT trigger on `synchronize`: one review per PR, so
# Claude's own fix pushes can never re-fire this workflow (no reviewer/fixer
# ping-pong, no claude-code-action#1299-style self-trigger failures).
Expand All @@ -27,25 +36,68 @@ concurrency:

jobs:
review:
# Only AI-loop PRs from claude/* branches in this repo. For `labeled`
# events, only the ai-loop label itself may fire the review.
# Two ways in, both same-repo PRs only: (a) AI-loop PRs from claude/*
# branches, or (b) any PR a triage+ user opts in via the `deep-review`
# label (role re-verified live in the perm step below). For `labeled`
# events, only the matching label itself may fire the review.
if: |
contains(github.event.pull_request.labels.*.name, 'ai-loop') &&
(github.event.action != 'labeled' || github.event.label.name == 'ai-loop') &&
github.event.pull_request.head.repo.full_name == github.repository &&
startsWith(github.event.pull_request.head.ref, 'claude/') &&
github.event.pull_request.state == 'open' &&
vars.AI_LOOP_ENABLED == 'true'
vars.AI_LOOP_ENABLED == 'true' &&
(
(
contains(github.event.pull_request.labels.*.name, 'ai-loop') &&
startsWith(github.event.pull_request.head.ref, 'claude/') &&
(github.event.action != 'labeled' || github.event.label.name == 'ai-loop')
) ||
(
contains(github.event.pull_request.labels.*.name, 'deep-review') &&
(github.event.action != 'labeled' || github.event.label.name == 'deep-review')
)
)
runs-on: ubuntu-latest
timeout-minutes: 30
steps:
# For the opt-in `deep-review` label only: GitHub already restricts
# labeling to triage+, but re-verify the labeler's live role
# defensively (same belt-and-braces pattern as on-slop.yml and
# claude-implement.yml) so an automated/template label can never spend
# Claude usage. Every other event shape is the unchanged ai-loop trust
# model and passes through as allowed.
- name: Check labeler permission
id: perm
env:
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
REPO: ${{ github.repository }}
SENDER: ${{ github.event.sender.login }}
ACTION: ${{ github.event.action }}
LABEL: ${{ github.event.label.name }}
run: |
set -euo pipefail
if [ "$ACTION" != "labeled" ] || [ "$LABEL" != "deep-review" ]; then
echo "allowed=true" >> "$GITHUB_OUTPUT"
echo "not a deep-review labeling event — no extra check needed"
exit 0
fi
ROLE=$(gh api "repos/$REPO/collaborators/$SENDER/permission" \
--jq '.role_name' || echo none)
case "$ROLE" in
admin|maintain|write|triage)
echo "allowed=true" >> "$GITHUB_OUTPUT"
echo "$SENDER has $ROLE access — running the deep review" ;;
*)
echo "allowed=false" >> "$GITHUB_OUTPUT"
echo "$SENDER lacks triage access ($ROLE) — not running" ;;
esac

# `opened` and `labeled` both fire when a PR is created with the label
# already attached — review exactly once. The concurrency group
# serializes the double-fire; the marker probe catches the second run.
# (The marker, not just any claude[bot] activity: the fix agent posts
# claude[bot] thread replies too, which must not count as "reviewed".)
- name: Skip if already reviewed
id: dedupe
if: steps.perm.outputs.allowed == 'true'
env:
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
REPO: ${{ github.repository }}
Expand All @@ -63,34 +115,35 @@ jobs:
fi

- name: Checkout PR head
if: steps.dedupe.outputs.skip == 'false'
if: steps.perm.outputs.allowed == 'true' && steps.dedupe.outputs.skip == 'false'
uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7
with:
ref: ${{ github.event.pull_request.head.ref }}
fetch-depth: 0
persist-credentials: false

- name: Setup pnpm
if: steps.dedupe.outputs.skip == 'false'
if: steps.perm.outputs.allowed == 'true' && steps.dedupe.outputs.skip == 'false'
uses: pnpm/action-setup@b906affcce14559ad1aafd4ab0e942779e9f58b1 # v4

- name: Setup Node.js
if: steps.dedupe.outputs.skip == 'false'
if: steps.perm.outputs.allowed == 'true' && steps.dedupe.outputs.skip == 'false'
uses: actions/setup-node@48b55a011bda9f5d6aeb4c2d9c7362e8dae4041e # v6
with:
node-version: '24'
cache: 'pnpm'

- name: Install dependencies
if: steps.dedupe.outputs.skip == 'false'
if: steps.perm.outputs.allowed == 'true' && steps.dedupe.outputs.skip == 'false'
run: pnpm install --frozen-lockfile

- name: Run Claude (deep review)
if: steps.dedupe.outputs.skip == 'false'
if: steps.perm.outputs.allowed == 'true' && steps.dedupe.outputs.skip == 'false'
uses: anthropics/claude-code-action@51c47629174b5dbca955cf93690fd39c8a41dadf # v1
with:
claude_code_oauth_token: ${{ secrets.CLAUDE_CODE_OAUTH_TOKEN }}
# The PR opener/labeler is claude[bot] (the implementer) by design.
# On loop PRs the opener/labeler is claude[bot] (the implementer)
# by design.
allowed_bots: 'claude'
claude_args: |
--max-turns 120
Expand All @@ -100,9 +153,10 @@ jobs:
REPO: ${{ github.repository }}
PR NUMBER: ${{ github.event.pull_request.number }}

You are the independent DEEP REVIEWER for this AI-authored PR.
The PR branch is checked out and dependencies are installed. You
did not write this code — review it adversarially.
You are the independent DEEP REVIEWER for this PR (typically
AI-authored or AI-assisted). The PR branch is checked out and
dependencies are installed. You did not write this code — review
it adversarially.

1. Read the diff (`gh pr diff`) and the changed files in full.
2. Verify every candidate finding against the actual code before
Expand Down Expand Up @@ -138,10 +192,10 @@ jobs:
- Include a ```mermaid diagram when a control-/data-flow picture
makes the bug clearer than prose (GitHub renders it).
5. ALWAYS finish — findings or not — by submitting ONE summary
review that certifies the deep review ran (the loop's gate
requires it before it may hand the PR to a human). Write the
body to a file first so the Markdown/tables survive intact,
then post it:
review that certifies the deep review ran (for AI-loop PRs,
the loop's gate requires it before it may hand the PR to a
human). Write the body to a file first so the Markdown/tables
survive intact, then post it:

(write this to /tmp/deep-review.md, filling in the values)
<!-- claude-deep-review -->
Expand Down
10 changes: 6 additions & 4 deletions CLAUDE.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -86,7 +86,9 @@ Issues labeled `claude-fix` (requires triage+ access, verified live) are impleme
`ai-loop`, CodeRabbit + a Claude deep review comment on it, and `claude-pr-loop.yml` drives
fix/verify rounds (cap 4) until CI is green and every AI review thread is resolved. Labels:
`ai-loop` (in the loop), `needs-human-review` (converged or contested — owner reviews and
squash-merges manually; the loop never merges), `ai-loop-paused` (cap/guard hit). Kill switches:
remove `ai-loop` (per PR) or set repo variable `AI_LOOP_ENABLED=false` (whole system). The
`<!-- ai-loop-state … -->` PR comment is machine-managed — never reformat its first line. The
loop refuses PRs that touch `.github/**` or the jest/commitlint/release/pnpm-workspace configs.
squash-merges manually; the loop never merges), `ai-loop-paused` (cap/guard hit). A triage+
user can also apply the opt-in `deep-review` label to any PR to run the one-shot Claude
deep review on it (once ever per PR). Kill switches: remove `ai-loop` (per PR) or set repo
Comment on lines +90 to +91

@coderabbitai coderabbitai Bot Jul 11, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🎯 Functional Correctness | 🟡 Minor | ⚡ Quick win

Document the same-repository restriction

“Any PR” is broader than the workflow contract: fork PRs are intentionally rejected. Clarify that deep-review applies to same-repository PRs so users do not expect labeling a fork PR to trigger a review.

Suggested wording 
- A triage+ user can also apply the opt-in `deep-review` label to any PR to run the one-shot Claude
+ A triage+ repository collaborator can also apply the opt-in `deep-review` label to any
+ same-repository PR to run the one-shot Claude
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
user can also apply the opt-in `deep-review` label to any PR to run the one-shot Claude
deep review on it (once ever per PR). Kill switches: remove `ai-loop` (per PR) or set repo
A triage+ repository collaborator can also apply the opt-in `deep-review` label to any
same-repository PR to run the one-shot Claude
deep review on it (once ever per PR). Kill switches: remove `ai-loop` (per PR) or set repo
🧰 Tools
🪛 LanguageTool

[style] ~90-~90: Consider a different adjective to strengthen your wording.
Context: ...el to any PR to run the one-shot Claude deep review on it (once ever per PR). Kill s...

(DEEP_PROFOUND)

🤖 Prompt for AI Agents 
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@CLAUDE.md` around lines 90 - 91, Clarify the `deep-review` documentation to
state that it applies only to same-repository PRs, not pull requests originating
from forks. Update the sentence describing labeling any PR while preserving the
one-shot behavior and existing kill-switch details.

variable `AI_LOOP_ENABLED=false` (whole system). The `<!-- ai-loop-state … -->` PR comment
is machine-managed — never reformat its first line. The loop refuses PRs that touch
`.github/**` or the jest/commitlint/release/pnpm-workspace configs.
13 changes: 7 additions & 6 deletions docs/docs/guides/getting-started/ai-development.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -47,12 +47,13 @@ the review-time requirements (Storybook story for UI changes, docs page for API

PRs authored by the loop move through a small label lifecycle:

| Label | Where | Meaning |
| -------------------- | ------ | ----------------------------------------------------------------------------- |
| `claude-fix` | issues | Maintainer-approved: Claude implements this issue and opens a PR |
| `ai-loop` | PRs | The PR is inside the autonomous review/fix loop |
| `needs-human-review` | PRs | The loop converged (or hit a disagreement) — awaiting maintainer review/merge |
| `ai-loop-paused` | PRs | The loop hit its iteration cap or a guardrail — a maintainer must intervene |
| Label | Where | Meaning |
| -------------------- | ------ | ----------------------------------------------------------------------------------------------------- |
| `claude-fix` | issues | Maintainer-approved: Claude implements this issue and opens a PR |
| `ai-loop` | PRs | The PR is inside the autonomous review/fix loop |
| `needs-human-review` | PRs | The loop converged (or hit a disagreement) — awaiting maintainer review/merge |
| `ai-loop-paused` | PRs | The loop hit its iteration cap or a guardrail — a maintainer must intervene |
| `deep-review` | PRs | Opt-in: a triage+ user applies it to run the one-shot Claude deep review on any PR (once ever per PR) |

The loop itself: Claude implements the issue and opens the PR → CodeRabbit reviews it and a
second, independent Claude review (a stronger model than the implementer) does a deep pass →
Expand Down
Loading