Repository navigation
ci: opt-in AI triage via ai-triage label (#274) - #277
Conversation
|
Caution Review failedThe pull request is closed. ℹ️ Recent review info⚙️ Run configurationConfiguration used: Path: .coderabbit.yaml Review profile: CHILL Plan: Pro Plus Run ID: 📒 Files selected for processing (2)
WalkthroughAdds an opt-in ChangesAI triage automation
Estimated code review effort: 4 (Complex) | ~45 minutes Sequence Diagram(s)sequenceDiagram
participant Maintainer as Authorized labeler
participant GitHub as GitHub Actions
participant PermissionAPI as Collaborator permission API
participant Claude as Claude triage action
participant Target as PR or issue
Maintainer->>GitHub: Apply ai-triage label
GitHub->>PermissionAPI: Verify collaborator permission
PermissionAPI-->>GitHub: Return allowed role
GitHub->>Claude: Start constrained triage session
Claude->>Target: Read target and search related items
Claude->>Target: Post one Markdown comment
GitHub->>Target: Remove ai-triage label
Possibly related issues
Possibly related PRs
🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✨ Finishing Touches🧪 Generate unit tests (beta)
Comment |
Preview DeploymentPreview URL: https://433d1bf7.bestax.pages.dev |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 2
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In @.github/workflows/ai-triage.yml:
- Around line 64-75: Remove the permission-output condition from the cleanup
step identified near the label-removal logic, so cleanup runs even when the
collaborator permission check in the workflow’s permission-evaluation step fails
and produces allowed=false. Preserve the existing label removal behavior and
other necessary guards, ensuring ai-triage is always cleaned up after
processing.
- Around line 48-52: Restrict the AI triage workflow’s write capabilities:
remove pull-requests: write and issues: write from the permissions used by the
agent, and prevent its Bash gh api access from issuing arbitrary mutations. Move
comment and label-removal operations into a separate deterministic workflow
step, or explicitly limit gh api to the required read and label-delete endpoints
while preserving only the minimum permissions needed.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: CHILL
Plan: Pro Plus
Run ID: ff27d159-1745-4e29-b417-eb34b4790dce
📒 Files selected for processing (3)
.github/workflows/ai-triage.ymlCLAUDE.mddocs/docs/guides/getting-started/ai-development.md
| permissions: | ||
| contents: read # checkout of the base default branch | ||
| pull-requests: write # post the triage comment on PRs; remove the label | ||
| issues: write # post the triage comment on issues; remove the label | ||
| id-token: write # OIDC exchange for the Claude OAuth token |
There was a problem hiding this comment.
🔒 Security & Privacy | 🟠 Major | 🏗️ Heavy lift
🧩 Analysis chain
🏁 Script executed:
#!/bin/bash
set -euo pipefail
# Inspect the workflow around the cited lines and any nearby triage/job config.
git ls-files '.github/workflows/ai-triage.yml' '.github/**' | sed -n '1,200p'
echo '--- ai-triage.yml ---'
cat -n .github/workflows/ai-triage.yml | sed -n '1,220p'
echo '--- search for gh api / tool config / permissions in github workflows ---'
rg -n --hidden --glob '.github/**' -e 'gh api' -e 'permissions:' -e 'id-token: write' -e 'pull-requests: write' -e 'issues: write' -e 'claude-code' -e 'tools?' .githubRepository: allxsmith/bestax
Length of output: 23452
Restrict the triage agent’s write surface. Bash(gh api:*) plus pull-requests: write / issues: write lets untrusted PR or issue content steer the model into arbitrary mutating REST calls. Keep the agent on read-only tools and move comment/label writes into a separate deterministic step, or narrow gh api to only the exact read and label-delete endpoints.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In @.github/workflows/ai-triage.yml around lines 48 - 52, Restrict the AI triage
workflow’s write capabilities: remove pull-requests: write and issues: write
from the permissions used by the agent, and prevent its Bash gh api access from
issuing arbitrary mutations. Move comment and label-removal operations into a
separate deterministic workflow step, or explicitly limit gh api to the required
read and label-delete endpoints while preserving only the minimum permissions
needed.
Preview DeploymentPreview URL: https://9c1b81f3.bestax.pages.dev |
|
@claude perform deep review Generated by Claude Code |
|
Claude encountered an error after 2m 44s —— View job Deep review in progress
|
Add .github/workflows/ai-triage.yml: labeling a PR or issue `ai-triage` (triage+ only, live-verified) runs one sonnet session that comments with related issues and likely duplicates, then removes the label so it acts as a re-pressable button. Document the label in the AI development docs table and CLAUDE.md. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Pohc8xLkdx4gwXkW3xd7up
Drop Bash(gh api:*) from --allowedTools: the prefix also matches POST/PATCH/DELETE and the session ingests untrusted issue/PR text, so an injected agent could mutate labels or machine-managed comments. Point the prompt's gh api hints at gh search / gh pr diff instead and note the GET-only posture in the header. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Pohc8xLkdx4gwXkW3xd7up
12d4091 to
cc49920
Compare
Preview DeploymentPreview URL: https://e1c1d61f.bestax.pages.dev |
There was a problem hiding this comment.
Actionable comments posted: 1
🧹 Nitpick comments (1)
docs/docs/guides/getting-started/ai-development.md (1)
50-57: 🎯 Functional Correctness | 🔵 Trivial | ⚡ Quick win
ai-triagerow omits the triage+ access requirement.Every other opt-in row (
deep-review) states who can trigger it. This row doesn't mention the triage+ permission gate thatai-triage.ymlenforces, and that CLAUDE.md's parallel description explicitly calls out ("triage+ only"). Worth aligning for consistency and accuracy.Proposed fix
-| `ai-triage` | PRs & issues | Opt-in: one-shot AI triage (related issues + duplicates); the label auto-removes after the run | +| `ai-triage` | PRs & issues | Opt-in: a triage+ user applies it to run one-shot AI triage (related issues + duplicates); the label auto-removes after the run |As per path instructions, "Check MDX/Markdown correctness, working internal links, and that documented APIs match the library."
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@docs/docs/guides/getting-started/ai-development.md` around lines 50 - 57, Update the ai-triage entry in the label table to explicitly state that it is opt-in for triage+ users only, matching the permission enforced by ai-triage.yml and the parallel description in CLAUDE.md.Source: Path instructions
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In @.github/workflows/ai-triage.yml:
- Around line 148-158: Remove the permission-result gate from the “Remove
ai-triage label” step in the workflow, retaining only always() so cleanup runs
even when the permission check fails. Keep the existing repository, issue/PR
number, token environment, and tolerant gh api deletion unchanged.
---
Nitpick comments:
In `@docs/docs/guides/getting-started/ai-development.md`:
- Around line 50-57: Update the ai-triage entry in the label table to explicitly
state that it is opt-in for triage+ users only, matching the permission enforced
by ai-triage.yml and the parallel description in CLAUDE.md.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: CHILL
Plan: Pro Plus
Run ID: df4159b6-53ff-4e9e-a290-1802fddb9b1a
📒 Files selected for processing (3)
.github/workflows/ai-triage.ymlCLAUDE.mddocs/docs/guides/getting-started/ai-development.md
![claude[bot]](https://avatars.githubusercontent.com/in/1236702?s=60&v=4){kind=small}
There was a problem hiding this comment.
Deep review — 0 finding(s)
| # | Severity | Area | Finding | Location |
|---|---|---|---|---|
| — | — | — | No blocking defects found. | — |
Overall: This is a well-constructed, opt-in triage workflow plus matching docs. The security posture is sound — it never checks out PR head code (base default branch only, so pull_request_target is safe), the two-layer gate (label + User + AI_LOOP_ENABLED job if, then a live collaborators-API role re-check) mirrors the three proven sibling workflows exactly, and the Claude tool allowlist is GET-only gh commands plus the two comment commands. Event handling is correct: NUMBER/IS_PR/concurrency all resolve properly for both pull_request_target and issues, PR/issue numbers are globally unique so no group collision, and the always() remove-label step keeps the button re-pressable even on failure. The only edge worth a human's eye — not a defect, effectively unreachable since GitHub restricts labeling to triage+ — is that a false result from the live perm re-check skips label removal, leaving the label applied; that's a reasonable "flag the anomaly" behavior. Docs and CLAUDE.md are updated in the same PR and there's no component/API/skills surface, so no story/skills obligation applies.
🏄 Clean sets rolling in on this one, dude — same lineup the other AI workflows already surf, no sketchy head-code checkout, gates locked in tight. Paddle it out, it's good to go.
) Addresses the CodeRabbit review on PR #277: the cleanup step's permission gate contradicted its own "never wedges on" comment — a transient collaborators-API failure would leave the label stuck with no way to re-fire via a labeled event. Removal uses only GITHUB_TOKEN, so dropping the gate spends no Claude usage. Also states the triage+ requirement in the docs label table, matching the deep-review row. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0142miDJt2AX9QUw5fNJt2xu
Preview DeploymentPreview URL: https://bad2f10d.bestax.pages.dev |
# Conflicts: # docs/docs/guides/getting-started/ai-development.md
Preview DeploymentPreview URL: https://20efa3f4.bestax.pages.dev |
There was a problem hiding this comment.
Deep review — 0 finding(s)
| # | Severity | Area | Finding | Location |
|---|---|---|---|---|
| – | – | – | No blocking defects found. | – |
Overall: This is a tightly-scoped, well-reasoned GitHub Actions workflow that faithfully mirrors the repo's established on-slop.yml / claude-review.yml / bestaxbot-reply.yml patterns. I checked the parts most likely to break: the two-layer permission gate (job if: + live collaborators/.../permission recheck) is sound; pull_request_target is safe because only the base default branch is checked out with persist-credentials: false and no project code ever runs; the concurrency group can't collide since GitHub issue/PR numbers share one namespace; the Claude allowlist is genuinely read-only + comment-only (no gh api), keeping any prompt-injection blast radius to a single comment; and the if: always() label removal via the issues API correctly covers both PRs and issues with GITHUB_TOKEN-only auth. The only non-blocking wart worth a human's awareness: when AI_LOOP_ENABLED != 'true' the whole job is skipped, so an applied ai-triage label is never auto-removed and stays stuck until a maintainer clears it — arguably intended (system off means nothing fires), but a maintainer may want the cleanup to run regardless. Docs (label table + CLAUDE.md) are updated; no runnable-code surface, so no tests apply.
🏄 Clean sets rolling in on this one, dude — solid two-layer break protection, no gnarly pull_request_target wipeouts, and the label just resets like a re-pressable button. Only ripple is a label that won't paddle back out if the kill switch is off, but that's mellow. Ship it. 🌊
# Conflicts: # CLAUDE.md # docs/docs/guides/getting-started/ai-development.md
Preview DeploymentPreview URL: https://25c87afa.bestax.pages.dev |
|
🎉 This PR is included in version 3.2.0 🎉 The release is available on: Your semantic-release bot 📦🚀 |
…des (#292) PR A of #289: Bun-parity fan-out triage. Always-on (config-driven) with a fail-closed daily budget counter on tracking issue #290, label mode kept, .claude/commands/ established (triage-dedupe, triage-find-issues, triage-find-duplicate-prs, pre-pr), docs updated. Refs #289 See #269, #274, #277
|
🎉 This PR is included in version 5.4.1 🎉 The release is available on: Your semantic-release bot 📦🚀 |
|
🎉 This PR is included in version 1.0.0 🎉 The release is available on: Your semantic-release bot 📦🚀 |
|
🎉 This PR is included in version 1.0.0 🎉 The release is available on: Your semantic-release bot 📦🚀 |
2 participants
Pull Request
Description
Adds an opt-in, label-triggered AI triage workflow (
.github/workflows/ai-triage.yml,name: AI Triage) — an adaptation of oven-sh/bun's always-onclaude-find-issues-for-pr.yml+claude-dedupe-issues.yml. A triage+ maintainer applies theai-triagelabel to a PR or issue; one cheap sonnet session searches the repo for related issues and likely duplicates, posts exactly one summary comment, and the label is auto-removed so it works as a re-pressable button.@allxsmith/bestax-bulma)create-bestax)@allxsmith/bestax-docs).github/workflows/ai-triage.yml) + docsKey properties, mirroring the house patterns in
on-slop.yml/claude-review.yml/bestaxbot-reply.yml:if:requires theai-triagelabel from a real user withAI_LOOP_ENABLED == 'true', and a livegh api .../collaborators/.../permissionre-check requires admin/maintain/write/triage — otherwise silent no-op. All later steps gate on that output.pull_request_target: only the base repo's default branch is checked out (never the PR head),persist-credentials: false, pinnedactions/checkoutSHA. No node/pnpm setup, no install — the job never runs project code.claude-code-actionSHA as the other workflows, OAuth token only (posts as claude[bot]),--max-turns 30,--model claude-sonnet-5,--allowedToolsrestricted to read-onlyghplusgh pr comment/gh issue comment. Prompt hard rules: one comment only, never label/close/merge/push/resolve, never write "@claude"/"@coderabbitai".if: always()step (still gated on the permission check) deletes the label via the issues API — which covers both PRs and issues — so the button never wedges "on".permissions: {}; job takescontents: read,pull-requests: write,issues: write,id-token: write(mirroringclaude-review.yml's OAuth-exchange needs).Docs: one new row in the label table in
docs/docs/guides/getting-started/ai-development.mdand one sentence in rootCLAUDE.md's "AI development loop" section.Related Issue(s)
Closes #274
Type of Change
Checklist
CLAUDE.mdfiles are updatedScreenshots / Demos
N/A — CI workflow. Flow: apply
ai-triage→ one triage comment (### AI triagewith duplicates/related lists or "No related issues or duplicates found.") → label auto-removed.Additional Context
ai-development.md, so trivial merge conflicts in that table are expected and easy to resolve (prettier realigned the table columns to fit the new row's width).ai-triagelabel: color#0e8a16, description "Run one-shot AI triage: related issues + duplicates (sonnet; triage+ only)".AI_LOOP_ENABLED=false; each run costs exactly one human-gated sonnet session.Bash(gh api:*)was removed because it also matches POST/PATCH/DELETE and the session ingests untrusted issue/PR text.🤖 Generated with Claude Code
https://claude.ai/code/session_01Pohc8xLkdx4gwXkW3xd7up
Summary by CodeRabbit
ai-triagelabel application for both issues and pull requests.ai-triagelabel is automatically removed after completion to allow re-running.ai-triagelabel lifecycle and behavior.