ci: add Claude Code Review action (ticket-alignment + Subconscious rubric)#105
ci: add Claude Code Review action (ticket-alignment + Subconscious rubric)#105aviyashchin wants to merge 2 commits into
Conversation
…bric) Adds (or upgrades) an automated PR reviewer that: - reads the linked Linear ticket from the linear-linkback PR comment and maps the diff 1:1 to acceptance criteria - applies a concrete Karpathy/harness rubric (no new abstractions with <2 callers, no new files <30 LOC, no speculative flags, no premature optimization, no committed TODOs) - applies Subconscious-specific checks (customer data integrations, prediction integrity, SOC 2/GDPR surface, secrets, public endpoints) - always posts exactly one comment per run (silence forbidden) Requires CLAUDE_CODE_OAUTH_TOKEN repo secret to function. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
WalkthroughA GitHub Actions workflow is added that automatically triggers Claude AI code reviews on pull requests. The workflow checks out repository history, invokes the Claude code review action with configured credentials, and posts formatted review comments containing ticket alignment, rubric discipline checks, Subconscious-specific risk assessments, and test coverage guidance. ChangesClaude Code Review Workflow
Poem
🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Review rate limit: 0/1 reviews remaining, refill in 60 minutes.Comment |
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: e6f406a901
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| id: claude-review | ||
| uses: anthropics/claude-code-action@v1 | ||
| with: | ||
| claude_code_oauth_token: ${{ secrets.CLAUDE_CODE_OAUTH_TOKEN }} |
There was a problem hiding this comment.
This workflow unconditionally requires secrets.CLAUDE_CODE_OAUTH_TOKEN, but it runs on every pull_request; on forked/Dependabot PRs GitHub does not pass repository secrets (and the token is read-only), so this value is empty and the Claude action fails instead of posting the “always one comment” result. Please gate the job/step for trusted PR sources or add a fallback path that cleanly skips when the secret is unavailable.
Useful? React with 👍 / 👎.
…cription) Org has Claude Code subscription access disabled at the Anthropic level, so the action can't use the CLAUDE_CODE_OAUTH_TOKEN. Switching to ANTHROPIC_API_KEY which uses pay-as-you-go billing and works regardless of org policy. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 3
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.
Inline comments:
In @.github/workflows/claude-code-review.yml:
- Around line 21-30: The workflow uses mutable action tags (actions/checkout@v4
and anthropics/claude-code-action@v1); replace both uses: entries with their
corresponding full-length commit SHAs to make the actions immutable. Look up the
commit SHA for the v4 branch of actions/checkout and the release commit for
anthropics/claude-code-action (via each repo's tag/branch on GitHub), then
update the uses: lines to use those full commit SHAs (and optionally add a
trailing comment noting the original tag for clarity).
- Around line 5-8: The claude-review job in the GitHub Actions workflow (job
name: claude-review) currently runs on pull_request which does not expose
secrets for forked or Dependabot PRs; update the workflow to either skip running
the claude-review job for forked/foreign repos by adding an if condition such as
checking github.event.pull_request.head.repo.full_name == github.repository on
the claude-review job, or migrate the job to run under pull_request_target but
add strict safeguards (validate PR author, checkout safe ref, and avoid
executing untrusted workflow code) to prevent secret leakage; apply the same
guard to any other jobs that rely on secrets (lines referenced around
claude-review).
- Around line 10-18: Add a concurrency group to the claude-review job so
simultaneous synchronize events don't race: update the claude-review job
definition (job name "claude-review") to include a concurrency block using
group: ${{ github.workflow }}-${{ github.event.pull_request.number }} and
cancel-in-progress: true; also stop relying on gh pr comment --edit-last by
embedding a unique HTML marker like <!-- claude-code-review --> in the generated
review body and change the update flow to find and PATCH the specific comment by
ID via the GitHub API (repos/{owner}/{repo}/issues/comments/{comment_id})
instead of using --edit-last so updates target the correct Claude comment.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro
Run ID: 523dfd7a-4a79-4a77-933d-3c730247b361
📒 Files selected for processing (1)
.github/workflows/claude-code-review.yml
| on: | ||
| pull_request: | ||
| types: [opened, synchronize, reopened, ready_for_review] | ||
|
|
There was a problem hiding this comment.
🧩 Analysis chain
🏁 Script executed:
cat -n .github/workflows/claude-code-review.ymlRepository: Subconscious-ai/design-system
Length of output: 10302
🌐 Web query:
GitHub Actions pull_request event secrets forked repositories 2024 2025
💡 Result:
In GitHub Actions, for workflows triggered by the pull_request event on pull requests from forked repositories, secrets from the base repository (except GITHUB_TOKEN) are not available. This is a security feature to prevent malicious actors from accessing secrets via PRs from forks. Secrets appear as empty strings in such workflows. This behavior has remained consistent through 2024, 2025, and into 2026, with no changes identified in official documentation or changelogs that alter this core restriction. To access secrets in workflows triggered by fork PRs: - Use the pull_request_target event instead. This runs the workflow in the context of the base branch, providing access to base repository secrets and a read/write GITHUB_TOKEN. However, it carries significant security risks: do not checkout or execute untrusted code from the PR (e.g., avoid actions/checkout with ref: ${{ github.event.pull_request.head.sha }} without persist-credentials: false and other mitigations), as it could exfiltrate secrets or the GITHUB_TOKEN. - Mitigations include: requiring manual approval (via repo/org settings for first-time contributors), using labels to trigger (e.g., on: pull_request_target: types: [labeled]), separate workflow_run workflows after initial CI, or GitHub Environments with required reviewers. - A November 2025 changelog updated pull_request_target to always use the default branch for workflow source and ref, and adjusted environment branch protections, effective Dec 8, 2025, to enhance security—but did not change secret availability. Fork repository secrets are never passed to the base repo's workflow on PRs; workflows run in the base repo context using base repo secrets only. Prioritize pull_request for safe CI (build/test/lint without secrets). Use pull_request_target sparingly for actions like labeling/commenting, with strict safeguards.
Citations:
- 1: https://docs.github.com/actions/security-guides/using-secrets-in-github-actions
- 2: https://2i2c.org/blog/github-action-secrets-forked-repositories/
- 3: https://www.wiz.io/blog/github-actions-security-threat-model-and-defenses
- 4: https://github.blog/changelog/2025-11-07-actions-pull_request_target-and-environment-branch-protections-changes
- 5: https://www.paulserban.eu/blog/post/pullrequest-vs-pullrequesttarget-the-github-actions-trigger-hiding-a-security-nightmare/
- 6: https://stackoverflow.com/questions/74957218/what-is-the-difference-between-pull-request-and-pull-request-target-event-in-git
Gate Claude reviews on forked and Dependabot PRs, or migrate to a split-workflow design.
pull_request workflows do not receive repository secrets on forked PRs and Dependabot PRs. Because this job requires secrets.ANTHROPIC_API_KEY, those reviews will fail when the secret resolves to an empty string. Add a condition to skip the claude-review job on forks (e.g., if: github.event.pull_request.head.repo.full_name == github.repository), or migrate to pull_request_target with strict safeguards to prevent untrusted code execution. (GitHub Actions documentation)
Also applies to: 29-31
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.
In @.github/workflows/claude-code-review.yml around lines 5 - 8, The
claude-review job in the GitHub Actions workflow (job name: claude-review)
currently runs on pull_request which does not expose secrets for forked or
Dependabot PRs; update the workflow to either skip running the claude-review job
for forked/foreign repos by adding an if condition such as checking
github.event.pull_request.head.repo.full_name == github.repository on the
claude-review job, or migrate the job to run under pull_request_target but add
strict safeguards (validate PR author, checkout safe ref, and avoid executing
untrusted workflow code) to prevent secret leakage; apply the same guard to any
other jobs that rely on secrets (lines referenced around claude-review).
| claude-review: | ||
| # Skip drafts unless explicitly marked ready | ||
| if: github.event.pull_request.draft == false | ||
| runs-on: ubuntu-latest | ||
| permissions: | ||
| contents: read | ||
| pull-requests: write | ||
| issues: read | ||
| id-token: write |
There was a problem hiding this comment.
🧩 Analysis chain
🏁 Script executed:
cd /tmp && find . -name "claude-code-review.yml" -o -name "*.yml" -path "*/.github/workflows/*" 2>/dev/null | head -20Repository: Subconscious-ai/design-system
Length of output: 55
🏁 Script executed:
git ls-files | grep -E "\.github/workflows.*\.yml" | head -20Repository: Subconscious-ai/design-system
Length of output: 169
🏁 Script executed:
cat -n .github/workflows/claude-code-review.yml | head -100Repository: Subconscious-ai/design-system
Length of output: 5784
🏁 Script executed:
cat -n .github/workflows/claude-code-review.yml | sed -n '55,75p'Repository: Subconscious-ai/design-system
Length of output: 1501
🏁 Script executed:
cat -n .github/workflows/claude-code-review.yml | sed -n '165,180p'Repository: Subconscious-ai/design-system
Length of output: 1047
🏁 Script executed:
wc -l .github/workflows/claude-code-review.ymlRepository: Subconscious-ai/design-system
Length of output: 118
🏁 Script executed:
grep -n "concurrency" .github/workflows/claude-code-review.ymlRepository: Subconscious-ai/design-system
Length of output: 55
🌐 Web query:
GitHub Actions concurrency group PR number best practice
💡 Result:
The best practice for using GitHub Actions concurrency groups with PR numbers is to set the concurrency group to ${{ github.workflow }}-${{ github.event.pull_request.number || github.ref }} at the workflow level, with cancel-in-progress: true. This ensures that only one workflow run per PR (or branch for non-PR events) executes at a time, automatically canceling previous in-progress or pending runs for the same PR when new commits are pushed. This provides fast feedback on the latest code changes. Key reasons and details: - Use github.event.pull_request.number for PR-specific grouping to cancel outdated runs within the same PR. - Prefix with github.workflow to make groups unique across multiple workflows in the repo, preventing cross-workflow cancellations. - Fallback to github.ref handles push events or other triggers where PR number is undefined. - Set at workflow level (not job level) for full workflow cancellation. - cancel-in-progress: true cancels running workflows; omit or set false for deployments to queue instead. Example YAML: name: CI on: pull_request: push: branches: [main] concurrency: group: ${{ github.workflow }}-${{ github.event.pull_request.number || github.ref }} cancel-in-progress: true jobs: test: runs-on: ubuntu-latest steps: - uses: actions/checkout@v4 - run: npm test This pattern is recommended in official GitHub docs for PR workflows and echoed in community best practices. For matrix jobs, append matrix variables to the group. Group names are case-insensitive.
Citations:
- 1: https://docs.github.com/en/actions/using-jobs/using-concurrency
- 2: https://oneuptime.com/blog/post/2026-01-25-github-actions-concurrency-control/view
- 3: https://docs.github.com/actions/writing-workflows/choosing-what-your-workflow-does/control-the-concurrency-of-workflows-and-jobs
- 4: https://docs.github.com/actions/using-workflows/workflow-syntax-for-github-actions
- 5: https://stackoverflow.com/questions/66335225/how-to-cancel-previous-runs-in-the-pr-when-you-push-new-commitsupdate-the-curre
Add concurrency group to prevent duplicate or lost review comments when multiple synchronize events trigger simultaneously.
The workflow lacks a concurrency group. Without it, fast successive pushes (triggering synchronize events) will spawn concurrent runs that can race: both may read the same "last comment" and conflict when editing, or one may overwrite the other, or duplicates may post. Combined with gh pr comment --edit-last targeting "the last comment by the current user" rather than a Claude-identified comment, the single-comment guarantee is not reliable.
Add PR-scoped concurrency at the job level:
jobs:
claude-review:
concurrency:
group: ${{ github.workflow }}-${{ github.event.pull_request.number }}
cancel-in-progress: trueAdditionally, replace --edit-last in the prompt (lines 59–63, 168–171) with a marker-based approach: add a unique HTML comment marker (e.g., <!-- claude-code-review -->) in the review body and update by comment ID via gh api repos/{owner}/{repo}/issues/comments/{comment_id} rather than targeting "the last comment."
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.
In @.github/workflows/claude-code-review.yml around lines 10 - 18, Add a
concurrency group to the claude-review job so simultaneous synchronize events
don't race: update the claude-review job definition (job name "claude-review")
to include a concurrency block using group: ${{ github.workflow }}-${{
github.event.pull_request.number }} and cancel-in-progress: true; also stop
relying on gh pr comment --edit-last by embedding a unique HTML marker like <!--
claude-code-review --> in the generated review body and change the update flow
to find and PATCH the specific comment by ID via the GitHub API
(repos/{owner}/{repo}/issues/comments/{comment_id}) instead of using --edit-last
so updates target the correct Claude comment.
| - name: Checkout repository | ||
| uses: actions/checkout@v4 | ||
| with: | ||
| fetch-depth: 0 # full history so reviewer can blame/log adjacent code | ||
|
|
||
| - name: Run Claude Code Review | ||
| id: claude-review | ||
| uses: anthropics/claude-code-action@v1 | ||
| with: | ||
| anthropic_api_key: ${{ secrets.ANTHROPIC_API_KEY }} |
There was a problem hiding this comment.
🧩 Analysis chain
🏁 Script executed:
find . -name "claude-code-review.yml" -type fRepository: Subconscious-ai/design-system
Length of output: 116
🌐 Web query:
GitHub Actions security hardening action pinning commit SHA mutable tags
💡 Result:
To harden GitHub Actions security, pin actions to full-length commit SHAs instead of mutable tags or versions. Mutable tags (e.g., @v1, @v2) can be rewritten by repository owners or attackers with access, pointing to malicious code that executes in your workflows with full permissions. This has been exploited in real incidents like the 2025 tj-actions/changed-files compromise (CVE-2025-30066), affecting thousands of repos; SHA-pinned workflows were unaffected. Official GitHub recommendation: "Pinning an action to a full-length commit SHA is currently the only way to use an action as an immutable release." SHAs are immutable (requiring impractical SHA-1 collisions to attack) and ensure exact code reproducibility. Example workflow change: Before: uses: actions/checkout@v4 After: uses: actions/checkout@692973e # v4.1.7 How to find SHAs: 1. Visit the action repo (e.g., github.com/actions/checkout). 2. Switch to the tag/branch (e.g., v4) via the branch dropdown; copy the full 40-char SHA. 3. Or run: git ls-remote --tags https://github.com/actions/checkout.git refs/tags/v4 4. Run the workflow once with tag, check logs for fetched SHA. 5. Verify SHA from official repo, not forks. Policies: Enable org/repo policies to require SHA pinning and block risky actions (new in 2025). Maintenance: - Use Dependabot/Renovate for SHA update PRs (reviews required; no auto-vuln alerts for SHAs natively). - Tools: pin-github-action, stepsecurity.io/securerepo, zgosalvez/ensure-sha-pinned-actions (fail builds on unpinned). - For high-risk/third-party actions first; tags OK for trusted like actions/* if necessary, but SHA preferred. - Composite actions/Docker images: Pin sub-actions/images to SHAs/digests too. - GitHub's 2025 Immutable Releases (maintainer opt-in) helps but doesn't replace consumer pinning. Prioritize: Review all workflows, pin everything, test, enforce via policy/tools.
Citations:
- 1: https://github.blog/changelog/2025-08-15-github-actions-policy-now-supports-blocking-and-sha-pinning-actions/
- 2: https://docs.github.com/en/actions/reference/security/secure-use
- 3: Pin GitHub Actions to commit SHAs in release workflow github/gh-aw#10021
- 4: https://www.stepsecurity.io/blog/pinning-github-actions-for-enhanced-security-a-complete-guide
- 5: Pin all GitHub Actions to immutable SHA commits for supply chain security github/gh-aw#5691
- 6: https://github.com/marketplace/actions/ensure-sha-pinned-actions
- 7: https://stackoverflow.com/questions/78903499/how-do-i-pin-an-action-to-a-specific-sha
- 8: https://actsense.dev/vulnerabilities/no_hash_pinning/
🏁 Script executed:
cat -n ./.github/workflows/claude-code-review.ymlRepository: Subconscious-ai/design-system
Length of output: 10302
Pin both actions to full-length commit SHAs instead of mutable tags.
This workflow hands repo contents and secrets to action code, but both uses: entries reference mutable tags (actions/checkout@v4, anthropics/claude-code-action@v1). Mutable tags can be retagged to point to malicious code. GitHub's official hardening guidance states: "Pinning an action to a full-length commit SHA is currently the only way to use an action as an immutable release." Real-world precedent: the 2025 tj-actions/changed-files compromise (CVE-2025-30066) exploited mutable tags to compromise thousands of workflows; SHA-pinned workflows remained unaffected.
Example fix:
uses: actions/checkout@692973e3d937129bcbf40652eb9f2f61becf3332 # v4.1.7
Locate the current SHA by visiting the release tag in the action's repo (e.g., github.com/actions/checkout, switch to v4 branch, copy the commit SHA) and add as a comment for clarity.
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.
In @.github/workflows/claude-code-review.yml around lines 21 - 30, The workflow
uses mutable action tags (actions/checkout@v4 and
anthropics/claude-code-action@v1); replace both uses: entries with their
corresponding full-length commit SHAs to make the actions immutable. Look up the
commit SHA for the v4 branch of actions/checkout and the release commit for
anthropics/claude-code-action (via each repo's tag/branch on GitHub), then
update the uses: lines to use those full commit SHAs (and optionally add a
trailing comment noting the original tag for clarity).
|
The Reopen once the secret is added under Settings → Secrets and variables → Actions. The branch |
1 participant
Summary
Adds
.github/workflows/claude-code-review.ymlso Claude reviews every PR with a ticket-alignment + Subconscious-specific rubric.What it does
linear-linkbackPR comment and maps the diff 1:1 to acceptance criteria--edit-lastto update prior comments on subsequent pushesRequired secret
CLAUDE_CODE_OAUTH_TOKENmust be set as a repo (or org) secret. Status in this repo: ❌ NOT SET — workflow will fail until addedTest plan
claude-reviewcheck runs🤖 Generated with Claude Code