Skip to content
Merged
Show file tree
Hide file tree
Changes from 1 commit
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
36 changes: 36 additions & 0 deletions .agents/recipes/pr-review/recipe.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,36 @@
---
name: pr-review
description: Review a pull request and post findings as a PR comment
trigger: pull_request
tool: claude-code
timeout_minutes: 15
max_turns: 30
permissions:
checks: write
contents: read
pull-requests: write
---

# PR Review

Review pull request #{{pr_number}} using the `review-code` skill.

## Instructions

1. Run `/review-code {{pr_number}}`
2. The skill writes the review to `/tmp/review-{{pr_number}}.md`. If it does
not, save the review output there yourself.
3. Before finishing, read `/tmp/review-{{pr_number}}.md` and verify it contains
a valid review (Summary, Findings, Verdict sections). If the file is empty
or malformed, write a brief "Review could not be completed" note to the file
instead.

## Constraints

- Do NOT post the review to GitHub yourself. The workflow handles posting via
`gh pr comment --body-file`.
- Do NOT approve or request changes on the PR.
- If the diff is extremely large (>100 changed files), focus on the most
critical files and note that a full review was not feasible in a single pass.
- If the PR only changes docs/markdown, focus on accuracy, broken links, and
Comment thread
andreatnvidia marked this conversation as resolved.
consistency with code. Skip linting.
152 changes: 152 additions & 0 deletions .github/workflows/agentic-ci-pr-review.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,152 @@
name: "Agentic CI: PR Review"

on:
pull_request:
types: [opened, synchronize, ready_for_review, labeled]
branches: [main]
workflow_dispatch:
inputs:
pr_number:
description: "PR number to review"
required: true

permissions:
checks: write
contents: read
pull-requests: write

concurrency:
group: agentic-ci-pr-review-${{ github.event.pull_request.number || github.event.inputs.pr_number }}
cancel-in-progress: true

jobs:
review:
# Only run for collaborators (or manual dispatch, which requires write access).
# Skip draft PRs on automatic triggers. Allow the re-review label to bypass draft check.
if: >-
github.event_name == 'workflow_dispatch'
|| (
contains(fromJSON('["MEMBER","COLLABORATOR","OWNER"]'), github.event.pull_request.author_association)
&& (
!github.event.pull_request.draft
|| (github.event.action == 'labeled' && github.event.label.name == 're-review')
)
)
Comment thread
andreatnvidia marked this conversation as resolved.
Outdated
runs-on: [self-hosted, agentic-ci]
timeout-minutes: 15
steps:
- name: Determine PR number
id: pr
run: |
if [ "${{ github.event_name }}" = "workflow_dispatch" ]; then
echo "number=${{ github.event.inputs.pr_number }}" >> "$GITHUB_OUTPUT"
else
echo "number=${{ github.event.pull_request.number }}" >> "$GITHUB_OUTPUT"
fi

- name: Check required config
run: |
if [ -z "${{ vars.AGENTIC_CI_MODEL }}" ]; then
echo "::error::AGENTIC_CI_MODEL variable is not set. Configure it in repo settings."
exit 1
fi

- name: Detect auth mode
id: auth
run: |
if [ -n "${{ secrets.AGENTIC_CI_API_BASE_URL }}" ] && [ -n "${{ secrets.AGENTIC_CI_API_KEY }}" ]; then
echo "mode=custom" >> "$GITHUB_OUTPUT"
else
echo "mode=oauth" >> "$GITHUB_OUTPUT"
fi

- name: Checkout PR branch
uses: actions/checkout@v4
with:
ref: ${{ github.event.pull_request.head.sha || '' }}
fetch-depth: 0
Comment on lines +125 to +129

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 security Recipe files from PR HEAD executed without trust boundary

The workflow checks out at steps.head.outputs.sha (the PR branch HEAD) and then reads recipe files β€” _runner.md and pr-review/recipe.md β€” from that checkout. For labeled events the gate validates the labeler (sender) rather than the PR author, so a trusted collaborator can intentionally trigger a review of an external contributor's PR. If that external PR modifies any file under .agents/recipes/, the modified recipe is the one that runs.

Because Claude Code is invoked with --max-turns 30 and has Bash tool access, a crafted recipe body could execute arbitrary shell commands on the self-hosted runner (e.g. exfiltrating $ANTHROPIC_API_KEY via curl, or calling gh with the valid GH_TOKEN).

The Phase 4 plan item ("CODEOWNERS for .agents/recipes/") would require review before merging a recipe change, but it would not prevent a pre-merge execution triggered by the agent-review label.

Fix: read recipe files from the base branch rather than from the checked-out PR HEAD:

- name: Fetch base-branch recipes
  run: |
    git fetch origin ${{ github.base_ref }}
    git checkout origin/${{ github.base_ref }} -- .agents/recipes/

Placing this step after checkout ensures that even when the PR modifies recipe files, the base-branch versions are what Claude receives.

Prompt To Fix With AI
This is a comment left during a code review.
Path: .github/workflows/agentic-ci-pr-review.yml
Line: 125-129

Comment:
**Recipe files from PR HEAD executed without trust boundary**

The workflow checks out at `steps.head.outputs.sha` (the PR branch HEAD) and then reads recipe files β€” `_runner.md` and `pr-review/recipe.md` β€” from that checkout. For `labeled` events the gate validates the *labeler* (sender) rather than the PR author, so a trusted collaborator can intentionally trigger a review of an external contributor's PR. If that external PR modifies any file under `.agents/recipes/`, the modified recipe is the one that runs.

Because Claude Code is invoked with `--max-turns 30` and has Bash tool access, a crafted recipe body could execute arbitrary shell commands on the self-hosted runner (e.g. exfiltrating `$ANTHROPIC_API_KEY` via curl, or calling `gh` with the valid `GH_TOKEN`).

The Phase 4 plan item ("CODEOWNERS for `.agents/recipes/`") would require review before *merging* a recipe change, but it would not prevent a pre-merge execution triggered by the `agent-review` label.

Fix: read recipe files from the base branch rather than from the checked-out PR HEAD:

```yaml
- name: Fetch base-branch recipes
  run: |
    git fetch origin ${{ github.base_ref }}
    git checkout origin/${{ github.base_ref }} -- .agents/recipes/
```

Placing this step after checkout ensures that even when the PR modifies recipe files, the base-branch versions are what Claude receives.

How can I resolve this? If you propose a fix, please make it concise.


# For workflow_dispatch, fetch and checkout the PR head.
- name: Checkout PR branch (dispatch)
if: github.event_name == 'workflow_dispatch'
env:
GH_TOKEN: ${{ github.token }}
run: |
PR=${{ steps.pr.outputs.number }}
HEAD_SHA=$(gh pr view "$PR" --json headRefOid -q '.headRefOid')
git fetch origin "$HEAD_SHA"
git checkout "$HEAD_SHA"

- name: Pre-flight checks
env:
ANTHROPIC_BASE_URL: ${{ secrets.AGENTIC_CI_API_BASE_URL }}
ANTHROPIC_API_KEY: ${{ secrets.AGENTIC_CI_API_KEY }}
run: |
if ! command -v claude &> /dev/null; then
echo "::error::claude CLI not found in PATH"
exit 1
fi
echo "Claude CLI version: $(claude --version 2>&1 || true)"

# Quick API check (custom endpoint only)
if [ -n "$ANTHROPIC_BASE_URL" ] && [ -n "$ANTHROPIC_API_KEY" ]; then
HTTP_CODE=$(curl -s -o /dev/null -w "%{http_code}" \
--max-time 10 \
-X POST "${ANTHROPIC_BASE_URL}/v1/messages" \
-H "Content-Type: application/json" \
-H "x-api-key: ${ANTHROPIC_API_KEY}" \
-H "anthropic-version: 2023-06-01" \
-d "{\"model\":\"${{ vars.AGENTIC_CI_MODEL }}\",\"max_tokens\":5,\"messages\":[{\"role\":\"user\",\"content\":\"hi\"}]}")
if [ "$HTTP_CODE" -lt 200 ] || [ "$HTTP_CODE" -ge 300 ]; then
echo "::error::API pre-flight failed with HTTP ${HTTP_CODE}"
exit 1
fi
echo "API pre-flight passed (HTTP ${HTTP_CODE})"
fi

- name: Run PR review recipe
env:
ANTHROPIC_BASE_URL: ${{ secrets.AGENTIC_CI_API_BASE_URL }}
ANTHROPIC_API_KEY: ${{ secrets.AGENTIC_CI_API_KEY }}
AGENTIC_CI_MODEL: ${{ vars.AGENTIC_CI_MODEL }}
GH_TOKEN: ${{ github.token }}
run: |
PR=${{ steps.pr.outputs.number }}
MODEL="${AGENTIC_CI_MODEL}"

# Build the prompt from _runner.md + recipe, substituting template vars.
RUNNER_CTX=$(cat .agents/recipes/_runner.md)
RECIPE_BODY=$(cat .agents/recipes/pr-review/recipe.md \
| sed '1,/^---$/{ /^---$/,/^---$/d }')

PROMPT=$(echo -e "${RUNNER_CTX}\n\n${RECIPE_BODY}" \
| sed "s/{{pr_number}}/${PR}/g")
Comment thread
andreatnvidia marked this conversation as resolved.
Outdated

claude \
--model "$MODEL" \
-p "$PROMPT" \
--max-turns 30 \
--output-format text \
--verbose \
2>&1 | tee /tmp/claude-review-log.txt || true

# Verify the review file was created.
if [ ! -s "/tmp/review-${PR}.md" ]; then
echo "::warning::Review file not created by agent. Extracting from output."
echo "Review could not be completed - the agent did not produce a review file." \
> "/tmp/review-${PR}.md"
fi

- name: Post review comment
env:
GH_TOKEN: ${{ github.token }}
run: |
PR=${{ steps.pr.outputs.number }}
gh pr comment "$PR" --body-file "/tmp/review-${PR}.md"

- name: Remove re-review label
if: github.event.action == 'labeled' && github.event.label.name == 're-review'
env:
GH_TOKEN: ${{ github.token }}
run: |
gh pr edit ${{ steps.pr.outputs.number }} --remove-label "re-review"
12 changes: 6 additions & 6 deletions plans/472/agentic-ci-plan.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
---
date: 2026-03-30
status: proposed
status: in-progress
authors:
- andreatgretel
---
Expand Down Expand Up @@ -553,13 +553,13 @@ is clear:

## Phased Rollout

### Phase 1: Foundation (this PR / next PR)
### Phase 1: Foundation

**Deliverables:**
- [ ] `.agents/recipes/` directory with `_runner.md` and recipe format spec
- [ ] First recipe: `pr-review/recipe.md`
- [ ] GitHub workflow: `agentic-ci-pr-review.yml` (self-hosted runner)
- [ ] API health probe workflow (`agentic-ci-health-probe.yml`) - pings the API
- [x] `.agents/recipes/` directory with `_runner.md` and recipe format spec
- [x] First recipe: `pr-review/recipe.md`
- [x] GitHub workflow: `agentic-ci-pr-review.yml` (self-hosted runner)
- [x] API health probe workflow (`agentic-ci-health-probe.yml`) - pings the API
on a schedule, fails the workflow run on error (GitHub's built-in
notifications handle alerting). Needed before relying on the API for real
work.
Expand Down
Loading