Skip to content
Merged
Show file tree
Hide file tree
Changes from 9 commits
Commits
Show all changes
16 commits
Select commit Hold shift + click to select a range
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
142 changes: 142 additions & 0 deletions .github/actions/pr-review/action.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,142 @@
---
name: OpenHands PR Review
description: Automated PR review using OpenHands agent
author: OpenHands

branding:
icon: code
color: blue

inputs:
llm-model:
description: LLM model to use for the review
required: false
default: anthropic/claude-sonnet-4-5-20250929
llm-base-url:
description: LLM base URL (optional, for custom LLM endpoints)
required: false
default: ''
review-style:
description: "Review style: 'standard' or 'roasted'"
required: false
default: standard
Comment thread
enyst marked this conversation as resolved.
Outdated
sdk-version:
description: SDK version tag or branch name to use (e.g., v1.0.0 or main)
required: false
default: main
Comment thread
enyst marked this conversation as resolved.
use-pr-branch-for-sdk:
description: When true and reviewing a PR on the SDK repo itself, use the PR's head branch instead of sdk-version
required: false
default: 'false'
llm-api-key:
description: LLM API key (required)
required: true
github-token:
description: GitHub token for API access (required)
required: true
Comment thread
enyst marked this conversation as resolved.

runs:
using: composite
steps:
- name: Determine SDK ref to use
id: sdk-ref
shell: bash
run: |
# If use-pr-branch-for-sdk is true and we're reviewing a PR on the SDK repo itself,
# use the PR's head branch instead of the sdk-version input
if [ "${{ inputs.use-pr-branch-for-sdk }}" = "true" ] && [ "${{ github.repository }}" = "OpenHands/software-agent-sdk" ]; then
echo "ref=${{ github.event.pull_request.head.ref }}" >> $GITHUB_OUTPUT
echo "Using PR branch '${{ github.event.pull_request.head.ref }}' for SDK (reviewing SDK repo PR)"
else
echo "ref=${{ inputs.sdk-version }}" >> $GITHUB_OUTPUT
echo "Using sdk-version '${{ inputs.sdk-version }}' for SDK"
fi

- name: Checkout software-agent-sdk repository
uses: actions/checkout@v4
with:
repository: OpenHands/software-agent-sdk
ref: ${{ steps.sdk-ref.outputs.ref }}
path: software-agent-sdk

- name: Checkout PR repository
uses: actions/checkout@v4
with:
repository: ${{ github.event.pull_request.head.repo.full_name }}
ref: ${{ github.event.pull_request.head.ref }}
fetch-depth: 0
persist-credentials: false
path: pr-repo

- name: Set up Python
uses: actions/setup-python@v5
with:
python-version: '3.13'
Comment thread
xingyaoww marked this conversation as resolved.
Outdated

- name: Install uv
uses: astral-sh/setup-uv@v6
with:
enable-cache: true

- name: Install GitHub CLI
shell: bash
run: |
sudo apt-get update
sudo apt-get install -y gh

- name: Install OpenHands dependencies
shell: bash
run: |

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟢 Nit: Using --system with uv pip is correct for CI environments, but consider adding a comment explaining why this is needed (to install into the system Python rather than creating a venv).

Suggested change
run: |
# Install to system Python (no venv needed in CI)
uv pip install --system ./software-agent-sdk/openhands-sdk ./software-agent-sdk/openhands-tools

uv pip install --system ./software-agent-sdk/openhands-sdk ./software-agent-sdk/openhands-tools

- name: Check required configuration
shell: bash
env:
LLM_API_KEY: ${{ inputs.llm-api-key }}
GITHUB_TOKEN: ${{ inputs.github-token }}
run: |
if [ -z "$LLM_API_KEY" ]; then
echo "Error: llm-api-key is required."
exit 1
fi
if [ -z "$GITHUB_TOKEN" ]; then
echo "Error: github-token is required."
exit 1
fi

echo "PR Number: ${{ github.event.pull_request.number }}"
echo "PR Title: ${{ github.event.pull_request.title }}"
echo "Repository: ${{ github.repository }}"
echo "SDK Version: ${{ inputs.sdk-version }}"
echo "LLM model: ${{ inputs.llm-model }}"
if [ -n "${{ inputs.llm-base-url }}" ]; then
echo "LLM base URL: ${{ inputs.llm-base-url }}"
fi

- name: Run PR review
shell: bash
env:
LLM_MODEL: ${{ inputs.llm-model }}
LLM_BASE_URL: ${{ inputs.llm-base-url }}

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟢 Nit: Empty string for LLM_BASE_URL is handled correctly by the Python script (agent_script.py:219-220 only sets base_url if truthy), but consider using null or omitting the env var entirely for clarity. Empty strings in environment variables can be ambiguous.

REVIEW_STYLE: ${{ inputs.review-style }}
LLM_API_KEY: ${{ inputs.llm-api-key }}
GITHUB_TOKEN: ${{ inputs.github-token }}
PR_NUMBER: ${{ github.event.pull_request.number }}
PR_TITLE: ${{ github.event.pull_request.title }}
PR_BODY: ${{ github.event.pull_request.body }}
PR_BASE_BRANCH: ${{ github.event.pull_request.base.ref }}
PR_HEAD_BRANCH: ${{ github.event.pull_request.head.ref }}
REPO_NAME: ${{ github.repository }}
run: |
cd pr-repo
uv run python ../software-agent-sdk/examples/03_github_workflows/02_pr_review/agent_script.py

- name: Upload logs as artifact
uses: actions/upload-artifact@v4
if: always()
with:
name: openhands-pr-review-logs
path: |
*.log
output/

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔴 Critical: The artifact paths are incorrect. The script runs with cd pr-repo (line 114), so logs are written to pr-repo/*.log and pr-repo/output/. However, in composite actions, each step starts from the original working directory, so this upload step will look in the wrong location.

Fix:

Suggested change
output/
pr-repo/*.log
pr-repo/output/

retention-days: 7
95 changes: 9 additions & 86 deletions .github/workflows/pr-review-by-openhands.yml
Original file line number Diff line number Diff line change
Expand Up @@ -37,91 +37,14 @@ jobs:
group: pr-review-${{ github.event.pull_request.number }}
cancel-in-progress: true
runs-on: ubuntu-24.04
env:
LLM_MODEL: litellm_proxy/claude-sonnet-4-5-20250929
LLM_BASE_URL: https://llm-proxy.app.all-hands.dev
# PR context will be automatically provided by the agent script
PR_NUMBER: ${{ github.event.pull_request.number }}
PR_TITLE: ${{ github.event.pull_request.title }}
PR_BODY: ${{ github.event.pull_request.body }}
PR_BASE_BRANCH: ${{ github.event.pull_request.base.ref }}
PR_HEAD_BRANCH: ${{ github.event.pull_request.head.ref }}
REPO_NAME: ${{ github.repository }}
steps:
- name: Checkout software-agent-sdk repository
uses: actions/checkout@v5
- name: Run PR Review
uses: OpenHands/software-agent-sdk/.github/actions/pr-review@main
with:
repository: OpenHands/software-agent-sdk
path: software-agent-sdk

- name: Checkout PR repository
uses: actions/checkout@v5
with:
# When using pull_request_target, explicitly checkout the PR branch
# This ensures we review the actual PR code (including fork PRs)
repository: ${{ github.event.pull_request.head.repo.full_name }}
ref: ${{ github.event.pull_request.head.ref }}
fetch-depth: 0
# Security: Don't persist credentials to prevent untrusted PR code from using them
persist-credentials: false
path: pr-repo

- name: Set up Python
uses: actions/setup-python@v6
with:
python-version: '3.13'

- name: Install uv
uses: astral-sh/setup-uv@v7
with:
enable-cache: true

- name: Install GitHub CLI
run: |
# Install GitHub CLI for posting review comments
sudo apt-get update
sudo apt-get install -y gh

- name: Install OpenHands dependencies
run: |
# Install OpenHands SDK and tools from local checkout
uv pip install --system ./software-agent-sdk/openhands-sdk ./software-agent-sdk/openhands-tools

- name: Check required configuration
env:
LLM_API_KEY: ${{ secrets.LLM_API_KEY }}
run: |
if [ -z "$LLM_API_KEY" ]; then
echo "Error: LLM_API_KEY secret is not set."
exit 1
fi

echo "PR Number: $PR_NUMBER"
echo "PR Title: $PR_TITLE"
echo "Repository: $REPO_NAME"
echo "LLM model: $LLM_MODEL"
if [ -n "$LLM_BASE_URL" ]; then
echo "LLM base URL: $LLM_BASE_URL"
fi

- name: Run PR review
env:
LLM_API_KEY: ${{ secrets.LLM_API_KEY }}
GITHUB_TOKEN: ${{ secrets.ALLHANDS_BOT_GITHUB_PAT }}
LMNR_PROJECT_API_KEY: ${{ secrets.LMNR_SKILLS_API_KEY }}
run: |
# Change to the PR repository directory so agent can analyze the code
cd pr-repo

# Run the PR review script from the software-agent-sdk checkout
uv run python ../software-agent-sdk/examples/03_github_workflows/02_pr_review/agent_script.py

- name: Upload logs as artifact
uses: actions/upload-artifact@v5
if: always()
with:
name: openhands-pr-review-logs
path: |
*.log
output/
retention-days: 7
llm-model: anthropic/claude-sonnet-4-5-20250929
llm-base-url: ''
review-style: standard
sdk-version: main
use-pr-branch-for-sdk: 'true'
llm-api-key: ${{ secrets.LLM_API_KEY }}
github-token: ${{ secrets.ALLHANDS_BOT_GITHUB_PAT }}
48 changes: 37 additions & 11 deletions examples/03_github_workflows/02_pr_review/README.md
Comment thread
xingyaoww marked this conversation as resolved.
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ This example demonstrates how to set up a GitHub Actions workflow for automated

## Files

- **`workflow.yml`**: GitHub Actions workflow file that triggers on PR labels
- **`workflow.yml`**: Example GitHub Actions workflow file that uses the composite action
- **`agent_script.py`**: Python script that runs the OpenHands agent for PR review
- **`prompt.py`**: The prompt asking the agent to write the PR review
- **`README.md`**: This documentation file
Expand Down Expand Up @@ -32,6 +32,7 @@ This example demonstrates how to set up a GitHub Actions workflow for automated
- Potential issues and security concerns
- Specific improvement suggestions
- **GitHub API Integration**: Uses the GitHub API to post inline review comments directly on specific lines of code
- **Version Control**: Use `sdk-version` to pin to a specific version tag or branch

## Setup

Expand All @@ -54,18 +55,22 @@ Set the following secrets in your GitHub repository settings:

### 3. Customize the workflow (optional)

Edit `.github/workflows/pr-review-by-openhands.yml` to customize the configuration in the `env` section:
Edit `.github/workflows/pr-review-by-openhands.yml` to customize the inputs:

```yaml
env:
# Optional: Use a different LLM model
LLM_MODEL: openhands/claude-sonnet-4-5-20250929
# Optional: Use a custom LLM base URL
# LLM_BASE_URL: 'https://custom-api.example.com'
# Optional: Choose review style ('standard' or 'roasted')
# - 'standard': Pragmatic, constructive feedback (default)
# - 'roasted': Linus Torvalds style brutally honest review
REVIEW_STYLE: standard
- name: Run PR Review
uses: ./.github/actions/pr-review
with:
# LLM configuration
llm-model: anthropic/claude-sonnet-4-5-20250929
llm-base-url: ''
# Review style: 'standard' or 'roasted' (Linus Torvalds style)
review-style: standard
# SDK version to use (version tag or branch name, e.g., 'v1.0.0' or 'main')
sdk-version: main
# Secrets
llm-api-key: ${{ secrets.LLM_API_KEY }}
github-token: ${{ secrets.GITHUB_TOKEN }}
```

### 4. Create the review label
Expand Down Expand Up @@ -101,3 +106,24 @@ There are two ways to trigger an automated review of a pull request:
5. Review comments will be posted to the PR when complete

**Note**: Both methods require write access to the repository, ensuring only authorized users can trigger the AI review.

## Composite Action

This workflow uses a reusable composite action located at `.github/actions/pr-review/action.yml` in the software-agent-sdk repository. The composite action handles:
Comment thread
enyst marked this conversation as resolved.

- Checking out the SDK at the specified version
- Setting up Python and dependencies
- Running the PR review agent
- Uploading logs as artifacts

### Action Inputs

| Input | Description | Required | Default |
|-------|-------------|----------|---------|
| `llm-model` | LLM model to use | No | `anthropic/claude-sonnet-4-5-20250929` |
| `llm-base-url` | LLM base URL (optional) | No | `''` |
| `review-style` | Review style: 'standard' or 'roasted' | No | `standard` |
| `sdk-version` | SDK version tag or branch name | No | `main` |
| `use-pr-branch-for-sdk` | Use PR branch for SDK when reviewing SDK repo | No | `false` |
| `llm-api-key` | LLM API key | Yes | - |
| `github-token` | GitHub token for API access | Yes | - |
Loading
Loading