-
Notifications
You must be signed in to change notification settings - Fork 11
Add dependency-review workflow for Dependabot/Renovate PR analysis #429
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Changes from 3 commits
756df55
f722ed1
48ad6e5
84dc2fd
3aed306
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
Large diffs are not rendered by default.
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,242 @@ | ||
| --- | ||
| inlined-imports: true | ||
| name: "Dependency Review" | ||
| description: "Analyze Dependabot and Renovate PRs for GitHub Actions and Buildkite dependency updates" | ||
| imports: | ||
| - gh-aw-fragments/elastic-tools.md | ||
| - gh-aw-fragments/runtime-setup.md | ||
| - gh-aw-fragments/formatting.md | ||
| - gh-aw-fragments/rigor.md | ||
| - gh-aw-fragments/mcp-pagination.md | ||
| - gh-aw-fragments/messages-footer.md | ||
| - gh-aw-fragments/safe-output-add-comment-pr.md | ||
| - gh-aw-fragments/network-ecosystems.md | ||
| engine: | ||
| id: copilot | ||
| model: ${{ inputs.model }} | ||
| concurrency: | ||
| group: "gh-aw-copilot-dependency-review-${{ github.event.pull_request.number }}" | ||
| on: | ||
| workflow_call: | ||
| inputs: | ||
| model: | ||
| description: "AI model to use" | ||
| type: string | ||
| required: false | ||
| default: "gpt-5.3-codex" | ||
| additional-instructions: | ||
| description: "Repo-specific instructions appended to the agent prompt" | ||
| type: string | ||
| required: false | ||
| default: "" | ||
| setup-commands: | ||
| description: "Shell commands to run before the agent starts (dependency install, build, etc.)" | ||
| type: string | ||
| required: false | ||
| default: "" | ||
| allowed-bot-users: | ||
| description: "Allowlisted bot actor usernames (comma-separated)" | ||
| type: string | ||
| required: false | ||
| default: "github-actions[bot]" | ||
| messages-footer: | ||
| description: "Footer appended to all agent comments and reviews" | ||
| type: string | ||
| required: false | ||
| default: "" | ||
| secrets: | ||
| COPILOT_GITHUB_TOKEN: | ||
| required: true | ||
| roles: [admin, maintainer, write] | ||
| bots: | ||
| - "${{ inputs.allowed-bot-users }}" | ||
| - "dependabot[bot]" | ||
| - "renovate[bot]" | ||
| concurrency: | ||
| group: dependency-review-${{ github.event.pull_request.number }} | ||
| cancel-in-progress: true | ||
| permissions: | ||
| actions: read | ||
| contents: read | ||
| issues: read | ||
| pull-requests: read | ||
| tools: | ||
| github: | ||
| toolsets: [repos, issues, pull_requests, search, actions] | ||
| bash: true | ||
| web-fetch: | ||
| safe-outputs: | ||
| activation-comments: false | ||
| add-labels: | ||
| max: 3 | ||
| allowed: | ||
| - "needs-human-review" | ||
| - "higher-risk" | ||
| strict: false | ||
| timeout-minutes: 60 | ||
| steps: | ||
| - name: Repo-specific setup | ||
| if: ${{ inputs.setup-commands != '' }} | ||
| env: | ||
| SETUP_COMMANDS: ${{ inputs.setup-commands }} | ||
| run: eval "$SETUP_COMMANDS" | ||
| --- | ||
|
|
||
| # Dependency Review Agent | ||
|
|
||
| Analyze dependency update pull requests (Dependabot, Renovate) in ${{ github.repository }} and provide a detailed analysis comment covering security, compatibility, and risk. | ||
|
|
||
| ## Context | ||
|
|
||
| - **Repository**: ${{ github.repository }} | ||
| - **PR**: #${{ github.event.pull_request.number }} — ${{ github.event.pull_request.title }} | ||
| - **PR Author**: ${{ github.actor }} | ||
|
|
||
| ## Constraints | ||
|
|
||
| This workflow is read-only. You can read files, search code, run commands, and comment on PRs — but your only outputs are an analysis comment and optional labels. | ||
|
|
||
| ## Instructions | ||
|
|
||
| ### Step 1: Gather Context | ||
|
|
||
| 1. Call `generate_agents_md` to get the repository's coding guidelines and conventions. If this fails, continue without it. | ||
| 2. Call `pull_request_read` with method `get` on PR #${{ github.event.pull_request.number }} to get full PR details (author, description, branches). | ||
| 3. Call `pull_request_read` with method `get_diff` to see exactly what changed. | ||
| 4. Call `pull_request_read` with method `get_files` to get the list of changed files. | ||
|
|
||
| ### Step 2: Identify Updated Dependencies | ||
|
|
||
| Parse the diff to identify each dependency being updated. For each dependency, extract: | ||
| - **Type**: GitHub Action, Buildkite plugin, or other | ||
| - **Source repository**: e.g. `actions/checkout`, `docker/build-push-action` | ||
| - **Old version**: tag, SHA, or version before the update | ||
| - **New version**: tag, SHA, or version after the update | ||
|
|
||
| If the PR does not update any GitHub Actions or Buildkite plugin dependencies, call `noop` with message "PR does not update GitHub Actions or Buildkite dependencies; nothing to analyze" and stop. | ||
|
|
||
| ### Step 3: Analyze Each Dependency | ||
|
|
||
| For each updated dependency, perform the following checks: | ||
|
|
||
| #### 3a: Commit Verification (GitHub Actions only) | ||
|
|
||
| If the action reference uses a commit SHA (e.g. `uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd`): | ||
|
|
||
| 1. Verify the commit is a verified commit by checking the GitHub API: | ||
| ```bash | ||
| gh api repos/{owner}/{repo}/commits/{sha} --jq '.commit.verification.verified' | ||
| ``` | ||
| 2. If the commit is **not verified**, flag this prominently. Unverified commits in pinned actions are a supply-chain risk — see [Imposter Commits in CI/CD](https://www.chainguard.dev/unchained/what-the-fork-imposter-commits-in-github-actions-and-ci-cd). | ||
| 3. Check whether the commit SHA corresponds to a known release tag: | ||
| ```bash | ||
| gh api repos/{owner}/{repo}/git/matching-refs/tags --jq '.[].ref' | head -20 | ||
| ``` | ||
| Then verify the tag points to the expected SHA. | ||
|
|
||
| #### 3b: Changelog and Release Notes | ||
|
|
||
| 1. Fetch the release notes for the new version from the dependency's repository: | ||
| ```bash | ||
| gh api repos/{owner}/{repo}/releases/tags/{new_tag} --jq '.body' 2>/dev/null | ||
| ``` | ||
| 2. If no release exists for the exact tag, check the latest releases: | ||
| ```bash | ||
| gh api repos/{owner}/{repo}/releases --jq '.[].tag_name' | head -10 | ||
| ``` | ||
| 3. Summarize key changes between the old and new versions, focusing on: | ||
| - Breaking changes or removed features | ||
| - New required inputs or changed defaults | ||
| - Security fixes | ||
| - Notable new features relevant to how this repo uses the action | ||
|
|
||
| #### 3c: Usage Analysis | ||
|
|
||
| 1. Search the repository for all places the dependency is used: | ||
| ```bash | ||
| grep -rn '{owner}/{repo}' .github/workflows/ --include='*.yml' --include='*.yaml' | ||
| ``` | ||
| 2. For each usage, note: | ||
| - Which workflow file and job uses it | ||
| - What inputs are passed to it | ||
| - What outputs are consumed from it | ||
| - What events trigger the workflow (push, pull_request, schedule, release, etc.) | ||
|
|
||
| 3. Cross-reference the usage against the changelog: | ||
| - Are any inputs used by this repo deprecated or removed in the new version? | ||
| - Are any outputs consumed by this repo changed in the new version? | ||
| - Are there new required inputs that are not provided? | ||
|
|
||
| #### 3d: Testability Assessment | ||
|
|
||
| 1. Check the trigger events for each workflow that uses the updated dependency. | ||
| 2. If a workflow is **only** triggered by `push` (to main/default branch), `release`, `schedule`, or `workflow_dispatch`, it **cannot be validated by the PR itself**. Flag this as higher risk. | ||
| 3. If a workflow is triggered by `pull_request` or `pull_request_target`, it can be exercised in the PR context. | ||
|
|
||
| #### 3e: Pin Format Check (Buildkite plugins) | ||
|
|
||
| For Buildkite plugin updates: | ||
| 1. Check if the update moves from a SHA-pinned version to a mutable tag (higher risk). | ||
| 2. Check if the update moves from one mutable tag to another mutable tag (moderate risk). | ||
| 3. SHA-to-SHA or tag-to-SHA-pinned updates are preferred. | ||
|
|
||
| ### Step 4: Determine Labels | ||
|
|
||
| Based on the analysis, determine if labels should be applied: | ||
|
|
||
| - **`needs-human-review`**: Apply when ANY of these conditions are met: | ||
| - A GitHub Action update introduces breaking input/output changes that affect this repo's usage | ||
| - A commit SHA is not verified | ||
| - A Buildkite plugin moves from SHA-pinned to mutable tag, or between mutable tags | ||
| - The changelog indicates breaking changes | ||
|
|
||
| - **`higher-risk`**: Apply when: | ||
| - The updated dependency is used only in workflows triggered by push-to-main, release, schedule, or workflow_dispatch (cannot be validated in PR context) | ||
|
|
||
| Only apply `needs-human-review` and `higher-risk` labels. | ||
|
|
||
| ### Step 5: Post Analysis Comment | ||
|
|
||
| Call `add_comment` on the PR with a structured analysis. Use the following format: | ||
|
|
||
| > ## Dependency Update Analysis | ||
| > | ||
| > **Summary**: [One-line summary of the update and overall risk assessment] | ||
| > | ||
| > ### [Dependency 1: owner/repo vOLD → vNEW] | ||
| > | ||
| > | Check | Result | | ||
| > | --- | --- | | ||
| > | Commit verified | ✅ Yes / ⚠️ No | | ||
| > | Breaking changes | ✅ None found / ⚠️ Found (details below) | | ||
| > | Testable in PR | ✅ Yes / ⚠️ No — workflow only runs on [events] | | ||
| > | Pin format | ✅ SHA-pinned / ⚠️ Mutable tag | | ||
| > | ||
| > <details> | ||
| > <summary>Changelog highlights (vOLD → vNEW)</summary> | ||
| > | ||
| > [Key changes from release notes] | ||
| > </details> | ||
| > | ||
| > <details> | ||
| > <summary>Usage in this repository</summary> | ||
| > | ||
| > [List of workflows/jobs using this dependency and relevant inputs/outputs] | ||
| > </details> | ||
| > | ||
| > <details> | ||
| > <summary>Compatibility assessment</summary> | ||
| > | ||
| > [Analysis of whether current usage is compatible with the new version] | ||
| > </details> | ||
| > | ||
| > ### Labels Applied | ||
| > [List of labels applied and why, or "No labels applied"] | ||
|
|
||
| If the analysis found no issues, keep the comment concise — do not pad with unnecessary detail. | ||
|
|
||
| ### Step 6: Apply Labels | ||
|
|
||
| If any labels were determined in Step 4, call `add_labels` to apply them to the PR. | ||
|
|
||
| ${{ inputs.additional-instructions }} | ||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,21 @@ | ||
| # This file is auto-generated by scripts/dogfood.sh. Do not edit directly. | ||
| # Edit gh-agent-workflows/dependency-review/example.yml and run 'make compile' to regenerate. | ||
| name: Trigger Dependency Review | ||
| on: | ||
| pull_request: | ||
| types: [opened, synchronize, reopened] | ||
|
|
||
| permissions: | ||
| actions: read | ||
| contents: read | ||
| issues: write | ||
| pull-requests: write | ||
|
|
||
| jobs: | ||
| run: | ||
| if: >- | ||
| github.event.pull_request.user.login == 'dependabot[bot]' || | ||
| github.event.pull_request.user.login == 'renovate[bot]' | ||
| uses: ./.github/workflows/gh-aw-dependency-review.lock.yml | ||
| secrets: | ||
| COPILOT_GITHUB_TOKEN: ${{ secrets.COPILOT_GITHUB_TOKEN }} |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,44 @@ | ||
| # Dependency Review | ||
|
|
||
| Analyze Dependabot and Renovate PRs across dependency types, with deeper checks for GitHub Actions and Buildkite updates. | ||
|
|
||
| ## How it works | ||
|
|
||
| Triggered when Dependabot or Renovate opens or updates a PR. Classifies each dependency update type, applies shared checks (changelog, usage compatibility, and testability), and runs extra checks for GitHub Actions commit verification and Buildkite pin-format risk. Posts a structured analysis comment and optionally labels the PR `needs-human-review` or `higher-risk`. | ||
|
|
||
| ## Quick Install | ||
|
|
||
| ```bash | ||
| mkdir -p .github/workflows && curl -sL \ | ||
| https://raw.githubusercontent.com/elastic/ai-github-actions/v0/gh-agent-workflows/dependency-review/example.yml \ | ||
| -o .github/workflows/dependency-review.yml | ||
| ``` | ||
|
|
||
| See [example.yml](example.yml) for the full workflow file. | ||
|
|
||
| ## Trigger | ||
|
|
||
| | Event | Types | Condition | | ||
| | --- | --- | --- | | ||
| | `pull_request` | `opened`, `synchronize`, `reopened` | PR author is `dependabot[bot]` or `renovate[bot]` | | ||
|
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. 🧩 Analysis chain🏁 Script executed: #!/bin/bash
set -euo pipefail
# First, verify the files exist
echo "=== Checking file existence ==="
for file in \
gh-agent-workflows/dependency-review/README.md \
gh-agent-workflows/dependency-review/example.yml \
.github/workflows/trigger-dependency-review.yml \
.github/workflows/gh-aw-dependency-review.md \
.github/workflows/gh-aw-dependency-review.lock.yml; do
if [ -f "$file" ]; then
echo "✓ $file exists"
else
echo "✗ $file NOT FOUND"
fi
done
echo ""
echo "=== Searching for bot references and allowed-bot-users ==="
rg -n -C2 'allowed-bot-users|dependabot\[bot\]|renovate\[bot\]|github-actions\[bot\]' \
--max-count=50 2>/dev/null || echo "Pattern search returned no results or error"Repository: elastic/ai-github-actions Length of output: 33207 🏁 Script executed: # Also check lines 23 and 31 of the README specifically
echo "=== Content around lines 23 and 31 of README ==="
if [ -f "gh-agent-workflows/dependency-review/README.md" ]; then
echo "Lines 20-26:"
sed -n '20,26p' gh-agent-workflows/dependency-review/README.md
echo ""
echo "Lines 28-34:"
sed -n '28,34p' gh-agent-workflows/dependency-review/README.md
fiRepository: elastic/ai-github-actions Length of output: 643 🏁 Script executed: # Check the actual defaults in example.yml or lock file
echo "=== Checking for defaults in workflow files ==="
if [ -f "gh-agent-workflows/dependency-review/example.yml" ]; then
echo "=== Content of example.yml ==="
cat gh-agent-workflows/dependency-review/example.yml
fiRepository: elastic/ai-github-actions Length of output: 861 Update The workflow triggers on 🤖 Prompt for AI Agents
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. @coderabbitai fix this for me There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. 🧩 Analysis chain
🏁 Script executed: Length of output: 9759
Task: Fix README documentation for allowed-bot-users defaultThe 🧠 Learnings used💡 Use |
||
|
|
||
| ## Inputs | ||
|
|
||
| | Input | Description | Required | Default | | ||
| | --- | --- | --- | --- | | ||
| | `additional-instructions` | Repo-specific instructions appended to the agent prompt | No | `""` | | ||
| | `setup-commands` | Shell commands run before the agent starts | No | `""` | | ||
| | `allowed-bot-users` | Allowlisted bot actor usernames (comma-separated) | No | `github-actions[bot]` | | ||
|
|
||
| ## Safe Outputs | ||
|
|
||
| - `add-comment` — post an analysis comment on the PR (max 1) | ||
| - `add-labels` — label the PR when human review or higher risk is detected (max 3) | ||
|
|
||
| ## Manual usage with mention-in-pr | ||
|
|
||
| You can also analyze any dependency update PR on demand using `mention-in-pr`. Comment on the PR with: | ||
|
|
||
| ``` | ||
| /ai Analyze this dependency update PR across dependency types: classify each update, run applicable risk checks, summarize changelog highlights, and note testability in PR context. | ||
| ``` | ||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,23 @@ | ||
| name: Dependency Review | ||
| on: | ||
| pull_request: | ||
| types: [opened, synchronize, reopened] | ||
|
|
||
| permissions: | ||
| actions: read | ||
| contents: read | ||
| issues: write | ||
| pull-requests: write | ||
|
|
||
| jobs: | ||
| run: | ||
| if: >- | ||
| github.event.pull_request.user.login == 'dependabot[bot]' || | ||
| github.event.pull_request.user.login == 'renovate[bot]' | ||
| uses: elastic/ai-github-actions/.github/workflows/gh-aw-dependency-review.lock.yml@v0 | ||
| with: | ||
| additional-instructions: | | ||
| Classify every updated dependency type and analyze all of them. | ||
| Apply GitHub Actions and Buildkite-specific checks only when those dependency types are present. | ||
| secrets: | ||
| COPILOT_GITHUB_TOKEN: ${{ secrets.COPILOT_GITHUB_TOKEN }} |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
remove chainguard reference
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Removed in 84dc2fd.