-
Notifications
You must be signed in to change notification settings - Fork 5
Add dependency-review workflow for Dependabot/Renovate PR analysis #429
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Merged
Changes from all commits
Commits
Show all changes
5 commits
Select commit
Hold shift + click to select a range
756df55
Initial plan
Copilot f722ed1
Add dependency-review workflow for analyzing Dependabot/Renovate PRs
Copilot 48ad6e5
Generalize dependency-review workflow guidance
Copilot 84dc2fd
Generalize dependency-review to cover Go, npm, Python, Java ecosystems
Copilot 3aed306
Merge branch 'main' into copilot/create-github-agent-workflow
strawgate File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
1,526 changes: 1,526 additions & 0 deletions
1,526
.github/workflows/gh-aw-dependency-review.lock.yml
Large diffs are not rendered by default.
Oops, something went wrong.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,283 @@ | ||
| --- | ||
| inlined-imports: true | ||
| name: "Dependency Review" | ||
| description: "Analyze Dependabot and Renovate dependency update PRs" | ||
| imports: | ||
| - gh-aw-fragments/elastic-tools.md | ||
| - gh-aw-fragments/runtime-setup.md | ||
| - gh-aw-fragments/formatting.md | ||
| - gh-aw-fragments/rigor.md | ||
| - gh-aw-fragments/mcp-pagination.md | ||
| - gh-aw-fragments/messages-footer.md | ||
| - gh-aw-fragments/safe-output-add-comment-pr.md | ||
| - gh-aw-fragments/network-ecosystems.md | ||
| engine: | ||
| id: copilot | ||
| model: ${{ inputs.model }} | ||
| concurrency: | ||
| group: "gh-aw-copilot-dependency-review-${{ github.event.pull_request.number }}" | ||
| on: | ||
| workflow_call: | ||
| inputs: | ||
| model: | ||
| description: "AI model to use" | ||
| type: string | ||
| required: false | ||
| default: "gpt-5.3-codex" | ||
| additional-instructions: | ||
| description: "Repo-specific instructions appended to the agent prompt" | ||
| type: string | ||
| required: false | ||
| default: "" | ||
| setup-commands: | ||
| description: "Shell commands to run before the agent starts (dependency install, build, etc.)" | ||
| type: string | ||
| required: false | ||
| default: "" | ||
| allowed-bot-users: | ||
| description: "Allowlisted bot actor usernames (comma-separated)" | ||
| type: string | ||
| required: false | ||
| default: "github-actions[bot]" | ||
| messages-footer: | ||
| description: "Footer appended to all agent comments and reviews" | ||
| type: string | ||
| required: false | ||
| default: "" | ||
| secrets: | ||
| COPILOT_GITHUB_TOKEN: | ||
| required: true | ||
| roles: [admin, maintainer, write] | ||
| bots: | ||
| - "${{ inputs.allowed-bot-users }}" | ||
| - "dependabot[bot]" | ||
| - "renovate[bot]" | ||
| concurrency: | ||
| group: dependency-review-${{ github.event.pull_request.number }} | ||
| cancel-in-progress: true | ||
| permissions: | ||
| actions: read | ||
| contents: read | ||
| issues: read | ||
| pull-requests: read | ||
| tools: | ||
| github: | ||
| toolsets: [repos, issues, pull_requests, search, actions] | ||
| bash: true | ||
| web-fetch: | ||
| safe-outputs: | ||
| activation-comments: false | ||
| add-labels: | ||
| max: 3 | ||
| allowed: | ||
| - "needs-human-review" | ||
| - "higher-risk" | ||
| strict: false | ||
| timeout-minutes: 60 | ||
| steps: | ||
| - name: Repo-specific setup | ||
| if: ${{ inputs.setup-commands != '' }} | ||
| env: | ||
| SETUP_COMMANDS: ${{ inputs.setup-commands }} | ||
| run: eval "$SETUP_COMMANDS" | ||
| --- | ||
|
|
||
| # Dependency Review Agent | ||
|
|
||
| Analyze dependency update pull requests (Dependabot, Renovate) in ${{ github.repository }}. Provide a detailed analysis comment covering changelog highlights, compatibility, risk, and ecosystem-specific checks. | ||
|
|
||
| ## Context | ||
|
|
||
| - **Repository**: ${{ github.repository }} | ||
| - **PR**: #${{ github.event.pull_request.number }} — ${{ github.event.pull_request.title }} | ||
| - **PR Author**: ${{ github.actor }} | ||
|
|
||
| ## Constraints | ||
|
|
||
| This workflow is read-only. You can read files, search code, run commands, and comment on PRs — but your only outputs are an analysis comment and optional labels. | ||
|
|
||
| ## Instructions | ||
|
|
||
| ### Step 1: Gather Context | ||
|
|
||
| 1. Call `generate_agents_md` to get the repository's coding guidelines and conventions. If this fails, continue without it. | ||
| 2. Call `pull_request_read` with method `get` on PR #${{ github.event.pull_request.number }} to get full PR details (author, description, branches). | ||
| 3. Call `pull_request_read` with method `get_diff` to see exactly what changed. | ||
| 4. Call `pull_request_read` with method `get_files` to get the list of changed files. | ||
|
|
||
| ### Step 2: Identify and Classify Updated Dependencies | ||
|
|
||
| Parse the diff to identify each dependency being updated. For each dependency, extract: | ||
| - **Ecosystem**: GitHub Actions, Buildkite plugin, Go module, npm package, Python (pip/Poetry/uv), Maven/Gradle (Java), or other | ||
| - **Package name**: e.g. `actions/checkout`, `golang.org/x/net`, `express`, `requests` | ||
| - **Old version**: tag, SHA, or version before the update | ||
| - **New version**: tag, SHA, or version after the update | ||
|
|
||
| Classify each dependency by looking at the files changed: | ||
| - `.github/workflows/*.yml` or `.github/workflows/*.yaml` → **GitHub Actions** | ||
| - `pipeline.yml`, `.buildkite/` files → **Buildkite plugin** | ||
| - `go.mod`, `go.sum` → **Go module** | ||
| - `package.json`, `package-lock.json`, `yarn.lock`, `pnpm-lock.yaml` → **npm/Node** | ||
| - `pyproject.toml`, `requirements*.txt`, `Pipfile*`, `poetry.lock`, `uv.lock` → **Python** | ||
| - `pom.xml`, `build.gradle`, `build.gradle.kts`, `gradle.lockfile` → **Java/Kotlin (Maven/Gradle)** | ||
| - Other manifest files → classify by ecosystem | ||
|
|
||
| ### Step 3: Analyze Each Dependency | ||
|
|
||
| For each updated dependency, perform the following checks: | ||
|
|
||
| #### 3a: Commit Verification (GitHub Actions only) | ||
|
|
||
| If the action reference uses a commit SHA (e.g. `uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd`): | ||
|
|
||
| 1. Verify the commit is a verified commit by checking the GitHub API: | ||
| ```bash | ||
| gh api repos/{owner}/{repo}/commits/{sha} --jq '.commit.verification.verified' | ||
| ``` | ||
| 2. If the commit is **not verified**, flag this prominently. Unverified commits in pinned actions are a supply-chain risk. | ||
| 3. Check whether the commit SHA corresponds to a known release tag: | ||
| ```bash | ||
| gh api repos/{owner}/{repo}/git/matching-refs/tags --jq '.[].ref' | head -20 | ||
| ``` | ||
| Then verify the tag points to the expected SHA. | ||
|
|
||
| #### 3b: Changelog and Release Notes | ||
|
|
||
| For dependencies hosted on GitHub, fetch the release notes: | ||
| 1. Fetch the release notes for the new version from the dependency's repository: | ||
| ```bash | ||
| gh api repos/{owner}/{repo}/releases/tags/{new_tag} --jq '.body' 2>/dev/null | ||
| ``` | ||
| 2. If no release exists for the exact tag, check the latest releases: | ||
| ```bash | ||
| gh api repos/{owner}/{repo}/releases --jq '.[].tag_name' | head -10 | ||
| ``` | ||
| 3. For non-GitHub dependencies, check the package registry or changelog files in the source repo when available. | ||
| 4. Summarize key changes between the old and new versions, focusing on: | ||
| - Breaking changes or removed features | ||
| - New required configuration or changed defaults | ||
| - Security fixes | ||
| - Deprecations | ||
| - Notable new features relevant to how this repo uses the dependency | ||
|
|
||
| #### 3c: Usage Analysis | ||
|
|
||
| 1. Search the repository for all places the dependency is used. The search method depends on the ecosystem: | ||
| - **GitHub Actions**: `grep -rn '{owner}/{repo}' .github/workflows/ --include='*.yml' --include='*.yaml'` | ||
| - **Go**: `grep -rn '{module}' --include='*.go'` (look for import statements) | ||
| - **npm/Node**: `grep -rn "require('{package}')\|from '{package}'" --include='*.js' --include='*.ts' --include='*.mjs' --include='*.cjs'` | ||
| - **Python**: `grep -rn "import {package}\|from {package}" --include='*.py'` | ||
| - **Java**: `grep -rn '{groupId}' --include='*.java' --include='*.kt' --include='*.gradle' --include='*.xml'` | ||
| 2. For each usage, note: | ||
| - Which files and modules use it | ||
| - What APIs, functions, or features are consumed | ||
| - For GitHub Actions: what inputs are passed and outputs consumed | ||
| - For GitHub Actions: what events trigger the workflow | ||
|
|
||
| 3. Cross-reference the usage against the changelog: | ||
| - Are any APIs, inputs, or features used by this repo deprecated or removed in the new version? | ||
| - Are there breaking changes to consumed interfaces? | ||
| - Are there new required configuration options that are not provided? | ||
|
|
||
| #### 3d: Testability Assessment | ||
|
|
||
| 1. Check the trigger events for each workflow that uses the updated dependency. | ||
| 2. If a workflow is **only** triggered by `push` (to main/default branch), `release`, `schedule`, or `workflow_dispatch`, it **cannot be validated by the PR itself**. Flag this as higher risk. | ||
| 3. If a workflow is triggered by `pull_request` or `pull_request_target`, it can be exercised in the PR context. | ||
|
|
||
| #### 3e: Pin Format Check (Buildkite plugins) | ||
|
|
||
| For Buildkite plugin updates: | ||
| 1. Check if the update moves from a SHA-pinned version to a mutable tag (higher risk). | ||
| 2. Check if the update moves from one mutable tag to another mutable tag (moderate risk). | ||
| 3. SHA-to-SHA or tag-to-SHA-pinned updates are preferred. | ||
|
|
||
| #### 3f: Ecosystem-Specific Guidance | ||
|
|
||
| Apply the following additional checks based on the dependency ecosystem: | ||
|
|
||
| **Go modules:** | ||
| - Check if this is a major version bump (e.g. v1 → v2) — Go major versions change the import path, which is a breaking change requiring code updates across the repo. | ||
| - For indirect dependency updates, note that these are transitive and generally lower risk. | ||
| - Check for `// Deprecated:` annotations in the module if accessible. | ||
|
|
||
| **npm / Node packages:** | ||
| - Check if this is a major semver bump — major versions typically signal breaking changes. | ||
| - Look for peer dependency conflicts that may arise from the update. | ||
| - For `devDependencies`, note that these only affect development and CI, not production. | ||
|
|
||
| **Python packages (pip, Poetry, uv):** | ||
| - Check if this is a major version bump — may indicate breaking API changes. | ||
| - Check for minimum Python version requirements that may have changed. | ||
| - For packages with native extensions (e.g. `numpy`, `cryptography`), note potential build or platform compatibility changes. | ||
|
|
||
| **Java / Kotlin (Maven, Gradle):** | ||
| - Check if this is a major version bump — may indicate breaking API changes. | ||
| - Note if the groupId or artifactId changed (dependency relocation). | ||
| - For Spring or framework dependencies, check for minimum JDK version changes. | ||
|
|
||
| ### Step 4: Determine Labels | ||
|
|
||
| Based on the analysis, determine if labels should be applied: | ||
|
|
||
| - **`needs-human-review`**: Apply when ANY of these conditions are met: | ||
| - A dependency update introduces breaking changes that affect this repo's usage | ||
| - A GitHub Actions commit SHA is not verified | ||
| - A Buildkite plugin moves from SHA-pinned to mutable tag, or between mutable tags | ||
| - The changelog indicates breaking changes | ||
| - A major version bump in any ecosystem (e.g. v1 → v2 in Go, major semver in npm/Python/Java) | ||
|
|
||
| - **`higher-risk`**: Apply when: | ||
| - The updated dependency is used only in workflows triggered by push-to-main, release, schedule, or workflow_dispatch (cannot be validated in PR context) | ||
|
|
||
| Only apply `needs-human-review` and `higher-risk` labels. | ||
|
|
||
| ### Step 5: Post Analysis Comment | ||
|
|
||
| Call `add_comment` on the PR with a structured analysis. Use the following format: | ||
|
|
||
| > ## Dependency Update Analysis | ||
| > | ||
| > **Summary**: [One-line summary of the update and overall risk assessment] | ||
| > | ||
| > ### [Dependency 1: package vOLD → vNEW] | ||
| > | ||
| > **Ecosystem**: [GitHub Actions / Go / npm / Python / Java / Buildkite / other] | ||
| > | ||
| > | Check | Result | | ||
| > | --- | --- | | ||
| > | Breaking changes | ✅ None found / ⚠️ Found (details below) | | ||
| > | Testable in PR | ✅ Yes / ⚠️ No — workflow only runs on [events] | | ||
| > | Commit verified | ✅ Yes / ⚠️ No *(GitHub Actions only)* | | ||
| > | Pin format | ✅ SHA-pinned / ⚠️ Mutable tag *(GitHub Actions / Buildkite only)* | | ||
| > | ||
| > Only include rows relevant to the dependency ecosystem. For example, "Commit verified" and "Pin format" only apply to GitHub Actions and Buildkite. | ||
| > | ||
| > <details> | ||
| > <summary>Changelog highlights (vOLD → vNEW)</summary> | ||
| > | ||
| > [Key changes from release notes] | ||
| > </details> | ||
| > | ||
| > <details> | ||
| > <summary>Usage in this repository</summary> | ||
| > | ||
| > [List of files/modules using this dependency and relevant APIs/inputs/outputs] | ||
| > </details> | ||
| > | ||
| > <details> | ||
| > <summary>Compatibility assessment</summary> | ||
| > | ||
| > [Analysis of whether current usage is compatible with the new version, including ecosystem-specific notes] | ||
| > </details> | ||
| > | ||
| > ### Labels Applied | ||
| > [List of labels applied and why, or "No labels applied"] | ||
|
|
||
| If the analysis found no issues, keep the comment concise — do not pad with unnecessary detail. | ||
|
|
||
| ### Step 6: Apply Labels | ||
|
|
||
| If any labels were determined in Step 4, call `add_labels` to apply them to the PR. | ||
|
|
||
| ${{ inputs.additional-instructions }} |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,21 @@ | ||
| # This file is auto-generated by scripts/dogfood.sh. Do not edit directly. | ||
| # Edit gh-agent-workflows/dependency-review/example.yml and run 'make compile' to regenerate. | ||
| name: Trigger Dependency Review | ||
| on: | ||
| pull_request: | ||
| types: [opened, synchronize, reopened] | ||
|
|
||
| permissions: | ||
| actions: read | ||
| contents: read | ||
| issues: write | ||
| pull-requests: write | ||
|
|
||
| jobs: | ||
| run: | ||
| if: >- | ||
| github.event.pull_request.user.login == 'dependabot[bot]' || | ||
| github.event.pull_request.user.login == 'renovate[bot]' | ||
| uses: ./.github/workflows/gh-aw-dependency-review.lock.yml | ||
| secrets: | ||
| COPILOT_GITHUB_TOKEN: ${{ secrets.COPILOT_GITHUB_TOKEN }} |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,44 @@ | ||
| # Dependency Review | ||
|
|
||
| Analyze Dependabot and Renovate dependency update PRs. | ||
|
|
||
| ## How it works | ||
|
|
||
| Triggered when Dependabot or Renovate opens or updates a PR. Classifies each dependency by ecosystem (GitHub Actions, Go, npm, Python, Java, Buildkite, etc.), runs shared checks (changelog, usage analysis, compatibility, testability), and applies ecosystem-specific checks where relevant. Posts a structured analysis comment and optionally labels the PR `needs-human-review` or `higher-risk`. | ||
|
|
||
| ## Quick Install | ||
|
|
||
| ```bash | ||
| mkdir -p .github/workflows && curl -sL \ | ||
| https://raw.githubusercontent.com/elastic/ai-github-actions/v0/gh-agent-workflows/dependency-review/example.yml \ | ||
| -o .github/workflows/dependency-review.yml | ||
| ``` | ||
|
|
||
| See [example.yml](example.yml) for the full workflow file. | ||
|
|
||
| ## Trigger | ||
|
|
||
| | Event | Types | Condition | | ||
| | --- | --- | --- | | ||
| | `pull_request` | `opened`, `synchronize`, `reopened` | PR author is `dependabot[bot]` or `renovate[bot]` | | ||
|
|
||
| ## Inputs | ||
|
|
||
| | Input | Description | Required | Default | | ||
| | --- | --- | --- | --- | | ||
| | `additional-instructions` | Repo-specific instructions appended to the agent prompt | No | `""` | | ||
| | `setup-commands` | Shell commands run before the agent starts | No | `""` | | ||
| | `allowed-bot-users` | Allowlisted bot actor usernames (comma-separated) | No | `github-actions[bot]` | | ||
|
|
||
| ## Safe Outputs | ||
|
|
||
| - `add-comment` — post an analysis comment on the PR (max 1) | ||
| - `add-labels` — label the PR when human review or higher risk is detected (max 3) | ||
|
|
||
| ## Manual usage with mention-in-pr | ||
|
|
||
| You can also analyze any dependency update PR on demand using `mention-in-pr`. Comment on the PR with: | ||
|
|
||
| ``` | ||
| /ai Analyze this dependency update PR: classify each dependency by ecosystem, check for breaking changes, summarize changelog highlights, assess usage compatibility, and note testability in PR context. | ||
| ``` | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,19 @@ | ||
| name: Dependency Review | ||
| on: | ||
| pull_request: | ||
| types: [opened, synchronize, reopened] | ||
|
|
||
| permissions: | ||
| actions: read | ||
| contents: read | ||
| issues: write | ||
| pull-requests: write | ||
|
|
||
| jobs: | ||
| run: | ||
| if: >- | ||
| github.event.pull_request.user.login == 'dependabot[bot]' || | ||
| github.event.pull_request.user.login == 'renovate[bot]' | ||
| uses: elastic/ai-github-actions/.github/workflows/gh-aw-dependency-review.lock.yml@v0 | ||
| secrets: | ||
| COPILOT_GITHUB_TOKEN: ${{ secrets.COPILOT_GITHUB_TOKEN }} |
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
🧩 Analysis chain
🏁 Script executed:
Repository: elastic/ai-github-actions
Length of output: 33207
🏁 Script executed:
Repository: elastic/ai-github-actions
Length of output: 643
🏁 Script executed:
Repository: elastic/ai-github-actions
Length of output: 861
Update
allowed-bot-usersdefault or clarify trigger mismatch.The workflow triggers on
dependabot[bot]andrenovate[bot]PRs (line 23), but theallowed-bot-usersinput defaults togithub-actions[bot](line 31). If users copy the example without overriding this parameter, the workflow will run on dependency PRs but exclude the bots that trigger it. Either adddependabot[bot],renovate[bot]to the default allowlist, or update the documentation to require users explicitly set this parameter.🤖 Prompt for AI Agents
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@coderabbitai fix this for me
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
🧩 Analysis chain
@strawgate, I'll fix this for you. Let me first examine the workflow source files to understand where the default is defined.🏁 Script executed:
Length of output: 9759
@strawgate, I've examined the workflow source. The actual behavior is correct –dependabot[bot]andrenovate[bot]are always allowed by default (hardcoded in the workflow'sbotsconfiguration), in addition to whatever is specified inallowed-bot-users. The issue is just that the README doesn't clarify this. Let me fix the documentation:Task: Fix README documentation for allowed-bot-users default
The
allowed-bot-usersinput defaults togithub-actions[bot], but the workflow also hardcodesdependabot[bot]andrenovate[bot]as allowed bots. Update the README to clarify this.🧠 Learnings used
💡 Use
@coderabbitaiplan to create a plan for code changes