diff --git a/.github/agents/dependabot-pr-reviewer.agent.md b/.github/agents/dependabot-pr-reviewer.agent.md index 4ec3b1c6..2a616892 100644 --- a/.github/agents/dependabot-pr-reviewer.agent.md +++ b/.github/agents/dependabot-pr-reviewer.agent.md @@ -91,43 +91,100 @@ Render the review body as markdown in this order: * The PR diff touches `.github/workflows/**`. * The PR author is not `dependabot[bot]`. -## Validation Execution - -When a high-risk trigger fires, execute the corresponding validation commands before finalizing the review. Report results inline so maintainers have concrete evidence rather than just advisory text. - -### Execution Rules - -* Run validation only for packages classified as high-risk by the surface rubric. -* Execute the validation command(s) listed in the surface table's "Validation advice" column. -* Cap total validation time at 5 minutes. If a command exceeds this, report the timeout and skip remaining checks. -* Never run validation for low-risk patches or dev-only bumps. - -### Per-Surface Validation Commands - -| Surface | Trigger condition | Commands to run | -| --- | --- | --- | -| python-runtime (dataviewer) | ABI-sensitive package major bump (`pyarrow`, `numpy`, `scipy`, `opencv*`, `pandas`) | `cd data-management/viewer && uv sync --extra dev --extra all && uv run ruff check backend/src/` then `uv run pytest backend/tests/ --tb=short -q` | -| python-runtime (evaluation) | ABI-sensitive package major bump | `cd evaluation && uv sync && uv run ruff check . && uv run pytest --tb=short -q` | -| python-runtime (training) | ABI-sensitive package major bump | `cd training && uv sync && uv run ruff check . && uv run pytest --tb=short -q` | -| dataviewer-frontend | Major bump of React, Vite, TypeScript, Tailwind, or peer-dep conflict | `cd data-management/viewer/frontend && npm ci && npm run validate` | -| terraform-providers | `azurerm` major bump or breaking-change boundary | `cd infrastructure/terraform && terraform init -backend=false && terraform validate` | -| gomod | Major version bump of direct dependency | `go mod verify && go vet ./... && go build ./...` | - -### Reporting Validation Results - -Include a `### Validation Results` block in the per-package section of the review body: - -* Show the command(s) executed. -* Report pass/fail counts (for example `411 passed, 110 skipped, 1 failed`). -* If all relevant tests pass, state: `✅ Automated validation passed — safe to merge based on lint and test results.` -* If tests fail, list the failing test names and indicate whether failures are related to the dependency bump or pre-existing. Distinguish by checking if the same tests fail on the base branch. -* If validation cannot run (missing tooling, timeout, environment limitation), state: `⚠️ Validation skipped: . Manual validation recommended.` +## Validation Signal + +The agent runs AFTER the `PR Validation` orchestrator finishes (`workflow_run` +trigger). Use the deterministic CI conclusion as the canonical validation +signal. Do not invoke `uv`, `pytest`, `npm ci`, `terraform`, or `go` from the +bash tool — those binaries live on the host runner and are not visible +inside the AWF firewall sandbox. + +The orchestrator's overall conclusion is injected into the prompt as +`PR_VALIDATION_CONCLUSION` (one of `success`, `failure`, `cancelled`, +`neutral`, `skipped`, `timed_out`, `action_required`). Map the touched +surfaces to the per-job check runs below and read each conclusion via the +`github` MCP `pull_requests` toolset (or `GET /repos/{owner}/{repo}/commits/{sha}/check-runs`). + +### Surface to Check Run Map + +| Surface | Authoritative check runs | +| --- | --- | +| dataviewer-frontend | `Dataviewer Frontend Tests` | +| python-runtime (dataviewer) | `Dataviewer Backend Pytest`, `Pytest Data Management Tools`, `Python Lint` | +| python-runtime (evaluation) | `Evaluation Pytest Tests`, `Pytest Inference`, `Python Lint` | +| python-runtime (training) | `Pytest Training`, `Python Lint` | +| training-rl-abi | `Pytest Training` (hosted CI cannot exercise Isaac Sim GPU paths) | +| terraform-providers | `Terraform Validation`, `Terraform Lint`, `Terraform Tests` | +| terraform-modules | `Terraform Tests`, `Terraform Validation` | +| gomod | `Go Tests`, `Go Lint` | +| docker | `Binary Integrity Check`, `Binary Dependency Freshness` | +| github-actions | `Workflow Permissions Scan`, `SHA Staleness Check`, `Dependency Pinning Scan` | + +### Static Impact Reasoning + +Complement the CI signal with manifest-level reasoning the sandbox CAN do +safely using `cat`, `grep`, `jq`, `npm view`, and `web-fetch`. These checks +must run regardless of CI conclusion: + +* **Isaac Sim ABI guard (training-rl-abi).** When the diff touches + `training/rl/requirements.txt` or `training/rl/pyproject.toml`, read + `training/rl/scripts/train.sh` and confirm the pin + `numpy>=1.26.0,<2.0.0` is still satisfied by the resolved version in + `training/rl/requirements.txt`. A `numpy` 2.x bump MUST be flagged as + high-risk regardless of advisory severity or CI conclusion. Cite both file + paths in the comment. +* **Torch / tensordict / onnxruntime-gpu.** A major bump invalidates GPU + smoke testing; flag as high-risk and note that hosted CI cannot validate + Isaac Sim behavior. +* **Dataviewer frontend peer-dep conflicts.** Run + `npm view @ peerDependencies` and compare against the + pinned `react`, `vite`, `typescript`, and `tailwindcss` versions in + `data-management/viewer/frontend/package.json`. Quote any peer-dep range + the new version breaches. +* **Terraform provider majors.** Read the upstream provider changelog via + `web-fetch` (registry.terraform.io or the provider repo `CHANGELOG.md`) + and quote any breaking input/output rename relevant to the modules under + `infrastructure/terraform/`. +* **Go module direct majors.** Quote the affected `go.mod` `module` line(s) + from the diff and note whether replace/retract directives changed. + +### Reporting + +Include a `### Validation Signal` block in the per-package section of the +review body with three parts: + +1. **Deterministic CI:** quote the orchestrator conclusion as + `PR Validation: ` followed by a bullet list of the relevant + per-surface check runs from the map above with name, conclusion, and + `html_url`. When `conclusion != success`, list the failing job names + first. +2. **Static impact reasoning:** one or two sentences citing the static + checks above. Always include the Isaac Sim ABI line when + `training/rl/requirements.txt` is in the diff, even on minor bumps. +3. **Banner:** if any high-risk trigger fired (advisory severity, ABI guard + violation, peer-dep conflict, breaking-changelog quote), prepend + `⚠️ Maintainer review recommended` to the top of the review body once. + +If the orchestrator conclusion is unavailable (workflow not yet completed, +PR resolution failed, or check-runs API returns empty), state: +`⚠️ Deterministic CI conclusion unavailable; verdict is advisory only.` +and keep the verdict at `COMMENT`. ### Verdict Adjustment -* When a high-risk bump passes all validation checks, the verdict MAY be upgraded from `COMMENT` to `APPROVE` with rationale: "High-risk trigger fired but automated validation confirms compatibility." -* When validation fails with bump-related errors, keep the verdict as `COMMENT` and include the failure details. -* When validation is skipped or inconclusive, keep the verdict as `COMMENT` with the original advisory text. +* `PR_VALIDATION_CONCLUSION == success` AND every relevant per-surface check + is `success` AND no static check raises a concern → verdict MAY upgrade + from `COMMENT` to `APPROVE`. Rationale must reference the orchestrator + conclusion plus the green per-surface checks by name. +* `PR_VALIDATION_CONCLUSION` is `failure`, `cancelled`, or `timed_out` → + verdict stays at `COMMENT`. Body MUST quote each failing per-surface + check name plus its `html_url`. Do NOT skip enrichment — maintainers rely + on the advisory output to triage which package in a grouped PR caused + the failure. +* `PR_VALIDATION_CONCLUSION` is `neutral`, `skipped`, or `action_required` + → verdict stays at `COMMENT`; body explains the inconclusive state. +* The Isaac Sim ABI guard is sticky: a `numpy` 2.x bump keeps the verdict + at `COMMENT` and forces the high-risk banner regardless of CI conclusion. ## Forbidden Actions diff --git a/.github/workflows/aw-dependabot-pr-review.lock.yml b/.github/workflows/aw-dependabot-pr-review.lock.yml index b9810d89..739014ed 100644 --- a/.github/workflows/aw-dependabot-pr-review.lock.yml +++ b/.github/workflows/aw-dependabot-pr-review.lock.yml @@ -1,4 +1,4 @@ -# gh-aw-metadata: {"schema_version":"v3","frontmatter_hash":"68ef551c4fc2aa15b4424db61966e109b10bc8c3d96879dc44c7669fcd3d0b0b","compiler_version":"v0.68.3","strict":true,"agent_id":"copilot"} +# gh-aw-metadata: {"schema_version":"v3","frontmatter_hash":"c966d2851ca55ead0d96c99a3a2459006eedf6227325f83cb4414955742356d2","compiler_version":"v0.68.3","strict":true,"agent_id":"copilot"} # gh-aw-manifest: {"version":1,"secrets":["COPILOT_GITHUB_TOKEN","GH_AW_GITHUB_MCP_SERVER_TOKEN","GH_AW_GITHUB_TOKEN","GITHUB_TOKEN"],"actions":[{"repo":"actions/checkout","sha":"de0fac2e4500dabe0009e67214ff5f5447ce83dd","version":"v6.0.2"},{"repo":"actions/download-artifact","sha":"3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c","version":"v8.0.1"},{"repo":"actions/github-script","sha":"373c709c69115d41ff229c7e5df9f8788daa9553","version":"v9"},{"repo":"actions/setup-go","sha":"4a3601121dd01d1626a1e23e37211e3254c1c06c","version":"v6.4.0"},{"repo":"actions/setup-node","sha":"53b83947a5a98c8d113130e565377fae1a50d02f","version":"v6.3.0"},{"repo":"actions/setup-python","sha":"a309ff8b426b58ec0e2a45f0f869d46889d02405","version":"v6.2.0"},{"repo":"actions/upload-artifact","sha":"043fb46d1a93c77aae656e7c1c64a875d1fc6a0a","version":"v7.0.1"},{"repo":"astral-sh/setup-uv","sha":"cec208311dfd045dd5311c1add060b2062131d57","version":"v8.0.0"},{"repo":"github/gh-aw-actions/setup","sha":"ba90f2186d7ad780ec640f364005fa24e797b360","version":"v0.68.3"},{"repo":"hashicorp/setup-terraform","sha":"5e8dbf3c6d9deaf4193ca7a8fb23f2ac83bb6c85","version":"5e8dbf3c6d9deaf4193ca7a8fb23f2ac83bb6c85"},{"repo":"terraform-linters/setup-tflint","sha":"b480b8fcdaa6f2c577f8e4fa799e89e756bb7c93","version":"b480b8fcdaa6f2c577f8e4fa799e89e756bb7c93"}],"containers":[{"image":"ghcr.io/github/gh-aw-firewall/agent:0.25.20"},{"image":"ghcr.io/github/gh-aw-firewall/api-proxy:0.25.20"},{"image":"ghcr.io/github/gh-aw-firewall/squid:0.25.20"},{"image":"ghcr.io/github/gh-aw-mcpg:v0.2.19"},{"image":"ghcr.io/github/github-mcp-server:v0.32.0"},{"image":"node:lts-alpine"}]} # ___ _ _ # / _ \ | | (_) @@ -57,60 +57,42 @@ name: "AW Dependabot PR Review" "on": - # bots: # Bots processed as bot check in pre-activation job - # - dependabot[bot] # Bots processed as bot check in pre-activation job - pull_request: - paths: - - "**/package.json" - - "**/package-lock.json" - - "**/pnpm-lock.yaml" - - "**/pyproject.toml" - - "**/uv.lock" - - "**/requirements*.txt" - - "**/go.mod" - - "**/go.sum" - - "**/*.tf" - - "**/*.tfvars" - - "**/Dockerfile*" - - "!.github/workflows/**" + workflow_run: + # zizmor: ignore[dangerous-triggers] - workflow_run trigger is secured with role and fork validation + branches: + - main types: - - opened - - synchronize - - reopened + - completed + workflows: + - PR Validation permissions: {} concurrency: - group: "gh-aw-${{ github.workflow }}-${{ github.event.pull_request.number || github.ref || github.run_id }}" - cancel-in-progress: true + group: "gh-aw-${{ github.workflow }}" run-name: "AW Dependabot PR Review" jobs: activation: needs: pre_activation + # zizmor: ignore[dangerous-triggers] - workflow_run trigger is secured with role and fork validation if: > - needs.pre_activation.outputs.activated == 'true' && ((github.event.pull_request.draft == false && github.event.pull_request.user.login == 'dependabot[bot]' - ) && (github.event_name != 'pull_request' || github.event.pull_request.head.repo.id == github.repository_id)) + (needs.pre_activation.outputs.activated == 'true' && (github.event.workflow_run.event == 'pull_request' && + github.event.workflow_run.conclusion != null)) && (github.event_name != 'workflow_run' || github.event.workflow_run.repository.id == github.repository_id && + (!(github.event.workflow_run.repository.fork))) runs-on: ubuntu-slim permissions: actions: read contents: read - discussions: write - issues: write - pull-requests: write outputs: - body: ${{ steps.sanitized.outputs.body }} - comment_id: ${{ steps.add-comment.outputs.comment-id }} - comment_repo: ${{ steps.add-comment.outputs.comment-repo }} - comment_url: ${{ steps.add-comment.outputs.comment-url }} + comment_id: "" + comment_repo: "" lockdown_check_failed: ${{ steps.generate_aw_info.outputs.lockdown_check_failed == 'true' }} model: ${{ steps.generate_aw_info.outputs.model }} secret_verification_result: ${{ steps.validate-secret.outputs.verification_result }} setup-trace-id: ${{ steps.setup.outputs.trace-id }} stale_lock_file_failed: ${{ steps.check-lock-file.outputs.stale_lock_file_failed == 'true' }} - text: ${{ steps.sanitized.outputs.text }} - title: ${{ steps.sanitized.outputs.title }} steps: - name: Setup Scripts id: setup @@ -145,19 +127,6 @@ jobs: setupGlobals(core, github, context, exec, io, getOctokit); const { main } = require('${{ runner.temp }}/gh-aw/actions/generate_aw_info.cjs'); await main(core, context); - - name: Add eyes reaction for immediate feedback - id: react - if: github.event_name == 'issues' || github.event_name == 'issue_comment' || github.event_name == 'pull_request_review_comment' || github.event_name == 'discussion' || github.event_name == 'discussion_comment' || github.event_name == 'pull_request' && github.event.pull_request.head.repo.id == github.repository_id - uses: actions/github-script@373c709c69115d41ff229c7e5df9f8788daa9553 # v9 - env: - GH_AW_REACTION: "eyes" - with: - github-token: ${{ secrets.GITHUB_TOKEN }} - script: | - const { setupGlobals } = require('${{ runner.temp }}/gh-aw/actions/setup_globals.cjs'); - setupGlobals(core, github, context, exec, io, getOctokit); - const { main } = require('${{ runner.temp }}/gh-aw/actions/add_reaction.cjs'); - await main(); - name: Validate COPILOT_GITHUB_TOKEN secret id: validate-secret run: bash "${RUNNER_TEMP}/gh-aw/actions/validate_multi_secret.sh" COPILOT_GITHUB_TOKEN 'GitHub Copilot CLI' https://github.github.com/gh-aw/reference/engines/#github-copilot-default @@ -194,29 +163,6 @@ jobs: setupGlobals(core, github, context, exec, io, getOctokit); const { main } = require('${{ runner.temp }}/gh-aw/actions/check_version_updates.cjs'); await main(); - - name: Compute current body text - id: sanitized - uses: actions/github-script@373c709c69115d41ff229c7e5df9f8788daa9553 # v9 - env: - GH_AW_ALLOWED_BOTS: "dependabot[bot]" - with: - script: | - const { setupGlobals } = require('${{ runner.temp }}/gh-aw/actions/setup_globals.cjs'); - setupGlobals(core, github, context, exec, io, getOctokit); - const { main } = require('${{ runner.temp }}/gh-aw/actions/compute_text.cjs'); - await main(); - - name: Add comment with workflow run link - id: add-comment - if: github.event_name == 'issues' || github.event_name == 'issue_comment' || github.event_name == 'pull_request_review_comment' || github.event_name == 'discussion' || github.event_name == 'discussion_comment' || github.event_name == 'pull_request' && github.event.pull_request.head.repo.id == github.repository_id - uses: actions/github-script@373c709c69115d41ff229c7e5df9f8788daa9553 # v9 - env: - GH_AW_WORKFLOW_NAME: "AW Dependabot PR Review" - with: - script: | - const { setupGlobals } = require('${{ runner.temp }}/gh-aw/actions/setup_globals.cjs'); - setupGlobals(core, github, context, exec, io, getOctokit); - const { main } = require('${{ runner.temp }}/gh-aw/actions/add_workflow_run_comment.cjs'); - await main(); - name: Create prompt with built-in context env: GH_AW_PROMPT: /tmp/gh-aw/aw-prompts/prompt.txt @@ -233,14 +179,14 @@ jobs: run: | bash "${RUNNER_TEMP}/gh-aw/actions/create_prompt_first.sh" { - cat << 'GH_AW_PROMPT_84acb7c6bbb4f5ce_EOF' + cat << 'GH_AW_PROMPT_1179d66472df9aa2_EOF' - GH_AW_PROMPT_84acb7c6bbb4f5ce_EOF + GH_AW_PROMPT_1179d66472df9aa2_EOF cat "${RUNNER_TEMP}/gh-aw/prompts/xpia.md" cat "${RUNNER_TEMP}/gh-aw/prompts/temp_folder_prompt.md" cat "${RUNNER_TEMP}/gh-aw/prompts/markdown.md" cat "${RUNNER_TEMP}/gh-aw/prompts/safe_outputs_prompt.md" - cat << 'GH_AW_PROMPT_84acb7c6bbb4f5ce_EOF' + cat << 'GH_AW_PROMPT_1179d66472df9aa2_EOF' Tools: add_comment(max:2), create_pull_request_review_comment(max:5), submit_pull_request_review, missing_tool, missing_data, noop @@ -272,13 +218,13 @@ jobs: {{/if}} - GH_AW_PROMPT_84acb7c6bbb4f5ce_EOF + GH_AW_PROMPT_1179d66472df9aa2_EOF cat "${RUNNER_TEMP}/gh-aw/prompts/github_mcp_tools_with_safeoutputs_prompt.md" - cat << 'GH_AW_PROMPT_84acb7c6bbb4f5ce_EOF' + cat << 'GH_AW_PROMPT_1179d66472df9aa2_EOF' {{#runtime-import .github/agents/dependabot-pr-reviewer.agent.md}} {{#runtime-import .github/workflows/aw-dependabot-pr-review.md}} - GH_AW_PROMPT_84acb7c6bbb4f5ce_EOF + GH_AW_PROMPT_1179d66472df9aa2_EOF } > "$GH_AW_PROMPT" - name: Interpolate variables and render templates uses: actions/github-script@373c709c69115d41ff229c7e5df9f8788daa9553 # v9 @@ -351,8 +297,11 @@ jobs: needs: activation runs-on: ubuntu-latest permissions: + actions: read contents: read pull-requests: read + concurrency: + group: "gh-aw-copilot-${{ github.workflow }}" env: DEFAULT_BRANCH: ${{ github.event.repository.default_branch }} GH_AW_ASSETS_ALLOWED_EXTS: "" @@ -445,6 +394,11 @@ jobs: uses: terraform-linters/setup-tflint@b480b8fcdaa6f2c577f8e4fa799e89e756bb7c93 # b480b8fcdaa6f2c577f8e4fa799e89e756bb7c93 with: tflint_version: latest + - id: resolve-pr + name: Resolve Dependabot PR context from triggering workflow_run + uses: actions/github-script@373c709c69115d41ff229c7e5df9f8788daa9553 # v9 + with: + script: "const wr = context.payload.workflow_run;\nif (!wr) {\n core.setFailed('workflow_run payload missing');\n return;\n}\ncore.exportVariable('PR_VALIDATION_CONCLUSION', wr.conclusion || 'unknown');\ncore.exportVariable('PR_VALIDATION_RUN_URL', wr.html_url || '');\ncore.exportVariable('PR_HEAD_SHA', wr.head_sha || '');\n\nconst prs = wr.pull_requests || [];\nlet prNumber = prs.length ? prs[0].number : null;\nif (!prNumber && wr.head_branch) {\n // workflow_run may not populate pull_requests for forks; resolve via search.\n const { data: search } = await github.rest.search.issuesAndPullRequests({\n q: `repo:${context.repo.owner}/${context.repo.repo} is:pr head:${wr.head_branch} state:open`,\n per_page: 1,\n });\n if (search.items.length) prNumber = search.items[0].number;\n}\nif (!prNumber) {\n core.warning('Could not resolve a PR for this workflow_run; emitting noop.');\n core.exportVariable('PR_DEPENDABOT_SKIP_REASON', 'no-pr-resolved');\n return;\n}\nconst { data: pr } = await github.rest.pulls.get({\n owner: context.repo.owner,\n repo: context.repo.repo,\n pull_number: prNumber,\n});\nif (pr.user.login !== 'dependabot[bot]') {\n core.exportVariable('PR_DEPENDABOT_SKIP_REASON', 'not-dependabot');\n return;\n}\nif (pr.draft) {\n core.exportVariable('PR_DEPENDABOT_SKIP_REASON', 'draft');\n return;\n}\ncore.exportVariable('PR_NUMBER', String(pr.number));\ncore.exportVariable('PR_TITLE', pr.title);\ncore.exportVariable('PR_HEAD_REF', pr.head.ref);\ncore.exportVariable('PR_BASE_REF', pr.base.ref);\ncore.exportVariable('PR_AUTHOR', pr.user.login);\ncore.info(`Resolved PR #${pr.number} (${pr.title}); PR Validation conclusion: ${wr.conclusion}`);\n" - name: Configure Git credentials env: @@ -496,9 +450,9 @@ jobs: mkdir -p "${RUNNER_TEMP}/gh-aw/safeoutputs" mkdir -p /tmp/gh-aw/safeoutputs mkdir -p /tmp/gh-aw/mcp-logs/safeoutputs - cat > "${RUNNER_TEMP}/gh-aw/safeoutputs/config.json" << 'GH_AW_SAFE_OUTPUTS_CONFIG_56b9412cccfc2269_EOF' + cat > "${RUNNER_TEMP}/gh-aw/safeoutputs/config.json" << 'GH_AW_SAFE_OUTPUTS_CONFIG_08db9facaf83d198_EOF' {"add_comment":{"max":2,"target":"triggering"},"create_pull_request_review_comment":{"max":5,"side":"RIGHT"},"create_report_incomplete_issue":{},"missing_data":{},"missing_tool":{},"noop":{"max":1,"report-as-issue":"true"},"report_incomplete":{},"submit_pull_request_review":{"max":1}} - GH_AW_SAFE_OUTPUTS_CONFIG_56b9412cccfc2269_EOF + GH_AW_SAFE_OUTPUTS_CONFIG_08db9facaf83d198_EOF - name: Write Safe Outputs Tools env: GH_AW_TOOLS_META_JSON: | @@ -739,7 +693,7 @@ jobs: export MCP_GATEWAY_DOCKER_COMMAND='docker run -i --rm --network host -v /var/run/docker.sock:/var/run/docker.sock -e MCP_GATEWAY_PORT -e MCP_GATEWAY_DOMAIN -e MCP_GATEWAY_API_KEY -e MCP_GATEWAY_PAYLOAD_DIR -e MCP_GATEWAY_PAYLOAD_SIZE_THRESHOLD -e DEBUG -e MCP_GATEWAY_LOG_DIR -e GH_AW_MCP_LOG_DIR -e GH_AW_SAFE_OUTPUTS -e GH_AW_SAFE_OUTPUTS_CONFIG_PATH -e GH_AW_SAFE_OUTPUTS_TOOLS_PATH -e GH_AW_ASSETS_BRANCH -e GH_AW_ASSETS_MAX_SIZE_KB -e GH_AW_ASSETS_ALLOWED_EXTS -e DEFAULT_BRANCH -e GITHUB_MCP_SERVER_TOKEN -e GITHUB_MCP_GUARD_MIN_INTEGRITY -e GITHUB_MCP_GUARD_REPOS -e GITHUB_REPOSITORY -e GITHUB_SERVER_URL -e GITHUB_SHA -e GITHUB_WORKSPACE -e GITHUB_TOKEN -e GITHUB_RUN_ID -e GITHUB_RUN_NUMBER -e GITHUB_RUN_ATTEMPT -e GITHUB_JOB -e GITHUB_ACTION -e GITHUB_EVENT_NAME -e GITHUB_EVENT_PATH -e GITHUB_ACTOR -e GITHUB_ACTOR_ID -e GITHUB_TRIGGERING_ACTOR -e GITHUB_WORKFLOW -e GITHUB_WORKFLOW_REF -e GITHUB_WORKFLOW_SHA -e GITHUB_REF -e GITHUB_REF_NAME -e GITHUB_REF_TYPE -e GITHUB_HEAD_REF -e GITHUB_BASE_REF -e GH_AW_SAFE_OUTPUTS_PORT -e GH_AW_SAFE_OUTPUTS_API_KEY -v /tmp/gh-aw/mcp-payloads:/tmp/gh-aw/mcp-payloads:rw -v /opt:/opt:ro -v /tmp:/tmp:rw -v '"${GITHUB_WORKSPACE}"':'"${GITHUB_WORKSPACE}"':rw ghcr.io/github/gh-aw-mcpg:v0.2.19' mkdir -p /home/runner/.copilot - cat << GH_AW_MCP_CONFIG_1ee5d9a7985d3147_EOF | bash "${RUNNER_TEMP}/gh-aw/actions/start_mcp_gateway.sh" + cat << GH_AW_MCP_CONFIG_a078df3ff30c98cc_EOF | bash "${RUNNER_TEMP}/gh-aw/actions/start_mcp_gateway.sh" { "mcpServers": { "github": { @@ -780,7 +734,7 @@ jobs: "payloadDir": "${MCP_GATEWAY_PAYLOAD_DIR}" } } - GH_AW_MCP_CONFIG_1ee5d9a7985d3147_EOF + GH_AW_MCP_CONFIG_a078df3ff30c98cc_EOF - name: Download activation artifact uses: actions/download-artifact@3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c # v8.0.1 with: @@ -798,19 +752,11 @@ jobs: # --allow-tool shell(cat **/*.tf) # --allow-tool shell(cat **/*.toml) # --allow-tool shell(cat **/go.mod) + # --allow-tool shell(cat training/rl/requirements.txt) + # --allow-tool shell(cat training/rl/scripts/train.sh) # --allow-tool shell(cat) - # --allow-tool shell(cd data-management/viewer && uv run pytest backend/tests/ --tb=short -q) - # --allow-tool shell(cd data-management/viewer && uv sync --extra dev --extra all && uv run ruff check backend/src/) - # --allow-tool shell(cd data-management/viewer/frontend && npm ci && npm run validate) - # --allow-tool shell(cd evaluation && uv sync && uv run ruff check . && uv run pytest --tb=short -q) - # --allow-tool shell(cd infrastructure/terraform && terraform fmt -check -recursive) - # --allow-tool shell(cd infrastructure/terraform && terraform init -backend=false && terraform validate) - # --allow-tool shell(cd training && uv sync && uv run ruff check . && uv run pytest --tb=short -q) # --allow-tool shell(date) # --allow-tool shell(echo) - # --allow-tool shell(go build ./...) - # --allow-tool shell(go mod verify) - # --allow-tool shell(go vet ./...) # --allow-tool shell(grep -R --line-number * -- :!node_modules :!.venv :!external) # --allow-tool shell(grep) # --allow-tool shell(head) @@ -821,7 +767,6 @@ jobs: # --allow-tool shell(sort) # --allow-tool shell(tail) # --allow-tool shell(uniq) - # --allow-tool shell(uv tree) # --allow-tool shell(wc) # --allow-tool shell(yq) # --allow-tool web_fetch @@ -833,7 +778,7 @@ jobs: (umask 177 && touch /tmp/gh-aw/agent-stdio.log) # shellcheck disable=SC1003 sudo -E awf --container-workdir "${GITHUB_WORKSPACE}" --mount "${RUNNER_TEMP}/gh-aw:${RUNNER_TEMP}/gh-aw:ro" --mount "${RUNNER_TEMP}/gh-aw:/host${RUNNER_TEMP}/gh-aw:ro" --env-all --exclude-env COPILOT_GITHUB_TOKEN --exclude-env GITHUB_MCP_SERVER_TOKEN --exclude-env MCP_GATEWAY_API_KEY --allow-domains '*.docker.com,*.docker.io,*.githubusercontent.com,*.pythonhosted.org,anaconda.org,api.business.githubcopilot.com,api.enterprise.githubcopilot.com,api.github.com,api.githubcopilot.com,api.individual.githubcopilot.com,api.npms.io,api.osv.dev,api.snapcraft.io,apt.releases.hashicorp.com,archive.ubuntu.com,auth.docker.io,azure.archive.ubuntu.com,binstar.org,bootstrap.pypa.io,bun.sh,cdn.jsdelivr.net,codeload.github.com,conda.anaconda.org,conda.binstar.org,crates.io,crl.geotrust.com,crl.globalsign.com,crl.identrust.com,crl.sectigo.com,crl.thawte.com,crl.usertrust.com,crl.verisign.com,crl3.digicert.com,crl4.digicert.com,crls.ssl.com,deb.nodesource.com,deno.land,dl.k8s.io,docs.github.com,esm.sh,files.pythonhosted.org,gcr.io,get.pnpm.io,ghcr.io,github-cloud.githubusercontent.com,github-cloud.s3.amazonaws.com,github.blog,github.com,github.githubassets.com,go.dev,golang.org,googleapis.deno.dev,googlechromelabs.github.io,goproxy.io,host.docker.internal,index.crates.io,json-schema.org,json.schemastore.org,jsr.io,keyserver.ubuntu.com,lfs.github.com,mcr.microsoft.com,nodejs.org,npm.pkg.github.com,npmjs.com,npmjs.org,objects.githubusercontent.com,ocsp.digicert.com,ocsp.geotrust.com,ocsp.globalsign.com,ocsp.identrust.com,ocsp.sectigo.com,ocsp.ssl.com,ocsp.thawte.com,ocsp.usertrust.com,ocsp.verisign.com,packagecloud.io,packages.cloud.google.com,packages.microsoft.com,pip.pypa.io,pkg.go.dev,pkgs.k8s.io,ppa.launchpad.net,production.cloudflare.docker.com,proxy.golang.org,pypi.org,pypi.python.org,quay.io,raw.githubusercontent.com,registry.bower.io,registry.hub.docker.com,registry.npmjs.com,registry.npmjs.org,registry.terraform.io,registry.yarnpkg.com,releases.hashicorp.com,repo.anaconda.com,repo.continuum.io,repo.yarnpkg.com,s.symcb.com,s.symcd.com,security.ubuntu.com,services.nvd.nist.gov,skimdb.npmjs.com,static.crates.io,storage.googleapis.com,sum.golang.org,telemetry.enterprise.githubcopilot.com,telemetry.vercel.com,ts-crl.ws.symantec.com,ts-ocsp.ws.symantec.com,www.googleapis.com,www.npmjs.com,www.npmjs.org,yarnpkg.com,yum.releases.hashicorp.com' --log-level info --proxy-logs-dir /tmp/gh-aw/sandbox/firewall/logs --audit-dir /tmp/gh-aw/sandbox/firewall/audit --enable-host-access --image-tag 0.25.20 --skip-pull --enable-api-proxy \ - -- /bin/bash -c 'node ${RUNNER_TEMP}/gh-aw/actions/copilot_driver.cjs /usr/local/bin/copilot --add-dir /tmp/gh-aw/ --log-level all --log-dir /tmp/gh-aw/sandbox/agent/logs/ --disable-builtin-mcps --no-ask-user --allow-tool github --allow-tool safeoutputs --allow-tool '\''shell(cat **/*.json)'\'' --allow-tool '\''shell(cat **/*.tf)'\'' --allow-tool '\''shell(cat **/*.toml)'\'' --allow-tool '\''shell(cat **/go.mod)'\'' --allow-tool '\''shell(cat)'\'' --allow-tool '\''shell(cd data-management/viewer && uv run pytest backend/tests/ --tb=short -q)'\'' --allow-tool '\''shell(cd data-management/viewer && uv sync --extra dev --extra all && uv run ruff check backend/src/)'\'' --allow-tool '\''shell(cd data-management/viewer/frontend && npm ci && npm run validate)'\'' --allow-tool '\''shell(cd evaluation && uv sync && uv run ruff check . && uv run pytest --tb=short -q)'\'' --allow-tool '\''shell(cd infrastructure/terraform && terraform fmt -check -recursive)'\'' --allow-tool '\''shell(cd infrastructure/terraform && terraform init -backend=false && terraform validate)'\'' --allow-tool '\''shell(cd training && uv sync && uv run ruff check . && uv run pytest --tb=short -q)'\'' --allow-tool '\''shell(date)'\'' --allow-tool '\''shell(echo)'\'' --allow-tool '\''shell(go build ./...)'\'' --allow-tool '\''shell(go mod verify)'\'' --allow-tool '\''shell(go vet ./...)'\'' --allow-tool '\''shell(grep -R --line-number * -- :!node_modules :!.venv :!external)'\'' --allow-tool '\''shell(grep)'\'' --allow-tool '\''shell(head)'\'' --allow-tool '\''shell(jq . **/*.json)'\'' --allow-tool '\''shell(ls)'\'' --allow-tool '\''shell(npm view *)'\'' --allow-tool '\''shell(pwd)'\'' --allow-tool '\''shell(sort)'\'' --allow-tool '\''shell(tail)'\'' --allow-tool '\''shell(uniq)'\'' --allow-tool '\''shell(uv tree)'\'' --allow-tool '\''shell(wc)'\'' --allow-tool '\''shell(yq)'\'' --allow-tool web_fetch --allow-tool write --allow-all-paths --add-dir "${GITHUB_WORKSPACE}" --prompt "$(cat /tmp/gh-aw/aw-prompts/prompt.txt)"' 2>&1 | tee -a /tmp/gh-aw/agent-stdio.log + -- /bin/bash -c 'node ${RUNNER_TEMP}/gh-aw/actions/copilot_driver.cjs /usr/local/bin/copilot --add-dir /tmp/gh-aw/ --log-level all --log-dir /tmp/gh-aw/sandbox/agent/logs/ --disable-builtin-mcps --no-ask-user --allow-tool github --allow-tool safeoutputs --allow-tool '\''shell(cat **/*.json)'\'' --allow-tool '\''shell(cat **/*.tf)'\'' --allow-tool '\''shell(cat **/*.toml)'\'' --allow-tool '\''shell(cat **/go.mod)'\'' --allow-tool '\''shell(cat training/rl/requirements.txt)'\'' --allow-tool '\''shell(cat training/rl/scripts/train.sh)'\'' --allow-tool '\''shell(cat)'\'' --allow-tool '\''shell(date)'\'' --allow-tool '\''shell(echo)'\'' --allow-tool '\''shell(grep -R --line-number * -- :!node_modules :!.venv :!external)'\'' --allow-tool '\''shell(grep)'\'' --allow-tool '\''shell(head)'\'' --allow-tool '\''shell(jq . **/*.json)'\'' --allow-tool '\''shell(ls)'\'' --allow-tool '\''shell(npm view *)'\'' --allow-tool '\''shell(pwd)'\'' --allow-tool '\''shell(sort)'\'' --allow-tool '\''shell(tail)'\'' --allow-tool '\''shell(uniq)'\'' --allow-tool '\''shell(wc)'\'' --allow-tool '\''shell(yq)'\'' --allow-tool web_fetch --allow-tool write --allow-all-paths --add-dir "${GITHUB_WORKSPACE}" --prompt "$(cat /tmp/gh-aw/aw-prompts/prompt.txt)"' 2>&1 | tee -a /tmp/gh-aw/agent-stdio.log env: COPILOT_AGENT_RUNNER_TYPE: STANDALONE COPILOT_GITHUB_TOKEN: ${{ secrets.COPILOT_GITHUB_TOKEN }} @@ -1137,25 +1082,6 @@ jobs: setupGlobals(core, github, context, exec, io, getOctokit); const { main } = require('${{ runner.temp }}/gh-aw/actions/handle_agent_failure.cjs'); await main(); - - name: Update reaction comment with completion status - id: conclusion - uses: actions/github-script@373c709c69115d41ff229c7e5df9f8788daa9553 # v9 - env: - GH_AW_AGENT_OUTPUT: ${{ steps.setup-agent-output-env.outputs.GH_AW_AGENT_OUTPUT }} - GH_AW_COMMENT_ID: ${{ needs.activation.outputs.comment_id }} - GH_AW_COMMENT_REPO: ${{ needs.activation.outputs.comment_repo }} - GH_AW_RUN_URL: ${{ github.server_url }}/${{ github.repository }}/actions/runs/${{ github.run_id }} - GH_AW_WORKFLOW_NAME: "AW Dependabot PR Review" - GH_AW_AGENT_CONCLUSION: ${{ needs.agent.result }} - GH_AW_DETECTION_CONCLUSION: ${{ needs.detection.outputs.detection_conclusion }} - GH_AW_DETECTION_REASON: ${{ needs.detection.outputs.detection_reason }} - with: - github-token: ${{ secrets.GH_AW_GITHUB_TOKEN || secrets.GITHUB_TOKEN }} - script: | - const { setupGlobals } = require('${{ runner.temp }}/gh-aw/actions/setup_globals.cjs'); - setupGlobals(core, github, context, exec, io, getOctokit); - const { main } = require('${{ runner.temp }}/gh-aw/actions/notify_comment_error.cjs'); - await main(); detection: needs: @@ -1316,8 +1242,7 @@ jobs: pre_activation: if: > - (github.event.pull_request.draft == false && github.event.pull_request.user.login == 'dependabot[bot]' - ) && (github.event_name != 'pull_request' || github.event.pull_request.head.repo.id == github.repository_id) + github.event.workflow_run.event == 'pull_request' && github.event.workflow_run.conclusion != null runs-on: ubuntu-slim outputs: activated: ${{ steps.check_membership.outputs.is_team_member == 'true' }} @@ -1335,7 +1260,6 @@ jobs: uses: actions/github-script@373c709c69115d41ff229c7e5df9f8788daa9553 # v9 env: GH_AW_REQUIRED_ROLES: "admin,maintainer,write" - GH_AW_ALLOWED_BOTS: "dependabot[bot]" with: github-token: ${{ secrets.GITHUB_TOKEN }} script: | diff --git a/.github/workflows/aw-dependabot-pr-review.md b/.github/workflows/aw-dependabot-pr-review.md index bcce93c2..7df75fbd 100644 --- a/.github/workflows/aw-dependabot-pr-review.md +++ b/.github/workflows/aw-dependabot-pr-review.md @@ -4,30 +4,18 @@ description: Advisory agentic review of Dependabot dependency update PRs for phy engine: copilot timeout-minutes: 15 if: > - github.event.pull_request.draft == false && - github.event.pull_request.user.login == 'dependabot[bot]' + github.event.workflow_run.event == 'pull_request' && + github.event.workflow_run.conclusion != null on: - pull_request: - types: [opened, synchronize, reopened] - paths: - - "**/package.json" - - "**/package-lock.json" - - "**/pnpm-lock.yaml" - - "**/pyproject.toml" - - "**/uv.lock" - - "**/requirements*.txt" - - "**/go.mod" - - "**/go.sum" - - "**/*.tf" - - "**/*.tfvars" - - "**/Dockerfile*" - - "!.github/workflows/**" - bots: ["dependabot[bot]"] - reaction: eyes - status-comment: true + workflow_run: + workflows: ["PR Validation"] + types: [completed] + branches: + - main permissions: contents: read pull-requests: read + actions: read network: allowed: - defaults @@ -73,6 +61,54 @@ steps: uses: terraform-linters/setup-tflint@b480b8fcdaa6f2c577f8e4fa799e89e756bb7c93 # v6.2.2 with: tflint_version: latest + - name: Resolve Dependabot PR context from triggering workflow_run + id: resolve-pr + uses: actions/github-script@373c709c69115d41ff229c7e5df9f8788daa9553 # v9.0.0 + with: + script: | + const wr = context.payload.workflow_run; + if (!wr) { + core.setFailed('workflow_run payload missing'); + return; + } + core.exportVariable('PR_VALIDATION_CONCLUSION', wr.conclusion || 'unknown'); + core.exportVariable('PR_VALIDATION_RUN_URL', wr.html_url || ''); + core.exportVariable('PR_HEAD_SHA', wr.head_sha || ''); + + const prs = wr.pull_requests || []; + let prNumber = prs.length ? prs[0].number : null; + if (!prNumber && wr.head_branch) { + // workflow_run may not populate pull_requests for forks; resolve via search. + const { data: search } = await github.rest.search.issuesAndPullRequests({ + q: `repo:${context.repo.owner}/${context.repo.repo} is:pr head:${wr.head_branch} state:open`, + per_page: 1, + }); + if (search.items.length) prNumber = search.items[0].number; + } + if (!prNumber) { + core.warning('Could not resolve a PR for this workflow_run; emitting noop.'); + core.exportVariable('PR_DEPENDABOT_SKIP_REASON', 'no-pr-resolved'); + return; + } + const { data: pr } = await github.rest.pulls.get({ + owner: context.repo.owner, + repo: context.repo.repo, + pull_number: prNumber, + }); + if (pr.user.login !== 'dependabot[bot]') { + core.exportVariable('PR_DEPENDABOT_SKIP_REASON', 'not-dependabot'); + return; + } + if (pr.draft) { + core.exportVariable('PR_DEPENDABOT_SKIP_REASON', 'draft'); + return; + } + core.exportVariable('PR_NUMBER', String(pr.number)); + core.exportVariable('PR_TITLE', pr.title); + core.exportVariable('PR_HEAD_REF', pr.head.ref); + core.exportVariable('PR_BASE_REF', pr.base.ref); + core.exportVariable('PR_AUTHOR', pr.user.login); + core.info(`Resolved PR #${pr.number} (${pr.title}); PR Validation conclusion: ${wr.conclusion}`); tools: github: toolsets: [context, repos, pull_requests] @@ -82,21 +118,11 @@ tools: - "cat **/*.toml" - "cat **/go.mod" - "cat **/*.tf" + - "cat training/rl/requirements.txt" + - "cat training/rl/scripts/train.sh" - "grep -R --line-number * -- :!node_modules :!.venv :!external" - "jq . **/*.json" - "npm view *" - - "uv tree" - # Validation commands for high-risk bumps - - "cd data-management/viewer && uv sync --extra dev --extra all && uv run ruff check backend/src/" - - "cd data-management/viewer && uv run pytest backend/tests/ --tb=short -q" - - "cd data-management/viewer/frontend && npm ci && npm run validate" - - "cd evaluation && uv sync && uv run ruff check . && uv run pytest --tb=short -q" - - "cd training && uv sync && uv run ruff check . && uv run pytest --tb=short -q" - - "cd infrastructure/terraform && terraform init -backend=false && terraform validate" - - "cd infrastructure/terraform && terraform fmt -check -recursive" - - "go vet ./..." - - "go build ./..." - - "go mod verify" safe-outputs: create-pull-request-review-comment: max: 5 @@ -113,7 +139,21 @@ imports: # Dependabot PR Review -Advisory-only review of Dependabot-authored pull requests in microsoft/physical-ai-toolchain. The agent classifies risk, enriches findings with GHSA/OSV/NVD intel and release notes, and posts a single review plus targeted inline comments. It never blocks merges. +Advisory-only review of Dependabot-authored pull requests in microsoft/physical-ai-toolchain. The agent classifies risk, enriches findings with GHSA/OSV/NVD intel and release notes, anchors validation on the deterministic `PR Validation` orchestrator that triggered this run, and posts a single review plus targeted inline comments. It never blocks merges. + +## Trigger Posture + +This workflow runs via `workflow_run` after the `PR Validation` orchestrator completes on a PR targeting `main`. The deterministic CI conclusion is the canonical validation signal — read it from the `PR_VALIDATION_CONCLUSION` environment variable injected by the resolver step. The agent must never attempt to run validation tooling (`uv`, `pytest`, `npm ci`, `terraform`, `go`) from the bash tool because those binaries are not visible inside the AWF firewall sandbox. + +The resolver step exports these environment variables for the agent to read: + +* `PR_NUMBER` — the Dependabot PR number under review +* `PR_TITLE`, `PR_HEAD_REF`, `PR_BASE_REF`, `PR_AUTHOR`, `PR_HEAD_SHA` +* `PR_VALIDATION_CONCLUSION` — `success`, `failure`, `cancelled`, `neutral`, `skipped`, `timed_out`, or `action_required` +* `PR_VALIDATION_RUN_URL` — direct link to the `PR Validation` run +* `PR_DEPENDABOT_SKIP_REASON` (optional) — set when the resolver determined the trigger should be skipped (`no-pr-resolved`, `not-dependabot`, `draft`) + +When `PR_DEPENDABOT_SKIP_REASON` is set, emit a `noop` with the reason as the rationale and stop. ## Posture @@ -125,8 +165,7 @@ Advisory-only review of Dependabot-authored pull requests in microsoft/physical- Skip the review and emit a `noop` when any of the following hold: -* Pull request author is not `dependabot[bot]`. -* Pull request is a draft (`github.event.pull_request.draft == true`). +* `PR_DEPENDABOT_SKIP_REASON` is set by the resolver step (PR could not be resolved, author is not `dependabot[bot]`, or PR is a draft). * Diff touches `.github/workflows/**` — workflow changes are reviewed by `dependency-review`, `workflow-permissions-scan`, and `sha-staleness-check` instead. * Diff contains no recognized dependency manifest change. @@ -136,9 +175,13 @@ The full reviewer persona, risk rubric, ecosystem-specific checks, and enrichmen ## Step-by-Step -1. **Parse.** Read the pull request title, body, and file diff. Extract package name, ecosystem, old/new versions, `GHSA-[0-9a-z]{4}-[0-9a-z]{4}-[0-9a-z]{4}` and `CVE-\d{4}-\d{4,7}` identifiers from the Dependabot body. -2. **Enrich.** Query GHSA (preferred), fall back to OSV (`api.osv.dev`) and NVD (`services.nvd.nist.gov`) for severity, affected ranges, and fixed versions. Fetch release notes or changelog via the relevant package registry (npm, PyPI, Go module proxy, Terraform registry). -3. **Classify.** Apply the persona's per-surface rubric. Flag ABI-sensitive pins (for example `numpy >=1.26.0,<2.0.0` in Isaac Sim training), pre-1.0 bumps, major version jumps, and missing upstream advisories. -4. **Review.** Post up to five inline `create-pull-request-review-comment` entries for specific risks, up to two `add-comment` status updates on the triggering PR, and exactly one `submit-pull-request-review` with `APPROVE` or `COMMENT`. When nothing actionable is found, emit `noop`. +1. **Resolve context.** Read `PR_NUMBER`, `PR_HEAD_SHA`, `PR_VALIDATION_CONCLUSION`, and `PR_VALIDATION_RUN_URL` from the environment. If `PR_DEPENDABOT_SKIP_REASON` is set, emit `noop` and stop. +2. **Read CI signal.** Use the `github` MCP `pull_requests` toolset (or `GET /repos/{owner}/{repo}/commits/{sha}/check-runs`) on `PR_HEAD_SHA` to enumerate per-surface check-run conclusions. Map them through the surface table in the persona. +3. **Parse.** Read the pull request title, body, and file diff. Extract package name, ecosystem, old/new versions, `GHSA-[0-9a-z]{4}-[0-9a-z]{4}-[0-9a-z]{4}` and `CVE-\d{4}-\d{4,7}` identifiers from the Dependabot body. +4. **Enrich.** Query GHSA (preferred), fall back to OSV (`api.osv.dev`) and NVD (`services.nvd.nist.gov`) for severity, affected ranges, and fixed versions. Fetch release notes or changelog via the relevant package registry (npm, PyPI, Go module proxy, Terraform registry). +5. **Classify.** Apply the persona's per-surface rubric. Flag ABI-sensitive pins (for example `numpy >=1.26.0,<2.0.0` in Isaac Sim training), pre-1.0 bumps, major version jumps, and missing upstream advisories. +6. **Review.** Post up to five inline `create-pull-request-review-comment` entries for specific risks, up to two `add-comment` status updates on the triggering PR, and exactly one `submit-pull-request-review` with `APPROVE` or `COMMENT`. + When `PR_VALIDATION_CONCLUSION` is anything other than `success`, the verdict MUST be `COMMENT` and the body MUST quote the failing per-surface check-run names plus their `html_url`. + Never skip enrichment on red CI — maintainers rely on advisory output to triage which package in a grouped PR caused the failure. Keep comments factual and concise. Cite the advisory identifier, affected versions, and the Dependabot PR URL.