Improve evaluate-pr-tests workflow: slash_command + workflow_dispatch, security hardening

.github/scripts/Checkout-GhAwPr.ps1

@@ -4,23 +4,23 @@
 
 .DESCRIPTION
     Checks out a PR branch and restores trusted agent infrastructure (skills,
-    instructions) from the base branch. Works for both same-repo and fork PRs.
+    instructions) from the base branch. This gives the agent the PR's code
+    changes with the latest skills and instructions from main.
 
-    This script is only invoked for workflow_dispatch triggers. For pull_request
-    and issue_comment, the gh-aw platform's checkout_pr_branch.cjs handles PR
-    checkout automatically (it runs as a platform step after all user steps).
-    workflow_dispatch skips the platform checkout entirely, so this script is
-    the only thing that gets the PR code onto disk.
+    This script is only invoked for workflow_dispatch triggers. For
+    pull_request_target and issue_comment, the gh-aw platform's
+    checkout_pr_branch.cjs handles PR checkout automatically.
+    workflow_dispatch skips the platform checkout entirely, so this script
+    is the only thing that gets the PR code onto disk.
 
-    SECURITY NOTE: This script checks out PR code onto disk. This is safe
-    because NO subsequent user steps execute workspace code — the gh-aw
-    platform copies the workspace into a sandboxed container with scrubbed
-    credentials before starting the agent. The classic "pwn-request" attack
-    requires checkout + execution; we only do checkout.
+    SECURITY: Before checkout, the script verifies the PR is not from a
+    fork and that the author has write access (write, maintain, or admin).
+    Fork PRs are evaluated via pull_request_target instead (where the
+    platform handles checkout safely inside a sandboxed container).
 
     DO NOT add steps after this that run scripts from the workspace
-    (e.g., ./build.sh, pwsh ./script.ps1). That would create an actual
-    fork code execution vulnerability. See:
+    (e.g., ./build.sh, pwsh ./script.ps1). That would create a code
+    execution vulnerability. See:
     https://securitylab.github.com/resources/github-actions-preventing-pwn-requests/
 
 .NOTES
@@ -42,16 +42,42 @@ if (-not $env:PR_NUMBER -or $env:PR_NUMBER -eq '0') {
 
 $PrNumber = $env:PR_NUMBER
 
+# ── Verify PR is same-repo and author has write access ───────────────────────
+
+$PrInfo = gh pr view $PrNumber --repo $env:GITHUB_REPOSITORY --json author,isCrossRepository --jq '{author: .author.login, isFork: .isCrossRepository}' | ConvertFrom-Json
+if ($LASTEXITCODE -ne 0) {
+    Write-Host "❌ Failed to fetch PR #$PrNumber metadata"
+    exit 1
+}
+
+if ($PrInfo.isFork) {
+    Write-Host "⏭️ PR #$PrNumber is from a fork. workflow_dispatch does not check out fork PRs."
+    Write-Host "   Fork PRs are evaluated automatically via pull_request_target."
+    exit 1
+}
+
+$Permission = gh api "repos/$($env:GITHUB_REPOSITORY)/collaborators/$($PrInfo.author)/permission" --jq '.permission'
+if ($LASTEXITCODE -ne 0) {
+    Write-Host "❌ Failed to check permissions for '$($PrInfo.author)'"
+    exit 1
+}
+
+$AllowedRoles = @('admin', 'write', 'maintain')
+if ($Permission -notin $AllowedRoles) {
+    Write-Host "⏭️ PR author '$($PrInfo.author)' has '$Permission' access. workflow_dispatch only processes PRs from authors with write access."
+    exit 1
+}
+Write-Host "✅ PR #$PrNumber by '$($PrInfo.author)' ($Permission access, same-repo)"
+
 # ── Save base branch SHA ─────────────────────────────────────────────────────
-# Must be captured BEFORE checkout replaces HEAD.
-# Exported for potential use by downstream platform steps (e.g., checkout_pr_branch.cjs)
 
 $BaseSha = git rev-parse HEAD
 if ($LASTEXITCODE -ne 0) {
     Write-Host "❌ Failed to get current HEAD SHA"
     exit 1
 }
 Add-Content -Path $env:GITHUB_ENV -Value "BASE_SHA=$BaseSha"
+Write-Host "Base branch SHA: $BaseSha"
 
 # ── Checkout PR branch ──────────────────────────────────────────────────────
 
@@ -65,10 +91,9 @@ Write-Host "✅ Checked out PR #$PrNumber"
 git log --oneline -1
 
 # ── Restore agent infrastructure from base branch ────────────────────────────
-# This script only runs for workflow_dispatch (other triggers use the platform's
-# checkout_pr_branch.cjs instead). For workflow_dispatch the platform checkout is
-# skipped, so this restore IS the final workspace state.
-# rm -rf first to prevent fork-added files from surviving the restore.
+# Replace skills and instructions with base branch versions to ensure the agent
+# always uses trusted infrastructure from main. Uses git checkout to read files
+# directly from the commit tree — works in shallow clones (no history traversal).
 
 if (Test-Path '.github/skills/') { Remove-Item -Recurse -Force '.github/skills/' }
 if (Test-Path '.github/instructions/') { Remove-Item -Recurse -Force '.github/instructions/' }

.github/workflows/copilot-evaluate-tests.md

@@ -1,9 +1,8 @@
 ---
 description: Evaluates test quality, coverage, and appropriateness on PRs that add or modify tests
 on:
-  pull_request:
-    types: [opened, synchronize, reopened, ready_for_review]
-    forks: ["*"]
+  pull_request_target:
+    types: [opened, synchronize, reopened]
     paths:
       - 'src/**/tests/**'
       - 'src/**/test/**'
@@ -15,9 +14,14 @@ on:
         description: 'PR number to evaluate'
         required: true
         type: number
+      suppress_comment:
+        description: 'Dry-run — evaluate but do not post a comment on the PR'
+        required: false
+        type: boolean
+        default: false
 
 if: >-
-  (github.event_name == 'pull_request' && github.event.pull_request.draft == false) ||
+  (github.event_name == 'pull_request_target' && github.event.pull_request.draft == false) ||
   github.event_name == 'workflow_dispatch' ||
   (github.event_name == 'issue_comment' &&
    github.event.issue.pull_request &&
@@ -57,7 +61,7 @@ timeout-minutes: 15
 
 steps:
   - name: Gate — skip if no test source files in diff
-    if: github.event_name == 'pull_request'
+    if: github.event_name == 'pull_request_target'
     env:
       GH_TOKEN: ${{ github.token }}
       PR_NUMBER: ${{ github.event.pull_request.number }}
@@ -73,9 +77,10 @@ steps:
       echo "✅ Found test files to evaluate:"
       echo "$TEST_FILES" | head -20
 
-  # Only needed for workflow_dispatch — for pull_request and issue_comment,
+  # Only needed for workflow_dispatch — for pull_request_target and issue_comment,
   # the gh-aw platform's checkout_pr_branch.cjs handles PR checkout automatically.
   # workflow_dispatch skips the platform checkout entirely, so we must do it here.
+  # The script gates on PR author having write access before checkout.
   - name: Checkout PR and restore agent infrastructure
     if: github.event_name == 'workflow_dispatch'
     env:
@@ -110,11 +115,27 @@ If the file is **missing**, the fork PR branch is likely not rebased on the late
 
 ❌ **Cannot evaluate**: this PR's branch does not include the evaluate-pr-tests skill (`.github/skills/evaluate-pr-tests/SKILL.md` is missing).
 
-**Fix**: rebase your fork on the latest `main` branch, or use the **workflow_dispatch** trigger (Actions tab → "Evaluate PR Tests" → "Run workflow" → enter PR number) which handles this automatically.
+**Fix**: rebase your fork on the latest `main` branch and push again. The evaluation will trigger automatically once the skill file is available.
 ```
 
 Then stop — do not proceed with the evaluation.
 
+## Dry-run mode
+
+When triggered via `workflow_dispatch` with `suppress_comment` = `${{ inputs.suppress_comment }}`:
+- If **true**, perform the full evaluation but **do not** post a comment on the PR. Write the evaluation to the workflow log only. This is useful for testing the skill without spamming the PR.
+- If **false** (default), post the comment as normal.
+
+## When no action is needed
+
+If there is nothing to evaluate (PR has no test files, PR is a docs-only change, etc.), you **must** call the `noop` tool with a message explaining why:
+
+```json
+{"noop": {"message": "No action needed: [brief explanation, e.g. 'PR contains no test files']"}}
+```
+
+Do not post a comment and do not silently exit — always use `noop` so the workflow run shows a clear reason.
+
 ## Running the skill
 
 1. Use `gh pr view <number>` to fetch PR metadata (title, body, labels, base branch). If `gh` CLI is unavailable, use the GitHub MCP tools instead.
@@ -124,7 +145,9 @@ Then stop — do not proceed with the evaluation.
 
 ## Posting Results
 
-Call `add_comment` with `item_number` set to the PR number. Wrap the report in a collapsible `<details>` block:
+If dry-run mode is active (`suppress_comment` is true), log the evaluation report to stdout and stop — do **not** call `add_comment`.
+
+Otherwise, call `add_comment` with `item_number` set to the PR number. Wrap the report in a collapsible `<details>` block:
 
 ```markdown
 ## 🧪 PR Test Evaluation