Skip to content
Merged
Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
29 commits
Select commit Hold shift + click to select a range
0cc2384
Switch evaluate-pr-tests to pull_request_target for fork PR support
github-actions[bot] Mar 26, 2026
2dfcf71
Gate workflow_dispatch checkout to PRs from authors with write access
github-actions[bot] Mar 26, 2026
d79ff20
Move write-access gate into Checkout-GhAwPr.ps1 for reuse
github-actions[bot] Mar 26, 2026
8234b33
Skip workflow_dispatch checkout for fork PRs
github-actions[bot] Mar 26, 2026
e1dee57
Restore entire .github/ from base branch instead of individual paths
github-actions[bot] Mar 26, 2026
f4c4791
Use merge instead of restore for workflow_dispatch checkout
github-actions[bot] Mar 26, 2026
c089534
Gate auto-evaluation on author_association for write access
github-actions[bot] Mar 27, 2026
e6004b7
Fix review findings: shallow clone, write-access gating, fork message
github-actions[bot] Mar 27, 2026
00af259
Add dry-run mode and noop guidance to evaluate-tests workflow
github-actions[bot] Mar 28, 2026
17c0f27
Add null guard for PrInfo in Checkout-GhAwPr.ps1
github-actions[bot] Apr 2, 2026
f4f02df
Update gh-aw security docs: accurate credential model, defense layers
github-actions[bot] Apr 2, 2026
27e14d6
Add copilot[bot] to bots allowlist for auto-evaluation
github-actions[bot] Apr 2, 2026
69c0a16
Allow fork PRs from write-access authors in workflow_dispatch
github-actions[bot] Apr 2, 2026
f3dc6a9
Address review feedback: rename suppress_output, fork guard, no-op re…
github-actions[bot] Apr 7, 2026
7bcc980
Fix bot identity: copilot[bot] → copilot-swe-agent[bot]
github-actions[bot] Apr 8, 2026
04bf387
Fix gate step for large PRs (300+ files)
github-actions[bot] Apr 8, 2026
082ed7a
Hide older evaluation comments when posting new ones
github-actions[bot] Apr 9, 2026
e7fdf22
Use slash_command trigger and add built-in feature discovery guide
github-actions[bot] Apr 13, 2026
b40ccb5
Add workflow labels and update fork PR behavior docs for slash_command
github-actions[bot] Apr 13, 2026
d142b43
Fix gate step to run for all triggers, bump timeout to 20min
github-actions[bot] Apr 13, 2026
cfee2bc
Fix gate to succeed cleanly for no-test PRs, fix permission denial ex…
github-actions[bot] Apr 14, 2026
e76545e
Skip checkout when gate finds no test files, fix bot author permissio…
github-actions[bot] Apr 14, 2026
bf19b8c
Gate exit 1 to stop workflow on no-test PRs, remove HAS_TEST_FILES
github-actions[bot] Apr 14, 2026
0faafa9
Limit evaluate-pr-tests to slash_command trigger only
github-actions[bot] Apr 14, 2026
f8ab3c5
Guard against non-PR issues and closed/merged PRs
github-actions[bot] Apr 14, 2026
6154bfd
Address review findings: fork gate, fatal restore, doc alignment
github-actions[bot] Apr 14, 2026
3b49c7a
Fix remaining review findings: gate error handling, REST fallback, ta…
github-actions[bot] Apr 14, 2026
72356a7
Surface REST API fallback errors instead of masking as 'no test files'
github-actions[bot] Apr 14, 2026
338d7c1
Re-enable workflow_dispatch for manual triggering
github-actions[bot] Apr 15, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
46 changes: 7 additions & 39 deletions .github/workflows/copilot-evaluate-tests.lock.yml

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

12 changes: 6 additions & 6 deletions .github/workflows/copilot-evaluate-tests.md
Original file line number Diff line number Diff line change
@@ -1,9 +1,8 @@
---
description: Evaluates test quality, coverage, and appropriateness on PRs that add or modify tests
on:
pull_request:
types: [opened, synchronize, reopened, ready_for_review]
forks: ["*"]
pull_request_target:
types: [opened, synchronize, reopened]

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I do think this will still lead to the 'Approve and run workflows' button showing up for PRs from untrusted forks. We need to solidify the guidance we give for when to hit that button. I really wish that button navigated into a list of workflows needing approval for the PR with boxes to select which to approve.

paths:
- 'src/**/tests/**'
- 'src/**/test/**'
Expand All @@ -15,9 +14,10 @@ on:
description: 'PR number to evaluate'
required: true
type: number
roles: all

Comment thread
PureWeen marked this conversation as resolved.
Outdated
if: >-
(github.event_name == 'pull_request' && github.event.pull_request.draft == false) ||
(github.event_name == 'pull_request_target' && github.event.pull_request.draft == false) ||
github.event_name == 'workflow_dispatch' ||
(github.event_name == 'issue_comment' &&
github.event.issue.pull_request &&

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I always guard against forks as well, preventing the workflow from running on forks except for the workflow_dispatch event. Otherwise, PRs within a fork will result in failing workflow runs (vs. starting the workflow and skipping all jobs).

Simple case that needs adapting to your scenario: if: (!github.event.repository.fork) || github.event_name == 'workflow_dispatch'.

Expand Down Expand Up @@ -57,7 +57,7 @@ timeout-minutes: 15

steps:
- name: Gate — skip if no test source files in diff
if: github.event_name == 'pull_request'
if: github.event_name == 'pull_request_target'
env:
GH_TOKEN: ${{ github.token }}
PR_NUMBER: ${{ github.event.pull_request.number }}
Expand All @@ -73,7 +73,7 @@ steps:
echo "✅ Found test files to evaluate:"
echo "$TEST_FILES" | head -20

# Only needed for workflow_dispatch — for pull_request and issue_comment,
# Only needed for workflow_dispatch — for pull_request_target and issue_comment,
# the gh-aw platform's checkout_pr_branch.cjs handles PR checkout automatically.
# workflow_dispatch skips the platform checkout entirely, so we must do it here.
- name: Checkout PR and restore agent infrastructure
Expand Down
Loading