Skip to content
Merged
Show file tree
Hide file tree
Changes from 42 commits
Commits
Show all changes
50 commits
Select commit Hold shift + click to select a range
7d87222
[CI] Refactor ci-copilot pipeline: scope env vars per task
Copilot May 22, 2026
29b1518
Fix review findings: persist regression data across phases
Copilot May 22, 2026
647786a
Fix detectedCategories routing: coalesce RunReview with RunGate
Copilot May 22, 2026
3fd884a
Fix null crash in Post/CopilotReview phases + add sentinel check
Copilot May 22, 2026
485e865
Add GH_TOKEN to Gate phase for PR metadata fetches
Copilot May 22, 2026
a22749d
[CI] Plug GH service-connection token leaks in copilot pipeline
T-Gro May 26, 2026
ca5ce6d
[CI] Strip GH_TOKEN from PR-code subprocesses; trust eng/scripts copy
T-Gro May 26, 2026
0d1acf3
[CI] Add security instructions for ci-copilot pipeline surface
T-Gro May 26, 2026
07565bb
Clean up stale MauiBot PR comments
Copilot May 26, 2026
63c4689
Merge remote-tracking branch 'origin/feature/refactor-copilot-yml' in…
Copilot May 26, 2026
d3bcebd
[CI] Compact ci-copilot security instructions
T-Gro May 26, 2026
7ade47d
[CI] Use YAML list + brace alternation for applyTo
T-Gro May 26, 2026
0e3ce9d
[CI] Use officially-documented applyTo syntax + drop scare line
T-Gro May 26, 2026
888febc
[CI] Fix Gate: move token-strip wrap inside verify-tests-fail.ps1
T-Gro May 26, 2026
d8ff0a1
Fix AI summary session replacement
Copilot May 27, 2026
3650971
Remove duplicate full-category UI test run from ReviewPR stage
Copilot May 28, 2026
d4deda6
Fix deep UI setup failure reporting
Copilot May 28, 2026
bbd6055
Hide no-op AI summary sections
Copilot May 28, 2026
d3069ce
Enhance MauiBot review posting
Copilot May 31, 2026
d461a5b
Update MauiBot AI summary layout
Copilot May 31, 2026
8fcd355
Merge try-fix guidance into AI summary review
Copilot May 31, 2026
5f3beaa
Run Windows device tests without VSTest
Copilot May 31, 2026
4b8e6dc
Add deterministic review rerun gate
Copilot May 31, 2026
19812a0
Verify rerun label application
Copilot May 31, 2026
87c03f0
Use JSON when adding agent labels
Copilot May 31, 2026
4148e72
Allow rerun workflow to label PRs
Copilot May 31, 2026
84421cf
Remove PR finalization from AI review flow
Copilot May 31, 2026
f1d8b1c
Include rerun activity in pre-flight context
Copilot Jun 1, 2026
16321bd
Move rerun implementation out of review UI PR
Copilot Jun 1, 2026
d6aed2d
Add deterministic review rerun gate
Copilot Jun 1, 2026
b20223d
Verify rerun label application
Copilot May 31, 2026
6706df5
Use JSON when adding agent labels
Copilot May 31, 2026
69238a9
Allow rerun workflow to label PRs
Copilot May 31, 2026
766c909
Include rerun activity in pre-flight context
Copilot Jun 1, 2026
9465d1c
Add gh-aw rerun review scanner
Copilot Jun 1, 2026
798f837
Make rerun helper importable
Copilot Jun 1, 2026
03a490e
Allow fork dry-runs for rerun scanner
Copilot Jun 1, 2026
b84cc7f
Clarify rerun scanner concurrency
Copilot Jun 2, 2026
a3412bc
Add review rerun in-progress lock
Copilot Jun 2, 2026
1dcdb74
Preserve rerun scanner target repository
Copilot Jun 2, 2026
8e71f44
Preserve review rerun platform and branch
Copilot Jun 2, 2026
530a7c5
Restrict review rerun to contributors
Copilot Jun 2, 2026
7f447e2
Address rerun review scanner feedback
Copilot Jun 4, 2026
5378fc4
Merge remote-tracking branch 'origin/main' into feature/rerun-review-…
Copilot Jun 4, 2026
7325fba
Use current repo for rerun scanner query
Copilot Jun 4, 2026
12cea3a
Allow non-first-time PR authors to rerun reviews
Copilot Jun 4, 2026
606c4e3
Add rerun scanner abuse limits
Copilot Jun 4, 2026
80da010
Harden rerun review scanner handler
Copilot Jun 4, 2026
29f62fb
Refine rerun handler input handling and edge cases
PureWeen Jun 4, 2026
3484700
Always check per-comment author_association in Test-ReviewCommandOpti…
PureWeen Jun 5, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
31 changes: 28 additions & 3 deletions .github/aw/actions-lock.json
Original file line number Diff line number Diff line change
@@ -1,5 +1,20 @@
{
"entries": {
"actions/checkout@v4": {
"repo": "actions/checkout",
"version": "v4",
"sha": "34e114876b0b11c390a56381ad16ebd13914f8d5"
},
"actions/checkout@v6.0.2": {
"repo": "actions/checkout",
"version": "v6.0.2",
"sha": "de0fac2e4500dabe0009e67214ff5f5447ce83dd"
},
"actions/download-artifact@v8.0.1": {
"repo": "actions/download-artifact",
"version": "v8.0.1",
"sha": "3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c"
},
"actions/github-script@v8": {
"repo": "actions/github-script",
"version": "v8",
Expand All @@ -10,10 +25,20 @@
"version": "v9.0.0",
"sha": "3a2844b7e9c422d3c10d287c895573f7108da1b3"
},
"github/gh-aw-actions/setup@v0.72.1": {
"actions/setup-node@v6.4.0": {
"repo": "actions/setup-node",
"version": "v6.4.0",
"sha": "48b55a011bda9f5d6aeb4c2d9c7362e8dae4041e"
},
"actions/upload-artifact@v7.0.1": {
"repo": "actions/upload-artifact",
"version": "v7.0.1",
"sha": "043fb46d1a93c77aae656e7c1c64a875d1fc6a0a"
},
"github/gh-aw-actions/setup@v0.77.5": {
"repo": "github/gh-aw-actions/setup",
"version": "v0.72.1",
"sha": "bc56a0cad2f450c562810785ef38649c04db812a"
"version": "v0.77.5",
"sha": "3ea13c02d765410340d533515cb31a7eef2baaf0"
},
"github/gh-aw/actions/setup@v0.43.19": {
"repo": "github/gh-aw/actions/setup",
Expand Down
20 changes: 14 additions & 6 deletions .github/docs/agent-labels.md
Original file line number Diff line number Diff line change
Expand Up @@ -41,13 +41,15 @@ Always applied on every completed agent run.
|-------|-------|-------------|--------------|
| `s/agent-reviewed` | 🔵 `#1565C0` | PR was reviewed by AI agent workflow (full 4-phase review) | Every completed agent run |

### Manual Label
### Manual / Queue Labels

Applied by MAUI maintainers, not by automation.
Manual labels are applied by MAUI maintainers. Queue labels are applied by deterministic automation, not by AI.

| Label | Color | Description | Applied When |
|-------|-------|-------------|--------------|
| `s/agent-fix-implemented` | 🟣 `#7B1FA2` | PR author implemented the agent's suggested fix | Maintainer applies when PR author adopts agent's recommendation |
| `s/agent-ready-for-rerun` | 🟣 `#5319E7` | AI review has new PR activity and is ready for rerun | `/review rerun` finds new comments or commits after the latest AI Summary / previous rerun request |
| `s/agent-review-in-progress` | 🟡 `#FBCA04` | AI review is currently running for this PR | Applied before triggering the async AzDO review pipeline and removed by pipeline cleanup; stale locks can be recovered after a conservative timeout |

---

Expand All @@ -62,16 +64,15 @@ Review-PR.ps1
│ ├── Validate → writes content.md
│ ├── Fix → writes content.md
│ └── Report → writes content.md
├── Phase 2: PR Finalize (optional)
├── Phase 3: Post Comments (optional)
└── Phase 4: Apply Labels ← labels are applied here
├── Phase 2: Post Comments (optional)
└── Phase 3: Apply Labels ← labels are applied here
├── Parse content.md files
├── Determine outcome + signal labels
├── Apply via GitHub REST API
└── Non-fatal: errors warn but don't fail the workflow
```

Labels are applied exclusively from `Review-PR.ps1` Phase 4. No other script applies agent labels. This single-source design avoids label conflicts and simplifies debugging.
Most review outcome labels are applied from `Review-PR.ps1` Phase 4. The exceptions are queue/lock labels: `s/agent-ready-for-rerun` is applied by the deterministic `/review rerun` GitHub Action path after checking for new comments or commits, and `s/agent-review-in-progress` is applied before triggering the async AzDO review pipeline. The rerun path does not use AI to decide whether these labels apply. The lock label normally clears in the AzDO cleanup stage; trigger paths treat very old locks as stale so a cancelled pipeline does not permanently block reviews.

### How Labels Are Parsed

Expand Down Expand Up @@ -140,6 +141,10 @@ is:pr label:s/agent-reviewed
|------|---------|
| `.github/scripts/shared/Update-AgentLabels.ps1` | Label helper module (all label logic) |
| `.github/scripts/Review-PR.ps1` | Orchestrator that calls `Apply-AgentLabels` in Phase 4 |
| `.github/scripts/Resolve-RerunEligibility.ps1` | Deterministic `/review rerun` checker that can apply `s/agent-ready-for-rerun` |
| `.github/scripts/Invoke-RerunReviewTrigger.ps1` | Safe-output handler that applies `s/agent-review-in-progress` before triggering AzDO reruns |
| `.github/workflows/review-trigger.yml` | Manual `/review` trigger that applies `s/agent-review-in-progress` before triggering AzDO reviews |
| `eng/pipelines/ci-copilot.yml` | AzDO review pipeline that removes `s/agent-review-in-progress` in final cleanup |
| `.github/skills/pr-review/SKILL.md` | Documents label system for the pr-review skill |

### Key Functions
Expand All @@ -151,6 +156,9 @@ is:pr label:s/agent-reviewed
| `Update-AgentOutcomeLabel` | Applies one outcome label, removes conflicting ones |
| `Update-AgentSignalLabels` | Adds/removes validate and fix signal labels |
| `Update-AgentReviewedLabel` | Ensures tracking label is present |
| `Set-AgentReviewInProgress` | Applies the async review lock label |
| `Clear-AgentReviewInProgress` | Removes the async review lock label |
| `Test-AgentReviewInProgressIsStale` | Checks whether a lock label is old enough to recover |
| `Ensure-LabelExists` | Creates or updates a label in the repository |

### Design Principles
Expand Down
52 changes: 52 additions & 0 deletions .github/instructions/ci-copilot-pipeline-security.instructions.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,52 @@
---
description: "Security rules for the Copilot PR-review pipeline. Read before editing."
applyTo: "eng/pipelines/ci-copilot.yml,eng/scripts/detect-ui-test-categories.ps1,.github/scripts/**,.github/pr-review/**,.github/skills/pr-review/**,.github/skills/verify-tests-fail-without-fix/**,.github/skills/try-fix/**,.github/skills/run-device-tests/**,.github/workflows/review-trigger.yml,.github/workflows/pr-review-queue.yml,.github/workflows/copilot-evaluate-tests.*"
---

# CI Copilot pipeline — security rules

This pipeline runs **untrusted PR code** on AzDO agents with these tokens in scope:

- `GH_COMMENT_TOKEN` / `GH_TOKEN` — `maui-bot` PAT (post comments, labels, reviews on any PR)
- `COPILOT_GITHUB_TOKEN` — Copilot CLI install token
- AzDO GitHub service-connection PAT — repo contents, PRs, checks, workflows

Once the PR is merged into the worktree, the author controls every `.csproj`, `Directory.Build.targets`, source generator, analyzer, test, `.ps1`, and `.yml` the pipeline subsequently runs.

## Rules

1. **Per-task `env:` scoping.** Only put tokens a task needs. The Copilot-agent task gets `COPILOT_GITHUB_TOKEN` only — never `GH_TOKEN`. Pass `--secret-env-vars=GH_TOKEN,GITHUB_TOKEN,COPILOT_GITHUB_TOKEN` to the Copilot CLI.

2. **`persistCredentials: false` on every `checkout: self`** unless the task pushes. Default checkout writes the service-connection PAT into `.git/config` as `extraheader`, readable by any subprocess.

3. **Trusted-copy scripts before merging the PR.** Setup task (still on `main`) copies `.github/scripts`, `.github/skills`, `eng/scripts` to `$(Build.ArtifactStagingDirectory)/trusted-github/`, then `chmod -R a-w`. Later tasks invoke scripts from `$TRUSTED/...`, never from the merged worktree. In PowerShell use `$ScriptsDir` / `$SkillsDir` / `$EngScriptsDir` (canonical impl in `Review-PR.ps1`). New post-merge scripts must be added to the Setup copy block.

4. **Strip tokens before invoking PR-controlled code.** Wrap every `dotnet build|test|run|pack`, `msbuild`, `dotnet cake`, `BuildAndRun*.ps1`, `Run-DeviceTests.ps1`, `Invoke-UITestWithRetry.ps1` in `Invoke-WithoutGhTokens { ... }` (defined in `Review-PR.ps1` and `verify-tests-fail.ps1` — saves/clears/restores `GH_TOKEN`, `GITHUB_TOKEN`, `COPILOT_GITHUB_TOKEN`). **Wrap as close to the subprocess as possible, not at the outer trusted-script boundary** — a trusted script may itself need `gh` for metadata (e.g., `verify-tests-fail.ps1` calls `Detect-TestsInDiff.ps1` which uses `gh api`), so wrapping the whole script breaks its detection path. Wrap only the line that launches the PR-controlled process. Exception: scripts that ONLY call `gh` for PR metadata (`Detect-TestsInDiff.ps1`, `Find-RegressionRisks.ps1`, `detect-ui-test-categories.ps1`) don't need wrapping at all — they keep the token.

5. **Cross-phase signal files in `$(Agent.TempDirectory)`** (or `$TRUSTED`), never `$RepoRoot/...`. PR code can overwrite anything in the worktree, including a gate verdict. Readers must not silently fall back to a worktree path if the trusted one is missing.

6. **Strip `##vso[...]` from PR-controlled stdout.** Pipe through `tr -d '\r' | sed -E 's/##vso\[[^]]*\]//g'` — bare `sed` misses CRLF lines and the agent will execute the directive.

7. **`gh-aw` workflows.** Pin compiler version (≥ v0.68.4 strips `pull-requests: write` per `gh-aw#28767`). Regenerate `.lock.yml` with `gh aw compile` in the **same commit** as any `.md` frontmatter edit (stale lock ⇒ all dispatches fail). `workflow_dispatch` triggers must restore trusted `.github/` from main (see `Checkout-GhAwPr.ps1`).

8. **No token republish.** Don't `setvariable` a token (visible to every later task, even with `issecret=true`). Don't write tokens to worktree files. Don't echo token names.

## Review checklist

- [ ] New `checkout: self` has `persistCredentials: false`.
- [ ] New `env:` block lists only the tokens that task needs; Copilot task has no `GH_TOKEN`.
- [ ] New post-merge script invoked via `$ScriptsDir` / `$SkillsDir` / `$EngScriptsDir`, not `$RepoRoot/...`, AND added to Setup copy block.
- [ ] New invocation of PR-controlled code (`dotnet test|build|run`, `BuildAndRun*`, `Run-DeviceTests`, `Invoke-UITestWithRetry`) is wrapped in `Invoke-WithoutGhTokens` AT THE CALL SITE (not at an outer boundary).
- [ ] New cross-phase state file lives under `$(Agent.TempDirectory)` / `$TRUSTED`.
- [ ] New PR-stdout pipe uses `tr -d '\r' | sed -E 's/##vso\[[^]]*\]//g'`.
- [ ] Edited `.github/workflows/*.md` has matching `.lock.yml` regenerated in same commit.

## Grep these during review

```bash
git grep -nE 'dotnet (test|build|run|pack)' eng/pipelines/ci-copilot.yml .github/scripts .github/skills | grep -v Invoke-WithoutGhTokens
git grep -nE 'Join-Path \$RepoRoot ".*\.(ps1|sh)"' .github/scripts .github/skills
git grep -nA1 'checkout: self' eng/pipelines/ci-copilot.yml | grep -v persistCredentials
git grep -nE 'Set-Content.*\$RepoRoot.*(gate-result|sentinel|verdict)' .github/scripts .github/skills
git grep -nE 'sed.*##vso' eng/pipelines/ci-copilot.yml | grep -v 'tr -d'
```
4 changes: 2 additions & 2 deletions .github/scripts/Find-RegressionRisks.ps1
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@
added → 🔴 REVERT. Same file but no line match → 🟡 OVERLAP. Otherwise → 🟢 CLEAN.

Outputs (when -OutputDir is provided):
- content.md Markdown summary suitable for the wall-of-text PR comment.
- content.md Markdown summary suitable for the wall-of-text PR review.
- risks.json Structured findings for downstream agents.
- result.txt One token: CLEAN | OVERLAP | REVERT (used by Review-PR.ps1
for branching).
Expand Down Expand Up @@ -726,7 +726,7 @@ if ($OutputDir) {
} | ConvertTo-Json -Depth 6
$payload | Set-Content (Join-Path $OutputDir 'risks.json') -Encoding UTF8

# content.md — markdown summary for the wall-of-text PR comment
# content.md — markdown summary for the wall-of-text PR review
$md = New-Object System.Text.StringBuilder
[void]$md.AppendLine("## 🔍 Regression Cross-Reference")
[void]$md.AppendLine()
Expand Down
Loading
Loading