From b4d6ed485bf85288d5d150265b4a1ca3cb8b2f85 Mon Sep 17 00:00:00 2001 From: Copilot <223556219+Copilot@users.noreply.github.com> Date: Tue, 2 Jun 2026 15:39:58 +0200 Subject: [PATCH 1/7] Document automated PR review workflow Add a guide for maintainers and community contributors explaining /review, /review rerun, /review tests, the review pipeline flow, comment outputs, and troubleshooting guidance. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> --- .github/docs/pr-review-workflow.md | 279 +++++++++++++++++++++++++++++ 1 file changed, 279 insertions(+) create mode 100644 .github/docs/pr-review-workflow.md diff --git a/.github/docs/pr-review-workflow.md b/.github/docs/pr-review-workflow.md new file mode 100644 index 000000000000..485d8d8c8c56 --- /dev/null +++ b/.github/docs/pr-review-workflow.md @@ -0,0 +1,279 @@ +# .NET MAUI automated PR review workflow + +This guide explains the automated review commands used in dotnet/maui pull requests: + +- `/review` +- `/review rerun` +- `/review tests` + +It is intended for Microsoft maintainers and community contributors who want to understand when to request an automated review, what the automation does, and how to interpret the resulting comments. + +## Quick command reference + +| Command | Who can run it | What it does | Output | +| --- | --- | --- | --- | +| `/review` | Repository users with write, maintain, or admin access | Queues the full MAUI Copilot PR review pipeline. | Updates the PR with an `AI Summary` comment. | +| `/review ` | Repository users with write, maintain, or admin access | Queues the full review pipeline for a specific platform: `android`, `ios`, `catalyst`, or `windows`. | Updates the PR with an `AI Summary` comment. | +| `/review rerun` | Repository users with write, maintain, or admin access | Requests a fresh full review after comments, commits, or CI context changed. It uses the same review pipeline as `/review`. | Adds or replaces a review session in the `AI Summary` comment. | +| `/review tests` | Repository users with write, maintain, or admin access | Reviews current CI/test failures and classifies whether they are likely PR-caused, unrelated, or insufficiently evidenced. | Adds or updates a `Test Failure Review` comment. | + +Community contributors cannot directly trigger these commands unless they have repository write access. If you are a community contributor, ask a maintainer to run the relevant command for your PR. + +## Choosing the right command + +Use `/review` when you want the complete automated PR review. This is the normal entry point for maintainers reviewing a PR. + +Use `/review rerun` when the PR already has an AI review but something changed enough that the previous review may be stale. Typical reasons: + +- the author pushed new commits; +- the author replied to review feedback; +- CI or test results changed; +- a maintainer wants a deterministic fresh review session without manually interpreting older output. + +Use `/review tests` when the question is specifically about CI/test failures, for example: + +- "Is this failure likely caused by the PR?" +- "Is CI red because of a known flaky test?" +- "Did this PR introduce a missing snapshot/baseline?" +- "Are these failures unrelated infrastructure or existing failures?" + +Do not use `/review tests` as a substitute for a code review. It does not approve, request changes, apply labels, trigger reruns, or change the PR. It only posts evidence-based failure classification. + +## `/review`: full PR review + +### Trigger + +Comment `/review` on a pull request. + +Optional platform argument: + +```text +/review android +/review ios +/review catalyst +/review windows +``` + +You can also use explicit flags: + +```text +/review --platform ios +/review --branch main +``` + +The trigger is implemented by `.github/workflows/review-trigger.yml`. It: + +1. checks that the comment is on a pull request; +2. verifies the actor has `write`, `maintain`, or `admin` repository permission; +3. parses the platform and optional pipeline branch; +4. infers the platform from `platform/*` labels when no platform was supplied; +5. queues the DevDiv `maui-copilot` Azure DevOps pipeline. + +The workflow intentionally does not handle `/review tests`; that subcommand is reserved for the test-failure review workflow. + +### Platform inference + +If you do not specify a platform, the trigger looks at PR labels: + +- `platform/iOS` -> `ios` +- `platform/macOS` -> `catalyst` +- `platform/android` -> `android` +- `platform/windows` -> `windows` + +If labels are inconclusive, it defaults to Android. If that is wrong for the PR, use an explicit platform argument. + +### What the review pipeline does + +The Azure DevOps review pipeline is defined in `eng/pipelines/ci-copilot.yml`. At a high level it has three stages: + +1. **ReviewPR**: checks out the PR, prepares the target platform, runs the Copilot PR review script, and publishes the initial review artifacts. +2. **RunDeepUITests**: runs detected UI test categories on the correct platform pool when the review identifies relevant UI tests. +3. **UpdateAISummaryComment**: updates the PR's `AI Summary` comment with review results and deep UI test results. + +The PR review script is `.github/scripts/Review-PR.ps1`. It orchestrates the core review phases: + +1. branch setup and PR merge for review; +2. UI category detection; +3. local/in-process UI test discovery and initial results; +4. regression cross-reference; +5. gate verification; +6. candidate review and fix exploration; +7. AI summary posting; +8. review labels. + +The generated PR comment is a single session-based `AI Summary` comment. New runs add or replace a session keyed by the reviewed commit, so readers can compare the latest run with older review sessions. + +## `/review rerun`: fresh full review + +Comment `/review rerun` when you want a new full review session after the PR changed. + +Operationally, this goes through the same `/review` trigger and review pipeline. The `rerun` token is used as an intent signal for humans and the AI summary UX: it tells readers that the command was meant to refresh the review after new information became available. + +Use it when: + +- a previous AI summary is stale; +- the author pushed a fix after review feedback; +- the previous run analyzed the wrong commit or incomplete context; +- a maintainer wants to replace an older session with a fresh one for the current PR head. + +Avoid using it repeatedly without new commits, comments, or CI results. It consumes CI and agent capacity, and repeated identical runs are unlikely to add useful information. + +## `/review tests`: test-failure review + +### Trigger + +Comment `/review tests` on a pull request. + +The trigger is implemented by `.github/workflows/copilot-review-tests.md`, compiled to `.github/workflows/copilot-review-tests.lock.yml`. + +Because gh-aw slash commands match only the first command token, the workflow listens for `/review` and then neutrally skips unless the comment uses the canonical `/review tests` subcommand. The regular `/review` trigger excludes `/review tests` so the two workflows do not both run. + +### What it does + +`/review tests` is comment-only. It does not: + +- approve or request changes; +- apply labels; +- trigger CI reruns; +- change code; +- start the full PR review pipeline. + +It gathers evidence from: + +- GitHub PR metadata, labels, changed files, and check rollup; +- Azure DevOps build metadata, timelines, and build logs; +- Helix references when available for device tests; +- optional authenticated AzDO data when `AZDO_TOKEN` or local Azure CLI auth is available; +- PR scope, including changed platforms, areas, and test files. + +Then it posts a `Test Failure Review` comment that classifies failures as: + +- **Likely PR-caused** +- **Likely unrelated** +- **Needs human investigation** +- **Insufficient data** + +The comment includes status badges, a short summary, a per-failure table, recommended action, and collapsible evidence details. + +### Local usage + +Maintainers can run the same flow locally: + +```powershell +pwsh .github/scripts/Review-Tests.ps1 -PRNumber 29800 -BuildId 1443464 +``` + +By default this writes local artifacts only: + +```text +CustomAgentLogsTmp/TestFailureReview//context.json +CustomAgentLogsTmp/TestFailureReview//context.md +CustomAgentLogsTmp/TestFailureReview//report.md +CustomAgentLogsTmp/TestFailureReview//comment.md +``` + +To post the generated comment: + +```powershell +pwsh .github/scripts/Review-Tests.ps1 -PRNumber 29800 -BuildId 1443464 -PostComment +``` + +To gather evidence without invoking Copilot: + +```powershell +pwsh .github/scripts/Review-Tests.ps1 -PRNumber 29800 -BuildId 1443464 -GatherOnly +``` + +Local runs can use Azure CLI to acquire an Azure DevOps bearer token. If available, the gatherer records that in the generated context. If builds are still inaccessible after authenticated access, the report should say so and classify affected checks as `Insufficient data`. + +### Interpreting `Insufficient data` + +`Insufficient data` means the workflow saw a failing check but did not have enough reliable evidence to attribute it. + +Common causes: + +- AzDO build records returned 404 or expired; +- logs were inaccessible; +- the build is still running; +- authenticated AzDO test APIs were unavailable; +- device-test failures may be hidden in Helix and no Helix data was available. + +Do not treat `Insufficient data` as "unrelated." It means a human or a rerun with better data is needed. + +## How to read the review comments + +### AI Summary + +The full `/review` and `/review rerun` pipeline posts an `AI Summary` comment. It may include: + +- gate status; +- UI test results; +- regression cross-reference; +- pre-flight context; +- code review findings; +- fix/candidate analysis; +- final recommendation. + +The latest session is expanded by default. Older sessions are retained for comparison. + +### Test Failure Review + +`/review tests` posts a separate `Test Failure Review` comment. This comment is intentionally separate from the `AI Summary` so readers can quickly answer, "Why is CI red?" without reading the full review. + +The top-level title is always: + +```markdown +## Test Failure Review +``` + +The verdict details live in badges and in the expanded review session. + +## Recommended workflow for maintainers + +1. Make sure the PR has appropriate `area-*` and `platform/*` labels. The agentic labeler normally handles this on PR open/reopen. +2. Run `/review` when a PR is ready for automated review. +3. Read the `AI Summary` comment and check whether the review found actionable issues. +4. If CI is red or ambiguous, run `/review tests` to get a focused failure-causality report. +5. If the author pushes fixes or comments materially change the context, run `/review rerun`. +6. Use human judgment for merge decisions. These workflows provide evidence and recommendations, not final approval authority. + +## Recommended workflow for community contributors + +1. Open the PR with a clear description and linked issue when possible. +2. Wait for labels and CI to run. +3. If you need an automated review, ask a maintainer to run `/review`. +4. If CI is red and you are unsure whether it is caused by your changes, ask a maintainer to run `/review tests`. +5. When an automated comment is posted, read the summary first, then expand evidence sections for details. +6. Push fixes or reply with clarifying information, then ask a maintainer whether `/review rerun` is useful. + +## Safety and trust boundaries + +The review automation analyzes untrusted PR code and untrusted comments. The workflows are designed so privileged writes happen through controlled steps and safe outputs. + +Important safeguards: + +- `/review` requires repository write-level permissions and queues a trusted AzDO pipeline. +- `/review tests` is comment-only and uses gh-aw safe outputs for PR comments. +- The full review pipeline keeps PR-controlled code separated from trusted scripts where possible. +- Review comments should be treated as assistant-generated evidence, not as a substitute for human review. + +## Troubleshooting + +| Symptom | Likely cause | What to do | +| --- | --- | --- | +| `/review` does nothing | The commenter does not have write/maintain/admin access, or the comment is not on a PR. | Ask a maintainer to run the command on the PR. | +| `/review` used the wrong platform | Platform labels were missing or ambiguous. | Re-run with an explicit platform, for example `/review ios`. | +| `/review tests` says `Insufficient data` | Build/log/Helix evidence was inaccessible or incomplete. | Re-run later, provide a build ID, or run locally with Azure CLI/AzDO auth. | +| The AI Summary looks stale | New commits or comments landed after the last review. | Run `/review rerun`. | +| There are multiple old sessions | The comment preserves prior review sessions for traceability. | Read the expanded latest session first. | + +## Related files + +- `.github/workflows/review-trigger.yml` — GitHub comment trigger for `/review`. +- `eng/pipelines/ci-copilot.yml` — Azure DevOps PR review pipeline. +- `.github/scripts/Review-PR.ps1` — local script orchestrating full PR review phases. +- `.github/scripts/post-ai-summary-comment.ps1` — AI Summary comment formatter. +- `.github/workflows/copilot-review-tests.md` — gh-aw source for `/review tests`. +- `.github/skills/review-test-failures/SKILL.md` — classification rubric for test-failure reviews. +- `.github/scripts/Review-Tests.ps1` — local runner for `/review tests`. +- `.github/docs/trigger-azdo-pipeline-setup.md` — OIDC setup for triggering AzDO pipelines from GitHub Actions. From ac98d15533f15d251a5b04f711bcb633de6cde2e Mon Sep 17 00:00:00 2001 From: Copilot <223556219+Copilot@users.noreply.github.com> Date: Fri, 19 Jun 2026 15:40:01 +0200 Subject: [PATCH 2/7] Update pr-review-workflow.md with recent changes Document recent workflow improvements: - Command comments are now minimized (collapsed as Resolved) after authorization (#35895, #36021) - /review rerun eligibility now requires PR author activity only (#35874) - Automated hourly rerun scanner processes queued reruns (#35685) - Add troubleshooting entries for rerun eligibility and command visibility Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> --- .github/docs/pr-review-workflow.md | 38 ++++++++++++++++++++++++++++-- 1 file changed, 36 insertions(+), 2 deletions(-) diff --git a/.github/docs/pr-review-workflow.md b/.github/docs/pr-review-workflow.md index 485d8d8c8c56..4929cf48421a 100644 --- a/.github/docs/pr-review-workflow.md +++ b/.github/docs/pr-review-workflow.md @@ -67,10 +67,13 @@ The trigger is implemented by `.github/workflows/review-trigger.yml`. It: 2. verifies the actor has `write`, `maintain`, or `admin` repository permission; 3. parses the platform and optional pipeline branch; 4. infers the platform from `platform/*` labels when no platform was supplied; -5. queues the DevDiv `maui-copilot` Azure DevOps pipeline. +5. queues the DevDiv `maui-copilot` Azure DevOps pipeline; +6. minimizes (collapses) the command comment as resolved once authorized. The workflow intentionally does not handle `/review tests`; that subcommand is reserved for the test-failure review workflow. +**Note**: Command comments are minimized (collapsed as "Resolved") after authorization to reduce conversation clutter while preserving the comment history for the automated rerun scanner. Unauthorized or malformed command comments remain fully visible. + ### Platform inference If you do not specify a platform, the trigger looks at PR labels: @@ -109,6 +112,30 @@ Comment `/review rerun` when you want a new full review session after the PR cha Operationally, this goes through the same `/review` trigger and review pipeline. The `rerun` token is used as an intent signal for humans and the AI summary UX: it tells readers that the command was meant to refresh the review after new information became available. +### Eligibility requirements + +The `/review rerun` workflow checks for **PR author activity** since the latest AI Summary or rerun checkpoint: + +- New commits (head SHA changed) +- New non-command comments from the PR author + +**Important**: Reviewer or maintainer reminder comments do NOT satisfy the rerun eligibility check. Only author activity triggers a rerun. + +When eligible, the workflow applies the `s/agent-ready-for-rerun` label. An automated hourly scanner processes queued reruns and triggers them when appropriate. + +### Automated rerun scanner + +The repository includes an hourly gh-aw workflow (`.github/workflows/rerun-review-scanner.md`) that: + +1. Queries PRs labeled `s/agent-ready-for-rerun` +2. Uses AI to decide `trigger` or `skip` for each PR +3. Triggers approved reruns via Azure DevOps +4. Cleans up queue labels and posts reactions + +This ensures reruns are processed automatically without manual intervention. + +### When to use `/review rerun` + Use it when: - a previous AI summary is stale; @@ -116,7 +143,7 @@ Use it when: - the previous run analyzed the wrong commit or incomplete context; - a maintainer wants to replace an older session with a fresh one for the current PR head. -Avoid using it repeatedly without new commits, comments, or CI results. It consumes CI and agent capacity, and repeated identical runs are unlikely to add useful information. +Avoid using it repeatedly without new commits or author comments. It consumes CI and agent capacity, and repeated identical runs are unlikely to add useful information. ## `/review tests`: test-failure review @@ -128,6 +155,8 @@ The trigger is implemented by `.github/workflows/copilot-review-tests.md`, compi Because gh-aw slash commands match only the first command token, the workflow listens for `/review` and then neutrally skips unless the comment uses the canonical `/review tests` subcommand. The regular `/review` trigger excludes `/review tests` so the two workflows do not both run. +**Note**: Like `/review`, the command comment is minimized (collapsed as "Resolved") after authorization to reduce conversation clutter. + ### What it does `/review tests` is comment-only. It does not: @@ -266,6 +295,8 @@ Important safeguards: | `/review tests` says `Insufficient data` | Build/log/Helix evidence was inaccessible or incomplete. | Re-run later, provide a build ID, or run locally with Azure CLI/AzDO auth. | | The AI Summary looks stale | New commits or comments landed after the last review. | Run `/review rerun`. | | There are multiple old sessions | The comment preserves prior review sessions for traceability. | Read the expanded latest session first. | +| `/review rerun` didn't trigger | Only PR author activity (commits or non-command comments) satisfies eligibility. Reviewer comments don't trigger reruns. | Wait for author activity or use `/review` to force a new review. | +| Command comment is still visible | The commenter may lack authorization, or the command was malformed. | Check actor permissions and command syntax. Authorized commands are minimized after processing. | ## Related files @@ -274,6 +305,9 @@ Important safeguards: - `.github/scripts/Review-PR.ps1` — local script orchestrating full PR review phases. - `.github/scripts/post-ai-summary-comment.ps1` — AI Summary comment formatter. - `.github/workflows/copilot-review-tests.md` — gh-aw source for `/review tests`. +- `.github/workflows/rerun-review-scanner.md` — gh-aw hourly scanner for queued `/review rerun` requests. +- `.github/scripts/Resolve-RerunEligibility.ps1` — determines if a PR is eligible for rerun based on author activity. +- `.github/scripts/Query-RerunReadyPRs.ps1` — queries PRs labeled `s/agent-ready-for-rerun`. - `.github/skills/review-test-failures/SKILL.md` — classification rubric for test-failure reviews. - `.github/scripts/Review-Tests.ps1` — local runner for `/review tests`. - `.github/docs/trigger-azdo-pipeline-setup.md` — OIDC setup for triggering AzDO pipelines from GitHub Actions. From 5818edfbbef48b16f96b5bd39e48699d631c08fe Mon Sep 17 00:00:00 2001 From: kubaflo Date: Sun, 21 Jun 2026 21:36:03 +0200 Subject: [PATCH 3/7] [Review] Improve emoji choices and reorder review structure MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit - Move Next Steps section below Review Sessions (user feedback) - Replace colorful emojis with more professional symbols: * 🤖 → 🔍 (review/analysis indicator) * 🟢🔴🟡 → ● (colored bullets based on context) * ❌ → ✗ (simpler cross) * 💡 → ℹ (information/note) Affects: - post-ai-summary-comment.ps1: swapped session and future action order - post-inline-review.ps1: updated AI marker emoji - Find-RegressionRisks.ps1: replaced colored circles - pr-preflight.md: updated finding indicators Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> --- .github/pr-review/pr-preflight.md | 2 +- .github/scripts/Find-RegressionRisks.ps1 | 28 ++++++++++----------- .github/scripts/post-ai-summary-comment.ps1 | 4 +-- .github/scripts/post-inline-review.ps1 | 5 +++- 4 files changed, 21 insertions(+), 18 deletions(-) diff --git a/.github/pr-review/pr-preflight.md b/.github/pr-review/pr-preflight.md index cef8034915c2..0491bc37fe2d 100644 --- a/.github/pr-review/pr-preflight.md +++ b/.github/pr-review/pr-preflight.md @@ -138,7 +138,7 @@ Write `content.md`: **Errors:** {count} | **Warnings:** {count} | **Suggestions:** {count} Key code review findings: -- {❌/⚠️/💡} {Brief finding with file:line reference} +- {✗/⚠/ℹ} {Brief finding with file:line reference} - ... *(If SKIPPED: "Code review sub-agent failed or timed out. Reason: {details}")* diff --git a/.github/scripts/Find-RegressionRisks.ps1 b/.github/scripts/Find-RegressionRisks.ps1 index efa5da467bac..6e3e2fb1a768 100644 --- a/.github/scripts/Find-RegressionRisks.ps1 +++ b/.github/scripts/Find-RegressionRisks.ps1 @@ -11,7 +11,7 @@ linked-issue label match). 4. Pulls each fix PR's diff and collects lines it ADDED to that same file. 5. Compares (whitespace-insensitive). If a removed line equals a line a fix PR - added → 🔴 REVERT. Same file but no line match → 🟡 OVERLAP. Otherwise → 🟢 CLEAN. + added → ● REVERT. Same file but no line match → ● OVERLAP. Otherwise → ● CLEAN. Outputs (when -OutputDir is provided): - content.md Markdown summary suitable for the wall-of-text PR review. @@ -340,10 +340,10 @@ if (-not $FilePaths -or $FilePaths.Count -eq 0) { } if ($FilePaths.Count -eq 0) { - Write-Host "🟢 No implementation files to check." -ForegroundColor Green + Write-Host "● No implementation files to check." -ForegroundColor Green if ($OutputDir) { New-Item -ItemType Directory -Force -Path $OutputDir | Out-Null - "🟢 No implementation files modified — skipping regression cross-reference." | + "● No implementation files modified — skipping regression cross-reference." | Set-Content (Join-Path $OutputDir "content.md") -Encoding UTF8 '{ "pr_number": ' + $PRNumber + ', "result": "CLEAN", "risks": [] }' | Set-Content (Join-Path $OutputDir "risks.json") -Encoding UTF8 @@ -433,7 +433,7 @@ foreach ($filePath in $FilePaths) { $commitLog = git log --oneline --follow --since="$sinceDate" --all -- $filePath 2>$null } if (-not $commitLog) { - Write-Host " 🟢 No recent commits." -ForegroundColor Green + Write-Host " ● No recent commits." -ForegroundColor Green continue } @@ -450,7 +450,7 @@ foreach ($filePath in $FilePaths) { } if ($recentPRs.Count -eq 0) { - Write-Host " 🟢 No recent PRs reference this file." -ForegroundColor Green + Write-Host " ● No recent PRs reference this file." -ForegroundColor Green continue } @@ -554,7 +554,7 @@ foreach ($filePath in $FilePaths) { $revertedArr = $reverted.ToArray() if ($revertCount -gt 0) { - Write-Host " 🔴 REVERT — $revertCount line(s) from #$recentPR being removed" -ForegroundColor Red + Write-Host " ● REVERT — $revertCount line(s) from #$recentPR being removed" -ForegroundColor Red foreach ($rl in $reverted) { Write-Host " - $($rl.Text.Trim())" -ForegroundColor Red } $riskEntry = [PSCustomObject]@{ File = $filePath @@ -650,7 +650,7 @@ $result = if ($reverts.Count -gt 0) { 'REVERT' } switch ($result) { 'REVERT' { - Write-Host "🔴 REVERT RISKS: $($reverts.Count)" -ForegroundColor Red + Write-Host "● REVERT RISKS: $($reverts.Count)" -ForegroundColor Red foreach ($r in $reverts) { Write-Host "" Write-Host " File: $($r.File)" -ForegroundColor Red @@ -666,13 +666,13 @@ switch ($result) { } } 'OVERLAP' { - Write-Host "🟡 OVERLAPS: $($overlaps.Count) (lower risk — same files, different lines)" -ForegroundColor Yellow + Write-Host "● OVERLAPS: $($overlaps.Count) (lower risk — same files, different lines)" -ForegroundColor Yellow foreach ($o in $overlaps) { Write-Host " $($o.File) — fix PR #$($o.RecentPR) ($($o.FixedIssues))" -ForegroundColor Yellow } } 'CLEAN' { - Write-Host "🟢 No regression risks detected." -ForegroundColor Green + Write-Host "● No regression risks detected." -ForegroundColor Green } } @@ -732,14 +732,14 @@ if ($OutputDir) { [void]$md.AppendLine() switch ($result) { 'REVERT' { - [void]$md.AppendLine("🔴 **Revert risks detected** — this PR removes $($reverts.Count) line(s) previously added by labeled bug-fix PRs.") + [void]$md.AppendLine("● **Revert risks detected** — this PR removes $($reverts.Count) line(s) previously added by labeled bug-fix PRs.") [void]$md.AppendLine() [void]$md.AppendLine("| File | Fix PR | Fixed issue(s) | Risk | Reverted line |") [void]$md.AppendLine("|---|---|---|---|---|") foreach ($r in $reverts) { $sample = @($r.RevertedLines) | Select-Object -First 1 | ForEach-Object { $_.Text.Trim() } $sampleEsc = ($sample -replace '\|', '\|') - [void]$md.AppendLine("| ``$($r.File)`` | #$($r.RecentPR) | $($r.FixedIssues) | 🔴 REVERT | ``$sampleEsc`` |") + [void]$md.AppendLine("| ``$($r.File)`` | #$($r.RecentPR) | $($r.FixedIssues) | ● REVERT | ``$sampleEsc`` |") } $allIssues = @($reverts | ForEach-Object { $_.FixedIssues -split ',\s*' } | Where-Object { $_ } | Select-Object -Unique | Sort-Object) @@ -767,7 +767,7 @@ if ($OutputDir) { } } 'OVERLAP' { - [void]$md.AppendLine("🟡 **Overlaps with prior bug-fix PRs** — same files modified, but no exact line revert detected.") + [void]$md.AppendLine("● **Overlaps with prior bug-fix PRs** — same files modified, but no exact line revert detected.") [void]$md.AppendLine() [void]$md.AppendLine("| File | Fix PR | Fixed issue(s) |") [void]$md.AppendLine("|---|---|---|") @@ -794,7 +794,7 @@ if ($OutputDir) { } } 'CLEAN' { - [void]$md.AppendLine("🟢 No regression risks detected. No labeled bug-fix PRs in the last $MonthsBack months touched the modified files.") + [void]$md.AppendLine("● No regression risks detected. No labeled bug-fix PRs in the last $MonthsBack months touched the modified files.") } } $md.ToString() | Set-Content (Join-Path $OutputDir 'content.md') -Encoding UTF8 @@ -806,7 +806,7 @@ if ($OutputDir) { foreach ($r in $reverts) { foreach ($rl in @($r.RevertedLines)) { $prUrl = "https://github.com/$Repo/pull/$($r.RecentPR)" - $body = "🔴 **Regression risk** — this line was added by [#$($r.RecentPR)]($prUrl) to fix $($r.FixedIssues). Removing it may re-introduce the original bug. Please confirm this removal is intentional and that the previously-fixed issue is covered by another mechanism." + $body = "● **Regression risk** — this line was added by [#$($r.RecentPR)]($prUrl) to fix $($r.FixedIssues). Removing it may re-introduce the original bug. Please confirm this removal is intentional and that the previously-fixed issue is covered by another mechanism." $inline += @{ path = $r.File line = $rl.Line diff --git a/.github/scripts/post-ai-summary-comment.ps1 b/.github/scripts/post-ai-summary-comment.ps1 index cb22f4de9481..444747054f22 100644 --- a/.github/scripts/post-ai-summary-comment.ps1 +++ b/.github/scripts/post-ai-summary-comment.ps1 @@ -618,9 +618,9 @@ $authorPing $statusChipRow -$newSessionBlock - $futureActionSection + +$newSessionBlock "@ # Clean up excessive blank lines diff --git a/.github/scripts/post-inline-review.ps1 b/.github/scripts/post-inline-review.ps1 index 6b77ae0b76b0..a3628488aedc 100644 --- a/.github/scripts/post-inline-review.ps1 +++ b/.github/scripts/post-inline-review.ps1 @@ -164,10 +164,13 @@ foreach ($f in $findings) { continue } + $rawBody = if ($f.body) { [string]$f.body } elseif ($f.message) { [string]$f.message } elseif ($f.content) { [string]$f.content } else { "(no description)" } + $aiMarker = "> 🔍 **AI-Generated Review** (multi-model: GPT-5.5, Claude Opus 4.8, Claude Opus 4.6, Gemini 3.1 Pro)`n`n" + $comment = @{ path = $p line = [int]$f.line - body = if ($f.body) { [string]$f.body } elseif ($f.message) { [string]$f.message } elseif ($f.content) { [string]$f.content } else { "(no description)" } + body = $aiMarker + $rawBody } # GitHub API requires 'side' for pull request review comments $comment['side'] = 'RIGHT' From ead1342b6d6f495cbf50dc16007a7c2e054ca8a0 Mon Sep 17 00:00:00 2001 From: kubaflo Date: Tue, 23 Jun 2026 00:30:54 +0200 Subject: [PATCH 4/7] Address review findings: fix /review rerun mechanics, permissions, and session retention Must-fix: - Document that /review rerun applies s/agent-ready-for-rerun label and is processed by hourly scanner (may delay ~1h), not direct pipeline trigger - Fix permission documentation: non-first-time PR authors can rerun their own PR without write access Should-fix: - Older AI Summary sessions are hidden as outdated, not retained side-by-side - Test comment heading is "## Tests Failure Analysis", not "## Test Failure Review" - Remove phantom phase from Review-PR.ps1 phase list and fix regression/gate order Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> --- .github/docs/pr-review-workflow.md | 27 ++++++++++++++------------- 1 file changed, 14 insertions(+), 13 deletions(-) diff --git a/.github/docs/pr-review-workflow.md b/.github/docs/pr-review-workflow.md index 4929cf48421a..a0650e61ee3a 100644 --- a/.github/docs/pr-review-workflow.md +++ b/.github/docs/pr-review-workflow.md @@ -14,10 +14,10 @@ It is intended for Microsoft maintainers and community contributors who want to | --- | --- | --- | --- | | `/review` | Repository users with write, maintain, or admin access | Queues the full MAUI Copilot PR review pipeline. | Updates the PR with an `AI Summary` comment. | | `/review ` | Repository users with write, maintain, or admin access | Queues the full review pipeline for a specific platform: `android`, `ios`, `catalyst`, or `windows`. | Updates the PR with an `AI Summary` comment. | -| `/review rerun` | Repository users with write, maintain, or admin access | Requests a fresh full review after comments, commits, or CI context changed. It uses the same review pipeline as `/review`. | Adds or replaces a review session in the `AI Summary` comment. | +| `/review rerun` | Repository users with write, maintain, or admin access; or non-first-time PR authors on their own PR | Requests a fresh full review after comments, commits, or CI context changed. Applies the `s/agent-ready-for-rerun` label; the hourly scanner then triggers the review pipeline. | Adds or replaces a review session in the `AI Summary` comment. | | `/review tests` | Repository users with write, maintain, or admin access | Reviews current CI/test failures and classifies whether they are likely PR-caused, unrelated, or insufficiently evidenced. | Adds or updates a `Test Failure Review` comment. | -Community contributors cannot directly trigger these commands unless they have repository write access. If you are a community contributor, ask a maintainer to run the relevant command for your PR. +Community contributors can trigger `/review rerun` on their own PR after their first contribution. For `/review` and `/review tests`, only repository users with write access can trigger these commands. If you are a first-time contributor, ask a maintainer to run the relevant command for your PR. ## Choosing the right command @@ -97,20 +97,21 @@ The PR review script is `.github/scripts/Review-PR.ps1`. It orchestrates the cor 1. branch setup and PR merge for review; 2. UI category detection; -3. local/in-process UI test discovery and initial results; -4. regression cross-reference; -5. gate verification; -6. candidate review and fix exploration; -7. AI summary posting; -8. review labels. +3. regression cross-reference; +4. gate verification; +5. candidate review and fix exploration; +6. AI summary posting; +7. review labels. -The generated PR comment is a single session-based `AI Summary` comment. New runs add or replace a session keyed by the reviewed commit, so readers can compare the latest run with older review sessions. +The generated PR comment is a single session-based `AI Summary` comment. New runs replace the review and hide older sessions, keyed by the reviewed commit. ## `/review rerun`: fresh full review Comment `/review rerun` when you want a new full review session after the PR changed. -Operationally, this goes through the same `/review` trigger and review pipeline. The `rerun` token is used as an intent signal for humans and the AI summary UX: it tells readers that the command was meant to refresh the review after new information became available. +Operationally, `/review rerun` applies the `s/agent-ready-for-rerun` label to your PR. The hourly `rerun-review-scanner` workflow checks for eligible PRs with this label and triggers the review pipeline. This means your review may be delayed up to ~1 hour, or may not run if the PR is ineligible. + +If you need an immediate review, use `/review` instead (write access required). ### Eligibility requirements @@ -243,7 +244,7 @@ The full `/review` and `/review rerun` pipeline posts an `AI Summary` comment. I - fix/candidate analysis; - final recommendation. -The latest session is expanded by default. Older sessions are retained for comparison. +The latest session is expanded by default. Older sessions are minimized and hidden as outdated. ### Test Failure Review @@ -252,10 +253,10 @@ The latest session is expanded by default. Older sessions are retained for compa The top-level title is always: ```markdown -## Test Failure Review +## Tests Failure Analysis ``` -The verdict details live in badges and in the expanded review session. +The verdict details and "Test Failure Review" label live in badges and in the expanded review session. ## Recommended workflow for maintainers From ac2dc2e77c067d7d62887da47935826120df76e2 Mon Sep 17 00:00:00 2001 From: kubaflo Date: Tue, 23 Jun 2026 18:29:01 +0200 Subject: [PATCH 5/7] Address Round 2 review: fix regression no-op suppression + emoji/section findings MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit PureWeen's Round 2 multi-model review flagged a regression introduced by the earlier emoji/reorder commit plus a few smaller issues: - [must-fix, 3/3] Regression no-op suppression broke: Find-RegressionRisks.ps1 emits '● No regression risks detected…' / '● No implementation files modified…' but Test-PhaseContentIsNoOp still matched the literal '🟢', so the empty regression section leaked into every clean PR's AI Summary. Updated both consumer regexes to '(?:●|🟢)' (alternation, not a char class — 🟢 is an astral surrogate pair that splits inside []) and refreshed the test fixtures to '●' with a 🟢 back-compat case. - [3/3] Section order now matches the commit's stated intent: Next Steps below Review Sessions (the earlier swap had inverted it). - [3/3] post-inline-review.ps1 marker no longer hardcodes a stale model roster — now a generic '(multi-model)' label. - [2/3] Distinct colorless severity glyphs in the regression markdown: ✗ REVERT (error), ⚠ OVERLAP (warning), ● CLEAN (status) — matching the ✗/⚠/ℹ set used elsewhere. Review-PR.ps1 keys off the JSON result field, not the glyph, so no new producer/consumer mismatch. - [1/3] Troubleshooting row corrected to match the session replace/hide wording. Validated: full .github/scripts Pester suite 241/241 pass. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> --- .github/docs/pr-review-workflow.md | 2 +- .github/scripts/Find-RegressionRisks.ps1 | 6 +++--- .github/scripts/Post-AISummaryComment.Tests.ps1 | 17 ++++++++++++++--- .github/scripts/post-ai-summary-comment.ps1 | 8 ++++---- .github/scripts/post-inline-review.ps1 | 2 +- 5 files changed, 23 insertions(+), 12 deletions(-) diff --git a/.github/docs/pr-review-workflow.md b/.github/docs/pr-review-workflow.md index a0650e61ee3a..09cd1de0bea2 100644 --- a/.github/docs/pr-review-workflow.md +++ b/.github/docs/pr-review-workflow.md @@ -295,7 +295,7 @@ Important safeguards: | `/review` used the wrong platform | Platform labels were missing or ambiguous. | Re-run with an explicit platform, for example `/review ios`. | | `/review tests` says `Insufficient data` | Build/log/Helix evidence was inaccessible or incomplete. | Re-run later, provide a build ID, or run locally with Azure CLI/AzDO auth. | | The AI Summary looks stale | New commits or comments landed after the last review. | Run `/review rerun`. | -| There are multiple old sessions | The comment preserves prior review sessions for traceability. | Read the expanded latest session first. | +| There are multiple old sessions | Older review sessions are minimized and hidden as outdated; only the latest session is kept. | Read the expanded latest session — it reflects the current HEAD commit. | | `/review rerun` didn't trigger | Only PR author activity (commits or non-command comments) satisfies eligibility. Reviewer comments don't trigger reruns. | Wait for author activity or use `/review` to force a new review. | | Command comment is still visible | The commenter may lack authorization, or the command was malformed. | Check actor permissions and command syntax. Authorized commands are minimized after processing. | diff --git a/.github/scripts/Find-RegressionRisks.ps1 b/.github/scripts/Find-RegressionRisks.ps1 index 6e3e2fb1a768..715bb974129f 100644 --- a/.github/scripts/Find-RegressionRisks.ps1 +++ b/.github/scripts/Find-RegressionRisks.ps1 @@ -732,14 +732,14 @@ if ($OutputDir) { [void]$md.AppendLine() switch ($result) { 'REVERT' { - [void]$md.AppendLine("● **Revert risks detected** — this PR removes $($reverts.Count) line(s) previously added by labeled bug-fix PRs.") + [void]$md.AppendLine("✗ **Revert risks detected** — this PR removes $($reverts.Count) line(s) previously added by labeled bug-fix PRs.") [void]$md.AppendLine() [void]$md.AppendLine("| File | Fix PR | Fixed issue(s) | Risk | Reverted line |") [void]$md.AppendLine("|---|---|---|---|---|") foreach ($r in $reverts) { $sample = @($r.RevertedLines) | Select-Object -First 1 | ForEach-Object { $_.Text.Trim() } $sampleEsc = ($sample -replace '\|', '\|') - [void]$md.AppendLine("| ``$($r.File)`` | #$($r.RecentPR) | $($r.FixedIssues) | ● REVERT | ``$sampleEsc`` |") + [void]$md.AppendLine("| ``$($r.File)`` | #$($r.RecentPR) | $($r.FixedIssues) | ✗ REVERT | ``$sampleEsc`` |") } $allIssues = @($reverts | ForEach-Object { $_.FixedIssues -split ',\s*' } | Where-Object { $_ } | Select-Object -Unique | Sort-Object) @@ -767,7 +767,7 @@ if ($OutputDir) { } } 'OVERLAP' { - [void]$md.AppendLine("● **Overlaps with prior bug-fix PRs** — same files modified, but no exact line revert detected.") + [void]$md.AppendLine("⚠ **Overlaps with prior bug-fix PRs** — same files modified, but no exact line revert detected.") [void]$md.AppendLine() [void]$md.AppendLine("| File | Fix PR | Fixed issue(s) |") [void]$md.AppendLine("|---|---|---|") diff --git a/.github/scripts/Post-AISummaryComment.Tests.ps1 b/.github/scripts/Post-AISummaryComment.Tests.ps1 index ee0d3767ba62..e3533539a26b 100644 --- a/.github/scripts/Post-AISummaryComment.Tests.ps1 +++ b/.github/scripts/Post-AISummaryComment.Tests.ps1 @@ -60,19 +60,30 @@ Describe 'Test-PhaseContentIsNoOp' { It 'suppresses regression placeholders when there are no implementation files or risks' { Test-PhaseContentIsNoOp ` -PhaseKey 'regression-check' ` - -Content '🟢 No implementation files modified — skipping regression cross-reference.' | + -Content '● No implementation files modified — skipping regression cross-reference.' | + Should -BeTrue + + Test-PhaseContentIsNoOp ` + -PhaseKey 'regression-check' ` + -Content "## 🔍 Regression Cross-Reference`n`n● No regression risks detected. No labeled bug-fix PRs in the last 6 months touched the modified files." | Should -BeTrue + # Back-compat: the legacy 🟢 glyph is still recognized as a no-op. Test-PhaseContentIsNoOp ` -PhaseKey 'regression-check' ` - -Content "## 🔍 Regression Cross-Reference`n`n🟢 No regression risks detected. No labeled bug-fix PRs in the last 6 months touched the modified files." | + -Content '🟢 No implementation files modified — skipping regression cross-reference.' | Should -BeTrue } It 'keeps actionable regression content' { Test-PhaseContentIsNoOp ` -PhaseKey 'regression-check' ` - -Content "## 🔍 Regression Cross-Reference`n`n🟡 **Overlaps with prior bug-fix PRs** — same files modified, but no exact line revert detected." | + -Content "## 🔍 Regression Cross-Reference`n`n⚠ **Overlaps with prior bug-fix PRs** — same files modified, but no exact line revert detected." | + Should -BeFalse + + Test-PhaseContentIsNoOp ` + -PhaseKey 'regression-check' ` + -Content "## 🔍 Regression Cross-Reference`n`n✗ **Revert risks detected** — this PR removes 2 line(s) previously added by labeled bug-fix PRs." | Should -BeFalse Test-PhaseContentIsNoOp ` diff --git a/.github/scripts/post-ai-summary-comment.ps1 b/.github/scripts/post-ai-summary-comment.ps1 index 444747054f22..2a07ba8af3fc 100644 --- a/.github/scripts/post-ai-summary-comment.ps1 +++ b/.github/scripts/post-ai-summary-comment.ps1 @@ -97,8 +97,8 @@ function Test-PhaseContentIsNoOp { "regression-check" { $withoutHeading = ($normalized -replace '(?m)^##\s+.*Regression Cross-Reference\s*\n+', '').Trim() return ( - $withoutHeading -match '^🟢\s+No implementation files modified\s+[—-]\s+skipping regression cross-reference\.\s*$' -or - $withoutHeading -match '^🟢\s+No regression risks detected\.\s+No labeled bug-fix PRs in the last \d+ months touched the modified files\.\s*$' + $withoutHeading -match '^(?:●|🟢)\s+No implementation files modified\s+[—-]\s+skipping regression cross-reference\.\s*$' -or + $withoutHeading -match '^(?:●|🟢)\s+No regression risks detected\.\s+No labeled bug-fix PRs in the last \d+ months touched the modified files\.\s*$' ) } default { @@ -618,9 +618,9 @@ $authorPing $statusChipRow -$futureActionSection - $newSessionBlock + +$futureActionSection "@ # Clean up excessive blank lines diff --git a/.github/scripts/post-inline-review.ps1 b/.github/scripts/post-inline-review.ps1 index a3628488aedc..f421c5c984b2 100644 --- a/.github/scripts/post-inline-review.ps1 +++ b/.github/scripts/post-inline-review.ps1 @@ -165,7 +165,7 @@ foreach ($f in $findings) { } $rawBody = if ($f.body) { [string]$f.body } elseif ($f.message) { [string]$f.message } elseif ($f.content) { [string]$f.content } else { "(no description)" } - $aiMarker = "> 🔍 **AI-Generated Review** (multi-model: GPT-5.5, Claude Opus 4.8, Claude Opus 4.6, Gemini 3.1 Pro)`n`n" + $aiMarker = "> 🔍 **AI-Generated Review** (multi-model)`n`n" $comment = @{ path = $p From bad5f62c7627e2c3ee2b59dadae8f55ac408a3f3 Mon Sep 17 00:00:00 2001 From: kubaflo Date: Wed, 24 Jun 2026 21:18:16 +0200 Subject: [PATCH 6/7] Doc: review sessions render collapsed (click to expand), not expanded by default MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Round-3 review noted that the AI Summary wraps the Review Sessions in a plain
(no 'open' attribute) with a 'click to expand' summary, so it renders collapsed — the doc's 'expanded by default' wording was inaccurate. That
block is pre-existing on the base branch and untouched by this PR, so the in-scope fix is the doc: describe the section as collapsible and tell readers to expand it. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> --- .github/docs/pr-review-workflow.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/.github/docs/pr-review-workflow.md b/.github/docs/pr-review-workflow.md index 09cd1de0bea2..6f9adb8ca088 100644 --- a/.github/docs/pr-review-workflow.md +++ b/.github/docs/pr-review-workflow.md @@ -244,7 +244,7 @@ The full `/review` and `/review rerun` pipeline posts an `AI Summary` comment. I - fix/candidate analysis; - final recommendation. -The latest session is expanded by default. Older sessions are minimized and hidden as outdated. +The review sessions are collapsed by default — expand the **Review Sessions** section to read them. Within it, the latest session appears first; older sessions are minimized and hidden as outdated. ### Test Failure Review @@ -295,7 +295,7 @@ Important safeguards: | `/review` used the wrong platform | Platform labels were missing or ambiguous. | Re-run with an explicit platform, for example `/review ios`. | | `/review tests` says `Insufficient data` | Build/log/Helix evidence was inaccessible or incomplete. | Re-run later, provide a build ID, or run locally with Azure CLI/AzDO auth. | | The AI Summary looks stale | New commits or comments landed after the last review. | Run `/review rerun`. | -| There are multiple old sessions | Older review sessions are minimized and hidden as outdated; only the latest session is kept. | Read the expanded latest session — it reflects the current HEAD commit. | +| There are multiple old sessions | Older review sessions are minimized and hidden as outdated; only the latest session is kept. | Expand the **Review Sessions** section and read the latest session — it reflects the current HEAD commit. | | `/review rerun` didn't trigger | Only PR author activity (commits or non-command comments) satisfies eligibility. Reviewer comments don't trigger reruns. | Wait for author activity or use `/review` to force a new review. | | Command comment is still visible | The commenter may lack authorization, or the command was malformed. | Check actor permissions and command syntax. Authorized commands are minimized after processing. | From 11c74c41b1949e3b303cb63aa67c55d630f6b54e Mon Sep 17 00:00:00 2001 From: kubaflo Date: Thu, 25 Jun 2026 17:20:41 +0200 Subject: [PATCH 7/7] Doc: correct the Review Sessions description (one session per comment) Round-4 review caught that the round-3 fix introduced a fresh inaccuracy: the clause 'the latest session appears first; older sessions are minimized within it' implied a nested latest-first history. In reality each AI Summary comment holds exactly one session (keyed to the current HEAD commit); previous review *comments* are separate and minimized via Hide-StaleMauiBot*. Reword so the doc matches the script's own synopsis ('the replacement review contains only the latest review session, keyed by the current HEAD commit SHA'). Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> --- .github/docs/pr-review-workflow.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/.github/docs/pr-review-workflow.md b/.github/docs/pr-review-workflow.md index 6f9adb8ca088..9f559313ce73 100644 --- a/.github/docs/pr-review-workflow.md +++ b/.github/docs/pr-review-workflow.md @@ -244,7 +244,7 @@ The full `/review` and `/review rerun` pipeline posts an `AI Summary` comment. I - fix/candidate analysis; - final recommendation. -The review sessions are collapsed by default — expand the **Review Sessions** section to read them. Within it, the latest session appears first; older sessions are minimized and hidden as outdated. +The review sessions are collapsed by default — expand the **Review Sessions** section to read the latest session, which is keyed to the current HEAD commit. Previous review comments are minimized and hidden as outdated. ### Test Failure Review @@ -295,7 +295,7 @@ Important safeguards: | `/review` used the wrong platform | Platform labels were missing or ambiguous. | Re-run with an explicit platform, for example `/review ios`. | | `/review tests` says `Insufficient data` | Build/log/Helix evidence was inaccessible or incomplete. | Re-run later, provide a build ID, or run locally with Azure CLI/AzDO auth. | | The AI Summary looks stale | New commits or comments landed after the last review. | Run `/review rerun`. | -| There are multiple old sessions | Older review sessions are minimized and hidden as outdated; only the latest session is kept. | Expand the **Review Sessions** section and read the latest session — it reflects the current HEAD commit. | +| There are multiple old AI Summary comments | Each comment holds only the latest session (keyed to its HEAD commit); previous review comments are minimized and hidden as outdated. | Expand the **Review Sessions** section in the newest comment — it reflects the current HEAD commit. | | `/review rerun` didn't trigger | Only PR author activity (commits or non-command comments) satisfies eligibility. Reviewer comments don't trigger reruns. | Wait for author activity or use `/review` to force a new review. | | Command comment is still visible | The commenter may lack authorization, or the command was malformed. | Check actor permissions and command syntax. Authorized commands are minimized after processing. |