Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
76 changes: 70 additions & 6 deletions .github/pr-review/pr-preflight.md
Original file line number Diff line number Diff line change
@@ -1,10 +1,10 @@
# PR Pre-Flight — Context Gathering
# PR Pre-Flight — Context Gathering & Code Review

> **SCOPE:** Document only. No code analysis. No fix opinions. No running tests.
> **SCOPE:** Gather context, classify files, and perform deep code review. No code changes. No fix selection. No test execution.

---

## Steps
## Part A: Context Gathering (Steps 1–6)

1. **Read the issue** — full body + ALL comments via GitHub MCP tools
2. **Find the PR** — read description, diff summary, review comments, inline feedback
Expand Down Expand Up @@ -35,7 +35,58 @@ gh pr view XXXXX --json comments --jq '.comments[] | select(.body | contains("Fi

---

## Output File
## Part B: Code Review (Step 7)

> **Purpose:** Perform deep code analysis using the `code-review` skill to surface correctness issues, safety concerns, and MAUI convention violations BEFORE Try-Fix explores alternatives. These findings guide Try-Fix models toward higher-quality fixes.

> **🚨 Independence-first requirement:** Step 7 MUST be invoked as a **separate sub-agent** (via the `task` tool with `agent_type: "general-purpose"`) so the code-review skill can form its assessment from the code BEFORE reading any PR narrative. The sub-agent receives ONLY the PR number — not the context gathered in Part A. This prevents anchoring bias.
>
> **Validation constraint:** The Step 7 prompt MUST NOT contain issue titles, root-cause descriptions, bug summaries, or any Part A content — only `PR #XXXXX`. If you find yourself adding context "to help" the sub-agent, you are violating independence-first.

7. **Invoke the code-review skill as a sub-agent:**

Use the `task` tool to launch a separate agent. The prompt MUST NOT contain issue titles, root-cause descriptions, or any Part A context — only the PR number.

```python
task(
name="code-review",
description="Code review for PR",
agent_type="general-purpose",
mode="sync",
prompt="""
Run the code-review skill for PR #XXXXX.
Follow the full 6-step workflow in .github/skills/code-review/SKILL.md.
Output the review in the format specified by that skill.
"""
)
```
Comment on lines +42 to +62
Copy link

Copilot AI Apr 16, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Step 7 says to invoke the code-review skill via the task tool as a separate sub-agent, but the example block only shows prompt, agent_type, and mode without indicating the actual task invocation shape used elsewhere (e.g., including the tool name/fields needed to launch the sub-agent, and—if required by the runner—the model selection). Tightening this snippet to match the real task call format would prevent confusion and make the “independence-first” requirement easier to follow correctly.

Copilot uses AI. Check for mistakes.

The sub-agent internally follows the code-review skill's 6-step workflow:
1. Gather code context (independence-first — reads code BEFORE PR description)
2. Load MAUI review rules from `.github/skills/code-review/references/review-rules.md`
3. Form independent assessment
4. Reconcile with PR narrative and prior reviews
5. Check CI status
6. Blast radius, failure-mode probing, and verdict

**If Step 7 fails, times out, or returns malformed output:**
- Write `pre-flight/code-review.md` with: `## Code Review: SKIPPED\n\nReason: {failure description}`
- Set verdict to `SKIPPED` in the Code Review Summary section of `content.md`
- Omit `hints` from Try-Fix prompts (the `hints` field becomes optional when code review is unavailable)
- Do NOT apply the code-review hard gate in Phase 3 (Report) — treat as if code review was not run

**Store the sub-agent's full output** in `pre-flight/code-review.md` — use the exact output format from the code-review skill (do NOT reformat or summarize into a different template).

**Extract key items for Try-Fix consumption** and add to `content.md`:
- All ❌ Error findings (with file:line references)
- All ⚠️ Warning findings (with file:line references)
- Failure-mode probes and their answers
- Blast radius assessment summary
- The overall verdict and confidence level

---

## Output Files

```bash
mkdir -p CustomAgentLogsTmp/PRState/{PRNumber}/PRAgent/pre-flight
Expand All @@ -52,16 +103,29 @@ Write `content.md`:
- {Finding 1}
- {Finding 2}

### Code Review Summary
**Verdict:** {LGTM / NEEDS_CHANGES / NEEDS_DISCUSSION / SKIPPED}
**Confidence:** {high / medium / low / N/A}
**Errors:** {count} | **Warnings:** {count} | **Suggestions:** {count}

Key code review findings:
- {❌/⚠️/💡} {Brief finding with file:line reference}
- ...
*(If SKIPPED: "Code review sub-agent failed or timed out. Reason: {details}")*

### Fix Candidates
| # | Source | Approach | Test Result | Files Changed | Notes |
|---|--------|----------|-------------|---------------|-------|
| PR | PR #XXXXX | {approach} | ⏳ PENDING (Gate) | `file.cs` | Original PR |
```

Write `code-review.md` — the exact output from the code-review sub-agent, in the format specified by `.github/skills/code-review/SKILL.md` (Review Output Format section). Do NOT reformat or create a custom template — preserve the skill's native output verbatim.

---

## Common Mistakes

- ❌ Researching root cause — save for Try-Fix phase
- ❌ Looking at implementation code — just gather context
- ❌ Skipping the code-review step — it provides critical findings for Try-Fix
- ❌ Reading the PR description before code in Step 7 — independence-first prevents anchoring bias
- ❌ Running tests — that's the Gate phase
- ❌ Proposing fixes — save fix ideas for Try-Fix phase
26 changes: 19 additions & 7 deletions .github/pr-review/pr-report.md
Original file line number Diff line number Diff line change
Expand Up @@ -12,18 +12,26 @@

- Phases 1-2 (Pre-Flight, Try-Fix) must be complete before starting
- Gate result is available from the prompt (ran separately before this skill)
- **Read `pre-flight/content.md`** to get the code-review summary (verdict, confidence, error/warning counts)
- Optionally read `pre-flight/code-review.md` for full findings if needed for the recommendation

---

## Steps

1. **Determine recommendation:**
1. **Determine recommendation** (rows evaluated in order — first match wins):

| Condition | Recommendation |
|-----------|----------------|
| PR's fix selected and Gate passed | `✅ APPROVE` |
| Alternative fix found via Try-Fix | `⚠️ REQUEST CHANGES` — suggest alternative |
| Gate failed | `⚠️ REQUEST CHANGES` — fix doesn't work |
| Priority | Condition | Recommendation |
|----------|-----------|----------------|
| 1 | Code review verdict is `NEEDS_CHANGES` (any ❌ errors) | `⚠️ REQUEST CHANGES` — code review found errors |
| 2 | Gate failed (tests fail with fix) | `⚠️ REQUEST CHANGES` — fix doesn't work |
| 3 | Alternative fix found via Try-Fix that is simpler/better | `⚠️ REQUEST CHANGES` — suggest alternative |
| 4 | Code review verdict is `NEEDS_DISCUSSION` | `⚠️ REQUEST CHANGES` — include code review concerns |
| 5 | PR's fix selected AND Gate passed AND code review LGTM or SKIPPED | `✅ APPROVE` |

**🚨 Hard gate:** If the code review (from Pre-Flight) has verdict `NEEDS_CHANGES`, the final recommendation MUST be `REQUEST CHANGES` regardless of Gate or Try-Fix results. Code-review ❌ Errors cannot be overridden by passing tests alone.

**Code review SKIPPED:** If the code-review sub-agent failed or timed out (verdict = `SKIPPED`), the hard gate does NOT apply. Proceed as if code review was not available — base the recommendation on Gate and Try-Fix results only. Note in the report that code review was unavailable.

2. **Write output files** — Save recommendation to `content.md`

Expand All @@ -47,18 +55,22 @@ Write `content.md`:
| Phase | Status | Notes |
|---|---|---|
| Pre-Flight | ✅ COMPLETE | {notes} |
| Code Review | {verdict} ({confidence}) | {error_count} errors, {warning_count} warnings |
| Gate | ✅ PASSED | {platform} |
| Try-Fix | ✅ COMPLETE | {N} attempts, {M} passing |
| Report | ✅ COMPLETE | |

### Code Review Impact on Try-Fix
{Brief description of how code-review findings influenced try-fix exploration. Did any model specifically address a code review ❌ Error? Did failure-mode probes reveal issues that guided fix approaches?}

### Summary
{Brief summary of the review}

### Root Cause
{Root cause analysis}

### Fix Quality
{Assessment of the fix}
{Assessment of the fix — informed by both gate results and code review findings}
```

---
Expand Down
11 changes: 7 additions & 4 deletions .github/scripts/Review-PR.ps1
Original file line number Diff line number Diff line change
Expand Up @@ -499,8 +499,11 @@ The agent will analyze the issue, determine the appropriate test type (UI test,
}
}

# Post gate result as a separate PR comment
$postGateScript = Join-Path $PSScriptRoot "post-gate-comment.ps1"
# Post gate result by updating (or creating) the unified AI Summary comment.
# The same script is called again in STEP 3 once the review phases finish; here
# it runs early so the PR author sees the gate outcome without waiting for the
# full review.
$postGateScript = Join-Path $PSScriptRoot "post-ai-summary-comment.ps1"
if (Test-Path $postGateScript) {
try {
if ($DryRun) {
Expand All @@ -509,10 +512,10 @@ if (Test-Path $postGateScript) {
& $postGateScript -PRNumber $PRNumber
}
} catch {
Write-Host " ⚠️ Failed to post gate comment (non-fatal): $_" -ForegroundColor Yellow
Write-Host " ⚠️ Failed to post gate section (non-fatal): $_" -ForegroundColor Yellow
}
} else {
Write-Host " ⚠️ post-gate-comment.ps1 not found" -ForegroundColor Yellow
Write-Host " ⚠️ post-ai-summary-comment.ps1 not found" -ForegroundColor Yellow
}

# Apply gate result label
Expand Down
87 changes: 76 additions & 11 deletions .github/scripts/post-ai-summary-comment.ps1
Original file line number Diff line number Diff line change
Expand Up @@ -13,9 +13,16 @@
After posting, the PR author is @-mentioned so they know to review.

Content is auto-loaded from PRAgent phase files:
CustomAgentLogsTmp/PRState/<PRNumber>/PRAgent/gate/content.md (always shown first, open)
CustomAgentLogsTmp/PRState/<PRNumber>/PRAgent/{pre-flight,try-fix,report}/content.md
CustomAgentLogsTmp/PRState/<PRNumber>/PRAgent/pre-flight/code-review.md

Gate is posted separately by post-gate-comment.ps1.
Gate is included as a section inside this unified comment — the script may
be called by Review-PR.ps1 twice per run: once after the gate completes
(gate-only update) and once after the review phases finish (full update).

Any standalone legacy "<!-- AI Gate -->" comment from older versions of
the script is deleted after a successful post to avoid duplicates.

.PARAMETER PRNumber
The pull request number (required)
Expand Down Expand Up @@ -60,9 +67,34 @@ if (-not (Test-Path $PRAgentDir)) {
}

$phases = [ordered]@{
"pre-flight" = @{ File = "pre-flight/content.md"; Icon = "🔍"; Title = "Pre-Flight — Context & Validation" }
"try-fix" = @{ File = "try-fix/content.md"; Icon = "🔧"; Title = "Fix — Analysis & Comparison" }
"report" = @{ File = "report/content.md"; Icon = "📋"; Title = "Report — Final Recommendation" }
"pre-flight" = @{ File = "pre-flight/content.md"; Icon = "🔍"; Title = "Pre-Flight — Context & Validation" }
"code-review" = @{ File = "pre-flight/code-review.md"; Icon = "🔬"; Title = "Code Review — Deep Analysis" }
"try-fix" = @{ File = "try-fix/content.md"; Icon = "🔧"; Title = "Fix — Analysis & Comparison" }
"report" = @{ File = "report/content.md"; Icon = "📋"; Title = "Report — Final Recommendation" }
}

# ─── Gate content (rendered first, always open) ───
$gateSection = $null
$gateFilePath = Join-Path $PRAgentDir "gate/content.md"
if (Test-Path $gateFilePath) {
$gateContent = Get-Content $gateFilePath -Raw -Encoding UTF8
if (-not [string]::IsNullOrWhiteSpace($gateContent)) {
Write-Host " ✅ gate ($((Get-Item $gateFilePath).Length) bytes)" -ForegroundColor Green
$gateSection = @"
<details open>
<summary>🚦 <strong>Gate — Test Before & After Fix</strong></summary>

---

$gateContent

</details>
"@
} else {
Write-Host " ⏭️ gate (empty)" -ForegroundColor Gray
}
} else {
Write-Host " ⏭️ gate (not found)" -ForegroundColor Gray
}

$phaseSections = @()
Expand Down Expand Up @@ -93,8 +125,8 @@ $content
}
}

if ($phaseSections.Count -eq 0) {
throw "No phase content found. Ensure at least one content.md exists in $PRAgentDir."
if (-not $gateSection -and $phaseSections.Count -eq 0) {
throw "No gate or phase content found. Ensure at least one of gate/content.md or {phase}/content.md exists in $PRAgentDir."
}

# ============================================================================
Expand Down Expand Up @@ -123,7 +155,13 @@ $timestamp = (Get-Date).ToUniversalTime().ToString("yyyy-MM-dd HH:mm UTC")
# BUILD NEW SESSION BLOCK
# ============================================================================

$phaseContent = $phaseSections -join "`n`n---`n`n"
# Combine gate (always first, open) with phases (collapsed). When only one
# kind of content is available, the session still renders cleanly.
$sessionParts = @()
if ($gateSection) { $sessionParts += $gateSection }
if ($phaseSections.Count -gt 0) { $sessionParts += ($phaseSections -join "`n`n---`n`n") }
$phaseContent = $sessionParts -join "`n`n---`n`n"

$sessionMarkerStart = "<!-- SESSION:$commitSha7 START -->"
$sessionMarkerEnd = "<!-- SESSION:$commitSha7 END -->"

Expand Down Expand Up @@ -176,12 +214,17 @@ function Merge-Sessions {
foreach ($sha in $orderedKeys) {
$block = $sessions[$sha]
if ($isFirst) {
# Ensure latest session has <details open>
$block = $block -replace '<details(?:\s+open)?>', '<details open>'
# Ensure ONLY the outer (session-wrapping) details tag is open. Inner
# phase tags must keep their original open/collapsed state — we used
# to re-open all of them via a global regex replace, which forced
# every phase to expand on each new session.
$rx = [regex]::new('<details(?:\s+open)?>')
$block = $rx.Replace($block, '<details open>', 1)
$isFirst = $false
} else {
# Collapse older sessions
$block = $block -replace '<details\s+open>', '<details>'
# Collapse the outer details of older sessions; leave inner phases alone.
$rx = [regex]::new('<details\s+open>')
$block = $rx.Replace($block, '<details>', 1)
}
$allSessions += $block
}
Expand Down Expand Up @@ -300,3 +343,25 @@ try {
} finally {
Remove-Item $tempFile -ErrorAction SilentlyContinue
}

# ============================================================================
# CLEAN UP LEGACY STANDALONE GATE COMMENTS
# ============================================================================
# Earlier versions of this workflow posted gate results in a separate comment
# marked with <!-- AI Gate -->. Now that the gate is included as a section in
# this unified comment, those legacy comments are duplicates and should go.

try {
$legacyMarker = "<!-- AI Gate -->"
$allRaw = gh api "repos/dotnet/maui/issues/$PRNumber/comments" --paginate 2>$null
if ($allRaw) {
$allComments = $allRaw | ConvertFrom-Json
$legacy = @($allComments | Where-Object { $_.body -and $_.body.Contains($legacyMarker) })
foreach ($lc in $legacy) {
Write-Host "🧹 Deleting legacy gate comment (ID: $($lc.id))..." -ForegroundColor Gray
gh api --method DELETE "repos/dotnet/maui/issues/comments/$($lc.id)" 2>&1 | Out-Null
}
}
} catch {
Write-Host "⚠️ Legacy gate-comment cleanup failed (non-fatal): $_" -ForegroundColor Yellow
}
Loading
Loading