Repository navigation
Auto-apply s/agent-ready-for-rerun from the PR Review Queue workflow #36358
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Closed
Closed
Changes from 3 commits
Commits
Show all changes
40 commits
Select commit
Hold shift + click to select a range
c98bc94
Auto-apply s/agent-ready-for-rerun from the PR Review Queue workflow
Copilot 152406b
Grant pull-requests: write so the auto-label step can apply the label
Copilot a2fc3e9
Allow scanner to trigger reruns from autonomous queue labels
Copilot fb233b4
Harden auto-rerun labeler: per-PR error isolation, arg-passing, fetch…
Copilot 9b0a5aa
Auto-rerun labeler: single-source label metadata, fail-loud label fet…
Copilot f2cd01e
auto-rerun labeler: fail loud on gh api errors in activity/commit fet…
Copilot b60eadf
Merge remote-tracking branch 'origin/main' into feature/auto-rerun-la…
Copilot 5c53948
auto-rerun labeler: fix -Limit help default, fail-loud label re-read,…
Copilot 5b4acfe
Address review: verify-read regression, label-desc drift, redundant l…
Copilot 8d72e52
Fix autonomous-rerun daily flap on scanner skip (last-declined checkp…
Copilot fe99a54
Doc: note head-SHA-differs as qualifying rerun activity
Copilot 6b46aac
Surface gh api stderr in Get-IssueLabels on failure
Copilot 0fa7212
Make -ApplyLabel post-apply verification failure-tolerant
Copilot 2637ec3
Merge origin/main into feature/auto-rerun-labeler
kubaflo 620a7d9
Merge main into feature/auto-rerun-labeler
Copilot 5886164
Harden autonomous rerun candidate scanning
d4ed6b3
Make rerun tests hermetic in full suite
3a3c1ab
Make rerun decline lifecycle race-safe
1967d88
Address auto-rerun review feedback
9657be3
Handle concurrent rerun label creation
e60d765
Harden autonomous rerun queue handling
fc4d474
Fix auto-rerun review activity checkpoints
89a9bae
Make rerun decline markers retry-safe
08ce06b
Harden PR queue validation
16a93c7
Fix rerun decline activity checkpoints
d29cfbd
Paginate rerun decline markers
d4a0773
Correct rerun queue guidance
e35aebd
Harden rerun scanner decisions
75a906c
Harden rerun scanner validation
d14abf2
Harden rerun queue activity handling
2a399db
Keep queue contract test outside skill validation
17dc7d3
Clarify scanner decline marker contract
5b9ec42
Paginate rerun label verification
b37011d
Harden rerun pagination and anti-flap
55879a1
Secure rerun logging and scanner coverage
0f44489
Disable queue checkout credential persistence
8f2a314
Harden rerun validation and live dispatch
94a3766
Discover queue validation test suites
98dc01d
Fix rerun eligibility and batch validation
d09eaa9
Fix mixed rerun decision batch validation
File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Some comments aren't visible on the classic Files Changed page.
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,188 @@ | ||
| #!/usr/bin/env pwsh | ||
| <# | ||
| .SYNOPSIS | ||
| Autonomously applies the s/agent-ready-for-rerun label to open PRs that have | ||
| genuinely new PR-author activity since their last AI review. | ||
|
|
||
| .DESCRIPTION | ||
| Driver for the PR Review Queue workflow. Enumerates open, non-draft PRs and | ||
| evaluates each with Resolve-AutonomousRerunEligibility — the deterministic, | ||
| AI-free counterpart of the `/review rerun` eligibility check. A PR qualifies | ||
| only when it already carries a MauiBot AI Summary AND has a new commit, a new | ||
| non-command author comment, or a head SHA that differs from the last reviewed | ||
| SHA since that summary. Eligible PRs are labelled s/agent-ready-for-rerun, | ||
| which the hourly rerun-review-scanner then picks up and re-reviews. | ||
|
|
||
| No AI is used and untrusted text is never inspected semantically. PRs that | ||
| were never AI-reviewed do not qualify. PRs that already carry the label, or | ||
| that have a non-stale s/agent-review-in-progress label, are skipped. | ||
|
|
||
| .PARAMETER Owner | ||
| Repository owner (default: dotnet). | ||
|
|
||
| .PARAMETER Repo | ||
| Repository name (default: maui). | ||
|
|
||
| .PARAMETER Limit | ||
| Maximum number of open PRs to inspect (default: 100). | ||
|
|
||
| .PARAMETER DryRun | ||
| Evaluate and report without applying any labels. | ||
|
|
||
| .PARAMETER OutputPath | ||
| Optional path to write a JSON summary of the decisions. | ||
|
|
||
| .EXAMPLE | ||
| ./Query-AutoRerunCandidates.ps1 -DryRun | ||
| # Read-only: classify open PRs without applying labels. | ||
| #> | ||
|
|
||
| param( | ||
| [string]$Owner = 'dotnet', | ||
| [string]$Repo = 'maui', | ||
| [int]$Limit = 100, | ||
| [switch]$DryRun, | ||
| [string]$OutputPath | ||
| ) | ||
|
|
||
| $ErrorActionPreference = 'Stop' | ||
|
|
||
| $ReadyForRerunLabel = 's/agent-ready-for-rerun' | ||
| $ReviewInProgressLabel = 's/agent-review-in-progress' | ||
| $ReadyForRerunLabelDescription = 'AI review has a new PR-author comment or commit and is ready for rerun' | ||
|
kubaflo marked this conversation as resolved.
Resolved
|
||
| $ReadyForRerunLabelColor = '5319E7' | ||
|
kubaflo marked this conversation as resolved.
Resolved
|
||
|
|
||
| . "$PSScriptRoot/Resolve-RerunEligibility.ps1" | ||
| . "$PSScriptRoot/shared/Update-AgentLabels.ps1" | ||
|
kubaflo marked this conversation as resolved.
Resolved
|
||
|
|
||
| function Get-IssueLabels { | ||
| param([int]$Number) | ||
|
|
||
| return @(gh api "repos/$Owner/$Repo/issues/$Number/labels" --jq '.[].name' 2>$null) | ||
| } | ||
|
kubaflo marked this conversation as resolved.
Resolved
|
||
|
|
||
| function Get-ActivityForPR { | ||
| param([int]$Number) | ||
|
|
||
| $issueComments = @(gh api "repos/$Owner/$Repo/issues/$Number/comments?per_page=100" --paginate --jq '.[]' | ForEach-Object { ConvertTo-RerunActivityItem -Item ($_ | ConvertFrom-Json) -Kind 'issue-comment' }) | ||
| $reviews = @(gh api "repos/$Owner/$Repo/pulls/$Number/reviews?per_page=100" --paginate --jq '.[]' | ForEach-Object { ConvertTo-RerunActivityItem -Item ($_ | ConvertFrom-Json) -Kind 'review' }) | ||
| $reviewComments = @(gh api "repos/$Owner/$Repo/pulls/$Number/comments?per_page=100" --paginate --jq '.[]' | ForEach-Object { ConvertTo-RerunActivityItem -Item ($_ | ConvertFrom-Json) -Kind 'review-comment' }) | ||
| return @($issueComments + $reviews + $reviewComments) | ||
| } | ||
|
|
||
| function Get-CommitsForPR { | ||
| param([int]$Number) | ||
|
|
||
| return @(gh api "repos/$Owner/$Repo/pulls/$Number/commits?per_page=100" --paginate --jq '.[]' | ForEach-Object { $_ | ConvertFrom-Json }) | ||
| } | ||
|
kubaflo marked this conversation as resolved.
Resolved
|
||
|
|
||
| $searchJson = gh pr list ` | ||
| --repo "$Owner/$Repo" ` | ||
| --state open ` | ||
| --limit $Limit ` | ||
| --json number,title,url,headRefOid,isDraft,labels,author | ||
| if ($LASTEXITCODE -ne 0) { | ||
| throw "Failed to list open PRs (gh pr list exited with code $LASTEXITCODE)." | ||
| } | ||
| $openPRs = @($searchJson | ConvertFrom-Json) | ||
|
|
||
| Write-Host "Inspecting $($openPRs.Count) open PR(s) for autonomous rerun eligibility..." | ||
|
|
||
| $labelEnsured = $false | ||
| $decisions = @() | ||
| $appliedCount = 0 | ||
|
|
||
| foreach ($pr in $openPRs) { | ||
|
kubaflo marked this conversation as resolved.
Resolved
|
||
| $number = [int]$pr.number | ||
| $title = [string]$pr.title | ||
|
|
||
| if ($pr.isDraft) { | ||
| $decisions += [pscustomobject]@{ prNumber = $number; title = $title; eligible = $false; reason = 'draft'; applied = $false } | ||
| continue | ||
| } | ||
|
|
||
| $labels = @(Get-IssueLabels -Number $number) | ||
|
|
||
|
kubaflo marked this conversation as resolved.
Resolved
|
||
| # Treat a stale in-progress label as absent so a wedged review can still be | ||
| # re-detected — the same staleness rule the rerun scanner uses. | ||
| $effectiveLabels = @($labels) | ||
| if ($labels -contains $ReviewInProgressLabel -and (Test-AgentReviewInProgressIsStale -PRNumber $number -Owner $Owner -Repo $Repo)) { | ||
| $effectiveLabels = @($labels | Where-Object { $_ -ne $ReviewInProgressLabel }) | ||
| } | ||
|
|
||
| $activity = @(Get-ActivityForPR -Number $number) | ||
| $commits = @(Get-CommitsForPR -Number $number) | ||
| $rawAuthorLogin = if ($pr.author -and $pr.author.login) { [string]$pr.author.login } else { '' } | ||
| $authorLogin = Normalize-GitHubActorLogin $rawAuthorLogin | ||
|
|
||
| $result = Resolve-AutonomousRerunEligibility ` | ||
| -Comments $activity ` | ||
| -Commits $commits ` | ||
| -CurrentHeadSha $pr.headRefOid ` | ||
| -PRAuthorLogin $authorLogin ` | ||
| -CurrentLabels $effectiveLabels | ||
|
|
||
| $alreadyPresent = @($labels | Where-Object { $_ -eq $ReadyForRerunLabel }).Count -gt 0 | ||
| $applied = $false | ||
|
|
||
| if ($result.Eligible -and -not $alreadyPresent) { | ||
| if ($DryRun) { | ||
| Write-Host " [dry-run] Would label #$number ($($result.Reason)): $title" | ||
| } else { | ||
| if (-not $labelEnsured) { | ||
| Ensure-LabelExists ` | ||
| -LabelName $ReadyForRerunLabel ` | ||
| -Description $ReadyForRerunLabelDescription ` | ||
| -Color $ReadyForRerunLabelColor ` | ||
| -Owner $Owner ` | ||
| -Repo $Repo | ||
| $labelEnsured = $true | ||
| } | ||
|
|
||
| $addSucceeded = Add-Label -PRNumber $number -LabelName $ReadyForRerunLabel -Owner $Owner -Repo $Repo | ||
| $updatedLabels = @(gh api "repos/$Owner/$Repo/issues/$number/labels" --jq '.[].name' 2>$null) | ||
| $labelIsPresent = @($updatedLabels | Where-Object { $_ -eq $ReadyForRerunLabel }).Count -gt 0 | ||
|
kubaflo marked this conversation as resolved.
Resolved
|
||
| if ($addSucceeded -or $labelIsPresent) { | ||
| $applied = $true | ||
| $appliedCount++ | ||
| Write-Host " ✅ Applied $ReadyForRerunLabel to #$number ($($result.Reason)): $title" -ForegroundColor Green | ||
| } else { | ||
| Write-Host " ⚠️ Failed to apply $ReadyForRerunLabel to #$number" -ForegroundColor Yellow | ||
|
kubaflo marked this conversation as resolved.
Resolved
|
||
| } | ||
| } | ||
| } elseif ($result.Eligible -and $alreadyPresent) { | ||
| Write-Host " ⏭️ #$number already has $ReadyForRerunLabel — skipping" | ||
| } | ||
|
|
||
| $decisions += [pscustomobject]@{ | ||
| prNumber = $number | ||
| title = $title | ||
| eligible = [bool]$result.Eligible | ||
| reason = [string]$result.Reason | ||
| alreadyPresent = $alreadyPresent | ||
| applied = $applied | ||
| } | ||
| } | ||
|
|
||
| $eligibleCount = @($decisions | Where-Object { $_.eligible -and -not $_.alreadyPresent }).Count | ||
|
kubaflo marked this conversation as resolved.
Resolved
|
||
| if ($DryRun) { | ||
| Write-Host "Autonomous rerun scan complete: $eligibleCount PR(s) eligible (dry-run, no labels applied)." | ||
| } else { | ||
| Write-Host "Autonomous rerun scan complete: applied $ReadyForRerunLabel to $appliedCount PR(s)." | ||
| } | ||
|
|
||
| if ($OutputPath) { | ||
| $outputDir = Split-Path -Parent $OutputPath | ||
| if ($outputDir) { | ||
| New-Item -ItemType Directory -Force -Path $outputDir | Out-Null | ||
| } | ||
| $summary = @{ | ||
| generatedAt = (Get-Date).ToUniversalTime().ToString('o') | ||
| dryRun = [bool]$DryRun | ||
| applied = $appliedCount | ||
| eligible = $eligibleCount | ||
| decisions = @($decisions) | ||
| } | ConvertTo-Json -Depth 10 | ||
| $summary | Set-Content -LiteralPath $OutputPath -Encoding UTF8 | ||
| Write-Host "Wrote decision summary to $OutputPath" | ||
| } | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.