diff --git a/.github/skills/release-readiness/SKILL.md b/.github/skills/release-readiness/SKILL.md index 0286b635790a..e5e5744d166a 100644 --- a/.github/skills/release-readiness/SKILL.md +++ b/.github/skills/release-readiness/SKILL.md @@ -34,7 +34,7 @@ This skill has **three** PowerShell entry points and one workflow: | Script | Branch type | Purpose | |--------|-------------|---------| | [`Find-ReleaseReadinessTrackers.ps1`](scripts/Find-ReleaseReadinessTrackers.ps1) | both | Detects active in-flight & candidate trackers (SR and Preview) across all active majors using a four-lane algorithm and the **tag-existence rule** ("a release is in flight unless its tag already exists"). Emits a single tracker JSON consumed by the workflow. | -| [`Get-ReleaseReadiness.ps1`](scripts/Get-ReleaseReadiness.ps1) | SR | Full readiness report for a single SR branch (in-flight or `-Candidate`). | +| [`Get-ReleaseReadiness.ps1`](scripts/Get-ReleaseReadiness.ps1) | SR | Full readiness report for a single SR branch (in-flight, `-Candidate`, or `-Shipped`). `-Shipped` is a display-only relabel — it surveys the SR branch exactly like in-flight but renders the header as `mode=shipped` for the post-ship tracker. | | [`Get-PreviewReadiness.ps1`](scripts/Get-PreviewReadiness.ps1) | Preview | Full readiness report for a single Preview branch (in-flight or candidate via `-Mode candidate -SurveyRef net.0`). | | [`release-readiness.yml`](../../workflows/release-readiness.yml) | both | Daily cron + manual dispatch + PR validation. Runs `Find-Trackers -AllActiveMajors`, fans out a matrix job per tracker, and writes idempotent `[Release Readiness]` issues per branch. | @@ -42,9 +42,11 @@ This skill has **three** PowerShell entry points and one workflow: The trackers detector is grounded in **tag existence as the source of truth for "shipped vs in-flight"**. A release is in-flight if and only if its expected tag has NOT been published — branch existence, commit recency, and milestone state are all secondary signals. -- SR shipped tag pattern: `.0.` (e.g. `10.0.71` shipped → SR7 no longer produces a tracker) +- SR shipped tag pattern: `.0.` (e.g. `10.0.71` shipped → SR7 retired, no longer produces a tracker) - Preview shipped tag pattern: `.0.0-preview..[.]` (e.g. `11.0.0-preview.5.26304.4` shipped → preview5 no longer produces a tracker) +**Post-ship lifecycle (`shipped` mode).** Most shipped SRs are retired the moment their tag exists. The **one exception** is the *most-recently-shipped* SR (highest shipped patch), which keeps emitting as `mode='shipped'` so its tracker issue stays useful through post-ship follow-up — adding the new build to the GitHub issue version dropdown, publishing release notes, closing out the milestone. The workflow treats `shipped` as **refresh-only**: it updates the tracker issue while it stays open, but **never (re)creates it**. Once a human closes the tracker, it stays closed and is not resurrected on the next scheduled run. This implements "keep updating until closed manually" without spamming a fresh issue after sign-off. Older shipped SRs are still retired. + ## Quick Start ### One-shot daily report (matches what the workflow runs) @@ -59,7 +61,7 @@ pwsh .github/skills/release-readiness/scripts/Find-ReleaseReadinessTrackers.ps1 # branchType: 'sr' | 'preview' # branchName: canonical proposed branch slug (always populated) # branchExists: true if the branch is on origin, false for candidates -# mode: 'in-flight' | 'candidate' +# mode: 'in-flight' | 'candidate' | 'shipped' (shipped = most-recently-shipped SR, refresh-only) # surveyRef: ref to actually survey (branch itself, or net.0 for candidates) # canonicalKey: stable join key (e.g. net10-sr8, net11-preview6) # issueTitle: title for the daily tracker issue @@ -172,7 +174,7 @@ pwsh .github/skills/release-readiness/scripts/Get-PreviewReadiness.ps1 \ - Dispatches to `Get-ReleaseReadiness.ps1` (SR) or `Get-PreviewReadiness.ps1` (Preview) based on `branchType`. - Looks for an open tracker issue by the canonical marker ``. - **Refresh path**: reuse the oldest open tracker issue (edit title + body); close any duplicates. - - **Create path**: open a new issue with `report` / `s/triaged` / `area-release-readiness` labels. + - **Create path**: open a new issue with `report` / `s/triaged` / `area-infrastructure` labels (each attached best-effort — a label missing from the repo is skipped with a warning rather than failing the create). - **Activity gate**: skip new-issue creation when `recentCommitCount == 0` AND no open tracker issue exists. (Existing open issues are still refreshed.) 3. **`validate`** — PR-trigger path. Runs the test suite + smoke-runs all three scripts. **Does not create or modify issues.** @@ -202,6 +204,7 @@ Each candidate fix PR is classified with confidence + evidence: | `merged-non-main-only` | Fix merged but only to `inflight/current` (or similar), not `main` | | `open-on-main` | Fix PR is OPEN against main, not yet merged | | `no-fix-yet` | No fix PR cross-referenced from the regression issue | +| `closed-fix-unlinked` | Issue is CLOSED and a closing comment **explicitly names** a fix PR (fix/resolve/close language) that is MERGED and present on the release branch, but the PR↔issue link was never recorded (no closing keyword / cross-reference). A bare mention of a PR (e.g. naming the *cause* PR for context) does **not** qualify. Non-blocking; action is to add a closing reference for traceability | | `needs-human-review` | Evidence is contradictory or weak | ## CI Status Categories diff --git a/.github/skills/release-readiness/references/methodology.md b/.github/skills/release-readiness/references/methodology.md index cdd3a59a930d..6a7628b6731d 100644 --- a/.github/skills/release-readiness/references/methodology.md +++ b/.github/skills/release-readiness/references/methodology.md @@ -170,6 +170,7 @@ The skill must derive which `regressed-in-X.Y.Z` labels matter for a given SR: | `merged-on-main-no-backport` | Fix PR state=MERGED, `onMain=true`, no backport PR to `$SrBranch` exists | | `open-on-main` | Fix PR state=OPEN, base=main | | `no-fix-yet` | No cross-referenced PR with high-confidence evidence found | +| `closed-fix-unlinked` | `no-fix-yet` would apply, BUT the issue is CLOSED and a closing comment **explicitly names** a fix PR (fix/resolve/close language) that is MERGED and whose commit is on `$SrBranch` (verified by SHA-ancestry OR the `(#)` squash-subject token). A bare PR mention (the cause-PR blame pattern) is rejected. Surfaces a missing PR↔issue link rather than a false "no fix" alarm. Non-blocking (Tier 3) | | `needs-human-review` | Only weak evidence; OR multiple candidate PRs with conflicting verdicts | ## CI Freshness diff --git a/.github/skills/release-readiness/scripts/Find-ReleaseReadinessTrackers.ps1 b/.github/skills/release-readiness/scripts/Find-ReleaseReadinessTrackers.ps1 index 6b68ee945524..dd5e4fae2364 100644 --- a/.github/skills/release-readiness/scripts/Find-ReleaseReadinessTrackers.ps1 +++ b/.github/skills/release-readiness/scripts/Find-ReleaseReadinessTrackers.ps1 @@ -18,7 +18,14 @@ does NOT exist on origin, the branch is in-flight — the release notes for that exact patch haven't been published yet, so it hasn't shipped. If the tag exists, the branch has already shipped that - patch and is skipped. + patch. Shipped SRs are normally retired, EXCEPT the most-recently- + shipped one (highest shipped patch), which is emitted as + mode='shipped' so its tracker keeps refreshing through post-ship + follow-up (adding the new build to the GitHub issue version dropdown, + release notes, milestone close-out). The workflow treats 'shipped' + as REFRESH-ONLY: it updates the tracker issue while it stays open but + never re-creates it, so once a human closes it, it stays closed. + Older shipped SRs are skipped. Lane 2 — next SR off main Identifies the highest SR (across in-flight branches AND shipped tags) @@ -538,7 +545,7 @@ function New-Tracker { param( [int]$Major, [int]$SrNumber, - [string]$Mode, # 'in-flight' or 'candidate' + [string]$Mode, # 'in-flight', 'candidate', or 'shipped' [string]$BranchName, # nullable for candidate without branch [string]$SurveyRef, # branch or development ref to survey [string]$PriorSrBranch, # nullable; used as -SrBranch for -Candidate mode @@ -561,6 +568,9 @@ function New-Tracker { if ($Mode -eq 'candidate') { $title = "[Release Readiness] .NET $Major SR$SrNumber — candidate from $SurveyRef" } + elseif ($Mode -eq 'shipped') { + $title = "[Release Readiness] .NET $Major SR$SrNumber — shipped ($branchDisplay)" + } return [pscustomobject]@{ branchType = 'sr' srNumber = $SrNumber @@ -728,7 +738,28 @@ function Invoke-DetectionForMajor { $inflightBranchesBySr[$sr] = $branch Write-Host " -> in-flight SR tracker: SR$sr (patch=$branchPatch, no tag $expectedTag yet, recent=$recent)" -ForegroundColor Green } else { - Write-Host " -> SR$sr branch '$branch' patch=$branchPatch already shipped (tag $expectedTag exists)" -ForegroundColor DarkGray + # Already shipped (the stable tag exists). We normally retire the + # tracker here, BUT the most-recently-shipped SR still has post-ship + # follow-up the tracker should keep surfacing until a human signs off + # (e.g. adding the new build to the GitHub issue version dropdown, + # release notes, closing out the milestone). So keep emitting the + # HIGHEST shipped SR as mode='shipped'. The workflow consumes this as + # REFRESH-ONLY: it refreshes the tracker issue while it stays open and + # never re-creates it once a human closes it — implementing + # "update until closed manually" without resurrecting a closed tracker. + # Lower (older) shipped SRs stay retired. + if ($branchPatch -eq $highestShippedPatch) { + $recent = Get-RecentCommitCount -Ref $branch -Days $ActivityWindowDays + $tracker = New-Tracker -Major $Major -SrNumber $sr -Mode 'shipped' ` + -BranchName $branch -SurveyRef $branch -PriorSrBranch $null ` + -PriorShippedPatch $highestShippedPatch -PriorShippedTag $highestShippedTag ` + -ExpectedPatch $branchPatch -ExpectedTag $expectedTag ` + -HasRecentActivityCount $recent + $trackers.Add($tracker) + Write-Host " -> shipped SR tracker (refresh-until-closed): SR$sr (patch=$branchPatch, tag $expectedTag exists, recent=$recent)" -ForegroundColor Yellow + } else { + Write-Host " -> SR$sr branch '$branch' patch=$branchPatch already shipped (tag $expectedTag exists)" -ForegroundColor DarkGray + } } } diff --git a/.github/skills/release-readiness/scripts/Get-PreviewReadiness.ps1 b/.github/skills/release-readiness/scripts/Get-PreviewReadiness.ps1 index ed276e103f21..96a8b29c0e40 100644 --- a/.github/skills/release-readiness/scripts/Get-PreviewReadiness.ps1 +++ b/.github/skills/release-readiness/scripts/Get-PreviewReadiness.ps1 @@ -156,6 +156,14 @@ if ([string]::IsNullOrWhiteSpace($SurveyRef)) { $SurveyRef = if ($Mode -eq 'candidate') { $mainBranch } else { $Branch } } +# Human-readable label for the daily-flow merge-up CHAIN that feeds the survey +# ref. The preview lane is a two-hop chain: main → net.0 → previewN. An +# in-flight preview surveys previewN, so BOTH hops feed it (main → net.0 → +# previewN). A candidate's survey ref IS net.0, so the chain it sees is the +# single main → net.0 hop. Used for the merge-up check Area, the high-priority +# blurb, and the carve-out list (which must stay string-equal to the check Area). +$mergeUpChainLabel = if ($Mode -eq 'candidate') { "main → $SurveyRef" } else { "main → $mainBranch → $SurveyRef" } + # Canonical tracker key. Default matches Find-Trackers' New-PreviewTracker. if ([string]::IsNullOrWhiteSpace($TrackerKey)) { $TrackerKey = "net$majorVersion-preview$previewNumber" @@ -923,20 +931,33 @@ function Get-CategorizedPullRequests { $maestroPRs = @($allReleasePRs | Where-Object { $_.author -and $_.author.login -match "dotnet-maestro" -and ($p0PrNumbers -notcontains $_.number) }) # Non-P/0, non-Maestro humans, split by scope. - $targetHumanPRsRaw = @($TargetPRs | Where-Object { -not ($_.author -and $_.author.login -match "dotnet-maestro") -and ($p0PrNumbers -notcontains $_.number) }) - $inflightHumanPRs = @($InflightPRs | Where-Object { -not ($_.author -and $_.author.login -match "dotnet-maestro") }) - - # 3. Merge-up: non-P/0, non-Maestro target PRs. MAUI convention: - # - head ref like `merge/main-to-net11.0` or `merge/preview4-to-net11.0` - # - title like "[automated] Merge branch 'main' => 'net11.0'" - $mergeUpPRs = @($targetHumanPRsRaw | Where-Object { + $targetHumanPRsRaw = @($TargetPRs | Where-Object { -not ($_.author -and $_.author.login -match "dotnet-maestro") -and ($p0PrNumbers -notcontains $_.number) }) + $inflightHumanPRsRaw = @($InflightPRs | Where-Object { -not ($_.author -and $_.author.login -match "dotnet-maestro") }) + + # 3. Merge-up: non-P/0, non-Maestro PRs from BOTH hops of the daily-flow chain + # that feeds the survey ref. The preview lane chains main → net.0 → + # previewN, so a stuck merge-up at EITHER hop starves the release of + # upstream fixes: + # - net.0 → previewN (base = survey ref) — from target PRs + # - main → net.0 (base = inflight branch) — from inflight PRs + # Both are hoisted to high priority rather than the second hop being buried + # as generic inflight-queue noise (the #36085 scenario). In candidate mode + # the inflight set is empty, so only the single base=net.0 hop is seen — + # no double counting across modes. + # MAUI convention: + # - head ref like `merge/main-to-net11.0` or `merge/preview4-to-net11.0` + # - title like "[automated] Merge branch 'main' => 'net11.0'" + $allHumanPRs = @($targetHumanPRsRaw) + @($inflightHumanPRsRaw) + $mergeUpPRs = @($allHumanPRs | Where-Object { ($_.headRefName -and $_.headRefName -match '^merge/.+-to-') -or ($_.title -and $_.title -match '^\[automated\] Merge branch') }) $mergeUpPrNumbers = @($mergeUpPRs | ForEach-Object { $_.number }) - # 4. Generic-human (target) = the remainder, counted/listed once. - $targetHumanPRs = @($targetHumanPRsRaw | Where-Object { $mergeUpPrNumbers -notcontains $_.number }) + # 4. Generic-human (target) and inflight-human = the remainders, with merge-up + # PRs removed from BOTH so a hoisted merge-up is never also listed below. + $targetHumanPRs = @($targetHumanPRsRaw | Where-Object { $mergeUpPrNumbers -notcontains $_.number }) + $inflightHumanPRs = @($inflightHumanPRsRaw | Where-Object { $mergeUpPrNumbers -notcontains $_.number }) return [PSCustomObject]@{ P0Prs = $p0Prs @@ -1373,9 +1394,9 @@ if ($p1Issues.Count -gt 0) { } if ($mergeUpPRs.Count -gt 0) { - $checks += New-Check -Area "Merge-up PRs (main → $SurveyRef)" -Status "BLOCKED" -Details "$($mergeUpPRs.Count) open merge-up PR(s). See 🔴 High-priority items at top. Stuck merge-up PRs block daily flow and accumulate conflicts." -NextAction "Resolve and merge each before shipping." + $checks += New-Check -Area "Merge-up PRs ($mergeUpChainLabel)" -Status "BLOCKED" -Details "$($mergeUpPRs.Count) open merge-up PR(s) in the ``$mergeUpChainLabel`` daily-flow chain. See 🔴 High-priority items at top. A stuck merge-up at any hop accumulates conflicts and starves the release of upstream fixes." -NextAction "Resolve and merge each before shipping." } else { - $checks += New-Check -Area "Merge-up PRs (main → $SurveyRef)" -Status "READY" -Details "No open merge-up PRs from ``main`` → ``$SurveyRef``." -NextAction "Continue monitoring." + $checks += New-Check -Area "Merge-up PRs ($mergeUpChainLabel)" -Status "READY" -Details "No open merge-up PRs in the ``$mergeUpChainLabel`` daily-flow chain." -NextAction "Continue monitoring." } if ($kbeIssues.Count -gt 0) { @@ -1579,10 +1600,10 @@ if ($nightlyFeedBanner) { # de-prioritized) before shipping. # 3. Maestro dependency-flow PRs — open Maestro PRs against the survey ref. # A stuck Maestro PR blocks all upstream dependency flow into this branch. -# 4. Merge-up PRs (main → survey ref) — daily-flow sync PRs whose head ref +# 4. Merge-up PRs (upstream → survey ref) — daily-flow sync PRs whose head ref # matches `merge/...-to-...` or title starts with "[automated] Merge branch". # A stuck merge-up PR accumulates conflicts and starves the release branch -# of new fixes from main. +# of new fixes from upstream (main on the SR lane, net11.0 on the preview lane). # Each item is itemized (one row per issue/PR) so the captain can see exactly # what's outstanding without drilling into the per-category PR tables below. $highPriorityRows = New-Object System.Collections.Generic.List[hashtable] @@ -1617,8 +1638,19 @@ foreach ($pr in $maestroPRs) { } foreach ($pr in $mergeUpPRs) { $action = Get-PRAction -PR $pr + # Name the specific hop from the PR's base: a survey-ref-based merge-up is the + # net.0 → previewN hop; an inflight-based one (base = net.0) is the + # main → net.0 hop. In candidate mode the survey ref IS net.0, so a + # base=net.0 PR is the main → net.0 hop (guarded by the candidate check). + $leg = if ($Mode -ne 'candidate' -and $pr.baseRefName -eq $SurveyRef) { + "$mainBranch → $SurveyRef" + } elseif ($pr.baseRefName -eq $mainBranch) { + "main → $mainBranch" + } else { + "merge-up" + } [void]$highPriorityRows.Add(@{ - kind = "🔀 Merge-up PR (main → $SurveyRef)" + kind = "🔀 Merge-up PR ($leg)" link = "[#$($pr.number)]($($pr.url))" title = $pr.title actor = "base ``$($pr.baseRefName)``, $($action.Age)d old" @@ -1629,7 +1661,7 @@ foreach ($pr in $mergeUpPRs) { if ($highPriorityRows.Count -gt 0) { [void]$md.AppendLine("## 🔴 High-priority items — $($highPriorityRows.Count) item(s)") [void]$md.AppendLine("") - [void]$md.AppendLine("_P/0 issues, P/0 PRs, Maestro PRs, and ``main`` → ``$SurveyRef`` merge-up PRs. Resolve these before treating the release as ready._") + [void]$md.AppendLine("_P/0 issues, P/0 PRs, Maestro PRs, and merge-up PRs in the ``$mergeUpChainLabel`` daily-flow chain. Resolve these before treating the release as ready._") [void]$md.AppendLine("") [void]$md.AppendLine("| Kind | Item | Title | Context | Next action |") [void]$md.AppendLine("|------|------|-------|---------|-------------|") @@ -1649,7 +1681,7 @@ $highPriorityCheckAreas = @( 'P/0 priority blockers', 'P/0 release-branch PRs', 'Maestro PRs', - "Merge-up PRs (main → $SurveyRef)" + "Merge-up PRs ($mergeUpChainLabel)" ) $blockingChecks = @($checks | Where-Object { $_.Status -eq 'BLOCKED' -and -not ($highPriorityCheckAreas -contains $_.Area) diff --git a/.github/skills/release-readiness/scripts/Get-ReleaseReadiness.ps1 b/.github/skills/release-readiness/scripts/Get-ReleaseReadiness.ps1 index 9b6d12649209..74edbbc9d069 100644 --- a/.github/skills/release-readiness/scripts/Get-ReleaseReadiness.ps1 +++ b/.github/skills/release-readiness/scripts/Get-ReleaseReadiness.ps1 @@ -133,6 +133,14 @@ param( # from main today. Requires -SrBranch to be the prior SR (used as the # exclude baseline). Treats origin/main as the "SR-to-be". [switch]$Candidate, + # Display-only: mark the survey as a SHIPPED SR (its stable tag already + # exists). Behaves exactly like in-flight for all survey/verdict logic + # (surveys the SR branch directly) — it ONLY relabels the rendered report + # header `mode=shipped` so a post-ship tracker doesn't misreport as + # in-flight. Set by the workflow for the most-recently-shipped SR, whose + # tracker keeps refreshing until a human closes it. Mutually exclusive with + # -Candidate. + [switch]$Shipped, # When set in -Candidate mode, model the dotnet/maui workflow where, after # cutting SRn+1 from main, the prior SR (-SrBranch) is merged in. The # candidate's "what's shipping" set = main-since-priorSR ∪ priorSR-only commits. @@ -1322,12 +1330,16 @@ function Get-CandidatePrChecks { function Resolve-Context { param([string]$SrBranch, [string]$Repo, [string]$MainBranch, [string[]]$ExcludeBranches, [switch]$NoFetch, [switch]$Candidate, - [switch]$InheritFromPriorSr) + [switch]$InheritFromPriorSr, [switch]$Shipped) if ($InheritFromPriorSr -and -not $Candidate) { throw "-InheritFromPriorSr is only valid with -Candidate (it models the SR cut-then-merge workflow)." } + if ($Shipped -and $Candidate) { + throw "-Shipped and -Candidate are mutually exclusive: -Shipped surveys an existing (already-tagged) SR branch; -Candidate pre-flights main as the next SR." + } + # HARD VALIDATION — refuse inflight/staging refs as SR sources. # See $Script:ForbiddenSrPatterns at top of file for the rule rationale. foreach ($pat in $Script:ForbiddenSrPatterns) { @@ -1375,6 +1387,13 @@ function Resolve-Context { $effectiveExcludes = @($priorSrRef) Write-Host "Candidate mode: surveying $effectiveSrRef vs prior SR $priorSrRef" -ForegroundColor Cyan } + elseif ($Shipped) { + # Display-only relabel. The survey is identical to in-flight (the SR + # branch surveyed directly); only the rendered header reads 'shipped' + # so the post-ship tracker doesn't misreport as in-flight. + $mode = 'shipped' + Write-Host "Shipped mode: surveying already-tagged SR branch $effectiveSrRef (display-only relabel)" -ForegroundColor Cyan + } if ($Candidate -and $InheritFromPriorSr) { Write-Host " -InheritFromPriorSr active: SR-to-be contents will be augmented with $priorSrRef-only commits" -ForegroundColor Cyan @@ -1448,6 +1467,21 @@ function Get-RevertedPrFromSubject { # Case-insensitive to also match hand-typed lowercase 'revert "..."' subjects. $m = [regex]::Match($Subject, '(?i)Revert\s+".*\(#(\d+)\)"') if ($m.Success) { return [int]$m.Groups[1].Value } + # Manual/hand-authored revert form (no GitHub quotes, and the body carries no + # "This reverts commit " line, so the SHA-override in the caller can't + # recover it either): "Revert - # (#)". + # Real example: `Revert - Fix Android stale ContainerView root leak #35372 (#36152)` + # → 35372 is the reverted fix, (#36152) is the revert's OWN squash PR. Without + # this, revertedPrSet records only 36152, the reverted fix's original commit + # still satisfies Test-PrNumberOnBranch, and a "fixed by #35372" comment on a + # CLOSED issue is falsely de-noised to closed-fix-unlinked ("No ship risk"). + # Anchored to a Revert subject AND to the bare `#N` immediately before the + # trailing `(#M)` so (a) non-revert subjects never match, (b) the trailing + # (#M) is never returned, and (c) incidental #refs earlier in the title are + # ignored. A rare misfire only ever marks a PR as reverted (safe direction — + # never a false "shipped"). + $m = [regex]::Match($Subject, '(?i)^(?:\[[^\]]+\]\s+)?Revert\b.*#(\d+)\s+\(#\d+\)\s*$') + if ($m.Success) { return [int]$m.Groups[1].Value } return $null } @@ -1836,6 +1870,76 @@ function Get-IssueTimelinePrs { return @($prs | Sort-Object -Unique) } +function Get-IssueCommentPrs { + <# + .SYNOPSIS + Extract PR numbers referenced in an issue's COMMENT BODIES (as opposed + to the structured timeline). Recovers fix linkage that lives ONLY in + human prose. + + .DESCRIPTION + Common QA close pattern in this repo: a maintainer closes a regression + with a comment like "This issue was fixed by PR #35028" but the fix PR + never used a closing keyword (`Fixes #NNNNN`) and GitHub therefore + recorded NO `cross-referenced` event on the ISSUE timeline. `Get-IssueTimelinePrs` + sees nothing, the classifier finds zero candidates, and the issue is + mislabelled `no-fix-yet` even though the fix shipped. This helper reads + the comment bodies so that linkage can be recovered. + + Returns @( @{ number = ; evidence = 'fix-phrase' | 'mention' } ). + `evidence` is 'fix-phrase' when the comment pairs the PR reference with + fix/resolve/close language (higher confidence), else 'mention'. + + IMPORTANT: this only surfaces CANDIDATES. Callers MUST still verify each + PR actually MERGED and that its commit is on the target branch + (`Test-CommitOnBranch`) before trusting it as a real fix — a bare prose + mention ("duplicate of #X", "see #Y") is not proof of anything. + #> + param($Repo, $IssueNumber) + $raw = Invoke-Gh @('api', "repos/$Repo/issues/$IssueNumber/comments", '--paginate') + if (-not $raw) { return @() } + $comments = $raw | ConvertFrom-Json -ErrorAction SilentlyContinue + if (-not $comments) { return @() } + + # strongest evidence seen per PR number ('fix-phrase' beats 'mention') + $byNum = @{} + foreach ($c in $comments) { + $body = Get-AzdoProp $c 'body' + if (-not $body) { continue } + # Extract PR references, rejecting CROSS-REPO ones. A maui regression is + # only de-noised by a fix that lives in THIS repo, so a cross-repo + # shorthand (`dotnet/runtime#123`), a github.com///pull/123 + # URL, or a scheme-less //pull/123 path must NOT be mistaken + # for maui#123. Same-repo shorthand (`dotnet/maui#123`), same-repo pull + # URLs/paths, bare `#123` and `PR#123` are all accepted. The + # `qual`/`urlrepo`/`pathrepo` groups capture any owner/repo qualifier so a + # foreign one can be skipped. + $refs = [regex]::Matches($body, '(?:(?[A-Za-z0-9._-]+/[A-Za-z0-9._-]+)#|github\.com/(?[A-Za-z0-9._-]+/[A-Za-z0-9._-]+)/pull/|(?[A-Za-z0-9._-]+/[A-Za-z0-9._-]+)/pull/|pull/|#)(\d+)') + foreach ($m in $refs) { + $qual = $m.Groups['qual'].Value + $urlRepo = $m.Groups['urlrepo'].Value + $pathRepo = $m.Groups['pathrepo'].Value + if ($qual -and $qual -ne $Repo) { continue } # cross-repo owner/repo#N shorthand + if ($urlRepo -and $urlRepo -ne $Repo) { continue } # cross-repo github.com/.../pull/N URL + if ($pathRepo -and $pathRepo -ne $Repo) { continue } # cross-repo scheme-less owner/repo/pull/N + $num = [int]$m.Groups[1].Value + # Does THIS comment pair the reference with fix/resolve/close language + # within a short window (tolerates the long ".../pull/" URL prefix)? + # The negative lookbehind drops ADJACENTLY-negated fix phrases ("not + # fixed by #X", "won't fix #Y", "isn't resolved by #Z") so they score as + # a bare 'mention', not high-confidence 'fix-phrase'. Because -match + # backtracks, a separate non-negated fix phrase for the same PR still + # matches; only a SOLELY-(adjacently-)negated reference is demoted. A + # non-adjacent negation ("won't be fixed by #X") is not caught here, but + # the caller's merged-AND-on-branch gates still bound the blast radius. + $isFix = $body -match "(?i)(?)` token GitHub stamps onto squash/merge subjects. + + .DESCRIPTION + Robust companion to Test-CommitOnBranch for the cross-branch flow. + A PR's own `mergeCommit.oid` is the SHA on the branch it merged INTO + (e.g. inflight/candidate). When that change later reaches the SR branch + via a branch-merge or cherry-pick it gets a DIFFERENT SHA, so a bare + `merge-base --is-ancestor ` returns false even + though the fix IS present. The `(#)` subject token survives all + three flows (squash, branch-merge, cherry-pick -x), so matching on it + recovers presence that SHA-ancestry misses. `--fixed-strings` keeps the + closing paren literal, preventing `(#3502)` from matching `(#35028)`. + #> + param([int]$PrNumber, [string]$BranchRef) + if (-not $PrNumber) { return $false } + $hit = Invoke-Git "log $BranchRef --fixed-strings --grep=(#$PrNumber) --format=%H -1" + return [bool]$hit +} + function Get-BackportPrsForSr { param($Repo, $SrBranch, $SourcePrNumber) # Look for any PR targeting the SR branch that mentions the source PR @@ -2041,6 +2168,78 @@ function Classify-RegressionCandidate { recommendedAction = "Inspect the revert chain manually: original fix → revert → (possible) revert-of-revert. Look for the actual fix PR in `gh pr list --search 'fixes #$($Issue.number)'` excluding revert titles." } } + + # ── FALLBACK: closed issue whose fix lives ONLY in comment prose ── + # No timeline-cross-referenced candidate survived the evidence filter, + # but the issue is CLOSED. Maintainers routinely close a regression with + # a plain-text comment ("fixed by PR #35028") that GitHub never turns + # into a structured link. Recover the cited PR and — ONLY when it + # actually MERGED and its commit is verifiably on THIS SR branch — + # classify 'closed-fix-unlinked': the fix is present (no ship risk), but + # the issue<->PR link is missing and should be added for traceability. + # + # Two gates make prose evidence safe, and BOTH are required: + # 1. fix-phrase ONLY — the comment must pair the PR with fix/resolve/ + # close language ("was fixed by PR #X"). A BARE mention is rejected + # because regression issues routinely name the CAUSE PR for context + # ("Before PR #32080 ... After PR #32080 the behavior changed"), and + # the cause naturally lives on the branch — it is not a fix. + # 2. merged AND on the SR branch — a cited PR that never merged, or + # merged elsewhere, is not proof the fix shipped here. + $issueState = Get-AzdoProp $Issue 'state' + if ($issueState -eq 'CLOSED') { + $commentPrs = Get-IssueCommentPrs -Repo $Ctx.repo -IssueNumber $Issue.number + $verifiedFixes = @() + foreach ($cp in $commentPrs) { + # Gate 1: require explicit fix language. Bare mentions (the cause-PR + # blame pattern) are NOT fixes and must not reclassify the issue. + if ($cp.evidence -ne 'fix-phrase') { continue } + if ($cp.number -eq $Issue.number) { continue } # self-reference + $info = Get-PrInfo -Repo $Ctx.repo -PrNumber $cp.number + if (-not $info) { continue } + if ($info.state -ne 'MERGED') { continue } + # A reverted fix is NOT a fix. Mirror the main SR-contents/candidate + # paths (see $revertedPrSet at the top of this function and the + # Revert-title skip in the candidate walk): drop PRs the SR later + # reverted, and drop PRs that are themselves rollbacks ("Revert ..." + # titles). Without this, the `(#)` on-branch token checked below + # matches the reverted fix's number inside the revert commit's own + # subject `Revert "... (#num)" (#N)`, so a rolled-back fix would pass + # the on-branch gate and be reported as "No ship risk". + if ($revertedPrSet.ContainsKey([int]$info.number)) { continue } + if (($info.title -match '(?i)^(?:\[[^\]]+\]\s+)?Revert\b') -or ($info.title -match '\[Revert\]')) { continue } + # Skip agent/skill/workflow PRs that only mention the issue for context. + if (Test-PrIsToolingOnly -Files $info.files) { continue } + $mergeSha = if ($info.mergeCommit) { $info.mergeCommit.oid } else { $null } + # Gate 2: presence on the SR branch via EITHER signal — direct SHA + # ancestry (fix merged straight to SR) OR the `(#)` subject + # token (fix flowed in from inflight/main under a different SHA — + # the common case). + $onSr = (Test-CommitOnBranch -Sha $mergeSha -BranchRef "origin/$($Ctx.srBranch)") ` + -or (Test-PrNumberOnBranch -PrNumber ([int]$info.number) -BranchRef "origin/$($Ctx.srBranch)") + if (-not $onSr) { continue } + $verifiedFixes += @{ + number = [int]$info.number + title = $info.title + state = $info.state + mergeSha = $mergeSha + evidenceType = "comment-$($cp.evidence)" + } + } + if ($verifiedFixes.Count -gt 0) { + $prList = (@($verifiedFixes | ForEach-Object { "#$($_.number)" }) | Sort-Object -Unique) -join ', ' + return @{ + classification = 'closed-fix-unlinked' + confidence = 'high' + evidence = @("Issue is CLOSED and fix PR $prList is MERGED and present on $($Ctx.srBranch), but was never linked to the issue (no closing keyword, no timeline cross-reference). Linkage recovered from a closing comment that explicitly names the fix.") + candidateFixPrs = @($verifiedFixes | ForEach-Object { + @{ number = $_.number; title = $_.title; state = $_.state; evidenceType = $_.evidenceType } + }) + recommendedAction = "No ship risk — fix is already in the SR. Add a closing reference for traceability (e.g. ``Fixes #$($Issue.number)`` in $prList, or link via the issue's Development panel) so future runs classify it automatically." + } + } + } + return @{ classification = 'no-fix-yet' confidence = 'high' @@ -2704,6 +2903,7 @@ function Get-VerdictTier { 'needs-human-review' { 2; break } 'in-sr-active' { 3; break } 'closed-as-duplicate' { 3; break } + 'closed-fix-unlinked' { 3; break } 'out-of-scope-future-sr' { 3; break } default { 2 } # unknown → treat as risk } @@ -3035,6 +3235,19 @@ function Get-ReportSemanticHash { # process against one computed now). Insertion order keeps the hash stable. $semantic = [ordered]@{ verdict = $Verdict.symbol + # Tracker lifecycle mode (candidate / in-flight / shipped). Folded in so a + # lifecycle TRANSITION always flips the hash and refreshes the tracker — + # even when every other hashed field is byte-for-byte identical across the + # flip. This matters most at in-flight -> shipped: `-Shipped` is a pure + # display relabel that surveys the SAME SR branch as in-flight, so srHead, + # ci, srPrs, regressions, shipChecks and nightlyFeed can all be unchanged + # at the moment the stable tag publishes. Without `mode` here, hash(shipped) + # == hash(in-flight), the workflow's idempotent no-op skips `gh issue edit`, + # and the tracker never visually flips to "shipped" (the whole point of the + # shipped lifecycle). `mode` is constant within a mode, so it adds NO daily + # churn — only the one-time transition refreshes. Default 'in-flight' when + # absent, matching Format-MarkdownReport's $mode default. + mode = if ($Data.metadata -and $Data.metadata.ContainsKey('mode') -and $Data.metadata['mode']) { $Data.metadata['mode'] } else { 'in-flight' } srHead = $Data.metadata.srHeadSha ciOverall = if ($Data.ContainsKey('ci') -and $Data['ci']) { $Data['ci'].overall } else { $null } srPrs = if ($Data.ContainsKey('srContents') -and $Data['srContents']) { @@ -3648,7 +3861,7 @@ function Format-MarkdownReport { $tier1Classes = @('in-sr-reverted', 'no-fix-yet') | Sort-Object $tier2Classes = @('rejected-from-sr', 'backport-in-progress', 'merged-on-main-no-backport', 'merged-non-main-only', 'open-on-main', 'needs-human-review') | Sort-Object - $tier3Classes = @('in-sr-active', 'closed-as-duplicate', 'no-fix-yet', 'out-of-scope-future-sr') | Sort-Object + $tier3Classes = @('in-sr-active', 'closed-as-duplicate', 'closed-fix-unlinked', 'no-fix-yet', 'out-of-scope-future-sr') | Sort-Object $emitTier = { param([string]$Header, [string[]]$Classes, [string]$EmptyLine, [string]$NoFixYetState) @@ -3844,7 +4057,7 @@ function Invoke-Main { $excludes = $ExcludeBranches -split ',' | ForEach-Object { $_.Trim() } | Where-Object { $_ } $ctx = Resolve-Context -SrBranch $SrBranch -Repo $Repo -MainBranch $MainBranch ` -ExcludeBranches $excludes -NoFetch:$NoFetch -Candidate:$Candidate ` - -InheritFromPriorSr:$InheritFromPriorSr + -InheritFromPriorSr:$InheritFromPriorSr -Shipped:$Shipped # Resolve regression labels $labelMode = 'explicit' diff --git a/.github/skills/release-readiness/tests/Test-ReleaseReadiness.ps1 b/.github/skills/release-readiness/tests/Test-ReleaseReadiness.ps1 index 667ebc60bd61..e279e11e85a1 100644 --- a/.github/skills/release-readiness/tests/Test-ReleaseReadiness.ps1 +++ b/.github/skills/release-readiness/tests/Test-ReleaseReadiness.ps1 @@ -429,6 +429,42 @@ if (-not (Test-Path $detectScriptPath)) { Assert-Eq -Label "no shipped tags (highest 0): patch 11 idle -> NOT stale" ` -Expected $false -Actual (Test-IsStaleSrBranch -BranchPatch 11 -HighestShippedPatch 0 -RecentActivityCount 0) + # ─────────── New-Tracker title + mode contract ─────────── + # The issue title encodes the tracker's lifecycle mode. Marker matching is by + # canonicalKey (not title), so title text is safe to vary, but downstream the + # 'shipped' title signals the refresh-until-closed lifecycle. Pin all three. + Write-Host "`n[Unit] New-Tracker title + mode" -ForegroundColor Cyan + $tkInflight = New-Tracker -Major 10 -SrNumber 8 -Mode 'in-flight' ` + -BranchName 'release/10.0.1xx-sr8' -SurveyRef 'release/10.0.1xx-sr8' -PriorSrBranch $null ` + -PriorShippedPatch 71 -PriorShippedTag '10.0.71' -ExpectedPatch 80 -ExpectedTag '10.0.80' ` + -HasRecentActivityCount 3 + Assert-Eq -Label "in-flight mode preserved" -Expected 'in-flight' -Actual $tkInflight.mode + Assert-Eq -Label "in-flight title = branch form" ` + -Expected '[Release Readiness] .NET 10 SR8 — release/10.0.1xx-sr8' -Actual $tkInflight.issueTitle + + $tkCandidate = New-Tracker -Major 10 -SrNumber 9 -Mode 'candidate' ` + -BranchName $null -SurveyRef 'main' -PriorSrBranch 'release/10.0.1xx-sr8' ` + -PriorShippedPatch 80 -PriorShippedTag '10.0.80' -ExpectedPatch 90 -ExpectedTag '10.0.90' ` + -HasRecentActivityCount 12 + Assert-Eq -Label "candidate mode preserved" -Expected 'candidate' -Actual $tkCandidate.mode + Assert-Eq -Label "candidate title = candidate-from form" ` + -Expected '[Release Readiness] .NET 10 SR9 — candidate from main' -Actual $tkCandidate.issueTitle + Assert-Eq -Label "candidate canonicalKey is title-independent" -Expected 'net10-sr9' -Actual $tkCandidate.canonicalKey + + $tkShipped = New-Tracker -Major 10 -SrNumber 8 -Mode 'shipped' ` + -BranchName 'release/10.0.1xx-sr8' -SurveyRef 'release/10.0.1xx-sr8' -PriorSrBranch $null ` + -PriorShippedPatch 80 -PriorShippedTag '10.0.80' -ExpectedPatch 80 -ExpectedTag '10.0.80' ` + -HasRecentActivityCount 1 + Assert-Eq -Label "shipped mode preserved" -Expected 'shipped' -Actual $tkShipped.mode + Assert-Eq -Label "shipped title = shipped form (signals refresh-until-closed)" ` + -Expected '[Release Readiness] .NET 10 SR8 — shipped (release/10.0.1xx-sr8)' -Actual $tkShipped.issueTitle + Assert-Eq -Label "shipped surveys its own branch (not a candidate ref)" ` + -Expected 'release/10.0.1xx-sr8' -Actual $tkShipped.surveyRef + Assert-Eq -Label "shipped canonicalKey matches in-flight (stable join key across lifecycle)" ` + -Expected 'net10-sr8' -Actual $tkShipped.canonicalKey + Assert-Eq -Label "shipped branchExists = true (the SR branch is real)" ` + -Expected $true -Actual $tkShipped.branchExists + # ─────────── Preview-tag regex contract ─────────── Write-Host "`n[Unit] Preview tag regex (.0.0-preview..[.])" -ForegroundColor Cyan $previewTagCases = @( @@ -637,13 +673,14 @@ function Get-ExpectedHighestShippedTag { if (-not $SkipE2E) { Write-Host "`n[E2E] Detection against live repo" -ForegroundColor Cyan - Write-Host " Under the tag-existence rule + Lane 1 staleness guard we expect ONE tracker:" -ForegroundColor DarkGray + Write-Host " Under the tag-existence rule + Lane 1 staleness guard we expect TWO trackers:" -ForegroundColor DarkGray + Write-Host " - SR8 (patch=80, tag 10.0.80 exists) - shipped, refresh-until-closed (highest shipped SR)" -ForegroundColor DarkGray Write-Host " - SR9 (candidate off main) - active" -ForegroundColor DarkGray Write-Host " DROPPED by the staleness guard (idle + below the shipped watermark 80):" -ForegroundColor DarkGray Write-Host " - SR2 (patch=21, no tag 10.0.21) - tag-absent but stale -> no matrix job" -ForegroundColor DarkGray Write-Host " - SR3 (patch=33, no tag 10.0.33) - tag-absent but stale -> no matrix job" -ForegroundColor DarkGray - Write-Host " NOTE: SR7 shipped 2026-06-05 (tag 10.0.71) and SR8 shipped 2026-07-03 (tag 10.0.80);" -ForegroundColor DarkGray - Write-Host " neither produces a tracker any longer." -ForegroundColor DarkGray + Write-Host " NOTE: SR7 shipped 2026-06-05 (tag 10.0.71); NOT the highest shipped SR -> retired (no tracker)." -ForegroundColor DarkGray + Write-Host " NOTE: SR8 shipped 2026-06-22 (tag 10.0.80); as the HIGHEST shipped SR it emits a 'shipped' tracker that refreshes until a human closes it." -ForegroundColor DarkGray $detectOut = Join-Path ([System.IO.Path]::GetTempPath()) "rr-detect-$(Get-Date -Format 'HHmmss').json" try { @@ -661,8 +698,8 @@ if (-not $SkipE2E) { -Expected $expectedHighestShipped -Actual $detected.highestShippedTag Assert-Eq -Label "highestShippedPreviewTag carries net10's last preview" ` -Expected '10.0.0-preview.7.25406.3' -Actual $detected.highestShippedPreviewTag - Assert-Eq -Label "tracker count is 1 (SR9 only — SR7+SR8 shipped; SR2/SR3 dropped as stale)" ` - -Expected 1 -Actual $detected.trackers.Count + Assert-Eq -Label "tracker count is 2 (SR8 shipped+refresh, SR9 candidate; SR7 retired; SR2/SR3 stale-dropped)" ` + -Expected 2 -Actual $detected.trackers.Count # All trackers in single-major net10 mode must be SR-flavored. (Net10's # previews 1–7 all shipped + no in-flight preview branch -> no preview tracker.) foreach ($t in $detected.trackers) { @@ -686,20 +723,34 @@ if (-not $SkipE2E) { Assert-Eq -Label "SR7 tracker absent (shipped as 10.0.71)" ` -Expected $false -Actual ($bySr.ContainsKey(7)) - # SR8 (shipped 2026-07-03 as 10.0.80 — its in-flight tracker retired on ship, - # exactly like SR7 before it). This transition is why the tracker set dropped - # from {SR8,SR9} to {SR9}. - Assert-Eq -Label "SR8 tracker absent (shipped as 10.0.80)" ` - -Expected $false -Actual ($bySr.ContainsKey(8)) - - # Drift-proof invariant behind the two lines above: ANY SR that surfaces as an - # active tracker must NOT already have its ship tag in git (a shipped SR is - # excluded), no matter how many ship. Uses the tracker's own expectedTag so the - # non-uniform patch convention (SR7 shipped as 10.0.71, SR8 as 10.0.80) is honored. - foreach ($active in @($detected.trackers | Where-Object branchType -eq 'sr')) { - Assert-Eq -Label "active SR$($active.srNumber) has no ship tag $($active.expectedTag) yet [shipped-exclusion invariant]" ` - -Expected $false ` - -Actual ([bool](& git -C $detected.repo tag --list $active.expectedTag)) + # SR8 (SHIPPED — highest shipped SR (tag 10.0.80) -> emits a + # 'shipped' tracker that refreshes until a human closes it). + if ($bySr.ContainsKey(8)) { + $sr8 = $bySr[8] + Assert-Eq -Label "SR8 mode = shipped" -Expected 'shipped' -Actual $sr8.mode + Assert-Eq -Label "SR8 canonicalKey" -Expected 'net10-sr8' -Actual $sr8.canonicalKey + Assert-Eq -Label "SR8 branchName" -Expected 'release/10.0.1xx-sr8' -Actual $sr8.branchName + Assert-Eq -Label "SR8 branchExists = true" -Expected $true -Actual $sr8.branchExists + Assert-Eq -Label "SR8 surveyRef = its own branch (shipped surveys the SR branch)" ` + -Expected 'release/10.0.1xx-sr8' -Actual $sr8.surveyRef + Assert-Eq -Label "SR8 issue title = shipped format" ` + -Expected '[Release Readiness] .NET 10 SR8 — shipped (release/10.0.1xx-sr8)' ` + -Actual $sr8.issueTitle + Assert-Eq -Label "SR8 expectedTag = 10.0.80" -Expected '10.0.80' -Actual $sr8.expectedTag + # hasRecentActivity is a 7-day-window signal (git log --since=7.days + # against the live branch), so its VALUE is wall-clock dependent and + # MUST NOT be pinned here — SR8 idling >7 days at the tail of a cycle + # is a NORMAL state that would (correctly) report $false. Assert only + # that the detector emits it as a real [bool]. The window math itself + # is covered deterministically by the synthetic-fixture unit test + # ([Unit] Get-RecentCommitCount recency window). + Assert-Eq -Label "SR8 hasRecentActivity is a [bool] (value is date-dependent)" ` + -Expected $true -Actual ($sr8.hasRecentActivity -is [bool]) + Assert-Eq -Label "SR8 regression labels" ` + -Expected 'regressed-in-10.0.70,regressed-in-10.0.80' ` + -Actual ($sr8.regressionLabels -join ',') + } else { + Write-Host " ❌ SR8 tracker missing" -ForegroundColor Red; $script:failed++ } # SR9 (candidate from main, ACTIVE) @@ -730,9 +781,10 @@ if (-not $SkipE2E) { # for "active": an active SR can legitimately sit idle for >7 days near # the tail of a cycle and report hasRecentActivity=$false. So assert the # flag is a real [bool] — never a hardcoded, date-dependent $true. SR7 - # shipped 2026-06-05 and SR8 shipped 2026-07-03; both left the tracker set, - # so only SR9 is active. - foreach ($srNum in @(9)) { + # shipped 2026-06-05 and (no longer the highest shipped SR) is retired; + # SR8 (shipped, highest -> refresh-until-closed) + SR9 (candidate) are + # the tracked set. + foreach ($srNum in @(8, 9)) { if ($bySr.ContainsKey($srNum)) { Assert-Eq -Label "SR$srNum hasRecentActivity is a [bool] (active SR; value date-dependent)" ` -Expected $true -Actual ($bySr[$srNum].hasRecentActivity -is [bool]) @@ -752,17 +804,22 @@ if (-not $SkipE2E) { # ──────────── E2E: -AllActiveMajors multi-major envelope ──────────── # In the unified post-consolidation shape, one invocation must surface every # active major (main's + any net.0 ≥ main). Expected current state: - # - net10 -> 1 SR tracker (SR9 candidate), no preview tracker - # (SR7 shipped 2026-06-05 + SR8 shipped 2026-07-03; SR2/SR3 dropped by the Lane 1 - # staleness guard; every net10 preview branch already shipped + net10.0 isn't in - # preview cycle) + # - net10 -> 2 SR trackers (SR8 shipped+refresh, SR9 candidate), no preview tracker + # (SR7 shipped 2026-06-05 and is no longer the highest shipped SR -> retired; + # SR8 shipped 2026-06-22 as the HIGHEST shipped SR -> 'shipped' tracker that + # refreshes until closed; SR2/SR3 dropped by the Lane 1 staleness guard; + # every net10 preview branch already shipped + net10.0 isn't in preview cycle) # - net11 -> 0 SR trackers (pre-GA: no `11.0.0` tag), 2 preview trackers - # (preview6 in-flight from release/11.0.1xx-preview6; preview7 candidate from net11.0) + # (preview6 in-flight: release/11.0.1xx-preview6 was cut, tag not yet published; + # preview7 candidate: net11.0 bumped to PreReleaseVersionIteration=7, so the + # detector now surfaces a candidate for the NEXT preview alongside in-flight + # preview6. Once preview7's branch is cut this becomes preview7 in-flight + + # preview8 candidate — update this snapshot then, same as the SR lane above.) Write-Host "`n[E2E] Detection with -AllActiveMajors" -ForegroundColor Cyan Write-Host " Expected:" -ForegroundColor DarkGray Write-Host " - majors[].length = 2 (net10 + net11)" -ForegroundColor DarkGray - Write-Host " - net10 trackers: 1 SR (sr9), 0 preview (SR7+SR8 shipped; SR2/SR3 stale-dropped)" -ForegroundColor DarkGray - Write-Host " - net11 trackers: 0 SR (pre-GA), 2 preview (preview6 in-flight, preview7 candidate)" -ForegroundColor DarkGray + Write-Host " - net10 trackers: 2 SR (sr8 shipped/sr9 candidate), 0 preview (SR7 retired; SR2/SR3 stale-dropped)" -ForegroundColor DarkGray + Write-Host " - net11 trackers: 0 SR (pre-GA), 2 preview (preview6 in-flight + preview7 candidate from net11.0)" -ForegroundColor DarkGray $multiOut = Join-Path ([System.IO.Path]::GetTempPath()) "rr-detect-allmajors-$(Get-Date -Format 'HHmmss').json" try { @@ -788,10 +845,10 @@ if (-not $SkipE2E) { $expectedNet10Highest = Get-ExpectedHighestShippedTag -RepoPath $multi.repo -Major 10 Assert-Eq -Label "net10 highestShippedTag matches highest stable 10.0.M tag (derived)" ` -Expected $expectedNet10Highest -Actual $net10.highestShippedTag - Assert-Eq -Label "net10 tracker count is 1 (no preview lane, SR7+SR8 shipped, SR2/SR3 stale-dropped)" -Expected 1 -Actual $net10.trackers.Count + Assert-Eq -Label "net10 tracker count is 2 (no preview lane; SR8 shipped+refresh, SR9 candidate; SR7 retired, SR2/SR3 stale-dropped)" -Expected 2 -Actual $net10.trackers.Count $srCount = @($net10.trackers | Where-Object branchType -eq 'sr').Count $previewCount = @($net10.trackers | Where-Object branchType -eq 'preview').Count - Assert-Eq -Label "net10 has 1 SR tracker" -Expected 1 -Actual $srCount + Assert-Eq -Label "net10 has 2 SR trackers" -Expected 2 -Actual $srCount Assert-Eq -Label "net10 has 0 preview trackers" -Expected 0 -Actual $previewCount } else { Write-Host " ❌ majors[] missing net10 entry" -ForegroundColor Red; $script:failed++ @@ -809,68 +866,99 @@ if (-not $SkipE2E) { -Expected '11.0.0-preview.5.26304.4' -Actual $net11.highestShippedPreviewTag Assert-Eq -Label "net11 tracker count is 2 (preview6 in-flight + preview7 candidate)" -Expected 2 -Actual $net11.trackers.Count $previewTrackers = @($net11.trackers | Where-Object branchType -eq 'preview') - Assert-Eq -Label "net11 has 2 preview trackers" -Expected 2 -Actual $previewTrackers.Count + Assert-Eq -Label "net11 has 2 preview trackers" -Expected 2 -Actual $previewTrackers.Count $srTrackers = @($net11.trackers | Where-Object branchType -eq 'sr') Assert-Eq -Label "net11 has 0 SR trackers (pre-GA -> Lane 2 skipped)" -Expected 0 -Actual $srTrackers.Count - # Select by previewNumber rather than array index — the tracker order is an - # implementation detail and must not silently swap the two assertions. - $preview6 = $previewTrackers | Where-Object previewNumber -eq 6 | Select-Object -First 1 - $preview7 = $previewTrackers | Where-Object previewNumber -eq 7 | Select-Object -First 1 - - # preview6 — IN-FLIGHT off its cut branch. - if ($preview6) { - Assert-Eq -Label "preview6 canonicalKey" -Expected 'net11-preview6' -Actual $preview6.canonicalKey - Assert-Eq -Label "preview6 mode = in-flight" -Expected 'in-flight' -Actual $preview6.mode - Assert-Eq -Label "preview6 surveyRef = its cut branch" -Expected 'release/11.0.1xx-preview6' -Actual $preview6.surveyRef - Assert-Eq -Label "preview6 expectedTagPrefix" -Expected '11.0.0-preview.6.' -Actual $preview6.expectedTagPrefix - Assert-Eq -Label "preview6 previewNumber = 6" -Expected 6 -Actual $preview6.previewNumber - Assert-Eq -Label "preview6 milestone name" -Expected '.NET 11.0-preview6' -Actual $preview6.milestoneName - Assert-Eq -Label "preview6 issue title format" ` - -Expected '[Release Readiness] .NET 11.0 preview6 — release/11.0.1xx-preview6' ` + # Select each preview by its number rather than array position so the + # assertions don't hinge on detector ordering. + $preview6 = $previewTrackers | Where-Object { [int]$_.previewNumber -eq 6 } | Select-Object -First 1 + $preview7 = $previewTrackers | Where-Object { [int]$_.previewNumber -eq 7 } | Select-Object -First 1 + if ($null -eq $preview6) { + Write-Host " ❌ net11 missing preview6 tracker" -ForegroundColor Red; $script:failed++ + } else { + # Lifecycle-INVARIANT fields (don't drift candidate<->in-flight): identity, + # tag prefix, preview number, milestone, and the canonical proposed branch slug. + Assert-Eq -Label "preview6 canonicalKey" -Expected 'net11-preview6' -Actual $preview6.canonicalKey + Assert-Eq -Label "preview6 expectedTagPrefix" -Expected '11.0.0-preview.6.' -Actual $preview6.expectedTagPrefix + Assert-Eq -Label "preview6 previewNumber = 6" -Expected 6 -Actual $preview6.previewNumber + Assert-Eq -Label "preview6 milestone name" -Expected '.NET 11.0-preview6' -Actual $preview6.milestoneName + Assert-Eq -Label "preview6 branchName = canonical slug" ` + -Expected 'release/11.0.1xx-preview6' -Actual $preview6.branchName + # preview6 transitions candidate (no branch) -> in-flight (branch cut) over its + # cycle. Don't hard-code EITHER state: read the detector's own branchExists and + # assert mode/surveyRef/issueTitle are CONSISTENT with it. This stays green across + # the candidate->in-flight cut instead of drifting the day the branch lands — same + # invariant-over-snapshot philosophy as the hasRecentActivity asserts below. + Assert-Eq -Label "preview6 branchExists is a [bool] (lifecycle pivot)" ` + -Expected $true -Actual ($preview6.branchExists -is [bool]) + if ($preview6.branchExists) { + # Branch has been cut -> in-flight: the tracker surveys the branch itself. + Assert-Eq -Label "preview6 mode = in-flight (branch exists)" -Expected 'in-flight' -Actual $preview6.mode + Assert-Eq -Label "preview6 surveyRef = branchName (branch exists)" ` + -Expected $preview6.branchName -Actual $preview6.surveyRef + Assert-Eq -Label "preview6 issue title = in-flight form" ` + -Expected "[Release Readiness] .NET 11.0 preview6 — $($preview6.branchName)" ` -Actual $preview6.issueTitle - Assert-Eq -Label "preview6 branchName = its cut branch" ` - -Expected 'release/11.0.1xx-preview6' -Actual $preview6.branchName - Assert-Eq -Label "preview6 branchExists = true (branch cut)" ` - -Expected $true -Actual $preview6.branchExists - # hasRecentActivity is a 7-day-window signal, not a marker of an - # "active preview cycle" — the branch can idle >7 days and report $false. - # Assert the flag's TYPE, not its date-dependent value. - Assert-Eq -Label "preview6 hasRecentActivity is a [bool] (value date-dependent)" ` - -Expected $true -Actual ($preview6.hasRecentActivity -is [bool]) - Assert-Eq -Label "preview6 hasRecentActivity == (recentCommitCount > 0) [mapping invariant]" ` - -Expected $true -Actual ($preview6.hasRecentActivity -eq ([int]$preview6.recentCommitCount -gt 0)) - Assert-Eq -Label "preview6 regressionLabels carries previewN-1 + previewN" ` - -Expected 'regressed-in-11.0.0-preview5,regressed-in-11.0.0-preview6' ` - -Actual ($preview6.regressionLabels -join ',') } else { - Write-Host " ❌ net11 preview6 tracker missing" -ForegroundColor Red; $script:failed++ + # No branch yet -> candidate: the tracker surveys the major's dev branch. + Assert-Eq -Label "preview6 mode = candidate (no branch yet)" -Expected 'candidate' -Actual $preview6.mode + Assert-Eq -Label "preview6 surveyRef = mainBranch (no branch yet)" ` + -Expected $net11.mainBranch -Actual $preview6.surveyRef + Assert-Eq -Label "preview6 issue title = candidate form" ` + -Expected "[Release Readiness] .NET 11.0 preview6 — candidate from $($net11.mainBranch)" ` + -Actual $preview6.issueTitle + } + # hasRecentActivity is a 7-day-window signal, not a marker of an + # "active preview cycle" — net11.0 can idle >7 days and report $false. + # Assert the flag's TYPE, not its date-dependent value. + Assert-Eq -Label "preview6 hasRecentActivity is a [bool] (value date-dependent)" ` + -Expected $true -Actual ($preview6.hasRecentActivity -is [bool]) + # Pin the count->flag WIRING (hasRecentActivity = recentCommitCount > 0) for the + # preview construction path too — same-instant fields, so date-independent yet it + # still trips on an inverted/hardcoded mapping. + Assert-Eq -Label "preview6 hasRecentActivity == (recentCommitCount > 0) [mapping invariant]" ` + -Expected $true -Actual ($preview6.hasRecentActivity -eq ([int]$preview6.recentCommitCount -gt 0)) + Assert-Eq -Label "preview6 regressionLabels carries previewN-1 + previewN" ` + -Expected 'regressed-in-11.0.0-preview5,regressed-in-11.0.0-preview6' ` + -Actual ($preview6.regressionLabels -join ',') } - # preview7 — CANDIDATE from net11.0 (no branch cut yet). - if ($preview7) { + # preview7 — candidate from net11.0. net11.0 carries + # PreReleaseVersionIteration=7, so the detector emits a candidate for the + # NEXT preview distinct from in-flight preview6. Branch not cut yet -> + # candidate surveying net11.0. Mirrors the preview6 invariants but for the + # candidate state; reads branchExists and asserts mode/surveyRef/title are + # CONSISTENT with it so it stays green across the candidate->in-flight cut. + if ($null -eq $preview7) { + Write-Host " ❌ net11 missing preview7 candidate tracker" -ForegroundColor Red; $script:failed++ + } else { Assert-Eq -Label "preview7 canonicalKey" -Expected 'net11-preview7' -Actual $preview7.canonicalKey - Assert-Eq -Label "preview7 mode = candidate" -Expected 'candidate' -Actual $preview7.mode - Assert-Eq -Label "preview7 surveyRef = net11.0" -Expected 'net11.0' -Actual $preview7.surveyRef Assert-Eq -Label "preview7 expectedTagPrefix" -Expected '11.0.0-preview.7.' -Actual $preview7.expectedTagPrefix Assert-Eq -Label "preview7 previewNumber = 7" -Expected 7 -Actual $preview7.previewNumber Assert-Eq -Label "preview7 milestone name" -Expected '.NET 11.0-preview7' -Actual $preview7.milestoneName - Assert-Eq -Label "preview7 issue title format" ` - -Expected '[Release Readiness] .NET 11.0 preview7 — candidate from net11.0' ` - -Actual $preview7.issueTitle - Assert-Eq -Label "preview7 branchName = canonical proposed slug" ` + Assert-Eq -Label "preview7 branchName = canonical slug" ` -Expected 'release/11.0.1xx-preview7' -Actual $preview7.branchName - Assert-Eq -Label "preview7 branchExists = false (no branch yet)" ` - -Expected $false -Actual $preview7.branchExists - Assert-Eq -Label "preview7 hasRecentActivity is a [bool] (value date-dependent)" ` - -Expected $true -Actual ($preview7.hasRecentActivity -is [bool]) - Assert-Eq -Label "preview7 hasRecentActivity == (recentCommitCount > 0) [mapping invariant]" ` - -Expected $true -Actual ($preview7.hasRecentActivity -eq ([int]$preview7.recentCommitCount -gt 0)) + Assert-Eq -Label "preview7 branchExists is a [bool] (lifecycle pivot)" ` + -Expected $true -Actual ($preview7.branchExists -is [bool]) + if ($preview7.branchExists) { + Assert-Eq -Label "preview7 mode = in-flight (branch exists)" -Expected 'in-flight' -Actual $preview7.mode + Assert-Eq -Label "preview7 surveyRef = branchName (branch exists)" ` + -Expected $preview7.branchName -Actual $preview7.surveyRef + Assert-Eq -Label "preview7 issue title = in-flight form" ` + -Expected "[Release Readiness] .NET 11.0 preview7 — $($preview7.branchName)" ` + -Actual $preview7.issueTitle + } else { + Assert-Eq -Label "preview7 mode = candidate (no branch yet)" -Expected 'candidate' -Actual $preview7.mode + Assert-Eq -Label "preview7 surveyRef = mainBranch (no branch yet)" ` + -Expected $net11.mainBranch -Actual $preview7.surveyRef + Assert-Eq -Label "preview7 issue title = candidate form" ` + -Expected "[Release Readiness] .NET 11.0 preview7 — candidate from $($net11.mainBranch)" ` + -Actual $preview7.issueTitle + } Assert-Eq -Label "preview7 regressionLabels carries previewN-1 + previewN" ` -Expected 'regressed-in-11.0.0-preview6,regressed-in-11.0.0-preview7' ` -Actual ($preview7.regressionLabels -join ',') - } else { - Write-Host " ❌ net11 preview7 tracker missing" -ForegroundColor Red; $script:failed++ } } else { Write-Host " ❌ majors[] missing net11 entry" -ForegroundColor Red; $script:failed++ @@ -1211,6 +1299,21 @@ Assert-Eq -Label "Reverted-PR from quoted title containing internal quotes" ` # Case-insensitive: a hand-typed lowercase 'revert "..."' subject must resolve. Assert-Eq -Label "Reverted-PR from lowercase 'revert' subject" ` -Expected 4321 -Actual (Get-RevertedPrFromSubject -Subject 'revert "fix thing (#4321)" (#8765)') +# Manual/hand-authored revert form (no GitHub quotes, no "This reverts commit" +# body): `Revert - #<reverted> (#<revertPR>)`. Real maui case: #36152 +# reverted #35372. Without recovering 35372, revertedPrSet holds only the revert +# PR (36152), the reverted fix's original commit still satisfies the on-branch +# gate, and a "fixed by #35372" comment on a CLOSED issue is falsely de-noised. +Assert-Eq -Label "Reverted-PR from manual 'Revert - <title> #N (#M)' form (real #36152/#35372)" ` + -Expected 35372 -Actual (Get-RevertedPrFromSubject -Subject 'Revert - Fix Android stale ContainerView root leak #35372 (#36152)') +Assert-Eq -Label "Manual revert form with branch prefix" ` + -Expected 40100 -Actual (Get-RevertedPrFromSubject -Subject '[release/10.0.1xx-sr9] Revert - Fix flaky test #40100 (#40200)') +# Safety: a manual-form pattern must NOT fire on a non-revert subject that merely +# ends with `#N (#M)`, nor return the trailing (#M) when no reverted # is present. +Assert-Eq -Label "Non-revert subject ending in '#N (#M)' still yields null" ` + -Expected $null -Actual (Get-RevertedPrFromSubject -Subject 'Fix layout regression #40300 (#40400)') +Assert-Eq -Label "Revert subject with only the trailing (#M) yields null (no false reverted-PR)" ` + -Expected $null -Actual (Get-RevertedPrFromSubject -Subject 'Revert - some cleanup (#40500)') # ───── Test-PrIsToolingOnly (false-positive guard #1) ───── Write-Host "`n[Unit] Test-PrIsToolingOnly (FP guard)" -ForegroundColor Cyan @@ -1408,6 +1511,266 @@ $cls2 = Classify-RegressionCandidate ` Assert-Eq -Label "Partial SrContents (no commits/fixedIssues) does not throw" ` -Expected 'no-fix-yet' -Actual $cls2.classification +# ───── Get-IssueCommentPrs (negation guard on fix-phrase scoring) ───── +# A maintainer comment that NEGATES a fix ("not fixed by #X", "won't fix #Y") must +# NOT be scored as high-confidence 'fix-phrase' — otherwise the closed-fix-unlinked +# fallback would treat a "still broken" comment as proof of a fix. Exercises the REAL +# Get-IssueCommentPrs (mocking only its gh call) so the regex itself is under test. +Write-Host "`n[Unit] Get-IssueCommentPrs (negated fix phrases score as 'mention')" -ForegroundColor Cyan + +$origInvokeGh = ${function:Invoke-Gh} +$script:mockCommentsJson = @' +[ + { "body": "Duplicate report — not fixed by #35028, still reproduces on SR8." }, + { "body": "This is actually fixed by #40001 in the nightly build." }, + { "body": "won't fix #50002 — working as intended." }, + { "body": "see #60003 for related context" }, + { "body": "not fixed by #70004 yet" }, + { "body": "update: now fixed by #70004" } +] +'@ +function Invoke-Gh { param([string[]]$GhArgs, [switch]$Quiet) return $script:mockCommentsJson } +try { + $scored = Get-IssueCommentPrs -Repo 'dotnet/maui' -IssueNumber 99999 + $byNum = @{}; foreach ($s in $scored) { $byNum[[int]$s.number] = $s.evidence } + + Assert-Eq -Label "negated 'not fixed by #35028' -> mention (not fix-phrase)" ` + -Expected 'mention' -Actual $byNum[35028] + Assert-Eq -Label "plain 'fixed by #40001' -> fix-phrase" ` + -Expected 'fix-phrase' -Actual $byNum[40001] + Assert-Eq -Label "negated 'won't fix #50002' -> mention" ` + -Expected 'mention' -Actual $byNum[50002] + Assert-Eq -Label "bare 'see #60003' (no fix word) -> mention" ` + -Expected 'mention' -Actual $byNum[60003] + # Strongest-evidence-wins still holds: #70004 is negated in one comment but + # confirmed in another -> the non-negated fix phrase upgrades it to fix-phrase. + Assert-Eq -Label "#70004 negated once + confirmed once -> fix-phrase wins" ` + -Expected 'fix-phrase' -Actual $byNum[70004] +} finally { + ${function:Invoke-Gh} = $origInvokeGh +} + +# ───── Get-IssueCommentPrs (cross-repo references are NOT local PRs) ───── +# A maui regression is only de-noised by a fix in THIS repo. A cross-repo +# shorthand (dotnet/runtime#N) or a github.com/<other>/<repo>/pull/N URL must +# NOT be mistaken for maui#N and reported as "No ship risk". Same-repo +# shorthand, same-repo pull URLs, bare #N and PR#N must still be extracted. +Write-Host "`n[Unit] Get-IssueCommentPrs (cross-repo references rejected)" -ForegroundColor Cyan +$origInvokeGh2 = ${function:Invoke-Gh} +$script:mockCrossRepoJson = @' +[ + { "body": "root cause is upstream, fixed by dotnet/runtime#35028" }, + { "body": "the real fix is https://github.com/dotnet/runtime/pull/41000" }, + { "body": "actually resolved by dotnet/maui#42000 on the SR" }, + { "body": "fixed by #43000" }, + { "body": "landed in https://github.com/dotnet/maui/pull/44000" }, + { "body": "closed by PR#45000" }, + { "body": "see dotnet/runtime/pull/46000 for the upstream fix" }, + { "body": "resolved by dotnet/maui/pull/47000" } +] +'@ +function Invoke-Gh { param([string[]]$GhArgs, [switch]$Quiet) return $script:mockCrossRepoJson } +try { + $scored2 = Get-IssueCommentPrs -Repo 'dotnet/maui' -IssueNumber 88888 + $nums = @($scored2 | ForEach-Object { [int]$_.number }) + + Assert-Eq -Label "cross-repo 'dotnet/runtime#35028' shorthand is NOT extracted" ` + -Expected $false -Actual ($nums -contains 35028) + Assert-Eq -Label "cross-repo runtime pull URL (41000) is NOT extracted" ` + -Expected $false -Actual ($nums -contains 41000) + Assert-Eq -Label "same-repo 'dotnet/maui#42000' shorthand IS extracted" ` + -Expected $true -Actual ($nums -contains 42000) + Assert-Eq -Label "bare '#43000' IS extracted" ` + -Expected $true -Actual ($nums -contains 43000) + Assert-Eq -Label "same-repo maui pull URL (44000) IS extracted" ` + -Expected $true -Actual ($nums -contains 44000) + Assert-Eq -Label "unqualified 'PR#45000' IS extracted (recall preserved)" ` + -Expected $true -Actual ($nums -contains 45000) + Assert-Eq -Label "scheme-less cross-repo 'dotnet/runtime/pull/46000' is NOT extracted" ` + -Expected $false -Actual ($nums -contains 46000) + Assert-Eq -Label "scheme-less same-repo 'dotnet/maui/pull/47000' IS extracted" ` + -Expected $true -Actual ($nums -contains 47000) + + $byNum2 = @{}; foreach ($s in $scored2) { $byNum2[[int]$s.number] = $s.evidence } + Assert-Eq -Label "same-repo 'resolved by dotnet/maui#42000' -> fix-phrase" ` + -Expected 'fix-phrase' -Actual $byNum2[42000] +} finally { + ${function:Invoke-Gh} = $origInvokeGh2 +} +# Real-world case driving this class: SR8 tracker #35876 flagged six CLOSED issues +# (#35252/#35253/#35254/#35255/#35291/#35409) as `no-fix-yet`/"Investigate" even +# though five of them were closed with a maintainer comment naming a MERGED fix PR +# that is already on release/10.0.1xx-sr8. The fix never used a closing keyword and +# GitHub recorded no timeline cross-reference, so Get-IssueTimelinePrs found nothing. +# The fallback recovers the cited PR from the comment, verifies it MERGED and sits on +# the SR branch, and reclassifies to the non-blocking `closed-fix-unlinked` (a missing +# link, not a missing fix). +Write-Host "`n[Unit] Classify-RegressionCandidate (closed-fix-unlinked)" -ForegroundColor Cyan + +# Mock the comment-PR recovery + the fix PR (#35028, merged into inflight/candidate +# and present on SR8) + branch membership for origin/release/10.0.1xx-sr8. +function Get-IssueCommentPrs { + param($Repo, $IssueNumber) + return @(@{ number = 35028; evidence = 'fix-phrase' }) +} +function Get-PrInfo { + param($Repo, $PrNumber) + return [pscustomobject]@{ + number = $PrNumber + title = 'Fix unstable CollectionView CI repro tests' + state = 'MERGED' + baseRefName = 'inflight/candidate' + mergedAt = '2026-06-01T00:00:00Z' + closedAt = '2026-06-01T00:00:00Z' + body = 'Fixes #35104' # links a DIFFERENT issue — never these five + mergeCommit = [pscustomobject]@{ oid = 'c1d6d72768c0ffee' } + files = @([pscustomobject]@{ path = 'src/Controls/tests/TestCases.HostApp/Issue35253.xaml.cs'; additions = 4; deletions = 0 }) + } +} +# Faithful to the real cross-branch flow: the fix squash-merged into the +# inflight/candidate side under SHA `c1d6...`, then flowed to SR8 under a +# DIFFERENT SHA. So direct SHA-ancestry of the PR's mergeCommit is FALSE; the +# `(#35028)` subject token is what proves presence on SR8. +function Test-CommitOnBranch { + param([string]$Sha, [string]$BranchRef) + return $false +} +function Test-PrNumberOnBranch { + param([int]$PrNumber, [string]$BranchRef) + return ($PrNumber -eq 35028 -and $BranchRef -eq 'origin/release/10.0.1xx-sr8') +} + +$clsUnlinked = Classify-RegressionCandidate ` + -Issue ([pscustomobject]@{ number = 35254; state = 'CLOSED' }) ` + -CandidatePrs @() ` + -Ctx @{ repo = 'dotnet/maui'; srBranch = 'release/10.0.1xx-sr8'; mainBranch = 'main' } ` + -SrContents @{ sourcePrs = @(); reverts = @() } + +Assert-Eq -Label "Closed issue + comment-cited merged PR on SR → closed-fix-unlinked (not no-fix-yet)" ` + -Expected 'closed-fix-unlinked' -Actual $clsUnlinked.classification +Assert-Eq -Label "closed-fix-unlinked → high confidence (fix-phrase required)" ` + -Expected 'high' -Actual $clsUnlinked.confidence +Assert-Eq -Label "closed-fix-unlinked → candidateFixPrs surfaces the cited PR (#35028)" ` + -Expected 35028 -Actual ([int]$clsUnlinked.candidateFixPrs[0].number) +Assert-Eq -Label "closed-fix-unlinked recovered via (#num) subject token when SHA-ancestry is false" ` + -Expected $true -Actual (($clsUnlinked.evidence -join "`n") -match 'present on release/10\.0\.1xx-sr8') +Assert-Eq -Label "closed-fix-unlinked → action is to add a closing reference (no ship risk)" ` + -Expected $true -Actual ($clsUnlinked.recommendedAction -match 'closing reference') +Assert-Eq -Label "closed-fix-unlinked is Tier 3 (non-blocking)" ` + -Expected 3 -Actual (Get-VerdictTier -Classification 'closed-fix-unlinked') + +# Guard A — bare 'mention' (no fix verb) must STAY no-fix-yet. Regression issues +# routinely name the CAUSE PR for context ("Before PR #X ... After PR #X"); the +# cause naturally sits on the branch, so the branch gate alone can't distinguish +# a fix from blame. The fix-phrase requirement is what rejects this. This is the +# #35291 false-positive guard: its comment blames #32080 (merged, on SR8) but +# names no fix → it must not reclassify. +function Get-IssueCommentPrs { param($Repo, $IssueNumber) return @(@{ number = 32080; evidence = 'mention' }) } +function Test-PrNumberOnBranch { param([int]$PrNumber, [string]$BranchRef) return $true } # cause PR IS on branch +$clsMention = Classify-RegressionCandidate ` + -Issue ([pscustomobject]@{ number = 35291; state = 'CLOSED' }) ` + -CandidatePrs @() ` + -Ctx @{ repo = 'dotnet/maui'; srBranch = 'release/10.0.1xx-sr8'; mainBranch = 'main' } ` + -SrContents @{ sourcePrs = @(); reverts = @() } +Assert-Eq -Label "Bare mention of a merged on-branch CAUSE PR → stays no-fix-yet (#35291 guard)" ` + -Expected 'no-fix-yet' -Actual $clsMention.classification + +# Guard B — #35291 was closed as by-design (real bug spun to #35310); its comments +# name NO fix PR. Must STAY no-fix-yet, not get a phantom reclassification. +function Get-IssueCommentPrs { param($Repo, $IssueNumber) return @() } +$clsByDesign = Classify-RegressionCandidate ` + -Issue ([pscustomobject]@{ number = 35291; state = 'CLOSED' }) ` + -CandidatePrs @() ` + -Ctx @{ repo = 'dotnet/maui'; srBranch = 'release/10.0.1xx-sr8'; mainBranch = 'main' } ` + -SrContents @{ sourcePrs = @(); reverts = @() } +Assert-Eq -Label "Closed issue with NO comment-cited PR → stays no-fix-yet (#35291)" ` + -Expected 'no-fix-yet' -Actual $clsByDesign.classification + +# Guard C — cited PR uses fix-phrase and is merged, but is NOT on the SR branch +# (neither SHA-ancestry nor `(#num)` subject token) → the branch gate rejects it, +# so it stays no-fix-yet (prevents a false 'fix is present' on a fix that landed +# on a different branch only). +function Get-IssueCommentPrs { param($Repo, $IssueNumber) return @(@{ number = 99001; evidence = 'fix-phrase' }) } +function Test-CommitOnBranch { param([string]$Sha, [string]$BranchRef) return $false } +function Test-PrNumberOnBranch { param([int]$PrNumber, [string]$BranchRef) return $false } +$clsNotOnSr = Classify-RegressionCandidate ` + -Issue ([pscustomobject]@{ number = 99100; state = 'CLOSED' }) ` + -CandidatePrs @() ` + -Ctx @{ repo = 'dotnet/maui'; srBranch = 'release/10.0.1xx-sr8'; mainBranch = 'main' } ` + -SrContents @{ sourcePrs = @(); reverts = @() } +Assert-Eq -Label "Comment-cited PR NOT on SR branch → stays no-fix-yet (branch gate)" ` + -Expected 'no-fix-yet' -Actual $clsNotOnSr.classification + +# Guard D — OPEN issue is never de-noised: a genuinely-open regression must keep +# blocking even if a comment happens to name a merged on-branch fix PR. +function Get-IssueCommentPrs { param($Repo, $IssueNumber) return @(@{ number = 35028; evidence = 'fix-phrase' }) } +function Test-CommitOnBranch { param([string]$Sha, [string]$BranchRef) return $true } +function Test-PrNumberOnBranch { param([int]$PrNumber, [string]$BranchRef) return $true } +$clsOpen = Classify-RegressionCandidate ` + -Issue ([pscustomobject]@{ number = 99200; state = 'OPEN' }) ` + -CandidatePrs @() ` + -Ctx @{ repo = 'dotnet/maui'; srBranch = 'release/10.0.1xx-sr8'; mainBranch = 'main' } ` + -SrContents @{ sourcePrs = @(); reverts = @() } +Assert-Eq -Label "OPEN issue is never reclassified to closed-fix-unlinked" ` + -Expected 'no-fix-yet' -Actual $clsOpen.classification + +# Guard E — a comment names a MERGED fix PR that IS on the SR by the `(#num)` +# subject token, BUT the SR later REVERTED it. A reverted fix is not a fix: the +# revertedPrSet parity with the main SR-contents/candidate paths must drop it, so +# the issue stays no-fix-yet instead of reporting a false "No ship risk". Extra +# teeth: Test-PrNumberOnBranch matches `(#35028)` which ALSO appears inside the +# revert commit's own subject, so without this guard the on-branch gate passes. +function Get-IssueCommentPrs { param($Repo, $IssueNumber) return @(@{ number = 35028; evidence = 'fix-phrase' }) } +function Get-PrInfo { + param($Repo, $PrNumber) + return [pscustomobject]@{ + number = $PrNumber + title = 'Fix flaky CollectionView test' + state = 'MERGED' + baseRefName = 'inflight/candidate' + mergedAt = '2026-06-01T00:00:00Z' + closedAt = '2026-06-01T00:00:00Z' + body = 'Fixes #35104' + mergeCommit = [pscustomobject]@{ oid = 'c1d6d72768c0ffee' } + files = @([pscustomobject]@{ path = 'src/Controls/src/Core/CollectionView.cs'; additions = 4; deletions = 0 }) + } +} +function Test-CommitOnBranch { param([string]$Sha, [string]$BranchRef) return $false } +function Test-PrNumberOnBranch { param([int]$PrNumber, [string]$BranchRef) return $true } +$clsReverted = Classify-RegressionCandidate ` + -Issue ([pscustomobject]@{ number = 35260; state = 'CLOSED' }) ` + -CandidatePrs @() ` + -Ctx @{ repo = 'dotnet/maui'; srBranch = 'release/10.0.1xx-sr8'; mainBranch = 'main' } ` + -SrContents @{ sourcePrs = @(); reverts = @(@{ revertsPr = 35028; revertBackportPr = $null }) } +Assert-Eq -Label "Comment-cited fix that the SR later REVERTED → stays no-fix-yet (not closed-fix-unlinked)" ` + -Expected 'no-fix-yet' -Actual $clsReverted.classification + +# Guard F — the comment's cited "fix" PR is ITSELF a Revert (a rollback), not a +# fix. Its title matches the Revert guard, so it must be skipped → no-fix-yet. +function Get-IssueCommentPrs { param($Repo, $IssueNumber) return @(@{ number = 40000; evidence = 'fix-phrase' }) } +function Get-PrInfo { + param($Repo, $PrNumber) + return [pscustomobject]@{ + number = $PrNumber + title = 'Revert "Fix flaky CollectionView test (#35028)" (#40000)' + state = 'MERGED' + baseRefName = 'release/10.0.1xx-sr8' + mergedAt = '2026-06-02T00:00:00Z' + closedAt = '2026-06-02T00:00:00Z' + body = 'Reverts #35028' + mergeCommit = [pscustomobject]@{ oid = 'deadbeefcafe0001' } + files = @([pscustomobject]@{ path = 'src/Controls/src/Core/CollectionView.cs'; additions = 0; deletions = 4 }) + } +} +function Test-PrNumberOnBranch { param([int]$PrNumber, [string]$BranchRef) return $true } +$clsRevertTitle = Classify-RegressionCandidate ` + -Issue ([pscustomobject]@{ number = 35261; state = 'CLOSED' }) ` + -CandidatePrs @() ` + -Ctx @{ repo = 'dotnet/maui'; srBranch = 'release/10.0.1xx-sr8'; mainBranch = 'main' } ` + -SrContents @{ sourcePrs = @(); reverts = @() } +Assert-Eq -Label "Comment-cited 'Revert ...' PR is a rollback, not a fix → stays no-fix-yet" ` + -Expected 'no-fix-yet' -Actual $clsRevertTitle.classification + # ───── Get-VerdictTier (deterministic tier table) ───── Write-Host "`n[Unit] Get-VerdictTier (deterministic tier table)" -ForegroundColor Cyan @@ -1422,6 +1785,7 @@ foreach ($case in @( @{ Cls = 'needs-human-review'; Tier = 2 } @{ Cls = 'in-sr-active'; Tier = 3 } @{ Cls = 'closed-as-duplicate'; Tier = 3 } + @{ Cls = 'closed-fix-unlinked'; Tier = 3 } @{ Cls = 'out-of-scope-future-sr'; Tier = 3 } @{ Cls = 'something-unknown'; Tier = 2 } # safe-default: risk )) { @@ -1618,6 +1982,55 @@ Assert-Eq -Label "Hash changes when verdict.symbol changes" -Expected $false -Ac $hashAgain = Get-ReportSemanticHash -Data $dataA -Verdict $verdictA Assert-Eq -Label "Hash is deterministic across runs" -Expected $hashA -Actual $hashAgain +# Lifecycle mode flip → DIFFERENT hash, even with byte-identical content. +# This guards the in-flight -> shipped transition: `-Shipped` surveys the SAME +# SR branch, so without folding `mode` into the hash the shipped run would +# collide with the last in-flight run and the workflow's no-op would skip the +# `gh issue edit`, freezing the tracker as "in-flight" and never flipping it to +# "shipped." Identical $Data except metadata.mode. +$dataInflight = @{ + metadata = @{ srHeadSha = 'cccccccc3333'; fetchedAt = '2025-01-01T00:00:00Z'; mode = 'in-flight' } + ci = @{ overall = 'green' } + srContents = @{ sourcePrs = @(35001, 35002) } + regressions = @( @{ issue = 35001; classification = 'in-sr-active' } ) + openSrPrs = @( @{ number = 35100 } ) +} +$dataShipped = @{ + metadata = @{ srHeadSha = 'cccccccc3333'; fetchedAt = '2025-01-01T00:00:00Z'; mode = 'shipped' } + ci = @{ overall = 'green' } + srContents = @{ sourcePrs = @(35001, 35002) } + regressions = @( @{ issue = 35001; classification = 'in-sr-active' } ) + openSrPrs = @( @{ number = 35100 } ) +} +$hInflight = Get-ReportSemanticHash -Data $dataInflight -Verdict $verdictA +$hShipped = Get-ReportSemanticHash -Data $dataShipped -Verdict $verdictA +Assert-Eq -Label "hash: in-flight vs shipped (identical content) → DIFFERENT (tracker flips to shipped)" ` + -Expected $false -Actual ($hInflight -eq $hShipped) +# Candidate is likewise distinct, and the fold is deterministic within a mode. +$dataCandidate = @{ + metadata = @{ srHeadSha = 'cccccccc3333'; fetchedAt = '2025-01-01T00:00:00Z'; mode = 'candidate' } + ci = @{ overall = 'green' } + srContents = @{ sourcePrs = @(35001, 35002) } + regressions = @( @{ issue = 35001; classification = 'in-sr-active' } ) + openSrPrs = @( @{ number = 35100 } ) +} +$hCandidate = Get-ReportSemanticHash -Data $dataCandidate -Verdict $verdictA +Assert-Eq -Label "hash: candidate vs in-flight (identical content) → DIFFERENT" ` + -Expected $false -Actual ($hCandidate -eq $hInflight) +Assert-Eq -Label "hash: mode fold is deterministic (shipped recomputed → SAME)" ` + -Expected $hShipped -Actual (Get-ReportSemanticHash -Data $dataShipped -Verdict $verdictA) +# Absent mode defaults to 'in-flight' → SAME as an explicit 'in-flight'. +$dataNoMode = @{ + metadata = @{ srHeadSha = 'cccccccc3333'; fetchedAt = '2025-01-01T00:00:00Z' } + ci = @{ overall = 'green' } + srContents = @{ sourcePrs = @(35001, 35002) } + regressions = @( @{ issue = 35001; classification = 'in-sr-active' } ) + openSrPrs = @( @{ number = 35100 } ) +} +$hNoMode = Get-ReportSemanticHash -Data $dataNoMode -Verdict $verdictA +Assert-Eq -Label "hash: absent mode defaults to in-flight → SAME as explicit in-flight" ` + -Expected $hInflight -Actual $hNoMode + # Order independence: source PRs in different order → SAME hash $dataReorder = $dataA.Clone() $dataReorder['srContents'] = @{ sourcePrs = @(35003, 35001, 35002) } # reordered @@ -1821,6 +2234,36 @@ Assert-Eq -Label "Without -TrackerKey: no visible Tracker line" -Expected $false Assert-Eq -Label "Without -TrackerKey: hash marker still present" -Expected $true ` -Actual ($mdNoTracker -match '<!-- release-readiness-hash:') +# ───── Body header `mode=` label: in-flight (default) / shipped / candidate ───── +# The rendered Tracker line and H1 must reflect metadata.mode so a post-ship +# tracker reads `mode=shipped` instead of misreporting as in-flight. Clone the +# metadata hashtable (shallow .Clone() shares it) so we don't pollute later tests. +Assert-Eq -Label 'Default render: Tracker line reads mode=in-flight' -Expected $true ` + -Actual ($md -match '\*\*Tracker:\*\* `net10-sr7` · mode=`in-flight`') +Assert-Eq -Label 'Default render: H1 is plain (not CANDIDATE)' -Expected $true ` + -Actual ($md -match '# Release Readiness — release/10\.0\.1xx-sr7') + +$mdDataShipped = $mdData.Clone() +$mdDataShipped.metadata = $mdData.metadata.Clone() +$mdDataShipped.metadata.mode = 'shipped' +$mdShipped = Format-MarkdownReport -Data $mdDataShipped -RepoUrl 'https://github.com/dotnet/maui' ` + -TrackerKey 'net10-sr7' -MaxBodyBytes 60000 +Assert-Eq -Label 'Shipped render: Tracker line reads mode=shipped' -Expected $true ` + -Actual ($mdShipped -match '\*\*Tracker:\*\* `net10-sr7` · mode=`shipped`') +Assert-Eq -Label 'Shipped render: H1 stays plain branch survey (not CANDIDATE)' -Expected $true ` + -Actual ($mdShipped -match '# Release Readiness — release/10\.0\.1xx-sr7' -and $mdShipped -notmatch '# Release Readiness — CANDIDATE') + +$mdDataCand = $mdData.Clone() +$mdDataCand.metadata = $mdData.metadata.Clone() +$mdDataCand.metadata.mode = 'candidate' +$mdDataCand.metadata.priorSrBranch = 'release/10.0.1xx-sr6' +$mdCand = Format-MarkdownReport -Data $mdDataCand -RepoUrl 'https://github.com/dotnet/maui' ` + -TrackerKey 'net10-sr7' -MaxBodyBytes 60000 +Assert-Eq -Label 'Candidate render: H1 shows CANDIDATE pre-flight' -Expected $true ` + -Actual ($mdCand -match '# Release Readiness — CANDIDATE for next SR') +Assert-Eq -Label 'Candidate render: Tracker line reads mode=candidate' -Expected $true ` + -Actual ($mdCand -match 'mode=`candidate`') + # Body cap: with very low cap, must truncate $mdCapped = Format-MarkdownReport -Data $mdData -RepoUrl 'https://github.com/dotnet/maui' ` -TrackerKey 'net10-sr7' -MaxBodyBytes 500 @@ -2098,6 +2541,28 @@ Assert-Eq -Label "CLOSED no-fix-yet (#96202) does NOT render in Tier 1" -Expecte Assert-Eq -Label "CLOSED no-fix-yet (#96202) renders in Tier 3 (not dropped)" -Expected $true ` -Actual ($tier3Block -match '#96202') +# (4c) closed-fix-unlinked renders in Tier 3 with its candidate PR + traceability action, +# and is NEVER counted as a blocker (the whole point: de-noise the false no-fix-yet alarm). +$mdDataCfu = @{} + $mdData +$mdDataCfu['regressions'] = @( + @{ issue = 96254; title = 'Closed CV repro, fix only in a comment'; state = 'CLOSED'; + classification = 'closed-fix-unlinked'; + candidateFixPrs = @(@{ number = 95028; title = 'Fix repro'; state = 'MERGED'; evidenceType = 'comment-fix-phrase' }); + recommendedAction = 'No ship risk — fix is already in the SR. Add a closing reference for traceability.' } +) +$mdDataCfu['summary'] = @{ 'closed-fix-unlinked' = 1 } +$mdCfu = Format-MarkdownReport -Data $mdDataCfu -RepoUrl 'https://github.com/dotnet/maui' ` + -TrackerKey 'net10-sr8' -MaxBodyBytes 60000 +$cfuLines = @($mdCfu -split "`r?`n") +$idxCfuT3 = ($cfuLines | Select-String -Pattern '🟢 Tier 3' | Select-Object -First 1).LineNumber - 1 +$cfuTier3Block = ($cfuLines[$idxCfuT3..($cfuLines.Count - 1)] -join "`n") +Assert-Eq -Label "closed-fix-unlinked renders in Tier 3 (#96254)" -Expected $true ` + -Actual ($cfuTier3Block -match '#96254') +Assert-Eq -Label "closed-fix-unlinked Tier-3 row links the recovered fix PR (#95028)" -Expected $true ` + -Actual ($cfuTier3Block -match '#95028') +Assert-Eq -Label "closed-fix-unlinked does NOT appear in a 🔴 Blocking section" -Expected $false ` + -Actual ($mdCfu -match '🔴 Blocking') + # (5) Marker-forgery via a TABLE cell: a Tier-1 title embedding the begin-marker between # newlines must NOT forge a second anchored marker line. $mdDataForgeTbl = @{} + $mdData @@ -3621,6 +4086,21 @@ Assert-Eq -Label "precedence: generic human keeps #5" -Expe Assert-Eq -Label "precedence: inflight-human = the inflight p/0 human (#8)" -Expected $true -Actual ($bInflight -contains 8) Assert-Eq -Label "precedence: inflight-human excludes inflight Maestro (#7)" -Expected $false -Actual ($bInflight -contains 7) +# Inflight merge-up hoist (#36085 scenario): a main → net<N>.0 automated merge PR +# (base = inflight branch) must be carved into the merge-up bucket — hoisted to +# high priority — and REMOVED from the inflight-human queue, not buried as generic +# inflight noise. The preview lane chains main → net<N>.0 → previewN, so a stuck +# main → net<N>.0 merge starves the preview branch of upstream fixes and belongs +# in the daily-flow merge-up chain alongside the net<N>.0 → previewN hop. +$prInflightMergeUp = [PSCustomObject]@{ number = 9; author = $humanLogin; labels = $plainLbl; headRefName = 'merge/main-to-net11.0'; title = "[automated] Merge branch 'main' => 'net11.0'" } +$bucketsIM = Get-CategorizedPullRequests -TargetPRs $targetSet -InflightPRs @($prInflightMaestro, $prInflightP0, $prInflightMergeUp) +$imMergeUp = @($bucketsIM.MergeUpPRs | ForEach-Object { $_.number }) +$imInflight = @($bucketsIM.InflightHumanPRs | ForEach-Object { $_.number }) +Assert-Eq -Label "inflight merge-up: hoisted into merge-up bucket (#9)" -Expected $true -Actual ($imMergeUp -contains 9) +Assert-Eq -Label "inflight merge-up: both hops in merge-up bucket (#6 target + #9 inflight)" -Expected 2 -Actual $bucketsIM.MergeUpPRs.Count +Assert-Eq -Label "inflight merge-up: removed from inflight-human queue (#9)" -Expected $false -Actual ($imInflight -contains 9) +Assert-Eq -Label "inflight merge-up: inflight-human still keeps the plain p/0 human (#8)" -Expected $true -Actual ($imInflight -contains 8) + # Empty-input safety: no PRs at all yields five empty buckets, no throw. $emptyBuckets = Get-CategorizedPullRequests -TargetPRs @() -InflightPRs @() Assert-Eq -Label "precedence: empty input → 0 p/0" -Expected 0 -Actual $emptyBuckets.P0Prs.Count diff --git a/.github/workflows/release-readiness.yml b/.github/workflows/release-readiness.yml index 518b83206a25..195154c2ffcb 100644 --- a/.github/workflows/release-readiness.yml +++ b/.github/workflows/release-readiness.yml @@ -184,6 +184,12 @@ jobs: fi SR_ARG="$PRIOR_SR" CANDIDATE_ARG=(-Candidate) + elif [ "$MODE" = "shipped" ]; then + # Already-tagged SR: survey the branch directly (same as in-flight), + # but pass -Shipped so the rendered header reads mode=shipped rather + # than misreporting the post-ship tracker as in-flight. + SR_ARG="$BRANCH_NAME" + CANDIDATE_ARG=(-Shipped) else SR_ARG="$BRANCH_NAME" fi @@ -242,6 +248,7 @@ jobs: MILESTONE_NAME: ${{ matrix.milestoneName }} BODY_FILE: ${{ steps.report.outputs.body-file }} RECENT_COMMIT_COUNT: ${{ matrix.recentCommitCount }} + MODE: ${{ matrix.mode }} shell: bash run: | set -euo pipefail @@ -258,6 +265,17 @@ jobs: --limit 50 \ --jq '. // [] | sort_by(.createdAt) | .[].number') + # Shipped trackers are REFRESH-ONLY. The most-recently-shipped SR keeps + # surfacing post-ship follow-up (adding the build to the GitHub issue + # version dropdown, release notes, milestone close-out) while its tracker + # issue stays OPEN, but we never (re)create it. Once a human closes the + # tracker, it stays closed — "update until closed manually" without + # resurrecting a closed issue on the next scheduled run. + if [ "$MODE" = "shipped" ] && [ -z "$EXISTING" ]; then + echo "Shipped tracker ${TRACKER_KEY}: no open issue (closed or never created) — refresh-only, nothing to do." + exit 0 + fi + # Activity gate: when there is no recent activity AND no open tracker issue, # skip new-issue creation. (If an existing issue is open, we still refresh it.) if [ "$RECENT_COMMIT_COUNT" -eq 0 ] && [ -z "$EXISTING" ]; then @@ -426,10 +444,19 @@ jobs: --repo "${{ github.repository }}" --title "$ISSUE_TITLE" --body-file "$BODY_FILE" - --label "report" - --label "s/triaged" - --label "area-release-readiness" ) + # Best-effort label attach — never fail the job for a label that does + # not exist in this repo. `gh issue create` hard-fails (422) on an + # unknown label, so attaching one that was renamed/removed would abort + # every auto-create. Probe each label and skip (with a warning) any + # that is missing, exactly like the milestone handling below. + for lbl in "report" "s/triaged" "area-infrastructure"; do + if gh api "repos/${{ github.repository }}/labels/${lbl//\//%2F}" --jq '.name' >/dev/null 2>&1; then + CREATE_ARGS+=(--label "$lbl") + else + echo "::warning::Label '$lbl' not found; creating issue without it." + fi + done # Best-effort milestone attach — never fail the job for a missing milestone. if [ -n "$MILESTONE_NAME" ]; then if gh api "repos/${{ github.repository }}/milestones?state=open&per_page=100" \ @@ -516,6 +543,12 @@ jobs: if [ "$MODE" = "candidate" ]; then SR_ARG="$PRIOR_SR" CANDIDATE_ARG=(-Candidate) + elif [ "$MODE" = "shipped" ]; then + # Mirror the production job: survey the branch directly but pass + # -Shipped so the smoke-run exercises the shipped code path (header + # relabel + semantic-hash mode fold) instead of running as in-flight. + SR_ARG="$BRANCH_NAME" + CANDIDATE_ARG=(-Shipped) else SR_ARG="$BRANCH_NAME" fi