Add closed-fix-unlinked regression classification to release-readiness

[PureWeen]

Note

Are you waiting for the changes in this PR to be merged?
It would be very helpful if you could test the resulting artifacts from this PR and let us know in a comment if this change resolves your issue. Thank you!

Problem

The release-readiness SR tracker falsely listed CLOSED regression issues as no-fix-yet / "Investigate" even when a merged fix was already on the release branch. Concrete case: the SR8 tracker (#35876) flagged six closed issues (#35252, #35253, #35254, #35255, #35291, #35409) as needing investigation, but five of them were closed with a maintainer comment naming a merged fix PR that already sits on release/10.0.1xx-sr8.

Root cause: the fix↔issue link lived only in human closing-comment prose ("This issue was fixed by PR #35028"). The fix PR never used a closing keyword (Fixes #NNNNN) and GitHub recorded no timeline cross-reference, so the timeline-only classifier (Get-IssueTimelinePrs) found zero candidates and fell through to no-fix-yet.

Fix

A new closed-fix-unlinked classification (Tier 3, non-blocking). For CLOSED issues, a fallback in Classify-RegressionCandidate recovers the fix PR from the closing-comment prose and verifies it before de-noising the alarm. Two gates keep it safe and always high-confidence:

1. fix-phrase only — the comment must pair the PR reference with fix/resolve/close language. A bare mention is rejected, because regression issues routinely name the cause PR for context ("Before PR #X … After PR #X"), and the cause naturally sits on the branch — it is not a fix.
2. merged AND on the SR branch — presence verified by SHA-ancestry OR the literal (#<num>) squash-subject token, which survives the cross-branch flow where the PR's mergeCommit SHA differs from the SR-side SHA.

The #35291 guard

#35291 was closed as by-design (the real bug spun off to #35310). Its comment blames #32080 ("Before PR #32080… After PR #32080") — #32080 is merged and on SR8, so the branch gate alone would have passed it. The fix-phrase requirement is what correctly keeps #35291 as no-fix-yet.

New helpers

• Get-IssueCommentPrs — scans issue comment bodies for PR references, tags each fix-phrase or mention.
• Test-PrNumberOnBranch — matches the (#<num>) subject token via git log --fixed-strings --grep, handling squash / cross-branch flow.

Also in this PR — shipped-SR tracker refresh-until-closed lifecycle (folded from #36113)

The most-recently-shipped SR no longer goes dark the moment it ships. Previously, once an SR's stable tag was published the detector dropped its tracker entirely — but post-ship there is still follow-up work (e.g. adding the new SR to the GitHub issue version dropdown). Now Find-ReleaseReadinessTrackers.ps1 emits a mode='shipped' tracker for the highest shipped SR that refreshes until a human closes the issue, with a distinct — shipped (release/…) title that signals the lifecycle. Lower already-shipped SRs stay retired. Get-ReleaseReadiness.ps1 gains a display-only -Shipped switch that surveys the SR's own branch.

(This was originally the separate stacked PR #36113. Because the live SR8 ship (10.0.80) made the two changes inseparable — #36111's E2E suite can only be green once it encodes the shipped-tracker behavior — #36113 was folded in here and closed. No force-push: it was applied as an additive commit.)

Review response (robust preview6 asserts + fix-phrase negation guard)

Addresses the multi-model review on this PR:

• Time-robust preview6 E2E asserts. The net11 preview6 asserts no longer hard-code a single lifecycle state. They read the detector's own branchExists and assert mode/surveyRef/issueTitle are consistent with it, so the suite stays green across the candidate → in-flight cut (which is what drifted CI when release/11.0.1xx-preview6 landed) instead of re-pinning to a snapshot that drifts again next transition.
• Negation guard on fix-phrase scoring. A negated maintainer comment (not fixed by #X, won't fix #Y, isn't resolved by #Z) was scored as high-confidence fix-phrase. A negative lookbehind now demotes a solely-negated reference to mention; -match backtracking still upgrades a PR that has a separate non-negated fix phrase. New unit tests exercise the real Get-IssueCommentPrs (mocking only its gh call).

Also in this PR — preview-lane merge-up hop is now a high-priority blocker

On an in-flight preview, the main → net<N>.0 merge-up hop (the codeflow that carries fixes from main into the active net<N>.0 development branch, which then merges into the preview branch) was rendered as ordinary informational context. But a preview cannot ship a fix that hasn't reached its own branch yet, so a pending merge-up is a real gate. Get-PreviewReadiness.ps1 now hoists that hop to the 🔴 High-priority section with a distinct 🔀 Merge-up PR (main → net<N>.0) label, and the label correctly names net<N>.0 (not main) as the upstream target. Concrete effect: PR #36085 (the live main → net11.0 merge) now surfaces as a High-priority blocker on the preview6 tracker instead of being buried.

Also in this PR — tracker auto-create label fix + net11 preview7 detection refresh

• Auto-create label bug. The release-readiness.yml workflow's create path applied --label area-release-readiness, a label that does not exist in the repo. gh issue create hard-fails (422) on an unknown label, so any scheduled first-time creation of a brand-new tracker would have aborted. The create path now uses the real area-infrastructure label and attaches all labels best-effort — each is probed via gh api repos/.../labels/<name> and skipped-with-warning if absent — mirroring how the workflow already handles a missing milestone. This makes new-tracker creation robust against label drift permanently. SKILL.md updated to match.
• net11 → preview7 detection snapshot. With net11.0 bumped to preview7, the live-detection E2E now legitimately emits two net11 preview trackers (preview6 in-flight + preview7 candidate). The hand-maintained detection snapshot was updated to select trackers by preview number (not array index) and assert each one's mode/surveyRef/issueTitle are consistent with its own branchExists, so it stays green across the next candidate → in-flight cut.

Also in this PR — semantic-hash now folds in tracker mode (resolves the in-flight→shipped blocker)

The shipped-SR lifecycle above had a latent ❌ that the multi-model review caught (carried over unfixed from #36113): Get-ReportSemanticHash hashed verdict / srHead / ci / srPrs / regressions / openSrPrs / shipChecks / nightlyFeed but not mode. Because -Shipped is a pure display relabel that surveys the same SR branch as in-flight, every hashed field can be byte-identical at the exact moment the stable tag publishes — so hash(shipped) == hash(in-flight), the daily workflow's idempotent no-op skips gh issue edit, and the tracker never visually flips to "shipped." The fix folds metadata.mode into the $semantic hash (StrictMode-safe ContainsKey access, defaulting absent → 'in-flight'). mode is constant within a mode, so it adds no daily churn — only the one-time lifecycle transitions refresh. Regression tests assert hash(shipped) != hash(in-flight) and hash(candidate) != hash(in-flight) on byte-identical content, plus mode-fold determinism and absent-mode back-compat.

Validation

• Live SR8 data: 5 of 6 flagged issues reclassify to closed-fix-unlinked with the correct fix PRs ([MAUI] Selected items are highlighted in orange and traces are preserved on Android #35252→[Android] [Candidate branch] Fix VerifySelectedItemClearsOnNullAssignment, CollectionViewSelectionShouldClear, SelectedItemVisualIsCleared UI test failure on Android  #34928; [MAUI] [Unstable] I7_Pull_to_Refresh - Number of items are increasing but the items displayed would not be incremented. #35253/[MAUI][Unstable] I8_Headers_and_Footers - Add/Clear buttons doesn't trigger the onClicked event handler #35254/[MAUI][Unstable] M4_Add/Delete - Adding or deleting any item doesn't work properly #35255/[unstable]Message does not appear in Messages Board after pressing Enter in Android Emulator #35409→Fix CI failure for CollectionView and CarouselView tests in April 14th Candidate #35028); by-design [MAUI] G4 - Back button glyph (FontImageSource) is stretched out #35291 correctly stays no-fix-yet.
• Offline suite green: 761/0, validated against live release state (SR8 shipped, preview6 in-flight, net11.0 bumped to preview7). New assertions cover the reclassification, the bare-mention/[MAUI] G4 - Back button glyph (FontImageSource) is stretched out #35291 guard, the not-on-branch guard, the OPEN-issue guard, the subject-token cross-branch path, the verdict-tier mapping, the Tier-3 markdown render, the shipped-tracker lifecycle, the branchExists-conditional preview6/preview7 invariants, the fix-phrase negation guard, the preview-lane merge-up hoist, the best-effort tracker-create labels, and the mode-folded semantic hash (hash(shipped) != hash(in-flight)).

Docs updated: SKILL.md and references/methodology.md classification tables.

This is a tooling/skill-only change — no product code, no public API.

Co-authored-by: Copilot 223556219+Copilot@users.noreply.github.com

[github-actions]

🚀 Dogfood this PR with:

⚠️ WARNING: Do not do this without first carefully reviewing the code of this PR to satisfy yourself it is safe.

curl -fsSL https://raw.githubusercontent.com/dotnet/maui/main/eng/scripts/get-maui-pr.sh | bash -s -- 36111

Or

• Run remotely in PowerShell:

iex "& { $(irm https://raw.githubusercontent.com/dotnet/maui/main/eng/scripts/get-maui-pr.ps1) } 36111"

[github-actions]

Skill Validation Results

@PureWeen — new skill validation results are available based on this last commit: 9d114d3.
To request a fresh validation after new comments or commits, comment /evaluate-skills.

✅ Skill Validation Results — 9d114d3 · Add closed-fix-unlinked regression classification to release-readiness · 2026-07-06T06:24:06Z

✅ Static Checks Passed

Skills: 20 | Eval specs linted: 7

Full lint output

── .github/skills/agentic-labeler/tests/eval.vally.yaml
npm warn deprecated prebuild-install@7.1.3: No longer maintained. Please contact the author of the relevant native addon; alternatives are available.
✔ .github/skills/agentic-labeler/tests/eval.vally.yaml is valid
── .github/skills/code-review/tests/eval.capability.vally.yaml
✔ .github/skills/code-review/tests/eval.capability.vally.yaml is valid
── .github/skills/code-review/tests/eval.vally.yaml
✔ .github/skills/code-review/tests/eval.vally.yaml is valid
── .github/skills/code-review/tests/hermeticity.vally.yaml
✔ .github/skills/code-review/tests/hermeticity.vally.yaml is valid
── .github/skills/evaluate-pr-tests/tests/eval.vally.yaml
✔ .github/skills/evaluate-pr-tests/tests/eval.vally.yaml is valid
── .github/skills/try-fix/tests/eval.vally.yaml
✔ .github/skills/try-fix/tests/eval.vally.yaml is valid
── .github/skills/verify-tests-fail-without-fix/tests/eval.vally.yaml
✔ .github/skills/verify-tests-fail-without-fix/tests/eval.vally.yaml is valid

⏭️ LLM Evaluation: Skipped

No changed skills with eval specs found.

🔍 Full results and investigation steps

[kubaflo]

Note

🤖 This review was generated by AI — an automated multi-model code review (Claude Opus 4.8 · GPT-5.5 · Gemini 3.1 Pro), independent reviews reconciled via cross-pollination.

---

🔍 AI Multi-Model Code Review — PR #36111

"Add closed-fix-unlinked regression classification to release-readiness" (base main, +279/−1).

🟡 Verdict: NEEDS_DISCUSSION (the code is sound; red CI is environmental)

Confidence: Medium — unanimous (all 3 models). CI classification: environmental (not PR-caused).

🔬 The red Validate (PR) is environmental — NOT caused by this PR

Validate (PR) reports Passed 716 / Failed 4. All 4 failures are pre-existing, time-sensitive "preview6" E2E asserts that this PR does not touch:

• preview6 mode = candidate, preview6 surveyRef = net11.0, preview6 issue title format, preview6 branchExists = false (no branch yet).

These assume no release/11.0.1xx-preview6 branch exists yet — but that branch has since been cut, so the "no branch yet" assumption is now stale and the candidate-mode asserts drift. The closed-fix-unlinked change is unrelated to preview detection, and the sibling stacked PR #36113 (same branch + more) passes the same suite. Recommendation: make these preview6 E2E asserts time-robust (don't hard-code "no branch yet") so CI goes green — ideally as a small follow-up, since they're pre-existing.

Per the code-review skill Rule #6, a red required check caps the verdict below LGTM even when the failures are environmental — hence NEEDS_DISCUSSION rather than LGTM. The code in this PR is sound.

✅ The closed-fix-unlinked classification is well-built

• Correctly identifies a closed regression whose fix PR is present on the SR branch but wasn't auto-linked, and routes it to an informational tier rather than a blocker.
• Has test coverage and clear docs (SKILL.md / methodology.md) matching the code.

💡 Non-blocking suggestions

1. Negated fix-phrases (inline @ Get-ReleaseReadiness.ps1:1838): not fixed by #X is scored as fix evidence. Bounded to Tier-3 (no blocker hidden), but worth a negation guard.
2. Test gaps: no direct unit test for the Test-PrNumberOnBranch squash-token grep, or the Test-PrIsToolingOnly rejection path in the closed-fix-unlinked fallback.
3. Get-IssueCommentPrs extraction regex also matches issue refs, so Get-PrInfo may be invoked on an issue number — harmless today, worth narrowing.

🏁 Recommendation

NEEDS_DISCUSSION → the code is approve-adjacent. No code changes are required by the classification itself; the only thing keeping it from green is the environmental preview6 CI drift (a pre-existing, time-sensitive assert that should be made robust). A maintainer can decide whether to fix those asserts here or in a follow-up.

---

Reviewed with the code-review skill: independence-first, full-file reading, local reproduction of the Validate (PR) failure to classify it as environmental, and cross-model reconciliation.

[kubaflo]

Note

🤖 This review was generated by AI — an automated multi-model code review (Claude Opus 4.8 · GPT-5.5 · Gemini 3.1 Pro).

---

🔍 AI Multi-Model Code Review — PR #36111 · Round 2

HEAD: 0d292b43 — now a combined PR: the closed-fix-unlinked classification + the refresh-most-recently-shipped-SR-tracker feature (folded in from the former #36113) + the R1 review fixes.

🔴 Verdict: NEEDS CHANGES

Confidence: High — unanimous (all 3 models).

✅ Two R1 items are resolved — nice

• Environmental CI fixed: Validate (PR) now passes — the preview6 E2E asserts were made robust (no longer hard-coding "no branch yet" after the preview6 branch was cut). 👍
• Negation guard added & correct: not fixed by #X is no longer scored as positive fix evidence (all 3 models confirmed negation_guard_correct: true). The R1 thread is resolved.

❌ Blocker — the #36113 semantic-hash bug came along unfixed (see inline @ Get-ReleaseReadiness.ps1:1352)

By folding the refresh-shipped-tracker feature into this PR, it inherits the confirmed ❌ error from the #36113 review, and all 3 models re-verified it is still unresolved: Get-ReportSemanticHash excludes mode, the -Shipped path surveys identically to in-flight, and the function isn't touched by this PR — so the daily workflow's hash-compare skips gh issue edit at the in-flight→shipped transition and the tracker never flips to "shipped." The headline behavior silently no-ops.

Fix: fold mode/the shipped tag into Get-ReportSemanticHash (or force the edit on the mode flip) + a regression test (hash(shipped) != hash(inflight) on identical content). Details inline.

💡 Non-blocking

• Negation guard residual (Opus): the guard catches not fixed by #X but misses an intervening adverb like not yet fixed by #X / still not resolved by #X. Minor — worth extending the negation lookbehind to span a short adverb.

🏁 Recommendation

REQUEST_CHANGES — one blocker: persist the in-flight→shipped flip (the semantic-hash fix + a guard test). The closed-fix-unlinked classification, the preview6 robustness, and the negation guard are all good; this is the same one-spot fix called out on #36113, which just needs to land here since the feature now lives in this PR.

---

Reviewed with the code-review skill: independence-first, full-file reading, semantic-hash control-flow re-verification (the prior confirmed error was re-checked against the combined branch by all 3 models).

[kubaflo]

Note

🤖 AI-generated approval (multi-model review automation, approving on @kubaflo's behalf).

✅ Approving — 3-model LGTM (Round 3). The last blocker is fixed.

This combined PR (closed-fix-unlinked classification + refresh-most-recently-shipped-SR-tracker) is now clean:

• ✅ The semantic-hash bug is fixed — mode is folded into Get-ReportSemanticHash, so an in-flight→shipped flip now changes the hash and the tracker actually refreshes to "shipped." All 3 models verified (semantic_hash_bug_fixed: true), including the one that originally found the bug. The default-when-absent is 'in-flight', so existing in-flight trackers don't mass re-churn on first run.
• ✅ A regression guard test was added — hash(in-flight) != hash(shipped) on identical content, plus a deterministic-recompute check (guard_test_present: true).
• ✅ Earlier rounds' items all resolved: preview6 E2E asserts made robust (Validate (PR) green), fix-phrase negation guard correct, closed-fix-unlinked classification sound.

One cosmetic non-blocking nit (Opus): Get-ReportSemanticHash's mode guard is -and $Data.metadata['mode'] (truthy) while Format-MarkdownReport uses ContainsKey only — a present-but-empty mode would hash as 'in-flight' but render as empty. Harmless in practice (mode is always set); align the two guards if you touch it.

Nicely iterated — from a real false-green class of bugs down to a clean, test-guarded gate.

[Copilot]

Pull request overview

This PR enhances the release-readiness tooling to better classify regressions and manage tracker lifecycles, primarily by (1) de-noising closed regressions whose fixes are only mentioned in human closing comments and (2) keeping the most-recently-shipped SR tracker refreshing until manually closed. It also improves preview readiness signaling around merge-up PRs, and hardens the workflow’s tracker creation behavior.

Changes:

• Add closed-fix-unlinked regression classification by extracting fix PR references from issue comment prose and verifying the fix is merged and present on the SR branch.
• Introduce/propagate a shipped tracker mode (refresh-only) for the most-recently-shipped SR, including workflow behavior and semantic-hash updates so the UI flips modes reliably.
• Improve preview readiness by hoisting main → net<N>.0 merge-up PRs into high-priority blockers and making detection/tests more time-robust.

Show a summary per file

File	Description
.github/workflows/release-readiness.yml	Adds refresh-only handling for mode=shipped and makes label attachment best-effort to avoid failing auto-create on missing labels.
.github/skills/release-readiness/tests/Test-ReleaseReadiness.ps1	Expands unit/E2E coverage for shipped tracker mode, mode-aware hashing/rendering, comment-based fix recovery, and preview merge-up hoisting.
.github/skills/release-readiness/SKILL.md	Documents -Shipped behavior, shipped tracker lifecycle, new classification, and updated label behavior.
.github/skills/release-readiness/scripts/Get-ReleaseReadiness.ps1	Implements -Shipped, adds comment-based PR extraction + branch-presence checks, adds closed-fix-unlinked, and folds mode into semantic hash.
.github/skills/release-readiness/scripts/Get-PreviewReadiness.ps1	Treats merge-up PRs in the full chain as high-priority blockers and improves labeling/text consistency for the merge-up chain.
.github/skills/release-readiness/scripts/Find-ReleaseReadinessTrackers.ps1	Emits mode='shipped' tracker for the highest shipped SR (refresh-until-closed), while retiring older shipped SRs.
.github/skills/release-readiness/references/methodology.md	Updates methodology to include closed-fix-unlinked classification semantics.

Copilot's findings

• Files reviewed: 7/7 changed files
• Comments generated: 1

[PureWeen]

🔬 Adversarial multi-model review — 3 reviewers × 2 rounds

Re-reviewed after the last push using independent reviewers (opus-4.8, gpt-5.5, opus-4.6) with adversarial consensus (verify factual claims against source; don't discard a finding on vote count alone). Any fixes were pushed to the branch; this is a COMMENT review only — approval is a human call.

Round 1 → fixed in 95fdff5077

#	Finding	Consensus	Action
A	closed-fix-unlinked fallback didn't skip reverted fix PRs	agree (2/3)	Fallback now skips $revertedPrSet members + Revert-titled PRs
C	Workflow validate job didn't cover -Shipped dispatch	verified	Added -Shipped branch to validate job
D	Cross-repo owner/repo#N / github.com/.../pull/N refs treated as maui#N	verified	Repo-aware extraction regex in Get-IssueCommentPrs
B	Non-adjacent negation window (fix-phrase guard)	disputed (1/3)	Doc-only reword; regex left as-is (bounded, documented)

Round 2 → fixed in 9d114d35af

gpt-5.5 returned NEEDS_CHANGES with two findings; both verified against live maui PRs and fixed:

• ❌ Manual reverts leaked a false green. #36152 ("Revert - Fix Android stale ContainerView root leak #35372") reverts #35372 but carries neither GitHub's quoted Revert "..." subject nor a This reverts commit <sha> body line, so Get-RevertedPrFromSubject returned null. revertedPrSet then held only the revert PR, the reverted fix's original commit still satisfied the on-branch gate, and a "fixed by #35372" comment on a closed issue was falsely de-noised. Added an anchored third pattern for the Revert - <title> #N (#M) form that recovers #N and never returns the revert's own (#M).
• ⚠️ Scheme-less cross-repo paths. dotnet/runtime/pull/N (no github.com host, no #) matched the bare pull/ alternative → mistaken for maui#N. Added a pathrepo owner/repo group before the bare pull/ alt and skip on a foreign repo; same-repo scheme-less paths still accepted.

opus-4.8 and opus-4.6 both returned LGTM in Round 2 (ran the suite 818/0, traced the Round-1 fixes, verified regex group numbering empirically).

Residuals (not fixed — bounded/known)

• 💡 Contrived cross-repo number-collision "fix-phrase bleed" — requires a same-number PR in another repo cited with fix language; very low likelihood, safe direction.
• 💡 Non-adjacent negation window — documented; bounded proximity window.

Verification

• Suite 824/0 locally (818 + 6 new regression tests: 4 revert-subject, 2 scheme-less cross-repo).
• Both edited files AST-parse clean.
• CI on 9d114d35af: Validate (PR) pass, Static validation pass, PR gate pass. maui-pr correctly skipping (tooling-only PR).

Verdict: all verified findings resolved. No outstanding ❌/⚠️. Deferring to a human for the actual approval.

[Copilot]

Copilot's findings

• Files reviewed: 7/7 changed files
• Comments generated: 1

[kubaflo]

Note

🔍 AI-generated review (automated multi-model orchestrator, on behalf of @kubaflo) — Opus 4.8 · GPT-5.5 · Gemini 3.1 Pro.

✅ Round 4 — APPROVED (new commit 9d114d35 reviewed)

New commit 9d114d35 ("Harden closed-fix-unlinked against manual reverts and scheme-less cross-repo refs") is a well-reasoned adversarial-hardening follow-up to the Tier-3 classifier. Two verified false-green paths closed:

1. Manual/hand-authored reverts (Revert - <title> #N (#M), no GitHub-quoted subject, no This reverts commit <sha> body) — Get-RevertedPrFromSubject returned null, so the reverted fix's original commit still satisfied the on-branch gate and a fixed by #N comment on a CLOSED issue was falsely de-noised. New anchored regex (?i)^(?:\[[^\]]+\]\s+)?Revert\b.*#(\d+)\s+\(#\d+\)\s*$ recovers #N while never returning the trailing (#M).
2. Scheme-less cross-repo paths (dotnet/runtime/pull/N, no host, no #) matched the bare pull/ alternative and were mistaken for maui#N. New pathrepo group ordered before the bare pull/ alternative, skipped when it names a foreign repo; same-repo scheme-less paths still accepted.

Verification

• Regex correctness confirmed. Greedy .* + end-anchor picks the trailing #N (#M) correctly; the char class excludes / so qual can't swallow a path; the pathrepo alternative precedes bare pull/ so the specific match wins. Both changes are safe-direction — a rare misfire only ever marks a PR reverted, never falsely shipped.
• 6 regression tests land exactly on the new paths: manual-revert recovery, [branch-prefix] Revert form, two safety negatives (non-revert #N (#M) → null; revert with only (#M) → null), scheme-less cross-repo reject + same-repo accept. Suite 824/0.
• CI green: Validate (PR) (Pester) ✅ 2m37s, PR gate ✅, Static validation ✅, Skill Validation ✅. evaluate jobs skipping (no eval entry for this skill). maui-pr skipping (by-design .github/**).

Verdict: Approve. Correct, safe-direction, and precisely tested. Re-approving on the refreshed head.