Release readiness skill — SR + Preview (supersedes #35754)

[PureWeen]

Note

Are you waiting for the changes in this PR to be merged?
It would be very helpful if you could test the resulting artifacts from this PR and let us know in a comment if this change resolves your issue. Thank you!

Summary

Adds a deterministic, evidence-backed release-readiness skill that produces a single "Is release/X.Y.Zxx-srN (or preview) ready to ship?" report for .NET MAUI release branches — both Servicing Releases (SR) and Previews, in both in-flight and candidate (pre-cut) modes.

Supersedes #35754.

What it does

Get-ReleaseReadiness.ps1 walks the SR branch, classifies open regressed-in-* issues against branch contents, computes the source-PR list (handling cherry-pick number swaps + non-main forward-flow), and rolls up an "is this ready to ship?" verdict with a Blocking summary hoisted to the top of the report.

Posts/refreshes a single [Release Readiness] GitHub tracking issue per release lane (idempotent via a semantic hash marker — only reposts when something meaningfully changed). See issue #35876 (SR8) for a live example.

Ship-readiness checks

A release captain sees these surface as 🟢 READY / 🟡 WATCH / 🔴 BLOCKED / ⚪ UNKNOWN rows. All BLOCKED rows roll up into the Blocking summary at the top.

Check	Catches
Versions.props bump	SR cycle hasn't been bumped on the SR branch
Versions.props servicing flip	PreReleaseVersionLabel=servicing + StabilizePackageVersion=true not applied — branch silently builds prerelease packages
Bug template lists SR version	Users can't file bugs against the new version
Main bumped to next SR cycle	Post-SR-cut PRs on main would falsely claim to ship in the SR being shipped
BAR default-channel mapping	SR branch not wired to .NET <band> SDK in BAR — caught the real SR8 outage
BAR build for SR HEAD	No published build at the SR HEAD commit
Milestone for current cycle	Fixed issues have nowhere to land
Milestone for next cycle	Open issues can't roll forward when current ships
Stale open milestones	Already-shipped releases accumulating untriaged issues (scoped to same major + same cycle type, 7-day grace)
CI Failure Scanner signals	Fresh ci-scan issues filed in the last 24h
Known Build Errors	Open KBE issues that may explain background CI noise

Each check that needs external tooling (darc, gh, milestone API) degrades to UNKNOWN with the exact verification command embedded — the report never silently skips.

Expected ship date

Header line surfaces the deadline. Cadence is patch-aware:

• PatchVersion ends in 0 (80, 90, 100…) or 0 (previews) → 2nd Tuesday of the month
• Anything else (81, 82, 91…) → ASAP hotfix, no cadence

Custom agent

.github/agents/release-readiness-agent.agent.md wraps the skill — handles regression-label confirmation, runs the script, then uses WorkIQ + maestro MCP to:

• Patch UNKNOWN BAR rows live (e.g. when darc isn't on CI's PATH)
• Add narrative context for rejected-from-sr PRs (chat history, review feedback)
• Present the final READY / Conditionally Ready / Not Ready verdict with citations

Testing

pwsh .github/skills/release-readiness/tests/Test-ReleaseReadiness.ps1
# 447 pass / 0 fail

Dogfooded live against SR7 + SR8 + the .NET 11 preview lane. Caught real-world bugs:

• SR8 missing from BAR default-channel mappings (verified via maestro_default_channels MCP)
• .NET 10 SR6 + .NET 10 SR7 milestones open with 76 + 63 open issues, past due
• .github/ISSUE_TEMPLATE/bug-report.yml missing 10.0.80 entry

Methodology gotchas (documented in references/methodology.md)

1. Cherry-pick number swap — SR backports get NEW PR numbers; can't naively grep source PR numbers
2. Timeline cross-references — closedByPullRequestsReferences returns empty for most MAUI issues; must walk gh api .../issues/N/timeline cross-referenced events
3. Forward-flow / non-main merges — a fix can merge into inflight/current only, not main (real example: PR [iOS / Mac] Fix CollectionView.ScrollTo(index) silently failing whenIsGrouped="True" #35609)

Files

• .github/skills/release-readiness/SKILL.md — skill entry point + reference docs
• .github/skills/release-readiness/scripts/Get-ReleaseReadiness.ps1 — main orchestrator (deterministic, no MCP)
• .github/skills/release-readiness/tests/Test-ReleaseReadiness.ps1 — 447-assertion test suite
• .github/skills/release-readiness/references/methodology.md — gotchas and patterns
• .github/agents/release-readiness-agent.agent.md — MCP-enriched agent wrapping the skill

[github-actions]

🚀 Dogfood this PR with:

⚠️ WARNING: Do not do this without first carefully reviewing the code of this PR to satisfy yourself it is safe.

curl -fsSL https://raw.githubusercontent.com/dotnet/maui/main/eng/scripts/get-maui-pr.sh | bash -s -- 35807

Or

• Run remotely in PowerShell:

iex "& { $(irm https://raw.githubusercontent.com/dotnet/maui/main/eng/scripts/get-maui-pr.ps1) } 35807"

[github-actions]

Skill Validation Results

@PureWeen — new skill validation results are available based on this last commit: bfc12a1.
To request a fresh validation after new comments or commits, comment /evaluate-skills.

✅ Skill Validation Results — bfc12a1 · Release readiness skill — SR + Preview (supersedes #35754) · 2026-06-17T02:39:14Z

✅ Static Checks Passed

Skills checked: 20 | Agents checked: 5

Full validator output

Found 1 skill(s)
[release-readiness] 📊 release-readiness: 5,530 BPE tokens [chars/4: 5,406] (comprehensive ✗), 40 sections, 5 code blocks
[release-readiness]    ⚠  Skill is 5,530 BPE tokens (chars/4 estimate: 5,406) — "comprehensive" skills hurt performance by 2.9pp on average. Consider splitting into 2–3 focused skills.
✅ All checks passed (1 skill(s))
Found 5 agent(s)
Validated 5 agent(s)
✅ All checks passed (5 agent(s))

⏭️ LLM Evaluation: Skipped

No changed skills with eval tests found.

🔍 Full results and investigation steps

[kubaflo]

/review -b feature/enhanced-reviewer

[MauiBot]

Expert Review — 8 findings

See inline comments for details.

[MauiBot]

[major] Logic and Correctness — False in-sr-active when GitHub API reports MERGED but the backport PR is absent from the git-derived sourcePrSet. The git log is the ground truth here; the GitHub state can lag a -NoFetch run or (rarely) reflect a merge to a different branch. Returning in-sr-active with the evidence string "SR contents may need refresh" tells the agent "fix is shipping" when the evidence is actually contradictory.

Suggested fix: downgrade to needs-human-review with confidence: 'low' for this branch:

} else {
    $verdict = 'needs-human-review'
    $evidence += "Backport PR #$($mergedBackport.number) marked merged by GitHub but NOT found in git-derived sourcePrSet — re-run without -NoFetch to confirm"
}

Contrast with line 581 (same block, different branch) which requires sourcePrSet membership before emitting in-sr-active.

[MauiBot]

[moderate] Logic and Correctness — CI verdict table documents red-known-flakes and red-new-failures (lines 99–100) but the script (Get-ReleaseReadiness.ps1 lines 350–352) never emits those values — it only ever emits red-needs-review. Agent consumers that key off red-known-flakes in the JSON will never match.

Either:

1. Remove red-known-flakes / red-new-failures from this table and document only red-needs-review, OR
2. Add the split logic to Get-CIStatus so the script actually emits distinct verdicts.

The script comment on line 352 (# downstream agent classifies known-flakes vs new) suggests option 1 is the intended design, in which case the two rows in this table should be collapsed to red-needs-review.

[MauiBot]

[moderate] Logic and Correctness — out-of-scope verdict is documented here and again in the E2E test assertion (Test-ReleaseReadiness.ps1 line 139: #35326 → out-of-scope) but Classify-RegressionCandidate never produces it. Issues that lack the relevant regressed-in-* label are simply never added to allIssues (they are filtered out by gh issue list --label), so the function is never called for them. The E2E test assertion on line 139 saying "#35326 → out-of-scope" will therefore never be validated by the test harness; the issue would just not appear in the results at all.

Options: (a) remove out-of-scope from the verdict table and update the test comment, or (b) add an explicit out-of-scope classification path (e.g. for issues that were pulled in via wildcard and don't match the label filter).

[MauiBot]

[moderate] Build & MSBuild / Cross-Platform — curl is invoked as a bare command on line 305. In PowerShell 7+, the curl → Invoke-WebRequest alias was removed, so this falls through to the system curl binary. On Windows runners (AzDO windows-latest or windows-2022) curl is in System32 since Windows 10 1803, but the AzDO Agent image may not guarantee it. More importantly, the AzDO public pipeline queryable here (dnceng-public) has open endpoints, so Invoke-RestMethod would be idiomatic, pipe-friendly, and not require parsing a JSON string out of raw text:

$obj = Invoke-RestMethod -Uri $url -TimeoutSec 30 -ErrorAction Stop

This also lets the catch block work with the native error record instead of checking $resp for null.

[MauiBot]

[minor] Logic and Correctness — Redundant equality fallback. git merge-base --is-ancestor X X exits 0 (every commit is an ancestor of itself), so the extra guard if (-not $isAtOrAhead -and $sourceSha -eq $Ctx.srHeadSha) is dead code. It can mislead a reader into thinking the equal case is NOT handled by --is-ancestor. Safe to remove; if kept, add a comment explaining why it is needed.

[MauiBot]

[minor] Logic and Correctness — confidence is hardcoded to 'high' for all non-empty verdicts, including needs-human-review. Emitting {classification: 'needs-human-review', confidence: 'high'} is self-contradictory and may mislead the downstream agent into treating the result as strongly evidenced. At minimum, needs-human-review verdicts (and the merged-non-main-only / merged-on-main-no-backport verdicts which depend on the ancestry check but have no backport) should carry confidence: 'medium'.

[MauiBot]

[minor] Complexity Reduction — Duplicate list number. This PR inserts 12. **release-readiness** at line 347 but the pre-existing 12. **try-fix** ("Internal Skills") remains at line 356, creating two 12. entries in the Reusable Skills list. The new release-readiness entry should be numbered 13. (or the entire internal-skills sub-list renumbered). This also causes the inline copilot system-prompt rendering (used for routing) to have an ambiguous duplicate key.

[MauiBot]

[minor] Build & MSBuild — /tmp/<branch-slug>-readiness is a hard-coded Unix path. On Windows CI agents pwsh maps /tmp to C:\Users\...\AppData\Local\Temp only when using certain builds; other environments will fail. Prefer $env:TEMP or PowerShell's cross-platform [System.IO.Path]::GetTempPath() in documentation examples:

-OutputDir "$(pwsh -c '[System.IO.Path]::GetTempPath()')/<branch-slug>-readiness"

(Same concern applies to the SKILL.md Quick Start examples.)

[kubaflo]

/review -b feature/regression-check

[kubaflo]

/review rerun

[kubaflo]

/review -b feature/enhanced-reviewer -p android

[MauiBot]

Expert Review — 7 findings

See inline comments for details.

[MauiBot]

[major] Release readiness — A reverted backport can still classify the source PR as in-sr-active. Get-CommitsForRevSpec adds both the backport PR and the original source PR to sourcePrs, but revert detection records the reverted backport PR number; this branch checks $sourcePrSet.ContainsKey($pr.number) first and only looks for that same source PR in $revertedPrSet. Scenario: source PR #35001 is backported as #36000 and #36000 is later reverted in the SR; #35001 remains in sourcePrSet and is reported active instead of reverted/needs review.

[MauiBot]

[major] Release readiness — -AllActiveMajors drops still-active servicing majors after main advances. Once origin/main moves from .NET 10 to .NET 11, the candidate -ge $mainMajor filter excludes net10.0, so daily tracker discovery stops producing .NET 10 SR trackers even though release/10.0.*-srN branches may still be active.

[MauiBot]

[moderate] Build/MSBuild — Early patch tags produce .NET <major>.0 SR1, but Get-MilestoneSortKey only parses .NET <major> SRn. As a result Compare-MauiMilestone (ConvertTo-Milestone 10.0.5) .NET 10 SR2 returns $null, so the earlier-milestone preservation guard in Fix-MilestoneDrift.ps1 can be bypassed for SR1 patch tags. Normalize the producer output or accept the .0 SR form in the sort key parser, and add the regression test from the try-fix.

[MauiBot]

[moderate] Release readiness — Candidate-mode label inference always fails here. Resolve-Context sets ctx.srBranch to the main branch in -Candidate mode and stores the real prior SR in ctx.priorSrBranch, but this parser only accepts release/...-srN; the workflow falls back to -InferRegressionLabels when tracker labels are empty, so candidate SR tracker runs skip the regressions phase instead of inferring labels from the prior SR.

[MauiBot]

[moderate] Build/MSBuild — This validation regex no longer matches the full GitHub issue URL form that Get-LinkedIssues accepts (Fixes https://github.com/dotnet/maui/issues/N). A PR that fixed an issue using the URL form can be found by the initial linked-issue logic but later fail Test-MilestoneValidForIssue, allowing a valid earlier milestone to be overwritten. Reuse the same URL-aware fixing-reference pattern here.

[MauiBot]

[moderate] Release readiness — The comment says issue templates are global and should be read from main, but this chooses net<major>.0. GitHub issue creation uses the default branch template, so preview readiness can report READY/BLOCKED based on a version dropdown that users never see. Read bug-report.yml from main (or the repo default branch) for this check.

[kubaflo]

/review -b feature/enhanced-reviewer -p android

[PureWeen]

Adversarial Multi-Model Review (3 reviewers, post-5ab551842a)

Methodology: 3 independent reviewers with adversarial consensus. Findings below were verified against source after initial flagging. Prior MauiBot review threads remain valid; this pass focuses on new findings in commits landed after the most recent MauiBot review (2026-06-13).

Summary of new findings

#	Severity	Area	Finding
1	❌	Logic	WATCH ship-checks never escalate the headline verdict, but the wiring comment says they do — and the Preview script does the opposite
2	⚠️	Error Handling	gh API failures (rate limit / outage / auth) silently become 🟢 READY for ci-scan + KBE
3	⚠️	Error Handling	gh pr list failure in Get-CandidatePrChecks is reported as "no Candidate PR found"
4	⚠️	Security	Any open PR on main with the word "Candidate" in its title spoofs the SR cut PR check
5	⚠️	Logic	In candidate mode, the cycle-after-next milestone (e.g. SR10 while surveying SR9-candidate) triggers BLOCKED → 🔴 Not Ready
6	⚠️	Logic	Internal dotnet-maui AzDO pipeline (dnceng/internal) is queried with no auth header → always 401 → silent fallback to unknown
7	⚠️	Testing	Newest helpers are NOT exercised — tests mirror-implement the regex/logic locally, so a bug in the real code leaves 493 tests green

Suggestions (lower severity)

• 💡 Get-ReleaseReadiness.ps1:480 — Get-MainBumpDateForCycle uses [DateTime]::Parse($parts[0]) without InvariantCulture. Inconsistent with the file's own ConvertTo-Utc helper (which exists specifically to avoid this). Low risk for ISO-8601, but feeds ship-date math. (Reviewer 1)
• 💡 release-readiness.yml:297 — gh api ... --jq ".[] | select(.title == \"$MILESTONE_NAME\") | .number" interpolates the milestone name into the jq filter. Today's milestone names (.NET 10 SR8) are quote-free so it's safe in practice, but the pattern is fragile. Consider piping through jq --arg m "$MILESTONE_NAME" '... select(.title == $m)'. (Reviewer 3)
• 💡 Find-ReleaseReadinessTrackers.ps1:446 + release-readiness.yml:170,369 — New-RegressionLabelList unconditionally returns ≥1 label, so the workflow's else REG_LABEL_ARG=(-InferRegressionLabels) branch is unreachable dead code. If the team ever adopts non-canonical patch labels (e.g. regressed-in-10.0.74), inference will silently never run. (Reviewer 3)
• 💡 Get-ReleaseReadiness.ps1:1651 — gh pr view --json files does not paginate; if a PR ever has >100 changed files, Test-PrIsToolingOnly sees only the first 100 and may misclassify a real product fix as tooling-only. Low practical impact (PRs >100 files are rare). Failing safe would be: if returned count == 100, return $false. (Reviewer 3)

User-suggested follow-up (not a defect)

The **Branch**: <name> body-marker parsing in Get-CiScanIssueBranch + Get-CiScanIssuesForSr is functionally correct but architecturally redundant — the label name itself already encodes the branch (ci-scan = main, ci-scan-net11 = net11.0, ci-scan-net12 = net12.0). A simpler design: branch → expected label, query that one label, skip body parsing. Leaving as-is also works.

Where this PR is solid (verified)

• Workflow permission surface — top-level contents: read, PR validate job is read-only, uses pull_request (not pull_request_target) so fork-modified scripts run with read-only token and no secrets.
• Invoke-Gh correctly checks $LASTEXITCODE and returns $null on failure (the false-green issues above are at the call sites, not in Invoke-Gh itself).
• Strict-mode discipline is strong: existence-guarded PSObject.Properties['x'], ,@() unary-comma returns to defeat array unwrap, [int] normalization at extraction.
• Resolve-Context correctly swaps srBranch and priorSrBranch semantics between candidate and in-flight modes — the new Get-CandidatePrChecks and milestone hygiene code use the right field.
• Test-PrIsToolingOnly false-positive guard against agent/skill PRs that mention regression issue numbers in their body for documentation purposes.

[PureWeen]

❌ Logic — The comment here promises that ci-scan signals "can escalate the verdict (fresh ci-scan → WATCH)", but Get-OverallVerdict at line 2516 only escalates the headline 🔴/🟡/🟢 when Status -eq 'BLOCKED'. WATCH ship-checks never move the verdict. Every signal these recent commits added is WATCH at most: Get-CiSignalChecks ci-scan (2342/2348), KBE (2367), Get-CandidatePrChecks (1148/1160), and the Maestro "no BAR build for HEAD" check.

Cross-script inconsistency: Get-PreviewReadiness.ps1's $StatusRank ranks WATCH = 1 above READY = 0 and Get-OverallStatus is worst-wins — so an identical signal degrades the Preview headline but stays 🟢 Ready on an SR.

Concrete trigger: In candidate mode the ci-scan scanner does match (ctx.srBranch = main / net11.0, which is what ci-status-net11 tags). A ci-scan-net11 issue filed 3 hours before the nightly run emits a WATCH check ("filed in the last 24h… Likely affects this SR"). The author's wiring comment says this escalates to WATCH; Get-OverallVerdict leaves it 🟢. Release captain reading only the headline ships without seeing the fresh CI regression the feature was built to surface.

Fix options: (a) make Get-OverallVerdict treat WATCH ship-checks as Tier-2 (🟡), matching Preview's worst-wins model and the comment; or (b) if WATCH-is-non-escalating is the deliberate design, fix the comment (drop "can escalate the verdict") and reconcile with the Preview script so the same signal yields the same operator-facing verdict in both.

Flagged by: 1/3 reviewers, verified against source.

[PureWeen]

⚠️ Error Handling — Get-CiScanLabels (and the parallel Get-ReleaseRelevantIssuesByLabel at 2198) return @() on gh failure with no out-of-band signal. Then Get-CiSignalChecks at lines 2353-2363 / 2372-2378 reports 🟢 READY: "No open ci-scan issues — scanner has not flagged recurring CI failures." / "No open Known Build Error issues found."

Concrete trigger: GitHub API rate limit / brief outage / token scope issue → gh label list exits non-zero → Invoke-Gh warns to stderr and returns $null → the helper returns @() → the report says 🟢 READY for both ci-scan and KBE. A release captain ships without seeing active blockers because the survey couldn't reach GitHub.

Fix: distinguish API failure from genuinely-empty results. Either return a structured @{Success=$true; Data=@()} envelope (like Invoke-DarcJson already does in this file) or emit an UNKNOWN New-ReadinessCheck with the retry command in NextAction so the captain sees "⚪ Could not query ci-scan labels" instead of false-green.

Flagged by: 1/3 reviewers, verified against source (see Invoke-Gh at line 190 and call sites 2235, 2198, 2353-2378).

[PureWeen]

⚠️ Error Handling — Same false-green pattern as the ci-scan/KBE checks: if gh pr list fails (rate limit, outage), $mainPrs stays @() and line 1147-1150 emits a WATCH with text "No open Candidate PR found". A release captain reading this in candidate mode could conclude the team simply hasn't opened one yet, when in reality the survey couldn't reach GitHub.

Fix: distinguish $raw -eq $null (gh failure) from $parsed.Count -eq 0 (genuinely empty). On failure, emit a separate UNKNOWN New-ReadinessCheck with NextAction = 'Retry: gh pr list --repo $($Ctx.repo) --base $($Ctx.mainBranch) --search Candidate' so the captain sees the survey was incomplete rather than a misleading WATCH.

Flagged by: 1/3 reviewers, verified against source.

[PureWeen]

⚠️ Security — The Candidate PR detector matches any open PR on main whose title contains the word "Candidate". The match is (?i)\bcandidate\b with no author/branch/label gating. Any contributor — including a fork PR author — can open a PR titled e.g. "Candidate refactor of foo" and have it surfaced as the SR cut PR in the next readiness report.

Real-world impact is bounded (WATCH severity only, doesn't escalate verdict, release captain reviews before cutting), but the report links to the spoofed PR by number from the official tracker issue — which can be embarrassing or confusing.

Fix: require at least one trusted signal — author membership in the dotnet/maui team, a specific label (e.g. release-candidate), or a strict title format the team controls (e.g. ^\[Candidate\] rather than \bcandidate\b). Document the chosen convention in the SKILL.md so the release captain knows how to mark a PR as the canonical Candidate.

Flagged by: 1/3 reviewers, verified against source.

[PureWeen]

⚠️ Logic — In candidate mode, this check escalates to 🔴 Not Ready when the cycle-after-next milestone is missing. With priorSrBranch = release/10.0.1xx-sr8, cycleNum++ yields current = SR9, next = SR10. Check 2 here BLOCKED-escalates if SR10 doesn't exist; BLOCKED ship-checks flow into Get-OverallVerdict Tier 1 → 🔴 Not Ready (line 2516-2519).

Concrete trigger: Early in the SR8→SR9 candidate window, the team has created SR9 milestone but not yet SR10 (the cycle two ahead — normal this early). The candidate tracker reports 🔴 Not Ready purely because a milestone for the SR-after-the-one-being-cut is absent.

The check's own rationale (line 1039: "Once SR9 ships, open issues will have nowhere to roll forward to") describes the in-flight shipping scenario; in candidate mode SR9 hasn't even been cut yet, so requiring SR10 is premature. Check 1 (current SR9 milestone missing → BLOCKED) is reasonable in both modes; only Check 2 is premature in candidate mode.

Fix: in candidate mode, downgrade the missing next-cycle milestone from BLOCKED to CLEANUP (or WATCH) so it surfaces without flipping the headline to Not Ready before the SR is even cut.

Needs human confirmation on MAUI's milestone pre-creation cadence — if SR10 is genuinely expected to exist before SR9 is cut, this is by design.

Flagged by: 1/3 reviewers, verified against source.

[PureWeen]

⚠️ Logic — Get-PipelineLatestBuilds uses Invoke-RestMethod with no -Headers. This works for the public dnceng-public/public pipelines (anonymous read), but $Script:InternalPipelines (line 173) lists dotnet-maui on dnceng/internal which requires AzDO authentication. The call will always 401 → catch block at 1479-1482 returns $null → Get-CIStatus at 1494-1500 emits verdict: 'unknown' with note "Could not query (auth or outage)".

That tips the CI overall into partial-unknown / unknown, which Get-OverallVerdict (line 2503-2506) escalates to Tier 2 (🟡 Conditionally Ready) every single run. So the SR can never report 🟢 Ready while this pipeline is in the list.

The Preview script acknowledges this scenario explicitly (Get-PreviewReadiness.ps1:1107: "Internal dnceng pipeline details are not queried in public workflow mode.") and gates on -IncludeInternal. The SR script does not.

Fix options: (a) skip the internal pipeline when no AzDO token is available (env-var check); (b) plumb a token via $env:SYSTEM_ACCESSTOKEN and conditionally set Authorization = "Bearer $token"; (c) shell out to az pipelines runs list which handles token lifecycle automatically. (a) is the simplest fix consistent with the Preview script's approach.

Flagged by: 1/3 reviewers, verified against source (pipeline at line 173, query at line 1475, escalation at 2503).

[PureWeen]

⚠️ Testing — "493 passing tests" overstates coverage of the most-recently-added code. The harness proves it can exercise real functions with mocks (Invoke-ShipChecksWithMockedVersions at ~1622 calls the real Get-ReleaseShipChecks; Invoke-MaestroChecksWithMocks at ~2034 overrides global:Invoke-DarcJson to call the real Get-MaestroOperationalChecks). Yet the helpers introduced by the newest commits are re-implemented inline in the test rather than invoked:

• ci-scan label regex: test defines a local $labelRegex = '^ci-scan(-|$)' here and asserts against the string — it never calls the production Get-CiScanLabels.
• next-SR label: test defines a local Get-NextSrLabel re-implementing the sr(\d+)$ regex (~line 1939) — it never calls the production Get-CandidatePrChecks.
• Get-CiScanIssuesForSr, Get-CiSignalChecks, Get-MainBumpDateForCycle (SR) and Get-CiScanIssues (Preview): zero test references.

A regression inside any of them (flipping ^ci-scan(-|$) to ^ci-scan- and losing main, or breaking the +/+++ diff filter in git log -S) leaves all 493 tests green because the tests assert against copies of the patterns, not the functions.

Fix: dot-source and invoke the real function with mocked Invoke-Gh / Invoke-Git (the existing 1622/2034 mock pattern is directly reusable), and delete the local $labelRegex / Get-NextSrLabel copies so the regex lives in exactly one place — the production code.

Flagged by: 2/3 reviewers (Reviewer 1 on mirror-impls, Reviewer 3 on coverage gaps for Get-MainBumpDateForCycle/Get-RegressionLabelsAuto/Test-CiScanIsFresh).

[kubaflo]

🤖 Multi-model code review — Request changes

Three models reviewed this PR independently (Claude Opus 4.8, GPT-5.5, Gemini 3.1 Pro), then cross-pollinated. (~10k lines, so the reviews prioritized security, workflow safety, and test execution over exhaustive line-by-line.)

Verdict: NEEDS_CHANGES (light) — the security and architecture are excellent; the asks are test-robustness fixes, not safety issues.

This is well-engineered 👍 (all three models agree)

• No injection. gh/git are invoked exclusively via argument arrays (& gh @GhArgs, & git -C $Repo @ArgList), so untrusted GitHub/AzDO data (PR titles, issue bodies, branch names) has no command-injection sink; the lone & $var is a local scriptblock formatter.
• Safe workflow. release-readiness.yml uses pull_request (not pull_request_target), keeps top-level permissions: contents: read, gates the only issues: write job to non-PR events, and routes untrusted matrix values through env: instead of interpolating into run:.
• Scripts launched via pwsh -NoProfile -File (the repo's recommended pattern), and the test suite is actually wired into CI.

Test-robustness (the asks)

1. CI will rot (release-readiness.yml:466) — the hard gate runs the E2E live known-answer suite (highestShippedTag == '10.0.71', tracker count == 4, SR8/SR9 specifics). These flip to failure as the repo evolves, reddening the validate job on every future PR that touches the skill (plus network flakiness). Make the deterministic offline unit tests the gate; put E2E behind -SkipE2E / opt-in.
2. return bypasses the final exit (Test-ReleaseReadiness.ps1:148) — a thrown E2E invocation records the failure but the top-level return skips the final exit 1, so CI can go green on that catastrophic path. Fall through to the final exit (or exit 1).
3. Two low nits: Invoke-Git splits args on spaces (mis-tokenizes paths with spaces; use [string[]]); one --jq filter interpolates $MILESTONE_NAME (jq-only, fails safe; prefer jq --arg).

_{Independent verdicts: GPT-5.5 — NEEDS_CHANGES (med-high) · Opus 4.8 — NEEDS_DISCUSSION (med) · Gemini 3.1 Pro — LGTM (high). All three found the security/architecture clean; the two non-LGTM verdicts converge on the E2E-in-CI test design.}

[kubaflo]

🤖 Re-review — down to one item

Re-reviewed at the final head (all five hardening rounds in). This is in great shape — the security and architecture were already clean, and rounds 2–5 (stable hash, notes-preservation/awk-anchoring, UTF-8 truncation, Preview hash freeze, concurrency) closed the substantive concerns. I'm withdrawing everything from my earlier review except one item, so this is essentially ready.

The one remaining ask (a real CI false-green)

Test-ReleaseReadiness.ps1:148 — return bypasses the final exit. When the Get-ReleaseReadiness.ps1 invocation raises a PowerShell-level error, the catch increments $script:failed then returns, which skips the terminal exit $(if ($script:failed) {1} else {0}) — so the test runner can exit 0 even though the script under test crashed. It's a narrow trigger (a parent-level throw, not a child non-zero exit), but it defeats the gate in exactly the catastrophic case. One-line fix: exit 1 instead of return (you can't fall through — the lines below assume $outDir exists). See inline.

Downgraded to non-blocking 👍

• Live known-answer E2E as the validate gate (highestShippedTag == '10.0.71', SR7 #35428/#35609, counts): I flagged this as rot-prone, but it's a deliberate, valuable signal and you clearly want live coverage — treating it as a non-blocking suggestion (consider -SkipE2E deterministic tests as the hard gate with E2E opt-in, so repo evolution doesn't red unrelated PRs). The two earlier low nits (Invoke-Git space-splitting; one --jq $MILESTONE_NAME interpolation) are fail-safe — not blocking.

@PureWeen — fix that one return and I'm good here. Thanks for the very thorough hardening passes. 🙏

_{Re-review by the 3-model panel (Opus 4.8 · GPT-5.5 · Gemini 3.1 Pro); the security/architecture remain clean across all three.}

[kubaflo]

🤖 Re-review — round 3 (head 6ae8cae)

Reviewed the new commit (portfolio / cross-release discovery).

New feature 👍

The portfolio mode is a clean, doc-only addition — enumerating active releases by reading the [Release Readiness] tracker issues (and correctly keying off the body marker <!-- release-readiness-tracker: rather than the title, which would also match the Epic) is a nice "always-on status board" that avoids a 60–120s survey re-run. The Release-Captain-Notes override path is well-documented. No concerns here.

Still open (carried from round 2)

One real item remains — this commit didn't touch it: Test-ReleaseReadiness.ps1:148 — return bypasses the final exit. The catch does $script:failed++ then return, which skips the terminal exit $(if ($script:failed) {1} else {0}), so a PowerShell-level failure of the Get-ReleaseReadiness.ps1 invocation can still exit 0 (CI green) even though the script under test crashed. One-line fix: exit 1 instead of return.

@PureWeen — just that one line and I'm good. Thanks!

_{3-model panel basis (Opus 4.8 · GPT-5.5 · Gemini 3.1 Pro); this round was a light re-review of the doc-only feature delta.}

All review feedback addressed — the E2E catch now exit 1s instead of return (commit bfc12a1), exactly as suggested; the E2E-as-CI-gate concern and the minor nits were previously downgraded to non-blocking. No outstanding asks from this automated multi-model review (LGTM). Dismissing this now-stale changes-request; final approval/merge intentionally deferred to a human maintainer.