Fix release-readiness tracker warnings: filter cross-repo PR cross-references and drop invalid authorAssociation gh field - #36029
Conversation
Get-IssueTimelinePrs collected cross-referenced PR numbers without checking the source repository, so PRs from forks/unrelated repos (e.g. #24 in a fork, #102 in an unrelated project) were looked up against dotnet/maui — 404ing (surfacing a 'gh pr view' warning embedded in the tracker issue) or silently matching an unrelated same-numbered PR. Filter cross-references to $Repo. Get-CandidatePrChecks queried 'gh pr list --json ...,authorAssociation' but authorAssociation is not a valid gh pr list field, so the query failed (warning in the tracker) and the maintainer spoof-gate degraded. Fetch author_association per title-matched candidate via 'gh api repos/{repo}/pulls/{n}' instead. Adds 8 deterministic gh-stubbed regression assertions for both fixes. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
|
🚀 Dogfood this PR with:
curl -fsSL https://raw.githubusercontent.com/dotnet/maui/main/eng/scripts/get-maui-pr.sh | bash -s -- 36029Or
iex "& { $(irm https://raw.githubusercontent.com/dotnet/maui/main/eng/scripts/get-maui-pr.ps1) } 36029" |
Skill Validation Results
✅ Skill Validation Results —
|
…n candidate gate
Adversarial multi-model review of the candidate spoof-gate surfaced a real
defect (3/3 consensus after a follow-up round): the per-title-match REST
author_association lookup added in the prior commit fails closed correctly,
but on a *transient* failure (rate-limit, 5xx, network blip, or a race where
the PR is transferred between the gh pr list and the gh api call) it produced
three misleading downstream effects for a LEGITIMATE maintainer Candidate PR:
1. The candidate vanished from the report (count -> 0).
2. The WATCH detail asserted it was a 'non-maintainer PR titled Candidate
excluded as not real cut PR' — mislabeling a transient API failure as a
CONFIRMED spoofer, which would tell a release captain their own PR isn't
from a maintainer during an actual SR cut.
3. Invoke-Gh's non-zero-exit Write-Warn embedded a raw 'gh ... exited'
line into $Script:Warnings, which is rendered into the tracker issue
body — reintroducing the exact warning noise this PR set out to remove.
Fix (security posture unchanged — still fail-closed; a spoof PR is never
accepted): replace the Where-Object gate with an explicit loop that classifies
each title match as maintainer / confirmed-spoofer / unverifiable, and report
unverifiable PRs honestly ('could not have their author association verified —
rerun to re-check') instead of as spoofers. Add a -Quiet switch to Invoke-Gh
so the transient association lookup doesn't leak a gh warning into the tracker
body; the structured WATCH note carries that signal instead.
Adds a deterministic offline regression test for the unverifiable path and a
guard asserting confirmed spoofers are not reported as unverifiable. Offline
suite: 522 passed / 0 failed; full E2E: 588 / 0.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Round-2 adversarial review follow-ups (both 💡, low-risk, no behavior regression): - Get-CandidatePrChecks: when a valid maintainer Candidate PR is accepted but a *second* title-matched PR is a confirmed spoofer or an unverifiable (transient gh REST failure) lookup, the accepted check now surfaces that excluded sibling in its Details and, for unverifiable cases, appends a "rerun to re-check" hint to NextAction. Previously those counts were silently dropped on the accepted path, so a transient blip on a co-existing Candidate PR was invisible to the release captain. Security posture unchanged — still fail-closed; only OWNER/MEMBER/COLLABORATOR are accepted. - Tests: add case (d) (accepted maintainer PR + co-existing unverifiable sibling must be surfaced) and a dedicated Invoke-Gh -Quiet test that exercises the REAL Invoke-Gh against a simulated failing gh to prove the warning-suppression contract directly (warns without -Quiet, silent with it) — the classification tests shadow Invoke-Gh and cannot observe it. Reword case (c)'s comment so it no longer claims to verify suppression. Offline suite: 530 passed / 0 failed. Full E2E: 596 passed / 0 failed. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
MauiBot
left a comment
There was a problem hiding this comment.
Expert Review — 1 findings
See inline comments for details.
| # matches an unrelated $Repo PR that happens to share the number. Filter | ||
| # to $Repo. The timeline API populates `repository.full_name` for both | ||
| # same-repo and cross-repo references, so this is reliable. | ||
| if (-not $iss.PSObject.Properties['repository']) { continue } |
There was a problem hiding this comment.
[major] GitHub API shape handling — This drops every cross-referenced PR unless the timeline payload contains source.issue.repository.full_name. The REST timeline issue payloads also expose repository_url, which identifies the source repo. Accept both repository.full_name and repository_url and add a repository_url-only same-repo regression fixture so same-repo PR references are not silently skipped if the nested repository object is absent.
MauiBot
left a comment
There was a problem hiding this comment.
AI Review Summary
@PureWeen — new AI review results are available based on this last commit:
1a5a2c0. To request a fresh review after new comments or commits, comment/review rerun.
Review Sessions — click to expand
Gate — Test Before & After Fix
Gate Result: ⚠️ SKIPPED
No tests were detected in this PR.
Recommendation: Add tests to verify the fix using the write-tests-agent.
Pre-Flight — Context & Validation
Issue: #36029 - PR-only infrastructure fix (no linked issue)
PR: #36029 - Fix release-readiness tracker warnings: filter cross-repo PR cross-references and drop invalid authorAssociation gh field
Platforms Affected: infrastructure / release-readiness tooling (Windows PowerShell validation)
Files Changed: 1 implementation, 1 test
Key Findings
- PR fixes two release-readiness tracker warning sources: foreign repository timeline cross-references being treated as dotnet/maui PR candidates, and an invalid
gh pr list --json authorAssociationprojection. - The implementation filters timeline PR references by
repository.full_namebefore collecting candidate PR numbers, removes the invalid list projection, and fetches RESTauthor_associationfor title-matched Candidate PRs. - Gate was already completed separately and skipped because no tests were detected by the gate script; the PR does include offline PowerShell regression assertions in
.github/skills/release-readiness/tests/Test-ReleaseReadiness.ps1. - GitHub CLI is unauthenticated in this environment, so PR checks and authenticated review surfaces could not be queried via
gh; public REST data and the local merge checkout were used instead.
Code Review Summary
Verdict: NEEDS_DISCUSSION
Confidence: low
Errors: 0 | Warnings: 0 | Suggestions: 0
Key code review findings:
- No actionable ❌ Error or
⚠️ Warning findings were identified by the independent code-review sub-agent. - CI status is undetermined because
gh pr checks --requiredrequires authentication in this environment; confidence is capped low. - Failure-mode probes confirmed the PR handles foreign low-number PRs, foreign same-number PRs, contributor Candidate PR spoofing, unverifiable maintainer Candidate PRs, and valid Candidate PRs with unverifiable siblings.
Fix Candidates
| # | Source | Approach | Test Result | Files Changed | Notes |
|---|---|---|---|---|---|
| PR | PR #36029 | Filter timeline cross-references by repository.full_name; fetch author_association per Candidate-titled PR using REST; suppress handled lookup warnings via Invoke-Gh -Quiet. |
.github/skills/release-readiness/scripts/Get-ReleaseReadiness.ps1, .github/skills/release-readiness/tests/Test-ReleaseReadiness.ps1 |
Original PR; gate skipped by prior step. |
Code Review — Deep Analysis
Code Review — PR #36029
Independent Assessment
What this changes: Release-readiness tooling now filters timeline cross-referenced PRs to the configured repo, removes unsupported authorAssociation from gh pr list --json, fetches REST author_association only for Candidate-titled PRs, and adds offline regression tests.
Inferred motivation: Prevent tracker-body warning spam and false fix-candidate detection from foreign PR references or invalid gh projections.
Reconciliation with PR Narrative
Author claims: Fixes cross-repo timeline leakage and invalid Candidate PR author-association lookup; adds deterministic tests.
Agreement/disagreement: Agree. The implementation matches the stated defects and tests the relevant failure modes.
Prior Review Reconciliation
| Prior ❌ Error Finding | Source | Status | Evidence |
|---|---|---|---|
| None found | MauiBot review, inline comments API, issue comments API | — | MauiBot had no ❌ findings; inline comments were empty; issue comments had dogfood + skill-validation only. |
gh is unauthenticated, so public REST endpoints and the local merge checkout were used. No GitHub comments were posted.
Blast Radius Assessment
- Runs for all instances: No — release-readiness script only.
- Startup impact: None.
- Static/shared state: Existing
$Script:Warnings; new-Quietsuppresses warnings only at selected handled lookup sites.
CI Status
- Required-check result: undetermined.
- Classification:
gh pr checks 36029 --requiredcould not run becausegh auth loginis required. Public status for head1a5a2c0showsskill-validationsuccess, but required checks could not be confirmed. - Action taken: confidence capped low; no posting.
Local validation observed by the sub-agent: Test-ReleaseReadiness.ps1 -SkipE2E = 528 passed / 2 failed. Baseline parent also fails the same two truncation-marker assertions, so failures appear pre-existing/unrelated.
Findings
No actionable ❌ Error or
Failure-Mode Probing
- Foreign timeline PR with low/nonexistent dotnet/maui number: filtered by
repository.full_namebeforeGet-PrInfo. - Foreign PR number exists in dotnet/maui: also filtered, preventing silent wrong-PR match.
- Contributor opens “Candidate” PR: excluded via REST author association.
- Maintainer Candidate association lookup fails: excluded fail-closed and surfaced as unverifiable with rerun guidance.
- Valid Candidate plus unverifiable sibling: valid PR remains reported; sibling is surfaced.
Verdict: NEEDS_DISCUSSION
Confidence: low
Summary: Code is targeted and appears sound, with relevant regression coverage. LGTM cannot be used because required CI status is undetermined due unauthenticated gh.
Fix — Analysis & Comparison
Fix Candidates
| # | Source | Approach | Test Result | Files Changed | Notes |
|---|---|---|---|---|---|
| 1 | try-fix-1 | REST pulls-list for Candidate PRs plus canonical timeline pull_request.url filtering. |
❌ FAILED | 2 files | Failed under PowerShell StrictMode because optional author_association was accessed directly when absent. |
| 2 | try-fix-2 | GraphQL-first: timeline CrossReferencedEvent.source.repository.nameWithOwner filtering plus Candidate PR PullRequest.authorAssociation in one GraphQL query. |
2 files | PR-specific assertions passed; full suite retained known unrelated 528/2 baseline failures. More complex than PR fix. | |
| PR | PR #36029 | Filter REST timeline cross-references by repository.full_name; remove invalid gh pr list projection; fetch author_association only for Candidate-titled PRs via REST with quiet handled failures. |
2 files | Original PR. Gate was skipped by prior step; code-review found no actionable code issues. |
Cross-Pollination
| Model | Round | New Ideas? | Details |
|---|---|---|---|
| gpt-5.5 / maui-expert-reviewer | 2 | Yes | Proposed GraphQL-first approach after try-fix-1 StrictMode failure. |
| gpt-5.5 / maui-expert-reviewer | 3 | No | No meaningfully different third approach remains; other options are field/fallback variants or less authoritative/brittle mechanisms. |
Exhausted: Yes
Selected Fix: PR #36029 — The best alternative (GraphQL-first) is broader and more complex, while the PR fix is smaller, targeted, authoritative, and avoids the invalid gh pr list field without changing the data-access architecture. No candidate passed all tests and was demonstrably better than the PR fix.
Recommended PR Title & Description
Assessment: ✏️ Recommend updating — the provided current title is unknown and the current description is empty, so the PR metadata does not describe the actual release-readiness tooling fix.
Recommended title
[Infrastructure] Release Readiness: Filter cross-repo PR references and fix Candidate author lookup
Recommended description
### Description of Change
Fixes two sources of warning noise and candidate misclassification in the release-readiness tracker tooling:
- `Get-IssueTimelinePrs` now keeps only timeline cross-referenced PRs from the target repo, preventing foreign PRs from being looked up as dotnet/maui fix candidates. This avoids both noisy 404 warnings for low foreign PR numbers and silent same-number mismatches against unrelated dotnet/maui PRs.
- `Get-CandidatePrChecks` no longer requests unsupported `authorAssociation` from `gh pr list --json`. It lists open PRs on the main branch using valid fields, then fetches REST `author_association` only for Candidate-titled PRs and fails closed when the association cannot be verified.
The fix also suppresses handled transient REST lookup failures from the tracker body via `Invoke-Gh -Quiet`, while still surfacing unverifiable Candidate-titled PRs as structured WATCH details.
### Key Technical Details
- Timeline cross-reference filtering uses the source issue/PR repository identity so only same-repo PRs are considered release fix candidates.
- Candidate PR spoof protection accepts only maintainer associations: `OWNER`, `MEMBER`, or `COLLABORATOR`.
- Unverifiable author-association lookups are excluded fail-closed and reported separately from confirmed non-maintainer spoofers.
### Tests
Adds deterministic offline PowerShell regression assertions for:
- Keeping same-repo timeline PR references while dropping foreign PR references and non-PR events.
- Returning no candidates for foreign-only timeline references.
- Accepting a maintainer-authored Candidate PR and excluding contributor-authored Candidate PRs.
- Reporting unverifiable Candidate-titled PRs without labeling them as confirmed non-maintainers.
- Suppressing handled `Invoke-Gh -Quiet` warnings from tracker output.
### Issues Fixed
PR-only infrastructure fix; no linked product issue.
Report — Final Recommendation
Comparative Report — PR #36029
Candidates Compared
| Candidate | Approach | Test Result | Assessment |
|---|---|---|---|
pr |
Filters timeline PR cross-references by repository.full_name; removes invalid gh pr list --json authorAssociation; fetches REST author_association only for Candidate-titled PRs with quiet handled failures. |
Small, targeted, and matches the observed public timeline payload shape. Expert review found one hardening gap around accepting repository_url if nested repository.full_name is absent. |
|
pr-plus-reviewer |
Same as pr, plus a fallback from repository_url to source repo full name and an offline repository-url-only regression fixture. |
-SkipE2E suite reports known unrelated 529/2 baseline. |
Best balance: preserves the PR's targeted fix while addressing the expert reviewer's only actionable concern with a small, local change. |
try-fix-1 |
REST pulls-list for Candidate PRs plus canonical timeline pull_request.url filtering. |
❌ Failed. | Must rank lower than candidates without regression failure. It throws under PowerShell StrictMode when optional author_association is absent. |
try-fix-2 |
GraphQL-first timeline and Candidate PR queries, including authorAssociation in one GraphQL flow. |
Viable but broader and more complex than necessary. Adds GraphQL pagination/query plumbing for a localized release-readiness script fix. |
Ranking
pr-plus-reviewer— wins. It is the raw PR's targeted fix plus the only expert-review hardening suggestion, with relevant regression coverage.pr— acceptable and targeted, but slightly less robust thanpr-plus-reviewerfor timeline payload shape variation.try-fix-2— relevant assertions passed, but it is a larger GraphQL rewrite with more moving parts and no clear benefit over the PR-lineage fix.try-fix-1— failed regression validation under StrictMode and must rank below candidates whose relevant assertions passed.
Winning Candidate
Winner: pr-plus-reviewer
pr-plus-reviewer should be selected because it keeps the PR's minimal REST/CLI architecture, avoids the invalid gh pr list field, filters foreign timeline PRs, and adds a low-risk repository_url fallback for the expert reviewer's API-shape concern. Candidates with failed regression tests were ranked lower as required.
Future Action — review latest findings
No alternative fix was selected for this run. Review the session findings and CI results before merging.
Note
Are you waiting for the changes in this PR to be merged?
It would be very helpful if you could test the resulting artifacts from this PR and let us know in a comment if this change resolves your issue. Thank you!
Problem
The
[Release Readiness]tracker issues were renderingghwarnings embedded in their bodies, two distinct root causes:1. Cross-repo timeline cross-references (e.g. SR8 tracker #35876).
Get-IssueTimelinePrswalked a regression issue'scross-referencedtimeline events and collected the source PR number without checking which repository the PR lives in. Real examples found live:zhollis21/AniSprinkles(an unrelated personal app)praveenkumarkarunanithi/maui(a fork)Looked up against
dotnet/mauithese 404 (gh pr view 24 ... Could not resolve to a PullRequest), and the warning gets written into the tracker issue. Worse, a foreign PR whose number happens to exist in dotnet/maui (e.g. #877 fromDIPSAS/DIPS.Mobile.UI) silently mis-matches an unrelated dotnet/maui PR as a "fix candidate" with no warning at all.2. Invalid
gh pr listfield (e.g. SR9 tracker #35867).Get-CandidatePrChecksrequestedgh pr list --json number,title,author,authorAssociation,updatedAt,url, butauthorAssociationis not a validgh pr listprojection field (it's REST-only). The whole query failed (Unknown JSON field: "authorAssociation"), surfacing a warning and silently degrading the maintainer spoof-gate.Fix
Get-IssueTimelinePrs: drop any cross-reference whoserepository.full_name≠ the target repo. The timeline API populatesrepository.full_namefor same-repo references too, so legitimate dotnet/maui fix PRs are preserved. This fixes both the noisy 404 warnings and the silent wrong-repo mis-matches.Get-CandidatePrChecks: removeauthorAssociationfrom thegh pr listprojection and fetchauthor_associationper title-matched candidate viagh api repos/{repo}/pulls/{n}(cheap — candidate matches are almost always 0–1). Fail closed: an unreadable association excludes the PR so aCandidate-titled spoof can't slip through.Tests
Adds 8 deterministic, offline (gh-stubbed) regression assertions: cross-repo timeline filtering (keeps same-repo, drops foreign #24/#102 and non-PR events; foreign-only → 0 candidates) and the REST author-association spoof-gate (MEMBER accepted, CONTRIBUTOR excluded, spoof-only reports exclusion). Offline suite: 492 passed / 0 failed.
Scope check: the sibling Preview engine (
Get-PreviewReadiness.ps1) has neither code path, so no change is needed there.