Skip to content

Surface nightly dogfood feed staleness at the top of release-readiness trackers - #36066

Merged
PureWeen merged 9 commits into
mainfrom
pureween/nightly-feed-staleness-banner
Jun 23, 2026
Merged

Surface nightly dogfood feed staleness at the top of release-readiness trackers#36066
PureWeen merged 9 commits into
mainfrom
pureween/nightly-feed-staleness-banner

Conversation

@PureWeen

Copy link
Copy Markdown
Member

Note

Are you waiting for the changes in this PR to be merged?
It would be very helpful if you could test the resulting artifacts from this PR and let us know in a comment if this change resolves your issue. Thank you!

Summary

Each [Release Readiness] tracker now opens with a nightly dogfood feed freshness banner for that release's lane, so a release captain can tell at a glance whether dogfooders are validating current bits or testing stale builds. A fresh nightly is expected daily; when one stops appearing, the tracker should say so loudly.

This is internal release-readiness tooling (.github/skills/release-readiness/) — no product/runtime code changes.

What it looks like

  • ✅ fresh (< 3 days): **Nightly dogfood feed:** ✅ \dotnet11` (net11 preview) — latest `11.0.0-preview.6.26322.3` built today.`
  • ⚠️ aging (3–6 days): blockquote with age + publish date.
  • ❌ stale (≥ 7 days): blockquote alarm — e.g. (live, SR8 band today): ❌ Nightly dogfood feed is STALE — 11 days … Latest build \10.0.80-ci.main.26310.5` published 2026-06-11 … builds appear to have stopped`.
  • Muted one-liners for unknown (feed query failed) and no matching build (band naming changed) — never a false alarm.

How it works

  • New shared helper scripts/NightlyFeed.ps1 (single source, dot-sourced by both engines + the test harness):
    • Get-NightlyFeedFreshness — queries the lane's Azure Artifacts feed (dotnet10, dotnet11, …) and returns the newest build whose version matches a band prefix, selected by catalogEntry.published. The feed orders versions by version number, not date, and mixes build families on one feed, so freshness must come from publish timestamps scoped by a version band. The network call is fail-open (any error → $null) and takes an injectable -Fetcher so it's fully unit-tested offline.
    • Format-NightlyFeedBannerpure, deterministic renderer (caller passes -Now); tiers fresh/aging/stale + muted unknown/no-match. Thresholds are parameters (default 3/7 days).
  • SR engine (Get-ReleaseReadiness.ps1): on full (all) runs, maps the SR lane to dotnet<Major> + the <Major>.0.<Patch> band — in-flight SR → the SR branch's band; candidate → main's band — queries freshness, and renders the banner under the Generated line. Band-number matching is deliberate: it's resilient to family-keyword churn (an SR8 .80 build is tagged ci.main, not ci.inflight, so a keyword match would miss it).
  • Preview engine (Get-PreviewReadiness.ps1): maps the preview to dotnet<Major> + the <Major>.0.0-preview.<N> band (iteration read from Versions.props at the survey ref), renders the banner under Overall status.
  • Defensive load: both engines dot-source the helper guarded by Test-Path + Get-Command; a missing/unloadable helper degrades to no banner rather than crashing the unattended nightly tracker job. The freshness query is wrapped in try/catch and gated to non-test code paths, so existing E2E tests stay network-free.

Testing

  • 26 new offline assertions for the helper: banner tiers, unknown / matched=$false, future-publish clamp, custom thresholds, deterministic publish date; and Get-NightlyFeedFreshness with a mocked -Fetcher covering newest-by-date (not version) selection, band-prefix filtering, fail-open on throw, and paged-registration @id follow-up. Plus SR render-path wiring assertions (banner appears after Generated; absent when the key isn't set).
  • Offline suite: 592/0.
  • Live validation against the real feeds (2026-06-22): dotnet10 ^10.0.90- → fresh today; dotnet10 ^10.0.80- → STALE 11d; dotnet11 ^11.0.0-preview. → fresh today.
  • Full suite is 602 passed / 2 pre-existing env-specific failures in the unmodified Find-ReleaseReadinessTrackers.ps1 E2E (its git-root guard fail-closes when run from a /tmp worktree without -Repo); identical on baseline, unrelated to this change.

Follow-ups (out of scope)

  • The companion Nightly-Builds wiki page is stale (mislabels which branch feeds dotnet9/dotnet10, missing dotnet11); a corrected draft is being reviewed separately.

Each release-readiness tracker now opens with a banner reflecting how fresh the
nightly dogfood feed for THAT lane is, so a release captain can see at a glance
whether dogfooders are validating current bits or testing stale builds.

- New shared helper scripts/NightlyFeed.ps1:
  - Get-NightlyFeedFreshness: queries the lane's Azure Artifacts feed (dotnet10,
    dotnet11, ...) and returns the newest build whose version matches a band
    prefix, by catalog publish date (the feed orders by version, NOT date, and
    mixes build families, so freshness must come from catalogEntry.published).
    Network call is FAIL-OPEN (any error -> null) and accepts an injectable
    -Fetcher so it is unit-tested offline.
  - Format-NightlyFeedBanner: PURE, deterministic renderer (caller passes -Now).
    Tiers: fresh (<3d), aging (3-6d), stale (>=7d), plus muted unknown / no-match.

- SR engine (Get-ReleaseReadiness.ps1): in full ('all') runs, maps the SR lane to
  dotnet<Major> + the <Major>.0.<Patch> band (in-flight SR -> SR branch band;
  candidate -> main band), queries freshness, and renders the banner under the
  **Generated** line. Band-number matching is resilient to family-keyword churn
  (an SR8 .80 build is tagged ci.main, not ci.inflight).

- Preview engine (Get-PreviewReadiness.ps1): maps the preview to dotnet<Major> +
  the <Major>.0.0-preview.<N> band (iteration read from Versions.props at the
  survey ref) and renders the banner under **Overall status**.

- Both engines defensively dot-source the helper (missing/unloadable helper
  degrades to no banner rather than crashing the unattended tracker job).

- Tests: 26 offline assertions for the helper (tiers, unknown/no-match,
  future-clamp, custom thresholds, deterministic date; mocked-fetcher coverage of
  date-not-version selection, band filtering, fail-open, paged registration) plus
  SR render-path wiring assertions. Offline suite 592/0.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@github-actions

github-actions Bot commented Jun 22, 2026

Copy link
Copy Markdown
Contributor

🚀 Dogfood this PR with:

⚠️ WARNING: Do not do this without first carefully reviewing the code of this PR to satisfy yourself it is safe.

curl -fsSL https://raw.githubusercontent.com/dotnet/maui/main/eng/scripts/get-maui-pr.sh | bash -s -- 36066

Or

  • Run remotely in PowerShell:
iex "& { $(irm https://raw.githubusercontent.com/dotnet/maui/main/eng/scripts/get-maui-pr.ps1) } 36066"

@github-actions

github-actions Bot commented Jun 22, 2026

Copy link
Copy Markdown
Contributor

Skill Validation Results

@PureWeen — new skill validation results are available based on this last commit: 772d01d.
To request a fresh validation after new comments or commits, comment /evaluate-skills.

Overall Passed Static Passed LLM Skipped Skills 20 Agents 6

Skill Validation Results772d01d · Surface nightly dogfood feed staleness at the top of release-readiness trackers · 2026-06-23T21:08:10Z

✅ Static Checks Passed

Skills: 20 | Eval specs linted: 7

Full lint output
── .github/skills/agentic-labeler/tests/eval.vally.yaml
npm warn deprecated prebuild-install@7.1.3: No longer maintained. Please contact the author of the relevant native addon; alternatives are available.
✔ .github/skills/agentic-labeler/tests/eval.vally.yaml is valid
── .github/skills/code-review/tests/eval.capability.vally.yaml
✔ .github/skills/code-review/tests/eval.capability.vally.yaml is valid
── .github/skills/code-review/tests/eval.vally.yaml
✔ .github/skills/code-review/tests/eval.vally.yaml is valid
── .github/skills/code-review/tests/hermeticity.vally.yaml
✔ .github/skills/code-review/tests/hermeticity.vally.yaml is valid
── .github/skills/evaluate-pr-tests/tests/eval.vally.yaml
✔ .github/skills/evaluate-pr-tests/tests/eval.vally.yaml is valid
── .github/skills/try-fix/tests/eval.vally.yaml
✔ .github/skills/try-fix/tests/eval.vally.yaml is valid
── .github/skills/verify-tests-fail-without-fix/tests/eval.vally.yaml
✔ .github/skills/verify-tests-fail-without-fix/tests/eval.vally.yaml is valid

⏭️ LLM Evaluation: Skipped

No changed skills with eval specs found.

🔍 Full results and investigation steps

@github-actions github-actions Bot added the area-infrastructure CI, Maestro / Coherency, upstream dependencies/versions label Jun 22, 2026
PureWeen and others added 5 commits June 22, 2026 13:45
The nightly-feed staleness banner watched the lane's version band, whose
freshest build is almost always the daily ci.main CI build (e.g.
10.0.90-ci.main, fresh every day). That masks the failure the banner exists
to catch: a stall in the *inflight* stream (ci.inflight — builds of the
inflight/current branch, the 'shipping next' dogfood bits). eng/Versions.props
on main switches the label to ci.inflight when BUILD_SOURCEBRANCH is
refs/heads/inflight/current; ordinary main CI is ci.main.

Add Resolve-NightlyDogfoodFreshness: prefer the newest ci.inflight build on
the feed (resolved feed-wide so it auto-follows band advances 10.0.80 ->
10.0.90), falling back to the lane band only when the feed has NO inflight
builds (e.g. a preview feed not yet in the inflight phase). A *transient*
inflight-query failure degrades to 'unknown' rather than falling through to
the always-fresh ci.main band, so a stalled feed is never painted green.

Both engines (SR + preview) now route through the resolver and label the
banner by build type (ci.inflight vs band). Live result today: dotnet10
shows STALE 15 days (10.0.80-ci.inflight stopped 2026-06-07, the failing
inflight build), while dotnet11-preview6 stays fresh via its preview build.

Tests: offline 600/0 (+8 resolver cases incl. the transient-error safety
path); full suite 610/2 (2 pre-existing env flakes in the unmodified
Find-ReleaseReadinessTrackers.ps1).

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
The nightly dogfood banner was excluded from Get-ReportSemanticHash, so on a
quiet SR in-flight tracker (no commit/CI/PR/regression change) the idempotent
no-op skipped the issue edit and the banner never appeared or refreshed — the
exact case the banner exists for (e.g. sr8 #35876: feed 15d stale, no banner).

Fix: add a pure Get-NightlyFeedTier helper (single source of truth for the
fresh/aging/stale/unknown/no-match bucket, sharing the renderer's thresholds)
and fold a non-drifting 'tier|version' signature into the semantic hash. Tier
crossings (ok->aging->stale) and new builds flip the hash and refresh the
tracker; a daily day-count tick within the same tier does not (no watcher spam).
Fail-open: if NightlyFeed.ps1 isn't loaded the field is '' and the hash is
unchanged from prior behaviour.

Tests: +11 Get-NightlyFeedTier bucket cases, +6 hash-integration cases
(ok vs stale differs, day-count drift invariant, new build differs, present vs
absent differs, unknown differs, deterministic). Offline suite 617/0.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…rd, fail-open telemetry

Adversarial-review fixes for the nightly dogfood feed staleness banner:

- Split-clock (SR idempotency hash): Get-ReportSemanticHash sampled
  [datetime]::UtcNow independently from the banner render, so a run that
  straddled a tier boundary could embed a hash tier that disagreed with the
  displayed banner tier and freeze a stale banner via the no-op gate.
  Add-SrNightlyFeedFreshness now captures one instant in $Data['nightlyFeedNow']
  and both the banner render and the hash tier reuse it (UtcNow fallback when unset).

- ci.main false-green guard (Resolve-NightlyDogfoodFreshness): an SR lane's band
  prefix (e.g. ^10\.0\.90-) also matches the always-fresh ci.main stream, so the
  no-inflight band fallback could surface a fresh ci.main build and paint a stalled
  inflight feed green. The fallback now reports matched=false for a ci.main-only
  band. Preview bands (preview.N) never match ci.main, so this is a no-op for preview.

- Fail-open telemetry (Get-NightlyFeedFreshness): the swallowing catch now emits a
  Write-Warning with the feed and exception so a real outage (401/503/DNS) isn't
  silently invisible in the unattended job's log.

- Honest lane label: an unknown/transient-failure result is labelled ci.inflight
  (the stream being measured) instead of implying the band carries the signal.

+7 regression asserts (2 fail-pre-fix for the ci.main guard, 1 for the split-clock
hash, plus over-filter and tier sanity guards). Offline suite 624/0.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…preview label parity

Round-2 adversarial-review follow-ups on the nightly dogfood feed staleness banner:

- NightlyFeed.ps1: the fail-open catch in Get-NightlyFeedFreshness now emits its
  diagnostic with -WarningAction Continue. Under an ambient $WarningPreference='Stop'
  (or inherited -WarningAction Stop) the bare Write-Warning would raise a terminating
  error that escapes the helper, violating its documented 'never throws -> returns $null'
  contract and breaking fail-open for the unattended nightly job.

- Get-PreviewReadiness.ps1: mirror the SR lane's 3-branch honest-label logic so an
  unknown / transient-inflight-failure case is labelled ci.inflight (the stream being
  measured) instead of implying the preview band carries the signal.

- Adds a regression assert that Get-NightlyFeedFreshness with a throwing fetcher under
  $WarningPreference='Stop' still returns $null without throwing. Offline suite 625/0.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Round-3 adversarial review (opus/gpt-5.5/gemini all 3 flagged this) found the round-2
fail-open fix was incomplete: only the NightlyFeed.ps1 helper's inner catch was hardened.
The two engine scripts' OUTER nightly-feed catches — and the load-time 'helper not found'
warnings — still used a bare Write-Warning. Under an ambient $WarningPreference='Stop',
a non-helper error inside the engine try (band resolution, [int] cast on a garbled
Versions.props, banner formatting) hits the catch, the bare Write-Warning is promoted to a
terminating error, and it escapes — crashing the unattended nightly job. Same contract the
round-2 fix protects, just one frame up.

- Get-ReleaseReadiness.ps1: -WarningAction Continue on the Add-SrNightlyFeedFreshness outer
  catch (line ~3701) and the load-time helper-not-found warning (line ~171).
- Get-PreviewReadiness.ps1: same two sites (lines ~1403 and ~137).
- Adds an engine-boundary regression test that drives a throw through
  Add-SrNightlyFeedFreshness under $WarningPreference='Stop' and asserts the catch swallows
  it (fails on pre-fix bare Write-Warning). Offline suite 626/0.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

@kubaflo kubaflo left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.


🤖 AI-GENERATED MULTI-MODEL CODE REVIEW 🤖

This review was independently performed by 3 AI models (Claude Opus 4.8, GPT-5.5, Gemini 3.1 Pro), then cross-pollinated to reconcile differences.

Round: 1
HEAD: 9b915f67
Timestamp: 2026-06-23T00:46 UTC+02


🎯 Verdict: LGTM ✅

Confidence: High (unanimous after cross-pollination)

Independent verdicts:

  • Claude Opus 4.8: LGTM/High
  • GPT-5.5: NEEDS_DISCUSSION/LowLGTM/High (upgraded after cross-pollination)
  • Gemini 3.1 Pro: LGTM/High

📊 Summary

This PR surfaces nightly dogfood feed staleness at the top of release-readiness trackers. The feature correctly:

✅ Fetches the nightly feed manifest and extracts the MAUI build date
✅ Compares against the current date to calculate staleness
✅ Displays a prominent banner when the feed is stale (>24h)
✅ Uses fail-open design (fetch errors don't block workflow)
✅ Integrates staleness into semantic hash to avoid daily spam while refreshing on transitions

🔍 Key Review Points

CI Status — Resolved Disagreement

Initial split: GPT flagged maui-pr: skipping as "undetermined CI coverage"

Resolution after investigation:

  • maui-pr skip is by-designeng/pipelines/ci.yml explicitly excludes .github/** under pr: paths: exclude
  • Actual validation: Skill Validation, Static validation, and Validate (PR) all pass
  • Validate (PR) runs the full test suite in CI: Passed: 695 Failed: 0
  • BLOCKED merge state is due to reviewDecision: REVIEW_REQUIRED (human approval), not CI failure

Consensus: All 3 models agree CI coverage is appropriate for .github/skills/** changes.

Test Failure Analysis

Local test failure (T8): Pre-existing timezone sensitivity (UTC+02:00 environment)

  • The test expects UTC timezone for date comparisons
  • Confirmed by all 3 models as unrelated to PR changes
  • CI passes with proper UTC configuration

Code Quality

✅ PowerShell script parses cleanly
✅ Fail-open design prevents workflow breakage
✅ Semantic hash integration avoids notification spam
✅ Clear comments and error handling

💡 Suggestions (Non-blocking)

2 minor suggestions for future maintainability (see inline comments):

  1. Documentation: Consider documenting NightlyFeed.ps1 and banner logic in SKILL.md
  2. Duplication: Lane label logic is duplicated between Preview and SR scripts — could extract to shared helper

These do not affect merge readiness.

🏁 Recommendation

Ready to merge pending human approval. All models agree the code is correct, well-tested, and follows best practices for fail-open infrastructure tooling.


This review used the code-review skill with independence-first assessment, full-context analysis, and empirical CI verification.

Comment thread .github/skills/release-readiness/scripts/Get-PreviewReadiness.ps1 Outdated
…eed in SKILL.md

Addresses the two non-blocking suggestions from the bot review on #36066:

- Dedupe the nightly-feed lane-label logic that was copy-pasted between the SR
  (Get-ReleaseReadiness.ps1) and Preview (Get-PreviewReadiness.ps1) engines into a
  single PURE helper, Format-NightlyFeedLaneLabel, in NightlyFeed.ps1. The
  honest-labeling rule (inflight->ci.inflight; band->caller-formatted note;
  unknown->ci.inflight) now has one source of truth, so the lanes can't drift the
  way the preview lane silently did once. Adds 5 direct unit tests covering both
  band-note shapes plus the unknown/other fallback (a mutation reintroducing the
  drift fails them). Offline suite 631/0.
- Document NightlyFeed.ps1 (functions, fail-open contract, determinism) and the
  staleness banner in the release-readiness SKILL.md.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

@kubaflo kubaflo left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Note

🤖 This review was generated by AI — an automated multi-model code review.


🔍 AI Multi-Model Code Review — Round 2

This review was independently performed by 3 AI models (Claude Opus 4.8, GPT-5.5, Gemini 3.1 Pro), then cross-pollinated to reconcile differences.

Round: 2 · HEAD: df609453 · Re-review after Round 1 (9b915f67, LGTM)


✅ Verdict: LGTM

Confidence: High — unanimous (all 3 models, both independently and after cross-pollination)

Model Independent After Cross-Pollination
Claude Opus 4.8 LGTM / High LGTM / High
GPT-5.5 LGTM / High LGTM / High
Gemini 3.1 Pro LGTM / High LGTM / High

🎯 Round 1 Suggestions — Both Resolved ✓

In Round 1 I posted 2 non-blocking suggestions. Both are cleanly addressed in this round:

  1. ✅ Extracted the duplicated lane-label construction → new pure helper Format-NightlyFeedLaneLabel in NightlyFeed.ps1. Both engines (Get-ReleaseReadiness.ps1 + Get-PreviewReadiness.ps1) now call the single source of truth instead of inlining the inflight/band/unknown mapping. This makes the "preview lane silently lost the band branch" drift class structurally impossible.

  2. ✅ Documented NightlyFeed.ps1 → new SKILL.md section with a full function table (Get-NightlyFeedFreshness, Resolve-NightlyDogfoodFreshness, Format-NightlyFeedLaneLabel, Get-NightlyFeedTier/Format-NightlyFeedBanner) plus the determinism/idempotency contract (one UtcNow per run, tier|version folded into the semantic hash, raw timestamp never hashed).

Bonus: 5 new discriminating unit tests for the extracted helper covering every branch (inflight, band-SR-shape, band-preview-shape, unknown, other-fallback).


🔬 Verification Performed

Opus built an empirical harness — dot-sourced NightlyFeed.ps1 at PR HEAD and compared Format-NightlyFeedLaneLabel against the verbatim Round 1 inline logic across a 7-input matrix (inflight / band / '' / mystery / 'BAND' / 'Inflight' / $null):

Byte-identical (case-sensitive) output for both SR and Preview lanes — no behavioral drift from the extraction
✅ The SR `$band` and Preview `$nfBand` (preview.$nfIteration) BandNote strings are passed through exactly as before
✅ Helper is pure (no clock, no network tokens) — confirmed
✅ All 4 modified .ps1 files AST-parse cleanly
✅ SKILL.md function table is factually accurate vs implementation (tier thresholds ✅ ≤2d · ⚠️ 3–6d · ❌ ≥7d = AgingDays=3/StaleDays=7)


🟢 CI Status

Check Result
Validate (PR) ✅ pass (2m42s — runs the release-readiness test suite incl. the 5 new asserts)
Skill Validation ✅ pass
Static validation ✅ pass
license/cla ✅ pass
maui-pr ⏭️ skipping — by-design (.github/** path exclusion in eng/pipelines/ci.yml)
Build Analysis ⏳ pending (informational, non-blocking)

The relevant validation for a .github/skills/** change ran and passed. The BLOCKED merge state reflects REVIEW_REQUIRED (human approval), not a CI failure.


💡 One Non-Blocking Suggestion (see inline)

The new unit tests lock the helper's branch logic but not the call-site -BandNote wiring end-to-end. This is a strict improvement over Round 1 and the residual risk is tiny (verified byte-identical at both call sites). An optional end-to-end assertion on each lane's rendered label would close it fully. Details inline.


🏁 Recommendation

LGTM — clean, well-tested refactor that fully resolves both Round 1 suggestions and eliminates a real drift class. Ready for human approval.


Reviewed with the code-review skill: independence-first assessment (code before narrative), full-context source reading, empirical verification, and CI cross-checking by all 3 models.

# ───── Format-NightlyFeedLaneLabel: honest-labeling rule (shared by both engines) ─────
# Direct guard for the rule that drifted once (the preview lane silently lost the band
# branch). Both engines now call this single helper, so these asserts cover both lanes.
Write-Host "`n[Unit] Format-NightlyFeedLaneLabel (honest labeling)" -ForegroundColor Cyan

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 💡 Suggestion (non-blocking): Tests lock the helper's branch logic but not the call-site -BandNote wiring

These 5 asserts exercise Format-NightlyFeedLaneLabel directly with hand-written -BandNote literals (`10.0.80` for the SR shape, `11.0.0-preview.6` (preview.6) for the preview shape), so they fully cover all three branches and would catch any future drop of the band branch in the single source of truth. 👍

What they don't assert is the call-site wiring — that the SR engine constructs "`$band`" (Get-ReleaseReadiness.ps1 ~L3677) and the Preview engine constructs "`$nfBand` (preview.$nfIteration)" (Get-PreviewReadiness.ps1 ~L1383) and passes it through as -BandNote. A call-site interpolation regression wouldn't be caught by these unit tests.

Strictly non-blocking — the drift-prone duplication is eliminated (single source of truth), and multi-model R2 verification proved byte-identical output at both current call sites. An optional end-to-end assertion on each lane's rendered laneLabel would lock the wiring as well as the logic.

— Raised by Claude Opus 4.8, adopted by Gemini 3.1 Pro during cross-pollination.

PureWeen and others added 2 commits June 23, 2026 15:09
Closed `no-fix-yet` regressions were counted in the SR Summary table but
rendered in no tier: `no-fix-yet` lived only in $tier1Classes and the emit
filtered it to OPEN, while it was absent from $tier3Classes. So closed entries
(triaged-away regressions with no fix PR cross-referenced) showed as e.g.
"no-fix-yet: 6" in the summary yet appeared nowhere — the live symptom on
tracker #35876 (.NET 10 SR8).

The verdict logic already downgrades closed no-fix-yet to Tier 3
(Get-VerdictTier path), so display and verdict disagreed. Make the tier emit
state-aware: OPEN no-fix-yet block in Tier 1, CLOSED-but-unresolved ones render
as Tier 3 informational — so the Summary count and the displayed rows agree and
closed-but-unresolved regressions stay visible for the release captain.

Adds a discriminating regression test (open → Tier 1, closed → Tier 3, closed
NOT in Tier 1). Offline suite 635/0.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
The Tier-3 display fix (306a296) renders closed no-fix-yet regressions in
Tier 3 (OPEN ones stay in Tier 1), but Get-ReportSemanticHash hashed each
regression as only issue:classification. When a no-fix-yet issue closed while
another blocker held the verdict 🔴, every hashed component was unchanged, so
the idempotent tracker updater skipped the edit and left a stale Tier-1 row.

Fold a state-derived tier bit (t1/t3) into the hash for the no-fix-yet class
ONLY — the single classification whose rendered tier depends on issue state.
A no-fix-yet OPEN→CLOSED flip now refreshes the tracker, while every other
classification stays state-insensitive, so unrelated state transitions (e.g.
a Tier-3 in-sr-active issue closing) don't churn the hash or spam watchers —
preserving the hash's deliberately conservative design.

Adds two discriminating tests: no-fix-yet state flip → different hash (under a
held verdict); non-no-fix-yet state flip → same hash. Mutation-verified.
Offline suite 637/0.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@PureWeen
PureWeen merged commit 676dbe9 into main Jun 23, 2026
17 of 18 checks passed
@PureWeen
PureWeen deleted the pureween/nightly-feed-staleness-banner branch June 23, 2026 21:42
@github-actions github-actions Bot added this to the .NET 10 SR9 milestone Jun 23, 2026
PureWeen added a commit that referenced this pull request Jul 6, 2026
<!-- Please let the below note in for people that find this PR -->
> [!NOTE]
> Are you waiting for the changes in this PR to be merged?
> It would be very helpful if you could [test the resulting
artifacts](https://github.com/dotnet/maui/wiki/Testing-PR-Builds) from
this PR and let us know in a comment if this change resolves your issue.
Thank you!

### Description

Docs/skill-only change. Adds a deterministic breadcrumb so a bare
request like **"nightly is broken please fix"** routes an agent (or a
human) straight to the right pipeline and the known failure surface,
instead of guessing.

Today nothing in the repo maps **"nightly" / "inflight feed" / "dogfood
feed"** to the official signed build. `copilot-instructions.md`
documents only the **PR** pipelines (`maui-pr`, `-devicetests`,
`-uitests`), and `azdo-build-investigator` mentioned `ci-official.yml`
only tangentially (CFSClean/Gradle). So an agent told the dogfood feed
is stale had no way to know the AzDO definition id (1095) or org
(`dnceng/internal`) to even query the builds.

This matters now because we just shipped the **nightly dogfood feed
staleness banner** (#36066) — when that banner turns ❌, the natural next
question is *"which pipeline do I go fix?"* and the repo couldn't answer
it.

### What changed

- **`.github/skills/azdo-build-investigator/SKILL.md`**
- Added `nightly` / `inflight feed` / `dogfood feed` / `official build`
trigger phrases to the skill `description` (fixes routing).
- New **"Nightly / Official Signed Build (inflight dogfood feed)"**
section: the pipeline identity (`dotnet-maui`, definition **1095**,
`dnceng`/`internal`, defined by `eng/pipelines/ci-official.yml`, cron
`05:00 UTC` on `inflight/current`), the AzDO investigation recipe
(`azdo_builds` by `definitionId`+branch → `azdo_search_timeline` →
`azdo_search_log`), and the recurring **`vs-workload.props`
(`MSB4019`)** failure in the `Pack Windows` → "Build Workloads, Sign &
Publish" step. Plus a note that the macOS leg can pass while Windows
fails, and that the release pipelines are distinct.
  - Added a matching row to the failure-pattern table.
- **`.github/skills/release-readiness/SKILL.md`**
- Cross-linked the ❌ staleness banner to the investigator section, so
the banner's existing "check the nightly pipeline" hint has a documented
target.

### Grounding

Every fact was confirmed against the live failing run — definition
`1095` / `dnceng-internal`, the `ci-official.yml` schedule,
`stage-pack.yml:150` ("Build Workloads, Sign & Publish"), and the
`MSB4019: vs-workload.props was not found` signature on
`inflight/current` (a multi-day streak as of this writing). No
product/runtime code is touched.

### Risk

Documentation only. No code paths, no tests affected. `ci-official.yml`
excludes `.github/**`, and `maui-pr` skips `.github/**`-only PRs by
design.

### Update — confirmed root cause folded in

A companion investigation (fix PR #36089, base `inflight/current`)
binlog-proved the actual cause of the current outage, now documented in
the breadcrumb:

- The workload packs (`Microsoft.NET.Sdk.Maui.Manifest`,
`Microsoft.Maui.Sdk`) target netstandard but ship, so they set
`<IsPackable>true</IsPackable>` explicitly.
- PR #32203's blanket `<IsPackable
Condition="$(TargetFramework.Contains('netstandard'))">false</IsPackable>`
in `Directory.Build.targets` (auto-imported last) overrode that →
`IsPackable=false`.
- The "Pack, Sign" step runs `-pack` **without** `-build`, so `Build`
never ran → `_GenerateVSWorkloadProps` (`AfterTargets="Build"`) never
wrote `vs-workload.props` → the line-30 import threw `MSB4019`.
- Regression is **`inflight/current`-only** (the #32203 commit isn't on
`main`/`net10.0` yet) — forward-port must carry the `and '$(IsPackable)'
== ''` guard or the break reappears.

The breadcrumb now teaches the real failure mechanism, not just the
symptom, and links the fix.

---------

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@github-actions github-actions Bot locked and limited conversation to collaborators Jul 24, 2026
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

area-infrastructure CI, Maestro / Coherency, upstream dependencies/versions

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants