to next cycle` for each remaining PR so the drop is a deliberate, logged decision. The continuous loop picks the deferred PRs up next @@ -419,8 +481,9 @@ For every open tracking issue in scope, converge on exactly one outcome: | Open first help-wanted draft `[ci-fix-net11]` PR | No open PR yet; a plausible candidate exists but cannot be runner-validated (device/UI tests) (attempt 1) | | Open first de-flake draft `[ci-fix-net11]` PR | No open PR yet; failure is intermittent (green-on-retry) from a genuine test-quality defect; a deterministic-synchronization fix is producible (attempt 1, Step 4.7 bucket b) | | Advance an existing PR (push attempt N+1) | Open PR's own CI settled red *because of the fix itself*, no human engaged, marker < 10 — push a NEW distinct fix onto the same branch (Step 3.5 → 5.6) | -| Surface a validated-green PR | Open PR's own CI settled green — comment "validated, attempt N/10; ready for review" and do NOT advance (Step 3.5) | -| Annotate an unrelated-flake red | Open PR is red only on baseline-flake / unrelated legs — comment which leg needs a re-run; do NOT burn an attempt (Step 3.5) | +| Surface a validated-green PR | Open PR's own CI settled green — comment "validated, attempt N/10; ready for review" and do NOT advance (Step 3.5), then mark the draft PR ready for review once the fixed test is confirmed green (Step 3.6) | +| Annotate an unrelated-flake red | Open PR is red only on baseline-flake / unrelated legs — comment which leg needs a re-run; do NOT burn an attempt (Step 3.5); if the SPECIFIC fixed test is confirmed green on that build, still mark the draft PR ready for review (Step 3.6) | +| Mark a validated PR ready for review | The specific test this PR fixed is confirmed green in the PR's own CI (even if unrelated legs are red) — comment the target-test result and transition the draft PR to ready for review; never approve or merge (Step 3.6) | | Wait | Open PR's CI is pending / not yet settled on the current head SHA — do nothing this cycle (Step 3.5) | | Hand-off skip (attempt cap) | Marker == 10 and the signature still reproduces — stop and defer to humans; the dedicated `[ci-fix-net11][needs-human]` PR is planned but currently deferred (Step 6) | | Recorded skip | Visual-regression, human engaged, already-handled, fixed-in-latest-build, infra-flake, out-of-bounds, only-mute-available, or no novel approach producible | @@ -695,14 +758,34 @@ Run these gates in order — the FIRST that fires decides this cycle's outcome: 1. **Human engaged.** If `C.humanEngaged` → `skipped: human engaged on PR #
; deferring` and stop. Never fight a human reviewer — the intentional hand-off boundary still holds the moment a person touches the PR. -2. **CI not settled / unknown / incomplete prefetch.** If `C.checksSettled == false` - OR `C.overallConclusion` is `pending`, `neutral`, or `unknown`, OR - `C.dataComplete == false` → the fix's CI has not finished on the current head SHA - (`C.headSha`), or the prefetch could not fully read this PR (a partial read can - understate `humanEngaged` / `attempt`). `skipped: PR #
CI pending / prefetch - incomplete on ; waiting` and stop. *(Round 1: this is where a - maintainer `/azp run` is awaited — the `/azp`-gated uitests/devicetests legs will - not have run until a human kicks them.)* +2. **CI not settled — but validate the target test first (target-focused readiness).** + If `C.checksSettled == false` OR `C.overallConclusion` is `pending`, `neutral`, or + `unknown`, OR `C.dataComplete == false` → the *overall* build has not finished on the + current head SHA (`C.headSha`), or the prefetch could not fully read this PR (a partial + read can understate `humanEngaged` / `attempt`). Unrelated legs still draining must NOT + keep an already-proven fix parked as a draft, so before waiting, try the target-focused + fast-path: + - **2a — Target-green fast-path (draft `[ci-fix-net11]` PRs only).** If `C.dataComplete + == true` AND the Step 3.6 preconditions hold (draft PR, `[ci-fix-net11] ` title prefix + AND the `agentic-workflows` label), run Step 3.6 **T1–T2** against `C.headSha` now: + identify the target test(s) and read the PR's OWN build **timeline / per-leg status** + (anonymous `_apis/build` — never `_apis/test`, per Hard-Rule 8). If EVERY target test + is **VALIDATED-GREEN** (per T2's all-platform definition below — the target's category + leg is `succeeded` on every platform that runs it, failed on none, and NO target + platform family the pipeline covers is still pending/unconcluded, per T2's "no platform + left unverified" rule; a platform that simply has no leg for the target's category — the + test doesn't run there — is fine, not a gap) AND every leg in + `C.failedLegs` that has ALREADY concluded classifies as **unrelated flake** (Step 4 / + Step 4.7 method — the fix is implicated in NO completed red), then the fix is proven + regardless of unrelated *pending* legs → run **Step 3.6 T3** (mark ready + 🎯 comment) + and stop. This early-out NEVER advances an attempt and NEVER acts on a red that could + be the fix's fault. + - **2b — Otherwise WAIT.** If the target test has not yet executed (pending/absent on + its leg), or a completed red is (or may be) caused by the fix, or `C.dataComplete == + false`, or this PR has no identifiable target test → `skipped: PR # CI pending / + target not yet validated on ; waiting` and stop. *(Round 1: this is where a + maintainer `/azp run` is awaited — the `/azp`-gated uitests/devicetests legs will not + have run until a human kicks them.)* 3. **Green → surface for review.** If `C.overallConclusion == "success"`: the checks that RAN are green. **Primary-gate check first:** the deterministic `success` verdict only certifies "at least one green check, nothing failing or @@ -714,18 +797,31 @@ Run these gates in order — the FIRST that fires decides this cycle's outcome: # primary CI gate (maui-pr) not green on ; waiting` and stop — do NOT surface. (The `/azp`-gated `maui-pr-uitests` (def 313) / `maui-pr-devicetests` (def 314) legs MAY still be un-run — that is expected and is named below, not a - reason to withhold the surface.) **Idempotency:** scan the PR's existing comments - for a prior bot `✅ … validated … on ` note for THIS head SHA — if one - already exists, record `already-surfaced PR # (head )` and stop - WITHOUT re-commenting (re-surfacing the same green every tick is noise and burns - the per-run comment budget other PRs need). Otherwise resolve the attempt number from - `C.effectiveAttempt` (the authoritative max(marker, bot-commit) counter; omit the - number only if it is somehow indeterminate). **Dry-run gate (Step 0):** if `dry_run == "true"`, do NOT emit any comment — instead print the intended `✅` validated-green body to the run log, tally `dry-run: would-surface-green PR #`, and stop. Otherwise `add_comment` on PR #: `✅ Attempt /10 - validated — the fix's CI is green on . Ready for human review.` - Name any `/azp`-gated legs (uitests def 313 / devicetests def 314) that have not - run and still need a maintainer `/azp run`. Do NOT advance. Record - `surfaced-green PR # (attempt /10)` and stop. *(This directly - attacks the real bottleneck — no reviews — so it is the highest-value outcome.)* + reason to withhold the surface.) **Comment idempotency + dry-run suppress the ✅ + comment ONLY — neither skips Step 3.6.** Scan the PR's existing comments for a prior + bot `✅ … validated … on ` note for THIS head SHA; if one exists, set + `SKIP_SURFACE_COMMENT = true`. Resolve the attempt number from `C.effectiveAttempt` + (the authoritative max(marker, bot-commit) counter; omit the number only if it is + somehow indeterminate). Then post the surface comment UNLESS suppressed: + - if `dry_run == "true"`: do NOT emit — print the intended `✅` validated-green body to + the run log and tally `dry-run: would-surface-green PR #`; + - else if `SKIP_SURFACE_COMMENT`: do NOT re-post — record `already-surfaced PR # + (head )` (re-surfacing the same green every tick is noise and burns the + per-run comment budget other PRs need); + - else `add_comment` on PR #: `✅ Attempt /10 validated — the fix's CI is + green on .` naming any `/azp`-gated legs (uitests def 313 / devicetests def + 314) that have not run and still need a maintainer `/azp run`; record `surfaced-green + PR # (attempt /10)`. (Do NOT assert "ready for human review" on a PR that + is still a draft — Step 3.6, not this comment, owns the draft→ready flip and posts its + own 🎯 announcement when it fires.) + Do NOT advance. Then — in ALL of the above cases — run **Step 3.6** (target-test + readiness gate) for this PR before stopping: its `/azp`-gated target legs may conclude + green on this SAME head SHA on a LATER sweep (an `/azp run` adds no commit), and Step + 3.6 is the ONLY place the draft→ready flip happens, so it MUST re-evaluate every sweep — + never `stop` here before it. *(This directly attacks the real bottleneck — no reviews — + so it is the highest-value outcome.)* 4. **Red → classify caused-by-fix vs unrelated-flake.** If `C.overallConclusion == "failure"`, analyze the PR's OWN failing build (NOT `net11.0`): find the AzDO `maui-pr` build for this PR (filter builds by `branchName=refs/pull//merge`, @@ -733,18 +829,26 @@ Run these gates in order — the FIRST that fires decides this cycle's outcome: method and Step 4.7 flake buckets to `C.failedLegs`. - **Unrelated flake only** (every failed leg is known-flaky / infra / a pre-existing baseline red NOT introduced by the fix): do NOT burn an attempt. - **Idempotency first:** scan the PR's existing comments for a prior bot - `♻️ … unrelated flake … on ` note for THIS head SHA — if one already - exists, record `already-annotated-flake PR # (head )` and stop - WITHOUT re-commenting (re-annotating the same flake every 12h sweep is noise and - burns the per-run comment budget other PRs need). Otherwise resolve the attempt - number from `C.effectiveAttempt` (the authoritative max(marker, bot-commit) - counter), omitting the `Attempt /10:` prefix only if it is somehow - indeterminate. **Dry-run gate (Step 0):** if `dry_run == "true"`, do NOT emit any comment — instead print the intended `♻️` unrelated-flake body to the run log, tally `dry-run: would-annotate-flake PR #`, and stop. Otherwise `add_comment` on PR #: `♻️ Attempt /10: red is - unrelated flake on leg(s) () on ; the fix itself is not - implicated. A maintainer re-run (/azp run ) should clear it.` Record - `annotated-flake PR # (head )` and stop. *(Round 1: human re-runs; - Round 2: auto re-trigger.)* + **Comment idempotency + dry-run suppress the ♻️ comment ONLY — neither skips Step + 3.6.** Scan the PR's existing comments for a prior bot `♻️ … unrelated flake … on + ` note for THIS head SHA; if one exists, set `SKIP_FLAKE_COMMENT = true`. + Resolve the attempt number from `C.effectiveAttempt` (the authoritative max(marker, + bot-commit) counter), omitting the `Attempt /10:` prefix only if it is + somehow indeterminate. Then post the flake note UNLESS suppressed: + - if `dry_run == "true"`: do NOT emit — print the intended `♻️` unrelated-flake body + to the run log and tally `dry-run: would-annotate-flake PR #`; + - else if `SKIP_FLAKE_COMMENT`: do NOT re-post — record `already-annotated-flake PR + # (head )` (re-annotating the same flake every 12h sweep is noise and + burns the per-run comment budget other PRs need); + - else `add_comment` on PR #: `♻️ Attempt /10: red is unrelated flake on + leg(s) () on ; the fix itself is not implicated. A + maintainer re-run (/azp run ) should clear it.` record `annotated-flake + PR # (head )`. + Then — in ALL of the above cases — run **Step 3.6** (target-test readiness gate) for + this PR before stopping: its `/azp`-gated target legs may conclude green on this SAME + head SHA on a LATER sweep, and Step 3.6 is the ONLY place the draft→ready flip + happens, so it MUST re-evaluate every sweep — never `stop` here before it. *(Round 1: + human re-runs; Round 2: auto re-trigger.)* - **Caused by the fix** (a failed leg still matches the original target signature, or the fix introduced a NEW failure): advance an attempt. - **Attempt count.** `attempt = C.effectiveAttempt` — the authoritative @@ -762,6 +866,188 @@ Run these gates in order — the FIRST that fires decides this cycle's outcome: from every prior commit on this branch**, emitted via the Step 5.6 ADVANCE path (push onto the same PR). +#### Step 3.6 — Target-test verification & mark-ready gate + +Reached from the Step 3 **green-surface** branch, the Step 4 **unrelated-flake** branch +(AFTER that branch has posted its comment), and the Step 3.5 **gate-2 target-focused +fast-path** (2a — while unrelated legs are still pending). Purpose: confirm the SPECIFIC +test(s) this PR was opened to fix now PASS in the PR's own CI, and — only when they do +— transition the draft `[ci-fix-net11]` PR to **ready for review** so a maintainer sees +a validated fix instead of a draft. This is the ONLY place the loop flips draft→ready; +it is a state transition, **never** an approval or a merge (a human still reviews and +merges). Overall red on *unrelated* legs must NOT gate readiness — we validate the fix, +not the base branch's flakiness. + +**Preconditions** (ALL must hold; otherwise record `skipped: readiness N/A PR # +()` and stop this gate): +- `C.isDraft == true` — if the PR is already ready-for-review, the draft→ready + transition is already done; do NOT re-mark **and do NOT re-apply `p/0`**. `p/0` is applied + exactly ONCE, atomically with the draft→ready flip (T3 — all-or-nothing with the 🎯 audit + comment + mark-ready), so an already-ready loop PR that is MISSING `p/0` has almost + certainly had it **removed by a maintainer** de-prioritizing the PR — a pure triage action + the human-engagement guard (which inspects comments/reviews/commits, NOT label changes) + cannot see. Re-adding `p/0` every sweep would fight that maintainer indefinitely, violating + the loop's "never override a human" contract. So the loop does NOT reconcile the label: + record `already-ready PR #` and stop this gate. (Trade-off: on the rare occasion a + transient API error drops `p/0` at flip time *after* the T3 atomic pre-check passed, it is + not auto-re-added — but the PR is still ready-for-review with its 🎯 audit comment, and + re-adding `p/0` is a trivial manual action; that is strictly preferable to steam-rolling a + maintainer's deliberate de-prioritization.) +- The PR is unmistakably THIS workflow's own: `[ci-fix-net11] ` title prefix AND the + `agentic-workflows` label (mirrors the safe-output lock; the compensating scope + control if the v0.80.9 compiler drops the declarative required-*). +- You reached this gate from Step 3 (green), Step 4 (**unrelated-flake**), or the Step 3.5 + gate-2 **target-focused fast-path** (2a) — i.e. the fix is NOT implicated in any red that + has concluded. If Step 4 classified a red as **caused by the fix** (ADVANCE), do NOT run + this gate — advance the attempt instead. + +**T1 — Identify the target test(s).** From the `[ci-scan-net11]` issue signature the fix +addresses (and the PR's own diff), extract the fully-qualified test method name(s) the +fix targets — e.g. `SafeAreaShouldWorkOnAllShellTabs`. For a de-flake it is the +de-flaked test; for a product fix it is the originally-failing test(s). If NO specific +test can be identified (e.g. a product build-break rather than a test failure), this is a +**build-only fix** — there is no single test to validate cross-platform, so validate at +**whole-build** granularity rather than stopping (Step 3 only *comments*; Step 3.6 is the +sole draft→ready flip, so a build-only fix is undrafted HERE or not at all). We only reach +this gate from Step 3 (green) / Step 4 (unrelated-flake) / gate-2a, so the fix is not +implicated in any concluded red; additionally require, via the T2 timeline method on +`C.headSha`, that the primary build pipeline `maui-pr` (def 302) has CONCLUDED with EVERY +one of its platform build legs `succeeded`/`completed` and NONE failed/canceled or still +pending — the build is green on **every** platform, not just the originally-broken one (the +cross-platform guard, applied to the build instead of a test). A build-only fix is undrafted +ONLY on the strength of the auto-running `maui-pr` (def 302) whole-build green, which the +loop CAN observe: if the PR's `[ci-scan]` issue signature or its own diff indicates the +ORIGINATING failure was in a `/azp`-gated pipeline (`maui-pr-uitests` def 313 or +`maui-pr-devicetests` def 314) rather than `maui-pr` (def 302), do NOT undraft on +`maui-pr`-green alone — those pipelines do not auto-run on this PR (GITHUB_TOKEN cannot +trigger them), so a green `maui-pr` build is NOT evidence the gated build break is fixed; +record `skipped: build-only fix PR # targets gated pipeline () not run — +deferring to human` and stop this gate. Otherwise, set `TARGET := "the maui-pr build +(build-only fix — no single target test)"` and proceed to **T3** to mark ready. If any `maui-pr` build leg is still unconcluded, record `skipped: build-only fix PR +# not yet whole-build green (leg(s) pending)` and stop this gate WITHOUT marking +ready (a green subset is not enough — a still-pending build leg could yet fail). + +**T2 — Verify the target test's platform legs are green (anonymous, leg-level).** Per-test +outcomes come from the AzDO **test-results** API (`_apis/test/...`), which is **NOT reachable +anonymously — Hard-Rule 8**: those endpoints 302-redirect to a sign-in page on this runner, so +a `curl` returns an HTML redirect (not JSON) and every target test would look "not executed". +Validate instead at **leg granularity** using ONLY the anonymous `_apis/build/...` timeline +(Hard-Rule 8) — a `succeeded` category leg means every test in that category **that actually +ran** passed on that platform. This is a strong bar **only if the target test genuinely +executes**: a job can go green while one specific test is skipped at runtime (`[Ignore]`, +`Assert.Ignore()`, `Assert.Inconclusive()`, a `Skip=`/conditional `[Fact]`, an `#if`-out, or +an early `return` before the asserts). So before trusting a green leg as proof the target +passed, **confirm from the PR's OWN diff that the fix does NOT skip, ignore, disable, or +short-circuit the target test** (it adds no `[Ignore]`/`[Explicit]`, `Assert.Ignore`/ +`Assert.Inconclusive`, `Skip=`, category-exclusion, `#if`-out, or early `return` around the +target). If the fix could cause the target to be runtime-skipped rather than genuinely pass, +leg-level green is NOT sufficient evidence — record `skipped: target test may be +runtime-skipped by this fix (diff adds ); leg-green insufficient, deferring to human +on PR #` and stop this gate WITHOUT marking ready. Otherwise (the target genuinely runs +and asserts, as a de-flake or product fix does) a green category leg cannot hide a +target-test failure, so treat it as authoritative. + +Map the target test to the CI leg(s) that run it. A leg name encodes platform + test category +(e.g. `Android UITests SafeAreaEdges,Shadow`, `iOS UITests SafeAreaEdges,Shadow`, `macOS +UITests SafeAreaEdges,Shadow`, `Windows UITests SafeAreaEdges,Shadow`). Determine the target +test's UI-test category — its `[Category(UITestCategories.X)]` in the test/HostApp source, +visible in the PR diff or the test file — to know which leg-name substring identifies its legs; +for a device test, the per-platform device-test legs. + +Using the SAME build-discovery as Step 4 (filter AzDO builds by `branchName=refs/pull//merge` +or `sourceVersion == C.headSha`), read each build's **timeline** on `C.headSha` for the +pipeline(s) that RUN the target test — `maui-pr` (def 302) for unit/integration tests, +`maui-pr-uitests` (def 313) for Appium UI tests, `maui-pr-devicetests` (def 314) for device +tests: + +```bash +ORG=dnceng-public; PROJ=public; BUILD= +# Anonymous + Hard-Rule-8-compliant. NEVER call _apis/test/... (it 302-redirects to sign-in). +# Each type=="Job" record is one platform leg: result (succeeded/failed/canceled/null), +# state (completed/inProgress/pending), name (encodes " UITests "): +curl -s "https://dev.azure.com/$ORG/$PROJ/_apis/build/builds/$BUILD/timeline?api-version=7.1" \ + | tee /tmp/gh-aw/agent/timeline_${BUILD}.json \ + | jq -r '.records[] | select(.type=="Job") | "\(.result)\t\(.state)\t\(.name)"' +``` + +(The prefetch already exposes per-leg status in `C.failedLegs` / the checks context, and +`gh pr checks ` lists the same per-leg rows with platform+category in the name — use +whichever is handy; the build timeline is the authoritative cross-check on the exact build.) + +Treat a target test as **VALIDATED-GREEN** only if, on `C.headSha`, the leg that runs its +category is green on **every platform it runs on** — a fix that repairs one platform must not +silently regress the same test's leg on another, so a green leg on the originally-red platform +alone is NOT enough. Across the target pipeline's platform legs (for a UI test: the Android, +iOS, Windows, and macOS/MacCatalyst legs running the target's category; for a device test: each +device-test platform), require ALL of: +- the target's category leg is `result == "succeeded"` **and** `state == "completed"` on + **every** platform that runs it, AND +- that leg is `failed` / `canceled` / aborted on **NO** platform, AND +- **no platform is left unverified.** For each platform family the target pipeline covers, that + platform's category leg must have CONCLUDED (`state == "completed"`) on `C.headSha`. If a + platform simply has no leg for the target's category, the test does not run there — that is + fine, not a gap. But if a platform's category leg has **not concluded** (`state` is + `inProgress` / pending, or an `/azp`-gated `maui-pr-uitests` / `maui-pr-devicetests` leg that + has not been kicked), the test's status on that platform is UNKNOWN → the fix is NOT yet + validated across platforms: record `skipped: target test green on but + not yet verified on (leg(s) pending / need /azp run) on PR #` + and stop this gate WITHOUT marking ready. + +A target test whose category leg never concluded on ANY platform (**not executed** anywhere — +e.g. its `/azp`-gated pipeline has not been kicked) is likewise NOT validated: record `skipped: +target test not yet executed on PR # ( not run — needs /azp run)` and stop +this gate WITHOUT marking ready. Do NOT overclaim — a green *sibling* leg (a different category +on the same platform) is not the target's leg, and a green leg on one platform is not a pass on +the others. + +**T3 — Mark ready + report.** If EVERY target test is VALIDATED-GREEN. The 🎯 comment, +the mark-ready, and the `p/0` label are THREE SEPARATE safe-outputs — the comment +existing does NOT prove the mark-ready took effect, so they are tracked independently: + +- **Comment idempotency (dup-suppress only — never gates the mark-ready).** We only reach + T3 while `C.isDraft == true` (the precondition stops an already-ready PR before here). So + if a prior bot comment for THIS head that carries the leading `🎯` anchor AND the + contiguous phrase `validated green on ` already exists (BOTH T3 variants — + `🎯 Target test validated green on ` and the build-only `🎯 Build validated green on + ` — contain that exact contiguous phrase, so this recognizes both; keying on + `target test` alone would instead miss the build-only comment and re-post it every sweep. + **Require the `🎯` anchor** so the match can NEVER be loosened to a gapped + `validated…green…on` form that would false-positive on the Step 3 `✅ … validated — the + fix's CI is green on ` surface comment and wrongly suppress the audit 🎯), the comment + landed on an earlier sweep but the **mark-ready did NOT take + effect** (the PR is still a draft) — set `SUPPRESS_COMMENT = true` (do not re-post the + duplicate 🎯 comment) but STILL complete the mark-ready + label below. Never treat the + comment's existence as "already marked ready" while the PR is still a draft. Record this + case as `re-marking PR # (🎯 present; mark-ready did not take on a prior sweep)`. +- **Atomicity budget pre-check (Hard-Rule 7).** Determine the outputs this gate will emit: + `mark_pull_request_as_ready_for_review` + `add_labels` always, plus `add_comment` unless + `SUPPRESS_COMMENT`. Verify a free per-run slot remains in EACH bucket you are about to + use. If ANY required bucket is exhausted, emit NONE of them — record `skipped: per-run + cap reached; deferring mark-ready PR # to next cycle` and stop this gate. Never mark a + PR ready (or label it) unless its 🎯 audit comment is GUARANTEED to exist for this exact + head SHA — either posted in this same sweep, or (in the `SUPPRESS_COMMENT` re-marking case) + already present from a prior sweep. The outputs you DO emit either all land or all defer + together; the audit trail must never be absent, but it is NOT re-posted when it already exists. +- **Dry-run gate (Step 0):** if `dry_run == "true"`, emit NOTHING — print the intended + readiness comment and "would mark ready + add p/0" to the run log, tally `dry-run: + would-mark-ready PR #`, and stop. +- Otherwise emit for THIS PR number ``: + 1. `add_comment` (ONLY if not `SUPPRESS_COMMENT`): `🎯 Target test validated green on + — passed on ALL platforms it runs on (, + buildId ). — not + caused by this fix."> Transitioning this PR from draft to ready for review and adding + `p/0`; a maintainer still reviews and merges.` (For a **build-only fix** — the T1 + whole-build fallback, no single target test — phrase the first clause as `🎯 Build + validated green on — the maui-pr build passed on ALL platforms (buildId + ).` instead of naming a test.) + 2. `mark_pull_request_as_ready_for_review` with `reason:` a one-line justification + naming the validated test(s) and ``. + 3. `add_labels` with `labels: ["p/0"]` for PR # — put the now-review-ready fix into + the team's p/0 priority queue so it is triaged, not lost in the draft backlog. (If + the PR somehow already carries `p/0`, this is a harmless no-op.) +- Record `marked-ready PR # (target green on ALL platforms on , + labeled p/0)` and stop. + #### Step 3.5.R — Maintainer change-request response (Track C) Reached from Step 3.5 gate 0. Goal: when an **eligible human reviewer** (Hard-Rule @@ -1395,7 +1681,19 @@ Filed by [`ci-status-fix-net11`](https://github.com/dotnet/maui/blob/main/.githu ### Step 8 — Per-issue tally + end-of-run summary -Per issue, append one outcome line to `/tmp/gh-aw/agent/coverage.txt`: +Per issue, append **one** outcome line to `/tmp/gh-aw/agent/coverage.txt` — the +terminal outcome for the cycle. When one cycle produces a chained pair — a PR gets a +same-cycle precursor line and **then** a Step 3.6 readiness line (`marked-ready` on the +draft→ready flip, or `already-ready` on a later steady-state sweep of an already-ready +PR) — record ONLY the terminal Step 3.6 readiness line: it supersedes ANY same-cycle +non-readiness precursor for that PR, so an aggregator keying on one-line-per-issue never +double-counts. The precursor may be a Step 3 green line (`surfaced-green` on the first ✅, +or `already-surfaced` when it already exists) OR — when an unrelated-flake red and a +target-green coincide — a Step 4 `annotated-flake` line; either way the readiness line is +terminal, and the superseded precursor's signal survives in the PR's 🎯/♻️ comment so no +information is lost. This covers the flip cycle (`surfaced-green` → `marked-ready`), the +post-flip steady state (`already-surfaced` → `already-ready`), and the flake-coincident +flip (`annotated-flake` → `marked-ready`): ``` # net11.0 attempt- @@ -1406,8 +1704,14 @@ Per issue, append one outcome line to `/tmp/gh-aw/agent/coverage.txt`: `advance-PR # attempt /10` (ADVANCE mode — new commit pushed to the existing PR), `surfaced-green PR #` (fix's own CI went green; commented for review, did not advance), `annotated-flake PR #` (red was unrelated flake; -commented, attempt NOT burned), `waiting PR #` (CI not settled yet), -`dry-run: would-`, `skipped: `. (The +commented, attempt NOT burned), `marked-ready PR #` (the specific fixed test +was confirmed green in the PR's own CI; draft flipped to ready for review), +`already-surfaced PR #` (the fix's ✅ green comment already existed this sweep; +re-post suppressed — superseded by the Step 3.6 readiness line when the PR also flips +ready that cycle), `already-ready PR #` (terminal steady state — the PR was flipped +ready on a prior cycle and remains green on an unchanged head; no action taken, +supersedes `already-surfaced`), `waiting PR #` (CI not settled yet), +`dry-run: would-`, `skipped: `. (The `needs-human-PR` outcome is reserved for the deferred hand-off — Step 6 currently records a skip instead, so it is not emitted.) diff --git a/.github/workflows/ci-status-fix.lock.yml b/.github/workflows/ci-status-fix.lock.yml index 93dc5969cb0d..65511f2b3259 100644 --- a/.github/workflows/ci-status-fix.lock.yml +++ b/.github/workflows/ci-status-fix.lock.yml @@ -1,4 +1,4 @@ -# gh-aw-metadata: {"schema_version":"v4","frontmatter_hash":"f014161967589ebcaf1ac5ec6f3c88ee5a4388d28b55a07dc36c45b7b2aebab6","body_hash":"86c6b2d4c3df13da14c6a3c8b211381fcc9f97bb1951ff7b80588a2c5338e00c","compiler_version":"v0.80.9","strict":true,"agent_id":"copilot","agent_model":"claude-opus-4.8","engine_versions":{"copilot":"1.0.63"}} +# gh-aw-metadata: {"schema_version":"v4","frontmatter_hash":"b0ebf8a2f179900cfe3638e994b27a43cec52bdbdd2331dc1bd4d65762fa2d6b","body_hash":"1825e2b1f7c7bd88241bf37580655489360f9bebc806edd00837a52ce4ad83a0","compiler_version":"v0.80.9","strict":true,"agent_id":"copilot","agent_model":"claude-opus-4.8","engine_versions":{"copilot":"1.0.63"}} # gh-aw-manifest: {"version":1,"secrets":["COPILOT_PAT_0","COPILOT_PAT_1","COPILOT_PAT_2","COPILOT_PAT_3","COPILOT_PAT_4","COPILOT_PAT_5","COPILOT_PAT_6","COPILOT_PAT_7","COPILOT_PAT_8","COPILOT_PAT_9","GH_AW_CI_TRIGGER_TOKEN","GH_AW_GITHUB_MCP_SERVER_TOKEN","GH_AW_GITHUB_TOKEN","GITHUB_TOKEN"],"actions":[{"repo":"actions/checkout","sha":"34e114876b0b11c390a56381ad16ebd13914f8d5","version":"v4"},{"repo":"actions/checkout","sha":"9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0","version":"v7.0.0"},{"repo":"actions/checkout","sha":"de0fac2e4500dabe0009e67214ff5f5447ce83dd","version":"v6.0.2"},{"repo":"actions/download-artifact","sha":"3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c","version":"v8.0.1"},{"repo":"actions/github-script","sha":"3a2844b7e9c422d3c10d287c895573f7108da1b3","version":"v9.0.0"},{"repo":"actions/setup-node","sha":"48b55a011bda9f5d6aeb4c2d9c7362e8dae4041e","version":"v6.4.0"},{"repo":"actions/upload-artifact","sha":"043fb46d1a93c77aae656e7c1c64a875d1fc6a0a","version":"v7.0.1"},{"repo":"github/gh-aw-actions/setup","sha":"8c7d04ebf1ece56cd381446125da3e0f6896294a","version":"v0.80.9"}],"containers":[{"image":"ghcr.io/github/gh-aw-firewall/agent:0.27.7","digest":"sha256:aae231e4635c8999d039c132f1602d3df850fe9b84a00aa2b5ac981179b5661c","pinned_image":"ghcr.io/github/gh-aw-firewall/agent:0.27.7@sha256:aae231e4635c8999d039c132f1602d3df850fe9b84a00aa2b5ac981179b5661c"},{"image":"ghcr.io/github/gh-aw-firewall/api-proxy:0.27.7","digest":"sha256:009caf2e3d88fa77b64e9a03a95a228fc58db0f1701c6d324b29ba5a3c7c79b6","pinned_image":"ghcr.io/github/gh-aw-firewall/api-proxy:0.27.7@sha256:009caf2e3d88fa77b64e9a03a95a228fc58db0f1701c6d324b29ba5a3c7c79b6"},{"image":"ghcr.io/github/gh-aw-firewall/squid:0.27.7","digest":"sha256:deb1d4e19de62d51cee0508057a596a19315c3423ada4d675cad136dc8037c96","pinned_image":"ghcr.io/github/gh-aw-firewall/squid:0.27.7@sha256:deb1d4e19de62d51cee0508057a596a19315c3423ada4d675cad136dc8037c96"},{"image":"ghcr.io/github/gh-aw-mcpg:v0.3.27","digest":"sha256:fe984bddde4ec05d756d9043edb0a32912e6b7b72f6a121b1082f29221421cc7","pinned_image":"ghcr.io/github/gh-aw-mcpg:v0.3.27@sha256:fe984bddde4ec05d756d9043edb0a32912e6b7b72f6a121b1082f29221421cc7"},{"image":"ghcr.io/github/gh-aw-node","digest":"sha256:529d02eb970b1161aa25c593a9c3df57fdfad5a8add328cb3b6eccef66f3183b","pinned_image":"ghcr.io/github/gh-aw-node@sha256:529d02eb970b1161aa25c593a9c3df57fdfad5a8add328cb3b6eccef66f3183b"},{"image":"ghcr.io/github/github-mcp-server:v1.4.0","digest":"sha256:2afb26356481d1a350e14544a6e160f7f7ec1561a1ea309b823665abf0309036","pinned_image":"ghcr.io/github/github-mcp-server:v1.4.0@sha256:2afb26356481d1a350e14544a6e160f7f7ec1561a1ea309b823665abf0309036"}]} # This file was automatically generated by gh-aw (v0.80.9). DO NOT EDIT. To debug this workflow, load the skill at https://github.com/github/gh-aw/blob/main/debug.md # @@ -37,7 +37,9 @@ # humans (the open tracking issue is the hand-off surface; a dedicated # [ci-fix][needs-human] PR is planned but currently deferred — see Step 6). CI on # the PR is kicked by a human `/azp run` for now; the loop watches, classifies, -# and re-fixes autonomously between kicks. +# and re-fixes autonomously between kicks. When the SPECIFIC test a PR fixed is +# confirmed green in that PR's own CI, the loop marks the draft PR ready for review +# (a state transition only — it never approves or merges). # Never mutes tests, but # de-flakes genuinely flaky ones (deterministic synchronization, no retries / # timeout bumps). Always skips visual-regression / screenshot issues. @@ -273,24 +275,24 @@ jobs: run: | bash "${RUNNER_TEMP}/gh-aw/actions/create_prompt_first.sh" { - cat << 'GH_AW_PROMPT_25cf75111b670167_EOF' + cat << 'GH_AW_PROMPT_2ac1d0b4083428f8_EOF' - GH_AW_PROMPT_25cf75111b670167_EOF + GH_AW_PROMPT_2ac1d0b4083428f8_EOF cat "${RUNNER_TEMP}/gh-aw/prompts/xpia.md" cat "${RUNNER_TEMP}/gh-aw/prompts/temp_folder_prompt.md" cat "${RUNNER_TEMP}/gh-aw/prompts/markdown.md" cat "${RUNNER_TEMP}/gh-aw/prompts/safe_outputs_prompt.md" - cat << 'GH_AW_PROMPT_25cf75111b670167_EOF' + cat << 'GH_AW_PROMPT_2ac1d0b4083428f8_EOF' - Tools: add_comment(max:3), create_pull_request(max:3), update_pull_request(max:3), push_to_pull_request_branch(max:3), missing_tool, missing_data, noop - GH_AW_PROMPT_25cf75111b670167_EOF + Tools: add_comment(max:6), create_pull_request(max:3), update_pull_request(max:3), mark_pull_request_as_ready_for_review(max:3), add_labels(max:3), push_to_pull_request_branch(max:3), missing_tool, missing_data, noop + GH_AW_PROMPT_2ac1d0b4083428f8_EOF cat "${RUNNER_TEMP}/gh-aw/prompts/safe_outputs_create_pull_request.md" cat "${RUNNER_TEMP}/gh-aw/prompts/safe_outputs_push_to_pr_branch.md" - cat << 'GH_AW_PROMPT_25cf75111b670167_EOF' + cat << 'GH_AW_PROMPT_2ac1d0b4083428f8_EOF' - GH_AW_PROMPT_25cf75111b670167_EOF + GH_AW_PROMPT_2ac1d0b4083428f8_EOF cat "${RUNNER_TEMP}/gh-aw/prompts/mcp_cli_tools_prompt.md" - cat << 'GH_AW_PROMPT_25cf75111b670167_EOF' + cat << 'GH_AW_PROMPT_2ac1d0b4083428f8_EOF' The following GitHub context information is available for this workflow: {{#if github.actor}} @@ -332,12 +334,12 @@ jobs: stop immediately and report the limitation rather than spending turns trying to work around it. - GH_AW_PROMPT_25cf75111b670167_EOF + GH_AW_PROMPT_2ac1d0b4083428f8_EOF cat "${RUNNER_TEMP}/gh-aw/prompts/github_mcp_tools_with_safeoutputs_prompt.md" - cat << 'GH_AW_PROMPT_25cf75111b670167_EOF' + cat << 'GH_AW_PROMPT_2ac1d0b4083428f8_EOF' {{#runtime-import .github/workflows/ci-status-fix.md}} - GH_AW_PROMPT_25cf75111b670167_EOF + GH_AW_PROMPT_2ac1d0b4083428f8_EOF } > "$GH_AW_PROMPT" - name: Interpolate variables and render templates uses: actions/github-script@3a2844b7e9c422d3c10d287c895573f7108da1b3 # v9.0.0 @@ -554,16 +556,18 @@ jobs: mkdir -p "${RUNNER_TEMP}/gh-aw/safeoutputs" mkdir -p /tmp/gh-aw/safeoutputs mkdir -p /tmp/gh-aw/mcp-logs/safeoutputs - cat > "${RUNNER_TEMP}/gh-aw/safeoutputs/config.json" << 'GH_AW_SAFE_OUTPUTS_CONFIG_fa2a69fad5fd0485_EOF' - {"add_comment":{"discussions":false,"max":3,"required_labels":["agentic-workflows"],"required_title_prefix":"[ci-fix] ","target":"*"},"create_pull_request":{"allowed_base_branches":["main"],"allowed_branches":["ci-fix/**"],"allowed_files":["src/Core/**","src/Controls/**","src/Essentials/**","src/BlazorWebView/**","src/TestUtils/**","src/Templates/**","**/PublicAPI.Unshipped.txt"],"base_branch":"main","draft":true,"labels":["agentic-workflows"],"max":3,"max_patch_files":100,"max_patch_size":4096,"protect_top_level_dot_folders":true,"protected_files":["package.json","bun.lockb","bunfig.toml","deno.json","deno.jsonc","deno.lock","global.json","NuGet.Config","Directory.Packages.props","mix.exs","mix.lock","go.mod","go.sum","stack.yaml","stack.yaml.lock","pom.xml","build.gradle","build.gradle.kts","settings.gradle","settings.gradle.kts","gradle.properties","package-lock.json","yarn.lock","pnpm-lock.yaml","npm-shrinkwrap.json","requirements.txt","Pipfile","Pipfile.lock","pyproject.toml","setup.py","setup.cfg","Gemfile","Gemfile.lock","uv.lock","CODEOWNERS","DESIGN.md","README.md","CONTRIBUTING.md","CHANGELOG.md","SECURITY.md","CODE_OF_CONDUCT.md","AGENTS.md","CLAUDE.md","GEMINI.md"],"protected_files_policy":"request_review","title_prefix":"[ci-fix] "},"create_report_incomplete_issue":{},"missing_data":{},"missing_tool":{},"noop":{"max":1,"report-as-issue":"true"},"push_to_pull_request_branch":{"allowed_files":["src/Core/**","src/Controls/**","src/Essentials/**","src/BlazorWebView/**","src/TestUtils/**","src/Templates/**","**/PublicAPI.Unshipped.txt"],"if_no_changes":"warn","max":3,"max_patch_size":4096,"protect_top_level_dot_folders":true,"protected_files":["package.json","bun.lockb","bunfig.toml","deno.json","deno.jsonc","deno.lock","global.json","NuGet.Config","Directory.Packages.props","mix.exs","mix.lock","go.mod","go.sum","stack.yaml","stack.yaml.lock","pom.xml","build.gradle","build.gradle.kts","settings.gradle","settings.gradle.kts","gradle.properties","package-lock.json","yarn.lock","pnpm-lock.yaml","npm-shrinkwrap.json","requirements.txt","Pipfile","Pipfile.lock","pyproject.toml","setup.py","setup.cfg","Gemfile","Gemfile.lock","uv.lock","CODEOWNERS","DESIGN.md","README.md","CONTRIBUTING.md","CHANGELOG.md","SECURITY.md","CODE_OF_CONDUCT.md","AGENTS.md","CLAUDE.md","GEMINI.md"],"required_labels":["agentic-workflows"],"target":"*","title_prefix":"[ci-fix] "},"report_incomplete":{},"update_pull_request":{"allow_body":true,"allow_title":false,"max":3,"target":"*","update_branch":false}} - GH_AW_SAFE_OUTPUTS_CONFIG_fa2a69fad5fd0485_EOF + cat > "${RUNNER_TEMP}/gh-aw/safeoutputs/config.json" << 'GH_AW_SAFE_OUTPUTS_CONFIG_0644bf1434ca0071_EOF' + {"add_comment":{"discussions":false,"max":6,"required_labels":["agentic-workflows"],"required_title_prefix":"[ci-fix] ","target":"*"},"add_labels":{"allowed":["p/0"],"max":3,"required_labels":["agentic-workflows"],"required_title_prefix":"[ci-fix] ","target":"*"},"create_pull_request":{"allowed_base_branches":["main"],"allowed_branches":["ci-fix/**"],"allowed_files":["src/Core/**","src/Controls/**","src/Essentials/**","src/BlazorWebView/**","src/TestUtils/**","src/Templates/**","**/PublicAPI.Unshipped.txt"],"base_branch":"main","draft":true,"labels":["agentic-workflows"],"max":3,"max_patch_files":100,"max_patch_size":4096,"protect_top_level_dot_folders":true,"protected_files":["package.json","bun.lockb","bunfig.toml","deno.json","deno.jsonc","deno.lock","global.json","NuGet.Config","Directory.Packages.props","mix.exs","mix.lock","go.mod","go.sum","stack.yaml","stack.yaml.lock","pom.xml","build.gradle","build.gradle.kts","settings.gradle","settings.gradle.kts","gradle.properties","package-lock.json","yarn.lock","pnpm-lock.yaml","npm-shrinkwrap.json","requirements.txt","Pipfile","Pipfile.lock","pyproject.toml","setup.py","setup.cfg","Gemfile","Gemfile.lock","uv.lock","CODEOWNERS","DESIGN.md","README.md","CONTRIBUTING.md","CHANGELOG.md","SECURITY.md","CODE_OF_CONDUCT.md","AGENTS.md","CLAUDE.md","GEMINI.md"],"protected_files_policy":"request_review","title_prefix":"[ci-fix] "},"create_report_incomplete_issue":{},"mark_pull_request_as_ready_for_review":{"max":3,"required_labels":["agentic-workflows"],"required_title_prefix":"[ci-fix] ","target":"*"},"missing_data":{},"missing_tool":{},"noop":{"max":1,"report-as-issue":"true"},"push_to_pull_request_branch":{"allowed_files":["src/Core/**","src/Controls/**","src/Essentials/**","src/BlazorWebView/**","src/TestUtils/**","src/Templates/**","**/PublicAPI.Unshipped.txt"],"if_no_changes":"warn","max":3,"max_patch_size":4096,"protect_top_level_dot_folders":true,"protected_files":["package.json","bun.lockb","bunfig.toml","deno.json","deno.jsonc","deno.lock","global.json","NuGet.Config","Directory.Packages.props","mix.exs","mix.lock","go.mod","go.sum","stack.yaml","stack.yaml.lock","pom.xml","build.gradle","build.gradle.kts","settings.gradle","settings.gradle.kts","gradle.properties","package-lock.json","yarn.lock","pnpm-lock.yaml","npm-shrinkwrap.json","requirements.txt","Pipfile","Pipfile.lock","pyproject.toml","setup.py","setup.cfg","Gemfile","Gemfile.lock","uv.lock","CODEOWNERS","DESIGN.md","README.md","CONTRIBUTING.md","CHANGELOG.md","SECURITY.md","CODE_OF_CONDUCT.md","AGENTS.md","CLAUDE.md","GEMINI.md"],"required_labels":["agentic-workflows"],"target":"*","title_prefix":"[ci-fix] "},"report_incomplete":{},"update_pull_request":{"allow_body":true,"allow_title":false,"max":3,"target":"*","update_branch":false}} + GH_AW_SAFE_OUTPUTS_CONFIG_0644bf1434ca0071_EOF - name: Generate Safe Outputs Tools env: GH_AW_TOOLS_META_JSON: | { "description_suffixes": { - "add_comment": " CONSTRAINTS: Maximum 3 comment(s) can be added. Target: *. Supports reply_to_id for discussion threading.", + "add_comment": " CONSTRAINTS: Maximum 6 comment(s) can be added. Target: *. Supports reply_to_id for discussion threading.", + "add_labels": " CONSTRAINTS: Maximum 3 label(s) can be added. Only these labels are allowed: [\"p/0\"]. Target: *.", "create_pull_request": " CONSTRAINTS: Maximum 3 pull request(s) can be created. Title will be prefixed with \"[ci-fix] \". Labels [\"agentic-workflows\"] will be automatically added. Only these labels are allowed: [\"agentic-workflows\"]. PRs will be created as drafts.", + "mark_pull_request_as_ready_for_review": " CONSTRAINTS: Maximum 3 pull request(s) can be marked as ready for review.", "push_to_pull_request_branch": " CONSTRAINTS: Maximum 3 push(es) can be made. The target pull request title must start with \"[ci-fix] \".", "update_pull_request": " CONSTRAINTS: Maximum 3 pull request(s) can be updated. Target: *." }, @@ -594,6 +598,25 @@ jobs: } } }, + "add_labels": { + "defaultMax": 5, + "fields": { + "item_number": { + "issueNumberOrTemporaryId": true + }, + "labels": { + "required": true, + "type": "array", + "itemType": "string", + "itemSanitize": true, + "itemMaxLength": 128 + }, + "repo": { + "type": "string", + "maxLength": 256 + } + } + }, "create_pull_request": { "defaultMax": 1, "fields": { @@ -635,6 +658,24 @@ jobs: } } }, + "mark_pull_request_as_ready_for_review": { + "defaultMax": 1, + "fields": { + "pull_request_number": { + "issueOrPRNumber": true + }, + "reason": { + "required": true, + "type": "string", + "sanitize": true, + "maxLength": 65000 + }, + "repo": { + "type": "string", + "maxLength": 256 + } + } + }, "missing_data": { "defaultMax": 20, "fields": { @@ -1494,7 +1535,7 @@ jobs: uses: actions/github-script@3a2844b7e9c422d3c10d287c895573f7108da1b3 # v9.0.0 env: WORKFLOW_NAME: "CI Failure Fixer (main)" - WORKFLOW_DESCRIPTION: "Periodic pass over open ci-scan tracking issues filed by the main-branch CI\nfailure scanner (.github/workflows/ci-status-main.md). This workflow targets\nthe `main` branch EXCLUSIVELY: it processes only issues labelled ci-scan and\nopens every PR against main. (The net11.0 branch is handled by the parallel\n.github/workflows/ci-status-fix-net11.md — the two are split because gh-aw can\nonly transport a fix relative to ONE static base branch per workflow, and the\nmain↔net11.0 divergence exceeds gh-aw's 10 MB transport-patch cap.) The fixer\nopens ONE draft [ci-fix] PR per actionable issue against main, then WATCHES\nthat PR's own CI on later runs: when the fix's CI comes back red and the red is\ncaused by the fix itself, it pushes a fresh follow-up fix onto the SAME PR\nbranch (never a second PR) — up to 10 attempts — then stops and defers to\nhumans (the open tracking issue is the hand-off surface; a dedicated\n[ci-fix][needs-human] PR is planned but currently deferred — see Step 6). CI on\nthe PR is kicked by a human `/azp run` for now; the loop watches, classifies,\nand re-fixes autonomously between kicks.\nNever mutes tests, but\nde-flakes genuinely flaky ones (deterministic synchronization, no retries /\ntimeout bumps). Always skips visual-regression / screenshot issues." + WORKFLOW_DESCRIPTION: "Periodic pass over open ci-scan tracking issues filed by the main-branch CI\nfailure scanner (.github/workflows/ci-status-main.md). This workflow targets\nthe `main` branch EXCLUSIVELY: it processes only issues labelled ci-scan and\nopens every PR against main. (The net11.0 branch is handled by the parallel\n.github/workflows/ci-status-fix-net11.md — the two are split because gh-aw can\nonly transport a fix relative to ONE static base branch per workflow, and the\nmain↔net11.0 divergence exceeds gh-aw's 10 MB transport-patch cap.) The fixer\nopens ONE draft [ci-fix] PR per actionable issue against main, then WATCHES\nthat PR's own CI on later runs: when the fix's CI comes back red and the red is\ncaused by the fix itself, it pushes a fresh follow-up fix onto the SAME PR\nbranch (never a second PR) — up to 10 attempts — then stops and defers to\nhumans (the open tracking issue is the hand-off surface; a dedicated\n[ci-fix][needs-human] PR is planned but currently deferred — see Step 6). CI on\nthe PR is kicked by a human `/azp run` for now; the loop watches, classifies,\nand re-fixes autonomously between kicks. When the SPECIFIC test a PR fixed is\nconfirmed green in that PR's own CI, the loop marks the draft PR ready for review\n(a state transition only — it never approves or merges).\nNever mutes tests, but\nde-flakes genuinely flaky ones (deterministic synchronization, no retries /\ntimeout bumps). Always skips visual-regression / screenshot issues." HAS_PATCH: ${{ needs.agent.outputs.has_patch }} with: script: | @@ -1910,7 +1951,7 @@ jobs: GH_AW_ALLOWED_DOMAINS: "*.blob.core.windows.net,*.githubusercontent.com,api.business.githubcopilot.com,api.enterprise.githubcopilot.com,api.github.com,api.githubcopilot.com,api.individual.githubcopilot.com,api.snapcraft.io,archive.ubuntu.com,azure.archive.ubuntu.com,codeload.github.com,crl.geotrust.com,crl.globalsign.com,crl.identrust.com,crl.sectigo.com,crl.thawte.com,crl.usertrust.com,crl.verisign.com,crl3.digicert.com,crl4.digicert.com,crls.ssl.com,dev.azure.com,docs.github.com,github-cloud.githubusercontent.com,github-cloud.s3.amazonaws.com,github.blog,github.com,github.githubassets.com,helix.dot.net,host.docker.internal,json-schema.org,json.schemastore.org,keyserver.ubuntu.com,lfs.github.com,objects.githubusercontent.com,ocsp.digicert.com,ocsp.geotrust.com,ocsp.globalsign.com,ocsp.identrust.com,ocsp.sectigo.com,ocsp.ssl.com,ocsp.thawte.com,ocsp.usertrust.com,ocsp.verisign.com,packagecloud.io,packages.cloud.google.com,packages.microsoft.com,patch-diff.githubusercontent.com,ppa.launchpad.net,raw.githubusercontent.com,registry.npmjs.org,s.symcb.com,s.symcd.com,security.ubuntu.com,telemetry.enterprise.githubcopilot.com,ts-crl.ws.symantec.com,ts-ocsp.ws.symantec.com,www.googleapis.com" GITHUB_SERVER_URL: ${{ github.server_url }} GITHUB_API_URL: ${{ github.api_url }} - GH_AW_SAFE_OUTPUTS_HANDLER_CONFIG: "{\"add_comment\":{\"discussions\":false,\"max\":3,\"required_labels\":[\"agentic-workflows\"],\"required_title_prefix\":\"[ci-fix] \",\"target\":\"*\"},\"create_pull_request\":{\"allowed_base_branches\":[\"main\"],\"allowed_branches\":[\"ci-fix/**\"],\"allowed_files\":[\"src/Core/**\",\"src/Controls/**\",\"src/Essentials/**\",\"src/BlazorWebView/**\",\"src/TestUtils/**\",\"src/Templates/**\",\"**/PublicAPI.Unshipped.txt\"],\"base_branch\":\"main\",\"draft\":true,\"labels\":[\"agentic-workflows\"],\"max\":3,\"max_patch_files\":100,\"max_patch_size\":4096,\"protect_top_level_dot_folders\":true,\"protected_files\":[\"package.json\",\"bun.lockb\",\"bunfig.toml\",\"deno.json\",\"deno.jsonc\",\"deno.lock\",\"global.json\",\"NuGet.Config\",\"Directory.Packages.props\",\"mix.exs\",\"mix.lock\",\"go.mod\",\"go.sum\",\"stack.yaml\",\"stack.yaml.lock\",\"pom.xml\",\"build.gradle\",\"build.gradle.kts\",\"settings.gradle\",\"settings.gradle.kts\",\"gradle.properties\",\"package-lock.json\",\"yarn.lock\",\"pnpm-lock.yaml\",\"npm-shrinkwrap.json\",\"requirements.txt\",\"Pipfile\",\"Pipfile.lock\",\"pyproject.toml\",\"setup.py\",\"setup.cfg\",\"Gemfile\",\"Gemfile.lock\",\"uv.lock\",\"CODEOWNERS\",\"DESIGN.md\",\"README.md\",\"CONTRIBUTING.md\",\"CHANGELOG.md\",\"SECURITY.md\",\"CODE_OF_CONDUCT.md\",\"AGENTS.md\",\"CLAUDE.md\",\"GEMINI.md\"],\"protected_files_policy\":\"request_review\",\"title_prefix\":\"[ci-fix] \"},\"create_report_incomplete_issue\":{},\"missing_data\":{},\"missing_tool\":{},\"noop\":{\"max\":1,\"report-as-issue\":\"true\"},\"push_to_pull_request_branch\":{\"allowed_files\":[\"src/Core/**\",\"src/Controls/**\",\"src/Essentials/**\",\"src/BlazorWebView/**\",\"src/TestUtils/**\",\"src/Templates/**\",\"**/PublicAPI.Unshipped.txt\"],\"if_no_changes\":\"warn\",\"max\":3,\"max_patch_size\":4096,\"protect_top_level_dot_folders\":true,\"protected_files\":[\"package.json\",\"bun.lockb\",\"bunfig.toml\",\"deno.json\",\"deno.jsonc\",\"deno.lock\",\"global.json\",\"NuGet.Config\",\"Directory.Packages.props\",\"mix.exs\",\"mix.lock\",\"go.mod\",\"go.sum\",\"stack.yaml\",\"stack.yaml.lock\",\"pom.xml\",\"build.gradle\",\"build.gradle.kts\",\"settings.gradle\",\"settings.gradle.kts\",\"gradle.properties\",\"package-lock.json\",\"yarn.lock\",\"pnpm-lock.yaml\",\"npm-shrinkwrap.json\",\"requirements.txt\",\"Pipfile\",\"Pipfile.lock\",\"pyproject.toml\",\"setup.py\",\"setup.cfg\",\"Gemfile\",\"Gemfile.lock\",\"uv.lock\",\"CODEOWNERS\",\"DESIGN.md\",\"README.md\",\"CONTRIBUTING.md\",\"CHANGELOG.md\",\"SECURITY.md\",\"CODE_OF_CONDUCT.md\",\"AGENTS.md\",\"CLAUDE.md\",\"GEMINI.md\"],\"required_labels\":[\"agentic-workflows\"],\"target\":\"*\",\"title_prefix\":\"[ci-fix] \"},\"report_incomplete\":{},\"update_pull_request\":{\"allow_body\":true,\"allow_title\":false,\"max\":3,\"target\":\"*\",\"update_branch\":false}}" + GH_AW_SAFE_OUTPUTS_HANDLER_CONFIG: "{\"add_comment\":{\"discussions\":false,\"max\":6,\"required_labels\":[\"agentic-workflows\"],\"required_title_prefix\":\"[ci-fix] \",\"target\":\"*\"},\"add_labels\":{\"allowed\":[\"p/0\"],\"max\":3,\"required_labels\":[\"agentic-workflows\"],\"required_title_prefix\":\"[ci-fix] \",\"target\":\"*\"},\"create_pull_request\":{\"allowed_base_branches\":[\"main\"],\"allowed_branches\":[\"ci-fix/**\"],\"allowed_files\":[\"src/Core/**\",\"src/Controls/**\",\"src/Essentials/**\",\"src/BlazorWebView/**\",\"src/TestUtils/**\",\"src/Templates/**\",\"**/PublicAPI.Unshipped.txt\"],\"base_branch\":\"main\",\"draft\":true,\"labels\":[\"agentic-workflows\"],\"max\":3,\"max_patch_files\":100,\"max_patch_size\":4096,\"protect_top_level_dot_folders\":true,\"protected_files\":[\"package.json\",\"bun.lockb\",\"bunfig.toml\",\"deno.json\",\"deno.jsonc\",\"deno.lock\",\"global.json\",\"NuGet.Config\",\"Directory.Packages.props\",\"mix.exs\",\"mix.lock\",\"go.mod\",\"go.sum\",\"stack.yaml\",\"stack.yaml.lock\",\"pom.xml\",\"build.gradle\",\"build.gradle.kts\",\"settings.gradle\",\"settings.gradle.kts\",\"gradle.properties\",\"package-lock.json\",\"yarn.lock\",\"pnpm-lock.yaml\",\"npm-shrinkwrap.json\",\"requirements.txt\",\"Pipfile\",\"Pipfile.lock\",\"pyproject.toml\",\"setup.py\",\"setup.cfg\",\"Gemfile\",\"Gemfile.lock\",\"uv.lock\",\"CODEOWNERS\",\"DESIGN.md\",\"README.md\",\"CONTRIBUTING.md\",\"CHANGELOG.md\",\"SECURITY.md\",\"CODE_OF_CONDUCT.md\",\"AGENTS.md\",\"CLAUDE.md\",\"GEMINI.md\"],\"protected_files_policy\":\"request_review\",\"title_prefix\":\"[ci-fix] \"},\"create_report_incomplete_issue\":{},\"mark_pull_request_as_ready_for_review\":{\"max\":3,\"required_labels\":[\"agentic-workflows\"],\"required_title_prefix\":\"[ci-fix] \",\"target\":\"*\"},\"missing_data\":{},\"missing_tool\":{},\"noop\":{\"max\":1,\"report-as-issue\":\"true\"},\"push_to_pull_request_branch\":{\"allowed_files\":[\"src/Core/**\",\"src/Controls/**\",\"src/Essentials/**\",\"src/BlazorWebView/**\",\"src/TestUtils/**\",\"src/Templates/**\",\"**/PublicAPI.Unshipped.txt\"],\"if_no_changes\":\"warn\",\"max\":3,\"max_patch_size\":4096,\"protect_top_level_dot_folders\":true,\"protected_files\":[\"package.json\",\"bun.lockb\",\"bunfig.toml\",\"deno.json\",\"deno.jsonc\",\"deno.lock\",\"global.json\",\"NuGet.Config\",\"Directory.Packages.props\",\"mix.exs\",\"mix.lock\",\"go.mod\",\"go.sum\",\"stack.yaml\",\"stack.yaml.lock\",\"pom.xml\",\"build.gradle\",\"build.gradle.kts\",\"settings.gradle\",\"settings.gradle.kts\",\"gradle.properties\",\"package-lock.json\",\"yarn.lock\",\"pnpm-lock.yaml\",\"npm-shrinkwrap.json\",\"requirements.txt\",\"Pipfile\",\"Pipfile.lock\",\"pyproject.toml\",\"setup.py\",\"setup.cfg\",\"Gemfile\",\"Gemfile.lock\",\"uv.lock\",\"CODEOWNERS\",\"DESIGN.md\",\"README.md\",\"CONTRIBUTING.md\",\"CHANGELOG.md\",\"SECURITY.md\",\"CODE_OF_CONDUCT.md\",\"AGENTS.md\",\"CLAUDE.md\",\"GEMINI.md\"],\"required_labels\":[\"agentic-workflows\"],\"target\":\"*\",\"title_prefix\":\"[ci-fix] \"},\"report_incomplete\":{},\"update_pull_request\":{\"allow_body\":true,\"allow_title\":false,\"max\":3,\"target\":\"*\",\"update_branch\":false}}" GH_AW_CI_TRIGGER_TOKEN: ${{ secrets.GH_AW_CI_TRIGGER_TOKEN }} with: github-token: ${{ secrets.GH_AW_GITHUB_TOKEN || secrets.GITHUB_TOKEN }} diff --git a/.github/workflows/ci-status-fix.md b/.github/workflows/ci-status-fix.md index d3227e25170c..9c2f46054029 100644 --- a/.github/workflows/ci-status-fix.md +++ b/.github/workflows/ci-status-fix.md @@ -15,7 +15,9 @@ description: | humans (the open tracking issue is the hand-off surface; a dedicated [ci-fix][needs-human] PR is planned but currently deferred — see Step 6). CI on the PR is kicked by a human `/azp run` for now; the loop watches, classifies, - and re-fixes autonomously between kicks. + and re-fixes autonomously between kicks. When the SPECIFIC test a PR fixed is + confirmed green in that PR's own CI, the loop marks the draft PR ready for review + (a state transition only — it never approves or merges). Never mutes tests, but de-flakes genuinely flaky ones (deterministic synchronization, no retries / timeout bumps). Always skips visual-regression / screenshot issues. @@ -239,8 +241,15 @@ safe-outputs: # PER-RUN total (not per-PR): a sweep may surface several green PRs and/or # annotate several flaky reds in one run. At max:1 all but the first comment # were silently dropped, starving the workflow's #1 value (surfacing green PRs - # for review). Raised to 3 to match the per-run throughput of the other outputs. - max: 3 + # for review). Sized to 6 (not 3) because a single green DRAFT PR that gets + # flipped ready in one sweep spends TWO comment slots — the Step 3 ✅ surface + # comment (which still names the /azp-gated legs a human must kick, so it is NOT + # redundant with 🎯) AND the Step 3.6 T3 🎯 readiness comment. At max:3 the shared + # bucket drained after ~1 draft flip and T3's atomicity pre-check then deferred + # every further mark-ready even while the mark-ready/add_labels buckets (also 3) + # sat idle; 6 lets ~3 draft flips (2 comments each) land per sweep, so the + # mark-ready:3 / add_labels:3 caps are actually reachable. + max: 6 target: "*" # Hard constraint (defense-in-depth): only comment on THIS workflow's own # ci-fix PRs — [ci-fix] title prefix AND agentic-workflows label. Mirrors the @@ -261,13 +270,51 @@ safe-outputs: # never the title, so disable title rewrites — this compiles to allow_title:false # and removes any ability to retitle an arbitrary PR. title: false - # NOTE: gh-aw v0.79.8 does NOT emit required-title-prefix/required-labels into + # NOTE: gh-aw v0.80.9 (the pinned compiler) does NOT emit required-title-prefix/required-labels into # the compiled config for update-pull-request (verified against the lock — it # silently drops them, unlike add-comment / push-to-pull-request-branch which # honor them). So which-PR scoping here relies on prompt Hard-Rule 6 + # min-integrity:approved; the residual is body-marker edits only (no code, no # merge, no close), capped at max:3. Revisit required-* if a future gh-aw # compiler emits them for this output. + mark-pull-request-as-ready-for-review: + # Flip a validated draft [ci-fix] PR to ready-for-review once the SPECIFIC test + # this PR was opened to fix is confirmed green in the PR's OWN CI (Step 3.6) — + # even when unrelated legs are red. The workflow is schedule/dispatch-triggered + # (no triggering-PR context), so target "*" lets the agent name the PR number it + # validated. This is a state transition ONLY: it never approves and never merges — + # a human still reviews and merges. + # PER-RUN total (not per-PR): a sweep may validate several PRs' target tests green. + max: 3 + target: "*" + # Hard constraint (defense-in-depth): only ever un-draft THIS workflow's own + # ci-fix PRs — [ci-fix] title prefix AND agentic-workflows label — so a confused + # or prompt-injected agent cannot mark an arbitrary PR ready. (If the v0.80.9 + # compiler silently drops these — as documented for update-pull-request above — + # the Step 3.6 preconditions + min-integrity:approved are the compensating scope + # controls; verify against the lock after compiling.) + required-title-prefix: "[ci-fix] " + required-labels: [agentic-workflows] + add-labels: + # Apply the p/0 priority label to a [ci-fix] PR at the exact moment the loop flips it + # from draft to ready-for-review (Step 3.6 T3) — so a validated, review-ready fix lands + # in the team's p/0 triage queue instead of sitting unseen in the draft backlog. Paired + # 1:1 with mark-pull-request-as-ready-for-review; target "*" lets the agent name the PR + # it just validated. + # allowed = HARD allowlist: the agent may ONLY ever add p/0, nothing else. This caps the + # blast radius of a confused/prompt-injected agent to exactly one benign priority label — + # it can never apply a downstream-triggering or destructive label. + allowed: [p/0] + # PER-RUN total (not per-PR): a sweep may mark several PRs ready in one run; each adds + # one label, so this matches the mark-ready per-run cap (3). + max: 3 + target: "*" + # Hard constraint (defense-in-depth): only ever label THIS workflow's own ci-fix PRs — + # [ci-fix] title prefix AND agentic-workflows label. (If the v0.80.9 compiler drops these + # — as documented for update-pull-request above — the Step 3.6 preconditions + + # min-integrity:approved + the allowed:[p/0] allowlist are the compensating controls.) + required-title-prefix: "[ci-fix] " + required-labels: [agentic-workflows] timeout-minutes: 90 @@ -339,33 +386,48 @@ through `safe-outputs`. 5. **One issue = one outcome per run.** Exactly one of: respond to a maintainer's change-request on the open PR (Track C, Step 3.5.R); open the first fix/help/ de-flake PR; advance an existing PR by one attempt (push a follow-up fix); - surface a validated-green PR for review; annotate an unrelated-flake red; wait + surface a validated-green PR for review; mark a target-validated draft PR + ready-for-review (Step 3.6 T3 — the terminal outcome that supersedes the same + run's surface/annotate precursor line), or record its `already-ready` + steady-state no-op; annotate an unrelated-flake red; wait (CI not yet settled); or a recorded skip (the dedicated needs-human PR is deferred — the attempt cap records a skip instead; see Step 6). Always prefer advancing or opening a PR over a skip when a non-mute diff is producible. 6. **All writes via `safe-outputs`.** Allowed outputs: `create_pull_request` (first attempt only), `push_to_pull_request_branch` (advance an existing PR by one attempt), `update_pull_request` (bump the attempt marker / refresh the - prior-attempts table), and `add_comment` (progress notes on the `[ci-fix]` PR - ONLY). NEVER comment on the tracking issue (issues are locked by + prior-attempts table), `add_comment` (progress notes on the `[ci-fix]` PR + ONLY), `mark_pull_request_as_ready_for_review` (flip a target-validated draft + `[ci-fix]` PR from draft to ready — Step 3.6 T3 only), and `add_labels` (add + ONLY the `p/0` label, ONLY on that same draft→ready transition — Step 3.6 T3). + NEVER comment on the tracking issue (issues are locked by `.github/workflows/ci-scan-lock-issues.yml`) — `add_comment` targets the PR. No `gh pr create`, no manual `git push`. **Defense-in-depth:** `add_comment` and `update_pull_request` use `target: "*"` (the agent supplies the PR number). - `add_comment` is now config-locked to `required-title-prefix: "[ci-fix] "` + + `add_comment`, `mark_pull_request_as_ready_for_review`, and `add_labels` are + config-locked to `required-title-prefix: "[ci-fix] "` + `required-labels: [agentic-workflows]` (hard-enforced by the handler, same as - `push_to_pull_request_branch`), so a comment can only ever land on THIS - workflow's own PRs. `update_pull_request` CANNOT be config-locked to a - title/label in gh-aw v0.79.8 (the compiler silently drops `required-*` for that - output — verified against the lock), so it keeps `title: false` (no retitles; - body-marker edits only) plus this prompt-level guard. Before emitting either, - VERIFY the target PR carries BOTH the `[ci-fix]` title prefix AND the - `agentic-workflows` label; never comment on or edit the body of any PR that - lacks both. + `push_to_pull_request_branch`), and `add_labels` additionally has an + `allowed: [p/0]` allowlist so it can ONLY ever add `p/0` — so these can only + ever land on THIS workflow's own PRs. `update_pull_request` CANNOT be + config-locked to a title/label in gh-aw v0.80.9 (the compiler silently drops + `required-*` for that output — verified against the lock), so it keeps + `title: false` (no retitles; body-marker edits only) plus this prompt-level + guard. Before emitting ANY of these, VERIFY the target PR carries BOTH the + `[ci-fix]` title prefix AND the `agentic-workflows` label; never comment on, + edit, un-draft, or label any PR that lacks both. 7. **Per-run safe-output caps (per RUN, not per PR).** Each output type is capped per run: `create_pull_request` 3, `push_to_pull_request_branch` 3, - `add_comment` 3, `update_pull_request` 3. Note an ADVANCE spends one push **and** - one update **and** one comment, so ≤ 3 advances/run; surfacing a green or - annotating a flake spends one comment. When a bucket is exhausted, do NOT keep + `add_comment` 6, `update_pull_request` 3, `mark_pull_request_as_ready_for_review` + 3, `add_labels` 3 (counts LABELS, not calls). Note an ADVANCE spends one push + **and** one update **and** one comment, so ≤ 3 advances/run; surfacing a green or + annotating a flake spends one comment; a **mark-ready (Step 3.6 T3)** of a + still-draft PR spends TWO comments (the Step 3 ✅ surface **and** the T3 🎯 + readiness note) **and** one mark-ready **and** one label as an ALL-OR-NOTHING set + (T3 pre-checks those buckets and defers the whole PR if any is exhausted — never + mark a PR ready without its 🎯 audit comment). `add_comment` is therefore sized 6 + (not 3) so ~3 draft flips, at 2 comments each, can land in one sweep instead of the + shared comment bucket starving the otherwise-idle mark-ready/add_labels buckets. When a bucket is exhausted, do NOT keep emitting (extras are silently dropped) — record `skipped: per-run cap reached; deferring PR # to next cycle` for each remaining PR so the drop is a deliberate, logged decision. The continuous loop picks the deferred PRs up next @@ -409,8 +471,9 @@ For every open tracking issue in scope, converge on exactly one outcome: | Open first help-wanted draft `[ci-fix]` PR | No open PR yet; a plausible candidate exists but cannot be runner-validated (device/UI tests) (attempt 1) | | Open first de-flake draft `[ci-fix]` PR | No open PR yet; failure is intermittent (green-on-retry) from a genuine test-quality defect; a deterministic-synchronization fix is producible (attempt 1, Step 4.7 bucket b) | | Advance an existing PR (push attempt N+1) | Open PR's own CI settled red *because of the fix itself*, no human engaged, marker < 10 — push a NEW distinct fix onto the same branch (Step 3.5 → 5.6) | -| Surface a validated-green PR | Open PR's own CI settled green — comment "validated, attempt N/10; ready for review" and do NOT advance (Step 3.5) | -| Annotate an unrelated-flake red | Open PR is red only on baseline-flake / unrelated legs — comment which leg needs a re-run; do NOT burn an attempt (Step 3.5) | +| Surface a validated-green PR | Open PR's own CI settled green — comment "validated, attempt N/10; ready for review" and do NOT advance (Step 3.5), then mark the draft PR ready for review once the fixed test is confirmed green (Step 3.6) | +| Annotate an unrelated-flake red | Open PR is red only on baseline-flake / unrelated legs — comment which leg needs a re-run; do NOT burn an attempt (Step 3.5); if the SPECIFIC fixed test is confirmed green on that build, still mark the draft PR ready for review (Step 3.6) | +| Mark a validated PR ready for review | The specific test this PR fixed is confirmed green in the PR's own CI (even if unrelated legs are red) — comment the target-test result and transition the draft PR to ready for review; never approve or merge (Step 3.6) | | Wait | Open PR's CI is pending / not yet settled on the current head SHA — do nothing this cycle (Step 3.5) | | Hand-off skip (attempt cap) | Marker == 10 and the signature still reproduces — stop and defer to humans; the dedicated `[ci-fix][needs-human]` PR is planned but currently deferred (Step 6) | | Recorded skip | Visual-regression, human engaged, already-handled, fixed-in-latest-build, infra-flake, out-of-bounds, only-mute-available, or no novel approach producible | @@ -685,14 +748,34 @@ Run these gates in order — the FIRST that fires decides this cycle's outcome: 1. **Human engaged.** If `C.humanEngaged` → `skipped: human engaged on PR #; deferring` and stop. Never fight a human reviewer — the intentional hand-off boundary still holds the moment a person touches the PR. -2. **CI not settled / unknown / incomplete prefetch.** If `C.checksSettled == false` - OR `C.overallConclusion` is `pending`, `neutral`, or `unknown`, OR - `C.dataComplete == false` → the fix's CI has not finished on the current head SHA - (`C.headSha`), or the prefetch could not fully read this PR (a partial read can - understate `humanEngaged` / `attempt`). `skipped: PR # CI pending / prefetch - incomplete on ; waiting` and stop. *(Round 1: this is where a - maintainer `/azp run` is awaited — the `/azp`-gated uitests/devicetests legs will - not have run until a human kicks them.)* +2. **CI not settled — but validate the target test first (target-focused readiness).** + If `C.checksSettled == false` OR `C.overallConclusion` is `pending`, `neutral`, or + `unknown`, OR `C.dataComplete == false` → the *overall* build has not finished on the + current head SHA (`C.headSha`), or the prefetch could not fully read this PR (a partial + read can understate `humanEngaged` / `attempt`). Unrelated legs still draining must NOT + keep an already-proven fix parked as a draft, so before waiting, try the target-focused + fast-path: + - **2a — Target-green fast-path (draft `[ci-fix]` PRs only).** If `C.dataComplete == + true` AND the Step 3.6 preconditions hold (draft PR, `[ci-fix] ` title prefix AND the + `agentic-workflows` label), run Step 3.6 **T1–T2** against `C.headSha` now: identify + the target test(s) and read the PR's OWN build **timeline / per-leg status** (anonymous + `_apis/build` — never `_apis/test`, per Hard-Rule 8). If EVERY target test is + **VALIDATED-GREEN** (per T2's all-platform definition below — the target's category leg + is `succeeded` on every platform that runs it, failed on none, and NO target platform + family the pipeline covers is still pending/unconcluded, per T2's "no platform left + unverified" rule; a platform that simply has no leg for the target's category — the test + doesn't run there — is fine, not a gap) AND every leg in + `C.failedLegs` that has ALREADY concluded classifies as **unrelated flake** (Step 4 / + Step 4.7 method — the fix is implicated in NO completed red), then the fix is proven + regardless of unrelated *pending* legs → run **Step 3.6 T3** (mark ready + 🎯 comment) + and stop. This early-out NEVER advances an attempt and NEVER acts on a red that could + be the fix's fault. + - **2b — Otherwise WAIT.** If the target test has not yet executed (pending/absent on + its leg), or a completed red is (or may be) caused by the fix, or `C.dataComplete == + false`, or this PR has no identifiable target test → `skipped: PR # CI pending / + target not yet validated on ; waiting` and stop. *(Round 1: this is where a + maintainer `/azp run` is awaited — the `/azp`-gated uitests/devicetests legs will not + have run until a human kicks them.)* 3. **Green → surface for review.** If `C.overallConclusion == "success"`: the checks that RAN are green. **Primary-gate check first:** the deterministic `success` verdict only certifies "at least one green check, nothing failing or @@ -704,18 +787,31 @@ Run these gates in order — the FIRST that fires decides this cycle's outcome: # primary CI gate (maui-pr) not green on ; waiting` and stop — do NOT surface. (The `/azp`-gated `maui-pr-uitests` (def 313) / `maui-pr-devicetests` (def 314) legs MAY still be un-run — that is expected and is named below, not a - reason to withhold the surface.) **Idempotency:** scan the PR's existing comments - for a prior bot `✅ … validated … on ` note for THIS head SHA — if one - already exists, record `already-surfaced PR # (head )` and stop - WITHOUT re-commenting (re-surfacing the same green every tick is noise and burns - the per-run comment budget other PRs need). Otherwise resolve the attempt number from - `C.effectiveAttempt` (the authoritative max(marker, bot-commit) counter; omit the - number only if it is somehow indeterminate). **Dry-run gate (Step 0):** if `dry_run == "true"`, do NOT emit any comment — instead print the intended `✅` validated-green body to the run log, tally `dry-run: would-surface-green PR #`, and stop. Otherwise `add_comment` on PR #: `✅ Attempt /10 validated - — the fix's CI is green on . Ready for human review.` - Name any `/azp`-gated legs (uitests def 313 / devicetests def 314) that have not - run and still need a maintainer `/azp run`. Do NOT advance. Record - `surfaced-green PR # (attempt /10)` and stop. *(This directly - attacks the real bottleneck — no reviews — so it is the highest-value outcome.)* + reason to withhold the surface.) **Comment idempotency + dry-run suppress the ✅ + comment ONLY — neither skips Step 3.6.** Scan the PR's existing comments for a prior + bot `✅ … validated … on ` note for THIS head SHA; if one exists, set + `SKIP_SURFACE_COMMENT = true`. Resolve the attempt number from `C.effectiveAttempt` + (the authoritative max(marker, bot-commit) counter; omit the number only if it is + somehow indeterminate). Then post the surface comment UNLESS suppressed: + - if `dry_run == "true"`: do NOT emit — print the intended `✅` validated-green body to + the run log and tally `dry-run: would-surface-green PR #`; + - else if `SKIP_SURFACE_COMMENT`: do NOT re-post — record `already-surfaced PR # + (head )` (re-surfacing the same green every tick is noise and burns the + per-run comment budget other PRs need); + - else `add_comment` on PR #: `✅ Attempt /10 validated — the fix's CI is + green on .` naming any `/azp`-gated legs (uitests def 313 / devicetests def + 314) that have not run and still need a maintainer `/azp run`; record `surfaced-green + PR # (attempt /10)`. (Do NOT assert "ready for human review" on a PR that + is still a draft — Step 3.6, not this comment, owns the draft→ready flip and posts its + own 🎯 announcement when it fires.) + Do NOT advance. Then — in ALL of the above cases — run **Step 3.6** (target-test + readiness gate) for this PR before stopping: its `/azp`-gated target legs may conclude + green on this SAME head SHA on a LATER sweep (an `/azp run` adds no commit), and Step + 3.6 is the ONLY place the draft→ready flip happens, so it MUST re-evaluate every sweep — + never `stop` here before it. *(This directly attacks the real bottleneck — no reviews — + so it is the highest-value outcome.)* 4. **Red → classify caused-by-fix vs unrelated-flake.** If `C.overallConclusion == "failure"`, analyze the PR's OWN failing build (NOT `main`): find the AzDO `maui-pr` build for this PR (filter builds by `branchName=refs/pull//merge`, @@ -723,18 +819,26 @@ Run these gates in order — the FIRST that fires decides this cycle's outcome: method and Step 4.7 flake buckets to `C.failedLegs`. - **Unrelated flake only** (every failed leg is known-flaky / infra / a pre-existing baseline red NOT introduced by the fix): do NOT burn an attempt. - **Idempotency first:** scan the PR's existing comments for a prior bot - `♻️ … unrelated flake … on ` note for THIS head SHA — if one already - exists, record `already-annotated-flake PR # (head )` and stop - WITHOUT re-commenting (re-annotating the same flake every 12h sweep is noise and - burns the per-run comment budget other PRs need). Otherwise resolve the attempt - number from `C.effectiveAttempt` (the authoritative max(marker, bot-commit) - counter), omitting the `Attempt /10:` prefix only if it is somehow - indeterminate. **Dry-run gate (Step 0):** if `dry_run == "true"`, do NOT emit any comment — instead print the intended `♻️` unrelated-flake body to the run log, tally `dry-run: would-annotate-flake PR #`, and stop. Otherwise `add_comment` on PR #: `♻️ Attempt /10: red is - unrelated flake on leg(s) () on ; the fix itself is not - implicated. A maintainer re-run (/azp run ) should clear it.` Record - `annotated-flake PR # (head )` and stop. *(Round 1: human re-runs; - Round 2: auto re-trigger.)* + **Comment idempotency + dry-run suppress the ♻️ comment ONLY — neither skips Step + 3.6.** Scan the PR's existing comments for a prior bot `♻️ … unrelated flake … on + ` note for THIS head SHA; if one exists, set `SKIP_FLAKE_COMMENT = true`. + Resolve the attempt number from `C.effectiveAttempt` (the authoritative max(marker, + bot-commit) counter), omitting the `Attempt /10:` prefix only if it is + somehow indeterminate. Then post the flake note UNLESS suppressed: + - if `dry_run == "true"`: do NOT emit — print the intended `♻️` unrelated-flake body + to the run log and tally `dry-run: would-annotate-flake PR #`; + - else if `SKIP_FLAKE_COMMENT`: do NOT re-post — record `already-annotated-flake PR + # (head )` (re-annotating the same flake every 12h sweep is noise and + burns the per-run comment budget other PRs need); + - else `add_comment` on PR #: `♻️ Attempt /10: red is unrelated flake on + leg(s) () on ; the fix itself is not implicated. A + maintainer re-run (/azp run ) should clear it.` record `annotated-flake + PR # (head )`. + Then — in ALL of the above cases — run **Step 3.6** (target-test readiness gate) for + this PR before stopping: its `/azp`-gated target legs may conclude green on this SAME + head SHA on a LATER sweep, and Step 3.6 is the ONLY place the draft→ready flip + happens, so it MUST re-evaluate every sweep — never `stop` here before it. *(Round 1: + human re-runs; Round 2: auto re-trigger.)* - **Caused by the fix** (a failed leg still matches the original target signature, or the fix introduced a NEW failure): advance an attempt. - **Attempt count.** `attempt = C.effectiveAttempt` — the authoritative @@ -752,6 +856,188 @@ Run these gates in order — the FIRST that fires decides this cycle's outcome: from every prior commit on this branch**, emitted via the Step 5.6 ADVANCE path (push onto the same PR). +#### Step 3.6 — Target-test verification & mark-ready gate + +Reached from the Step 3 **green-surface** branch, the Step 4 **unrelated-flake** branch +(AFTER that branch has posted its comment), and the Step 3.5 **gate-2 target-focused +fast-path** (2a — while unrelated legs are still pending). Purpose: confirm the SPECIFIC +test(s) this PR was opened to fix now PASS in the PR's own CI, and — only when they do +— transition the draft `[ci-fix]` PR to **ready for review** so a maintainer sees a +validated fix instead of a draft. This is the ONLY place the loop flips draft→ready; it +is a state transition, **never** an approval or a merge (a human still reviews and +merges). Overall red on *unrelated* legs must NOT gate readiness — we validate the fix, +not the base branch's flakiness. + +**Preconditions** (ALL must hold; otherwise record `skipped: readiness N/A PR # +()` and stop this gate): +- `C.isDraft == true` — if the PR is already ready-for-review, the draft→ready + transition is already done; do NOT re-mark **and do NOT re-apply `p/0`**. `p/0` is applied + exactly ONCE, atomically with the draft→ready flip (T3 — all-or-nothing with the 🎯 audit + comment + mark-ready), so an already-ready loop PR that is MISSING `p/0` has almost + certainly had it **removed by a maintainer** de-prioritizing the PR — a pure triage action + the human-engagement guard (which inspects comments/reviews/commits, NOT label changes) + cannot see. Re-adding `p/0` every sweep would fight that maintainer indefinitely, violating + the loop's "never override a human" contract. So the loop does NOT reconcile the label: + record `already-ready PR #` and stop this gate. (Trade-off: on the rare occasion a + transient API error drops `p/0` at flip time *after* the T3 atomic pre-check passed, it is + not auto-re-added — but the PR is still ready-for-review with its 🎯 audit comment, and + re-adding `p/0` is a trivial manual action; that is strictly preferable to steam-rolling a + maintainer's deliberate de-prioritization.) +- The PR is unmistakably THIS workflow's own: `[ci-fix] ` title prefix AND the + `agentic-workflows` label (mirrors the safe-output lock; the compensating scope + control if the v0.80.9 compiler drops the declarative required-*). +- You reached this gate from Step 3 (green), Step 4 (**unrelated-flake**), or the Step 3.5 + gate-2 **target-focused fast-path** (2a) — i.e. the fix is NOT implicated in any red that + has concluded. If Step 4 classified a red as **caused by the fix** (ADVANCE), do NOT run + this gate — advance the attempt instead. + +**T1 — Identify the target test(s).** From the `[ci-scan]` issue signature the fix +addresses (and the PR's own diff), extract the fully-qualified test method name(s) the +fix targets — e.g. `SafeAreaShouldWorkOnAllShellTabs`. For a de-flake it is the +de-flaked test; for a product fix it is the originally-failing test(s). If NO specific +test can be identified (e.g. a product build-break rather than a test failure), this is a +**build-only fix** — there is no single test to validate cross-platform, so validate at +**whole-build** granularity rather than stopping (Step 3 only *comments*; Step 3.6 is the +sole draft→ready flip, so a build-only fix is undrafted HERE or not at all). We only reach +this gate from Step 3 (green) / Step 4 (unrelated-flake) / gate-2a, so the fix is not +implicated in any concluded red; additionally require, via the T2 timeline method on +`C.headSha`, that the primary build pipeline `maui-pr` (def 302) has CONCLUDED with EVERY +one of its platform build legs `succeeded`/`completed` and NONE failed/canceled or still +pending — the build is green on **every** platform, not just the originally-broken one (the +cross-platform guard, applied to the build instead of a test). A build-only fix is undrafted +ONLY on the strength of the auto-running `maui-pr` (def 302) whole-build green, which the +loop CAN observe: if the PR's `[ci-scan]` issue signature or its own diff indicates the +ORIGINATING failure was in a `/azp`-gated pipeline (`maui-pr-uitests` def 313 or +`maui-pr-devicetests` def 314) rather than `maui-pr` (def 302), do NOT undraft on +`maui-pr`-green alone — those pipelines do not auto-run on this PR (GITHUB_TOKEN cannot +trigger them), so a green `maui-pr` build is NOT evidence the gated build break is fixed; +record `skipped: build-only fix PR # targets gated pipeline () not run — +deferring to human` and stop this gate. Otherwise, set `TARGET := "the maui-pr build +(build-only fix — no single target test)"` and proceed to **T3** to mark ready. If any `maui-pr` build leg is still unconcluded, record `skipped: build-only fix PR +# not yet whole-build green (leg(s) pending)` and stop this gate WITHOUT marking +ready (a green subset is not enough — a still-pending build leg could yet fail). + +**T2 — Verify the target test's platform legs are green (anonymous, leg-level).** Per-test +outcomes come from the AzDO **test-results** API (`_apis/test/...`), which is **NOT reachable +anonymously — Hard-Rule 8**: those endpoints 302-redirect to a sign-in page on this runner, so +a `curl` returns an HTML redirect (not JSON) and every target test would look "not executed". +Validate instead at **leg granularity** using ONLY the anonymous `_apis/build/...` timeline +(Hard-Rule 8) — a `succeeded` category leg means every test in that category **that actually +ran** passed on that platform. This is a strong bar **only if the target test genuinely +executes**: a job can go green while one specific test is skipped at runtime (`[Ignore]`, +`Assert.Ignore()`, `Assert.Inconclusive()`, a `Skip=`/conditional `[Fact]`, an `#if`-out, or +an early `return` before the asserts). So before trusting a green leg as proof the target +passed, **confirm from the PR's OWN diff that the fix does NOT skip, ignore, disable, or +short-circuit the target test** (it adds no `[Ignore]`/`[Explicit]`, `Assert.Ignore`/ +`Assert.Inconclusive`, `Skip=`, category-exclusion, `#if`-out, or early `return` around the +target). If the fix could cause the target to be runtime-skipped rather than genuinely pass, +leg-level green is NOT sufficient evidence — record `skipped: target test may be +runtime-skipped by this fix (diff adds ); leg-green insufficient, deferring to human +on PR #` and stop this gate WITHOUT marking ready. Otherwise (the target genuinely runs +and asserts, as a de-flake or product fix does) a green category leg cannot hide a +target-test failure, so treat it as authoritative. + +Map the target test to the CI leg(s) that run it. A leg name encodes platform + test category +(e.g. `Android UITests SafeAreaEdges,Shadow`, `iOS UITests SafeAreaEdges,Shadow`, `macOS +UITests SafeAreaEdges,Shadow`, `Windows UITests SafeAreaEdges,Shadow`). Determine the target +test's UI-test category — its `[Category(UITestCategories.X)]` in the test/HostApp source, +visible in the PR diff or the test file — to know which leg-name substring identifies its legs; +for a device test, the per-platform device-test legs. + +Using the SAME build-discovery as Step 4 (filter AzDO builds by `branchName=refs/pull//merge` +or `sourceVersion == C.headSha`), read each build's **timeline** on `C.headSha` for the +pipeline(s) that RUN the target test — `maui-pr` (def 302) for unit/integration tests, +`maui-pr-uitests` (def 313) for Appium UI tests, `maui-pr-devicetests` (def 314) for device +tests: + +```bash +ORG=dnceng-public; PROJ=public; BUILD= +# Anonymous + Hard-Rule-8-compliant. NEVER call _apis/test/... (it 302-redirects to sign-in). +# Each type=="Job" record is one platform leg: result (succeeded/failed/canceled/null), +# state (completed/inProgress/pending), name (encodes " UITests "): +curl -s "https://dev.azure.com/$ORG/$PROJ/_apis/build/builds/$BUILD/timeline?api-version=7.1" \ + | tee /tmp/gh-aw/agent/timeline_${BUILD}.json \ + | jq -r '.records[] | select(.type=="Job") | "\(.result)\t\(.state)\t\(.name)"' +``` + +(The prefetch already exposes per-leg status in `C.failedLegs` / the checks context, and +`gh pr checks ` lists the same per-leg rows with platform+category in the name — use +whichever is handy; the build timeline is the authoritative cross-check on the exact build.) + +Treat a target test as **VALIDATED-GREEN** only if, on `C.headSha`, the leg that runs its +category is green on **every platform it runs on** — a fix that repairs one platform must not +silently regress the same test's leg on another, so a green leg on the originally-red platform +alone is NOT enough. Across the target pipeline's platform legs (for a UI test: the Android, +iOS, Windows, and macOS/MacCatalyst legs running the target's category; for a device test: each +device-test platform), require ALL of: +- the target's category leg is `result == "succeeded"` **and** `state == "completed"` on + **every** platform that runs it, AND +- that leg is `failed` / `canceled` / aborted on **NO** platform, AND +- **no platform is left unverified.** For each platform family the target pipeline covers, that + platform's category leg must have CONCLUDED (`state == "completed"`) on `C.headSha`. If a + platform simply has no leg for the target's category, the test does not run there — that is + fine, not a gap. But if a platform's category leg has **not concluded** (`state` is + `inProgress` / pending, or an `/azp`-gated `maui-pr-uitests` / `maui-pr-devicetests` leg that + has not been kicked), the test's status on that platform is UNKNOWN → the fix is NOT yet + validated across platforms: record `skipped: target test green on but + not yet verified on (leg(s) pending / need /azp run) on PR #` + and stop this gate WITHOUT marking ready. + +A target test whose category leg never concluded on ANY platform (**not executed** anywhere — +e.g. its `/azp`-gated pipeline has not been kicked) is likewise NOT validated: record `skipped: +target test not yet executed on PR # ( not run — needs /azp run)` and stop +this gate WITHOUT marking ready. Do NOT overclaim — a green *sibling* leg (a different category +on the same platform) is not the target's leg, and a green leg on one platform is not a pass on +the others. + +**T3 — Mark ready + report.** If EVERY target test is VALIDATED-GREEN. The 🎯 comment, +the mark-ready, and the `p/0` label are THREE SEPARATE safe-outputs — the comment +existing does NOT prove the mark-ready took effect, so they are tracked independently: + +- **Comment idempotency (dup-suppress only — never gates the mark-ready).** We only reach + T3 while `C.isDraft == true` (the precondition stops an already-ready PR before here). So + if a prior bot comment for THIS head that carries the leading `🎯` anchor AND the + contiguous phrase `validated green on ` already exists (BOTH T3 variants — + `🎯 Target test validated green on ` and the build-only `🎯 Build validated green on + ` — contain that exact contiguous phrase, so this recognizes both; keying on + `target test` alone would instead miss the build-only comment and re-post it every sweep. + **Require the `🎯` anchor** so the match can NEVER be loosened to a gapped + `validated…green…on` form that would false-positive on the Step 3 `✅ … validated — the + fix's CI is green on ` surface comment and wrongly suppress the audit 🎯), the comment + landed on an earlier sweep but the **mark-ready did NOT take + effect** (the PR is still a draft) — set `SUPPRESS_COMMENT = true` (do not re-post the + duplicate 🎯 comment) but STILL complete the mark-ready + label below. Never treat the + comment's existence as "already marked ready" while the PR is still a draft. Record this + case as `re-marking PR # (🎯 present; mark-ready did not take on a prior sweep)`. +- **Atomicity budget pre-check (Hard-Rule 7).** Determine the outputs this gate will emit: + `mark_pull_request_as_ready_for_review` + `add_labels` always, plus `add_comment` unless + `SUPPRESS_COMMENT`. Verify a free per-run slot remains in EACH bucket you are about to + use. If ANY required bucket is exhausted, emit NONE of them — record `skipped: per-run + cap reached; deferring mark-ready PR # to next cycle` and stop this gate. Never mark a + PR ready (or label it) unless its 🎯 audit comment is GUARANTEED to exist for this exact + head SHA — either posted in this same sweep, or (in the `SUPPRESS_COMMENT` re-marking case) + already present from a prior sweep. The outputs you DO emit either all land or all defer + together; the audit trail must never be absent, but it is NOT re-posted when it already exists. +- **Dry-run gate (Step 0):** if `dry_run == "true"`, emit NOTHING — print the intended + readiness comment and "would mark ready + add p/0" to the run log, tally `dry-run: + would-mark-ready PR #`, and stop. +- Otherwise emit for THIS PR number ``: + 1. `add_comment` (ONLY if not `SUPPRESS_COMMENT`): `🎯 Target test validated green on + — passed on ALL platforms it runs on (, + buildId ). — not + caused by this fix."> Transitioning this PR from draft to ready for review and adding + `p/0`; a maintainer still reviews and merges.` (For a **build-only fix** — the T1 + whole-build fallback, no single target test — phrase the first clause as `🎯 Build + validated green on — the maui-pr build passed on ALL platforms (buildId + ).` instead of naming a test.) + 2. `mark_pull_request_as_ready_for_review` with `reason:` a one-line justification + naming the validated test(s) and ``. + 3. `add_labels` with `labels: ["p/0"]` for PR # — put the now-review-ready fix into + the team's p/0 priority queue so it is triaged, not lost in the draft backlog. (If + the PR somehow already carries `p/0`, this is a harmless no-op.) +- Record `marked-ready PR # (target green on ALL platforms on , + labeled p/0)` and stop. + #### Step 3.5.R — Maintainer change-request response (Track C) Reached from Step 3.5 gate 0. Goal: when an **eligible human reviewer** (Hard-Rule @@ -1383,7 +1669,19 @@ Filed by [`ci-status-fix`](https://github.com/dotnet/maui/blob/main/.github/work ### Step 8 — Per-issue tally + end-of-run summary -Per issue, append one outcome line to `/tmp/gh-aw/agent/coverage.txt`: +Per issue, append **one** outcome line to `/tmp/gh-aw/agent/coverage.txt` — the +terminal outcome for the cycle. When one cycle produces a chained pair — a PR gets a +same-cycle precursor line and **then** a Step 3.6 readiness line (`marked-ready` on the +draft→ready flip, or `already-ready` on a later steady-state sweep of an already-ready +PR) — record ONLY the terminal Step 3.6 readiness line: it supersedes ANY same-cycle +non-readiness precursor for that PR, so an aggregator keying on one-line-per-issue never +double-counts. The precursor may be a Step 3 green line (`surfaced-green` on the first ✅, +or `already-surfaced` when it already exists) OR — when an unrelated-flake red and a +target-green coincide — a Step 4 `annotated-flake` line; either way the readiness line is +terminal, and the superseded precursor's signal survives in the PR's 🎯/♻️ comment so no +information is lost. This covers the flip cycle (`surfaced-green` → `marked-ready`), the +post-flip steady state (`already-surfaced` → `already-ready`), and the flake-coincident +flip (`annotated-flake` → `marked-ready`): ``` # main attempt- @@ -1394,8 +1692,14 @@ Per issue, append one outcome line to `/tmp/gh-aw/agent/coverage.txt`: `advance-PR # attempt /10` (ADVANCE mode — new commit pushed to the existing PR), `surfaced-green PR #` (fix's own CI went green; commented for review, did not advance), `annotated-flake PR #` (red was unrelated flake; -commented, attempt NOT burned), `waiting PR #` (CI not settled yet), -`dry-run: would-`, `skipped: `. (The +commented, attempt NOT burned), `marked-ready PR #` (the specific fixed test +was confirmed green in the PR's own CI; draft flipped to ready for review), +`already-surfaced PR #` (the fix's ✅ green comment already existed this sweep; +re-post suppressed — superseded by the Step 3.6 readiness line when the PR also flips +ready that cycle), `already-ready PR #` (terminal steady state — the PR was flipped +ready on a prior cycle and remains green on an unchanged head; no action taken, +supersedes `already-surfaced`), `waiting PR #` (CI not settled yet), +`dry-run: would-`, `skipped: `. (The `needs-human-PR` outcome is reserved for the deferred hand-off — Step 6 currently records a skip instead, so it is not emitted.)
CI pending / + target not yet validated on ; waiting` and stop. *(Round 1: this is where a + maintainer `/azp run` is awaited — the `/azp`-gated uitests/devicetests legs will not + have run until a human kicks them.)* 3. **Green → surface for review.** If `C.overallConclusion == "success"`: the checks that RAN are green. **Primary-gate check first:** the deterministic `success` verdict only certifies "at least one green check, nothing failing or @@ -714,18 +797,31 @@ Run these gates in order — the FIRST that fires decides this cycle's outcome: # primary CI gate (maui-pr) not green on ; waiting` and stop — do NOT surface. (The `/azp`-gated `maui-pr-uitests` (def 313) / `maui-pr-devicetests` (def 314) legs MAY still be un-run — that is expected and is named below, not a - reason to withhold the surface.) **Idempotency:** scan the PR's existing comments - for a prior bot `✅ … validated … on ` note for THIS head SHA — if one - already exists, record `already-surfaced PR # (head )` and stop - WITHOUT re-commenting (re-surfacing the same green every tick is noise and burns - the per-run comment budget other PRs need). Otherwise resolve the attempt number from - `C.effectiveAttempt` (the authoritative max(marker, bot-commit) counter; omit the - number only if it is somehow indeterminate). **Dry-run gate (Step 0):** if `dry_run == "true"`, do NOT emit any comment — instead print the intended `✅` validated-green body to the run log, tally `dry-run: would-surface-green PR #`, and stop. Otherwise `add_comment` on PR #: `✅ Attempt /10 - validated — the fix's CI is green on . Ready for human review.` - Name any `/azp`-gated legs (uitests def 313 / devicetests def 314) that have not - run and still need a maintainer `/azp run`. Do NOT advance. Record - `surfaced-green PR # (attempt /10)` and stop. *(This directly - attacks the real bottleneck — no reviews — so it is the highest-value outcome.)* + reason to withhold the surface.) **Comment idempotency + dry-run suppress the ✅ + comment ONLY — neither skips Step 3.6.** Scan the PR's existing comments for a prior + bot `✅ … validated … on ` note for THIS head SHA; if one exists, set + `SKIP_SURFACE_COMMENT = true`. Resolve the attempt number from `C.effectiveAttempt` + (the authoritative max(marker, bot-commit) counter; omit the number only if it is + somehow indeterminate). Then post the surface comment UNLESS suppressed: + - if `dry_run == "true"`: do NOT emit — print the intended `✅` validated-green body to + the run log and tally `dry-run: would-surface-green PR #`; + - else if `SKIP_SURFACE_COMMENT`: do NOT re-post — record `already-surfaced PR # + (head )` (re-surfacing the same green every tick is noise and burns the + per-run comment budget other PRs need); + - else `add_comment` on PR #: `✅ Attempt /10 validated — the fix's CI is + green on .` naming any `/azp`-gated legs (uitests def 313 / devicetests def + 314) that have not run and still need a maintainer `/azp run`; record `surfaced-green + PR # (attempt /10)`. (Do NOT assert "ready for human review" on a PR that + is still a draft — Step 3.6, not this comment, owns the draft→ready flip and posts its + own 🎯 announcement when it fires.) + Do NOT advance. Then — in ALL of the above cases — run **Step 3.6** (target-test + readiness gate) for this PR before stopping: its `/azp`-gated target legs may conclude + green on this SAME head SHA on a LATER sweep (an `/azp run` adds no commit), and Step + 3.6 is the ONLY place the draft→ready flip happens, so it MUST re-evaluate every sweep — + never `stop` here before it. *(This directly attacks the real bottleneck — no reviews — + so it is the highest-value outcome.)* 4. **Red → classify caused-by-fix vs unrelated-flake.** If `C.overallConclusion == "failure"`, analyze the PR's OWN failing build (NOT `net11.0`): find the AzDO `maui-pr` build for this PR (filter builds by `branchName=refs/pull//merge`, @@ -733,18 +829,26 @@ Run these gates in order — the FIRST that fires decides this cycle's outcome: method and Step 4.7 flake buckets to `C.failedLegs`. - **Unrelated flake only** (every failed leg is known-flaky / infra / a pre-existing baseline red NOT introduced by the fix): do NOT burn an attempt. - **Idempotency first:** scan the PR's existing comments for a prior bot - `♻️ … unrelated flake … on ` note for THIS head SHA — if one already - exists, record `already-annotated-flake PR # (head )` and stop - WITHOUT re-commenting (re-annotating the same flake every 12h sweep is noise and - burns the per-run comment budget other PRs need). Otherwise resolve the attempt - number from `C.effectiveAttempt` (the authoritative max(marker, bot-commit) - counter), omitting the `Attempt /10:` prefix only if it is somehow - indeterminate. **Dry-run gate (Step 0):** if `dry_run == "true"`, do NOT emit any comment — instead print the intended `♻️` unrelated-flake body to the run log, tally `dry-run: would-annotate-flake PR #`, and stop. Otherwise `add_comment` on PR #: `♻️ Attempt /10: red is - unrelated flake on leg(s) () on ; the fix itself is not - implicated. A maintainer re-run (/azp run ) should clear it.` Record - `annotated-flake PR # (head )` and stop. *(Round 1: human re-runs; - Round 2: auto re-trigger.)* + **Comment idempotency + dry-run suppress the ♻️ comment ONLY — neither skips Step + 3.6.** Scan the PR's existing comments for a prior bot `♻️ … unrelated flake … on + ` note for THIS head SHA; if one exists, set `SKIP_FLAKE_COMMENT = true`. + Resolve the attempt number from `C.effectiveAttempt` (the authoritative max(marker, + bot-commit) counter), omitting the `Attempt /10:` prefix only if it is + somehow indeterminate. Then post the flake note UNLESS suppressed: + - if `dry_run == "true"`: do NOT emit — print the intended `♻️` unrelated-flake body + to the run log and tally `dry-run: would-annotate-flake PR #`; + - else if `SKIP_FLAKE_COMMENT`: do NOT re-post — record `already-annotated-flake PR + # (head )` (re-annotating the same flake every 12h sweep is noise and + burns the per-run comment budget other PRs need); + - else `add_comment` on PR #: `♻️ Attempt /10: red is unrelated flake on + leg(s) () on ; the fix itself is not implicated. A + maintainer re-run (/azp run ) should clear it.` record `annotated-flake + PR # (head )`. + Then — in ALL of the above cases — run **Step 3.6** (target-test readiness gate) for + this PR before stopping: its `/azp`-gated target legs may conclude green on this SAME + head SHA on a LATER sweep, and Step 3.6 is the ONLY place the draft→ready flip + happens, so it MUST re-evaluate every sweep — never `stop` here before it. *(Round 1: + human re-runs; Round 2: auto re-trigger.)* - **Caused by the fix** (a failed leg still matches the original target signature, or the fix introduced a NEW failure): advance an attempt. - **Attempt count.** `attempt = C.effectiveAttempt` — the authoritative @@ -762,6 +866,188 @@ Run these gates in order — the FIRST that fires decides this cycle's outcome: from every prior commit on this branch**, emitted via the Step 5.6 ADVANCE path (push onto the same PR). +#### Step 3.6 — Target-test verification & mark-ready gate + +Reached from the Step 3 **green-surface** branch, the Step 4 **unrelated-flake** branch +(AFTER that branch has posted its comment), and the Step 3.5 **gate-2 target-focused +fast-path** (2a — while unrelated legs are still pending). Purpose: confirm the SPECIFIC +test(s) this PR was opened to fix now PASS in the PR's own CI, and — only when they do +— transition the draft `[ci-fix-net11]` PR to **ready for review** so a maintainer sees +a validated fix instead of a draft. This is the ONLY place the loop flips draft→ready; +it is a state transition, **never** an approval or a merge (a human still reviews and +merges). Overall red on *unrelated* legs must NOT gate readiness — we validate the fix, +not the base branch's flakiness. + +**Preconditions** (ALL must hold; otherwise record `skipped: readiness N/A PR # +()` and stop this gate): +- `C.isDraft == true` — if the PR is already ready-for-review, the draft→ready + transition is already done; do NOT re-mark **and do NOT re-apply `p/0`**. `p/0` is applied + exactly ONCE, atomically with the draft→ready flip (T3 — all-or-nothing with the 🎯 audit + comment + mark-ready), so an already-ready loop PR that is MISSING `p/0` has almost + certainly had it **removed by a maintainer** de-prioritizing the PR — a pure triage action + the human-engagement guard (which inspects comments/reviews/commits, NOT label changes) + cannot see. Re-adding `p/0` every sweep would fight that maintainer indefinitely, violating + the loop's "never override a human" contract. So the loop does NOT reconcile the label: + record `already-ready PR #` and stop this gate. (Trade-off: on the rare occasion a + transient API error drops `p/0` at flip time *after* the T3 atomic pre-check passed, it is + not auto-re-added — but the PR is still ready-for-review with its 🎯 audit comment, and + re-adding `p/0` is a trivial manual action; that is strictly preferable to steam-rolling a + maintainer's deliberate de-prioritization.) +- The PR is unmistakably THIS workflow's own: `[ci-fix-net11] ` title prefix AND the + `agentic-workflows` label (mirrors the safe-output lock; the compensating scope + control if the v0.80.9 compiler drops the declarative required-*). +- You reached this gate from Step 3 (green), Step 4 (**unrelated-flake**), or the Step 3.5 + gate-2 **target-focused fast-path** (2a) — i.e. the fix is NOT implicated in any red that + has concluded. If Step 4 classified a red as **caused by the fix** (ADVANCE), do NOT run + this gate — advance the attempt instead. + +**T1 — Identify the target test(s).** From the `[ci-scan-net11]` issue signature the fix +addresses (and the PR's own diff), extract the fully-qualified test method name(s) the +fix targets — e.g. `SafeAreaShouldWorkOnAllShellTabs`. For a de-flake it is the +de-flaked test; for a product fix it is the originally-failing test(s). If NO specific +test can be identified (e.g. a product build-break rather than a test failure), this is a +**build-only fix** — there is no single test to validate cross-platform, so validate at +**whole-build** granularity rather than stopping (Step 3 only *comments*; Step 3.6 is the +sole draft→ready flip, so a build-only fix is undrafted HERE or not at all). We only reach +this gate from Step 3 (green) / Step 4 (unrelated-flake) / gate-2a, so the fix is not +implicated in any concluded red; additionally require, via the T2 timeline method on +`C.headSha`, that the primary build pipeline `maui-pr` (def 302) has CONCLUDED with EVERY +one of its platform build legs `succeeded`/`completed` and NONE failed/canceled or still +pending — the build is green on **every** platform, not just the originally-broken one (the +cross-platform guard, applied to the build instead of a test). A build-only fix is undrafted +ONLY on the strength of the auto-running `maui-pr` (def 302) whole-build green, which the +loop CAN observe: if the PR's `[ci-scan]` issue signature or its own diff indicates the +ORIGINATING failure was in a `/azp`-gated pipeline (`maui-pr-uitests` def 313 or +`maui-pr-devicetests` def 314) rather than `maui-pr` (def 302), do NOT undraft on +`maui-pr`-green alone — those pipelines do not auto-run on this PR (GITHUB_TOKEN cannot +trigger them), so a green `maui-pr` build is NOT evidence the gated build break is fixed; +record `skipped: build-only fix PR # targets gated pipeline () not run — +deferring to human` and stop this gate. Otherwise, set `TARGET := "the maui-pr build +(build-only fix — no single target test)"` and proceed to **T3** to mark ready. If any `maui-pr` build leg is still unconcluded, record `skipped: build-only fix PR +# not yet whole-build green (leg(s) pending)` and stop this gate WITHOUT marking +ready (a green subset is not enough — a still-pending build leg could yet fail). + +**T2 — Verify the target test's platform legs are green (anonymous, leg-level).** Per-test +outcomes come from the AzDO **test-results** API (`_apis/test/...`), which is **NOT reachable +anonymously — Hard-Rule 8**: those endpoints 302-redirect to a sign-in page on this runner, so +a `curl` returns an HTML redirect (not JSON) and every target test would look "not executed". +Validate instead at **leg granularity** using ONLY the anonymous `_apis/build/...` timeline +(Hard-Rule 8) — a `succeeded` category leg means every test in that category **that actually +ran** passed on that platform. This is a strong bar **only if the target test genuinely +executes**: a job can go green while one specific test is skipped at runtime (`[Ignore]`, +`Assert.Ignore()`, `Assert.Inconclusive()`, a `Skip=`/conditional `[Fact]`, an `#if`-out, or +an early `return` before the asserts). So before trusting a green leg as proof the target +passed, **confirm from the PR's OWN diff that the fix does NOT skip, ignore, disable, or +short-circuit the target test** (it adds no `[Ignore]`/`[Explicit]`, `Assert.Ignore`/ +`Assert.Inconclusive`, `Skip=`, category-exclusion, `#if`-out, or early `return` around the +target). If the fix could cause the target to be runtime-skipped rather than genuinely pass, +leg-level green is NOT sufficient evidence — record `skipped: target test may be +runtime-skipped by this fix (diff adds ); leg-green insufficient, deferring to human +on PR #` and stop this gate WITHOUT marking ready. Otherwise (the target genuinely runs +and asserts, as a de-flake or product fix does) a green category leg cannot hide a +target-test failure, so treat it as authoritative. + +Map the target test to the CI leg(s) that run it. A leg name encodes platform + test category +(e.g. `Android UITests SafeAreaEdges,Shadow`, `iOS UITests SafeAreaEdges,Shadow`, `macOS +UITests SafeAreaEdges,Shadow`, `Windows UITests SafeAreaEdges,Shadow`). Determine the target +test's UI-test category — its `[Category(UITestCategories.X)]` in the test/HostApp source, +visible in the PR diff or the test file — to know which leg-name substring identifies its legs; +for a device test, the per-platform device-test legs. + +Using the SAME build-discovery as Step 4 (filter AzDO builds by `branchName=refs/pull//merge` +or `sourceVersion == C.headSha`), read each build's **timeline** on `C.headSha` for the +pipeline(s) that RUN the target test — `maui-pr` (def 302) for unit/integration tests, +`maui-pr-uitests` (def 313) for Appium UI tests, `maui-pr-devicetests` (def 314) for device +tests: + +```bash +ORG=dnceng-public; PROJ=public; BUILD= +# Anonymous + Hard-Rule-8-compliant. NEVER call _apis/test/... (it 302-redirects to sign-in). +# Each type=="Job" record is one platform leg: result (succeeded/failed/canceled/null), +# state (completed/inProgress/pending), name (encodes " UITests "): +curl -s "https://dev.azure.com/$ORG/$PROJ/_apis/build/builds/$BUILD/timeline?api-version=7.1" \ + | tee /tmp/gh-aw/agent/timeline_${BUILD}.json \ + | jq -r '.records[] | select(.type=="Job") | "\(.result)\t\(.state)\t\(.name)"' +``` + +(The prefetch already exposes per-leg status in `C.failedLegs` / the checks context, and +`gh pr checks ` lists the same per-leg rows with platform+category in the name — use +whichever is handy; the build timeline is the authoritative cross-check on the exact build.) + +Treat a target test as **VALIDATED-GREEN** only if, on `C.headSha`, the leg that runs its +category is green on **every platform it runs on** — a fix that repairs one platform must not +silently regress the same test's leg on another, so a green leg on the originally-red platform +alone is NOT enough. Across the target pipeline's platform legs (for a UI test: the Android, +iOS, Windows, and macOS/MacCatalyst legs running the target's category; for a device test: each +device-test platform), require ALL of: +- the target's category leg is `result == "succeeded"` **and** `state == "completed"` on + **every** platform that runs it, AND +- that leg is `failed` / `canceled` / aborted on **NO** platform, AND +- **no platform is left unverified.** For each platform family the target pipeline covers, that + platform's category leg must have CONCLUDED (`state == "completed"`) on `C.headSha`. If a + platform simply has no leg for the target's category, the test does not run there — that is + fine, not a gap. But if a platform's category leg has **not concluded** (`state` is + `inProgress` / pending, or an `/azp`-gated `maui-pr-uitests` / `maui-pr-devicetests` leg that + has not been kicked), the test's status on that platform is UNKNOWN → the fix is NOT yet + validated across platforms: record `skipped: target test green on but + not yet verified on (leg(s) pending / need /azp run) on PR #` + and stop this gate WITHOUT marking ready. + +A target test whose category leg never concluded on ANY platform (**not executed** anywhere — +e.g. its `/azp`-gated pipeline has not been kicked) is likewise NOT validated: record `skipped: +target test not yet executed on PR # ( not run — needs /azp run)` and stop +this gate WITHOUT marking ready. Do NOT overclaim — a green *sibling* leg (a different category +on the same platform) is not the target's leg, and a green leg on one platform is not a pass on +the others. + +**T3 — Mark ready + report.** If EVERY target test is VALIDATED-GREEN. The 🎯 comment, +the mark-ready, and the `p/0` label are THREE SEPARATE safe-outputs — the comment +existing does NOT prove the mark-ready took effect, so they are tracked independently: + +- **Comment idempotency (dup-suppress only — never gates the mark-ready).** We only reach + T3 while `C.isDraft == true` (the precondition stops an already-ready PR before here). So + if a prior bot comment for THIS head that carries the leading `🎯` anchor AND the + contiguous phrase `validated green on ` already exists (BOTH T3 variants — + `🎯 Target test validated green on ` and the build-only `🎯 Build validated green on + ` — contain that exact contiguous phrase, so this recognizes both; keying on + `target test` alone would instead miss the build-only comment and re-post it every sweep. + **Require the `🎯` anchor** so the match can NEVER be loosened to a gapped + `validated…green…on` form that would false-positive on the Step 3 `✅ … validated — the + fix's CI is green on ` surface comment and wrongly suppress the audit 🎯), the comment + landed on an earlier sweep but the **mark-ready did NOT take + effect** (the PR is still a draft) — set `SUPPRESS_COMMENT = true` (do not re-post the + duplicate 🎯 comment) but STILL complete the mark-ready + label below. Never treat the + comment's existence as "already marked ready" while the PR is still a draft. Record this + case as `re-marking PR # (🎯 present; mark-ready did not take on a prior sweep)`. +- **Atomicity budget pre-check (Hard-Rule 7).** Determine the outputs this gate will emit: + `mark_pull_request_as_ready_for_review` + `add_labels` always, plus `add_comment` unless + `SUPPRESS_COMMENT`. Verify a free per-run slot remains in EACH bucket you are about to + use. If ANY required bucket is exhausted, emit NONE of them — record `skipped: per-run + cap reached; deferring mark-ready PR # to next cycle` and stop this gate. Never mark a + PR ready (or label it) unless its 🎯 audit comment is GUARANTEED to exist for this exact + head SHA — either posted in this same sweep, or (in the `SUPPRESS_COMMENT` re-marking case) + already present from a prior sweep. The outputs you DO emit either all land or all defer + together; the audit trail must never be absent, but it is NOT re-posted when it already exists. +- **Dry-run gate (Step 0):** if `dry_run == "true"`, emit NOTHING — print the intended + readiness comment and "would mark ready + add p/0" to the run log, tally `dry-run: + would-mark-ready PR #`, and stop. +- Otherwise emit for THIS PR number ``: + 1. `add_comment` (ONLY if not `SUPPRESS_COMMENT`): `🎯 Target test validated green on + — passed on ALL platforms it runs on (, + buildId ). — not + caused by this fix."> Transitioning this PR from draft to ready for review and adding + `p/0`; a maintainer still reviews and merges.` (For a **build-only fix** — the T1 + whole-build fallback, no single target test — phrase the first clause as `🎯 Build + validated green on — the maui-pr build passed on ALL platforms (buildId + ).` instead of naming a test.) + 2. `mark_pull_request_as_ready_for_review` with `reason:` a one-line justification + naming the validated test(s) and ``. + 3. `add_labels` with `labels: ["p/0"]` for PR # — put the now-review-ready fix into + the team's p/0 priority queue so it is triaged, not lost in the draft backlog. (If + the PR somehow already carries `p/0`, this is a harmless no-op.) +- Record `marked-ready PR # (target green on ALL platforms on , + labeled p/0)` and stop. + #### Step 3.5.R — Maintainer change-request response (Track C) Reached from Step 3.5 gate 0. Goal: when an **eligible human reviewer** (Hard-Rule @@ -1395,7 +1681,19 @@ Filed by [`ci-status-fix-net11`](https://github.com/dotnet/maui/blob/main/.githu ### Step 8 — Per-issue tally + end-of-run summary -Per issue, append one outcome line to `/tmp/gh-aw/agent/coverage.txt`: +Per issue, append **one** outcome line to `/tmp/gh-aw/agent/coverage.txt` — the +terminal outcome for the cycle. When one cycle produces a chained pair — a PR gets a +same-cycle precursor line and **then** a Step 3.6 readiness line (`marked-ready` on the +draft→ready flip, or `already-ready` on a later steady-state sweep of an already-ready +PR) — record ONLY the terminal Step 3.6 readiness line: it supersedes ANY same-cycle +non-readiness precursor for that PR, so an aggregator keying on one-line-per-issue never +double-counts. The precursor may be a Step 3 green line (`surfaced-green` on the first ✅, +or `already-surfaced` when it already exists) OR — when an unrelated-flake red and a +target-green coincide — a Step 4 `annotated-flake` line; either way the readiness line is +terminal, and the superseded precursor's signal survives in the PR's 🎯/♻️ comment so no +information is lost. This covers the flip cycle (`surfaced-green` → `marked-ready`), the +post-flip steady state (`already-surfaced` → `already-ready`), and the flake-coincident +flip (`annotated-flake` → `marked-ready`): ``` # net11.0 attempt- @@ -1406,8 +1704,14 @@ Per issue, append one outcome line to `/tmp/gh-aw/agent/coverage.txt`: `advance-PR # attempt /10` (ADVANCE mode — new commit pushed to the existing PR), `surfaced-green PR #` (fix's own CI went green; commented for review, did not advance), `annotated-flake PR #` (red was unrelated flake; -commented, attempt NOT burned), `waiting PR #` (CI not settled yet), -`dry-run: would-`, `skipped: `. (The +commented, attempt NOT burned), `marked-ready PR #` (the specific fixed test +was confirmed green in the PR's own CI; draft flipped to ready for review), +`already-surfaced PR #` (the fix's ✅ green comment already existed this sweep; +re-post suppressed — superseded by the Step 3.6 readiness line when the PR also flips +ready that cycle), `already-ready PR #` (terminal steady state — the PR was flipped +ready on a prior cycle and remains green on an unchanged head; no action taken, +supersedes `already-surfaced`), `waiting PR #` (CI not settled yet), +`dry-run: would-`, `skipped: `. (The `needs-human-PR` outcome is reserved for the deferred hand-off — Step 6 currently records a skip instead, so it is not emitted.) diff --git a/.github/workflows/ci-status-fix.lock.yml b/.github/workflows/ci-status-fix.lock.yml index 93dc5969cb0d..65511f2b3259 100644 --- a/.github/workflows/ci-status-fix.lock.yml +++ b/.github/workflows/ci-status-fix.lock.yml @@ -1,4 +1,4 @@ -# gh-aw-metadata: {"schema_version":"v4","frontmatter_hash":"f014161967589ebcaf1ac5ec6f3c88ee5a4388d28b55a07dc36c45b7b2aebab6","body_hash":"86c6b2d4c3df13da14c6a3c8b211381fcc9f97bb1951ff7b80588a2c5338e00c","compiler_version":"v0.80.9","strict":true,"agent_id":"copilot","agent_model":"claude-opus-4.8","engine_versions":{"copilot":"1.0.63"}} +# gh-aw-metadata: {"schema_version":"v4","frontmatter_hash":"b0ebf8a2f179900cfe3638e994b27a43cec52bdbdd2331dc1bd4d65762fa2d6b","body_hash":"1825e2b1f7c7bd88241bf37580655489360f9bebc806edd00837a52ce4ad83a0","compiler_version":"v0.80.9","strict":true,"agent_id":"copilot","agent_model":"claude-opus-4.8","engine_versions":{"copilot":"1.0.63"}} # gh-aw-manifest: {"version":1,"secrets":["COPILOT_PAT_0","COPILOT_PAT_1","COPILOT_PAT_2","COPILOT_PAT_3","COPILOT_PAT_4","COPILOT_PAT_5","COPILOT_PAT_6","COPILOT_PAT_7","COPILOT_PAT_8","COPILOT_PAT_9","GH_AW_CI_TRIGGER_TOKEN","GH_AW_GITHUB_MCP_SERVER_TOKEN","GH_AW_GITHUB_TOKEN","GITHUB_TOKEN"],"actions":[{"repo":"actions/checkout","sha":"34e114876b0b11c390a56381ad16ebd13914f8d5","version":"v4"},{"repo":"actions/checkout","sha":"9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0","version":"v7.0.0"},{"repo":"actions/checkout","sha":"de0fac2e4500dabe0009e67214ff5f5447ce83dd","version":"v6.0.2"},{"repo":"actions/download-artifact","sha":"3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c","version":"v8.0.1"},{"repo":"actions/github-script","sha":"3a2844b7e9c422d3c10d287c895573f7108da1b3","version":"v9.0.0"},{"repo":"actions/setup-node","sha":"48b55a011bda9f5d6aeb4c2d9c7362e8dae4041e","version":"v6.4.0"},{"repo":"actions/upload-artifact","sha":"043fb46d1a93c77aae656e7c1c64a875d1fc6a0a","version":"v7.0.1"},{"repo":"github/gh-aw-actions/setup","sha":"8c7d04ebf1ece56cd381446125da3e0f6896294a","version":"v0.80.9"}],"containers":[{"image":"ghcr.io/github/gh-aw-firewall/agent:0.27.7","digest":"sha256:aae231e4635c8999d039c132f1602d3df850fe9b84a00aa2b5ac981179b5661c","pinned_image":"ghcr.io/github/gh-aw-firewall/agent:0.27.7@sha256:aae231e4635c8999d039c132f1602d3df850fe9b84a00aa2b5ac981179b5661c"},{"image":"ghcr.io/github/gh-aw-firewall/api-proxy:0.27.7","digest":"sha256:009caf2e3d88fa77b64e9a03a95a228fc58db0f1701c6d324b29ba5a3c7c79b6","pinned_image":"ghcr.io/github/gh-aw-firewall/api-proxy:0.27.7@sha256:009caf2e3d88fa77b64e9a03a95a228fc58db0f1701c6d324b29ba5a3c7c79b6"},{"image":"ghcr.io/github/gh-aw-firewall/squid:0.27.7","digest":"sha256:deb1d4e19de62d51cee0508057a596a19315c3423ada4d675cad136dc8037c96","pinned_image":"ghcr.io/github/gh-aw-firewall/squid:0.27.7@sha256:deb1d4e19de62d51cee0508057a596a19315c3423ada4d675cad136dc8037c96"},{"image":"ghcr.io/github/gh-aw-mcpg:v0.3.27","digest":"sha256:fe984bddde4ec05d756d9043edb0a32912e6b7b72f6a121b1082f29221421cc7","pinned_image":"ghcr.io/github/gh-aw-mcpg:v0.3.27@sha256:fe984bddde4ec05d756d9043edb0a32912e6b7b72f6a121b1082f29221421cc7"},{"image":"ghcr.io/github/gh-aw-node","digest":"sha256:529d02eb970b1161aa25c593a9c3df57fdfad5a8add328cb3b6eccef66f3183b","pinned_image":"ghcr.io/github/gh-aw-node@sha256:529d02eb970b1161aa25c593a9c3df57fdfad5a8add328cb3b6eccef66f3183b"},{"image":"ghcr.io/github/github-mcp-server:v1.4.0","digest":"sha256:2afb26356481d1a350e14544a6e160f7f7ec1561a1ea309b823665abf0309036","pinned_image":"ghcr.io/github/github-mcp-server:v1.4.0@sha256:2afb26356481d1a350e14544a6e160f7f7ec1561a1ea309b823665abf0309036"}]} # This file was automatically generated by gh-aw (v0.80.9). DO NOT EDIT. To debug this workflow, load the skill at https://github.com/github/gh-aw/blob/main/debug.md # @@ -37,7 +37,9 @@ # humans (the open tracking issue is the hand-off surface; a dedicated # [ci-fix][needs-human] PR is planned but currently deferred — see Step 6). CI on # the PR is kicked by a human `/azp run` for now; the loop watches, classifies, -# and re-fixes autonomously between kicks. +# and re-fixes autonomously between kicks. When the SPECIFIC test a PR fixed is +# confirmed green in that PR's own CI, the loop marks the draft PR ready for review +# (a state transition only — it never approves or merges). # Never mutes tests, but # de-flakes genuinely flaky ones (deterministic synchronization, no retries / # timeout bumps). Always skips visual-regression / screenshot issues. @@ -273,24 +275,24 @@ jobs: run: | bash "${RUNNER_TEMP}/gh-aw/actions/create_prompt_first.sh" { - cat << 'GH_AW_PROMPT_25cf75111b670167_EOF' + cat << 'GH_AW_PROMPT_2ac1d0b4083428f8_EOF' - GH_AW_PROMPT_25cf75111b670167_EOF + GH_AW_PROMPT_2ac1d0b4083428f8_EOF cat "${RUNNER_TEMP}/gh-aw/prompts/xpia.md" cat "${RUNNER_TEMP}/gh-aw/prompts/temp_folder_prompt.md" cat "${RUNNER_TEMP}/gh-aw/prompts/markdown.md" cat "${RUNNER_TEMP}/gh-aw/prompts/safe_outputs_prompt.md" - cat << 'GH_AW_PROMPT_25cf75111b670167_EOF' + cat << 'GH_AW_PROMPT_2ac1d0b4083428f8_EOF' - Tools: add_comment(max:3), create_pull_request(max:3), update_pull_request(max:3), push_to_pull_request_branch(max:3), missing_tool, missing_data, noop - GH_AW_PROMPT_25cf75111b670167_EOF + Tools: add_comment(max:6), create_pull_request(max:3), update_pull_request(max:3), mark_pull_request_as_ready_for_review(max:3), add_labels(max:3), push_to_pull_request_branch(max:3), missing_tool, missing_data, noop + GH_AW_PROMPT_2ac1d0b4083428f8_EOF cat "${RUNNER_TEMP}/gh-aw/prompts/safe_outputs_create_pull_request.md" cat "${RUNNER_TEMP}/gh-aw/prompts/safe_outputs_push_to_pr_branch.md" - cat << 'GH_AW_PROMPT_25cf75111b670167_EOF' + cat << 'GH_AW_PROMPT_2ac1d0b4083428f8_EOF' - GH_AW_PROMPT_25cf75111b670167_EOF + GH_AW_PROMPT_2ac1d0b4083428f8_EOF cat "${RUNNER_TEMP}/gh-aw/prompts/mcp_cli_tools_prompt.md" - cat << 'GH_AW_PROMPT_25cf75111b670167_EOF' + cat << 'GH_AW_PROMPT_2ac1d0b4083428f8_EOF' The following GitHub context information is available for this workflow: {{#if github.actor}} @@ -332,12 +334,12 @@ jobs: stop immediately and report the limitation rather than spending turns trying to work around it. - GH_AW_PROMPT_25cf75111b670167_EOF + GH_AW_PROMPT_2ac1d0b4083428f8_EOF cat "${RUNNER_TEMP}/gh-aw/prompts/github_mcp_tools_with_safeoutputs_prompt.md" - cat << 'GH_AW_PROMPT_25cf75111b670167_EOF' + cat << 'GH_AW_PROMPT_2ac1d0b4083428f8_EOF' {{#runtime-import .github/workflows/ci-status-fix.md}} - GH_AW_PROMPT_25cf75111b670167_EOF + GH_AW_PROMPT_2ac1d0b4083428f8_EOF } > "$GH_AW_PROMPT" - name: Interpolate variables and render templates uses: actions/github-script@3a2844b7e9c422d3c10d287c895573f7108da1b3 # v9.0.0 @@ -554,16 +556,18 @@ jobs: mkdir -p "${RUNNER_TEMP}/gh-aw/safeoutputs" mkdir -p /tmp/gh-aw/safeoutputs mkdir -p /tmp/gh-aw/mcp-logs/safeoutputs - cat > "${RUNNER_TEMP}/gh-aw/safeoutputs/config.json" << 'GH_AW_SAFE_OUTPUTS_CONFIG_fa2a69fad5fd0485_EOF' - {"add_comment":{"discussions":false,"max":3,"required_labels":["agentic-workflows"],"required_title_prefix":"[ci-fix] ","target":"*"},"create_pull_request":{"allowed_base_branches":["main"],"allowed_branches":["ci-fix/**"],"allowed_files":["src/Core/**","src/Controls/**","src/Essentials/**","src/BlazorWebView/**","src/TestUtils/**","src/Templates/**","**/PublicAPI.Unshipped.txt"],"base_branch":"main","draft":true,"labels":["agentic-workflows"],"max":3,"max_patch_files":100,"max_patch_size":4096,"protect_top_level_dot_folders":true,"protected_files":["package.json","bun.lockb","bunfig.toml","deno.json","deno.jsonc","deno.lock","global.json","NuGet.Config","Directory.Packages.props","mix.exs","mix.lock","go.mod","go.sum","stack.yaml","stack.yaml.lock","pom.xml","build.gradle","build.gradle.kts","settings.gradle","settings.gradle.kts","gradle.properties","package-lock.json","yarn.lock","pnpm-lock.yaml","npm-shrinkwrap.json","requirements.txt","Pipfile","Pipfile.lock","pyproject.toml","setup.py","setup.cfg","Gemfile","Gemfile.lock","uv.lock","CODEOWNERS","DESIGN.md","README.md","CONTRIBUTING.md","CHANGELOG.md","SECURITY.md","CODE_OF_CONDUCT.md","AGENTS.md","CLAUDE.md","GEMINI.md"],"protected_files_policy":"request_review","title_prefix":"[ci-fix] "},"create_report_incomplete_issue":{},"missing_data":{},"missing_tool":{},"noop":{"max":1,"report-as-issue":"true"},"push_to_pull_request_branch":{"allowed_files":["src/Core/**","src/Controls/**","src/Essentials/**","src/BlazorWebView/**","src/TestUtils/**","src/Templates/**","**/PublicAPI.Unshipped.txt"],"if_no_changes":"warn","max":3,"max_patch_size":4096,"protect_top_level_dot_folders":true,"protected_files":["package.json","bun.lockb","bunfig.toml","deno.json","deno.jsonc","deno.lock","global.json","NuGet.Config","Directory.Packages.props","mix.exs","mix.lock","go.mod","go.sum","stack.yaml","stack.yaml.lock","pom.xml","build.gradle","build.gradle.kts","settings.gradle","settings.gradle.kts","gradle.properties","package-lock.json","yarn.lock","pnpm-lock.yaml","npm-shrinkwrap.json","requirements.txt","Pipfile","Pipfile.lock","pyproject.toml","setup.py","setup.cfg","Gemfile","Gemfile.lock","uv.lock","CODEOWNERS","DESIGN.md","README.md","CONTRIBUTING.md","CHANGELOG.md","SECURITY.md","CODE_OF_CONDUCT.md","AGENTS.md","CLAUDE.md","GEMINI.md"],"required_labels":["agentic-workflows"],"target":"*","title_prefix":"[ci-fix] "},"report_incomplete":{},"update_pull_request":{"allow_body":true,"allow_title":false,"max":3,"target":"*","update_branch":false}} - GH_AW_SAFE_OUTPUTS_CONFIG_fa2a69fad5fd0485_EOF + cat > "${RUNNER_TEMP}/gh-aw/safeoutputs/config.json" << 'GH_AW_SAFE_OUTPUTS_CONFIG_0644bf1434ca0071_EOF' + {"add_comment":{"discussions":false,"max":6,"required_labels":["agentic-workflows"],"required_title_prefix":"[ci-fix] ","target":"*"},"add_labels":{"allowed":["p/0"],"max":3,"required_labels":["agentic-workflows"],"required_title_prefix":"[ci-fix] ","target":"*"},"create_pull_request":{"allowed_base_branches":["main"],"allowed_branches":["ci-fix/**"],"allowed_files":["src/Core/**","src/Controls/**","src/Essentials/**","src/BlazorWebView/**","src/TestUtils/**","src/Templates/**","**/PublicAPI.Unshipped.txt"],"base_branch":"main","draft":true,"labels":["agentic-workflows"],"max":3,"max_patch_files":100,"max_patch_size":4096,"protect_top_level_dot_folders":true,"protected_files":["package.json","bun.lockb","bunfig.toml","deno.json","deno.jsonc","deno.lock","global.json","NuGet.Config","Directory.Packages.props","mix.exs","mix.lock","go.mod","go.sum","stack.yaml","stack.yaml.lock","pom.xml","build.gradle","build.gradle.kts","settings.gradle","settings.gradle.kts","gradle.properties","package-lock.json","yarn.lock","pnpm-lock.yaml","npm-shrinkwrap.json","requirements.txt","Pipfile","Pipfile.lock","pyproject.toml","setup.py","setup.cfg","Gemfile","Gemfile.lock","uv.lock","CODEOWNERS","DESIGN.md","README.md","CONTRIBUTING.md","CHANGELOG.md","SECURITY.md","CODE_OF_CONDUCT.md","AGENTS.md","CLAUDE.md","GEMINI.md"],"protected_files_policy":"request_review","title_prefix":"[ci-fix] "},"create_report_incomplete_issue":{},"mark_pull_request_as_ready_for_review":{"max":3,"required_labels":["agentic-workflows"],"required_title_prefix":"[ci-fix] ","target":"*"},"missing_data":{},"missing_tool":{},"noop":{"max":1,"report-as-issue":"true"},"push_to_pull_request_branch":{"allowed_files":["src/Core/**","src/Controls/**","src/Essentials/**","src/BlazorWebView/**","src/TestUtils/**","src/Templates/**","**/PublicAPI.Unshipped.txt"],"if_no_changes":"warn","max":3,"max_patch_size":4096,"protect_top_level_dot_folders":true,"protected_files":["package.json","bun.lockb","bunfig.toml","deno.json","deno.jsonc","deno.lock","global.json","NuGet.Config","Directory.Packages.props","mix.exs","mix.lock","go.mod","go.sum","stack.yaml","stack.yaml.lock","pom.xml","build.gradle","build.gradle.kts","settings.gradle","settings.gradle.kts","gradle.properties","package-lock.json","yarn.lock","pnpm-lock.yaml","npm-shrinkwrap.json","requirements.txt","Pipfile","Pipfile.lock","pyproject.toml","setup.py","setup.cfg","Gemfile","Gemfile.lock","uv.lock","CODEOWNERS","DESIGN.md","README.md","CONTRIBUTING.md","CHANGELOG.md","SECURITY.md","CODE_OF_CONDUCT.md","AGENTS.md","CLAUDE.md","GEMINI.md"],"required_labels":["agentic-workflows"],"target":"*","title_prefix":"[ci-fix] "},"report_incomplete":{},"update_pull_request":{"allow_body":true,"allow_title":false,"max":3,"target":"*","update_branch":false}} + GH_AW_SAFE_OUTPUTS_CONFIG_0644bf1434ca0071_EOF - name: Generate Safe Outputs Tools env: GH_AW_TOOLS_META_JSON: | { "description_suffixes": { - "add_comment": " CONSTRAINTS: Maximum 3 comment(s) can be added. Target: *. Supports reply_to_id for discussion threading.", + "add_comment": " CONSTRAINTS: Maximum 6 comment(s) can be added. Target: *. Supports reply_to_id for discussion threading.", + "add_labels": " CONSTRAINTS: Maximum 3 label(s) can be added. Only these labels are allowed: [\"p/0\"]. Target: *.", "create_pull_request": " CONSTRAINTS: Maximum 3 pull request(s) can be created. Title will be prefixed with \"[ci-fix] \". Labels [\"agentic-workflows\"] will be automatically added. Only these labels are allowed: [\"agentic-workflows\"]. PRs will be created as drafts.", + "mark_pull_request_as_ready_for_review": " CONSTRAINTS: Maximum 3 pull request(s) can be marked as ready for review.", "push_to_pull_request_branch": " CONSTRAINTS: Maximum 3 push(es) can be made. The target pull request title must start with \"[ci-fix] \".", "update_pull_request": " CONSTRAINTS: Maximum 3 pull request(s) can be updated. Target: *." }, @@ -594,6 +598,25 @@ jobs: } } }, + "add_labels": { + "defaultMax": 5, + "fields": { + "item_number": { + "issueNumberOrTemporaryId": true + }, + "labels": { + "required": true, + "type": "array", + "itemType": "string", + "itemSanitize": true, + "itemMaxLength": 128 + }, + "repo": { + "type": "string", + "maxLength": 256 + } + } + }, "create_pull_request": { "defaultMax": 1, "fields": { @@ -635,6 +658,24 @@ jobs: } } }, + "mark_pull_request_as_ready_for_review": { + "defaultMax": 1, + "fields": { + "pull_request_number": { + "issueOrPRNumber": true + }, + "reason": { + "required": true, + "type": "string", + "sanitize": true, + "maxLength": 65000 + }, + "repo": { + "type": "string", + "maxLength": 256 + } + } + }, "missing_data": { "defaultMax": 20, "fields": { @@ -1494,7 +1535,7 @@ jobs: uses: actions/github-script@3a2844b7e9c422d3c10d287c895573f7108da1b3 # v9.0.0 env: WORKFLOW_NAME: "CI Failure Fixer (main)" - WORKFLOW_DESCRIPTION: "Periodic pass over open ci-scan tracking issues filed by the main-branch CI\nfailure scanner (.github/workflows/ci-status-main.md). This workflow targets\nthe `main` branch EXCLUSIVELY: it processes only issues labelled ci-scan and\nopens every PR against main. (The net11.0 branch is handled by the parallel\n.github/workflows/ci-status-fix-net11.md — the two are split because gh-aw can\nonly transport a fix relative to ONE static base branch per workflow, and the\nmain↔net11.0 divergence exceeds gh-aw's 10 MB transport-patch cap.) The fixer\nopens ONE draft [ci-fix] PR per actionable issue against main, then WATCHES\nthat PR's own CI on later runs: when the fix's CI comes back red and the red is\ncaused by the fix itself, it pushes a fresh follow-up fix onto the SAME PR\nbranch (never a second PR) — up to 10 attempts — then stops and defers to\nhumans (the open tracking issue is the hand-off surface; a dedicated\n[ci-fix][needs-human] PR is planned but currently deferred — see Step 6). CI on\nthe PR is kicked by a human `/azp run` for now; the loop watches, classifies,\nand re-fixes autonomously between kicks.\nNever mutes tests, but\nde-flakes genuinely flaky ones (deterministic synchronization, no retries /\ntimeout bumps). Always skips visual-regression / screenshot issues." + WORKFLOW_DESCRIPTION: "Periodic pass over open ci-scan tracking issues filed by the main-branch CI\nfailure scanner (.github/workflows/ci-status-main.md). This workflow targets\nthe `main` branch EXCLUSIVELY: it processes only issues labelled ci-scan and\nopens every PR against main. (The net11.0 branch is handled by the parallel\n.github/workflows/ci-status-fix-net11.md — the two are split because gh-aw can\nonly transport a fix relative to ONE static base branch per workflow, and the\nmain↔net11.0 divergence exceeds gh-aw's 10 MB transport-patch cap.) The fixer\nopens ONE draft [ci-fix] PR per actionable issue against main, then WATCHES\nthat PR's own CI on later runs: when the fix's CI comes back red and the red is\ncaused by the fix itself, it pushes a fresh follow-up fix onto the SAME PR\nbranch (never a second PR) — up to 10 attempts — then stops and defers to\nhumans (the open tracking issue is the hand-off surface; a dedicated\n[ci-fix][needs-human] PR is planned but currently deferred — see Step 6). CI on\nthe PR is kicked by a human `/azp run` for now; the loop watches, classifies,\nand re-fixes autonomously between kicks. When the SPECIFIC test a PR fixed is\nconfirmed green in that PR's own CI, the loop marks the draft PR ready for review\n(a state transition only — it never approves or merges).\nNever mutes tests, but\nde-flakes genuinely flaky ones (deterministic synchronization, no retries /\ntimeout bumps). Always skips visual-regression / screenshot issues." HAS_PATCH: ${{ needs.agent.outputs.has_patch }} with: script: | @@ -1910,7 +1951,7 @@ jobs: GH_AW_ALLOWED_DOMAINS: "*.blob.core.windows.net,*.githubusercontent.com,api.business.githubcopilot.com,api.enterprise.githubcopilot.com,api.github.com,api.githubcopilot.com,api.individual.githubcopilot.com,api.snapcraft.io,archive.ubuntu.com,azure.archive.ubuntu.com,codeload.github.com,crl.geotrust.com,crl.globalsign.com,crl.identrust.com,crl.sectigo.com,crl.thawte.com,crl.usertrust.com,crl.verisign.com,crl3.digicert.com,crl4.digicert.com,crls.ssl.com,dev.azure.com,docs.github.com,github-cloud.githubusercontent.com,github-cloud.s3.amazonaws.com,github.blog,github.com,github.githubassets.com,helix.dot.net,host.docker.internal,json-schema.org,json.schemastore.org,keyserver.ubuntu.com,lfs.github.com,objects.githubusercontent.com,ocsp.digicert.com,ocsp.geotrust.com,ocsp.globalsign.com,ocsp.identrust.com,ocsp.sectigo.com,ocsp.ssl.com,ocsp.thawte.com,ocsp.usertrust.com,ocsp.verisign.com,packagecloud.io,packages.cloud.google.com,packages.microsoft.com,patch-diff.githubusercontent.com,ppa.launchpad.net,raw.githubusercontent.com,registry.npmjs.org,s.symcb.com,s.symcd.com,security.ubuntu.com,telemetry.enterprise.githubcopilot.com,ts-crl.ws.symantec.com,ts-ocsp.ws.symantec.com,www.googleapis.com" GITHUB_SERVER_URL: ${{ github.server_url }} GITHUB_API_URL: ${{ github.api_url }} - GH_AW_SAFE_OUTPUTS_HANDLER_CONFIG: "{\"add_comment\":{\"discussions\":false,\"max\":3,\"required_labels\":[\"agentic-workflows\"],\"required_title_prefix\":\"[ci-fix] \",\"target\":\"*\"},\"create_pull_request\":{\"allowed_base_branches\":[\"main\"],\"allowed_branches\":[\"ci-fix/**\"],\"allowed_files\":[\"src/Core/**\",\"src/Controls/**\",\"src/Essentials/**\",\"src/BlazorWebView/**\",\"src/TestUtils/**\",\"src/Templates/**\",\"**/PublicAPI.Unshipped.txt\"],\"base_branch\":\"main\",\"draft\":true,\"labels\":[\"agentic-workflows\"],\"max\":3,\"max_patch_files\":100,\"max_patch_size\":4096,\"protect_top_level_dot_folders\":true,\"protected_files\":[\"package.json\",\"bun.lockb\",\"bunfig.toml\",\"deno.json\",\"deno.jsonc\",\"deno.lock\",\"global.json\",\"NuGet.Config\",\"Directory.Packages.props\",\"mix.exs\",\"mix.lock\",\"go.mod\",\"go.sum\",\"stack.yaml\",\"stack.yaml.lock\",\"pom.xml\",\"build.gradle\",\"build.gradle.kts\",\"settings.gradle\",\"settings.gradle.kts\",\"gradle.properties\",\"package-lock.json\",\"yarn.lock\",\"pnpm-lock.yaml\",\"npm-shrinkwrap.json\",\"requirements.txt\",\"Pipfile\",\"Pipfile.lock\",\"pyproject.toml\",\"setup.py\",\"setup.cfg\",\"Gemfile\",\"Gemfile.lock\",\"uv.lock\",\"CODEOWNERS\",\"DESIGN.md\",\"README.md\",\"CONTRIBUTING.md\",\"CHANGELOG.md\",\"SECURITY.md\",\"CODE_OF_CONDUCT.md\",\"AGENTS.md\",\"CLAUDE.md\",\"GEMINI.md\"],\"protected_files_policy\":\"request_review\",\"title_prefix\":\"[ci-fix] \"},\"create_report_incomplete_issue\":{},\"missing_data\":{},\"missing_tool\":{},\"noop\":{\"max\":1,\"report-as-issue\":\"true\"},\"push_to_pull_request_branch\":{\"allowed_files\":[\"src/Core/**\",\"src/Controls/**\",\"src/Essentials/**\",\"src/BlazorWebView/**\",\"src/TestUtils/**\",\"src/Templates/**\",\"**/PublicAPI.Unshipped.txt\"],\"if_no_changes\":\"warn\",\"max\":3,\"max_patch_size\":4096,\"protect_top_level_dot_folders\":true,\"protected_files\":[\"package.json\",\"bun.lockb\",\"bunfig.toml\",\"deno.json\",\"deno.jsonc\",\"deno.lock\",\"global.json\",\"NuGet.Config\",\"Directory.Packages.props\",\"mix.exs\",\"mix.lock\",\"go.mod\",\"go.sum\",\"stack.yaml\",\"stack.yaml.lock\",\"pom.xml\",\"build.gradle\",\"build.gradle.kts\",\"settings.gradle\",\"settings.gradle.kts\",\"gradle.properties\",\"package-lock.json\",\"yarn.lock\",\"pnpm-lock.yaml\",\"npm-shrinkwrap.json\",\"requirements.txt\",\"Pipfile\",\"Pipfile.lock\",\"pyproject.toml\",\"setup.py\",\"setup.cfg\",\"Gemfile\",\"Gemfile.lock\",\"uv.lock\",\"CODEOWNERS\",\"DESIGN.md\",\"README.md\",\"CONTRIBUTING.md\",\"CHANGELOG.md\",\"SECURITY.md\",\"CODE_OF_CONDUCT.md\",\"AGENTS.md\",\"CLAUDE.md\",\"GEMINI.md\"],\"required_labels\":[\"agentic-workflows\"],\"target\":\"*\",\"title_prefix\":\"[ci-fix] \"},\"report_incomplete\":{},\"update_pull_request\":{\"allow_body\":true,\"allow_title\":false,\"max\":3,\"target\":\"*\",\"update_branch\":false}}" + GH_AW_SAFE_OUTPUTS_HANDLER_CONFIG: "{\"add_comment\":{\"discussions\":false,\"max\":6,\"required_labels\":[\"agentic-workflows\"],\"required_title_prefix\":\"[ci-fix] \",\"target\":\"*\"},\"add_labels\":{\"allowed\":[\"p/0\"],\"max\":3,\"required_labels\":[\"agentic-workflows\"],\"required_title_prefix\":\"[ci-fix] \",\"target\":\"*\"},\"create_pull_request\":{\"allowed_base_branches\":[\"main\"],\"allowed_branches\":[\"ci-fix/**\"],\"allowed_files\":[\"src/Core/**\",\"src/Controls/**\",\"src/Essentials/**\",\"src/BlazorWebView/**\",\"src/TestUtils/**\",\"src/Templates/**\",\"**/PublicAPI.Unshipped.txt\"],\"base_branch\":\"main\",\"draft\":true,\"labels\":[\"agentic-workflows\"],\"max\":3,\"max_patch_files\":100,\"max_patch_size\":4096,\"protect_top_level_dot_folders\":true,\"protected_files\":[\"package.json\",\"bun.lockb\",\"bunfig.toml\",\"deno.json\",\"deno.jsonc\",\"deno.lock\",\"global.json\",\"NuGet.Config\",\"Directory.Packages.props\",\"mix.exs\",\"mix.lock\",\"go.mod\",\"go.sum\",\"stack.yaml\",\"stack.yaml.lock\",\"pom.xml\",\"build.gradle\",\"build.gradle.kts\",\"settings.gradle\",\"settings.gradle.kts\",\"gradle.properties\",\"package-lock.json\",\"yarn.lock\",\"pnpm-lock.yaml\",\"npm-shrinkwrap.json\",\"requirements.txt\",\"Pipfile\",\"Pipfile.lock\",\"pyproject.toml\",\"setup.py\",\"setup.cfg\",\"Gemfile\",\"Gemfile.lock\",\"uv.lock\",\"CODEOWNERS\",\"DESIGN.md\",\"README.md\",\"CONTRIBUTING.md\",\"CHANGELOG.md\",\"SECURITY.md\",\"CODE_OF_CONDUCT.md\",\"AGENTS.md\",\"CLAUDE.md\",\"GEMINI.md\"],\"protected_files_policy\":\"request_review\",\"title_prefix\":\"[ci-fix] \"},\"create_report_incomplete_issue\":{},\"mark_pull_request_as_ready_for_review\":{\"max\":3,\"required_labels\":[\"agentic-workflows\"],\"required_title_prefix\":\"[ci-fix] \",\"target\":\"*\"},\"missing_data\":{},\"missing_tool\":{},\"noop\":{\"max\":1,\"report-as-issue\":\"true\"},\"push_to_pull_request_branch\":{\"allowed_files\":[\"src/Core/**\",\"src/Controls/**\",\"src/Essentials/**\",\"src/BlazorWebView/**\",\"src/TestUtils/**\",\"src/Templates/**\",\"**/PublicAPI.Unshipped.txt\"],\"if_no_changes\":\"warn\",\"max\":3,\"max_patch_size\":4096,\"protect_top_level_dot_folders\":true,\"protected_files\":[\"package.json\",\"bun.lockb\",\"bunfig.toml\",\"deno.json\",\"deno.jsonc\",\"deno.lock\",\"global.json\",\"NuGet.Config\",\"Directory.Packages.props\",\"mix.exs\",\"mix.lock\",\"go.mod\",\"go.sum\",\"stack.yaml\",\"stack.yaml.lock\",\"pom.xml\",\"build.gradle\",\"build.gradle.kts\",\"settings.gradle\",\"settings.gradle.kts\",\"gradle.properties\",\"package-lock.json\",\"yarn.lock\",\"pnpm-lock.yaml\",\"npm-shrinkwrap.json\",\"requirements.txt\",\"Pipfile\",\"Pipfile.lock\",\"pyproject.toml\",\"setup.py\",\"setup.cfg\",\"Gemfile\",\"Gemfile.lock\",\"uv.lock\",\"CODEOWNERS\",\"DESIGN.md\",\"README.md\",\"CONTRIBUTING.md\",\"CHANGELOG.md\",\"SECURITY.md\",\"CODE_OF_CONDUCT.md\",\"AGENTS.md\",\"CLAUDE.md\",\"GEMINI.md\"],\"required_labels\":[\"agentic-workflows\"],\"target\":\"*\",\"title_prefix\":\"[ci-fix] \"},\"report_incomplete\":{},\"update_pull_request\":{\"allow_body\":true,\"allow_title\":false,\"max\":3,\"target\":\"*\",\"update_branch\":false}}" GH_AW_CI_TRIGGER_TOKEN: ${{ secrets.GH_AW_CI_TRIGGER_TOKEN }} with: github-token: ${{ secrets.GH_AW_GITHUB_TOKEN || secrets.GITHUB_TOKEN }} diff --git a/.github/workflows/ci-status-fix.md b/.github/workflows/ci-status-fix.md index d3227e25170c..9c2f46054029 100644 --- a/.github/workflows/ci-status-fix.md +++ b/.github/workflows/ci-status-fix.md @@ -15,7 +15,9 @@ description: | humans (the open tracking issue is the hand-off surface; a dedicated [ci-fix][needs-human] PR is planned but currently deferred — see Step 6). CI on the PR is kicked by a human `/azp run` for now; the loop watches, classifies, - and re-fixes autonomously between kicks. + and re-fixes autonomously between kicks. When the SPECIFIC test a PR fixed is + confirmed green in that PR's own CI, the loop marks the draft PR ready for review + (a state transition only — it never approves or merges). Never mutes tests, but de-flakes genuinely flaky ones (deterministic synchronization, no retries / timeout bumps). Always skips visual-regression / screenshot issues. @@ -239,8 +241,15 @@ safe-outputs: # PER-RUN total (not per-PR): a sweep may surface several green PRs and/or # annotate several flaky reds in one run. At max:1 all but the first comment # were silently dropped, starving the workflow's #1 value (surfacing green PRs - # for review). Raised to 3 to match the per-run throughput of the other outputs. - max: 3 + # for review). Sized to 6 (not 3) because a single green DRAFT PR that gets + # flipped ready in one sweep spends TWO comment slots — the Step 3 ✅ surface + # comment (which still names the /azp-gated legs a human must kick, so it is NOT + # redundant with 🎯) AND the Step 3.6 T3 🎯 readiness comment. At max:3 the shared + # bucket drained after ~1 draft flip and T3's atomicity pre-check then deferred + # every further mark-ready even while the mark-ready/add_labels buckets (also 3) + # sat idle; 6 lets ~3 draft flips (2 comments each) land per sweep, so the + # mark-ready:3 / add_labels:3 caps are actually reachable. + max: 6 target: "*" # Hard constraint (defense-in-depth): only comment on THIS workflow's own # ci-fix PRs — [ci-fix] title prefix AND agentic-workflows label. Mirrors the @@ -261,13 +270,51 @@ safe-outputs: # never the title, so disable title rewrites — this compiles to allow_title:false # and removes any ability to retitle an arbitrary PR. title: false - # NOTE: gh-aw v0.79.8 does NOT emit required-title-prefix/required-labels into + # NOTE: gh-aw v0.80.9 (the pinned compiler) does NOT emit required-title-prefix/required-labels into # the compiled config for update-pull-request (verified against the lock — it # silently drops them, unlike add-comment / push-to-pull-request-branch which # honor them). So which-PR scoping here relies on prompt Hard-Rule 6 + # min-integrity:approved; the residual is body-marker edits only (no code, no # merge, no close), capped at max:3. Revisit required-* if a future gh-aw # compiler emits them for this output. + mark-pull-request-as-ready-for-review: + # Flip a validated draft [ci-fix] PR to ready-for-review once the SPECIFIC test + # this PR was opened to fix is confirmed green in the PR's OWN CI (Step 3.6) — + # even when unrelated legs are red. The workflow is schedule/dispatch-triggered + # (no triggering-PR context), so target "*" lets the agent name the PR number it + # validated. This is a state transition ONLY: it never approves and never merges — + # a human still reviews and merges. + # PER-RUN total (not per-PR): a sweep may validate several PRs' target tests green. + max: 3 + target: "*" + # Hard constraint (defense-in-depth): only ever un-draft THIS workflow's own + # ci-fix PRs — [ci-fix] title prefix AND agentic-workflows label — so a confused + # or prompt-injected agent cannot mark an arbitrary PR ready. (If the v0.80.9 + # compiler silently drops these — as documented for update-pull-request above — + # the Step 3.6 preconditions + min-integrity:approved are the compensating scope + # controls; verify against the lock after compiling.) + required-title-prefix: "[ci-fix] " + required-labels: [agentic-workflows] + add-labels: + # Apply the p/0 priority label to a [ci-fix] PR at the exact moment the loop flips it + # from draft to ready-for-review (Step 3.6 T3) — so a validated, review-ready fix lands + # in the team's p/0 triage queue instead of sitting unseen in the draft backlog. Paired + # 1:1 with mark-pull-request-as-ready-for-review; target "*" lets the agent name the PR + # it just validated. + # allowed = HARD allowlist: the agent may ONLY ever add p/0, nothing else. This caps the + # blast radius of a confused/prompt-injected agent to exactly one benign priority label — + # it can never apply a downstream-triggering or destructive label. + allowed: [p/0] + # PER-RUN total (not per-PR): a sweep may mark several PRs ready in one run; each adds + # one label, so this matches the mark-ready per-run cap (3). + max: 3 + target: "*" + # Hard constraint (defense-in-depth): only ever label THIS workflow's own ci-fix PRs — + # [ci-fix] title prefix AND agentic-workflows label. (If the v0.80.9 compiler drops these + # — as documented for update-pull-request above — the Step 3.6 preconditions + + # min-integrity:approved + the allowed:[p/0] allowlist are the compensating controls.) + required-title-prefix: "[ci-fix] " + required-labels: [agentic-workflows] timeout-minutes: 90 @@ -339,33 +386,48 @@ through `safe-outputs`. 5. **One issue = one outcome per run.** Exactly one of: respond to a maintainer's change-request on the open PR (Track C, Step 3.5.R); open the first fix/help/ de-flake PR; advance an existing PR by one attempt (push a follow-up fix); - surface a validated-green PR for review; annotate an unrelated-flake red; wait + surface a validated-green PR for review; mark a target-validated draft PR + ready-for-review (Step 3.6 T3 — the terminal outcome that supersedes the same + run's surface/annotate precursor line), or record its `already-ready` + steady-state no-op; annotate an unrelated-flake red; wait (CI not yet settled); or a recorded skip (the dedicated needs-human PR is deferred — the attempt cap records a skip instead; see Step 6). Always prefer advancing or opening a PR over a skip when a non-mute diff is producible. 6. **All writes via `safe-outputs`.** Allowed outputs: `create_pull_request` (first attempt only), `push_to_pull_request_branch` (advance an existing PR by one attempt), `update_pull_request` (bump the attempt marker / refresh the - prior-attempts table), and `add_comment` (progress notes on the `[ci-fix]` PR - ONLY). NEVER comment on the tracking issue (issues are locked by + prior-attempts table), `add_comment` (progress notes on the `[ci-fix]` PR + ONLY), `mark_pull_request_as_ready_for_review` (flip a target-validated draft + `[ci-fix]` PR from draft to ready — Step 3.6 T3 only), and `add_labels` (add + ONLY the `p/0` label, ONLY on that same draft→ready transition — Step 3.6 T3). + NEVER comment on the tracking issue (issues are locked by `.github/workflows/ci-scan-lock-issues.yml`) — `add_comment` targets the PR. No `gh pr create`, no manual `git push`. **Defense-in-depth:** `add_comment` and `update_pull_request` use `target: "*"` (the agent supplies the PR number). - `add_comment` is now config-locked to `required-title-prefix: "[ci-fix] "` + + `add_comment`, `mark_pull_request_as_ready_for_review`, and `add_labels` are + config-locked to `required-title-prefix: "[ci-fix] "` + `required-labels: [agentic-workflows]` (hard-enforced by the handler, same as - `push_to_pull_request_branch`), so a comment can only ever land on THIS - workflow's own PRs. `update_pull_request` CANNOT be config-locked to a - title/label in gh-aw v0.79.8 (the compiler silently drops `required-*` for that - output — verified against the lock), so it keeps `title: false` (no retitles; - body-marker edits only) plus this prompt-level guard. Before emitting either, - VERIFY the target PR carries BOTH the `[ci-fix]` title prefix AND the - `agentic-workflows` label; never comment on or edit the body of any PR that - lacks both. + `push_to_pull_request_branch`), and `add_labels` additionally has an + `allowed: [p/0]` allowlist so it can ONLY ever add `p/0` — so these can only + ever land on THIS workflow's own PRs. `update_pull_request` CANNOT be + config-locked to a title/label in gh-aw v0.80.9 (the compiler silently drops + `required-*` for that output — verified against the lock), so it keeps + `title: false` (no retitles; body-marker edits only) plus this prompt-level + guard. Before emitting ANY of these, VERIFY the target PR carries BOTH the + `[ci-fix]` title prefix AND the `agentic-workflows` label; never comment on, + edit, un-draft, or label any PR that lacks both. 7. **Per-run safe-output caps (per RUN, not per PR).** Each output type is capped per run: `create_pull_request` 3, `push_to_pull_request_branch` 3, - `add_comment` 3, `update_pull_request` 3. Note an ADVANCE spends one push **and** - one update **and** one comment, so ≤ 3 advances/run; surfacing a green or - annotating a flake spends one comment. When a bucket is exhausted, do NOT keep + `add_comment` 6, `update_pull_request` 3, `mark_pull_request_as_ready_for_review` + 3, `add_labels` 3 (counts LABELS, not calls). Note an ADVANCE spends one push + **and** one update **and** one comment, so ≤ 3 advances/run; surfacing a green or + annotating a flake spends one comment; a **mark-ready (Step 3.6 T3)** of a + still-draft PR spends TWO comments (the Step 3 ✅ surface **and** the T3 🎯 + readiness note) **and** one mark-ready **and** one label as an ALL-OR-NOTHING set + (T3 pre-checks those buckets and defers the whole PR if any is exhausted — never + mark a PR ready without its 🎯 audit comment). `add_comment` is therefore sized 6 + (not 3) so ~3 draft flips, at 2 comments each, can land in one sweep instead of the + shared comment bucket starving the otherwise-idle mark-ready/add_labels buckets. When a bucket is exhausted, do NOT keep emitting (extras are silently dropped) — record `skipped: per-run cap reached; deferring PR # to next cycle` for each remaining PR so the drop is a deliberate, logged decision. The continuous loop picks the deferred PRs up next @@ -409,8 +471,9 @@ For every open tracking issue in scope, converge on exactly one outcome: | Open first help-wanted draft `[ci-fix]` PR | No open PR yet; a plausible candidate exists but cannot be runner-validated (device/UI tests) (attempt 1) | | Open first de-flake draft `[ci-fix]` PR | No open PR yet; failure is intermittent (green-on-retry) from a genuine test-quality defect; a deterministic-synchronization fix is producible (attempt 1, Step 4.7 bucket b) | | Advance an existing PR (push attempt N+1) | Open PR's own CI settled red *because of the fix itself*, no human engaged, marker < 10 — push a NEW distinct fix onto the same branch (Step 3.5 → 5.6) | -| Surface a validated-green PR | Open PR's own CI settled green — comment "validated, attempt N/10; ready for review" and do NOT advance (Step 3.5) | -| Annotate an unrelated-flake red | Open PR is red only on baseline-flake / unrelated legs — comment which leg needs a re-run; do NOT burn an attempt (Step 3.5) | +| Surface a validated-green PR | Open PR's own CI settled green — comment "validated, attempt N/10; ready for review" and do NOT advance (Step 3.5), then mark the draft PR ready for review once the fixed test is confirmed green (Step 3.6) | +| Annotate an unrelated-flake red | Open PR is red only on baseline-flake / unrelated legs — comment which leg needs a re-run; do NOT burn an attempt (Step 3.5); if the SPECIFIC fixed test is confirmed green on that build, still mark the draft PR ready for review (Step 3.6) | +| Mark a validated PR ready for review | The specific test this PR fixed is confirmed green in the PR's own CI (even if unrelated legs are red) — comment the target-test result and transition the draft PR to ready for review; never approve or merge (Step 3.6) | | Wait | Open PR's CI is pending / not yet settled on the current head SHA — do nothing this cycle (Step 3.5) | | Hand-off skip (attempt cap) | Marker == 10 and the signature still reproduces — stop and defer to humans; the dedicated `[ci-fix][needs-human]` PR is planned but currently deferred (Step 6) | | Recorded skip | Visual-regression, human engaged, already-handled, fixed-in-latest-build, infra-flake, out-of-bounds, only-mute-available, or no novel approach producible | @@ -685,14 +748,34 @@ Run these gates in order — the FIRST that fires decides this cycle's outcome: 1. **Human engaged.** If `C.humanEngaged` → `skipped: human engaged on PR #; deferring` and stop. Never fight a human reviewer — the intentional hand-off boundary still holds the moment a person touches the PR. -2. **CI not settled / unknown / incomplete prefetch.** If `C.checksSettled == false` - OR `C.overallConclusion` is `pending`, `neutral`, or `unknown`, OR - `C.dataComplete == false` → the fix's CI has not finished on the current head SHA - (`C.headSha`), or the prefetch could not fully read this PR (a partial read can - understate `humanEngaged` / `attempt`). `skipped: PR # CI pending / prefetch - incomplete on ; waiting` and stop. *(Round 1: this is where a - maintainer `/azp run` is awaited — the `/azp`-gated uitests/devicetests legs will - not have run until a human kicks them.)* +2. **CI not settled — but validate the target test first (target-focused readiness).** + If `C.checksSettled == false` OR `C.overallConclusion` is `pending`, `neutral`, or + `unknown`, OR `C.dataComplete == false` → the *overall* build has not finished on the + current head SHA (`C.headSha`), or the prefetch could not fully read this PR (a partial + read can understate `humanEngaged` / `attempt`). Unrelated legs still draining must NOT + keep an already-proven fix parked as a draft, so before waiting, try the target-focused + fast-path: + - **2a — Target-green fast-path (draft `[ci-fix]` PRs only).** If `C.dataComplete == + true` AND the Step 3.6 preconditions hold (draft PR, `[ci-fix] ` title prefix AND the + `agentic-workflows` label), run Step 3.6 **T1–T2** against `C.headSha` now: identify + the target test(s) and read the PR's OWN build **timeline / per-leg status** (anonymous + `_apis/build` — never `_apis/test`, per Hard-Rule 8). If EVERY target test is + **VALIDATED-GREEN** (per T2's all-platform definition below — the target's category leg + is `succeeded` on every platform that runs it, failed on none, and NO target platform + family the pipeline covers is still pending/unconcluded, per T2's "no platform left + unverified" rule; a platform that simply has no leg for the target's category — the test + doesn't run there — is fine, not a gap) AND every leg in + `C.failedLegs` that has ALREADY concluded classifies as **unrelated flake** (Step 4 / + Step 4.7 method — the fix is implicated in NO completed red), then the fix is proven + regardless of unrelated *pending* legs → run **Step 3.6 T3** (mark ready + 🎯 comment) + and stop. This early-out NEVER advances an attempt and NEVER acts on a red that could + be the fix's fault. + - **2b — Otherwise WAIT.** If the target test has not yet executed (pending/absent on + its leg), or a completed red is (or may be) caused by the fix, or `C.dataComplete == + false`, or this PR has no identifiable target test → `skipped: PR # CI pending / + target not yet validated on ; waiting` and stop. *(Round 1: this is where a + maintainer `/azp run` is awaited — the `/azp`-gated uitests/devicetests legs will not + have run until a human kicks them.)* 3. **Green → surface for review.** If `C.overallConclusion == "success"`: the checks that RAN are green. **Primary-gate check first:** the deterministic `success` verdict only certifies "at least one green check, nothing failing or @@ -704,18 +787,31 @@ Run these gates in order — the FIRST that fires decides this cycle's outcome: # primary CI gate (maui-pr) not green on ; waiting` and stop — do NOT surface. (The `/azp`-gated `maui-pr-uitests` (def 313) / `maui-pr-devicetests` (def 314) legs MAY still be un-run — that is expected and is named below, not a - reason to withhold the surface.) **Idempotency:** scan the PR's existing comments - for a prior bot `✅ … validated … on ` note for THIS head SHA — if one - already exists, record `already-surfaced PR # (head )` and stop - WITHOUT re-commenting (re-surfacing the same green every tick is noise and burns - the per-run comment budget other PRs need). Otherwise resolve the attempt number from - `C.effectiveAttempt` (the authoritative max(marker, bot-commit) counter; omit the - number only if it is somehow indeterminate). **Dry-run gate (Step 0):** if `dry_run == "true"`, do NOT emit any comment — instead print the intended `✅` validated-green body to the run log, tally `dry-run: would-surface-green PR #`, and stop. Otherwise `add_comment` on PR #: `✅ Attempt /10 validated - — the fix's CI is green on . Ready for human review.` - Name any `/azp`-gated legs (uitests def 313 / devicetests def 314) that have not - run and still need a maintainer `/azp run`. Do NOT advance. Record - `surfaced-green PR # (attempt /10)` and stop. *(This directly - attacks the real bottleneck — no reviews — so it is the highest-value outcome.)* + reason to withhold the surface.) **Comment idempotency + dry-run suppress the ✅ + comment ONLY — neither skips Step 3.6.** Scan the PR's existing comments for a prior + bot `✅ … validated … on ` note for THIS head SHA; if one exists, set + `SKIP_SURFACE_COMMENT = true`. Resolve the attempt number from `C.effectiveAttempt` + (the authoritative max(marker, bot-commit) counter; omit the number only if it is + somehow indeterminate). Then post the surface comment UNLESS suppressed: + - if `dry_run == "true"`: do NOT emit — print the intended `✅` validated-green body to + the run log and tally `dry-run: would-surface-green PR #`; + - else if `SKIP_SURFACE_COMMENT`: do NOT re-post — record `already-surfaced PR # + (head )` (re-surfacing the same green every tick is noise and burns the + per-run comment budget other PRs need); + - else `add_comment` on PR #: `✅ Attempt /10 validated — the fix's CI is + green on .` naming any `/azp`-gated legs (uitests def 313 / devicetests def + 314) that have not run and still need a maintainer `/azp run`; record `surfaced-green + PR # (attempt /10)`. (Do NOT assert "ready for human review" on a PR that + is still a draft — Step 3.6, not this comment, owns the draft→ready flip and posts its + own 🎯 announcement when it fires.) + Do NOT advance. Then — in ALL of the above cases — run **Step 3.6** (target-test + readiness gate) for this PR before stopping: its `/azp`-gated target legs may conclude + green on this SAME head SHA on a LATER sweep (an `/azp run` adds no commit), and Step + 3.6 is the ONLY place the draft→ready flip happens, so it MUST re-evaluate every sweep — + never `stop` here before it. *(This directly attacks the real bottleneck — no reviews — + so it is the highest-value outcome.)* 4. **Red → classify caused-by-fix vs unrelated-flake.** If `C.overallConclusion == "failure"`, analyze the PR's OWN failing build (NOT `main`): find the AzDO `maui-pr` build for this PR (filter builds by `branchName=refs/pull//merge`, @@ -723,18 +819,26 @@ Run these gates in order — the FIRST that fires decides this cycle's outcome: method and Step 4.7 flake buckets to `C.failedLegs`. - **Unrelated flake only** (every failed leg is known-flaky / infra / a pre-existing baseline red NOT introduced by the fix): do NOT burn an attempt. - **Idempotency first:** scan the PR's existing comments for a prior bot - `♻️ … unrelated flake … on ` note for THIS head SHA — if one already - exists, record `already-annotated-flake PR # (head )` and stop - WITHOUT re-commenting (re-annotating the same flake every 12h sweep is noise and - burns the per-run comment budget other PRs need). Otherwise resolve the attempt - number from `C.effectiveAttempt` (the authoritative max(marker, bot-commit) - counter), omitting the `Attempt /10:` prefix only if it is somehow - indeterminate. **Dry-run gate (Step 0):** if `dry_run == "true"`, do NOT emit any comment — instead print the intended `♻️` unrelated-flake body to the run log, tally `dry-run: would-annotate-flake PR #`, and stop. Otherwise `add_comment` on PR #: `♻️ Attempt /10: red is - unrelated flake on leg(s) () on ; the fix itself is not - implicated. A maintainer re-run (/azp run ) should clear it.` Record - `annotated-flake PR # (head )` and stop. *(Round 1: human re-runs; - Round 2: auto re-trigger.)* + **Comment idempotency + dry-run suppress the ♻️ comment ONLY — neither skips Step + 3.6.** Scan the PR's existing comments for a prior bot `♻️ … unrelated flake … on + ` note for THIS head SHA; if one exists, set `SKIP_FLAKE_COMMENT = true`. + Resolve the attempt number from `C.effectiveAttempt` (the authoritative max(marker, + bot-commit) counter), omitting the `Attempt /10:` prefix only if it is + somehow indeterminate. Then post the flake note UNLESS suppressed: + - if `dry_run == "true"`: do NOT emit — print the intended `♻️` unrelated-flake body + to the run log and tally `dry-run: would-annotate-flake PR #`; + - else if `SKIP_FLAKE_COMMENT`: do NOT re-post — record `already-annotated-flake PR + # (head )` (re-annotating the same flake every 12h sweep is noise and + burns the per-run comment budget other PRs need); + - else `add_comment` on PR #: `♻️ Attempt /10: red is unrelated flake on + leg(s) () on ; the fix itself is not implicated. A + maintainer re-run (/azp run ) should clear it.` record `annotated-flake + PR # (head )`. + Then — in ALL of the above cases — run **Step 3.6** (target-test readiness gate) for + this PR before stopping: its `/azp`-gated target legs may conclude green on this SAME + head SHA on a LATER sweep, and Step 3.6 is the ONLY place the draft→ready flip + happens, so it MUST re-evaluate every sweep — never `stop` here before it. *(Round 1: + human re-runs; Round 2: auto re-trigger.)* - **Caused by the fix** (a failed leg still matches the original target signature, or the fix introduced a NEW failure): advance an attempt. - **Attempt count.** `attempt = C.effectiveAttempt` — the authoritative @@ -752,6 +856,188 @@ Run these gates in order — the FIRST that fires decides this cycle's outcome: from every prior commit on this branch**, emitted via the Step 5.6 ADVANCE path (push onto the same PR). +#### Step 3.6 — Target-test verification & mark-ready gate + +Reached from the Step 3 **green-surface** branch, the Step 4 **unrelated-flake** branch +(AFTER that branch has posted its comment), and the Step 3.5 **gate-2 target-focused +fast-path** (2a — while unrelated legs are still pending). Purpose: confirm the SPECIFIC +test(s) this PR was opened to fix now PASS in the PR's own CI, and — only when they do +— transition the draft `[ci-fix]` PR to **ready for review** so a maintainer sees a +validated fix instead of a draft. This is the ONLY place the loop flips draft→ready; it +is a state transition, **never** an approval or a merge (a human still reviews and +merges). Overall red on *unrelated* legs must NOT gate readiness — we validate the fix, +not the base branch's flakiness. + +**Preconditions** (ALL must hold; otherwise record `skipped: readiness N/A PR # +()` and stop this gate): +- `C.isDraft == true` — if the PR is already ready-for-review, the draft→ready + transition is already done; do NOT re-mark **and do NOT re-apply `p/0`**. `p/0` is applied + exactly ONCE, atomically with the draft→ready flip (T3 — all-or-nothing with the 🎯 audit + comment + mark-ready), so an already-ready loop PR that is MISSING `p/0` has almost + certainly had it **removed by a maintainer** de-prioritizing the PR — a pure triage action + the human-engagement guard (which inspects comments/reviews/commits, NOT label changes) + cannot see. Re-adding `p/0` every sweep would fight that maintainer indefinitely, violating + the loop's "never override a human" contract. So the loop does NOT reconcile the label: + record `already-ready PR #` and stop this gate. (Trade-off: on the rare occasion a + transient API error drops `p/0` at flip time *after* the T3 atomic pre-check passed, it is + not auto-re-added — but the PR is still ready-for-review with its 🎯 audit comment, and + re-adding `p/0` is a trivial manual action; that is strictly preferable to steam-rolling a + maintainer's deliberate de-prioritization.) +- The PR is unmistakably THIS workflow's own: `[ci-fix] ` title prefix AND the + `agentic-workflows` label (mirrors the safe-output lock; the compensating scope + control if the v0.80.9 compiler drops the declarative required-*). +- You reached this gate from Step 3 (green), Step 4 (**unrelated-flake**), or the Step 3.5 + gate-2 **target-focused fast-path** (2a) — i.e. the fix is NOT implicated in any red that + has concluded. If Step 4 classified a red as **caused by the fix** (ADVANCE), do NOT run + this gate — advance the attempt instead. + +**T1 — Identify the target test(s).** From the `[ci-scan]` issue signature the fix +addresses (and the PR's own diff), extract the fully-qualified test method name(s) the +fix targets — e.g. `SafeAreaShouldWorkOnAllShellTabs`. For a de-flake it is the +de-flaked test; for a product fix it is the originally-failing test(s). If NO specific +test can be identified (e.g. a product build-break rather than a test failure), this is a +**build-only fix** — there is no single test to validate cross-platform, so validate at +**whole-build** granularity rather than stopping (Step 3 only *comments*; Step 3.6 is the +sole draft→ready flip, so a build-only fix is undrafted HERE or not at all). We only reach +this gate from Step 3 (green) / Step 4 (unrelated-flake) / gate-2a, so the fix is not +implicated in any concluded red; additionally require, via the T2 timeline method on +`C.headSha`, that the primary build pipeline `maui-pr` (def 302) has CONCLUDED with EVERY +one of its platform build legs `succeeded`/`completed` and NONE failed/canceled or still +pending — the build is green on **every** platform, not just the originally-broken one (the +cross-platform guard, applied to the build instead of a test). A build-only fix is undrafted +ONLY on the strength of the auto-running `maui-pr` (def 302) whole-build green, which the +loop CAN observe: if the PR's `[ci-scan]` issue signature or its own diff indicates the +ORIGINATING failure was in a `/azp`-gated pipeline (`maui-pr-uitests` def 313 or +`maui-pr-devicetests` def 314) rather than `maui-pr` (def 302), do NOT undraft on +`maui-pr`-green alone — those pipelines do not auto-run on this PR (GITHUB_TOKEN cannot +trigger them), so a green `maui-pr` build is NOT evidence the gated build break is fixed; +record `skipped: build-only fix PR # targets gated pipeline () not run — +deferring to human` and stop this gate. Otherwise, set `TARGET := "the maui-pr build +(build-only fix — no single target test)"` and proceed to **T3** to mark ready. If any `maui-pr` build leg is still unconcluded, record `skipped: build-only fix PR +# not yet whole-build green (leg(s) pending)` and stop this gate WITHOUT marking +ready (a green subset is not enough — a still-pending build leg could yet fail). + +**T2 — Verify the target test's platform legs are green (anonymous, leg-level).** Per-test +outcomes come from the AzDO **test-results** API (`_apis/test/...`), which is **NOT reachable +anonymously — Hard-Rule 8**: those endpoints 302-redirect to a sign-in page on this runner, so +a `curl` returns an HTML redirect (not JSON) and every target test would look "not executed". +Validate instead at **leg granularity** using ONLY the anonymous `_apis/build/...` timeline +(Hard-Rule 8) — a `succeeded` category leg means every test in that category **that actually +ran** passed on that platform. This is a strong bar **only if the target test genuinely +executes**: a job can go green while one specific test is skipped at runtime (`[Ignore]`, +`Assert.Ignore()`, `Assert.Inconclusive()`, a `Skip=`/conditional `[Fact]`, an `#if`-out, or +an early `return` before the asserts). So before trusting a green leg as proof the target +passed, **confirm from the PR's OWN diff that the fix does NOT skip, ignore, disable, or +short-circuit the target test** (it adds no `[Ignore]`/`[Explicit]`, `Assert.Ignore`/ +`Assert.Inconclusive`, `Skip=`, category-exclusion, `#if`-out, or early `return` around the +target). If the fix could cause the target to be runtime-skipped rather than genuinely pass, +leg-level green is NOT sufficient evidence — record `skipped: target test may be +runtime-skipped by this fix (diff adds ); leg-green insufficient, deferring to human +on PR #` and stop this gate WITHOUT marking ready. Otherwise (the target genuinely runs +and asserts, as a de-flake or product fix does) a green category leg cannot hide a +target-test failure, so treat it as authoritative. + +Map the target test to the CI leg(s) that run it. A leg name encodes platform + test category +(e.g. `Android UITests SafeAreaEdges,Shadow`, `iOS UITests SafeAreaEdges,Shadow`, `macOS +UITests SafeAreaEdges,Shadow`, `Windows UITests SafeAreaEdges,Shadow`). Determine the target +test's UI-test category — its `[Category(UITestCategories.X)]` in the test/HostApp source, +visible in the PR diff or the test file — to know which leg-name substring identifies its legs; +for a device test, the per-platform device-test legs. + +Using the SAME build-discovery as Step 4 (filter AzDO builds by `branchName=refs/pull//merge` +or `sourceVersion == C.headSha`), read each build's **timeline** on `C.headSha` for the +pipeline(s) that RUN the target test — `maui-pr` (def 302) for unit/integration tests, +`maui-pr-uitests` (def 313) for Appium UI tests, `maui-pr-devicetests` (def 314) for device +tests: + +```bash +ORG=dnceng-public; PROJ=public; BUILD= +# Anonymous + Hard-Rule-8-compliant. NEVER call _apis/test/... (it 302-redirects to sign-in). +# Each type=="Job" record is one platform leg: result (succeeded/failed/canceled/null), +# state (completed/inProgress/pending), name (encodes " UITests "): +curl -s "https://dev.azure.com/$ORG/$PROJ/_apis/build/builds/$BUILD/timeline?api-version=7.1" \ + | tee /tmp/gh-aw/agent/timeline_${BUILD}.json \ + | jq -r '.records[] | select(.type=="Job") | "\(.result)\t\(.state)\t\(.name)"' +``` + +(The prefetch already exposes per-leg status in `C.failedLegs` / the checks context, and +`gh pr checks ` lists the same per-leg rows with platform+category in the name — use +whichever is handy; the build timeline is the authoritative cross-check on the exact build.) + +Treat a target test as **VALIDATED-GREEN** only if, on `C.headSha`, the leg that runs its +category is green on **every platform it runs on** — a fix that repairs one platform must not +silently regress the same test's leg on another, so a green leg on the originally-red platform +alone is NOT enough. Across the target pipeline's platform legs (for a UI test: the Android, +iOS, Windows, and macOS/MacCatalyst legs running the target's category; for a device test: each +device-test platform), require ALL of: +- the target's category leg is `result == "succeeded"` **and** `state == "completed"` on + **every** platform that runs it, AND +- that leg is `failed` / `canceled` / aborted on **NO** platform, AND +- **no platform is left unverified.** For each platform family the target pipeline covers, that + platform's category leg must have CONCLUDED (`state == "completed"`) on `C.headSha`. If a + platform simply has no leg for the target's category, the test does not run there — that is + fine, not a gap. But if a platform's category leg has **not concluded** (`state` is + `inProgress` / pending, or an `/azp`-gated `maui-pr-uitests` / `maui-pr-devicetests` leg that + has not been kicked), the test's status on that platform is UNKNOWN → the fix is NOT yet + validated across platforms: record `skipped: target test green on but + not yet verified on (leg(s) pending / need /azp run) on PR #` + and stop this gate WITHOUT marking ready. + +A target test whose category leg never concluded on ANY platform (**not executed** anywhere — +e.g. its `/azp`-gated pipeline has not been kicked) is likewise NOT validated: record `skipped: +target test not yet executed on PR # ( not run — needs /azp run)` and stop +this gate WITHOUT marking ready. Do NOT overclaim — a green *sibling* leg (a different category +on the same platform) is not the target's leg, and a green leg on one platform is not a pass on +the others. + +**T3 — Mark ready + report.** If EVERY target test is VALIDATED-GREEN. The 🎯 comment, +the mark-ready, and the `p/0` label are THREE SEPARATE safe-outputs — the comment +existing does NOT prove the mark-ready took effect, so they are tracked independently: + +- **Comment idempotency (dup-suppress only — never gates the mark-ready).** We only reach + T3 while `C.isDraft == true` (the precondition stops an already-ready PR before here). So + if a prior bot comment for THIS head that carries the leading `🎯` anchor AND the + contiguous phrase `validated green on ` already exists (BOTH T3 variants — + `🎯 Target test validated green on ` and the build-only `🎯 Build validated green on + ` — contain that exact contiguous phrase, so this recognizes both; keying on + `target test` alone would instead miss the build-only comment and re-post it every sweep. + **Require the `🎯` anchor** so the match can NEVER be loosened to a gapped + `validated…green…on` form that would false-positive on the Step 3 `✅ … validated — the + fix's CI is green on ` surface comment and wrongly suppress the audit 🎯), the comment + landed on an earlier sweep but the **mark-ready did NOT take + effect** (the PR is still a draft) — set `SUPPRESS_COMMENT = true` (do not re-post the + duplicate 🎯 comment) but STILL complete the mark-ready + label below. Never treat the + comment's existence as "already marked ready" while the PR is still a draft. Record this + case as `re-marking PR # (🎯 present; mark-ready did not take on a prior sweep)`. +- **Atomicity budget pre-check (Hard-Rule 7).** Determine the outputs this gate will emit: + `mark_pull_request_as_ready_for_review` + `add_labels` always, plus `add_comment` unless + `SUPPRESS_COMMENT`. Verify a free per-run slot remains in EACH bucket you are about to + use. If ANY required bucket is exhausted, emit NONE of them — record `skipped: per-run + cap reached; deferring mark-ready PR # to next cycle` and stop this gate. Never mark a + PR ready (or label it) unless its 🎯 audit comment is GUARANTEED to exist for this exact + head SHA — either posted in this same sweep, or (in the `SUPPRESS_COMMENT` re-marking case) + already present from a prior sweep. The outputs you DO emit either all land or all defer + together; the audit trail must never be absent, but it is NOT re-posted when it already exists. +- **Dry-run gate (Step 0):** if `dry_run == "true"`, emit NOTHING — print the intended + readiness comment and "would mark ready + add p/0" to the run log, tally `dry-run: + would-mark-ready PR #`, and stop. +- Otherwise emit for THIS PR number ``: + 1. `add_comment` (ONLY if not `SUPPRESS_COMMENT`): `🎯 Target test validated green on + — passed on ALL platforms it runs on (, + buildId ). — not + caused by this fix."> Transitioning this PR from draft to ready for review and adding + `p/0`; a maintainer still reviews and merges.` (For a **build-only fix** — the T1 + whole-build fallback, no single target test — phrase the first clause as `🎯 Build + validated green on — the maui-pr build passed on ALL platforms (buildId + ).` instead of naming a test.) + 2. `mark_pull_request_as_ready_for_review` with `reason:` a one-line justification + naming the validated test(s) and ``. + 3. `add_labels` with `labels: ["p/0"]` for PR # — put the now-review-ready fix into + the team's p/0 priority queue so it is triaged, not lost in the draft backlog. (If + the PR somehow already carries `p/0`, this is a harmless no-op.) +- Record `marked-ready PR # (target green on ALL platforms on , + labeled p/0)` and stop. + #### Step 3.5.R — Maintainer change-request response (Track C) Reached from Step 3.5 gate 0. Goal: when an **eligible human reviewer** (Hard-Rule @@ -1383,7 +1669,19 @@ Filed by [`ci-status-fix`](https://github.com/dotnet/maui/blob/main/.github/work ### Step 8 — Per-issue tally + end-of-run summary -Per issue, append one outcome line to `/tmp/gh-aw/agent/coverage.txt`: +Per issue, append **one** outcome line to `/tmp/gh-aw/agent/coverage.txt` — the +terminal outcome for the cycle. When one cycle produces a chained pair — a PR gets a +same-cycle precursor line and **then** a Step 3.6 readiness line (`marked-ready` on the +draft→ready flip, or `already-ready` on a later steady-state sweep of an already-ready +PR) — record ONLY the terminal Step 3.6 readiness line: it supersedes ANY same-cycle +non-readiness precursor for that PR, so an aggregator keying on one-line-per-issue never +double-counts. The precursor may be a Step 3 green line (`surfaced-green` on the first ✅, +or `already-surfaced` when it already exists) OR — when an unrelated-flake red and a +target-green coincide — a Step 4 `annotated-flake` line; either way the readiness line is +terminal, and the superseded precursor's signal survives in the PR's 🎯/♻️ comment so no +information is lost. This covers the flip cycle (`surfaced-green` → `marked-ready`), the +post-flip steady state (`already-surfaced` → `already-ready`), and the flake-coincident +flip (`annotated-flake` → `marked-ready`): ``` # main attempt- @@ -1394,8 +1692,14 @@ Per issue, append one outcome line to `/tmp/gh-aw/agent/coverage.txt`: `advance-PR # attempt /10` (ADVANCE mode — new commit pushed to the existing PR), `surfaced-green PR #` (fix's own CI went green; commented for review, did not advance), `annotated-flake PR #` (red was unrelated flake; -commented, attempt NOT burned), `waiting PR #` (CI not settled yet), -`dry-run: would-`, `skipped: `. (The +commented, attempt NOT burned), `marked-ready PR #` (the specific fixed test +was confirmed green in the PR's own CI; draft flipped to ready for review), +`already-surfaced PR #` (the fix's ✅ green comment already existed this sweep; +re-post suppressed — superseded by the Step 3.6 readiness line when the PR also flips +ready that cycle), `already-ready PR #` (terminal steady state — the PR was flipped +ready on a prior cycle and remains green on an unchanged head; no action taken, +supersedes `already-surfaced`), `waiting PR #` (CI not settled yet), +`dry-run: would-`, `skipped: `. (The `needs-human-PR` outcome is reserved for the deferred hand-off — Step 6 currently records a skip instead, so it is not emitted.)
primary CI gate (maui-pr) not green on ; waiting` and stop — do NOT surface. (The `/azp`-gated `maui-pr-uitests` (def 313) / `maui-pr-devicetests` (def 314) legs MAY still be un-run — that is expected and is named below, not a - reason to withhold the surface.) **Idempotency:** scan the PR's existing comments - for a prior bot `✅ … validated … on ` note for THIS head SHA — if one - already exists, record `already-surfaced PR # (head )` and stop - WITHOUT re-commenting (re-surfacing the same green every tick is noise and burns - the per-run comment budget other PRs need). Otherwise resolve the attempt number from - `C.effectiveAttempt` (the authoritative max(marker, bot-commit) counter; omit the - number only if it is somehow indeterminate). **Dry-run gate (Step 0):** if `dry_run == "true"`, do NOT emit any comment — instead print the intended `✅` validated-green body to the run log, tally `dry-run: would-surface-green PR #`, and stop. Otherwise `add_comment` on PR #: `✅ Attempt /10 - validated — the fix's CI is green on . Ready for human review.` - Name any `/azp`-gated legs (uitests def 313 / devicetests def 314) that have not - run and still need a maintainer `/azp run`. Do NOT advance. Record - `surfaced-green PR # (attempt /10)` and stop. *(This directly - attacks the real bottleneck — no reviews — so it is the highest-value outcome.)* + reason to withhold the surface.) **Comment idempotency + dry-run suppress the ✅ + comment ONLY — neither skips Step 3.6.** Scan the PR's existing comments for a prior + bot `✅ … validated … on ` note for THIS head SHA; if one exists, set + `SKIP_SURFACE_COMMENT = true`. Resolve the attempt number from `C.effectiveAttempt` + (the authoritative max(marker, bot-commit) counter; omit the number only if it is + somehow indeterminate). Then post the surface comment UNLESS suppressed: + - if `dry_run == "true"`: do NOT emit — print the intended `✅` validated-green body to + the run log and tally `dry-run: would-surface-green PR #`; + - else if `SKIP_SURFACE_COMMENT`: do NOT re-post — record `already-surfaced PR # + (head )` (re-surfacing the same green every tick is noise and burns the + per-run comment budget other PRs need); + - else `add_comment` on PR #: `✅ Attempt /10 validated — the fix's CI is + green on .` naming any `/azp`-gated legs (uitests def 313 / devicetests def + 314) that have not run and still need a maintainer `/azp run`; record `surfaced-green + PR # (attempt /10)`. (Do NOT assert "ready for human review" on a PR that + is still a draft — Step 3.6, not this comment, owns the draft→ready flip and posts its + own 🎯 announcement when it fires.) + Do NOT advance. Then — in ALL of the above cases — run **Step 3.6** (target-test + readiness gate) for this PR before stopping: its `/azp`-gated target legs may conclude + green on this SAME head SHA on a LATER sweep (an `/azp run` adds no commit), and Step + 3.6 is the ONLY place the draft→ready flip happens, so it MUST re-evaluate every sweep — + never `stop` here before it. *(This directly attacks the real bottleneck — no reviews — + so it is the highest-value outcome.)* 4. **Red → classify caused-by-fix vs unrelated-flake.** If `C.overallConclusion == "failure"`, analyze the PR's OWN failing build (NOT `net11.0`): find the AzDO `maui-pr` build for this PR (filter builds by `branchName=refs/pull//merge`, @@ -733,18 +829,26 @@ Run these gates in order — the FIRST that fires decides this cycle's outcome: method and Step 4.7 flake buckets to `C.failedLegs`. - **Unrelated flake only** (every failed leg is known-flaky / infra / a pre-existing baseline red NOT introduced by the fix): do NOT burn an attempt. - **Idempotency first:** scan the PR's existing comments for a prior bot - `♻️ … unrelated flake … on ` note for THIS head SHA — if one already - exists, record `already-annotated-flake PR # (head )` and stop - WITHOUT re-commenting (re-annotating the same flake every 12h sweep is noise and - burns the per-run comment budget other PRs need). Otherwise resolve the attempt - number from `C.effectiveAttempt` (the authoritative max(marker, bot-commit) - counter), omitting the `Attempt /10:` prefix only if it is somehow - indeterminate. **Dry-run gate (Step 0):** if `dry_run == "true"`, do NOT emit any comment — instead print the intended `♻️` unrelated-flake body to the run log, tally `dry-run: would-annotate-flake PR #`, and stop. Otherwise `add_comment` on PR #: `♻️ Attempt /10: red is - unrelated flake on leg(s) () on ; the fix itself is not - implicated. A maintainer re-run (/azp run ) should clear it.` Record - `annotated-flake PR # (head )` and stop. *(Round 1: human re-runs; - Round 2: auto re-trigger.)* + **Comment idempotency + dry-run suppress the ♻️ comment ONLY — neither skips Step + 3.6.** Scan the PR's existing comments for a prior bot `♻️ … unrelated flake … on + ` note for THIS head SHA; if one exists, set `SKIP_FLAKE_COMMENT = true`. + Resolve the attempt number from `C.effectiveAttempt` (the authoritative max(marker, + bot-commit) counter), omitting the `Attempt /10:` prefix only if it is + somehow indeterminate. Then post the flake note UNLESS suppressed: + - if `dry_run == "true"`: do NOT emit — print the intended `♻️` unrelated-flake body + to the run log and tally `dry-run: would-annotate-flake PR #`; + - else if `SKIP_FLAKE_COMMENT`: do NOT re-post — record `already-annotated-flake PR + # (head )` (re-annotating the same flake every 12h sweep is noise and + burns the per-run comment budget other PRs need); + - else `add_comment` on PR #: `♻️ Attempt /10: red is unrelated flake on + leg(s) () on ; the fix itself is not implicated. A + maintainer re-run (/azp run ) should clear it.` record `annotated-flake + PR # (head )`. + Then — in ALL of the above cases — run **Step 3.6** (target-test readiness gate) for + this PR before stopping: its `/azp`-gated target legs may conclude green on this SAME + head SHA on a LATER sweep, and Step 3.6 is the ONLY place the draft→ready flip + happens, so it MUST re-evaluate every sweep — never `stop` here before it. *(Round 1: + human re-runs; Round 2: auto re-trigger.)* - **Caused by the fix** (a failed leg still matches the original target signature, or the fix introduced a NEW failure): advance an attempt. - **Attempt count.** `attempt = C.effectiveAttempt` — the authoritative @@ -762,6 +866,188 @@ Run these gates in order — the FIRST that fires decides this cycle's outcome: from every prior commit on this branch**, emitted via the Step 5.6 ADVANCE path (push onto the same PR). +#### Step 3.6 — Target-test verification & mark-ready gate + +Reached from the Step 3 **green-surface** branch, the Step 4 **unrelated-flake** branch +(AFTER that branch has posted its comment), and the Step 3.5 **gate-2 target-focused +fast-path** (2a — while unrelated legs are still pending). Purpose: confirm the SPECIFIC +test(s) this PR was opened to fix now PASS in the PR's own CI, and — only when they do +— transition the draft `[ci-fix-net11]` PR to **ready for review** so a maintainer sees +a validated fix instead of a draft. This is the ONLY place the loop flips draft→ready; +it is a state transition, **never** an approval or a merge (a human still reviews and +merges). Overall red on *unrelated* legs must NOT gate readiness — we validate the fix, +not the base branch's flakiness. + +**Preconditions** (ALL must hold; otherwise record `skipped: readiness N/A PR # +()` and stop this gate): +- `C.isDraft == true` — if the PR is already ready-for-review, the draft→ready + transition is already done; do NOT re-mark **and do NOT re-apply `p/0`**. `p/0` is applied + exactly ONCE, atomically with the draft→ready flip (T3 — all-or-nothing with the 🎯 audit + comment + mark-ready), so an already-ready loop PR that is MISSING `p/0` has almost + certainly had it **removed by a maintainer** de-prioritizing the PR — a pure triage action + the human-engagement guard (which inspects comments/reviews/commits, NOT label changes) + cannot see. Re-adding `p/0` every sweep would fight that maintainer indefinitely, violating + the loop's "never override a human" contract. So the loop does NOT reconcile the label: + record `already-ready PR #` and stop this gate. (Trade-off: on the rare occasion a + transient API error drops `p/0` at flip time *after* the T3 atomic pre-check passed, it is + not auto-re-added — but the PR is still ready-for-review with its 🎯 audit comment, and + re-adding `p/0` is a trivial manual action; that is strictly preferable to steam-rolling a + maintainer's deliberate de-prioritization.) +- The PR is unmistakably THIS workflow's own: `[ci-fix-net11] ` title prefix AND the + `agentic-workflows` label (mirrors the safe-output lock; the compensating scope + control if the v0.80.9 compiler drops the declarative required-*). +- You reached this gate from Step 3 (green), Step 4 (**unrelated-flake**), or the Step 3.5 + gate-2 **target-focused fast-path** (2a) — i.e. the fix is NOT implicated in any red that + has concluded. If Step 4 classified a red as **caused by the fix** (ADVANCE), do NOT run + this gate — advance the attempt instead. + +**T1 — Identify the target test(s).** From the `[ci-scan-net11]` issue signature the fix +addresses (and the PR's own diff), extract the fully-qualified test method name(s) the +fix targets — e.g. `SafeAreaShouldWorkOnAllShellTabs`. For a de-flake it is the +de-flaked test; for a product fix it is the originally-failing test(s). If NO specific +test can be identified (e.g. a product build-break rather than a test failure), this is a +**build-only fix** — there is no single test to validate cross-platform, so validate at +**whole-build** granularity rather than stopping (Step 3 only *comments*; Step 3.6 is the +sole draft→ready flip, so a build-only fix is undrafted HERE or not at all). We only reach +this gate from Step 3 (green) / Step 4 (unrelated-flake) / gate-2a, so the fix is not +implicated in any concluded red; additionally require, via the T2 timeline method on +`C.headSha`, that the primary build pipeline `maui-pr` (def 302) has CONCLUDED with EVERY +one of its platform build legs `succeeded`/`completed` and NONE failed/canceled or still +pending — the build is green on **every** platform, not just the originally-broken one (the +cross-platform guard, applied to the build instead of a test). A build-only fix is undrafted +ONLY on the strength of the auto-running `maui-pr` (def 302) whole-build green, which the +loop CAN observe: if the PR's `[ci-scan]` issue signature or its own diff indicates the +ORIGINATING failure was in a `/azp`-gated pipeline (`maui-pr-uitests` def 313 or +`maui-pr-devicetests` def 314) rather than `maui-pr` (def 302), do NOT undraft on +`maui-pr`-green alone — those pipelines do not auto-run on this PR (GITHUB_TOKEN cannot +trigger them), so a green `maui-pr` build is NOT evidence the gated build break is fixed; +record `skipped: build-only fix PR # targets gated pipeline () not run — +deferring to human` and stop this gate. Otherwise, set `TARGET := "the maui-pr build +(build-only fix — no single target test)"` and proceed to **T3** to mark ready. If any `maui-pr` build leg is still unconcluded, record `skipped: build-only fix PR +# not yet whole-build green (leg(s) pending)` and stop this gate WITHOUT marking +ready (a green subset is not enough — a still-pending build leg could yet fail). + +**T2 — Verify the target test's platform legs are green (anonymous, leg-level).** Per-test +outcomes come from the AzDO **test-results** API (`_apis/test/...`), which is **NOT reachable +anonymously — Hard-Rule 8**: those endpoints 302-redirect to a sign-in page on this runner, so +a `curl` returns an HTML redirect (not JSON) and every target test would look "not executed". +Validate instead at **leg granularity** using ONLY the anonymous `_apis/build/...` timeline +(Hard-Rule 8) — a `succeeded` category leg means every test in that category **that actually +ran** passed on that platform. This is a strong bar **only if the target test genuinely +executes**: a job can go green while one specific test is skipped at runtime (`[Ignore]`, +`Assert.Ignore()`, `Assert.Inconclusive()`, a `Skip=`/conditional `[Fact]`, an `#if`-out, or +an early `return` before the asserts). So before trusting a green leg as proof the target +passed, **confirm from the PR's OWN diff that the fix does NOT skip, ignore, disable, or +short-circuit the target test** (it adds no `[Ignore]`/`[Explicit]`, `Assert.Ignore`/ +`Assert.Inconclusive`, `Skip=`, category-exclusion, `#if`-out, or early `return` around the +target). If the fix could cause the target to be runtime-skipped rather than genuinely pass, +leg-level green is NOT sufficient evidence — record `skipped: target test may be +runtime-skipped by this fix (diff adds ); leg-green insufficient, deferring to human +on PR #` and stop this gate WITHOUT marking ready. Otherwise (the target genuinely runs +and asserts, as a de-flake or product fix does) a green category leg cannot hide a +target-test failure, so treat it as authoritative. + +Map the target test to the CI leg(s) that run it. A leg name encodes platform + test category +(e.g. `Android UITests SafeAreaEdges,Shadow`, `iOS UITests SafeAreaEdges,Shadow`, `macOS +UITests SafeAreaEdges,Shadow`, `Windows UITests SafeAreaEdges,Shadow`). Determine the target +test's UI-test category — its `[Category(UITestCategories.X)]` in the test/HostApp source, +visible in the PR diff or the test file — to know which leg-name substring identifies its legs; +for a device test, the per-platform device-test legs. + +Using the SAME build-discovery as Step 4 (filter AzDO builds by `branchName=refs/pull//merge` +or `sourceVersion == C.headSha`), read each build's **timeline** on `C.headSha` for the +pipeline(s) that RUN the target test — `maui-pr` (def 302) for unit/integration tests, +`maui-pr-uitests` (def 313) for Appium UI tests, `maui-pr-devicetests` (def 314) for device +tests: + +```bash +ORG=dnceng-public; PROJ=public; BUILD= +# Anonymous + Hard-Rule-8-compliant. NEVER call _apis/test/... (it 302-redirects to sign-in). +# Each type=="Job" record is one platform leg: result (succeeded/failed/canceled/null), +# state (completed/inProgress/pending), name (encodes " UITests "): +curl -s "https://dev.azure.com/$ORG/$PROJ/_apis/build/builds/$BUILD/timeline?api-version=7.1" \ + | tee /tmp/gh-aw/agent/timeline_${BUILD}.json \ + | jq -r '.records[] | select(.type=="Job") | "\(.result)\t\(.state)\t\(.name)"' +``` + +(The prefetch already exposes per-leg status in `C.failedLegs` / the checks context, and +`gh pr checks ` lists the same per-leg rows with platform+category in the name — use +whichever is handy; the build timeline is the authoritative cross-check on the exact build.) + +Treat a target test as **VALIDATED-GREEN** only if, on `C.headSha`, the leg that runs its +category is green on **every platform it runs on** — a fix that repairs one platform must not +silently regress the same test's leg on another, so a green leg on the originally-red platform +alone is NOT enough. Across the target pipeline's platform legs (for a UI test: the Android, +iOS, Windows, and macOS/MacCatalyst legs running the target's category; for a device test: each +device-test platform), require ALL of: +- the target's category leg is `result == "succeeded"` **and** `state == "completed"` on + **every** platform that runs it, AND +- that leg is `failed` / `canceled` / aborted on **NO** platform, AND +- **no platform is left unverified.** For each platform family the target pipeline covers, that + platform's category leg must have CONCLUDED (`state == "completed"`) on `C.headSha`. If a + platform simply has no leg for the target's category, the test does not run there — that is + fine, not a gap. But if a platform's category leg has **not concluded** (`state` is + `inProgress` / pending, or an `/azp`-gated `maui-pr-uitests` / `maui-pr-devicetests` leg that + has not been kicked), the test's status on that platform is UNKNOWN → the fix is NOT yet + validated across platforms: record `skipped: target test green on but + not yet verified on (leg(s) pending / need /azp run) on PR #` + and stop this gate WITHOUT marking ready. + +A target test whose category leg never concluded on ANY platform (**not executed** anywhere — +e.g. its `/azp`-gated pipeline has not been kicked) is likewise NOT validated: record `skipped: +target test not yet executed on PR # ( not run — needs /azp run)` and stop +this gate WITHOUT marking ready. Do NOT overclaim — a green *sibling* leg (a different category +on the same platform) is not the target's leg, and a green leg on one platform is not a pass on +the others. + +**T3 — Mark ready + report.** If EVERY target test is VALIDATED-GREEN. The 🎯 comment, +the mark-ready, and the `p/0` label are THREE SEPARATE safe-outputs — the comment +existing does NOT prove the mark-ready took effect, so they are tracked independently: + +- **Comment idempotency (dup-suppress only — never gates the mark-ready).** We only reach + T3 while `C.isDraft == true` (the precondition stops an already-ready PR before here). So + if a prior bot comment for THIS head that carries the leading `🎯` anchor AND the + contiguous phrase `validated green on ` already exists (BOTH T3 variants — + `🎯 Target test validated green on ` and the build-only `🎯 Build validated green on + ` — contain that exact contiguous phrase, so this recognizes both; keying on + `target test` alone would instead miss the build-only comment and re-post it every sweep. + **Require the `🎯` anchor** so the match can NEVER be loosened to a gapped + `validated…green…on` form that would false-positive on the Step 3 `✅ … validated — the + fix's CI is green on ` surface comment and wrongly suppress the audit 🎯), the comment + landed on an earlier sweep but the **mark-ready did NOT take + effect** (the PR is still a draft) — set `SUPPRESS_COMMENT = true` (do not re-post the + duplicate 🎯 comment) but STILL complete the mark-ready + label below. Never treat the + comment's existence as "already marked ready" while the PR is still a draft. Record this + case as `re-marking PR # (🎯 present; mark-ready did not take on a prior sweep)`. +- **Atomicity budget pre-check (Hard-Rule 7).** Determine the outputs this gate will emit: + `mark_pull_request_as_ready_for_review` + `add_labels` always, plus `add_comment` unless + `SUPPRESS_COMMENT`. Verify a free per-run slot remains in EACH bucket you are about to + use. If ANY required bucket is exhausted, emit NONE of them — record `skipped: per-run + cap reached; deferring mark-ready PR # to next cycle` and stop this gate. Never mark a + PR ready (or label it) unless its 🎯 audit comment is GUARANTEED to exist for this exact + head SHA — either posted in this same sweep, or (in the `SUPPRESS_COMMENT` re-marking case) + already present from a prior sweep. The outputs you DO emit either all land or all defer + together; the audit trail must never be absent, but it is NOT re-posted when it already exists. +- **Dry-run gate (Step 0):** if `dry_run == "true"`, emit NOTHING — print the intended + readiness comment and "would mark ready + add p/0" to the run log, tally `dry-run: + would-mark-ready PR #`, and stop. +- Otherwise emit for THIS PR number ``: + 1. `add_comment` (ONLY if not `SUPPRESS_COMMENT`): `🎯 Target test validated green on + — passed on ALL platforms it runs on (, + buildId ). — not + caused by this fix."> Transitioning this PR from draft to ready for review and adding + `p/0`; a maintainer still reviews and merges.` (For a **build-only fix** — the T1 + whole-build fallback, no single target test — phrase the first clause as `🎯 Build + validated green on — the maui-pr build passed on ALL platforms (buildId + ).` instead of naming a test.) + 2. `mark_pull_request_as_ready_for_review` with `reason:` a one-line justification + naming the validated test(s) and ``. + 3. `add_labels` with `labels: ["p/0"]` for PR # — put the now-review-ready fix into + the team's p/0 priority queue so it is triaged, not lost in the draft backlog. (If + the PR somehow already carries `p/0`, this is a harmless no-op.) +- Record `marked-ready PR # (target green on ALL platforms on , + labeled p/0)` and stop. + #### Step 3.5.R — Maintainer change-request response (Track C) Reached from Step 3.5 gate 0. Goal: when an **eligible human reviewer** (Hard-Rule @@ -1395,7 +1681,19 @@ Filed by [`ci-status-fix-net11`](https://github.com/dotnet/maui/blob/main/.githu ### Step 8 — Per-issue tally + end-of-run summary -Per issue, append one outcome line to `/tmp/gh-aw/agent/coverage.txt`: +Per issue, append **one** outcome line to `/tmp/gh-aw/agent/coverage.txt` — the +terminal outcome for the cycle. When one cycle produces a chained pair — a PR gets a +same-cycle precursor line and **then** a Step 3.6 readiness line (`marked-ready` on the +draft→ready flip, or `already-ready` on a later steady-state sweep of an already-ready +PR) — record ONLY the terminal Step 3.6 readiness line: it supersedes ANY same-cycle +non-readiness precursor for that PR, so an aggregator keying on one-line-per-issue never +double-counts. The precursor may be a Step 3 green line (`surfaced-green` on the first ✅, +or `already-surfaced` when it already exists) OR — when an unrelated-flake red and a +target-green coincide — a Step 4 `annotated-flake` line; either way the readiness line is +terminal, and the superseded precursor's signal survives in the PR's 🎯/♻️ comment so no +information is lost. This covers the flip cycle (`surfaced-green` → `marked-ready`), the +post-flip steady state (`already-surfaced` → `already-ready`), and the flake-coincident +flip (`annotated-flake` → `marked-ready`): ``` # net11.0 attempt- @@ -1406,8 +1704,14 @@ Per issue, append one outcome line to `/tmp/gh-aw/agent/coverage.txt`: `advance-PR # attempt /10` (ADVANCE mode — new commit pushed to the existing PR), `surfaced-green PR #` (fix's own CI went green; commented for review, did not advance), `annotated-flake PR #` (red was unrelated flake; -commented, attempt NOT burned), `waiting PR #` (CI not settled yet), -`dry-run: would-`, `skipped: `. (The +commented, attempt NOT burned), `marked-ready PR #` (the specific fixed test +was confirmed green in the PR's own CI; draft flipped to ready for review), +`already-surfaced PR #` (the fix's ✅ green comment already existed this sweep; +re-post suppressed — superseded by the Step 3.6 readiness line when the PR also flips +ready that cycle), `already-ready PR #` (terminal steady state — the PR was flipped +ready on a prior cycle and remains green on an unchanged head; no action taken, +supersedes `already-surfaced`), `waiting PR #` (CI not settled yet), +`dry-run: would-`, `skipped: `. (The `needs-human-PR` outcome is reserved for the deferred hand-off — Step 6 currently records a skip instead, so it is not emitted.) diff --git a/.github/workflows/ci-status-fix.lock.yml b/.github/workflows/ci-status-fix.lock.yml index 93dc5969cb0d..65511f2b3259 100644 --- a/.github/workflows/ci-status-fix.lock.yml +++ b/.github/workflows/ci-status-fix.lock.yml @@ -1,4 +1,4 @@ -# gh-aw-metadata: {"schema_version":"v4","frontmatter_hash":"f014161967589ebcaf1ac5ec6f3c88ee5a4388d28b55a07dc36c45b7b2aebab6","body_hash":"86c6b2d4c3df13da14c6a3c8b211381fcc9f97bb1951ff7b80588a2c5338e00c","compiler_version":"v0.80.9","strict":true,"agent_id":"copilot","agent_model":"claude-opus-4.8","engine_versions":{"copilot":"1.0.63"}} +# gh-aw-metadata: {"schema_version":"v4","frontmatter_hash":"b0ebf8a2f179900cfe3638e994b27a43cec52bdbdd2331dc1bd4d65762fa2d6b","body_hash":"1825e2b1f7c7bd88241bf37580655489360f9bebc806edd00837a52ce4ad83a0","compiler_version":"v0.80.9","strict":true,"agent_id":"copilot","agent_model":"claude-opus-4.8","engine_versions":{"copilot":"1.0.63"}} # gh-aw-manifest: {"version":1,"secrets":["COPILOT_PAT_0","COPILOT_PAT_1","COPILOT_PAT_2","COPILOT_PAT_3","COPILOT_PAT_4","COPILOT_PAT_5","COPILOT_PAT_6","COPILOT_PAT_7","COPILOT_PAT_8","COPILOT_PAT_9","GH_AW_CI_TRIGGER_TOKEN","GH_AW_GITHUB_MCP_SERVER_TOKEN","GH_AW_GITHUB_TOKEN","GITHUB_TOKEN"],"actions":[{"repo":"actions/checkout","sha":"34e114876b0b11c390a56381ad16ebd13914f8d5","version":"v4"},{"repo":"actions/checkout","sha":"9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0","version":"v7.0.0"},{"repo":"actions/checkout","sha":"de0fac2e4500dabe0009e67214ff5f5447ce83dd","version":"v6.0.2"},{"repo":"actions/download-artifact","sha":"3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c","version":"v8.0.1"},{"repo":"actions/github-script","sha":"3a2844b7e9c422d3c10d287c895573f7108da1b3","version":"v9.0.0"},{"repo":"actions/setup-node","sha":"48b55a011bda9f5d6aeb4c2d9c7362e8dae4041e","version":"v6.4.0"},{"repo":"actions/upload-artifact","sha":"043fb46d1a93c77aae656e7c1c64a875d1fc6a0a","version":"v7.0.1"},{"repo":"github/gh-aw-actions/setup","sha":"8c7d04ebf1ece56cd381446125da3e0f6896294a","version":"v0.80.9"}],"containers":[{"image":"ghcr.io/github/gh-aw-firewall/agent:0.27.7","digest":"sha256:aae231e4635c8999d039c132f1602d3df850fe9b84a00aa2b5ac981179b5661c","pinned_image":"ghcr.io/github/gh-aw-firewall/agent:0.27.7@sha256:aae231e4635c8999d039c132f1602d3df850fe9b84a00aa2b5ac981179b5661c"},{"image":"ghcr.io/github/gh-aw-firewall/api-proxy:0.27.7","digest":"sha256:009caf2e3d88fa77b64e9a03a95a228fc58db0f1701c6d324b29ba5a3c7c79b6","pinned_image":"ghcr.io/github/gh-aw-firewall/api-proxy:0.27.7@sha256:009caf2e3d88fa77b64e9a03a95a228fc58db0f1701c6d324b29ba5a3c7c79b6"},{"image":"ghcr.io/github/gh-aw-firewall/squid:0.27.7","digest":"sha256:deb1d4e19de62d51cee0508057a596a19315c3423ada4d675cad136dc8037c96","pinned_image":"ghcr.io/github/gh-aw-firewall/squid:0.27.7@sha256:deb1d4e19de62d51cee0508057a596a19315c3423ada4d675cad136dc8037c96"},{"image":"ghcr.io/github/gh-aw-mcpg:v0.3.27","digest":"sha256:fe984bddde4ec05d756d9043edb0a32912e6b7b72f6a121b1082f29221421cc7","pinned_image":"ghcr.io/github/gh-aw-mcpg:v0.3.27@sha256:fe984bddde4ec05d756d9043edb0a32912e6b7b72f6a121b1082f29221421cc7"},{"image":"ghcr.io/github/gh-aw-node","digest":"sha256:529d02eb970b1161aa25c593a9c3df57fdfad5a8add328cb3b6eccef66f3183b","pinned_image":"ghcr.io/github/gh-aw-node@sha256:529d02eb970b1161aa25c593a9c3df57fdfad5a8add328cb3b6eccef66f3183b"},{"image":"ghcr.io/github/github-mcp-server:v1.4.0","digest":"sha256:2afb26356481d1a350e14544a6e160f7f7ec1561a1ea309b823665abf0309036","pinned_image":"ghcr.io/github/github-mcp-server:v1.4.0@sha256:2afb26356481d1a350e14544a6e160f7f7ec1561a1ea309b823665abf0309036"}]} # This file was automatically generated by gh-aw (v0.80.9). DO NOT EDIT. To debug this workflow, load the skill at https://github.com/github/gh-aw/blob/main/debug.md # @@ -37,7 +37,9 @@ # humans (the open tracking issue is the hand-off surface; a dedicated # [ci-fix][needs-human] PR is planned but currently deferred — see Step 6). CI on # the PR is kicked by a human `/azp run` for now; the loop watches, classifies, -# and re-fixes autonomously between kicks. +# and re-fixes autonomously between kicks. When the SPECIFIC test a PR fixed is +# confirmed green in that PR's own CI, the loop marks the draft PR ready for review +# (a state transition only — it never approves or merges). # Never mutes tests, but # de-flakes genuinely flaky ones (deterministic synchronization, no retries / # timeout bumps). Always skips visual-regression / screenshot issues. @@ -273,24 +275,24 @@ jobs: run: | bash "${RUNNER_TEMP}/gh-aw/actions/create_prompt_first.sh" { - cat << 'GH_AW_PROMPT_25cf75111b670167_EOF' + cat << 'GH_AW_PROMPT_2ac1d0b4083428f8_EOF' - GH_AW_PROMPT_25cf75111b670167_EOF + GH_AW_PROMPT_2ac1d0b4083428f8_EOF cat "${RUNNER_TEMP}/gh-aw/prompts/xpia.md" cat "${RUNNER_TEMP}/gh-aw/prompts/temp_folder_prompt.md" cat "${RUNNER_TEMP}/gh-aw/prompts/markdown.md" cat "${RUNNER_TEMP}/gh-aw/prompts/safe_outputs_prompt.md" - cat << 'GH_AW_PROMPT_25cf75111b670167_EOF' + cat << 'GH_AW_PROMPT_2ac1d0b4083428f8_EOF' - Tools: add_comment(max:3), create_pull_request(max:3), update_pull_request(max:3), push_to_pull_request_branch(max:3), missing_tool, missing_data, noop - GH_AW_PROMPT_25cf75111b670167_EOF + Tools: add_comment(max:6), create_pull_request(max:3), update_pull_request(max:3), mark_pull_request_as_ready_for_review(max:3), add_labels(max:3), push_to_pull_request_branch(max:3), missing_tool, missing_data, noop + GH_AW_PROMPT_2ac1d0b4083428f8_EOF cat "${RUNNER_TEMP}/gh-aw/prompts/safe_outputs_create_pull_request.md" cat "${RUNNER_TEMP}/gh-aw/prompts/safe_outputs_push_to_pr_branch.md" - cat << 'GH_AW_PROMPT_25cf75111b670167_EOF' + cat << 'GH_AW_PROMPT_2ac1d0b4083428f8_EOF' - GH_AW_PROMPT_25cf75111b670167_EOF + GH_AW_PROMPT_2ac1d0b4083428f8_EOF cat "${RUNNER_TEMP}/gh-aw/prompts/mcp_cli_tools_prompt.md" - cat << 'GH_AW_PROMPT_25cf75111b670167_EOF' + cat << 'GH_AW_PROMPT_2ac1d0b4083428f8_EOF' The following GitHub context information is available for this workflow: {{#if github.actor}} @@ -332,12 +334,12 @@ jobs: stop immediately and report the limitation rather than spending turns trying to work around it. - GH_AW_PROMPT_25cf75111b670167_EOF + GH_AW_PROMPT_2ac1d0b4083428f8_EOF cat "${RUNNER_TEMP}/gh-aw/prompts/github_mcp_tools_with_safeoutputs_prompt.md" - cat << 'GH_AW_PROMPT_25cf75111b670167_EOF' + cat << 'GH_AW_PROMPT_2ac1d0b4083428f8_EOF' {{#runtime-import .github/workflows/ci-status-fix.md}} - GH_AW_PROMPT_25cf75111b670167_EOF + GH_AW_PROMPT_2ac1d0b4083428f8_EOF } > "$GH_AW_PROMPT" - name: Interpolate variables and render templates uses: actions/github-script@3a2844b7e9c422d3c10d287c895573f7108da1b3 # v9.0.0 @@ -554,16 +556,18 @@ jobs: mkdir -p "${RUNNER_TEMP}/gh-aw/safeoutputs" mkdir -p /tmp/gh-aw/safeoutputs mkdir -p /tmp/gh-aw/mcp-logs/safeoutputs - cat > "${RUNNER_TEMP}/gh-aw/safeoutputs/config.json" << 'GH_AW_SAFE_OUTPUTS_CONFIG_fa2a69fad5fd0485_EOF' - {"add_comment":{"discussions":false,"max":3,"required_labels":["agentic-workflows"],"required_title_prefix":"[ci-fix] ","target":"*"},"create_pull_request":{"allowed_base_branches":["main"],"allowed_branches":["ci-fix/**"],"allowed_files":["src/Core/**","src/Controls/**","src/Essentials/**","src/BlazorWebView/**","src/TestUtils/**","src/Templates/**","**/PublicAPI.Unshipped.txt"],"base_branch":"main","draft":true,"labels":["agentic-workflows"],"max":3,"max_patch_files":100,"max_patch_size":4096,"protect_top_level_dot_folders":true,"protected_files":["package.json","bun.lockb","bunfig.toml","deno.json","deno.jsonc","deno.lock","global.json","NuGet.Config","Directory.Packages.props","mix.exs","mix.lock","go.mod","go.sum","stack.yaml","stack.yaml.lock","pom.xml","build.gradle","build.gradle.kts","settings.gradle","settings.gradle.kts","gradle.properties","package-lock.json","yarn.lock","pnpm-lock.yaml","npm-shrinkwrap.json","requirements.txt","Pipfile","Pipfile.lock","pyproject.toml","setup.py","setup.cfg","Gemfile","Gemfile.lock","uv.lock","CODEOWNERS","DESIGN.md","README.md","CONTRIBUTING.md","CHANGELOG.md","SECURITY.md","CODE_OF_CONDUCT.md","AGENTS.md","CLAUDE.md","GEMINI.md"],"protected_files_policy":"request_review","title_prefix":"[ci-fix] "},"create_report_incomplete_issue":{},"missing_data":{},"missing_tool":{},"noop":{"max":1,"report-as-issue":"true"},"push_to_pull_request_branch":{"allowed_files":["src/Core/**","src/Controls/**","src/Essentials/**","src/BlazorWebView/**","src/TestUtils/**","src/Templates/**","**/PublicAPI.Unshipped.txt"],"if_no_changes":"warn","max":3,"max_patch_size":4096,"protect_top_level_dot_folders":true,"protected_files":["package.json","bun.lockb","bunfig.toml","deno.json","deno.jsonc","deno.lock","global.json","NuGet.Config","Directory.Packages.props","mix.exs","mix.lock","go.mod","go.sum","stack.yaml","stack.yaml.lock","pom.xml","build.gradle","build.gradle.kts","settings.gradle","settings.gradle.kts","gradle.properties","package-lock.json","yarn.lock","pnpm-lock.yaml","npm-shrinkwrap.json","requirements.txt","Pipfile","Pipfile.lock","pyproject.toml","setup.py","setup.cfg","Gemfile","Gemfile.lock","uv.lock","CODEOWNERS","DESIGN.md","README.md","CONTRIBUTING.md","CHANGELOG.md","SECURITY.md","CODE_OF_CONDUCT.md","AGENTS.md","CLAUDE.md","GEMINI.md"],"required_labels":["agentic-workflows"],"target":"*","title_prefix":"[ci-fix] "},"report_incomplete":{},"update_pull_request":{"allow_body":true,"allow_title":false,"max":3,"target":"*","update_branch":false}} - GH_AW_SAFE_OUTPUTS_CONFIG_fa2a69fad5fd0485_EOF + cat > "${RUNNER_TEMP}/gh-aw/safeoutputs/config.json" << 'GH_AW_SAFE_OUTPUTS_CONFIG_0644bf1434ca0071_EOF' + {"add_comment":{"discussions":false,"max":6,"required_labels":["agentic-workflows"],"required_title_prefix":"[ci-fix] ","target":"*"},"add_labels":{"allowed":["p/0"],"max":3,"required_labels":["agentic-workflows"],"required_title_prefix":"[ci-fix] ","target":"*"},"create_pull_request":{"allowed_base_branches":["main"],"allowed_branches":["ci-fix/**"],"allowed_files":["src/Core/**","src/Controls/**","src/Essentials/**","src/BlazorWebView/**","src/TestUtils/**","src/Templates/**","**/PublicAPI.Unshipped.txt"],"base_branch":"main","draft":true,"labels":["agentic-workflows"],"max":3,"max_patch_files":100,"max_patch_size":4096,"protect_top_level_dot_folders":true,"protected_files":["package.json","bun.lockb","bunfig.toml","deno.json","deno.jsonc","deno.lock","global.json","NuGet.Config","Directory.Packages.props","mix.exs","mix.lock","go.mod","go.sum","stack.yaml","stack.yaml.lock","pom.xml","build.gradle","build.gradle.kts","settings.gradle","settings.gradle.kts","gradle.properties","package-lock.json","yarn.lock","pnpm-lock.yaml","npm-shrinkwrap.json","requirements.txt","Pipfile","Pipfile.lock","pyproject.toml","setup.py","setup.cfg","Gemfile","Gemfile.lock","uv.lock","CODEOWNERS","DESIGN.md","README.md","CONTRIBUTING.md","CHANGELOG.md","SECURITY.md","CODE_OF_CONDUCT.md","AGENTS.md","CLAUDE.md","GEMINI.md"],"protected_files_policy":"request_review","title_prefix":"[ci-fix] "},"create_report_incomplete_issue":{},"mark_pull_request_as_ready_for_review":{"max":3,"required_labels":["agentic-workflows"],"required_title_prefix":"[ci-fix] ","target":"*"},"missing_data":{},"missing_tool":{},"noop":{"max":1,"report-as-issue":"true"},"push_to_pull_request_branch":{"allowed_files":["src/Core/**","src/Controls/**","src/Essentials/**","src/BlazorWebView/**","src/TestUtils/**","src/Templates/**","**/PublicAPI.Unshipped.txt"],"if_no_changes":"warn","max":3,"max_patch_size":4096,"protect_top_level_dot_folders":true,"protected_files":["package.json","bun.lockb","bunfig.toml","deno.json","deno.jsonc","deno.lock","global.json","NuGet.Config","Directory.Packages.props","mix.exs","mix.lock","go.mod","go.sum","stack.yaml","stack.yaml.lock","pom.xml","build.gradle","build.gradle.kts","settings.gradle","settings.gradle.kts","gradle.properties","package-lock.json","yarn.lock","pnpm-lock.yaml","npm-shrinkwrap.json","requirements.txt","Pipfile","Pipfile.lock","pyproject.toml","setup.py","setup.cfg","Gemfile","Gemfile.lock","uv.lock","CODEOWNERS","DESIGN.md","README.md","CONTRIBUTING.md","CHANGELOG.md","SECURITY.md","CODE_OF_CONDUCT.md","AGENTS.md","CLAUDE.md","GEMINI.md"],"required_labels":["agentic-workflows"],"target":"*","title_prefix":"[ci-fix] "},"report_incomplete":{},"update_pull_request":{"allow_body":true,"allow_title":false,"max":3,"target":"*","update_branch":false}} + GH_AW_SAFE_OUTPUTS_CONFIG_0644bf1434ca0071_EOF - name: Generate Safe Outputs Tools env: GH_AW_TOOLS_META_JSON: | { "description_suffixes": { - "add_comment": " CONSTRAINTS: Maximum 3 comment(s) can be added. Target: *. Supports reply_to_id for discussion threading.", + "add_comment": " CONSTRAINTS: Maximum 6 comment(s) can be added. Target: *. Supports reply_to_id for discussion threading.", + "add_labels": " CONSTRAINTS: Maximum 3 label(s) can be added. Only these labels are allowed: [\"p/0\"]. Target: *.", "create_pull_request": " CONSTRAINTS: Maximum 3 pull request(s) can be created. Title will be prefixed with \"[ci-fix] \". Labels [\"agentic-workflows\"] will be automatically added. Only these labels are allowed: [\"agentic-workflows\"]. PRs will be created as drafts.", + "mark_pull_request_as_ready_for_review": " CONSTRAINTS: Maximum 3 pull request(s) can be marked as ready for review.", "push_to_pull_request_branch": " CONSTRAINTS: Maximum 3 push(es) can be made. The target pull request title must start with \"[ci-fix] \".", "update_pull_request": " CONSTRAINTS: Maximum 3 pull request(s) can be updated. Target: *." }, @@ -594,6 +598,25 @@ jobs: } } }, + "add_labels": { + "defaultMax": 5, + "fields": { + "item_number": { + "issueNumberOrTemporaryId": true + }, + "labels": { + "required": true, + "type": "array", + "itemType": "string", + "itemSanitize": true, + "itemMaxLength": 128 + }, + "repo": { + "type": "string", + "maxLength": 256 + } + } + }, "create_pull_request": { "defaultMax": 1, "fields": { @@ -635,6 +658,24 @@ jobs: } } }, + "mark_pull_request_as_ready_for_review": { + "defaultMax": 1, + "fields": { + "pull_request_number": { + "issueOrPRNumber": true + }, + "reason": { + "required": true, + "type": "string", + "sanitize": true, + "maxLength": 65000 + }, + "repo": { + "type": "string", + "maxLength": 256 + } + } + }, "missing_data": { "defaultMax": 20, "fields": { @@ -1494,7 +1535,7 @@ jobs: uses: actions/github-script@3a2844b7e9c422d3c10d287c895573f7108da1b3 # v9.0.0 env: WORKFLOW_NAME: "CI Failure Fixer (main)" - WORKFLOW_DESCRIPTION: "Periodic pass over open ci-scan tracking issues filed by the main-branch CI\nfailure scanner (.github/workflows/ci-status-main.md). This workflow targets\nthe `main` branch EXCLUSIVELY: it processes only issues labelled ci-scan and\nopens every PR against main. (The net11.0 branch is handled by the parallel\n.github/workflows/ci-status-fix-net11.md — the two are split because gh-aw can\nonly transport a fix relative to ONE static base branch per workflow, and the\nmain↔net11.0 divergence exceeds gh-aw's 10 MB transport-patch cap.) The fixer\nopens ONE draft [ci-fix] PR per actionable issue against main, then WATCHES\nthat PR's own CI on later runs: when the fix's CI comes back red and the red is\ncaused by the fix itself, it pushes a fresh follow-up fix onto the SAME PR\nbranch (never a second PR) — up to 10 attempts — then stops and defers to\nhumans (the open tracking issue is the hand-off surface; a dedicated\n[ci-fix][needs-human] PR is planned but currently deferred — see Step 6). CI on\nthe PR is kicked by a human `/azp run` for now; the loop watches, classifies,\nand re-fixes autonomously between kicks.\nNever mutes tests, but\nde-flakes genuinely flaky ones (deterministic synchronization, no retries /\ntimeout bumps). Always skips visual-regression / screenshot issues." + WORKFLOW_DESCRIPTION: "Periodic pass over open ci-scan tracking issues filed by the main-branch CI\nfailure scanner (.github/workflows/ci-status-main.md). This workflow targets\nthe `main` branch EXCLUSIVELY: it processes only issues labelled ci-scan and\nopens every PR against main. (The net11.0 branch is handled by the parallel\n.github/workflows/ci-status-fix-net11.md — the two are split because gh-aw can\nonly transport a fix relative to ONE static base branch per workflow, and the\nmain↔net11.0 divergence exceeds gh-aw's 10 MB transport-patch cap.) The fixer\nopens ONE draft [ci-fix] PR per actionable issue against main, then WATCHES\nthat PR's own CI on later runs: when the fix's CI comes back red and the red is\ncaused by the fix itself, it pushes a fresh follow-up fix onto the SAME PR\nbranch (never a second PR) — up to 10 attempts — then stops and defers to\nhumans (the open tracking issue is the hand-off surface; a dedicated\n[ci-fix][needs-human] PR is planned but currently deferred — see Step 6). CI on\nthe PR is kicked by a human `/azp run` for now; the loop watches, classifies,\nand re-fixes autonomously between kicks. When the SPECIFIC test a PR fixed is\nconfirmed green in that PR's own CI, the loop marks the draft PR ready for review\n(a state transition only — it never approves or merges).\nNever mutes tests, but\nde-flakes genuinely flaky ones (deterministic synchronization, no retries /\ntimeout bumps). Always skips visual-regression / screenshot issues." HAS_PATCH: ${{ needs.agent.outputs.has_patch }} with: script: | @@ -1910,7 +1951,7 @@ jobs: GH_AW_ALLOWED_DOMAINS: "*.blob.core.windows.net,*.githubusercontent.com,api.business.githubcopilot.com,api.enterprise.githubcopilot.com,api.github.com,api.githubcopilot.com,api.individual.githubcopilot.com,api.snapcraft.io,archive.ubuntu.com,azure.archive.ubuntu.com,codeload.github.com,crl.geotrust.com,crl.globalsign.com,crl.identrust.com,crl.sectigo.com,crl.thawte.com,crl.usertrust.com,crl.verisign.com,crl3.digicert.com,crl4.digicert.com,crls.ssl.com,dev.azure.com,docs.github.com,github-cloud.githubusercontent.com,github-cloud.s3.amazonaws.com,github.blog,github.com,github.githubassets.com,helix.dot.net,host.docker.internal,json-schema.org,json.schemastore.org,keyserver.ubuntu.com,lfs.github.com,objects.githubusercontent.com,ocsp.digicert.com,ocsp.geotrust.com,ocsp.globalsign.com,ocsp.identrust.com,ocsp.sectigo.com,ocsp.ssl.com,ocsp.thawte.com,ocsp.usertrust.com,ocsp.verisign.com,packagecloud.io,packages.cloud.google.com,packages.microsoft.com,patch-diff.githubusercontent.com,ppa.launchpad.net,raw.githubusercontent.com,registry.npmjs.org,s.symcb.com,s.symcd.com,security.ubuntu.com,telemetry.enterprise.githubcopilot.com,ts-crl.ws.symantec.com,ts-ocsp.ws.symantec.com,www.googleapis.com" GITHUB_SERVER_URL: ${{ github.server_url }} GITHUB_API_URL: ${{ github.api_url }} - GH_AW_SAFE_OUTPUTS_HANDLER_CONFIG: "{\"add_comment\":{\"discussions\":false,\"max\":3,\"required_labels\":[\"agentic-workflows\"],\"required_title_prefix\":\"[ci-fix] \",\"target\":\"*\"},\"create_pull_request\":{\"allowed_base_branches\":[\"main\"],\"allowed_branches\":[\"ci-fix/**\"],\"allowed_files\":[\"src/Core/**\",\"src/Controls/**\",\"src/Essentials/**\",\"src/BlazorWebView/**\",\"src/TestUtils/**\",\"src/Templates/**\",\"**/PublicAPI.Unshipped.txt\"],\"base_branch\":\"main\",\"draft\":true,\"labels\":[\"agentic-workflows\"],\"max\":3,\"max_patch_files\":100,\"max_patch_size\":4096,\"protect_top_level_dot_folders\":true,\"protected_files\":[\"package.json\",\"bun.lockb\",\"bunfig.toml\",\"deno.json\",\"deno.jsonc\",\"deno.lock\",\"global.json\",\"NuGet.Config\",\"Directory.Packages.props\",\"mix.exs\",\"mix.lock\",\"go.mod\",\"go.sum\",\"stack.yaml\",\"stack.yaml.lock\",\"pom.xml\",\"build.gradle\",\"build.gradle.kts\",\"settings.gradle\",\"settings.gradle.kts\",\"gradle.properties\",\"package-lock.json\",\"yarn.lock\",\"pnpm-lock.yaml\",\"npm-shrinkwrap.json\",\"requirements.txt\",\"Pipfile\",\"Pipfile.lock\",\"pyproject.toml\",\"setup.py\",\"setup.cfg\",\"Gemfile\",\"Gemfile.lock\",\"uv.lock\",\"CODEOWNERS\",\"DESIGN.md\",\"README.md\",\"CONTRIBUTING.md\",\"CHANGELOG.md\",\"SECURITY.md\",\"CODE_OF_CONDUCT.md\",\"AGENTS.md\",\"CLAUDE.md\",\"GEMINI.md\"],\"protected_files_policy\":\"request_review\",\"title_prefix\":\"[ci-fix] \"},\"create_report_incomplete_issue\":{},\"missing_data\":{},\"missing_tool\":{},\"noop\":{\"max\":1,\"report-as-issue\":\"true\"},\"push_to_pull_request_branch\":{\"allowed_files\":[\"src/Core/**\",\"src/Controls/**\",\"src/Essentials/**\",\"src/BlazorWebView/**\",\"src/TestUtils/**\",\"src/Templates/**\",\"**/PublicAPI.Unshipped.txt\"],\"if_no_changes\":\"warn\",\"max\":3,\"max_patch_size\":4096,\"protect_top_level_dot_folders\":true,\"protected_files\":[\"package.json\",\"bun.lockb\",\"bunfig.toml\",\"deno.json\",\"deno.jsonc\",\"deno.lock\",\"global.json\",\"NuGet.Config\",\"Directory.Packages.props\",\"mix.exs\",\"mix.lock\",\"go.mod\",\"go.sum\",\"stack.yaml\",\"stack.yaml.lock\",\"pom.xml\",\"build.gradle\",\"build.gradle.kts\",\"settings.gradle\",\"settings.gradle.kts\",\"gradle.properties\",\"package-lock.json\",\"yarn.lock\",\"pnpm-lock.yaml\",\"npm-shrinkwrap.json\",\"requirements.txt\",\"Pipfile\",\"Pipfile.lock\",\"pyproject.toml\",\"setup.py\",\"setup.cfg\",\"Gemfile\",\"Gemfile.lock\",\"uv.lock\",\"CODEOWNERS\",\"DESIGN.md\",\"README.md\",\"CONTRIBUTING.md\",\"CHANGELOG.md\",\"SECURITY.md\",\"CODE_OF_CONDUCT.md\",\"AGENTS.md\",\"CLAUDE.md\",\"GEMINI.md\"],\"required_labels\":[\"agentic-workflows\"],\"target\":\"*\",\"title_prefix\":\"[ci-fix] \"},\"report_incomplete\":{},\"update_pull_request\":{\"allow_body\":true,\"allow_title\":false,\"max\":3,\"target\":\"*\",\"update_branch\":false}}" + GH_AW_SAFE_OUTPUTS_HANDLER_CONFIG: "{\"add_comment\":{\"discussions\":false,\"max\":6,\"required_labels\":[\"agentic-workflows\"],\"required_title_prefix\":\"[ci-fix] \",\"target\":\"*\"},\"add_labels\":{\"allowed\":[\"p/0\"],\"max\":3,\"required_labels\":[\"agentic-workflows\"],\"required_title_prefix\":\"[ci-fix] \",\"target\":\"*\"},\"create_pull_request\":{\"allowed_base_branches\":[\"main\"],\"allowed_branches\":[\"ci-fix/**\"],\"allowed_files\":[\"src/Core/**\",\"src/Controls/**\",\"src/Essentials/**\",\"src/BlazorWebView/**\",\"src/TestUtils/**\",\"src/Templates/**\",\"**/PublicAPI.Unshipped.txt\"],\"base_branch\":\"main\",\"draft\":true,\"labels\":[\"agentic-workflows\"],\"max\":3,\"max_patch_files\":100,\"max_patch_size\":4096,\"protect_top_level_dot_folders\":true,\"protected_files\":[\"package.json\",\"bun.lockb\",\"bunfig.toml\",\"deno.json\",\"deno.jsonc\",\"deno.lock\",\"global.json\",\"NuGet.Config\",\"Directory.Packages.props\",\"mix.exs\",\"mix.lock\",\"go.mod\",\"go.sum\",\"stack.yaml\",\"stack.yaml.lock\",\"pom.xml\",\"build.gradle\",\"build.gradle.kts\",\"settings.gradle\",\"settings.gradle.kts\",\"gradle.properties\",\"package-lock.json\",\"yarn.lock\",\"pnpm-lock.yaml\",\"npm-shrinkwrap.json\",\"requirements.txt\",\"Pipfile\",\"Pipfile.lock\",\"pyproject.toml\",\"setup.py\",\"setup.cfg\",\"Gemfile\",\"Gemfile.lock\",\"uv.lock\",\"CODEOWNERS\",\"DESIGN.md\",\"README.md\",\"CONTRIBUTING.md\",\"CHANGELOG.md\",\"SECURITY.md\",\"CODE_OF_CONDUCT.md\",\"AGENTS.md\",\"CLAUDE.md\",\"GEMINI.md\"],\"protected_files_policy\":\"request_review\",\"title_prefix\":\"[ci-fix] \"},\"create_report_incomplete_issue\":{},\"mark_pull_request_as_ready_for_review\":{\"max\":3,\"required_labels\":[\"agentic-workflows\"],\"required_title_prefix\":\"[ci-fix] \",\"target\":\"*\"},\"missing_data\":{},\"missing_tool\":{},\"noop\":{\"max\":1,\"report-as-issue\":\"true\"},\"push_to_pull_request_branch\":{\"allowed_files\":[\"src/Core/**\",\"src/Controls/**\",\"src/Essentials/**\",\"src/BlazorWebView/**\",\"src/TestUtils/**\",\"src/Templates/**\",\"**/PublicAPI.Unshipped.txt\"],\"if_no_changes\":\"warn\",\"max\":3,\"max_patch_size\":4096,\"protect_top_level_dot_folders\":true,\"protected_files\":[\"package.json\",\"bun.lockb\",\"bunfig.toml\",\"deno.json\",\"deno.jsonc\",\"deno.lock\",\"global.json\",\"NuGet.Config\",\"Directory.Packages.props\",\"mix.exs\",\"mix.lock\",\"go.mod\",\"go.sum\",\"stack.yaml\",\"stack.yaml.lock\",\"pom.xml\",\"build.gradle\",\"build.gradle.kts\",\"settings.gradle\",\"settings.gradle.kts\",\"gradle.properties\",\"package-lock.json\",\"yarn.lock\",\"pnpm-lock.yaml\",\"npm-shrinkwrap.json\",\"requirements.txt\",\"Pipfile\",\"Pipfile.lock\",\"pyproject.toml\",\"setup.py\",\"setup.cfg\",\"Gemfile\",\"Gemfile.lock\",\"uv.lock\",\"CODEOWNERS\",\"DESIGN.md\",\"README.md\",\"CONTRIBUTING.md\",\"CHANGELOG.md\",\"SECURITY.md\",\"CODE_OF_CONDUCT.md\",\"AGENTS.md\",\"CLAUDE.md\",\"GEMINI.md\"],\"required_labels\":[\"agentic-workflows\"],\"target\":\"*\",\"title_prefix\":\"[ci-fix] \"},\"report_incomplete\":{},\"update_pull_request\":{\"allow_body\":true,\"allow_title\":false,\"max\":3,\"target\":\"*\",\"update_branch\":false}}" GH_AW_CI_TRIGGER_TOKEN: ${{ secrets.GH_AW_CI_TRIGGER_TOKEN }} with: github-token: ${{ secrets.GH_AW_GITHUB_TOKEN || secrets.GITHUB_TOKEN }} diff --git a/.github/workflows/ci-status-fix.md b/.github/workflows/ci-status-fix.md index d3227e25170c..9c2f46054029 100644 --- a/.github/workflows/ci-status-fix.md +++ b/.github/workflows/ci-status-fix.md @@ -15,7 +15,9 @@ description: | humans (the open tracking issue is the hand-off surface; a dedicated [ci-fix][needs-human] PR is planned but currently deferred — see Step 6). CI on the PR is kicked by a human `/azp run` for now; the loop watches, classifies, - and re-fixes autonomously between kicks. + and re-fixes autonomously between kicks. When the SPECIFIC test a PR fixed is + confirmed green in that PR's own CI, the loop marks the draft PR ready for review + (a state transition only — it never approves or merges). Never mutes tests, but de-flakes genuinely flaky ones (deterministic synchronization, no retries / timeout bumps). Always skips visual-regression / screenshot issues. @@ -239,8 +241,15 @@ safe-outputs: # PER-RUN total (not per-PR): a sweep may surface several green PRs and/or # annotate several flaky reds in one run. At max:1 all but the first comment # were silently dropped, starving the workflow's #1 value (surfacing green PRs - # for review). Raised to 3 to match the per-run throughput of the other outputs. - max: 3 + # for review). Sized to 6 (not 3) because a single green DRAFT PR that gets + # flipped ready in one sweep spends TWO comment slots — the Step 3 ✅ surface + # comment (which still names the /azp-gated legs a human must kick, so it is NOT + # redundant with 🎯) AND the Step 3.6 T3 🎯 readiness comment. At max:3 the shared + # bucket drained after ~1 draft flip and T3's atomicity pre-check then deferred + # every further mark-ready even while the mark-ready/add_labels buckets (also 3) + # sat idle; 6 lets ~3 draft flips (2 comments each) land per sweep, so the + # mark-ready:3 / add_labels:3 caps are actually reachable. + max: 6 target: "*" # Hard constraint (defense-in-depth): only comment on THIS workflow's own # ci-fix PRs — [ci-fix] title prefix AND agentic-workflows label. Mirrors the @@ -261,13 +270,51 @@ safe-outputs: # never the title, so disable title rewrites — this compiles to allow_title:false # and removes any ability to retitle an arbitrary PR. title: false - # NOTE: gh-aw v0.79.8 does NOT emit required-title-prefix/required-labels into + # NOTE: gh-aw v0.80.9 (the pinned compiler) does NOT emit required-title-prefix/required-labels into # the compiled config for update-pull-request (verified against the lock — it # silently drops them, unlike add-comment / push-to-pull-request-branch which # honor them). So which-PR scoping here relies on prompt Hard-Rule 6 + # min-integrity:approved; the residual is body-marker edits only (no code, no # merge, no close), capped at max:3. Revisit required-* if a future gh-aw # compiler emits them for this output. + mark-pull-request-as-ready-for-review: + # Flip a validated draft [ci-fix] PR to ready-for-review once the SPECIFIC test + # this PR was opened to fix is confirmed green in the PR's OWN CI (Step 3.6) — + # even when unrelated legs are red. The workflow is schedule/dispatch-triggered + # (no triggering-PR context), so target "*" lets the agent name the PR number it + # validated. This is a state transition ONLY: it never approves and never merges — + # a human still reviews and merges. + # PER-RUN total (not per-PR): a sweep may validate several PRs' target tests green. + max: 3 + target: "*" + # Hard constraint (defense-in-depth): only ever un-draft THIS workflow's own + # ci-fix PRs — [ci-fix] title prefix AND agentic-workflows label — so a confused + # or prompt-injected agent cannot mark an arbitrary PR ready. (If the v0.80.9 + # compiler silently drops these — as documented for update-pull-request above — + # the Step 3.6 preconditions + min-integrity:approved are the compensating scope + # controls; verify against the lock after compiling.) + required-title-prefix: "[ci-fix] " + required-labels: [agentic-workflows] + add-labels: + # Apply the p/0 priority label to a [ci-fix] PR at the exact moment the loop flips it + # from draft to ready-for-review (Step 3.6 T3) — so a validated, review-ready fix lands + # in the team's p/0 triage queue instead of sitting unseen in the draft backlog. Paired + # 1:1 with mark-pull-request-as-ready-for-review; target "*" lets the agent name the PR + # it just validated. + # allowed = HARD allowlist: the agent may ONLY ever add p/0, nothing else. This caps the + # blast radius of a confused/prompt-injected agent to exactly one benign priority label — + # it can never apply a downstream-triggering or destructive label. + allowed: [p/0] + # PER-RUN total (not per-PR): a sweep may mark several PRs ready in one run; each adds + # one label, so this matches the mark-ready per-run cap (3). + max: 3 + target: "*" + # Hard constraint (defense-in-depth): only ever label THIS workflow's own ci-fix PRs — + # [ci-fix] title prefix AND agentic-workflows label. (If the v0.80.9 compiler drops these + # — as documented for update-pull-request above — the Step 3.6 preconditions + + # min-integrity:approved + the allowed:[p/0] allowlist are the compensating controls.) + required-title-prefix: "[ci-fix] " + required-labels: [agentic-workflows] timeout-minutes: 90 @@ -339,33 +386,48 @@ through `safe-outputs`. 5. **One issue = one outcome per run.** Exactly one of: respond to a maintainer's change-request on the open PR (Track C, Step 3.5.R); open the first fix/help/ de-flake PR; advance an existing PR by one attempt (push a follow-up fix); - surface a validated-green PR for review; annotate an unrelated-flake red; wait + surface a validated-green PR for review; mark a target-validated draft PR + ready-for-review (Step 3.6 T3 — the terminal outcome that supersedes the same + run's surface/annotate precursor line), or record its `already-ready` + steady-state no-op; annotate an unrelated-flake red; wait (CI not yet settled); or a recorded skip (the dedicated needs-human PR is deferred — the attempt cap records a skip instead; see Step 6). Always prefer advancing or opening a PR over a skip when a non-mute diff is producible. 6. **All writes via `safe-outputs`.** Allowed outputs: `create_pull_request` (first attempt only), `push_to_pull_request_branch` (advance an existing PR by one attempt), `update_pull_request` (bump the attempt marker / refresh the - prior-attempts table), and `add_comment` (progress notes on the `[ci-fix]` PR - ONLY). NEVER comment on the tracking issue (issues are locked by + prior-attempts table), `add_comment` (progress notes on the `[ci-fix]` PR + ONLY), `mark_pull_request_as_ready_for_review` (flip a target-validated draft + `[ci-fix]` PR from draft to ready — Step 3.6 T3 only), and `add_labels` (add + ONLY the `p/0` label, ONLY on that same draft→ready transition — Step 3.6 T3). + NEVER comment on the tracking issue (issues are locked by `.github/workflows/ci-scan-lock-issues.yml`) — `add_comment` targets the PR. No `gh pr create`, no manual `git push`. **Defense-in-depth:** `add_comment` and `update_pull_request` use `target: "*"` (the agent supplies the PR number). - `add_comment` is now config-locked to `required-title-prefix: "[ci-fix] "` + + `add_comment`, `mark_pull_request_as_ready_for_review`, and `add_labels` are + config-locked to `required-title-prefix: "[ci-fix] "` + `required-labels: [agentic-workflows]` (hard-enforced by the handler, same as - `push_to_pull_request_branch`), so a comment can only ever land on THIS - workflow's own PRs. `update_pull_request` CANNOT be config-locked to a - title/label in gh-aw v0.79.8 (the compiler silently drops `required-*` for that - output — verified against the lock), so it keeps `title: false` (no retitles; - body-marker edits only) plus this prompt-level guard. Before emitting either, - VERIFY the target PR carries BOTH the `[ci-fix]` title prefix AND the - `agentic-workflows` label; never comment on or edit the body of any PR that - lacks both. + `push_to_pull_request_branch`), and `add_labels` additionally has an + `allowed: [p/0]` allowlist so it can ONLY ever add `p/0` — so these can only + ever land on THIS workflow's own PRs. `update_pull_request` CANNOT be + config-locked to a title/label in gh-aw v0.80.9 (the compiler silently drops + `required-*` for that output — verified against the lock), so it keeps + `title: false` (no retitles; body-marker edits only) plus this prompt-level + guard. Before emitting ANY of these, VERIFY the target PR carries BOTH the + `[ci-fix]` title prefix AND the `agentic-workflows` label; never comment on, + edit, un-draft, or label any PR that lacks both. 7. **Per-run safe-output caps (per RUN, not per PR).** Each output type is capped per run: `create_pull_request` 3, `push_to_pull_request_branch` 3, - `add_comment` 3, `update_pull_request` 3. Note an ADVANCE spends one push **and** - one update **and** one comment, so ≤ 3 advances/run; surfacing a green or - annotating a flake spends one comment. When a bucket is exhausted, do NOT keep + `add_comment` 6, `update_pull_request` 3, `mark_pull_request_as_ready_for_review` + 3, `add_labels` 3 (counts LABELS, not calls). Note an ADVANCE spends one push + **and** one update **and** one comment, so ≤ 3 advances/run; surfacing a green or + annotating a flake spends one comment; a **mark-ready (Step 3.6 T3)** of a + still-draft PR spends TWO comments (the Step 3 ✅ surface **and** the T3 🎯 + readiness note) **and** one mark-ready **and** one label as an ALL-OR-NOTHING set + (T3 pre-checks those buckets and defers the whole PR if any is exhausted — never + mark a PR ready without its 🎯 audit comment). `add_comment` is therefore sized 6 + (not 3) so ~3 draft flips, at 2 comments each, can land in one sweep instead of the + shared comment bucket starving the otherwise-idle mark-ready/add_labels buckets. When a bucket is exhausted, do NOT keep emitting (extras are silently dropped) — record `skipped: per-run cap reached; deferring PR # to next cycle` for each remaining PR so the drop is a deliberate, logged decision. The continuous loop picks the deferred PRs up next @@ -409,8 +471,9 @@ For every open tracking issue in scope, converge on exactly one outcome: | Open first help-wanted draft `[ci-fix]` PR | No open PR yet; a plausible candidate exists but cannot be runner-validated (device/UI tests) (attempt 1) | | Open first de-flake draft `[ci-fix]` PR | No open PR yet; failure is intermittent (green-on-retry) from a genuine test-quality defect; a deterministic-synchronization fix is producible (attempt 1, Step 4.7 bucket b) | | Advance an existing PR (push attempt N+1) | Open PR's own CI settled red *because of the fix itself*, no human engaged, marker < 10 — push a NEW distinct fix onto the same branch (Step 3.5 → 5.6) | -| Surface a validated-green PR | Open PR's own CI settled green — comment "validated, attempt N/10; ready for review" and do NOT advance (Step 3.5) | -| Annotate an unrelated-flake red | Open PR is red only on baseline-flake / unrelated legs — comment which leg needs a re-run; do NOT burn an attempt (Step 3.5) | +| Surface a validated-green PR | Open PR's own CI settled green — comment "validated, attempt N/10; ready for review" and do NOT advance (Step 3.5), then mark the draft PR ready for review once the fixed test is confirmed green (Step 3.6) | +| Annotate an unrelated-flake red | Open PR is red only on baseline-flake / unrelated legs — comment which leg needs a re-run; do NOT burn an attempt (Step 3.5); if the SPECIFIC fixed test is confirmed green on that build, still mark the draft PR ready for review (Step 3.6) | +| Mark a validated PR ready for review | The specific test this PR fixed is confirmed green in the PR's own CI (even if unrelated legs are red) — comment the target-test result and transition the draft PR to ready for review; never approve or merge (Step 3.6) | | Wait | Open PR's CI is pending / not yet settled on the current head SHA — do nothing this cycle (Step 3.5) | | Hand-off skip (attempt cap) | Marker == 10 and the signature still reproduces — stop and defer to humans; the dedicated `[ci-fix][needs-human]` PR is planned but currently deferred (Step 6) | | Recorded skip | Visual-regression, human engaged, already-handled, fixed-in-latest-build, infra-flake, out-of-bounds, only-mute-available, or no novel approach producible | @@ -685,14 +748,34 @@ Run these gates in order — the FIRST that fires decides this cycle's outcome: 1. **Human engaged.** If `C.humanEngaged` → `skipped: human engaged on PR #; deferring` and stop. Never fight a human reviewer — the intentional hand-off boundary still holds the moment a person touches the PR. -2. **CI not settled / unknown / incomplete prefetch.** If `C.checksSettled == false` - OR `C.overallConclusion` is `pending`, `neutral`, or `unknown`, OR - `C.dataComplete == false` → the fix's CI has not finished on the current head SHA - (`C.headSha`), or the prefetch could not fully read this PR (a partial read can - understate `humanEngaged` / `attempt`). `skipped: PR # CI pending / prefetch - incomplete on ; waiting` and stop. *(Round 1: this is where a - maintainer `/azp run` is awaited — the `/azp`-gated uitests/devicetests legs will - not have run until a human kicks them.)* +2. **CI not settled — but validate the target test first (target-focused readiness).** + If `C.checksSettled == false` OR `C.overallConclusion` is `pending`, `neutral`, or + `unknown`, OR `C.dataComplete == false` → the *overall* build has not finished on the + current head SHA (`C.headSha`), or the prefetch could not fully read this PR (a partial + read can understate `humanEngaged` / `attempt`). Unrelated legs still draining must NOT + keep an already-proven fix parked as a draft, so before waiting, try the target-focused + fast-path: + - **2a — Target-green fast-path (draft `[ci-fix]` PRs only).** If `C.dataComplete == + true` AND the Step 3.6 preconditions hold (draft PR, `[ci-fix] ` title prefix AND the + `agentic-workflows` label), run Step 3.6 **T1–T2** against `C.headSha` now: identify + the target test(s) and read the PR's OWN build **timeline / per-leg status** (anonymous + `_apis/build` — never `_apis/test`, per Hard-Rule 8). If EVERY target test is + **VALIDATED-GREEN** (per T2's all-platform definition below — the target's category leg + is `succeeded` on every platform that runs it, failed on none, and NO target platform + family the pipeline covers is still pending/unconcluded, per T2's "no platform left + unverified" rule; a platform that simply has no leg for the target's category — the test + doesn't run there — is fine, not a gap) AND every leg in + `C.failedLegs` that has ALREADY concluded classifies as **unrelated flake** (Step 4 / + Step 4.7 method — the fix is implicated in NO completed red), then the fix is proven + regardless of unrelated *pending* legs → run **Step 3.6 T3** (mark ready + 🎯 comment) + and stop. This early-out NEVER advances an attempt and NEVER acts on a red that could + be the fix's fault. + - **2b — Otherwise WAIT.** If the target test has not yet executed (pending/absent on + its leg), or a completed red is (or may be) caused by the fix, or `C.dataComplete == + false`, or this PR has no identifiable target test → `skipped: PR # CI pending / + target not yet validated on ; waiting` and stop. *(Round 1: this is where a + maintainer `/azp run` is awaited — the `/azp`-gated uitests/devicetests legs will not + have run until a human kicks them.)* 3. **Green → surface for review.** If `C.overallConclusion == "success"`: the checks that RAN are green. **Primary-gate check first:** the deterministic `success` verdict only certifies "at least one green check, nothing failing or @@ -704,18 +787,31 @@ Run these gates in order — the FIRST that fires decides this cycle's outcome: # primary CI gate (maui-pr) not green on ; waiting` and stop — do NOT surface. (The `/azp`-gated `maui-pr-uitests` (def 313) / `maui-pr-devicetests` (def 314) legs MAY still be un-run — that is expected and is named below, not a - reason to withhold the surface.) **Idempotency:** scan the PR's existing comments - for a prior bot `✅ … validated … on ` note for THIS head SHA — if one - already exists, record `already-surfaced PR # (head )` and stop - WITHOUT re-commenting (re-surfacing the same green every tick is noise and burns - the per-run comment budget other PRs need). Otherwise resolve the attempt number from - `C.effectiveAttempt` (the authoritative max(marker, bot-commit) counter; omit the - number only if it is somehow indeterminate). **Dry-run gate (Step 0):** if `dry_run == "true"`, do NOT emit any comment — instead print the intended `✅` validated-green body to the run log, tally `dry-run: would-surface-green PR #`, and stop. Otherwise `add_comment` on PR #: `✅ Attempt /10 validated - — the fix's CI is green on . Ready for human review.` - Name any `/azp`-gated legs (uitests def 313 / devicetests def 314) that have not - run and still need a maintainer `/azp run`. Do NOT advance. Record - `surfaced-green PR # (attempt /10)` and stop. *(This directly - attacks the real bottleneck — no reviews — so it is the highest-value outcome.)* + reason to withhold the surface.) **Comment idempotency + dry-run suppress the ✅ + comment ONLY — neither skips Step 3.6.** Scan the PR's existing comments for a prior + bot `✅ … validated … on ` note for THIS head SHA; if one exists, set + `SKIP_SURFACE_COMMENT = true`. Resolve the attempt number from `C.effectiveAttempt` + (the authoritative max(marker, bot-commit) counter; omit the number only if it is + somehow indeterminate). Then post the surface comment UNLESS suppressed: + - if `dry_run == "true"`: do NOT emit — print the intended `✅` validated-green body to + the run log and tally `dry-run: would-surface-green PR #`; + - else if `SKIP_SURFACE_COMMENT`: do NOT re-post — record `already-surfaced PR # + (head )` (re-surfacing the same green every tick is noise and burns the + per-run comment budget other PRs need); + - else `add_comment` on PR #: `✅ Attempt /10 validated — the fix's CI is + green on .` naming any `/azp`-gated legs (uitests def 313 / devicetests def + 314) that have not run and still need a maintainer `/azp run`; record `surfaced-green + PR # (attempt /10)`. (Do NOT assert "ready for human review" on a PR that + is still a draft — Step 3.6, not this comment, owns the draft→ready flip and posts its + own 🎯 announcement when it fires.) + Do NOT advance. Then — in ALL of the above cases — run **Step 3.6** (target-test + readiness gate) for this PR before stopping: its `/azp`-gated target legs may conclude + green on this SAME head SHA on a LATER sweep (an `/azp run` adds no commit), and Step + 3.6 is the ONLY place the draft→ready flip happens, so it MUST re-evaluate every sweep — + never `stop` here before it. *(This directly attacks the real bottleneck — no reviews — + so it is the highest-value outcome.)* 4. **Red → classify caused-by-fix vs unrelated-flake.** If `C.overallConclusion == "failure"`, analyze the PR's OWN failing build (NOT `main`): find the AzDO `maui-pr` build for this PR (filter builds by `branchName=refs/pull//merge`, @@ -723,18 +819,26 @@ Run these gates in order — the FIRST that fires decides this cycle's outcome: method and Step 4.7 flake buckets to `C.failedLegs`. - **Unrelated flake only** (every failed leg is known-flaky / infra / a pre-existing baseline red NOT introduced by the fix): do NOT burn an attempt. - **Idempotency first:** scan the PR's existing comments for a prior bot - `♻️ … unrelated flake … on ` note for THIS head SHA — if one already - exists, record `already-annotated-flake PR # (head )` and stop - WITHOUT re-commenting (re-annotating the same flake every 12h sweep is noise and - burns the per-run comment budget other PRs need). Otherwise resolve the attempt - number from `C.effectiveAttempt` (the authoritative max(marker, bot-commit) - counter), omitting the `Attempt /10:` prefix only if it is somehow - indeterminate. **Dry-run gate (Step 0):** if `dry_run == "true"`, do NOT emit any comment — instead print the intended `♻️` unrelated-flake body to the run log, tally `dry-run: would-annotate-flake PR #`, and stop. Otherwise `add_comment` on PR #: `♻️ Attempt /10: red is - unrelated flake on leg(s) () on ; the fix itself is not - implicated. A maintainer re-run (/azp run ) should clear it.` Record - `annotated-flake PR # (head )` and stop. *(Round 1: human re-runs; - Round 2: auto re-trigger.)* + **Comment idempotency + dry-run suppress the ♻️ comment ONLY — neither skips Step + 3.6.** Scan the PR's existing comments for a prior bot `♻️ … unrelated flake … on + ` note for THIS head SHA; if one exists, set `SKIP_FLAKE_COMMENT = true`. + Resolve the attempt number from `C.effectiveAttempt` (the authoritative max(marker, + bot-commit) counter), omitting the `Attempt /10:` prefix only if it is + somehow indeterminate. Then post the flake note UNLESS suppressed: + - if `dry_run == "true"`: do NOT emit — print the intended `♻️` unrelated-flake body + to the run log and tally `dry-run: would-annotate-flake PR #`; + - else if `SKIP_FLAKE_COMMENT`: do NOT re-post — record `already-annotated-flake PR + # (head )` (re-annotating the same flake every 12h sweep is noise and + burns the per-run comment budget other PRs need); + - else `add_comment` on PR #: `♻️ Attempt /10: red is unrelated flake on + leg(s) () on ; the fix itself is not implicated. A + maintainer re-run (/azp run ) should clear it.` record `annotated-flake + PR # (head )`. + Then — in ALL of the above cases — run **Step 3.6** (target-test readiness gate) for + this PR before stopping: its `/azp`-gated target legs may conclude green on this SAME + head SHA on a LATER sweep, and Step 3.6 is the ONLY place the draft→ready flip + happens, so it MUST re-evaluate every sweep — never `stop` here before it. *(Round 1: + human re-runs; Round 2: auto re-trigger.)* - **Caused by the fix** (a failed leg still matches the original target signature, or the fix introduced a NEW failure): advance an attempt. - **Attempt count.** `attempt = C.effectiveAttempt` — the authoritative @@ -752,6 +856,188 @@ Run these gates in order — the FIRST that fires decides this cycle's outcome: from every prior commit on this branch**, emitted via the Step 5.6 ADVANCE path (push onto the same PR). +#### Step 3.6 — Target-test verification & mark-ready gate + +Reached from the Step 3 **green-surface** branch, the Step 4 **unrelated-flake** branch +(AFTER that branch has posted its comment), and the Step 3.5 **gate-2 target-focused +fast-path** (2a — while unrelated legs are still pending). Purpose: confirm the SPECIFIC +test(s) this PR was opened to fix now PASS in the PR's own CI, and — only when they do +— transition the draft `[ci-fix]` PR to **ready for review** so a maintainer sees a +validated fix instead of a draft. This is the ONLY place the loop flips draft→ready; it +is a state transition, **never** an approval or a merge (a human still reviews and +merges). Overall red on *unrelated* legs must NOT gate readiness — we validate the fix, +not the base branch's flakiness. + +**Preconditions** (ALL must hold; otherwise record `skipped: readiness N/A PR # +()` and stop this gate): +- `C.isDraft == true` — if the PR is already ready-for-review, the draft→ready + transition is already done; do NOT re-mark **and do NOT re-apply `p/0`**. `p/0` is applied + exactly ONCE, atomically with the draft→ready flip (T3 — all-or-nothing with the 🎯 audit + comment + mark-ready), so an already-ready loop PR that is MISSING `p/0` has almost + certainly had it **removed by a maintainer** de-prioritizing the PR — a pure triage action + the human-engagement guard (which inspects comments/reviews/commits, NOT label changes) + cannot see. Re-adding `p/0` every sweep would fight that maintainer indefinitely, violating + the loop's "never override a human" contract. So the loop does NOT reconcile the label: + record `already-ready PR #` and stop this gate. (Trade-off: on the rare occasion a + transient API error drops `p/0` at flip time *after* the T3 atomic pre-check passed, it is + not auto-re-added — but the PR is still ready-for-review with its 🎯 audit comment, and + re-adding `p/0` is a trivial manual action; that is strictly preferable to steam-rolling a + maintainer's deliberate de-prioritization.) +- The PR is unmistakably THIS workflow's own: `[ci-fix] ` title prefix AND the + `agentic-workflows` label (mirrors the safe-output lock; the compensating scope + control if the v0.80.9 compiler drops the declarative required-*). +- You reached this gate from Step 3 (green), Step 4 (**unrelated-flake**), or the Step 3.5 + gate-2 **target-focused fast-path** (2a) — i.e. the fix is NOT implicated in any red that + has concluded. If Step 4 classified a red as **caused by the fix** (ADVANCE), do NOT run + this gate — advance the attempt instead. + +**T1 — Identify the target test(s).** From the `[ci-scan]` issue signature the fix +addresses (and the PR's own diff), extract the fully-qualified test method name(s) the +fix targets — e.g. `SafeAreaShouldWorkOnAllShellTabs`. For a de-flake it is the +de-flaked test; for a product fix it is the originally-failing test(s). If NO specific +test can be identified (e.g. a product build-break rather than a test failure), this is a +**build-only fix** — there is no single test to validate cross-platform, so validate at +**whole-build** granularity rather than stopping (Step 3 only *comments*; Step 3.6 is the +sole draft→ready flip, so a build-only fix is undrafted HERE or not at all). We only reach +this gate from Step 3 (green) / Step 4 (unrelated-flake) / gate-2a, so the fix is not +implicated in any concluded red; additionally require, via the T2 timeline method on +`C.headSha`, that the primary build pipeline `maui-pr` (def 302) has CONCLUDED with EVERY +one of its platform build legs `succeeded`/`completed` and NONE failed/canceled or still +pending — the build is green on **every** platform, not just the originally-broken one (the +cross-platform guard, applied to the build instead of a test). A build-only fix is undrafted +ONLY on the strength of the auto-running `maui-pr` (def 302) whole-build green, which the +loop CAN observe: if the PR's `[ci-scan]` issue signature or its own diff indicates the +ORIGINATING failure was in a `/azp`-gated pipeline (`maui-pr-uitests` def 313 or +`maui-pr-devicetests` def 314) rather than `maui-pr` (def 302), do NOT undraft on +`maui-pr`-green alone — those pipelines do not auto-run on this PR (GITHUB_TOKEN cannot +trigger them), so a green `maui-pr` build is NOT evidence the gated build break is fixed; +record `skipped: build-only fix PR # targets gated pipeline () not run — +deferring to human` and stop this gate. Otherwise, set `TARGET := "the maui-pr build +(build-only fix — no single target test)"` and proceed to **T3** to mark ready. If any `maui-pr` build leg is still unconcluded, record `skipped: build-only fix PR +# not yet whole-build green (leg(s) pending)` and stop this gate WITHOUT marking +ready (a green subset is not enough — a still-pending build leg could yet fail). + +**T2 — Verify the target test's platform legs are green (anonymous, leg-level).** Per-test +outcomes come from the AzDO **test-results** API (`_apis/test/...`), which is **NOT reachable +anonymously — Hard-Rule 8**: those endpoints 302-redirect to a sign-in page on this runner, so +a `curl` returns an HTML redirect (not JSON) and every target test would look "not executed". +Validate instead at **leg granularity** using ONLY the anonymous `_apis/build/...` timeline +(Hard-Rule 8) — a `succeeded` category leg means every test in that category **that actually +ran** passed on that platform. This is a strong bar **only if the target test genuinely +executes**: a job can go green while one specific test is skipped at runtime (`[Ignore]`, +`Assert.Ignore()`, `Assert.Inconclusive()`, a `Skip=`/conditional `[Fact]`, an `#if`-out, or +an early `return` before the asserts). So before trusting a green leg as proof the target +passed, **confirm from the PR's OWN diff that the fix does NOT skip, ignore, disable, or +short-circuit the target test** (it adds no `[Ignore]`/`[Explicit]`, `Assert.Ignore`/ +`Assert.Inconclusive`, `Skip=`, category-exclusion, `#if`-out, or early `return` around the +target). If the fix could cause the target to be runtime-skipped rather than genuinely pass, +leg-level green is NOT sufficient evidence — record `skipped: target test may be +runtime-skipped by this fix (diff adds ); leg-green insufficient, deferring to human +on PR #` and stop this gate WITHOUT marking ready. Otherwise (the target genuinely runs +and asserts, as a de-flake or product fix does) a green category leg cannot hide a +target-test failure, so treat it as authoritative. + +Map the target test to the CI leg(s) that run it. A leg name encodes platform + test category +(e.g. `Android UITests SafeAreaEdges,Shadow`, `iOS UITests SafeAreaEdges,Shadow`, `macOS +UITests SafeAreaEdges,Shadow`, `Windows UITests SafeAreaEdges,Shadow`). Determine the target +test's UI-test category — its `[Category(UITestCategories.X)]` in the test/HostApp source, +visible in the PR diff or the test file — to know which leg-name substring identifies its legs; +for a device test, the per-platform device-test legs. + +Using the SAME build-discovery as Step 4 (filter AzDO builds by `branchName=refs/pull//merge` +or `sourceVersion == C.headSha`), read each build's **timeline** on `C.headSha` for the +pipeline(s) that RUN the target test — `maui-pr` (def 302) for unit/integration tests, +`maui-pr-uitests` (def 313) for Appium UI tests, `maui-pr-devicetests` (def 314) for device +tests: + +```bash +ORG=dnceng-public; PROJ=public; BUILD= +# Anonymous + Hard-Rule-8-compliant. NEVER call _apis/test/... (it 302-redirects to sign-in). +# Each type=="Job" record is one platform leg: result (succeeded/failed/canceled/null), +# state (completed/inProgress/pending), name (encodes " UITests "): +curl -s "https://dev.azure.com/$ORG/$PROJ/_apis/build/builds/$BUILD/timeline?api-version=7.1" \ + | tee /tmp/gh-aw/agent/timeline_${BUILD}.json \ + | jq -r '.records[] | select(.type=="Job") | "\(.result)\t\(.state)\t\(.name)"' +``` + +(The prefetch already exposes per-leg status in `C.failedLegs` / the checks context, and +`gh pr checks ` lists the same per-leg rows with platform+category in the name — use +whichever is handy; the build timeline is the authoritative cross-check on the exact build.) + +Treat a target test as **VALIDATED-GREEN** only if, on `C.headSha`, the leg that runs its +category is green on **every platform it runs on** — a fix that repairs one platform must not +silently regress the same test's leg on another, so a green leg on the originally-red platform +alone is NOT enough. Across the target pipeline's platform legs (for a UI test: the Android, +iOS, Windows, and macOS/MacCatalyst legs running the target's category; for a device test: each +device-test platform), require ALL of: +- the target's category leg is `result == "succeeded"` **and** `state == "completed"` on + **every** platform that runs it, AND +- that leg is `failed` / `canceled` / aborted on **NO** platform, AND +- **no platform is left unverified.** For each platform family the target pipeline covers, that + platform's category leg must have CONCLUDED (`state == "completed"`) on `C.headSha`. If a + platform simply has no leg for the target's category, the test does not run there — that is + fine, not a gap. But if a platform's category leg has **not concluded** (`state` is + `inProgress` / pending, or an `/azp`-gated `maui-pr-uitests` / `maui-pr-devicetests` leg that + has not been kicked), the test's status on that platform is UNKNOWN → the fix is NOT yet + validated across platforms: record `skipped: target test green on but + not yet verified on (leg(s) pending / need /azp run) on PR #` + and stop this gate WITHOUT marking ready. + +A target test whose category leg never concluded on ANY platform (**not executed** anywhere — +e.g. its `/azp`-gated pipeline has not been kicked) is likewise NOT validated: record `skipped: +target test not yet executed on PR # ( not run — needs /azp run)` and stop +this gate WITHOUT marking ready. Do NOT overclaim — a green *sibling* leg (a different category +on the same platform) is not the target's leg, and a green leg on one platform is not a pass on +the others. + +**T3 — Mark ready + report.** If EVERY target test is VALIDATED-GREEN. The 🎯 comment, +the mark-ready, and the `p/0` label are THREE SEPARATE safe-outputs — the comment +existing does NOT prove the mark-ready took effect, so they are tracked independently: + +- **Comment idempotency (dup-suppress only — never gates the mark-ready).** We only reach + T3 while `C.isDraft == true` (the precondition stops an already-ready PR before here). So + if a prior bot comment for THIS head that carries the leading `🎯` anchor AND the + contiguous phrase `validated green on ` already exists (BOTH T3 variants — + `🎯 Target test validated green on ` and the build-only `🎯 Build validated green on + ` — contain that exact contiguous phrase, so this recognizes both; keying on + `target test` alone would instead miss the build-only comment and re-post it every sweep. + **Require the `🎯` anchor** so the match can NEVER be loosened to a gapped + `validated…green…on` form that would false-positive on the Step 3 `✅ … validated — the + fix's CI is green on ` surface comment and wrongly suppress the audit 🎯), the comment + landed on an earlier sweep but the **mark-ready did NOT take + effect** (the PR is still a draft) — set `SUPPRESS_COMMENT = true` (do not re-post the + duplicate 🎯 comment) but STILL complete the mark-ready + label below. Never treat the + comment's existence as "already marked ready" while the PR is still a draft. Record this + case as `re-marking PR # (🎯 present; mark-ready did not take on a prior sweep)`. +- **Atomicity budget pre-check (Hard-Rule 7).** Determine the outputs this gate will emit: + `mark_pull_request_as_ready_for_review` + `add_labels` always, plus `add_comment` unless + `SUPPRESS_COMMENT`. Verify a free per-run slot remains in EACH bucket you are about to + use. If ANY required bucket is exhausted, emit NONE of them — record `skipped: per-run + cap reached; deferring mark-ready PR # to next cycle` and stop this gate. Never mark a + PR ready (or label it) unless its 🎯 audit comment is GUARANTEED to exist for this exact + head SHA — either posted in this same sweep, or (in the `SUPPRESS_COMMENT` re-marking case) + already present from a prior sweep. The outputs you DO emit either all land or all defer + together; the audit trail must never be absent, but it is NOT re-posted when it already exists. +- **Dry-run gate (Step 0):** if `dry_run == "true"`, emit NOTHING — print the intended + readiness comment and "would mark ready + add p/0" to the run log, tally `dry-run: + would-mark-ready PR #`, and stop. +- Otherwise emit for THIS PR number ``: + 1. `add_comment` (ONLY if not `SUPPRESS_COMMENT`): `🎯 Target test validated green on + — passed on ALL platforms it runs on (, + buildId ). — not + caused by this fix."> Transitioning this PR from draft to ready for review and adding + `p/0`; a maintainer still reviews and merges.` (For a **build-only fix** — the T1 + whole-build fallback, no single target test — phrase the first clause as `🎯 Build + validated green on — the maui-pr build passed on ALL platforms (buildId + ).` instead of naming a test.) + 2. `mark_pull_request_as_ready_for_review` with `reason:` a one-line justification + naming the validated test(s) and ``. + 3. `add_labels` with `labels: ["p/0"]` for PR # — put the now-review-ready fix into + the team's p/0 priority queue so it is triaged, not lost in the draft backlog. (If + the PR somehow already carries `p/0`, this is a harmless no-op.) +- Record `marked-ready PR # (target green on ALL platforms on , + labeled p/0)` and stop. + #### Step 3.5.R — Maintainer change-request response (Track C) Reached from Step 3.5 gate 0. Goal: when an **eligible human reviewer** (Hard-Rule @@ -1383,7 +1669,19 @@ Filed by [`ci-status-fix`](https://github.com/dotnet/maui/blob/main/.github/work ### Step 8 — Per-issue tally + end-of-run summary -Per issue, append one outcome line to `/tmp/gh-aw/agent/coverage.txt`: +Per issue, append **one** outcome line to `/tmp/gh-aw/agent/coverage.txt` — the +terminal outcome for the cycle. When one cycle produces a chained pair — a PR gets a +same-cycle precursor line and **then** a Step 3.6 readiness line (`marked-ready` on the +draft→ready flip, or `already-ready` on a later steady-state sweep of an already-ready +PR) — record ONLY the terminal Step 3.6 readiness line: it supersedes ANY same-cycle +non-readiness precursor for that PR, so an aggregator keying on one-line-per-issue never +double-counts. The precursor may be a Step 3 green line (`surfaced-green` on the first ✅, +or `already-surfaced` when it already exists) OR — when an unrelated-flake red and a +target-green coincide — a Step 4 `annotated-flake` line; either way the readiness line is +terminal, and the superseded precursor's signal survives in the PR's 🎯/♻️ comment so no +information is lost. This covers the flip cycle (`surfaced-green` → `marked-ready`), the +post-flip steady state (`already-surfaced` → `already-ready`), and the flake-coincident +flip (`annotated-flake` → `marked-ready`): ``` # main attempt- @@ -1394,8 +1692,14 @@ Per issue, append one outcome line to `/tmp/gh-aw/agent/coverage.txt`: `advance-PR # attempt /10` (ADVANCE mode — new commit pushed to the existing PR), `surfaced-green PR #` (fix's own CI went green; commented for review, did not advance), `annotated-flake PR #` (red was unrelated flake; -commented, attempt NOT burned), `waiting PR #` (CI not settled yet), -`dry-run: would-`, `skipped: `. (The +commented, attempt NOT burned), `marked-ready PR #` (the specific fixed test +was confirmed green in the PR's own CI; draft flipped to ready for review), +`already-surfaced PR #` (the fix's ✅ green comment already existed this sweep; +re-post suppressed — superseded by the Step 3.6 readiness line when the PR also flips +ready that cycle), `already-ready PR #` (terminal steady state — the PR was flipped +ready on a prior cycle and remains green on an unchanged head; no action taken, +supersedes `already-surfaced`), `waiting PR #` (CI not settled yet), +`dry-run: would-`, `skipped: `. (The `needs-human-PR` outcome is reserved for the deferred hand-off — Step 6 currently records a skip instead, so it is not emitted.)
(head )` and stop - WITHOUT re-commenting (re-surfacing the same green every tick is noise and burns - the per-run comment budget other PRs need). Otherwise resolve the attempt number from - `C.effectiveAttempt` (the authoritative max(marker, bot-commit) counter; omit the - number only if it is somehow indeterminate). **Dry-run gate (Step 0):** if `dry_run == "true"`, do NOT emit any comment — instead print the intended `✅` validated-green body to the run log, tally `dry-run: would-surface-green PR #`, and stop. Otherwise `add_comment` on PR #: `✅ Attempt /10 - validated — the fix's CI is green on . Ready for human review.` - Name any `/azp`-gated legs (uitests def 313 / devicetests def 314) that have not - run and still need a maintainer `/azp run`. Do NOT advance. Record - `surfaced-green PR # (attempt /10)` and stop. *(This directly - attacks the real bottleneck — no reviews — so it is the highest-value outcome.)* + reason to withhold the surface.) **Comment idempotency + dry-run suppress the ✅ + comment ONLY — neither skips Step 3.6.** Scan the PR's existing comments for a prior + bot `✅ … validated … on ` note for THIS head SHA; if one exists, set + `SKIP_SURFACE_COMMENT = true`. Resolve the attempt number from `C.effectiveAttempt` + (the authoritative max(marker, bot-commit) counter; omit the number only if it is + somehow indeterminate). Then post the surface comment UNLESS suppressed: + - if `dry_run == "true"`: do NOT emit — print the intended `✅` validated-green body to + the run log and tally `dry-run: would-surface-green PR #`; + - else if `SKIP_SURFACE_COMMENT`: do NOT re-post — record `already-surfaced PR # + (head )` (re-surfacing the same green every tick is noise and burns the + per-run comment budget other PRs need); + - else `add_comment` on PR #: `✅ Attempt /10 validated — the fix's CI is + green on .` naming any `/azp`-gated legs (uitests def 313 / devicetests def + 314) that have not run and still need a maintainer `/azp run`; record `surfaced-green + PR # (attempt /10)`. (Do NOT assert "ready for human review" on a PR that + is still a draft — Step 3.6, not this comment, owns the draft→ready flip and posts its + own 🎯 announcement when it fires.) + Do NOT advance. Then — in ALL of the above cases — run **Step 3.6** (target-test + readiness gate) for this PR before stopping: its `/azp`-gated target legs may conclude + green on this SAME head SHA on a LATER sweep (an `/azp run` adds no commit), and Step + 3.6 is the ONLY place the draft→ready flip happens, so it MUST re-evaluate every sweep — + never `stop` here before it. *(This directly attacks the real bottleneck — no reviews — + so it is the highest-value outcome.)* 4. **Red → classify caused-by-fix vs unrelated-flake.** If `C.overallConclusion == "failure"`, analyze the PR's OWN failing build (NOT `net11.0`): find the AzDO `maui-pr` build for this PR (filter builds by `branchName=refs/pull//merge`, @@ -733,18 +829,26 @@ Run these gates in order — the FIRST that fires decides this cycle's outcome: method and Step 4.7 flake buckets to `C.failedLegs`. - **Unrelated flake only** (every failed leg is known-flaky / infra / a pre-existing baseline red NOT introduced by the fix): do NOT burn an attempt. - **Idempotency first:** scan the PR's existing comments for a prior bot - `♻️ … unrelated flake … on ` note for THIS head SHA — if one already - exists, record `already-annotated-flake PR # (head )` and stop - WITHOUT re-commenting (re-annotating the same flake every 12h sweep is noise and - burns the per-run comment budget other PRs need). Otherwise resolve the attempt - number from `C.effectiveAttempt` (the authoritative max(marker, bot-commit) - counter), omitting the `Attempt /10:` prefix only if it is somehow - indeterminate. **Dry-run gate (Step 0):** if `dry_run == "true"`, do NOT emit any comment — instead print the intended `♻️` unrelated-flake body to the run log, tally `dry-run: would-annotate-flake PR #`, and stop. Otherwise `add_comment` on PR #: `♻️ Attempt /10: red is - unrelated flake on leg(s) () on ; the fix itself is not - implicated. A maintainer re-run (/azp run ) should clear it.` Record - `annotated-flake PR # (head )` and stop. *(Round 1: human re-runs; - Round 2: auto re-trigger.)* + **Comment idempotency + dry-run suppress the ♻️ comment ONLY — neither skips Step + 3.6.** Scan the PR's existing comments for a prior bot `♻️ … unrelated flake … on + ` note for THIS head SHA; if one exists, set `SKIP_FLAKE_COMMENT = true`. + Resolve the attempt number from `C.effectiveAttempt` (the authoritative max(marker, + bot-commit) counter), omitting the `Attempt /10:` prefix only if it is + somehow indeterminate. Then post the flake note UNLESS suppressed: + - if `dry_run == "true"`: do NOT emit — print the intended `♻️` unrelated-flake body + to the run log and tally `dry-run: would-annotate-flake PR #`; + - else if `SKIP_FLAKE_COMMENT`: do NOT re-post — record `already-annotated-flake PR + # (head )` (re-annotating the same flake every 12h sweep is noise and + burns the per-run comment budget other PRs need); + - else `add_comment` on PR #: `♻️ Attempt /10: red is unrelated flake on + leg(s) () on ; the fix itself is not implicated. A + maintainer re-run (/azp run ) should clear it.` record `annotated-flake + PR # (head )`. + Then — in ALL of the above cases — run **Step 3.6** (target-test readiness gate) for + this PR before stopping: its `/azp`-gated target legs may conclude green on this SAME + head SHA on a LATER sweep, and Step 3.6 is the ONLY place the draft→ready flip + happens, so it MUST re-evaluate every sweep — never `stop` here before it. *(Round 1: + human re-runs; Round 2: auto re-trigger.)* - **Caused by the fix** (a failed leg still matches the original target signature, or the fix introduced a NEW failure): advance an attempt. - **Attempt count.** `attempt = C.effectiveAttempt` — the authoritative @@ -762,6 +866,188 @@ Run these gates in order — the FIRST that fires decides this cycle's outcome: from every prior commit on this branch**, emitted via the Step 5.6 ADVANCE path (push onto the same PR). +#### Step 3.6 — Target-test verification & mark-ready gate + +Reached from the Step 3 **green-surface** branch, the Step 4 **unrelated-flake** branch +(AFTER that branch has posted its comment), and the Step 3.5 **gate-2 target-focused +fast-path** (2a — while unrelated legs are still pending). Purpose: confirm the SPECIFIC +test(s) this PR was opened to fix now PASS in the PR's own CI, and — only when they do +— transition the draft `[ci-fix-net11]` PR to **ready for review** so a maintainer sees +a validated fix instead of a draft. This is the ONLY place the loop flips draft→ready; +it is a state transition, **never** an approval or a merge (a human still reviews and +merges). Overall red on *unrelated* legs must NOT gate readiness — we validate the fix, +not the base branch's flakiness. + +**Preconditions** (ALL must hold; otherwise record `skipped: readiness N/A PR # +()` and stop this gate): +- `C.isDraft == true` — if the PR is already ready-for-review, the draft→ready + transition is already done; do NOT re-mark **and do NOT re-apply `p/0`**. `p/0` is applied + exactly ONCE, atomically with the draft→ready flip (T3 — all-or-nothing with the 🎯 audit + comment + mark-ready), so an already-ready loop PR that is MISSING `p/0` has almost + certainly had it **removed by a maintainer** de-prioritizing the PR — a pure triage action + the human-engagement guard (which inspects comments/reviews/commits, NOT label changes) + cannot see. Re-adding `p/0` every sweep would fight that maintainer indefinitely, violating + the loop's "never override a human" contract. So the loop does NOT reconcile the label: + record `already-ready PR #` and stop this gate. (Trade-off: on the rare occasion a + transient API error drops `p/0` at flip time *after* the T3 atomic pre-check passed, it is + not auto-re-added — but the PR is still ready-for-review with its 🎯 audit comment, and + re-adding `p/0` is a trivial manual action; that is strictly preferable to steam-rolling a + maintainer's deliberate de-prioritization.) +- The PR is unmistakably THIS workflow's own: `[ci-fix-net11] ` title prefix AND the + `agentic-workflows` label (mirrors the safe-output lock; the compensating scope + control if the v0.80.9 compiler drops the declarative required-*). +- You reached this gate from Step 3 (green), Step 4 (**unrelated-flake**), or the Step 3.5 + gate-2 **target-focused fast-path** (2a) — i.e. the fix is NOT implicated in any red that + has concluded. If Step 4 classified a red as **caused by the fix** (ADVANCE), do NOT run + this gate — advance the attempt instead. + +**T1 — Identify the target test(s).** From the `[ci-scan-net11]` issue signature the fix +addresses (and the PR's own diff), extract the fully-qualified test method name(s) the +fix targets — e.g. `SafeAreaShouldWorkOnAllShellTabs`. For a de-flake it is the +de-flaked test; for a product fix it is the originally-failing test(s). If NO specific +test can be identified (e.g. a product build-break rather than a test failure), this is a +**build-only fix** — there is no single test to validate cross-platform, so validate at +**whole-build** granularity rather than stopping (Step 3 only *comments*; Step 3.6 is the +sole draft→ready flip, so a build-only fix is undrafted HERE or not at all). We only reach +this gate from Step 3 (green) / Step 4 (unrelated-flake) / gate-2a, so the fix is not +implicated in any concluded red; additionally require, via the T2 timeline method on +`C.headSha`, that the primary build pipeline `maui-pr` (def 302) has CONCLUDED with EVERY +one of its platform build legs `succeeded`/`completed` and NONE failed/canceled or still +pending — the build is green on **every** platform, not just the originally-broken one (the +cross-platform guard, applied to the build instead of a test). A build-only fix is undrafted +ONLY on the strength of the auto-running `maui-pr` (def 302) whole-build green, which the +loop CAN observe: if the PR's `[ci-scan]` issue signature or its own diff indicates the +ORIGINATING failure was in a `/azp`-gated pipeline (`maui-pr-uitests` def 313 or +`maui-pr-devicetests` def 314) rather than `maui-pr` (def 302), do NOT undraft on +`maui-pr`-green alone — those pipelines do not auto-run on this PR (GITHUB_TOKEN cannot +trigger them), so a green `maui-pr` build is NOT evidence the gated build break is fixed; +record `skipped: build-only fix PR # targets gated pipeline () not run — +deferring to human` and stop this gate. Otherwise, set `TARGET := "the maui-pr build +(build-only fix — no single target test)"` and proceed to **T3** to mark ready. If any `maui-pr` build leg is still unconcluded, record `skipped: build-only fix PR +# not yet whole-build green (leg(s) pending)` and stop this gate WITHOUT marking +ready (a green subset is not enough — a still-pending build leg could yet fail). + +**T2 — Verify the target test's platform legs are green (anonymous, leg-level).** Per-test +outcomes come from the AzDO **test-results** API (`_apis/test/...`), which is **NOT reachable +anonymously — Hard-Rule 8**: those endpoints 302-redirect to a sign-in page on this runner, so +a `curl` returns an HTML redirect (not JSON) and every target test would look "not executed". +Validate instead at **leg granularity** using ONLY the anonymous `_apis/build/...` timeline +(Hard-Rule 8) — a `succeeded` category leg means every test in that category **that actually +ran** passed on that platform. This is a strong bar **only if the target test genuinely +executes**: a job can go green while one specific test is skipped at runtime (`[Ignore]`, +`Assert.Ignore()`, `Assert.Inconclusive()`, a `Skip=`/conditional `[Fact]`, an `#if`-out, or +an early `return` before the asserts). So before trusting a green leg as proof the target +passed, **confirm from the PR's OWN diff that the fix does NOT skip, ignore, disable, or +short-circuit the target test** (it adds no `[Ignore]`/`[Explicit]`, `Assert.Ignore`/ +`Assert.Inconclusive`, `Skip=`, category-exclusion, `#if`-out, or early `return` around the +target). If the fix could cause the target to be runtime-skipped rather than genuinely pass, +leg-level green is NOT sufficient evidence — record `skipped: target test may be +runtime-skipped by this fix (diff adds ); leg-green insufficient, deferring to human +on PR #` and stop this gate WITHOUT marking ready. Otherwise (the target genuinely runs +and asserts, as a de-flake or product fix does) a green category leg cannot hide a +target-test failure, so treat it as authoritative. + +Map the target test to the CI leg(s) that run it. A leg name encodes platform + test category +(e.g. `Android UITests SafeAreaEdges,Shadow`, `iOS UITests SafeAreaEdges,Shadow`, `macOS +UITests SafeAreaEdges,Shadow`, `Windows UITests SafeAreaEdges,Shadow`). Determine the target +test's UI-test category — its `[Category(UITestCategories.X)]` in the test/HostApp source, +visible in the PR diff or the test file — to know which leg-name substring identifies its legs; +for a device test, the per-platform device-test legs. + +Using the SAME build-discovery as Step 4 (filter AzDO builds by `branchName=refs/pull//merge` +or `sourceVersion == C.headSha`), read each build's **timeline** on `C.headSha` for the +pipeline(s) that RUN the target test — `maui-pr` (def 302) for unit/integration tests, +`maui-pr-uitests` (def 313) for Appium UI tests, `maui-pr-devicetests` (def 314) for device +tests: + +```bash +ORG=dnceng-public; PROJ=public; BUILD= +# Anonymous + Hard-Rule-8-compliant. NEVER call _apis/test/... (it 302-redirects to sign-in). +# Each type=="Job" record is one platform leg: result (succeeded/failed/canceled/null), +# state (completed/inProgress/pending), name (encodes " UITests "): +curl -s "https://dev.azure.com/$ORG/$PROJ/_apis/build/builds/$BUILD/timeline?api-version=7.1" \ + | tee /tmp/gh-aw/agent/timeline_${BUILD}.json \ + | jq -r '.records[] | select(.type=="Job") | "\(.result)\t\(.state)\t\(.name)"' +``` + +(The prefetch already exposes per-leg status in `C.failedLegs` / the checks context, and +`gh pr checks ` lists the same per-leg rows with platform+category in the name — use +whichever is handy; the build timeline is the authoritative cross-check on the exact build.) + +Treat a target test as **VALIDATED-GREEN** only if, on `C.headSha`, the leg that runs its +category is green on **every platform it runs on** — a fix that repairs one platform must not +silently regress the same test's leg on another, so a green leg on the originally-red platform +alone is NOT enough. Across the target pipeline's platform legs (for a UI test: the Android, +iOS, Windows, and macOS/MacCatalyst legs running the target's category; for a device test: each +device-test platform), require ALL of: +- the target's category leg is `result == "succeeded"` **and** `state == "completed"` on + **every** platform that runs it, AND +- that leg is `failed` / `canceled` / aborted on **NO** platform, AND +- **no platform is left unverified.** For each platform family the target pipeline covers, that + platform's category leg must have CONCLUDED (`state == "completed"`) on `C.headSha`. If a + platform simply has no leg for the target's category, the test does not run there — that is + fine, not a gap. But if a platform's category leg has **not concluded** (`state` is + `inProgress` / pending, or an `/azp`-gated `maui-pr-uitests` / `maui-pr-devicetests` leg that + has not been kicked), the test's status on that platform is UNKNOWN → the fix is NOT yet + validated across platforms: record `skipped: target test green on but + not yet verified on (leg(s) pending / need /azp run) on PR #` + and stop this gate WITHOUT marking ready. + +A target test whose category leg never concluded on ANY platform (**not executed** anywhere — +e.g. its `/azp`-gated pipeline has not been kicked) is likewise NOT validated: record `skipped: +target test not yet executed on PR # ( not run — needs /azp run)` and stop +this gate WITHOUT marking ready. Do NOT overclaim — a green *sibling* leg (a different category +on the same platform) is not the target's leg, and a green leg on one platform is not a pass on +the others. + +**T3 — Mark ready + report.** If EVERY target test is VALIDATED-GREEN. The 🎯 comment, +the mark-ready, and the `p/0` label are THREE SEPARATE safe-outputs — the comment +existing does NOT prove the mark-ready took effect, so they are tracked independently: + +- **Comment idempotency (dup-suppress only — never gates the mark-ready).** We only reach + T3 while `C.isDraft == true` (the precondition stops an already-ready PR before here). So + if a prior bot comment for THIS head that carries the leading `🎯` anchor AND the + contiguous phrase `validated green on ` already exists (BOTH T3 variants — + `🎯 Target test validated green on ` and the build-only `🎯 Build validated green on + ` — contain that exact contiguous phrase, so this recognizes both; keying on + `target test` alone would instead miss the build-only comment and re-post it every sweep. + **Require the `🎯` anchor** so the match can NEVER be loosened to a gapped + `validated…green…on` form that would false-positive on the Step 3 `✅ … validated — the + fix's CI is green on ` surface comment and wrongly suppress the audit 🎯), the comment + landed on an earlier sweep but the **mark-ready did NOT take + effect** (the PR is still a draft) — set `SUPPRESS_COMMENT = true` (do not re-post the + duplicate 🎯 comment) but STILL complete the mark-ready + label below. Never treat the + comment's existence as "already marked ready" while the PR is still a draft. Record this + case as `re-marking PR # (🎯 present; mark-ready did not take on a prior sweep)`. +- **Atomicity budget pre-check (Hard-Rule 7).** Determine the outputs this gate will emit: + `mark_pull_request_as_ready_for_review` + `add_labels` always, plus `add_comment` unless + `SUPPRESS_COMMENT`. Verify a free per-run slot remains in EACH bucket you are about to + use. If ANY required bucket is exhausted, emit NONE of them — record `skipped: per-run + cap reached; deferring mark-ready PR # to next cycle` and stop this gate. Never mark a + PR ready (or label it) unless its 🎯 audit comment is GUARANTEED to exist for this exact + head SHA — either posted in this same sweep, or (in the `SUPPRESS_COMMENT` re-marking case) + already present from a prior sweep. The outputs you DO emit either all land or all defer + together; the audit trail must never be absent, but it is NOT re-posted when it already exists. +- **Dry-run gate (Step 0):** if `dry_run == "true"`, emit NOTHING — print the intended + readiness comment and "would mark ready + add p/0" to the run log, tally `dry-run: + would-mark-ready PR #`, and stop. +- Otherwise emit for THIS PR number ``: + 1. `add_comment` (ONLY if not `SUPPRESS_COMMENT`): `🎯 Target test validated green on + — passed on ALL platforms it runs on (, + buildId ). — not + caused by this fix."> Transitioning this PR from draft to ready for review and adding + `p/0`; a maintainer still reviews and merges.` (For a **build-only fix** — the T1 + whole-build fallback, no single target test — phrase the first clause as `🎯 Build + validated green on — the maui-pr build passed on ALL platforms (buildId + ).` instead of naming a test.) + 2. `mark_pull_request_as_ready_for_review` with `reason:` a one-line justification + naming the validated test(s) and ``. + 3. `add_labels` with `labels: ["p/0"]` for PR # — put the now-review-ready fix into + the team's p/0 priority queue so it is triaged, not lost in the draft backlog. (If + the PR somehow already carries `p/0`, this is a harmless no-op.) +- Record `marked-ready PR # (target green on ALL platforms on , + labeled p/0)` and stop. + #### Step 3.5.R — Maintainer change-request response (Track C) Reached from Step 3.5 gate 0. Goal: when an **eligible human reviewer** (Hard-Rule @@ -1395,7 +1681,19 @@ Filed by [`ci-status-fix-net11`](https://github.com/dotnet/maui/blob/main/.githu ### Step 8 — Per-issue tally + end-of-run summary -Per issue, append one outcome line to `/tmp/gh-aw/agent/coverage.txt`: +Per issue, append **one** outcome line to `/tmp/gh-aw/agent/coverage.txt` — the +terminal outcome for the cycle. When one cycle produces a chained pair — a PR gets a +same-cycle precursor line and **then** a Step 3.6 readiness line (`marked-ready` on the +draft→ready flip, or `already-ready` on a later steady-state sweep of an already-ready +PR) — record ONLY the terminal Step 3.6 readiness line: it supersedes ANY same-cycle +non-readiness precursor for that PR, so an aggregator keying on one-line-per-issue never +double-counts. The precursor may be a Step 3 green line (`surfaced-green` on the first ✅, +or `already-surfaced` when it already exists) OR — when an unrelated-flake red and a +target-green coincide — a Step 4 `annotated-flake` line; either way the readiness line is +terminal, and the superseded precursor's signal survives in the PR's 🎯/♻️ comment so no +information is lost. This covers the flip cycle (`surfaced-green` → `marked-ready`), the +post-flip steady state (`already-surfaced` → `already-ready`), and the flake-coincident +flip (`annotated-flake` → `marked-ready`): ``` # net11.0 attempt- @@ -1406,8 +1704,14 @@ Per issue, append one outcome line to `/tmp/gh-aw/agent/coverage.txt`: `advance-PR # attempt /10` (ADVANCE mode — new commit pushed to the existing PR), `surfaced-green PR #` (fix's own CI went green; commented for review, did not advance), `annotated-flake PR #` (red was unrelated flake; -commented, attempt NOT burned), `waiting PR #` (CI not settled yet), -`dry-run: would-`, `skipped: `. (The +commented, attempt NOT burned), `marked-ready PR #` (the specific fixed test +was confirmed green in the PR's own CI; draft flipped to ready for review), +`already-surfaced PR #` (the fix's ✅ green comment already existed this sweep; +re-post suppressed — superseded by the Step 3.6 readiness line when the PR also flips +ready that cycle), `already-ready PR #` (terminal steady state — the PR was flipped +ready on a prior cycle and remains green on an unchanged head; no action taken, +supersedes `already-surfaced`), `waiting PR #` (CI not settled yet), +`dry-run: would-`, `skipped: `. (The `needs-human-PR` outcome is reserved for the deferred hand-off — Step 6 currently records a skip instead, so it is not emitted.) diff --git a/.github/workflows/ci-status-fix.lock.yml b/.github/workflows/ci-status-fix.lock.yml index 93dc5969cb0d..65511f2b3259 100644 --- a/.github/workflows/ci-status-fix.lock.yml +++ b/.github/workflows/ci-status-fix.lock.yml @@ -1,4 +1,4 @@ -# gh-aw-metadata: {"schema_version":"v4","frontmatter_hash":"f014161967589ebcaf1ac5ec6f3c88ee5a4388d28b55a07dc36c45b7b2aebab6","body_hash":"86c6b2d4c3df13da14c6a3c8b211381fcc9f97bb1951ff7b80588a2c5338e00c","compiler_version":"v0.80.9","strict":true,"agent_id":"copilot","agent_model":"claude-opus-4.8","engine_versions":{"copilot":"1.0.63"}} +# gh-aw-metadata: {"schema_version":"v4","frontmatter_hash":"b0ebf8a2f179900cfe3638e994b27a43cec52bdbdd2331dc1bd4d65762fa2d6b","body_hash":"1825e2b1f7c7bd88241bf37580655489360f9bebc806edd00837a52ce4ad83a0","compiler_version":"v0.80.9","strict":true,"agent_id":"copilot","agent_model":"claude-opus-4.8","engine_versions":{"copilot":"1.0.63"}} # gh-aw-manifest: {"version":1,"secrets":["COPILOT_PAT_0","COPILOT_PAT_1","COPILOT_PAT_2","COPILOT_PAT_3","COPILOT_PAT_4","COPILOT_PAT_5","COPILOT_PAT_6","COPILOT_PAT_7","COPILOT_PAT_8","COPILOT_PAT_9","GH_AW_CI_TRIGGER_TOKEN","GH_AW_GITHUB_MCP_SERVER_TOKEN","GH_AW_GITHUB_TOKEN","GITHUB_TOKEN"],"actions":[{"repo":"actions/checkout","sha":"34e114876b0b11c390a56381ad16ebd13914f8d5","version":"v4"},{"repo":"actions/checkout","sha":"9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0","version":"v7.0.0"},{"repo":"actions/checkout","sha":"de0fac2e4500dabe0009e67214ff5f5447ce83dd","version":"v6.0.2"},{"repo":"actions/download-artifact","sha":"3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c","version":"v8.0.1"},{"repo":"actions/github-script","sha":"3a2844b7e9c422d3c10d287c895573f7108da1b3","version":"v9.0.0"},{"repo":"actions/setup-node","sha":"48b55a011bda9f5d6aeb4c2d9c7362e8dae4041e","version":"v6.4.0"},{"repo":"actions/upload-artifact","sha":"043fb46d1a93c77aae656e7c1c64a875d1fc6a0a","version":"v7.0.1"},{"repo":"github/gh-aw-actions/setup","sha":"8c7d04ebf1ece56cd381446125da3e0f6896294a","version":"v0.80.9"}],"containers":[{"image":"ghcr.io/github/gh-aw-firewall/agent:0.27.7","digest":"sha256:aae231e4635c8999d039c132f1602d3df850fe9b84a00aa2b5ac981179b5661c","pinned_image":"ghcr.io/github/gh-aw-firewall/agent:0.27.7@sha256:aae231e4635c8999d039c132f1602d3df850fe9b84a00aa2b5ac981179b5661c"},{"image":"ghcr.io/github/gh-aw-firewall/api-proxy:0.27.7","digest":"sha256:009caf2e3d88fa77b64e9a03a95a228fc58db0f1701c6d324b29ba5a3c7c79b6","pinned_image":"ghcr.io/github/gh-aw-firewall/api-proxy:0.27.7@sha256:009caf2e3d88fa77b64e9a03a95a228fc58db0f1701c6d324b29ba5a3c7c79b6"},{"image":"ghcr.io/github/gh-aw-firewall/squid:0.27.7","digest":"sha256:deb1d4e19de62d51cee0508057a596a19315c3423ada4d675cad136dc8037c96","pinned_image":"ghcr.io/github/gh-aw-firewall/squid:0.27.7@sha256:deb1d4e19de62d51cee0508057a596a19315c3423ada4d675cad136dc8037c96"},{"image":"ghcr.io/github/gh-aw-mcpg:v0.3.27","digest":"sha256:fe984bddde4ec05d756d9043edb0a32912e6b7b72f6a121b1082f29221421cc7","pinned_image":"ghcr.io/github/gh-aw-mcpg:v0.3.27@sha256:fe984bddde4ec05d756d9043edb0a32912e6b7b72f6a121b1082f29221421cc7"},{"image":"ghcr.io/github/gh-aw-node","digest":"sha256:529d02eb970b1161aa25c593a9c3df57fdfad5a8add328cb3b6eccef66f3183b","pinned_image":"ghcr.io/github/gh-aw-node@sha256:529d02eb970b1161aa25c593a9c3df57fdfad5a8add328cb3b6eccef66f3183b"},{"image":"ghcr.io/github/github-mcp-server:v1.4.0","digest":"sha256:2afb26356481d1a350e14544a6e160f7f7ec1561a1ea309b823665abf0309036","pinned_image":"ghcr.io/github/github-mcp-server:v1.4.0@sha256:2afb26356481d1a350e14544a6e160f7f7ec1561a1ea309b823665abf0309036"}]} # This file was automatically generated by gh-aw (v0.80.9). DO NOT EDIT. To debug this workflow, load the skill at https://github.com/github/gh-aw/blob/main/debug.md # @@ -37,7 +37,9 @@ # humans (the open tracking issue is the hand-off surface; a dedicated # [ci-fix][needs-human] PR is planned but currently deferred — see Step 6). CI on # the PR is kicked by a human `/azp run` for now; the loop watches, classifies, -# and re-fixes autonomously between kicks. +# and re-fixes autonomously between kicks. When the SPECIFIC test a PR fixed is +# confirmed green in that PR's own CI, the loop marks the draft PR ready for review +# (a state transition only — it never approves or merges). # Never mutes tests, but # de-flakes genuinely flaky ones (deterministic synchronization, no retries / # timeout bumps). Always skips visual-regression / screenshot issues. @@ -273,24 +275,24 @@ jobs: run: | bash "${RUNNER_TEMP}/gh-aw/actions/create_prompt_first.sh" { - cat << 'GH_AW_PROMPT_25cf75111b670167_EOF' + cat << 'GH_AW_PROMPT_2ac1d0b4083428f8_EOF' - GH_AW_PROMPT_25cf75111b670167_EOF + GH_AW_PROMPT_2ac1d0b4083428f8_EOF cat "${RUNNER_TEMP}/gh-aw/prompts/xpia.md" cat "${RUNNER_TEMP}/gh-aw/prompts/temp_folder_prompt.md" cat "${RUNNER_TEMP}/gh-aw/prompts/markdown.md" cat "${RUNNER_TEMP}/gh-aw/prompts/safe_outputs_prompt.md" - cat << 'GH_AW_PROMPT_25cf75111b670167_EOF' + cat << 'GH_AW_PROMPT_2ac1d0b4083428f8_EOF' - Tools: add_comment(max:3), create_pull_request(max:3), update_pull_request(max:3), push_to_pull_request_branch(max:3), missing_tool, missing_data, noop - GH_AW_PROMPT_25cf75111b670167_EOF + Tools: add_comment(max:6), create_pull_request(max:3), update_pull_request(max:3), mark_pull_request_as_ready_for_review(max:3), add_labels(max:3), push_to_pull_request_branch(max:3), missing_tool, missing_data, noop + GH_AW_PROMPT_2ac1d0b4083428f8_EOF cat "${RUNNER_TEMP}/gh-aw/prompts/safe_outputs_create_pull_request.md" cat "${RUNNER_TEMP}/gh-aw/prompts/safe_outputs_push_to_pr_branch.md" - cat << 'GH_AW_PROMPT_25cf75111b670167_EOF' + cat << 'GH_AW_PROMPT_2ac1d0b4083428f8_EOF' - GH_AW_PROMPT_25cf75111b670167_EOF + GH_AW_PROMPT_2ac1d0b4083428f8_EOF cat "${RUNNER_TEMP}/gh-aw/prompts/mcp_cli_tools_prompt.md" - cat << 'GH_AW_PROMPT_25cf75111b670167_EOF' + cat << 'GH_AW_PROMPT_2ac1d0b4083428f8_EOF' The following GitHub context information is available for this workflow: {{#if github.actor}} @@ -332,12 +334,12 @@ jobs: stop immediately and report the limitation rather than spending turns trying to work around it. - GH_AW_PROMPT_25cf75111b670167_EOF + GH_AW_PROMPT_2ac1d0b4083428f8_EOF cat "${RUNNER_TEMP}/gh-aw/prompts/github_mcp_tools_with_safeoutputs_prompt.md" - cat << 'GH_AW_PROMPT_25cf75111b670167_EOF' + cat << 'GH_AW_PROMPT_2ac1d0b4083428f8_EOF' {{#runtime-import .github/workflows/ci-status-fix.md}} - GH_AW_PROMPT_25cf75111b670167_EOF + GH_AW_PROMPT_2ac1d0b4083428f8_EOF } > "$GH_AW_PROMPT" - name: Interpolate variables and render templates uses: actions/github-script@3a2844b7e9c422d3c10d287c895573f7108da1b3 # v9.0.0 @@ -554,16 +556,18 @@ jobs: mkdir -p "${RUNNER_TEMP}/gh-aw/safeoutputs" mkdir -p /tmp/gh-aw/safeoutputs mkdir -p /tmp/gh-aw/mcp-logs/safeoutputs - cat > "${RUNNER_TEMP}/gh-aw/safeoutputs/config.json" << 'GH_AW_SAFE_OUTPUTS_CONFIG_fa2a69fad5fd0485_EOF' - {"add_comment":{"discussions":false,"max":3,"required_labels":["agentic-workflows"],"required_title_prefix":"[ci-fix] ","target":"*"},"create_pull_request":{"allowed_base_branches":["main"],"allowed_branches":["ci-fix/**"],"allowed_files":["src/Core/**","src/Controls/**","src/Essentials/**","src/BlazorWebView/**","src/TestUtils/**","src/Templates/**","**/PublicAPI.Unshipped.txt"],"base_branch":"main","draft":true,"labels":["agentic-workflows"],"max":3,"max_patch_files":100,"max_patch_size":4096,"protect_top_level_dot_folders":true,"protected_files":["package.json","bun.lockb","bunfig.toml","deno.json","deno.jsonc","deno.lock","global.json","NuGet.Config","Directory.Packages.props","mix.exs","mix.lock","go.mod","go.sum","stack.yaml","stack.yaml.lock","pom.xml","build.gradle","build.gradle.kts","settings.gradle","settings.gradle.kts","gradle.properties","package-lock.json","yarn.lock","pnpm-lock.yaml","npm-shrinkwrap.json","requirements.txt","Pipfile","Pipfile.lock","pyproject.toml","setup.py","setup.cfg","Gemfile","Gemfile.lock","uv.lock","CODEOWNERS","DESIGN.md","README.md","CONTRIBUTING.md","CHANGELOG.md","SECURITY.md","CODE_OF_CONDUCT.md","AGENTS.md","CLAUDE.md","GEMINI.md"],"protected_files_policy":"request_review","title_prefix":"[ci-fix] "},"create_report_incomplete_issue":{},"missing_data":{},"missing_tool":{},"noop":{"max":1,"report-as-issue":"true"},"push_to_pull_request_branch":{"allowed_files":["src/Core/**","src/Controls/**","src/Essentials/**","src/BlazorWebView/**","src/TestUtils/**","src/Templates/**","**/PublicAPI.Unshipped.txt"],"if_no_changes":"warn","max":3,"max_patch_size":4096,"protect_top_level_dot_folders":true,"protected_files":["package.json","bun.lockb","bunfig.toml","deno.json","deno.jsonc","deno.lock","global.json","NuGet.Config","Directory.Packages.props","mix.exs","mix.lock","go.mod","go.sum","stack.yaml","stack.yaml.lock","pom.xml","build.gradle","build.gradle.kts","settings.gradle","settings.gradle.kts","gradle.properties","package-lock.json","yarn.lock","pnpm-lock.yaml","npm-shrinkwrap.json","requirements.txt","Pipfile","Pipfile.lock","pyproject.toml","setup.py","setup.cfg","Gemfile","Gemfile.lock","uv.lock","CODEOWNERS","DESIGN.md","README.md","CONTRIBUTING.md","CHANGELOG.md","SECURITY.md","CODE_OF_CONDUCT.md","AGENTS.md","CLAUDE.md","GEMINI.md"],"required_labels":["agentic-workflows"],"target":"*","title_prefix":"[ci-fix] "},"report_incomplete":{},"update_pull_request":{"allow_body":true,"allow_title":false,"max":3,"target":"*","update_branch":false}} - GH_AW_SAFE_OUTPUTS_CONFIG_fa2a69fad5fd0485_EOF + cat > "${RUNNER_TEMP}/gh-aw/safeoutputs/config.json" << 'GH_AW_SAFE_OUTPUTS_CONFIG_0644bf1434ca0071_EOF' + {"add_comment":{"discussions":false,"max":6,"required_labels":["agentic-workflows"],"required_title_prefix":"[ci-fix] ","target":"*"},"add_labels":{"allowed":["p/0"],"max":3,"required_labels":["agentic-workflows"],"required_title_prefix":"[ci-fix] ","target":"*"},"create_pull_request":{"allowed_base_branches":["main"],"allowed_branches":["ci-fix/**"],"allowed_files":["src/Core/**","src/Controls/**","src/Essentials/**","src/BlazorWebView/**","src/TestUtils/**","src/Templates/**","**/PublicAPI.Unshipped.txt"],"base_branch":"main","draft":true,"labels":["agentic-workflows"],"max":3,"max_patch_files":100,"max_patch_size":4096,"protect_top_level_dot_folders":true,"protected_files":["package.json","bun.lockb","bunfig.toml","deno.json","deno.jsonc","deno.lock","global.json","NuGet.Config","Directory.Packages.props","mix.exs","mix.lock","go.mod","go.sum","stack.yaml","stack.yaml.lock","pom.xml","build.gradle","build.gradle.kts","settings.gradle","settings.gradle.kts","gradle.properties","package-lock.json","yarn.lock","pnpm-lock.yaml","npm-shrinkwrap.json","requirements.txt","Pipfile","Pipfile.lock","pyproject.toml","setup.py","setup.cfg","Gemfile","Gemfile.lock","uv.lock","CODEOWNERS","DESIGN.md","README.md","CONTRIBUTING.md","CHANGELOG.md","SECURITY.md","CODE_OF_CONDUCT.md","AGENTS.md","CLAUDE.md","GEMINI.md"],"protected_files_policy":"request_review","title_prefix":"[ci-fix] "},"create_report_incomplete_issue":{},"mark_pull_request_as_ready_for_review":{"max":3,"required_labels":["agentic-workflows"],"required_title_prefix":"[ci-fix] ","target":"*"},"missing_data":{},"missing_tool":{},"noop":{"max":1,"report-as-issue":"true"},"push_to_pull_request_branch":{"allowed_files":["src/Core/**","src/Controls/**","src/Essentials/**","src/BlazorWebView/**","src/TestUtils/**","src/Templates/**","**/PublicAPI.Unshipped.txt"],"if_no_changes":"warn","max":3,"max_patch_size":4096,"protect_top_level_dot_folders":true,"protected_files":["package.json","bun.lockb","bunfig.toml","deno.json","deno.jsonc","deno.lock","global.json","NuGet.Config","Directory.Packages.props","mix.exs","mix.lock","go.mod","go.sum","stack.yaml","stack.yaml.lock","pom.xml","build.gradle","build.gradle.kts","settings.gradle","settings.gradle.kts","gradle.properties","package-lock.json","yarn.lock","pnpm-lock.yaml","npm-shrinkwrap.json","requirements.txt","Pipfile","Pipfile.lock","pyproject.toml","setup.py","setup.cfg","Gemfile","Gemfile.lock","uv.lock","CODEOWNERS","DESIGN.md","README.md","CONTRIBUTING.md","CHANGELOG.md","SECURITY.md","CODE_OF_CONDUCT.md","AGENTS.md","CLAUDE.md","GEMINI.md"],"required_labels":["agentic-workflows"],"target":"*","title_prefix":"[ci-fix] "},"report_incomplete":{},"update_pull_request":{"allow_body":true,"allow_title":false,"max":3,"target":"*","update_branch":false}} + GH_AW_SAFE_OUTPUTS_CONFIG_0644bf1434ca0071_EOF - name: Generate Safe Outputs Tools env: GH_AW_TOOLS_META_JSON: | { "description_suffixes": { - "add_comment": " CONSTRAINTS: Maximum 3 comment(s) can be added. Target: *. Supports reply_to_id for discussion threading.", + "add_comment": " CONSTRAINTS: Maximum 6 comment(s) can be added. Target: *. Supports reply_to_id for discussion threading.", + "add_labels": " CONSTRAINTS: Maximum 3 label(s) can be added. Only these labels are allowed: [\"p/0\"]. Target: *.", "create_pull_request": " CONSTRAINTS: Maximum 3 pull request(s) can be created. Title will be prefixed with \"[ci-fix] \". Labels [\"agentic-workflows\"] will be automatically added. Only these labels are allowed: [\"agentic-workflows\"]. PRs will be created as drafts.", + "mark_pull_request_as_ready_for_review": " CONSTRAINTS: Maximum 3 pull request(s) can be marked as ready for review.", "push_to_pull_request_branch": " CONSTRAINTS: Maximum 3 push(es) can be made. The target pull request title must start with \"[ci-fix] \".", "update_pull_request": " CONSTRAINTS: Maximum 3 pull request(s) can be updated. Target: *." }, @@ -594,6 +598,25 @@ jobs: } } }, + "add_labels": { + "defaultMax": 5, + "fields": { + "item_number": { + "issueNumberOrTemporaryId": true + }, + "labels": { + "required": true, + "type": "array", + "itemType": "string", + "itemSanitize": true, + "itemMaxLength": 128 + }, + "repo": { + "type": "string", + "maxLength": 256 + } + } + }, "create_pull_request": { "defaultMax": 1, "fields": { @@ -635,6 +658,24 @@ jobs: } } }, + "mark_pull_request_as_ready_for_review": { + "defaultMax": 1, + "fields": { + "pull_request_number": { + "issueOrPRNumber": true + }, + "reason": { + "required": true, + "type": "string", + "sanitize": true, + "maxLength": 65000 + }, + "repo": { + "type": "string", + "maxLength": 256 + } + } + }, "missing_data": { "defaultMax": 20, "fields": { @@ -1494,7 +1535,7 @@ jobs: uses: actions/github-script@3a2844b7e9c422d3c10d287c895573f7108da1b3 # v9.0.0 env: WORKFLOW_NAME: "CI Failure Fixer (main)" - WORKFLOW_DESCRIPTION: "Periodic pass over open ci-scan tracking issues filed by the main-branch CI\nfailure scanner (.github/workflows/ci-status-main.md). This workflow targets\nthe `main` branch EXCLUSIVELY: it processes only issues labelled ci-scan and\nopens every PR against main. (The net11.0 branch is handled by the parallel\n.github/workflows/ci-status-fix-net11.md — the two are split because gh-aw can\nonly transport a fix relative to ONE static base branch per workflow, and the\nmain↔net11.0 divergence exceeds gh-aw's 10 MB transport-patch cap.) The fixer\nopens ONE draft [ci-fix] PR per actionable issue against main, then WATCHES\nthat PR's own CI on later runs: when the fix's CI comes back red and the red is\ncaused by the fix itself, it pushes a fresh follow-up fix onto the SAME PR\nbranch (never a second PR) — up to 10 attempts — then stops and defers to\nhumans (the open tracking issue is the hand-off surface; a dedicated\n[ci-fix][needs-human] PR is planned but currently deferred — see Step 6). CI on\nthe PR is kicked by a human `/azp run` for now; the loop watches, classifies,\nand re-fixes autonomously between kicks.\nNever mutes tests, but\nde-flakes genuinely flaky ones (deterministic synchronization, no retries /\ntimeout bumps). Always skips visual-regression / screenshot issues." + WORKFLOW_DESCRIPTION: "Periodic pass over open ci-scan tracking issues filed by the main-branch CI\nfailure scanner (.github/workflows/ci-status-main.md). This workflow targets\nthe `main` branch EXCLUSIVELY: it processes only issues labelled ci-scan and\nopens every PR against main. (The net11.0 branch is handled by the parallel\n.github/workflows/ci-status-fix-net11.md — the two are split because gh-aw can\nonly transport a fix relative to ONE static base branch per workflow, and the\nmain↔net11.0 divergence exceeds gh-aw's 10 MB transport-patch cap.) The fixer\nopens ONE draft [ci-fix] PR per actionable issue against main, then WATCHES\nthat PR's own CI on later runs: when the fix's CI comes back red and the red is\ncaused by the fix itself, it pushes a fresh follow-up fix onto the SAME PR\nbranch (never a second PR) — up to 10 attempts — then stops and defers to\nhumans (the open tracking issue is the hand-off surface; a dedicated\n[ci-fix][needs-human] PR is planned but currently deferred — see Step 6). CI on\nthe PR is kicked by a human `/azp run` for now; the loop watches, classifies,\nand re-fixes autonomously between kicks. When the SPECIFIC test a PR fixed is\nconfirmed green in that PR's own CI, the loop marks the draft PR ready for review\n(a state transition only — it never approves or merges).\nNever mutes tests, but\nde-flakes genuinely flaky ones (deterministic synchronization, no retries /\ntimeout bumps). Always skips visual-regression / screenshot issues." HAS_PATCH: ${{ needs.agent.outputs.has_patch }} with: script: | @@ -1910,7 +1951,7 @@ jobs: GH_AW_ALLOWED_DOMAINS: "*.blob.core.windows.net,*.githubusercontent.com,api.business.githubcopilot.com,api.enterprise.githubcopilot.com,api.github.com,api.githubcopilot.com,api.individual.githubcopilot.com,api.snapcraft.io,archive.ubuntu.com,azure.archive.ubuntu.com,codeload.github.com,crl.geotrust.com,crl.globalsign.com,crl.identrust.com,crl.sectigo.com,crl.thawte.com,crl.usertrust.com,crl.verisign.com,crl3.digicert.com,crl4.digicert.com,crls.ssl.com,dev.azure.com,docs.github.com,github-cloud.githubusercontent.com,github-cloud.s3.amazonaws.com,github.blog,github.com,github.githubassets.com,helix.dot.net,host.docker.internal,json-schema.org,json.schemastore.org,keyserver.ubuntu.com,lfs.github.com,objects.githubusercontent.com,ocsp.digicert.com,ocsp.geotrust.com,ocsp.globalsign.com,ocsp.identrust.com,ocsp.sectigo.com,ocsp.ssl.com,ocsp.thawte.com,ocsp.usertrust.com,ocsp.verisign.com,packagecloud.io,packages.cloud.google.com,packages.microsoft.com,patch-diff.githubusercontent.com,ppa.launchpad.net,raw.githubusercontent.com,registry.npmjs.org,s.symcb.com,s.symcd.com,security.ubuntu.com,telemetry.enterprise.githubcopilot.com,ts-crl.ws.symantec.com,ts-ocsp.ws.symantec.com,www.googleapis.com" GITHUB_SERVER_URL: ${{ github.server_url }} GITHUB_API_URL: ${{ github.api_url }} - GH_AW_SAFE_OUTPUTS_HANDLER_CONFIG: "{\"add_comment\":{\"discussions\":false,\"max\":3,\"required_labels\":[\"agentic-workflows\"],\"required_title_prefix\":\"[ci-fix] \",\"target\":\"*\"},\"create_pull_request\":{\"allowed_base_branches\":[\"main\"],\"allowed_branches\":[\"ci-fix/**\"],\"allowed_files\":[\"src/Core/**\",\"src/Controls/**\",\"src/Essentials/**\",\"src/BlazorWebView/**\",\"src/TestUtils/**\",\"src/Templates/**\",\"**/PublicAPI.Unshipped.txt\"],\"base_branch\":\"main\",\"draft\":true,\"labels\":[\"agentic-workflows\"],\"max\":3,\"max_patch_files\":100,\"max_patch_size\":4096,\"protect_top_level_dot_folders\":true,\"protected_files\":[\"package.json\",\"bun.lockb\",\"bunfig.toml\",\"deno.json\",\"deno.jsonc\",\"deno.lock\",\"global.json\",\"NuGet.Config\",\"Directory.Packages.props\",\"mix.exs\",\"mix.lock\",\"go.mod\",\"go.sum\",\"stack.yaml\",\"stack.yaml.lock\",\"pom.xml\",\"build.gradle\",\"build.gradle.kts\",\"settings.gradle\",\"settings.gradle.kts\",\"gradle.properties\",\"package-lock.json\",\"yarn.lock\",\"pnpm-lock.yaml\",\"npm-shrinkwrap.json\",\"requirements.txt\",\"Pipfile\",\"Pipfile.lock\",\"pyproject.toml\",\"setup.py\",\"setup.cfg\",\"Gemfile\",\"Gemfile.lock\",\"uv.lock\",\"CODEOWNERS\",\"DESIGN.md\",\"README.md\",\"CONTRIBUTING.md\",\"CHANGELOG.md\",\"SECURITY.md\",\"CODE_OF_CONDUCT.md\",\"AGENTS.md\",\"CLAUDE.md\",\"GEMINI.md\"],\"protected_files_policy\":\"request_review\",\"title_prefix\":\"[ci-fix] \"},\"create_report_incomplete_issue\":{},\"missing_data\":{},\"missing_tool\":{},\"noop\":{\"max\":1,\"report-as-issue\":\"true\"},\"push_to_pull_request_branch\":{\"allowed_files\":[\"src/Core/**\",\"src/Controls/**\",\"src/Essentials/**\",\"src/BlazorWebView/**\",\"src/TestUtils/**\",\"src/Templates/**\",\"**/PublicAPI.Unshipped.txt\"],\"if_no_changes\":\"warn\",\"max\":3,\"max_patch_size\":4096,\"protect_top_level_dot_folders\":true,\"protected_files\":[\"package.json\",\"bun.lockb\",\"bunfig.toml\",\"deno.json\",\"deno.jsonc\",\"deno.lock\",\"global.json\",\"NuGet.Config\",\"Directory.Packages.props\",\"mix.exs\",\"mix.lock\",\"go.mod\",\"go.sum\",\"stack.yaml\",\"stack.yaml.lock\",\"pom.xml\",\"build.gradle\",\"build.gradle.kts\",\"settings.gradle\",\"settings.gradle.kts\",\"gradle.properties\",\"package-lock.json\",\"yarn.lock\",\"pnpm-lock.yaml\",\"npm-shrinkwrap.json\",\"requirements.txt\",\"Pipfile\",\"Pipfile.lock\",\"pyproject.toml\",\"setup.py\",\"setup.cfg\",\"Gemfile\",\"Gemfile.lock\",\"uv.lock\",\"CODEOWNERS\",\"DESIGN.md\",\"README.md\",\"CONTRIBUTING.md\",\"CHANGELOG.md\",\"SECURITY.md\",\"CODE_OF_CONDUCT.md\",\"AGENTS.md\",\"CLAUDE.md\",\"GEMINI.md\"],\"required_labels\":[\"agentic-workflows\"],\"target\":\"*\",\"title_prefix\":\"[ci-fix] \"},\"report_incomplete\":{},\"update_pull_request\":{\"allow_body\":true,\"allow_title\":false,\"max\":3,\"target\":\"*\",\"update_branch\":false}}" + GH_AW_SAFE_OUTPUTS_HANDLER_CONFIG: "{\"add_comment\":{\"discussions\":false,\"max\":6,\"required_labels\":[\"agentic-workflows\"],\"required_title_prefix\":\"[ci-fix] \",\"target\":\"*\"},\"add_labels\":{\"allowed\":[\"p/0\"],\"max\":3,\"required_labels\":[\"agentic-workflows\"],\"required_title_prefix\":\"[ci-fix] \",\"target\":\"*\"},\"create_pull_request\":{\"allowed_base_branches\":[\"main\"],\"allowed_branches\":[\"ci-fix/**\"],\"allowed_files\":[\"src/Core/**\",\"src/Controls/**\",\"src/Essentials/**\",\"src/BlazorWebView/**\",\"src/TestUtils/**\",\"src/Templates/**\",\"**/PublicAPI.Unshipped.txt\"],\"base_branch\":\"main\",\"draft\":true,\"labels\":[\"agentic-workflows\"],\"max\":3,\"max_patch_files\":100,\"max_patch_size\":4096,\"protect_top_level_dot_folders\":true,\"protected_files\":[\"package.json\",\"bun.lockb\",\"bunfig.toml\",\"deno.json\",\"deno.jsonc\",\"deno.lock\",\"global.json\",\"NuGet.Config\",\"Directory.Packages.props\",\"mix.exs\",\"mix.lock\",\"go.mod\",\"go.sum\",\"stack.yaml\",\"stack.yaml.lock\",\"pom.xml\",\"build.gradle\",\"build.gradle.kts\",\"settings.gradle\",\"settings.gradle.kts\",\"gradle.properties\",\"package-lock.json\",\"yarn.lock\",\"pnpm-lock.yaml\",\"npm-shrinkwrap.json\",\"requirements.txt\",\"Pipfile\",\"Pipfile.lock\",\"pyproject.toml\",\"setup.py\",\"setup.cfg\",\"Gemfile\",\"Gemfile.lock\",\"uv.lock\",\"CODEOWNERS\",\"DESIGN.md\",\"README.md\",\"CONTRIBUTING.md\",\"CHANGELOG.md\",\"SECURITY.md\",\"CODE_OF_CONDUCT.md\",\"AGENTS.md\",\"CLAUDE.md\",\"GEMINI.md\"],\"protected_files_policy\":\"request_review\",\"title_prefix\":\"[ci-fix] \"},\"create_report_incomplete_issue\":{},\"mark_pull_request_as_ready_for_review\":{\"max\":3,\"required_labels\":[\"agentic-workflows\"],\"required_title_prefix\":\"[ci-fix] \",\"target\":\"*\"},\"missing_data\":{},\"missing_tool\":{},\"noop\":{\"max\":1,\"report-as-issue\":\"true\"},\"push_to_pull_request_branch\":{\"allowed_files\":[\"src/Core/**\",\"src/Controls/**\",\"src/Essentials/**\",\"src/BlazorWebView/**\",\"src/TestUtils/**\",\"src/Templates/**\",\"**/PublicAPI.Unshipped.txt\"],\"if_no_changes\":\"warn\",\"max\":3,\"max_patch_size\":4096,\"protect_top_level_dot_folders\":true,\"protected_files\":[\"package.json\",\"bun.lockb\",\"bunfig.toml\",\"deno.json\",\"deno.jsonc\",\"deno.lock\",\"global.json\",\"NuGet.Config\",\"Directory.Packages.props\",\"mix.exs\",\"mix.lock\",\"go.mod\",\"go.sum\",\"stack.yaml\",\"stack.yaml.lock\",\"pom.xml\",\"build.gradle\",\"build.gradle.kts\",\"settings.gradle\",\"settings.gradle.kts\",\"gradle.properties\",\"package-lock.json\",\"yarn.lock\",\"pnpm-lock.yaml\",\"npm-shrinkwrap.json\",\"requirements.txt\",\"Pipfile\",\"Pipfile.lock\",\"pyproject.toml\",\"setup.py\",\"setup.cfg\",\"Gemfile\",\"Gemfile.lock\",\"uv.lock\",\"CODEOWNERS\",\"DESIGN.md\",\"README.md\",\"CONTRIBUTING.md\",\"CHANGELOG.md\",\"SECURITY.md\",\"CODE_OF_CONDUCT.md\",\"AGENTS.md\",\"CLAUDE.md\",\"GEMINI.md\"],\"required_labels\":[\"agentic-workflows\"],\"target\":\"*\",\"title_prefix\":\"[ci-fix] \"},\"report_incomplete\":{},\"update_pull_request\":{\"allow_body\":true,\"allow_title\":false,\"max\":3,\"target\":\"*\",\"update_branch\":false}}" GH_AW_CI_TRIGGER_TOKEN: ${{ secrets.GH_AW_CI_TRIGGER_TOKEN }} with: github-token: ${{ secrets.GH_AW_GITHUB_TOKEN || secrets.GITHUB_TOKEN }} diff --git a/.github/workflows/ci-status-fix.md b/.github/workflows/ci-status-fix.md index d3227e25170c..9c2f46054029 100644 --- a/.github/workflows/ci-status-fix.md +++ b/.github/workflows/ci-status-fix.md @@ -15,7 +15,9 @@ description: | humans (the open tracking issue is the hand-off surface; a dedicated [ci-fix][needs-human] PR is planned but currently deferred — see Step 6). CI on the PR is kicked by a human `/azp run` for now; the loop watches, classifies, - and re-fixes autonomously between kicks. + and re-fixes autonomously between kicks. When the SPECIFIC test a PR fixed is + confirmed green in that PR's own CI, the loop marks the draft PR ready for review + (a state transition only — it never approves or merges). Never mutes tests, but de-flakes genuinely flaky ones (deterministic synchronization, no retries / timeout bumps). Always skips visual-regression / screenshot issues. @@ -239,8 +241,15 @@ safe-outputs: # PER-RUN total (not per-PR): a sweep may surface several green PRs and/or # annotate several flaky reds in one run. At max:1 all but the first comment # were silently dropped, starving the workflow's #1 value (surfacing green PRs - # for review). Raised to 3 to match the per-run throughput of the other outputs. - max: 3 + # for review). Sized to 6 (not 3) because a single green DRAFT PR that gets + # flipped ready in one sweep spends TWO comment slots — the Step 3 ✅ surface + # comment (which still names the /azp-gated legs a human must kick, so it is NOT + # redundant with 🎯) AND the Step 3.6 T3 🎯 readiness comment. At max:3 the shared + # bucket drained after ~1 draft flip and T3's atomicity pre-check then deferred + # every further mark-ready even while the mark-ready/add_labels buckets (also 3) + # sat idle; 6 lets ~3 draft flips (2 comments each) land per sweep, so the + # mark-ready:3 / add_labels:3 caps are actually reachable. + max: 6 target: "*" # Hard constraint (defense-in-depth): only comment on THIS workflow's own # ci-fix PRs — [ci-fix] title prefix AND agentic-workflows label. Mirrors the @@ -261,13 +270,51 @@ safe-outputs: # never the title, so disable title rewrites — this compiles to allow_title:false # and removes any ability to retitle an arbitrary PR. title: false - # NOTE: gh-aw v0.79.8 does NOT emit required-title-prefix/required-labels into + # NOTE: gh-aw v0.80.9 (the pinned compiler) does NOT emit required-title-prefix/required-labels into # the compiled config for update-pull-request (verified against the lock — it # silently drops them, unlike add-comment / push-to-pull-request-branch which # honor them). So which-PR scoping here relies on prompt Hard-Rule 6 + # min-integrity:approved; the residual is body-marker edits only (no code, no # merge, no close), capped at max:3. Revisit required-* if a future gh-aw # compiler emits them for this output. + mark-pull-request-as-ready-for-review: + # Flip a validated draft [ci-fix] PR to ready-for-review once the SPECIFIC test + # this PR was opened to fix is confirmed green in the PR's OWN CI (Step 3.6) — + # even when unrelated legs are red. The workflow is schedule/dispatch-triggered + # (no triggering-PR context), so target "*" lets the agent name the PR number it + # validated. This is a state transition ONLY: it never approves and never merges — + # a human still reviews and merges. + # PER-RUN total (not per-PR): a sweep may validate several PRs' target tests green. + max: 3 + target: "*" + # Hard constraint (defense-in-depth): only ever un-draft THIS workflow's own + # ci-fix PRs — [ci-fix] title prefix AND agentic-workflows label — so a confused + # or prompt-injected agent cannot mark an arbitrary PR ready. (If the v0.80.9 + # compiler silently drops these — as documented for update-pull-request above — + # the Step 3.6 preconditions + min-integrity:approved are the compensating scope + # controls; verify against the lock after compiling.) + required-title-prefix: "[ci-fix] " + required-labels: [agentic-workflows] + add-labels: + # Apply the p/0 priority label to a [ci-fix] PR at the exact moment the loop flips it + # from draft to ready-for-review (Step 3.6 T3) — so a validated, review-ready fix lands + # in the team's p/0 triage queue instead of sitting unseen in the draft backlog. Paired + # 1:1 with mark-pull-request-as-ready-for-review; target "*" lets the agent name the PR + # it just validated. + # allowed = HARD allowlist: the agent may ONLY ever add p/0, nothing else. This caps the + # blast radius of a confused/prompt-injected agent to exactly one benign priority label — + # it can never apply a downstream-triggering or destructive label. + allowed: [p/0] + # PER-RUN total (not per-PR): a sweep may mark several PRs ready in one run; each adds + # one label, so this matches the mark-ready per-run cap (3). + max: 3 + target: "*" + # Hard constraint (defense-in-depth): only ever label THIS workflow's own ci-fix PRs — + # [ci-fix] title prefix AND agentic-workflows label. (If the v0.80.9 compiler drops these + # — as documented for update-pull-request above — the Step 3.6 preconditions + + # min-integrity:approved + the allowed:[p/0] allowlist are the compensating controls.) + required-title-prefix: "[ci-fix] " + required-labels: [agentic-workflows] timeout-minutes: 90 @@ -339,33 +386,48 @@ through `safe-outputs`. 5. **One issue = one outcome per run.** Exactly one of: respond to a maintainer's change-request on the open PR (Track C, Step 3.5.R); open the first fix/help/ de-flake PR; advance an existing PR by one attempt (push a follow-up fix); - surface a validated-green PR for review; annotate an unrelated-flake red; wait + surface a validated-green PR for review; mark a target-validated draft PR + ready-for-review (Step 3.6 T3 — the terminal outcome that supersedes the same + run's surface/annotate precursor line), or record its `already-ready` + steady-state no-op; annotate an unrelated-flake red; wait (CI not yet settled); or a recorded skip (the dedicated needs-human PR is deferred — the attempt cap records a skip instead; see Step 6). Always prefer advancing or opening a PR over a skip when a non-mute diff is producible. 6. **All writes via `safe-outputs`.** Allowed outputs: `create_pull_request` (first attempt only), `push_to_pull_request_branch` (advance an existing PR by one attempt), `update_pull_request` (bump the attempt marker / refresh the - prior-attempts table), and `add_comment` (progress notes on the `[ci-fix]` PR - ONLY). NEVER comment on the tracking issue (issues are locked by + prior-attempts table), `add_comment` (progress notes on the `[ci-fix]` PR + ONLY), `mark_pull_request_as_ready_for_review` (flip a target-validated draft + `[ci-fix]` PR from draft to ready — Step 3.6 T3 only), and `add_labels` (add + ONLY the `p/0` label, ONLY on that same draft→ready transition — Step 3.6 T3). + NEVER comment on the tracking issue (issues are locked by `.github/workflows/ci-scan-lock-issues.yml`) — `add_comment` targets the PR. No `gh pr create`, no manual `git push`. **Defense-in-depth:** `add_comment` and `update_pull_request` use `target: "*"` (the agent supplies the PR number). - `add_comment` is now config-locked to `required-title-prefix: "[ci-fix] "` + + `add_comment`, `mark_pull_request_as_ready_for_review`, and `add_labels` are + config-locked to `required-title-prefix: "[ci-fix] "` + `required-labels: [agentic-workflows]` (hard-enforced by the handler, same as - `push_to_pull_request_branch`), so a comment can only ever land on THIS - workflow's own PRs. `update_pull_request` CANNOT be config-locked to a - title/label in gh-aw v0.79.8 (the compiler silently drops `required-*` for that - output — verified against the lock), so it keeps `title: false` (no retitles; - body-marker edits only) plus this prompt-level guard. Before emitting either, - VERIFY the target PR carries BOTH the `[ci-fix]` title prefix AND the - `agentic-workflows` label; never comment on or edit the body of any PR that - lacks both. + `push_to_pull_request_branch`), and `add_labels` additionally has an + `allowed: [p/0]` allowlist so it can ONLY ever add `p/0` — so these can only + ever land on THIS workflow's own PRs. `update_pull_request` CANNOT be + config-locked to a title/label in gh-aw v0.80.9 (the compiler silently drops + `required-*` for that output — verified against the lock), so it keeps + `title: false` (no retitles; body-marker edits only) plus this prompt-level + guard. Before emitting ANY of these, VERIFY the target PR carries BOTH the + `[ci-fix]` title prefix AND the `agentic-workflows` label; never comment on, + edit, un-draft, or label any PR that lacks both. 7. **Per-run safe-output caps (per RUN, not per PR).** Each output type is capped per run: `create_pull_request` 3, `push_to_pull_request_branch` 3, - `add_comment` 3, `update_pull_request` 3. Note an ADVANCE spends one push **and** - one update **and** one comment, so ≤ 3 advances/run; surfacing a green or - annotating a flake spends one comment. When a bucket is exhausted, do NOT keep + `add_comment` 6, `update_pull_request` 3, `mark_pull_request_as_ready_for_review` + 3, `add_labels` 3 (counts LABELS, not calls). Note an ADVANCE spends one push + **and** one update **and** one comment, so ≤ 3 advances/run; surfacing a green or + annotating a flake spends one comment; a **mark-ready (Step 3.6 T3)** of a + still-draft PR spends TWO comments (the Step 3 ✅ surface **and** the T3 🎯 + readiness note) **and** one mark-ready **and** one label as an ALL-OR-NOTHING set + (T3 pre-checks those buckets and defers the whole PR if any is exhausted — never + mark a PR ready without its 🎯 audit comment). `add_comment` is therefore sized 6 + (not 3) so ~3 draft flips, at 2 comments each, can land in one sweep instead of the + shared comment bucket starving the otherwise-idle mark-ready/add_labels buckets. When a bucket is exhausted, do NOT keep emitting (extras are silently dropped) — record `skipped: per-run cap reached; deferring PR # to next cycle` for each remaining PR so the drop is a deliberate, logged decision. The continuous loop picks the deferred PRs up next @@ -409,8 +471,9 @@ For every open tracking issue in scope, converge on exactly one outcome: | Open first help-wanted draft `[ci-fix]` PR | No open PR yet; a plausible candidate exists but cannot be runner-validated (device/UI tests) (attempt 1) | | Open first de-flake draft `[ci-fix]` PR | No open PR yet; failure is intermittent (green-on-retry) from a genuine test-quality defect; a deterministic-synchronization fix is producible (attempt 1, Step 4.7 bucket b) | | Advance an existing PR (push attempt N+1) | Open PR's own CI settled red *because of the fix itself*, no human engaged, marker < 10 — push a NEW distinct fix onto the same branch (Step 3.5 → 5.6) | -| Surface a validated-green PR | Open PR's own CI settled green — comment "validated, attempt N/10; ready for review" and do NOT advance (Step 3.5) | -| Annotate an unrelated-flake red | Open PR is red only on baseline-flake / unrelated legs — comment which leg needs a re-run; do NOT burn an attempt (Step 3.5) | +| Surface a validated-green PR | Open PR's own CI settled green — comment "validated, attempt N/10; ready for review" and do NOT advance (Step 3.5), then mark the draft PR ready for review once the fixed test is confirmed green (Step 3.6) | +| Annotate an unrelated-flake red | Open PR is red only on baseline-flake / unrelated legs — comment which leg needs a re-run; do NOT burn an attempt (Step 3.5); if the SPECIFIC fixed test is confirmed green on that build, still mark the draft PR ready for review (Step 3.6) | +| Mark a validated PR ready for review | The specific test this PR fixed is confirmed green in the PR's own CI (even if unrelated legs are red) — comment the target-test result and transition the draft PR to ready for review; never approve or merge (Step 3.6) | | Wait | Open PR's CI is pending / not yet settled on the current head SHA — do nothing this cycle (Step 3.5) | | Hand-off skip (attempt cap) | Marker == 10 and the signature still reproduces — stop and defer to humans; the dedicated `[ci-fix][needs-human]` PR is planned but currently deferred (Step 6) | | Recorded skip | Visual-regression, human engaged, already-handled, fixed-in-latest-build, infra-flake, out-of-bounds, only-mute-available, or no novel approach producible | @@ -685,14 +748,34 @@ Run these gates in order — the FIRST that fires decides this cycle's outcome: 1. **Human engaged.** If `C.humanEngaged` → `skipped: human engaged on PR #; deferring` and stop. Never fight a human reviewer — the intentional hand-off boundary still holds the moment a person touches the PR. -2. **CI not settled / unknown / incomplete prefetch.** If `C.checksSettled == false` - OR `C.overallConclusion` is `pending`, `neutral`, or `unknown`, OR - `C.dataComplete == false` → the fix's CI has not finished on the current head SHA - (`C.headSha`), or the prefetch could not fully read this PR (a partial read can - understate `humanEngaged` / `attempt`). `skipped: PR # CI pending / prefetch - incomplete on ; waiting` and stop. *(Round 1: this is where a - maintainer `/azp run` is awaited — the `/azp`-gated uitests/devicetests legs will - not have run until a human kicks them.)* +2. **CI not settled — but validate the target test first (target-focused readiness).** + If `C.checksSettled == false` OR `C.overallConclusion` is `pending`, `neutral`, or + `unknown`, OR `C.dataComplete == false` → the *overall* build has not finished on the + current head SHA (`C.headSha`), or the prefetch could not fully read this PR (a partial + read can understate `humanEngaged` / `attempt`). Unrelated legs still draining must NOT + keep an already-proven fix parked as a draft, so before waiting, try the target-focused + fast-path: + - **2a — Target-green fast-path (draft `[ci-fix]` PRs only).** If `C.dataComplete == + true` AND the Step 3.6 preconditions hold (draft PR, `[ci-fix] ` title prefix AND the + `agentic-workflows` label), run Step 3.6 **T1–T2** against `C.headSha` now: identify + the target test(s) and read the PR's OWN build **timeline / per-leg status** (anonymous + `_apis/build` — never `_apis/test`, per Hard-Rule 8). If EVERY target test is + **VALIDATED-GREEN** (per T2's all-platform definition below — the target's category leg + is `succeeded` on every platform that runs it, failed on none, and NO target platform + family the pipeline covers is still pending/unconcluded, per T2's "no platform left + unverified" rule; a platform that simply has no leg for the target's category — the test + doesn't run there — is fine, not a gap) AND every leg in + `C.failedLegs` that has ALREADY concluded classifies as **unrelated flake** (Step 4 / + Step 4.7 method — the fix is implicated in NO completed red), then the fix is proven + regardless of unrelated *pending* legs → run **Step 3.6 T3** (mark ready + 🎯 comment) + and stop. This early-out NEVER advances an attempt and NEVER acts on a red that could + be the fix's fault. + - **2b — Otherwise WAIT.** If the target test has not yet executed (pending/absent on + its leg), or a completed red is (or may be) caused by the fix, or `C.dataComplete == + false`, or this PR has no identifiable target test → `skipped: PR # CI pending / + target not yet validated on ; waiting` and stop. *(Round 1: this is where a + maintainer `/azp run` is awaited — the `/azp`-gated uitests/devicetests legs will not + have run until a human kicks them.)* 3. **Green → surface for review.** If `C.overallConclusion == "success"`: the checks that RAN are green. **Primary-gate check first:** the deterministic `success` verdict only certifies "at least one green check, nothing failing or @@ -704,18 +787,31 @@ Run these gates in order — the FIRST that fires decides this cycle's outcome: # primary CI gate (maui-pr) not green on ; waiting` and stop — do NOT surface. (The `/azp`-gated `maui-pr-uitests` (def 313) / `maui-pr-devicetests` (def 314) legs MAY still be un-run — that is expected and is named below, not a - reason to withhold the surface.) **Idempotency:** scan the PR's existing comments - for a prior bot `✅ … validated … on ` note for THIS head SHA — if one - already exists, record `already-surfaced PR # (head )` and stop - WITHOUT re-commenting (re-surfacing the same green every tick is noise and burns - the per-run comment budget other PRs need). Otherwise resolve the attempt number from - `C.effectiveAttempt` (the authoritative max(marker, bot-commit) counter; omit the - number only if it is somehow indeterminate). **Dry-run gate (Step 0):** if `dry_run == "true"`, do NOT emit any comment — instead print the intended `✅` validated-green body to the run log, tally `dry-run: would-surface-green PR #`, and stop. Otherwise `add_comment` on PR #: `✅ Attempt /10 validated - — the fix's CI is green on . Ready for human review.` - Name any `/azp`-gated legs (uitests def 313 / devicetests def 314) that have not - run and still need a maintainer `/azp run`. Do NOT advance. Record - `surfaced-green PR # (attempt /10)` and stop. *(This directly - attacks the real bottleneck — no reviews — so it is the highest-value outcome.)* + reason to withhold the surface.) **Comment idempotency + dry-run suppress the ✅ + comment ONLY — neither skips Step 3.6.** Scan the PR's existing comments for a prior + bot `✅ … validated … on ` note for THIS head SHA; if one exists, set + `SKIP_SURFACE_COMMENT = true`. Resolve the attempt number from `C.effectiveAttempt` + (the authoritative max(marker, bot-commit) counter; omit the number only if it is + somehow indeterminate). Then post the surface comment UNLESS suppressed: + - if `dry_run == "true"`: do NOT emit — print the intended `✅` validated-green body to + the run log and tally `dry-run: would-surface-green PR #`; + - else if `SKIP_SURFACE_COMMENT`: do NOT re-post — record `already-surfaced PR # + (head )` (re-surfacing the same green every tick is noise and burns the + per-run comment budget other PRs need); + - else `add_comment` on PR #: `✅ Attempt /10 validated — the fix's CI is + green on .` naming any `/azp`-gated legs (uitests def 313 / devicetests def + 314) that have not run and still need a maintainer `/azp run`; record `surfaced-green + PR # (attempt /10)`. (Do NOT assert "ready for human review" on a PR that + is still a draft — Step 3.6, not this comment, owns the draft→ready flip and posts its + own 🎯 announcement when it fires.) + Do NOT advance. Then — in ALL of the above cases — run **Step 3.6** (target-test + readiness gate) for this PR before stopping: its `/azp`-gated target legs may conclude + green on this SAME head SHA on a LATER sweep (an `/azp run` adds no commit), and Step + 3.6 is the ONLY place the draft→ready flip happens, so it MUST re-evaluate every sweep — + never `stop` here before it. *(This directly attacks the real bottleneck — no reviews — + so it is the highest-value outcome.)* 4. **Red → classify caused-by-fix vs unrelated-flake.** If `C.overallConclusion == "failure"`, analyze the PR's OWN failing build (NOT `main`): find the AzDO `maui-pr` build for this PR (filter builds by `branchName=refs/pull//merge`, @@ -723,18 +819,26 @@ Run these gates in order — the FIRST that fires decides this cycle's outcome: method and Step 4.7 flake buckets to `C.failedLegs`. - **Unrelated flake only** (every failed leg is known-flaky / infra / a pre-existing baseline red NOT introduced by the fix): do NOT burn an attempt. - **Idempotency first:** scan the PR's existing comments for a prior bot - `♻️ … unrelated flake … on ` note for THIS head SHA — if one already - exists, record `already-annotated-flake PR # (head )` and stop - WITHOUT re-commenting (re-annotating the same flake every 12h sweep is noise and - burns the per-run comment budget other PRs need). Otherwise resolve the attempt - number from `C.effectiveAttempt` (the authoritative max(marker, bot-commit) - counter), omitting the `Attempt /10:` prefix only if it is somehow - indeterminate. **Dry-run gate (Step 0):** if `dry_run == "true"`, do NOT emit any comment — instead print the intended `♻️` unrelated-flake body to the run log, tally `dry-run: would-annotate-flake PR #`, and stop. Otherwise `add_comment` on PR #: `♻️ Attempt /10: red is - unrelated flake on leg(s) () on ; the fix itself is not - implicated. A maintainer re-run (/azp run ) should clear it.` Record - `annotated-flake PR # (head )` and stop. *(Round 1: human re-runs; - Round 2: auto re-trigger.)* + **Comment idempotency + dry-run suppress the ♻️ comment ONLY — neither skips Step + 3.6.** Scan the PR's existing comments for a prior bot `♻️ … unrelated flake … on + ` note for THIS head SHA; if one exists, set `SKIP_FLAKE_COMMENT = true`. + Resolve the attempt number from `C.effectiveAttempt` (the authoritative max(marker, + bot-commit) counter), omitting the `Attempt /10:` prefix only if it is + somehow indeterminate. Then post the flake note UNLESS suppressed: + - if `dry_run == "true"`: do NOT emit — print the intended `♻️` unrelated-flake body + to the run log and tally `dry-run: would-annotate-flake PR #`; + - else if `SKIP_FLAKE_COMMENT`: do NOT re-post — record `already-annotated-flake PR + # (head )` (re-annotating the same flake every 12h sweep is noise and + burns the per-run comment budget other PRs need); + - else `add_comment` on PR #: `♻️ Attempt /10: red is unrelated flake on + leg(s) () on ; the fix itself is not implicated. A + maintainer re-run (/azp run ) should clear it.` record `annotated-flake + PR # (head )`. + Then — in ALL of the above cases — run **Step 3.6** (target-test readiness gate) for + this PR before stopping: its `/azp`-gated target legs may conclude green on this SAME + head SHA on a LATER sweep, and Step 3.6 is the ONLY place the draft→ready flip + happens, so it MUST re-evaluate every sweep — never `stop` here before it. *(Round 1: + human re-runs; Round 2: auto re-trigger.)* - **Caused by the fix** (a failed leg still matches the original target signature, or the fix introduced a NEW failure): advance an attempt. - **Attempt count.** `attempt = C.effectiveAttempt` — the authoritative @@ -752,6 +856,188 @@ Run these gates in order — the FIRST that fires decides this cycle's outcome: from every prior commit on this branch**, emitted via the Step 5.6 ADVANCE path (push onto the same PR). +#### Step 3.6 — Target-test verification & mark-ready gate + +Reached from the Step 3 **green-surface** branch, the Step 4 **unrelated-flake** branch +(AFTER that branch has posted its comment), and the Step 3.5 **gate-2 target-focused +fast-path** (2a — while unrelated legs are still pending). Purpose: confirm the SPECIFIC +test(s) this PR was opened to fix now PASS in the PR's own CI, and — only when they do +— transition the draft `[ci-fix]` PR to **ready for review** so a maintainer sees a +validated fix instead of a draft. This is the ONLY place the loop flips draft→ready; it +is a state transition, **never** an approval or a merge (a human still reviews and +merges). Overall red on *unrelated* legs must NOT gate readiness — we validate the fix, +not the base branch's flakiness. + +**Preconditions** (ALL must hold; otherwise record `skipped: readiness N/A PR # +()` and stop this gate): +- `C.isDraft == true` — if the PR is already ready-for-review, the draft→ready + transition is already done; do NOT re-mark **and do NOT re-apply `p/0`**. `p/0` is applied + exactly ONCE, atomically with the draft→ready flip (T3 — all-or-nothing with the 🎯 audit + comment + mark-ready), so an already-ready loop PR that is MISSING `p/0` has almost + certainly had it **removed by a maintainer** de-prioritizing the PR — a pure triage action + the human-engagement guard (which inspects comments/reviews/commits, NOT label changes) + cannot see. Re-adding `p/0` every sweep would fight that maintainer indefinitely, violating + the loop's "never override a human" contract. So the loop does NOT reconcile the label: + record `already-ready PR #` and stop this gate. (Trade-off: on the rare occasion a + transient API error drops `p/0` at flip time *after* the T3 atomic pre-check passed, it is + not auto-re-added — but the PR is still ready-for-review with its 🎯 audit comment, and + re-adding `p/0` is a trivial manual action; that is strictly preferable to steam-rolling a + maintainer's deliberate de-prioritization.) +- The PR is unmistakably THIS workflow's own: `[ci-fix] ` title prefix AND the + `agentic-workflows` label (mirrors the safe-output lock; the compensating scope + control if the v0.80.9 compiler drops the declarative required-*). +- You reached this gate from Step 3 (green), Step 4 (**unrelated-flake**), or the Step 3.5 + gate-2 **target-focused fast-path** (2a) — i.e. the fix is NOT implicated in any red that + has concluded. If Step 4 classified a red as **caused by the fix** (ADVANCE), do NOT run + this gate — advance the attempt instead. + +**T1 — Identify the target test(s).** From the `[ci-scan]` issue signature the fix +addresses (and the PR's own diff), extract the fully-qualified test method name(s) the +fix targets — e.g. `SafeAreaShouldWorkOnAllShellTabs`. For a de-flake it is the +de-flaked test; for a product fix it is the originally-failing test(s). If NO specific +test can be identified (e.g. a product build-break rather than a test failure), this is a +**build-only fix** — there is no single test to validate cross-platform, so validate at +**whole-build** granularity rather than stopping (Step 3 only *comments*; Step 3.6 is the +sole draft→ready flip, so a build-only fix is undrafted HERE or not at all). We only reach +this gate from Step 3 (green) / Step 4 (unrelated-flake) / gate-2a, so the fix is not +implicated in any concluded red; additionally require, via the T2 timeline method on +`C.headSha`, that the primary build pipeline `maui-pr` (def 302) has CONCLUDED with EVERY +one of its platform build legs `succeeded`/`completed` and NONE failed/canceled or still +pending — the build is green on **every** platform, not just the originally-broken one (the +cross-platform guard, applied to the build instead of a test). A build-only fix is undrafted +ONLY on the strength of the auto-running `maui-pr` (def 302) whole-build green, which the +loop CAN observe: if the PR's `[ci-scan]` issue signature or its own diff indicates the +ORIGINATING failure was in a `/azp`-gated pipeline (`maui-pr-uitests` def 313 or +`maui-pr-devicetests` def 314) rather than `maui-pr` (def 302), do NOT undraft on +`maui-pr`-green alone — those pipelines do not auto-run on this PR (GITHUB_TOKEN cannot +trigger them), so a green `maui-pr` build is NOT evidence the gated build break is fixed; +record `skipped: build-only fix PR # targets gated pipeline () not run — +deferring to human` and stop this gate. Otherwise, set `TARGET := "the maui-pr build +(build-only fix — no single target test)"` and proceed to **T3** to mark ready. If any `maui-pr` build leg is still unconcluded, record `skipped: build-only fix PR +# not yet whole-build green (leg(s) pending)` and stop this gate WITHOUT marking +ready (a green subset is not enough — a still-pending build leg could yet fail). + +**T2 — Verify the target test's platform legs are green (anonymous, leg-level).** Per-test +outcomes come from the AzDO **test-results** API (`_apis/test/...`), which is **NOT reachable +anonymously — Hard-Rule 8**: those endpoints 302-redirect to a sign-in page on this runner, so +a `curl` returns an HTML redirect (not JSON) and every target test would look "not executed". +Validate instead at **leg granularity** using ONLY the anonymous `_apis/build/...` timeline +(Hard-Rule 8) — a `succeeded` category leg means every test in that category **that actually +ran** passed on that platform. This is a strong bar **only if the target test genuinely +executes**: a job can go green while one specific test is skipped at runtime (`[Ignore]`, +`Assert.Ignore()`, `Assert.Inconclusive()`, a `Skip=`/conditional `[Fact]`, an `#if`-out, or +an early `return` before the asserts). So before trusting a green leg as proof the target +passed, **confirm from the PR's OWN diff that the fix does NOT skip, ignore, disable, or +short-circuit the target test** (it adds no `[Ignore]`/`[Explicit]`, `Assert.Ignore`/ +`Assert.Inconclusive`, `Skip=`, category-exclusion, `#if`-out, or early `return` around the +target). If the fix could cause the target to be runtime-skipped rather than genuinely pass, +leg-level green is NOT sufficient evidence — record `skipped: target test may be +runtime-skipped by this fix (diff adds ); leg-green insufficient, deferring to human +on PR #` and stop this gate WITHOUT marking ready. Otherwise (the target genuinely runs +and asserts, as a de-flake or product fix does) a green category leg cannot hide a +target-test failure, so treat it as authoritative. + +Map the target test to the CI leg(s) that run it. A leg name encodes platform + test category +(e.g. `Android UITests SafeAreaEdges,Shadow`, `iOS UITests SafeAreaEdges,Shadow`, `macOS +UITests SafeAreaEdges,Shadow`, `Windows UITests SafeAreaEdges,Shadow`). Determine the target +test's UI-test category — its `[Category(UITestCategories.X)]` in the test/HostApp source, +visible in the PR diff or the test file — to know which leg-name substring identifies its legs; +for a device test, the per-platform device-test legs. + +Using the SAME build-discovery as Step 4 (filter AzDO builds by `branchName=refs/pull//merge` +or `sourceVersion == C.headSha`), read each build's **timeline** on `C.headSha` for the +pipeline(s) that RUN the target test — `maui-pr` (def 302) for unit/integration tests, +`maui-pr-uitests` (def 313) for Appium UI tests, `maui-pr-devicetests` (def 314) for device +tests: + +```bash +ORG=dnceng-public; PROJ=public; BUILD= +# Anonymous + Hard-Rule-8-compliant. NEVER call _apis/test/... (it 302-redirects to sign-in). +# Each type=="Job" record is one platform leg: result (succeeded/failed/canceled/null), +# state (completed/inProgress/pending), name (encodes " UITests "): +curl -s "https://dev.azure.com/$ORG/$PROJ/_apis/build/builds/$BUILD/timeline?api-version=7.1" \ + | tee /tmp/gh-aw/agent/timeline_${BUILD}.json \ + | jq -r '.records[] | select(.type=="Job") | "\(.result)\t\(.state)\t\(.name)"' +``` + +(The prefetch already exposes per-leg status in `C.failedLegs` / the checks context, and +`gh pr checks ` lists the same per-leg rows with platform+category in the name — use +whichever is handy; the build timeline is the authoritative cross-check on the exact build.) + +Treat a target test as **VALIDATED-GREEN** only if, on `C.headSha`, the leg that runs its +category is green on **every platform it runs on** — a fix that repairs one platform must not +silently regress the same test's leg on another, so a green leg on the originally-red platform +alone is NOT enough. Across the target pipeline's platform legs (for a UI test: the Android, +iOS, Windows, and macOS/MacCatalyst legs running the target's category; for a device test: each +device-test platform), require ALL of: +- the target's category leg is `result == "succeeded"` **and** `state == "completed"` on + **every** platform that runs it, AND +- that leg is `failed` / `canceled` / aborted on **NO** platform, AND +- **no platform is left unverified.** For each platform family the target pipeline covers, that + platform's category leg must have CONCLUDED (`state == "completed"`) on `C.headSha`. If a + platform simply has no leg for the target's category, the test does not run there — that is + fine, not a gap. But if a platform's category leg has **not concluded** (`state` is + `inProgress` / pending, or an `/azp`-gated `maui-pr-uitests` / `maui-pr-devicetests` leg that + has not been kicked), the test's status on that platform is UNKNOWN → the fix is NOT yet + validated across platforms: record `skipped: target test green on but + not yet verified on (leg(s) pending / need /azp run) on PR #` + and stop this gate WITHOUT marking ready. + +A target test whose category leg never concluded on ANY platform (**not executed** anywhere — +e.g. its `/azp`-gated pipeline has not been kicked) is likewise NOT validated: record `skipped: +target test not yet executed on PR # ( not run — needs /azp run)` and stop +this gate WITHOUT marking ready. Do NOT overclaim — a green *sibling* leg (a different category +on the same platform) is not the target's leg, and a green leg on one platform is not a pass on +the others. + +**T3 — Mark ready + report.** If EVERY target test is VALIDATED-GREEN. The 🎯 comment, +the mark-ready, and the `p/0` label are THREE SEPARATE safe-outputs — the comment +existing does NOT prove the mark-ready took effect, so they are tracked independently: + +- **Comment idempotency (dup-suppress only — never gates the mark-ready).** We only reach + T3 while `C.isDraft == true` (the precondition stops an already-ready PR before here). So + if a prior bot comment for THIS head that carries the leading `🎯` anchor AND the + contiguous phrase `validated green on ` already exists (BOTH T3 variants — + `🎯 Target test validated green on ` and the build-only `🎯 Build validated green on + ` — contain that exact contiguous phrase, so this recognizes both; keying on + `target test` alone would instead miss the build-only comment and re-post it every sweep. + **Require the `🎯` anchor** so the match can NEVER be loosened to a gapped + `validated…green…on` form that would false-positive on the Step 3 `✅ … validated — the + fix's CI is green on ` surface comment and wrongly suppress the audit 🎯), the comment + landed on an earlier sweep but the **mark-ready did NOT take + effect** (the PR is still a draft) — set `SUPPRESS_COMMENT = true` (do not re-post the + duplicate 🎯 comment) but STILL complete the mark-ready + label below. Never treat the + comment's existence as "already marked ready" while the PR is still a draft. Record this + case as `re-marking PR # (🎯 present; mark-ready did not take on a prior sweep)`. +- **Atomicity budget pre-check (Hard-Rule 7).** Determine the outputs this gate will emit: + `mark_pull_request_as_ready_for_review` + `add_labels` always, plus `add_comment` unless + `SUPPRESS_COMMENT`. Verify a free per-run slot remains in EACH bucket you are about to + use. If ANY required bucket is exhausted, emit NONE of them — record `skipped: per-run + cap reached; deferring mark-ready PR # to next cycle` and stop this gate. Never mark a + PR ready (or label it) unless its 🎯 audit comment is GUARANTEED to exist for this exact + head SHA — either posted in this same sweep, or (in the `SUPPRESS_COMMENT` re-marking case) + already present from a prior sweep. The outputs you DO emit either all land or all defer + together; the audit trail must never be absent, but it is NOT re-posted when it already exists. +- **Dry-run gate (Step 0):** if `dry_run == "true"`, emit NOTHING — print the intended + readiness comment and "would mark ready + add p/0" to the run log, tally `dry-run: + would-mark-ready PR #`, and stop. +- Otherwise emit for THIS PR number ``: + 1. `add_comment` (ONLY if not `SUPPRESS_COMMENT`): `🎯 Target test validated green on + — passed on ALL platforms it runs on (, + buildId ). — not + caused by this fix."> Transitioning this PR from draft to ready for review and adding + `p/0`; a maintainer still reviews and merges.` (For a **build-only fix** — the T1 + whole-build fallback, no single target test — phrase the first clause as `🎯 Build + validated green on — the maui-pr build passed on ALL platforms (buildId + ).` instead of naming a test.) + 2. `mark_pull_request_as_ready_for_review` with `reason:` a one-line justification + naming the validated test(s) and ``. + 3. `add_labels` with `labels: ["p/0"]` for PR # — put the now-review-ready fix into + the team's p/0 priority queue so it is triaged, not lost in the draft backlog. (If + the PR somehow already carries `p/0`, this is a harmless no-op.) +- Record `marked-ready PR # (target green on ALL platforms on , + labeled p/0)` and stop. + #### Step 3.5.R — Maintainer change-request response (Track C) Reached from Step 3.5 gate 0. Goal: when an **eligible human reviewer** (Hard-Rule @@ -1383,7 +1669,19 @@ Filed by [`ci-status-fix`](https://github.com/dotnet/maui/blob/main/.github/work ### Step 8 — Per-issue tally + end-of-run summary -Per issue, append one outcome line to `/tmp/gh-aw/agent/coverage.txt`: +Per issue, append **one** outcome line to `/tmp/gh-aw/agent/coverage.txt` — the +terminal outcome for the cycle. When one cycle produces a chained pair — a PR gets a +same-cycle precursor line and **then** a Step 3.6 readiness line (`marked-ready` on the +draft→ready flip, or `already-ready` on a later steady-state sweep of an already-ready +PR) — record ONLY the terminal Step 3.6 readiness line: it supersedes ANY same-cycle +non-readiness precursor for that PR, so an aggregator keying on one-line-per-issue never +double-counts. The precursor may be a Step 3 green line (`surfaced-green` on the first ✅, +or `already-surfaced` when it already exists) OR — when an unrelated-flake red and a +target-green coincide — a Step 4 `annotated-flake` line; either way the readiness line is +terminal, and the superseded precursor's signal survives in the PR's 🎯/♻️ comment so no +information is lost. This covers the flip cycle (`surfaced-green` → `marked-ready`), the +post-flip steady state (`already-surfaced` → `already-ready`), and the flake-coincident +flip (`annotated-flake` → `marked-ready`): ``` # main attempt- @@ -1394,8 +1692,14 @@ Per issue, append one outcome line to `/tmp/gh-aw/agent/coverage.txt`: `advance-PR # attempt /10` (ADVANCE mode — new commit pushed to the existing PR), `surfaced-green PR #` (fix's own CI went green; commented for review, did not advance), `annotated-flake PR #` (red was unrelated flake; -commented, attempt NOT burned), `waiting PR #` (CI not settled yet), -`dry-run: would-`, `skipped: `. (The +commented, attempt NOT burned), `marked-ready PR #` (the specific fixed test +was confirmed green in the PR's own CI; draft flipped to ready for review), +`already-surfaced PR #` (the fix's ✅ green comment already existed this sweep; +re-post suppressed — superseded by the Step 3.6 readiness line when the PR also flips +ready that cycle), `already-ready PR #` (terminal steady state — the PR was flipped +ready on a prior cycle and remains green on an unchanged head; no action taken, +supersedes `already-surfaced`), `waiting PR #` (CI not settled yet), +`dry-run: would-`, `skipped: `. (The `needs-human-PR` outcome is reserved for the deferred hand-off — Step 6 currently records a skip instead, so it is not emitted.)
`, and stop. Otherwise `add_comment` on PR #
: `✅ Attempt /10 - validated — the fix's CI is green on . Ready for human review.` - Name any `/azp`-gated legs (uitests def 313 / devicetests def 314) that have not - run and still need a maintainer `/azp run`. Do NOT advance. Record - `surfaced-green PR # (attempt /10)` and stop. *(This directly - attacks the real bottleneck — no reviews — so it is the highest-value outcome.)* + reason to withhold the surface.) **Comment idempotency + dry-run suppress the ✅ + comment ONLY — neither skips Step 3.6.** Scan the PR's existing comments for a prior + bot `✅ … validated … on ` note for THIS head SHA; if one exists, set + `SKIP_SURFACE_COMMENT = true`. Resolve the attempt number from `C.effectiveAttempt` + (the authoritative max(marker, bot-commit) counter; omit the number only if it is + somehow indeterminate). Then post the surface comment UNLESS suppressed: + - if `dry_run == "true"`: do NOT emit — print the intended `✅` validated-green body to + the run log and tally `dry-run: would-surface-green PR #`; + - else if `SKIP_SURFACE_COMMENT`: do NOT re-post — record `already-surfaced PR # + (head )` (re-surfacing the same green every tick is noise and burns the + per-run comment budget other PRs need); + - else `add_comment` on PR #: `✅ Attempt /10 validated — the fix's CI is + green on .` naming any `/azp`-gated legs (uitests def 313 / devicetests def + 314) that have not run and still need a maintainer `/azp run`; record `surfaced-green + PR # (attempt /10)`. (Do NOT assert "ready for human review" on a PR that + is still a draft — Step 3.6, not this comment, owns the draft→ready flip and posts its + own 🎯 announcement when it fires.) + Do NOT advance. Then — in ALL of the above cases — run **Step 3.6** (target-test + readiness gate) for this PR before stopping: its `/azp`-gated target legs may conclude + green on this SAME head SHA on a LATER sweep (an `/azp run` adds no commit), and Step + 3.6 is the ONLY place the draft→ready flip happens, so it MUST re-evaluate every sweep — + never `stop` here before it. *(This directly attacks the real bottleneck — no reviews — + so it is the highest-value outcome.)* 4. **Red → classify caused-by-fix vs unrelated-flake.** If `C.overallConclusion == "failure"`, analyze the PR's OWN failing build (NOT `net11.0`): find the AzDO `maui-pr` build for this PR (filter builds by `branchName=refs/pull//merge`, @@ -733,18 +829,26 @@ Run these gates in order — the FIRST that fires decides this cycle's outcome: method and Step 4.7 flake buckets to `C.failedLegs`. - **Unrelated flake only** (every failed leg is known-flaky / infra / a pre-existing baseline red NOT introduced by the fix): do NOT burn an attempt. - **Idempotency first:** scan the PR's existing comments for a prior bot - `♻️ … unrelated flake … on ` note for THIS head SHA — if one already - exists, record `already-annotated-flake PR # (head )` and stop - WITHOUT re-commenting (re-annotating the same flake every 12h sweep is noise and - burns the per-run comment budget other PRs need). Otherwise resolve the attempt - number from `C.effectiveAttempt` (the authoritative max(marker, bot-commit) - counter), omitting the `Attempt /10:` prefix only if it is somehow - indeterminate. **Dry-run gate (Step 0):** if `dry_run == "true"`, do NOT emit any comment — instead print the intended `♻️` unrelated-flake body to the run log, tally `dry-run: would-annotate-flake PR #`, and stop. Otherwise `add_comment` on PR #: `♻️ Attempt /10: red is - unrelated flake on leg(s) () on ; the fix itself is not - implicated. A maintainer re-run (/azp run ) should clear it.` Record - `annotated-flake PR # (head )` and stop. *(Round 1: human re-runs; - Round 2: auto re-trigger.)* + **Comment idempotency + dry-run suppress the ♻️ comment ONLY — neither skips Step + 3.6.** Scan the PR's existing comments for a prior bot `♻️ … unrelated flake … on + ` note for THIS head SHA; if one exists, set `SKIP_FLAKE_COMMENT = true`. + Resolve the attempt number from `C.effectiveAttempt` (the authoritative max(marker, + bot-commit) counter), omitting the `Attempt /10:` prefix only if it is + somehow indeterminate. Then post the flake note UNLESS suppressed: + - if `dry_run == "true"`: do NOT emit — print the intended `♻️` unrelated-flake body + to the run log and tally `dry-run: would-annotate-flake PR #`; + - else if `SKIP_FLAKE_COMMENT`: do NOT re-post — record `already-annotated-flake PR + # (head )` (re-annotating the same flake every 12h sweep is noise and + burns the per-run comment budget other PRs need); + - else `add_comment` on PR #: `♻️ Attempt /10: red is unrelated flake on + leg(s) () on ; the fix itself is not implicated. A + maintainer re-run (/azp run ) should clear it.` record `annotated-flake + PR # (head )`. + Then — in ALL of the above cases — run **Step 3.6** (target-test readiness gate) for + this PR before stopping: its `/azp`-gated target legs may conclude green on this SAME + head SHA on a LATER sweep, and Step 3.6 is the ONLY place the draft→ready flip + happens, so it MUST re-evaluate every sweep — never `stop` here before it. *(Round 1: + human re-runs; Round 2: auto re-trigger.)* - **Caused by the fix** (a failed leg still matches the original target signature, or the fix introduced a NEW failure): advance an attempt. - **Attempt count.** `attempt = C.effectiveAttempt` — the authoritative @@ -762,6 +866,188 @@ Run these gates in order — the FIRST that fires decides this cycle's outcome: from every prior commit on this branch**, emitted via the Step 5.6 ADVANCE path (push onto the same PR). +#### Step 3.6 — Target-test verification & mark-ready gate + +Reached from the Step 3 **green-surface** branch, the Step 4 **unrelated-flake** branch +(AFTER that branch has posted its comment), and the Step 3.5 **gate-2 target-focused +fast-path** (2a — while unrelated legs are still pending). Purpose: confirm the SPECIFIC +test(s) this PR was opened to fix now PASS in the PR's own CI, and — only when they do +— transition the draft `[ci-fix-net11]` PR to **ready for review** so a maintainer sees +a validated fix instead of a draft. This is the ONLY place the loop flips draft→ready; +it is a state transition, **never** an approval or a merge (a human still reviews and +merges). Overall red on *unrelated* legs must NOT gate readiness — we validate the fix, +not the base branch's flakiness. + +**Preconditions** (ALL must hold; otherwise record `skipped: readiness N/A PR # +()` and stop this gate): +- `C.isDraft == true` — if the PR is already ready-for-review, the draft→ready + transition is already done; do NOT re-mark **and do NOT re-apply `p/0`**. `p/0` is applied + exactly ONCE, atomically with the draft→ready flip (T3 — all-or-nothing with the 🎯 audit + comment + mark-ready), so an already-ready loop PR that is MISSING `p/0` has almost + certainly had it **removed by a maintainer** de-prioritizing the PR — a pure triage action + the human-engagement guard (which inspects comments/reviews/commits, NOT label changes) + cannot see. Re-adding `p/0` every sweep would fight that maintainer indefinitely, violating + the loop's "never override a human" contract. So the loop does NOT reconcile the label: + record `already-ready PR #` and stop this gate. (Trade-off: on the rare occasion a + transient API error drops `p/0` at flip time *after* the T3 atomic pre-check passed, it is + not auto-re-added — but the PR is still ready-for-review with its 🎯 audit comment, and + re-adding `p/0` is a trivial manual action; that is strictly preferable to steam-rolling a + maintainer's deliberate de-prioritization.) +- The PR is unmistakably THIS workflow's own: `[ci-fix-net11] ` title prefix AND the + `agentic-workflows` label (mirrors the safe-output lock; the compensating scope + control if the v0.80.9 compiler drops the declarative required-*). +- You reached this gate from Step 3 (green), Step 4 (**unrelated-flake**), or the Step 3.5 + gate-2 **target-focused fast-path** (2a) — i.e. the fix is NOT implicated in any red that + has concluded. If Step 4 classified a red as **caused by the fix** (ADVANCE), do NOT run + this gate — advance the attempt instead. + +**T1 — Identify the target test(s).** From the `[ci-scan-net11]` issue signature the fix +addresses (and the PR's own diff), extract the fully-qualified test method name(s) the +fix targets — e.g. `SafeAreaShouldWorkOnAllShellTabs`. For a de-flake it is the +de-flaked test; for a product fix it is the originally-failing test(s). If NO specific +test can be identified (e.g. a product build-break rather than a test failure), this is a +**build-only fix** — there is no single test to validate cross-platform, so validate at +**whole-build** granularity rather than stopping (Step 3 only *comments*; Step 3.6 is the +sole draft→ready flip, so a build-only fix is undrafted HERE or not at all). We only reach +this gate from Step 3 (green) / Step 4 (unrelated-flake) / gate-2a, so the fix is not +implicated in any concluded red; additionally require, via the T2 timeline method on +`C.headSha`, that the primary build pipeline `maui-pr` (def 302) has CONCLUDED with EVERY +one of its platform build legs `succeeded`/`completed` and NONE failed/canceled or still +pending — the build is green on **every** platform, not just the originally-broken one (the +cross-platform guard, applied to the build instead of a test). A build-only fix is undrafted +ONLY on the strength of the auto-running `maui-pr` (def 302) whole-build green, which the +loop CAN observe: if the PR's `[ci-scan]` issue signature or its own diff indicates the +ORIGINATING failure was in a `/azp`-gated pipeline (`maui-pr-uitests` def 313 or +`maui-pr-devicetests` def 314) rather than `maui-pr` (def 302), do NOT undraft on +`maui-pr`-green alone — those pipelines do not auto-run on this PR (GITHUB_TOKEN cannot +trigger them), so a green `maui-pr` build is NOT evidence the gated build break is fixed; +record `skipped: build-only fix PR # targets gated pipeline () not run — +deferring to human` and stop this gate. Otherwise, set `TARGET := "the maui-pr build +(build-only fix — no single target test)"` and proceed to **T3** to mark ready. If any `maui-pr` build leg is still unconcluded, record `skipped: build-only fix PR +# not yet whole-build green (leg(s) pending)` and stop this gate WITHOUT marking +ready (a green subset is not enough — a still-pending build leg could yet fail). + +**T2 — Verify the target test's platform legs are green (anonymous, leg-level).** Per-test +outcomes come from the AzDO **test-results** API (`_apis/test/...`), which is **NOT reachable +anonymously — Hard-Rule 8**: those endpoints 302-redirect to a sign-in page on this runner, so +a `curl` returns an HTML redirect (not JSON) and every target test would look "not executed". +Validate instead at **leg granularity** using ONLY the anonymous `_apis/build/...` timeline +(Hard-Rule 8) — a `succeeded` category leg means every test in that category **that actually +ran** passed on that platform. This is a strong bar **only if the target test genuinely +executes**: a job can go green while one specific test is skipped at runtime (`[Ignore]`, +`Assert.Ignore()`, `Assert.Inconclusive()`, a `Skip=`/conditional `[Fact]`, an `#if`-out, or +an early `return` before the asserts). So before trusting a green leg as proof the target +passed, **confirm from the PR's OWN diff that the fix does NOT skip, ignore, disable, or +short-circuit the target test** (it adds no `[Ignore]`/`[Explicit]`, `Assert.Ignore`/ +`Assert.Inconclusive`, `Skip=`, category-exclusion, `#if`-out, or early `return` around the +target). If the fix could cause the target to be runtime-skipped rather than genuinely pass, +leg-level green is NOT sufficient evidence — record `skipped: target test may be +runtime-skipped by this fix (diff adds ); leg-green insufficient, deferring to human +on PR #` and stop this gate WITHOUT marking ready. Otherwise (the target genuinely runs +and asserts, as a de-flake or product fix does) a green category leg cannot hide a +target-test failure, so treat it as authoritative. + +Map the target test to the CI leg(s) that run it. A leg name encodes platform + test category +(e.g. `Android UITests SafeAreaEdges,Shadow`, `iOS UITests SafeAreaEdges,Shadow`, `macOS +UITests SafeAreaEdges,Shadow`, `Windows UITests SafeAreaEdges,Shadow`). Determine the target +test's UI-test category — its `[Category(UITestCategories.X)]` in the test/HostApp source, +visible in the PR diff or the test file — to know which leg-name substring identifies its legs; +for a device test, the per-platform device-test legs. + +Using the SAME build-discovery as Step 4 (filter AzDO builds by `branchName=refs/pull//merge` +or `sourceVersion == C.headSha`), read each build's **timeline** on `C.headSha` for the +pipeline(s) that RUN the target test — `maui-pr` (def 302) for unit/integration tests, +`maui-pr-uitests` (def 313) for Appium UI tests, `maui-pr-devicetests` (def 314) for device +tests: + +```bash +ORG=dnceng-public; PROJ=public; BUILD= +# Anonymous + Hard-Rule-8-compliant. NEVER call _apis/test/... (it 302-redirects to sign-in). +# Each type=="Job" record is one platform leg: result (succeeded/failed/canceled/null), +# state (completed/inProgress/pending), name (encodes " UITests "): +curl -s "https://dev.azure.com/$ORG/$PROJ/_apis/build/builds/$BUILD/timeline?api-version=7.1" \ + | tee /tmp/gh-aw/agent/timeline_${BUILD}.json \ + | jq -r '.records[] | select(.type=="Job") | "\(.result)\t\(.state)\t\(.name)"' +``` + +(The prefetch already exposes per-leg status in `C.failedLegs` / the checks context, and +`gh pr checks ` lists the same per-leg rows with platform+category in the name — use +whichever is handy; the build timeline is the authoritative cross-check on the exact build.) + +Treat a target test as **VALIDATED-GREEN** only if, on `C.headSha`, the leg that runs its +category is green on **every platform it runs on** — a fix that repairs one platform must not +silently regress the same test's leg on another, so a green leg on the originally-red platform +alone is NOT enough. Across the target pipeline's platform legs (for a UI test: the Android, +iOS, Windows, and macOS/MacCatalyst legs running the target's category; for a device test: each +device-test platform), require ALL of: +- the target's category leg is `result == "succeeded"` **and** `state == "completed"` on + **every** platform that runs it, AND +- that leg is `failed` / `canceled` / aborted on **NO** platform, AND +- **no platform is left unverified.** For each platform family the target pipeline covers, that + platform's category leg must have CONCLUDED (`state == "completed"`) on `C.headSha`. If a + platform simply has no leg for the target's category, the test does not run there — that is + fine, not a gap. But if a platform's category leg has **not concluded** (`state` is + `inProgress` / pending, or an `/azp`-gated `maui-pr-uitests` / `maui-pr-devicetests` leg that + has not been kicked), the test's status on that platform is UNKNOWN → the fix is NOT yet + validated across platforms: record `skipped: target test green on but + not yet verified on (leg(s) pending / need /azp run) on PR #` + and stop this gate WITHOUT marking ready. + +A target test whose category leg never concluded on ANY platform (**not executed** anywhere — +e.g. its `/azp`-gated pipeline has not been kicked) is likewise NOT validated: record `skipped: +target test not yet executed on PR # ( not run — needs /azp run)` and stop +this gate WITHOUT marking ready. Do NOT overclaim — a green *sibling* leg (a different category +on the same platform) is not the target's leg, and a green leg on one platform is not a pass on +the others. + +**T3 — Mark ready + report.** If EVERY target test is VALIDATED-GREEN. The 🎯 comment, +the mark-ready, and the `p/0` label are THREE SEPARATE safe-outputs — the comment +existing does NOT prove the mark-ready took effect, so they are tracked independently: + +- **Comment idempotency (dup-suppress only — never gates the mark-ready).** We only reach + T3 while `C.isDraft == true` (the precondition stops an already-ready PR before here). So + if a prior bot comment for THIS head that carries the leading `🎯` anchor AND the + contiguous phrase `validated green on ` already exists (BOTH T3 variants — + `🎯 Target test validated green on ` and the build-only `🎯 Build validated green on + ` — contain that exact contiguous phrase, so this recognizes both; keying on + `target test` alone would instead miss the build-only comment and re-post it every sweep. + **Require the `🎯` anchor** so the match can NEVER be loosened to a gapped + `validated…green…on` form that would false-positive on the Step 3 `✅ … validated — the + fix's CI is green on ` surface comment and wrongly suppress the audit 🎯), the comment + landed on an earlier sweep but the **mark-ready did NOT take + effect** (the PR is still a draft) — set `SUPPRESS_COMMENT = true` (do not re-post the + duplicate 🎯 comment) but STILL complete the mark-ready + label below. Never treat the + comment's existence as "already marked ready" while the PR is still a draft. Record this + case as `re-marking PR # (🎯 present; mark-ready did not take on a prior sweep)`. +- **Atomicity budget pre-check (Hard-Rule 7).** Determine the outputs this gate will emit: + `mark_pull_request_as_ready_for_review` + `add_labels` always, plus `add_comment` unless + `SUPPRESS_COMMENT`. Verify a free per-run slot remains in EACH bucket you are about to + use. If ANY required bucket is exhausted, emit NONE of them — record `skipped: per-run + cap reached; deferring mark-ready PR # to next cycle` and stop this gate. Never mark a + PR ready (or label it) unless its 🎯 audit comment is GUARANTEED to exist for this exact + head SHA — either posted in this same sweep, or (in the `SUPPRESS_COMMENT` re-marking case) + already present from a prior sweep. The outputs you DO emit either all land or all defer + together; the audit trail must never be absent, but it is NOT re-posted when it already exists. +- **Dry-run gate (Step 0):** if `dry_run == "true"`, emit NOTHING — print the intended + readiness comment and "would mark ready + add p/0" to the run log, tally `dry-run: + would-mark-ready PR #`, and stop. +- Otherwise emit for THIS PR number ``: + 1. `add_comment` (ONLY if not `SUPPRESS_COMMENT`): `🎯 Target test validated green on + — passed on ALL platforms it runs on (, + buildId ). — not + caused by this fix."> Transitioning this PR from draft to ready for review and adding + `p/0`; a maintainer still reviews and merges.` (For a **build-only fix** — the T1 + whole-build fallback, no single target test — phrase the first clause as `🎯 Build + validated green on — the maui-pr build passed on ALL platforms (buildId + ).` instead of naming a test.) + 2. `mark_pull_request_as_ready_for_review` with `reason:` a one-line justification + naming the validated test(s) and ``. + 3. `add_labels` with `labels: ["p/0"]` for PR # — put the now-review-ready fix into + the team's p/0 priority queue so it is triaged, not lost in the draft backlog. (If + the PR somehow already carries `p/0`, this is a harmless no-op.) +- Record `marked-ready PR # (target green on ALL platforms on , + labeled p/0)` and stop. + #### Step 3.5.R — Maintainer change-request response (Track C) Reached from Step 3.5 gate 0. Goal: when an **eligible human reviewer** (Hard-Rule @@ -1395,7 +1681,19 @@ Filed by [`ci-status-fix-net11`](https://github.com/dotnet/maui/blob/main/.githu ### Step 8 — Per-issue tally + end-of-run summary -Per issue, append one outcome line to `/tmp/gh-aw/agent/coverage.txt`: +Per issue, append **one** outcome line to `/tmp/gh-aw/agent/coverage.txt` — the +terminal outcome for the cycle. When one cycle produces a chained pair — a PR gets a +same-cycle precursor line and **then** a Step 3.6 readiness line (`marked-ready` on the +draft→ready flip, or `already-ready` on a later steady-state sweep of an already-ready +PR) — record ONLY the terminal Step 3.6 readiness line: it supersedes ANY same-cycle +non-readiness precursor for that PR, so an aggregator keying on one-line-per-issue never +double-counts. The precursor may be a Step 3 green line (`surfaced-green` on the first ✅, +or `already-surfaced` when it already exists) OR — when an unrelated-flake red and a +target-green coincide — a Step 4 `annotated-flake` line; either way the readiness line is +terminal, and the superseded precursor's signal survives in the PR's 🎯/♻️ comment so no +information is lost. This covers the flip cycle (`surfaced-green` → `marked-ready`), the +post-flip steady state (`already-surfaced` → `already-ready`), and the flake-coincident +flip (`annotated-flake` → `marked-ready`): ``` # net11.0 attempt- @@ -1406,8 +1704,14 @@ Per issue, append one outcome line to `/tmp/gh-aw/agent/coverage.txt`: `advance-PR # attempt /10` (ADVANCE mode — new commit pushed to the existing PR), `surfaced-green PR #` (fix's own CI went green; commented for review, did not advance), `annotated-flake PR #` (red was unrelated flake; -commented, attempt NOT burned), `waiting PR #` (CI not settled yet), -`dry-run: would-`, `skipped: `. (The +commented, attempt NOT burned), `marked-ready PR #` (the specific fixed test +was confirmed green in the PR's own CI; draft flipped to ready for review), +`already-surfaced PR #` (the fix's ✅ green comment already existed this sweep; +re-post suppressed — superseded by the Step 3.6 readiness line when the PR also flips +ready that cycle), `already-ready PR #` (terminal steady state — the PR was flipped +ready on a prior cycle and remains green on an unchanged head; no action taken, +supersedes `already-surfaced`), `waiting PR #` (CI not settled yet), +`dry-run: would-`, `skipped: `. (The `needs-human-PR` outcome is reserved for the deferred hand-off — Step 6 currently records a skip instead, so it is not emitted.) diff --git a/.github/workflows/ci-status-fix.lock.yml b/.github/workflows/ci-status-fix.lock.yml index 93dc5969cb0d..65511f2b3259 100644 --- a/.github/workflows/ci-status-fix.lock.yml +++ b/.github/workflows/ci-status-fix.lock.yml @@ -1,4 +1,4 @@ -# gh-aw-metadata: {"schema_version":"v4","frontmatter_hash":"f014161967589ebcaf1ac5ec6f3c88ee5a4388d28b55a07dc36c45b7b2aebab6","body_hash":"86c6b2d4c3df13da14c6a3c8b211381fcc9f97bb1951ff7b80588a2c5338e00c","compiler_version":"v0.80.9","strict":true,"agent_id":"copilot","agent_model":"claude-opus-4.8","engine_versions":{"copilot":"1.0.63"}} +# gh-aw-metadata: {"schema_version":"v4","frontmatter_hash":"b0ebf8a2f179900cfe3638e994b27a43cec52bdbdd2331dc1bd4d65762fa2d6b","body_hash":"1825e2b1f7c7bd88241bf37580655489360f9bebc806edd00837a52ce4ad83a0","compiler_version":"v0.80.9","strict":true,"agent_id":"copilot","agent_model":"claude-opus-4.8","engine_versions":{"copilot":"1.0.63"}} # gh-aw-manifest: {"version":1,"secrets":["COPILOT_PAT_0","COPILOT_PAT_1","COPILOT_PAT_2","COPILOT_PAT_3","COPILOT_PAT_4","COPILOT_PAT_5","COPILOT_PAT_6","COPILOT_PAT_7","COPILOT_PAT_8","COPILOT_PAT_9","GH_AW_CI_TRIGGER_TOKEN","GH_AW_GITHUB_MCP_SERVER_TOKEN","GH_AW_GITHUB_TOKEN","GITHUB_TOKEN"],"actions":[{"repo":"actions/checkout","sha":"34e114876b0b11c390a56381ad16ebd13914f8d5","version":"v4"},{"repo":"actions/checkout","sha":"9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0","version":"v7.0.0"},{"repo":"actions/checkout","sha":"de0fac2e4500dabe0009e67214ff5f5447ce83dd","version":"v6.0.2"},{"repo":"actions/download-artifact","sha":"3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c","version":"v8.0.1"},{"repo":"actions/github-script","sha":"3a2844b7e9c422d3c10d287c895573f7108da1b3","version":"v9.0.0"},{"repo":"actions/setup-node","sha":"48b55a011bda9f5d6aeb4c2d9c7362e8dae4041e","version":"v6.4.0"},{"repo":"actions/upload-artifact","sha":"043fb46d1a93c77aae656e7c1c64a875d1fc6a0a","version":"v7.0.1"},{"repo":"github/gh-aw-actions/setup","sha":"8c7d04ebf1ece56cd381446125da3e0f6896294a","version":"v0.80.9"}],"containers":[{"image":"ghcr.io/github/gh-aw-firewall/agent:0.27.7","digest":"sha256:aae231e4635c8999d039c132f1602d3df850fe9b84a00aa2b5ac981179b5661c","pinned_image":"ghcr.io/github/gh-aw-firewall/agent:0.27.7@sha256:aae231e4635c8999d039c132f1602d3df850fe9b84a00aa2b5ac981179b5661c"},{"image":"ghcr.io/github/gh-aw-firewall/api-proxy:0.27.7","digest":"sha256:009caf2e3d88fa77b64e9a03a95a228fc58db0f1701c6d324b29ba5a3c7c79b6","pinned_image":"ghcr.io/github/gh-aw-firewall/api-proxy:0.27.7@sha256:009caf2e3d88fa77b64e9a03a95a228fc58db0f1701c6d324b29ba5a3c7c79b6"},{"image":"ghcr.io/github/gh-aw-firewall/squid:0.27.7","digest":"sha256:deb1d4e19de62d51cee0508057a596a19315c3423ada4d675cad136dc8037c96","pinned_image":"ghcr.io/github/gh-aw-firewall/squid:0.27.7@sha256:deb1d4e19de62d51cee0508057a596a19315c3423ada4d675cad136dc8037c96"},{"image":"ghcr.io/github/gh-aw-mcpg:v0.3.27","digest":"sha256:fe984bddde4ec05d756d9043edb0a32912e6b7b72f6a121b1082f29221421cc7","pinned_image":"ghcr.io/github/gh-aw-mcpg:v0.3.27@sha256:fe984bddde4ec05d756d9043edb0a32912e6b7b72f6a121b1082f29221421cc7"},{"image":"ghcr.io/github/gh-aw-node","digest":"sha256:529d02eb970b1161aa25c593a9c3df57fdfad5a8add328cb3b6eccef66f3183b","pinned_image":"ghcr.io/github/gh-aw-node@sha256:529d02eb970b1161aa25c593a9c3df57fdfad5a8add328cb3b6eccef66f3183b"},{"image":"ghcr.io/github/github-mcp-server:v1.4.0","digest":"sha256:2afb26356481d1a350e14544a6e160f7f7ec1561a1ea309b823665abf0309036","pinned_image":"ghcr.io/github/github-mcp-server:v1.4.0@sha256:2afb26356481d1a350e14544a6e160f7f7ec1561a1ea309b823665abf0309036"}]} # This file was automatically generated by gh-aw (v0.80.9). DO NOT EDIT. To debug this workflow, load the skill at https://github.com/github/gh-aw/blob/main/debug.md # @@ -37,7 +37,9 @@ # humans (the open tracking issue is the hand-off surface; a dedicated # [ci-fix][needs-human] PR is planned but currently deferred — see Step 6). CI on # the PR is kicked by a human `/azp run` for now; the loop watches, classifies, -# and re-fixes autonomously between kicks. +# and re-fixes autonomously between kicks. When the SPECIFIC test a PR fixed is +# confirmed green in that PR's own CI, the loop marks the draft PR ready for review +# (a state transition only — it never approves or merges). # Never mutes tests, but # de-flakes genuinely flaky ones (deterministic synchronization, no retries / # timeout bumps). Always skips visual-regression / screenshot issues. @@ -273,24 +275,24 @@ jobs: run: | bash "${RUNNER_TEMP}/gh-aw/actions/create_prompt_first.sh" { - cat << 'GH_AW_PROMPT_25cf75111b670167_EOF' + cat << 'GH_AW_PROMPT_2ac1d0b4083428f8_EOF' - GH_AW_PROMPT_25cf75111b670167_EOF + GH_AW_PROMPT_2ac1d0b4083428f8_EOF cat "${RUNNER_TEMP}/gh-aw/prompts/xpia.md" cat "${RUNNER_TEMP}/gh-aw/prompts/temp_folder_prompt.md" cat "${RUNNER_TEMP}/gh-aw/prompts/markdown.md" cat "${RUNNER_TEMP}/gh-aw/prompts/safe_outputs_prompt.md" - cat << 'GH_AW_PROMPT_25cf75111b670167_EOF' + cat << 'GH_AW_PROMPT_2ac1d0b4083428f8_EOF' - Tools: add_comment(max:3), create_pull_request(max:3), update_pull_request(max:3), push_to_pull_request_branch(max:3), missing_tool, missing_data, noop - GH_AW_PROMPT_25cf75111b670167_EOF + Tools: add_comment(max:6), create_pull_request(max:3), update_pull_request(max:3), mark_pull_request_as_ready_for_review(max:3), add_labels(max:3), push_to_pull_request_branch(max:3), missing_tool, missing_data, noop + GH_AW_PROMPT_2ac1d0b4083428f8_EOF cat "${RUNNER_TEMP}/gh-aw/prompts/safe_outputs_create_pull_request.md" cat "${RUNNER_TEMP}/gh-aw/prompts/safe_outputs_push_to_pr_branch.md" - cat << 'GH_AW_PROMPT_25cf75111b670167_EOF' + cat << 'GH_AW_PROMPT_2ac1d0b4083428f8_EOF' - GH_AW_PROMPT_25cf75111b670167_EOF + GH_AW_PROMPT_2ac1d0b4083428f8_EOF cat "${RUNNER_TEMP}/gh-aw/prompts/mcp_cli_tools_prompt.md" - cat << 'GH_AW_PROMPT_25cf75111b670167_EOF' + cat << 'GH_AW_PROMPT_2ac1d0b4083428f8_EOF' The following GitHub context information is available for this workflow: {{#if github.actor}} @@ -332,12 +334,12 @@ jobs: stop immediately and report the limitation rather than spending turns trying to work around it. - GH_AW_PROMPT_25cf75111b670167_EOF + GH_AW_PROMPT_2ac1d0b4083428f8_EOF cat "${RUNNER_TEMP}/gh-aw/prompts/github_mcp_tools_with_safeoutputs_prompt.md" - cat << 'GH_AW_PROMPT_25cf75111b670167_EOF' + cat << 'GH_AW_PROMPT_2ac1d0b4083428f8_EOF' {{#runtime-import .github/workflows/ci-status-fix.md}} - GH_AW_PROMPT_25cf75111b670167_EOF + GH_AW_PROMPT_2ac1d0b4083428f8_EOF } > "$GH_AW_PROMPT" - name: Interpolate variables and render templates uses: actions/github-script@3a2844b7e9c422d3c10d287c895573f7108da1b3 # v9.0.0 @@ -554,16 +556,18 @@ jobs: mkdir -p "${RUNNER_TEMP}/gh-aw/safeoutputs" mkdir -p /tmp/gh-aw/safeoutputs mkdir -p /tmp/gh-aw/mcp-logs/safeoutputs - cat > "${RUNNER_TEMP}/gh-aw/safeoutputs/config.json" << 'GH_AW_SAFE_OUTPUTS_CONFIG_fa2a69fad5fd0485_EOF' - {"add_comment":{"discussions":false,"max":3,"required_labels":["agentic-workflows"],"required_title_prefix":"[ci-fix] ","target":"*"},"create_pull_request":{"allowed_base_branches":["main"],"allowed_branches":["ci-fix/**"],"allowed_files":["src/Core/**","src/Controls/**","src/Essentials/**","src/BlazorWebView/**","src/TestUtils/**","src/Templates/**","**/PublicAPI.Unshipped.txt"],"base_branch":"main","draft":true,"labels":["agentic-workflows"],"max":3,"max_patch_files":100,"max_patch_size":4096,"protect_top_level_dot_folders":true,"protected_files":["package.json","bun.lockb","bunfig.toml","deno.json","deno.jsonc","deno.lock","global.json","NuGet.Config","Directory.Packages.props","mix.exs","mix.lock","go.mod","go.sum","stack.yaml","stack.yaml.lock","pom.xml","build.gradle","build.gradle.kts","settings.gradle","settings.gradle.kts","gradle.properties","package-lock.json","yarn.lock","pnpm-lock.yaml","npm-shrinkwrap.json","requirements.txt","Pipfile","Pipfile.lock","pyproject.toml","setup.py","setup.cfg","Gemfile","Gemfile.lock","uv.lock","CODEOWNERS","DESIGN.md","README.md","CONTRIBUTING.md","CHANGELOG.md","SECURITY.md","CODE_OF_CONDUCT.md","AGENTS.md","CLAUDE.md","GEMINI.md"],"protected_files_policy":"request_review","title_prefix":"[ci-fix] "},"create_report_incomplete_issue":{},"missing_data":{},"missing_tool":{},"noop":{"max":1,"report-as-issue":"true"},"push_to_pull_request_branch":{"allowed_files":["src/Core/**","src/Controls/**","src/Essentials/**","src/BlazorWebView/**","src/TestUtils/**","src/Templates/**","**/PublicAPI.Unshipped.txt"],"if_no_changes":"warn","max":3,"max_patch_size":4096,"protect_top_level_dot_folders":true,"protected_files":["package.json","bun.lockb","bunfig.toml","deno.json","deno.jsonc","deno.lock","global.json","NuGet.Config","Directory.Packages.props","mix.exs","mix.lock","go.mod","go.sum","stack.yaml","stack.yaml.lock","pom.xml","build.gradle","build.gradle.kts","settings.gradle","settings.gradle.kts","gradle.properties","package-lock.json","yarn.lock","pnpm-lock.yaml","npm-shrinkwrap.json","requirements.txt","Pipfile","Pipfile.lock","pyproject.toml","setup.py","setup.cfg","Gemfile","Gemfile.lock","uv.lock","CODEOWNERS","DESIGN.md","README.md","CONTRIBUTING.md","CHANGELOG.md","SECURITY.md","CODE_OF_CONDUCT.md","AGENTS.md","CLAUDE.md","GEMINI.md"],"required_labels":["agentic-workflows"],"target":"*","title_prefix":"[ci-fix] "},"report_incomplete":{},"update_pull_request":{"allow_body":true,"allow_title":false,"max":3,"target":"*","update_branch":false}} - GH_AW_SAFE_OUTPUTS_CONFIG_fa2a69fad5fd0485_EOF + cat > "${RUNNER_TEMP}/gh-aw/safeoutputs/config.json" << 'GH_AW_SAFE_OUTPUTS_CONFIG_0644bf1434ca0071_EOF' + {"add_comment":{"discussions":false,"max":6,"required_labels":["agentic-workflows"],"required_title_prefix":"[ci-fix] ","target":"*"},"add_labels":{"allowed":["p/0"],"max":3,"required_labels":["agentic-workflows"],"required_title_prefix":"[ci-fix] ","target":"*"},"create_pull_request":{"allowed_base_branches":["main"],"allowed_branches":["ci-fix/**"],"allowed_files":["src/Core/**","src/Controls/**","src/Essentials/**","src/BlazorWebView/**","src/TestUtils/**","src/Templates/**","**/PublicAPI.Unshipped.txt"],"base_branch":"main","draft":true,"labels":["agentic-workflows"],"max":3,"max_patch_files":100,"max_patch_size":4096,"protect_top_level_dot_folders":true,"protected_files":["package.json","bun.lockb","bunfig.toml","deno.json","deno.jsonc","deno.lock","global.json","NuGet.Config","Directory.Packages.props","mix.exs","mix.lock","go.mod","go.sum","stack.yaml","stack.yaml.lock","pom.xml","build.gradle","build.gradle.kts","settings.gradle","settings.gradle.kts","gradle.properties","package-lock.json","yarn.lock","pnpm-lock.yaml","npm-shrinkwrap.json","requirements.txt","Pipfile","Pipfile.lock","pyproject.toml","setup.py","setup.cfg","Gemfile","Gemfile.lock","uv.lock","CODEOWNERS","DESIGN.md","README.md","CONTRIBUTING.md","CHANGELOG.md","SECURITY.md","CODE_OF_CONDUCT.md","AGENTS.md","CLAUDE.md","GEMINI.md"],"protected_files_policy":"request_review","title_prefix":"[ci-fix] "},"create_report_incomplete_issue":{},"mark_pull_request_as_ready_for_review":{"max":3,"required_labels":["agentic-workflows"],"required_title_prefix":"[ci-fix] ","target":"*"},"missing_data":{},"missing_tool":{},"noop":{"max":1,"report-as-issue":"true"},"push_to_pull_request_branch":{"allowed_files":["src/Core/**","src/Controls/**","src/Essentials/**","src/BlazorWebView/**","src/TestUtils/**","src/Templates/**","**/PublicAPI.Unshipped.txt"],"if_no_changes":"warn","max":3,"max_patch_size":4096,"protect_top_level_dot_folders":true,"protected_files":["package.json","bun.lockb","bunfig.toml","deno.json","deno.jsonc","deno.lock","global.json","NuGet.Config","Directory.Packages.props","mix.exs","mix.lock","go.mod","go.sum","stack.yaml","stack.yaml.lock","pom.xml","build.gradle","build.gradle.kts","settings.gradle","settings.gradle.kts","gradle.properties","package-lock.json","yarn.lock","pnpm-lock.yaml","npm-shrinkwrap.json","requirements.txt","Pipfile","Pipfile.lock","pyproject.toml","setup.py","setup.cfg","Gemfile","Gemfile.lock","uv.lock","CODEOWNERS","DESIGN.md","README.md","CONTRIBUTING.md","CHANGELOG.md","SECURITY.md","CODE_OF_CONDUCT.md","AGENTS.md","CLAUDE.md","GEMINI.md"],"required_labels":["agentic-workflows"],"target":"*","title_prefix":"[ci-fix] "},"report_incomplete":{},"update_pull_request":{"allow_body":true,"allow_title":false,"max":3,"target":"*","update_branch":false}} + GH_AW_SAFE_OUTPUTS_CONFIG_0644bf1434ca0071_EOF - name: Generate Safe Outputs Tools env: GH_AW_TOOLS_META_JSON: | { "description_suffixes": { - "add_comment": " CONSTRAINTS: Maximum 3 comment(s) can be added. Target: *. Supports reply_to_id for discussion threading.", + "add_comment": " CONSTRAINTS: Maximum 6 comment(s) can be added. Target: *. Supports reply_to_id for discussion threading.", + "add_labels": " CONSTRAINTS: Maximum 3 label(s) can be added. Only these labels are allowed: [\"p/0\"]. Target: *.", "create_pull_request": " CONSTRAINTS: Maximum 3 pull request(s) can be created. Title will be prefixed with \"[ci-fix] \". Labels [\"agentic-workflows\"] will be automatically added. Only these labels are allowed: [\"agentic-workflows\"]. PRs will be created as drafts.", + "mark_pull_request_as_ready_for_review": " CONSTRAINTS: Maximum 3 pull request(s) can be marked as ready for review.", "push_to_pull_request_branch": " CONSTRAINTS: Maximum 3 push(es) can be made. The target pull request title must start with \"[ci-fix] \".", "update_pull_request": " CONSTRAINTS: Maximum 3 pull request(s) can be updated. Target: *." }, @@ -594,6 +598,25 @@ jobs: } } }, + "add_labels": { + "defaultMax": 5, + "fields": { + "item_number": { + "issueNumberOrTemporaryId": true + }, + "labels": { + "required": true, + "type": "array", + "itemType": "string", + "itemSanitize": true, + "itemMaxLength": 128 + }, + "repo": { + "type": "string", + "maxLength": 256 + } + } + }, "create_pull_request": { "defaultMax": 1, "fields": { @@ -635,6 +658,24 @@ jobs: } } }, + "mark_pull_request_as_ready_for_review": { + "defaultMax": 1, + "fields": { + "pull_request_number": { + "issueOrPRNumber": true + }, + "reason": { + "required": true, + "type": "string", + "sanitize": true, + "maxLength": 65000 + }, + "repo": { + "type": "string", + "maxLength": 256 + } + } + }, "missing_data": { "defaultMax": 20, "fields": { @@ -1494,7 +1535,7 @@ jobs: uses: actions/github-script@3a2844b7e9c422d3c10d287c895573f7108da1b3 # v9.0.0 env: WORKFLOW_NAME: "CI Failure Fixer (main)" - WORKFLOW_DESCRIPTION: "Periodic pass over open ci-scan tracking issues filed by the main-branch CI\nfailure scanner (.github/workflows/ci-status-main.md). This workflow targets\nthe `main` branch EXCLUSIVELY: it processes only issues labelled ci-scan and\nopens every PR against main. (The net11.0 branch is handled by the parallel\n.github/workflows/ci-status-fix-net11.md — the two are split because gh-aw can\nonly transport a fix relative to ONE static base branch per workflow, and the\nmain↔net11.0 divergence exceeds gh-aw's 10 MB transport-patch cap.) The fixer\nopens ONE draft [ci-fix] PR per actionable issue against main, then WATCHES\nthat PR's own CI on later runs: when the fix's CI comes back red and the red is\ncaused by the fix itself, it pushes a fresh follow-up fix onto the SAME PR\nbranch (never a second PR) — up to 10 attempts — then stops and defers to\nhumans (the open tracking issue is the hand-off surface; a dedicated\n[ci-fix][needs-human] PR is planned but currently deferred — see Step 6). CI on\nthe PR is kicked by a human `/azp run` for now; the loop watches, classifies,\nand re-fixes autonomously between kicks.\nNever mutes tests, but\nde-flakes genuinely flaky ones (deterministic synchronization, no retries /\ntimeout bumps). Always skips visual-regression / screenshot issues." + WORKFLOW_DESCRIPTION: "Periodic pass over open ci-scan tracking issues filed by the main-branch CI\nfailure scanner (.github/workflows/ci-status-main.md). This workflow targets\nthe `main` branch EXCLUSIVELY: it processes only issues labelled ci-scan and\nopens every PR against main. (The net11.0 branch is handled by the parallel\n.github/workflows/ci-status-fix-net11.md — the two are split because gh-aw can\nonly transport a fix relative to ONE static base branch per workflow, and the\nmain↔net11.0 divergence exceeds gh-aw's 10 MB transport-patch cap.) The fixer\nopens ONE draft [ci-fix] PR per actionable issue against main, then WATCHES\nthat PR's own CI on later runs: when the fix's CI comes back red and the red is\ncaused by the fix itself, it pushes a fresh follow-up fix onto the SAME PR\nbranch (never a second PR) — up to 10 attempts — then stops and defers to\nhumans (the open tracking issue is the hand-off surface; a dedicated\n[ci-fix][needs-human] PR is planned but currently deferred — see Step 6). CI on\nthe PR is kicked by a human `/azp run` for now; the loop watches, classifies,\nand re-fixes autonomously between kicks. When the SPECIFIC test a PR fixed is\nconfirmed green in that PR's own CI, the loop marks the draft PR ready for review\n(a state transition only — it never approves or merges).\nNever mutes tests, but\nde-flakes genuinely flaky ones (deterministic synchronization, no retries /\ntimeout bumps). Always skips visual-regression / screenshot issues." HAS_PATCH: ${{ needs.agent.outputs.has_patch }} with: script: | @@ -1910,7 +1951,7 @@ jobs: GH_AW_ALLOWED_DOMAINS: "*.blob.core.windows.net,*.githubusercontent.com,api.business.githubcopilot.com,api.enterprise.githubcopilot.com,api.github.com,api.githubcopilot.com,api.individual.githubcopilot.com,api.snapcraft.io,archive.ubuntu.com,azure.archive.ubuntu.com,codeload.github.com,crl.geotrust.com,crl.globalsign.com,crl.identrust.com,crl.sectigo.com,crl.thawte.com,crl.usertrust.com,crl.verisign.com,crl3.digicert.com,crl4.digicert.com,crls.ssl.com,dev.azure.com,docs.github.com,github-cloud.githubusercontent.com,github-cloud.s3.amazonaws.com,github.blog,github.com,github.githubassets.com,helix.dot.net,host.docker.internal,json-schema.org,json.schemastore.org,keyserver.ubuntu.com,lfs.github.com,objects.githubusercontent.com,ocsp.digicert.com,ocsp.geotrust.com,ocsp.globalsign.com,ocsp.identrust.com,ocsp.sectigo.com,ocsp.ssl.com,ocsp.thawte.com,ocsp.usertrust.com,ocsp.verisign.com,packagecloud.io,packages.cloud.google.com,packages.microsoft.com,patch-diff.githubusercontent.com,ppa.launchpad.net,raw.githubusercontent.com,registry.npmjs.org,s.symcb.com,s.symcd.com,security.ubuntu.com,telemetry.enterprise.githubcopilot.com,ts-crl.ws.symantec.com,ts-ocsp.ws.symantec.com,www.googleapis.com" GITHUB_SERVER_URL: ${{ github.server_url }} GITHUB_API_URL: ${{ github.api_url }} - GH_AW_SAFE_OUTPUTS_HANDLER_CONFIG: "{\"add_comment\":{\"discussions\":false,\"max\":3,\"required_labels\":[\"agentic-workflows\"],\"required_title_prefix\":\"[ci-fix] \",\"target\":\"*\"},\"create_pull_request\":{\"allowed_base_branches\":[\"main\"],\"allowed_branches\":[\"ci-fix/**\"],\"allowed_files\":[\"src/Core/**\",\"src/Controls/**\",\"src/Essentials/**\",\"src/BlazorWebView/**\",\"src/TestUtils/**\",\"src/Templates/**\",\"**/PublicAPI.Unshipped.txt\"],\"base_branch\":\"main\",\"draft\":true,\"labels\":[\"agentic-workflows\"],\"max\":3,\"max_patch_files\":100,\"max_patch_size\":4096,\"protect_top_level_dot_folders\":true,\"protected_files\":[\"package.json\",\"bun.lockb\",\"bunfig.toml\",\"deno.json\",\"deno.jsonc\",\"deno.lock\",\"global.json\",\"NuGet.Config\",\"Directory.Packages.props\",\"mix.exs\",\"mix.lock\",\"go.mod\",\"go.sum\",\"stack.yaml\",\"stack.yaml.lock\",\"pom.xml\",\"build.gradle\",\"build.gradle.kts\",\"settings.gradle\",\"settings.gradle.kts\",\"gradle.properties\",\"package-lock.json\",\"yarn.lock\",\"pnpm-lock.yaml\",\"npm-shrinkwrap.json\",\"requirements.txt\",\"Pipfile\",\"Pipfile.lock\",\"pyproject.toml\",\"setup.py\",\"setup.cfg\",\"Gemfile\",\"Gemfile.lock\",\"uv.lock\",\"CODEOWNERS\",\"DESIGN.md\",\"README.md\",\"CONTRIBUTING.md\",\"CHANGELOG.md\",\"SECURITY.md\",\"CODE_OF_CONDUCT.md\",\"AGENTS.md\",\"CLAUDE.md\",\"GEMINI.md\"],\"protected_files_policy\":\"request_review\",\"title_prefix\":\"[ci-fix] \"},\"create_report_incomplete_issue\":{},\"missing_data\":{},\"missing_tool\":{},\"noop\":{\"max\":1,\"report-as-issue\":\"true\"},\"push_to_pull_request_branch\":{\"allowed_files\":[\"src/Core/**\",\"src/Controls/**\",\"src/Essentials/**\",\"src/BlazorWebView/**\",\"src/TestUtils/**\",\"src/Templates/**\",\"**/PublicAPI.Unshipped.txt\"],\"if_no_changes\":\"warn\",\"max\":3,\"max_patch_size\":4096,\"protect_top_level_dot_folders\":true,\"protected_files\":[\"package.json\",\"bun.lockb\",\"bunfig.toml\",\"deno.json\",\"deno.jsonc\",\"deno.lock\",\"global.json\",\"NuGet.Config\",\"Directory.Packages.props\",\"mix.exs\",\"mix.lock\",\"go.mod\",\"go.sum\",\"stack.yaml\",\"stack.yaml.lock\",\"pom.xml\",\"build.gradle\",\"build.gradle.kts\",\"settings.gradle\",\"settings.gradle.kts\",\"gradle.properties\",\"package-lock.json\",\"yarn.lock\",\"pnpm-lock.yaml\",\"npm-shrinkwrap.json\",\"requirements.txt\",\"Pipfile\",\"Pipfile.lock\",\"pyproject.toml\",\"setup.py\",\"setup.cfg\",\"Gemfile\",\"Gemfile.lock\",\"uv.lock\",\"CODEOWNERS\",\"DESIGN.md\",\"README.md\",\"CONTRIBUTING.md\",\"CHANGELOG.md\",\"SECURITY.md\",\"CODE_OF_CONDUCT.md\",\"AGENTS.md\",\"CLAUDE.md\",\"GEMINI.md\"],\"required_labels\":[\"agentic-workflows\"],\"target\":\"*\",\"title_prefix\":\"[ci-fix] \"},\"report_incomplete\":{},\"update_pull_request\":{\"allow_body\":true,\"allow_title\":false,\"max\":3,\"target\":\"*\",\"update_branch\":false}}" + GH_AW_SAFE_OUTPUTS_HANDLER_CONFIG: "{\"add_comment\":{\"discussions\":false,\"max\":6,\"required_labels\":[\"agentic-workflows\"],\"required_title_prefix\":\"[ci-fix] \",\"target\":\"*\"},\"add_labels\":{\"allowed\":[\"p/0\"],\"max\":3,\"required_labels\":[\"agentic-workflows\"],\"required_title_prefix\":\"[ci-fix] \",\"target\":\"*\"},\"create_pull_request\":{\"allowed_base_branches\":[\"main\"],\"allowed_branches\":[\"ci-fix/**\"],\"allowed_files\":[\"src/Core/**\",\"src/Controls/**\",\"src/Essentials/**\",\"src/BlazorWebView/**\",\"src/TestUtils/**\",\"src/Templates/**\",\"**/PublicAPI.Unshipped.txt\"],\"base_branch\":\"main\",\"draft\":true,\"labels\":[\"agentic-workflows\"],\"max\":3,\"max_patch_files\":100,\"max_patch_size\":4096,\"protect_top_level_dot_folders\":true,\"protected_files\":[\"package.json\",\"bun.lockb\",\"bunfig.toml\",\"deno.json\",\"deno.jsonc\",\"deno.lock\",\"global.json\",\"NuGet.Config\",\"Directory.Packages.props\",\"mix.exs\",\"mix.lock\",\"go.mod\",\"go.sum\",\"stack.yaml\",\"stack.yaml.lock\",\"pom.xml\",\"build.gradle\",\"build.gradle.kts\",\"settings.gradle\",\"settings.gradle.kts\",\"gradle.properties\",\"package-lock.json\",\"yarn.lock\",\"pnpm-lock.yaml\",\"npm-shrinkwrap.json\",\"requirements.txt\",\"Pipfile\",\"Pipfile.lock\",\"pyproject.toml\",\"setup.py\",\"setup.cfg\",\"Gemfile\",\"Gemfile.lock\",\"uv.lock\",\"CODEOWNERS\",\"DESIGN.md\",\"README.md\",\"CONTRIBUTING.md\",\"CHANGELOG.md\",\"SECURITY.md\",\"CODE_OF_CONDUCT.md\",\"AGENTS.md\",\"CLAUDE.md\",\"GEMINI.md\"],\"protected_files_policy\":\"request_review\",\"title_prefix\":\"[ci-fix] \"},\"create_report_incomplete_issue\":{},\"mark_pull_request_as_ready_for_review\":{\"max\":3,\"required_labels\":[\"agentic-workflows\"],\"required_title_prefix\":\"[ci-fix] \",\"target\":\"*\"},\"missing_data\":{},\"missing_tool\":{},\"noop\":{\"max\":1,\"report-as-issue\":\"true\"},\"push_to_pull_request_branch\":{\"allowed_files\":[\"src/Core/**\",\"src/Controls/**\",\"src/Essentials/**\",\"src/BlazorWebView/**\",\"src/TestUtils/**\",\"src/Templates/**\",\"**/PublicAPI.Unshipped.txt\"],\"if_no_changes\":\"warn\",\"max\":3,\"max_patch_size\":4096,\"protect_top_level_dot_folders\":true,\"protected_files\":[\"package.json\",\"bun.lockb\",\"bunfig.toml\",\"deno.json\",\"deno.jsonc\",\"deno.lock\",\"global.json\",\"NuGet.Config\",\"Directory.Packages.props\",\"mix.exs\",\"mix.lock\",\"go.mod\",\"go.sum\",\"stack.yaml\",\"stack.yaml.lock\",\"pom.xml\",\"build.gradle\",\"build.gradle.kts\",\"settings.gradle\",\"settings.gradle.kts\",\"gradle.properties\",\"package-lock.json\",\"yarn.lock\",\"pnpm-lock.yaml\",\"npm-shrinkwrap.json\",\"requirements.txt\",\"Pipfile\",\"Pipfile.lock\",\"pyproject.toml\",\"setup.py\",\"setup.cfg\",\"Gemfile\",\"Gemfile.lock\",\"uv.lock\",\"CODEOWNERS\",\"DESIGN.md\",\"README.md\",\"CONTRIBUTING.md\",\"CHANGELOG.md\",\"SECURITY.md\",\"CODE_OF_CONDUCT.md\",\"AGENTS.md\",\"CLAUDE.md\",\"GEMINI.md\"],\"required_labels\":[\"agentic-workflows\"],\"target\":\"*\",\"title_prefix\":\"[ci-fix] \"},\"report_incomplete\":{},\"update_pull_request\":{\"allow_body\":true,\"allow_title\":false,\"max\":3,\"target\":\"*\",\"update_branch\":false}}" GH_AW_CI_TRIGGER_TOKEN: ${{ secrets.GH_AW_CI_TRIGGER_TOKEN }} with: github-token: ${{ secrets.GH_AW_GITHUB_TOKEN || secrets.GITHUB_TOKEN }} diff --git a/.github/workflows/ci-status-fix.md b/.github/workflows/ci-status-fix.md index d3227e25170c..9c2f46054029 100644 --- a/.github/workflows/ci-status-fix.md +++ b/.github/workflows/ci-status-fix.md @@ -15,7 +15,9 @@ description: | humans (the open tracking issue is the hand-off surface; a dedicated [ci-fix][needs-human] PR is planned but currently deferred — see Step 6). CI on the PR is kicked by a human `/azp run` for now; the loop watches, classifies, - and re-fixes autonomously between kicks. + and re-fixes autonomously between kicks. When the SPECIFIC test a PR fixed is + confirmed green in that PR's own CI, the loop marks the draft PR ready for review + (a state transition only — it never approves or merges). Never mutes tests, but de-flakes genuinely flaky ones (deterministic synchronization, no retries / timeout bumps). Always skips visual-regression / screenshot issues. @@ -239,8 +241,15 @@ safe-outputs: # PER-RUN total (not per-PR): a sweep may surface several green PRs and/or # annotate several flaky reds in one run. At max:1 all but the first comment # were silently dropped, starving the workflow's #1 value (surfacing green PRs - # for review). Raised to 3 to match the per-run throughput of the other outputs. - max: 3 + # for review). Sized to 6 (not 3) because a single green DRAFT PR that gets + # flipped ready in one sweep spends TWO comment slots — the Step 3 ✅ surface + # comment (which still names the /azp-gated legs a human must kick, so it is NOT + # redundant with 🎯) AND the Step 3.6 T3 🎯 readiness comment. At max:3 the shared + # bucket drained after ~1 draft flip and T3's atomicity pre-check then deferred + # every further mark-ready even while the mark-ready/add_labels buckets (also 3) + # sat idle; 6 lets ~3 draft flips (2 comments each) land per sweep, so the + # mark-ready:3 / add_labels:3 caps are actually reachable. + max: 6 target: "*" # Hard constraint (defense-in-depth): only comment on THIS workflow's own # ci-fix PRs — [ci-fix] title prefix AND agentic-workflows label. Mirrors the @@ -261,13 +270,51 @@ safe-outputs: # never the title, so disable title rewrites — this compiles to allow_title:false # and removes any ability to retitle an arbitrary PR. title: false - # NOTE: gh-aw v0.79.8 does NOT emit required-title-prefix/required-labels into + # NOTE: gh-aw v0.80.9 (the pinned compiler) does NOT emit required-title-prefix/required-labels into # the compiled config for update-pull-request (verified against the lock — it # silently drops them, unlike add-comment / push-to-pull-request-branch which # honor them). So which-PR scoping here relies on prompt Hard-Rule 6 + # min-integrity:approved; the residual is body-marker edits only (no code, no # merge, no close), capped at max:3. Revisit required-* if a future gh-aw # compiler emits them for this output. + mark-pull-request-as-ready-for-review: + # Flip a validated draft [ci-fix] PR to ready-for-review once the SPECIFIC test + # this PR was opened to fix is confirmed green in the PR's OWN CI (Step 3.6) — + # even when unrelated legs are red. The workflow is schedule/dispatch-triggered + # (no triggering-PR context), so target "*" lets the agent name the PR number it + # validated. This is a state transition ONLY: it never approves and never merges — + # a human still reviews and merges. + # PER-RUN total (not per-PR): a sweep may validate several PRs' target tests green. + max: 3 + target: "*" + # Hard constraint (defense-in-depth): only ever un-draft THIS workflow's own + # ci-fix PRs — [ci-fix] title prefix AND agentic-workflows label — so a confused + # or prompt-injected agent cannot mark an arbitrary PR ready. (If the v0.80.9 + # compiler silently drops these — as documented for update-pull-request above — + # the Step 3.6 preconditions + min-integrity:approved are the compensating scope + # controls; verify against the lock after compiling.) + required-title-prefix: "[ci-fix] " + required-labels: [agentic-workflows] + add-labels: + # Apply the p/0 priority label to a [ci-fix] PR at the exact moment the loop flips it + # from draft to ready-for-review (Step 3.6 T3) — so a validated, review-ready fix lands + # in the team's p/0 triage queue instead of sitting unseen in the draft backlog. Paired + # 1:1 with mark-pull-request-as-ready-for-review; target "*" lets the agent name the PR + # it just validated. + # allowed = HARD allowlist: the agent may ONLY ever add p/0, nothing else. This caps the + # blast radius of a confused/prompt-injected agent to exactly one benign priority label — + # it can never apply a downstream-triggering or destructive label. + allowed: [p/0] + # PER-RUN total (not per-PR): a sweep may mark several PRs ready in one run; each adds + # one label, so this matches the mark-ready per-run cap (3). + max: 3 + target: "*" + # Hard constraint (defense-in-depth): only ever label THIS workflow's own ci-fix PRs — + # [ci-fix] title prefix AND agentic-workflows label. (If the v0.80.9 compiler drops these + # — as documented for update-pull-request above — the Step 3.6 preconditions + + # min-integrity:approved + the allowed:[p/0] allowlist are the compensating controls.) + required-title-prefix: "[ci-fix] " + required-labels: [agentic-workflows] timeout-minutes: 90 @@ -339,33 +386,48 @@ through `safe-outputs`. 5. **One issue = one outcome per run.** Exactly one of: respond to a maintainer's change-request on the open PR (Track C, Step 3.5.R); open the first fix/help/ de-flake PR; advance an existing PR by one attempt (push a follow-up fix); - surface a validated-green PR for review; annotate an unrelated-flake red; wait + surface a validated-green PR for review; mark a target-validated draft PR + ready-for-review (Step 3.6 T3 — the terminal outcome that supersedes the same + run's surface/annotate precursor line), or record its `already-ready` + steady-state no-op; annotate an unrelated-flake red; wait (CI not yet settled); or a recorded skip (the dedicated needs-human PR is deferred — the attempt cap records a skip instead; see Step 6). Always prefer advancing or opening a PR over a skip when a non-mute diff is producible. 6. **All writes via `safe-outputs`.** Allowed outputs: `create_pull_request` (first attempt only), `push_to_pull_request_branch` (advance an existing PR by one attempt), `update_pull_request` (bump the attempt marker / refresh the - prior-attempts table), and `add_comment` (progress notes on the `[ci-fix]` PR - ONLY). NEVER comment on the tracking issue (issues are locked by + prior-attempts table), `add_comment` (progress notes on the `[ci-fix]` PR + ONLY), `mark_pull_request_as_ready_for_review` (flip a target-validated draft + `[ci-fix]` PR from draft to ready — Step 3.6 T3 only), and `add_labels` (add + ONLY the `p/0` label, ONLY on that same draft→ready transition — Step 3.6 T3). + NEVER comment on the tracking issue (issues are locked by `.github/workflows/ci-scan-lock-issues.yml`) — `add_comment` targets the PR. No `gh pr create`, no manual `git push`. **Defense-in-depth:** `add_comment` and `update_pull_request` use `target: "*"` (the agent supplies the PR number). - `add_comment` is now config-locked to `required-title-prefix: "[ci-fix] "` + + `add_comment`, `mark_pull_request_as_ready_for_review`, and `add_labels` are + config-locked to `required-title-prefix: "[ci-fix] "` + `required-labels: [agentic-workflows]` (hard-enforced by the handler, same as - `push_to_pull_request_branch`), so a comment can only ever land on THIS - workflow's own PRs. `update_pull_request` CANNOT be config-locked to a - title/label in gh-aw v0.79.8 (the compiler silently drops `required-*` for that - output — verified against the lock), so it keeps `title: false` (no retitles; - body-marker edits only) plus this prompt-level guard. Before emitting either, - VERIFY the target PR carries BOTH the `[ci-fix]` title prefix AND the - `agentic-workflows` label; never comment on or edit the body of any PR that - lacks both. + `push_to_pull_request_branch`), and `add_labels` additionally has an + `allowed: [p/0]` allowlist so it can ONLY ever add `p/0` — so these can only + ever land on THIS workflow's own PRs. `update_pull_request` CANNOT be + config-locked to a title/label in gh-aw v0.80.9 (the compiler silently drops + `required-*` for that output — verified against the lock), so it keeps + `title: false` (no retitles; body-marker edits only) plus this prompt-level + guard. Before emitting ANY of these, VERIFY the target PR carries BOTH the + `[ci-fix]` title prefix AND the `agentic-workflows` label; never comment on, + edit, un-draft, or label any PR that lacks both. 7. **Per-run safe-output caps (per RUN, not per PR).** Each output type is capped per run: `create_pull_request` 3, `push_to_pull_request_branch` 3, - `add_comment` 3, `update_pull_request` 3. Note an ADVANCE spends one push **and** - one update **and** one comment, so ≤ 3 advances/run; surfacing a green or - annotating a flake spends one comment. When a bucket is exhausted, do NOT keep + `add_comment` 6, `update_pull_request` 3, `mark_pull_request_as_ready_for_review` + 3, `add_labels` 3 (counts LABELS, not calls). Note an ADVANCE spends one push + **and** one update **and** one comment, so ≤ 3 advances/run; surfacing a green or + annotating a flake spends one comment; a **mark-ready (Step 3.6 T3)** of a + still-draft PR spends TWO comments (the Step 3 ✅ surface **and** the T3 🎯 + readiness note) **and** one mark-ready **and** one label as an ALL-OR-NOTHING set + (T3 pre-checks those buckets and defers the whole PR if any is exhausted — never + mark a PR ready without its 🎯 audit comment). `add_comment` is therefore sized 6 + (not 3) so ~3 draft flips, at 2 comments each, can land in one sweep instead of the + shared comment bucket starving the otherwise-idle mark-ready/add_labels buckets. When a bucket is exhausted, do NOT keep emitting (extras are silently dropped) — record `skipped: per-run cap reached; deferring PR # to next cycle` for each remaining PR so the drop is a deliberate, logged decision. The continuous loop picks the deferred PRs up next @@ -409,8 +471,9 @@ For every open tracking issue in scope, converge on exactly one outcome: | Open first help-wanted draft `[ci-fix]` PR | No open PR yet; a plausible candidate exists but cannot be runner-validated (device/UI tests) (attempt 1) | | Open first de-flake draft `[ci-fix]` PR | No open PR yet; failure is intermittent (green-on-retry) from a genuine test-quality defect; a deterministic-synchronization fix is producible (attempt 1, Step 4.7 bucket b) | | Advance an existing PR (push attempt N+1) | Open PR's own CI settled red *because of the fix itself*, no human engaged, marker < 10 — push a NEW distinct fix onto the same branch (Step 3.5 → 5.6) | -| Surface a validated-green PR | Open PR's own CI settled green — comment "validated, attempt N/10; ready for review" and do NOT advance (Step 3.5) | -| Annotate an unrelated-flake red | Open PR is red only on baseline-flake / unrelated legs — comment which leg needs a re-run; do NOT burn an attempt (Step 3.5) | +| Surface a validated-green PR | Open PR's own CI settled green — comment "validated, attempt N/10; ready for review" and do NOT advance (Step 3.5), then mark the draft PR ready for review once the fixed test is confirmed green (Step 3.6) | +| Annotate an unrelated-flake red | Open PR is red only on baseline-flake / unrelated legs — comment which leg needs a re-run; do NOT burn an attempt (Step 3.5); if the SPECIFIC fixed test is confirmed green on that build, still mark the draft PR ready for review (Step 3.6) | +| Mark a validated PR ready for review | The specific test this PR fixed is confirmed green in the PR's own CI (even if unrelated legs are red) — comment the target-test result and transition the draft PR to ready for review; never approve or merge (Step 3.6) | | Wait | Open PR's CI is pending / not yet settled on the current head SHA — do nothing this cycle (Step 3.5) | | Hand-off skip (attempt cap) | Marker == 10 and the signature still reproduces — stop and defer to humans; the dedicated `[ci-fix][needs-human]` PR is planned but currently deferred (Step 6) | | Recorded skip | Visual-regression, human engaged, already-handled, fixed-in-latest-build, infra-flake, out-of-bounds, only-mute-available, or no novel approach producible | @@ -685,14 +748,34 @@ Run these gates in order — the FIRST that fires decides this cycle's outcome: 1. **Human engaged.** If `C.humanEngaged` → `skipped: human engaged on PR #; deferring` and stop. Never fight a human reviewer — the intentional hand-off boundary still holds the moment a person touches the PR. -2. **CI not settled / unknown / incomplete prefetch.** If `C.checksSettled == false` - OR `C.overallConclusion` is `pending`, `neutral`, or `unknown`, OR - `C.dataComplete == false` → the fix's CI has not finished on the current head SHA - (`C.headSha`), or the prefetch could not fully read this PR (a partial read can - understate `humanEngaged` / `attempt`). `skipped: PR # CI pending / prefetch - incomplete on ; waiting` and stop. *(Round 1: this is where a - maintainer `/azp run` is awaited — the `/azp`-gated uitests/devicetests legs will - not have run until a human kicks them.)* +2. **CI not settled — but validate the target test first (target-focused readiness).** + If `C.checksSettled == false` OR `C.overallConclusion` is `pending`, `neutral`, or + `unknown`, OR `C.dataComplete == false` → the *overall* build has not finished on the + current head SHA (`C.headSha`), or the prefetch could not fully read this PR (a partial + read can understate `humanEngaged` / `attempt`). Unrelated legs still draining must NOT + keep an already-proven fix parked as a draft, so before waiting, try the target-focused + fast-path: + - **2a — Target-green fast-path (draft `[ci-fix]` PRs only).** If `C.dataComplete == + true` AND the Step 3.6 preconditions hold (draft PR, `[ci-fix] ` title prefix AND the + `agentic-workflows` label), run Step 3.6 **T1–T2** against `C.headSha` now: identify + the target test(s) and read the PR's OWN build **timeline / per-leg status** (anonymous + `_apis/build` — never `_apis/test`, per Hard-Rule 8). If EVERY target test is + **VALIDATED-GREEN** (per T2's all-platform definition below — the target's category leg + is `succeeded` on every platform that runs it, failed on none, and NO target platform + family the pipeline covers is still pending/unconcluded, per T2's "no platform left + unverified" rule; a platform that simply has no leg for the target's category — the test + doesn't run there — is fine, not a gap) AND every leg in + `C.failedLegs` that has ALREADY concluded classifies as **unrelated flake** (Step 4 / + Step 4.7 method — the fix is implicated in NO completed red), then the fix is proven + regardless of unrelated *pending* legs → run **Step 3.6 T3** (mark ready + 🎯 comment) + and stop. This early-out NEVER advances an attempt and NEVER acts on a red that could + be the fix's fault. + - **2b — Otherwise WAIT.** If the target test has not yet executed (pending/absent on + its leg), or a completed red is (or may be) caused by the fix, or `C.dataComplete == + false`, or this PR has no identifiable target test → `skipped: PR # CI pending / + target not yet validated on ; waiting` and stop. *(Round 1: this is where a + maintainer `/azp run` is awaited — the `/azp`-gated uitests/devicetests legs will not + have run until a human kicks them.)* 3. **Green → surface for review.** If `C.overallConclusion == "success"`: the checks that RAN are green. **Primary-gate check first:** the deterministic `success` verdict only certifies "at least one green check, nothing failing or @@ -704,18 +787,31 @@ Run these gates in order — the FIRST that fires decides this cycle's outcome: # primary CI gate (maui-pr) not green on ; waiting` and stop — do NOT surface. (The `/azp`-gated `maui-pr-uitests` (def 313) / `maui-pr-devicetests` (def 314) legs MAY still be un-run — that is expected and is named below, not a - reason to withhold the surface.) **Idempotency:** scan the PR's existing comments - for a prior bot `✅ … validated … on ` note for THIS head SHA — if one - already exists, record `already-surfaced PR # (head )` and stop - WITHOUT re-commenting (re-surfacing the same green every tick is noise and burns - the per-run comment budget other PRs need). Otherwise resolve the attempt number from - `C.effectiveAttempt` (the authoritative max(marker, bot-commit) counter; omit the - number only if it is somehow indeterminate). **Dry-run gate (Step 0):** if `dry_run == "true"`, do NOT emit any comment — instead print the intended `✅` validated-green body to the run log, tally `dry-run: would-surface-green PR #`, and stop. Otherwise `add_comment` on PR #: `✅ Attempt /10 validated - — the fix's CI is green on . Ready for human review.` - Name any `/azp`-gated legs (uitests def 313 / devicetests def 314) that have not - run and still need a maintainer `/azp run`. Do NOT advance. Record - `surfaced-green PR # (attempt /10)` and stop. *(This directly - attacks the real bottleneck — no reviews — so it is the highest-value outcome.)* + reason to withhold the surface.) **Comment idempotency + dry-run suppress the ✅ + comment ONLY — neither skips Step 3.6.** Scan the PR's existing comments for a prior + bot `✅ … validated … on ` note for THIS head SHA; if one exists, set + `SKIP_SURFACE_COMMENT = true`. Resolve the attempt number from `C.effectiveAttempt` + (the authoritative max(marker, bot-commit) counter; omit the number only if it is + somehow indeterminate). Then post the surface comment UNLESS suppressed: + - if `dry_run == "true"`: do NOT emit — print the intended `✅` validated-green body to + the run log and tally `dry-run: would-surface-green PR #`; + - else if `SKIP_SURFACE_COMMENT`: do NOT re-post — record `already-surfaced PR # + (head )` (re-surfacing the same green every tick is noise and burns the + per-run comment budget other PRs need); + - else `add_comment` on PR #: `✅ Attempt /10 validated — the fix's CI is + green on .` naming any `/azp`-gated legs (uitests def 313 / devicetests def + 314) that have not run and still need a maintainer `/azp run`; record `surfaced-green + PR # (attempt /10)`. (Do NOT assert "ready for human review" on a PR that + is still a draft — Step 3.6, not this comment, owns the draft→ready flip and posts its + own 🎯 announcement when it fires.) + Do NOT advance. Then — in ALL of the above cases — run **Step 3.6** (target-test + readiness gate) for this PR before stopping: its `/azp`-gated target legs may conclude + green on this SAME head SHA on a LATER sweep (an `/azp run` adds no commit), and Step + 3.6 is the ONLY place the draft→ready flip happens, so it MUST re-evaluate every sweep — + never `stop` here before it. *(This directly attacks the real bottleneck — no reviews — + so it is the highest-value outcome.)* 4. **Red → classify caused-by-fix vs unrelated-flake.** If `C.overallConclusion == "failure"`, analyze the PR's OWN failing build (NOT `main`): find the AzDO `maui-pr` build for this PR (filter builds by `branchName=refs/pull//merge`, @@ -723,18 +819,26 @@ Run these gates in order — the FIRST that fires decides this cycle's outcome: method and Step 4.7 flake buckets to `C.failedLegs`. - **Unrelated flake only** (every failed leg is known-flaky / infra / a pre-existing baseline red NOT introduced by the fix): do NOT burn an attempt. - **Idempotency first:** scan the PR's existing comments for a prior bot - `♻️ … unrelated flake … on ` note for THIS head SHA — if one already - exists, record `already-annotated-flake PR # (head )` and stop - WITHOUT re-commenting (re-annotating the same flake every 12h sweep is noise and - burns the per-run comment budget other PRs need). Otherwise resolve the attempt - number from `C.effectiveAttempt` (the authoritative max(marker, bot-commit) - counter), omitting the `Attempt /10:` prefix only if it is somehow - indeterminate. **Dry-run gate (Step 0):** if `dry_run == "true"`, do NOT emit any comment — instead print the intended `♻️` unrelated-flake body to the run log, tally `dry-run: would-annotate-flake PR #`, and stop. Otherwise `add_comment` on PR #: `♻️ Attempt /10: red is - unrelated flake on leg(s) () on ; the fix itself is not - implicated. A maintainer re-run (/azp run ) should clear it.` Record - `annotated-flake PR # (head )` and stop. *(Round 1: human re-runs; - Round 2: auto re-trigger.)* + **Comment idempotency + dry-run suppress the ♻️ comment ONLY — neither skips Step + 3.6.** Scan the PR's existing comments for a prior bot `♻️ … unrelated flake … on + ` note for THIS head SHA; if one exists, set `SKIP_FLAKE_COMMENT = true`. + Resolve the attempt number from `C.effectiveAttempt` (the authoritative max(marker, + bot-commit) counter), omitting the `Attempt /10:` prefix only if it is + somehow indeterminate. Then post the flake note UNLESS suppressed: + - if `dry_run == "true"`: do NOT emit — print the intended `♻️` unrelated-flake body + to the run log and tally `dry-run: would-annotate-flake PR #`; + - else if `SKIP_FLAKE_COMMENT`: do NOT re-post — record `already-annotated-flake PR + # (head )` (re-annotating the same flake every 12h sweep is noise and + burns the per-run comment budget other PRs need); + - else `add_comment` on PR #: `♻️ Attempt /10: red is unrelated flake on + leg(s) () on ; the fix itself is not implicated. A + maintainer re-run (/azp run ) should clear it.` record `annotated-flake + PR # (head )`. + Then — in ALL of the above cases — run **Step 3.6** (target-test readiness gate) for + this PR before stopping: its `/azp`-gated target legs may conclude green on this SAME + head SHA on a LATER sweep, and Step 3.6 is the ONLY place the draft→ready flip + happens, so it MUST re-evaluate every sweep — never `stop` here before it. *(Round 1: + human re-runs; Round 2: auto re-trigger.)* - **Caused by the fix** (a failed leg still matches the original target signature, or the fix introduced a NEW failure): advance an attempt. - **Attempt count.** `attempt = C.effectiveAttempt` — the authoritative @@ -752,6 +856,188 @@ Run these gates in order — the FIRST that fires decides this cycle's outcome: from every prior commit on this branch**, emitted via the Step 5.6 ADVANCE path (push onto the same PR). +#### Step 3.6 — Target-test verification & mark-ready gate + +Reached from the Step 3 **green-surface** branch, the Step 4 **unrelated-flake** branch +(AFTER that branch has posted its comment), and the Step 3.5 **gate-2 target-focused +fast-path** (2a — while unrelated legs are still pending). Purpose: confirm the SPECIFIC +test(s) this PR was opened to fix now PASS in the PR's own CI, and — only when they do +— transition the draft `[ci-fix]` PR to **ready for review** so a maintainer sees a +validated fix instead of a draft. This is the ONLY place the loop flips draft→ready; it +is a state transition, **never** an approval or a merge (a human still reviews and +merges). Overall red on *unrelated* legs must NOT gate readiness — we validate the fix, +not the base branch's flakiness. + +**Preconditions** (ALL must hold; otherwise record `skipped: readiness N/A PR # +()` and stop this gate): +- `C.isDraft == true` — if the PR is already ready-for-review, the draft→ready + transition is already done; do NOT re-mark **and do NOT re-apply `p/0`**. `p/0` is applied + exactly ONCE, atomically with the draft→ready flip (T3 — all-or-nothing with the 🎯 audit + comment + mark-ready), so an already-ready loop PR that is MISSING `p/0` has almost + certainly had it **removed by a maintainer** de-prioritizing the PR — a pure triage action + the human-engagement guard (which inspects comments/reviews/commits, NOT label changes) + cannot see. Re-adding `p/0` every sweep would fight that maintainer indefinitely, violating + the loop's "never override a human" contract. So the loop does NOT reconcile the label: + record `already-ready PR #` and stop this gate. (Trade-off: on the rare occasion a + transient API error drops `p/0` at flip time *after* the T3 atomic pre-check passed, it is + not auto-re-added — but the PR is still ready-for-review with its 🎯 audit comment, and + re-adding `p/0` is a trivial manual action; that is strictly preferable to steam-rolling a + maintainer's deliberate de-prioritization.) +- The PR is unmistakably THIS workflow's own: `[ci-fix] ` title prefix AND the + `agentic-workflows` label (mirrors the safe-output lock; the compensating scope + control if the v0.80.9 compiler drops the declarative required-*). +- You reached this gate from Step 3 (green), Step 4 (**unrelated-flake**), or the Step 3.5 + gate-2 **target-focused fast-path** (2a) — i.e. the fix is NOT implicated in any red that + has concluded. If Step 4 classified a red as **caused by the fix** (ADVANCE), do NOT run + this gate — advance the attempt instead. + +**T1 — Identify the target test(s).** From the `[ci-scan]` issue signature the fix +addresses (and the PR's own diff), extract the fully-qualified test method name(s) the +fix targets — e.g. `SafeAreaShouldWorkOnAllShellTabs`. For a de-flake it is the +de-flaked test; for a product fix it is the originally-failing test(s). If NO specific +test can be identified (e.g. a product build-break rather than a test failure), this is a +**build-only fix** — there is no single test to validate cross-platform, so validate at +**whole-build** granularity rather than stopping (Step 3 only *comments*; Step 3.6 is the +sole draft→ready flip, so a build-only fix is undrafted HERE or not at all). We only reach +this gate from Step 3 (green) / Step 4 (unrelated-flake) / gate-2a, so the fix is not +implicated in any concluded red; additionally require, via the T2 timeline method on +`C.headSha`, that the primary build pipeline `maui-pr` (def 302) has CONCLUDED with EVERY +one of its platform build legs `succeeded`/`completed` and NONE failed/canceled or still +pending — the build is green on **every** platform, not just the originally-broken one (the +cross-platform guard, applied to the build instead of a test). A build-only fix is undrafted +ONLY on the strength of the auto-running `maui-pr` (def 302) whole-build green, which the +loop CAN observe: if the PR's `[ci-scan]` issue signature or its own diff indicates the +ORIGINATING failure was in a `/azp`-gated pipeline (`maui-pr-uitests` def 313 or +`maui-pr-devicetests` def 314) rather than `maui-pr` (def 302), do NOT undraft on +`maui-pr`-green alone — those pipelines do not auto-run on this PR (GITHUB_TOKEN cannot +trigger them), so a green `maui-pr` build is NOT evidence the gated build break is fixed; +record `skipped: build-only fix PR # targets gated pipeline () not run — +deferring to human` and stop this gate. Otherwise, set `TARGET := "the maui-pr build +(build-only fix — no single target test)"` and proceed to **T3** to mark ready. If any `maui-pr` build leg is still unconcluded, record `skipped: build-only fix PR +# not yet whole-build green (leg(s) pending)` and stop this gate WITHOUT marking +ready (a green subset is not enough — a still-pending build leg could yet fail). + +**T2 — Verify the target test's platform legs are green (anonymous, leg-level).** Per-test +outcomes come from the AzDO **test-results** API (`_apis/test/...`), which is **NOT reachable +anonymously — Hard-Rule 8**: those endpoints 302-redirect to a sign-in page on this runner, so +a `curl` returns an HTML redirect (not JSON) and every target test would look "not executed". +Validate instead at **leg granularity** using ONLY the anonymous `_apis/build/...` timeline +(Hard-Rule 8) — a `succeeded` category leg means every test in that category **that actually +ran** passed on that platform. This is a strong bar **only if the target test genuinely +executes**: a job can go green while one specific test is skipped at runtime (`[Ignore]`, +`Assert.Ignore()`, `Assert.Inconclusive()`, a `Skip=`/conditional `[Fact]`, an `#if`-out, or +an early `return` before the asserts). So before trusting a green leg as proof the target +passed, **confirm from the PR's OWN diff that the fix does NOT skip, ignore, disable, or +short-circuit the target test** (it adds no `[Ignore]`/`[Explicit]`, `Assert.Ignore`/ +`Assert.Inconclusive`, `Skip=`, category-exclusion, `#if`-out, or early `return` around the +target). If the fix could cause the target to be runtime-skipped rather than genuinely pass, +leg-level green is NOT sufficient evidence — record `skipped: target test may be +runtime-skipped by this fix (diff adds ); leg-green insufficient, deferring to human +on PR #` and stop this gate WITHOUT marking ready. Otherwise (the target genuinely runs +and asserts, as a de-flake or product fix does) a green category leg cannot hide a +target-test failure, so treat it as authoritative. + +Map the target test to the CI leg(s) that run it. A leg name encodes platform + test category +(e.g. `Android UITests SafeAreaEdges,Shadow`, `iOS UITests SafeAreaEdges,Shadow`, `macOS +UITests SafeAreaEdges,Shadow`, `Windows UITests SafeAreaEdges,Shadow`). Determine the target +test's UI-test category — its `[Category(UITestCategories.X)]` in the test/HostApp source, +visible in the PR diff or the test file — to know which leg-name substring identifies its legs; +for a device test, the per-platform device-test legs. + +Using the SAME build-discovery as Step 4 (filter AzDO builds by `branchName=refs/pull//merge` +or `sourceVersion == C.headSha`), read each build's **timeline** on `C.headSha` for the +pipeline(s) that RUN the target test — `maui-pr` (def 302) for unit/integration tests, +`maui-pr-uitests` (def 313) for Appium UI tests, `maui-pr-devicetests` (def 314) for device +tests: + +```bash +ORG=dnceng-public; PROJ=public; BUILD= +# Anonymous + Hard-Rule-8-compliant. NEVER call _apis/test/... (it 302-redirects to sign-in). +# Each type=="Job" record is one platform leg: result (succeeded/failed/canceled/null), +# state (completed/inProgress/pending), name (encodes " UITests "): +curl -s "https://dev.azure.com/$ORG/$PROJ/_apis/build/builds/$BUILD/timeline?api-version=7.1" \ + | tee /tmp/gh-aw/agent/timeline_${BUILD}.json \ + | jq -r '.records[] | select(.type=="Job") | "\(.result)\t\(.state)\t\(.name)"' +``` + +(The prefetch already exposes per-leg status in `C.failedLegs` / the checks context, and +`gh pr checks ` lists the same per-leg rows with platform+category in the name — use +whichever is handy; the build timeline is the authoritative cross-check on the exact build.) + +Treat a target test as **VALIDATED-GREEN** only if, on `C.headSha`, the leg that runs its +category is green on **every platform it runs on** — a fix that repairs one platform must not +silently regress the same test's leg on another, so a green leg on the originally-red platform +alone is NOT enough. Across the target pipeline's platform legs (for a UI test: the Android, +iOS, Windows, and macOS/MacCatalyst legs running the target's category; for a device test: each +device-test platform), require ALL of: +- the target's category leg is `result == "succeeded"` **and** `state == "completed"` on + **every** platform that runs it, AND +- that leg is `failed` / `canceled` / aborted on **NO** platform, AND +- **no platform is left unverified.** For each platform family the target pipeline covers, that + platform's category leg must have CONCLUDED (`state == "completed"`) on `C.headSha`. If a + platform simply has no leg for the target's category, the test does not run there — that is + fine, not a gap. But if a platform's category leg has **not concluded** (`state` is + `inProgress` / pending, or an `/azp`-gated `maui-pr-uitests` / `maui-pr-devicetests` leg that + has not been kicked), the test's status on that platform is UNKNOWN → the fix is NOT yet + validated across platforms: record `skipped: target test green on but + not yet verified on (leg(s) pending / need /azp run) on PR #` + and stop this gate WITHOUT marking ready. + +A target test whose category leg never concluded on ANY platform (**not executed** anywhere — +e.g. its `/azp`-gated pipeline has not been kicked) is likewise NOT validated: record `skipped: +target test not yet executed on PR # ( not run — needs /azp run)` and stop +this gate WITHOUT marking ready. Do NOT overclaim — a green *sibling* leg (a different category +on the same platform) is not the target's leg, and a green leg on one platform is not a pass on +the others. + +**T3 — Mark ready + report.** If EVERY target test is VALIDATED-GREEN. The 🎯 comment, +the mark-ready, and the `p/0` label are THREE SEPARATE safe-outputs — the comment +existing does NOT prove the mark-ready took effect, so they are tracked independently: + +- **Comment idempotency (dup-suppress only — never gates the mark-ready).** We only reach + T3 while `C.isDraft == true` (the precondition stops an already-ready PR before here). So + if a prior bot comment for THIS head that carries the leading `🎯` anchor AND the + contiguous phrase `validated green on ` already exists (BOTH T3 variants — + `🎯 Target test validated green on ` and the build-only `🎯 Build validated green on + ` — contain that exact contiguous phrase, so this recognizes both; keying on + `target test` alone would instead miss the build-only comment and re-post it every sweep. + **Require the `🎯` anchor** so the match can NEVER be loosened to a gapped + `validated…green…on` form that would false-positive on the Step 3 `✅ … validated — the + fix's CI is green on ` surface comment and wrongly suppress the audit 🎯), the comment + landed on an earlier sweep but the **mark-ready did NOT take + effect** (the PR is still a draft) — set `SUPPRESS_COMMENT = true` (do not re-post the + duplicate 🎯 comment) but STILL complete the mark-ready + label below. Never treat the + comment's existence as "already marked ready" while the PR is still a draft. Record this + case as `re-marking PR # (🎯 present; mark-ready did not take on a prior sweep)`. +- **Atomicity budget pre-check (Hard-Rule 7).** Determine the outputs this gate will emit: + `mark_pull_request_as_ready_for_review` + `add_labels` always, plus `add_comment` unless + `SUPPRESS_COMMENT`. Verify a free per-run slot remains in EACH bucket you are about to + use. If ANY required bucket is exhausted, emit NONE of them — record `skipped: per-run + cap reached; deferring mark-ready PR # to next cycle` and stop this gate. Never mark a + PR ready (or label it) unless its 🎯 audit comment is GUARANTEED to exist for this exact + head SHA — either posted in this same sweep, or (in the `SUPPRESS_COMMENT` re-marking case) + already present from a prior sweep. The outputs you DO emit either all land or all defer + together; the audit trail must never be absent, but it is NOT re-posted when it already exists. +- **Dry-run gate (Step 0):** if `dry_run == "true"`, emit NOTHING — print the intended + readiness comment and "would mark ready + add p/0" to the run log, tally `dry-run: + would-mark-ready PR #`, and stop. +- Otherwise emit for THIS PR number ``: + 1. `add_comment` (ONLY if not `SUPPRESS_COMMENT`): `🎯 Target test validated green on + — passed on ALL platforms it runs on (, + buildId ). — not + caused by this fix."> Transitioning this PR from draft to ready for review and adding + `p/0`; a maintainer still reviews and merges.` (For a **build-only fix** — the T1 + whole-build fallback, no single target test — phrase the first clause as `🎯 Build + validated green on — the maui-pr build passed on ALL platforms (buildId + ).` instead of naming a test.) + 2. `mark_pull_request_as_ready_for_review` with `reason:` a one-line justification + naming the validated test(s) and ``. + 3. `add_labels` with `labels: ["p/0"]` for PR # — put the now-review-ready fix into + the team's p/0 priority queue so it is triaged, not lost in the draft backlog. (If + the PR somehow already carries `p/0`, this is a harmless no-op.) +- Record `marked-ready PR # (target green on ALL platforms on , + labeled p/0)` and stop. + #### Step 3.5.R — Maintainer change-request response (Track C) Reached from Step 3.5 gate 0. Goal: when an **eligible human reviewer** (Hard-Rule @@ -1383,7 +1669,19 @@ Filed by [`ci-status-fix`](https://github.com/dotnet/maui/blob/main/.github/work ### Step 8 — Per-issue tally + end-of-run summary -Per issue, append one outcome line to `/tmp/gh-aw/agent/coverage.txt`: +Per issue, append **one** outcome line to `/tmp/gh-aw/agent/coverage.txt` — the +terminal outcome for the cycle. When one cycle produces a chained pair — a PR gets a +same-cycle precursor line and **then** a Step 3.6 readiness line (`marked-ready` on the +draft→ready flip, or `already-ready` on a later steady-state sweep of an already-ready +PR) — record ONLY the terminal Step 3.6 readiness line: it supersedes ANY same-cycle +non-readiness precursor for that PR, so an aggregator keying on one-line-per-issue never +double-counts. The precursor may be a Step 3 green line (`surfaced-green` on the first ✅, +or `already-surfaced` when it already exists) OR — when an unrelated-flake red and a +target-green coincide — a Step 4 `annotated-flake` line; either way the readiness line is +terminal, and the superseded precursor's signal survives in the PR's 🎯/♻️ comment so no +information is lost. This covers the flip cycle (`surfaced-green` → `marked-ready`), the +post-flip steady state (`already-surfaced` → `already-ready`), and the flake-coincident +flip (`annotated-flake` → `marked-ready`): ``` # main attempt- @@ -1394,8 +1692,14 @@ Per issue, append one outcome line to `/tmp/gh-aw/agent/coverage.txt`: `advance-PR # attempt /10` (ADVANCE mode — new commit pushed to the existing PR), `surfaced-green PR #` (fix's own CI went green; commented for review, did not advance), `annotated-flake PR #` (red was unrelated flake; -commented, attempt NOT burned), `waiting PR #` (CI not settled yet), -`dry-run: would-`, `skipped: `. (The +commented, attempt NOT burned), `marked-ready PR #` (the specific fixed test +was confirmed green in the PR's own CI; draft flipped to ready for review), +`already-surfaced PR #` (the fix's ✅ green comment already existed this sweep; +re-post suppressed — superseded by the Step 3.6 readiness line when the PR also flips +ready that cycle), `already-ready PR #` (terminal steady state — the PR was flipped +ready on a prior cycle and remains green on an unchanged head; no action taken, +supersedes `already-surfaced`), `waiting PR #` (CI not settled yet), +`dry-run: would-`, `skipped: `. (The `needs-human-PR` outcome is reserved for the deferred hand-off — Step 6 currently records a skip instead, so it is not emitted.)
(attempt /10)` and stop. *(This directly - attacks the real bottleneck — no reviews — so it is the highest-value outcome.)* + reason to withhold the surface.) **Comment idempotency + dry-run suppress the ✅ + comment ONLY — neither skips Step 3.6.** Scan the PR's existing comments for a prior + bot `✅ … validated … on ` note for THIS head SHA; if one exists, set + `SKIP_SURFACE_COMMENT = true`. Resolve the attempt number from `C.effectiveAttempt` + (the authoritative max(marker, bot-commit) counter; omit the number only if it is + somehow indeterminate). Then post the surface comment UNLESS suppressed: + - if `dry_run == "true"`: do NOT emit — print the intended `✅` validated-green body to + the run log and tally `dry-run: would-surface-green PR #`; + - else if `SKIP_SURFACE_COMMENT`: do NOT re-post — record `already-surfaced PR # + (head )` (re-surfacing the same green every tick is noise and burns the + per-run comment budget other PRs need); + - else `add_comment` on PR #: `✅ Attempt /10 validated — the fix's CI is + green on .` naming any `/azp`-gated legs (uitests def 313 / devicetests def + 314) that have not run and still need a maintainer `/azp run`; record `surfaced-green + PR # (attempt /10)`. (Do NOT assert "ready for human review" on a PR that + is still a draft — Step 3.6, not this comment, owns the draft→ready flip and posts its + own 🎯 announcement when it fires.) + Do NOT advance. Then — in ALL of the above cases — run **Step 3.6** (target-test + readiness gate) for this PR before stopping: its `/azp`-gated target legs may conclude + green on this SAME head SHA on a LATER sweep (an `/azp run` adds no commit), and Step + 3.6 is the ONLY place the draft→ready flip happens, so it MUST re-evaluate every sweep — + never `stop` here before it. *(This directly attacks the real bottleneck — no reviews — + so it is the highest-value outcome.)* 4. **Red → classify caused-by-fix vs unrelated-flake.** If `C.overallConclusion == "failure"`, analyze the PR's OWN failing build (NOT `net11.0`): find the AzDO `maui-pr` build for this PR (filter builds by `branchName=refs/pull//merge`, @@ -733,18 +829,26 @@ Run these gates in order — the FIRST that fires decides this cycle's outcome: method and Step 4.7 flake buckets to `C.failedLegs`. - **Unrelated flake only** (every failed leg is known-flaky / infra / a pre-existing baseline red NOT introduced by the fix): do NOT burn an attempt. - **Idempotency first:** scan the PR's existing comments for a prior bot - `♻️ … unrelated flake … on ` note for THIS head SHA — if one already - exists, record `already-annotated-flake PR # (head )` and stop - WITHOUT re-commenting (re-annotating the same flake every 12h sweep is noise and - burns the per-run comment budget other PRs need). Otherwise resolve the attempt - number from `C.effectiveAttempt` (the authoritative max(marker, bot-commit) - counter), omitting the `Attempt /10:` prefix only if it is somehow - indeterminate. **Dry-run gate (Step 0):** if `dry_run == "true"`, do NOT emit any comment — instead print the intended `♻️` unrelated-flake body to the run log, tally `dry-run: would-annotate-flake PR #`, and stop. Otherwise `add_comment` on PR #: `♻️ Attempt /10: red is - unrelated flake on leg(s) () on ; the fix itself is not - implicated. A maintainer re-run (/azp run ) should clear it.` Record - `annotated-flake PR # (head )` and stop. *(Round 1: human re-runs; - Round 2: auto re-trigger.)* + **Comment idempotency + dry-run suppress the ♻️ comment ONLY — neither skips Step + 3.6.** Scan the PR's existing comments for a prior bot `♻️ … unrelated flake … on + ` note for THIS head SHA; if one exists, set `SKIP_FLAKE_COMMENT = true`. + Resolve the attempt number from `C.effectiveAttempt` (the authoritative max(marker, + bot-commit) counter), omitting the `Attempt /10:` prefix only if it is + somehow indeterminate. Then post the flake note UNLESS suppressed: + - if `dry_run == "true"`: do NOT emit — print the intended `♻️` unrelated-flake body + to the run log and tally `dry-run: would-annotate-flake PR #`; + - else if `SKIP_FLAKE_COMMENT`: do NOT re-post — record `already-annotated-flake PR + # (head )` (re-annotating the same flake every 12h sweep is noise and + burns the per-run comment budget other PRs need); + - else `add_comment` on PR #: `♻️ Attempt /10: red is unrelated flake on + leg(s) () on ; the fix itself is not implicated. A + maintainer re-run (/azp run ) should clear it.` record `annotated-flake + PR # (head )`. + Then — in ALL of the above cases — run **Step 3.6** (target-test readiness gate) for + this PR before stopping: its `/azp`-gated target legs may conclude green on this SAME + head SHA on a LATER sweep, and Step 3.6 is the ONLY place the draft→ready flip + happens, so it MUST re-evaluate every sweep — never `stop` here before it. *(Round 1: + human re-runs; Round 2: auto re-trigger.)* - **Caused by the fix** (a failed leg still matches the original target signature, or the fix introduced a NEW failure): advance an attempt. - **Attempt count.** `attempt = C.effectiveAttempt` — the authoritative @@ -762,6 +866,188 @@ Run these gates in order — the FIRST that fires decides this cycle's outcome: from every prior commit on this branch**, emitted via the Step 5.6 ADVANCE path (push onto the same PR). +#### Step 3.6 — Target-test verification & mark-ready gate + +Reached from the Step 3 **green-surface** branch, the Step 4 **unrelated-flake** branch +(AFTER that branch has posted its comment), and the Step 3.5 **gate-2 target-focused +fast-path** (2a — while unrelated legs are still pending). Purpose: confirm the SPECIFIC +test(s) this PR was opened to fix now PASS in the PR's own CI, and — only when they do +— transition the draft `[ci-fix-net11]` PR to **ready for review** so a maintainer sees +a validated fix instead of a draft. This is the ONLY place the loop flips draft→ready; +it is a state transition, **never** an approval or a merge (a human still reviews and +merges). Overall red on *unrelated* legs must NOT gate readiness — we validate the fix, +not the base branch's flakiness. + +**Preconditions** (ALL must hold; otherwise record `skipped: readiness N/A PR # +()` and stop this gate): +- `C.isDraft == true` — if the PR is already ready-for-review, the draft→ready + transition is already done; do NOT re-mark **and do NOT re-apply `p/0`**. `p/0` is applied + exactly ONCE, atomically with the draft→ready flip (T3 — all-or-nothing with the 🎯 audit + comment + mark-ready), so an already-ready loop PR that is MISSING `p/0` has almost + certainly had it **removed by a maintainer** de-prioritizing the PR — a pure triage action + the human-engagement guard (which inspects comments/reviews/commits, NOT label changes) + cannot see. Re-adding `p/0` every sweep would fight that maintainer indefinitely, violating + the loop's "never override a human" contract. So the loop does NOT reconcile the label: + record `already-ready PR #` and stop this gate. (Trade-off: on the rare occasion a + transient API error drops `p/0` at flip time *after* the T3 atomic pre-check passed, it is + not auto-re-added — but the PR is still ready-for-review with its 🎯 audit comment, and + re-adding `p/0` is a trivial manual action; that is strictly preferable to steam-rolling a + maintainer's deliberate de-prioritization.) +- The PR is unmistakably THIS workflow's own: `[ci-fix-net11] ` title prefix AND the + `agentic-workflows` label (mirrors the safe-output lock; the compensating scope + control if the v0.80.9 compiler drops the declarative required-*). +- You reached this gate from Step 3 (green), Step 4 (**unrelated-flake**), or the Step 3.5 + gate-2 **target-focused fast-path** (2a) — i.e. the fix is NOT implicated in any red that + has concluded. If Step 4 classified a red as **caused by the fix** (ADVANCE), do NOT run + this gate — advance the attempt instead. + +**T1 — Identify the target test(s).** From the `[ci-scan-net11]` issue signature the fix +addresses (and the PR's own diff), extract the fully-qualified test method name(s) the +fix targets — e.g. `SafeAreaShouldWorkOnAllShellTabs`. For a de-flake it is the +de-flaked test; for a product fix it is the originally-failing test(s). If NO specific +test can be identified (e.g. a product build-break rather than a test failure), this is a +**build-only fix** — there is no single test to validate cross-platform, so validate at +**whole-build** granularity rather than stopping (Step 3 only *comments*; Step 3.6 is the +sole draft→ready flip, so a build-only fix is undrafted HERE or not at all). We only reach +this gate from Step 3 (green) / Step 4 (unrelated-flake) / gate-2a, so the fix is not +implicated in any concluded red; additionally require, via the T2 timeline method on +`C.headSha`, that the primary build pipeline `maui-pr` (def 302) has CONCLUDED with EVERY +one of its platform build legs `succeeded`/`completed` and NONE failed/canceled or still +pending — the build is green on **every** platform, not just the originally-broken one (the +cross-platform guard, applied to the build instead of a test). A build-only fix is undrafted +ONLY on the strength of the auto-running `maui-pr` (def 302) whole-build green, which the +loop CAN observe: if the PR's `[ci-scan]` issue signature or its own diff indicates the +ORIGINATING failure was in a `/azp`-gated pipeline (`maui-pr-uitests` def 313 or +`maui-pr-devicetests` def 314) rather than `maui-pr` (def 302), do NOT undraft on +`maui-pr`-green alone — those pipelines do not auto-run on this PR (GITHUB_TOKEN cannot +trigger them), so a green `maui-pr` build is NOT evidence the gated build break is fixed; +record `skipped: build-only fix PR # targets gated pipeline () not run — +deferring to human` and stop this gate. Otherwise, set `TARGET := "the maui-pr build +(build-only fix — no single target test)"` and proceed to **T3** to mark ready. If any `maui-pr` build leg is still unconcluded, record `skipped: build-only fix PR +# not yet whole-build green (leg(s) pending)` and stop this gate WITHOUT marking +ready (a green subset is not enough — a still-pending build leg could yet fail). + +**T2 — Verify the target test's platform legs are green (anonymous, leg-level).** Per-test +outcomes come from the AzDO **test-results** API (`_apis/test/...`), which is **NOT reachable +anonymously — Hard-Rule 8**: those endpoints 302-redirect to a sign-in page on this runner, so +a `curl` returns an HTML redirect (not JSON) and every target test would look "not executed". +Validate instead at **leg granularity** using ONLY the anonymous `_apis/build/...` timeline +(Hard-Rule 8) — a `succeeded` category leg means every test in that category **that actually +ran** passed on that platform. This is a strong bar **only if the target test genuinely +executes**: a job can go green while one specific test is skipped at runtime (`[Ignore]`, +`Assert.Ignore()`, `Assert.Inconclusive()`, a `Skip=`/conditional `[Fact]`, an `#if`-out, or +an early `return` before the asserts). So before trusting a green leg as proof the target +passed, **confirm from the PR's OWN diff that the fix does NOT skip, ignore, disable, or +short-circuit the target test** (it adds no `[Ignore]`/`[Explicit]`, `Assert.Ignore`/ +`Assert.Inconclusive`, `Skip=`, category-exclusion, `#if`-out, or early `return` around the +target). If the fix could cause the target to be runtime-skipped rather than genuinely pass, +leg-level green is NOT sufficient evidence — record `skipped: target test may be +runtime-skipped by this fix (diff adds ); leg-green insufficient, deferring to human +on PR #` and stop this gate WITHOUT marking ready. Otherwise (the target genuinely runs +and asserts, as a de-flake or product fix does) a green category leg cannot hide a +target-test failure, so treat it as authoritative. + +Map the target test to the CI leg(s) that run it. A leg name encodes platform + test category +(e.g. `Android UITests SafeAreaEdges,Shadow`, `iOS UITests SafeAreaEdges,Shadow`, `macOS +UITests SafeAreaEdges,Shadow`, `Windows UITests SafeAreaEdges,Shadow`). Determine the target +test's UI-test category — its `[Category(UITestCategories.X)]` in the test/HostApp source, +visible in the PR diff or the test file — to know which leg-name substring identifies its legs; +for a device test, the per-platform device-test legs. + +Using the SAME build-discovery as Step 4 (filter AzDO builds by `branchName=refs/pull//merge` +or `sourceVersion == C.headSha`), read each build's **timeline** on `C.headSha` for the +pipeline(s) that RUN the target test — `maui-pr` (def 302) for unit/integration tests, +`maui-pr-uitests` (def 313) for Appium UI tests, `maui-pr-devicetests` (def 314) for device +tests: + +```bash +ORG=dnceng-public; PROJ=public; BUILD= +# Anonymous + Hard-Rule-8-compliant. NEVER call _apis/test/... (it 302-redirects to sign-in). +# Each type=="Job" record is one platform leg: result (succeeded/failed/canceled/null), +# state (completed/inProgress/pending), name (encodes " UITests "): +curl -s "https://dev.azure.com/$ORG/$PROJ/_apis/build/builds/$BUILD/timeline?api-version=7.1" \ + | tee /tmp/gh-aw/agent/timeline_${BUILD}.json \ + | jq -r '.records[] | select(.type=="Job") | "\(.result)\t\(.state)\t\(.name)"' +``` + +(The prefetch already exposes per-leg status in `C.failedLegs` / the checks context, and +`gh pr checks ` lists the same per-leg rows with platform+category in the name — use +whichever is handy; the build timeline is the authoritative cross-check on the exact build.) + +Treat a target test as **VALIDATED-GREEN** only if, on `C.headSha`, the leg that runs its +category is green on **every platform it runs on** — a fix that repairs one platform must not +silently regress the same test's leg on another, so a green leg on the originally-red platform +alone is NOT enough. Across the target pipeline's platform legs (for a UI test: the Android, +iOS, Windows, and macOS/MacCatalyst legs running the target's category; for a device test: each +device-test platform), require ALL of: +- the target's category leg is `result == "succeeded"` **and** `state == "completed"` on + **every** platform that runs it, AND +- that leg is `failed` / `canceled` / aborted on **NO** platform, AND +- **no platform is left unverified.** For each platform family the target pipeline covers, that + platform's category leg must have CONCLUDED (`state == "completed"`) on `C.headSha`. If a + platform simply has no leg for the target's category, the test does not run there — that is + fine, not a gap. But if a platform's category leg has **not concluded** (`state` is + `inProgress` / pending, or an `/azp`-gated `maui-pr-uitests` / `maui-pr-devicetests` leg that + has not been kicked), the test's status on that platform is UNKNOWN → the fix is NOT yet + validated across platforms: record `skipped: target test green on but + not yet verified on (leg(s) pending / need /azp run) on PR #` + and stop this gate WITHOUT marking ready. + +A target test whose category leg never concluded on ANY platform (**not executed** anywhere — +e.g. its `/azp`-gated pipeline has not been kicked) is likewise NOT validated: record `skipped: +target test not yet executed on PR # ( not run — needs /azp run)` and stop +this gate WITHOUT marking ready. Do NOT overclaim — a green *sibling* leg (a different category +on the same platform) is not the target's leg, and a green leg on one platform is not a pass on +the others. + +**T3 — Mark ready + report.** If EVERY target test is VALIDATED-GREEN. The 🎯 comment, +the mark-ready, and the `p/0` label are THREE SEPARATE safe-outputs — the comment +existing does NOT prove the mark-ready took effect, so they are tracked independently: + +- **Comment idempotency (dup-suppress only — never gates the mark-ready).** We only reach + T3 while `C.isDraft == true` (the precondition stops an already-ready PR before here). So + if a prior bot comment for THIS head that carries the leading `🎯` anchor AND the + contiguous phrase `validated green on ` already exists (BOTH T3 variants — + `🎯 Target test validated green on ` and the build-only `🎯 Build validated green on + ` — contain that exact contiguous phrase, so this recognizes both; keying on + `target test` alone would instead miss the build-only comment and re-post it every sweep. + **Require the `🎯` anchor** so the match can NEVER be loosened to a gapped + `validated…green…on` form that would false-positive on the Step 3 `✅ … validated — the + fix's CI is green on ` surface comment and wrongly suppress the audit 🎯), the comment + landed on an earlier sweep but the **mark-ready did NOT take + effect** (the PR is still a draft) — set `SUPPRESS_COMMENT = true` (do not re-post the + duplicate 🎯 comment) but STILL complete the mark-ready + label below. Never treat the + comment's existence as "already marked ready" while the PR is still a draft. Record this + case as `re-marking PR # (🎯 present; mark-ready did not take on a prior sweep)`. +- **Atomicity budget pre-check (Hard-Rule 7).** Determine the outputs this gate will emit: + `mark_pull_request_as_ready_for_review` + `add_labels` always, plus `add_comment` unless + `SUPPRESS_COMMENT`. Verify a free per-run slot remains in EACH bucket you are about to + use. If ANY required bucket is exhausted, emit NONE of them — record `skipped: per-run + cap reached; deferring mark-ready PR # to next cycle` and stop this gate. Never mark a + PR ready (or label it) unless its 🎯 audit comment is GUARANTEED to exist for this exact + head SHA — either posted in this same sweep, or (in the `SUPPRESS_COMMENT` re-marking case) + already present from a prior sweep. The outputs you DO emit either all land or all defer + together; the audit trail must never be absent, but it is NOT re-posted when it already exists. +- **Dry-run gate (Step 0):** if `dry_run == "true"`, emit NOTHING — print the intended + readiness comment and "would mark ready + add p/0" to the run log, tally `dry-run: + would-mark-ready PR #`, and stop. +- Otherwise emit for THIS PR number ``: + 1. `add_comment` (ONLY if not `SUPPRESS_COMMENT`): `🎯 Target test validated green on + — passed on ALL platforms it runs on (, + buildId ). — not + caused by this fix."> Transitioning this PR from draft to ready for review and adding + `p/0`; a maintainer still reviews and merges.` (For a **build-only fix** — the T1 + whole-build fallback, no single target test — phrase the first clause as `🎯 Build + validated green on — the maui-pr build passed on ALL platforms (buildId + ).` instead of naming a test.) + 2. `mark_pull_request_as_ready_for_review` with `reason:` a one-line justification + naming the validated test(s) and ``. + 3. `add_labels` with `labels: ["p/0"]` for PR # — put the now-review-ready fix into + the team's p/0 priority queue so it is triaged, not lost in the draft backlog. (If + the PR somehow already carries `p/0`, this is a harmless no-op.) +- Record `marked-ready PR # (target green on ALL platforms on , + labeled p/0)` and stop. + #### Step 3.5.R — Maintainer change-request response (Track C) Reached from Step 3.5 gate 0. Goal: when an **eligible human reviewer** (Hard-Rule @@ -1395,7 +1681,19 @@ Filed by [`ci-status-fix-net11`](https://github.com/dotnet/maui/blob/main/.githu ### Step 8 — Per-issue tally + end-of-run summary -Per issue, append one outcome line to `/tmp/gh-aw/agent/coverage.txt`: +Per issue, append **one** outcome line to `/tmp/gh-aw/agent/coverage.txt` — the +terminal outcome for the cycle. When one cycle produces a chained pair — a PR gets a +same-cycle precursor line and **then** a Step 3.6 readiness line (`marked-ready` on the +draft→ready flip, or `already-ready` on a later steady-state sweep of an already-ready +PR) — record ONLY the terminal Step 3.6 readiness line: it supersedes ANY same-cycle +non-readiness precursor for that PR, so an aggregator keying on one-line-per-issue never +double-counts. The precursor may be a Step 3 green line (`surfaced-green` on the first ✅, +or `already-surfaced` when it already exists) OR — when an unrelated-flake red and a +target-green coincide — a Step 4 `annotated-flake` line; either way the readiness line is +terminal, and the superseded precursor's signal survives in the PR's 🎯/♻️ comment so no +information is lost. This covers the flip cycle (`surfaced-green` → `marked-ready`), the +post-flip steady state (`already-surfaced` → `already-ready`), and the flake-coincident +flip (`annotated-flake` → `marked-ready`): ``` # net11.0 attempt- @@ -1406,8 +1704,14 @@ Per issue, append one outcome line to `/tmp/gh-aw/agent/coverage.txt`: `advance-PR # attempt /10` (ADVANCE mode — new commit pushed to the existing PR), `surfaced-green PR #` (fix's own CI went green; commented for review, did not advance), `annotated-flake PR #` (red was unrelated flake; -commented, attempt NOT burned), `waiting PR #` (CI not settled yet), -`dry-run: would-`, `skipped: `. (The +commented, attempt NOT burned), `marked-ready PR #` (the specific fixed test +was confirmed green in the PR's own CI; draft flipped to ready for review), +`already-surfaced PR #` (the fix's ✅ green comment already existed this sweep; +re-post suppressed — superseded by the Step 3.6 readiness line when the PR also flips +ready that cycle), `already-ready PR #` (terminal steady state — the PR was flipped +ready on a prior cycle and remains green on an unchanged head; no action taken, +supersedes `already-surfaced`), `waiting PR #` (CI not settled yet), +`dry-run: would-`, `skipped: `. (The `needs-human-PR` outcome is reserved for the deferred hand-off — Step 6 currently records a skip instead, so it is not emitted.) diff --git a/.github/workflows/ci-status-fix.lock.yml b/.github/workflows/ci-status-fix.lock.yml index 93dc5969cb0d..65511f2b3259 100644 --- a/.github/workflows/ci-status-fix.lock.yml +++ b/.github/workflows/ci-status-fix.lock.yml @@ -1,4 +1,4 @@ -# gh-aw-metadata: {"schema_version":"v4","frontmatter_hash":"f014161967589ebcaf1ac5ec6f3c88ee5a4388d28b55a07dc36c45b7b2aebab6","body_hash":"86c6b2d4c3df13da14c6a3c8b211381fcc9f97bb1951ff7b80588a2c5338e00c","compiler_version":"v0.80.9","strict":true,"agent_id":"copilot","agent_model":"claude-opus-4.8","engine_versions":{"copilot":"1.0.63"}} +# gh-aw-metadata: {"schema_version":"v4","frontmatter_hash":"b0ebf8a2f179900cfe3638e994b27a43cec52bdbdd2331dc1bd4d65762fa2d6b","body_hash":"1825e2b1f7c7bd88241bf37580655489360f9bebc806edd00837a52ce4ad83a0","compiler_version":"v0.80.9","strict":true,"agent_id":"copilot","agent_model":"claude-opus-4.8","engine_versions":{"copilot":"1.0.63"}} # gh-aw-manifest: {"version":1,"secrets":["COPILOT_PAT_0","COPILOT_PAT_1","COPILOT_PAT_2","COPILOT_PAT_3","COPILOT_PAT_4","COPILOT_PAT_5","COPILOT_PAT_6","COPILOT_PAT_7","COPILOT_PAT_8","COPILOT_PAT_9","GH_AW_CI_TRIGGER_TOKEN","GH_AW_GITHUB_MCP_SERVER_TOKEN","GH_AW_GITHUB_TOKEN","GITHUB_TOKEN"],"actions":[{"repo":"actions/checkout","sha":"34e114876b0b11c390a56381ad16ebd13914f8d5","version":"v4"},{"repo":"actions/checkout","sha":"9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0","version":"v7.0.0"},{"repo":"actions/checkout","sha":"de0fac2e4500dabe0009e67214ff5f5447ce83dd","version":"v6.0.2"},{"repo":"actions/download-artifact","sha":"3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c","version":"v8.0.1"},{"repo":"actions/github-script","sha":"3a2844b7e9c422d3c10d287c895573f7108da1b3","version":"v9.0.0"},{"repo":"actions/setup-node","sha":"48b55a011bda9f5d6aeb4c2d9c7362e8dae4041e","version":"v6.4.0"},{"repo":"actions/upload-artifact","sha":"043fb46d1a93c77aae656e7c1c64a875d1fc6a0a","version":"v7.0.1"},{"repo":"github/gh-aw-actions/setup","sha":"8c7d04ebf1ece56cd381446125da3e0f6896294a","version":"v0.80.9"}],"containers":[{"image":"ghcr.io/github/gh-aw-firewall/agent:0.27.7","digest":"sha256:aae231e4635c8999d039c132f1602d3df850fe9b84a00aa2b5ac981179b5661c","pinned_image":"ghcr.io/github/gh-aw-firewall/agent:0.27.7@sha256:aae231e4635c8999d039c132f1602d3df850fe9b84a00aa2b5ac981179b5661c"},{"image":"ghcr.io/github/gh-aw-firewall/api-proxy:0.27.7","digest":"sha256:009caf2e3d88fa77b64e9a03a95a228fc58db0f1701c6d324b29ba5a3c7c79b6","pinned_image":"ghcr.io/github/gh-aw-firewall/api-proxy:0.27.7@sha256:009caf2e3d88fa77b64e9a03a95a228fc58db0f1701c6d324b29ba5a3c7c79b6"},{"image":"ghcr.io/github/gh-aw-firewall/squid:0.27.7","digest":"sha256:deb1d4e19de62d51cee0508057a596a19315c3423ada4d675cad136dc8037c96","pinned_image":"ghcr.io/github/gh-aw-firewall/squid:0.27.7@sha256:deb1d4e19de62d51cee0508057a596a19315c3423ada4d675cad136dc8037c96"},{"image":"ghcr.io/github/gh-aw-mcpg:v0.3.27","digest":"sha256:fe984bddde4ec05d756d9043edb0a32912e6b7b72f6a121b1082f29221421cc7","pinned_image":"ghcr.io/github/gh-aw-mcpg:v0.3.27@sha256:fe984bddde4ec05d756d9043edb0a32912e6b7b72f6a121b1082f29221421cc7"},{"image":"ghcr.io/github/gh-aw-node","digest":"sha256:529d02eb970b1161aa25c593a9c3df57fdfad5a8add328cb3b6eccef66f3183b","pinned_image":"ghcr.io/github/gh-aw-node@sha256:529d02eb970b1161aa25c593a9c3df57fdfad5a8add328cb3b6eccef66f3183b"},{"image":"ghcr.io/github/github-mcp-server:v1.4.0","digest":"sha256:2afb26356481d1a350e14544a6e160f7f7ec1561a1ea309b823665abf0309036","pinned_image":"ghcr.io/github/github-mcp-server:v1.4.0@sha256:2afb26356481d1a350e14544a6e160f7f7ec1561a1ea309b823665abf0309036"}]} # This file was automatically generated by gh-aw (v0.80.9). DO NOT EDIT. To debug this workflow, load the skill at https://github.com/github/gh-aw/blob/main/debug.md # @@ -37,7 +37,9 @@ # humans (the open tracking issue is the hand-off surface; a dedicated # [ci-fix][needs-human] PR is planned but currently deferred — see Step 6). CI on # the PR is kicked by a human `/azp run` for now; the loop watches, classifies, -# and re-fixes autonomously between kicks. +# and re-fixes autonomously between kicks. When the SPECIFIC test a PR fixed is +# confirmed green in that PR's own CI, the loop marks the draft PR ready for review +# (a state transition only — it never approves or merges). # Never mutes tests, but # de-flakes genuinely flaky ones (deterministic synchronization, no retries / # timeout bumps). Always skips visual-regression / screenshot issues. @@ -273,24 +275,24 @@ jobs: run: | bash "${RUNNER_TEMP}/gh-aw/actions/create_prompt_first.sh" { - cat << 'GH_AW_PROMPT_25cf75111b670167_EOF' + cat << 'GH_AW_PROMPT_2ac1d0b4083428f8_EOF' - GH_AW_PROMPT_25cf75111b670167_EOF + GH_AW_PROMPT_2ac1d0b4083428f8_EOF cat "${RUNNER_TEMP}/gh-aw/prompts/xpia.md" cat "${RUNNER_TEMP}/gh-aw/prompts/temp_folder_prompt.md" cat "${RUNNER_TEMP}/gh-aw/prompts/markdown.md" cat "${RUNNER_TEMP}/gh-aw/prompts/safe_outputs_prompt.md" - cat << 'GH_AW_PROMPT_25cf75111b670167_EOF' + cat << 'GH_AW_PROMPT_2ac1d0b4083428f8_EOF' - Tools: add_comment(max:3), create_pull_request(max:3), update_pull_request(max:3), push_to_pull_request_branch(max:3), missing_tool, missing_data, noop - GH_AW_PROMPT_25cf75111b670167_EOF + Tools: add_comment(max:6), create_pull_request(max:3), update_pull_request(max:3), mark_pull_request_as_ready_for_review(max:3), add_labels(max:3), push_to_pull_request_branch(max:3), missing_tool, missing_data, noop + GH_AW_PROMPT_2ac1d0b4083428f8_EOF cat "${RUNNER_TEMP}/gh-aw/prompts/safe_outputs_create_pull_request.md" cat "${RUNNER_TEMP}/gh-aw/prompts/safe_outputs_push_to_pr_branch.md" - cat << 'GH_AW_PROMPT_25cf75111b670167_EOF' + cat << 'GH_AW_PROMPT_2ac1d0b4083428f8_EOF' - GH_AW_PROMPT_25cf75111b670167_EOF + GH_AW_PROMPT_2ac1d0b4083428f8_EOF cat "${RUNNER_TEMP}/gh-aw/prompts/mcp_cli_tools_prompt.md" - cat << 'GH_AW_PROMPT_25cf75111b670167_EOF' + cat << 'GH_AW_PROMPT_2ac1d0b4083428f8_EOF' The following GitHub context information is available for this workflow: {{#if github.actor}} @@ -332,12 +334,12 @@ jobs: stop immediately and report the limitation rather than spending turns trying to work around it. - GH_AW_PROMPT_25cf75111b670167_EOF + GH_AW_PROMPT_2ac1d0b4083428f8_EOF cat "${RUNNER_TEMP}/gh-aw/prompts/github_mcp_tools_with_safeoutputs_prompt.md" - cat << 'GH_AW_PROMPT_25cf75111b670167_EOF' + cat << 'GH_AW_PROMPT_2ac1d0b4083428f8_EOF' {{#runtime-import .github/workflows/ci-status-fix.md}} - GH_AW_PROMPT_25cf75111b670167_EOF + GH_AW_PROMPT_2ac1d0b4083428f8_EOF } > "$GH_AW_PROMPT" - name: Interpolate variables and render templates uses: actions/github-script@3a2844b7e9c422d3c10d287c895573f7108da1b3 # v9.0.0 @@ -554,16 +556,18 @@ jobs: mkdir -p "${RUNNER_TEMP}/gh-aw/safeoutputs" mkdir -p /tmp/gh-aw/safeoutputs mkdir -p /tmp/gh-aw/mcp-logs/safeoutputs - cat > "${RUNNER_TEMP}/gh-aw/safeoutputs/config.json" << 'GH_AW_SAFE_OUTPUTS_CONFIG_fa2a69fad5fd0485_EOF' - {"add_comment":{"discussions":false,"max":3,"required_labels":["agentic-workflows"],"required_title_prefix":"[ci-fix] ","target":"*"},"create_pull_request":{"allowed_base_branches":["main"],"allowed_branches":["ci-fix/**"],"allowed_files":["src/Core/**","src/Controls/**","src/Essentials/**","src/BlazorWebView/**","src/TestUtils/**","src/Templates/**","**/PublicAPI.Unshipped.txt"],"base_branch":"main","draft":true,"labels":["agentic-workflows"],"max":3,"max_patch_files":100,"max_patch_size":4096,"protect_top_level_dot_folders":true,"protected_files":["package.json","bun.lockb","bunfig.toml","deno.json","deno.jsonc","deno.lock","global.json","NuGet.Config","Directory.Packages.props","mix.exs","mix.lock","go.mod","go.sum","stack.yaml","stack.yaml.lock","pom.xml","build.gradle","build.gradle.kts","settings.gradle","settings.gradle.kts","gradle.properties","package-lock.json","yarn.lock","pnpm-lock.yaml","npm-shrinkwrap.json","requirements.txt","Pipfile","Pipfile.lock","pyproject.toml","setup.py","setup.cfg","Gemfile","Gemfile.lock","uv.lock","CODEOWNERS","DESIGN.md","README.md","CONTRIBUTING.md","CHANGELOG.md","SECURITY.md","CODE_OF_CONDUCT.md","AGENTS.md","CLAUDE.md","GEMINI.md"],"protected_files_policy":"request_review","title_prefix":"[ci-fix] "},"create_report_incomplete_issue":{},"missing_data":{},"missing_tool":{},"noop":{"max":1,"report-as-issue":"true"},"push_to_pull_request_branch":{"allowed_files":["src/Core/**","src/Controls/**","src/Essentials/**","src/BlazorWebView/**","src/TestUtils/**","src/Templates/**","**/PublicAPI.Unshipped.txt"],"if_no_changes":"warn","max":3,"max_patch_size":4096,"protect_top_level_dot_folders":true,"protected_files":["package.json","bun.lockb","bunfig.toml","deno.json","deno.jsonc","deno.lock","global.json","NuGet.Config","Directory.Packages.props","mix.exs","mix.lock","go.mod","go.sum","stack.yaml","stack.yaml.lock","pom.xml","build.gradle","build.gradle.kts","settings.gradle","settings.gradle.kts","gradle.properties","package-lock.json","yarn.lock","pnpm-lock.yaml","npm-shrinkwrap.json","requirements.txt","Pipfile","Pipfile.lock","pyproject.toml","setup.py","setup.cfg","Gemfile","Gemfile.lock","uv.lock","CODEOWNERS","DESIGN.md","README.md","CONTRIBUTING.md","CHANGELOG.md","SECURITY.md","CODE_OF_CONDUCT.md","AGENTS.md","CLAUDE.md","GEMINI.md"],"required_labels":["agentic-workflows"],"target":"*","title_prefix":"[ci-fix] "},"report_incomplete":{},"update_pull_request":{"allow_body":true,"allow_title":false,"max":3,"target":"*","update_branch":false}} - GH_AW_SAFE_OUTPUTS_CONFIG_fa2a69fad5fd0485_EOF + cat > "${RUNNER_TEMP}/gh-aw/safeoutputs/config.json" << 'GH_AW_SAFE_OUTPUTS_CONFIG_0644bf1434ca0071_EOF' + {"add_comment":{"discussions":false,"max":6,"required_labels":["agentic-workflows"],"required_title_prefix":"[ci-fix] ","target":"*"},"add_labels":{"allowed":["p/0"],"max":3,"required_labels":["agentic-workflows"],"required_title_prefix":"[ci-fix] ","target":"*"},"create_pull_request":{"allowed_base_branches":["main"],"allowed_branches":["ci-fix/**"],"allowed_files":["src/Core/**","src/Controls/**","src/Essentials/**","src/BlazorWebView/**","src/TestUtils/**","src/Templates/**","**/PublicAPI.Unshipped.txt"],"base_branch":"main","draft":true,"labels":["agentic-workflows"],"max":3,"max_patch_files":100,"max_patch_size":4096,"protect_top_level_dot_folders":true,"protected_files":["package.json","bun.lockb","bunfig.toml","deno.json","deno.jsonc","deno.lock","global.json","NuGet.Config","Directory.Packages.props","mix.exs","mix.lock","go.mod","go.sum","stack.yaml","stack.yaml.lock","pom.xml","build.gradle","build.gradle.kts","settings.gradle","settings.gradle.kts","gradle.properties","package-lock.json","yarn.lock","pnpm-lock.yaml","npm-shrinkwrap.json","requirements.txt","Pipfile","Pipfile.lock","pyproject.toml","setup.py","setup.cfg","Gemfile","Gemfile.lock","uv.lock","CODEOWNERS","DESIGN.md","README.md","CONTRIBUTING.md","CHANGELOG.md","SECURITY.md","CODE_OF_CONDUCT.md","AGENTS.md","CLAUDE.md","GEMINI.md"],"protected_files_policy":"request_review","title_prefix":"[ci-fix] "},"create_report_incomplete_issue":{},"mark_pull_request_as_ready_for_review":{"max":3,"required_labels":["agentic-workflows"],"required_title_prefix":"[ci-fix] ","target":"*"},"missing_data":{},"missing_tool":{},"noop":{"max":1,"report-as-issue":"true"},"push_to_pull_request_branch":{"allowed_files":["src/Core/**","src/Controls/**","src/Essentials/**","src/BlazorWebView/**","src/TestUtils/**","src/Templates/**","**/PublicAPI.Unshipped.txt"],"if_no_changes":"warn","max":3,"max_patch_size":4096,"protect_top_level_dot_folders":true,"protected_files":["package.json","bun.lockb","bunfig.toml","deno.json","deno.jsonc","deno.lock","global.json","NuGet.Config","Directory.Packages.props","mix.exs","mix.lock","go.mod","go.sum","stack.yaml","stack.yaml.lock","pom.xml","build.gradle","build.gradle.kts","settings.gradle","settings.gradle.kts","gradle.properties","package-lock.json","yarn.lock","pnpm-lock.yaml","npm-shrinkwrap.json","requirements.txt","Pipfile","Pipfile.lock","pyproject.toml","setup.py","setup.cfg","Gemfile","Gemfile.lock","uv.lock","CODEOWNERS","DESIGN.md","README.md","CONTRIBUTING.md","CHANGELOG.md","SECURITY.md","CODE_OF_CONDUCT.md","AGENTS.md","CLAUDE.md","GEMINI.md"],"required_labels":["agentic-workflows"],"target":"*","title_prefix":"[ci-fix] "},"report_incomplete":{},"update_pull_request":{"allow_body":true,"allow_title":false,"max":3,"target":"*","update_branch":false}} + GH_AW_SAFE_OUTPUTS_CONFIG_0644bf1434ca0071_EOF - name: Generate Safe Outputs Tools env: GH_AW_TOOLS_META_JSON: | { "description_suffixes": { - "add_comment": " CONSTRAINTS: Maximum 3 comment(s) can be added. Target: *. Supports reply_to_id for discussion threading.", + "add_comment": " CONSTRAINTS: Maximum 6 comment(s) can be added. Target: *. Supports reply_to_id for discussion threading.", + "add_labels": " CONSTRAINTS: Maximum 3 label(s) can be added. Only these labels are allowed: [\"p/0\"]. Target: *.", "create_pull_request": " CONSTRAINTS: Maximum 3 pull request(s) can be created. Title will be prefixed with \"[ci-fix] \". Labels [\"agentic-workflows\"] will be automatically added. Only these labels are allowed: [\"agentic-workflows\"]. PRs will be created as drafts.", + "mark_pull_request_as_ready_for_review": " CONSTRAINTS: Maximum 3 pull request(s) can be marked as ready for review.", "push_to_pull_request_branch": " CONSTRAINTS: Maximum 3 push(es) can be made. The target pull request title must start with \"[ci-fix] \".", "update_pull_request": " CONSTRAINTS: Maximum 3 pull request(s) can be updated. Target: *." }, @@ -594,6 +598,25 @@ jobs: } } }, + "add_labels": { + "defaultMax": 5, + "fields": { + "item_number": { + "issueNumberOrTemporaryId": true + }, + "labels": { + "required": true, + "type": "array", + "itemType": "string", + "itemSanitize": true, + "itemMaxLength": 128 + }, + "repo": { + "type": "string", + "maxLength": 256 + } + } + }, "create_pull_request": { "defaultMax": 1, "fields": { @@ -635,6 +658,24 @@ jobs: } } }, + "mark_pull_request_as_ready_for_review": { + "defaultMax": 1, + "fields": { + "pull_request_number": { + "issueOrPRNumber": true + }, + "reason": { + "required": true, + "type": "string", + "sanitize": true, + "maxLength": 65000 + }, + "repo": { + "type": "string", + "maxLength": 256 + } + } + }, "missing_data": { "defaultMax": 20, "fields": { @@ -1494,7 +1535,7 @@ jobs: uses: actions/github-script@3a2844b7e9c422d3c10d287c895573f7108da1b3 # v9.0.0 env: WORKFLOW_NAME: "CI Failure Fixer (main)" - WORKFLOW_DESCRIPTION: "Periodic pass over open ci-scan tracking issues filed by the main-branch CI\nfailure scanner (.github/workflows/ci-status-main.md). This workflow targets\nthe `main` branch EXCLUSIVELY: it processes only issues labelled ci-scan and\nopens every PR against main. (The net11.0 branch is handled by the parallel\n.github/workflows/ci-status-fix-net11.md — the two are split because gh-aw can\nonly transport a fix relative to ONE static base branch per workflow, and the\nmain↔net11.0 divergence exceeds gh-aw's 10 MB transport-patch cap.) The fixer\nopens ONE draft [ci-fix] PR per actionable issue against main, then WATCHES\nthat PR's own CI on later runs: when the fix's CI comes back red and the red is\ncaused by the fix itself, it pushes a fresh follow-up fix onto the SAME PR\nbranch (never a second PR) — up to 10 attempts — then stops and defers to\nhumans (the open tracking issue is the hand-off surface; a dedicated\n[ci-fix][needs-human] PR is planned but currently deferred — see Step 6). CI on\nthe PR is kicked by a human `/azp run` for now; the loop watches, classifies,\nand re-fixes autonomously between kicks.\nNever mutes tests, but\nde-flakes genuinely flaky ones (deterministic synchronization, no retries /\ntimeout bumps). Always skips visual-regression / screenshot issues." + WORKFLOW_DESCRIPTION: "Periodic pass over open ci-scan tracking issues filed by the main-branch CI\nfailure scanner (.github/workflows/ci-status-main.md). This workflow targets\nthe `main` branch EXCLUSIVELY: it processes only issues labelled ci-scan and\nopens every PR against main. (The net11.0 branch is handled by the parallel\n.github/workflows/ci-status-fix-net11.md — the two are split because gh-aw can\nonly transport a fix relative to ONE static base branch per workflow, and the\nmain↔net11.0 divergence exceeds gh-aw's 10 MB transport-patch cap.) The fixer\nopens ONE draft [ci-fix] PR per actionable issue against main, then WATCHES\nthat PR's own CI on later runs: when the fix's CI comes back red and the red is\ncaused by the fix itself, it pushes a fresh follow-up fix onto the SAME PR\nbranch (never a second PR) — up to 10 attempts — then stops and defers to\nhumans (the open tracking issue is the hand-off surface; a dedicated\n[ci-fix][needs-human] PR is planned but currently deferred — see Step 6). CI on\nthe PR is kicked by a human `/azp run` for now; the loop watches, classifies,\nand re-fixes autonomously between kicks. When the SPECIFIC test a PR fixed is\nconfirmed green in that PR's own CI, the loop marks the draft PR ready for review\n(a state transition only — it never approves or merges).\nNever mutes tests, but\nde-flakes genuinely flaky ones (deterministic synchronization, no retries /\ntimeout bumps). Always skips visual-regression / screenshot issues." HAS_PATCH: ${{ needs.agent.outputs.has_patch }} with: script: | @@ -1910,7 +1951,7 @@ jobs: GH_AW_ALLOWED_DOMAINS: "*.blob.core.windows.net,*.githubusercontent.com,api.business.githubcopilot.com,api.enterprise.githubcopilot.com,api.github.com,api.githubcopilot.com,api.individual.githubcopilot.com,api.snapcraft.io,archive.ubuntu.com,azure.archive.ubuntu.com,codeload.github.com,crl.geotrust.com,crl.globalsign.com,crl.identrust.com,crl.sectigo.com,crl.thawte.com,crl.usertrust.com,crl.verisign.com,crl3.digicert.com,crl4.digicert.com,crls.ssl.com,dev.azure.com,docs.github.com,github-cloud.githubusercontent.com,github-cloud.s3.amazonaws.com,github.blog,github.com,github.githubassets.com,helix.dot.net,host.docker.internal,json-schema.org,json.schemastore.org,keyserver.ubuntu.com,lfs.github.com,objects.githubusercontent.com,ocsp.digicert.com,ocsp.geotrust.com,ocsp.globalsign.com,ocsp.identrust.com,ocsp.sectigo.com,ocsp.ssl.com,ocsp.thawte.com,ocsp.usertrust.com,ocsp.verisign.com,packagecloud.io,packages.cloud.google.com,packages.microsoft.com,patch-diff.githubusercontent.com,ppa.launchpad.net,raw.githubusercontent.com,registry.npmjs.org,s.symcb.com,s.symcd.com,security.ubuntu.com,telemetry.enterprise.githubcopilot.com,ts-crl.ws.symantec.com,ts-ocsp.ws.symantec.com,www.googleapis.com" GITHUB_SERVER_URL: ${{ github.server_url }} GITHUB_API_URL: ${{ github.api_url }} - GH_AW_SAFE_OUTPUTS_HANDLER_CONFIG: "{\"add_comment\":{\"discussions\":false,\"max\":3,\"required_labels\":[\"agentic-workflows\"],\"required_title_prefix\":\"[ci-fix] \",\"target\":\"*\"},\"create_pull_request\":{\"allowed_base_branches\":[\"main\"],\"allowed_branches\":[\"ci-fix/**\"],\"allowed_files\":[\"src/Core/**\",\"src/Controls/**\",\"src/Essentials/**\",\"src/BlazorWebView/**\",\"src/TestUtils/**\",\"src/Templates/**\",\"**/PublicAPI.Unshipped.txt\"],\"base_branch\":\"main\",\"draft\":true,\"labels\":[\"agentic-workflows\"],\"max\":3,\"max_patch_files\":100,\"max_patch_size\":4096,\"protect_top_level_dot_folders\":true,\"protected_files\":[\"package.json\",\"bun.lockb\",\"bunfig.toml\",\"deno.json\",\"deno.jsonc\",\"deno.lock\",\"global.json\",\"NuGet.Config\",\"Directory.Packages.props\",\"mix.exs\",\"mix.lock\",\"go.mod\",\"go.sum\",\"stack.yaml\",\"stack.yaml.lock\",\"pom.xml\",\"build.gradle\",\"build.gradle.kts\",\"settings.gradle\",\"settings.gradle.kts\",\"gradle.properties\",\"package-lock.json\",\"yarn.lock\",\"pnpm-lock.yaml\",\"npm-shrinkwrap.json\",\"requirements.txt\",\"Pipfile\",\"Pipfile.lock\",\"pyproject.toml\",\"setup.py\",\"setup.cfg\",\"Gemfile\",\"Gemfile.lock\",\"uv.lock\",\"CODEOWNERS\",\"DESIGN.md\",\"README.md\",\"CONTRIBUTING.md\",\"CHANGELOG.md\",\"SECURITY.md\",\"CODE_OF_CONDUCT.md\",\"AGENTS.md\",\"CLAUDE.md\",\"GEMINI.md\"],\"protected_files_policy\":\"request_review\",\"title_prefix\":\"[ci-fix] \"},\"create_report_incomplete_issue\":{},\"missing_data\":{},\"missing_tool\":{},\"noop\":{\"max\":1,\"report-as-issue\":\"true\"},\"push_to_pull_request_branch\":{\"allowed_files\":[\"src/Core/**\",\"src/Controls/**\",\"src/Essentials/**\",\"src/BlazorWebView/**\",\"src/TestUtils/**\",\"src/Templates/**\",\"**/PublicAPI.Unshipped.txt\"],\"if_no_changes\":\"warn\",\"max\":3,\"max_patch_size\":4096,\"protect_top_level_dot_folders\":true,\"protected_files\":[\"package.json\",\"bun.lockb\",\"bunfig.toml\",\"deno.json\",\"deno.jsonc\",\"deno.lock\",\"global.json\",\"NuGet.Config\",\"Directory.Packages.props\",\"mix.exs\",\"mix.lock\",\"go.mod\",\"go.sum\",\"stack.yaml\",\"stack.yaml.lock\",\"pom.xml\",\"build.gradle\",\"build.gradle.kts\",\"settings.gradle\",\"settings.gradle.kts\",\"gradle.properties\",\"package-lock.json\",\"yarn.lock\",\"pnpm-lock.yaml\",\"npm-shrinkwrap.json\",\"requirements.txt\",\"Pipfile\",\"Pipfile.lock\",\"pyproject.toml\",\"setup.py\",\"setup.cfg\",\"Gemfile\",\"Gemfile.lock\",\"uv.lock\",\"CODEOWNERS\",\"DESIGN.md\",\"README.md\",\"CONTRIBUTING.md\",\"CHANGELOG.md\",\"SECURITY.md\",\"CODE_OF_CONDUCT.md\",\"AGENTS.md\",\"CLAUDE.md\",\"GEMINI.md\"],\"required_labels\":[\"agentic-workflows\"],\"target\":\"*\",\"title_prefix\":\"[ci-fix] \"},\"report_incomplete\":{},\"update_pull_request\":{\"allow_body\":true,\"allow_title\":false,\"max\":3,\"target\":\"*\",\"update_branch\":false}}" + GH_AW_SAFE_OUTPUTS_HANDLER_CONFIG: "{\"add_comment\":{\"discussions\":false,\"max\":6,\"required_labels\":[\"agentic-workflows\"],\"required_title_prefix\":\"[ci-fix] \",\"target\":\"*\"},\"add_labels\":{\"allowed\":[\"p/0\"],\"max\":3,\"required_labels\":[\"agentic-workflows\"],\"required_title_prefix\":\"[ci-fix] \",\"target\":\"*\"},\"create_pull_request\":{\"allowed_base_branches\":[\"main\"],\"allowed_branches\":[\"ci-fix/**\"],\"allowed_files\":[\"src/Core/**\",\"src/Controls/**\",\"src/Essentials/**\",\"src/BlazorWebView/**\",\"src/TestUtils/**\",\"src/Templates/**\",\"**/PublicAPI.Unshipped.txt\"],\"base_branch\":\"main\",\"draft\":true,\"labels\":[\"agentic-workflows\"],\"max\":3,\"max_patch_files\":100,\"max_patch_size\":4096,\"protect_top_level_dot_folders\":true,\"protected_files\":[\"package.json\",\"bun.lockb\",\"bunfig.toml\",\"deno.json\",\"deno.jsonc\",\"deno.lock\",\"global.json\",\"NuGet.Config\",\"Directory.Packages.props\",\"mix.exs\",\"mix.lock\",\"go.mod\",\"go.sum\",\"stack.yaml\",\"stack.yaml.lock\",\"pom.xml\",\"build.gradle\",\"build.gradle.kts\",\"settings.gradle\",\"settings.gradle.kts\",\"gradle.properties\",\"package-lock.json\",\"yarn.lock\",\"pnpm-lock.yaml\",\"npm-shrinkwrap.json\",\"requirements.txt\",\"Pipfile\",\"Pipfile.lock\",\"pyproject.toml\",\"setup.py\",\"setup.cfg\",\"Gemfile\",\"Gemfile.lock\",\"uv.lock\",\"CODEOWNERS\",\"DESIGN.md\",\"README.md\",\"CONTRIBUTING.md\",\"CHANGELOG.md\",\"SECURITY.md\",\"CODE_OF_CONDUCT.md\",\"AGENTS.md\",\"CLAUDE.md\",\"GEMINI.md\"],\"protected_files_policy\":\"request_review\",\"title_prefix\":\"[ci-fix] \"},\"create_report_incomplete_issue\":{},\"mark_pull_request_as_ready_for_review\":{\"max\":3,\"required_labels\":[\"agentic-workflows\"],\"required_title_prefix\":\"[ci-fix] \",\"target\":\"*\"},\"missing_data\":{},\"missing_tool\":{},\"noop\":{\"max\":1,\"report-as-issue\":\"true\"},\"push_to_pull_request_branch\":{\"allowed_files\":[\"src/Core/**\",\"src/Controls/**\",\"src/Essentials/**\",\"src/BlazorWebView/**\",\"src/TestUtils/**\",\"src/Templates/**\",\"**/PublicAPI.Unshipped.txt\"],\"if_no_changes\":\"warn\",\"max\":3,\"max_patch_size\":4096,\"protect_top_level_dot_folders\":true,\"protected_files\":[\"package.json\",\"bun.lockb\",\"bunfig.toml\",\"deno.json\",\"deno.jsonc\",\"deno.lock\",\"global.json\",\"NuGet.Config\",\"Directory.Packages.props\",\"mix.exs\",\"mix.lock\",\"go.mod\",\"go.sum\",\"stack.yaml\",\"stack.yaml.lock\",\"pom.xml\",\"build.gradle\",\"build.gradle.kts\",\"settings.gradle\",\"settings.gradle.kts\",\"gradle.properties\",\"package-lock.json\",\"yarn.lock\",\"pnpm-lock.yaml\",\"npm-shrinkwrap.json\",\"requirements.txt\",\"Pipfile\",\"Pipfile.lock\",\"pyproject.toml\",\"setup.py\",\"setup.cfg\",\"Gemfile\",\"Gemfile.lock\",\"uv.lock\",\"CODEOWNERS\",\"DESIGN.md\",\"README.md\",\"CONTRIBUTING.md\",\"CHANGELOG.md\",\"SECURITY.md\",\"CODE_OF_CONDUCT.md\",\"AGENTS.md\",\"CLAUDE.md\",\"GEMINI.md\"],\"required_labels\":[\"agentic-workflows\"],\"target\":\"*\",\"title_prefix\":\"[ci-fix] \"},\"report_incomplete\":{},\"update_pull_request\":{\"allow_body\":true,\"allow_title\":false,\"max\":3,\"target\":\"*\",\"update_branch\":false}}" GH_AW_CI_TRIGGER_TOKEN: ${{ secrets.GH_AW_CI_TRIGGER_TOKEN }} with: github-token: ${{ secrets.GH_AW_GITHUB_TOKEN || secrets.GITHUB_TOKEN }} diff --git a/.github/workflows/ci-status-fix.md b/.github/workflows/ci-status-fix.md index d3227e25170c..9c2f46054029 100644 --- a/.github/workflows/ci-status-fix.md +++ b/.github/workflows/ci-status-fix.md @@ -15,7 +15,9 @@ description: | humans (the open tracking issue is the hand-off surface; a dedicated [ci-fix][needs-human] PR is planned but currently deferred — see Step 6). CI on the PR is kicked by a human `/azp run` for now; the loop watches, classifies, - and re-fixes autonomously between kicks. + and re-fixes autonomously between kicks. When the SPECIFIC test a PR fixed is + confirmed green in that PR's own CI, the loop marks the draft PR ready for review + (a state transition only — it never approves or merges). Never mutes tests, but de-flakes genuinely flaky ones (deterministic synchronization, no retries / timeout bumps). Always skips visual-regression / screenshot issues. @@ -239,8 +241,15 @@ safe-outputs: # PER-RUN total (not per-PR): a sweep may surface several green PRs and/or # annotate several flaky reds in one run. At max:1 all but the first comment # were silently dropped, starving the workflow's #1 value (surfacing green PRs - # for review). Raised to 3 to match the per-run throughput of the other outputs. - max: 3 + # for review). Sized to 6 (not 3) because a single green DRAFT PR that gets + # flipped ready in one sweep spends TWO comment slots — the Step 3 ✅ surface + # comment (which still names the /azp-gated legs a human must kick, so it is NOT + # redundant with 🎯) AND the Step 3.6 T3 🎯 readiness comment. At max:3 the shared + # bucket drained after ~1 draft flip and T3's atomicity pre-check then deferred + # every further mark-ready even while the mark-ready/add_labels buckets (also 3) + # sat idle; 6 lets ~3 draft flips (2 comments each) land per sweep, so the + # mark-ready:3 / add_labels:3 caps are actually reachable. + max: 6 target: "*" # Hard constraint (defense-in-depth): only comment on THIS workflow's own # ci-fix PRs — [ci-fix] title prefix AND agentic-workflows label. Mirrors the @@ -261,13 +270,51 @@ safe-outputs: # never the title, so disable title rewrites — this compiles to allow_title:false # and removes any ability to retitle an arbitrary PR. title: false - # NOTE: gh-aw v0.79.8 does NOT emit required-title-prefix/required-labels into + # NOTE: gh-aw v0.80.9 (the pinned compiler) does NOT emit required-title-prefix/required-labels into # the compiled config for update-pull-request (verified against the lock — it # silently drops them, unlike add-comment / push-to-pull-request-branch which # honor them). So which-PR scoping here relies on prompt Hard-Rule 6 + # min-integrity:approved; the residual is body-marker edits only (no code, no # merge, no close), capped at max:3. Revisit required-* if a future gh-aw # compiler emits them for this output. + mark-pull-request-as-ready-for-review: + # Flip a validated draft [ci-fix] PR to ready-for-review once the SPECIFIC test + # this PR was opened to fix is confirmed green in the PR's OWN CI (Step 3.6) — + # even when unrelated legs are red. The workflow is schedule/dispatch-triggered + # (no triggering-PR context), so target "*" lets the agent name the PR number it + # validated. This is a state transition ONLY: it never approves and never merges — + # a human still reviews and merges. + # PER-RUN total (not per-PR): a sweep may validate several PRs' target tests green. + max: 3 + target: "*" + # Hard constraint (defense-in-depth): only ever un-draft THIS workflow's own + # ci-fix PRs — [ci-fix] title prefix AND agentic-workflows label — so a confused + # or prompt-injected agent cannot mark an arbitrary PR ready. (If the v0.80.9 + # compiler silently drops these — as documented for update-pull-request above — + # the Step 3.6 preconditions + min-integrity:approved are the compensating scope + # controls; verify against the lock after compiling.) + required-title-prefix: "[ci-fix] " + required-labels: [agentic-workflows] + add-labels: + # Apply the p/0 priority label to a [ci-fix] PR at the exact moment the loop flips it + # from draft to ready-for-review (Step 3.6 T3) — so a validated, review-ready fix lands + # in the team's p/0 triage queue instead of sitting unseen in the draft backlog. Paired + # 1:1 with mark-pull-request-as-ready-for-review; target "*" lets the agent name the PR + # it just validated. + # allowed = HARD allowlist: the agent may ONLY ever add p/0, nothing else. This caps the + # blast radius of a confused/prompt-injected agent to exactly one benign priority label — + # it can never apply a downstream-triggering or destructive label. + allowed: [p/0] + # PER-RUN total (not per-PR): a sweep may mark several PRs ready in one run; each adds + # one label, so this matches the mark-ready per-run cap (3). + max: 3 + target: "*" + # Hard constraint (defense-in-depth): only ever label THIS workflow's own ci-fix PRs — + # [ci-fix] title prefix AND agentic-workflows label. (If the v0.80.9 compiler drops these + # — as documented for update-pull-request above — the Step 3.6 preconditions + + # min-integrity:approved + the allowed:[p/0] allowlist are the compensating controls.) + required-title-prefix: "[ci-fix] " + required-labels: [agentic-workflows] timeout-minutes: 90 @@ -339,33 +386,48 @@ through `safe-outputs`. 5. **One issue = one outcome per run.** Exactly one of: respond to a maintainer's change-request on the open PR (Track C, Step 3.5.R); open the first fix/help/ de-flake PR; advance an existing PR by one attempt (push a follow-up fix); - surface a validated-green PR for review; annotate an unrelated-flake red; wait + surface a validated-green PR for review; mark a target-validated draft PR + ready-for-review (Step 3.6 T3 — the terminal outcome that supersedes the same + run's surface/annotate precursor line), or record its `already-ready` + steady-state no-op; annotate an unrelated-flake red; wait (CI not yet settled); or a recorded skip (the dedicated needs-human PR is deferred — the attempt cap records a skip instead; see Step 6). Always prefer advancing or opening a PR over a skip when a non-mute diff is producible. 6. **All writes via `safe-outputs`.** Allowed outputs: `create_pull_request` (first attempt only), `push_to_pull_request_branch` (advance an existing PR by one attempt), `update_pull_request` (bump the attempt marker / refresh the - prior-attempts table), and `add_comment` (progress notes on the `[ci-fix]` PR - ONLY). NEVER comment on the tracking issue (issues are locked by + prior-attempts table), `add_comment` (progress notes on the `[ci-fix]` PR + ONLY), `mark_pull_request_as_ready_for_review` (flip a target-validated draft + `[ci-fix]` PR from draft to ready — Step 3.6 T3 only), and `add_labels` (add + ONLY the `p/0` label, ONLY on that same draft→ready transition — Step 3.6 T3). + NEVER comment on the tracking issue (issues are locked by `.github/workflows/ci-scan-lock-issues.yml`) — `add_comment` targets the PR. No `gh pr create`, no manual `git push`. **Defense-in-depth:** `add_comment` and `update_pull_request` use `target: "*"` (the agent supplies the PR number). - `add_comment` is now config-locked to `required-title-prefix: "[ci-fix] "` + + `add_comment`, `mark_pull_request_as_ready_for_review`, and `add_labels` are + config-locked to `required-title-prefix: "[ci-fix] "` + `required-labels: [agentic-workflows]` (hard-enforced by the handler, same as - `push_to_pull_request_branch`), so a comment can only ever land on THIS - workflow's own PRs. `update_pull_request` CANNOT be config-locked to a - title/label in gh-aw v0.79.8 (the compiler silently drops `required-*` for that - output — verified against the lock), so it keeps `title: false` (no retitles; - body-marker edits only) plus this prompt-level guard. Before emitting either, - VERIFY the target PR carries BOTH the `[ci-fix]` title prefix AND the - `agentic-workflows` label; never comment on or edit the body of any PR that - lacks both. + `push_to_pull_request_branch`), and `add_labels` additionally has an + `allowed: [p/0]` allowlist so it can ONLY ever add `p/0` — so these can only + ever land on THIS workflow's own PRs. `update_pull_request` CANNOT be + config-locked to a title/label in gh-aw v0.80.9 (the compiler silently drops + `required-*` for that output — verified against the lock), so it keeps + `title: false` (no retitles; body-marker edits only) plus this prompt-level + guard. Before emitting ANY of these, VERIFY the target PR carries BOTH the + `[ci-fix]` title prefix AND the `agentic-workflows` label; never comment on, + edit, un-draft, or label any PR that lacks both. 7. **Per-run safe-output caps (per RUN, not per PR).** Each output type is capped per run: `create_pull_request` 3, `push_to_pull_request_branch` 3, - `add_comment` 3, `update_pull_request` 3. Note an ADVANCE spends one push **and** - one update **and** one comment, so ≤ 3 advances/run; surfacing a green or - annotating a flake spends one comment. When a bucket is exhausted, do NOT keep + `add_comment` 6, `update_pull_request` 3, `mark_pull_request_as_ready_for_review` + 3, `add_labels` 3 (counts LABELS, not calls). Note an ADVANCE spends one push + **and** one update **and** one comment, so ≤ 3 advances/run; surfacing a green or + annotating a flake spends one comment; a **mark-ready (Step 3.6 T3)** of a + still-draft PR spends TWO comments (the Step 3 ✅ surface **and** the T3 🎯 + readiness note) **and** one mark-ready **and** one label as an ALL-OR-NOTHING set + (T3 pre-checks those buckets and defers the whole PR if any is exhausted — never + mark a PR ready without its 🎯 audit comment). `add_comment` is therefore sized 6 + (not 3) so ~3 draft flips, at 2 comments each, can land in one sweep instead of the + shared comment bucket starving the otherwise-idle mark-ready/add_labels buckets. When a bucket is exhausted, do NOT keep emitting (extras are silently dropped) — record `skipped: per-run cap reached; deferring PR # to next cycle` for each remaining PR so the drop is a deliberate, logged decision. The continuous loop picks the deferred PRs up next @@ -409,8 +471,9 @@ For every open tracking issue in scope, converge on exactly one outcome: | Open first help-wanted draft `[ci-fix]` PR | No open PR yet; a plausible candidate exists but cannot be runner-validated (device/UI tests) (attempt 1) | | Open first de-flake draft `[ci-fix]` PR | No open PR yet; failure is intermittent (green-on-retry) from a genuine test-quality defect; a deterministic-synchronization fix is producible (attempt 1, Step 4.7 bucket b) | | Advance an existing PR (push attempt N+1) | Open PR's own CI settled red *because of the fix itself*, no human engaged, marker < 10 — push a NEW distinct fix onto the same branch (Step 3.5 → 5.6) | -| Surface a validated-green PR | Open PR's own CI settled green — comment "validated, attempt N/10; ready for review" and do NOT advance (Step 3.5) | -| Annotate an unrelated-flake red | Open PR is red only on baseline-flake / unrelated legs — comment which leg needs a re-run; do NOT burn an attempt (Step 3.5) | +| Surface a validated-green PR | Open PR's own CI settled green — comment "validated, attempt N/10; ready for review" and do NOT advance (Step 3.5), then mark the draft PR ready for review once the fixed test is confirmed green (Step 3.6) | +| Annotate an unrelated-flake red | Open PR is red only on baseline-flake / unrelated legs — comment which leg needs a re-run; do NOT burn an attempt (Step 3.5); if the SPECIFIC fixed test is confirmed green on that build, still mark the draft PR ready for review (Step 3.6) | +| Mark a validated PR ready for review | The specific test this PR fixed is confirmed green in the PR's own CI (even if unrelated legs are red) — comment the target-test result and transition the draft PR to ready for review; never approve or merge (Step 3.6) | | Wait | Open PR's CI is pending / not yet settled on the current head SHA — do nothing this cycle (Step 3.5) | | Hand-off skip (attempt cap) | Marker == 10 and the signature still reproduces — stop and defer to humans; the dedicated `[ci-fix][needs-human]` PR is planned but currently deferred (Step 6) | | Recorded skip | Visual-regression, human engaged, already-handled, fixed-in-latest-build, infra-flake, out-of-bounds, only-mute-available, or no novel approach producible | @@ -685,14 +748,34 @@ Run these gates in order — the FIRST that fires decides this cycle's outcome: 1. **Human engaged.** If `C.humanEngaged` → `skipped: human engaged on PR #; deferring` and stop. Never fight a human reviewer — the intentional hand-off boundary still holds the moment a person touches the PR. -2. **CI not settled / unknown / incomplete prefetch.** If `C.checksSettled == false` - OR `C.overallConclusion` is `pending`, `neutral`, or `unknown`, OR - `C.dataComplete == false` → the fix's CI has not finished on the current head SHA - (`C.headSha`), or the prefetch could not fully read this PR (a partial read can - understate `humanEngaged` / `attempt`). `skipped: PR # CI pending / prefetch - incomplete on ; waiting` and stop. *(Round 1: this is where a - maintainer `/azp run` is awaited — the `/azp`-gated uitests/devicetests legs will - not have run until a human kicks them.)* +2. **CI not settled — but validate the target test first (target-focused readiness).** + If `C.checksSettled == false` OR `C.overallConclusion` is `pending`, `neutral`, or + `unknown`, OR `C.dataComplete == false` → the *overall* build has not finished on the + current head SHA (`C.headSha`), or the prefetch could not fully read this PR (a partial + read can understate `humanEngaged` / `attempt`). Unrelated legs still draining must NOT + keep an already-proven fix parked as a draft, so before waiting, try the target-focused + fast-path: + - **2a — Target-green fast-path (draft `[ci-fix]` PRs only).** If `C.dataComplete == + true` AND the Step 3.6 preconditions hold (draft PR, `[ci-fix] ` title prefix AND the + `agentic-workflows` label), run Step 3.6 **T1–T2** against `C.headSha` now: identify + the target test(s) and read the PR's OWN build **timeline / per-leg status** (anonymous + `_apis/build` — never `_apis/test`, per Hard-Rule 8). If EVERY target test is + **VALIDATED-GREEN** (per T2's all-platform definition below — the target's category leg + is `succeeded` on every platform that runs it, failed on none, and NO target platform + family the pipeline covers is still pending/unconcluded, per T2's "no platform left + unverified" rule; a platform that simply has no leg for the target's category — the test + doesn't run there — is fine, not a gap) AND every leg in + `C.failedLegs` that has ALREADY concluded classifies as **unrelated flake** (Step 4 / + Step 4.7 method — the fix is implicated in NO completed red), then the fix is proven + regardless of unrelated *pending* legs → run **Step 3.6 T3** (mark ready + 🎯 comment) + and stop. This early-out NEVER advances an attempt and NEVER acts on a red that could + be the fix's fault. + - **2b — Otherwise WAIT.** If the target test has not yet executed (pending/absent on + its leg), or a completed red is (or may be) caused by the fix, or `C.dataComplete == + false`, or this PR has no identifiable target test → `skipped: PR # CI pending / + target not yet validated on ; waiting` and stop. *(Round 1: this is where a + maintainer `/azp run` is awaited — the `/azp`-gated uitests/devicetests legs will not + have run until a human kicks them.)* 3. **Green → surface for review.** If `C.overallConclusion == "success"`: the checks that RAN are green. **Primary-gate check first:** the deterministic `success` verdict only certifies "at least one green check, nothing failing or @@ -704,18 +787,31 @@ Run these gates in order — the FIRST that fires decides this cycle's outcome: # primary CI gate (maui-pr) not green on ; waiting` and stop — do NOT surface. (The `/azp`-gated `maui-pr-uitests` (def 313) / `maui-pr-devicetests` (def 314) legs MAY still be un-run — that is expected and is named below, not a - reason to withhold the surface.) **Idempotency:** scan the PR's existing comments - for a prior bot `✅ … validated … on ` note for THIS head SHA — if one - already exists, record `already-surfaced PR # (head )` and stop - WITHOUT re-commenting (re-surfacing the same green every tick is noise and burns - the per-run comment budget other PRs need). Otherwise resolve the attempt number from - `C.effectiveAttempt` (the authoritative max(marker, bot-commit) counter; omit the - number only if it is somehow indeterminate). **Dry-run gate (Step 0):** if `dry_run == "true"`, do NOT emit any comment — instead print the intended `✅` validated-green body to the run log, tally `dry-run: would-surface-green PR #`, and stop. Otherwise `add_comment` on PR #: `✅ Attempt /10 validated - — the fix's CI is green on . Ready for human review.` - Name any `/azp`-gated legs (uitests def 313 / devicetests def 314) that have not - run and still need a maintainer `/azp run`. Do NOT advance. Record - `surfaced-green PR # (attempt /10)` and stop. *(This directly - attacks the real bottleneck — no reviews — so it is the highest-value outcome.)* + reason to withhold the surface.) **Comment idempotency + dry-run suppress the ✅ + comment ONLY — neither skips Step 3.6.** Scan the PR's existing comments for a prior + bot `✅ … validated … on ` note for THIS head SHA; if one exists, set + `SKIP_SURFACE_COMMENT = true`. Resolve the attempt number from `C.effectiveAttempt` + (the authoritative max(marker, bot-commit) counter; omit the number only if it is + somehow indeterminate). Then post the surface comment UNLESS suppressed: + - if `dry_run == "true"`: do NOT emit — print the intended `✅` validated-green body to + the run log and tally `dry-run: would-surface-green PR #`; + - else if `SKIP_SURFACE_COMMENT`: do NOT re-post — record `already-surfaced PR # + (head )` (re-surfacing the same green every tick is noise and burns the + per-run comment budget other PRs need); + - else `add_comment` on PR #: `✅ Attempt /10 validated — the fix's CI is + green on .` naming any `/azp`-gated legs (uitests def 313 / devicetests def + 314) that have not run and still need a maintainer `/azp run`; record `surfaced-green + PR # (attempt /10)`. (Do NOT assert "ready for human review" on a PR that + is still a draft — Step 3.6, not this comment, owns the draft→ready flip and posts its + own 🎯 announcement when it fires.) + Do NOT advance. Then — in ALL of the above cases — run **Step 3.6** (target-test + readiness gate) for this PR before stopping: its `/azp`-gated target legs may conclude + green on this SAME head SHA on a LATER sweep (an `/azp run` adds no commit), and Step + 3.6 is the ONLY place the draft→ready flip happens, so it MUST re-evaluate every sweep — + never `stop` here before it. *(This directly attacks the real bottleneck — no reviews — + so it is the highest-value outcome.)* 4. **Red → classify caused-by-fix vs unrelated-flake.** If `C.overallConclusion == "failure"`, analyze the PR's OWN failing build (NOT `main`): find the AzDO `maui-pr` build for this PR (filter builds by `branchName=refs/pull//merge`, @@ -723,18 +819,26 @@ Run these gates in order — the FIRST that fires decides this cycle's outcome: method and Step 4.7 flake buckets to `C.failedLegs`. - **Unrelated flake only** (every failed leg is known-flaky / infra / a pre-existing baseline red NOT introduced by the fix): do NOT burn an attempt. - **Idempotency first:** scan the PR's existing comments for a prior bot - `♻️ … unrelated flake … on ` note for THIS head SHA — if one already - exists, record `already-annotated-flake PR # (head )` and stop - WITHOUT re-commenting (re-annotating the same flake every 12h sweep is noise and - burns the per-run comment budget other PRs need). Otherwise resolve the attempt - number from `C.effectiveAttempt` (the authoritative max(marker, bot-commit) - counter), omitting the `Attempt /10:` prefix only if it is somehow - indeterminate. **Dry-run gate (Step 0):** if `dry_run == "true"`, do NOT emit any comment — instead print the intended `♻️` unrelated-flake body to the run log, tally `dry-run: would-annotate-flake PR #`, and stop. Otherwise `add_comment` on PR #: `♻️ Attempt /10: red is - unrelated flake on leg(s) () on ; the fix itself is not - implicated. A maintainer re-run (/azp run ) should clear it.` Record - `annotated-flake PR # (head )` and stop. *(Round 1: human re-runs; - Round 2: auto re-trigger.)* + **Comment idempotency + dry-run suppress the ♻️ comment ONLY — neither skips Step + 3.6.** Scan the PR's existing comments for a prior bot `♻️ … unrelated flake … on + ` note for THIS head SHA; if one exists, set `SKIP_FLAKE_COMMENT = true`. + Resolve the attempt number from `C.effectiveAttempt` (the authoritative max(marker, + bot-commit) counter), omitting the `Attempt /10:` prefix only if it is + somehow indeterminate. Then post the flake note UNLESS suppressed: + - if `dry_run == "true"`: do NOT emit — print the intended `♻️` unrelated-flake body + to the run log and tally `dry-run: would-annotate-flake PR #`; + - else if `SKIP_FLAKE_COMMENT`: do NOT re-post — record `already-annotated-flake PR + # (head )` (re-annotating the same flake every 12h sweep is noise and + burns the per-run comment budget other PRs need); + - else `add_comment` on PR #: `♻️ Attempt /10: red is unrelated flake on + leg(s) () on ; the fix itself is not implicated. A + maintainer re-run (/azp run ) should clear it.` record `annotated-flake + PR # (head )`. + Then — in ALL of the above cases — run **Step 3.6** (target-test readiness gate) for + this PR before stopping: its `/azp`-gated target legs may conclude green on this SAME + head SHA on a LATER sweep, and Step 3.6 is the ONLY place the draft→ready flip + happens, so it MUST re-evaluate every sweep — never `stop` here before it. *(Round 1: + human re-runs; Round 2: auto re-trigger.)* - **Caused by the fix** (a failed leg still matches the original target signature, or the fix introduced a NEW failure): advance an attempt. - **Attempt count.** `attempt = C.effectiveAttempt` — the authoritative @@ -752,6 +856,188 @@ Run these gates in order — the FIRST that fires decides this cycle's outcome: from every prior commit on this branch**, emitted via the Step 5.6 ADVANCE path (push onto the same PR). +#### Step 3.6 — Target-test verification & mark-ready gate + +Reached from the Step 3 **green-surface** branch, the Step 4 **unrelated-flake** branch +(AFTER that branch has posted its comment), and the Step 3.5 **gate-2 target-focused +fast-path** (2a — while unrelated legs are still pending). Purpose: confirm the SPECIFIC +test(s) this PR was opened to fix now PASS in the PR's own CI, and — only when they do +— transition the draft `[ci-fix]` PR to **ready for review** so a maintainer sees a +validated fix instead of a draft. This is the ONLY place the loop flips draft→ready; it +is a state transition, **never** an approval or a merge (a human still reviews and +merges). Overall red on *unrelated* legs must NOT gate readiness — we validate the fix, +not the base branch's flakiness. + +**Preconditions** (ALL must hold; otherwise record `skipped: readiness N/A PR # +()` and stop this gate): +- `C.isDraft == true` — if the PR is already ready-for-review, the draft→ready + transition is already done; do NOT re-mark **and do NOT re-apply `p/0`**. `p/0` is applied + exactly ONCE, atomically with the draft→ready flip (T3 — all-or-nothing with the 🎯 audit + comment + mark-ready), so an already-ready loop PR that is MISSING `p/0` has almost + certainly had it **removed by a maintainer** de-prioritizing the PR — a pure triage action + the human-engagement guard (which inspects comments/reviews/commits, NOT label changes) + cannot see. Re-adding `p/0` every sweep would fight that maintainer indefinitely, violating + the loop's "never override a human" contract. So the loop does NOT reconcile the label: + record `already-ready PR #` and stop this gate. (Trade-off: on the rare occasion a + transient API error drops `p/0` at flip time *after* the T3 atomic pre-check passed, it is + not auto-re-added — but the PR is still ready-for-review with its 🎯 audit comment, and + re-adding `p/0` is a trivial manual action; that is strictly preferable to steam-rolling a + maintainer's deliberate de-prioritization.) +- The PR is unmistakably THIS workflow's own: `[ci-fix] ` title prefix AND the + `agentic-workflows` label (mirrors the safe-output lock; the compensating scope + control if the v0.80.9 compiler drops the declarative required-*). +- You reached this gate from Step 3 (green), Step 4 (**unrelated-flake**), or the Step 3.5 + gate-2 **target-focused fast-path** (2a) — i.e. the fix is NOT implicated in any red that + has concluded. If Step 4 classified a red as **caused by the fix** (ADVANCE), do NOT run + this gate — advance the attempt instead. + +**T1 — Identify the target test(s).** From the `[ci-scan]` issue signature the fix +addresses (and the PR's own diff), extract the fully-qualified test method name(s) the +fix targets — e.g. `SafeAreaShouldWorkOnAllShellTabs`. For a de-flake it is the +de-flaked test; for a product fix it is the originally-failing test(s). If NO specific +test can be identified (e.g. a product build-break rather than a test failure), this is a +**build-only fix** — there is no single test to validate cross-platform, so validate at +**whole-build** granularity rather than stopping (Step 3 only *comments*; Step 3.6 is the +sole draft→ready flip, so a build-only fix is undrafted HERE or not at all). We only reach +this gate from Step 3 (green) / Step 4 (unrelated-flake) / gate-2a, so the fix is not +implicated in any concluded red; additionally require, via the T2 timeline method on +`C.headSha`, that the primary build pipeline `maui-pr` (def 302) has CONCLUDED with EVERY +one of its platform build legs `succeeded`/`completed` and NONE failed/canceled or still +pending — the build is green on **every** platform, not just the originally-broken one (the +cross-platform guard, applied to the build instead of a test). A build-only fix is undrafted +ONLY on the strength of the auto-running `maui-pr` (def 302) whole-build green, which the +loop CAN observe: if the PR's `[ci-scan]` issue signature or its own diff indicates the +ORIGINATING failure was in a `/azp`-gated pipeline (`maui-pr-uitests` def 313 or +`maui-pr-devicetests` def 314) rather than `maui-pr` (def 302), do NOT undraft on +`maui-pr`-green alone — those pipelines do not auto-run on this PR (GITHUB_TOKEN cannot +trigger them), so a green `maui-pr` build is NOT evidence the gated build break is fixed; +record `skipped: build-only fix PR # targets gated pipeline () not run — +deferring to human` and stop this gate. Otherwise, set `TARGET := "the maui-pr build +(build-only fix — no single target test)"` and proceed to **T3** to mark ready. If any `maui-pr` build leg is still unconcluded, record `skipped: build-only fix PR +# not yet whole-build green (leg(s) pending)` and stop this gate WITHOUT marking +ready (a green subset is not enough — a still-pending build leg could yet fail). + +**T2 — Verify the target test's platform legs are green (anonymous, leg-level).** Per-test +outcomes come from the AzDO **test-results** API (`_apis/test/...`), which is **NOT reachable +anonymously — Hard-Rule 8**: those endpoints 302-redirect to a sign-in page on this runner, so +a `curl` returns an HTML redirect (not JSON) and every target test would look "not executed". +Validate instead at **leg granularity** using ONLY the anonymous `_apis/build/...` timeline +(Hard-Rule 8) — a `succeeded` category leg means every test in that category **that actually +ran** passed on that platform. This is a strong bar **only if the target test genuinely +executes**: a job can go green while one specific test is skipped at runtime (`[Ignore]`, +`Assert.Ignore()`, `Assert.Inconclusive()`, a `Skip=`/conditional `[Fact]`, an `#if`-out, or +an early `return` before the asserts). So before trusting a green leg as proof the target +passed, **confirm from the PR's OWN diff that the fix does NOT skip, ignore, disable, or +short-circuit the target test** (it adds no `[Ignore]`/`[Explicit]`, `Assert.Ignore`/ +`Assert.Inconclusive`, `Skip=`, category-exclusion, `#if`-out, or early `return` around the +target). If the fix could cause the target to be runtime-skipped rather than genuinely pass, +leg-level green is NOT sufficient evidence — record `skipped: target test may be +runtime-skipped by this fix (diff adds ); leg-green insufficient, deferring to human +on PR #` and stop this gate WITHOUT marking ready. Otherwise (the target genuinely runs +and asserts, as a de-flake or product fix does) a green category leg cannot hide a +target-test failure, so treat it as authoritative. + +Map the target test to the CI leg(s) that run it. A leg name encodes platform + test category +(e.g. `Android UITests SafeAreaEdges,Shadow`, `iOS UITests SafeAreaEdges,Shadow`, `macOS +UITests SafeAreaEdges,Shadow`, `Windows UITests SafeAreaEdges,Shadow`). Determine the target +test's UI-test category — its `[Category(UITestCategories.X)]` in the test/HostApp source, +visible in the PR diff or the test file — to know which leg-name substring identifies its legs; +for a device test, the per-platform device-test legs. + +Using the SAME build-discovery as Step 4 (filter AzDO builds by `branchName=refs/pull//merge` +or `sourceVersion == C.headSha`), read each build's **timeline** on `C.headSha` for the +pipeline(s) that RUN the target test — `maui-pr` (def 302) for unit/integration tests, +`maui-pr-uitests` (def 313) for Appium UI tests, `maui-pr-devicetests` (def 314) for device +tests: + +```bash +ORG=dnceng-public; PROJ=public; BUILD= +# Anonymous + Hard-Rule-8-compliant. NEVER call _apis/test/... (it 302-redirects to sign-in). +# Each type=="Job" record is one platform leg: result (succeeded/failed/canceled/null), +# state (completed/inProgress/pending), name (encodes " UITests "): +curl -s "https://dev.azure.com/$ORG/$PROJ/_apis/build/builds/$BUILD/timeline?api-version=7.1" \ + | tee /tmp/gh-aw/agent/timeline_${BUILD}.json \ + | jq -r '.records[] | select(.type=="Job") | "\(.result)\t\(.state)\t\(.name)"' +``` + +(The prefetch already exposes per-leg status in `C.failedLegs` / the checks context, and +`gh pr checks ` lists the same per-leg rows with platform+category in the name — use +whichever is handy; the build timeline is the authoritative cross-check on the exact build.) + +Treat a target test as **VALIDATED-GREEN** only if, on `C.headSha`, the leg that runs its +category is green on **every platform it runs on** — a fix that repairs one platform must not +silently regress the same test's leg on another, so a green leg on the originally-red platform +alone is NOT enough. Across the target pipeline's platform legs (for a UI test: the Android, +iOS, Windows, and macOS/MacCatalyst legs running the target's category; for a device test: each +device-test platform), require ALL of: +- the target's category leg is `result == "succeeded"` **and** `state == "completed"` on + **every** platform that runs it, AND +- that leg is `failed` / `canceled` / aborted on **NO** platform, AND +- **no platform is left unverified.** For each platform family the target pipeline covers, that + platform's category leg must have CONCLUDED (`state == "completed"`) on `C.headSha`. If a + platform simply has no leg for the target's category, the test does not run there — that is + fine, not a gap. But if a platform's category leg has **not concluded** (`state` is + `inProgress` / pending, or an `/azp`-gated `maui-pr-uitests` / `maui-pr-devicetests` leg that + has not been kicked), the test's status on that platform is UNKNOWN → the fix is NOT yet + validated across platforms: record `skipped: target test green on but + not yet verified on (leg(s) pending / need /azp run) on PR #` + and stop this gate WITHOUT marking ready. + +A target test whose category leg never concluded on ANY platform (**not executed** anywhere — +e.g. its `/azp`-gated pipeline has not been kicked) is likewise NOT validated: record `skipped: +target test not yet executed on PR # ( not run — needs /azp run)` and stop +this gate WITHOUT marking ready. Do NOT overclaim — a green *sibling* leg (a different category +on the same platform) is not the target's leg, and a green leg on one platform is not a pass on +the others. + +**T3 — Mark ready + report.** If EVERY target test is VALIDATED-GREEN. The 🎯 comment, +the mark-ready, and the `p/0` label are THREE SEPARATE safe-outputs — the comment +existing does NOT prove the mark-ready took effect, so they are tracked independently: + +- **Comment idempotency (dup-suppress only — never gates the mark-ready).** We only reach + T3 while `C.isDraft == true` (the precondition stops an already-ready PR before here). So + if a prior bot comment for THIS head that carries the leading `🎯` anchor AND the + contiguous phrase `validated green on ` already exists (BOTH T3 variants — + `🎯 Target test validated green on ` and the build-only `🎯 Build validated green on + ` — contain that exact contiguous phrase, so this recognizes both; keying on + `target test` alone would instead miss the build-only comment and re-post it every sweep. + **Require the `🎯` anchor** so the match can NEVER be loosened to a gapped + `validated…green…on` form that would false-positive on the Step 3 `✅ … validated — the + fix's CI is green on ` surface comment and wrongly suppress the audit 🎯), the comment + landed on an earlier sweep but the **mark-ready did NOT take + effect** (the PR is still a draft) — set `SUPPRESS_COMMENT = true` (do not re-post the + duplicate 🎯 comment) but STILL complete the mark-ready + label below. Never treat the + comment's existence as "already marked ready" while the PR is still a draft. Record this + case as `re-marking PR # (🎯 present; mark-ready did not take on a prior sweep)`. +- **Atomicity budget pre-check (Hard-Rule 7).** Determine the outputs this gate will emit: + `mark_pull_request_as_ready_for_review` + `add_labels` always, plus `add_comment` unless + `SUPPRESS_COMMENT`. Verify a free per-run slot remains in EACH bucket you are about to + use. If ANY required bucket is exhausted, emit NONE of them — record `skipped: per-run + cap reached; deferring mark-ready PR # to next cycle` and stop this gate. Never mark a + PR ready (or label it) unless its 🎯 audit comment is GUARANTEED to exist for this exact + head SHA — either posted in this same sweep, or (in the `SUPPRESS_COMMENT` re-marking case) + already present from a prior sweep. The outputs you DO emit either all land or all defer + together; the audit trail must never be absent, but it is NOT re-posted when it already exists. +- **Dry-run gate (Step 0):** if `dry_run == "true"`, emit NOTHING — print the intended + readiness comment and "would mark ready + add p/0" to the run log, tally `dry-run: + would-mark-ready PR #`, and stop. +- Otherwise emit for THIS PR number ``: + 1. `add_comment` (ONLY if not `SUPPRESS_COMMENT`): `🎯 Target test validated green on + — passed on ALL platforms it runs on (, + buildId ). — not + caused by this fix."> Transitioning this PR from draft to ready for review and adding + `p/0`; a maintainer still reviews and merges.` (For a **build-only fix** — the T1 + whole-build fallback, no single target test — phrase the first clause as `🎯 Build + validated green on — the maui-pr build passed on ALL platforms (buildId + ).` instead of naming a test.) + 2. `mark_pull_request_as_ready_for_review` with `reason:` a one-line justification + naming the validated test(s) and ``. + 3. `add_labels` with `labels: ["p/0"]` for PR # — put the now-review-ready fix into + the team's p/0 priority queue so it is triaged, not lost in the draft backlog. (If + the PR somehow already carries `p/0`, this is a harmless no-op.) +- Record `marked-ready PR # (target green on ALL platforms on , + labeled p/0)` and stop. + #### Step 3.5.R — Maintainer change-request response (Track C) Reached from Step 3.5 gate 0. Goal: when an **eligible human reviewer** (Hard-Rule @@ -1383,7 +1669,19 @@ Filed by [`ci-status-fix`](https://github.com/dotnet/maui/blob/main/.github/work ### Step 8 — Per-issue tally + end-of-run summary -Per issue, append one outcome line to `/tmp/gh-aw/agent/coverage.txt`: +Per issue, append **one** outcome line to `/tmp/gh-aw/agent/coverage.txt` — the +terminal outcome for the cycle. When one cycle produces a chained pair — a PR gets a +same-cycle precursor line and **then** a Step 3.6 readiness line (`marked-ready` on the +draft→ready flip, or `already-ready` on a later steady-state sweep of an already-ready +PR) — record ONLY the terminal Step 3.6 readiness line: it supersedes ANY same-cycle +non-readiness precursor for that PR, so an aggregator keying on one-line-per-issue never +double-counts. The precursor may be a Step 3 green line (`surfaced-green` on the first ✅, +or `already-surfaced` when it already exists) OR — when an unrelated-flake red and a +target-green coincide — a Step 4 `annotated-flake` line; either way the readiness line is +terminal, and the superseded precursor's signal survives in the PR's 🎯/♻️ comment so no +information is lost. This covers the flip cycle (`surfaced-green` → `marked-ready`), the +post-flip steady state (`already-surfaced` → `already-ready`), and the flake-coincident +flip (`annotated-flake` → `marked-ready`): ``` # main attempt- @@ -1394,8 +1692,14 @@ Per issue, append one outcome line to `/tmp/gh-aw/agent/coverage.txt`: `advance-PR # attempt /10` (ADVANCE mode — new commit pushed to the existing PR), `surfaced-green PR #` (fix's own CI went green; commented for review, did not advance), `annotated-flake PR #` (red was unrelated flake; -commented, attempt NOT burned), `waiting PR #` (CI not settled yet), -`dry-run: would-`, `skipped: `. (The +commented, attempt NOT burned), `marked-ready PR #` (the specific fixed test +was confirmed green in the PR's own CI; draft flipped to ready for review), +`already-surfaced PR #` (the fix's ✅ green comment already existed this sweep; +re-post suppressed — superseded by the Step 3.6 readiness line when the PR also flips +ready that cycle), `already-ready PR #` (terminal steady state — the PR was flipped +ready on a prior cycle and remains green on an unchanged head; no action taken, +supersedes `already-surfaced`), `waiting PR #` (CI not settled yet), +`dry-run: would-`, `skipped: `. (The `needs-human-PR` outcome is reserved for the deferred hand-off — Step 6 currently records a skip instead, so it is not emitted.)
`; + - else if `SKIP_SURFACE_COMMENT`: do NOT re-post — record `already-surfaced PR #
+ (head )` (re-surfacing the same green every tick is noise and burns the + per-run comment budget other PRs need); + - else `add_comment` on PR #: `✅ Attempt /10 validated — the fix's CI is + green on .` naming any `/azp`-gated legs (uitests def 313 / devicetests def + 314) that have not run and still need a maintainer `/azp run`; record `surfaced-green + PR # (attempt /10)`. (Do NOT assert "ready for human review" on a PR that + is still a draft — Step 3.6, not this comment, owns the draft→ready flip and posts its + own 🎯 announcement when it fires.) + Do NOT advance. Then — in ALL of the above cases — run **Step 3.6** (target-test + readiness gate) for this PR before stopping: its `/azp`-gated target legs may conclude + green on this SAME head SHA on a LATER sweep (an `/azp run` adds no commit), and Step + 3.6 is the ONLY place the draft→ready flip happens, so it MUST re-evaluate every sweep — + never `stop` here before it. *(This directly attacks the real bottleneck — no reviews — + so it is the highest-value outcome.)* 4. **Red → classify caused-by-fix vs unrelated-flake.** If `C.overallConclusion == "failure"`, analyze the PR's OWN failing build (NOT `net11.0`): find the AzDO `maui-pr` build for this PR (filter builds by `branchName=refs/pull//merge`, @@ -733,18 +829,26 @@ Run these gates in order — the FIRST that fires decides this cycle's outcome: method and Step 4.7 flake buckets to `C.failedLegs`. - **Unrelated flake only** (every failed leg is known-flaky / infra / a pre-existing baseline red NOT introduced by the fix): do NOT burn an attempt. - **Idempotency first:** scan the PR's existing comments for a prior bot - `♻️ … unrelated flake … on ` note for THIS head SHA — if one already - exists, record `already-annotated-flake PR # (head )` and stop - WITHOUT re-commenting (re-annotating the same flake every 12h sweep is noise and - burns the per-run comment budget other PRs need). Otherwise resolve the attempt - number from `C.effectiveAttempt` (the authoritative max(marker, bot-commit) - counter), omitting the `Attempt /10:` prefix only if it is somehow - indeterminate. **Dry-run gate (Step 0):** if `dry_run == "true"`, do NOT emit any comment — instead print the intended `♻️` unrelated-flake body to the run log, tally `dry-run: would-annotate-flake PR #`, and stop. Otherwise `add_comment` on PR #: `♻️ Attempt /10: red is - unrelated flake on leg(s) () on ; the fix itself is not - implicated. A maintainer re-run (/azp run ) should clear it.` Record - `annotated-flake PR # (head )` and stop. *(Round 1: human re-runs; - Round 2: auto re-trigger.)* + **Comment idempotency + dry-run suppress the ♻️ comment ONLY — neither skips Step + 3.6.** Scan the PR's existing comments for a prior bot `♻️ … unrelated flake … on + ` note for THIS head SHA; if one exists, set `SKIP_FLAKE_COMMENT = true`. + Resolve the attempt number from `C.effectiveAttempt` (the authoritative max(marker, + bot-commit) counter), omitting the `Attempt /10:` prefix only if it is + somehow indeterminate. Then post the flake note UNLESS suppressed: + - if `dry_run == "true"`: do NOT emit — print the intended `♻️` unrelated-flake body + to the run log and tally `dry-run: would-annotate-flake PR #`; + - else if `SKIP_FLAKE_COMMENT`: do NOT re-post — record `already-annotated-flake PR + # (head )` (re-annotating the same flake every 12h sweep is noise and + burns the per-run comment budget other PRs need); + - else `add_comment` on PR #: `♻️ Attempt /10: red is unrelated flake on + leg(s) () on ; the fix itself is not implicated. A + maintainer re-run (/azp run ) should clear it.` record `annotated-flake + PR # (head )`. + Then — in ALL of the above cases — run **Step 3.6** (target-test readiness gate) for + this PR before stopping: its `/azp`-gated target legs may conclude green on this SAME + head SHA on a LATER sweep, and Step 3.6 is the ONLY place the draft→ready flip + happens, so it MUST re-evaluate every sweep — never `stop` here before it. *(Round 1: + human re-runs; Round 2: auto re-trigger.)* - **Caused by the fix** (a failed leg still matches the original target signature, or the fix introduced a NEW failure): advance an attempt. - **Attempt count.** `attempt = C.effectiveAttempt` — the authoritative @@ -762,6 +866,188 @@ Run these gates in order — the FIRST that fires decides this cycle's outcome: from every prior commit on this branch**, emitted via the Step 5.6 ADVANCE path (push onto the same PR). +#### Step 3.6 — Target-test verification & mark-ready gate + +Reached from the Step 3 **green-surface** branch, the Step 4 **unrelated-flake** branch +(AFTER that branch has posted its comment), and the Step 3.5 **gate-2 target-focused +fast-path** (2a — while unrelated legs are still pending). Purpose: confirm the SPECIFIC +test(s) this PR was opened to fix now PASS in the PR's own CI, and — only when they do +— transition the draft `[ci-fix-net11]` PR to **ready for review** so a maintainer sees +a validated fix instead of a draft. This is the ONLY place the loop flips draft→ready; +it is a state transition, **never** an approval or a merge (a human still reviews and +merges). Overall red on *unrelated* legs must NOT gate readiness — we validate the fix, +not the base branch's flakiness. + +**Preconditions** (ALL must hold; otherwise record `skipped: readiness N/A PR # +()` and stop this gate): +- `C.isDraft == true` — if the PR is already ready-for-review, the draft→ready + transition is already done; do NOT re-mark **and do NOT re-apply `p/0`**. `p/0` is applied + exactly ONCE, atomically with the draft→ready flip (T3 — all-or-nothing with the 🎯 audit + comment + mark-ready), so an already-ready loop PR that is MISSING `p/0` has almost + certainly had it **removed by a maintainer** de-prioritizing the PR — a pure triage action + the human-engagement guard (which inspects comments/reviews/commits, NOT label changes) + cannot see. Re-adding `p/0` every sweep would fight that maintainer indefinitely, violating + the loop's "never override a human" contract. So the loop does NOT reconcile the label: + record `already-ready PR #` and stop this gate. (Trade-off: on the rare occasion a + transient API error drops `p/0` at flip time *after* the T3 atomic pre-check passed, it is + not auto-re-added — but the PR is still ready-for-review with its 🎯 audit comment, and + re-adding `p/0` is a trivial manual action; that is strictly preferable to steam-rolling a + maintainer's deliberate de-prioritization.) +- The PR is unmistakably THIS workflow's own: `[ci-fix-net11] ` title prefix AND the + `agentic-workflows` label (mirrors the safe-output lock; the compensating scope + control if the v0.80.9 compiler drops the declarative required-*). +- You reached this gate from Step 3 (green), Step 4 (**unrelated-flake**), or the Step 3.5 + gate-2 **target-focused fast-path** (2a) — i.e. the fix is NOT implicated in any red that + has concluded. If Step 4 classified a red as **caused by the fix** (ADVANCE), do NOT run + this gate — advance the attempt instead. + +**T1 — Identify the target test(s).** From the `[ci-scan-net11]` issue signature the fix +addresses (and the PR's own diff), extract the fully-qualified test method name(s) the +fix targets — e.g. `SafeAreaShouldWorkOnAllShellTabs`. For a de-flake it is the +de-flaked test; for a product fix it is the originally-failing test(s). If NO specific +test can be identified (e.g. a product build-break rather than a test failure), this is a +**build-only fix** — there is no single test to validate cross-platform, so validate at +**whole-build** granularity rather than stopping (Step 3 only *comments*; Step 3.6 is the +sole draft→ready flip, so a build-only fix is undrafted HERE or not at all). We only reach +this gate from Step 3 (green) / Step 4 (unrelated-flake) / gate-2a, so the fix is not +implicated in any concluded red; additionally require, via the T2 timeline method on +`C.headSha`, that the primary build pipeline `maui-pr` (def 302) has CONCLUDED with EVERY +one of its platform build legs `succeeded`/`completed` and NONE failed/canceled or still +pending — the build is green on **every** platform, not just the originally-broken one (the +cross-platform guard, applied to the build instead of a test). A build-only fix is undrafted +ONLY on the strength of the auto-running `maui-pr` (def 302) whole-build green, which the +loop CAN observe: if the PR's `[ci-scan]` issue signature or its own diff indicates the +ORIGINATING failure was in a `/azp`-gated pipeline (`maui-pr-uitests` def 313 or +`maui-pr-devicetests` def 314) rather than `maui-pr` (def 302), do NOT undraft on +`maui-pr`-green alone — those pipelines do not auto-run on this PR (GITHUB_TOKEN cannot +trigger them), so a green `maui-pr` build is NOT evidence the gated build break is fixed; +record `skipped: build-only fix PR # targets gated pipeline () not run — +deferring to human` and stop this gate. Otherwise, set `TARGET := "the maui-pr build +(build-only fix — no single target test)"` and proceed to **T3** to mark ready. If any `maui-pr` build leg is still unconcluded, record `skipped: build-only fix PR +# not yet whole-build green (leg(s) pending)` and stop this gate WITHOUT marking +ready (a green subset is not enough — a still-pending build leg could yet fail). + +**T2 — Verify the target test's platform legs are green (anonymous, leg-level).** Per-test +outcomes come from the AzDO **test-results** API (`_apis/test/...`), which is **NOT reachable +anonymously — Hard-Rule 8**: those endpoints 302-redirect to a sign-in page on this runner, so +a `curl` returns an HTML redirect (not JSON) and every target test would look "not executed". +Validate instead at **leg granularity** using ONLY the anonymous `_apis/build/...` timeline +(Hard-Rule 8) — a `succeeded` category leg means every test in that category **that actually +ran** passed on that platform. This is a strong bar **only if the target test genuinely +executes**: a job can go green while one specific test is skipped at runtime (`[Ignore]`, +`Assert.Ignore()`, `Assert.Inconclusive()`, a `Skip=`/conditional `[Fact]`, an `#if`-out, or +an early `return` before the asserts). So before trusting a green leg as proof the target +passed, **confirm from the PR's OWN diff that the fix does NOT skip, ignore, disable, or +short-circuit the target test** (it adds no `[Ignore]`/`[Explicit]`, `Assert.Ignore`/ +`Assert.Inconclusive`, `Skip=`, category-exclusion, `#if`-out, or early `return` around the +target). If the fix could cause the target to be runtime-skipped rather than genuinely pass, +leg-level green is NOT sufficient evidence — record `skipped: target test may be +runtime-skipped by this fix (diff adds ); leg-green insufficient, deferring to human +on PR #` and stop this gate WITHOUT marking ready. Otherwise (the target genuinely runs +and asserts, as a de-flake or product fix does) a green category leg cannot hide a +target-test failure, so treat it as authoritative. + +Map the target test to the CI leg(s) that run it. A leg name encodes platform + test category +(e.g. `Android UITests SafeAreaEdges,Shadow`, `iOS UITests SafeAreaEdges,Shadow`, `macOS +UITests SafeAreaEdges,Shadow`, `Windows UITests SafeAreaEdges,Shadow`). Determine the target +test's UI-test category — its `[Category(UITestCategories.X)]` in the test/HostApp source, +visible in the PR diff or the test file — to know which leg-name substring identifies its legs; +for a device test, the per-platform device-test legs. + +Using the SAME build-discovery as Step 4 (filter AzDO builds by `branchName=refs/pull//merge` +or `sourceVersion == C.headSha`), read each build's **timeline** on `C.headSha` for the +pipeline(s) that RUN the target test — `maui-pr` (def 302) for unit/integration tests, +`maui-pr-uitests` (def 313) for Appium UI tests, `maui-pr-devicetests` (def 314) for device +tests: + +```bash +ORG=dnceng-public; PROJ=public; BUILD= +# Anonymous + Hard-Rule-8-compliant. NEVER call _apis/test/... (it 302-redirects to sign-in). +# Each type=="Job" record is one platform leg: result (succeeded/failed/canceled/null), +# state (completed/inProgress/pending), name (encodes " UITests "): +curl -s "https://dev.azure.com/$ORG/$PROJ/_apis/build/builds/$BUILD/timeline?api-version=7.1" \ + | tee /tmp/gh-aw/agent/timeline_${BUILD}.json \ + | jq -r '.records[] | select(.type=="Job") | "\(.result)\t\(.state)\t\(.name)"' +``` + +(The prefetch already exposes per-leg status in `C.failedLegs` / the checks context, and +`gh pr checks ` lists the same per-leg rows with platform+category in the name — use +whichever is handy; the build timeline is the authoritative cross-check on the exact build.) + +Treat a target test as **VALIDATED-GREEN** only if, on `C.headSha`, the leg that runs its +category is green on **every platform it runs on** — a fix that repairs one platform must not +silently regress the same test's leg on another, so a green leg on the originally-red platform +alone is NOT enough. Across the target pipeline's platform legs (for a UI test: the Android, +iOS, Windows, and macOS/MacCatalyst legs running the target's category; for a device test: each +device-test platform), require ALL of: +- the target's category leg is `result == "succeeded"` **and** `state == "completed"` on + **every** platform that runs it, AND +- that leg is `failed` / `canceled` / aborted on **NO** platform, AND +- **no platform is left unverified.** For each platform family the target pipeline covers, that + platform's category leg must have CONCLUDED (`state == "completed"`) on `C.headSha`. If a + platform simply has no leg for the target's category, the test does not run there — that is + fine, not a gap. But if a platform's category leg has **not concluded** (`state` is + `inProgress` / pending, or an `/azp`-gated `maui-pr-uitests` / `maui-pr-devicetests` leg that + has not been kicked), the test's status on that platform is UNKNOWN → the fix is NOT yet + validated across platforms: record `skipped: target test green on but + not yet verified on (leg(s) pending / need /azp run) on PR #` + and stop this gate WITHOUT marking ready. + +A target test whose category leg never concluded on ANY platform (**not executed** anywhere — +e.g. its `/azp`-gated pipeline has not been kicked) is likewise NOT validated: record `skipped: +target test not yet executed on PR # ( not run — needs /azp run)` and stop +this gate WITHOUT marking ready. Do NOT overclaim — a green *sibling* leg (a different category +on the same platform) is not the target's leg, and a green leg on one platform is not a pass on +the others. + +**T3 — Mark ready + report.** If EVERY target test is VALIDATED-GREEN. The 🎯 comment, +the mark-ready, and the `p/0` label are THREE SEPARATE safe-outputs — the comment +existing does NOT prove the mark-ready took effect, so they are tracked independently: + +- **Comment idempotency (dup-suppress only — never gates the mark-ready).** We only reach + T3 while `C.isDraft == true` (the precondition stops an already-ready PR before here). So + if a prior bot comment for THIS head that carries the leading `🎯` anchor AND the + contiguous phrase `validated green on ` already exists (BOTH T3 variants — + `🎯 Target test validated green on ` and the build-only `🎯 Build validated green on + ` — contain that exact contiguous phrase, so this recognizes both; keying on + `target test` alone would instead miss the build-only comment and re-post it every sweep. + **Require the `🎯` anchor** so the match can NEVER be loosened to a gapped + `validated…green…on` form that would false-positive on the Step 3 `✅ … validated — the + fix's CI is green on ` surface comment and wrongly suppress the audit 🎯), the comment + landed on an earlier sweep but the **mark-ready did NOT take + effect** (the PR is still a draft) — set `SUPPRESS_COMMENT = true` (do not re-post the + duplicate 🎯 comment) but STILL complete the mark-ready + label below. Never treat the + comment's existence as "already marked ready" while the PR is still a draft. Record this + case as `re-marking PR # (🎯 present; mark-ready did not take on a prior sweep)`. +- **Atomicity budget pre-check (Hard-Rule 7).** Determine the outputs this gate will emit: + `mark_pull_request_as_ready_for_review` + `add_labels` always, plus `add_comment` unless + `SUPPRESS_COMMENT`. Verify a free per-run slot remains in EACH bucket you are about to + use. If ANY required bucket is exhausted, emit NONE of them — record `skipped: per-run + cap reached; deferring mark-ready PR # to next cycle` and stop this gate. Never mark a + PR ready (or label it) unless its 🎯 audit comment is GUARANTEED to exist for this exact + head SHA — either posted in this same sweep, or (in the `SUPPRESS_COMMENT` re-marking case) + already present from a prior sweep. The outputs you DO emit either all land or all defer + together; the audit trail must never be absent, but it is NOT re-posted when it already exists. +- **Dry-run gate (Step 0):** if `dry_run == "true"`, emit NOTHING — print the intended + readiness comment and "would mark ready + add p/0" to the run log, tally `dry-run: + would-mark-ready PR #`, and stop. +- Otherwise emit for THIS PR number ``: + 1. `add_comment` (ONLY if not `SUPPRESS_COMMENT`): `🎯 Target test validated green on + — passed on ALL platforms it runs on (, + buildId ). — not + caused by this fix."> Transitioning this PR from draft to ready for review and adding + `p/0`; a maintainer still reviews and merges.` (For a **build-only fix** — the T1 + whole-build fallback, no single target test — phrase the first clause as `🎯 Build + validated green on — the maui-pr build passed on ALL platforms (buildId + ).` instead of naming a test.) + 2. `mark_pull_request_as_ready_for_review` with `reason:` a one-line justification + naming the validated test(s) and ``. + 3. `add_labels` with `labels: ["p/0"]` for PR # — put the now-review-ready fix into + the team's p/0 priority queue so it is triaged, not lost in the draft backlog. (If + the PR somehow already carries `p/0`, this is a harmless no-op.) +- Record `marked-ready PR # (target green on ALL platforms on , + labeled p/0)` and stop. + #### Step 3.5.R — Maintainer change-request response (Track C) Reached from Step 3.5 gate 0. Goal: when an **eligible human reviewer** (Hard-Rule @@ -1395,7 +1681,19 @@ Filed by [`ci-status-fix-net11`](https://github.com/dotnet/maui/blob/main/.githu ### Step 8 — Per-issue tally + end-of-run summary -Per issue, append one outcome line to `/tmp/gh-aw/agent/coverage.txt`: +Per issue, append **one** outcome line to `/tmp/gh-aw/agent/coverage.txt` — the +terminal outcome for the cycle. When one cycle produces a chained pair — a PR gets a +same-cycle precursor line and **then** a Step 3.6 readiness line (`marked-ready` on the +draft→ready flip, or `already-ready` on a later steady-state sweep of an already-ready +PR) — record ONLY the terminal Step 3.6 readiness line: it supersedes ANY same-cycle +non-readiness precursor for that PR, so an aggregator keying on one-line-per-issue never +double-counts. The precursor may be a Step 3 green line (`surfaced-green` on the first ✅, +or `already-surfaced` when it already exists) OR — when an unrelated-flake red and a +target-green coincide — a Step 4 `annotated-flake` line; either way the readiness line is +terminal, and the superseded precursor's signal survives in the PR's 🎯/♻️ comment so no +information is lost. This covers the flip cycle (`surfaced-green` → `marked-ready`), the +post-flip steady state (`already-surfaced` → `already-ready`), and the flake-coincident +flip (`annotated-flake` → `marked-ready`): ``` # net11.0 attempt- @@ -1406,8 +1704,14 @@ Per issue, append one outcome line to `/tmp/gh-aw/agent/coverage.txt`: `advance-PR # attempt /10` (ADVANCE mode — new commit pushed to the existing PR), `surfaced-green PR #` (fix's own CI went green; commented for review, did not advance), `annotated-flake PR #` (red was unrelated flake; -commented, attempt NOT burned), `waiting PR #` (CI not settled yet), -`dry-run: would-`, `skipped: `. (The +commented, attempt NOT burned), `marked-ready PR #` (the specific fixed test +was confirmed green in the PR's own CI; draft flipped to ready for review), +`already-surfaced PR #` (the fix's ✅ green comment already existed this sweep; +re-post suppressed — superseded by the Step 3.6 readiness line when the PR also flips +ready that cycle), `already-ready PR #` (terminal steady state — the PR was flipped +ready on a prior cycle and remains green on an unchanged head; no action taken, +supersedes `already-surfaced`), `waiting PR #` (CI not settled yet), +`dry-run: would-`, `skipped: `. (The `needs-human-PR` outcome is reserved for the deferred hand-off — Step 6 currently records a skip instead, so it is not emitted.) diff --git a/.github/workflows/ci-status-fix.lock.yml b/.github/workflows/ci-status-fix.lock.yml index 93dc5969cb0d..65511f2b3259 100644 --- a/.github/workflows/ci-status-fix.lock.yml +++ b/.github/workflows/ci-status-fix.lock.yml @@ -1,4 +1,4 @@ -# gh-aw-metadata: {"schema_version":"v4","frontmatter_hash":"f014161967589ebcaf1ac5ec6f3c88ee5a4388d28b55a07dc36c45b7b2aebab6","body_hash":"86c6b2d4c3df13da14c6a3c8b211381fcc9f97bb1951ff7b80588a2c5338e00c","compiler_version":"v0.80.9","strict":true,"agent_id":"copilot","agent_model":"claude-opus-4.8","engine_versions":{"copilot":"1.0.63"}} +# gh-aw-metadata: {"schema_version":"v4","frontmatter_hash":"b0ebf8a2f179900cfe3638e994b27a43cec52bdbdd2331dc1bd4d65762fa2d6b","body_hash":"1825e2b1f7c7bd88241bf37580655489360f9bebc806edd00837a52ce4ad83a0","compiler_version":"v0.80.9","strict":true,"agent_id":"copilot","agent_model":"claude-opus-4.8","engine_versions":{"copilot":"1.0.63"}} # gh-aw-manifest: {"version":1,"secrets":["COPILOT_PAT_0","COPILOT_PAT_1","COPILOT_PAT_2","COPILOT_PAT_3","COPILOT_PAT_4","COPILOT_PAT_5","COPILOT_PAT_6","COPILOT_PAT_7","COPILOT_PAT_8","COPILOT_PAT_9","GH_AW_CI_TRIGGER_TOKEN","GH_AW_GITHUB_MCP_SERVER_TOKEN","GH_AW_GITHUB_TOKEN","GITHUB_TOKEN"],"actions":[{"repo":"actions/checkout","sha":"34e114876b0b11c390a56381ad16ebd13914f8d5","version":"v4"},{"repo":"actions/checkout","sha":"9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0","version":"v7.0.0"},{"repo":"actions/checkout","sha":"de0fac2e4500dabe0009e67214ff5f5447ce83dd","version":"v6.0.2"},{"repo":"actions/download-artifact","sha":"3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c","version":"v8.0.1"},{"repo":"actions/github-script","sha":"3a2844b7e9c422d3c10d287c895573f7108da1b3","version":"v9.0.0"},{"repo":"actions/setup-node","sha":"48b55a011bda9f5d6aeb4c2d9c7362e8dae4041e","version":"v6.4.0"},{"repo":"actions/upload-artifact","sha":"043fb46d1a93c77aae656e7c1c64a875d1fc6a0a","version":"v7.0.1"},{"repo":"github/gh-aw-actions/setup","sha":"8c7d04ebf1ece56cd381446125da3e0f6896294a","version":"v0.80.9"}],"containers":[{"image":"ghcr.io/github/gh-aw-firewall/agent:0.27.7","digest":"sha256:aae231e4635c8999d039c132f1602d3df850fe9b84a00aa2b5ac981179b5661c","pinned_image":"ghcr.io/github/gh-aw-firewall/agent:0.27.7@sha256:aae231e4635c8999d039c132f1602d3df850fe9b84a00aa2b5ac981179b5661c"},{"image":"ghcr.io/github/gh-aw-firewall/api-proxy:0.27.7","digest":"sha256:009caf2e3d88fa77b64e9a03a95a228fc58db0f1701c6d324b29ba5a3c7c79b6","pinned_image":"ghcr.io/github/gh-aw-firewall/api-proxy:0.27.7@sha256:009caf2e3d88fa77b64e9a03a95a228fc58db0f1701c6d324b29ba5a3c7c79b6"},{"image":"ghcr.io/github/gh-aw-firewall/squid:0.27.7","digest":"sha256:deb1d4e19de62d51cee0508057a596a19315c3423ada4d675cad136dc8037c96","pinned_image":"ghcr.io/github/gh-aw-firewall/squid:0.27.7@sha256:deb1d4e19de62d51cee0508057a596a19315c3423ada4d675cad136dc8037c96"},{"image":"ghcr.io/github/gh-aw-mcpg:v0.3.27","digest":"sha256:fe984bddde4ec05d756d9043edb0a32912e6b7b72f6a121b1082f29221421cc7","pinned_image":"ghcr.io/github/gh-aw-mcpg:v0.3.27@sha256:fe984bddde4ec05d756d9043edb0a32912e6b7b72f6a121b1082f29221421cc7"},{"image":"ghcr.io/github/gh-aw-node","digest":"sha256:529d02eb970b1161aa25c593a9c3df57fdfad5a8add328cb3b6eccef66f3183b","pinned_image":"ghcr.io/github/gh-aw-node@sha256:529d02eb970b1161aa25c593a9c3df57fdfad5a8add328cb3b6eccef66f3183b"},{"image":"ghcr.io/github/github-mcp-server:v1.4.0","digest":"sha256:2afb26356481d1a350e14544a6e160f7f7ec1561a1ea309b823665abf0309036","pinned_image":"ghcr.io/github/github-mcp-server:v1.4.0@sha256:2afb26356481d1a350e14544a6e160f7f7ec1561a1ea309b823665abf0309036"}]} # This file was automatically generated by gh-aw (v0.80.9). DO NOT EDIT. To debug this workflow, load the skill at https://github.com/github/gh-aw/blob/main/debug.md # @@ -37,7 +37,9 @@ # humans (the open tracking issue is the hand-off surface; a dedicated # [ci-fix][needs-human] PR is planned but currently deferred — see Step 6). CI on # the PR is kicked by a human `/azp run` for now; the loop watches, classifies, -# and re-fixes autonomously between kicks. +# and re-fixes autonomously between kicks. When the SPECIFIC test a PR fixed is +# confirmed green in that PR's own CI, the loop marks the draft PR ready for review +# (a state transition only — it never approves or merges). # Never mutes tests, but # de-flakes genuinely flaky ones (deterministic synchronization, no retries / # timeout bumps). Always skips visual-regression / screenshot issues. @@ -273,24 +275,24 @@ jobs: run: | bash "${RUNNER_TEMP}/gh-aw/actions/create_prompt_first.sh" { - cat << 'GH_AW_PROMPT_25cf75111b670167_EOF' + cat << 'GH_AW_PROMPT_2ac1d0b4083428f8_EOF' - GH_AW_PROMPT_25cf75111b670167_EOF + GH_AW_PROMPT_2ac1d0b4083428f8_EOF cat "${RUNNER_TEMP}/gh-aw/prompts/xpia.md" cat "${RUNNER_TEMP}/gh-aw/prompts/temp_folder_prompt.md" cat "${RUNNER_TEMP}/gh-aw/prompts/markdown.md" cat "${RUNNER_TEMP}/gh-aw/prompts/safe_outputs_prompt.md" - cat << 'GH_AW_PROMPT_25cf75111b670167_EOF' + cat << 'GH_AW_PROMPT_2ac1d0b4083428f8_EOF' - Tools: add_comment(max:3), create_pull_request(max:3), update_pull_request(max:3), push_to_pull_request_branch(max:3), missing_tool, missing_data, noop - GH_AW_PROMPT_25cf75111b670167_EOF + Tools: add_comment(max:6), create_pull_request(max:3), update_pull_request(max:3), mark_pull_request_as_ready_for_review(max:3), add_labels(max:3), push_to_pull_request_branch(max:3), missing_tool, missing_data, noop + GH_AW_PROMPT_2ac1d0b4083428f8_EOF cat "${RUNNER_TEMP}/gh-aw/prompts/safe_outputs_create_pull_request.md" cat "${RUNNER_TEMP}/gh-aw/prompts/safe_outputs_push_to_pr_branch.md" - cat << 'GH_AW_PROMPT_25cf75111b670167_EOF' + cat << 'GH_AW_PROMPT_2ac1d0b4083428f8_EOF' - GH_AW_PROMPT_25cf75111b670167_EOF + GH_AW_PROMPT_2ac1d0b4083428f8_EOF cat "${RUNNER_TEMP}/gh-aw/prompts/mcp_cli_tools_prompt.md" - cat << 'GH_AW_PROMPT_25cf75111b670167_EOF' + cat << 'GH_AW_PROMPT_2ac1d0b4083428f8_EOF' The following GitHub context information is available for this workflow: {{#if github.actor}} @@ -332,12 +334,12 @@ jobs: stop immediately and report the limitation rather than spending turns trying to work around it. - GH_AW_PROMPT_25cf75111b670167_EOF + GH_AW_PROMPT_2ac1d0b4083428f8_EOF cat "${RUNNER_TEMP}/gh-aw/prompts/github_mcp_tools_with_safeoutputs_prompt.md" - cat << 'GH_AW_PROMPT_25cf75111b670167_EOF' + cat << 'GH_AW_PROMPT_2ac1d0b4083428f8_EOF' {{#runtime-import .github/workflows/ci-status-fix.md}} - GH_AW_PROMPT_25cf75111b670167_EOF + GH_AW_PROMPT_2ac1d0b4083428f8_EOF } > "$GH_AW_PROMPT" - name: Interpolate variables and render templates uses: actions/github-script@3a2844b7e9c422d3c10d287c895573f7108da1b3 # v9.0.0 @@ -554,16 +556,18 @@ jobs: mkdir -p "${RUNNER_TEMP}/gh-aw/safeoutputs" mkdir -p /tmp/gh-aw/safeoutputs mkdir -p /tmp/gh-aw/mcp-logs/safeoutputs - cat > "${RUNNER_TEMP}/gh-aw/safeoutputs/config.json" << 'GH_AW_SAFE_OUTPUTS_CONFIG_fa2a69fad5fd0485_EOF' - {"add_comment":{"discussions":false,"max":3,"required_labels":["agentic-workflows"],"required_title_prefix":"[ci-fix] ","target":"*"},"create_pull_request":{"allowed_base_branches":["main"],"allowed_branches":["ci-fix/**"],"allowed_files":["src/Core/**","src/Controls/**","src/Essentials/**","src/BlazorWebView/**","src/TestUtils/**","src/Templates/**","**/PublicAPI.Unshipped.txt"],"base_branch":"main","draft":true,"labels":["agentic-workflows"],"max":3,"max_patch_files":100,"max_patch_size":4096,"protect_top_level_dot_folders":true,"protected_files":["package.json","bun.lockb","bunfig.toml","deno.json","deno.jsonc","deno.lock","global.json","NuGet.Config","Directory.Packages.props","mix.exs","mix.lock","go.mod","go.sum","stack.yaml","stack.yaml.lock","pom.xml","build.gradle","build.gradle.kts","settings.gradle","settings.gradle.kts","gradle.properties","package-lock.json","yarn.lock","pnpm-lock.yaml","npm-shrinkwrap.json","requirements.txt","Pipfile","Pipfile.lock","pyproject.toml","setup.py","setup.cfg","Gemfile","Gemfile.lock","uv.lock","CODEOWNERS","DESIGN.md","README.md","CONTRIBUTING.md","CHANGELOG.md","SECURITY.md","CODE_OF_CONDUCT.md","AGENTS.md","CLAUDE.md","GEMINI.md"],"protected_files_policy":"request_review","title_prefix":"[ci-fix] "},"create_report_incomplete_issue":{},"missing_data":{},"missing_tool":{},"noop":{"max":1,"report-as-issue":"true"},"push_to_pull_request_branch":{"allowed_files":["src/Core/**","src/Controls/**","src/Essentials/**","src/BlazorWebView/**","src/TestUtils/**","src/Templates/**","**/PublicAPI.Unshipped.txt"],"if_no_changes":"warn","max":3,"max_patch_size":4096,"protect_top_level_dot_folders":true,"protected_files":["package.json","bun.lockb","bunfig.toml","deno.json","deno.jsonc","deno.lock","global.json","NuGet.Config","Directory.Packages.props","mix.exs","mix.lock","go.mod","go.sum","stack.yaml","stack.yaml.lock","pom.xml","build.gradle","build.gradle.kts","settings.gradle","settings.gradle.kts","gradle.properties","package-lock.json","yarn.lock","pnpm-lock.yaml","npm-shrinkwrap.json","requirements.txt","Pipfile","Pipfile.lock","pyproject.toml","setup.py","setup.cfg","Gemfile","Gemfile.lock","uv.lock","CODEOWNERS","DESIGN.md","README.md","CONTRIBUTING.md","CHANGELOG.md","SECURITY.md","CODE_OF_CONDUCT.md","AGENTS.md","CLAUDE.md","GEMINI.md"],"required_labels":["agentic-workflows"],"target":"*","title_prefix":"[ci-fix] "},"report_incomplete":{},"update_pull_request":{"allow_body":true,"allow_title":false,"max":3,"target":"*","update_branch":false}} - GH_AW_SAFE_OUTPUTS_CONFIG_fa2a69fad5fd0485_EOF + cat > "${RUNNER_TEMP}/gh-aw/safeoutputs/config.json" << 'GH_AW_SAFE_OUTPUTS_CONFIG_0644bf1434ca0071_EOF' + {"add_comment":{"discussions":false,"max":6,"required_labels":["agentic-workflows"],"required_title_prefix":"[ci-fix] ","target":"*"},"add_labels":{"allowed":["p/0"],"max":3,"required_labels":["agentic-workflows"],"required_title_prefix":"[ci-fix] ","target":"*"},"create_pull_request":{"allowed_base_branches":["main"],"allowed_branches":["ci-fix/**"],"allowed_files":["src/Core/**","src/Controls/**","src/Essentials/**","src/BlazorWebView/**","src/TestUtils/**","src/Templates/**","**/PublicAPI.Unshipped.txt"],"base_branch":"main","draft":true,"labels":["agentic-workflows"],"max":3,"max_patch_files":100,"max_patch_size":4096,"protect_top_level_dot_folders":true,"protected_files":["package.json","bun.lockb","bunfig.toml","deno.json","deno.jsonc","deno.lock","global.json","NuGet.Config","Directory.Packages.props","mix.exs","mix.lock","go.mod","go.sum","stack.yaml","stack.yaml.lock","pom.xml","build.gradle","build.gradle.kts","settings.gradle","settings.gradle.kts","gradle.properties","package-lock.json","yarn.lock","pnpm-lock.yaml","npm-shrinkwrap.json","requirements.txt","Pipfile","Pipfile.lock","pyproject.toml","setup.py","setup.cfg","Gemfile","Gemfile.lock","uv.lock","CODEOWNERS","DESIGN.md","README.md","CONTRIBUTING.md","CHANGELOG.md","SECURITY.md","CODE_OF_CONDUCT.md","AGENTS.md","CLAUDE.md","GEMINI.md"],"protected_files_policy":"request_review","title_prefix":"[ci-fix] "},"create_report_incomplete_issue":{},"mark_pull_request_as_ready_for_review":{"max":3,"required_labels":["agentic-workflows"],"required_title_prefix":"[ci-fix] ","target":"*"},"missing_data":{},"missing_tool":{},"noop":{"max":1,"report-as-issue":"true"},"push_to_pull_request_branch":{"allowed_files":["src/Core/**","src/Controls/**","src/Essentials/**","src/BlazorWebView/**","src/TestUtils/**","src/Templates/**","**/PublicAPI.Unshipped.txt"],"if_no_changes":"warn","max":3,"max_patch_size":4096,"protect_top_level_dot_folders":true,"protected_files":["package.json","bun.lockb","bunfig.toml","deno.json","deno.jsonc","deno.lock","global.json","NuGet.Config","Directory.Packages.props","mix.exs","mix.lock","go.mod","go.sum","stack.yaml","stack.yaml.lock","pom.xml","build.gradle","build.gradle.kts","settings.gradle","settings.gradle.kts","gradle.properties","package-lock.json","yarn.lock","pnpm-lock.yaml","npm-shrinkwrap.json","requirements.txt","Pipfile","Pipfile.lock","pyproject.toml","setup.py","setup.cfg","Gemfile","Gemfile.lock","uv.lock","CODEOWNERS","DESIGN.md","README.md","CONTRIBUTING.md","CHANGELOG.md","SECURITY.md","CODE_OF_CONDUCT.md","AGENTS.md","CLAUDE.md","GEMINI.md"],"required_labels":["agentic-workflows"],"target":"*","title_prefix":"[ci-fix] "},"report_incomplete":{},"update_pull_request":{"allow_body":true,"allow_title":false,"max":3,"target":"*","update_branch":false}} + GH_AW_SAFE_OUTPUTS_CONFIG_0644bf1434ca0071_EOF - name: Generate Safe Outputs Tools env: GH_AW_TOOLS_META_JSON: | { "description_suffixes": { - "add_comment": " CONSTRAINTS: Maximum 3 comment(s) can be added. Target: *. Supports reply_to_id for discussion threading.", + "add_comment": " CONSTRAINTS: Maximum 6 comment(s) can be added. Target: *. Supports reply_to_id for discussion threading.", + "add_labels": " CONSTRAINTS: Maximum 3 label(s) can be added. Only these labels are allowed: [\"p/0\"]. Target: *.", "create_pull_request": " CONSTRAINTS: Maximum 3 pull request(s) can be created. Title will be prefixed with \"[ci-fix] \". Labels [\"agentic-workflows\"] will be automatically added. Only these labels are allowed: [\"agentic-workflows\"]. PRs will be created as drafts.", + "mark_pull_request_as_ready_for_review": " CONSTRAINTS: Maximum 3 pull request(s) can be marked as ready for review.", "push_to_pull_request_branch": " CONSTRAINTS: Maximum 3 push(es) can be made. The target pull request title must start with \"[ci-fix] \".", "update_pull_request": " CONSTRAINTS: Maximum 3 pull request(s) can be updated. Target: *." }, @@ -594,6 +598,25 @@ jobs: } } }, + "add_labels": { + "defaultMax": 5, + "fields": { + "item_number": { + "issueNumberOrTemporaryId": true + }, + "labels": { + "required": true, + "type": "array", + "itemType": "string", + "itemSanitize": true, + "itemMaxLength": 128 + }, + "repo": { + "type": "string", + "maxLength": 256 + } + } + }, "create_pull_request": { "defaultMax": 1, "fields": { @@ -635,6 +658,24 @@ jobs: } } }, + "mark_pull_request_as_ready_for_review": { + "defaultMax": 1, + "fields": { + "pull_request_number": { + "issueOrPRNumber": true + }, + "reason": { + "required": true, + "type": "string", + "sanitize": true, + "maxLength": 65000 + }, + "repo": { + "type": "string", + "maxLength": 256 + } + } + }, "missing_data": { "defaultMax": 20, "fields": { @@ -1494,7 +1535,7 @@ jobs: uses: actions/github-script@3a2844b7e9c422d3c10d287c895573f7108da1b3 # v9.0.0 env: WORKFLOW_NAME: "CI Failure Fixer (main)" - WORKFLOW_DESCRIPTION: "Periodic pass over open ci-scan tracking issues filed by the main-branch CI\nfailure scanner (.github/workflows/ci-status-main.md). This workflow targets\nthe `main` branch EXCLUSIVELY: it processes only issues labelled ci-scan and\nopens every PR against main. (The net11.0 branch is handled by the parallel\n.github/workflows/ci-status-fix-net11.md — the two are split because gh-aw can\nonly transport a fix relative to ONE static base branch per workflow, and the\nmain↔net11.0 divergence exceeds gh-aw's 10 MB transport-patch cap.) The fixer\nopens ONE draft [ci-fix] PR per actionable issue against main, then WATCHES\nthat PR's own CI on later runs: when the fix's CI comes back red and the red is\ncaused by the fix itself, it pushes a fresh follow-up fix onto the SAME PR\nbranch (never a second PR) — up to 10 attempts — then stops and defers to\nhumans (the open tracking issue is the hand-off surface; a dedicated\n[ci-fix][needs-human] PR is planned but currently deferred — see Step 6). CI on\nthe PR is kicked by a human `/azp run` for now; the loop watches, classifies,\nand re-fixes autonomously between kicks.\nNever mutes tests, but\nde-flakes genuinely flaky ones (deterministic synchronization, no retries /\ntimeout bumps). Always skips visual-regression / screenshot issues." + WORKFLOW_DESCRIPTION: "Periodic pass over open ci-scan tracking issues filed by the main-branch CI\nfailure scanner (.github/workflows/ci-status-main.md). This workflow targets\nthe `main` branch EXCLUSIVELY: it processes only issues labelled ci-scan and\nopens every PR against main. (The net11.0 branch is handled by the parallel\n.github/workflows/ci-status-fix-net11.md — the two are split because gh-aw can\nonly transport a fix relative to ONE static base branch per workflow, and the\nmain↔net11.0 divergence exceeds gh-aw's 10 MB transport-patch cap.) The fixer\nopens ONE draft [ci-fix] PR per actionable issue against main, then WATCHES\nthat PR's own CI on later runs: when the fix's CI comes back red and the red is\ncaused by the fix itself, it pushes a fresh follow-up fix onto the SAME PR\nbranch (never a second PR) — up to 10 attempts — then stops and defers to\nhumans (the open tracking issue is the hand-off surface; a dedicated\n[ci-fix][needs-human] PR is planned but currently deferred — see Step 6). CI on\nthe PR is kicked by a human `/azp run` for now; the loop watches, classifies,\nand re-fixes autonomously between kicks. When the SPECIFIC test a PR fixed is\nconfirmed green in that PR's own CI, the loop marks the draft PR ready for review\n(a state transition only — it never approves or merges).\nNever mutes tests, but\nde-flakes genuinely flaky ones (deterministic synchronization, no retries /\ntimeout bumps). Always skips visual-regression / screenshot issues." HAS_PATCH: ${{ needs.agent.outputs.has_patch }} with: script: | @@ -1910,7 +1951,7 @@ jobs: GH_AW_ALLOWED_DOMAINS: "*.blob.core.windows.net,*.githubusercontent.com,api.business.githubcopilot.com,api.enterprise.githubcopilot.com,api.github.com,api.githubcopilot.com,api.individual.githubcopilot.com,api.snapcraft.io,archive.ubuntu.com,azure.archive.ubuntu.com,codeload.github.com,crl.geotrust.com,crl.globalsign.com,crl.identrust.com,crl.sectigo.com,crl.thawte.com,crl.usertrust.com,crl.verisign.com,crl3.digicert.com,crl4.digicert.com,crls.ssl.com,dev.azure.com,docs.github.com,github-cloud.githubusercontent.com,github-cloud.s3.amazonaws.com,github.blog,github.com,github.githubassets.com,helix.dot.net,host.docker.internal,json-schema.org,json.schemastore.org,keyserver.ubuntu.com,lfs.github.com,objects.githubusercontent.com,ocsp.digicert.com,ocsp.geotrust.com,ocsp.globalsign.com,ocsp.identrust.com,ocsp.sectigo.com,ocsp.ssl.com,ocsp.thawte.com,ocsp.usertrust.com,ocsp.verisign.com,packagecloud.io,packages.cloud.google.com,packages.microsoft.com,patch-diff.githubusercontent.com,ppa.launchpad.net,raw.githubusercontent.com,registry.npmjs.org,s.symcb.com,s.symcd.com,security.ubuntu.com,telemetry.enterprise.githubcopilot.com,ts-crl.ws.symantec.com,ts-ocsp.ws.symantec.com,www.googleapis.com" GITHUB_SERVER_URL: ${{ github.server_url }} GITHUB_API_URL: ${{ github.api_url }} - GH_AW_SAFE_OUTPUTS_HANDLER_CONFIG: "{\"add_comment\":{\"discussions\":false,\"max\":3,\"required_labels\":[\"agentic-workflows\"],\"required_title_prefix\":\"[ci-fix] \",\"target\":\"*\"},\"create_pull_request\":{\"allowed_base_branches\":[\"main\"],\"allowed_branches\":[\"ci-fix/**\"],\"allowed_files\":[\"src/Core/**\",\"src/Controls/**\",\"src/Essentials/**\",\"src/BlazorWebView/**\",\"src/TestUtils/**\",\"src/Templates/**\",\"**/PublicAPI.Unshipped.txt\"],\"base_branch\":\"main\",\"draft\":true,\"labels\":[\"agentic-workflows\"],\"max\":3,\"max_patch_files\":100,\"max_patch_size\":4096,\"protect_top_level_dot_folders\":true,\"protected_files\":[\"package.json\",\"bun.lockb\",\"bunfig.toml\",\"deno.json\",\"deno.jsonc\",\"deno.lock\",\"global.json\",\"NuGet.Config\",\"Directory.Packages.props\",\"mix.exs\",\"mix.lock\",\"go.mod\",\"go.sum\",\"stack.yaml\",\"stack.yaml.lock\",\"pom.xml\",\"build.gradle\",\"build.gradle.kts\",\"settings.gradle\",\"settings.gradle.kts\",\"gradle.properties\",\"package-lock.json\",\"yarn.lock\",\"pnpm-lock.yaml\",\"npm-shrinkwrap.json\",\"requirements.txt\",\"Pipfile\",\"Pipfile.lock\",\"pyproject.toml\",\"setup.py\",\"setup.cfg\",\"Gemfile\",\"Gemfile.lock\",\"uv.lock\",\"CODEOWNERS\",\"DESIGN.md\",\"README.md\",\"CONTRIBUTING.md\",\"CHANGELOG.md\",\"SECURITY.md\",\"CODE_OF_CONDUCT.md\",\"AGENTS.md\",\"CLAUDE.md\",\"GEMINI.md\"],\"protected_files_policy\":\"request_review\",\"title_prefix\":\"[ci-fix] \"},\"create_report_incomplete_issue\":{},\"missing_data\":{},\"missing_tool\":{},\"noop\":{\"max\":1,\"report-as-issue\":\"true\"},\"push_to_pull_request_branch\":{\"allowed_files\":[\"src/Core/**\",\"src/Controls/**\",\"src/Essentials/**\",\"src/BlazorWebView/**\",\"src/TestUtils/**\",\"src/Templates/**\",\"**/PublicAPI.Unshipped.txt\"],\"if_no_changes\":\"warn\",\"max\":3,\"max_patch_size\":4096,\"protect_top_level_dot_folders\":true,\"protected_files\":[\"package.json\",\"bun.lockb\",\"bunfig.toml\",\"deno.json\",\"deno.jsonc\",\"deno.lock\",\"global.json\",\"NuGet.Config\",\"Directory.Packages.props\",\"mix.exs\",\"mix.lock\",\"go.mod\",\"go.sum\",\"stack.yaml\",\"stack.yaml.lock\",\"pom.xml\",\"build.gradle\",\"build.gradle.kts\",\"settings.gradle\",\"settings.gradle.kts\",\"gradle.properties\",\"package-lock.json\",\"yarn.lock\",\"pnpm-lock.yaml\",\"npm-shrinkwrap.json\",\"requirements.txt\",\"Pipfile\",\"Pipfile.lock\",\"pyproject.toml\",\"setup.py\",\"setup.cfg\",\"Gemfile\",\"Gemfile.lock\",\"uv.lock\",\"CODEOWNERS\",\"DESIGN.md\",\"README.md\",\"CONTRIBUTING.md\",\"CHANGELOG.md\",\"SECURITY.md\",\"CODE_OF_CONDUCT.md\",\"AGENTS.md\",\"CLAUDE.md\",\"GEMINI.md\"],\"required_labels\":[\"agentic-workflows\"],\"target\":\"*\",\"title_prefix\":\"[ci-fix] \"},\"report_incomplete\":{},\"update_pull_request\":{\"allow_body\":true,\"allow_title\":false,\"max\":3,\"target\":\"*\",\"update_branch\":false}}" + GH_AW_SAFE_OUTPUTS_HANDLER_CONFIG: "{\"add_comment\":{\"discussions\":false,\"max\":6,\"required_labels\":[\"agentic-workflows\"],\"required_title_prefix\":\"[ci-fix] \",\"target\":\"*\"},\"add_labels\":{\"allowed\":[\"p/0\"],\"max\":3,\"required_labels\":[\"agentic-workflows\"],\"required_title_prefix\":\"[ci-fix] \",\"target\":\"*\"},\"create_pull_request\":{\"allowed_base_branches\":[\"main\"],\"allowed_branches\":[\"ci-fix/**\"],\"allowed_files\":[\"src/Core/**\",\"src/Controls/**\",\"src/Essentials/**\",\"src/BlazorWebView/**\",\"src/TestUtils/**\",\"src/Templates/**\",\"**/PublicAPI.Unshipped.txt\"],\"base_branch\":\"main\",\"draft\":true,\"labels\":[\"agentic-workflows\"],\"max\":3,\"max_patch_files\":100,\"max_patch_size\":4096,\"protect_top_level_dot_folders\":true,\"protected_files\":[\"package.json\",\"bun.lockb\",\"bunfig.toml\",\"deno.json\",\"deno.jsonc\",\"deno.lock\",\"global.json\",\"NuGet.Config\",\"Directory.Packages.props\",\"mix.exs\",\"mix.lock\",\"go.mod\",\"go.sum\",\"stack.yaml\",\"stack.yaml.lock\",\"pom.xml\",\"build.gradle\",\"build.gradle.kts\",\"settings.gradle\",\"settings.gradle.kts\",\"gradle.properties\",\"package-lock.json\",\"yarn.lock\",\"pnpm-lock.yaml\",\"npm-shrinkwrap.json\",\"requirements.txt\",\"Pipfile\",\"Pipfile.lock\",\"pyproject.toml\",\"setup.py\",\"setup.cfg\",\"Gemfile\",\"Gemfile.lock\",\"uv.lock\",\"CODEOWNERS\",\"DESIGN.md\",\"README.md\",\"CONTRIBUTING.md\",\"CHANGELOG.md\",\"SECURITY.md\",\"CODE_OF_CONDUCT.md\",\"AGENTS.md\",\"CLAUDE.md\",\"GEMINI.md\"],\"protected_files_policy\":\"request_review\",\"title_prefix\":\"[ci-fix] \"},\"create_report_incomplete_issue\":{},\"mark_pull_request_as_ready_for_review\":{\"max\":3,\"required_labels\":[\"agentic-workflows\"],\"required_title_prefix\":\"[ci-fix] \",\"target\":\"*\"},\"missing_data\":{},\"missing_tool\":{},\"noop\":{\"max\":1,\"report-as-issue\":\"true\"},\"push_to_pull_request_branch\":{\"allowed_files\":[\"src/Core/**\",\"src/Controls/**\",\"src/Essentials/**\",\"src/BlazorWebView/**\",\"src/TestUtils/**\",\"src/Templates/**\",\"**/PublicAPI.Unshipped.txt\"],\"if_no_changes\":\"warn\",\"max\":3,\"max_patch_size\":4096,\"protect_top_level_dot_folders\":true,\"protected_files\":[\"package.json\",\"bun.lockb\",\"bunfig.toml\",\"deno.json\",\"deno.jsonc\",\"deno.lock\",\"global.json\",\"NuGet.Config\",\"Directory.Packages.props\",\"mix.exs\",\"mix.lock\",\"go.mod\",\"go.sum\",\"stack.yaml\",\"stack.yaml.lock\",\"pom.xml\",\"build.gradle\",\"build.gradle.kts\",\"settings.gradle\",\"settings.gradle.kts\",\"gradle.properties\",\"package-lock.json\",\"yarn.lock\",\"pnpm-lock.yaml\",\"npm-shrinkwrap.json\",\"requirements.txt\",\"Pipfile\",\"Pipfile.lock\",\"pyproject.toml\",\"setup.py\",\"setup.cfg\",\"Gemfile\",\"Gemfile.lock\",\"uv.lock\",\"CODEOWNERS\",\"DESIGN.md\",\"README.md\",\"CONTRIBUTING.md\",\"CHANGELOG.md\",\"SECURITY.md\",\"CODE_OF_CONDUCT.md\",\"AGENTS.md\",\"CLAUDE.md\",\"GEMINI.md\"],\"required_labels\":[\"agentic-workflows\"],\"target\":\"*\",\"title_prefix\":\"[ci-fix] \"},\"report_incomplete\":{},\"update_pull_request\":{\"allow_body\":true,\"allow_title\":false,\"max\":3,\"target\":\"*\",\"update_branch\":false}}" GH_AW_CI_TRIGGER_TOKEN: ${{ secrets.GH_AW_CI_TRIGGER_TOKEN }} with: github-token: ${{ secrets.GH_AW_GITHUB_TOKEN || secrets.GITHUB_TOKEN }} diff --git a/.github/workflows/ci-status-fix.md b/.github/workflows/ci-status-fix.md index d3227e25170c..9c2f46054029 100644 --- a/.github/workflows/ci-status-fix.md +++ b/.github/workflows/ci-status-fix.md @@ -15,7 +15,9 @@ description: | humans (the open tracking issue is the hand-off surface; a dedicated [ci-fix][needs-human] PR is planned but currently deferred — see Step 6). CI on the PR is kicked by a human `/azp run` for now; the loop watches, classifies, - and re-fixes autonomously between kicks. + and re-fixes autonomously between kicks. When the SPECIFIC test a PR fixed is + confirmed green in that PR's own CI, the loop marks the draft PR ready for review + (a state transition only — it never approves or merges). Never mutes tests, but de-flakes genuinely flaky ones (deterministic synchronization, no retries / timeout bumps). Always skips visual-regression / screenshot issues. @@ -239,8 +241,15 @@ safe-outputs: # PER-RUN total (not per-PR): a sweep may surface several green PRs and/or # annotate several flaky reds in one run. At max:1 all but the first comment # were silently dropped, starving the workflow's #1 value (surfacing green PRs - # for review). Raised to 3 to match the per-run throughput of the other outputs. - max: 3 + # for review). Sized to 6 (not 3) because a single green DRAFT PR that gets + # flipped ready in one sweep spends TWO comment slots — the Step 3 ✅ surface + # comment (which still names the /azp-gated legs a human must kick, so it is NOT + # redundant with 🎯) AND the Step 3.6 T3 🎯 readiness comment. At max:3 the shared + # bucket drained after ~1 draft flip and T3's atomicity pre-check then deferred + # every further mark-ready even while the mark-ready/add_labels buckets (also 3) + # sat idle; 6 lets ~3 draft flips (2 comments each) land per sweep, so the + # mark-ready:3 / add_labels:3 caps are actually reachable. + max: 6 target: "*" # Hard constraint (defense-in-depth): only comment on THIS workflow's own # ci-fix PRs — [ci-fix] title prefix AND agentic-workflows label. Mirrors the @@ -261,13 +270,51 @@ safe-outputs: # never the title, so disable title rewrites — this compiles to allow_title:false # and removes any ability to retitle an arbitrary PR. title: false - # NOTE: gh-aw v0.79.8 does NOT emit required-title-prefix/required-labels into + # NOTE: gh-aw v0.80.9 (the pinned compiler) does NOT emit required-title-prefix/required-labels into # the compiled config for update-pull-request (verified against the lock — it # silently drops them, unlike add-comment / push-to-pull-request-branch which # honor them). So which-PR scoping here relies on prompt Hard-Rule 6 + # min-integrity:approved; the residual is body-marker edits only (no code, no # merge, no close), capped at max:3. Revisit required-* if a future gh-aw # compiler emits them for this output. + mark-pull-request-as-ready-for-review: + # Flip a validated draft [ci-fix] PR to ready-for-review once the SPECIFIC test + # this PR was opened to fix is confirmed green in the PR's OWN CI (Step 3.6) — + # even when unrelated legs are red. The workflow is schedule/dispatch-triggered + # (no triggering-PR context), so target "*" lets the agent name the PR number it + # validated. This is a state transition ONLY: it never approves and never merges — + # a human still reviews and merges. + # PER-RUN total (not per-PR): a sweep may validate several PRs' target tests green. + max: 3 + target: "*" + # Hard constraint (defense-in-depth): only ever un-draft THIS workflow's own + # ci-fix PRs — [ci-fix] title prefix AND agentic-workflows label — so a confused + # or prompt-injected agent cannot mark an arbitrary PR ready. (If the v0.80.9 + # compiler silently drops these — as documented for update-pull-request above — + # the Step 3.6 preconditions + min-integrity:approved are the compensating scope + # controls; verify against the lock after compiling.) + required-title-prefix: "[ci-fix] " + required-labels: [agentic-workflows] + add-labels: + # Apply the p/0 priority label to a [ci-fix] PR at the exact moment the loop flips it + # from draft to ready-for-review (Step 3.6 T3) — so a validated, review-ready fix lands + # in the team's p/0 triage queue instead of sitting unseen in the draft backlog. Paired + # 1:1 with mark-pull-request-as-ready-for-review; target "*" lets the agent name the PR + # it just validated. + # allowed = HARD allowlist: the agent may ONLY ever add p/0, nothing else. This caps the + # blast radius of a confused/prompt-injected agent to exactly one benign priority label — + # it can never apply a downstream-triggering or destructive label. + allowed: [p/0] + # PER-RUN total (not per-PR): a sweep may mark several PRs ready in one run; each adds + # one label, so this matches the mark-ready per-run cap (3). + max: 3 + target: "*" + # Hard constraint (defense-in-depth): only ever label THIS workflow's own ci-fix PRs — + # [ci-fix] title prefix AND agentic-workflows label. (If the v0.80.9 compiler drops these + # — as documented for update-pull-request above — the Step 3.6 preconditions + + # min-integrity:approved + the allowed:[p/0] allowlist are the compensating controls.) + required-title-prefix: "[ci-fix] " + required-labels: [agentic-workflows] timeout-minutes: 90 @@ -339,33 +386,48 @@ through `safe-outputs`. 5. **One issue = one outcome per run.** Exactly one of: respond to a maintainer's change-request on the open PR (Track C, Step 3.5.R); open the first fix/help/ de-flake PR; advance an existing PR by one attempt (push a follow-up fix); - surface a validated-green PR for review; annotate an unrelated-flake red; wait + surface a validated-green PR for review; mark a target-validated draft PR + ready-for-review (Step 3.6 T3 — the terminal outcome that supersedes the same + run's surface/annotate precursor line), or record its `already-ready` + steady-state no-op; annotate an unrelated-flake red; wait (CI not yet settled); or a recorded skip (the dedicated needs-human PR is deferred — the attempt cap records a skip instead; see Step 6). Always prefer advancing or opening a PR over a skip when a non-mute diff is producible. 6. **All writes via `safe-outputs`.** Allowed outputs: `create_pull_request` (first attempt only), `push_to_pull_request_branch` (advance an existing PR by one attempt), `update_pull_request` (bump the attempt marker / refresh the - prior-attempts table), and `add_comment` (progress notes on the `[ci-fix]` PR - ONLY). NEVER comment on the tracking issue (issues are locked by + prior-attempts table), `add_comment` (progress notes on the `[ci-fix]` PR + ONLY), `mark_pull_request_as_ready_for_review` (flip a target-validated draft + `[ci-fix]` PR from draft to ready — Step 3.6 T3 only), and `add_labels` (add + ONLY the `p/0` label, ONLY on that same draft→ready transition — Step 3.6 T3). + NEVER comment on the tracking issue (issues are locked by `.github/workflows/ci-scan-lock-issues.yml`) — `add_comment` targets the PR. No `gh pr create`, no manual `git push`. **Defense-in-depth:** `add_comment` and `update_pull_request` use `target: "*"` (the agent supplies the PR number). - `add_comment` is now config-locked to `required-title-prefix: "[ci-fix] "` + + `add_comment`, `mark_pull_request_as_ready_for_review`, and `add_labels` are + config-locked to `required-title-prefix: "[ci-fix] "` + `required-labels: [agentic-workflows]` (hard-enforced by the handler, same as - `push_to_pull_request_branch`), so a comment can only ever land on THIS - workflow's own PRs. `update_pull_request` CANNOT be config-locked to a - title/label in gh-aw v0.79.8 (the compiler silently drops `required-*` for that - output — verified against the lock), so it keeps `title: false` (no retitles; - body-marker edits only) plus this prompt-level guard. Before emitting either, - VERIFY the target PR carries BOTH the `[ci-fix]` title prefix AND the - `agentic-workflows` label; never comment on or edit the body of any PR that - lacks both. + `push_to_pull_request_branch`), and `add_labels` additionally has an + `allowed: [p/0]` allowlist so it can ONLY ever add `p/0` — so these can only + ever land on THIS workflow's own PRs. `update_pull_request` CANNOT be + config-locked to a title/label in gh-aw v0.80.9 (the compiler silently drops + `required-*` for that output — verified against the lock), so it keeps + `title: false` (no retitles; body-marker edits only) plus this prompt-level + guard. Before emitting ANY of these, VERIFY the target PR carries BOTH the + `[ci-fix]` title prefix AND the `agentic-workflows` label; never comment on, + edit, un-draft, or label any PR that lacks both. 7. **Per-run safe-output caps (per RUN, not per PR).** Each output type is capped per run: `create_pull_request` 3, `push_to_pull_request_branch` 3, - `add_comment` 3, `update_pull_request` 3. Note an ADVANCE spends one push **and** - one update **and** one comment, so ≤ 3 advances/run; surfacing a green or - annotating a flake spends one comment. When a bucket is exhausted, do NOT keep + `add_comment` 6, `update_pull_request` 3, `mark_pull_request_as_ready_for_review` + 3, `add_labels` 3 (counts LABELS, not calls). Note an ADVANCE spends one push + **and** one update **and** one comment, so ≤ 3 advances/run; surfacing a green or + annotating a flake spends one comment; a **mark-ready (Step 3.6 T3)** of a + still-draft PR spends TWO comments (the Step 3 ✅ surface **and** the T3 🎯 + readiness note) **and** one mark-ready **and** one label as an ALL-OR-NOTHING set + (T3 pre-checks those buckets and defers the whole PR if any is exhausted — never + mark a PR ready without its 🎯 audit comment). `add_comment` is therefore sized 6 + (not 3) so ~3 draft flips, at 2 comments each, can land in one sweep instead of the + shared comment bucket starving the otherwise-idle mark-ready/add_labels buckets. When a bucket is exhausted, do NOT keep emitting (extras are silently dropped) — record `skipped: per-run cap reached; deferring PR # to next cycle` for each remaining PR so the drop is a deliberate, logged decision. The continuous loop picks the deferred PRs up next @@ -409,8 +471,9 @@ For every open tracking issue in scope, converge on exactly one outcome: | Open first help-wanted draft `[ci-fix]` PR | No open PR yet; a plausible candidate exists but cannot be runner-validated (device/UI tests) (attempt 1) | | Open first de-flake draft `[ci-fix]` PR | No open PR yet; failure is intermittent (green-on-retry) from a genuine test-quality defect; a deterministic-synchronization fix is producible (attempt 1, Step 4.7 bucket b) | | Advance an existing PR (push attempt N+1) | Open PR's own CI settled red *because of the fix itself*, no human engaged, marker < 10 — push a NEW distinct fix onto the same branch (Step 3.5 → 5.6) | -| Surface a validated-green PR | Open PR's own CI settled green — comment "validated, attempt N/10; ready for review" and do NOT advance (Step 3.5) | -| Annotate an unrelated-flake red | Open PR is red only on baseline-flake / unrelated legs — comment which leg needs a re-run; do NOT burn an attempt (Step 3.5) | +| Surface a validated-green PR | Open PR's own CI settled green — comment "validated, attempt N/10; ready for review" and do NOT advance (Step 3.5), then mark the draft PR ready for review once the fixed test is confirmed green (Step 3.6) | +| Annotate an unrelated-flake red | Open PR is red only on baseline-flake / unrelated legs — comment which leg needs a re-run; do NOT burn an attempt (Step 3.5); if the SPECIFIC fixed test is confirmed green on that build, still mark the draft PR ready for review (Step 3.6) | +| Mark a validated PR ready for review | The specific test this PR fixed is confirmed green in the PR's own CI (even if unrelated legs are red) — comment the target-test result and transition the draft PR to ready for review; never approve or merge (Step 3.6) | | Wait | Open PR's CI is pending / not yet settled on the current head SHA — do nothing this cycle (Step 3.5) | | Hand-off skip (attempt cap) | Marker == 10 and the signature still reproduces — stop and defer to humans; the dedicated `[ci-fix][needs-human]` PR is planned but currently deferred (Step 6) | | Recorded skip | Visual-regression, human engaged, already-handled, fixed-in-latest-build, infra-flake, out-of-bounds, only-mute-available, or no novel approach producible | @@ -685,14 +748,34 @@ Run these gates in order — the FIRST that fires decides this cycle's outcome: 1. **Human engaged.** If `C.humanEngaged` → `skipped: human engaged on PR #; deferring` and stop. Never fight a human reviewer — the intentional hand-off boundary still holds the moment a person touches the PR. -2. **CI not settled / unknown / incomplete prefetch.** If `C.checksSettled == false` - OR `C.overallConclusion` is `pending`, `neutral`, or `unknown`, OR - `C.dataComplete == false` → the fix's CI has not finished on the current head SHA - (`C.headSha`), or the prefetch could not fully read this PR (a partial read can - understate `humanEngaged` / `attempt`). `skipped: PR # CI pending / prefetch - incomplete on ; waiting` and stop. *(Round 1: this is where a - maintainer `/azp run` is awaited — the `/azp`-gated uitests/devicetests legs will - not have run until a human kicks them.)* +2. **CI not settled — but validate the target test first (target-focused readiness).** + If `C.checksSettled == false` OR `C.overallConclusion` is `pending`, `neutral`, or + `unknown`, OR `C.dataComplete == false` → the *overall* build has not finished on the + current head SHA (`C.headSha`), or the prefetch could not fully read this PR (a partial + read can understate `humanEngaged` / `attempt`). Unrelated legs still draining must NOT + keep an already-proven fix parked as a draft, so before waiting, try the target-focused + fast-path: + - **2a — Target-green fast-path (draft `[ci-fix]` PRs only).** If `C.dataComplete == + true` AND the Step 3.6 preconditions hold (draft PR, `[ci-fix] ` title prefix AND the + `agentic-workflows` label), run Step 3.6 **T1–T2** against `C.headSha` now: identify + the target test(s) and read the PR's OWN build **timeline / per-leg status** (anonymous + `_apis/build` — never `_apis/test`, per Hard-Rule 8). If EVERY target test is + **VALIDATED-GREEN** (per T2's all-platform definition below — the target's category leg + is `succeeded` on every platform that runs it, failed on none, and NO target platform + family the pipeline covers is still pending/unconcluded, per T2's "no platform left + unverified" rule; a platform that simply has no leg for the target's category — the test + doesn't run there — is fine, not a gap) AND every leg in + `C.failedLegs` that has ALREADY concluded classifies as **unrelated flake** (Step 4 / + Step 4.7 method — the fix is implicated in NO completed red), then the fix is proven + regardless of unrelated *pending* legs → run **Step 3.6 T3** (mark ready + 🎯 comment) + and stop. This early-out NEVER advances an attempt and NEVER acts on a red that could + be the fix's fault. + - **2b — Otherwise WAIT.** If the target test has not yet executed (pending/absent on + its leg), or a completed red is (or may be) caused by the fix, or `C.dataComplete == + false`, or this PR has no identifiable target test → `skipped: PR # CI pending / + target not yet validated on ; waiting` and stop. *(Round 1: this is where a + maintainer `/azp run` is awaited — the `/azp`-gated uitests/devicetests legs will not + have run until a human kicks them.)* 3. **Green → surface for review.** If `C.overallConclusion == "success"`: the checks that RAN are green. **Primary-gate check first:** the deterministic `success` verdict only certifies "at least one green check, nothing failing or @@ -704,18 +787,31 @@ Run these gates in order — the FIRST that fires decides this cycle's outcome: # primary CI gate (maui-pr) not green on ; waiting` and stop — do NOT surface. (The `/azp`-gated `maui-pr-uitests` (def 313) / `maui-pr-devicetests` (def 314) legs MAY still be un-run — that is expected and is named below, not a - reason to withhold the surface.) **Idempotency:** scan the PR's existing comments - for a prior bot `✅ … validated … on ` note for THIS head SHA — if one - already exists, record `already-surfaced PR # (head )` and stop - WITHOUT re-commenting (re-surfacing the same green every tick is noise and burns - the per-run comment budget other PRs need). Otherwise resolve the attempt number from - `C.effectiveAttempt` (the authoritative max(marker, bot-commit) counter; omit the - number only if it is somehow indeterminate). **Dry-run gate (Step 0):** if `dry_run == "true"`, do NOT emit any comment — instead print the intended `✅` validated-green body to the run log, tally `dry-run: would-surface-green PR #`, and stop. Otherwise `add_comment` on PR #: `✅ Attempt /10 validated - — the fix's CI is green on . Ready for human review.` - Name any `/azp`-gated legs (uitests def 313 / devicetests def 314) that have not - run and still need a maintainer `/azp run`. Do NOT advance. Record - `surfaced-green PR # (attempt /10)` and stop. *(This directly - attacks the real bottleneck — no reviews — so it is the highest-value outcome.)* + reason to withhold the surface.) **Comment idempotency + dry-run suppress the ✅ + comment ONLY — neither skips Step 3.6.** Scan the PR's existing comments for a prior + bot `✅ … validated … on ` note for THIS head SHA; if one exists, set + `SKIP_SURFACE_COMMENT = true`. Resolve the attempt number from `C.effectiveAttempt` + (the authoritative max(marker, bot-commit) counter; omit the number only if it is + somehow indeterminate). Then post the surface comment UNLESS suppressed: + - if `dry_run == "true"`: do NOT emit — print the intended `✅` validated-green body to + the run log and tally `dry-run: would-surface-green PR #`; + - else if `SKIP_SURFACE_COMMENT`: do NOT re-post — record `already-surfaced PR # + (head )` (re-surfacing the same green every tick is noise and burns the + per-run comment budget other PRs need); + - else `add_comment` on PR #: `✅ Attempt /10 validated — the fix's CI is + green on .` naming any `/azp`-gated legs (uitests def 313 / devicetests def + 314) that have not run and still need a maintainer `/azp run`; record `surfaced-green + PR # (attempt /10)`. (Do NOT assert "ready for human review" on a PR that + is still a draft — Step 3.6, not this comment, owns the draft→ready flip and posts its + own 🎯 announcement when it fires.) + Do NOT advance. Then — in ALL of the above cases — run **Step 3.6** (target-test + readiness gate) for this PR before stopping: its `/azp`-gated target legs may conclude + green on this SAME head SHA on a LATER sweep (an `/azp run` adds no commit), and Step + 3.6 is the ONLY place the draft→ready flip happens, so it MUST re-evaluate every sweep — + never `stop` here before it. *(This directly attacks the real bottleneck — no reviews — + so it is the highest-value outcome.)* 4. **Red → classify caused-by-fix vs unrelated-flake.** If `C.overallConclusion == "failure"`, analyze the PR's OWN failing build (NOT `main`): find the AzDO `maui-pr` build for this PR (filter builds by `branchName=refs/pull//merge`, @@ -723,18 +819,26 @@ Run these gates in order — the FIRST that fires decides this cycle's outcome: method and Step 4.7 flake buckets to `C.failedLegs`. - **Unrelated flake only** (every failed leg is known-flaky / infra / a pre-existing baseline red NOT introduced by the fix): do NOT burn an attempt. - **Idempotency first:** scan the PR's existing comments for a prior bot - `♻️ … unrelated flake … on ` note for THIS head SHA — if one already - exists, record `already-annotated-flake PR # (head )` and stop - WITHOUT re-commenting (re-annotating the same flake every 12h sweep is noise and - burns the per-run comment budget other PRs need). Otherwise resolve the attempt - number from `C.effectiveAttempt` (the authoritative max(marker, bot-commit) - counter), omitting the `Attempt /10:` prefix only if it is somehow - indeterminate. **Dry-run gate (Step 0):** if `dry_run == "true"`, do NOT emit any comment — instead print the intended `♻️` unrelated-flake body to the run log, tally `dry-run: would-annotate-flake PR #`, and stop. Otherwise `add_comment` on PR #: `♻️ Attempt /10: red is - unrelated flake on leg(s) () on ; the fix itself is not - implicated. A maintainer re-run (/azp run ) should clear it.` Record - `annotated-flake PR # (head )` and stop. *(Round 1: human re-runs; - Round 2: auto re-trigger.)* + **Comment idempotency + dry-run suppress the ♻️ comment ONLY — neither skips Step + 3.6.** Scan the PR's existing comments for a prior bot `♻️ … unrelated flake … on + ` note for THIS head SHA; if one exists, set `SKIP_FLAKE_COMMENT = true`. + Resolve the attempt number from `C.effectiveAttempt` (the authoritative max(marker, + bot-commit) counter), omitting the `Attempt /10:` prefix only if it is + somehow indeterminate. Then post the flake note UNLESS suppressed: + - if `dry_run == "true"`: do NOT emit — print the intended `♻️` unrelated-flake body + to the run log and tally `dry-run: would-annotate-flake PR #`; + - else if `SKIP_FLAKE_COMMENT`: do NOT re-post — record `already-annotated-flake PR + # (head )` (re-annotating the same flake every 12h sweep is noise and + burns the per-run comment budget other PRs need); + - else `add_comment` on PR #: `♻️ Attempt /10: red is unrelated flake on + leg(s) () on ; the fix itself is not implicated. A + maintainer re-run (/azp run ) should clear it.` record `annotated-flake + PR # (head )`. + Then — in ALL of the above cases — run **Step 3.6** (target-test readiness gate) for + this PR before stopping: its `/azp`-gated target legs may conclude green on this SAME + head SHA on a LATER sweep, and Step 3.6 is the ONLY place the draft→ready flip + happens, so it MUST re-evaluate every sweep — never `stop` here before it. *(Round 1: + human re-runs; Round 2: auto re-trigger.)* - **Caused by the fix** (a failed leg still matches the original target signature, or the fix introduced a NEW failure): advance an attempt. - **Attempt count.** `attempt = C.effectiveAttempt` — the authoritative @@ -752,6 +856,188 @@ Run these gates in order — the FIRST that fires decides this cycle's outcome: from every prior commit on this branch**, emitted via the Step 5.6 ADVANCE path (push onto the same PR). +#### Step 3.6 — Target-test verification & mark-ready gate + +Reached from the Step 3 **green-surface** branch, the Step 4 **unrelated-flake** branch +(AFTER that branch has posted its comment), and the Step 3.5 **gate-2 target-focused +fast-path** (2a — while unrelated legs are still pending). Purpose: confirm the SPECIFIC +test(s) this PR was opened to fix now PASS in the PR's own CI, and — only when they do +— transition the draft `[ci-fix]` PR to **ready for review** so a maintainer sees a +validated fix instead of a draft. This is the ONLY place the loop flips draft→ready; it +is a state transition, **never** an approval or a merge (a human still reviews and +merges). Overall red on *unrelated* legs must NOT gate readiness — we validate the fix, +not the base branch's flakiness. + +**Preconditions** (ALL must hold; otherwise record `skipped: readiness N/A PR # +()` and stop this gate): +- `C.isDraft == true` — if the PR is already ready-for-review, the draft→ready + transition is already done; do NOT re-mark **and do NOT re-apply `p/0`**. `p/0` is applied + exactly ONCE, atomically with the draft→ready flip (T3 — all-or-nothing with the 🎯 audit + comment + mark-ready), so an already-ready loop PR that is MISSING `p/0` has almost + certainly had it **removed by a maintainer** de-prioritizing the PR — a pure triage action + the human-engagement guard (which inspects comments/reviews/commits, NOT label changes) + cannot see. Re-adding `p/0` every sweep would fight that maintainer indefinitely, violating + the loop's "never override a human" contract. So the loop does NOT reconcile the label: + record `already-ready PR #` and stop this gate. (Trade-off: on the rare occasion a + transient API error drops `p/0` at flip time *after* the T3 atomic pre-check passed, it is + not auto-re-added — but the PR is still ready-for-review with its 🎯 audit comment, and + re-adding `p/0` is a trivial manual action; that is strictly preferable to steam-rolling a + maintainer's deliberate de-prioritization.) +- The PR is unmistakably THIS workflow's own: `[ci-fix] ` title prefix AND the + `agentic-workflows` label (mirrors the safe-output lock; the compensating scope + control if the v0.80.9 compiler drops the declarative required-*). +- You reached this gate from Step 3 (green), Step 4 (**unrelated-flake**), or the Step 3.5 + gate-2 **target-focused fast-path** (2a) — i.e. the fix is NOT implicated in any red that + has concluded. If Step 4 classified a red as **caused by the fix** (ADVANCE), do NOT run + this gate — advance the attempt instead. + +**T1 — Identify the target test(s).** From the `[ci-scan]` issue signature the fix +addresses (and the PR's own diff), extract the fully-qualified test method name(s) the +fix targets — e.g. `SafeAreaShouldWorkOnAllShellTabs`. For a de-flake it is the +de-flaked test; for a product fix it is the originally-failing test(s). If NO specific +test can be identified (e.g. a product build-break rather than a test failure), this is a +**build-only fix** — there is no single test to validate cross-platform, so validate at +**whole-build** granularity rather than stopping (Step 3 only *comments*; Step 3.6 is the +sole draft→ready flip, so a build-only fix is undrafted HERE or not at all). We only reach +this gate from Step 3 (green) / Step 4 (unrelated-flake) / gate-2a, so the fix is not +implicated in any concluded red; additionally require, via the T2 timeline method on +`C.headSha`, that the primary build pipeline `maui-pr` (def 302) has CONCLUDED with EVERY +one of its platform build legs `succeeded`/`completed` and NONE failed/canceled or still +pending — the build is green on **every** platform, not just the originally-broken one (the +cross-platform guard, applied to the build instead of a test). A build-only fix is undrafted +ONLY on the strength of the auto-running `maui-pr` (def 302) whole-build green, which the +loop CAN observe: if the PR's `[ci-scan]` issue signature or its own diff indicates the +ORIGINATING failure was in a `/azp`-gated pipeline (`maui-pr-uitests` def 313 or +`maui-pr-devicetests` def 314) rather than `maui-pr` (def 302), do NOT undraft on +`maui-pr`-green alone — those pipelines do not auto-run on this PR (GITHUB_TOKEN cannot +trigger them), so a green `maui-pr` build is NOT evidence the gated build break is fixed; +record `skipped: build-only fix PR # targets gated pipeline () not run — +deferring to human` and stop this gate. Otherwise, set `TARGET := "the maui-pr build +(build-only fix — no single target test)"` and proceed to **T3** to mark ready. If any `maui-pr` build leg is still unconcluded, record `skipped: build-only fix PR +# not yet whole-build green (leg(s) pending)` and stop this gate WITHOUT marking +ready (a green subset is not enough — a still-pending build leg could yet fail). + +**T2 — Verify the target test's platform legs are green (anonymous, leg-level).** Per-test +outcomes come from the AzDO **test-results** API (`_apis/test/...`), which is **NOT reachable +anonymously — Hard-Rule 8**: those endpoints 302-redirect to a sign-in page on this runner, so +a `curl` returns an HTML redirect (not JSON) and every target test would look "not executed". +Validate instead at **leg granularity** using ONLY the anonymous `_apis/build/...` timeline +(Hard-Rule 8) — a `succeeded` category leg means every test in that category **that actually +ran** passed on that platform. This is a strong bar **only if the target test genuinely +executes**: a job can go green while one specific test is skipped at runtime (`[Ignore]`, +`Assert.Ignore()`, `Assert.Inconclusive()`, a `Skip=`/conditional `[Fact]`, an `#if`-out, or +an early `return` before the asserts). So before trusting a green leg as proof the target +passed, **confirm from the PR's OWN diff that the fix does NOT skip, ignore, disable, or +short-circuit the target test** (it adds no `[Ignore]`/`[Explicit]`, `Assert.Ignore`/ +`Assert.Inconclusive`, `Skip=`, category-exclusion, `#if`-out, or early `return` around the +target). If the fix could cause the target to be runtime-skipped rather than genuinely pass, +leg-level green is NOT sufficient evidence — record `skipped: target test may be +runtime-skipped by this fix (diff adds ); leg-green insufficient, deferring to human +on PR #` and stop this gate WITHOUT marking ready. Otherwise (the target genuinely runs +and asserts, as a de-flake or product fix does) a green category leg cannot hide a +target-test failure, so treat it as authoritative. + +Map the target test to the CI leg(s) that run it. A leg name encodes platform + test category +(e.g. `Android UITests SafeAreaEdges,Shadow`, `iOS UITests SafeAreaEdges,Shadow`, `macOS +UITests SafeAreaEdges,Shadow`, `Windows UITests SafeAreaEdges,Shadow`). Determine the target +test's UI-test category — its `[Category(UITestCategories.X)]` in the test/HostApp source, +visible in the PR diff or the test file — to know which leg-name substring identifies its legs; +for a device test, the per-platform device-test legs. + +Using the SAME build-discovery as Step 4 (filter AzDO builds by `branchName=refs/pull//merge` +or `sourceVersion == C.headSha`), read each build's **timeline** on `C.headSha` for the +pipeline(s) that RUN the target test — `maui-pr` (def 302) for unit/integration tests, +`maui-pr-uitests` (def 313) for Appium UI tests, `maui-pr-devicetests` (def 314) for device +tests: + +```bash +ORG=dnceng-public; PROJ=public; BUILD= +# Anonymous + Hard-Rule-8-compliant. NEVER call _apis/test/... (it 302-redirects to sign-in). +# Each type=="Job" record is one platform leg: result (succeeded/failed/canceled/null), +# state (completed/inProgress/pending), name (encodes " UITests "): +curl -s "https://dev.azure.com/$ORG/$PROJ/_apis/build/builds/$BUILD/timeline?api-version=7.1" \ + | tee /tmp/gh-aw/agent/timeline_${BUILD}.json \ + | jq -r '.records[] | select(.type=="Job") | "\(.result)\t\(.state)\t\(.name)"' +``` + +(The prefetch already exposes per-leg status in `C.failedLegs` / the checks context, and +`gh pr checks ` lists the same per-leg rows with platform+category in the name — use +whichever is handy; the build timeline is the authoritative cross-check on the exact build.) + +Treat a target test as **VALIDATED-GREEN** only if, on `C.headSha`, the leg that runs its +category is green on **every platform it runs on** — a fix that repairs one platform must not +silently regress the same test's leg on another, so a green leg on the originally-red platform +alone is NOT enough. Across the target pipeline's platform legs (for a UI test: the Android, +iOS, Windows, and macOS/MacCatalyst legs running the target's category; for a device test: each +device-test platform), require ALL of: +- the target's category leg is `result == "succeeded"` **and** `state == "completed"` on + **every** platform that runs it, AND +- that leg is `failed` / `canceled` / aborted on **NO** platform, AND +- **no platform is left unverified.** For each platform family the target pipeline covers, that + platform's category leg must have CONCLUDED (`state == "completed"`) on `C.headSha`. If a + platform simply has no leg for the target's category, the test does not run there — that is + fine, not a gap. But if a platform's category leg has **not concluded** (`state` is + `inProgress` / pending, or an `/azp`-gated `maui-pr-uitests` / `maui-pr-devicetests` leg that + has not been kicked), the test's status on that platform is UNKNOWN → the fix is NOT yet + validated across platforms: record `skipped: target test green on but + not yet verified on (leg(s) pending / need /azp run) on PR #` + and stop this gate WITHOUT marking ready. + +A target test whose category leg never concluded on ANY platform (**not executed** anywhere — +e.g. its `/azp`-gated pipeline has not been kicked) is likewise NOT validated: record `skipped: +target test not yet executed on PR # ( not run — needs /azp run)` and stop +this gate WITHOUT marking ready. Do NOT overclaim — a green *sibling* leg (a different category +on the same platform) is not the target's leg, and a green leg on one platform is not a pass on +the others. + +**T3 — Mark ready + report.** If EVERY target test is VALIDATED-GREEN. The 🎯 comment, +the mark-ready, and the `p/0` label are THREE SEPARATE safe-outputs — the comment +existing does NOT prove the mark-ready took effect, so they are tracked independently: + +- **Comment idempotency (dup-suppress only — never gates the mark-ready).** We only reach + T3 while `C.isDraft == true` (the precondition stops an already-ready PR before here). So + if a prior bot comment for THIS head that carries the leading `🎯` anchor AND the + contiguous phrase `validated green on ` already exists (BOTH T3 variants — + `🎯 Target test validated green on ` and the build-only `🎯 Build validated green on + ` — contain that exact contiguous phrase, so this recognizes both; keying on + `target test` alone would instead miss the build-only comment and re-post it every sweep. + **Require the `🎯` anchor** so the match can NEVER be loosened to a gapped + `validated…green…on` form that would false-positive on the Step 3 `✅ … validated — the + fix's CI is green on ` surface comment and wrongly suppress the audit 🎯), the comment + landed on an earlier sweep but the **mark-ready did NOT take + effect** (the PR is still a draft) — set `SUPPRESS_COMMENT = true` (do not re-post the + duplicate 🎯 comment) but STILL complete the mark-ready + label below. Never treat the + comment's existence as "already marked ready" while the PR is still a draft. Record this + case as `re-marking PR # (🎯 present; mark-ready did not take on a prior sweep)`. +- **Atomicity budget pre-check (Hard-Rule 7).** Determine the outputs this gate will emit: + `mark_pull_request_as_ready_for_review` + `add_labels` always, plus `add_comment` unless + `SUPPRESS_COMMENT`. Verify a free per-run slot remains in EACH bucket you are about to + use. If ANY required bucket is exhausted, emit NONE of them — record `skipped: per-run + cap reached; deferring mark-ready PR # to next cycle` and stop this gate. Never mark a + PR ready (or label it) unless its 🎯 audit comment is GUARANTEED to exist for this exact + head SHA — either posted in this same sweep, or (in the `SUPPRESS_COMMENT` re-marking case) + already present from a prior sweep. The outputs you DO emit either all land or all defer + together; the audit trail must never be absent, but it is NOT re-posted when it already exists. +- **Dry-run gate (Step 0):** if `dry_run == "true"`, emit NOTHING — print the intended + readiness comment and "would mark ready + add p/0" to the run log, tally `dry-run: + would-mark-ready PR #`, and stop. +- Otherwise emit for THIS PR number ``: + 1. `add_comment` (ONLY if not `SUPPRESS_COMMENT`): `🎯 Target test validated green on + — passed on ALL platforms it runs on (, + buildId ). — not + caused by this fix."> Transitioning this PR from draft to ready for review and adding + `p/0`; a maintainer still reviews and merges.` (For a **build-only fix** — the T1 + whole-build fallback, no single target test — phrase the first clause as `🎯 Build + validated green on — the maui-pr build passed on ALL platforms (buildId + ).` instead of naming a test.) + 2. `mark_pull_request_as_ready_for_review` with `reason:` a one-line justification + naming the validated test(s) and ``. + 3. `add_labels` with `labels: ["p/0"]` for PR # — put the now-review-ready fix into + the team's p/0 priority queue so it is triaged, not lost in the draft backlog. (If + the PR somehow already carries `p/0`, this is a harmless no-op.) +- Record `marked-ready PR # (target green on ALL platforms on , + labeled p/0)` and stop. + #### Step 3.5.R — Maintainer change-request response (Track C) Reached from Step 3.5 gate 0. Goal: when an **eligible human reviewer** (Hard-Rule @@ -1383,7 +1669,19 @@ Filed by [`ci-status-fix`](https://github.com/dotnet/maui/blob/main/.github/work ### Step 8 — Per-issue tally + end-of-run summary -Per issue, append one outcome line to `/tmp/gh-aw/agent/coverage.txt`: +Per issue, append **one** outcome line to `/tmp/gh-aw/agent/coverage.txt` — the +terminal outcome for the cycle. When one cycle produces a chained pair — a PR gets a +same-cycle precursor line and **then** a Step 3.6 readiness line (`marked-ready` on the +draft→ready flip, or `already-ready` on a later steady-state sweep of an already-ready +PR) — record ONLY the terminal Step 3.6 readiness line: it supersedes ANY same-cycle +non-readiness precursor for that PR, so an aggregator keying on one-line-per-issue never +double-counts. The precursor may be a Step 3 green line (`surfaced-green` on the first ✅, +or `already-surfaced` when it already exists) OR — when an unrelated-flake red and a +target-green coincide — a Step 4 `annotated-flake` line; either way the readiness line is +terminal, and the superseded precursor's signal survives in the PR's 🎯/♻️ comment so no +information is lost. This covers the flip cycle (`surfaced-green` → `marked-ready`), the +post-flip steady state (`already-surfaced` → `already-ready`), and the flake-coincident +flip (`annotated-flake` → `marked-ready`): ``` # main attempt- @@ -1394,8 +1692,14 @@ Per issue, append one outcome line to `/tmp/gh-aw/agent/coverage.txt`: `advance-PR # attempt /10` (ADVANCE mode — new commit pushed to the existing PR), `surfaced-green PR #` (fix's own CI went green; commented for review, did not advance), `annotated-flake PR #` (red was unrelated flake; -commented, attempt NOT burned), `waiting PR #` (CI not settled yet), -`dry-run: would-`, `skipped: `. (The +commented, attempt NOT burned), `marked-ready PR #` (the specific fixed test +was confirmed green in the PR's own CI; draft flipped to ready for review), +`already-surfaced PR #` (the fix's ✅ green comment already existed this sweep; +re-post suppressed — superseded by the Step 3.6 readiness line when the PR also flips +ready that cycle), `already-ready PR #` (terminal steady state — the PR was flipped +ready on a prior cycle and remains green on an unchanged head; no action taken, +supersedes `already-surfaced`), `waiting PR #` (CI not settled yet), +`dry-run: would-`, `skipped: `. (The `needs-human-PR` outcome is reserved for the deferred hand-off — Step 6 currently records a skip instead, so it is not emitted.)
: `✅ Attempt /10 validated — the fix's CI is + green on .` naming any `/azp`-gated legs (uitests def 313 / devicetests def + 314) that have not run and still need a maintainer `/azp run`; record `surfaced-green + PR # (attempt /10)`. (Do NOT assert "ready for human review" on a PR that + is still a draft — Step 3.6, not this comment, owns the draft→ready flip and posts its + own 🎯 announcement when it fires.) + Do NOT advance. Then — in ALL of the above cases — run **Step 3.6** (target-test + readiness gate) for this PR before stopping: its `/azp`-gated target legs may conclude + green on this SAME head SHA on a LATER sweep (an `/azp run` adds no commit), and Step + 3.6 is the ONLY place the draft→ready flip happens, so it MUST re-evaluate every sweep — + never `stop` here before it. *(This directly attacks the real bottleneck — no reviews — + so it is the highest-value outcome.)* 4. **Red → classify caused-by-fix vs unrelated-flake.** If `C.overallConclusion == "failure"`, analyze the PR's OWN failing build (NOT `net11.0`): find the AzDO `maui-pr` build for this PR (filter builds by `branchName=refs/pull//merge`, @@ -733,18 +829,26 @@ Run these gates in order — the FIRST that fires decides this cycle's outcome: method and Step 4.7 flake buckets to `C.failedLegs`. - **Unrelated flake only** (every failed leg is known-flaky / infra / a pre-existing baseline red NOT introduced by the fix): do NOT burn an attempt. - **Idempotency first:** scan the PR's existing comments for a prior bot - `♻️ … unrelated flake … on ` note for THIS head SHA — if one already - exists, record `already-annotated-flake PR # (head )` and stop - WITHOUT re-commenting (re-annotating the same flake every 12h sweep is noise and - burns the per-run comment budget other PRs need). Otherwise resolve the attempt - number from `C.effectiveAttempt` (the authoritative max(marker, bot-commit) - counter), omitting the `Attempt /10:` prefix only if it is somehow - indeterminate. **Dry-run gate (Step 0):** if `dry_run == "true"`, do NOT emit any comment — instead print the intended `♻️` unrelated-flake body to the run log, tally `dry-run: would-annotate-flake PR #`, and stop. Otherwise `add_comment` on PR #: `♻️ Attempt /10: red is - unrelated flake on leg(s) () on ; the fix itself is not - implicated. A maintainer re-run (/azp run ) should clear it.` Record - `annotated-flake PR # (head )` and stop. *(Round 1: human re-runs; - Round 2: auto re-trigger.)* + **Comment idempotency + dry-run suppress the ♻️ comment ONLY — neither skips Step + 3.6.** Scan the PR's existing comments for a prior bot `♻️ … unrelated flake … on + ` note for THIS head SHA; if one exists, set `SKIP_FLAKE_COMMENT = true`. + Resolve the attempt number from `C.effectiveAttempt` (the authoritative max(marker, + bot-commit) counter), omitting the `Attempt /10:` prefix only if it is + somehow indeterminate. Then post the flake note UNLESS suppressed: + - if `dry_run == "true"`: do NOT emit — print the intended `♻️` unrelated-flake body + to the run log and tally `dry-run: would-annotate-flake PR #`; + - else if `SKIP_FLAKE_COMMENT`: do NOT re-post — record `already-annotated-flake PR + # (head )` (re-annotating the same flake every 12h sweep is noise and + burns the per-run comment budget other PRs need); + - else `add_comment` on PR #: `♻️ Attempt /10: red is unrelated flake on + leg(s) () on ; the fix itself is not implicated. A + maintainer re-run (/azp run ) should clear it.` record `annotated-flake + PR # (head )`. + Then — in ALL of the above cases — run **Step 3.6** (target-test readiness gate) for + this PR before stopping: its `/azp`-gated target legs may conclude green on this SAME + head SHA on a LATER sweep, and Step 3.6 is the ONLY place the draft→ready flip + happens, so it MUST re-evaluate every sweep — never `stop` here before it. *(Round 1: + human re-runs; Round 2: auto re-trigger.)* - **Caused by the fix** (a failed leg still matches the original target signature, or the fix introduced a NEW failure): advance an attempt. - **Attempt count.** `attempt = C.effectiveAttempt` — the authoritative @@ -762,6 +866,188 @@ Run these gates in order — the FIRST that fires decides this cycle's outcome: from every prior commit on this branch**, emitted via the Step 5.6 ADVANCE path (push onto the same PR). +#### Step 3.6 — Target-test verification & mark-ready gate + +Reached from the Step 3 **green-surface** branch, the Step 4 **unrelated-flake** branch +(AFTER that branch has posted its comment), and the Step 3.5 **gate-2 target-focused +fast-path** (2a — while unrelated legs are still pending). Purpose: confirm the SPECIFIC +test(s) this PR was opened to fix now PASS in the PR's own CI, and — only when they do +— transition the draft `[ci-fix-net11]` PR to **ready for review** so a maintainer sees +a validated fix instead of a draft. This is the ONLY place the loop flips draft→ready; +it is a state transition, **never** an approval or a merge (a human still reviews and +merges). Overall red on *unrelated* legs must NOT gate readiness — we validate the fix, +not the base branch's flakiness. + +**Preconditions** (ALL must hold; otherwise record `skipped: readiness N/A PR # +()` and stop this gate): +- `C.isDraft == true` — if the PR is already ready-for-review, the draft→ready + transition is already done; do NOT re-mark **and do NOT re-apply `p/0`**. `p/0` is applied + exactly ONCE, atomically with the draft→ready flip (T3 — all-or-nothing with the 🎯 audit + comment + mark-ready), so an already-ready loop PR that is MISSING `p/0` has almost + certainly had it **removed by a maintainer** de-prioritizing the PR — a pure triage action + the human-engagement guard (which inspects comments/reviews/commits, NOT label changes) + cannot see. Re-adding `p/0` every sweep would fight that maintainer indefinitely, violating + the loop's "never override a human" contract. So the loop does NOT reconcile the label: + record `already-ready PR #` and stop this gate. (Trade-off: on the rare occasion a + transient API error drops `p/0` at flip time *after* the T3 atomic pre-check passed, it is + not auto-re-added — but the PR is still ready-for-review with its 🎯 audit comment, and + re-adding `p/0` is a trivial manual action; that is strictly preferable to steam-rolling a + maintainer's deliberate de-prioritization.) +- The PR is unmistakably THIS workflow's own: `[ci-fix-net11] ` title prefix AND the + `agentic-workflows` label (mirrors the safe-output lock; the compensating scope + control if the v0.80.9 compiler drops the declarative required-*). +- You reached this gate from Step 3 (green), Step 4 (**unrelated-flake**), or the Step 3.5 + gate-2 **target-focused fast-path** (2a) — i.e. the fix is NOT implicated in any red that + has concluded. If Step 4 classified a red as **caused by the fix** (ADVANCE), do NOT run + this gate — advance the attempt instead. + +**T1 — Identify the target test(s).** From the `[ci-scan-net11]` issue signature the fix +addresses (and the PR's own diff), extract the fully-qualified test method name(s) the +fix targets — e.g. `SafeAreaShouldWorkOnAllShellTabs`. For a de-flake it is the +de-flaked test; for a product fix it is the originally-failing test(s). If NO specific +test can be identified (e.g. a product build-break rather than a test failure), this is a +**build-only fix** — there is no single test to validate cross-platform, so validate at +**whole-build** granularity rather than stopping (Step 3 only *comments*; Step 3.6 is the +sole draft→ready flip, so a build-only fix is undrafted HERE or not at all). We only reach +this gate from Step 3 (green) / Step 4 (unrelated-flake) / gate-2a, so the fix is not +implicated in any concluded red; additionally require, via the T2 timeline method on +`C.headSha`, that the primary build pipeline `maui-pr` (def 302) has CONCLUDED with EVERY +one of its platform build legs `succeeded`/`completed` and NONE failed/canceled or still +pending — the build is green on **every** platform, not just the originally-broken one (the +cross-platform guard, applied to the build instead of a test). A build-only fix is undrafted +ONLY on the strength of the auto-running `maui-pr` (def 302) whole-build green, which the +loop CAN observe: if the PR's `[ci-scan]` issue signature or its own diff indicates the +ORIGINATING failure was in a `/azp`-gated pipeline (`maui-pr-uitests` def 313 or +`maui-pr-devicetests` def 314) rather than `maui-pr` (def 302), do NOT undraft on +`maui-pr`-green alone — those pipelines do not auto-run on this PR (GITHUB_TOKEN cannot +trigger them), so a green `maui-pr` build is NOT evidence the gated build break is fixed; +record `skipped: build-only fix PR # targets gated pipeline () not run — +deferring to human` and stop this gate. Otherwise, set `TARGET := "the maui-pr build +(build-only fix — no single target test)"` and proceed to **T3** to mark ready. If any `maui-pr` build leg is still unconcluded, record `skipped: build-only fix PR +# not yet whole-build green (leg(s) pending)` and stop this gate WITHOUT marking +ready (a green subset is not enough — a still-pending build leg could yet fail). + +**T2 — Verify the target test's platform legs are green (anonymous, leg-level).** Per-test +outcomes come from the AzDO **test-results** API (`_apis/test/...`), which is **NOT reachable +anonymously — Hard-Rule 8**: those endpoints 302-redirect to a sign-in page on this runner, so +a `curl` returns an HTML redirect (not JSON) and every target test would look "not executed". +Validate instead at **leg granularity** using ONLY the anonymous `_apis/build/...` timeline +(Hard-Rule 8) — a `succeeded` category leg means every test in that category **that actually +ran** passed on that platform. This is a strong bar **only if the target test genuinely +executes**: a job can go green while one specific test is skipped at runtime (`[Ignore]`, +`Assert.Ignore()`, `Assert.Inconclusive()`, a `Skip=`/conditional `[Fact]`, an `#if`-out, or +an early `return` before the asserts). So before trusting a green leg as proof the target +passed, **confirm from the PR's OWN diff that the fix does NOT skip, ignore, disable, or +short-circuit the target test** (it adds no `[Ignore]`/`[Explicit]`, `Assert.Ignore`/ +`Assert.Inconclusive`, `Skip=`, category-exclusion, `#if`-out, or early `return` around the +target). If the fix could cause the target to be runtime-skipped rather than genuinely pass, +leg-level green is NOT sufficient evidence — record `skipped: target test may be +runtime-skipped by this fix (diff adds ); leg-green insufficient, deferring to human +on PR #` and stop this gate WITHOUT marking ready. Otherwise (the target genuinely runs +and asserts, as a de-flake or product fix does) a green category leg cannot hide a +target-test failure, so treat it as authoritative. + +Map the target test to the CI leg(s) that run it. A leg name encodes platform + test category +(e.g. `Android UITests SafeAreaEdges,Shadow`, `iOS UITests SafeAreaEdges,Shadow`, `macOS +UITests SafeAreaEdges,Shadow`, `Windows UITests SafeAreaEdges,Shadow`). Determine the target +test's UI-test category — its `[Category(UITestCategories.X)]` in the test/HostApp source, +visible in the PR diff or the test file — to know which leg-name substring identifies its legs; +for a device test, the per-platform device-test legs. + +Using the SAME build-discovery as Step 4 (filter AzDO builds by `branchName=refs/pull//merge` +or `sourceVersion == C.headSha`), read each build's **timeline** on `C.headSha` for the +pipeline(s) that RUN the target test — `maui-pr` (def 302) for unit/integration tests, +`maui-pr-uitests` (def 313) for Appium UI tests, `maui-pr-devicetests` (def 314) for device +tests: + +```bash +ORG=dnceng-public; PROJ=public; BUILD= +# Anonymous + Hard-Rule-8-compliant. NEVER call _apis/test/... (it 302-redirects to sign-in). +# Each type=="Job" record is one platform leg: result (succeeded/failed/canceled/null), +# state (completed/inProgress/pending), name (encodes " UITests "): +curl -s "https://dev.azure.com/$ORG/$PROJ/_apis/build/builds/$BUILD/timeline?api-version=7.1" \ + | tee /tmp/gh-aw/agent/timeline_${BUILD}.json \ + | jq -r '.records[] | select(.type=="Job") | "\(.result)\t\(.state)\t\(.name)"' +``` + +(The prefetch already exposes per-leg status in `C.failedLegs` / the checks context, and +`gh pr checks ` lists the same per-leg rows with platform+category in the name — use +whichever is handy; the build timeline is the authoritative cross-check on the exact build.) + +Treat a target test as **VALIDATED-GREEN** only if, on `C.headSha`, the leg that runs its +category is green on **every platform it runs on** — a fix that repairs one platform must not +silently regress the same test's leg on another, so a green leg on the originally-red platform +alone is NOT enough. Across the target pipeline's platform legs (for a UI test: the Android, +iOS, Windows, and macOS/MacCatalyst legs running the target's category; for a device test: each +device-test platform), require ALL of: +- the target's category leg is `result == "succeeded"` **and** `state == "completed"` on + **every** platform that runs it, AND +- that leg is `failed` / `canceled` / aborted on **NO** platform, AND +- **no platform is left unverified.** For each platform family the target pipeline covers, that + platform's category leg must have CONCLUDED (`state == "completed"`) on `C.headSha`. If a + platform simply has no leg for the target's category, the test does not run there — that is + fine, not a gap. But if a platform's category leg has **not concluded** (`state` is + `inProgress` / pending, or an `/azp`-gated `maui-pr-uitests` / `maui-pr-devicetests` leg that + has not been kicked), the test's status on that platform is UNKNOWN → the fix is NOT yet + validated across platforms: record `skipped: target test green on but + not yet verified on (leg(s) pending / need /azp run) on PR #` + and stop this gate WITHOUT marking ready. + +A target test whose category leg never concluded on ANY platform (**not executed** anywhere — +e.g. its `/azp`-gated pipeline has not been kicked) is likewise NOT validated: record `skipped: +target test not yet executed on PR # ( not run — needs /azp run)` and stop +this gate WITHOUT marking ready. Do NOT overclaim — a green *sibling* leg (a different category +on the same platform) is not the target's leg, and a green leg on one platform is not a pass on +the others. + +**T3 — Mark ready + report.** If EVERY target test is VALIDATED-GREEN. The 🎯 comment, +the mark-ready, and the `p/0` label are THREE SEPARATE safe-outputs — the comment +existing does NOT prove the mark-ready took effect, so they are tracked independently: + +- **Comment idempotency (dup-suppress only — never gates the mark-ready).** We only reach + T3 while `C.isDraft == true` (the precondition stops an already-ready PR before here). So + if a prior bot comment for THIS head that carries the leading `🎯` anchor AND the + contiguous phrase `validated green on ` already exists (BOTH T3 variants — + `🎯 Target test validated green on ` and the build-only `🎯 Build validated green on + ` — contain that exact contiguous phrase, so this recognizes both; keying on + `target test` alone would instead miss the build-only comment and re-post it every sweep. + **Require the `🎯` anchor** so the match can NEVER be loosened to a gapped + `validated…green…on` form that would false-positive on the Step 3 `✅ … validated — the + fix's CI is green on ` surface comment and wrongly suppress the audit 🎯), the comment + landed on an earlier sweep but the **mark-ready did NOT take + effect** (the PR is still a draft) — set `SUPPRESS_COMMENT = true` (do not re-post the + duplicate 🎯 comment) but STILL complete the mark-ready + label below. Never treat the + comment's existence as "already marked ready" while the PR is still a draft. Record this + case as `re-marking PR # (🎯 present; mark-ready did not take on a prior sweep)`. +- **Atomicity budget pre-check (Hard-Rule 7).** Determine the outputs this gate will emit: + `mark_pull_request_as_ready_for_review` + `add_labels` always, plus `add_comment` unless + `SUPPRESS_COMMENT`. Verify a free per-run slot remains in EACH bucket you are about to + use. If ANY required bucket is exhausted, emit NONE of them — record `skipped: per-run + cap reached; deferring mark-ready PR # to next cycle` and stop this gate. Never mark a + PR ready (or label it) unless its 🎯 audit comment is GUARANTEED to exist for this exact + head SHA — either posted in this same sweep, or (in the `SUPPRESS_COMMENT` re-marking case) + already present from a prior sweep. The outputs you DO emit either all land or all defer + together; the audit trail must never be absent, but it is NOT re-posted when it already exists. +- **Dry-run gate (Step 0):** if `dry_run == "true"`, emit NOTHING — print the intended + readiness comment and "would mark ready + add p/0" to the run log, tally `dry-run: + would-mark-ready PR #`, and stop. +- Otherwise emit for THIS PR number ``: + 1. `add_comment` (ONLY if not `SUPPRESS_COMMENT`): `🎯 Target test validated green on + — passed on ALL platforms it runs on (, + buildId ). — not + caused by this fix."> Transitioning this PR from draft to ready for review and adding + `p/0`; a maintainer still reviews and merges.` (For a **build-only fix** — the T1 + whole-build fallback, no single target test — phrase the first clause as `🎯 Build + validated green on — the maui-pr build passed on ALL platforms (buildId + ).` instead of naming a test.) + 2. `mark_pull_request_as_ready_for_review` with `reason:` a one-line justification + naming the validated test(s) and ``. + 3. `add_labels` with `labels: ["p/0"]` for PR # — put the now-review-ready fix into + the team's p/0 priority queue so it is triaged, not lost in the draft backlog. (If + the PR somehow already carries `p/0`, this is a harmless no-op.) +- Record `marked-ready PR # (target green on ALL platforms on , + labeled p/0)` and stop. + #### Step 3.5.R — Maintainer change-request response (Track C) Reached from Step 3.5 gate 0. Goal: when an **eligible human reviewer** (Hard-Rule @@ -1395,7 +1681,19 @@ Filed by [`ci-status-fix-net11`](https://github.com/dotnet/maui/blob/main/.githu ### Step 8 — Per-issue tally + end-of-run summary -Per issue, append one outcome line to `/tmp/gh-aw/agent/coverage.txt`: +Per issue, append **one** outcome line to `/tmp/gh-aw/agent/coverage.txt` — the +terminal outcome for the cycle. When one cycle produces a chained pair — a PR gets a +same-cycle precursor line and **then** a Step 3.6 readiness line (`marked-ready` on the +draft→ready flip, or `already-ready` on a later steady-state sweep of an already-ready +PR) — record ONLY the terminal Step 3.6 readiness line: it supersedes ANY same-cycle +non-readiness precursor for that PR, so an aggregator keying on one-line-per-issue never +double-counts. The precursor may be a Step 3 green line (`surfaced-green` on the first ✅, +or `already-surfaced` when it already exists) OR — when an unrelated-flake red and a +target-green coincide — a Step 4 `annotated-flake` line; either way the readiness line is +terminal, and the superseded precursor's signal survives in the PR's 🎯/♻️ comment so no +information is lost. This covers the flip cycle (`surfaced-green` → `marked-ready`), the +post-flip steady state (`already-surfaced` → `already-ready`), and the flake-coincident +flip (`annotated-flake` → `marked-ready`): ``` # net11.0 attempt- @@ -1406,8 +1704,14 @@ Per issue, append one outcome line to `/tmp/gh-aw/agent/coverage.txt`: `advance-PR # attempt /10` (ADVANCE mode — new commit pushed to the existing PR), `surfaced-green PR #` (fix's own CI went green; commented for review, did not advance), `annotated-flake PR #` (red was unrelated flake; -commented, attempt NOT burned), `waiting PR #` (CI not settled yet), -`dry-run: would-`, `skipped: `. (The +commented, attempt NOT burned), `marked-ready PR #` (the specific fixed test +was confirmed green in the PR's own CI; draft flipped to ready for review), +`already-surfaced PR #` (the fix's ✅ green comment already existed this sweep; +re-post suppressed — superseded by the Step 3.6 readiness line when the PR also flips +ready that cycle), `already-ready PR #` (terminal steady state — the PR was flipped +ready on a prior cycle and remains green on an unchanged head; no action taken, +supersedes `already-surfaced`), `waiting PR #` (CI not settled yet), +`dry-run: would-`, `skipped: `. (The `needs-human-PR` outcome is reserved for the deferred hand-off — Step 6 currently records a skip instead, so it is not emitted.) diff --git a/.github/workflows/ci-status-fix.lock.yml b/.github/workflows/ci-status-fix.lock.yml index 93dc5969cb0d..65511f2b3259 100644 --- a/.github/workflows/ci-status-fix.lock.yml +++ b/.github/workflows/ci-status-fix.lock.yml @@ -1,4 +1,4 @@ -# gh-aw-metadata: {"schema_version":"v4","frontmatter_hash":"f014161967589ebcaf1ac5ec6f3c88ee5a4388d28b55a07dc36c45b7b2aebab6","body_hash":"86c6b2d4c3df13da14c6a3c8b211381fcc9f97bb1951ff7b80588a2c5338e00c","compiler_version":"v0.80.9","strict":true,"agent_id":"copilot","agent_model":"claude-opus-4.8","engine_versions":{"copilot":"1.0.63"}} +# gh-aw-metadata: {"schema_version":"v4","frontmatter_hash":"b0ebf8a2f179900cfe3638e994b27a43cec52bdbdd2331dc1bd4d65762fa2d6b","body_hash":"1825e2b1f7c7bd88241bf37580655489360f9bebc806edd00837a52ce4ad83a0","compiler_version":"v0.80.9","strict":true,"agent_id":"copilot","agent_model":"claude-opus-4.8","engine_versions":{"copilot":"1.0.63"}} # gh-aw-manifest: {"version":1,"secrets":["COPILOT_PAT_0","COPILOT_PAT_1","COPILOT_PAT_2","COPILOT_PAT_3","COPILOT_PAT_4","COPILOT_PAT_5","COPILOT_PAT_6","COPILOT_PAT_7","COPILOT_PAT_8","COPILOT_PAT_9","GH_AW_CI_TRIGGER_TOKEN","GH_AW_GITHUB_MCP_SERVER_TOKEN","GH_AW_GITHUB_TOKEN","GITHUB_TOKEN"],"actions":[{"repo":"actions/checkout","sha":"34e114876b0b11c390a56381ad16ebd13914f8d5","version":"v4"},{"repo":"actions/checkout","sha":"9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0","version":"v7.0.0"},{"repo":"actions/checkout","sha":"de0fac2e4500dabe0009e67214ff5f5447ce83dd","version":"v6.0.2"},{"repo":"actions/download-artifact","sha":"3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c","version":"v8.0.1"},{"repo":"actions/github-script","sha":"3a2844b7e9c422d3c10d287c895573f7108da1b3","version":"v9.0.0"},{"repo":"actions/setup-node","sha":"48b55a011bda9f5d6aeb4c2d9c7362e8dae4041e","version":"v6.4.0"},{"repo":"actions/upload-artifact","sha":"043fb46d1a93c77aae656e7c1c64a875d1fc6a0a","version":"v7.0.1"},{"repo":"github/gh-aw-actions/setup","sha":"8c7d04ebf1ece56cd381446125da3e0f6896294a","version":"v0.80.9"}],"containers":[{"image":"ghcr.io/github/gh-aw-firewall/agent:0.27.7","digest":"sha256:aae231e4635c8999d039c132f1602d3df850fe9b84a00aa2b5ac981179b5661c","pinned_image":"ghcr.io/github/gh-aw-firewall/agent:0.27.7@sha256:aae231e4635c8999d039c132f1602d3df850fe9b84a00aa2b5ac981179b5661c"},{"image":"ghcr.io/github/gh-aw-firewall/api-proxy:0.27.7","digest":"sha256:009caf2e3d88fa77b64e9a03a95a228fc58db0f1701c6d324b29ba5a3c7c79b6","pinned_image":"ghcr.io/github/gh-aw-firewall/api-proxy:0.27.7@sha256:009caf2e3d88fa77b64e9a03a95a228fc58db0f1701c6d324b29ba5a3c7c79b6"},{"image":"ghcr.io/github/gh-aw-firewall/squid:0.27.7","digest":"sha256:deb1d4e19de62d51cee0508057a596a19315c3423ada4d675cad136dc8037c96","pinned_image":"ghcr.io/github/gh-aw-firewall/squid:0.27.7@sha256:deb1d4e19de62d51cee0508057a596a19315c3423ada4d675cad136dc8037c96"},{"image":"ghcr.io/github/gh-aw-mcpg:v0.3.27","digest":"sha256:fe984bddde4ec05d756d9043edb0a32912e6b7b72f6a121b1082f29221421cc7","pinned_image":"ghcr.io/github/gh-aw-mcpg:v0.3.27@sha256:fe984bddde4ec05d756d9043edb0a32912e6b7b72f6a121b1082f29221421cc7"},{"image":"ghcr.io/github/gh-aw-node","digest":"sha256:529d02eb970b1161aa25c593a9c3df57fdfad5a8add328cb3b6eccef66f3183b","pinned_image":"ghcr.io/github/gh-aw-node@sha256:529d02eb970b1161aa25c593a9c3df57fdfad5a8add328cb3b6eccef66f3183b"},{"image":"ghcr.io/github/github-mcp-server:v1.4.0","digest":"sha256:2afb26356481d1a350e14544a6e160f7f7ec1561a1ea309b823665abf0309036","pinned_image":"ghcr.io/github/github-mcp-server:v1.4.0@sha256:2afb26356481d1a350e14544a6e160f7f7ec1561a1ea309b823665abf0309036"}]} # This file was automatically generated by gh-aw (v0.80.9). DO NOT EDIT. To debug this workflow, load the skill at https://github.com/github/gh-aw/blob/main/debug.md # @@ -37,7 +37,9 @@ # humans (the open tracking issue is the hand-off surface; a dedicated # [ci-fix][needs-human] PR is planned but currently deferred — see Step 6). CI on # the PR is kicked by a human `/azp run` for now; the loop watches, classifies, -# and re-fixes autonomously between kicks. +# and re-fixes autonomously between kicks. When the SPECIFIC test a PR fixed is +# confirmed green in that PR's own CI, the loop marks the draft PR ready for review +# (a state transition only — it never approves or merges). # Never mutes tests, but # de-flakes genuinely flaky ones (deterministic synchronization, no retries / # timeout bumps). Always skips visual-regression / screenshot issues. @@ -273,24 +275,24 @@ jobs: run: | bash "${RUNNER_TEMP}/gh-aw/actions/create_prompt_first.sh" { - cat << 'GH_AW_PROMPT_25cf75111b670167_EOF' + cat << 'GH_AW_PROMPT_2ac1d0b4083428f8_EOF' - GH_AW_PROMPT_25cf75111b670167_EOF + GH_AW_PROMPT_2ac1d0b4083428f8_EOF cat "${RUNNER_TEMP}/gh-aw/prompts/xpia.md" cat "${RUNNER_TEMP}/gh-aw/prompts/temp_folder_prompt.md" cat "${RUNNER_TEMP}/gh-aw/prompts/markdown.md" cat "${RUNNER_TEMP}/gh-aw/prompts/safe_outputs_prompt.md" - cat << 'GH_AW_PROMPT_25cf75111b670167_EOF' + cat << 'GH_AW_PROMPT_2ac1d0b4083428f8_EOF' - Tools: add_comment(max:3), create_pull_request(max:3), update_pull_request(max:3), push_to_pull_request_branch(max:3), missing_tool, missing_data, noop - GH_AW_PROMPT_25cf75111b670167_EOF + Tools: add_comment(max:6), create_pull_request(max:3), update_pull_request(max:3), mark_pull_request_as_ready_for_review(max:3), add_labels(max:3), push_to_pull_request_branch(max:3), missing_tool, missing_data, noop + GH_AW_PROMPT_2ac1d0b4083428f8_EOF cat "${RUNNER_TEMP}/gh-aw/prompts/safe_outputs_create_pull_request.md" cat "${RUNNER_TEMP}/gh-aw/prompts/safe_outputs_push_to_pr_branch.md" - cat << 'GH_AW_PROMPT_25cf75111b670167_EOF' + cat << 'GH_AW_PROMPT_2ac1d0b4083428f8_EOF' - GH_AW_PROMPT_25cf75111b670167_EOF + GH_AW_PROMPT_2ac1d0b4083428f8_EOF cat "${RUNNER_TEMP}/gh-aw/prompts/mcp_cli_tools_prompt.md" - cat << 'GH_AW_PROMPT_25cf75111b670167_EOF' + cat << 'GH_AW_PROMPT_2ac1d0b4083428f8_EOF' The following GitHub context information is available for this workflow: {{#if github.actor}} @@ -332,12 +334,12 @@ jobs: stop immediately and report the limitation rather than spending turns trying to work around it. - GH_AW_PROMPT_25cf75111b670167_EOF + GH_AW_PROMPT_2ac1d0b4083428f8_EOF cat "${RUNNER_TEMP}/gh-aw/prompts/github_mcp_tools_with_safeoutputs_prompt.md" - cat << 'GH_AW_PROMPT_25cf75111b670167_EOF' + cat << 'GH_AW_PROMPT_2ac1d0b4083428f8_EOF' {{#runtime-import .github/workflows/ci-status-fix.md}} - GH_AW_PROMPT_25cf75111b670167_EOF + GH_AW_PROMPT_2ac1d0b4083428f8_EOF } > "$GH_AW_PROMPT" - name: Interpolate variables and render templates uses: actions/github-script@3a2844b7e9c422d3c10d287c895573f7108da1b3 # v9.0.0 @@ -554,16 +556,18 @@ jobs: mkdir -p "${RUNNER_TEMP}/gh-aw/safeoutputs" mkdir -p /tmp/gh-aw/safeoutputs mkdir -p /tmp/gh-aw/mcp-logs/safeoutputs - cat > "${RUNNER_TEMP}/gh-aw/safeoutputs/config.json" << 'GH_AW_SAFE_OUTPUTS_CONFIG_fa2a69fad5fd0485_EOF' - {"add_comment":{"discussions":false,"max":3,"required_labels":["agentic-workflows"],"required_title_prefix":"[ci-fix] ","target":"*"},"create_pull_request":{"allowed_base_branches":["main"],"allowed_branches":["ci-fix/**"],"allowed_files":["src/Core/**","src/Controls/**","src/Essentials/**","src/BlazorWebView/**","src/TestUtils/**","src/Templates/**","**/PublicAPI.Unshipped.txt"],"base_branch":"main","draft":true,"labels":["agentic-workflows"],"max":3,"max_patch_files":100,"max_patch_size":4096,"protect_top_level_dot_folders":true,"protected_files":["package.json","bun.lockb","bunfig.toml","deno.json","deno.jsonc","deno.lock","global.json","NuGet.Config","Directory.Packages.props","mix.exs","mix.lock","go.mod","go.sum","stack.yaml","stack.yaml.lock","pom.xml","build.gradle","build.gradle.kts","settings.gradle","settings.gradle.kts","gradle.properties","package-lock.json","yarn.lock","pnpm-lock.yaml","npm-shrinkwrap.json","requirements.txt","Pipfile","Pipfile.lock","pyproject.toml","setup.py","setup.cfg","Gemfile","Gemfile.lock","uv.lock","CODEOWNERS","DESIGN.md","README.md","CONTRIBUTING.md","CHANGELOG.md","SECURITY.md","CODE_OF_CONDUCT.md","AGENTS.md","CLAUDE.md","GEMINI.md"],"protected_files_policy":"request_review","title_prefix":"[ci-fix] "},"create_report_incomplete_issue":{},"missing_data":{},"missing_tool":{},"noop":{"max":1,"report-as-issue":"true"},"push_to_pull_request_branch":{"allowed_files":["src/Core/**","src/Controls/**","src/Essentials/**","src/BlazorWebView/**","src/TestUtils/**","src/Templates/**","**/PublicAPI.Unshipped.txt"],"if_no_changes":"warn","max":3,"max_patch_size":4096,"protect_top_level_dot_folders":true,"protected_files":["package.json","bun.lockb","bunfig.toml","deno.json","deno.jsonc","deno.lock","global.json","NuGet.Config","Directory.Packages.props","mix.exs","mix.lock","go.mod","go.sum","stack.yaml","stack.yaml.lock","pom.xml","build.gradle","build.gradle.kts","settings.gradle","settings.gradle.kts","gradle.properties","package-lock.json","yarn.lock","pnpm-lock.yaml","npm-shrinkwrap.json","requirements.txt","Pipfile","Pipfile.lock","pyproject.toml","setup.py","setup.cfg","Gemfile","Gemfile.lock","uv.lock","CODEOWNERS","DESIGN.md","README.md","CONTRIBUTING.md","CHANGELOG.md","SECURITY.md","CODE_OF_CONDUCT.md","AGENTS.md","CLAUDE.md","GEMINI.md"],"required_labels":["agentic-workflows"],"target":"*","title_prefix":"[ci-fix] "},"report_incomplete":{},"update_pull_request":{"allow_body":true,"allow_title":false,"max":3,"target":"*","update_branch":false}} - GH_AW_SAFE_OUTPUTS_CONFIG_fa2a69fad5fd0485_EOF + cat > "${RUNNER_TEMP}/gh-aw/safeoutputs/config.json" << 'GH_AW_SAFE_OUTPUTS_CONFIG_0644bf1434ca0071_EOF' + {"add_comment":{"discussions":false,"max":6,"required_labels":["agentic-workflows"],"required_title_prefix":"[ci-fix] ","target":"*"},"add_labels":{"allowed":["p/0"],"max":3,"required_labels":["agentic-workflows"],"required_title_prefix":"[ci-fix] ","target":"*"},"create_pull_request":{"allowed_base_branches":["main"],"allowed_branches":["ci-fix/**"],"allowed_files":["src/Core/**","src/Controls/**","src/Essentials/**","src/BlazorWebView/**","src/TestUtils/**","src/Templates/**","**/PublicAPI.Unshipped.txt"],"base_branch":"main","draft":true,"labels":["agentic-workflows"],"max":3,"max_patch_files":100,"max_patch_size":4096,"protect_top_level_dot_folders":true,"protected_files":["package.json","bun.lockb","bunfig.toml","deno.json","deno.jsonc","deno.lock","global.json","NuGet.Config","Directory.Packages.props","mix.exs","mix.lock","go.mod","go.sum","stack.yaml","stack.yaml.lock","pom.xml","build.gradle","build.gradle.kts","settings.gradle","settings.gradle.kts","gradle.properties","package-lock.json","yarn.lock","pnpm-lock.yaml","npm-shrinkwrap.json","requirements.txt","Pipfile","Pipfile.lock","pyproject.toml","setup.py","setup.cfg","Gemfile","Gemfile.lock","uv.lock","CODEOWNERS","DESIGN.md","README.md","CONTRIBUTING.md","CHANGELOG.md","SECURITY.md","CODE_OF_CONDUCT.md","AGENTS.md","CLAUDE.md","GEMINI.md"],"protected_files_policy":"request_review","title_prefix":"[ci-fix] "},"create_report_incomplete_issue":{},"mark_pull_request_as_ready_for_review":{"max":3,"required_labels":["agentic-workflows"],"required_title_prefix":"[ci-fix] ","target":"*"},"missing_data":{},"missing_tool":{},"noop":{"max":1,"report-as-issue":"true"},"push_to_pull_request_branch":{"allowed_files":["src/Core/**","src/Controls/**","src/Essentials/**","src/BlazorWebView/**","src/TestUtils/**","src/Templates/**","**/PublicAPI.Unshipped.txt"],"if_no_changes":"warn","max":3,"max_patch_size":4096,"protect_top_level_dot_folders":true,"protected_files":["package.json","bun.lockb","bunfig.toml","deno.json","deno.jsonc","deno.lock","global.json","NuGet.Config","Directory.Packages.props","mix.exs","mix.lock","go.mod","go.sum","stack.yaml","stack.yaml.lock","pom.xml","build.gradle","build.gradle.kts","settings.gradle","settings.gradle.kts","gradle.properties","package-lock.json","yarn.lock","pnpm-lock.yaml","npm-shrinkwrap.json","requirements.txt","Pipfile","Pipfile.lock","pyproject.toml","setup.py","setup.cfg","Gemfile","Gemfile.lock","uv.lock","CODEOWNERS","DESIGN.md","README.md","CONTRIBUTING.md","CHANGELOG.md","SECURITY.md","CODE_OF_CONDUCT.md","AGENTS.md","CLAUDE.md","GEMINI.md"],"required_labels":["agentic-workflows"],"target":"*","title_prefix":"[ci-fix] "},"report_incomplete":{},"update_pull_request":{"allow_body":true,"allow_title":false,"max":3,"target":"*","update_branch":false}} + GH_AW_SAFE_OUTPUTS_CONFIG_0644bf1434ca0071_EOF - name: Generate Safe Outputs Tools env: GH_AW_TOOLS_META_JSON: | { "description_suffixes": { - "add_comment": " CONSTRAINTS: Maximum 3 comment(s) can be added. Target: *. Supports reply_to_id for discussion threading.", + "add_comment": " CONSTRAINTS: Maximum 6 comment(s) can be added. Target: *. Supports reply_to_id for discussion threading.", + "add_labels": " CONSTRAINTS: Maximum 3 label(s) can be added. Only these labels are allowed: [\"p/0\"]. Target: *.", "create_pull_request": " CONSTRAINTS: Maximum 3 pull request(s) can be created. Title will be prefixed with \"[ci-fix] \". Labels [\"agentic-workflows\"] will be automatically added. Only these labels are allowed: [\"agentic-workflows\"]. PRs will be created as drafts.", + "mark_pull_request_as_ready_for_review": " CONSTRAINTS: Maximum 3 pull request(s) can be marked as ready for review.", "push_to_pull_request_branch": " CONSTRAINTS: Maximum 3 push(es) can be made. The target pull request title must start with \"[ci-fix] \".", "update_pull_request": " CONSTRAINTS: Maximum 3 pull request(s) can be updated. Target: *." }, @@ -594,6 +598,25 @@ jobs: } } }, + "add_labels": { + "defaultMax": 5, + "fields": { + "item_number": { + "issueNumberOrTemporaryId": true + }, + "labels": { + "required": true, + "type": "array", + "itemType": "string", + "itemSanitize": true, + "itemMaxLength": 128 + }, + "repo": { + "type": "string", + "maxLength": 256 + } + } + }, "create_pull_request": { "defaultMax": 1, "fields": { @@ -635,6 +658,24 @@ jobs: } } }, + "mark_pull_request_as_ready_for_review": { + "defaultMax": 1, + "fields": { + "pull_request_number": { + "issueOrPRNumber": true + }, + "reason": { + "required": true, + "type": "string", + "sanitize": true, + "maxLength": 65000 + }, + "repo": { + "type": "string", + "maxLength": 256 + } + } + }, "missing_data": { "defaultMax": 20, "fields": { @@ -1494,7 +1535,7 @@ jobs: uses: actions/github-script@3a2844b7e9c422d3c10d287c895573f7108da1b3 # v9.0.0 env: WORKFLOW_NAME: "CI Failure Fixer (main)" - WORKFLOW_DESCRIPTION: "Periodic pass over open ci-scan tracking issues filed by the main-branch CI\nfailure scanner (.github/workflows/ci-status-main.md). This workflow targets\nthe `main` branch EXCLUSIVELY: it processes only issues labelled ci-scan and\nopens every PR against main. (The net11.0 branch is handled by the parallel\n.github/workflows/ci-status-fix-net11.md — the two are split because gh-aw can\nonly transport a fix relative to ONE static base branch per workflow, and the\nmain↔net11.0 divergence exceeds gh-aw's 10 MB transport-patch cap.) The fixer\nopens ONE draft [ci-fix] PR per actionable issue against main, then WATCHES\nthat PR's own CI on later runs: when the fix's CI comes back red and the red is\ncaused by the fix itself, it pushes a fresh follow-up fix onto the SAME PR\nbranch (never a second PR) — up to 10 attempts — then stops and defers to\nhumans (the open tracking issue is the hand-off surface; a dedicated\n[ci-fix][needs-human] PR is planned but currently deferred — see Step 6). CI on\nthe PR is kicked by a human `/azp run` for now; the loop watches, classifies,\nand re-fixes autonomously between kicks.\nNever mutes tests, but\nde-flakes genuinely flaky ones (deterministic synchronization, no retries /\ntimeout bumps). Always skips visual-regression / screenshot issues." + WORKFLOW_DESCRIPTION: "Periodic pass over open ci-scan tracking issues filed by the main-branch CI\nfailure scanner (.github/workflows/ci-status-main.md). This workflow targets\nthe `main` branch EXCLUSIVELY: it processes only issues labelled ci-scan and\nopens every PR against main. (The net11.0 branch is handled by the parallel\n.github/workflows/ci-status-fix-net11.md — the two are split because gh-aw can\nonly transport a fix relative to ONE static base branch per workflow, and the\nmain↔net11.0 divergence exceeds gh-aw's 10 MB transport-patch cap.) The fixer\nopens ONE draft [ci-fix] PR per actionable issue against main, then WATCHES\nthat PR's own CI on later runs: when the fix's CI comes back red and the red is\ncaused by the fix itself, it pushes a fresh follow-up fix onto the SAME PR\nbranch (never a second PR) — up to 10 attempts — then stops and defers to\nhumans (the open tracking issue is the hand-off surface; a dedicated\n[ci-fix][needs-human] PR is planned but currently deferred — see Step 6). CI on\nthe PR is kicked by a human `/azp run` for now; the loop watches, classifies,\nand re-fixes autonomously between kicks. When the SPECIFIC test a PR fixed is\nconfirmed green in that PR's own CI, the loop marks the draft PR ready for review\n(a state transition only — it never approves or merges).\nNever mutes tests, but\nde-flakes genuinely flaky ones (deterministic synchronization, no retries /\ntimeout bumps). Always skips visual-regression / screenshot issues." HAS_PATCH: ${{ needs.agent.outputs.has_patch }} with: script: | @@ -1910,7 +1951,7 @@ jobs: GH_AW_ALLOWED_DOMAINS: "*.blob.core.windows.net,*.githubusercontent.com,api.business.githubcopilot.com,api.enterprise.githubcopilot.com,api.github.com,api.githubcopilot.com,api.individual.githubcopilot.com,api.snapcraft.io,archive.ubuntu.com,azure.archive.ubuntu.com,codeload.github.com,crl.geotrust.com,crl.globalsign.com,crl.identrust.com,crl.sectigo.com,crl.thawte.com,crl.usertrust.com,crl.verisign.com,crl3.digicert.com,crl4.digicert.com,crls.ssl.com,dev.azure.com,docs.github.com,github-cloud.githubusercontent.com,github-cloud.s3.amazonaws.com,github.blog,github.com,github.githubassets.com,helix.dot.net,host.docker.internal,json-schema.org,json.schemastore.org,keyserver.ubuntu.com,lfs.github.com,objects.githubusercontent.com,ocsp.digicert.com,ocsp.geotrust.com,ocsp.globalsign.com,ocsp.identrust.com,ocsp.sectigo.com,ocsp.ssl.com,ocsp.thawte.com,ocsp.usertrust.com,ocsp.verisign.com,packagecloud.io,packages.cloud.google.com,packages.microsoft.com,patch-diff.githubusercontent.com,ppa.launchpad.net,raw.githubusercontent.com,registry.npmjs.org,s.symcb.com,s.symcd.com,security.ubuntu.com,telemetry.enterprise.githubcopilot.com,ts-crl.ws.symantec.com,ts-ocsp.ws.symantec.com,www.googleapis.com" GITHUB_SERVER_URL: ${{ github.server_url }} GITHUB_API_URL: ${{ github.api_url }} - GH_AW_SAFE_OUTPUTS_HANDLER_CONFIG: "{\"add_comment\":{\"discussions\":false,\"max\":3,\"required_labels\":[\"agentic-workflows\"],\"required_title_prefix\":\"[ci-fix] \",\"target\":\"*\"},\"create_pull_request\":{\"allowed_base_branches\":[\"main\"],\"allowed_branches\":[\"ci-fix/**\"],\"allowed_files\":[\"src/Core/**\",\"src/Controls/**\",\"src/Essentials/**\",\"src/BlazorWebView/**\",\"src/TestUtils/**\",\"src/Templates/**\",\"**/PublicAPI.Unshipped.txt\"],\"base_branch\":\"main\",\"draft\":true,\"labels\":[\"agentic-workflows\"],\"max\":3,\"max_patch_files\":100,\"max_patch_size\":4096,\"protect_top_level_dot_folders\":true,\"protected_files\":[\"package.json\",\"bun.lockb\",\"bunfig.toml\",\"deno.json\",\"deno.jsonc\",\"deno.lock\",\"global.json\",\"NuGet.Config\",\"Directory.Packages.props\",\"mix.exs\",\"mix.lock\",\"go.mod\",\"go.sum\",\"stack.yaml\",\"stack.yaml.lock\",\"pom.xml\",\"build.gradle\",\"build.gradle.kts\",\"settings.gradle\",\"settings.gradle.kts\",\"gradle.properties\",\"package-lock.json\",\"yarn.lock\",\"pnpm-lock.yaml\",\"npm-shrinkwrap.json\",\"requirements.txt\",\"Pipfile\",\"Pipfile.lock\",\"pyproject.toml\",\"setup.py\",\"setup.cfg\",\"Gemfile\",\"Gemfile.lock\",\"uv.lock\",\"CODEOWNERS\",\"DESIGN.md\",\"README.md\",\"CONTRIBUTING.md\",\"CHANGELOG.md\",\"SECURITY.md\",\"CODE_OF_CONDUCT.md\",\"AGENTS.md\",\"CLAUDE.md\",\"GEMINI.md\"],\"protected_files_policy\":\"request_review\",\"title_prefix\":\"[ci-fix] \"},\"create_report_incomplete_issue\":{},\"missing_data\":{},\"missing_tool\":{},\"noop\":{\"max\":1,\"report-as-issue\":\"true\"},\"push_to_pull_request_branch\":{\"allowed_files\":[\"src/Core/**\",\"src/Controls/**\",\"src/Essentials/**\",\"src/BlazorWebView/**\",\"src/TestUtils/**\",\"src/Templates/**\",\"**/PublicAPI.Unshipped.txt\"],\"if_no_changes\":\"warn\",\"max\":3,\"max_patch_size\":4096,\"protect_top_level_dot_folders\":true,\"protected_files\":[\"package.json\",\"bun.lockb\",\"bunfig.toml\",\"deno.json\",\"deno.jsonc\",\"deno.lock\",\"global.json\",\"NuGet.Config\",\"Directory.Packages.props\",\"mix.exs\",\"mix.lock\",\"go.mod\",\"go.sum\",\"stack.yaml\",\"stack.yaml.lock\",\"pom.xml\",\"build.gradle\",\"build.gradle.kts\",\"settings.gradle\",\"settings.gradle.kts\",\"gradle.properties\",\"package-lock.json\",\"yarn.lock\",\"pnpm-lock.yaml\",\"npm-shrinkwrap.json\",\"requirements.txt\",\"Pipfile\",\"Pipfile.lock\",\"pyproject.toml\",\"setup.py\",\"setup.cfg\",\"Gemfile\",\"Gemfile.lock\",\"uv.lock\",\"CODEOWNERS\",\"DESIGN.md\",\"README.md\",\"CONTRIBUTING.md\",\"CHANGELOG.md\",\"SECURITY.md\",\"CODE_OF_CONDUCT.md\",\"AGENTS.md\",\"CLAUDE.md\",\"GEMINI.md\"],\"required_labels\":[\"agentic-workflows\"],\"target\":\"*\",\"title_prefix\":\"[ci-fix] \"},\"report_incomplete\":{},\"update_pull_request\":{\"allow_body\":true,\"allow_title\":false,\"max\":3,\"target\":\"*\",\"update_branch\":false}}" + GH_AW_SAFE_OUTPUTS_HANDLER_CONFIG: "{\"add_comment\":{\"discussions\":false,\"max\":6,\"required_labels\":[\"agentic-workflows\"],\"required_title_prefix\":\"[ci-fix] \",\"target\":\"*\"},\"add_labels\":{\"allowed\":[\"p/0\"],\"max\":3,\"required_labels\":[\"agentic-workflows\"],\"required_title_prefix\":\"[ci-fix] \",\"target\":\"*\"},\"create_pull_request\":{\"allowed_base_branches\":[\"main\"],\"allowed_branches\":[\"ci-fix/**\"],\"allowed_files\":[\"src/Core/**\",\"src/Controls/**\",\"src/Essentials/**\",\"src/BlazorWebView/**\",\"src/TestUtils/**\",\"src/Templates/**\",\"**/PublicAPI.Unshipped.txt\"],\"base_branch\":\"main\",\"draft\":true,\"labels\":[\"agentic-workflows\"],\"max\":3,\"max_patch_files\":100,\"max_patch_size\":4096,\"protect_top_level_dot_folders\":true,\"protected_files\":[\"package.json\",\"bun.lockb\",\"bunfig.toml\",\"deno.json\",\"deno.jsonc\",\"deno.lock\",\"global.json\",\"NuGet.Config\",\"Directory.Packages.props\",\"mix.exs\",\"mix.lock\",\"go.mod\",\"go.sum\",\"stack.yaml\",\"stack.yaml.lock\",\"pom.xml\",\"build.gradle\",\"build.gradle.kts\",\"settings.gradle\",\"settings.gradle.kts\",\"gradle.properties\",\"package-lock.json\",\"yarn.lock\",\"pnpm-lock.yaml\",\"npm-shrinkwrap.json\",\"requirements.txt\",\"Pipfile\",\"Pipfile.lock\",\"pyproject.toml\",\"setup.py\",\"setup.cfg\",\"Gemfile\",\"Gemfile.lock\",\"uv.lock\",\"CODEOWNERS\",\"DESIGN.md\",\"README.md\",\"CONTRIBUTING.md\",\"CHANGELOG.md\",\"SECURITY.md\",\"CODE_OF_CONDUCT.md\",\"AGENTS.md\",\"CLAUDE.md\",\"GEMINI.md\"],\"protected_files_policy\":\"request_review\",\"title_prefix\":\"[ci-fix] \"},\"create_report_incomplete_issue\":{},\"mark_pull_request_as_ready_for_review\":{\"max\":3,\"required_labels\":[\"agentic-workflows\"],\"required_title_prefix\":\"[ci-fix] \",\"target\":\"*\"},\"missing_data\":{},\"missing_tool\":{},\"noop\":{\"max\":1,\"report-as-issue\":\"true\"},\"push_to_pull_request_branch\":{\"allowed_files\":[\"src/Core/**\",\"src/Controls/**\",\"src/Essentials/**\",\"src/BlazorWebView/**\",\"src/TestUtils/**\",\"src/Templates/**\",\"**/PublicAPI.Unshipped.txt\"],\"if_no_changes\":\"warn\",\"max\":3,\"max_patch_size\":4096,\"protect_top_level_dot_folders\":true,\"protected_files\":[\"package.json\",\"bun.lockb\",\"bunfig.toml\",\"deno.json\",\"deno.jsonc\",\"deno.lock\",\"global.json\",\"NuGet.Config\",\"Directory.Packages.props\",\"mix.exs\",\"mix.lock\",\"go.mod\",\"go.sum\",\"stack.yaml\",\"stack.yaml.lock\",\"pom.xml\",\"build.gradle\",\"build.gradle.kts\",\"settings.gradle\",\"settings.gradle.kts\",\"gradle.properties\",\"package-lock.json\",\"yarn.lock\",\"pnpm-lock.yaml\",\"npm-shrinkwrap.json\",\"requirements.txt\",\"Pipfile\",\"Pipfile.lock\",\"pyproject.toml\",\"setup.py\",\"setup.cfg\",\"Gemfile\",\"Gemfile.lock\",\"uv.lock\",\"CODEOWNERS\",\"DESIGN.md\",\"README.md\",\"CONTRIBUTING.md\",\"CHANGELOG.md\",\"SECURITY.md\",\"CODE_OF_CONDUCT.md\",\"AGENTS.md\",\"CLAUDE.md\",\"GEMINI.md\"],\"required_labels\":[\"agentic-workflows\"],\"target\":\"*\",\"title_prefix\":\"[ci-fix] \"},\"report_incomplete\":{},\"update_pull_request\":{\"allow_body\":true,\"allow_title\":false,\"max\":3,\"target\":\"*\",\"update_branch\":false}}" GH_AW_CI_TRIGGER_TOKEN: ${{ secrets.GH_AW_CI_TRIGGER_TOKEN }} with: github-token: ${{ secrets.GH_AW_GITHUB_TOKEN || secrets.GITHUB_TOKEN }} diff --git a/.github/workflows/ci-status-fix.md b/.github/workflows/ci-status-fix.md index d3227e25170c..9c2f46054029 100644 --- a/.github/workflows/ci-status-fix.md +++ b/.github/workflows/ci-status-fix.md @@ -15,7 +15,9 @@ description: | humans (the open tracking issue is the hand-off surface; a dedicated [ci-fix][needs-human] PR is planned but currently deferred — see Step 6). CI on the PR is kicked by a human `/azp run` for now; the loop watches, classifies, - and re-fixes autonomously between kicks. + and re-fixes autonomously between kicks. When the SPECIFIC test a PR fixed is + confirmed green in that PR's own CI, the loop marks the draft PR ready for review + (a state transition only — it never approves or merges). Never mutes tests, but de-flakes genuinely flaky ones (deterministic synchronization, no retries / timeout bumps). Always skips visual-regression / screenshot issues. @@ -239,8 +241,15 @@ safe-outputs: # PER-RUN total (not per-PR): a sweep may surface several green PRs and/or # annotate several flaky reds in one run. At max:1 all but the first comment # were silently dropped, starving the workflow's #1 value (surfacing green PRs - # for review). Raised to 3 to match the per-run throughput of the other outputs. - max: 3 + # for review). Sized to 6 (not 3) because a single green DRAFT PR that gets + # flipped ready in one sweep spends TWO comment slots — the Step 3 ✅ surface + # comment (which still names the /azp-gated legs a human must kick, so it is NOT + # redundant with 🎯) AND the Step 3.6 T3 🎯 readiness comment. At max:3 the shared + # bucket drained after ~1 draft flip and T3's atomicity pre-check then deferred + # every further mark-ready even while the mark-ready/add_labels buckets (also 3) + # sat idle; 6 lets ~3 draft flips (2 comments each) land per sweep, so the + # mark-ready:3 / add_labels:3 caps are actually reachable. + max: 6 target: "*" # Hard constraint (defense-in-depth): only comment on THIS workflow's own # ci-fix PRs — [ci-fix] title prefix AND agentic-workflows label. Mirrors the @@ -261,13 +270,51 @@ safe-outputs: # never the title, so disable title rewrites — this compiles to allow_title:false # and removes any ability to retitle an arbitrary PR. title: false - # NOTE: gh-aw v0.79.8 does NOT emit required-title-prefix/required-labels into + # NOTE: gh-aw v0.80.9 (the pinned compiler) does NOT emit required-title-prefix/required-labels into # the compiled config for update-pull-request (verified against the lock — it # silently drops them, unlike add-comment / push-to-pull-request-branch which # honor them). So which-PR scoping here relies on prompt Hard-Rule 6 + # min-integrity:approved; the residual is body-marker edits only (no code, no # merge, no close), capped at max:3. Revisit required-* if a future gh-aw # compiler emits them for this output. + mark-pull-request-as-ready-for-review: + # Flip a validated draft [ci-fix] PR to ready-for-review once the SPECIFIC test + # this PR was opened to fix is confirmed green in the PR's OWN CI (Step 3.6) — + # even when unrelated legs are red. The workflow is schedule/dispatch-triggered + # (no triggering-PR context), so target "*" lets the agent name the PR number it + # validated. This is a state transition ONLY: it never approves and never merges — + # a human still reviews and merges. + # PER-RUN total (not per-PR): a sweep may validate several PRs' target tests green. + max: 3 + target: "*" + # Hard constraint (defense-in-depth): only ever un-draft THIS workflow's own + # ci-fix PRs — [ci-fix] title prefix AND agentic-workflows label — so a confused + # or prompt-injected agent cannot mark an arbitrary PR ready. (If the v0.80.9 + # compiler silently drops these — as documented for update-pull-request above — + # the Step 3.6 preconditions + min-integrity:approved are the compensating scope + # controls; verify against the lock after compiling.) + required-title-prefix: "[ci-fix] " + required-labels: [agentic-workflows] + add-labels: + # Apply the p/0 priority label to a [ci-fix] PR at the exact moment the loop flips it + # from draft to ready-for-review (Step 3.6 T3) — so a validated, review-ready fix lands + # in the team's p/0 triage queue instead of sitting unseen in the draft backlog. Paired + # 1:1 with mark-pull-request-as-ready-for-review; target "*" lets the agent name the PR + # it just validated. + # allowed = HARD allowlist: the agent may ONLY ever add p/0, nothing else. This caps the + # blast radius of a confused/prompt-injected agent to exactly one benign priority label — + # it can never apply a downstream-triggering or destructive label. + allowed: [p/0] + # PER-RUN total (not per-PR): a sweep may mark several PRs ready in one run; each adds + # one label, so this matches the mark-ready per-run cap (3). + max: 3 + target: "*" + # Hard constraint (defense-in-depth): only ever label THIS workflow's own ci-fix PRs — + # [ci-fix] title prefix AND agentic-workflows label. (If the v0.80.9 compiler drops these + # — as documented for update-pull-request above — the Step 3.6 preconditions + + # min-integrity:approved + the allowed:[p/0] allowlist are the compensating controls.) + required-title-prefix: "[ci-fix] " + required-labels: [agentic-workflows] timeout-minutes: 90 @@ -339,33 +386,48 @@ through `safe-outputs`. 5. **One issue = one outcome per run.** Exactly one of: respond to a maintainer's change-request on the open PR (Track C, Step 3.5.R); open the first fix/help/ de-flake PR; advance an existing PR by one attempt (push a follow-up fix); - surface a validated-green PR for review; annotate an unrelated-flake red; wait + surface a validated-green PR for review; mark a target-validated draft PR + ready-for-review (Step 3.6 T3 — the terminal outcome that supersedes the same + run's surface/annotate precursor line), or record its `already-ready` + steady-state no-op; annotate an unrelated-flake red; wait (CI not yet settled); or a recorded skip (the dedicated needs-human PR is deferred — the attempt cap records a skip instead; see Step 6). Always prefer advancing or opening a PR over a skip when a non-mute diff is producible. 6. **All writes via `safe-outputs`.** Allowed outputs: `create_pull_request` (first attempt only), `push_to_pull_request_branch` (advance an existing PR by one attempt), `update_pull_request` (bump the attempt marker / refresh the - prior-attempts table), and `add_comment` (progress notes on the `[ci-fix]` PR - ONLY). NEVER comment on the tracking issue (issues are locked by + prior-attempts table), `add_comment` (progress notes on the `[ci-fix]` PR + ONLY), `mark_pull_request_as_ready_for_review` (flip a target-validated draft + `[ci-fix]` PR from draft to ready — Step 3.6 T3 only), and `add_labels` (add + ONLY the `p/0` label, ONLY on that same draft→ready transition — Step 3.6 T3). + NEVER comment on the tracking issue (issues are locked by `.github/workflows/ci-scan-lock-issues.yml`) — `add_comment` targets the PR. No `gh pr create`, no manual `git push`. **Defense-in-depth:** `add_comment` and `update_pull_request` use `target: "*"` (the agent supplies the PR number). - `add_comment` is now config-locked to `required-title-prefix: "[ci-fix] "` + + `add_comment`, `mark_pull_request_as_ready_for_review`, and `add_labels` are + config-locked to `required-title-prefix: "[ci-fix] "` + `required-labels: [agentic-workflows]` (hard-enforced by the handler, same as - `push_to_pull_request_branch`), so a comment can only ever land on THIS - workflow's own PRs. `update_pull_request` CANNOT be config-locked to a - title/label in gh-aw v0.79.8 (the compiler silently drops `required-*` for that - output — verified against the lock), so it keeps `title: false` (no retitles; - body-marker edits only) plus this prompt-level guard. Before emitting either, - VERIFY the target PR carries BOTH the `[ci-fix]` title prefix AND the - `agentic-workflows` label; never comment on or edit the body of any PR that - lacks both. + `push_to_pull_request_branch`), and `add_labels` additionally has an + `allowed: [p/0]` allowlist so it can ONLY ever add `p/0` — so these can only + ever land on THIS workflow's own PRs. `update_pull_request` CANNOT be + config-locked to a title/label in gh-aw v0.80.9 (the compiler silently drops + `required-*` for that output — verified against the lock), so it keeps + `title: false` (no retitles; body-marker edits only) plus this prompt-level + guard. Before emitting ANY of these, VERIFY the target PR carries BOTH the + `[ci-fix]` title prefix AND the `agentic-workflows` label; never comment on, + edit, un-draft, or label any PR that lacks both. 7. **Per-run safe-output caps (per RUN, not per PR).** Each output type is capped per run: `create_pull_request` 3, `push_to_pull_request_branch` 3, - `add_comment` 3, `update_pull_request` 3. Note an ADVANCE spends one push **and** - one update **and** one comment, so ≤ 3 advances/run; surfacing a green or - annotating a flake spends one comment. When a bucket is exhausted, do NOT keep + `add_comment` 6, `update_pull_request` 3, `mark_pull_request_as_ready_for_review` + 3, `add_labels` 3 (counts LABELS, not calls). Note an ADVANCE spends one push + **and** one update **and** one comment, so ≤ 3 advances/run; surfacing a green or + annotating a flake spends one comment; a **mark-ready (Step 3.6 T3)** of a + still-draft PR spends TWO comments (the Step 3 ✅ surface **and** the T3 🎯 + readiness note) **and** one mark-ready **and** one label as an ALL-OR-NOTHING set + (T3 pre-checks those buckets and defers the whole PR if any is exhausted — never + mark a PR ready without its 🎯 audit comment). `add_comment` is therefore sized 6 + (not 3) so ~3 draft flips, at 2 comments each, can land in one sweep instead of the + shared comment bucket starving the otherwise-idle mark-ready/add_labels buckets. When a bucket is exhausted, do NOT keep emitting (extras are silently dropped) — record `skipped: per-run cap reached; deferring PR # to next cycle` for each remaining PR so the drop is a deliberate, logged decision. The continuous loop picks the deferred PRs up next @@ -409,8 +471,9 @@ For every open tracking issue in scope, converge on exactly one outcome: | Open first help-wanted draft `[ci-fix]` PR | No open PR yet; a plausible candidate exists but cannot be runner-validated (device/UI tests) (attempt 1) | | Open first de-flake draft `[ci-fix]` PR | No open PR yet; failure is intermittent (green-on-retry) from a genuine test-quality defect; a deterministic-synchronization fix is producible (attempt 1, Step 4.7 bucket b) | | Advance an existing PR (push attempt N+1) | Open PR's own CI settled red *because of the fix itself*, no human engaged, marker < 10 — push a NEW distinct fix onto the same branch (Step 3.5 → 5.6) | -| Surface a validated-green PR | Open PR's own CI settled green — comment "validated, attempt N/10; ready for review" and do NOT advance (Step 3.5) | -| Annotate an unrelated-flake red | Open PR is red only on baseline-flake / unrelated legs — comment which leg needs a re-run; do NOT burn an attempt (Step 3.5) | +| Surface a validated-green PR | Open PR's own CI settled green — comment "validated, attempt N/10; ready for review" and do NOT advance (Step 3.5), then mark the draft PR ready for review once the fixed test is confirmed green (Step 3.6) | +| Annotate an unrelated-flake red | Open PR is red only on baseline-flake / unrelated legs — comment which leg needs a re-run; do NOT burn an attempt (Step 3.5); if the SPECIFIC fixed test is confirmed green on that build, still mark the draft PR ready for review (Step 3.6) | +| Mark a validated PR ready for review | The specific test this PR fixed is confirmed green in the PR's own CI (even if unrelated legs are red) — comment the target-test result and transition the draft PR to ready for review; never approve or merge (Step 3.6) | | Wait | Open PR's CI is pending / not yet settled on the current head SHA — do nothing this cycle (Step 3.5) | | Hand-off skip (attempt cap) | Marker == 10 and the signature still reproduces — stop and defer to humans; the dedicated `[ci-fix][needs-human]` PR is planned but currently deferred (Step 6) | | Recorded skip | Visual-regression, human engaged, already-handled, fixed-in-latest-build, infra-flake, out-of-bounds, only-mute-available, or no novel approach producible | @@ -685,14 +748,34 @@ Run these gates in order — the FIRST that fires decides this cycle's outcome: 1. **Human engaged.** If `C.humanEngaged` → `skipped: human engaged on PR #; deferring` and stop. Never fight a human reviewer — the intentional hand-off boundary still holds the moment a person touches the PR. -2. **CI not settled / unknown / incomplete prefetch.** If `C.checksSettled == false` - OR `C.overallConclusion` is `pending`, `neutral`, or `unknown`, OR - `C.dataComplete == false` → the fix's CI has not finished on the current head SHA - (`C.headSha`), or the prefetch could not fully read this PR (a partial read can - understate `humanEngaged` / `attempt`). `skipped: PR # CI pending / prefetch - incomplete on ; waiting` and stop. *(Round 1: this is where a - maintainer `/azp run` is awaited — the `/azp`-gated uitests/devicetests legs will - not have run until a human kicks them.)* +2. **CI not settled — but validate the target test first (target-focused readiness).** + If `C.checksSettled == false` OR `C.overallConclusion` is `pending`, `neutral`, or + `unknown`, OR `C.dataComplete == false` → the *overall* build has not finished on the + current head SHA (`C.headSha`), or the prefetch could not fully read this PR (a partial + read can understate `humanEngaged` / `attempt`). Unrelated legs still draining must NOT + keep an already-proven fix parked as a draft, so before waiting, try the target-focused + fast-path: + - **2a — Target-green fast-path (draft `[ci-fix]` PRs only).** If `C.dataComplete == + true` AND the Step 3.6 preconditions hold (draft PR, `[ci-fix] ` title prefix AND the + `agentic-workflows` label), run Step 3.6 **T1–T2** against `C.headSha` now: identify + the target test(s) and read the PR's OWN build **timeline / per-leg status** (anonymous + `_apis/build` — never `_apis/test`, per Hard-Rule 8). If EVERY target test is + **VALIDATED-GREEN** (per T2's all-platform definition below — the target's category leg + is `succeeded` on every platform that runs it, failed on none, and NO target platform + family the pipeline covers is still pending/unconcluded, per T2's "no platform left + unverified" rule; a platform that simply has no leg for the target's category — the test + doesn't run there — is fine, not a gap) AND every leg in + `C.failedLegs` that has ALREADY concluded classifies as **unrelated flake** (Step 4 / + Step 4.7 method — the fix is implicated in NO completed red), then the fix is proven + regardless of unrelated *pending* legs → run **Step 3.6 T3** (mark ready + 🎯 comment) + and stop. This early-out NEVER advances an attempt and NEVER acts on a red that could + be the fix's fault. + - **2b — Otherwise WAIT.** If the target test has not yet executed (pending/absent on + its leg), or a completed red is (or may be) caused by the fix, or `C.dataComplete == + false`, or this PR has no identifiable target test → `skipped: PR # CI pending / + target not yet validated on ; waiting` and stop. *(Round 1: this is where a + maintainer `/azp run` is awaited — the `/azp`-gated uitests/devicetests legs will not + have run until a human kicks them.)* 3. **Green → surface for review.** If `C.overallConclusion == "success"`: the checks that RAN are green. **Primary-gate check first:** the deterministic `success` verdict only certifies "at least one green check, nothing failing or @@ -704,18 +787,31 @@ Run these gates in order — the FIRST that fires decides this cycle's outcome: # primary CI gate (maui-pr) not green on ; waiting` and stop — do NOT surface. (The `/azp`-gated `maui-pr-uitests` (def 313) / `maui-pr-devicetests` (def 314) legs MAY still be un-run — that is expected and is named below, not a - reason to withhold the surface.) **Idempotency:** scan the PR's existing comments - for a prior bot `✅ … validated … on ` note for THIS head SHA — if one - already exists, record `already-surfaced PR # (head )` and stop - WITHOUT re-commenting (re-surfacing the same green every tick is noise and burns - the per-run comment budget other PRs need). Otherwise resolve the attempt number from - `C.effectiveAttempt` (the authoritative max(marker, bot-commit) counter; omit the - number only if it is somehow indeterminate). **Dry-run gate (Step 0):** if `dry_run == "true"`, do NOT emit any comment — instead print the intended `✅` validated-green body to the run log, tally `dry-run: would-surface-green PR #`, and stop. Otherwise `add_comment` on PR #: `✅ Attempt /10 validated - — the fix's CI is green on . Ready for human review.` - Name any `/azp`-gated legs (uitests def 313 / devicetests def 314) that have not - run and still need a maintainer `/azp run`. Do NOT advance. Record - `surfaced-green PR # (attempt /10)` and stop. *(This directly - attacks the real bottleneck — no reviews — so it is the highest-value outcome.)* + reason to withhold the surface.) **Comment idempotency + dry-run suppress the ✅ + comment ONLY — neither skips Step 3.6.** Scan the PR's existing comments for a prior + bot `✅ … validated … on ` note for THIS head SHA; if one exists, set + `SKIP_SURFACE_COMMENT = true`. Resolve the attempt number from `C.effectiveAttempt` + (the authoritative max(marker, bot-commit) counter; omit the number only if it is + somehow indeterminate). Then post the surface comment UNLESS suppressed: + - if `dry_run == "true"`: do NOT emit — print the intended `✅` validated-green body to + the run log and tally `dry-run: would-surface-green PR #`; + - else if `SKIP_SURFACE_COMMENT`: do NOT re-post — record `already-surfaced PR # + (head )` (re-surfacing the same green every tick is noise and burns the + per-run comment budget other PRs need); + - else `add_comment` on PR #: `✅ Attempt /10 validated — the fix's CI is + green on .` naming any `/azp`-gated legs (uitests def 313 / devicetests def + 314) that have not run and still need a maintainer `/azp run`; record `surfaced-green + PR # (attempt /10)`. (Do NOT assert "ready for human review" on a PR that + is still a draft — Step 3.6, not this comment, owns the draft→ready flip and posts its + own 🎯 announcement when it fires.) + Do NOT advance. Then — in ALL of the above cases — run **Step 3.6** (target-test + readiness gate) for this PR before stopping: its `/azp`-gated target legs may conclude + green on this SAME head SHA on a LATER sweep (an `/azp run` adds no commit), and Step + 3.6 is the ONLY place the draft→ready flip happens, so it MUST re-evaluate every sweep — + never `stop` here before it. *(This directly attacks the real bottleneck — no reviews — + so it is the highest-value outcome.)* 4. **Red → classify caused-by-fix vs unrelated-flake.** If `C.overallConclusion == "failure"`, analyze the PR's OWN failing build (NOT `main`): find the AzDO `maui-pr` build for this PR (filter builds by `branchName=refs/pull//merge`, @@ -723,18 +819,26 @@ Run these gates in order — the FIRST that fires decides this cycle's outcome: method and Step 4.7 flake buckets to `C.failedLegs`. - **Unrelated flake only** (every failed leg is known-flaky / infra / a pre-existing baseline red NOT introduced by the fix): do NOT burn an attempt. - **Idempotency first:** scan the PR's existing comments for a prior bot - `♻️ … unrelated flake … on ` note for THIS head SHA — if one already - exists, record `already-annotated-flake PR # (head )` and stop - WITHOUT re-commenting (re-annotating the same flake every 12h sweep is noise and - burns the per-run comment budget other PRs need). Otherwise resolve the attempt - number from `C.effectiveAttempt` (the authoritative max(marker, bot-commit) - counter), omitting the `Attempt /10:` prefix only if it is somehow - indeterminate. **Dry-run gate (Step 0):** if `dry_run == "true"`, do NOT emit any comment — instead print the intended `♻️` unrelated-flake body to the run log, tally `dry-run: would-annotate-flake PR #`, and stop. Otherwise `add_comment` on PR #: `♻️ Attempt /10: red is - unrelated flake on leg(s) () on ; the fix itself is not - implicated. A maintainer re-run (/azp run ) should clear it.` Record - `annotated-flake PR # (head )` and stop. *(Round 1: human re-runs; - Round 2: auto re-trigger.)* + **Comment idempotency + dry-run suppress the ♻️ comment ONLY — neither skips Step + 3.6.** Scan the PR's existing comments for a prior bot `♻️ … unrelated flake … on + ` note for THIS head SHA; if one exists, set `SKIP_FLAKE_COMMENT = true`. + Resolve the attempt number from `C.effectiveAttempt` (the authoritative max(marker, + bot-commit) counter), omitting the `Attempt /10:` prefix only if it is + somehow indeterminate. Then post the flake note UNLESS suppressed: + - if `dry_run == "true"`: do NOT emit — print the intended `♻️` unrelated-flake body + to the run log and tally `dry-run: would-annotate-flake PR #`; + - else if `SKIP_FLAKE_COMMENT`: do NOT re-post — record `already-annotated-flake PR + # (head )` (re-annotating the same flake every 12h sweep is noise and + burns the per-run comment budget other PRs need); + - else `add_comment` on PR #: `♻️ Attempt /10: red is unrelated flake on + leg(s) () on ; the fix itself is not implicated. A + maintainer re-run (/azp run ) should clear it.` record `annotated-flake + PR # (head )`. + Then — in ALL of the above cases — run **Step 3.6** (target-test readiness gate) for + this PR before stopping: its `/azp`-gated target legs may conclude green on this SAME + head SHA on a LATER sweep, and Step 3.6 is the ONLY place the draft→ready flip + happens, so it MUST re-evaluate every sweep — never `stop` here before it. *(Round 1: + human re-runs; Round 2: auto re-trigger.)* - **Caused by the fix** (a failed leg still matches the original target signature, or the fix introduced a NEW failure): advance an attempt. - **Attempt count.** `attempt = C.effectiveAttempt` — the authoritative @@ -752,6 +856,188 @@ Run these gates in order — the FIRST that fires decides this cycle's outcome: from every prior commit on this branch**, emitted via the Step 5.6 ADVANCE path (push onto the same PR). +#### Step 3.6 — Target-test verification & mark-ready gate + +Reached from the Step 3 **green-surface** branch, the Step 4 **unrelated-flake** branch +(AFTER that branch has posted its comment), and the Step 3.5 **gate-2 target-focused +fast-path** (2a — while unrelated legs are still pending). Purpose: confirm the SPECIFIC +test(s) this PR was opened to fix now PASS in the PR's own CI, and — only when they do +— transition the draft `[ci-fix]` PR to **ready for review** so a maintainer sees a +validated fix instead of a draft. This is the ONLY place the loop flips draft→ready; it +is a state transition, **never** an approval or a merge (a human still reviews and +merges). Overall red on *unrelated* legs must NOT gate readiness — we validate the fix, +not the base branch's flakiness. + +**Preconditions** (ALL must hold; otherwise record `skipped: readiness N/A PR # +()` and stop this gate): +- `C.isDraft == true` — if the PR is already ready-for-review, the draft→ready + transition is already done; do NOT re-mark **and do NOT re-apply `p/0`**. `p/0` is applied + exactly ONCE, atomically with the draft→ready flip (T3 — all-or-nothing with the 🎯 audit + comment + mark-ready), so an already-ready loop PR that is MISSING `p/0` has almost + certainly had it **removed by a maintainer** de-prioritizing the PR — a pure triage action + the human-engagement guard (which inspects comments/reviews/commits, NOT label changes) + cannot see. Re-adding `p/0` every sweep would fight that maintainer indefinitely, violating + the loop's "never override a human" contract. So the loop does NOT reconcile the label: + record `already-ready PR #` and stop this gate. (Trade-off: on the rare occasion a + transient API error drops `p/0` at flip time *after* the T3 atomic pre-check passed, it is + not auto-re-added — but the PR is still ready-for-review with its 🎯 audit comment, and + re-adding `p/0` is a trivial manual action; that is strictly preferable to steam-rolling a + maintainer's deliberate de-prioritization.) +- The PR is unmistakably THIS workflow's own: `[ci-fix] ` title prefix AND the + `agentic-workflows` label (mirrors the safe-output lock; the compensating scope + control if the v0.80.9 compiler drops the declarative required-*). +- You reached this gate from Step 3 (green), Step 4 (**unrelated-flake**), or the Step 3.5 + gate-2 **target-focused fast-path** (2a) — i.e. the fix is NOT implicated in any red that + has concluded. If Step 4 classified a red as **caused by the fix** (ADVANCE), do NOT run + this gate — advance the attempt instead. + +**T1 — Identify the target test(s).** From the `[ci-scan]` issue signature the fix +addresses (and the PR's own diff), extract the fully-qualified test method name(s) the +fix targets — e.g. `SafeAreaShouldWorkOnAllShellTabs`. For a de-flake it is the +de-flaked test; for a product fix it is the originally-failing test(s). If NO specific +test can be identified (e.g. a product build-break rather than a test failure), this is a +**build-only fix** — there is no single test to validate cross-platform, so validate at +**whole-build** granularity rather than stopping (Step 3 only *comments*; Step 3.6 is the +sole draft→ready flip, so a build-only fix is undrafted HERE or not at all). We only reach +this gate from Step 3 (green) / Step 4 (unrelated-flake) / gate-2a, so the fix is not +implicated in any concluded red; additionally require, via the T2 timeline method on +`C.headSha`, that the primary build pipeline `maui-pr` (def 302) has CONCLUDED with EVERY +one of its platform build legs `succeeded`/`completed` and NONE failed/canceled or still +pending — the build is green on **every** platform, not just the originally-broken one (the +cross-platform guard, applied to the build instead of a test). A build-only fix is undrafted +ONLY on the strength of the auto-running `maui-pr` (def 302) whole-build green, which the +loop CAN observe: if the PR's `[ci-scan]` issue signature or its own diff indicates the +ORIGINATING failure was in a `/azp`-gated pipeline (`maui-pr-uitests` def 313 or +`maui-pr-devicetests` def 314) rather than `maui-pr` (def 302), do NOT undraft on +`maui-pr`-green alone — those pipelines do not auto-run on this PR (GITHUB_TOKEN cannot +trigger them), so a green `maui-pr` build is NOT evidence the gated build break is fixed; +record `skipped: build-only fix PR # targets gated pipeline () not run — +deferring to human` and stop this gate. Otherwise, set `TARGET := "the maui-pr build +(build-only fix — no single target test)"` and proceed to **T3** to mark ready. If any `maui-pr` build leg is still unconcluded, record `skipped: build-only fix PR +# not yet whole-build green (leg(s) pending)` and stop this gate WITHOUT marking +ready (a green subset is not enough — a still-pending build leg could yet fail). + +**T2 — Verify the target test's platform legs are green (anonymous, leg-level).** Per-test +outcomes come from the AzDO **test-results** API (`_apis/test/...`), which is **NOT reachable +anonymously — Hard-Rule 8**: those endpoints 302-redirect to a sign-in page on this runner, so +a `curl` returns an HTML redirect (not JSON) and every target test would look "not executed". +Validate instead at **leg granularity** using ONLY the anonymous `_apis/build/...` timeline +(Hard-Rule 8) — a `succeeded` category leg means every test in that category **that actually +ran** passed on that platform. This is a strong bar **only if the target test genuinely +executes**: a job can go green while one specific test is skipped at runtime (`[Ignore]`, +`Assert.Ignore()`, `Assert.Inconclusive()`, a `Skip=`/conditional `[Fact]`, an `#if`-out, or +an early `return` before the asserts). So before trusting a green leg as proof the target +passed, **confirm from the PR's OWN diff that the fix does NOT skip, ignore, disable, or +short-circuit the target test** (it adds no `[Ignore]`/`[Explicit]`, `Assert.Ignore`/ +`Assert.Inconclusive`, `Skip=`, category-exclusion, `#if`-out, or early `return` around the +target). If the fix could cause the target to be runtime-skipped rather than genuinely pass, +leg-level green is NOT sufficient evidence — record `skipped: target test may be +runtime-skipped by this fix (diff adds ); leg-green insufficient, deferring to human +on PR #` and stop this gate WITHOUT marking ready. Otherwise (the target genuinely runs +and asserts, as a de-flake or product fix does) a green category leg cannot hide a +target-test failure, so treat it as authoritative. + +Map the target test to the CI leg(s) that run it. A leg name encodes platform + test category +(e.g. `Android UITests SafeAreaEdges,Shadow`, `iOS UITests SafeAreaEdges,Shadow`, `macOS +UITests SafeAreaEdges,Shadow`, `Windows UITests SafeAreaEdges,Shadow`). Determine the target +test's UI-test category — its `[Category(UITestCategories.X)]` in the test/HostApp source, +visible in the PR diff or the test file — to know which leg-name substring identifies its legs; +for a device test, the per-platform device-test legs. + +Using the SAME build-discovery as Step 4 (filter AzDO builds by `branchName=refs/pull//merge` +or `sourceVersion == C.headSha`), read each build's **timeline** on `C.headSha` for the +pipeline(s) that RUN the target test — `maui-pr` (def 302) for unit/integration tests, +`maui-pr-uitests` (def 313) for Appium UI tests, `maui-pr-devicetests` (def 314) for device +tests: + +```bash +ORG=dnceng-public; PROJ=public; BUILD= +# Anonymous + Hard-Rule-8-compliant. NEVER call _apis/test/... (it 302-redirects to sign-in). +# Each type=="Job" record is one platform leg: result (succeeded/failed/canceled/null), +# state (completed/inProgress/pending), name (encodes " UITests "): +curl -s "https://dev.azure.com/$ORG/$PROJ/_apis/build/builds/$BUILD/timeline?api-version=7.1" \ + | tee /tmp/gh-aw/agent/timeline_${BUILD}.json \ + | jq -r '.records[] | select(.type=="Job") | "\(.result)\t\(.state)\t\(.name)"' +``` + +(The prefetch already exposes per-leg status in `C.failedLegs` / the checks context, and +`gh pr checks ` lists the same per-leg rows with platform+category in the name — use +whichever is handy; the build timeline is the authoritative cross-check on the exact build.) + +Treat a target test as **VALIDATED-GREEN** only if, on `C.headSha`, the leg that runs its +category is green on **every platform it runs on** — a fix that repairs one platform must not +silently regress the same test's leg on another, so a green leg on the originally-red platform +alone is NOT enough. Across the target pipeline's platform legs (for a UI test: the Android, +iOS, Windows, and macOS/MacCatalyst legs running the target's category; for a device test: each +device-test platform), require ALL of: +- the target's category leg is `result == "succeeded"` **and** `state == "completed"` on + **every** platform that runs it, AND +- that leg is `failed` / `canceled` / aborted on **NO** platform, AND +- **no platform is left unverified.** For each platform family the target pipeline covers, that + platform's category leg must have CONCLUDED (`state == "completed"`) on `C.headSha`. If a + platform simply has no leg for the target's category, the test does not run there — that is + fine, not a gap. But if a platform's category leg has **not concluded** (`state` is + `inProgress` / pending, or an `/azp`-gated `maui-pr-uitests` / `maui-pr-devicetests` leg that + has not been kicked), the test's status on that platform is UNKNOWN → the fix is NOT yet + validated across platforms: record `skipped: target test green on but + not yet verified on (leg(s) pending / need /azp run) on PR #` + and stop this gate WITHOUT marking ready. + +A target test whose category leg never concluded on ANY platform (**not executed** anywhere — +e.g. its `/azp`-gated pipeline has not been kicked) is likewise NOT validated: record `skipped: +target test not yet executed on PR # ( not run — needs /azp run)` and stop +this gate WITHOUT marking ready. Do NOT overclaim — a green *sibling* leg (a different category +on the same platform) is not the target's leg, and a green leg on one platform is not a pass on +the others. + +**T3 — Mark ready + report.** If EVERY target test is VALIDATED-GREEN. The 🎯 comment, +the mark-ready, and the `p/0` label are THREE SEPARATE safe-outputs — the comment +existing does NOT prove the mark-ready took effect, so they are tracked independently: + +- **Comment idempotency (dup-suppress only — never gates the mark-ready).** We only reach + T3 while `C.isDraft == true` (the precondition stops an already-ready PR before here). So + if a prior bot comment for THIS head that carries the leading `🎯` anchor AND the + contiguous phrase `validated green on ` already exists (BOTH T3 variants — + `🎯 Target test validated green on ` and the build-only `🎯 Build validated green on + ` — contain that exact contiguous phrase, so this recognizes both; keying on + `target test` alone would instead miss the build-only comment and re-post it every sweep. + **Require the `🎯` anchor** so the match can NEVER be loosened to a gapped + `validated…green…on` form that would false-positive on the Step 3 `✅ … validated — the + fix's CI is green on ` surface comment and wrongly suppress the audit 🎯), the comment + landed on an earlier sweep but the **mark-ready did NOT take + effect** (the PR is still a draft) — set `SUPPRESS_COMMENT = true` (do not re-post the + duplicate 🎯 comment) but STILL complete the mark-ready + label below. Never treat the + comment's existence as "already marked ready" while the PR is still a draft. Record this + case as `re-marking PR # (🎯 present; mark-ready did not take on a prior sweep)`. +- **Atomicity budget pre-check (Hard-Rule 7).** Determine the outputs this gate will emit: + `mark_pull_request_as_ready_for_review` + `add_labels` always, plus `add_comment` unless + `SUPPRESS_COMMENT`. Verify a free per-run slot remains in EACH bucket you are about to + use. If ANY required bucket is exhausted, emit NONE of them — record `skipped: per-run + cap reached; deferring mark-ready PR # to next cycle` and stop this gate. Never mark a + PR ready (or label it) unless its 🎯 audit comment is GUARANTEED to exist for this exact + head SHA — either posted in this same sweep, or (in the `SUPPRESS_COMMENT` re-marking case) + already present from a prior sweep. The outputs you DO emit either all land or all defer + together; the audit trail must never be absent, but it is NOT re-posted when it already exists. +- **Dry-run gate (Step 0):** if `dry_run == "true"`, emit NOTHING — print the intended + readiness comment and "would mark ready + add p/0" to the run log, tally `dry-run: + would-mark-ready PR #`, and stop. +- Otherwise emit for THIS PR number ``: + 1. `add_comment` (ONLY if not `SUPPRESS_COMMENT`): `🎯 Target test validated green on + — passed on ALL platforms it runs on (, + buildId ). — not + caused by this fix."> Transitioning this PR from draft to ready for review and adding + `p/0`; a maintainer still reviews and merges.` (For a **build-only fix** — the T1 + whole-build fallback, no single target test — phrase the first clause as `🎯 Build + validated green on — the maui-pr build passed on ALL platforms (buildId + ).` instead of naming a test.) + 2. `mark_pull_request_as_ready_for_review` with `reason:` a one-line justification + naming the validated test(s) and ``. + 3. `add_labels` with `labels: ["p/0"]` for PR # — put the now-review-ready fix into + the team's p/0 priority queue so it is triaged, not lost in the draft backlog. (If + the PR somehow already carries `p/0`, this is a harmless no-op.) +- Record `marked-ready PR # (target green on ALL platforms on , + labeled p/0)` and stop. + #### Step 3.5.R — Maintainer change-request response (Track C) Reached from Step 3.5 gate 0. Goal: when an **eligible human reviewer** (Hard-Rule @@ -1383,7 +1669,19 @@ Filed by [`ci-status-fix`](https://github.com/dotnet/maui/blob/main/.github/work ### Step 8 — Per-issue tally + end-of-run summary -Per issue, append one outcome line to `/tmp/gh-aw/agent/coverage.txt`: +Per issue, append **one** outcome line to `/tmp/gh-aw/agent/coverage.txt` — the +terminal outcome for the cycle. When one cycle produces a chained pair — a PR gets a +same-cycle precursor line and **then** a Step 3.6 readiness line (`marked-ready` on the +draft→ready flip, or `already-ready` on a later steady-state sweep of an already-ready +PR) — record ONLY the terminal Step 3.6 readiness line: it supersedes ANY same-cycle +non-readiness precursor for that PR, so an aggregator keying on one-line-per-issue never +double-counts. The precursor may be a Step 3 green line (`surfaced-green` on the first ✅, +or `already-surfaced` when it already exists) OR — when an unrelated-flake red and a +target-green coincide — a Step 4 `annotated-flake` line; either way the readiness line is +terminal, and the superseded precursor's signal survives in the PR's 🎯/♻️ comment so no +information is lost. This covers the flip cycle (`surfaced-green` → `marked-ready`), the +post-flip steady state (`already-surfaced` → `already-ready`), and the flake-coincident +flip (`annotated-flake` → `marked-ready`): ``` # main attempt- @@ -1394,8 +1692,14 @@ Per issue, append one outcome line to `/tmp/gh-aw/agent/coverage.txt`: `advance-PR # attempt /10` (ADVANCE mode — new commit pushed to the existing PR), `surfaced-green PR #` (fix's own CI went green; commented for review, did not advance), `annotated-flake PR #` (red was unrelated flake; -commented, attempt NOT burned), `waiting PR #` (CI not settled yet), -`dry-run: would-`, `skipped: `. (The +commented, attempt NOT burned), `marked-ready PR #` (the specific fixed test +was confirmed green in the PR's own CI; draft flipped to ready for review), +`already-surfaced PR #` (the fix's ✅ green comment already existed this sweep; +re-post suppressed — superseded by the Step 3.6 readiness line when the PR also flips +ready that cycle), `already-ready PR #` (terminal steady state — the PR was flipped +ready on a prior cycle and remains green on an unchanged head; no action taken, +supersedes `already-surfaced`), `waiting PR #` (CI not settled yet), +`dry-run: would-`, `skipped: `. (The `needs-human-PR` outcome is reserved for the deferred hand-off — Step 6 currently records a skip instead, so it is not emitted.)
(attempt /10)`. (Do NOT assert "ready for human review" on a PR that + is still a draft — Step 3.6, not this comment, owns the draft→ready flip and posts its + own 🎯 announcement when it fires.) + Do NOT advance. Then — in ALL of the above cases — run **Step 3.6** (target-test + readiness gate) for this PR before stopping: its `/azp`-gated target legs may conclude + green on this SAME head SHA on a LATER sweep (an `/azp run` adds no commit), and Step + 3.6 is the ONLY place the draft→ready flip happens, so it MUST re-evaluate every sweep — + never `stop` here before it. *(This directly attacks the real bottleneck — no reviews — + so it is the highest-value outcome.)* 4. **Red → classify caused-by-fix vs unrelated-flake.** If `C.overallConclusion == "failure"`, analyze the PR's OWN failing build (NOT `net11.0`): find the AzDO `maui-pr` build for this PR (filter builds by `branchName=refs/pull//merge`, @@ -733,18 +829,26 @@ Run these gates in order — the FIRST that fires decides this cycle's outcome: method and Step 4.7 flake buckets to `C.failedLegs`. - **Unrelated flake only** (every failed leg is known-flaky / infra / a pre-existing baseline red NOT introduced by the fix): do NOT burn an attempt. - **Idempotency first:** scan the PR's existing comments for a prior bot - `♻️ … unrelated flake … on ` note for THIS head SHA — if one already - exists, record `already-annotated-flake PR # (head )` and stop - WITHOUT re-commenting (re-annotating the same flake every 12h sweep is noise and - burns the per-run comment budget other PRs need). Otherwise resolve the attempt - number from `C.effectiveAttempt` (the authoritative max(marker, bot-commit) - counter), omitting the `Attempt /10:` prefix only if it is somehow - indeterminate. **Dry-run gate (Step 0):** if `dry_run == "true"`, do NOT emit any comment — instead print the intended `♻️` unrelated-flake body to the run log, tally `dry-run: would-annotate-flake PR #`, and stop. Otherwise `add_comment` on PR #: `♻️ Attempt /10: red is - unrelated flake on leg(s) () on ; the fix itself is not - implicated. A maintainer re-run (/azp run ) should clear it.` Record - `annotated-flake PR # (head )` and stop. *(Round 1: human re-runs; - Round 2: auto re-trigger.)* + **Comment idempotency + dry-run suppress the ♻️ comment ONLY — neither skips Step + 3.6.** Scan the PR's existing comments for a prior bot `♻️ … unrelated flake … on + ` note for THIS head SHA; if one exists, set `SKIP_FLAKE_COMMENT = true`. + Resolve the attempt number from `C.effectiveAttempt` (the authoritative max(marker, + bot-commit) counter), omitting the `Attempt /10:` prefix only if it is + somehow indeterminate. Then post the flake note UNLESS suppressed: + - if `dry_run == "true"`: do NOT emit — print the intended `♻️` unrelated-flake body + to the run log and tally `dry-run: would-annotate-flake PR #`; + - else if `SKIP_FLAKE_COMMENT`: do NOT re-post — record `already-annotated-flake PR + # (head )` (re-annotating the same flake every 12h sweep is noise and + burns the per-run comment budget other PRs need); + - else `add_comment` on PR #: `♻️ Attempt /10: red is unrelated flake on + leg(s) () on ; the fix itself is not implicated. A + maintainer re-run (/azp run ) should clear it.` record `annotated-flake + PR # (head )`. + Then — in ALL of the above cases — run **Step 3.6** (target-test readiness gate) for + this PR before stopping: its `/azp`-gated target legs may conclude green on this SAME + head SHA on a LATER sweep, and Step 3.6 is the ONLY place the draft→ready flip + happens, so it MUST re-evaluate every sweep — never `stop` here before it. *(Round 1: + human re-runs; Round 2: auto re-trigger.)* - **Caused by the fix** (a failed leg still matches the original target signature, or the fix introduced a NEW failure): advance an attempt. - **Attempt count.** `attempt = C.effectiveAttempt` — the authoritative @@ -762,6 +866,188 @@ Run these gates in order — the FIRST that fires decides this cycle's outcome: from every prior commit on this branch**, emitted via the Step 5.6 ADVANCE path (push onto the same PR). +#### Step 3.6 — Target-test verification & mark-ready gate + +Reached from the Step 3 **green-surface** branch, the Step 4 **unrelated-flake** branch +(AFTER that branch has posted its comment), and the Step 3.5 **gate-2 target-focused +fast-path** (2a — while unrelated legs are still pending). Purpose: confirm the SPECIFIC +test(s) this PR was opened to fix now PASS in the PR's own CI, and — only when they do +— transition the draft `[ci-fix-net11]` PR to **ready for review** so a maintainer sees +a validated fix instead of a draft. This is the ONLY place the loop flips draft→ready; +it is a state transition, **never** an approval or a merge (a human still reviews and +merges). Overall red on *unrelated* legs must NOT gate readiness — we validate the fix, +not the base branch's flakiness. + +**Preconditions** (ALL must hold; otherwise record `skipped: readiness N/A PR # +()` and stop this gate): +- `C.isDraft == true` — if the PR is already ready-for-review, the draft→ready + transition is already done; do NOT re-mark **and do NOT re-apply `p/0`**. `p/0` is applied + exactly ONCE, atomically with the draft→ready flip (T3 — all-or-nothing with the 🎯 audit + comment + mark-ready), so an already-ready loop PR that is MISSING `p/0` has almost + certainly had it **removed by a maintainer** de-prioritizing the PR — a pure triage action + the human-engagement guard (which inspects comments/reviews/commits, NOT label changes) + cannot see. Re-adding `p/0` every sweep would fight that maintainer indefinitely, violating + the loop's "never override a human" contract. So the loop does NOT reconcile the label: + record `already-ready PR #` and stop this gate. (Trade-off: on the rare occasion a + transient API error drops `p/0` at flip time *after* the T3 atomic pre-check passed, it is + not auto-re-added — but the PR is still ready-for-review with its 🎯 audit comment, and + re-adding `p/0` is a trivial manual action; that is strictly preferable to steam-rolling a + maintainer's deliberate de-prioritization.) +- The PR is unmistakably THIS workflow's own: `[ci-fix-net11] ` title prefix AND the + `agentic-workflows` label (mirrors the safe-output lock; the compensating scope + control if the v0.80.9 compiler drops the declarative required-*). +- You reached this gate from Step 3 (green), Step 4 (**unrelated-flake**), or the Step 3.5 + gate-2 **target-focused fast-path** (2a) — i.e. the fix is NOT implicated in any red that + has concluded. If Step 4 classified a red as **caused by the fix** (ADVANCE), do NOT run + this gate — advance the attempt instead. + +**T1 — Identify the target test(s).** From the `[ci-scan-net11]` issue signature the fix +addresses (and the PR's own diff), extract the fully-qualified test method name(s) the +fix targets — e.g. `SafeAreaShouldWorkOnAllShellTabs`. For a de-flake it is the +de-flaked test; for a product fix it is the originally-failing test(s). If NO specific +test can be identified (e.g. a product build-break rather than a test failure), this is a +**build-only fix** — there is no single test to validate cross-platform, so validate at +**whole-build** granularity rather than stopping (Step 3 only *comments*; Step 3.6 is the +sole draft→ready flip, so a build-only fix is undrafted HERE or not at all). We only reach +this gate from Step 3 (green) / Step 4 (unrelated-flake) / gate-2a, so the fix is not +implicated in any concluded red; additionally require, via the T2 timeline method on +`C.headSha`, that the primary build pipeline `maui-pr` (def 302) has CONCLUDED with EVERY +one of its platform build legs `succeeded`/`completed` and NONE failed/canceled or still +pending — the build is green on **every** platform, not just the originally-broken one (the +cross-platform guard, applied to the build instead of a test). A build-only fix is undrafted +ONLY on the strength of the auto-running `maui-pr` (def 302) whole-build green, which the +loop CAN observe: if the PR's `[ci-scan]` issue signature or its own diff indicates the +ORIGINATING failure was in a `/azp`-gated pipeline (`maui-pr-uitests` def 313 or +`maui-pr-devicetests` def 314) rather than `maui-pr` (def 302), do NOT undraft on +`maui-pr`-green alone — those pipelines do not auto-run on this PR (GITHUB_TOKEN cannot +trigger them), so a green `maui-pr` build is NOT evidence the gated build break is fixed; +record `skipped: build-only fix PR # targets gated pipeline () not run — +deferring to human` and stop this gate. Otherwise, set `TARGET := "the maui-pr build +(build-only fix — no single target test)"` and proceed to **T3** to mark ready. If any `maui-pr` build leg is still unconcluded, record `skipped: build-only fix PR +# not yet whole-build green (leg(s) pending)` and stop this gate WITHOUT marking +ready (a green subset is not enough — a still-pending build leg could yet fail). + +**T2 — Verify the target test's platform legs are green (anonymous, leg-level).** Per-test +outcomes come from the AzDO **test-results** API (`_apis/test/...`), which is **NOT reachable +anonymously — Hard-Rule 8**: those endpoints 302-redirect to a sign-in page on this runner, so +a `curl` returns an HTML redirect (not JSON) and every target test would look "not executed". +Validate instead at **leg granularity** using ONLY the anonymous `_apis/build/...` timeline +(Hard-Rule 8) — a `succeeded` category leg means every test in that category **that actually +ran** passed on that platform. This is a strong bar **only if the target test genuinely +executes**: a job can go green while one specific test is skipped at runtime (`[Ignore]`, +`Assert.Ignore()`, `Assert.Inconclusive()`, a `Skip=`/conditional `[Fact]`, an `#if`-out, or +an early `return` before the asserts). So before trusting a green leg as proof the target +passed, **confirm from the PR's OWN diff that the fix does NOT skip, ignore, disable, or +short-circuit the target test** (it adds no `[Ignore]`/`[Explicit]`, `Assert.Ignore`/ +`Assert.Inconclusive`, `Skip=`, category-exclusion, `#if`-out, or early `return` around the +target). If the fix could cause the target to be runtime-skipped rather than genuinely pass, +leg-level green is NOT sufficient evidence — record `skipped: target test may be +runtime-skipped by this fix (diff adds ); leg-green insufficient, deferring to human +on PR #` and stop this gate WITHOUT marking ready. Otherwise (the target genuinely runs +and asserts, as a de-flake or product fix does) a green category leg cannot hide a +target-test failure, so treat it as authoritative. + +Map the target test to the CI leg(s) that run it. A leg name encodes platform + test category +(e.g. `Android UITests SafeAreaEdges,Shadow`, `iOS UITests SafeAreaEdges,Shadow`, `macOS +UITests SafeAreaEdges,Shadow`, `Windows UITests SafeAreaEdges,Shadow`). Determine the target +test's UI-test category — its `[Category(UITestCategories.X)]` in the test/HostApp source, +visible in the PR diff or the test file — to know which leg-name substring identifies its legs; +for a device test, the per-platform device-test legs. + +Using the SAME build-discovery as Step 4 (filter AzDO builds by `branchName=refs/pull//merge` +or `sourceVersion == C.headSha`), read each build's **timeline** on `C.headSha` for the +pipeline(s) that RUN the target test — `maui-pr` (def 302) for unit/integration tests, +`maui-pr-uitests` (def 313) for Appium UI tests, `maui-pr-devicetests` (def 314) for device +tests: + +```bash +ORG=dnceng-public; PROJ=public; BUILD= +# Anonymous + Hard-Rule-8-compliant. NEVER call _apis/test/... (it 302-redirects to sign-in). +# Each type=="Job" record is one platform leg: result (succeeded/failed/canceled/null), +# state (completed/inProgress/pending), name (encodes " UITests "): +curl -s "https://dev.azure.com/$ORG/$PROJ/_apis/build/builds/$BUILD/timeline?api-version=7.1" \ + | tee /tmp/gh-aw/agent/timeline_${BUILD}.json \ + | jq -r '.records[] | select(.type=="Job") | "\(.result)\t\(.state)\t\(.name)"' +``` + +(The prefetch already exposes per-leg status in `C.failedLegs` / the checks context, and +`gh pr checks ` lists the same per-leg rows with platform+category in the name — use +whichever is handy; the build timeline is the authoritative cross-check on the exact build.) + +Treat a target test as **VALIDATED-GREEN** only if, on `C.headSha`, the leg that runs its +category is green on **every platform it runs on** — a fix that repairs one platform must not +silently regress the same test's leg on another, so a green leg on the originally-red platform +alone is NOT enough. Across the target pipeline's platform legs (for a UI test: the Android, +iOS, Windows, and macOS/MacCatalyst legs running the target's category; for a device test: each +device-test platform), require ALL of: +- the target's category leg is `result == "succeeded"` **and** `state == "completed"` on + **every** platform that runs it, AND +- that leg is `failed` / `canceled` / aborted on **NO** platform, AND +- **no platform is left unverified.** For each platform family the target pipeline covers, that + platform's category leg must have CONCLUDED (`state == "completed"`) on `C.headSha`. If a + platform simply has no leg for the target's category, the test does not run there — that is + fine, not a gap. But if a platform's category leg has **not concluded** (`state` is + `inProgress` / pending, or an `/azp`-gated `maui-pr-uitests` / `maui-pr-devicetests` leg that + has not been kicked), the test's status on that platform is UNKNOWN → the fix is NOT yet + validated across platforms: record `skipped: target test green on but + not yet verified on (leg(s) pending / need /azp run) on PR #` + and stop this gate WITHOUT marking ready. + +A target test whose category leg never concluded on ANY platform (**not executed** anywhere — +e.g. its `/azp`-gated pipeline has not been kicked) is likewise NOT validated: record `skipped: +target test not yet executed on PR # ( not run — needs /azp run)` and stop +this gate WITHOUT marking ready. Do NOT overclaim — a green *sibling* leg (a different category +on the same platform) is not the target's leg, and a green leg on one platform is not a pass on +the others. + +**T3 — Mark ready + report.** If EVERY target test is VALIDATED-GREEN. The 🎯 comment, +the mark-ready, and the `p/0` label are THREE SEPARATE safe-outputs — the comment +existing does NOT prove the mark-ready took effect, so they are tracked independently: + +- **Comment idempotency (dup-suppress only — never gates the mark-ready).** We only reach + T3 while `C.isDraft == true` (the precondition stops an already-ready PR before here). So + if a prior bot comment for THIS head that carries the leading `🎯` anchor AND the + contiguous phrase `validated green on ` already exists (BOTH T3 variants — + `🎯 Target test validated green on ` and the build-only `🎯 Build validated green on + ` — contain that exact contiguous phrase, so this recognizes both; keying on + `target test` alone would instead miss the build-only comment and re-post it every sweep. + **Require the `🎯` anchor** so the match can NEVER be loosened to a gapped + `validated…green…on` form that would false-positive on the Step 3 `✅ … validated — the + fix's CI is green on ` surface comment and wrongly suppress the audit 🎯), the comment + landed on an earlier sweep but the **mark-ready did NOT take + effect** (the PR is still a draft) — set `SUPPRESS_COMMENT = true` (do not re-post the + duplicate 🎯 comment) but STILL complete the mark-ready + label below. Never treat the + comment's existence as "already marked ready" while the PR is still a draft. Record this + case as `re-marking PR # (🎯 present; mark-ready did not take on a prior sweep)`. +- **Atomicity budget pre-check (Hard-Rule 7).** Determine the outputs this gate will emit: + `mark_pull_request_as_ready_for_review` + `add_labels` always, plus `add_comment` unless + `SUPPRESS_COMMENT`. Verify a free per-run slot remains in EACH bucket you are about to + use. If ANY required bucket is exhausted, emit NONE of them — record `skipped: per-run + cap reached; deferring mark-ready PR # to next cycle` and stop this gate. Never mark a + PR ready (or label it) unless its 🎯 audit comment is GUARANTEED to exist for this exact + head SHA — either posted in this same sweep, or (in the `SUPPRESS_COMMENT` re-marking case) + already present from a prior sweep. The outputs you DO emit either all land or all defer + together; the audit trail must never be absent, but it is NOT re-posted when it already exists. +- **Dry-run gate (Step 0):** if `dry_run == "true"`, emit NOTHING — print the intended + readiness comment and "would mark ready + add p/0" to the run log, tally `dry-run: + would-mark-ready PR #`, and stop. +- Otherwise emit for THIS PR number ``: + 1. `add_comment` (ONLY if not `SUPPRESS_COMMENT`): `🎯 Target test validated green on + — passed on ALL platforms it runs on (, + buildId ). — not + caused by this fix."> Transitioning this PR from draft to ready for review and adding + `p/0`; a maintainer still reviews and merges.` (For a **build-only fix** — the T1 + whole-build fallback, no single target test — phrase the first clause as `🎯 Build + validated green on — the maui-pr build passed on ALL platforms (buildId + ).` instead of naming a test.) + 2. `mark_pull_request_as_ready_for_review` with `reason:` a one-line justification + naming the validated test(s) and ``. + 3. `add_labels` with `labels: ["p/0"]` for PR # — put the now-review-ready fix into + the team's p/0 priority queue so it is triaged, not lost in the draft backlog. (If + the PR somehow already carries `p/0`, this is a harmless no-op.) +- Record `marked-ready PR # (target green on ALL platforms on , + labeled p/0)` and stop. + #### Step 3.5.R — Maintainer change-request response (Track C) Reached from Step 3.5 gate 0. Goal: when an **eligible human reviewer** (Hard-Rule @@ -1395,7 +1681,19 @@ Filed by [`ci-status-fix-net11`](https://github.com/dotnet/maui/blob/main/.githu ### Step 8 — Per-issue tally + end-of-run summary -Per issue, append one outcome line to `/tmp/gh-aw/agent/coverage.txt`: +Per issue, append **one** outcome line to `/tmp/gh-aw/agent/coverage.txt` — the +terminal outcome for the cycle. When one cycle produces a chained pair — a PR gets a +same-cycle precursor line and **then** a Step 3.6 readiness line (`marked-ready` on the +draft→ready flip, or `already-ready` on a later steady-state sweep of an already-ready +PR) — record ONLY the terminal Step 3.6 readiness line: it supersedes ANY same-cycle +non-readiness precursor for that PR, so an aggregator keying on one-line-per-issue never +double-counts. The precursor may be a Step 3 green line (`surfaced-green` on the first ✅, +or `already-surfaced` when it already exists) OR — when an unrelated-flake red and a +target-green coincide — a Step 4 `annotated-flake` line; either way the readiness line is +terminal, and the superseded precursor's signal survives in the PR's 🎯/♻️ comment so no +information is lost. This covers the flip cycle (`surfaced-green` → `marked-ready`), the +post-flip steady state (`already-surfaced` → `already-ready`), and the flake-coincident +flip (`annotated-flake` → `marked-ready`): ``` # net11.0 attempt- @@ -1406,8 +1704,14 @@ Per issue, append one outcome line to `/tmp/gh-aw/agent/coverage.txt`: `advance-PR # attempt /10` (ADVANCE mode — new commit pushed to the existing PR), `surfaced-green PR #` (fix's own CI went green; commented for review, did not advance), `annotated-flake PR #` (red was unrelated flake; -commented, attempt NOT burned), `waiting PR #` (CI not settled yet), -`dry-run: would-`, `skipped: `. (The +commented, attempt NOT burned), `marked-ready PR #` (the specific fixed test +was confirmed green in the PR's own CI; draft flipped to ready for review), +`already-surfaced PR #` (the fix's ✅ green comment already existed this sweep; +re-post suppressed — superseded by the Step 3.6 readiness line when the PR also flips +ready that cycle), `already-ready PR #` (terminal steady state — the PR was flipped +ready on a prior cycle and remains green on an unchanged head; no action taken, +supersedes `already-surfaced`), `waiting PR #` (CI not settled yet), +`dry-run: would-`, `skipped: `. (The `needs-human-PR` outcome is reserved for the deferred hand-off — Step 6 currently records a skip instead, so it is not emitted.) diff --git a/.github/workflows/ci-status-fix.lock.yml b/.github/workflows/ci-status-fix.lock.yml index 93dc5969cb0d..65511f2b3259 100644 --- a/.github/workflows/ci-status-fix.lock.yml +++ b/.github/workflows/ci-status-fix.lock.yml @@ -1,4 +1,4 @@ -# gh-aw-metadata: {"schema_version":"v4","frontmatter_hash":"f014161967589ebcaf1ac5ec6f3c88ee5a4388d28b55a07dc36c45b7b2aebab6","body_hash":"86c6b2d4c3df13da14c6a3c8b211381fcc9f97bb1951ff7b80588a2c5338e00c","compiler_version":"v0.80.9","strict":true,"agent_id":"copilot","agent_model":"claude-opus-4.8","engine_versions":{"copilot":"1.0.63"}} +# gh-aw-metadata: {"schema_version":"v4","frontmatter_hash":"b0ebf8a2f179900cfe3638e994b27a43cec52bdbdd2331dc1bd4d65762fa2d6b","body_hash":"1825e2b1f7c7bd88241bf37580655489360f9bebc806edd00837a52ce4ad83a0","compiler_version":"v0.80.9","strict":true,"agent_id":"copilot","agent_model":"claude-opus-4.8","engine_versions":{"copilot":"1.0.63"}} # gh-aw-manifest: {"version":1,"secrets":["COPILOT_PAT_0","COPILOT_PAT_1","COPILOT_PAT_2","COPILOT_PAT_3","COPILOT_PAT_4","COPILOT_PAT_5","COPILOT_PAT_6","COPILOT_PAT_7","COPILOT_PAT_8","COPILOT_PAT_9","GH_AW_CI_TRIGGER_TOKEN","GH_AW_GITHUB_MCP_SERVER_TOKEN","GH_AW_GITHUB_TOKEN","GITHUB_TOKEN"],"actions":[{"repo":"actions/checkout","sha":"34e114876b0b11c390a56381ad16ebd13914f8d5","version":"v4"},{"repo":"actions/checkout","sha":"9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0","version":"v7.0.0"},{"repo":"actions/checkout","sha":"de0fac2e4500dabe0009e67214ff5f5447ce83dd","version":"v6.0.2"},{"repo":"actions/download-artifact","sha":"3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c","version":"v8.0.1"},{"repo":"actions/github-script","sha":"3a2844b7e9c422d3c10d287c895573f7108da1b3","version":"v9.0.0"},{"repo":"actions/setup-node","sha":"48b55a011bda9f5d6aeb4c2d9c7362e8dae4041e","version":"v6.4.0"},{"repo":"actions/upload-artifact","sha":"043fb46d1a93c77aae656e7c1c64a875d1fc6a0a","version":"v7.0.1"},{"repo":"github/gh-aw-actions/setup","sha":"8c7d04ebf1ece56cd381446125da3e0f6896294a","version":"v0.80.9"}],"containers":[{"image":"ghcr.io/github/gh-aw-firewall/agent:0.27.7","digest":"sha256:aae231e4635c8999d039c132f1602d3df850fe9b84a00aa2b5ac981179b5661c","pinned_image":"ghcr.io/github/gh-aw-firewall/agent:0.27.7@sha256:aae231e4635c8999d039c132f1602d3df850fe9b84a00aa2b5ac981179b5661c"},{"image":"ghcr.io/github/gh-aw-firewall/api-proxy:0.27.7","digest":"sha256:009caf2e3d88fa77b64e9a03a95a228fc58db0f1701c6d324b29ba5a3c7c79b6","pinned_image":"ghcr.io/github/gh-aw-firewall/api-proxy:0.27.7@sha256:009caf2e3d88fa77b64e9a03a95a228fc58db0f1701c6d324b29ba5a3c7c79b6"},{"image":"ghcr.io/github/gh-aw-firewall/squid:0.27.7","digest":"sha256:deb1d4e19de62d51cee0508057a596a19315c3423ada4d675cad136dc8037c96","pinned_image":"ghcr.io/github/gh-aw-firewall/squid:0.27.7@sha256:deb1d4e19de62d51cee0508057a596a19315c3423ada4d675cad136dc8037c96"},{"image":"ghcr.io/github/gh-aw-mcpg:v0.3.27","digest":"sha256:fe984bddde4ec05d756d9043edb0a32912e6b7b72f6a121b1082f29221421cc7","pinned_image":"ghcr.io/github/gh-aw-mcpg:v0.3.27@sha256:fe984bddde4ec05d756d9043edb0a32912e6b7b72f6a121b1082f29221421cc7"},{"image":"ghcr.io/github/gh-aw-node","digest":"sha256:529d02eb970b1161aa25c593a9c3df57fdfad5a8add328cb3b6eccef66f3183b","pinned_image":"ghcr.io/github/gh-aw-node@sha256:529d02eb970b1161aa25c593a9c3df57fdfad5a8add328cb3b6eccef66f3183b"},{"image":"ghcr.io/github/github-mcp-server:v1.4.0","digest":"sha256:2afb26356481d1a350e14544a6e160f7f7ec1561a1ea309b823665abf0309036","pinned_image":"ghcr.io/github/github-mcp-server:v1.4.0@sha256:2afb26356481d1a350e14544a6e160f7f7ec1561a1ea309b823665abf0309036"}]} # This file was automatically generated by gh-aw (v0.80.9). DO NOT EDIT. To debug this workflow, load the skill at https://github.com/github/gh-aw/blob/main/debug.md # @@ -37,7 +37,9 @@ # humans (the open tracking issue is the hand-off surface; a dedicated # [ci-fix][needs-human] PR is planned but currently deferred — see Step 6). CI on # the PR is kicked by a human `/azp run` for now; the loop watches, classifies, -# and re-fixes autonomously between kicks. +# and re-fixes autonomously between kicks. When the SPECIFIC test a PR fixed is +# confirmed green in that PR's own CI, the loop marks the draft PR ready for review +# (a state transition only — it never approves or merges). # Never mutes tests, but # de-flakes genuinely flaky ones (deterministic synchronization, no retries / # timeout bumps). Always skips visual-regression / screenshot issues. @@ -273,24 +275,24 @@ jobs: run: | bash "${RUNNER_TEMP}/gh-aw/actions/create_prompt_first.sh" { - cat << 'GH_AW_PROMPT_25cf75111b670167_EOF' + cat << 'GH_AW_PROMPT_2ac1d0b4083428f8_EOF' - GH_AW_PROMPT_25cf75111b670167_EOF + GH_AW_PROMPT_2ac1d0b4083428f8_EOF cat "${RUNNER_TEMP}/gh-aw/prompts/xpia.md" cat "${RUNNER_TEMP}/gh-aw/prompts/temp_folder_prompt.md" cat "${RUNNER_TEMP}/gh-aw/prompts/markdown.md" cat "${RUNNER_TEMP}/gh-aw/prompts/safe_outputs_prompt.md" - cat << 'GH_AW_PROMPT_25cf75111b670167_EOF' + cat << 'GH_AW_PROMPT_2ac1d0b4083428f8_EOF' - Tools: add_comment(max:3), create_pull_request(max:3), update_pull_request(max:3), push_to_pull_request_branch(max:3), missing_tool, missing_data, noop - GH_AW_PROMPT_25cf75111b670167_EOF + Tools: add_comment(max:6), create_pull_request(max:3), update_pull_request(max:3), mark_pull_request_as_ready_for_review(max:3), add_labels(max:3), push_to_pull_request_branch(max:3), missing_tool, missing_data, noop + GH_AW_PROMPT_2ac1d0b4083428f8_EOF cat "${RUNNER_TEMP}/gh-aw/prompts/safe_outputs_create_pull_request.md" cat "${RUNNER_TEMP}/gh-aw/prompts/safe_outputs_push_to_pr_branch.md" - cat << 'GH_AW_PROMPT_25cf75111b670167_EOF' + cat << 'GH_AW_PROMPT_2ac1d0b4083428f8_EOF' - GH_AW_PROMPT_25cf75111b670167_EOF + GH_AW_PROMPT_2ac1d0b4083428f8_EOF cat "${RUNNER_TEMP}/gh-aw/prompts/mcp_cli_tools_prompt.md" - cat << 'GH_AW_PROMPT_25cf75111b670167_EOF' + cat << 'GH_AW_PROMPT_2ac1d0b4083428f8_EOF' The following GitHub context information is available for this workflow: {{#if github.actor}} @@ -332,12 +334,12 @@ jobs: stop immediately and report the limitation rather than spending turns trying to work around it. - GH_AW_PROMPT_25cf75111b670167_EOF + GH_AW_PROMPT_2ac1d0b4083428f8_EOF cat "${RUNNER_TEMP}/gh-aw/prompts/github_mcp_tools_with_safeoutputs_prompt.md" - cat << 'GH_AW_PROMPT_25cf75111b670167_EOF' + cat << 'GH_AW_PROMPT_2ac1d0b4083428f8_EOF' {{#runtime-import .github/workflows/ci-status-fix.md}} - GH_AW_PROMPT_25cf75111b670167_EOF + GH_AW_PROMPT_2ac1d0b4083428f8_EOF } > "$GH_AW_PROMPT" - name: Interpolate variables and render templates uses: actions/github-script@3a2844b7e9c422d3c10d287c895573f7108da1b3 # v9.0.0 @@ -554,16 +556,18 @@ jobs: mkdir -p "${RUNNER_TEMP}/gh-aw/safeoutputs" mkdir -p /tmp/gh-aw/safeoutputs mkdir -p /tmp/gh-aw/mcp-logs/safeoutputs - cat > "${RUNNER_TEMP}/gh-aw/safeoutputs/config.json" << 'GH_AW_SAFE_OUTPUTS_CONFIG_fa2a69fad5fd0485_EOF' - {"add_comment":{"discussions":false,"max":3,"required_labels":["agentic-workflows"],"required_title_prefix":"[ci-fix] ","target":"*"},"create_pull_request":{"allowed_base_branches":["main"],"allowed_branches":["ci-fix/**"],"allowed_files":["src/Core/**","src/Controls/**","src/Essentials/**","src/BlazorWebView/**","src/TestUtils/**","src/Templates/**","**/PublicAPI.Unshipped.txt"],"base_branch":"main","draft":true,"labels":["agentic-workflows"],"max":3,"max_patch_files":100,"max_patch_size":4096,"protect_top_level_dot_folders":true,"protected_files":["package.json","bun.lockb","bunfig.toml","deno.json","deno.jsonc","deno.lock","global.json","NuGet.Config","Directory.Packages.props","mix.exs","mix.lock","go.mod","go.sum","stack.yaml","stack.yaml.lock","pom.xml","build.gradle","build.gradle.kts","settings.gradle","settings.gradle.kts","gradle.properties","package-lock.json","yarn.lock","pnpm-lock.yaml","npm-shrinkwrap.json","requirements.txt","Pipfile","Pipfile.lock","pyproject.toml","setup.py","setup.cfg","Gemfile","Gemfile.lock","uv.lock","CODEOWNERS","DESIGN.md","README.md","CONTRIBUTING.md","CHANGELOG.md","SECURITY.md","CODE_OF_CONDUCT.md","AGENTS.md","CLAUDE.md","GEMINI.md"],"protected_files_policy":"request_review","title_prefix":"[ci-fix] "},"create_report_incomplete_issue":{},"missing_data":{},"missing_tool":{},"noop":{"max":1,"report-as-issue":"true"},"push_to_pull_request_branch":{"allowed_files":["src/Core/**","src/Controls/**","src/Essentials/**","src/BlazorWebView/**","src/TestUtils/**","src/Templates/**","**/PublicAPI.Unshipped.txt"],"if_no_changes":"warn","max":3,"max_patch_size":4096,"protect_top_level_dot_folders":true,"protected_files":["package.json","bun.lockb","bunfig.toml","deno.json","deno.jsonc","deno.lock","global.json","NuGet.Config","Directory.Packages.props","mix.exs","mix.lock","go.mod","go.sum","stack.yaml","stack.yaml.lock","pom.xml","build.gradle","build.gradle.kts","settings.gradle","settings.gradle.kts","gradle.properties","package-lock.json","yarn.lock","pnpm-lock.yaml","npm-shrinkwrap.json","requirements.txt","Pipfile","Pipfile.lock","pyproject.toml","setup.py","setup.cfg","Gemfile","Gemfile.lock","uv.lock","CODEOWNERS","DESIGN.md","README.md","CONTRIBUTING.md","CHANGELOG.md","SECURITY.md","CODE_OF_CONDUCT.md","AGENTS.md","CLAUDE.md","GEMINI.md"],"required_labels":["agentic-workflows"],"target":"*","title_prefix":"[ci-fix] "},"report_incomplete":{},"update_pull_request":{"allow_body":true,"allow_title":false,"max":3,"target":"*","update_branch":false}} - GH_AW_SAFE_OUTPUTS_CONFIG_fa2a69fad5fd0485_EOF + cat > "${RUNNER_TEMP}/gh-aw/safeoutputs/config.json" << 'GH_AW_SAFE_OUTPUTS_CONFIG_0644bf1434ca0071_EOF' + {"add_comment":{"discussions":false,"max":6,"required_labels":["agentic-workflows"],"required_title_prefix":"[ci-fix] ","target":"*"},"add_labels":{"allowed":["p/0"],"max":3,"required_labels":["agentic-workflows"],"required_title_prefix":"[ci-fix] ","target":"*"},"create_pull_request":{"allowed_base_branches":["main"],"allowed_branches":["ci-fix/**"],"allowed_files":["src/Core/**","src/Controls/**","src/Essentials/**","src/BlazorWebView/**","src/TestUtils/**","src/Templates/**","**/PublicAPI.Unshipped.txt"],"base_branch":"main","draft":true,"labels":["agentic-workflows"],"max":3,"max_patch_files":100,"max_patch_size":4096,"protect_top_level_dot_folders":true,"protected_files":["package.json","bun.lockb","bunfig.toml","deno.json","deno.jsonc","deno.lock","global.json","NuGet.Config","Directory.Packages.props","mix.exs","mix.lock","go.mod","go.sum","stack.yaml","stack.yaml.lock","pom.xml","build.gradle","build.gradle.kts","settings.gradle","settings.gradle.kts","gradle.properties","package-lock.json","yarn.lock","pnpm-lock.yaml","npm-shrinkwrap.json","requirements.txt","Pipfile","Pipfile.lock","pyproject.toml","setup.py","setup.cfg","Gemfile","Gemfile.lock","uv.lock","CODEOWNERS","DESIGN.md","README.md","CONTRIBUTING.md","CHANGELOG.md","SECURITY.md","CODE_OF_CONDUCT.md","AGENTS.md","CLAUDE.md","GEMINI.md"],"protected_files_policy":"request_review","title_prefix":"[ci-fix] "},"create_report_incomplete_issue":{},"mark_pull_request_as_ready_for_review":{"max":3,"required_labels":["agentic-workflows"],"required_title_prefix":"[ci-fix] ","target":"*"},"missing_data":{},"missing_tool":{},"noop":{"max":1,"report-as-issue":"true"},"push_to_pull_request_branch":{"allowed_files":["src/Core/**","src/Controls/**","src/Essentials/**","src/BlazorWebView/**","src/TestUtils/**","src/Templates/**","**/PublicAPI.Unshipped.txt"],"if_no_changes":"warn","max":3,"max_patch_size":4096,"protect_top_level_dot_folders":true,"protected_files":["package.json","bun.lockb","bunfig.toml","deno.json","deno.jsonc","deno.lock","global.json","NuGet.Config","Directory.Packages.props","mix.exs","mix.lock","go.mod","go.sum","stack.yaml","stack.yaml.lock","pom.xml","build.gradle","build.gradle.kts","settings.gradle","settings.gradle.kts","gradle.properties","package-lock.json","yarn.lock","pnpm-lock.yaml","npm-shrinkwrap.json","requirements.txt","Pipfile","Pipfile.lock","pyproject.toml","setup.py","setup.cfg","Gemfile","Gemfile.lock","uv.lock","CODEOWNERS","DESIGN.md","README.md","CONTRIBUTING.md","CHANGELOG.md","SECURITY.md","CODE_OF_CONDUCT.md","AGENTS.md","CLAUDE.md","GEMINI.md"],"required_labels":["agentic-workflows"],"target":"*","title_prefix":"[ci-fix] "},"report_incomplete":{},"update_pull_request":{"allow_body":true,"allow_title":false,"max":3,"target":"*","update_branch":false}} + GH_AW_SAFE_OUTPUTS_CONFIG_0644bf1434ca0071_EOF - name: Generate Safe Outputs Tools env: GH_AW_TOOLS_META_JSON: | { "description_suffixes": { - "add_comment": " CONSTRAINTS: Maximum 3 comment(s) can be added. Target: *. Supports reply_to_id for discussion threading.", + "add_comment": " CONSTRAINTS: Maximum 6 comment(s) can be added. Target: *. Supports reply_to_id for discussion threading.", + "add_labels": " CONSTRAINTS: Maximum 3 label(s) can be added. Only these labels are allowed: [\"p/0\"]. Target: *.", "create_pull_request": " CONSTRAINTS: Maximum 3 pull request(s) can be created. Title will be prefixed with \"[ci-fix] \". Labels [\"agentic-workflows\"] will be automatically added. Only these labels are allowed: [\"agentic-workflows\"]. PRs will be created as drafts.", + "mark_pull_request_as_ready_for_review": " CONSTRAINTS: Maximum 3 pull request(s) can be marked as ready for review.", "push_to_pull_request_branch": " CONSTRAINTS: Maximum 3 push(es) can be made. The target pull request title must start with \"[ci-fix] \".", "update_pull_request": " CONSTRAINTS: Maximum 3 pull request(s) can be updated. Target: *." }, @@ -594,6 +598,25 @@ jobs: } } }, + "add_labels": { + "defaultMax": 5, + "fields": { + "item_number": { + "issueNumberOrTemporaryId": true + }, + "labels": { + "required": true, + "type": "array", + "itemType": "string", + "itemSanitize": true, + "itemMaxLength": 128 + }, + "repo": { + "type": "string", + "maxLength": 256 + } + } + }, "create_pull_request": { "defaultMax": 1, "fields": { @@ -635,6 +658,24 @@ jobs: } } }, + "mark_pull_request_as_ready_for_review": { + "defaultMax": 1, + "fields": { + "pull_request_number": { + "issueOrPRNumber": true + }, + "reason": { + "required": true, + "type": "string", + "sanitize": true, + "maxLength": 65000 + }, + "repo": { + "type": "string", + "maxLength": 256 + } + } + }, "missing_data": { "defaultMax": 20, "fields": { @@ -1494,7 +1535,7 @@ jobs: uses: actions/github-script@3a2844b7e9c422d3c10d287c895573f7108da1b3 # v9.0.0 env: WORKFLOW_NAME: "CI Failure Fixer (main)" - WORKFLOW_DESCRIPTION: "Periodic pass over open ci-scan tracking issues filed by the main-branch CI\nfailure scanner (.github/workflows/ci-status-main.md). This workflow targets\nthe `main` branch EXCLUSIVELY: it processes only issues labelled ci-scan and\nopens every PR against main. (The net11.0 branch is handled by the parallel\n.github/workflows/ci-status-fix-net11.md — the two are split because gh-aw can\nonly transport a fix relative to ONE static base branch per workflow, and the\nmain↔net11.0 divergence exceeds gh-aw's 10 MB transport-patch cap.) The fixer\nopens ONE draft [ci-fix] PR per actionable issue against main, then WATCHES\nthat PR's own CI on later runs: when the fix's CI comes back red and the red is\ncaused by the fix itself, it pushes a fresh follow-up fix onto the SAME PR\nbranch (never a second PR) — up to 10 attempts — then stops and defers to\nhumans (the open tracking issue is the hand-off surface; a dedicated\n[ci-fix][needs-human] PR is planned but currently deferred — see Step 6). CI on\nthe PR is kicked by a human `/azp run` for now; the loop watches, classifies,\nand re-fixes autonomously between kicks.\nNever mutes tests, but\nde-flakes genuinely flaky ones (deterministic synchronization, no retries /\ntimeout bumps). Always skips visual-regression / screenshot issues." + WORKFLOW_DESCRIPTION: "Periodic pass over open ci-scan tracking issues filed by the main-branch CI\nfailure scanner (.github/workflows/ci-status-main.md). This workflow targets\nthe `main` branch EXCLUSIVELY: it processes only issues labelled ci-scan and\nopens every PR against main. (The net11.0 branch is handled by the parallel\n.github/workflows/ci-status-fix-net11.md — the two are split because gh-aw can\nonly transport a fix relative to ONE static base branch per workflow, and the\nmain↔net11.0 divergence exceeds gh-aw's 10 MB transport-patch cap.) The fixer\nopens ONE draft [ci-fix] PR per actionable issue against main, then WATCHES\nthat PR's own CI on later runs: when the fix's CI comes back red and the red is\ncaused by the fix itself, it pushes a fresh follow-up fix onto the SAME PR\nbranch (never a second PR) — up to 10 attempts — then stops and defers to\nhumans (the open tracking issue is the hand-off surface; a dedicated\n[ci-fix][needs-human] PR is planned but currently deferred — see Step 6). CI on\nthe PR is kicked by a human `/azp run` for now; the loop watches, classifies,\nand re-fixes autonomously between kicks. When the SPECIFIC test a PR fixed is\nconfirmed green in that PR's own CI, the loop marks the draft PR ready for review\n(a state transition only — it never approves or merges).\nNever mutes tests, but\nde-flakes genuinely flaky ones (deterministic synchronization, no retries /\ntimeout bumps). Always skips visual-regression / screenshot issues." HAS_PATCH: ${{ needs.agent.outputs.has_patch }} with: script: | @@ -1910,7 +1951,7 @@ jobs: GH_AW_ALLOWED_DOMAINS: "*.blob.core.windows.net,*.githubusercontent.com,api.business.githubcopilot.com,api.enterprise.githubcopilot.com,api.github.com,api.githubcopilot.com,api.individual.githubcopilot.com,api.snapcraft.io,archive.ubuntu.com,azure.archive.ubuntu.com,codeload.github.com,crl.geotrust.com,crl.globalsign.com,crl.identrust.com,crl.sectigo.com,crl.thawte.com,crl.usertrust.com,crl.verisign.com,crl3.digicert.com,crl4.digicert.com,crls.ssl.com,dev.azure.com,docs.github.com,github-cloud.githubusercontent.com,github-cloud.s3.amazonaws.com,github.blog,github.com,github.githubassets.com,helix.dot.net,host.docker.internal,json-schema.org,json.schemastore.org,keyserver.ubuntu.com,lfs.github.com,objects.githubusercontent.com,ocsp.digicert.com,ocsp.geotrust.com,ocsp.globalsign.com,ocsp.identrust.com,ocsp.sectigo.com,ocsp.ssl.com,ocsp.thawte.com,ocsp.usertrust.com,ocsp.verisign.com,packagecloud.io,packages.cloud.google.com,packages.microsoft.com,patch-diff.githubusercontent.com,ppa.launchpad.net,raw.githubusercontent.com,registry.npmjs.org,s.symcb.com,s.symcd.com,security.ubuntu.com,telemetry.enterprise.githubcopilot.com,ts-crl.ws.symantec.com,ts-ocsp.ws.symantec.com,www.googleapis.com" GITHUB_SERVER_URL: ${{ github.server_url }} GITHUB_API_URL: ${{ github.api_url }} - GH_AW_SAFE_OUTPUTS_HANDLER_CONFIG: "{\"add_comment\":{\"discussions\":false,\"max\":3,\"required_labels\":[\"agentic-workflows\"],\"required_title_prefix\":\"[ci-fix] \",\"target\":\"*\"},\"create_pull_request\":{\"allowed_base_branches\":[\"main\"],\"allowed_branches\":[\"ci-fix/**\"],\"allowed_files\":[\"src/Core/**\",\"src/Controls/**\",\"src/Essentials/**\",\"src/BlazorWebView/**\",\"src/TestUtils/**\",\"src/Templates/**\",\"**/PublicAPI.Unshipped.txt\"],\"base_branch\":\"main\",\"draft\":true,\"labels\":[\"agentic-workflows\"],\"max\":3,\"max_patch_files\":100,\"max_patch_size\":4096,\"protect_top_level_dot_folders\":true,\"protected_files\":[\"package.json\",\"bun.lockb\",\"bunfig.toml\",\"deno.json\",\"deno.jsonc\",\"deno.lock\",\"global.json\",\"NuGet.Config\",\"Directory.Packages.props\",\"mix.exs\",\"mix.lock\",\"go.mod\",\"go.sum\",\"stack.yaml\",\"stack.yaml.lock\",\"pom.xml\",\"build.gradle\",\"build.gradle.kts\",\"settings.gradle\",\"settings.gradle.kts\",\"gradle.properties\",\"package-lock.json\",\"yarn.lock\",\"pnpm-lock.yaml\",\"npm-shrinkwrap.json\",\"requirements.txt\",\"Pipfile\",\"Pipfile.lock\",\"pyproject.toml\",\"setup.py\",\"setup.cfg\",\"Gemfile\",\"Gemfile.lock\",\"uv.lock\",\"CODEOWNERS\",\"DESIGN.md\",\"README.md\",\"CONTRIBUTING.md\",\"CHANGELOG.md\",\"SECURITY.md\",\"CODE_OF_CONDUCT.md\",\"AGENTS.md\",\"CLAUDE.md\",\"GEMINI.md\"],\"protected_files_policy\":\"request_review\",\"title_prefix\":\"[ci-fix] \"},\"create_report_incomplete_issue\":{},\"missing_data\":{},\"missing_tool\":{},\"noop\":{\"max\":1,\"report-as-issue\":\"true\"},\"push_to_pull_request_branch\":{\"allowed_files\":[\"src/Core/**\",\"src/Controls/**\",\"src/Essentials/**\",\"src/BlazorWebView/**\",\"src/TestUtils/**\",\"src/Templates/**\",\"**/PublicAPI.Unshipped.txt\"],\"if_no_changes\":\"warn\",\"max\":3,\"max_patch_size\":4096,\"protect_top_level_dot_folders\":true,\"protected_files\":[\"package.json\",\"bun.lockb\",\"bunfig.toml\",\"deno.json\",\"deno.jsonc\",\"deno.lock\",\"global.json\",\"NuGet.Config\",\"Directory.Packages.props\",\"mix.exs\",\"mix.lock\",\"go.mod\",\"go.sum\",\"stack.yaml\",\"stack.yaml.lock\",\"pom.xml\",\"build.gradle\",\"build.gradle.kts\",\"settings.gradle\",\"settings.gradle.kts\",\"gradle.properties\",\"package-lock.json\",\"yarn.lock\",\"pnpm-lock.yaml\",\"npm-shrinkwrap.json\",\"requirements.txt\",\"Pipfile\",\"Pipfile.lock\",\"pyproject.toml\",\"setup.py\",\"setup.cfg\",\"Gemfile\",\"Gemfile.lock\",\"uv.lock\",\"CODEOWNERS\",\"DESIGN.md\",\"README.md\",\"CONTRIBUTING.md\",\"CHANGELOG.md\",\"SECURITY.md\",\"CODE_OF_CONDUCT.md\",\"AGENTS.md\",\"CLAUDE.md\",\"GEMINI.md\"],\"required_labels\":[\"agentic-workflows\"],\"target\":\"*\",\"title_prefix\":\"[ci-fix] \"},\"report_incomplete\":{},\"update_pull_request\":{\"allow_body\":true,\"allow_title\":false,\"max\":3,\"target\":\"*\",\"update_branch\":false}}" + GH_AW_SAFE_OUTPUTS_HANDLER_CONFIG: "{\"add_comment\":{\"discussions\":false,\"max\":6,\"required_labels\":[\"agentic-workflows\"],\"required_title_prefix\":\"[ci-fix] \",\"target\":\"*\"},\"add_labels\":{\"allowed\":[\"p/0\"],\"max\":3,\"required_labels\":[\"agentic-workflows\"],\"required_title_prefix\":\"[ci-fix] \",\"target\":\"*\"},\"create_pull_request\":{\"allowed_base_branches\":[\"main\"],\"allowed_branches\":[\"ci-fix/**\"],\"allowed_files\":[\"src/Core/**\",\"src/Controls/**\",\"src/Essentials/**\",\"src/BlazorWebView/**\",\"src/TestUtils/**\",\"src/Templates/**\",\"**/PublicAPI.Unshipped.txt\"],\"base_branch\":\"main\",\"draft\":true,\"labels\":[\"agentic-workflows\"],\"max\":3,\"max_patch_files\":100,\"max_patch_size\":4096,\"protect_top_level_dot_folders\":true,\"protected_files\":[\"package.json\",\"bun.lockb\",\"bunfig.toml\",\"deno.json\",\"deno.jsonc\",\"deno.lock\",\"global.json\",\"NuGet.Config\",\"Directory.Packages.props\",\"mix.exs\",\"mix.lock\",\"go.mod\",\"go.sum\",\"stack.yaml\",\"stack.yaml.lock\",\"pom.xml\",\"build.gradle\",\"build.gradle.kts\",\"settings.gradle\",\"settings.gradle.kts\",\"gradle.properties\",\"package-lock.json\",\"yarn.lock\",\"pnpm-lock.yaml\",\"npm-shrinkwrap.json\",\"requirements.txt\",\"Pipfile\",\"Pipfile.lock\",\"pyproject.toml\",\"setup.py\",\"setup.cfg\",\"Gemfile\",\"Gemfile.lock\",\"uv.lock\",\"CODEOWNERS\",\"DESIGN.md\",\"README.md\",\"CONTRIBUTING.md\",\"CHANGELOG.md\",\"SECURITY.md\",\"CODE_OF_CONDUCT.md\",\"AGENTS.md\",\"CLAUDE.md\",\"GEMINI.md\"],\"protected_files_policy\":\"request_review\",\"title_prefix\":\"[ci-fix] \"},\"create_report_incomplete_issue\":{},\"mark_pull_request_as_ready_for_review\":{\"max\":3,\"required_labels\":[\"agentic-workflows\"],\"required_title_prefix\":\"[ci-fix] \",\"target\":\"*\"},\"missing_data\":{},\"missing_tool\":{},\"noop\":{\"max\":1,\"report-as-issue\":\"true\"},\"push_to_pull_request_branch\":{\"allowed_files\":[\"src/Core/**\",\"src/Controls/**\",\"src/Essentials/**\",\"src/BlazorWebView/**\",\"src/TestUtils/**\",\"src/Templates/**\",\"**/PublicAPI.Unshipped.txt\"],\"if_no_changes\":\"warn\",\"max\":3,\"max_patch_size\":4096,\"protect_top_level_dot_folders\":true,\"protected_files\":[\"package.json\",\"bun.lockb\",\"bunfig.toml\",\"deno.json\",\"deno.jsonc\",\"deno.lock\",\"global.json\",\"NuGet.Config\",\"Directory.Packages.props\",\"mix.exs\",\"mix.lock\",\"go.mod\",\"go.sum\",\"stack.yaml\",\"stack.yaml.lock\",\"pom.xml\",\"build.gradle\",\"build.gradle.kts\",\"settings.gradle\",\"settings.gradle.kts\",\"gradle.properties\",\"package-lock.json\",\"yarn.lock\",\"pnpm-lock.yaml\",\"npm-shrinkwrap.json\",\"requirements.txt\",\"Pipfile\",\"Pipfile.lock\",\"pyproject.toml\",\"setup.py\",\"setup.cfg\",\"Gemfile\",\"Gemfile.lock\",\"uv.lock\",\"CODEOWNERS\",\"DESIGN.md\",\"README.md\",\"CONTRIBUTING.md\",\"CHANGELOG.md\",\"SECURITY.md\",\"CODE_OF_CONDUCT.md\",\"AGENTS.md\",\"CLAUDE.md\",\"GEMINI.md\"],\"required_labels\":[\"agentic-workflows\"],\"target\":\"*\",\"title_prefix\":\"[ci-fix] \"},\"report_incomplete\":{},\"update_pull_request\":{\"allow_body\":true,\"allow_title\":false,\"max\":3,\"target\":\"*\",\"update_branch\":false}}" GH_AW_CI_TRIGGER_TOKEN: ${{ secrets.GH_AW_CI_TRIGGER_TOKEN }} with: github-token: ${{ secrets.GH_AW_GITHUB_TOKEN || secrets.GITHUB_TOKEN }} diff --git a/.github/workflows/ci-status-fix.md b/.github/workflows/ci-status-fix.md index d3227e25170c..9c2f46054029 100644 --- a/.github/workflows/ci-status-fix.md +++ b/.github/workflows/ci-status-fix.md @@ -15,7 +15,9 @@ description: | humans (the open tracking issue is the hand-off surface; a dedicated [ci-fix][needs-human] PR is planned but currently deferred — see Step 6). CI on the PR is kicked by a human `/azp run` for now; the loop watches, classifies, - and re-fixes autonomously between kicks. + and re-fixes autonomously between kicks. When the SPECIFIC test a PR fixed is + confirmed green in that PR's own CI, the loop marks the draft PR ready for review + (a state transition only — it never approves or merges). Never mutes tests, but de-flakes genuinely flaky ones (deterministic synchronization, no retries / timeout bumps). Always skips visual-regression / screenshot issues. @@ -239,8 +241,15 @@ safe-outputs: # PER-RUN total (not per-PR): a sweep may surface several green PRs and/or # annotate several flaky reds in one run. At max:1 all but the first comment # were silently dropped, starving the workflow's #1 value (surfacing green PRs - # for review). Raised to 3 to match the per-run throughput of the other outputs. - max: 3 + # for review). Sized to 6 (not 3) because a single green DRAFT PR that gets + # flipped ready in one sweep spends TWO comment slots — the Step 3 ✅ surface + # comment (which still names the /azp-gated legs a human must kick, so it is NOT + # redundant with 🎯) AND the Step 3.6 T3 🎯 readiness comment. At max:3 the shared + # bucket drained after ~1 draft flip and T3's atomicity pre-check then deferred + # every further mark-ready even while the mark-ready/add_labels buckets (also 3) + # sat idle; 6 lets ~3 draft flips (2 comments each) land per sweep, so the + # mark-ready:3 / add_labels:3 caps are actually reachable. + max: 6 target: "*" # Hard constraint (defense-in-depth): only comment on THIS workflow's own # ci-fix PRs — [ci-fix] title prefix AND agentic-workflows label. Mirrors the @@ -261,13 +270,51 @@ safe-outputs: # never the title, so disable title rewrites — this compiles to allow_title:false # and removes any ability to retitle an arbitrary PR. title: false - # NOTE: gh-aw v0.79.8 does NOT emit required-title-prefix/required-labels into + # NOTE: gh-aw v0.80.9 (the pinned compiler) does NOT emit required-title-prefix/required-labels into # the compiled config for update-pull-request (verified against the lock — it # silently drops them, unlike add-comment / push-to-pull-request-branch which # honor them). So which-PR scoping here relies on prompt Hard-Rule 6 + # min-integrity:approved; the residual is body-marker edits only (no code, no # merge, no close), capped at max:3. Revisit required-* if a future gh-aw # compiler emits them for this output. + mark-pull-request-as-ready-for-review: + # Flip a validated draft [ci-fix] PR to ready-for-review once the SPECIFIC test + # this PR was opened to fix is confirmed green in the PR's OWN CI (Step 3.6) — + # even when unrelated legs are red. The workflow is schedule/dispatch-triggered + # (no triggering-PR context), so target "*" lets the agent name the PR number it + # validated. This is a state transition ONLY: it never approves and never merges — + # a human still reviews and merges. + # PER-RUN total (not per-PR): a sweep may validate several PRs' target tests green. + max: 3 + target: "*" + # Hard constraint (defense-in-depth): only ever un-draft THIS workflow's own + # ci-fix PRs — [ci-fix] title prefix AND agentic-workflows label — so a confused + # or prompt-injected agent cannot mark an arbitrary PR ready. (If the v0.80.9 + # compiler silently drops these — as documented for update-pull-request above — + # the Step 3.6 preconditions + min-integrity:approved are the compensating scope + # controls; verify against the lock after compiling.) + required-title-prefix: "[ci-fix] " + required-labels: [agentic-workflows] + add-labels: + # Apply the p/0 priority label to a [ci-fix] PR at the exact moment the loop flips it + # from draft to ready-for-review (Step 3.6 T3) — so a validated, review-ready fix lands + # in the team's p/0 triage queue instead of sitting unseen in the draft backlog. Paired + # 1:1 with mark-pull-request-as-ready-for-review; target "*" lets the agent name the PR + # it just validated. + # allowed = HARD allowlist: the agent may ONLY ever add p/0, nothing else. This caps the + # blast radius of a confused/prompt-injected agent to exactly one benign priority label — + # it can never apply a downstream-triggering or destructive label. + allowed: [p/0] + # PER-RUN total (not per-PR): a sweep may mark several PRs ready in one run; each adds + # one label, so this matches the mark-ready per-run cap (3). + max: 3 + target: "*" + # Hard constraint (defense-in-depth): only ever label THIS workflow's own ci-fix PRs — + # [ci-fix] title prefix AND agentic-workflows label. (If the v0.80.9 compiler drops these + # — as documented for update-pull-request above — the Step 3.6 preconditions + + # min-integrity:approved + the allowed:[p/0] allowlist are the compensating controls.) + required-title-prefix: "[ci-fix] " + required-labels: [agentic-workflows] timeout-minutes: 90 @@ -339,33 +386,48 @@ through `safe-outputs`. 5. **One issue = one outcome per run.** Exactly one of: respond to a maintainer's change-request on the open PR (Track C, Step 3.5.R); open the first fix/help/ de-flake PR; advance an existing PR by one attempt (push a follow-up fix); - surface a validated-green PR for review; annotate an unrelated-flake red; wait + surface a validated-green PR for review; mark a target-validated draft PR + ready-for-review (Step 3.6 T3 — the terminal outcome that supersedes the same + run's surface/annotate precursor line), or record its `already-ready` + steady-state no-op; annotate an unrelated-flake red; wait (CI not yet settled); or a recorded skip (the dedicated needs-human PR is deferred — the attempt cap records a skip instead; see Step 6). Always prefer advancing or opening a PR over a skip when a non-mute diff is producible. 6. **All writes via `safe-outputs`.** Allowed outputs: `create_pull_request` (first attempt only), `push_to_pull_request_branch` (advance an existing PR by one attempt), `update_pull_request` (bump the attempt marker / refresh the - prior-attempts table), and `add_comment` (progress notes on the `[ci-fix]` PR - ONLY). NEVER comment on the tracking issue (issues are locked by + prior-attempts table), `add_comment` (progress notes on the `[ci-fix]` PR + ONLY), `mark_pull_request_as_ready_for_review` (flip a target-validated draft + `[ci-fix]` PR from draft to ready — Step 3.6 T3 only), and `add_labels` (add + ONLY the `p/0` label, ONLY on that same draft→ready transition — Step 3.6 T3). + NEVER comment on the tracking issue (issues are locked by `.github/workflows/ci-scan-lock-issues.yml`) — `add_comment` targets the PR. No `gh pr create`, no manual `git push`. **Defense-in-depth:** `add_comment` and `update_pull_request` use `target: "*"` (the agent supplies the PR number). - `add_comment` is now config-locked to `required-title-prefix: "[ci-fix] "` + + `add_comment`, `mark_pull_request_as_ready_for_review`, and `add_labels` are + config-locked to `required-title-prefix: "[ci-fix] "` + `required-labels: [agentic-workflows]` (hard-enforced by the handler, same as - `push_to_pull_request_branch`), so a comment can only ever land on THIS - workflow's own PRs. `update_pull_request` CANNOT be config-locked to a - title/label in gh-aw v0.79.8 (the compiler silently drops `required-*` for that - output — verified against the lock), so it keeps `title: false` (no retitles; - body-marker edits only) plus this prompt-level guard. Before emitting either, - VERIFY the target PR carries BOTH the `[ci-fix]` title prefix AND the - `agentic-workflows` label; never comment on or edit the body of any PR that - lacks both. + `push_to_pull_request_branch`), and `add_labels` additionally has an + `allowed: [p/0]` allowlist so it can ONLY ever add `p/0` — so these can only + ever land on THIS workflow's own PRs. `update_pull_request` CANNOT be + config-locked to a title/label in gh-aw v0.80.9 (the compiler silently drops + `required-*` for that output — verified against the lock), so it keeps + `title: false` (no retitles; body-marker edits only) plus this prompt-level + guard. Before emitting ANY of these, VERIFY the target PR carries BOTH the + `[ci-fix]` title prefix AND the `agentic-workflows` label; never comment on, + edit, un-draft, or label any PR that lacks both. 7. **Per-run safe-output caps (per RUN, not per PR).** Each output type is capped per run: `create_pull_request` 3, `push_to_pull_request_branch` 3, - `add_comment` 3, `update_pull_request` 3. Note an ADVANCE spends one push **and** - one update **and** one comment, so ≤ 3 advances/run; surfacing a green or - annotating a flake spends one comment. When a bucket is exhausted, do NOT keep + `add_comment` 6, `update_pull_request` 3, `mark_pull_request_as_ready_for_review` + 3, `add_labels` 3 (counts LABELS, not calls). Note an ADVANCE spends one push + **and** one update **and** one comment, so ≤ 3 advances/run; surfacing a green or + annotating a flake spends one comment; a **mark-ready (Step 3.6 T3)** of a + still-draft PR spends TWO comments (the Step 3 ✅ surface **and** the T3 🎯 + readiness note) **and** one mark-ready **and** one label as an ALL-OR-NOTHING set + (T3 pre-checks those buckets and defers the whole PR if any is exhausted — never + mark a PR ready without its 🎯 audit comment). `add_comment` is therefore sized 6 + (not 3) so ~3 draft flips, at 2 comments each, can land in one sweep instead of the + shared comment bucket starving the otherwise-idle mark-ready/add_labels buckets. When a bucket is exhausted, do NOT keep emitting (extras are silently dropped) — record `skipped: per-run cap reached; deferring PR # to next cycle` for each remaining PR so the drop is a deliberate, logged decision. The continuous loop picks the deferred PRs up next @@ -409,8 +471,9 @@ For every open tracking issue in scope, converge on exactly one outcome: | Open first help-wanted draft `[ci-fix]` PR | No open PR yet; a plausible candidate exists but cannot be runner-validated (device/UI tests) (attempt 1) | | Open first de-flake draft `[ci-fix]` PR | No open PR yet; failure is intermittent (green-on-retry) from a genuine test-quality defect; a deterministic-synchronization fix is producible (attempt 1, Step 4.7 bucket b) | | Advance an existing PR (push attempt N+1) | Open PR's own CI settled red *because of the fix itself*, no human engaged, marker < 10 — push a NEW distinct fix onto the same branch (Step 3.5 → 5.6) | -| Surface a validated-green PR | Open PR's own CI settled green — comment "validated, attempt N/10; ready for review" and do NOT advance (Step 3.5) | -| Annotate an unrelated-flake red | Open PR is red only on baseline-flake / unrelated legs — comment which leg needs a re-run; do NOT burn an attempt (Step 3.5) | +| Surface a validated-green PR | Open PR's own CI settled green — comment "validated, attempt N/10; ready for review" and do NOT advance (Step 3.5), then mark the draft PR ready for review once the fixed test is confirmed green (Step 3.6) | +| Annotate an unrelated-flake red | Open PR is red only on baseline-flake / unrelated legs — comment which leg needs a re-run; do NOT burn an attempt (Step 3.5); if the SPECIFIC fixed test is confirmed green on that build, still mark the draft PR ready for review (Step 3.6) | +| Mark a validated PR ready for review | The specific test this PR fixed is confirmed green in the PR's own CI (even if unrelated legs are red) — comment the target-test result and transition the draft PR to ready for review; never approve or merge (Step 3.6) | | Wait | Open PR's CI is pending / not yet settled on the current head SHA — do nothing this cycle (Step 3.5) | | Hand-off skip (attempt cap) | Marker == 10 and the signature still reproduces — stop and defer to humans; the dedicated `[ci-fix][needs-human]` PR is planned but currently deferred (Step 6) | | Recorded skip | Visual-regression, human engaged, already-handled, fixed-in-latest-build, infra-flake, out-of-bounds, only-mute-available, or no novel approach producible | @@ -685,14 +748,34 @@ Run these gates in order — the FIRST that fires decides this cycle's outcome: 1. **Human engaged.** If `C.humanEngaged` → `skipped: human engaged on PR #; deferring` and stop. Never fight a human reviewer — the intentional hand-off boundary still holds the moment a person touches the PR. -2. **CI not settled / unknown / incomplete prefetch.** If `C.checksSettled == false` - OR `C.overallConclusion` is `pending`, `neutral`, or `unknown`, OR - `C.dataComplete == false` → the fix's CI has not finished on the current head SHA - (`C.headSha`), or the prefetch could not fully read this PR (a partial read can - understate `humanEngaged` / `attempt`). `skipped: PR # CI pending / prefetch - incomplete on ; waiting` and stop. *(Round 1: this is where a - maintainer `/azp run` is awaited — the `/azp`-gated uitests/devicetests legs will - not have run until a human kicks them.)* +2. **CI not settled — but validate the target test first (target-focused readiness).** + If `C.checksSettled == false` OR `C.overallConclusion` is `pending`, `neutral`, or + `unknown`, OR `C.dataComplete == false` → the *overall* build has not finished on the + current head SHA (`C.headSha`), or the prefetch could not fully read this PR (a partial + read can understate `humanEngaged` / `attempt`). Unrelated legs still draining must NOT + keep an already-proven fix parked as a draft, so before waiting, try the target-focused + fast-path: + - **2a — Target-green fast-path (draft `[ci-fix]` PRs only).** If `C.dataComplete == + true` AND the Step 3.6 preconditions hold (draft PR, `[ci-fix] ` title prefix AND the + `agentic-workflows` label), run Step 3.6 **T1–T2** against `C.headSha` now: identify + the target test(s) and read the PR's OWN build **timeline / per-leg status** (anonymous + `_apis/build` — never `_apis/test`, per Hard-Rule 8). If EVERY target test is + **VALIDATED-GREEN** (per T2's all-platform definition below — the target's category leg + is `succeeded` on every platform that runs it, failed on none, and NO target platform + family the pipeline covers is still pending/unconcluded, per T2's "no platform left + unverified" rule; a platform that simply has no leg for the target's category — the test + doesn't run there — is fine, not a gap) AND every leg in + `C.failedLegs` that has ALREADY concluded classifies as **unrelated flake** (Step 4 / + Step 4.7 method — the fix is implicated in NO completed red), then the fix is proven + regardless of unrelated *pending* legs → run **Step 3.6 T3** (mark ready + 🎯 comment) + and stop. This early-out NEVER advances an attempt and NEVER acts on a red that could + be the fix's fault. + - **2b — Otherwise WAIT.** If the target test has not yet executed (pending/absent on + its leg), or a completed red is (or may be) caused by the fix, or `C.dataComplete == + false`, or this PR has no identifiable target test → `skipped: PR # CI pending / + target not yet validated on ; waiting` and stop. *(Round 1: this is where a + maintainer `/azp run` is awaited — the `/azp`-gated uitests/devicetests legs will not + have run until a human kicks them.)* 3. **Green → surface for review.** If `C.overallConclusion == "success"`: the checks that RAN are green. **Primary-gate check first:** the deterministic `success` verdict only certifies "at least one green check, nothing failing or @@ -704,18 +787,31 @@ Run these gates in order — the FIRST that fires decides this cycle's outcome: # primary CI gate (maui-pr) not green on ; waiting` and stop — do NOT surface. (The `/azp`-gated `maui-pr-uitests` (def 313) / `maui-pr-devicetests` (def 314) legs MAY still be un-run — that is expected and is named below, not a - reason to withhold the surface.) **Idempotency:** scan the PR's existing comments - for a prior bot `✅ … validated … on ` note for THIS head SHA — if one - already exists, record `already-surfaced PR # (head )` and stop - WITHOUT re-commenting (re-surfacing the same green every tick is noise and burns - the per-run comment budget other PRs need). Otherwise resolve the attempt number from - `C.effectiveAttempt` (the authoritative max(marker, bot-commit) counter; omit the - number only if it is somehow indeterminate). **Dry-run gate (Step 0):** if `dry_run == "true"`, do NOT emit any comment — instead print the intended `✅` validated-green body to the run log, tally `dry-run: would-surface-green PR #`, and stop. Otherwise `add_comment` on PR #: `✅ Attempt /10 validated - — the fix's CI is green on . Ready for human review.` - Name any `/azp`-gated legs (uitests def 313 / devicetests def 314) that have not - run and still need a maintainer `/azp run`. Do NOT advance. Record - `surfaced-green PR # (attempt /10)` and stop. *(This directly - attacks the real bottleneck — no reviews — so it is the highest-value outcome.)* + reason to withhold the surface.) **Comment idempotency + dry-run suppress the ✅ + comment ONLY — neither skips Step 3.6.** Scan the PR's existing comments for a prior + bot `✅ … validated … on ` note for THIS head SHA; if one exists, set + `SKIP_SURFACE_COMMENT = true`. Resolve the attempt number from `C.effectiveAttempt` + (the authoritative max(marker, bot-commit) counter; omit the number only if it is + somehow indeterminate). Then post the surface comment UNLESS suppressed: + - if `dry_run == "true"`: do NOT emit — print the intended `✅` validated-green body to + the run log and tally `dry-run: would-surface-green PR #`; + - else if `SKIP_SURFACE_COMMENT`: do NOT re-post — record `already-surfaced PR # + (head )` (re-surfacing the same green every tick is noise and burns the + per-run comment budget other PRs need); + - else `add_comment` on PR #: `✅ Attempt /10 validated — the fix's CI is + green on .` naming any `/azp`-gated legs (uitests def 313 / devicetests def + 314) that have not run and still need a maintainer `/azp run`; record `surfaced-green + PR # (attempt /10)`. (Do NOT assert "ready for human review" on a PR that + is still a draft — Step 3.6, not this comment, owns the draft→ready flip and posts its + own 🎯 announcement when it fires.) + Do NOT advance. Then — in ALL of the above cases — run **Step 3.6** (target-test + readiness gate) for this PR before stopping: its `/azp`-gated target legs may conclude + green on this SAME head SHA on a LATER sweep (an `/azp run` adds no commit), and Step + 3.6 is the ONLY place the draft→ready flip happens, so it MUST re-evaluate every sweep — + never `stop` here before it. *(This directly attacks the real bottleneck — no reviews — + so it is the highest-value outcome.)* 4. **Red → classify caused-by-fix vs unrelated-flake.** If `C.overallConclusion == "failure"`, analyze the PR's OWN failing build (NOT `main`): find the AzDO `maui-pr` build for this PR (filter builds by `branchName=refs/pull//merge`, @@ -723,18 +819,26 @@ Run these gates in order — the FIRST that fires decides this cycle's outcome: method and Step 4.7 flake buckets to `C.failedLegs`. - **Unrelated flake only** (every failed leg is known-flaky / infra / a pre-existing baseline red NOT introduced by the fix): do NOT burn an attempt. - **Idempotency first:** scan the PR's existing comments for a prior bot - `♻️ … unrelated flake … on ` note for THIS head SHA — if one already - exists, record `already-annotated-flake PR # (head )` and stop - WITHOUT re-commenting (re-annotating the same flake every 12h sweep is noise and - burns the per-run comment budget other PRs need). Otherwise resolve the attempt - number from `C.effectiveAttempt` (the authoritative max(marker, bot-commit) - counter), omitting the `Attempt /10:` prefix only if it is somehow - indeterminate. **Dry-run gate (Step 0):** if `dry_run == "true"`, do NOT emit any comment — instead print the intended `♻️` unrelated-flake body to the run log, tally `dry-run: would-annotate-flake PR #`, and stop. Otherwise `add_comment` on PR #: `♻️ Attempt /10: red is - unrelated flake on leg(s) () on ; the fix itself is not - implicated. A maintainer re-run (/azp run ) should clear it.` Record - `annotated-flake PR # (head )` and stop. *(Round 1: human re-runs; - Round 2: auto re-trigger.)* + **Comment idempotency + dry-run suppress the ♻️ comment ONLY — neither skips Step + 3.6.** Scan the PR's existing comments for a prior bot `♻️ … unrelated flake … on + ` note for THIS head SHA; if one exists, set `SKIP_FLAKE_COMMENT = true`. + Resolve the attempt number from `C.effectiveAttempt` (the authoritative max(marker, + bot-commit) counter), omitting the `Attempt /10:` prefix only if it is + somehow indeterminate. Then post the flake note UNLESS suppressed: + - if `dry_run == "true"`: do NOT emit — print the intended `♻️` unrelated-flake body + to the run log and tally `dry-run: would-annotate-flake PR #`; + - else if `SKIP_FLAKE_COMMENT`: do NOT re-post — record `already-annotated-flake PR + # (head )` (re-annotating the same flake every 12h sweep is noise and + burns the per-run comment budget other PRs need); + - else `add_comment` on PR #: `♻️ Attempt /10: red is unrelated flake on + leg(s) () on ; the fix itself is not implicated. A + maintainer re-run (/azp run ) should clear it.` record `annotated-flake + PR # (head )`. + Then — in ALL of the above cases — run **Step 3.6** (target-test readiness gate) for + this PR before stopping: its `/azp`-gated target legs may conclude green on this SAME + head SHA on a LATER sweep, and Step 3.6 is the ONLY place the draft→ready flip + happens, so it MUST re-evaluate every sweep — never `stop` here before it. *(Round 1: + human re-runs; Round 2: auto re-trigger.)* - **Caused by the fix** (a failed leg still matches the original target signature, or the fix introduced a NEW failure): advance an attempt. - **Attempt count.** `attempt = C.effectiveAttempt` — the authoritative @@ -752,6 +856,188 @@ Run these gates in order — the FIRST that fires decides this cycle's outcome: from every prior commit on this branch**, emitted via the Step 5.6 ADVANCE path (push onto the same PR). +#### Step 3.6 — Target-test verification & mark-ready gate + +Reached from the Step 3 **green-surface** branch, the Step 4 **unrelated-flake** branch +(AFTER that branch has posted its comment), and the Step 3.5 **gate-2 target-focused +fast-path** (2a — while unrelated legs are still pending). Purpose: confirm the SPECIFIC +test(s) this PR was opened to fix now PASS in the PR's own CI, and — only when they do +— transition the draft `[ci-fix]` PR to **ready for review** so a maintainer sees a +validated fix instead of a draft. This is the ONLY place the loop flips draft→ready; it +is a state transition, **never** an approval or a merge (a human still reviews and +merges). Overall red on *unrelated* legs must NOT gate readiness — we validate the fix, +not the base branch's flakiness. + +**Preconditions** (ALL must hold; otherwise record `skipped: readiness N/A PR # +()` and stop this gate): +- `C.isDraft == true` — if the PR is already ready-for-review, the draft→ready + transition is already done; do NOT re-mark **and do NOT re-apply `p/0`**. `p/0` is applied + exactly ONCE, atomically with the draft→ready flip (T3 — all-or-nothing with the 🎯 audit + comment + mark-ready), so an already-ready loop PR that is MISSING `p/0` has almost + certainly had it **removed by a maintainer** de-prioritizing the PR — a pure triage action + the human-engagement guard (which inspects comments/reviews/commits, NOT label changes) + cannot see. Re-adding `p/0` every sweep would fight that maintainer indefinitely, violating + the loop's "never override a human" contract. So the loop does NOT reconcile the label: + record `already-ready PR #` and stop this gate. (Trade-off: on the rare occasion a + transient API error drops `p/0` at flip time *after* the T3 atomic pre-check passed, it is + not auto-re-added — but the PR is still ready-for-review with its 🎯 audit comment, and + re-adding `p/0` is a trivial manual action; that is strictly preferable to steam-rolling a + maintainer's deliberate de-prioritization.) +- The PR is unmistakably THIS workflow's own: `[ci-fix] ` title prefix AND the + `agentic-workflows` label (mirrors the safe-output lock; the compensating scope + control if the v0.80.9 compiler drops the declarative required-*). +- You reached this gate from Step 3 (green), Step 4 (**unrelated-flake**), or the Step 3.5 + gate-2 **target-focused fast-path** (2a) — i.e. the fix is NOT implicated in any red that + has concluded. If Step 4 classified a red as **caused by the fix** (ADVANCE), do NOT run + this gate — advance the attempt instead. + +**T1 — Identify the target test(s).** From the `[ci-scan]` issue signature the fix +addresses (and the PR's own diff), extract the fully-qualified test method name(s) the +fix targets — e.g. `SafeAreaShouldWorkOnAllShellTabs`. For a de-flake it is the +de-flaked test; for a product fix it is the originally-failing test(s). If NO specific +test can be identified (e.g. a product build-break rather than a test failure), this is a +**build-only fix** — there is no single test to validate cross-platform, so validate at +**whole-build** granularity rather than stopping (Step 3 only *comments*; Step 3.6 is the +sole draft→ready flip, so a build-only fix is undrafted HERE or not at all). We only reach +this gate from Step 3 (green) / Step 4 (unrelated-flake) / gate-2a, so the fix is not +implicated in any concluded red; additionally require, via the T2 timeline method on +`C.headSha`, that the primary build pipeline `maui-pr` (def 302) has CONCLUDED with EVERY +one of its platform build legs `succeeded`/`completed` and NONE failed/canceled or still +pending — the build is green on **every** platform, not just the originally-broken one (the +cross-platform guard, applied to the build instead of a test). A build-only fix is undrafted +ONLY on the strength of the auto-running `maui-pr` (def 302) whole-build green, which the +loop CAN observe: if the PR's `[ci-scan]` issue signature or its own diff indicates the +ORIGINATING failure was in a `/azp`-gated pipeline (`maui-pr-uitests` def 313 or +`maui-pr-devicetests` def 314) rather than `maui-pr` (def 302), do NOT undraft on +`maui-pr`-green alone — those pipelines do not auto-run on this PR (GITHUB_TOKEN cannot +trigger them), so a green `maui-pr` build is NOT evidence the gated build break is fixed; +record `skipped: build-only fix PR # targets gated pipeline () not run — +deferring to human` and stop this gate. Otherwise, set `TARGET := "the maui-pr build +(build-only fix — no single target test)"` and proceed to **T3** to mark ready. If any `maui-pr` build leg is still unconcluded, record `skipped: build-only fix PR +# not yet whole-build green (leg(s) pending)` and stop this gate WITHOUT marking +ready (a green subset is not enough — a still-pending build leg could yet fail). + +**T2 — Verify the target test's platform legs are green (anonymous, leg-level).** Per-test +outcomes come from the AzDO **test-results** API (`_apis/test/...`), which is **NOT reachable +anonymously — Hard-Rule 8**: those endpoints 302-redirect to a sign-in page on this runner, so +a `curl` returns an HTML redirect (not JSON) and every target test would look "not executed". +Validate instead at **leg granularity** using ONLY the anonymous `_apis/build/...` timeline +(Hard-Rule 8) — a `succeeded` category leg means every test in that category **that actually +ran** passed on that platform. This is a strong bar **only if the target test genuinely +executes**: a job can go green while one specific test is skipped at runtime (`[Ignore]`, +`Assert.Ignore()`, `Assert.Inconclusive()`, a `Skip=`/conditional `[Fact]`, an `#if`-out, or +an early `return` before the asserts). So before trusting a green leg as proof the target +passed, **confirm from the PR's OWN diff that the fix does NOT skip, ignore, disable, or +short-circuit the target test** (it adds no `[Ignore]`/`[Explicit]`, `Assert.Ignore`/ +`Assert.Inconclusive`, `Skip=`, category-exclusion, `#if`-out, or early `return` around the +target). If the fix could cause the target to be runtime-skipped rather than genuinely pass, +leg-level green is NOT sufficient evidence — record `skipped: target test may be +runtime-skipped by this fix (diff adds ); leg-green insufficient, deferring to human +on PR #` and stop this gate WITHOUT marking ready. Otherwise (the target genuinely runs +and asserts, as a de-flake or product fix does) a green category leg cannot hide a +target-test failure, so treat it as authoritative. + +Map the target test to the CI leg(s) that run it. A leg name encodes platform + test category +(e.g. `Android UITests SafeAreaEdges,Shadow`, `iOS UITests SafeAreaEdges,Shadow`, `macOS +UITests SafeAreaEdges,Shadow`, `Windows UITests SafeAreaEdges,Shadow`). Determine the target +test's UI-test category — its `[Category(UITestCategories.X)]` in the test/HostApp source, +visible in the PR diff or the test file — to know which leg-name substring identifies its legs; +for a device test, the per-platform device-test legs. + +Using the SAME build-discovery as Step 4 (filter AzDO builds by `branchName=refs/pull//merge` +or `sourceVersion == C.headSha`), read each build's **timeline** on `C.headSha` for the +pipeline(s) that RUN the target test — `maui-pr` (def 302) for unit/integration tests, +`maui-pr-uitests` (def 313) for Appium UI tests, `maui-pr-devicetests` (def 314) for device +tests: + +```bash +ORG=dnceng-public; PROJ=public; BUILD= +# Anonymous + Hard-Rule-8-compliant. NEVER call _apis/test/... (it 302-redirects to sign-in). +# Each type=="Job" record is one platform leg: result (succeeded/failed/canceled/null), +# state (completed/inProgress/pending), name (encodes " UITests "): +curl -s "https://dev.azure.com/$ORG/$PROJ/_apis/build/builds/$BUILD/timeline?api-version=7.1" \ + | tee /tmp/gh-aw/agent/timeline_${BUILD}.json \ + | jq -r '.records[] | select(.type=="Job") | "\(.result)\t\(.state)\t\(.name)"' +``` + +(The prefetch already exposes per-leg status in `C.failedLegs` / the checks context, and +`gh pr checks ` lists the same per-leg rows with platform+category in the name — use +whichever is handy; the build timeline is the authoritative cross-check on the exact build.) + +Treat a target test as **VALIDATED-GREEN** only if, on `C.headSha`, the leg that runs its +category is green on **every platform it runs on** — a fix that repairs one platform must not +silently regress the same test's leg on another, so a green leg on the originally-red platform +alone is NOT enough. Across the target pipeline's platform legs (for a UI test: the Android, +iOS, Windows, and macOS/MacCatalyst legs running the target's category; for a device test: each +device-test platform), require ALL of: +- the target's category leg is `result == "succeeded"` **and** `state == "completed"` on + **every** platform that runs it, AND +- that leg is `failed` / `canceled` / aborted on **NO** platform, AND +- **no platform is left unverified.** For each platform family the target pipeline covers, that + platform's category leg must have CONCLUDED (`state == "completed"`) on `C.headSha`. If a + platform simply has no leg for the target's category, the test does not run there — that is + fine, not a gap. But if a platform's category leg has **not concluded** (`state` is + `inProgress` / pending, or an `/azp`-gated `maui-pr-uitests` / `maui-pr-devicetests` leg that + has not been kicked), the test's status on that platform is UNKNOWN → the fix is NOT yet + validated across platforms: record `skipped: target test green on but + not yet verified on (leg(s) pending / need /azp run) on PR #` + and stop this gate WITHOUT marking ready. + +A target test whose category leg never concluded on ANY platform (**not executed** anywhere — +e.g. its `/azp`-gated pipeline has not been kicked) is likewise NOT validated: record `skipped: +target test not yet executed on PR # ( not run — needs /azp run)` and stop +this gate WITHOUT marking ready. Do NOT overclaim — a green *sibling* leg (a different category +on the same platform) is not the target's leg, and a green leg on one platform is not a pass on +the others. + +**T3 — Mark ready + report.** If EVERY target test is VALIDATED-GREEN. The 🎯 comment, +the mark-ready, and the `p/0` label are THREE SEPARATE safe-outputs — the comment +existing does NOT prove the mark-ready took effect, so they are tracked independently: + +- **Comment idempotency (dup-suppress only — never gates the mark-ready).** We only reach + T3 while `C.isDraft == true` (the precondition stops an already-ready PR before here). So + if a prior bot comment for THIS head that carries the leading `🎯` anchor AND the + contiguous phrase `validated green on ` already exists (BOTH T3 variants — + `🎯 Target test validated green on ` and the build-only `🎯 Build validated green on + ` — contain that exact contiguous phrase, so this recognizes both; keying on + `target test` alone would instead miss the build-only comment and re-post it every sweep. + **Require the `🎯` anchor** so the match can NEVER be loosened to a gapped + `validated…green…on` form that would false-positive on the Step 3 `✅ … validated — the + fix's CI is green on ` surface comment and wrongly suppress the audit 🎯), the comment + landed on an earlier sweep but the **mark-ready did NOT take + effect** (the PR is still a draft) — set `SUPPRESS_COMMENT = true` (do not re-post the + duplicate 🎯 comment) but STILL complete the mark-ready + label below. Never treat the + comment's existence as "already marked ready" while the PR is still a draft. Record this + case as `re-marking PR # (🎯 present; mark-ready did not take on a prior sweep)`. +- **Atomicity budget pre-check (Hard-Rule 7).** Determine the outputs this gate will emit: + `mark_pull_request_as_ready_for_review` + `add_labels` always, plus `add_comment` unless + `SUPPRESS_COMMENT`. Verify a free per-run slot remains in EACH bucket you are about to + use. If ANY required bucket is exhausted, emit NONE of them — record `skipped: per-run + cap reached; deferring mark-ready PR # to next cycle` and stop this gate. Never mark a + PR ready (or label it) unless its 🎯 audit comment is GUARANTEED to exist for this exact + head SHA — either posted in this same sweep, or (in the `SUPPRESS_COMMENT` re-marking case) + already present from a prior sweep. The outputs you DO emit either all land or all defer + together; the audit trail must never be absent, but it is NOT re-posted when it already exists. +- **Dry-run gate (Step 0):** if `dry_run == "true"`, emit NOTHING — print the intended + readiness comment and "would mark ready + add p/0" to the run log, tally `dry-run: + would-mark-ready PR #`, and stop. +- Otherwise emit for THIS PR number ``: + 1. `add_comment` (ONLY if not `SUPPRESS_COMMENT`): `🎯 Target test validated green on + — passed on ALL platforms it runs on (, + buildId ). — not + caused by this fix."> Transitioning this PR from draft to ready for review and adding + `p/0`; a maintainer still reviews and merges.` (For a **build-only fix** — the T1 + whole-build fallback, no single target test — phrase the first clause as `🎯 Build + validated green on — the maui-pr build passed on ALL platforms (buildId + ).` instead of naming a test.) + 2. `mark_pull_request_as_ready_for_review` with `reason:` a one-line justification + naming the validated test(s) and ``. + 3. `add_labels` with `labels: ["p/0"]` for PR # — put the now-review-ready fix into + the team's p/0 priority queue so it is triaged, not lost in the draft backlog. (If + the PR somehow already carries `p/0`, this is a harmless no-op.) +- Record `marked-ready PR # (target green on ALL platforms on , + labeled p/0)` and stop. + #### Step 3.5.R — Maintainer change-request response (Track C) Reached from Step 3.5 gate 0. Goal: when an **eligible human reviewer** (Hard-Rule @@ -1383,7 +1669,19 @@ Filed by [`ci-status-fix`](https://github.com/dotnet/maui/blob/main/.github/work ### Step 8 — Per-issue tally + end-of-run summary -Per issue, append one outcome line to `/tmp/gh-aw/agent/coverage.txt`: +Per issue, append **one** outcome line to `/tmp/gh-aw/agent/coverage.txt` — the +terminal outcome for the cycle. When one cycle produces a chained pair — a PR gets a +same-cycle precursor line and **then** a Step 3.6 readiness line (`marked-ready` on the +draft→ready flip, or `already-ready` on a later steady-state sweep of an already-ready +PR) — record ONLY the terminal Step 3.6 readiness line: it supersedes ANY same-cycle +non-readiness precursor for that PR, so an aggregator keying on one-line-per-issue never +double-counts. The precursor may be a Step 3 green line (`surfaced-green` on the first ✅, +or `already-surfaced` when it already exists) OR — when an unrelated-flake red and a +target-green coincide — a Step 4 `annotated-flake` line; either way the readiness line is +terminal, and the superseded precursor's signal survives in the PR's 🎯/♻️ comment so no +information is lost. This covers the flip cycle (`surfaced-green` → `marked-ready`), the +post-flip steady state (`already-surfaced` → `already-ready`), and the flake-coincident +flip (`annotated-flake` → `marked-ready`): ``` # main attempt- @@ -1394,8 +1692,14 @@ Per issue, append one outcome line to `/tmp/gh-aw/agent/coverage.txt`: `advance-PR # attempt /10` (ADVANCE mode — new commit pushed to the existing PR), `surfaced-green PR #` (fix's own CI went green; commented for review, did not advance), `annotated-flake PR #` (red was unrelated flake; -commented, attempt NOT burned), `waiting PR #` (CI not settled yet), -`dry-run: would-`, `skipped: `. (The +commented, attempt NOT burned), `marked-ready PR #` (the specific fixed test +was confirmed green in the PR's own CI; draft flipped to ready for review), +`already-surfaced PR #` (the fix's ✅ green comment already existed this sweep; +re-post suppressed — superseded by the Step 3.6 readiness line when the PR also flips +ready that cycle), `already-ready PR #` (terminal steady state — the PR was flipped +ready on a prior cycle and remains green on an unchanged head; no action taken, +supersedes `already-surfaced`), `waiting PR #` (CI not settled yet), +`dry-run: would-`, `skipped: `. (The `needs-human-PR` outcome is reserved for the deferred hand-off — Step 6 currently records a skip instead, so it is not emitted.)
/merge`, @@ -733,18 +829,26 @@ Run these gates in order — the FIRST that fires decides this cycle's outcome: method and Step 4.7 flake buckets to `C.failedLegs`. - **Unrelated flake only** (every failed leg is known-flaky / infra / a pre-existing baseline red NOT introduced by the fix): do NOT burn an attempt. - **Idempotency first:** scan the PR's existing comments for a prior bot - `♻️ … unrelated flake … on ` note for THIS head SHA — if one already - exists, record `already-annotated-flake PR # (head )` and stop - WITHOUT re-commenting (re-annotating the same flake every 12h sweep is noise and - burns the per-run comment budget other PRs need). Otherwise resolve the attempt - number from `C.effectiveAttempt` (the authoritative max(marker, bot-commit) - counter), omitting the `Attempt /10:` prefix only if it is somehow - indeterminate. **Dry-run gate (Step 0):** if `dry_run == "true"`, do NOT emit any comment — instead print the intended `♻️` unrelated-flake body to the run log, tally `dry-run: would-annotate-flake PR #`, and stop. Otherwise `add_comment` on PR #: `♻️ Attempt /10: red is - unrelated flake on leg(s) () on ; the fix itself is not - implicated. A maintainer re-run (/azp run ) should clear it.` Record - `annotated-flake PR # (head )` and stop. *(Round 1: human re-runs; - Round 2: auto re-trigger.)* + **Comment idempotency + dry-run suppress the ♻️ comment ONLY — neither skips Step + 3.6.** Scan the PR's existing comments for a prior bot `♻️ … unrelated flake … on + ` note for THIS head SHA; if one exists, set `SKIP_FLAKE_COMMENT = true`. + Resolve the attempt number from `C.effectiveAttempt` (the authoritative max(marker, + bot-commit) counter), omitting the `Attempt /10:` prefix only if it is + somehow indeterminate. Then post the flake note UNLESS suppressed: + - if `dry_run == "true"`: do NOT emit — print the intended `♻️` unrelated-flake body + to the run log and tally `dry-run: would-annotate-flake PR #`; + - else if `SKIP_FLAKE_COMMENT`: do NOT re-post — record `already-annotated-flake PR + # (head )` (re-annotating the same flake every 12h sweep is noise and + burns the per-run comment budget other PRs need); + - else `add_comment` on PR #: `♻️ Attempt /10: red is unrelated flake on + leg(s) () on ; the fix itself is not implicated. A + maintainer re-run (/azp run ) should clear it.` record `annotated-flake + PR # (head )`. + Then — in ALL of the above cases — run **Step 3.6** (target-test readiness gate) for + this PR before stopping: its `/azp`-gated target legs may conclude green on this SAME + head SHA on a LATER sweep, and Step 3.6 is the ONLY place the draft→ready flip + happens, so it MUST re-evaluate every sweep — never `stop` here before it. *(Round 1: + human re-runs; Round 2: auto re-trigger.)* - **Caused by the fix** (a failed leg still matches the original target signature, or the fix introduced a NEW failure): advance an attempt. - **Attempt count.** `attempt = C.effectiveAttempt` — the authoritative @@ -762,6 +866,188 @@ Run these gates in order — the FIRST that fires decides this cycle's outcome: from every prior commit on this branch**, emitted via the Step 5.6 ADVANCE path (push onto the same PR). +#### Step 3.6 — Target-test verification & mark-ready gate + +Reached from the Step 3 **green-surface** branch, the Step 4 **unrelated-flake** branch +(AFTER that branch has posted its comment), and the Step 3.5 **gate-2 target-focused +fast-path** (2a — while unrelated legs are still pending). Purpose: confirm the SPECIFIC +test(s) this PR was opened to fix now PASS in the PR's own CI, and — only when they do +— transition the draft `[ci-fix-net11]` PR to **ready for review** so a maintainer sees +a validated fix instead of a draft. This is the ONLY place the loop flips draft→ready; +it is a state transition, **never** an approval or a merge (a human still reviews and +merges). Overall red on *unrelated* legs must NOT gate readiness — we validate the fix, +not the base branch's flakiness. + +**Preconditions** (ALL must hold; otherwise record `skipped: readiness N/A PR # +()` and stop this gate): +- `C.isDraft == true` — if the PR is already ready-for-review, the draft→ready + transition is already done; do NOT re-mark **and do NOT re-apply `p/0`**. `p/0` is applied + exactly ONCE, atomically with the draft→ready flip (T3 — all-or-nothing with the 🎯 audit + comment + mark-ready), so an already-ready loop PR that is MISSING `p/0` has almost + certainly had it **removed by a maintainer** de-prioritizing the PR — a pure triage action + the human-engagement guard (which inspects comments/reviews/commits, NOT label changes) + cannot see. Re-adding `p/0` every sweep would fight that maintainer indefinitely, violating + the loop's "never override a human" contract. So the loop does NOT reconcile the label: + record `already-ready PR #` and stop this gate. (Trade-off: on the rare occasion a + transient API error drops `p/0` at flip time *after* the T3 atomic pre-check passed, it is + not auto-re-added — but the PR is still ready-for-review with its 🎯 audit comment, and + re-adding `p/0` is a trivial manual action; that is strictly preferable to steam-rolling a + maintainer's deliberate de-prioritization.) +- The PR is unmistakably THIS workflow's own: `[ci-fix-net11] ` title prefix AND the + `agentic-workflows` label (mirrors the safe-output lock; the compensating scope + control if the v0.80.9 compiler drops the declarative required-*). +- You reached this gate from Step 3 (green), Step 4 (**unrelated-flake**), or the Step 3.5 + gate-2 **target-focused fast-path** (2a) — i.e. the fix is NOT implicated in any red that + has concluded. If Step 4 classified a red as **caused by the fix** (ADVANCE), do NOT run + this gate — advance the attempt instead. + +**T1 — Identify the target test(s).** From the `[ci-scan-net11]` issue signature the fix +addresses (and the PR's own diff), extract the fully-qualified test method name(s) the +fix targets — e.g. `SafeAreaShouldWorkOnAllShellTabs`. For a de-flake it is the +de-flaked test; for a product fix it is the originally-failing test(s). If NO specific +test can be identified (e.g. a product build-break rather than a test failure), this is a +**build-only fix** — there is no single test to validate cross-platform, so validate at +**whole-build** granularity rather than stopping (Step 3 only *comments*; Step 3.6 is the +sole draft→ready flip, so a build-only fix is undrafted HERE or not at all). We only reach +this gate from Step 3 (green) / Step 4 (unrelated-flake) / gate-2a, so the fix is not +implicated in any concluded red; additionally require, via the T2 timeline method on +`C.headSha`, that the primary build pipeline `maui-pr` (def 302) has CONCLUDED with EVERY +one of its platform build legs `succeeded`/`completed` and NONE failed/canceled or still +pending — the build is green on **every** platform, not just the originally-broken one (the +cross-platform guard, applied to the build instead of a test). A build-only fix is undrafted +ONLY on the strength of the auto-running `maui-pr` (def 302) whole-build green, which the +loop CAN observe: if the PR's `[ci-scan]` issue signature or its own diff indicates the +ORIGINATING failure was in a `/azp`-gated pipeline (`maui-pr-uitests` def 313 or +`maui-pr-devicetests` def 314) rather than `maui-pr` (def 302), do NOT undraft on +`maui-pr`-green alone — those pipelines do not auto-run on this PR (GITHUB_TOKEN cannot +trigger them), so a green `maui-pr` build is NOT evidence the gated build break is fixed; +record `skipped: build-only fix PR # targets gated pipeline () not run — +deferring to human` and stop this gate. Otherwise, set `TARGET := "the maui-pr build +(build-only fix — no single target test)"` and proceed to **T3** to mark ready. If any `maui-pr` build leg is still unconcluded, record `skipped: build-only fix PR +# not yet whole-build green (leg(s) pending)` and stop this gate WITHOUT marking +ready (a green subset is not enough — a still-pending build leg could yet fail). + +**T2 — Verify the target test's platform legs are green (anonymous, leg-level).** Per-test +outcomes come from the AzDO **test-results** API (`_apis/test/...`), which is **NOT reachable +anonymously — Hard-Rule 8**: those endpoints 302-redirect to a sign-in page on this runner, so +a `curl` returns an HTML redirect (not JSON) and every target test would look "not executed". +Validate instead at **leg granularity** using ONLY the anonymous `_apis/build/...` timeline +(Hard-Rule 8) — a `succeeded` category leg means every test in that category **that actually +ran** passed on that platform. This is a strong bar **only if the target test genuinely +executes**: a job can go green while one specific test is skipped at runtime (`[Ignore]`, +`Assert.Ignore()`, `Assert.Inconclusive()`, a `Skip=`/conditional `[Fact]`, an `#if`-out, or +an early `return` before the asserts). So before trusting a green leg as proof the target +passed, **confirm from the PR's OWN diff that the fix does NOT skip, ignore, disable, or +short-circuit the target test** (it adds no `[Ignore]`/`[Explicit]`, `Assert.Ignore`/ +`Assert.Inconclusive`, `Skip=`, category-exclusion, `#if`-out, or early `return` around the +target). If the fix could cause the target to be runtime-skipped rather than genuinely pass, +leg-level green is NOT sufficient evidence — record `skipped: target test may be +runtime-skipped by this fix (diff adds ); leg-green insufficient, deferring to human +on PR #` and stop this gate WITHOUT marking ready. Otherwise (the target genuinely runs +and asserts, as a de-flake or product fix does) a green category leg cannot hide a +target-test failure, so treat it as authoritative. + +Map the target test to the CI leg(s) that run it. A leg name encodes platform + test category +(e.g. `Android UITests SafeAreaEdges,Shadow`, `iOS UITests SafeAreaEdges,Shadow`, `macOS +UITests SafeAreaEdges,Shadow`, `Windows UITests SafeAreaEdges,Shadow`). Determine the target +test's UI-test category — its `[Category(UITestCategories.X)]` in the test/HostApp source, +visible in the PR diff or the test file — to know which leg-name substring identifies its legs; +for a device test, the per-platform device-test legs. + +Using the SAME build-discovery as Step 4 (filter AzDO builds by `branchName=refs/pull//merge` +or `sourceVersion == C.headSha`), read each build's **timeline** on `C.headSha` for the +pipeline(s) that RUN the target test — `maui-pr` (def 302) for unit/integration tests, +`maui-pr-uitests` (def 313) for Appium UI tests, `maui-pr-devicetests` (def 314) for device +tests: + +```bash +ORG=dnceng-public; PROJ=public; BUILD= +# Anonymous + Hard-Rule-8-compliant. NEVER call _apis/test/... (it 302-redirects to sign-in). +# Each type=="Job" record is one platform leg: result (succeeded/failed/canceled/null), +# state (completed/inProgress/pending), name (encodes " UITests "): +curl -s "https://dev.azure.com/$ORG/$PROJ/_apis/build/builds/$BUILD/timeline?api-version=7.1" \ + | tee /tmp/gh-aw/agent/timeline_${BUILD}.json \ + | jq -r '.records[] | select(.type=="Job") | "\(.result)\t\(.state)\t\(.name)"' +``` + +(The prefetch already exposes per-leg status in `C.failedLegs` / the checks context, and +`gh pr checks ` lists the same per-leg rows with platform+category in the name — use +whichever is handy; the build timeline is the authoritative cross-check on the exact build.) + +Treat a target test as **VALIDATED-GREEN** only if, on `C.headSha`, the leg that runs its +category is green on **every platform it runs on** — a fix that repairs one platform must not +silently regress the same test's leg on another, so a green leg on the originally-red platform +alone is NOT enough. Across the target pipeline's platform legs (for a UI test: the Android, +iOS, Windows, and macOS/MacCatalyst legs running the target's category; for a device test: each +device-test platform), require ALL of: +- the target's category leg is `result == "succeeded"` **and** `state == "completed"` on + **every** platform that runs it, AND +- that leg is `failed` / `canceled` / aborted on **NO** platform, AND +- **no platform is left unverified.** For each platform family the target pipeline covers, that + platform's category leg must have CONCLUDED (`state == "completed"`) on `C.headSha`. If a + platform simply has no leg for the target's category, the test does not run there — that is + fine, not a gap. But if a platform's category leg has **not concluded** (`state` is + `inProgress` / pending, or an `/azp`-gated `maui-pr-uitests` / `maui-pr-devicetests` leg that + has not been kicked), the test's status on that platform is UNKNOWN → the fix is NOT yet + validated across platforms: record `skipped: target test green on but + not yet verified on (leg(s) pending / need /azp run) on PR #` + and stop this gate WITHOUT marking ready. + +A target test whose category leg never concluded on ANY platform (**not executed** anywhere — +e.g. its `/azp`-gated pipeline has not been kicked) is likewise NOT validated: record `skipped: +target test not yet executed on PR # ( not run — needs /azp run)` and stop +this gate WITHOUT marking ready. Do NOT overclaim — a green *sibling* leg (a different category +on the same platform) is not the target's leg, and a green leg on one platform is not a pass on +the others. + +**T3 — Mark ready + report.** If EVERY target test is VALIDATED-GREEN. The 🎯 comment, +the mark-ready, and the `p/0` label are THREE SEPARATE safe-outputs — the comment +existing does NOT prove the mark-ready took effect, so they are tracked independently: + +- **Comment idempotency (dup-suppress only — never gates the mark-ready).** We only reach + T3 while `C.isDraft == true` (the precondition stops an already-ready PR before here). So + if a prior bot comment for THIS head that carries the leading `🎯` anchor AND the + contiguous phrase `validated green on ` already exists (BOTH T3 variants — + `🎯 Target test validated green on ` and the build-only `🎯 Build validated green on + ` — contain that exact contiguous phrase, so this recognizes both; keying on + `target test` alone would instead miss the build-only comment and re-post it every sweep. + **Require the `🎯` anchor** so the match can NEVER be loosened to a gapped + `validated…green…on` form that would false-positive on the Step 3 `✅ … validated — the + fix's CI is green on ` surface comment and wrongly suppress the audit 🎯), the comment + landed on an earlier sweep but the **mark-ready did NOT take + effect** (the PR is still a draft) — set `SUPPRESS_COMMENT = true` (do not re-post the + duplicate 🎯 comment) but STILL complete the mark-ready + label below. Never treat the + comment's existence as "already marked ready" while the PR is still a draft. Record this + case as `re-marking PR # (🎯 present; mark-ready did not take on a prior sweep)`. +- **Atomicity budget pre-check (Hard-Rule 7).** Determine the outputs this gate will emit: + `mark_pull_request_as_ready_for_review` + `add_labels` always, plus `add_comment` unless + `SUPPRESS_COMMENT`. Verify a free per-run slot remains in EACH bucket you are about to + use. If ANY required bucket is exhausted, emit NONE of them — record `skipped: per-run + cap reached; deferring mark-ready PR # to next cycle` and stop this gate. Never mark a + PR ready (or label it) unless its 🎯 audit comment is GUARANTEED to exist for this exact + head SHA — either posted in this same sweep, or (in the `SUPPRESS_COMMENT` re-marking case) + already present from a prior sweep. The outputs you DO emit either all land or all defer + together; the audit trail must never be absent, but it is NOT re-posted when it already exists. +- **Dry-run gate (Step 0):** if `dry_run == "true"`, emit NOTHING — print the intended + readiness comment and "would mark ready + add p/0" to the run log, tally `dry-run: + would-mark-ready PR #`, and stop. +- Otherwise emit for THIS PR number ``: + 1. `add_comment` (ONLY if not `SUPPRESS_COMMENT`): `🎯 Target test validated green on + — passed on ALL platforms it runs on (, + buildId ). — not + caused by this fix."> Transitioning this PR from draft to ready for review and adding + `p/0`; a maintainer still reviews and merges.` (For a **build-only fix** — the T1 + whole-build fallback, no single target test — phrase the first clause as `🎯 Build + validated green on — the maui-pr build passed on ALL platforms (buildId + ).` instead of naming a test.) + 2. `mark_pull_request_as_ready_for_review` with `reason:` a one-line justification + naming the validated test(s) and ``. + 3. `add_labels` with `labels: ["p/0"]` for PR # — put the now-review-ready fix into + the team's p/0 priority queue so it is triaged, not lost in the draft backlog. (If + the PR somehow already carries `p/0`, this is a harmless no-op.) +- Record `marked-ready PR # (target green on ALL platforms on , + labeled p/0)` and stop. + #### Step 3.5.R — Maintainer change-request response (Track C) Reached from Step 3.5 gate 0. Goal: when an **eligible human reviewer** (Hard-Rule @@ -1395,7 +1681,19 @@ Filed by [`ci-status-fix-net11`](https://github.com/dotnet/maui/blob/main/.githu ### Step 8 — Per-issue tally + end-of-run summary -Per issue, append one outcome line to `/tmp/gh-aw/agent/coverage.txt`: +Per issue, append **one** outcome line to `/tmp/gh-aw/agent/coverage.txt` — the +terminal outcome for the cycle. When one cycle produces a chained pair — a PR gets a +same-cycle precursor line and **then** a Step 3.6 readiness line (`marked-ready` on the +draft→ready flip, or `already-ready` on a later steady-state sweep of an already-ready +PR) — record ONLY the terminal Step 3.6 readiness line: it supersedes ANY same-cycle +non-readiness precursor for that PR, so an aggregator keying on one-line-per-issue never +double-counts. The precursor may be a Step 3 green line (`surfaced-green` on the first ✅, +or `already-surfaced` when it already exists) OR — when an unrelated-flake red and a +target-green coincide — a Step 4 `annotated-flake` line; either way the readiness line is +terminal, and the superseded precursor's signal survives in the PR's 🎯/♻️ comment so no +information is lost. This covers the flip cycle (`surfaced-green` → `marked-ready`), the +post-flip steady state (`already-surfaced` → `already-ready`), and the flake-coincident +flip (`annotated-flake` → `marked-ready`): ``` # net11.0 attempt- @@ -1406,8 +1704,14 @@ Per issue, append one outcome line to `/tmp/gh-aw/agent/coverage.txt`: `advance-PR # attempt /10` (ADVANCE mode — new commit pushed to the existing PR), `surfaced-green PR #` (fix's own CI went green; commented for review, did not advance), `annotated-flake PR #` (red was unrelated flake; -commented, attempt NOT burned), `waiting PR #` (CI not settled yet), -`dry-run: would-`, `skipped: `. (The +commented, attempt NOT burned), `marked-ready PR #` (the specific fixed test +was confirmed green in the PR's own CI; draft flipped to ready for review), +`already-surfaced PR #` (the fix's ✅ green comment already existed this sweep; +re-post suppressed — superseded by the Step 3.6 readiness line when the PR also flips +ready that cycle), `already-ready PR #` (terminal steady state — the PR was flipped +ready on a prior cycle and remains green on an unchanged head; no action taken, +supersedes `already-surfaced`), `waiting PR #` (CI not settled yet), +`dry-run: would-`, `skipped: `. (The `needs-human-PR` outcome is reserved for the deferred hand-off — Step 6 currently records a skip instead, so it is not emitted.) diff --git a/.github/workflows/ci-status-fix.lock.yml b/.github/workflows/ci-status-fix.lock.yml index 93dc5969cb0d..65511f2b3259 100644 --- a/.github/workflows/ci-status-fix.lock.yml +++ b/.github/workflows/ci-status-fix.lock.yml @@ -1,4 +1,4 @@ -# gh-aw-metadata: {"schema_version":"v4","frontmatter_hash":"f014161967589ebcaf1ac5ec6f3c88ee5a4388d28b55a07dc36c45b7b2aebab6","body_hash":"86c6b2d4c3df13da14c6a3c8b211381fcc9f97bb1951ff7b80588a2c5338e00c","compiler_version":"v0.80.9","strict":true,"agent_id":"copilot","agent_model":"claude-opus-4.8","engine_versions":{"copilot":"1.0.63"}} +# gh-aw-metadata: {"schema_version":"v4","frontmatter_hash":"b0ebf8a2f179900cfe3638e994b27a43cec52bdbdd2331dc1bd4d65762fa2d6b","body_hash":"1825e2b1f7c7bd88241bf37580655489360f9bebc806edd00837a52ce4ad83a0","compiler_version":"v0.80.9","strict":true,"agent_id":"copilot","agent_model":"claude-opus-4.8","engine_versions":{"copilot":"1.0.63"}} # gh-aw-manifest: {"version":1,"secrets":["COPILOT_PAT_0","COPILOT_PAT_1","COPILOT_PAT_2","COPILOT_PAT_3","COPILOT_PAT_4","COPILOT_PAT_5","COPILOT_PAT_6","COPILOT_PAT_7","COPILOT_PAT_8","COPILOT_PAT_9","GH_AW_CI_TRIGGER_TOKEN","GH_AW_GITHUB_MCP_SERVER_TOKEN","GH_AW_GITHUB_TOKEN","GITHUB_TOKEN"],"actions":[{"repo":"actions/checkout","sha":"34e114876b0b11c390a56381ad16ebd13914f8d5","version":"v4"},{"repo":"actions/checkout","sha":"9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0","version":"v7.0.0"},{"repo":"actions/checkout","sha":"de0fac2e4500dabe0009e67214ff5f5447ce83dd","version":"v6.0.2"},{"repo":"actions/download-artifact","sha":"3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c","version":"v8.0.1"},{"repo":"actions/github-script","sha":"3a2844b7e9c422d3c10d287c895573f7108da1b3","version":"v9.0.0"},{"repo":"actions/setup-node","sha":"48b55a011bda9f5d6aeb4c2d9c7362e8dae4041e","version":"v6.4.0"},{"repo":"actions/upload-artifact","sha":"043fb46d1a93c77aae656e7c1c64a875d1fc6a0a","version":"v7.0.1"},{"repo":"github/gh-aw-actions/setup","sha":"8c7d04ebf1ece56cd381446125da3e0f6896294a","version":"v0.80.9"}],"containers":[{"image":"ghcr.io/github/gh-aw-firewall/agent:0.27.7","digest":"sha256:aae231e4635c8999d039c132f1602d3df850fe9b84a00aa2b5ac981179b5661c","pinned_image":"ghcr.io/github/gh-aw-firewall/agent:0.27.7@sha256:aae231e4635c8999d039c132f1602d3df850fe9b84a00aa2b5ac981179b5661c"},{"image":"ghcr.io/github/gh-aw-firewall/api-proxy:0.27.7","digest":"sha256:009caf2e3d88fa77b64e9a03a95a228fc58db0f1701c6d324b29ba5a3c7c79b6","pinned_image":"ghcr.io/github/gh-aw-firewall/api-proxy:0.27.7@sha256:009caf2e3d88fa77b64e9a03a95a228fc58db0f1701c6d324b29ba5a3c7c79b6"},{"image":"ghcr.io/github/gh-aw-firewall/squid:0.27.7","digest":"sha256:deb1d4e19de62d51cee0508057a596a19315c3423ada4d675cad136dc8037c96","pinned_image":"ghcr.io/github/gh-aw-firewall/squid:0.27.7@sha256:deb1d4e19de62d51cee0508057a596a19315c3423ada4d675cad136dc8037c96"},{"image":"ghcr.io/github/gh-aw-mcpg:v0.3.27","digest":"sha256:fe984bddde4ec05d756d9043edb0a32912e6b7b72f6a121b1082f29221421cc7","pinned_image":"ghcr.io/github/gh-aw-mcpg:v0.3.27@sha256:fe984bddde4ec05d756d9043edb0a32912e6b7b72f6a121b1082f29221421cc7"},{"image":"ghcr.io/github/gh-aw-node","digest":"sha256:529d02eb970b1161aa25c593a9c3df57fdfad5a8add328cb3b6eccef66f3183b","pinned_image":"ghcr.io/github/gh-aw-node@sha256:529d02eb970b1161aa25c593a9c3df57fdfad5a8add328cb3b6eccef66f3183b"},{"image":"ghcr.io/github/github-mcp-server:v1.4.0","digest":"sha256:2afb26356481d1a350e14544a6e160f7f7ec1561a1ea309b823665abf0309036","pinned_image":"ghcr.io/github/github-mcp-server:v1.4.0@sha256:2afb26356481d1a350e14544a6e160f7f7ec1561a1ea309b823665abf0309036"}]} # This file was automatically generated by gh-aw (v0.80.9). DO NOT EDIT. To debug this workflow, load the skill at https://github.com/github/gh-aw/blob/main/debug.md # @@ -37,7 +37,9 @@ # humans (the open tracking issue is the hand-off surface; a dedicated # [ci-fix][needs-human] PR is planned but currently deferred — see Step 6). CI on # the PR is kicked by a human `/azp run` for now; the loop watches, classifies, -# and re-fixes autonomously between kicks. +# and re-fixes autonomously between kicks. When the SPECIFIC test a PR fixed is +# confirmed green in that PR's own CI, the loop marks the draft PR ready for review +# (a state transition only — it never approves or merges). # Never mutes tests, but # de-flakes genuinely flaky ones (deterministic synchronization, no retries / # timeout bumps). Always skips visual-regression / screenshot issues. @@ -273,24 +275,24 @@ jobs: run: | bash "${RUNNER_TEMP}/gh-aw/actions/create_prompt_first.sh" { - cat << 'GH_AW_PROMPT_25cf75111b670167_EOF' + cat << 'GH_AW_PROMPT_2ac1d0b4083428f8_EOF' - GH_AW_PROMPT_25cf75111b670167_EOF + GH_AW_PROMPT_2ac1d0b4083428f8_EOF cat "${RUNNER_TEMP}/gh-aw/prompts/xpia.md" cat "${RUNNER_TEMP}/gh-aw/prompts/temp_folder_prompt.md" cat "${RUNNER_TEMP}/gh-aw/prompts/markdown.md" cat "${RUNNER_TEMP}/gh-aw/prompts/safe_outputs_prompt.md" - cat << 'GH_AW_PROMPT_25cf75111b670167_EOF' + cat << 'GH_AW_PROMPT_2ac1d0b4083428f8_EOF' - Tools: add_comment(max:3), create_pull_request(max:3), update_pull_request(max:3), push_to_pull_request_branch(max:3), missing_tool, missing_data, noop - GH_AW_PROMPT_25cf75111b670167_EOF + Tools: add_comment(max:6), create_pull_request(max:3), update_pull_request(max:3), mark_pull_request_as_ready_for_review(max:3), add_labels(max:3), push_to_pull_request_branch(max:3), missing_tool, missing_data, noop + GH_AW_PROMPT_2ac1d0b4083428f8_EOF cat "${RUNNER_TEMP}/gh-aw/prompts/safe_outputs_create_pull_request.md" cat "${RUNNER_TEMP}/gh-aw/prompts/safe_outputs_push_to_pr_branch.md" - cat << 'GH_AW_PROMPT_25cf75111b670167_EOF' + cat << 'GH_AW_PROMPT_2ac1d0b4083428f8_EOF' - GH_AW_PROMPT_25cf75111b670167_EOF + GH_AW_PROMPT_2ac1d0b4083428f8_EOF cat "${RUNNER_TEMP}/gh-aw/prompts/mcp_cli_tools_prompt.md" - cat << 'GH_AW_PROMPT_25cf75111b670167_EOF' + cat << 'GH_AW_PROMPT_2ac1d0b4083428f8_EOF' The following GitHub context information is available for this workflow: {{#if github.actor}} @@ -332,12 +334,12 @@ jobs: stop immediately and report the limitation rather than spending turns trying to work around it. - GH_AW_PROMPT_25cf75111b670167_EOF + GH_AW_PROMPT_2ac1d0b4083428f8_EOF cat "${RUNNER_TEMP}/gh-aw/prompts/github_mcp_tools_with_safeoutputs_prompt.md" - cat << 'GH_AW_PROMPT_25cf75111b670167_EOF' + cat << 'GH_AW_PROMPT_2ac1d0b4083428f8_EOF' {{#runtime-import .github/workflows/ci-status-fix.md}} - GH_AW_PROMPT_25cf75111b670167_EOF + GH_AW_PROMPT_2ac1d0b4083428f8_EOF } > "$GH_AW_PROMPT" - name: Interpolate variables and render templates uses: actions/github-script@3a2844b7e9c422d3c10d287c895573f7108da1b3 # v9.0.0 @@ -554,16 +556,18 @@ jobs: mkdir -p "${RUNNER_TEMP}/gh-aw/safeoutputs" mkdir -p /tmp/gh-aw/safeoutputs mkdir -p /tmp/gh-aw/mcp-logs/safeoutputs - cat > "${RUNNER_TEMP}/gh-aw/safeoutputs/config.json" << 'GH_AW_SAFE_OUTPUTS_CONFIG_fa2a69fad5fd0485_EOF' - {"add_comment":{"discussions":false,"max":3,"required_labels":["agentic-workflows"],"required_title_prefix":"[ci-fix] ","target":"*"},"create_pull_request":{"allowed_base_branches":["main"],"allowed_branches":["ci-fix/**"],"allowed_files":["src/Core/**","src/Controls/**","src/Essentials/**","src/BlazorWebView/**","src/TestUtils/**","src/Templates/**","**/PublicAPI.Unshipped.txt"],"base_branch":"main","draft":true,"labels":["agentic-workflows"],"max":3,"max_patch_files":100,"max_patch_size":4096,"protect_top_level_dot_folders":true,"protected_files":["package.json","bun.lockb","bunfig.toml","deno.json","deno.jsonc","deno.lock","global.json","NuGet.Config","Directory.Packages.props","mix.exs","mix.lock","go.mod","go.sum","stack.yaml","stack.yaml.lock","pom.xml","build.gradle","build.gradle.kts","settings.gradle","settings.gradle.kts","gradle.properties","package-lock.json","yarn.lock","pnpm-lock.yaml","npm-shrinkwrap.json","requirements.txt","Pipfile","Pipfile.lock","pyproject.toml","setup.py","setup.cfg","Gemfile","Gemfile.lock","uv.lock","CODEOWNERS","DESIGN.md","README.md","CONTRIBUTING.md","CHANGELOG.md","SECURITY.md","CODE_OF_CONDUCT.md","AGENTS.md","CLAUDE.md","GEMINI.md"],"protected_files_policy":"request_review","title_prefix":"[ci-fix] "},"create_report_incomplete_issue":{},"missing_data":{},"missing_tool":{},"noop":{"max":1,"report-as-issue":"true"},"push_to_pull_request_branch":{"allowed_files":["src/Core/**","src/Controls/**","src/Essentials/**","src/BlazorWebView/**","src/TestUtils/**","src/Templates/**","**/PublicAPI.Unshipped.txt"],"if_no_changes":"warn","max":3,"max_patch_size":4096,"protect_top_level_dot_folders":true,"protected_files":["package.json","bun.lockb","bunfig.toml","deno.json","deno.jsonc","deno.lock","global.json","NuGet.Config","Directory.Packages.props","mix.exs","mix.lock","go.mod","go.sum","stack.yaml","stack.yaml.lock","pom.xml","build.gradle","build.gradle.kts","settings.gradle","settings.gradle.kts","gradle.properties","package-lock.json","yarn.lock","pnpm-lock.yaml","npm-shrinkwrap.json","requirements.txt","Pipfile","Pipfile.lock","pyproject.toml","setup.py","setup.cfg","Gemfile","Gemfile.lock","uv.lock","CODEOWNERS","DESIGN.md","README.md","CONTRIBUTING.md","CHANGELOG.md","SECURITY.md","CODE_OF_CONDUCT.md","AGENTS.md","CLAUDE.md","GEMINI.md"],"required_labels":["agentic-workflows"],"target":"*","title_prefix":"[ci-fix] "},"report_incomplete":{},"update_pull_request":{"allow_body":true,"allow_title":false,"max":3,"target":"*","update_branch":false}} - GH_AW_SAFE_OUTPUTS_CONFIG_fa2a69fad5fd0485_EOF + cat > "${RUNNER_TEMP}/gh-aw/safeoutputs/config.json" << 'GH_AW_SAFE_OUTPUTS_CONFIG_0644bf1434ca0071_EOF' + {"add_comment":{"discussions":false,"max":6,"required_labels":["agentic-workflows"],"required_title_prefix":"[ci-fix] ","target":"*"},"add_labels":{"allowed":["p/0"],"max":3,"required_labels":["agentic-workflows"],"required_title_prefix":"[ci-fix] ","target":"*"},"create_pull_request":{"allowed_base_branches":["main"],"allowed_branches":["ci-fix/**"],"allowed_files":["src/Core/**","src/Controls/**","src/Essentials/**","src/BlazorWebView/**","src/TestUtils/**","src/Templates/**","**/PublicAPI.Unshipped.txt"],"base_branch":"main","draft":true,"labels":["agentic-workflows"],"max":3,"max_patch_files":100,"max_patch_size":4096,"protect_top_level_dot_folders":true,"protected_files":["package.json","bun.lockb","bunfig.toml","deno.json","deno.jsonc","deno.lock","global.json","NuGet.Config","Directory.Packages.props","mix.exs","mix.lock","go.mod","go.sum","stack.yaml","stack.yaml.lock","pom.xml","build.gradle","build.gradle.kts","settings.gradle","settings.gradle.kts","gradle.properties","package-lock.json","yarn.lock","pnpm-lock.yaml","npm-shrinkwrap.json","requirements.txt","Pipfile","Pipfile.lock","pyproject.toml","setup.py","setup.cfg","Gemfile","Gemfile.lock","uv.lock","CODEOWNERS","DESIGN.md","README.md","CONTRIBUTING.md","CHANGELOG.md","SECURITY.md","CODE_OF_CONDUCT.md","AGENTS.md","CLAUDE.md","GEMINI.md"],"protected_files_policy":"request_review","title_prefix":"[ci-fix] "},"create_report_incomplete_issue":{},"mark_pull_request_as_ready_for_review":{"max":3,"required_labels":["agentic-workflows"],"required_title_prefix":"[ci-fix] ","target":"*"},"missing_data":{},"missing_tool":{},"noop":{"max":1,"report-as-issue":"true"},"push_to_pull_request_branch":{"allowed_files":["src/Core/**","src/Controls/**","src/Essentials/**","src/BlazorWebView/**","src/TestUtils/**","src/Templates/**","**/PublicAPI.Unshipped.txt"],"if_no_changes":"warn","max":3,"max_patch_size":4096,"protect_top_level_dot_folders":true,"protected_files":["package.json","bun.lockb","bunfig.toml","deno.json","deno.jsonc","deno.lock","global.json","NuGet.Config","Directory.Packages.props","mix.exs","mix.lock","go.mod","go.sum","stack.yaml","stack.yaml.lock","pom.xml","build.gradle","build.gradle.kts","settings.gradle","settings.gradle.kts","gradle.properties","package-lock.json","yarn.lock","pnpm-lock.yaml","npm-shrinkwrap.json","requirements.txt","Pipfile","Pipfile.lock","pyproject.toml","setup.py","setup.cfg","Gemfile","Gemfile.lock","uv.lock","CODEOWNERS","DESIGN.md","README.md","CONTRIBUTING.md","CHANGELOG.md","SECURITY.md","CODE_OF_CONDUCT.md","AGENTS.md","CLAUDE.md","GEMINI.md"],"required_labels":["agentic-workflows"],"target":"*","title_prefix":"[ci-fix] "},"report_incomplete":{},"update_pull_request":{"allow_body":true,"allow_title":false,"max":3,"target":"*","update_branch":false}} + GH_AW_SAFE_OUTPUTS_CONFIG_0644bf1434ca0071_EOF - name: Generate Safe Outputs Tools env: GH_AW_TOOLS_META_JSON: | { "description_suffixes": { - "add_comment": " CONSTRAINTS: Maximum 3 comment(s) can be added. Target: *. Supports reply_to_id for discussion threading.", + "add_comment": " CONSTRAINTS: Maximum 6 comment(s) can be added. Target: *. Supports reply_to_id for discussion threading.", + "add_labels": " CONSTRAINTS: Maximum 3 label(s) can be added. Only these labels are allowed: [\"p/0\"]. Target: *.", "create_pull_request": " CONSTRAINTS: Maximum 3 pull request(s) can be created. Title will be prefixed with \"[ci-fix] \". Labels [\"agentic-workflows\"] will be automatically added. Only these labels are allowed: [\"agentic-workflows\"]. PRs will be created as drafts.", + "mark_pull_request_as_ready_for_review": " CONSTRAINTS: Maximum 3 pull request(s) can be marked as ready for review.", "push_to_pull_request_branch": " CONSTRAINTS: Maximum 3 push(es) can be made. The target pull request title must start with \"[ci-fix] \".", "update_pull_request": " CONSTRAINTS: Maximum 3 pull request(s) can be updated. Target: *." }, @@ -594,6 +598,25 @@ jobs: } } }, + "add_labels": { + "defaultMax": 5, + "fields": { + "item_number": { + "issueNumberOrTemporaryId": true + }, + "labels": { + "required": true, + "type": "array", + "itemType": "string", + "itemSanitize": true, + "itemMaxLength": 128 + }, + "repo": { + "type": "string", + "maxLength": 256 + } + } + }, "create_pull_request": { "defaultMax": 1, "fields": { @@ -635,6 +658,24 @@ jobs: } } }, + "mark_pull_request_as_ready_for_review": { + "defaultMax": 1, + "fields": { + "pull_request_number": { + "issueOrPRNumber": true + }, + "reason": { + "required": true, + "type": "string", + "sanitize": true, + "maxLength": 65000 + }, + "repo": { + "type": "string", + "maxLength": 256 + } + } + }, "missing_data": { "defaultMax": 20, "fields": { @@ -1494,7 +1535,7 @@ jobs: uses: actions/github-script@3a2844b7e9c422d3c10d287c895573f7108da1b3 # v9.0.0 env: WORKFLOW_NAME: "CI Failure Fixer (main)" - WORKFLOW_DESCRIPTION: "Periodic pass over open ci-scan tracking issues filed by the main-branch CI\nfailure scanner (.github/workflows/ci-status-main.md). This workflow targets\nthe `main` branch EXCLUSIVELY: it processes only issues labelled ci-scan and\nopens every PR against main. (The net11.0 branch is handled by the parallel\n.github/workflows/ci-status-fix-net11.md — the two are split because gh-aw can\nonly transport a fix relative to ONE static base branch per workflow, and the\nmain↔net11.0 divergence exceeds gh-aw's 10 MB transport-patch cap.) The fixer\nopens ONE draft [ci-fix] PR per actionable issue against main, then WATCHES\nthat PR's own CI on later runs: when the fix's CI comes back red and the red is\ncaused by the fix itself, it pushes a fresh follow-up fix onto the SAME PR\nbranch (never a second PR) — up to 10 attempts — then stops and defers to\nhumans (the open tracking issue is the hand-off surface; a dedicated\n[ci-fix][needs-human] PR is planned but currently deferred — see Step 6). CI on\nthe PR is kicked by a human `/azp run` for now; the loop watches, classifies,\nand re-fixes autonomously between kicks.\nNever mutes tests, but\nde-flakes genuinely flaky ones (deterministic synchronization, no retries /\ntimeout bumps). Always skips visual-regression / screenshot issues." + WORKFLOW_DESCRIPTION: "Periodic pass over open ci-scan tracking issues filed by the main-branch CI\nfailure scanner (.github/workflows/ci-status-main.md). This workflow targets\nthe `main` branch EXCLUSIVELY: it processes only issues labelled ci-scan and\nopens every PR against main. (The net11.0 branch is handled by the parallel\n.github/workflows/ci-status-fix-net11.md — the two are split because gh-aw can\nonly transport a fix relative to ONE static base branch per workflow, and the\nmain↔net11.0 divergence exceeds gh-aw's 10 MB transport-patch cap.) The fixer\nopens ONE draft [ci-fix] PR per actionable issue against main, then WATCHES\nthat PR's own CI on later runs: when the fix's CI comes back red and the red is\ncaused by the fix itself, it pushes a fresh follow-up fix onto the SAME PR\nbranch (never a second PR) — up to 10 attempts — then stops and defers to\nhumans (the open tracking issue is the hand-off surface; a dedicated\n[ci-fix][needs-human] PR is planned but currently deferred — see Step 6). CI on\nthe PR is kicked by a human `/azp run` for now; the loop watches, classifies,\nand re-fixes autonomously between kicks. When the SPECIFIC test a PR fixed is\nconfirmed green in that PR's own CI, the loop marks the draft PR ready for review\n(a state transition only — it never approves or merges).\nNever mutes tests, but\nde-flakes genuinely flaky ones (deterministic synchronization, no retries /\ntimeout bumps). Always skips visual-regression / screenshot issues." HAS_PATCH: ${{ needs.agent.outputs.has_patch }} with: script: | @@ -1910,7 +1951,7 @@ jobs: GH_AW_ALLOWED_DOMAINS: "*.blob.core.windows.net,*.githubusercontent.com,api.business.githubcopilot.com,api.enterprise.githubcopilot.com,api.github.com,api.githubcopilot.com,api.individual.githubcopilot.com,api.snapcraft.io,archive.ubuntu.com,azure.archive.ubuntu.com,codeload.github.com,crl.geotrust.com,crl.globalsign.com,crl.identrust.com,crl.sectigo.com,crl.thawte.com,crl.usertrust.com,crl.verisign.com,crl3.digicert.com,crl4.digicert.com,crls.ssl.com,dev.azure.com,docs.github.com,github-cloud.githubusercontent.com,github-cloud.s3.amazonaws.com,github.blog,github.com,github.githubassets.com,helix.dot.net,host.docker.internal,json-schema.org,json.schemastore.org,keyserver.ubuntu.com,lfs.github.com,objects.githubusercontent.com,ocsp.digicert.com,ocsp.geotrust.com,ocsp.globalsign.com,ocsp.identrust.com,ocsp.sectigo.com,ocsp.ssl.com,ocsp.thawte.com,ocsp.usertrust.com,ocsp.verisign.com,packagecloud.io,packages.cloud.google.com,packages.microsoft.com,patch-diff.githubusercontent.com,ppa.launchpad.net,raw.githubusercontent.com,registry.npmjs.org,s.symcb.com,s.symcd.com,security.ubuntu.com,telemetry.enterprise.githubcopilot.com,ts-crl.ws.symantec.com,ts-ocsp.ws.symantec.com,www.googleapis.com" GITHUB_SERVER_URL: ${{ github.server_url }} GITHUB_API_URL: ${{ github.api_url }} - GH_AW_SAFE_OUTPUTS_HANDLER_CONFIG: "{\"add_comment\":{\"discussions\":false,\"max\":3,\"required_labels\":[\"agentic-workflows\"],\"required_title_prefix\":\"[ci-fix] \",\"target\":\"*\"},\"create_pull_request\":{\"allowed_base_branches\":[\"main\"],\"allowed_branches\":[\"ci-fix/**\"],\"allowed_files\":[\"src/Core/**\",\"src/Controls/**\",\"src/Essentials/**\",\"src/BlazorWebView/**\",\"src/TestUtils/**\",\"src/Templates/**\",\"**/PublicAPI.Unshipped.txt\"],\"base_branch\":\"main\",\"draft\":true,\"labels\":[\"agentic-workflows\"],\"max\":3,\"max_patch_files\":100,\"max_patch_size\":4096,\"protect_top_level_dot_folders\":true,\"protected_files\":[\"package.json\",\"bun.lockb\",\"bunfig.toml\",\"deno.json\",\"deno.jsonc\",\"deno.lock\",\"global.json\",\"NuGet.Config\",\"Directory.Packages.props\",\"mix.exs\",\"mix.lock\",\"go.mod\",\"go.sum\",\"stack.yaml\",\"stack.yaml.lock\",\"pom.xml\",\"build.gradle\",\"build.gradle.kts\",\"settings.gradle\",\"settings.gradle.kts\",\"gradle.properties\",\"package-lock.json\",\"yarn.lock\",\"pnpm-lock.yaml\",\"npm-shrinkwrap.json\",\"requirements.txt\",\"Pipfile\",\"Pipfile.lock\",\"pyproject.toml\",\"setup.py\",\"setup.cfg\",\"Gemfile\",\"Gemfile.lock\",\"uv.lock\",\"CODEOWNERS\",\"DESIGN.md\",\"README.md\",\"CONTRIBUTING.md\",\"CHANGELOG.md\",\"SECURITY.md\",\"CODE_OF_CONDUCT.md\",\"AGENTS.md\",\"CLAUDE.md\",\"GEMINI.md\"],\"protected_files_policy\":\"request_review\",\"title_prefix\":\"[ci-fix] \"},\"create_report_incomplete_issue\":{},\"missing_data\":{},\"missing_tool\":{},\"noop\":{\"max\":1,\"report-as-issue\":\"true\"},\"push_to_pull_request_branch\":{\"allowed_files\":[\"src/Core/**\",\"src/Controls/**\",\"src/Essentials/**\",\"src/BlazorWebView/**\",\"src/TestUtils/**\",\"src/Templates/**\",\"**/PublicAPI.Unshipped.txt\"],\"if_no_changes\":\"warn\",\"max\":3,\"max_patch_size\":4096,\"protect_top_level_dot_folders\":true,\"protected_files\":[\"package.json\",\"bun.lockb\",\"bunfig.toml\",\"deno.json\",\"deno.jsonc\",\"deno.lock\",\"global.json\",\"NuGet.Config\",\"Directory.Packages.props\",\"mix.exs\",\"mix.lock\",\"go.mod\",\"go.sum\",\"stack.yaml\",\"stack.yaml.lock\",\"pom.xml\",\"build.gradle\",\"build.gradle.kts\",\"settings.gradle\",\"settings.gradle.kts\",\"gradle.properties\",\"package-lock.json\",\"yarn.lock\",\"pnpm-lock.yaml\",\"npm-shrinkwrap.json\",\"requirements.txt\",\"Pipfile\",\"Pipfile.lock\",\"pyproject.toml\",\"setup.py\",\"setup.cfg\",\"Gemfile\",\"Gemfile.lock\",\"uv.lock\",\"CODEOWNERS\",\"DESIGN.md\",\"README.md\",\"CONTRIBUTING.md\",\"CHANGELOG.md\",\"SECURITY.md\",\"CODE_OF_CONDUCT.md\",\"AGENTS.md\",\"CLAUDE.md\",\"GEMINI.md\"],\"required_labels\":[\"agentic-workflows\"],\"target\":\"*\",\"title_prefix\":\"[ci-fix] \"},\"report_incomplete\":{},\"update_pull_request\":{\"allow_body\":true,\"allow_title\":false,\"max\":3,\"target\":\"*\",\"update_branch\":false}}" + GH_AW_SAFE_OUTPUTS_HANDLER_CONFIG: "{\"add_comment\":{\"discussions\":false,\"max\":6,\"required_labels\":[\"agentic-workflows\"],\"required_title_prefix\":\"[ci-fix] \",\"target\":\"*\"},\"add_labels\":{\"allowed\":[\"p/0\"],\"max\":3,\"required_labels\":[\"agentic-workflows\"],\"required_title_prefix\":\"[ci-fix] \",\"target\":\"*\"},\"create_pull_request\":{\"allowed_base_branches\":[\"main\"],\"allowed_branches\":[\"ci-fix/**\"],\"allowed_files\":[\"src/Core/**\",\"src/Controls/**\",\"src/Essentials/**\",\"src/BlazorWebView/**\",\"src/TestUtils/**\",\"src/Templates/**\",\"**/PublicAPI.Unshipped.txt\"],\"base_branch\":\"main\",\"draft\":true,\"labels\":[\"agentic-workflows\"],\"max\":3,\"max_patch_files\":100,\"max_patch_size\":4096,\"protect_top_level_dot_folders\":true,\"protected_files\":[\"package.json\",\"bun.lockb\",\"bunfig.toml\",\"deno.json\",\"deno.jsonc\",\"deno.lock\",\"global.json\",\"NuGet.Config\",\"Directory.Packages.props\",\"mix.exs\",\"mix.lock\",\"go.mod\",\"go.sum\",\"stack.yaml\",\"stack.yaml.lock\",\"pom.xml\",\"build.gradle\",\"build.gradle.kts\",\"settings.gradle\",\"settings.gradle.kts\",\"gradle.properties\",\"package-lock.json\",\"yarn.lock\",\"pnpm-lock.yaml\",\"npm-shrinkwrap.json\",\"requirements.txt\",\"Pipfile\",\"Pipfile.lock\",\"pyproject.toml\",\"setup.py\",\"setup.cfg\",\"Gemfile\",\"Gemfile.lock\",\"uv.lock\",\"CODEOWNERS\",\"DESIGN.md\",\"README.md\",\"CONTRIBUTING.md\",\"CHANGELOG.md\",\"SECURITY.md\",\"CODE_OF_CONDUCT.md\",\"AGENTS.md\",\"CLAUDE.md\",\"GEMINI.md\"],\"protected_files_policy\":\"request_review\",\"title_prefix\":\"[ci-fix] \"},\"create_report_incomplete_issue\":{},\"mark_pull_request_as_ready_for_review\":{\"max\":3,\"required_labels\":[\"agentic-workflows\"],\"required_title_prefix\":\"[ci-fix] \",\"target\":\"*\"},\"missing_data\":{},\"missing_tool\":{},\"noop\":{\"max\":1,\"report-as-issue\":\"true\"},\"push_to_pull_request_branch\":{\"allowed_files\":[\"src/Core/**\",\"src/Controls/**\",\"src/Essentials/**\",\"src/BlazorWebView/**\",\"src/TestUtils/**\",\"src/Templates/**\",\"**/PublicAPI.Unshipped.txt\"],\"if_no_changes\":\"warn\",\"max\":3,\"max_patch_size\":4096,\"protect_top_level_dot_folders\":true,\"protected_files\":[\"package.json\",\"bun.lockb\",\"bunfig.toml\",\"deno.json\",\"deno.jsonc\",\"deno.lock\",\"global.json\",\"NuGet.Config\",\"Directory.Packages.props\",\"mix.exs\",\"mix.lock\",\"go.mod\",\"go.sum\",\"stack.yaml\",\"stack.yaml.lock\",\"pom.xml\",\"build.gradle\",\"build.gradle.kts\",\"settings.gradle\",\"settings.gradle.kts\",\"gradle.properties\",\"package-lock.json\",\"yarn.lock\",\"pnpm-lock.yaml\",\"npm-shrinkwrap.json\",\"requirements.txt\",\"Pipfile\",\"Pipfile.lock\",\"pyproject.toml\",\"setup.py\",\"setup.cfg\",\"Gemfile\",\"Gemfile.lock\",\"uv.lock\",\"CODEOWNERS\",\"DESIGN.md\",\"README.md\",\"CONTRIBUTING.md\",\"CHANGELOG.md\",\"SECURITY.md\",\"CODE_OF_CONDUCT.md\",\"AGENTS.md\",\"CLAUDE.md\",\"GEMINI.md\"],\"required_labels\":[\"agentic-workflows\"],\"target\":\"*\",\"title_prefix\":\"[ci-fix] \"},\"report_incomplete\":{},\"update_pull_request\":{\"allow_body\":true,\"allow_title\":false,\"max\":3,\"target\":\"*\",\"update_branch\":false}}" GH_AW_CI_TRIGGER_TOKEN: ${{ secrets.GH_AW_CI_TRIGGER_TOKEN }} with: github-token: ${{ secrets.GH_AW_GITHUB_TOKEN || secrets.GITHUB_TOKEN }} diff --git a/.github/workflows/ci-status-fix.md b/.github/workflows/ci-status-fix.md index d3227e25170c..9c2f46054029 100644 --- a/.github/workflows/ci-status-fix.md +++ b/.github/workflows/ci-status-fix.md @@ -15,7 +15,9 @@ description: | humans (the open tracking issue is the hand-off surface; a dedicated [ci-fix][needs-human] PR is planned but currently deferred — see Step 6). CI on the PR is kicked by a human `/azp run` for now; the loop watches, classifies, - and re-fixes autonomously between kicks. + and re-fixes autonomously between kicks. When the SPECIFIC test a PR fixed is + confirmed green in that PR's own CI, the loop marks the draft PR ready for review + (a state transition only — it never approves or merges). Never mutes tests, but de-flakes genuinely flaky ones (deterministic synchronization, no retries / timeout bumps). Always skips visual-regression / screenshot issues. @@ -239,8 +241,15 @@ safe-outputs: # PER-RUN total (not per-PR): a sweep may surface several green PRs and/or # annotate several flaky reds in one run. At max:1 all but the first comment # were silently dropped, starving the workflow's #1 value (surfacing green PRs - # for review). Raised to 3 to match the per-run throughput of the other outputs. - max: 3 + # for review). Sized to 6 (not 3) because a single green DRAFT PR that gets + # flipped ready in one sweep spends TWO comment slots — the Step 3 ✅ surface + # comment (which still names the /azp-gated legs a human must kick, so it is NOT + # redundant with 🎯) AND the Step 3.6 T3 🎯 readiness comment. At max:3 the shared + # bucket drained after ~1 draft flip and T3's atomicity pre-check then deferred + # every further mark-ready even while the mark-ready/add_labels buckets (also 3) + # sat idle; 6 lets ~3 draft flips (2 comments each) land per sweep, so the + # mark-ready:3 / add_labels:3 caps are actually reachable. + max: 6 target: "*" # Hard constraint (defense-in-depth): only comment on THIS workflow's own # ci-fix PRs — [ci-fix] title prefix AND agentic-workflows label. Mirrors the @@ -261,13 +270,51 @@ safe-outputs: # never the title, so disable title rewrites — this compiles to allow_title:false # and removes any ability to retitle an arbitrary PR. title: false - # NOTE: gh-aw v0.79.8 does NOT emit required-title-prefix/required-labels into + # NOTE: gh-aw v0.80.9 (the pinned compiler) does NOT emit required-title-prefix/required-labels into # the compiled config for update-pull-request (verified against the lock — it # silently drops them, unlike add-comment / push-to-pull-request-branch which # honor them). So which-PR scoping here relies on prompt Hard-Rule 6 + # min-integrity:approved; the residual is body-marker edits only (no code, no # merge, no close), capped at max:3. Revisit required-* if a future gh-aw # compiler emits them for this output. + mark-pull-request-as-ready-for-review: + # Flip a validated draft [ci-fix] PR to ready-for-review once the SPECIFIC test + # this PR was opened to fix is confirmed green in the PR's OWN CI (Step 3.6) — + # even when unrelated legs are red. The workflow is schedule/dispatch-triggered + # (no triggering-PR context), so target "*" lets the agent name the PR number it + # validated. This is a state transition ONLY: it never approves and never merges — + # a human still reviews and merges. + # PER-RUN total (not per-PR): a sweep may validate several PRs' target tests green. + max: 3 + target: "*" + # Hard constraint (defense-in-depth): only ever un-draft THIS workflow's own + # ci-fix PRs — [ci-fix] title prefix AND agentic-workflows label — so a confused + # or prompt-injected agent cannot mark an arbitrary PR ready. (If the v0.80.9 + # compiler silently drops these — as documented for update-pull-request above — + # the Step 3.6 preconditions + min-integrity:approved are the compensating scope + # controls; verify against the lock after compiling.) + required-title-prefix: "[ci-fix] " + required-labels: [agentic-workflows] + add-labels: + # Apply the p/0 priority label to a [ci-fix] PR at the exact moment the loop flips it + # from draft to ready-for-review (Step 3.6 T3) — so a validated, review-ready fix lands + # in the team's p/0 triage queue instead of sitting unseen in the draft backlog. Paired + # 1:1 with mark-pull-request-as-ready-for-review; target "*" lets the agent name the PR + # it just validated. + # allowed = HARD allowlist: the agent may ONLY ever add p/0, nothing else. This caps the + # blast radius of a confused/prompt-injected agent to exactly one benign priority label — + # it can never apply a downstream-triggering or destructive label. + allowed: [p/0] + # PER-RUN total (not per-PR): a sweep may mark several PRs ready in one run; each adds + # one label, so this matches the mark-ready per-run cap (3). + max: 3 + target: "*" + # Hard constraint (defense-in-depth): only ever label THIS workflow's own ci-fix PRs — + # [ci-fix] title prefix AND agentic-workflows label. (If the v0.80.9 compiler drops these + # — as documented for update-pull-request above — the Step 3.6 preconditions + + # min-integrity:approved + the allowed:[p/0] allowlist are the compensating controls.) + required-title-prefix: "[ci-fix] " + required-labels: [agentic-workflows] timeout-minutes: 90 @@ -339,33 +386,48 @@ through `safe-outputs`. 5. **One issue = one outcome per run.** Exactly one of: respond to a maintainer's change-request on the open PR (Track C, Step 3.5.R); open the first fix/help/ de-flake PR; advance an existing PR by one attempt (push a follow-up fix); - surface a validated-green PR for review; annotate an unrelated-flake red; wait + surface a validated-green PR for review; mark a target-validated draft PR + ready-for-review (Step 3.6 T3 — the terminal outcome that supersedes the same + run's surface/annotate precursor line), or record its `already-ready` + steady-state no-op; annotate an unrelated-flake red; wait (CI not yet settled); or a recorded skip (the dedicated needs-human PR is deferred — the attempt cap records a skip instead; see Step 6). Always prefer advancing or opening a PR over a skip when a non-mute diff is producible. 6. **All writes via `safe-outputs`.** Allowed outputs: `create_pull_request` (first attempt only), `push_to_pull_request_branch` (advance an existing PR by one attempt), `update_pull_request` (bump the attempt marker / refresh the - prior-attempts table), and `add_comment` (progress notes on the `[ci-fix]` PR - ONLY). NEVER comment on the tracking issue (issues are locked by + prior-attempts table), `add_comment` (progress notes on the `[ci-fix]` PR + ONLY), `mark_pull_request_as_ready_for_review` (flip a target-validated draft + `[ci-fix]` PR from draft to ready — Step 3.6 T3 only), and `add_labels` (add + ONLY the `p/0` label, ONLY on that same draft→ready transition — Step 3.6 T3). + NEVER comment on the tracking issue (issues are locked by `.github/workflows/ci-scan-lock-issues.yml`) — `add_comment` targets the PR. No `gh pr create`, no manual `git push`. **Defense-in-depth:** `add_comment` and `update_pull_request` use `target: "*"` (the agent supplies the PR number). - `add_comment` is now config-locked to `required-title-prefix: "[ci-fix] "` + + `add_comment`, `mark_pull_request_as_ready_for_review`, and `add_labels` are + config-locked to `required-title-prefix: "[ci-fix] "` + `required-labels: [agentic-workflows]` (hard-enforced by the handler, same as - `push_to_pull_request_branch`), so a comment can only ever land on THIS - workflow's own PRs. `update_pull_request` CANNOT be config-locked to a - title/label in gh-aw v0.79.8 (the compiler silently drops `required-*` for that - output — verified against the lock), so it keeps `title: false` (no retitles; - body-marker edits only) plus this prompt-level guard. Before emitting either, - VERIFY the target PR carries BOTH the `[ci-fix]` title prefix AND the - `agentic-workflows` label; never comment on or edit the body of any PR that - lacks both. + `push_to_pull_request_branch`), and `add_labels` additionally has an + `allowed: [p/0]` allowlist so it can ONLY ever add `p/0` — so these can only + ever land on THIS workflow's own PRs. `update_pull_request` CANNOT be + config-locked to a title/label in gh-aw v0.80.9 (the compiler silently drops + `required-*` for that output — verified against the lock), so it keeps + `title: false` (no retitles; body-marker edits only) plus this prompt-level + guard. Before emitting ANY of these, VERIFY the target PR carries BOTH the + `[ci-fix]` title prefix AND the `agentic-workflows` label; never comment on, + edit, un-draft, or label any PR that lacks both. 7. **Per-run safe-output caps (per RUN, not per PR).** Each output type is capped per run: `create_pull_request` 3, `push_to_pull_request_branch` 3, - `add_comment` 3, `update_pull_request` 3. Note an ADVANCE spends one push **and** - one update **and** one comment, so ≤ 3 advances/run; surfacing a green or - annotating a flake spends one comment. When a bucket is exhausted, do NOT keep + `add_comment` 6, `update_pull_request` 3, `mark_pull_request_as_ready_for_review` + 3, `add_labels` 3 (counts LABELS, not calls). Note an ADVANCE spends one push + **and** one update **and** one comment, so ≤ 3 advances/run; surfacing a green or + annotating a flake spends one comment; a **mark-ready (Step 3.6 T3)** of a + still-draft PR spends TWO comments (the Step 3 ✅ surface **and** the T3 🎯 + readiness note) **and** one mark-ready **and** one label as an ALL-OR-NOTHING set + (T3 pre-checks those buckets and defers the whole PR if any is exhausted — never + mark a PR ready without its 🎯 audit comment). `add_comment` is therefore sized 6 + (not 3) so ~3 draft flips, at 2 comments each, can land in one sweep instead of the + shared comment bucket starving the otherwise-idle mark-ready/add_labels buckets. When a bucket is exhausted, do NOT keep emitting (extras are silently dropped) — record `skipped: per-run cap reached; deferring PR # to next cycle` for each remaining PR so the drop is a deliberate, logged decision. The continuous loop picks the deferred PRs up next @@ -409,8 +471,9 @@ For every open tracking issue in scope, converge on exactly one outcome: | Open first help-wanted draft `[ci-fix]` PR | No open PR yet; a plausible candidate exists but cannot be runner-validated (device/UI tests) (attempt 1) | | Open first de-flake draft `[ci-fix]` PR | No open PR yet; failure is intermittent (green-on-retry) from a genuine test-quality defect; a deterministic-synchronization fix is producible (attempt 1, Step 4.7 bucket b) | | Advance an existing PR (push attempt N+1) | Open PR's own CI settled red *because of the fix itself*, no human engaged, marker < 10 — push a NEW distinct fix onto the same branch (Step 3.5 → 5.6) | -| Surface a validated-green PR | Open PR's own CI settled green — comment "validated, attempt N/10; ready for review" and do NOT advance (Step 3.5) | -| Annotate an unrelated-flake red | Open PR is red only on baseline-flake / unrelated legs — comment which leg needs a re-run; do NOT burn an attempt (Step 3.5) | +| Surface a validated-green PR | Open PR's own CI settled green — comment "validated, attempt N/10; ready for review" and do NOT advance (Step 3.5), then mark the draft PR ready for review once the fixed test is confirmed green (Step 3.6) | +| Annotate an unrelated-flake red | Open PR is red only on baseline-flake / unrelated legs — comment which leg needs a re-run; do NOT burn an attempt (Step 3.5); if the SPECIFIC fixed test is confirmed green on that build, still mark the draft PR ready for review (Step 3.6) | +| Mark a validated PR ready for review | The specific test this PR fixed is confirmed green in the PR's own CI (even if unrelated legs are red) — comment the target-test result and transition the draft PR to ready for review; never approve or merge (Step 3.6) | | Wait | Open PR's CI is pending / not yet settled on the current head SHA — do nothing this cycle (Step 3.5) | | Hand-off skip (attempt cap) | Marker == 10 and the signature still reproduces — stop and defer to humans; the dedicated `[ci-fix][needs-human]` PR is planned but currently deferred (Step 6) | | Recorded skip | Visual-regression, human engaged, already-handled, fixed-in-latest-build, infra-flake, out-of-bounds, only-mute-available, or no novel approach producible | @@ -685,14 +748,34 @@ Run these gates in order — the FIRST that fires decides this cycle's outcome: 1. **Human engaged.** If `C.humanEngaged` → `skipped: human engaged on PR #; deferring` and stop. Never fight a human reviewer — the intentional hand-off boundary still holds the moment a person touches the PR. -2. **CI not settled / unknown / incomplete prefetch.** If `C.checksSettled == false` - OR `C.overallConclusion` is `pending`, `neutral`, or `unknown`, OR - `C.dataComplete == false` → the fix's CI has not finished on the current head SHA - (`C.headSha`), or the prefetch could not fully read this PR (a partial read can - understate `humanEngaged` / `attempt`). `skipped: PR # CI pending / prefetch - incomplete on ; waiting` and stop. *(Round 1: this is where a - maintainer `/azp run` is awaited — the `/azp`-gated uitests/devicetests legs will - not have run until a human kicks them.)* +2. **CI not settled — but validate the target test first (target-focused readiness).** + If `C.checksSettled == false` OR `C.overallConclusion` is `pending`, `neutral`, or + `unknown`, OR `C.dataComplete == false` → the *overall* build has not finished on the + current head SHA (`C.headSha`), or the prefetch could not fully read this PR (a partial + read can understate `humanEngaged` / `attempt`). Unrelated legs still draining must NOT + keep an already-proven fix parked as a draft, so before waiting, try the target-focused + fast-path: + - **2a — Target-green fast-path (draft `[ci-fix]` PRs only).** If `C.dataComplete == + true` AND the Step 3.6 preconditions hold (draft PR, `[ci-fix] ` title prefix AND the + `agentic-workflows` label), run Step 3.6 **T1–T2** against `C.headSha` now: identify + the target test(s) and read the PR's OWN build **timeline / per-leg status** (anonymous + `_apis/build` — never `_apis/test`, per Hard-Rule 8). If EVERY target test is + **VALIDATED-GREEN** (per T2's all-platform definition below — the target's category leg + is `succeeded` on every platform that runs it, failed on none, and NO target platform + family the pipeline covers is still pending/unconcluded, per T2's "no platform left + unverified" rule; a platform that simply has no leg for the target's category — the test + doesn't run there — is fine, not a gap) AND every leg in + `C.failedLegs` that has ALREADY concluded classifies as **unrelated flake** (Step 4 / + Step 4.7 method — the fix is implicated in NO completed red), then the fix is proven + regardless of unrelated *pending* legs → run **Step 3.6 T3** (mark ready + 🎯 comment) + and stop. This early-out NEVER advances an attempt and NEVER acts on a red that could + be the fix's fault. + - **2b — Otherwise WAIT.** If the target test has not yet executed (pending/absent on + its leg), or a completed red is (or may be) caused by the fix, or `C.dataComplete == + false`, or this PR has no identifiable target test → `skipped: PR # CI pending / + target not yet validated on ; waiting` and stop. *(Round 1: this is where a + maintainer `/azp run` is awaited — the `/azp`-gated uitests/devicetests legs will not + have run until a human kicks them.)* 3. **Green → surface for review.** If `C.overallConclusion == "success"`: the checks that RAN are green. **Primary-gate check first:** the deterministic `success` verdict only certifies "at least one green check, nothing failing or @@ -704,18 +787,31 @@ Run these gates in order — the FIRST that fires decides this cycle's outcome: # primary CI gate (maui-pr) not green on ; waiting` and stop — do NOT surface. (The `/azp`-gated `maui-pr-uitests` (def 313) / `maui-pr-devicetests` (def 314) legs MAY still be un-run — that is expected and is named below, not a - reason to withhold the surface.) **Idempotency:** scan the PR's existing comments - for a prior bot `✅ … validated … on ` note for THIS head SHA — if one - already exists, record `already-surfaced PR # (head )` and stop - WITHOUT re-commenting (re-surfacing the same green every tick is noise and burns - the per-run comment budget other PRs need). Otherwise resolve the attempt number from - `C.effectiveAttempt` (the authoritative max(marker, bot-commit) counter; omit the - number only if it is somehow indeterminate). **Dry-run gate (Step 0):** if `dry_run == "true"`, do NOT emit any comment — instead print the intended `✅` validated-green body to the run log, tally `dry-run: would-surface-green PR #`, and stop. Otherwise `add_comment` on PR #: `✅ Attempt /10 validated - — the fix's CI is green on . Ready for human review.` - Name any `/azp`-gated legs (uitests def 313 / devicetests def 314) that have not - run and still need a maintainer `/azp run`. Do NOT advance. Record - `surfaced-green PR # (attempt /10)` and stop. *(This directly - attacks the real bottleneck — no reviews — so it is the highest-value outcome.)* + reason to withhold the surface.) **Comment idempotency + dry-run suppress the ✅ + comment ONLY — neither skips Step 3.6.** Scan the PR's existing comments for a prior + bot `✅ … validated … on ` note for THIS head SHA; if one exists, set + `SKIP_SURFACE_COMMENT = true`. Resolve the attempt number from `C.effectiveAttempt` + (the authoritative max(marker, bot-commit) counter; omit the number only if it is + somehow indeterminate). Then post the surface comment UNLESS suppressed: + - if `dry_run == "true"`: do NOT emit — print the intended `✅` validated-green body to + the run log and tally `dry-run: would-surface-green PR #`; + - else if `SKIP_SURFACE_COMMENT`: do NOT re-post — record `already-surfaced PR # + (head )` (re-surfacing the same green every tick is noise and burns the + per-run comment budget other PRs need); + - else `add_comment` on PR #: `✅ Attempt /10 validated — the fix's CI is + green on .` naming any `/azp`-gated legs (uitests def 313 / devicetests def + 314) that have not run and still need a maintainer `/azp run`; record `surfaced-green + PR # (attempt /10)`. (Do NOT assert "ready for human review" on a PR that + is still a draft — Step 3.6, not this comment, owns the draft→ready flip and posts its + own 🎯 announcement when it fires.) + Do NOT advance. Then — in ALL of the above cases — run **Step 3.6** (target-test + readiness gate) for this PR before stopping: its `/azp`-gated target legs may conclude + green on this SAME head SHA on a LATER sweep (an `/azp run` adds no commit), and Step + 3.6 is the ONLY place the draft→ready flip happens, so it MUST re-evaluate every sweep — + never `stop` here before it. *(This directly attacks the real bottleneck — no reviews — + so it is the highest-value outcome.)* 4. **Red → classify caused-by-fix vs unrelated-flake.** If `C.overallConclusion == "failure"`, analyze the PR's OWN failing build (NOT `main`): find the AzDO `maui-pr` build for this PR (filter builds by `branchName=refs/pull//merge`, @@ -723,18 +819,26 @@ Run these gates in order — the FIRST that fires decides this cycle's outcome: method and Step 4.7 flake buckets to `C.failedLegs`. - **Unrelated flake only** (every failed leg is known-flaky / infra / a pre-existing baseline red NOT introduced by the fix): do NOT burn an attempt. - **Idempotency first:** scan the PR's existing comments for a prior bot - `♻️ … unrelated flake … on ` note for THIS head SHA — if one already - exists, record `already-annotated-flake PR # (head )` and stop - WITHOUT re-commenting (re-annotating the same flake every 12h sweep is noise and - burns the per-run comment budget other PRs need). Otherwise resolve the attempt - number from `C.effectiveAttempt` (the authoritative max(marker, bot-commit) - counter), omitting the `Attempt /10:` prefix only if it is somehow - indeterminate. **Dry-run gate (Step 0):** if `dry_run == "true"`, do NOT emit any comment — instead print the intended `♻️` unrelated-flake body to the run log, tally `dry-run: would-annotate-flake PR #`, and stop. Otherwise `add_comment` on PR #: `♻️ Attempt /10: red is - unrelated flake on leg(s) () on ; the fix itself is not - implicated. A maintainer re-run (/azp run ) should clear it.` Record - `annotated-flake PR # (head )` and stop. *(Round 1: human re-runs; - Round 2: auto re-trigger.)* + **Comment idempotency + dry-run suppress the ♻️ comment ONLY — neither skips Step + 3.6.** Scan the PR's existing comments for a prior bot `♻️ … unrelated flake … on + ` note for THIS head SHA; if one exists, set `SKIP_FLAKE_COMMENT = true`. + Resolve the attempt number from `C.effectiveAttempt` (the authoritative max(marker, + bot-commit) counter), omitting the `Attempt /10:` prefix only if it is + somehow indeterminate. Then post the flake note UNLESS suppressed: + - if `dry_run == "true"`: do NOT emit — print the intended `♻️` unrelated-flake body + to the run log and tally `dry-run: would-annotate-flake PR #`; + - else if `SKIP_FLAKE_COMMENT`: do NOT re-post — record `already-annotated-flake PR + # (head )` (re-annotating the same flake every 12h sweep is noise and + burns the per-run comment budget other PRs need); + - else `add_comment` on PR #: `♻️ Attempt /10: red is unrelated flake on + leg(s) () on ; the fix itself is not implicated. A + maintainer re-run (/azp run ) should clear it.` record `annotated-flake + PR # (head )`. + Then — in ALL of the above cases — run **Step 3.6** (target-test readiness gate) for + this PR before stopping: its `/azp`-gated target legs may conclude green on this SAME + head SHA on a LATER sweep, and Step 3.6 is the ONLY place the draft→ready flip + happens, so it MUST re-evaluate every sweep — never `stop` here before it. *(Round 1: + human re-runs; Round 2: auto re-trigger.)* - **Caused by the fix** (a failed leg still matches the original target signature, or the fix introduced a NEW failure): advance an attempt. - **Attempt count.** `attempt = C.effectiveAttempt` — the authoritative @@ -752,6 +856,188 @@ Run these gates in order — the FIRST that fires decides this cycle's outcome: from every prior commit on this branch**, emitted via the Step 5.6 ADVANCE path (push onto the same PR). +#### Step 3.6 — Target-test verification & mark-ready gate + +Reached from the Step 3 **green-surface** branch, the Step 4 **unrelated-flake** branch +(AFTER that branch has posted its comment), and the Step 3.5 **gate-2 target-focused +fast-path** (2a — while unrelated legs are still pending). Purpose: confirm the SPECIFIC +test(s) this PR was opened to fix now PASS in the PR's own CI, and — only when they do +— transition the draft `[ci-fix]` PR to **ready for review** so a maintainer sees a +validated fix instead of a draft. This is the ONLY place the loop flips draft→ready; it +is a state transition, **never** an approval or a merge (a human still reviews and +merges). Overall red on *unrelated* legs must NOT gate readiness — we validate the fix, +not the base branch's flakiness. + +**Preconditions** (ALL must hold; otherwise record `skipped: readiness N/A PR # +()` and stop this gate): +- `C.isDraft == true` — if the PR is already ready-for-review, the draft→ready + transition is already done; do NOT re-mark **and do NOT re-apply `p/0`**. `p/0` is applied + exactly ONCE, atomically with the draft→ready flip (T3 — all-or-nothing with the 🎯 audit + comment + mark-ready), so an already-ready loop PR that is MISSING `p/0` has almost + certainly had it **removed by a maintainer** de-prioritizing the PR — a pure triage action + the human-engagement guard (which inspects comments/reviews/commits, NOT label changes) + cannot see. Re-adding `p/0` every sweep would fight that maintainer indefinitely, violating + the loop's "never override a human" contract. So the loop does NOT reconcile the label: + record `already-ready PR #` and stop this gate. (Trade-off: on the rare occasion a + transient API error drops `p/0` at flip time *after* the T3 atomic pre-check passed, it is + not auto-re-added — but the PR is still ready-for-review with its 🎯 audit comment, and + re-adding `p/0` is a trivial manual action; that is strictly preferable to steam-rolling a + maintainer's deliberate de-prioritization.) +- The PR is unmistakably THIS workflow's own: `[ci-fix] ` title prefix AND the + `agentic-workflows` label (mirrors the safe-output lock; the compensating scope + control if the v0.80.9 compiler drops the declarative required-*). +- You reached this gate from Step 3 (green), Step 4 (**unrelated-flake**), or the Step 3.5 + gate-2 **target-focused fast-path** (2a) — i.e. the fix is NOT implicated in any red that + has concluded. If Step 4 classified a red as **caused by the fix** (ADVANCE), do NOT run + this gate — advance the attempt instead. + +**T1 — Identify the target test(s).** From the `[ci-scan]` issue signature the fix +addresses (and the PR's own diff), extract the fully-qualified test method name(s) the +fix targets — e.g. `SafeAreaShouldWorkOnAllShellTabs`. For a de-flake it is the +de-flaked test; for a product fix it is the originally-failing test(s). If NO specific +test can be identified (e.g. a product build-break rather than a test failure), this is a +**build-only fix** — there is no single test to validate cross-platform, so validate at +**whole-build** granularity rather than stopping (Step 3 only *comments*; Step 3.6 is the +sole draft→ready flip, so a build-only fix is undrafted HERE or not at all). We only reach +this gate from Step 3 (green) / Step 4 (unrelated-flake) / gate-2a, so the fix is not +implicated in any concluded red; additionally require, via the T2 timeline method on +`C.headSha`, that the primary build pipeline `maui-pr` (def 302) has CONCLUDED with EVERY +one of its platform build legs `succeeded`/`completed` and NONE failed/canceled or still +pending — the build is green on **every** platform, not just the originally-broken one (the +cross-platform guard, applied to the build instead of a test). A build-only fix is undrafted +ONLY on the strength of the auto-running `maui-pr` (def 302) whole-build green, which the +loop CAN observe: if the PR's `[ci-scan]` issue signature or its own diff indicates the +ORIGINATING failure was in a `/azp`-gated pipeline (`maui-pr-uitests` def 313 or +`maui-pr-devicetests` def 314) rather than `maui-pr` (def 302), do NOT undraft on +`maui-pr`-green alone — those pipelines do not auto-run on this PR (GITHUB_TOKEN cannot +trigger them), so a green `maui-pr` build is NOT evidence the gated build break is fixed; +record `skipped: build-only fix PR # targets gated pipeline () not run — +deferring to human` and stop this gate. Otherwise, set `TARGET := "the maui-pr build +(build-only fix — no single target test)"` and proceed to **T3** to mark ready. If any `maui-pr` build leg is still unconcluded, record `skipped: build-only fix PR +# not yet whole-build green (leg(s) pending)` and stop this gate WITHOUT marking +ready (a green subset is not enough — a still-pending build leg could yet fail). + +**T2 — Verify the target test's platform legs are green (anonymous, leg-level).** Per-test +outcomes come from the AzDO **test-results** API (`_apis/test/...`), which is **NOT reachable +anonymously — Hard-Rule 8**: those endpoints 302-redirect to a sign-in page on this runner, so +a `curl` returns an HTML redirect (not JSON) and every target test would look "not executed". +Validate instead at **leg granularity** using ONLY the anonymous `_apis/build/...` timeline +(Hard-Rule 8) — a `succeeded` category leg means every test in that category **that actually +ran** passed on that platform. This is a strong bar **only if the target test genuinely +executes**: a job can go green while one specific test is skipped at runtime (`[Ignore]`, +`Assert.Ignore()`, `Assert.Inconclusive()`, a `Skip=`/conditional `[Fact]`, an `#if`-out, or +an early `return` before the asserts). So before trusting a green leg as proof the target +passed, **confirm from the PR's OWN diff that the fix does NOT skip, ignore, disable, or +short-circuit the target test** (it adds no `[Ignore]`/`[Explicit]`, `Assert.Ignore`/ +`Assert.Inconclusive`, `Skip=`, category-exclusion, `#if`-out, or early `return` around the +target). If the fix could cause the target to be runtime-skipped rather than genuinely pass, +leg-level green is NOT sufficient evidence — record `skipped: target test may be +runtime-skipped by this fix (diff adds ); leg-green insufficient, deferring to human +on PR #` and stop this gate WITHOUT marking ready. Otherwise (the target genuinely runs +and asserts, as a de-flake or product fix does) a green category leg cannot hide a +target-test failure, so treat it as authoritative. + +Map the target test to the CI leg(s) that run it. A leg name encodes platform + test category +(e.g. `Android UITests SafeAreaEdges,Shadow`, `iOS UITests SafeAreaEdges,Shadow`, `macOS +UITests SafeAreaEdges,Shadow`, `Windows UITests SafeAreaEdges,Shadow`). Determine the target +test's UI-test category — its `[Category(UITestCategories.X)]` in the test/HostApp source, +visible in the PR diff or the test file — to know which leg-name substring identifies its legs; +for a device test, the per-platform device-test legs. + +Using the SAME build-discovery as Step 4 (filter AzDO builds by `branchName=refs/pull//merge` +or `sourceVersion == C.headSha`), read each build's **timeline** on `C.headSha` for the +pipeline(s) that RUN the target test — `maui-pr` (def 302) for unit/integration tests, +`maui-pr-uitests` (def 313) for Appium UI tests, `maui-pr-devicetests` (def 314) for device +tests: + +```bash +ORG=dnceng-public; PROJ=public; BUILD= +# Anonymous + Hard-Rule-8-compliant. NEVER call _apis/test/... (it 302-redirects to sign-in). +# Each type=="Job" record is one platform leg: result (succeeded/failed/canceled/null), +# state (completed/inProgress/pending), name (encodes " UITests "): +curl -s "https://dev.azure.com/$ORG/$PROJ/_apis/build/builds/$BUILD/timeline?api-version=7.1" \ + | tee /tmp/gh-aw/agent/timeline_${BUILD}.json \ + | jq -r '.records[] | select(.type=="Job") | "\(.result)\t\(.state)\t\(.name)"' +``` + +(The prefetch already exposes per-leg status in `C.failedLegs` / the checks context, and +`gh pr checks ` lists the same per-leg rows with platform+category in the name — use +whichever is handy; the build timeline is the authoritative cross-check on the exact build.) + +Treat a target test as **VALIDATED-GREEN** only if, on `C.headSha`, the leg that runs its +category is green on **every platform it runs on** — a fix that repairs one platform must not +silently regress the same test's leg on another, so a green leg on the originally-red platform +alone is NOT enough. Across the target pipeline's platform legs (for a UI test: the Android, +iOS, Windows, and macOS/MacCatalyst legs running the target's category; for a device test: each +device-test platform), require ALL of: +- the target's category leg is `result == "succeeded"` **and** `state == "completed"` on + **every** platform that runs it, AND +- that leg is `failed` / `canceled` / aborted on **NO** platform, AND +- **no platform is left unverified.** For each platform family the target pipeline covers, that + platform's category leg must have CONCLUDED (`state == "completed"`) on `C.headSha`. If a + platform simply has no leg for the target's category, the test does not run there — that is + fine, not a gap. But if a platform's category leg has **not concluded** (`state` is + `inProgress` / pending, or an `/azp`-gated `maui-pr-uitests` / `maui-pr-devicetests` leg that + has not been kicked), the test's status on that platform is UNKNOWN → the fix is NOT yet + validated across platforms: record `skipped: target test green on but + not yet verified on (leg(s) pending / need /azp run) on PR #` + and stop this gate WITHOUT marking ready. + +A target test whose category leg never concluded on ANY platform (**not executed** anywhere — +e.g. its `/azp`-gated pipeline has not been kicked) is likewise NOT validated: record `skipped: +target test not yet executed on PR # ( not run — needs /azp run)` and stop +this gate WITHOUT marking ready. Do NOT overclaim — a green *sibling* leg (a different category +on the same platform) is not the target's leg, and a green leg on one platform is not a pass on +the others. + +**T3 — Mark ready + report.** If EVERY target test is VALIDATED-GREEN. The 🎯 comment, +the mark-ready, and the `p/0` label are THREE SEPARATE safe-outputs — the comment +existing does NOT prove the mark-ready took effect, so they are tracked independently: + +- **Comment idempotency (dup-suppress only — never gates the mark-ready).** We only reach + T3 while `C.isDraft == true` (the precondition stops an already-ready PR before here). So + if a prior bot comment for THIS head that carries the leading `🎯` anchor AND the + contiguous phrase `validated green on ` already exists (BOTH T3 variants — + `🎯 Target test validated green on ` and the build-only `🎯 Build validated green on + ` — contain that exact contiguous phrase, so this recognizes both; keying on + `target test` alone would instead miss the build-only comment and re-post it every sweep. + **Require the `🎯` anchor** so the match can NEVER be loosened to a gapped + `validated…green…on` form that would false-positive on the Step 3 `✅ … validated — the + fix's CI is green on ` surface comment and wrongly suppress the audit 🎯), the comment + landed on an earlier sweep but the **mark-ready did NOT take + effect** (the PR is still a draft) — set `SUPPRESS_COMMENT = true` (do not re-post the + duplicate 🎯 comment) but STILL complete the mark-ready + label below. Never treat the + comment's existence as "already marked ready" while the PR is still a draft. Record this + case as `re-marking PR # (🎯 present; mark-ready did not take on a prior sweep)`. +- **Atomicity budget pre-check (Hard-Rule 7).** Determine the outputs this gate will emit: + `mark_pull_request_as_ready_for_review` + `add_labels` always, plus `add_comment` unless + `SUPPRESS_COMMENT`. Verify a free per-run slot remains in EACH bucket you are about to + use. If ANY required bucket is exhausted, emit NONE of them — record `skipped: per-run + cap reached; deferring mark-ready PR # to next cycle` and stop this gate. Never mark a + PR ready (or label it) unless its 🎯 audit comment is GUARANTEED to exist for this exact + head SHA — either posted in this same sweep, or (in the `SUPPRESS_COMMENT` re-marking case) + already present from a prior sweep. The outputs you DO emit either all land or all defer + together; the audit trail must never be absent, but it is NOT re-posted when it already exists. +- **Dry-run gate (Step 0):** if `dry_run == "true"`, emit NOTHING — print the intended + readiness comment and "would mark ready + add p/0" to the run log, tally `dry-run: + would-mark-ready PR #`, and stop. +- Otherwise emit for THIS PR number ``: + 1. `add_comment` (ONLY if not `SUPPRESS_COMMENT`): `🎯 Target test validated green on + — passed on ALL platforms it runs on (, + buildId ). — not + caused by this fix."> Transitioning this PR from draft to ready for review and adding + `p/0`; a maintainer still reviews and merges.` (For a **build-only fix** — the T1 + whole-build fallback, no single target test — phrase the first clause as `🎯 Build + validated green on — the maui-pr build passed on ALL platforms (buildId + ).` instead of naming a test.) + 2. `mark_pull_request_as_ready_for_review` with `reason:` a one-line justification + naming the validated test(s) and ``. + 3. `add_labels` with `labels: ["p/0"]` for PR # — put the now-review-ready fix into + the team's p/0 priority queue so it is triaged, not lost in the draft backlog. (If + the PR somehow already carries `p/0`, this is a harmless no-op.) +- Record `marked-ready PR # (target green on ALL platforms on , + labeled p/0)` and stop. + #### Step 3.5.R — Maintainer change-request response (Track C) Reached from Step 3.5 gate 0. Goal: when an **eligible human reviewer** (Hard-Rule @@ -1383,7 +1669,19 @@ Filed by [`ci-status-fix`](https://github.com/dotnet/maui/blob/main/.github/work ### Step 8 — Per-issue tally + end-of-run summary -Per issue, append one outcome line to `/tmp/gh-aw/agent/coverage.txt`: +Per issue, append **one** outcome line to `/tmp/gh-aw/agent/coverage.txt` — the +terminal outcome for the cycle. When one cycle produces a chained pair — a PR gets a +same-cycle precursor line and **then** a Step 3.6 readiness line (`marked-ready` on the +draft→ready flip, or `already-ready` on a later steady-state sweep of an already-ready +PR) — record ONLY the terminal Step 3.6 readiness line: it supersedes ANY same-cycle +non-readiness precursor for that PR, so an aggregator keying on one-line-per-issue never +double-counts. The precursor may be a Step 3 green line (`surfaced-green` on the first ✅, +or `already-surfaced` when it already exists) OR — when an unrelated-flake red and a +target-green coincide — a Step 4 `annotated-flake` line; either way the readiness line is +terminal, and the superseded precursor's signal survives in the PR's 🎯/♻️ comment so no +information is lost. This covers the flip cycle (`surfaced-green` → `marked-ready`), the +post-flip steady state (`already-surfaced` → `already-ready`), and the flake-coincident +flip (`annotated-flake` → `marked-ready`): ``` # main attempt- @@ -1394,8 +1692,14 @@ Per issue, append one outcome line to `/tmp/gh-aw/agent/coverage.txt`: `advance-PR # attempt /10` (ADVANCE mode — new commit pushed to the existing PR), `surfaced-green PR #` (fix's own CI went green; commented for review, did not advance), `annotated-flake PR #` (red was unrelated flake; -commented, attempt NOT burned), `waiting PR #` (CI not settled yet), -`dry-run: would-`, `skipped: `. (The +commented, attempt NOT burned), `marked-ready PR #` (the specific fixed test +was confirmed green in the PR's own CI; draft flipped to ready for review), +`already-surfaced PR #` (the fix's ✅ green comment already existed this sweep; +re-post suppressed — superseded by the Step 3.6 readiness line when the PR also flips +ready that cycle), `already-ready PR #` (terminal steady state — the PR was flipped +ready on a prior cycle and remains green on an unchanged head; no action taken, +supersedes `already-surfaced`), `waiting PR #` (CI not settled yet), +`dry-run: would-`, `skipped: `. (The `needs-human-PR` outcome is reserved for the deferred hand-off — Step 6 currently records a skip instead, so it is not emitted.)
(head )` and stop - WITHOUT re-commenting (re-annotating the same flake every 12h sweep is noise and - burns the per-run comment budget other PRs need). Otherwise resolve the attempt - number from `C.effectiveAttempt` (the authoritative max(marker, bot-commit) - counter), omitting the `Attempt /10:` prefix only if it is somehow - indeterminate. **Dry-run gate (Step 0):** if `dry_run == "true"`, do NOT emit any comment — instead print the intended `♻️` unrelated-flake body to the run log, tally `dry-run: would-annotate-flake PR #`, and stop. Otherwise `add_comment` on PR #: `♻️ Attempt /10: red is - unrelated flake on leg(s) () on ; the fix itself is not - implicated. A maintainer re-run (/azp run ) should clear it.` Record - `annotated-flake PR # (head )` and stop. *(Round 1: human re-runs; - Round 2: auto re-trigger.)* + **Comment idempotency + dry-run suppress the ♻️ comment ONLY — neither skips Step + 3.6.** Scan the PR's existing comments for a prior bot `♻️ … unrelated flake … on + ` note for THIS head SHA; if one exists, set `SKIP_FLAKE_COMMENT = true`. + Resolve the attempt number from `C.effectiveAttempt` (the authoritative max(marker, + bot-commit) counter), omitting the `Attempt /10:` prefix only if it is + somehow indeterminate. Then post the flake note UNLESS suppressed: + - if `dry_run == "true"`: do NOT emit — print the intended `♻️` unrelated-flake body + to the run log and tally `dry-run: would-annotate-flake PR #`; + - else if `SKIP_FLAKE_COMMENT`: do NOT re-post — record `already-annotated-flake PR + # (head )` (re-annotating the same flake every 12h sweep is noise and + burns the per-run comment budget other PRs need); + - else `add_comment` on PR #: `♻️ Attempt /10: red is unrelated flake on + leg(s) () on ; the fix itself is not implicated. A + maintainer re-run (/azp run ) should clear it.` record `annotated-flake + PR # (head )`. + Then — in ALL of the above cases — run **Step 3.6** (target-test readiness gate) for + this PR before stopping: its `/azp`-gated target legs may conclude green on this SAME + head SHA on a LATER sweep, and Step 3.6 is the ONLY place the draft→ready flip + happens, so it MUST re-evaluate every sweep — never `stop` here before it. *(Round 1: + human re-runs; Round 2: auto re-trigger.)* - **Caused by the fix** (a failed leg still matches the original target signature, or the fix introduced a NEW failure): advance an attempt. - **Attempt count.** `attempt = C.effectiveAttempt` — the authoritative @@ -762,6 +866,188 @@ Run these gates in order — the FIRST that fires decides this cycle's outcome: from every prior commit on this branch**, emitted via the Step 5.6 ADVANCE path (push onto the same PR). +#### Step 3.6 — Target-test verification & mark-ready gate + +Reached from the Step 3 **green-surface** branch, the Step 4 **unrelated-flake** branch +(AFTER that branch has posted its comment), and the Step 3.5 **gate-2 target-focused +fast-path** (2a — while unrelated legs are still pending). Purpose: confirm the SPECIFIC +test(s) this PR was opened to fix now PASS in the PR's own CI, and — only when they do +— transition the draft `[ci-fix-net11]` PR to **ready for review** so a maintainer sees +a validated fix instead of a draft. This is the ONLY place the loop flips draft→ready; +it is a state transition, **never** an approval or a merge (a human still reviews and +merges). Overall red on *unrelated* legs must NOT gate readiness — we validate the fix, +not the base branch's flakiness. + +**Preconditions** (ALL must hold; otherwise record `skipped: readiness N/A PR # +()` and stop this gate): +- `C.isDraft == true` — if the PR is already ready-for-review, the draft→ready + transition is already done; do NOT re-mark **and do NOT re-apply `p/0`**. `p/0` is applied + exactly ONCE, atomically with the draft→ready flip (T3 — all-or-nothing with the 🎯 audit + comment + mark-ready), so an already-ready loop PR that is MISSING `p/0` has almost + certainly had it **removed by a maintainer** de-prioritizing the PR — a pure triage action + the human-engagement guard (which inspects comments/reviews/commits, NOT label changes) + cannot see. Re-adding `p/0` every sweep would fight that maintainer indefinitely, violating + the loop's "never override a human" contract. So the loop does NOT reconcile the label: + record `already-ready PR #` and stop this gate. (Trade-off: on the rare occasion a + transient API error drops `p/0` at flip time *after* the T3 atomic pre-check passed, it is + not auto-re-added — but the PR is still ready-for-review with its 🎯 audit comment, and + re-adding `p/0` is a trivial manual action; that is strictly preferable to steam-rolling a + maintainer's deliberate de-prioritization.) +- The PR is unmistakably THIS workflow's own: `[ci-fix-net11] ` title prefix AND the + `agentic-workflows` label (mirrors the safe-output lock; the compensating scope + control if the v0.80.9 compiler drops the declarative required-*). +- You reached this gate from Step 3 (green), Step 4 (**unrelated-flake**), or the Step 3.5 + gate-2 **target-focused fast-path** (2a) — i.e. the fix is NOT implicated in any red that + has concluded. If Step 4 classified a red as **caused by the fix** (ADVANCE), do NOT run + this gate — advance the attempt instead. + +**T1 — Identify the target test(s).** From the `[ci-scan-net11]` issue signature the fix +addresses (and the PR's own diff), extract the fully-qualified test method name(s) the +fix targets — e.g. `SafeAreaShouldWorkOnAllShellTabs`. For a de-flake it is the +de-flaked test; for a product fix it is the originally-failing test(s). If NO specific +test can be identified (e.g. a product build-break rather than a test failure), this is a +**build-only fix** — there is no single test to validate cross-platform, so validate at +**whole-build** granularity rather than stopping (Step 3 only *comments*; Step 3.6 is the +sole draft→ready flip, so a build-only fix is undrafted HERE or not at all). We only reach +this gate from Step 3 (green) / Step 4 (unrelated-flake) / gate-2a, so the fix is not +implicated in any concluded red; additionally require, via the T2 timeline method on +`C.headSha`, that the primary build pipeline `maui-pr` (def 302) has CONCLUDED with EVERY +one of its platform build legs `succeeded`/`completed` and NONE failed/canceled or still +pending — the build is green on **every** platform, not just the originally-broken one (the +cross-platform guard, applied to the build instead of a test). A build-only fix is undrafted +ONLY on the strength of the auto-running `maui-pr` (def 302) whole-build green, which the +loop CAN observe: if the PR's `[ci-scan]` issue signature or its own diff indicates the +ORIGINATING failure was in a `/azp`-gated pipeline (`maui-pr-uitests` def 313 or +`maui-pr-devicetests` def 314) rather than `maui-pr` (def 302), do NOT undraft on +`maui-pr`-green alone — those pipelines do not auto-run on this PR (GITHUB_TOKEN cannot +trigger them), so a green `maui-pr` build is NOT evidence the gated build break is fixed; +record `skipped: build-only fix PR # targets gated pipeline () not run — +deferring to human` and stop this gate. Otherwise, set `TARGET := "the maui-pr build +(build-only fix — no single target test)"` and proceed to **T3** to mark ready. If any `maui-pr` build leg is still unconcluded, record `skipped: build-only fix PR +# not yet whole-build green (leg(s) pending)` and stop this gate WITHOUT marking +ready (a green subset is not enough — a still-pending build leg could yet fail). + +**T2 — Verify the target test's platform legs are green (anonymous, leg-level).** Per-test +outcomes come from the AzDO **test-results** API (`_apis/test/...`), which is **NOT reachable +anonymously — Hard-Rule 8**: those endpoints 302-redirect to a sign-in page on this runner, so +a `curl` returns an HTML redirect (not JSON) and every target test would look "not executed". +Validate instead at **leg granularity** using ONLY the anonymous `_apis/build/...` timeline +(Hard-Rule 8) — a `succeeded` category leg means every test in that category **that actually +ran** passed on that platform. This is a strong bar **only if the target test genuinely +executes**: a job can go green while one specific test is skipped at runtime (`[Ignore]`, +`Assert.Ignore()`, `Assert.Inconclusive()`, a `Skip=`/conditional `[Fact]`, an `#if`-out, or +an early `return` before the asserts). So before trusting a green leg as proof the target +passed, **confirm from the PR's OWN diff that the fix does NOT skip, ignore, disable, or +short-circuit the target test** (it adds no `[Ignore]`/`[Explicit]`, `Assert.Ignore`/ +`Assert.Inconclusive`, `Skip=`, category-exclusion, `#if`-out, or early `return` around the +target). If the fix could cause the target to be runtime-skipped rather than genuinely pass, +leg-level green is NOT sufficient evidence — record `skipped: target test may be +runtime-skipped by this fix (diff adds ); leg-green insufficient, deferring to human +on PR #` and stop this gate WITHOUT marking ready. Otherwise (the target genuinely runs +and asserts, as a de-flake or product fix does) a green category leg cannot hide a +target-test failure, so treat it as authoritative. + +Map the target test to the CI leg(s) that run it. A leg name encodes platform + test category +(e.g. `Android UITests SafeAreaEdges,Shadow`, `iOS UITests SafeAreaEdges,Shadow`, `macOS +UITests SafeAreaEdges,Shadow`, `Windows UITests SafeAreaEdges,Shadow`). Determine the target +test's UI-test category — its `[Category(UITestCategories.X)]` in the test/HostApp source, +visible in the PR diff or the test file — to know which leg-name substring identifies its legs; +for a device test, the per-platform device-test legs. + +Using the SAME build-discovery as Step 4 (filter AzDO builds by `branchName=refs/pull//merge` +or `sourceVersion == C.headSha`), read each build's **timeline** on `C.headSha` for the +pipeline(s) that RUN the target test — `maui-pr` (def 302) for unit/integration tests, +`maui-pr-uitests` (def 313) for Appium UI tests, `maui-pr-devicetests` (def 314) for device +tests: + +```bash +ORG=dnceng-public; PROJ=public; BUILD= +# Anonymous + Hard-Rule-8-compliant. NEVER call _apis/test/... (it 302-redirects to sign-in). +# Each type=="Job" record is one platform leg: result (succeeded/failed/canceled/null), +# state (completed/inProgress/pending), name (encodes " UITests "): +curl -s "https://dev.azure.com/$ORG/$PROJ/_apis/build/builds/$BUILD/timeline?api-version=7.1" \ + | tee /tmp/gh-aw/agent/timeline_${BUILD}.json \ + | jq -r '.records[] | select(.type=="Job") | "\(.result)\t\(.state)\t\(.name)"' +``` + +(The prefetch already exposes per-leg status in `C.failedLegs` / the checks context, and +`gh pr checks ` lists the same per-leg rows with platform+category in the name — use +whichever is handy; the build timeline is the authoritative cross-check on the exact build.) + +Treat a target test as **VALIDATED-GREEN** only if, on `C.headSha`, the leg that runs its +category is green on **every platform it runs on** — a fix that repairs one platform must not +silently regress the same test's leg on another, so a green leg on the originally-red platform +alone is NOT enough. Across the target pipeline's platform legs (for a UI test: the Android, +iOS, Windows, and macOS/MacCatalyst legs running the target's category; for a device test: each +device-test platform), require ALL of: +- the target's category leg is `result == "succeeded"` **and** `state == "completed"` on + **every** platform that runs it, AND +- that leg is `failed` / `canceled` / aborted on **NO** platform, AND +- **no platform is left unverified.** For each platform family the target pipeline covers, that + platform's category leg must have CONCLUDED (`state == "completed"`) on `C.headSha`. If a + platform simply has no leg for the target's category, the test does not run there — that is + fine, not a gap. But if a platform's category leg has **not concluded** (`state` is + `inProgress` / pending, or an `/azp`-gated `maui-pr-uitests` / `maui-pr-devicetests` leg that + has not been kicked), the test's status on that platform is UNKNOWN → the fix is NOT yet + validated across platforms: record `skipped: target test green on but + not yet verified on (leg(s) pending / need /azp run) on PR #` + and stop this gate WITHOUT marking ready. + +A target test whose category leg never concluded on ANY platform (**not executed** anywhere — +e.g. its `/azp`-gated pipeline has not been kicked) is likewise NOT validated: record `skipped: +target test not yet executed on PR # ( not run — needs /azp run)` and stop +this gate WITHOUT marking ready. Do NOT overclaim — a green *sibling* leg (a different category +on the same platform) is not the target's leg, and a green leg on one platform is not a pass on +the others. + +**T3 — Mark ready + report.** If EVERY target test is VALIDATED-GREEN. The 🎯 comment, +the mark-ready, and the `p/0` label are THREE SEPARATE safe-outputs — the comment +existing does NOT prove the mark-ready took effect, so they are tracked independently: + +- **Comment idempotency (dup-suppress only — never gates the mark-ready).** We only reach + T3 while `C.isDraft == true` (the precondition stops an already-ready PR before here). So + if a prior bot comment for THIS head that carries the leading `🎯` anchor AND the + contiguous phrase `validated green on ` already exists (BOTH T3 variants — + `🎯 Target test validated green on ` and the build-only `🎯 Build validated green on + ` — contain that exact contiguous phrase, so this recognizes both; keying on + `target test` alone would instead miss the build-only comment and re-post it every sweep. + **Require the `🎯` anchor** so the match can NEVER be loosened to a gapped + `validated…green…on` form that would false-positive on the Step 3 `✅ … validated — the + fix's CI is green on ` surface comment and wrongly suppress the audit 🎯), the comment + landed on an earlier sweep but the **mark-ready did NOT take + effect** (the PR is still a draft) — set `SUPPRESS_COMMENT = true` (do not re-post the + duplicate 🎯 comment) but STILL complete the mark-ready + label below. Never treat the + comment's existence as "already marked ready" while the PR is still a draft. Record this + case as `re-marking PR # (🎯 present; mark-ready did not take on a prior sweep)`. +- **Atomicity budget pre-check (Hard-Rule 7).** Determine the outputs this gate will emit: + `mark_pull_request_as_ready_for_review` + `add_labels` always, plus `add_comment` unless + `SUPPRESS_COMMENT`. Verify a free per-run slot remains in EACH bucket you are about to + use. If ANY required bucket is exhausted, emit NONE of them — record `skipped: per-run + cap reached; deferring mark-ready PR # to next cycle` and stop this gate. Never mark a + PR ready (or label it) unless its 🎯 audit comment is GUARANTEED to exist for this exact + head SHA — either posted in this same sweep, or (in the `SUPPRESS_COMMENT` re-marking case) + already present from a prior sweep. The outputs you DO emit either all land or all defer + together; the audit trail must never be absent, but it is NOT re-posted when it already exists. +- **Dry-run gate (Step 0):** if `dry_run == "true"`, emit NOTHING — print the intended + readiness comment and "would mark ready + add p/0" to the run log, tally `dry-run: + would-mark-ready PR #`, and stop. +- Otherwise emit for THIS PR number ``: + 1. `add_comment` (ONLY if not `SUPPRESS_COMMENT`): `🎯 Target test validated green on + — passed on ALL platforms it runs on (, + buildId ). — not + caused by this fix."> Transitioning this PR from draft to ready for review and adding + `p/0`; a maintainer still reviews and merges.` (For a **build-only fix** — the T1 + whole-build fallback, no single target test — phrase the first clause as `🎯 Build + validated green on — the maui-pr build passed on ALL platforms (buildId + ).` instead of naming a test.) + 2. `mark_pull_request_as_ready_for_review` with `reason:` a one-line justification + naming the validated test(s) and ``. + 3. `add_labels` with `labels: ["p/0"]` for PR # — put the now-review-ready fix into + the team's p/0 priority queue so it is triaged, not lost in the draft backlog. (If + the PR somehow already carries `p/0`, this is a harmless no-op.) +- Record `marked-ready PR # (target green on ALL platforms on , + labeled p/0)` and stop. + #### Step 3.5.R — Maintainer change-request response (Track C) Reached from Step 3.5 gate 0. Goal: when an **eligible human reviewer** (Hard-Rule @@ -1395,7 +1681,19 @@ Filed by [`ci-status-fix-net11`](https://github.com/dotnet/maui/blob/main/.githu ### Step 8 — Per-issue tally + end-of-run summary -Per issue, append one outcome line to `/tmp/gh-aw/agent/coverage.txt`: +Per issue, append **one** outcome line to `/tmp/gh-aw/agent/coverage.txt` — the +terminal outcome for the cycle. When one cycle produces a chained pair — a PR gets a +same-cycle precursor line and **then** a Step 3.6 readiness line (`marked-ready` on the +draft→ready flip, or `already-ready` on a later steady-state sweep of an already-ready +PR) — record ONLY the terminal Step 3.6 readiness line: it supersedes ANY same-cycle +non-readiness precursor for that PR, so an aggregator keying on one-line-per-issue never +double-counts. The precursor may be a Step 3 green line (`surfaced-green` on the first ✅, +or `already-surfaced` when it already exists) OR — when an unrelated-flake red and a +target-green coincide — a Step 4 `annotated-flake` line; either way the readiness line is +terminal, and the superseded precursor's signal survives in the PR's 🎯/♻️ comment so no +information is lost. This covers the flip cycle (`surfaced-green` → `marked-ready`), the +post-flip steady state (`already-surfaced` → `already-ready`), and the flake-coincident +flip (`annotated-flake` → `marked-ready`): ``` # net11.0 attempt- @@ -1406,8 +1704,14 @@ Per issue, append one outcome line to `/tmp/gh-aw/agent/coverage.txt`: `advance-PR # attempt /10` (ADVANCE mode — new commit pushed to the existing PR), `surfaced-green PR #` (fix's own CI went green; commented for review, did not advance), `annotated-flake PR #` (red was unrelated flake; -commented, attempt NOT burned), `waiting PR #` (CI not settled yet), -`dry-run: would-`, `skipped: `. (The +commented, attempt NOT burned), `marked-ready PR #` (the specific fixed test +was confirmed green in the PR's own CI; draft flipped to ready for review), +`already-surfaced PR #` (the fix's ✅ green comment already existed this sweep; +re-post suppressed — superseded by the Step 3.6 readiness line when the PR also flips +ready that cycle), `already-ready PR #` (terminal steady state — the PR was flipped +ready on a prior cycle and remains green on an unchanged head; no action taken, +supersedes `already-surfaced`), `waiting PR #` (CI not settled yet), +`dry-run: would-`, `skipped: `. (The `needs-human-PR` outcome is reserved for the deferred hand-off — Step 6 currently records a skip instead, so it is not emitted.) diff --git a/.github/workflows/ci-status-fix.lock.yml b/.github/workflows/ci-status-fix.lock.yml index 93dc5969cb0d..65511f2b3259 100644 --- a/.github/workflows/ci-status-fix.lock.yml +++ b/.github/workflows/ci-status-fix.lock.yml @@ -1,4 +1,4 @@ -# gh-aw-metadata: {"schema_version":"v4","frontmatter_hash":"f014161967589ebcaf1ac5ec6f3c88ee5a4388d28b55a07dc36c45b7b2aebab6","body_hash":"86c6b2d4c3df13da14c6a3c8b211381fcc9f97bb1951ff7b80588a2c5338e00c","compiler_version":"v0.80.9","strict":true,"agent_id":"copilot","agent_model":"claude-opus-4.8","engine_versions":{"copilot":"1.0.63"}} +# gh-aw-metadata: {"schema_version":"v4","frontmatter_hash":"b0ebf8a2f179900cfe3638e994b27a43cec52bdbdd2331dc1bd4d65762fa2d6b","body_hash":"1825e2b1f7c7bd88241bf37580655489360f9bebc806edd00837a52ce4ad83a0","compiler_version":"v0.80.9","strict":true,"agent_id":"copilot","agent_model":"claude-opus-4.8","engine_versions":{"copilot":"1.0.63"}} # gh-aw-manifest: {"version":1,"secrets":["COPILOT_PAT_0","COPILOT_PAT_1","COPILOT_PAT_2","COPILOT_PAT_3","COPILOT_PAT_4","COPILOT_PAT_5","COPILOT_PAT_6","COPILOT_PAT_7","COPILOT_PAT_8","COPILOT_PAT_9","GH_AW_CI_TRIGGER_TOKEN","GH_AW_GITHUB_MCP_SERVER_TOKEN","GH_AW_GITHUB_TOKEN","GITHUB_TOKEN"],"actions":[{"repo":"actions/checkout","sha":"34e114876b0b11c390a56381ad16ebd13914f8d5","version":"v4"},{"repo":"actions/checkout","sha":"9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0","version":"v7.0.0"},{"repo":"actions/checkout","sha":"de0fac2e4500dabe0009e67214ff5f5447ce83dd","version":"v6.0.2"},{"repo":"actions/download-artifact","sha":"3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c","version":"v8.0.1"},{"repo":"actions/github-script","sha":"3a2844b7e9c422d3c10d287c895573f7108da1b3","version":"v9.0.0"},{"repo":"actions/setup-node","sha":"48b55a011bda9f5d6aeb4c2d9c7362e8dae4041e","version":"v6.4.0"},{"repo":"actions/upload-artifact","sha":"043fb46d1a93c77aae656e7c1c64a875d1fc6a0a","version":"v7.0.1"},{"repo":"github/gh-aw-actions/setup","sha":"8c7d04ebf1ece56cd381446125da3e0f6896294a","version":"v0.80.9"}],"containers":[{"image":"ghcr.io/github/gh-aw-firewall/agent:0.27.7","digest":"sha256:aae231e4635c8999d039c132f1602d3df850fe9b84a00aa2b5ac981179b5661c","pinned_image":"ghcr.io/github/gh-aw-firewall/agent:0.27.7@sha256:aae231e4635c8999d039c132f1602d3df850fe9b84a00aa2b5ac981179b5661c"},{"image":"ghcr.io/github/gh-aw-firewall/api-proxy:0.27.7","digest":"sha256:009caf2e3d88fa77b64e9a03a95a228fc58db0f1701c6d324b29ba5a3c7c79b6","pinned_image":"ghcr.io/github/gh-aw-firewall/api-proxy:0.27.7@sha256:009caf2e3d88fa77b64e9a03a95a228fc58db0f1701c6d324b29ba5a3c7c79b6"},{"image":"ghcr.io/github/gh-aw-firewall/squid:0.27.7","digest":"sha256:deb1d4e19de62d51cee0508057a596a19315c3423ada4d675cad136dc8037c96","pinned_image":"ghcr.io/github/gh-aw-firewall/squid:0.27.7@sha256:deb1d4e19de62d51cee0508057a596a19315c3423ada4d675cad136dc8037c96"},{"image":"ghcr.io/github/gh-aw-mcpg:v0.3.27","digest":"sha256:fe984bddde4ec05d756d9043edb0a32912e6b7b72f6a121b1082f29221421cc7","pinned_image":"ghcr.io/github/gh-aw-mcpg:v0.3.27@sha256:fe984bddde4ec05d756d9043edb0a32912e6b7b72f6a121b1082f29221421cc7"},{"image":"ghcr.io/github/gh-aw-node","digest":"sha256:529d02eb970b1161aa25c593a9c3df57fdfad5a8add328cb3b6eccef66f3183b","pinned_image":"ghcr.io/github/gh-aw-node@sha256:529d02eb970b1161aa25c593a9c3df57fdfad5a8add328cb3b6eccef66f3183b"},{"image":"ghcr.io/github/github-mcp-server:v1.4.0","digest":"sha256:2afb26356481d1a350e14544a6e160f7f7ec1561a1ea309b823665abf0309036","pinned_image":"ghcr.io/github/github-mcp-server:v1.4.0@sha256:2afb26356481d1a350e14544a6e160f7f7ec1561a1ea309b823665abf0309036"}]} # This file was automatically generated by gh-aw (v0.80.9). DO NOT EDIT. To debug this workflow, load the skill at https://github.com/github/gh-aw/blob/main/debug.md # @@ -37,7 +37,9 @@ # humans (the open tracking issue is the hand-off surface; a dedicated # [ci-fix][needs-human] PR is planned but currently deferred — see Step 6). CI on # the PR is kicked by a human `/azp run` for now; the loop watches, classifies, -# and re-fixes autonomously between kicks. +# and re-fixes autonomously between kicks. When the SPECIFIC test a PR fixed is +# confirmed green in that PR's own CI, the loop marks the draft PR ready for review +# (a state transition only — it never approves or merges). # Never mutes tests, but # de-flakes genuinely flaky ones (deterministic synchronization, no retries / # timeout bumps). Always skips visual-regression / screenshot issues. @@ -273,24 +275,24 @@ jobs: run: | bash "${RUNNER_TEMP}/gh-aw/actions/create_prompt_first.sh" { - cat << 'GH_AW_PROMPT_25cf75111b670167_EOF' + cat << 'GH_AW_PROMPT_2ac1d0b4083428f8_EOF' - GH_AW_PROMPT_25cf75111b670167_EOF + GH_AW_PROMPT_2ac1d0b4083428f8_EOF cat "${RUNNER_TEMP}/gh-aw/prompts/xpia.md" cat "${RUNNER_TEMP}/gh-aw/prompts/temp_folder_prompt.md" cat "${RUNNER_TEMP}/gh-aw/prompts/markdown.md" cat "${RUNNER_TEMP}/gh-aw/prompts/safe_outputs_prompt.md" - cat << 'GH_AW_PROMPT_25cf75111b670167_EOF' + cat << 'GH_AW_PROMPT_2ac1d0b4083428f8_EOF' - Tools: add_comment(max:3), create_pull_request(max:3), update_pull_request(max:3), push_to_pull_request_branch(max:3), missing_tool, missing_data, noop - GH_AW_PROMPT_25cf75111b670167_EOF + Tools: add_comment(max:6), create_pull_request(max:3), update_pull_request(max:3), mark_pull_request_as_ready_for_review(max:3), add_labels(max:3), push_to_pull_request_branch(max:3), missing_tool, missing_data, noop + GH_AW_PROMPT_2ac1d0b4083428f8_EOF cat "${RUNNER_TEMP}/gh-aw/prompts/safe_outputs_create_pull_request.md" cat "${RUNNER_TEMP}/gh-aw/prompts/safe_outputs_push_to_pr_branch.md" - cat << 'GH_AW_PROMPT_25cf75111b670167_EOF' + cat << 'GH_AW_PROMPT_2ac1d0b4083428f8_EOF' - GH_AW_PROMPT_25cf75111b670167_EOF + GH_AW_PROMPT_2ac1d0b4083428f8_EOF cat "${RUNNER_TEMP}/gh-aw/prompts/mcp_cli_tools_prompt.md" - cat << 'GH_AW_PROMPT_25cf75111b670167_EOF' + cat << 'GH_AW_PROMPT_2ac1d0b4083428f8_EOF' The following GitHub context information is available for this workflow: {{#if github.actor}} @@ -332,12 +334,12 @@ jobs: stop immediately and report the limitation rather than spending turns trying to work around it. - GH_AW_PROMPT_25cf75111b670167_EOF + GH_AW_PROMPT_2ac1d0b4083428f8_EOF cat "${RUNNER_TEMP}/gh-aw/prompts/github_mcp_tools_with_safeoutputs_prompt.md" - cat << 'GH_AW_PROMPT_25cf75111b670167_EOF' + cat << 'GH_AW_PROMPT_2ac1d0b4083428f8_EOF' {{#runtime-import .github/workflows/ci-status-fix.md}} - GH_AW_PROMPT_25cf75111b670167_EOF + GH_AW_PROMPT_2ac1d0b4083428f8_EOF } > "$GH_AW_PROMPT" - name: Interpolate variables and render templates uses: actions/github-script@3a2844b7e9c422d3c10d287c895573f7108da1b3 # v9.0.0 @@ -554,16 +556,18 @@ jobs: mkdir -p "${RUNNER_TEMP}/gh-aw/safeoutputs" mkdir -p /tmp/gh-aw/safeoutputs mkdir -p /tmp/gh-aw/mcp-logs/safeoutputs - cat > "${RUNNER_TEMP}/gh-aw/safeoutputs/config.json" << 'GH_AW_SAFE_OUTPUTS_CONFIG_fa2a69fad5fd0485_EOF' - {"add_comment":{"discussions":false,"max":3,"required_labels":["agentic-workflows"],"required_title_prefix":"[ci-fix] ","target":"*"},"create_pull_request":{"allowed_base_branches":["main"],"allowed_branches":["ci-fix/**"],"allowed_files":["src/Core/**","src/Controls/**","src/Essentials/**","src/BlazorWebView/**","src/TestUtils/**","src/Templates/**","**/PublicAPI.Unshipped.txt"],"base_branch":"main","draft":true,"labels":["agentic-workflows"],"max":3,"max_patch_files":100,"max_patch_size":4096,"protect_top_level_dot_folders":true,"protected_files":["package.json","bun.lockb","bunfig.toml","deno.json","deno.jsonc","deno.lock","global.json","NuGet.Config","Directory.Packages.props","mix.exs","mix.lock","go.mod","go.sum","stack.yaml","stack.yaml.lock","pom.xml","build.gradle","build.gradle.kts","settings.gradle","settings.gradle.kts","gradle.properties","package-lock.json","yarn.lock","pnpm-lock.yaml","npm-shrinkwrap.json","requirements.txt","Pipfile","Pipfile.lock","pyproject.toml","setup.py","setup.cfg","Gemfile","Gemfile.lock","uv.lock","CODEOWNERS","DESIGN.md","README.md","CONTRIBUTING.md","CHANGELOG.md","SECURITY.md","CODE_OF_CONDUCT.md","AGENTS.md","CLAUDE.md","GEMINI.md"],"protected_files_policy":"request_review","title_prefix":"[ci-fix] "},"create_report_incomplete_issue":{},"missing_data":{},"missing_tool":{},"noop":{"max":1,"report-as-issue":"true"},"push_to_pull_request_branch":{"allowed_files":["src/Core/**","src/Controls/**","src/Essentials/**","src/BlazorWebView/**","src/TestUtils/**","src/Templates/**","**/PublicAPI.Unshipped.txt"],"if_no_changes":"warn","max":3,"max_patch_size":4096,"protect_top_level_dot_folders":true,"protected_files":["package.json","bun.lockb","bunfig.toml","deno.json","deno.jsonc","deno.lock","global.json","NuGet.Config","Directory.Packages.props","mix.exs","mix.lock","go.mod","go.sum","stack.yaml","stack.yaml.lock","pom.xml","build.gradle","build.gradle.kts","settings.gradle","settings.gradle.kts","gradle.properties","package-lock.json","yarn.lock","pnpm-lock.yaml","npm-shrinkwrap.json","requirements.txt","Pipfile","Pipfile.lock","pyproject.toml","setup.py","setup.cfg","Gemfile","Gemfile.lock","uv.lock","CODEOWNERS","DESIGN.md","README.md","CONTRIBUTING.md","CHANGELOG.md","SECURITY.md","CODE_OF_CONDUCT.md","AGENTS.md","CLAUDE.md","GEMINI.md"],"required_labels":["agentic-workflows"],"target":"*","title_prefix":"[ci-fix] "},"report_incomplete":{},"update_pull_request":{"allow_body":true,"allow_title":false,"max":3,"target":"*","update_branch":false}} - GH_AW_SAFE_OUTPUTS_CONFIG_fa2a69fad5fd0485_EOF + cat > "${RUNNER_TEMP}/gh-aw/safeoutputs/config.json" << 'GH_AW_SAFE_OUTPUTS_CONFIG_0644bf1434ca0071_EOF' + {"add_comment":{"discussions":false,"max":6,"required_labels":["agentic-workflows"],"required_title_prefix":"[ci-fix] ","target":"*"},"add_labels":{"allowed":["p/0"],"max":3,"required_labels":["agentic-workflows"],"required_title_prefix":"[ci-fix] ","target":"*"},"create_pull_request":{"allowed_base_branches":["main"],"allowed_branches":["ci-fix/**"],"allowed_files":["src/Core/**","src/Controls/**","src/Essentials/**","src/BlazorWebView/**","src/TestUtils/**","src/Templates/**","**/PublicAPI.Unshipped.txt"],"base_branch":"main","draft":true,"labels":["agentic-workflows"],"max":3,"max_patch_files":100,"max_patch_size":4096,"protect_top_level_dot_folders":true,"protected_files":["package.json","bun.lockb","bunfig.toml","deno.json","deno.jsonc","deno.lock","global.json","NuGet.Config","Directory.Packages.props","mix.exs","mix.lock","go.mod","go.sum","stack.yaml","stack.yaml.lock","pom.xml","build.gradle","build.gradle.kts","settings.gradle","settings.gradle.kts","gradle.properties","package-lock.json","yarn.lock","pnpm-lock.yaml","npm-shrinkwrap.json","requirements.txt","Pipfile","Pipfile.lock","pyproject.toml","setup.py","setup.cfg","Gemfile","Gemfile.lock","uv.lock","CODEOWNERS","DESIGN.md","README.md","CONTRIBUTING.md","CHANGELOG.md","SECURITY.md","CODE_OF_CONDUCT.md","AGENTS.md","CLAUDE.md","GEMINI.md"],"protected_files_policy":"request_review","title_prefix":"[ci-fix] "},"create_report_incomplete_issue":{},"mark_pull_request_as_ready_for_review":{"max":3,"required_labels":["agentic-workflows"],"required_title_prefix":"[ci-fix] ","target":"*"},"missing_data":{},"missing_tool":{},"noop":{"max":1,"report-as-issue":"true"},"push_to_pull_request_branch":{"allowed_files":["src/Core/**","src/Controls/**","src/Essentials/**","src/BlazorWebView/**","src/TestUtils/**","src/Templates/**","**/PublicAPI.Unshipped.txt"],"if_no_changes":"warn","max":3,"max_patch_size":4096,"protect_top_level_dot_folders":true,"protected_files":["package.json","bun.lockb","bunfig.toml","deno.json","deno.jsonc","deno.lock","global.json","NuGet.Config","Directory.Packages.props","mix.exs","mix.lock","go.mod","go.sum","stack.yaml","stack.yaml.lock","pom.xml","build.gradle","build.gradle.kts","settings.gradle","settings.gradle.kts","gradle.properties","package-lock.json","yarn.lock","pnpm-lock.yaml","npm-shrinkwrap.json","requirements.txt","Pipfile","Pipfile.lock","pyproject.toml","setup.py","setup.cfg","Gemfile","Gemfile.lock","uv.lock","CODEOWNERS","DESIGN.md","README.md","CONTRIBUTING.md","CHANGELOG.md","SECURITY.md","CODE_OF_CONDUCT.md","AGENTS.md","CLAUDE.md","GEMINI.md"],"required_labels":["agentic-workflows"],"target":"*","title_prefix":"[ci-fix] "},"report_incomplete":{},"update_pull_request":{"allow_body":true,"allow_title":false,"max":3,"target":"*","update_branch":false}} + GH_AW_SAFE_OUTPUTS_CONFIG_0644bf1434ca0071_EOF - name: Generate Safe Outputs Tools env: GH_AW_TOOLS_META_JSON: | { "description_suffixes": { - "add_comment": " CONSTRAINTS: Maximum 3 comment(s) can be added. Target: *. Supports reply_to_id for discussion threading.", + "add_comment": " CONSTRAINTS: Maximum 6 comment(s) can be added. Target: *. Supports reply_to_id for discussion threading.", + "add_labels": " CONSTRAINTS: Maximum 3 label(s) can be added. Only these labels are allowed: [\"p/0\"]. Target: *.", "create_pull_request": " CONSTRAINTS: Maximum 3 pull request(s) can be created. Title will be prefixed with \"[ci-fix] \". Labels [\"agentic-workflows\"] will be automatically added. Only these labels are allowed: [\"agentic-workflows\"]. PRs will be created as drafts.", + "mark_pull_request_as_ready_for_review": " CONSTRAINTS: Maximum 3 pull request(s) can be marked as ready for review.", "push_to_pull_request_branch": " CONSTRAINTS: Maximum 3 push(es) can be made. The target pull request title must start with \"[ci-fix] \".", "update_pull_request": " CONSTRAINTS: Maximum 3 pull request(s) can be updated. Target: *." }, @@ -594,6 +598,25 @@ jobs: } } }, + "add_labels": { + "defaultMax": 5, + "fields": { + "item_number": { + "issueNumberOrTemporaryId": true + }, + "labels": { + "required": true, + "type": "array", + "itemType": "string", + "itemSanitize": true, + "itemMaxLength": 128 + }, + "repo": { + "type": "string", + "maxLength": 256 + } + } + }, "create_pull_request": { "defaultMax": 1, "fields": { @@ -635,6 +658,24 @@ jobs: } } }, + "mark_pull_request_as_ready_for_review": { + "defaultMax": 1, + "fields": { + "pull_request_number": { + "issueOrPRNumber": true + }, + "reason": { + "required": true, + "type": "string", + "sanitize": true, + "maxLength": 65000 + }, + "repo": { + "type": "string", + "maxLength": 256 + } + } + }, "missing_data": { "defaultMax": 20, "fields": { @@ -1494,7 +1535,7 @@ jobs: uses: actions/github-script@3a2844b7e9c422d3c10d287c895573f7108da1b3 # v9.0.0 env: WORKFLOW_NAME: "CI Failure Fixer (main)" - WORKFLOW_DESCRIPTION: "Periodic pass over open ci-scan tracking issues filed by the main-branch CI\nfailure scanner (.github/workflows/ci-status-main.md). This workflow targets\nthe `main` branch EXCLUSIVELY: it processes only issues labelled ci-scan and\nopens every PR against main. (The net11.0 branch is handled by the parallel\n.github/workflows/ci-status-fix-net11.md — the two are split because gh-aw can\nonly transport a fix relative to ONE static base branch per workflow, and the\nmain↔net11.0 divergence exceeds gh-aw's 10 MB transport-patch cap.) The fixer\nopens ONE draft [ci-fix] PR per actionable issue against main, then WATCHES\nthat PR's own CI on later runs: when the fix's CI comes back red and the red is\ncaused by the fix itself, it pushes a fresh follow-up fix onto the SAME PR\nbranch (never a second PR) — up to 10 attempts — then stops and defers to\nhumans (the open tracking issue is the hand-off surface; a dedicated\n[ci-fix][needs-human] PR is planned but currently deferred — see Step 6). CI on\nthe PR is kicked by a human `/azp run` for now; the loop watches, classifies,\nand re-fixes autonomously between kicks.\nNever mutes tests, but\nde-flakes genuinely flaky ones (deterministic synchronization, no retries /\ntimeout bumps). Always skips visual-regression / screenshot issues." + WORKFLOW_DESCRIPTION: "Periodic pass over open ci-scan tracking issues filed by the main-branch CI\nfailure scanner (.github/workflows/ci-status-main.md). This workflow targets\nthe `main` branch EXCLUSIVELY: it processes only issues labelled ci-scan and\nopens every PR against main. (The net11.0 branch is handled by the parallel\n.github/workflows/ci-status-fix-net11.md — the two are split because gh-aw can\nonly transport a fix relative to ONE static base branch per workflow, and the\nmain↔net11.0 divergence exceeds gh-aw's 10 MB transport-patch cap.) The fixer\nopens ONE draft [ci-fix] PR per actionable issue against main, then WATCHES\nthat PR's own CI on later runs: when the fix's CI comes back red and the red is\ncaused by the fix itself, it pushes a fresh follow-up fix onto the SAME PR\nbranch (never a second PR) — up to 10 attempts — then stops and defers to\nhumans (the open tracking issue is the hand-off surface; a dedicated\n[ci-fix][needs-human] PR is planned but currently deferred — see Step 6). CI on\nthe PR is kicked by a human `/azp run` for now; the loop watches, classifies,\nand re-fixes autonomously between kicks. When the SPECIFIC test a PR fixed is\nconfirmed green in that PR's own CI, the loop marks the draft PR ready for review\n(a state transition only — it never approves or merges).\nNever mutes tests, but\nde-flakes genuinely flaky ones (deterministic synchronization, no retries /\ntimeout bumps). Always skips visual-regression / screenshot issues." HAS_PATCH: ${{ needs.agent.outputs.has_patch }} with: script: | @@ -1910,7 +1951,7 @@ jobs: GH_AW_ALLOWED_DOMAINS: "*.blob.core.windows.net,*.githubusercontent.com,api.business.githubcopilot.com,api.enterprise.githubcopilot.com,api.github.com,api.githubcopilot.com,api.individual.githubcopilot.com,api.snapcraft.io,archive.ubuntu.com,azure.archive.ubuntu.com,codeload.github.com,crl.geotrust.com,crl.globalsign.com,crl.identrust.com,crl.sectigo.com,crl.thawte.com,crl.usertrust.com,crl.verisign.com,crl3.digicert.com,crl4.digicert.com,crls.ssl.com,dev.azure.com,docs.github.com,github-cloud.githubusercontent.com,github-cloud.s3.amazonaws.com,github.blog,github.com,github.githubassets.com,helix.dot.net,host.docker.internal,json-schema.org,json.schemastore.org,keyserver.ubuntu.com,lfs.github.com,objects.githubusercontent.com,ocsp.digicert.com,ocsp.geotrust.com,ocsp.globalsign.com,ocsp.identrust.com,ocsp.sectigo.com,ocsp.ssl.com,ocsp.thawte.com,ocsp.usertrust.com,ocsp.verisign.com,packagecloud.io,packages.cloud.google.com,packages.microsoft.com,patch-diff.githubusercontent.com,ppa.launchpad.net,raw.githubusercontent.com,registry.npmjs.org,s.symcb.com,s.symcd.com,security.ubuntu.com,telemetry.enterprise.githubcopilot.com,ts-crl.ws.symantec.com,ts-ocsp.ws.symantec.com,www.googleapis.com" GITHUB_SERVER_URL: ${{ github.server_url }} GITHUB_API_URL: ${{ github.api_url }} - GH_AW_SAFE_OUTPUTS_HANDLER_CONFIG: "{\"add_comment\":{\"discussions\":false,\"max\":3,\"required_labels\":[\"agentic-workflows\"],\"required_title_prefix\":\"[ci-fix] \",\"target\":\"*\"},\"create_pull_request\":{\"allowed_base_branches\":[\"main\"],\"allowed_branches\":[\"ci-fix/**\"],\"allowed_files\":[\"src/Core/**\",\"src/Controls/**\",\"src/Essentials/**\",\"src/BlazorWebView/**\",\"src/TestUtils/**\",\"src/Templates/**\",\"**/PublicAPI.Unshipped.txt\"],\"base_branch\":\"main\",\"draft\":true,\"labels\":[\"agentic-workflows\"],\"max\":3,\"max_patch_files\":100,\"max_patch_size\":4096,\"protect_top_level_dot_folders\":true,\"protected_files\":[\"package.json\",\"bun.lockb\",\"bunfig.toml\",\"deno.json\",\"deno.jsonc\",\"deno.lock\",\"global.json\",\"NuGet.Config\",\"Directory.Packages.props\",\"mix.exs\",\"mix.lock\",\"go.mod\",\"go.sum\",\"stack.yaml\",\"stack.yaml.lock\",\"pom.xml\",\"build.gradle\",\"build.gradle.kts\",\"settings.gradle\",\"settings.gradle.kts\",\"gradle.properties\",\"package-lock.json\",\"yarn.lock\",\"pnpm-lock.yaml\",\"npm-shrinkwrap.json\",\"requirements.txt\",\"Pipfile\",\"Pipfile.lock\",\"pyproject.toml\",\"setup.py\",\"setup.cfg\",\"Gemfile\",\"Gemfile.lock\",\"uv.lock\",\"CODEOWNERS\",\"DESIGN.md\",\"README.md\",\"CONTRIBUTING.md\",\"CHANGELOG.md\",\"SECURITY.md\",\"CODE_OF_CONDUCT.md\",\"AGENTS.md\",\"CLAUDE.md\",\"GEMINI.md\"],\"protected_files_policy\":\"request_review\",\"title_prefix\":\"[ci-fix] \"},\"create_report_incomplete_issue\":{},\"missing_data\":{},\"missing_tool\":{},\"noop\":{\"max\":1,\"report-as-issue\":\"true\"},\"push_to_pull_request_branch\":{\"allowed_files\":[\"src/Core/**\",\"src/Controls/**\",\"src/Essentials/**\",\"src/BlazorWebView/**\",\"src/TestUtils/**\",\"src/Templates/**\",\"**/PublicAPI.Unshipped.txt\"],\"if_no_changes\":\"warn\",\"max\":3,\"max_patch_size\":4096,\"protect_top_level_dot_folders\":true,\"protected_files\":[\"package.json\",\"bun.lockb\",\"bunfig.toml\",\"deno.json\",\"deno.jsonc\",\"deno.lock\",\"global.json\",\"NuGet.Config\",\"Directory.Packages.props\",\"mix.exs\",\"mix.lock\",\"go.mod\",\"go.sum\",\"stack.yaml\",\"stack.yaml.lock\",\"pom.xml\",\"build.gradle\",\"build.gradle.kts\",\"settings.gradle\",\"settings.gradle.kts\",\"gradle.properties\",\"package-lock.json\",\"yarn.lock\",\"pnpm-lock.yaml\",\"npm-shrinkwrap.json\",\"requirements.txt\",\"Pipfile\",\"Pipfile.lock\",\"pyproject.toml\",\"setup.py\",\"setup.cfg\",\"Gemfile\",\"Gemfile.lock\",\"uv.lock\",\"CODEOWNERS\",\"DESIGN.md\",\"README.md\",\"CONTRIBUTING.md\",\"CHANGELOG.md\",\"SECURITY.md\",\"CODE_OF_CONDUCT.md\",\"AGENTS.md\",\"CLAUDE.md\",\"GEMINI.md\"],\"required_labels\":[\"agentic-workflows\"],\"target\":\"*\",\"title_prefix\":\"[ci-fix] \"},\"report_incomplete\":{},\"update_pull_request\":{\"allow_body\":true,\"allow_title\":false,\"max\":3,\"target\":\"*\",\"update_branch\":false}}" + GH_AW_SAFE_OUTPUTS_HANDLER_CONFIG: "{\"add_comment\":{\"discussions\":false,\"max\":6,\"required_labels\":[\"agentic-workflows\"],\"required_title_prefix\":\"[ci-fix] \",\"target\":\"*\"},\"add_labels\":{\"allowed\":[\"p/0\"],\"max\":3,\"required_labels\":[\"agentic-workflows\"],\"required_title_prefix\":\"[ci-fix] \",\"target\":\"*\"},\"create_pull_request\":{\"allowed_base_branches\":[\"main\"],\"allowed_branches\":[\"ci-fix/**\"],\"allowed_files\":[\"src/Core/**\",\"src/Controls/**\",\"src/Essentials/**\",\"src/BlazorWebView/**\",\"src/TestUtils/**\",\"src/Templates/**\",\"**/PublicAPI.Unshipped.txt\"],\"base_branch\":\"main\",\"draft\":true,\"labels\":[\"agentic-workflows\"],\"max\":3,\"max_patch_files\":100,\"max_patch_size\":4096,\"protect_top_level_dot_folders\":true,\"protected_files\":[\"package.json\",\"bun.lockb\",\"bunfig.toml\",\"deno.json\",\"deno.jsonc\",\"deno.lock\",\"global.json\",\"NuGet.Config\",\"Directory.Packages.props\",\"mix.exs\",\"mix.lock\",\"go.mod\",\"go.sum\",\"stack.yaml\",\"stack.yaml.lock\",\"pom.xml\",\"build.gradle\",\"build.gradle.kts\",\"settings.gradle\",\"settings.gradle.kts\",\"gradle.properties\",\"package-lock.json\",\"yarn.lock\",\"pnpm-lock.yaml\",\"npm-shrinkwrap.json\",\"requirements.txt\",\"Pipfile\",\"Pipfile.lock\",\"pyproject.toml\",\"setup.py\",\"setup.cfg\",\"Gemfile\",\"Gemfile.lock\",\"uv.lock\",\"CODEOWNERS\",\"DESIGN.md\",\"README.md\",\"CONTRIBUTING.md\",\"CHANGELOG.md\",\"SECURITY.md\",\"CODE_OF_CONDUCT.md\",\"AGENTS.md\",\"CLAUDE.md\",\"GEMINI.md\"],\"protected_files_policy\":\"request_review\",\"title_prefix\":\"[ci-fix] \"},\"create_report_incomplete_issue\":{},\"mark_pull_request_as_ready_for_review\":{\"max\":3,\"required_labels\":[\"agentic-workflows\"],\"required_title_prefix\":\"[ci-fix] \",\"target\":\"*\"},\"missing_data\":{},\"missing_tool\":{},\"noop\":{\"max\":1,\"report-as-issue\":\"true\"},\"push_to_pull_request_branch\":{\"allowed_files\":[\"src/Core/**\",\"src/Controls/**\",\"src/Essentials/**\",\"src/BlazorWebView/**\",\"src/TestUtils/**\",\"src/Templates/**\",\"**/PublicAPI.Unshipped.txt\"],\"if_no_changes\":\"warn\",\"max\":3,\"max_patch_size\":4096,\"protect_top_level_dot_folders\":true,\"protected_files\":[\"package.json\",\"bun.lockb\",\"bunfig.toml\",\"deno.json\",\"deno.jsonc\",\"deno.lock\",\"global.json\",\"NuGet.Config\",\"Directory.Packages.props\",\"mix.exs\",\"mix.lock\",\"go.mod\",\"go.sum\",\"stack.yaml\",\"stack.yaml.lock\",\"pom.xml\",\"build.gradle\",\"build.gradle.kts\",\"settings.gradle\",\"settings.gradle.kts\",\"gradle.properties\",\"package-lock.json\",\"yarn.lock\",\"pnpm-lock.yaml\",\"npm-shrinkwrap.json\",\"requirements.txt\",\"Pipfile\",\"Pipfile.lock\",\"pyproject.toml\",\"setup.py\",\"setup.cfg\",\"Gemfile\",\"Gemfile.lock\",\"uv.lock\",\"CODEOWNERS\",\"DESIGN.md\",\"README.md\",\"CONTRIBUTING.md\",\"CHANGELOG.md\",\"SECURITY.md\",\"CODE_OF_CONDUCT.md\",\"AGENTS.md\",\"CLAUDE.md\",\"GEMINI.md\"],\"required_labels\":[\"agentic-workflows\"],\"target\":\"*\",\"title_prefix\":\"[ci-fix] \"},\"report_incomplete\":{},\"update_pull_request\":{\"allow_body\":true,\"allow_title\":false,\"max\":3,\"target\":\"*\",\"update_branch\":false}}" GH_AW_CI_TRIGGER_TOKEN: ${{ secrets.GH_AW_CI_TRIGGER_TOKEN }} with: github-token: ${{ secrets.GH_AW_GITHUB_TOKEN || secrets.GITHUB_TOKEN }} diff --git a/.github/workflows/ci-status-fix.md b/.github/workflows/ci-status-fix.md index d3227e25170c..9c2f46054029 100644 --- a/.github/workflows/ci-status-fix.md +++ b/.github/workflows/ci-status-fix.md @@ -15,7 +15,9 @@ description: | humans (the open tracking issue is the hand-off surface; a dedicated [ci-fix][needs-human] PR is planned but currently deferred — see Step 6). CI on the PR is kicked by a human `/azp run` for now; the loop watches, classifies, - and re-fixes autonomously between kicks. + and re-fixes autonomously between kicks. When the SPECIFIC test a PR fixed is + confirmed green in that PR's own CI, the loop marks the draft PR ready for review + (a state transition only — it never approves or merges). Never mutes tests, but de-flakes genuinely flaky ones (deterministic synchronization, no retries / timeout bumps). Always skips visual-regression / screenshot issues. @@ -239,8 +241,15 @@ safe-outputs: # PER-RUN total (not per-PR): a sweep may surface several green PRs and/or # annotate several flaky reds in one run. At max:1 all but the first comment # were silently dropped, starving the workflow's #1 value (surfacing green PRs - # for review). Raised to 3 to match the per-run throughput of the other outputs. - max: 3 + # for review). Sized to 6 (not 3) because a single green DRAFT PR that gets + # flipped ready in one sweep spends TWO comment slots — the Step 3 ✅ surface + # comment (which still names the /azp-gated legs a human must kick, so it is NOT + # redundant with 🎯) AND the Step 3.6 T3 🎯 readiness comment. At max:3 the shared + # bucket drained after ~1 draft flip and T3's atomicity pre-check then deferred + # every further mark-ready even while the mark-ready/add_labels buckets (also 3) + # sat idle; 6 lets ~3 draft flips (2 comments each) land per sweep, so the + # mark-ready:3 / add_labels:3 caps are actually reachable. + max: 6 target: "*" # Hard constraint (defense-in-depth): only comment on THIS workflow's own # ci-fix PRs — [ci-fix] title prefix AND agentic-workflows label. Mirrors the @@ -261,13 +270,51 @@ safe-outputs: # never the title, so disable title rewrites — this compiles to allow_title:false # and removes any ability to retitle an arbitrary PR. title: false - # NOTE: gh-aw v0.79.8 does NOT emit required-title-prefix/required-labels into + # NOTE: gh-aw v0.80.9 (the pinned compiler) does NOT emit required-title-prefix/required-labels into # the compiled config for update-pull-request (verified against the lock — it # silently drops them, unlike add-comment / push-to-pull-request-branch which # honor them). So which-PR scoping here relies on prompt Hard-Rule 6 + # min-integrity:approved; the residual is body-marker edits only (no code, no # merge, no close), capped at max:3. Revisit required-* if a future gh-aw # compiler emits them for this output. + mark-pull-request-as-ready-for-review: + # Flip a validated draft [ci-fix] PR to ready-for-review once the SPECIFIC test + # this PR was opened to fix is confirmed green in the PR's OWN CI (Step 3.6) — + # even when unrelated legs are red. The workflow is schedule/dispatch-triggered + # (no triggering-PR context), so target "*" lets the agent name the PR number it + # validated. This is a state transition ONLY: it never approves and never merges — + # a human still reviews and merges. + # PER-RUN total (not per-PR): a sweep may validate several PRs' target tests green. + max: 3 + target: "*" + # Hard constraint (defense-in-depth): only ever un-draft THIS workflow's own + # ci-fix PRs — [ci-fix] title prefix AND agentic-workflows label — so a confused + # or prompt-injected agent cannot mark an arbitrary PR ready. (If the v0.80.9 + # compiler silently drops these — as documented for update-pull-request above — + # the Step 3.6 preconditions + min-integrity:approved are the compensating scope + # controls; verify against the lock after compiling.) + required-title-prefix: "[ci-fix] " + required-labels: [agentic-workflows] + add-labels: + # Apply the p/0 priority label to a [ci-fix] PR at the exact moment the loop flips it + # from draft to ready-for-review (Step 3.6 T3) — so a validated, review-ready fix lands + # in the team's p/0 triage queue instead of sitting unseen in the draft backlog. Paired + # 1:1 with mark-pull-request-as-ready-for-review; target "*" lets the agent name the PR + # it just validated. + # allowed = HARD allowlist: the agent may ONLY ever add p/0, nothing else. This caps the + # blast radius of a confused/prompt-injected agent to exactly one benign priority label — + # it can never apply a downstream-triggering or destructive label. + allowed: [p/0] + # PER-RUN total (not per-PR): a sweep may mark several PRs ready in one run; each adds + # one label, so this matches the mark-ready per-run cap (3). + max: 3 + target: "*" + # Hard constraint (defense-in-depth): only ever label THIS workflow's own ci-fix PRs — + # [ci-fix] title prefix AND agentic-workflows label. (If the v0.80.9 compiler drops these + # — as documented for update-pull-request above — the Step 3.6 preconditions + + # min-integrity:approved + the allowed:[p/0] allowlist are the compensating controls.) + required-title-prefix: "[ci-fix] " + required-labels: [agentic-workflows] timeout-minutes: 90 @@ -339,33 +386,48 @@ through `safe-outputs`. 5. **One issue = one outcome per run.** Exactly one of: respond to a maintainer's change-request on the open PR (Track C, Step 3.5.R); open the first fix/help/ de-flake PR; advance an existing PR by one attempt (push a follow-up fix); - surface a validated-green PR for review; annotate an unrelated-flake red; wait + surface a validated-green PR for review; mark a target-validated draft PR + ready-for-review (Step 3.6 T3 — the terminal outcome that supersedes the same + run's surface/annotate precursor line), or record its `already-ready` + steady-state no-op; annotate an unrelated-flake red; wait (CI not yet settled); or a recorded skip (the dedicated needs-human PR is deferred — the attempt cap records a skip instead; see Step 6). Always prefer advancing or opening a PR over a skip when a non-mute diff is producible. 6. **All writes via `safe-outputs`.** Allowed outputs: `create_pull_request` (first attempt only), `push_to_pull_request_branch` (advance an existing PR by one attempt), `update_pull_request` (bump the attempt marker / refresh the - prior-attempts table), and `add_comment` (progress notes on the `[ci-fix]` PR - ONLY). NEVER comment on the tracking issue (issues are locked by + prior-attempts table), `add_comment` (progress notes on the `[ci-fix]` PR + ONLY), `mark_pull_request_as_ready_for_review` (flip a target-validated draft + `[ci-fix]` PR from draft to ready — Step 3.6 T3 only), and `add_labels` (add + ONLY the `p/0` label, ONLY on that same draft→ready transition — Step 3.6 T3). + NEVER comment on the tracking issue (issues are locked by `.github/workflows/ci-scan-lock-issues.yml`) — `add_comment` targets the PR. No `gh pr create`, no manual `git push`. **Defense-in-depth:** `add_comment` and `update_pull_request` use `target: "*"` (the agent supplies the PR number). - `add_comment` is now config-locked to `required-title-prefix: "[ci-fix] "` + + `add_comment`, `mark_pull_request_as_ready_for_review`, and `add_labels` are + config-locked to `required-title-prefix: "[ci-fix] "` + `required-labels: [agentic-workflows]` (hard-enforced by the handler, same as - `push_to_pull_request_branch`), so a comment can only ever land on THIS - workflow's own PRs. `update_pull_request` CANNOT be config-locked to a - title/label in gh-aw v0.79.8 (the compiler silently drops `required-*` for that - output — verified against the lock), so it keeps `title: false` (no retitles; - body-marker edits only) plus this prompt-level guard. Before emitting either, - VERIFY the target PR carries BOTH the `[ci-fix]` title prefix AND the - `agentic-workflows` label; never comment on or edit the body of any PR that - lacks both. + `push_to_pull_request_branch`), and `add_labels` additionally has an + `allowed: [p/0]` allowlist so it can ONLY ever add `p/0` — so these can only + ever land on THIS workflow's own PRs. `update_pull_request` CANNOT be + config-locked to a title/label in gh-aw v0.80.9 (the compiler silently drops + `required-*` for that output — verified against the lock), so it keeps + `title: false` (no retitles; body-marker edits only) plus this prompt-level + guard. Before emitting ANY of these, VERIFY the target PR carries BOTH the + `[ci-fix]` title prefix AND the `agentic-workflows` label; never comment on, + edit, un-draft, or label any PR that lacks both. 7. **Per-run safe-output caps (per RUN, not per PR).** Each output type is capped per run: `create_pull_request` 3, `push_to_pull_request_branch` 3, - `add_comment` 3, `update_pull_request` 3. Note an ADVANCE spends one push **and** - one update **and** one comment, so ≤ 3 advances/run; surfacing a green or - annotating a flake spends one comment. When a bucket is exhausted, do NOT keep + `add_comment` 6, `update_pull_request` 3, `mark_pull_request_as_ready_for_review` + 3, `add_labels` 3 (counts LABELS, not calls). Note an ADVANCE spends one push + **and** one update **and** one comment, so ≤ 3 advances/run; surfacing a green or + annotating a flake spends one comment; a **mark-ready (Step 3.6 T3)** of a + still-draft PR spends TWO comments (the Step 3 ✅ surface **and** the T3 🎯 + readiness note) **and** one mark-ready **and** one label as an ALL-OR-NOTHING set + (T3 pre-checks those buckets and defers the whole PR if any is exhausted — never + mark a PR ready without its 🎯 audit comment). `add_comment` is therefore sized 6 + (not 3) so ~3 draft flips, at 2 comments each, can land in one sweep instead of the + shared comment bucket starving the otherwise-idle mark-ready/add_labels buckets. When a bucket is exhausted, do NOT keep emitting (extras are silently dropped) — record `skipped: per-run cap reached; deferring PR # to next cycle` for each remaining PR so the drop is a deliberate, logged decision. The continuous loop picks the deferred PRs up next @@ -409,8 +471,9 @@ For every open tracking issue in scope, converge on exactly one outcome: | Open first help-wanted draft `[ci-fix]` PR | No open PR yet; a plausible candidate exists but cannot be runner-validated (device/UI tests) (attempt 1) | | Open first de-flake draft `[ci-fix]` PR | No open PR yet; failure is intermittent (green-on-retry) from a genuine test-quality defect; a deterministic-synchronization fix is producible (attempt 1, Step 4.7 bucket b) | | Advance an existing PR (push attempt N+1) | Open PR's own CI settled red *because of the fix itself*, no human engaged, marker < 10 — push a NEW distinct fix onto the same branch (Step 3.5 → 5.6) | -| Surface a validated-green PR | Open PR's own CI settled green — comment "validated, attempt N/10; ready for review" and do NOT advance (Step 3.5) | -| Annotate an unrelated-flake red | Open PR is red only on baseline-flake / unrelated legs — comment which leg needs a re-run; do NOT burn an attempt (Step 3.5) | +| Surface a validated-green PR | Open PR's own CI settled green — comment "validated, attempt N/10; ready for review" and do NOT advance (Step 3.5), then mark the draft PR ready for review once the fixed test is confirmed green (Step 3.6) | +| Annotate an unrelated-flake red | Open PR is red only on baseline-flake / unrelated legs — comment which leg needs a re-run; do NOT burn an attempt (Step 3.5); if the SPECIFIC fixed test is confirmed green on that build, still mark the draft PR ready for review (Step 3.6) | +| Mark a validated PR ready for review | The specific test this PR fixed is confirmed green in the PR's own CI (even if unrelated legs are red) — comment the target-test result and transition the draft PR to ready for review; never approve or merge (Step 3.6) | | Wait | Open PR's CI is pending / not yet settled on the current head SHA — do nothing this cycle (Step 3.5) | | Hand-off skip (attempt cap) | Marker == 10 and the signature still reproduces — stop and defer to humans; the dedicated `[ci-fix][needs-human]` PR is planned but currently deferred (Step 6) | | Recorded skip | Visual-regression, human engaged, already-handled, fixed-in-latest-build, infra-flake, out-of-bounds, only-mute-available, or no novel approach producible | @@ -685,14 +748,34 @@ Run these gates in order — the FIRST that fires decides this cycle's outcome: 1. **Human engaged.** If `C.humanEngaged` → `skipped: human engaged on PR #; deferring` and stop. Never fight a human reviewer — the intentional hand-off boundary still holds the moment a person touches the PR. -2. **CI not settled / unknown / incomplete prefetch.** If `C.checksSettled == false` - OR `C.overallConclusion` is `pending`, `neutral`, or `unknown`, OR - `C.dataComplete == false` → the fix's CI has not finished on the current head SHA - (`C.headSha`), or the prefetch could not fully read this PR (a partial read can - understate `humanEngaged` / `attempt`). `skipped: PR # CI pending / prefetch - incomplete on ; waiting` and stop. *(Round 1: this is where a - maintainer `/azp run` is awaited — the `/azp`-gated uitests/devicetests legs will - not have run until a human kicks them.)* +2. **CI not settled — but validate the target test first (target-focused readiness).** + If `C.checksSettled == false` OR `C.overallConclusion` is `pending`, `neutral`, or + `unknown`, OR `C.dataComplete == false` → the *overall* build has not finished on the + current head SHA (`C.headSha`), or the prefetch could not fully read this PR (a partial + read can understate `humanEngaged` / `attempt`). Unrelated legs still draining must NOT + keep an already-proven fix parked as a draft, so before waiting, try the target-focused + fast-path: + - **2a — Target-green fast-path (draft `[ci-fix]` PRs only).** If `C.dataComplete == + true` AND the Step 3.6 preconditions hold (draft PR, `[ci-fix] ` title prefix AND the + `agentic-workflows` label), run Step 3.6 **T1–T2** against `C.headSha` now: identify + the target test(s) and read the PR's OWN build **timeline / per-leg status** (anonymous + `_apis/build` — never `_apis/test`, per Hard-Rule 8). If EVERY target test is + **VALIDATED-GREEN** (per T2's all-platform definition below — the target's category leg + is `succeeded` on every platform that runs it, failed on none, and NO target platform + family the pipeline covers is still pending/unconcluded, per T2's "no platform left + unverified" rule; a platform that simply has no leg for the target's category — the test + doesn't run there — is fine, not a gap) AND every leg in + `C.failedLegs` that has ALREADY concluded classifies as **unrelated flake** (Step 4 / + Step 4.7 method — the fix is implicated in NO completed red), then the fix is proven + regardless of unrelated *pending* legs → run **Step 3.6 T3** (mark ready + 🎯 comment) + and stop. This early-out NEVER advances an attempt and NEVER acts on a red that could + be the fix's fault. + - **2b — Otherwise WAIT.** If the target test has not yet executed (pending/absent on + its leg), or a completed red is (or may be) caused by the fix, or `C.dataComplete == + false`, or this PR has no identifiable target test → `skipped: PR # CI pending / + target not yet validated on ; waiting` and stop. *(Round 1: this is where a + maintainer `/azp run` is awaited — the `/azp`-gated uitests/devicetests legs will not + have run until a human kicks them.)* 3. **Green → surface for review.** If `C.overallConclusion == "success"`: the checks that RAN are green. **Primary-gate check first:** the deterministic `success` verdict only certifies "at least one green check, nothing failing or @@ -704,18 +787,31 @@ Run these gates in order — the FIRST that fires decides this cycle's outcome: # primary CI gate (maui-pr) not green on ; waiting` and stop — do NOT surface. (The `/azp`-gated `maui-pr-uitests` (def 313) / `maui-pr-devicetests` (def 314) legs MAY still be un-run — that is expected and is named below, not a - reason to withhold the surface.) **Idempotency:** scan the PR's existing comments - for a prior bot `✅ … validated … on ` note for THIS head SHA — if one - already exists, record `already-surfaced PR # (head )` and stop - WITHOUT re-commenting (re-surfacing the same green every tick is noise and burns - the per-run comment budget other PRs need). Otherwise resolve the attempt number from - `C.effectiveAttempt` (the authoritative max(marker, bot-commit) counter; omit the - number only if it is somehow indeterminate). **Dry-run gate (Step 0):** if `dry_run == "true"`, do NOT emit any comment — instead print the intended `✅` validated-green body to the run log, tally `dry-run: would-surface-green PR #`, and stop. Otherwise `add_comment` on PR #: `✅ Attempt /10 validated - — the fix's CI is green on . Ready for human review.` - Name any `/azp`-gated legs (uitests def 313 / devicetests def 314) that have not - run and still need a maintainer `/azp run`. Do NOT advance. Record - `surfaced-green PR # (attempt /10)` and stop. *(This directly - attacks the real bottleneck — no reviews — so it is the highest-value outcome.)* + reason to withhold the surface.) **Comment idempotency + dry-run suppress the ✅ + comment ONLY — neither skips Step 3.6.** Scan the PR's existing comments for a prior + bot `✅ … validated … on ` note for THIS head SHA; if one exists, set + `SKIP_SURFACE_COMMENT = true`. Resolve the attempt number from `C.effectiveAttempt` + (the authoritative max(marker, bot-commit) counter; omit the number only if it is + somehow indeterminate). Then post the surface comment UNLESS suppressed: + - if `dry_run == "true"`: do NOT emit — print the intended `✅` validated-green body to + the run log and tally `dry-run: would-surface-green PR #`; + - else if `SKIP_SURFACE_COMMENT`: do NOT re-post — record `already-surfaced PR # + (head )` (re-surfacing the same green every tick is noise and burns the + per-run comment budget other PRs need); + - else `add_comment` on PR #: `✅ Attempt /10 validated — the fix's CI is + green on .` naming any `/azp`-gated legs (uitests def 313 / devicetests def + 314) that have not run and still need a maintainer `/azp run`; record `surfaced-green + PR # (attempt /10)`. (Do NOT assert "ready for human review" on a PR that + is still a draft — Step 3.6, not this comment, owns the draft→ready flip and posts its + own 🎯 announcement when it fires.) + Do NOT advance. Then — in ALL of the above cases — run **Step 3.6** (target-test + readiness gate) for this PR before stopping: its `/azp`-gated target legs may conclude + green on this SAME head SHA on a LATER sweep (an `/azp run` adds no commit), and Step + 3.6 is the ONLY place the draft→ready flip happens, so it MUST re-evaluate every sweep — + never `stop` here before it. *(This directly attacks the real bottleneck — no reviews — + so it is the highest-value outcome.)* 4. **Red → classify caused-by-fix vs unrelated-flake.** If `C.overallConclusion == "failure"`, analyze the PR's OWN failing build (NOT `main`): find the AzDO `maui-pr` build for this PR (filter builds by `branchName=refs/pull//merge`, @@ -723,18 +819,26 @@ Run these gates in order — the FIRST that fires decides this cycle's outcome: method and Step 4.7 flake buckets to `C.failedLegs`. - **Unrelated flake only** (every failed leg is known-flaky / infra / a pre-existing baseline red NOT introduced by the fix): do NOT burn an attempt. - **Idempotency first:** scan the PR's existing comments for a prior bot - `♻️ … unrelated flake … on ` note for THIS head SHA — if one already - exists, record `already-annotated-flake PR # (head )` and stop - WITHOUT re-commenting (re-annotating the same flake every 12h sweep is noise and - burns the per-run comment budget other PRs need). Otherwise resolve the attempt - number from `C.effectiveAttempt` (the authoritative max(marker, bot-commit) - counter), omitting the `Attempt /10:` prefix only if it is somehow - indeterminate. **Dry-run gate (Step 0):** if `dry_run == "true"`, do NOT emit any comment — instead print the intended `♻️` unrelated-flake body to the run log, tally `dry-run: would-annotate-flake PR #`, and stop. Otherwise `add_comment` on PR #: `♻️ Attempt /10: red is - unrelated flake on leg(s) () on ; the fix itself is not - implicated. A maintainer re-run (/azp run ) should clear it.` Record - `annotated-flake PR # (head )` and stop. *(Round 1: human re-runs; - Round 2: auto re-trigger.)* + **Comment idempotency + dry-run suppress the ♻️ comment ONLY — neither skips Step + 3.6.** Scan the PR's existing comments for a prior bot `♻️ … unrelated flake … on + ` note for THIS head SHA; if one exists, set `SKIP_FLAKE_COMMENT = true`. + Resolve the attempt number from `C.effectiveAttempt` (the authoritative max(marker, + bot-commit) counter), omitting the `Attempt /10:` prefix only if it is + somehow indeterminate. Then post the flake note UNLESS suppressed: + - if `dry_run == "true"`: do NOT emit — print the intended `♻️` unrelated-flake body + to the run log and tally `dry-run: would-annotate-flake PR #`; + - else if `SKIP_FLAKE_COMMENT`: do NOT re-post — record `already-annotated-flake PR + # (head )` (re-annotating the same flake every 12h sweep is noise and + burns the per-run comment budget other PRs need); + - else `add_comment` on PR #: `♻️ Attempt /10: red is unrelated flake on + leg(s) () on ; the fix itself is not implicated. A + maintainer re-run (/azp run ) should clear it.` record `annotated-flake + PR # (head )`. + Then — in ALL of the above cases — run **Step 3.6** (target-test readiness gate) for + this PR before stopping: its `/azp`-gated target legs may conclude green on this SAME + head SHA on a LATER sweep, and Step 3.6 is the ONLY place the draft→ready flip + happens, so it MUST re-evaluate every sweep — never `stop` here before it. *(Round 1: + human re-runs; Round 2: auto re-trigger.)* - **Caused by the fix** (a failed leg still matches the original target signature, or the fix introduced a NEW failure): advance an attempt. - **Attempt count.** `attempt = C.effectiveAttempt` — the authoritative @@ -752,6 +856,188 @@ Run these gates in order — the FIRST that fires decides this cycle's outcome: from every prior commit on this branch**, emitted via the Step 5.6 ADVANCE path (push onto the same PR). +#### Step 3.6 — Target-test verification & mark-ready gate + +Reached from the Step 3 **green-surface** branch, the Step 4 **unrelated-flake** branch +(AFTER that branch has posted its comment), and the Step 3.5 **gate-2 target-focused +fast-path** (2a — while unrelated legs are still pending). Purpose: confirm the SPECIFIC +test(s) this PR was opened to fix now PASS in the PR's own CI, and — only when they do +— transition the draft `[ci-fix]` PR to **ready for review** so a maintainer sees a +validated fix instead of a draft. This is the ONLY place the loop flips draft→ready; it +is a state transition, **never** an approval or a merge (a human still reviews and +merges). Overall red on *unrelated* legs must NOT gate readiness — we validate the fix, +not the base branch's flakiness. + +**Preconditions** (ALL must hold; otherwise record `skipped: readiness N/A PR # +()` and stop this gate): +- `C.isDraft == true` — if the PR is already ready-for-review, the draft→ready + transition is already done; do NOT re-mark **and do NOT re-apply `p/0`**. `p/0` is applied + exactly ONCE, atomically with the draft→ready flip (T3 — all-or-nothing with the 🎯 audit + comment + mark-ready), so an already-ready loop PR that is MISSING `p/0` has almost + certainly had it **removed by a maintainer** de-prioritizing the PR — a pure triage action + the human-engagement guard (which inspects comments/reviews/commits, NOT label changes) + cannot see. Re-adding `p/0` every sweep would fight that maintainer indefinitely, violating + the loop's "never override a human" contract. So the loop does NOT reconcile the label: + record `already-ready PR #` and stop this gate. (Trade-off: on the rare occasion a + transient API error drops `p/0` at flip time *after* the T3 atomic pre-check passed, it is + not auto-re-added — but the PR is still ready-for-review with its 🎯 audit comment, and + re-adding `p/0` is a trivial manual action; that is strictly preferable to steam-rolling a + maintainer's deliberate de-prioritization.) +- The PR is unmistakably THIS workflow's own: `[ci-fix] ` title prefix AND the + `agentic-workflows` label (mirrors the safe-output lock; the compensating scope + control if the v0.80.9 compiler drops the declarative required-*). +- You reached this gate from Step 3 (green), Step 4 (**unrelated-flake**), or the Step 3.5 + gate-2 **target-focused fast-path** (2a) — i.e. the fix is NOT implicated in any red that + has concluded. If Step 4 classified a red as **caused by the fix** (ADVANCE), do NOT run + this gate — advance the attempt instead. + +**T1 — Identify the target test(s).** From the `[ci-scan]` issue signature the fix +addresses (and the PR's own diff), extract the fully-qualified test method name(s) the +fix targets — e.g. `SafeAreaShouldWorkOnAllShellTabs`. For a de-flake it is the +de-flaked test; for a product fix it is the originally-failing test(s). If NO specific +test can be identified (e.g. a product build-break rather than a test failure), this is a +**build-only fix** — there is no single test to validate cross-platform, so validate at +**whole-build** granularity rather than stopping (Step 3 only *comments*; Step 3.6 is the +sole draft→ready flip, so a build-only fix is undrafted HERE or not at all). We only reach +this gate from Step 3 (green) / Step 4 (unrelated-flake) / gate-2a, so the fix is not +implicated in any concluded red; additionally require, via the T2 timeline method on +`C.headSha`, that the primary build pipeline `maui-pr` (def 302) has CONCLUDED with EVERY +one of its platform build legs `succeeded`/`completed` and NONE failed/canceled or still +pending — the build is green on **every** platform, not just the originally-broken one (the +cross-platform guard, applied to the build instead of a test). A build-only fix is undrafted +ONLY on the strength of the auto-running `maui-pr` (def 302) whole-build green, which the +loop CAN observe: if the PR's `[ci-scan]` issue signature or its own diff indicates the +ORIGINATING failure was in a `/azp`-gated pipeline (`maui-pr-uitests` def 313 or +`maui-pr-devicetests` def 314) rather than `maui-pr` (def 302), do NOT undraft on +`maui-pr`-green alone — those pipelines do not auto-run on this PR (GITHUB_TOKEN cannot +trigger them), so a green `maui-pr` build is NOT evidence the gated build break is fixed; +record `skipped: build-only fix PR # targets gated pipeline () not run — +deferring to human` and stop this gate. Otherwise, set `TARGET := "the maui-pr build +(build-only fix — no single target test)"` and proceed to **T3** to mark ready. If any `maui-pr` build leg is still unconcluded, record `skipped: build-only fix PR +# not yet whole-build green (leg(s) pending)` and stop this gate WITHOUT marking +ready (a green subset is not enough — a still-pending build leg could yet fail). + +**T2 — Verify the target test's platform legs are green (anonymous, leg-level).** Per-test +outcomes come from the AzDO **test-results** API (`_apis/test/...`), which is **NOT reachable +anonymously — Hard-Rule 8**: those endpoints 302-redirect to a sign-in page on this runner, so +a `curl` returns an HTML redirect (not JSON) and every target test would look "not executed". +Validate instead at **leg granularity** using ONLY the anonymous `_apis/build/...` timeline +(Hard-Rule 8) — a `succeeded` category leg means every test in that category **that actually +ran** passed on that platform. This is a strong bar **only if the target test genuinely +executes**: a job can go green while one specific test is skipped at runtime (`[Ignore]`, +`Assert.Ignore()`, `Assert.Inconclusive()`, a `Skip=`/conditional `[Fact]`, an `#if`-out, or +an early `return` before the asserts). So before trusting a green leg as proof the target +passed, **confirm from the PR's OWN diff that the fix does NOT skip, ignore, disable, or +short-circuit the target test** (it adds no `[Ignore]`/`[Explicit]`, `Assert.Ignore`/ +`Assert.Inconclusive`, `Skip=`, category-exclusion, `#if`-out, or early `return` around the +target). If the fix could cause the target to be runtime-skipped rather than genuinely pass, +leg-level green is NOT sufficient evidence — record `skipped: target test may be +runtime-skipped by this fix (diff adds ); leg-green insufficient, deferring to human +on PR #` and stop this gate WITHOUT marking ready. Otherwise (the target genuinely runs +and asserts, as a de-flake or product fix does) a green category leg cannot hide a +target-test failure, so treat it as authoritative. + +Map the target test to the CI leg(s) that run it. A leg name encodes platform + test category +(e.g. `Android UITests SafeAreaEdges,Shadow`, `iOS UITests SafeAreaEdges,Shadow`, `macOS +UITests SafeAreaEdges,Shadow`, `Windows UITests SafeAreaEdges,Shadow`). Determine the target +test's UI-test category — its `[Category(UITestCategories.X)]` in the test/HostApp source, +visible in the PR diff or the test file — to know which leg-name substring identifies its legs; +for a device test, the per-platform device-test legs. + +Using the SAME build-discovery as Step 4 (filter AzDO builds by `branchName=refs/pull//merge` +or `sourceVersion == C.headSha`), read each build's **timeline** on `C.headSha` for the +pipeline(s) that RUN the target test — `maui-pr` (def 302) for unit/integration tests, +`maui-pr-uitests` (def 313) for Appium UI tests, `maui-pr-devicetests` (def 314) for device +tests: + +```bash +ORG=dnceng-public; PROJ=public; BUILD= +# Anonymous + Hard-Rule-8-compliant. NEVER call _apis/test/... (it 302-redirects to sign-in). +# Each type=="Job" record is one platform leg: result (succeeded/failed/canceled/null), +# state (completed/inProgress/pending), name (encodes " UITests "): +curl -s "https://dev.azure.com/$ORG/$PROJ/_apis/build/builds/$BUILD/timeline?api-version=7.1" \ + | tee /tmp/gh-aw/agent/timeline_${BUILD}.json \ + | jq -r '.records[] | select(.type=="Job") | "\(.result)\t\(.state)\t\(.name)"' +``` + +(The prefetch already exposes per-leg status in `C.failedLegs` / the checks context, and +`gh pr checks ` lists the same per-leg rows with platform+category in the name — use +whichever is handy; the build timeline is the authoritative cross-check on the exact build.) + +Treat a target test as **VALIDATED-GREEN** only if, on `C.headSha`, the leg that runs its +category is green on **every platform it runs on** — a fix that repairs one platform must not +silently regress the same test's leg on another, so a green leg on the originally-red platform +alone is NOT enough. Across the target pipeline's platform legs (for a UI test: the Android, +iOS, Windows, and macOS/MacCatalyst legs running the target's category; for a device test: each +device-test platform), require ALL of: +- the target's category leg is `result == "succeeded"` **and** `state == "completed"` on + **every** platform that runs it, AND +- that leg is `failed` / `canceled` / aborted on **NO** platform, AND +- **no platform is left unverified.** For each platform family the target pipeline covers, that + platform's category leg must have CONCLUDED (`state == "completed"`) on `C.headSha`. If a + platform simply has no leg for the target's category, the test does not run there — that is + fine, not a gap. But if a platform's category leg has **not concluded** (`state` is + `inProgress` / pending, or an `/azp`-gated `maui-pr-uitests` / `maui-pr-devicetests` leg that + has not been kicked), the test's status on that platform is UNKNOWN → the fix is NOT yet + validated across platforms: record `skipped: target test green on but + not yet verified on (leg(s) pending / need /azp run) on PR #` + and stop this gate WITHOUT marking ready. + +A target test whose category leg never concluded on ANY platform (**not executed** anywhere — +e.g. its `/azp`-gated pipeline has not been kicked) is likewise NOT validated: record `skipped: +target test not yet executed on PR # ( not run — needs /azp run)` and stop +this gate WITHOUT marking ready. Do NOT overclaim — a green *sibling* leg (a different category +on the same platform) is not the target's leg, and a green leg on one platform is not a pass on +the others. + +**T3 — Mark ready + report.** If EVERY target test is VALIDATED-GREEN. The 🎯 comment, +the mark-ready, and the `p/0` label are THREE SEPARATE safe-outputs — the comment +existing does NOT prove the mark-ready took effect, so they are tracked independently: + +- **Comment idempotency (dup-suppress only — never gates the mark-ready).** We only reach + T3 while `C.isDraft == true` (the precondition stops an already-ready PR before here). So + if a prior bot comment for THIS head that carries the leading `🎯` anchor AND the + contiguous phrase `validated green on ` already exists (BOTH T3 variants — + `🎯 Target test validated green on ` and the build-only `🎯 Build validated green on + ` — contain that exact contiguous phrase, so this recognizes both; keying on + `target test` alone would instead miss the build-only comment and re-post it every sweep. + **Require the `🎯` anchor** so the match can NEVER be loosened to a gapped + `validated…green…on` form that would false-positive on the Step 3 `✅ … validated — the + fix's CI is green on ` surface comment and wrongly suppress the audit 🎯), the comment + landed on an earlier sweep but the **mark-ready did NOT take + effect** (the PR is still a draft) — set `SUPPRESS_COMMENT = true` (do not re-post the + duplicate 🎯 comment) but STILL complete the mark-ready + label below. Never treat the + comment's existence as "already marked ready" while the PR is still a draft. Record this + case as `re-marking PR # (🎯 present; mark-ready did not take on a prior sweep)`. +- **Atomicity budget pre-check (Hard-Rule 7).** Determine the outputs this gate will emit: + `mark_pull_request_as_ready_for_review` + `add_labels` always, plus `add_comment` unless + `SUPPRESS_COMMENT`. Verify a free per-run slot remains in EACH bucket you are about to + use. If ANY required bucket is exhausted, emit NONE of them — record `skipped: per-run + cap reached; deferring mark-ready PR # to next cycle` and stop this gate. Never mark a + PR ready (or label it) unless its 🎯 audit comment is GUARANTEED to exist for this exact + head SHA — either posted in this same sweep, or (in the `SUPPRESS_COMMENT` re-marking case) + already present from a prior sweep. The outputs you DO emit either all land or all defer + together; the audit trail must never be absent, but it is NOT re-posted when it already exists. +- **Dry-run gate (Step 0):** if `dry_run == "true"`, emit NOTHING — print the intended + readiness comment and "would mark ready + add p/0" to the run log, tally `dry-run: + would-mark-ready PR #`, and stop. +- Otherwise emit for THIS PR number ``: + 1. `add_comment` (ONLY if not `SUPPRESS_COMMENT`): `🎯 Target test validated green on + — passed on ALL platforms it runs on (, + buildId ). — not + caused by this fix."> Transitioning this PR from draft to ready for review and adding + `p/0`; a maintainer still reviews and merges.` (For a **build-only fix** — the T1 + whole-build fallback, no single target test — phrase the first clause as `🎯 Build + validated green on — the maui-pr build passed on ALL platforms (buildId + ).` instead of naming a test.) + 2. `mark_pull_request_as_ready_for_review` with `reason:` a one-line justification + naming the validated test(s) and ``. + 3. `add_labels` with `labels: ["p/0"]` for PR # — put the now-review-ready fix into + the team's p/0 priority queue so it is triaged, not lost in the draft backlog. (If + the PR somehow already carries `p/0`, this is a harmless no-op.) +- Record `marked-ready PR # (target green on ALL platforms on , + labeled p/0)` and stop. + #### Step 3.5.R — Maintainer change-request response (Track C) Reached from Step 3.5 gate 0. Goal: when an **eligible human reviewer** (Hard-Rule @@ -1383,7 +1669,19 @@ Filed by [`ci-status-fix`](https://github.com/dotnet/maui/blob/main/.github/work ### Step 8 — Per-issue tally + end-of-run summary -Per issue, append one outcome line to `/tmp/gh-aw/agent/coverage.txt`: +Per issue, append **one** outcome line to `/tmp/gh-aw/agent/coverage.txt` — the +terminal outcome for the cycle. When one cycle produces a chained pair — a PR gets a +same-cycle precursor line and **then** a Step 3.6 readiness line (`marked-ready` on the +draft→ready flip, or `already-ready` on a later steady-state sweep of an already-ready +PR) — record ONLY the terminal Step 3.6 readiness line: it supersedes ANY same-cycle +non-readiness precursor for that PR, so an aggregator keying on one-line-per-issue never +double-counts. The precursor may be a Step 3 green line (`surfaced-green` on the first ✅, +or `already-surfaced` when it already exists) OR — when an unrelated-flake red and a +target-green coincide — a Step 4 `annotated-flake` line; either way the readiness line is +terminal, and the superseded precursor's signal survives in the PR's 🎯/♻️ comment so no +information is lost. This covers the flip cycle (`surfaced-green` → `marked-ready`), the +post-flip steady state (`already-surfaced` → `already-ready`), and the flake-coincident +flip (`annotated-flake` → `marked-ready`): ``` # main attempt- @@ -1394,8 +1692,14 @@ Per issue, append one outcome line to `/tmp/gh-aw/agent/coverage.txt`: `advance-PR # attempt /10` (ADVANCE mode — new commit pushed to the existing PR), `surfaced-green PR #` (fix's own CI went green; commented for review, did not advance), `annotated-flake PR #` (red was unrelated flake; -commented, attempt NOT burned), `waiting PR #` (CI not settled yet), -`dry-run: would-`, `skipped: `. (The +commented, attempt NOT burned), `marked-ready PR #` (the specific fixed test +was confirmed green in the PR's own CI; draft flipped to ready for review), +`already-surfaced PR #` (the fix's ✅ green comment already existed this sweep; +re-post suppressed — superseded by the Step 3.6 readiness line when the PR also flips +ready that cycle), `already-ready PR #` (terminal steady state — the PR was flipped +ready on a prior cycle and remains green on an unchanged head; no action taken, +supersedes `already-surfaced`), `waiting PR #` (CI not settled yet), +`dry-run: would-`, `skipped: `. (The `needs-human-PR` outcome is reserved for the deferred hand-off — Step 6 currently records a skip instead, so it is not emitted.)
: `♻️ Attempt /10: red is - unrelated flake on leg(s) () on ; the fix itself is not - implicated. A maintainer re-run (/azp run ) should clear it.` Record - `annotated-flake PR # (head )` and stop. *(Round 1: human re-runs; - Round 2: auto re-trigger.)* + **Comment idempotency + dry-run suppress the ♻️ comment ONLY — neither skips Step + 3.6.** Scan the PR's existing comments for a prior bot `♻️ … unrelated flake … on + ` note for THIS head SHA; if one exists, set `SKIP_FLAKE_COMMENT = true`. + Resolve the attempt number from `C.effectiveAttempt` (the authoritative max(marker, + bot-commit) counter), omitting the `Attempt /10:` prefix only if it is + somehow indeterminate. Then post the flake note UNLESS suppressed: + - if `dry_run == "true"`: do NOT emit — print the intended `♻️` unrelated-flake body + to the run log and tally `dry-run: would-annotate-flake PR #`; + - else if `SKIP_FLAKE_COMMENT`: do NOT re-post — record `already-annotated-flake PR + # (head )` (re-annotating the same flake every 12h sweep is noise and + burns the per-run comment budget other PRs need); + - else `add_comment` on PR #: `♻️ Attempt /10: red is unrelated flake on + leg(s) () on ; the fix itself is not implicated. A + maintainer re-run (/azp run ) should clear it.` record `annotated-flake + PR # (head )`. + Then — in ALL of the above cases — run **Step 3.6** (target-test readiness gate) for + this PR before stopping: its `/azp`-gated target legs may conclude green on this SAME + head SHA on a LATER sweep, and Step 3.6 is the ONLY place the draft→ready flip + happens, so it MUST re-evaluate every sweep — never `stop` here before it. *(Round 1: + human re-runs; Round 2: auto re-trigger.)* - **Caused by the fix** (a failed leg still matches the original target signature, or the fix introduced a NEW failure): advance an attempt. - **Attempt count.** `attempt = C.effectiveAttempt` — the authoritative @@ -762,6 +866,188 @@ Run these gates in order — the FIRST that fires decides this cycle's outcome: from every prior commit on this branch**, emitted via the Step 5.6 ADVANCE path (push onto the same PR). +#### Step 3.6 — Target-test verification & mark-ready gate + +Reached from the Step 3 **green-surface** branch, the Step 4 **unrelated-flake** branch +(AFTER that branch has posted its comment), and the Step 3.5 **gate-2 target-focused +fast-path** (2a — while unrelated legs are still pending). Purpose: confirm the SPECIFIC +test(s) this PR was opened to fix now PASS in the PR's own CI, and — only when they do +— transition the draft `[ci-fix-net11]` PR to **ready for review** so a maintainer sees +a validated fix instead of a draft. This is the ONLY place the loop flips draft→ready; +it is a state transition, **never** an approval or a merge (a human still reviews and +merges). Overall red on *unrelated* legs must NOT gate readiness — we validate the fix, +not the base branch's flakiness. + +**Preconditions** (ALL must hold; otherwise record `skipped: readiness N/A PR # +()` and stop this gate): +- `C.isDraft == true` — if the PR is already ready-for-review, the draft→ready + transition is already done; do NOT re-mark **and do NOT re-apply `p/0`**. `p/0` is applied + exactly ONCE, atomically with the draft→ready flip (T3 — all-or-nothing with the 🎯 audit + comment + mark-ready), so an already-ready loop PR that is MISSING `p/0` has almost + certainly had it **removed by a maintainer** de-prioritizing the PR — a pure triage action + the human-engagement guard (which inspects comments/reviews/commits, NOT label changes) + cannot see. Re-adding `p/0` every sweep would fight that maintainer indefinitely, violating + the loop's "never override a human" contract. So the loop does NOT reconcile the label: + record `already-ready PR #` and stop this gate. (Trade-off: on the rare occasion a + transient API error drops `p/0` at flip time *after* the T3 atomic pre-check passed, it is + not auto-re-added — but the PR is still ready-for-review with its 🎯 audit comment, and + re-adding `p/0` is a trivial manual action; that is strictly preferable to steam-rolling a + maintainer's deliberate de-prioritization.) +- The PR is unmistakably THIS workflow's own: `[ci-fix-net11] ` title prefix AND the + `agentic-workflows` label (mirrors the safe-output lock; the compensating scope + control if the v0.80.9 compiler drops the declarative required-*). +- You reached this gate from Step 3 (green), Step 4 (**unrelated-flake**), or the Step 3.5 + gate-2 **target-focused fast-path** (2a) — i.e. the fix is NOT implicated in any red that + has concluded. If Step 4 classified a red as **caused by the fix** (ADVANCE), do NOT run + this gate — advance the attempt instead. + +**T1 — Identify the target test(s).** From the `[ci-scan-net11]` issue signature the fix +addresses (and the PR's own diff), extract the fully-qualified test method name(s) the +fix targets — e.g. `SafeAreaShouldWorkOnAllShellTabs`. For a de-flake it is the +de-flaked test; for a product fix it is the originally-failing test(s). If NO specific +test can be identified (e.g. a product build-break rather than a test failure), this is a +**build-only fix** — there is no single test to validate cross-platform, so validate at +**whole-build** granularity rather than stopping (Step 3 only *comments*; Step 3.6 is the +sole draft→ready flip, so a build-only fix is undrafted HERE or not at all). We only reach +this gate from Step 3 (green) / Step 4 (unrelated-flake) / gate-2a, so the fix is not +implicated in any concluded red; additionally require, via the T2 timeline method on +`C.headSha`, that the primary build pipeline `maui-pr` (def 302) has CONCLUDED with EVERY +one of its platform build legs `succeeded`/`completed` and NONE failed/canceled or still +pending — the build is green on **every** platform, not just the originally-broken one (the +cross-platform guard, applied to the build instead of a test). A build-only fix is undrafted +ONLY on the strength of the auto-running `maui-pr` (def 302) whole-build green, which the +loop CAN observe: if the PR's `[ci-scan]` issue signature or its own diff indicates the +ORIGINATING failure was in a `/azp`-gated pipeline (`maui-pr-uitests` def 313 or +`maui-pr-devicetests` def 314) rather than `maui-pr` (def 302), do NOT undraft on +`maui-pr`-green alone — those pipelines do not auto-run on this PR (GITHUB_TOKEN cannot +trigger them), so a green `maui-pr` build is NOT evidence the gated build break is fixed; +record `skipped: build-only fix PR # targets gated pipeline () not run — +deferring to human` and stop this gate. Otherwise, set `TARGET := "the maui-pr build +(build-only fix — no single target test)"` and proceed to **T3** to mark ready. If any `maui-pr` build leg is still unconcluded, record `skipped: build-only fix PR +# not yet whole-build green (leg(s) pending)` and stop this gate WITHOUT marking +ready (a green subset is not enough — a still-pending build leg could yet fail). + +**T2 — Verify the target test's platform legs are green (anonymous, leg-level).** Per-test +outcomes come from the AzDO **test-results** API (`_apis/test/...`), which is **NOT reachable +anonymously — Hard-Rule 8**: those endpoints 302-redirect to a sign-in page on this runner, so +a `curl` returns an HTML redirect (not JSON) and every target test would look "not executed". +Validate instead at **leg granularity** using ONLY the anonymous `_apis/build/...` timeline +(Hard-Rule 8) — a `succeeded` category leg means every test in that category **that actually +ran** passed on that platform. This is a strong bar **only if the target test genuinely +executes**: a job can go green while one specific test is skipped at runtime (`[Ignore]`, +`Assert.Ignore()`, `Assert.Inconclusive()`, a `Skip=`/conditional `[Fact]`, an `#if`-out, or +an early `return` before the asserts). So before trusting a green leg as proof the target +passed, **confirm from the PR's OWN diff that the fix does NOT skip, ignore, disable, or +short-circuit the target test** (it adds no `[Ignore]`/`[Explicit]`, `Assert.Ignore`/ +`Assert.Inconclusive`, `Skip=`, category-exclusion, `#if`-out, or early `return` around the +target). If the fix could cause the target to be runtime-skipped rather than genuinely pass, +leg-level green is NOT sufficient evidence — record `skipped: target test may be +runtime-skipped by this fix (diff adds ); leg-green insufficient, deferring to human +on PR #` and stop this gate WITHOUT marking ready. Otherwise (the target genuinely runs +and asserts, as a de-flake or product fix does) a green category leg cannot hide a +target-test failure, so treat it as authoritative. + +Map the target test to the CI leg(s) that run it. A leg name encodes platform + test category +(e.g. `Android UITests SafeAreaEdges,Shadow`, `iOS UITests SafeAreaEdges,Shadow`, `macOS +UITests SafeAreaEdges,Shadow`, `Windows UITests SafeAreaEdges,Shadow`). Determine the target +test's UI-test category — its `[Category(UITestCategories.X)]` in the test/HostApp source, +visible in the PR diff or the test file — to know which leg-name substring identifies its legs; +for a device test, the per-platform device-test legs. + +Using the SAME build-discovery as Step 4 (filter AzDO builds by `branchName=refs/pull//merge` +or `sourceVersion == C.headSha`), read each build's **timeline** on `C.headSha` for the +pipeline(s) that RUN the target test — `maui-pr` (def 302) for unit/integration tests, +`maui-pr-uitests` (def 313) for Appium UI tests, `maui-pr-devicetests` (def 314) for device +tests: + +```bash +ORG=dnceng-public; PROJ=public; BUILD= +# Anonymous + Hard-Rule-8-compliant. NEVER call _apis/test/... (it 302-redirects to sign-in). +# Each type=="Job" record is one platform leg: result (succeeded/failed/canceled/null), +# state (completed/inProgress/pending), name (encodes " UITests "): +curl -s "https://dev.azure.com/$ORG/$PROJ/_apis/build/builds/$BUILD/timeline?api-version=7.1" \ + | tee /tmp/gh-aw/agent/timeline_${BUILD}.json \ + | jq -r '.records[] | select(.type=="Job") | "\(.result)\t\(.state)\t\(.name)"' +``` + +(The prefetch already exposes per-leg status in `C.failedLegs` / the checks context, and +`gh pr checks ` lists the same per-leg rows with platform+category in the name — use +whichever is handy; the build timeline is the authoritative cross-check on the exact build.) + +Treat a target test as **VALIDATED-GREEN** only if, on `C.headSha`, the leg that runs its +category is green on **every platform it runs on** — a fix that repairs one platform must not +silently regress the same test's leg on another, so a green leg on the originally-red platform +alone is NOT enough. Across the target pipeline's platform legs (for a UI test: the Android, +iOS, Windows, and macOS/MacCatalyst legs running the target's category; for a device test: each +device-test platform), require ALL of: +- the target's category leg is `result == "succeeded"` **and** `state == "completed"` on + **every** platform that runs it, AND +- that leg is `failed` / `canceled` / aborted on **NO** platform, AND +- **no platform is left unverified.** For each platform family the target pipeline covers, that + platform's category leg must have CONCLUDED (`state == "completed"`) on `C.headSha`. If a + platform simply has no leg for the target's category, the test does not run there — that is + fine, not a gap. But if a platform's category leg has **not concluded** (`state` is + `inProgress` / pending, or an `/azp`-gated `maui-pr-uitests` / `maui-pr-devicetests` leg that + has not been kicked), the test's status on that platform is UNKNOWN → the fix is NOT yet + validated across platforms: record `skipped: target test green on but + not yet verified on (leg(s) pending / need /azp run) on PR #` + and stop this gate WITHOUT marking ready. + +A target test whose category leg never concluded on ANY platform (**not executed** anywhere — +e.g. its `/azp`-gated pipeline has not been kicked) is likewise NOT validated: record `skipped: +target test not yet executed on PR # ( not run — needs /azp run)` and stop +this gate WITHOUT marking ready. Do NOT overclaim — a green *sibling* leg (a different category +on the same platform) is not the target's leg, and a green leg on one platform is not a pass on +the others. + +**T3 — Mark ready + report.** If EVERY target test is VALIDATED-GREEN. The 🎯 comment, +the mark-ready, and the `p/0` label are THREE SEPARATE safe-outputs — the comment +existing does NOT prove the mark-ready took effect, so they are tracked independently: + +- **Comment idempotency (dup-suppress only — never gates the mark-ready).** We only reach + T3 while `C.isDraft == true` (the precondition stops an already-ready PR before here). So + if a prior bot comment for THIS head that carries the leading `🎯` anchor AND the + contiguous phrase `validated green on ` already exists (BOTH T3 variants — + `🎯 Target test validated green on ` and the build-only `🎯 Build validated green on + ` — contain that exact contiguous phrase, so this recognizes both; keying on + `target test` alone would instead miss the build-only comment and re-post it every sweep. + **Require the `🎯` anchor** so the match can NEVER be loosened to a gapped + `validated…green…on` form that would false-positive on the Step 3 `✅ … validated — the + fix's CI is green on ` surface comment and wrongly suppress the audit 🎯), the comment + landed on an earlier sweep but the **mark-ready did NOT take + effect** (the PR is still a draft) — set `SUPPRESS_COMMENT = true` (do not re-post the + duplicate 🎯 comment) but STILL complete the mark-ready + label below. Never treat the + comment's existence as "already marked ready" while the PR is still a draft. Record this + case as `re-marking PR # (🎯 present; mark-ready did not take on a prior sweep)`. +- **Atomicity budget pre-check (Hard-Rule 7).** Determine the outputs this gate will emit: + `mark_pull_request_as_ready_for_review` + `add_labels` always, plus `add_comment` unless + `SUPPRESS_COMMENT`. Verify a free per-run slot remains in EACH bucket you are about to + use. If ANY required bucket is exhausted, emit NONE of them — record `skipped: per-run + cap reached; deferring mark-ready PR # to next cycle` and stop this gate. Never mark a + PR ready (or label it) unless its 🎯 audit comment is GUARANTEED to exist for this exact + head SHA — either posted in this same sweep, or (in the `SUPPRESS_COMMENT` re-marking case) + already present from a prior sweep. The outputs you DO emit either all land or all defer + together; the audit trail must never be absent, but it is NOT re-posted when it already exists. +- **Dry-run gate (Step 0):** if `dry_run == "true"`, emit NOTHING — print the intended + readiness comment and "would mark ready + add p/0" to the run log, tally `dry-run: + would-mark-ready PR #`, and stop. +- Otherwise emit for THIS PR number ``: + 1. `add_comment` (ONLY if not `SUPPRESS_COMMENT`): `🎯 Target test validated green on + — passed on ALL platforms it runs on (, + buildId ). — not + caused by this fix."> Transitioning this PR from draft to ready for review and adding + `p/0`; a maintainer still reviews and merges.` (For a **build-only fix** — the T1 + whole-build fallback, no single target test — phrase the first clause as `🎯 Build + validated green on — the maui-pr build passed on ALL platforms (buildId + ).` instead of naming a test.) + 2. `mark_pull_request_as_ready_for_review` with `reason:` a one-line justification + naming the validated test(s) and ``. + 3. `add_labels` with `labels: ["p/0"]` for PR # — put the now-review-ready fix into + the team's p/0 priority queue so it is triaged, not lost in the draft backlog. (If + the PR somehow already carries `p/0`, this is a harmless no-op.) +- Record `marked-ready PR # (target green on ALL platforms on , + labeled p/0)` and stop. + #### Step 3.5.R — Maintainer change-request response (Track C) Reached from Step 3.5 gate 0. Goal: when an **eligible human reviewer** (Hard-Rule @@ -1395,7 +1681,19 @@ Filed by [`ci-status-fix-net11`](https://github.com/dotnet/maui/blob/main/.githu ### Step 8 — Per-issue tally + end-of-run summary -Per issue, append one outcome line to `/tmp/gh-aw/agent/coverage.txt`: +Per issue, append **one** outcome line to `/tmp/gh-aw/agent/coverage.txt` — the +terminal outcome for the cycle. When one cycle produces a chained pair — a PR gets a +same-cycle precursor line and **then** a Step 3.6 readiness line (`marked-ready` on the +draft→ready flip, or `already-ready` on a later steady-state sweep of an already-ready +PR) — record ONLY the terminal Step 3.6 readiness line: it supersedes ANY same-cycle +non-readiness precursor for that PR, so an aggregator keying on one-line-per-issue never +double-counts. The precursor may be a Step 3 green line (`surfaced-green` on the first ✅, +or `already-surfaced` when it already exists) OR — when an unrelated-flake red and a +target-green coincide — a Step 4 `annotated-flake` line; either way the readiness line is +terminal, and the superseded precursor's signal survives in the PR's 🎯/♻️ comment so no +information is lost. This covers the flip cycle (`surfaced-green` → `marked-ready`), the +post-flip steady state (`already-surfaced` → `already-ready`), and the flake-coincident +flip (`annotated-flake` → `marked-ready`): ``` # net11.0 attempt- @@ -1406,8 +1704,14 @@ Per issue, append one outcome line to `/tmp/gh-aw/agent/coverage.txt`: `advance-PR # attempt /10` (ADVANCE mode — new commit pushed to the existing PR), `surfaced-green PR #` (fix's own CI went green; commented for review, did not advance), `annotated-flake PR #` (red was unrelated flake; -commented, attempt NOT burned), `waiting PR #` (CI not settled yet), -`dry-run: would-`, `skipped: `. (The +commented, attempt NOT burned), `marked-ready PR #` (the specific fixed test +was confirmed green in the PR's own CI; draft flipped to ready for review), +`already-surfaced PR #` (the fix's ✅ green comment already existed this sweep; +re-post suppressed — superseded by the Step 3.6 readiness line when the PR also flips +ready that cycle), `already-ready PR #` (terminal steady state — the PR was flipped +ready on a prior cycle and remains green on an unchanged head; no action taken, +supersedes `already-surfaced`), `waiting PR #` (CI not settled yet), +`dry-run: would-`, `skipped: `. (The `needs-human-PR` outcome is reserved for the deferred hand-off — Step 6 currently records a skip instead, so it is not emitted.) diff --git a/.github/workflows/ci-status-fix.lock.yml b/.github/workflows/ci-status-fix.lock.yml index 93dc5969cb0d..65511f2b3259 100644 --- a/.github/workflows/ci-status-fix.lock.yml +++ b/.github/workflows/ci-status-fix.lock.yml @@ -1,4 +1,4 @@ -# gh-aw-metadata: {"schema_version":"v4","frontmatter_hash":"f014161967589ebcaf1ac5ec6f3c88ee5a4388d28b55a07dc36c45b7b2aebab6","body_hash":"86c6b2d4c3df13da14c6a3c8b211381fcc9f97bb1951ff7b80588a2c5338e00c","compiler_version":"v0.80.9","strict":true,"agent_id":"copilot","agent_model":"claude-opus-4.8","engine_versions":{"copilot":"1.0.63"}} +# gh-aw-metadata: {"schema_version":"v4","frontmatter_hash":"b0ebf8a2f179900cfe3638e994b27a43cec52bdbdd2331dc1bd4d65762fa2d6b","body_hash":"1825e2b1f7c7bd88241bf37580655489360f9bebc806edd00837a52ce4ad83a0","compiler_version":"v0.80.9","strict":true,"agent_id":"copilot","agent_model":"claude-opus-4.8","engine_versions":{"copilot":"1.0.63"}} # gh-aw-manifest: {"version":1,"secrets":["COPILOT_PAT_0","COPILOT_PAT_1","COPILOT_PAT_2","COPILOT_PAT_3","COPILOT_PAT_4","COPILOT_PAT_5","COPILOT_PAT_6","COPILOT_PAT_7","COPILOT_PAT_8","COPILOT_PAT_9","GH_AW_CI_TRIGGER_TOKEN","GH_AW_GITHUB_MCP_SERVER_TOKEN","GH_AW_GITHUB_TOKEN","GITHUB_TOKEN"],"actions":[{"repo":"actions/checkout","sha":"34e114876b0b11c390a56381ad16ebd13914f8d5","version":"v4"},{"repo":"actions/checkout","sha":"9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0","version":"v7.0.0"},{"repo":"actions/checkout","sha":"de0fac2e4500dabe0009e67214ff5f5447ce83dd","version":"v6.0.2"},{"repo":"actions/download-artifact","sha":"3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c","version":"v8.0.1"},{"repo":"actions/github-script","sha":"3a2844b7e9c422d3c10d287c895573f7108da1b3","version":"v9.0.0"},{"repo":"actions/setup-node","sha":"48b55a011bda9f5d6aeb4c2d9c7362e8dae4041e","version":"v6.4.0"},{"repo":"actions/upload-artifact","sha":"043fb46d1a93c77aae656e7c1c64a875d1fc6a0a","version":"v7.0.1"},{"repo":"github/gh-aw-actions/setup","sha":"8c7d04ebf1ece56cd381446125da3e0f6896294a","version":"v0.80.9"}],"containers":[{"image":"ghcr.io/github/gh-aw-firewall/agent:0.27.7","digest":"sha256:aae231e4635c8999d039c132f1602d3df850fe9b84a00aa2b5ac981179b5661c","pinned_image":"ghcr.io/github/gh-aw-firewall/agent:0.27.7@sha256:aae231e4635c8999d039c132f1602d3df850fe9b84a00aa2b5ac981179b5661c"},{"image":"ghcr.io/github/gh-aw-firewall/api-proxy:0.27.7","digest":"sha256:009caf2e3d88fa77b64e9a03a95a228fc58db0f1701c6d324b29ba5a3c7c79b6","pinned_image":"ghcr.io/github/gh-aw-firewall/api-proxy:0.27.7@sha256:009caf2e3d88fa77b64e9a03a95a228fc58db0f1701c6d324b29ba5a3c7c79b6"},{"image":"ghcr.io/github/gh-aw-firewall/squid:0.27.7","digest":"sha256:deb1d4e19de62d51cee0508057a596a19315c3423ada4d675cad136dc8037c96","pinned_image":"ghcr.io/github/gh-aw-firewall/squid:0.27.7@sha256:deb1d4e19de62d51cee0508057a596a19315c3423ada4d675cad136dc8037c96"},{"image":"ghcr.io/github/gh-aw-mcpg:v0.3.27","digest":"sha256:fe984bddde4ec05d756d9043edb0a32912e6b7b72f6a121b1082f29221421cc7","pinned_image":"ghcr.io/github/gh-aw-mcpg:v0.3.27@sha256:fe984bddde4ec05d756d9043edb0a32912e6b7b72f6a121b1082f29221421cc7"},{"image":"ghcr.io/github/gh-aw-node","digest":"sha256:529d02eb970b1161aa25c593a9c3df57fdfad5a8add328cb3b6eccef66f3183b","pinned_image":"ghcr.io/github/gh-aw-node@sha256:529d02eb970b1161aa25c593a9c3df57fdfad5a8add328cb3b6eccef66f3183b"},{"image":"ghcr.io/github/github-mcp-server:v1.4.0","digest":"sha256:2afb26356481d1a350e14544a6e160f7f7ec1561a1ea309b823665abf0309036","pinned_image":"ghcr.io/github/github-mcp-server:v1.4.0@sha256:2afb26356481d1a350e14544a6e160f7f7ec1561a1ea309b823665abf0309036"}]} # This file was automatically generated by gh-aw (v0.80.9). DO NOT EDIT. To debug this workflow, load the skill at https://github.com/github/gh-aw/blob/main/debug.md # @@ -37,7 +37,9 @@ # humans (the open tracking issue is the hand-off surface; a dedicated # [ci-fix][needs-human] PR is planned but currently deferred — see Step 6). CI on # the PR is kicked by a human `/azp run` for now; the loop watches, classifies, -# and re-fixes autonomously between kicks. +# and re-fixes autonomously between kicks. When the SPECIFIC test a PR fixed is +# confirmed green in that PR's own CI, the loop marks the draft PR ready for review +# (a state transition only — it never approves or merges). # Never mutes tests, but # de-flakes genuinely flaky ones (deterministic synchronization, no retries / # timeout bumps). Always skips visual-regression / screenshot issues. @@ -273,24 +275,24 @@ jobs: run: | bash "${RUNNER_TEMP}/gh-aw/actions/create_prompt_first.sh" { - cat << 'GH_AW_PROMPT_25cf75111b670167_EOF' + cat << 'GH_AW_PROMPT_2ac1d0b4083428f8_EOF' - GH_AW_PROMPT_25cf75111b670167_EOF + GH_AW_PROMPT_2ac1d0b4083428f8_EOF cat "${RUNNER_TEMP}/gh-aw/prompts/xpia.md" cat "${RUNNER_TEMP}/gh-aw/prompts/temp_folder_prompt.md" cat "${RUNNER_TEMP}/gh-aw/prompts/markdown.md" cat "${RUNNER_TEMP}/gh-aw/prompts/safe_outputs_prompt.md" - cat << 'GH_AW_PROMPT_25cf75111b670167_EOF' + cat << 'GH_AW_PROMPT_2ac1d0b4083428f8_EOF' - Tools: add_comment(max:3), create_pull_request(max:3), update_pull_request(max:3), push_to_pull_request_branch(max:3), missing_tool, missing_data, noop - GH_AW_PROMPT_25cf75111b670167_EOF + Tools: add_comment(max:6), create_pull_request(max:3), update_pull_request(max:3), mark_pull_request_as_ready_for_review(max:3), add_labels(max:3), push_to_pull_request_branch(max:3), missing_tool, missing_data, noop + GH_AW_PROMPT_2ac1d0b4083428f8_EOF cat "${RUNNER_TEMP}/gh-aw/prompts/safe_outputs_create_pull_request.md" cat "${RUNNER_TEMP}/gh-aw/prompts/safe_outputs_push_to_pr_branch.md" - cat << 'GH_AW_PROMPT_25cf75111b670167_EOF' + cat << 'GH_AW_PROMPT_2ac1d0b4083428f8_EOF' - GH_AW_PROMPT_25cf75111b670167_EOF + GH_AW_PROMPT_2ac1d0b4083428f8_EOF cat "${RUNNER_TEMP}/gh-aw/prompts/mcp_cli_tools_prompt.md" - cat << 'GH_AW_PROMPT_25cf75111b670167_EOF' + cat << 'GH_AW_PROMPT_2ac1d0b4083428f8_EOF' The following GitHub context information is available for this workflow: {{#if github.actor}} @@ -332,12 +334,12 @@ jobs: stop immediately and report the limitation rather than spending turns trying to work around it. - GH_AW_PROMPT_25cf75111b670167_EOF + GH_AW_PROMPT_2ac1d0b4083428f8_EOF cat "${RUNNER_TEMP}/gh-aw/prompts/github_mcp_tools_with_safeoutputs_prompt.md" - cat << 'GH_AW_PROMPT_25cf75111b670167_EOF' + cat << 'GH_AW_PROMPT_2ac1d0b4083428f8_EOF' {{#runtime-import .github/workflows/ci-status-fix.md}} - GH_AW_PROMPT_25cf75111b670167_EOF + GH_AW_PROMPT_2ac1d0b4083428f8_EOF } > "$GH_AW_PROMPT" - name: Interpolate variables and render templates uses: actions/github-script@3a2844b7e9c422d3c10d287c895573f7108da1b3 # v9.0.0 @@ -554,16 +556,18 @@ jobs: mkdir -p "${RUNNER_TEMP}/gh-aw/safeoutputs" mkdir -p /tmp/gh-aw/safeoutputs mkdir -p /tmp/gh-aw/mcp-logs/safeoutputs - cat > "${RUNNER_TEMP}/gh-aw/safeoutputs/config.json" << 'GH_AW_SAFE_OUTPUTS_CONFIG_fa2a69fad5fd0485_EOF' - {"add_comment":{"discussions":false,"max":3,"required_labels":["agentic-workflows"],"required_title_prefix":"[ci-fix] ","target":"*"},"create_pull_request":{"allowed_base_branches":["main"],"allowed_branches":["ci-fix/**"],"allowed_files":["src/Core/**","src/Controls/**","src/Essentials/**","src/BlazorWebView/**","src/TestUtils/**","src/Templates/**","**/PublicAPI.Unshipped.txt"],"base_branch":"main","draft":true,"labels":["agentic-workflows"],"max":3,"max_patch_files":100,"max_patch_size":4096,"protect_top_level_dot_folders":true,"protected_files":["package.json","bun.lockb","bunfig.toml","deno.json","deno.jsonc","deno.lock","global.json","NuGet.Config","Directory.Packages.props","mix.exs","mix.lock","go.mod","go.sum","stack.yaml","stack.yaml.lock","pom.xml","build.gradle","build.gradle.kts","settings.gradle","settings.gradle.kts","gradle.properties","package-lock.json","yarn.lock","pnpm-lock.yaml","npm-shrinkwrap.json","requirements.txt","Pipfile","Pipfile.lock","pyproject.toml","setup.py","setup.cfg","Gemfile","Gemfile.lock","uv.lock","CODEOWNERS","DESIGN.md","README.md","CONTRIBUTING.md","CHANGELOG.md","SECURITY.md","CODE_OF_CONDUCT.md","AGENTS.md","CLAUDE.md","GEMINI.md"],"protected_files_policy":"request_review","title_prefix":"[ci-fix] "},"create_report_incomplete_issue":{},"missing_data":{},"missing_tool":{},"noop":{"max":1,"report-as-issue":"true"},"push_to_pull_request_branch":{"allowed_files":["src/Core/**","src/Controls/**","src/Essentials/**","src/BlazorWebView/**","src/TestUtils/**","src/Templates/**","**/PublicAPI.Unshipped.txt"],"if_no_changes":"warn","max":3,"max_patch_size":4096,"protect_top_level_dot_folders":true,"protected_files":["package.json","bun.lockb","bunfig.toml","deno.json","deno.jsonc","deno.lock","global.json","NuGet.Config","Directory.Packages.props","mix.exs","mix.lock","go.mod","go.sum","stack.yaml","stack.yaml.lock","pom.xml","build.gradle","build.gradle.kts","settings.gradle","settings.gradle.kts","gradle.properties","package-lock.json","yarn.lock","pnpm-lock.yaml","npm-shrinkwrap.json","requirements.txt","Pipfile","Pipfile.lock","pyproject.toml","setup.py","setup.cfg","Gemfile","Gemfile.lock","uv.lock","CODEOWNERS","DESIGN.md","README.md","CONTRIBUTING.md","CHANGELOG.md","SECURITY.md","CODE_OF_CONDUCT.md","AGENTS.md","CLAUDE.md","GEMINI.md"],"required_labels":["agentic-workflows"],"target":"*","title_prefix":"[ci-fix] "},"report_incomplete":{},"update_pull_request":{"allow_body":true,"allow_title":false,"max":3,"target":"*","update_branch":false}} - GH_AW_SAFE_OUTPUTS_CONFIG_fa2a69fad5fd0485_EOF + cat > "${RUNNER_TEMP}/gh-aw/safeoutputs/config.json" << 'GH_AW_SAFE_OUTPUTS_CONFIG_0644bf1434ca0071_EOF' + {"add_comment":{"discussions":false,"max":6,"required_labels":["agentic-workflows"],"required_title_prefix":"[ci-fix] ","target":"*"},"add_labels":{"allowed":["p/0"],"max":3,"required_labels":["agentic-workflows"],"required_title_prefix":"[ci-fix] ","target":"*"},"create_pull_request":{"allowed_base_branches":["main"],"allowed_branches":["ci-fix/**"],"allowed_files":["src/Core/**","src/Controls/**","src/Essentials/**","src/BlazorWebView/**","src/TestUtils/**","src/Templates/**","**/PublicAPI.Unshipped.txt"],"base_branch":"main","draft":true,"labels":["agentic-workflows"],"max":3,"max_patch_files":100,"max_patch_size":4096,"protect_top_level_dot_folders":true,"protected_files":["package.json","bun.lockb","bunfig.toml","deno.json","deno.jsonc","deno.lock","global.json","NuGet.Config","Directory.Packages.props","mix.exs","mix.lock","go.mod","go.sum","stack.yaml","stack.yaml.lock","pom.xml","build.gradle","build.gradle.kts","settings.gradle","settings.gradle.kts","gradle.properties","package-lock.json","yarn.lock","pnpm-lock.yaml","npm-shrinkwrap.json","requirements.txt","Pipfile","Pipfile.lock","pyproject.toml","setup.py","setup.cfg","Gemfile","Gemfile.lock","uv.lock","CODEOWNERS","DESIGN.md","README.md","CONTRIBUTING.md","CHANGELOG.md","SECURITY.md","CODE_OF_CONDUCT.md","AGENTS.md","CLAUDE.md","GEMINI.md"],"protected_files_policy":"request_review","title_prefix":"[ci-fix] "},"create_report_incomplete_issue":{},"mark_pull_request_as_ready_for_review":{"max":3,"required_labels":["agentic-workflows"],"required_title_prefix":"[ci-fix] ","target":"*"},"missing_data":{},"missing_tool":{},"noop":{"max":1,"report-as-issue":"true"},"push_to_pull_request_branch":{"allowed_files":["src/Core/**","src/Controls/**","src/Essentials/**","src/BlazorWebView/**","src/TestUtils/**","src/Templates/**","**/PublicAPI.Unshipped.txt"],"if_no_changes":"warn","max":3,"max_patch_size":4096,"protect_top_level_dot_folders":true,"protected_files":["package.json","bun.lockb","bunfig.toml","deno.json","deno.jsonc","deno.lock","global.json","NuGet.Config","Directory.Packages.props","mix.exs","mix.lock","go.mod","go.sum","stack.yaml","stack.yaml.lock","pom.xml","build.gradle","build.gradle.kts","settings.gradle","settings.gradle.kts","gradle.properties","package-lock.json","yarn.lock","pnpm-lock.yaml","npm-shrinkwrap.json","requirements.txt","Pipfile","Pipfile.lock","pyproject.toml","setup.py","setup.cfg","Gemfile","Gemfile.lock","uv.lock","CODEOWNERS","DESIGN.md","README.md","CONTRIBUTING.md","CHANGELOG.md","SECURITY.md","CODE_OF_CONDUCT.md","AGENTS.md","CLAUDE.md","GEMINI.md"],"required_labels":["agentic-workflows"],"target":"*","title_prefix":"[ci-fix] "},"report_incomplete":{},"update_pull_request":{"allow_body":true,"allow_title":false,"max":3,"target":"*","update_branch":false}} + GH_AW_SAFE_OUTPUTS_CONFIG_0644bf1434ca0071_EOF - name: Generate Safe Outputs Tools env: GH_AW_TOOLS_META_JSON: | { "description_suffixes": { - "add_comment": " CONSTRAINTS: Maximum 3 comment(s) can be added. Target: *. Supports reply_to_id for discussion threading.", + "add_comment": " CONSTRAINTS: Maximum 6 comment(s) can be added. Target: *. Supports reply_to_id for discussion threading.", + "add_labels": " CONSTRAINTS: Maximum 3 label(s) can be added. Only these labels are allowed: [\"p/0\"]. Target: *.", "create_pull_request": " CONSTRAINTS: Maximum 3 pull request(s) can be created. Title will be prefixed with \"[ci-fix] \". Labels [\"agentic-workflows\"] will be automatically added. Only these labels are allowed: [\"agentic-workflows\"]. PRs will be created as drafts.", + "mark_pull_request_as_ready_for_review": " CONSTRAINTS: Maximum 3 pull request(s) can be marked as ready for review.", "push_to_pull_request_branch": " CONSTRAINTS: Maximum 3 push(es) can be made. The target pull request title must start with \"[ci-fix] \".", "update_pull_request": " CONSTRAINTS: Maximum 3 pull request(s) can be updated. Target: *." }, @@ -594,6 +598,25 @@ jobs: } } }, + "add_labels": { + "defaultMax": 5, + "fields": { + "item_number": { + "issueNumberOrTemporaryId": true + }, + "labels": { + "required": true, + "type": "array", + "itemType": "string", + "itemSanitize": true, + "itemMaxLength": 128 + }, + "repo": { + "type": "string", + "maxLength": 256 + } + } + }, "create_pull_request": { "defaultMax": 1, "fields": { @@ -635,6 +658,24 @@ jobs: } } }, + "mark_pull_request_as_ready_for_review": { + "defaultMax": 1, + "fields": { + "pull_request_number": { + "issueOrPRNumber": true + }, + "reason": { + "required": true, + "type": "string", + "sanitize": true, + "maxLength": 65000 + }, + "repo": { + "type": "string", + "maxLength": 256 + } + } + }, "missing_data": { "defaultMax": 20, "fields": { @@ -1494,7 +1535,7 @@ jobs: uses: actions/github-script@3a2844b7e9c422d3c10d287c895573f7108da1b3 # v9.0.0 env: WORKFLOW_NAME: "CI Failure Fixer (main)" - WORKFLOW_DESCRIPTION: "Periodic pass over open ci-scan tracking issues filed by the main-branch CI\nfailure scanner (.github/workflows/ci-status-main.md). This workflow targets\nthe `main` branch EXCLUSIVELY: it processes only issues labelled ci-scan and\nopens every PR against main. (The net11.0 branch is handled by the parallel\n.github/workflows/ci-status-fix-net11.md — the two are split because gh-aw can\nonly transport a fix relative to ONE static base branch per workflow, and the\nmain↔net11.0 divergence exceeds gh-aw's 10 MB transport-patch cap.) The fixer\nopens ONE draft [ci-fix] PR per actionable issue against main, then WATCHES\nthat PR's own CI on later runs: when the fix's CI comes back red and the red is\ncaused by the fix itself, it pushes a fresh follow-up fix onto the SAME PR\nbranch (never a second PR) — up to 10 attempts — then stops and defers to\nhumans (the open tracking issue is the hand-off surface; a dedicated\n[ci-fix][needs-human] PR is planned but currently deferred — see Step 6). CI on\nthe PR is kicked by a human `/azp run` for now; the loop watches, classifies,\nand re-fixes autonomously between kicks.\nNever mutes tests, but\nde-flakes genuinely flaky ones (deterministic synchronization, no retries /\ntimeout bumps). Always skips visual-regression / screenshot issues." + WORKFLOW_DESCRIPTION: "Periodic pass over open ci-scan tracking issues filed by the main-branch CI\nfailure scanner (.github/workflows/ci-status-main.md). This workflow targets\nthe `main` branch EXCLUSIVELY: it processes only issues labelled ci-scan and\nopens every PR against main. (The net11.0 branch is handled by the parallel\n.github/workflows/ci-status-fix-net11.md — the two are split because gh-aw can\nonly transport a fix relative to ONE static base branch per workflow, and the\nmain↔net11.0 divergence exceeds gh-aw's 10 MB transport-patch cap.) The fixer\nopens ONE draft [ci-fix] PR per actionable issue against main, then WATCHES\nthat PR's own CI on later runs: when the fix's CI comes back red and the red is\ncaused by the fix itself, it pushes a fresh follow-up fix onto the SAME PR\nbranch (never a second PR) — up to 10 attempts — then stops and defers to\nhumans (the open tracking issue is the hand-off surface; a dedicated\n[ci-fix][needs-human] PR is planned but currently deferred — see Step 6). CI on\nthe PR is kicked by a human `/azp run` for now; the loop watches, classifies,\nand re-fixes autonomously between kicks. When the SPECIFIC test a PR fixed is\nconfirmed green in that PR's own CI, the loop marks the draft PR ready for review\n(a state transition only — it never approves or merges).\nNever mutes tests, but\nde-flakes genuinely flaky ones (deterministic synchronization, no retries /\ntimeout bumps). Always skips visual-regression / screenshot issues." HAS_PATCH: ${{ needs.agent.outputs.has_patch }} with: script: | @@ -1910,7 +1951,7 @@ jobs: GH_AW_ALLOWED_DOMAINS: "*.blob.core.windows.net,*.githubusercontent.com,api.business.githubcopilot.com,api.enterprise.githubcopilot.com,api.github.com,api.githubcopilot.com,api.individual.githubcopilot.com,api.snapcraft.io,archive.ubuntu.com,azure.archive.ubuntu.com,codeload.github.com,crl.geotrust.com,crl.globalsign.com,crl.identrust.com,crl.sectigo.com,crl.thawte.com,crl.usertrust.com,crl.verisign.com,crl3.digicert.com,crl4.digicert.com,crls.ssl.com,dev.azure.com,docs.github.com,github-cloud.githubusercontent.com,github-cloud.s3.amazonaws.com,github.blog,github.com,github.githubassets.com,helix.dot.net,host.docker.internal,json-schema.org,json.schemastore.org,keyserver.ubuntu.com,lfs.github.com,objects.githubusercontent.com,ocsp.digicert.com,ocsp.geotrust.com,ocsp.globalsign.com,ocsp.identrust.com,ocsp.sectigo.com,ocsp.ssl.com,ocsp.thawte.com,ocsp.usertrust.com,ocsp.verisign.com,packagecloud.io,packages.cloud.google.com,packages.microsoft.com,patch-diff.githubusercontent.com,ppa.launchpad.net,raw.githubusercontent.com,registry.npmjs.org,s.symcb.com,s.symcd.com,security.ubuntu.com,telemetry.enterprise.githubcopilot.com,ts-crl.ws.symantec.com,ts-ocsp.ws.symantec.com,www.googleapis.com" GITHUB_SERVER_URL: ${{ github.server_url }} GITHUB_API_URL: ${{ github.api_url }} - GH_AW_SAFE_OUTPUTS_HANDLER_CONFIG: "{\"add_comment\":{\"discussions\":false,\"max\":3,\"required_labels\":[\"agentic-workflows\"],\"required_title_prefix\":\"[ci-fix] \",\"target\":\"*\"},\"create_pull_request\":{\"allowed_base_branches\":[\"main\"],\"allowed_branches\":[\"ci-fix/**\"],\"allowed_files\":[\"src/Core/**\",\"src/Controls/**\",\"src/Essentials/**\",\"src/BlazorWebView/**\",\"src/TestUtils/**\",\"src/Templates/**\",\"**/PublicAPI.Unshipped.txt\"],\"base_branch\":\"main\",\"draft\":true,\"labels\":[\"agentic-workflows\"],\"max\":3,\"max_patch_files\":100,\"max_patch_size\":4096,\"protect_top_level_dot_folders\":true,\"protected_files\":[\"package.json\",\"bun.lockb\",\"bunfig.toml\",\"deno.json\",\"deno.jsonc\",\"deno.lock\",\"global.json\",\"NuGet.Config\",\"Directory.Packages.props\",\"mix.exs\",\"mix.lock\",\"go.mod\",\"go.sum\",\"stack.yaml\",\"stack.yaml.lock\",\"pom.xml\",\"build.gradle\",\"build.gradle.kts\",\"settings.gradle\",\"settings.gradle.kts\",\"gradle.properties\",\"package-lock.json\",\"yarn.lock\",\"pnpm-lock.yaml\",\"npm-shrinkwrap.json\",\"requirements.txt\",\"Pipfile\",\"Pipfile.lock\",\"pyproject.toml\",\"setup.py\",\"setup.cfg\",\"Gemfile\",\"Gemfile.lock\",\"uv.lock\",\"CODEOWNERS\",\"DESIGN.md\",\"README.md\",\"CONTRIBUTING.md\",\"CHANGELOG.md\",\"SECURITY.md\",\"CODE_OF_CONDUCT.md\",\"AGENTS.md\",\"CLAUDE.md\",\"GEMINI.md\"],\"protected_files_policy\":\"request_review\",\"title_prefix\":\"[ci-fix] \"},\"create_report_incomplete_issue\":{},\"missing_data\":{},\"missing_tool\":{},\"noop\":{\"max\":1,\"report-as-issue\":\"true\"},\"push_to_pull_request_branch\":{\"allowed_files\":[\"src/Core/**\",\"src/Controls/**\",\"src/Essentials/**\",\"src/BlazorWebView/**\",\"src/TestUtils/**\",\"src/Templates/**\",\"**/PublicAPI.Unshipped.txt\"],\"if_no_changes\":\"warn\",\"max\":3,\"max_patch_size\":4096,\"protect_top_level_dot_folders\":true,\"protected_files\":[\"package.json\",\"bun.lockb\",\"bunfig.toml\",\"deno.json\",\"deno.jsonc\",\"deno.lock\",\"global.json\",\"NuGet.Config\",\"Directory.Packages.props\",\"mix.exs\",\"mix.lock\",\"go.mod\",\"go.sum\",\"stack.yaml\",\"stack.yaml.lock\",\"pom.xml\",\"build.gradle\",\"build.gradle.kts\",\"settings.gradle\",\"settings.gradle.kts\",\"gradle.properties\",\"package-lock.json\",\"yarn.lock\",\"pnpm-lock.yaml\",\"npm-shrinkwrap.json\",\"requirements.txt\",\"Pipfile\",\"Pipfile.lock\",\"pyproject.toml\",\"setup.py\",\"setup.cfg\",\"Gemfile\",\"Gemfile.lock\",\"uv.lock\",\"CODEOWNERS\",\"DESIGN.md\",\"README.md\",\"CONTRIBUTING.md\",\"CHANGELOG.md\",\"SECURITY.md\",\"CODE_OF_CONDUCT.md\",\"AGENTS.md\",\"CLAUDE.md\",\"GEMINI.md\"],\"required_labels\":[\"agentic-workflows\"],\"target\":\"*\",\"title_prefix\":\"[ci-fix] \"},\"report_incomplete\":{},\"update_pull_request\":{\"allow_body\":true,\"allow_title\":false,\"max\":3,\"target\":\"*\",\"update_branch\":false}}" + GH_AW_SAFE_OUTPUTS_HANDLER_CONFIG: "{\"add_comment\":{\"discussions\":false,\"max\":6,\"required_labels\":[\"agentic-workflows\"],\"required_title_prefix\":\"[ci-fix] \",\"target\":\"*\"},\"add_labels\":{\"allowed\":[\"p/0\"],\"max\":3,\"required_labels\":[\"agentic-workflows\"],\"required_title_prefix\":\"[ci-fix] \",\"target\":\"*\"},\"create_pull_request\":{\"allowed_base_branches\":[\"main\"],\"allowed_branches\":[\"ci-fix/**\"],\"allowed_files\":[\"src/Core/**\",\"src/Controls/**\",\"src/Essentials/**\",\"src/BlazorWebView/**\",\"src/TestUtils/**\",\"src/Templates/**\",\"**/PublicAPI.Unshipped.txt\"],\"base_branch\":\"main\",\"draft\":true,\"labels\":[\"agentic-workflows\"],\"max\":3,\"max_patch_files\":100,\"max_patch_size\":4096,\"protect_top_level_dot_folders\":true,\"protected_files\":[\"package.json\",\"bun.lockb\",\"bunfig.toml\",\"deno.json\",\"deno.jsonc\",\"deno.lock\",\"global.json\",\"NuGet.Config\",\"Directory.Packages.props\",\"mix.exs\",\"mix.lock\",\"go.mod\",\"go.sum\",\"stack.yaml\",\"stack.yaml.lock\",\"pom.xml\",\"build.gradle\",\"build.gradle.kts\",\"settings.gradle\",\"settings.gradle.kts\",\"gradle.properties\",\"package-lock.json\",\"yarn.lock\",\"pnpm-lock.yaml\",\"npm-shrinkwrap.json\",\"requirements.txt\",\"Pipfile\",\"Pipfile.lock\",\"pyproject.toml\",\"setup.py\",\"setup.cfg\",\"Gemfile\",\"Gemfile.lock\",\"uv.lock\",\"CODEOWNERS\",\"DESIGN.md\",\"README.md\",\"CONTRIBUTING.md\",\"CHANGELOG.md\",\"SECURITY.md\",\"CODE_OF_CONDUCT.md\",\"AGENTS.md\",\"CLAUDE.md\",\"GEMINI.md\"],\"protected_files_policy\":\"request_review\",\"title_prefix\":\"[ci-fix] \"},\"create_report_incomplete_issue\":{},\"mark_pull_request_as_ready_for_review\":{\"max\":3,\"required_labels\":[\"agentic-workflows\"],\"required_title_prefix\":\"[ci-fix] \",\"target\":\"*\"},\"missing_data\":{},\"missing_tool\":{},\"noop\":{\"max\":1,\"report-as-issue\":\"true\"},\"push_to_pull_request_branch\":{\"allowed_files\":[\"src/Core/**\",\"src/Controls/**\",\"src/Essentials/**\",\"src/BlazorWebView/**\",\"src/TestUtils/**\",\"src/Templates/**\",\"**/PublicAPI.Unshipped.txt\"],\"if_no_changes\":\"warn\",\"max\":3,\"max_patch_size\":4096,\"protect_top_level_dot_folders\":true,\"protected_files\":[\"package.json\",\"bun.lockb\",\"bunfig.toml\",\"deno.json\",\"deno.jsonc\",\"deno.lock\",\"global.json\",\"NuGet.Config\",\"Directory.Packages.props\",\"mix.exs\",\"mix.lock\",\"go.mod\",\"go.sum\",\"stack.yaml\",\"stack.yaml.lock\",\"pom.xml\",\"build.gradle\",\"build.gradle.kts\",\"settings.gradle\",\"settings.gradle.kts\",\"gradle.properties\",\"package-lock.json\",\"yarn.lock\",\"pnpm-lock.yaml\",\"npm-shrinkwrap.json\",\"requirements.txt\",\"Pipfile\",\"Pipfile.lock\",\"pyproject.toml\",\"setup.py\",\"setup.cfg\",\"Gemfile\",\"Gemfile.lock\",\"uv.lock\",\"CODEOWNERS\",\"DESIGN.md\",\"README.md\",\"CONTRIBUTING.md\",\"CHANGELOG.md\",\"SECURITY.md\",\"CODE_OF_CONDUCT.md\",\"AGENTS.md\",\"CLAUDE.md\",\"GEMINI.md\"],\"required_labels\":[\"agentic-workflows\"],\"target\":\"*\",\"title_prefix\":\"[ci-fix] \"},\"report_incomplete\":{},\"update_pull_request\":{\"allow_body\":true,\"allow_title\":false,\"max\":3,\"target\":\"*\",\"update_branch\":false}}" GH_AW_CI_TRIGGER_TOKEN: ${{ secrets.GH_AW_CI_TRIGGER_TOKEN }} with: github-token: ${{ secrets.GH_AW_GITHUB_TOKEN || secrets.GITHUB_TOKEN }} diff --git a/.github/workflows/ci-status-fix.md b/.github/workflows/ci-status-fix.md index d3227e25170c..9c2f46054029 100644 --- a/.github/workflows/ci-status-fix.md +++ b/.github/workflows/ci-status-fix.md @@ -15,7 +15,9 @@ description: | humans (the open tracking issue is the hand-off surface; a dedicated [ci-fix][needs-human] PR is planned but currently deferred — see Step 6). CI on the PR is kicked by a human `/azp run` for now; the loop watches, classifies, - and re-fixes autonomously between kicks. + and re-fixes autonomously between kicks. When the SPECIFIC test a PR fixed is + confirmed green in that PR's own CI, the loop marks the draft PR ready for review + (a state transition only — it never approves or merges). Never mutes tests, but de-flakes genuinely flaky ones (deterministic synchronization, no retries / timeout bumps). Always skips visual-regression / screenshot issues. @@ -239,8 +241,15 @@ safe-outputs: # PER-RUN total (not per-PR): a sweep may surface several green PRs and/or # annotate several flaky reds in one run. At max:1 all but the first comment # were silently dropped, starving the workflow's #1 value (surfacing green PRs - # for review). Raised to 3 to match the per-run throughput of the other outputs. - max: 3 + # for review). Sized to 6 (not 3) because a single green DRAFT PR that gets + # flipped ready in one sweep spends TWO comment slots — the Step 3 ✅ surface + # comment (which still names the /azp-gated legs a human must kick, so it is NOT + # redundant with 🎯) AND the Step 3.6 T3 🎯 readiness comment. At max:3 the shared + # bucket drained after ~1 draft flip and T3's atomicity pre-check then deferred + # every further mark-ready even while the mark-ready/add_labels buckets (also 3) + # sat idle; 6 lets ~3 draft flips (2 comments each) land per sweep, so the + # mark-ready:3 / add_labels:3 caps are actually reachable. + max: 6 target: "*" # Hard constraint (defense-in-depth): only comment on THIS workflow's own # ci-fix PRs — [ci-fix] title prefix AND agentic-workflows label. Mirrors the @@ -261,13 +270,51 @@ safe-outputs: # never the title, so disable title rewrites — this compiles to allow_title:false # and removes any ability to retitle an arbitrary PR. title: false - # NOTE: gh-aw v0.79.8 does NOT emit required-title-prefix/required-labels into + # NOTE: gh-aw v0.80.9 (the pinned compiler) does NOT emit required-title-prefix/required-labels into # the compiled config for update-pull-request (verified against the lock — it # silently drops them, unlike add-comment / push-to-pull-request-branch which # honor them). So which-PR scoping here relies on prompt Hard-Rule 6 + # min-integrity:approved; the residual is body-marker edits only (no code, no # merge, no close), capped at max:3. Revisit required-* if a future gh-aw # compiler emits them for this output. + mark-pull-request-as-ready-for-review: + # Flip a validated draft [ci-fix] PR to ready-for-review once the SPECIFIC test + # this PR was opened to fix is confirmed green in the PR's OWN CI (Step 3.6) — + # even when unrelated legs are red. The workflow is schedule/dispatch-triggered + # (no triggering-PR context), so target "*" lets the agent name the PR number it + # validated. This is a state transition ONLY: it never approves and never merges — + # a human still reviews and merges. + # PER-RUN total (not per-PR): a sweep may validate several PRs' target tests green. + max: 3 + target: "*" + # Hard constraint (defense-in-depth): only ever un-draft THIS workflow's own + # ci-fix PRs — [ci-fix] title prefix AND agentic-workflows label — so a confused + # or prompt-injected agent cannot mark an arbitrary PR ready. (If the v0.80.9 + # compiler silently drops these — as documented for update-pull-request above — + # the Step 3.6 preconditions + min-integrity:approved are the compensating scope + # controls; verify against the lock after compiling.) + required-title-prefix: "[ci-fix] " + required-labels: [agentic-workflows] + add-labels: + # Apply the p/0 priority label to a [ci-fix] PR at the exact moment the loop flips it + # from draft to ready-for-review (Step 3.6 T3) — so a validated, review-ready fix lands + # in the team's p/0 triage queue instead of sitting unseen in the draft backlog. Paired + # 1:1 with mark-pull-request-as-ready-for-review; target "*" lets the agent name the PR + # it just validated. + # allowed = HARD allowlist: the agent may ONLY ever add p/0, nothing else. This caps the + # blast radius of a confused/prompt-injected agent to exactly one benign priority label — + # it can never apply a downstream-triggering or destructive label. + allowed: [p/0] + # PER-RUN total (not per-PR): a sweep may mark several PRs ready in one run; each adds + # one label, so this matches the mark-ready per-run cap (3). + max: 3 + target: "*" + # Hard constraint (defense-in-depth): only ever label THIS workflow's own ci-fix PRs — + # [ci-fix] title prefix AND agentic-workflows label. (If the v0.80.9 compiler drops these + # — as documented for update-pull-request above — the Step 3.6 preconditions + + # min-integrity:approved + the allowed:[p/0] allowlist are the compensating controls.) + required-title-prefix: "[ci-fix] " + required-labels: [agentic-workflows] timeout-minutes: 90 @@ -339,33 +386,48 @@ through `safe-outputs`. 5. **One issue = one outcome per run.** Exactly one of: respond to a maintainer's change-request on the open PR (Track C, Step 3.5.R); open the first fix/help/ de-flake PR; advance an existing PR by one attempt (push a follow-up fix); - surface a validated-green PR for review; annotate an unrelated-flake red; wait + surface a validated-green PR for review; mark a target-validated draft PR + ready-for-review (Step 3.6 T3 — the terminal outcome that supersedes the same + run's surface/annotate precursor line), or record its `already-ready` + steady-state no-op; annotate an unrelated-flake red; wait (CI not yet settled); or a recorded skip (the dedicated needs-human PR is deferred — the attempt cap records a skip instead; see Step 6). Always prefer advancing or opening a PR over a skip when a non-mute diff is producible. 6. **All writes via `safe-outputs`.** Allowed outputs: `create_pull_request` (first attempt only), `push_to_pull_request_branch` (advance an existing PR by one attempt), `update_pull_request` (bump the attempt marker / refresh the - prior-attempts table), and `add_comment` (progress notes on the `[ci-fix]` PR - ONLY). NEVER comment on the tracking issue (issues are locked by + prior-attempts table), `add_comment` (progress notes on the `[ci-fix]` PR + ONLY), `mark_pull_request_as_ready_for_review` (flip a target-validated draft + `[ci-fix]` PR from draft to ready — Step 3.6 T3 only), and `add_labels` (add + ONLY the `p/0` label, ONLY on that same draft→ready transition — Step 3.6 T3). + NEVER comment on the tracking issue (issues are locked by `.github/workflows/ci-scan-lock-issues.yml`) — `add_comment` targets the PR. No `gh pr create`, no manual `git push`. **Defense-in-depth:** `add_comment` and `update_pull_request` use `target: "*"` (the agent supplies the PR number). - `add_comment` is now config-locked to `required-title-prefix: "[ci-fix] "` + + `add_comment`, `mark_pull_request_as_ready_for_review`, and `add_labels` are + config-locked to `required-title-prefix: "[ci-fix] "` + `required-labels: [agentic-workflows]` (hard-enforced by the handler, same as - `push_to_pull_request_branch`), so a comment can only ever land on THIS - workflow's own PRs. `update_pull_request` CANNOT be config-locked to a - title/label in gh-aw v0.79.8 (the compiler silently drops `required-*` for that - output — verified against the lock), so it keeps `title: false` (no retitles; - body-marker edits only) plus this prompt-level guard. Before emitting either, - VERIFY the target PR carries BOTH the `[ci-fix]` title prefix AND the - `agentic-workflows` label; never comment on or edit the body of any PR that - lacks both. + `push_to_pull_request_branch`), and `add_labels` additionally has an + `allowed: [p/0]` allowlist so it can ONLY ever add `p/0` — so these can only + ever land on THIS workflow's own PRs. `update_pull_request` CANNOT be + config-locked to a title/label in gh-aw v0.80.9 (the compiler silently drops + `required-*` for that output — verified against the lock), so it keeps + `title: false` (no retitles; body-marker edits only) plus this prompt-level + guard. Before emitting ANY of these, VERIFY the target PR carries BOTH the + `[ci-fix]` title prefix AND the `agentic-workflows` label; never comment on, + edit, un-draft, or label any PR that lacks both. 7. **Per-run safe-output caps (per RUN, not per PR).** Each output type is capped per run: `create_pull_request` 3, `push_to_pull_request_branch` 3, - `add_comment` 3, `update_pull_request` 3. Note an ADVANCE spends one push **and** - one update **and** one comment, so ≤ 3 advances/run; surfacing a green or - annotating a flake spends one comment. When a bucket is exhausted, do NOT keep + `add_comment` 6, `update_pull_request` 3, `mark_pull_request_as_ready_for_review` + 3, `add_labels` 3 (counts LABELS, not calls). Note an ADVANCE spends one push + **and** one update **and** one comment, so ≤ 3 advances/run; surfacing a green or + annotating a flake spends one comment; a **mark-ready (Step 3.6 T3)** of a + still-draft PR spends TWO comments (the Step 3 ✅ surface **and** the T3 🎯 + readiness note) **and** one mark-ready **and** one label as an ALL-OR-NOTHING set + (T3 pre-checks those buckets and defers the whole PR if any is exhausted — never + mark a PR ready without its 🎯 audit comment). `add_comment` is therefore sized 6 + (not 3) so ~3 draft flips, at 2 comments each, can land in one sweep instead of the + shared comment bucket starving the otherwise-idle mark-ready/add_labels buckets. When a bucket is exhausted, do NOT keep emitting (extras are silently dropped) — record `skipped: per-run cap reached; deferring PR # to next cycle` for each remaining PR so the drop is a deliberate, logged decision. The continuous loop picks the deferred PRs up next @@ -409,8 +471,9 @@ For every open tracking issue in scope, converge on exactly one outcome: | Open first help-wanted draft `[ci-fix]` PR | No open PR yet; a plausible candidate exists but cannot be runner-validated (device/UI tests) (attempt 1) | | Open first de-flake draft `[ci-fix]` PR | No open PR yet; failure is intermittent (green-on-retry) from a genuine test-quality defect; a deterministic-synchronization fix is producible (attempt 1, Step 4.7 bucket b) | | Advance an existing PR (push attempt N+1) | Open PR's own CI settled red *because of the fix itself*, no human engaged, marker < 10 — push a NEW distinct fix onto the same branch (Step 3.5 → 5.6) | -| Surface a validated-green PR | Open PR's own CI settled green — comment "validated, attempt N/10; ready for review" and do NOT advance (Step 3.5) | -| Annotate an unrelated-flake red | Open PR is red only on baseline-flake / unrelated legs — comment which leg needs a re-run; do NOT burn an attempt (Step 3.5) | +| Surface a validated-green PR | Open PR's own CI settled green — comment "validated, attempt N/10; ready for review" and do NOT advance (Step 3.5), then mark the draft PR ready for review once the fixed test is confirmed green (Step 3.6) | +| Annotate an unrelated-flake red | Open PR is red only on baseline-flake / unrelated legs — comment which leg needs a re-run; do NOT burn an attempt (Step 3.5); if the SPECIFIC fixed test is confirmed green on that build, still mark the draft PR ready for review (Step 3.6) | +| Mark a validated PR ready for review | The specific test this PR fixed is confirmed green in the PR's own CI (even if unrelated legs are red) — comment the target-test result and transition the draft PR to ready for review; never approve or merge (Step 3.6) | | Wait | Open PR's CI is pending / not yet settled on the current head SHA — do nothing this cycle (Step 3.5) | | Hand-off skip (attempt cap) | Marker == 10 and the signature still reproduces — stop and defer to humans; the dedicated `[ci-fix][needs-human]` PR is planned but currently deferred (Step 6) | | Recorded skip | Visual-regression, human engaged, already-handled, fixed-in-latest-build, infra-flake, out-of-bounds, only-mute-available, or no novel approach producible | @@ -685,14 +748,34 @@ Run these gates in order — the FIRST that fires decides this cycle's outcome: 1. **Human engaged.** If `C.humanEngaged` → `skipped: human engaged on PR #; deferring` and stop. Never fight a human reviewer — the intentional hand-off boundary still holds the moment a person touches the PR. -2. **CI not settled / unknown / incomplete prefetch.** If `C.checksSettled == false` - OR `C.overallConclusion` is `pending`, `neutral`, or `unknown`, OR - `C.dataComplete == false` → the fix's CI has not finished on the current head SHA - (`C.headSha`), or the prefetch could not fully read this PR (a partial read can - understate `humanEngaged` / `attempt`). `skipped: PR # CI pending / prefetch - incomplete on ; waiting` and stop. *(Round 1: this is where a - maintainer `/azp run` is awaited — the `/azp`-gated uitests/devicetests legs will - not have run until a human kicks them.)* +2. **CI not settled — but validate the target test first (target-focused readiness).** + If `C.checksSettled == false` OR `C.overallConclusion` is `pending`, `neutral`, or + `unknown`, OR `C.dataComplete == false` → the *overall* build has not finished on the + current head SHA (`C.headSha`), or the prefetch could not fully read this PR (a partial + read can understate `humanEngaged` / `attempt`). Unrelated legs still draining must NOT + keep an already-proven fix parked as a draft, so before waiting, try the target-focused + fast-path: + - **2a — Target-green fast-path (draft `[ci-fix]` PRs only).** If `C.dataComplete == + true` AND the Step 3.6 preconditions hold (draft PR, `[ci-fix] ` title prefix AND the + `agentic-workflows` label), run Step 3.6 **T1–T2** against `C.headSha` now: identify + the target test(s) and read the PR's OWN build **timeline / per-leg status** (anonymous + `_apis/build` — never `_apis/test`, per Hard-Rule 8). If EVERY target test is + **VALIDATED-GREEN** (per T2's all-platform definition below — the target's category leg + is `succeeded` on every platform that runs it, failed on none, and NO target platform + family the pipeline covers is still pending/unconcluded, per T2's "no platform left + unverified" rule; a platform that simply has no leg for the target's category — the test + doesn't run there — is fine, not a gap) AND every leg in + `C.failedLegs` that has ALREADY concluded classifies as **unrelated flake** (Step 4 / + Step 4.7 method — the fix is implicated in NO completed red), then the fix is proven + regardless of unrelated *pending* legs → run **Step 3.6 T3** (mark ready + 🎯 comment) + and stop. This early-out NEVER advances an attempt and NEVER acts on a red that could + be the fix's fault. + - **2b — Otherwise WAIT.** If the target test has not yet executed (pending/absent on + its leg), or a completed red is (or may be) caused by the fix, or `C.dataComplete == + false`, or this PR has no identifiable target test → `skipped: PR # CI pending / + target not yet validated on ; waiting` and stop. *(Round 1: this is where a + maintainer `/azp run` is awaited — the `/azp`-gated uitests/devicetests legs will not + have run until a human kicks them.)* 3. **Green → surface for review.** If `C.overallConclusion == "success"`: the checks that RAN are green. **Primary-gate check first:** the deterministic `success` verdict only certifies "at least one green check, nothing failing or @@ -704,18 +787,31 @@ Run these gates in order — the FIRST that fires decides this cycle's outcome: # primary CI gate (maui-pr) not green on ; waiting` and stop — do NOT surface. (The `/azp`-gated `maui-pr-uitests` (def 313) / `maui-pr-devicetests` (def 314) legs MAY still be un-run — that is expected and is named below, not a - reason to withhold the surface.) **Idempotency:** scan the PR's existing comments - for a prior bot `✅ … validated … on ` note for THIS head SHA — if one - already exists, record `already-surfaced PR # (head )` and stop - WITHOUT re-commenting (re-surfacing the same green every tick is noise and burns - the per-run comment budget other PRs need). Otherwise resolve the attempt number from - `C.effectiveAttempt` (the authoritative max(marker, bot-commit) counter; omit the - number only if it is somehow indeterminate). **Dry-run gate (Step 0):** if `dry_run == "true"`, do NOT emit any comment — instead print the intended `✅` validated-green body to the run log, tally `dry-run: would-surface-green PR #`, and stop. Otherwise `add_comment` on PR #: `✅ Attempt /10 validated - — the fix's CI is green on . Ready for human review.` - Name any `/azp`-gated legs (uitests def 313 / devicetests def 314) that have not - run and still need a maintainer `/azp run`. Do NOT advance. Record - `surfaced-green PR # (attempt /10)` and stop. *(This directly - attacks the real bottleneck — no reviews — so it is the highest-value outcome.)* + reason to withhold the surface.) **Comment idempotency + dry-run suppress the ✅ + comment ONLY — neither skips Step 3.6.** Scan the PR's existing comments for a prior + bot `✅ … validated … on ` note for THIS head SHA; if one exists, set + `SKIP_SURFACE_COMMENT = true`. Resolve the attempt number from `C.effectiveAttempt` + (the authoritative max(marker, bot-commit) counter; omit the number only if it is + somehow indeterminate). Then post the surface comment UNLESS suppressed: + - if `dry_run == "true"`: do NOT emit — print the intended `✅` validated-green body to + the run log and tally `dry-run: would-surface-green PR #`; + - else if `SKIP_SURFACE_COMMENT`: do NOT re-post — record `already-surfaced PR # + (head )` (re-surfacing the same green every tick is noise and burns the + per-run comment budget other PRs need); + - else `add_comment` on PR #: `✅ Attempt /10 validated — the fix's CI is + green on .` naming any `/azp`-gated legs (uitests def 313 / devicetests def + 314) that have not run and still need a maintainer `/azp run`; record `surfaced-green + PR # (attempt /10)`. (Do NOT assert "ready for human review" on a PR that + is still a draft — Step 3.6, not this comment, owns the draft→ready flip and posts its + own 🎯 announcement when it fires.) + Do NOT advance. Then — in ALL of the above cases — run **Step 3.6** (target-test + readiness gate) for this PR before stopping: its `/azp`-gated target legs may conclude + green on this SAME head SHA on a LATER sweep (an `/azp run` adds no commit), and Step + 3.6 is the ONLY place the draft→ready flip happens, so it MUST re-evaluate every sweep — + never `stop` here before it. *(This directly attacks the real bottleneck — no reviews — + so it is the highest-value outcome.)* 4. **Red → classify caused-by-fix vs unrelated-flake.** If `C.overallConclusion == "failure"`, analyze the PR's OWN failing build (NOT `main`): find the AzDO `maui-pr` build for this PR (filter builds by `branchName=refs/pull//merge`, @@ -723,18 +819,26 @@ Run these gates in order — the FIRST that fires decides this cycle's outcome: method and Step 4.7 flake buckets to `C.failedLegs`. - **Unrelated flake only** (every failed leg is known-flaky / infra / a pre-existing baseline red NOT introduced by the fix): do NOT burn an attempt. - **Idempotency first:** scan the PR's existing comments for a prior bot - `♻️ … unrelated flake … on ` note for THIS head SHA — if one already - exists, record `already-annotated-flake PR # (head )` and stop - WITHOUT re-commenting (re-annotating the same flake every 12h sweep is noise and - burns the per-run comment budget other PRs need). Otherwise resolve the attempt - number from `C.effectiveAttempt` (the authoritative max(marker, bot-commit) - counter), omitting the `Attempt /10:` prefix only if it is somehow - indeterminate. **Dry-run gate (Step 0):** if `dry_run == "true"`, do NOT emit any comment — instead print the intended `♻️` unrelated-flake body to the run log, tally `dry-run: would-annotate-flake PR #`, and stop. Otherwise `add_comment` on PR #: `♻️ Attempt /10: red is - unrelated flake on leg(s) () on ; the fix itself is not - implicated. A maintainer re-run (/azp run ) should clear it.` Record - `annotated-flake PR # (head )` and stop. *(Round 1: human re-runs; - Round 2: auto re-trigger.)* + **Comment idempotency + dry-run suppress the ♻️ comment ONLY — neither skips Step + 3.6.** Scan the PR's existing comments for a prior bot `♻️ … unrelated flake … on + ` note for THIS head SHA; if one exists, set `SKIP_FLAKE_COMMENT = true`. + Resolve the attempt number from `C.effectiveAttempt` (the authoritative max(marker, + bot-commit) counter), omitting the `Attempt /10:` prefix only if it is + somehow indeterminate. Then post the flake note UNLESS suppressed: + - if `dry_run == "true"`: do NOT emit — print the intended `♻️` unrelated-flake body + to the run log and tally `dry-run: would-annotate-flake PR #`; + - else if `SKIP_FLAKE_COMMENT`: do NOT re-post — record `already-annotated-flake PR + # (head )` (re-annotating the same flake every 12h sweep is noise and + burns the per-run comment budget other PRs need); + - else `add_comment` on PR #: `♻️ Attempt /10: red is unrelated flake on + leg(s) () on ; the fix itself is not implicated. A + maintainer re-run (/azp run ) should clear it.` record `annotated-flake + PR # (head )`. + Then — in ALL of the above cases — run **Step 3.6** (target-test readiness gate) for + this PR before stopping: its `/azp`-gated target legs may conclude green on this SAME + head SHA on a LATER sweep, and Step 3.6 is the ONLY place the draft→ready flip + happens, so it MUST re-evaluate every sweep — never `stop` here before it. *(Round 1: + human re-runs; Round 2: auto re-trigger.)* - **Caused by the fix** (a failed leg still matches the original target signature, or the fix introduced a NEW failure): advance an attempt. - **Attempt count.** `attempt = C.effectiveAttempt` — the authoritative @@ -752,6 +856,188 @@ Run these gates in order — the FIRST that fires decides this cycle's outcome: from every prior commit on this branch**, emitted via the Step 5.6 ADVANCE path (push onto the same PR). +#### Step 3.6 — Target-test verification & mark-ready gate + +Reached from the Step 3 **green-surface** branch, the Step 4 **unrelated-flake** branch +(AFTER that branch has posted its comment), and the Step 3.5 **gate-2 target-focused +fast-path** (2a — while unrelated legs are still pending). Purpose: confirm the SPECIFIC +test(s) this PR was opened to fix now PASS in the PR's own CI, and — only when they do +— transition the draft `[ci-fix]` PR to **ready for review** so a maintainer sees a +validated fix instead of a draft. This is the ONLY place the loop flips draft→ready; it +is a state transition, **never** an approval or a merge (a human still reviews and +merges). Overall red on *unrelated* legs must NOT gate readiness — we validate the fix, +not the base branch's flakiness. + +**Preconditions** (ALL must hold; otherwise record `skipped: readiness N/A PR # +()` and stop this gate): +- `C.isDraft == true` — if the PR is already ready-for-review, the draft→ready + transition is already done; do NOT re-mark **and do NOT re-apply `p/0`**. `p/0` is applied + exactly ONCE, atomically with the draft→ready flip (T3 — all-or-nothing with the 🎯 audit + comment + mark-ready), so an already-ready loop PR that is MISSING `p/0` has almost + certainly had it **removed by a maintainer** de-prioritizing the PR — a pure triage action + the human-engagement guard (which inspects comments/reviews/commits, NOT label changes) + cannot see. Re-adding `p/0` every sweep would fight that maintainer indefinitely, violating + the loop's "never override a human" contract. So the loop does NOT reconcile the label: + record `already-ready PR #` and stop this gate. (Trade-off: on the rare occasion a + transient API error drops `p/0` at flip time *after* the T3 atomic pre-check passed, it is + not auto-re-added — but the PR is still ready-for-review with its 🎯 audit comment, and + re-adding `p/0` is a trivial manual action; that is strictly preferable to steam-rolling a + maintainer's deliberate de-prioritization.) +- The PR is unmistakably THIS workflow's own: `[ci-fix] ` title prefix AND the + `agentic-workflows` label (mirrors the safe-output lock; the compensating scope + control if the v0.80.9 compiler drops the declarative required-*). +- You reached this gate from Step 3 (green), Step 4 (**unrelated-flake**), or the Step 3.5 + gate-2 **target-focused fast-path** (2a) — i.e. the fix is NOT implicated in any red that + has concluded. If Step 4 classified a red as **caused by the fix** (ADVANCE), do NOT run + this gate — advance the attempt instead. + +**T1 — Identify the target test(s).** From the `[ci-scan]` issue signature the fix +addresses (and the PR's own diff), extract the fully-qualified test method name(s) the +fix targets — e.g. `SafeAreaShouldWorkOnAllShellTabs`. For a de-flake it is the +de-flaked test; for a product fix it is the originally-failing test(s). If NO specific +test can be identified (e.g. a product build-break rather than a test failure), this is a +**build-only fix** — there is no single test to validate cross-platform, so validate at +**whole-build** granularity rather than stopping (Step 3 only *comments*; Step 3.6 is the +sole draft→ready flip, so a build-only fix is undrafted HERE or not at all). We only reach +this gate from Step 3 (green) / Step 4 (unrelated-flake) / gate-2a, so the fix is not +implicated in any concluded red; additionally require, via the T2 timeline method on +`C.headSha`, that the primary build pipeline `maui-pr` (def 302) has CONCLUDED with EVERY +one of its platform build legs `succeeded`/`completed` and NONE failed/canceled or still +pending — the build is green on **every** platform, not just the originally-broken one (the +cross-platform guard, applied to the build instead of a test). A build-only fix is undrafted +ONLY on the strength of the auto-running `maui-pr` (def 302) whole-build green, which the +loop CAN observe: if the PR's `[ci-scan]` issue signature or its own diff indicates the +ORIGINATING failure was in a `/azp`-gated pipeline (`maui-pr-uitests` def 313 or +`maui-pr-devicetests` def 314) rather than `maui-pr` (def 302), do NOT undraft on +`maui-pr`-green alone — those pipelines do not auto-run on this PR (GITHUB_TOKEN cannot +trigger them), so a green `maui-pr` build is NOT evidence the gated build break is fixed; +record `skipped: build-only fix PR # targets gated pipeline () not run — +deferring to human` and stop this gate. Otherwise, set `TARGET := "the maui-pr build +(build-only fix — no single target test)"` and proceed to **T3** to mark ready. If any `maui-pr` build leg is still unconcluded, record `skipped: build-only fix PR +# not yet whole-build green (leg(s) pending)` and stop this gate WITHOUT marking +ready (a green subset is not enough — a still-pending build leg could yet fail). + +**T2 — Verify the target test's platform legs are green (anonymous, leg-level).** Per-test +outcomes come from the AzDO **test-results** API (`_apis/test/...`), which is **NOT reachable +anonymously — Hard-Rule 8**: those endpoints 302-redirect to a sign-in page on this runner, so +a `curl` returns an HTML redirect (not JSON) and every target test would look "not executed". +Validate instead at **leg granularity** using ONLY the anonymous `_apis/build/...` timeline +(Hard-Rule 8) — a `succeeded` category leg means every test in that category **that actually +ran** passed on that platform. This is a strong bar **only if the target test genuinely +executes**: a job can go green while one specific test is skipped at runtime (`[Ignore]`, +`Assert.Ignore()`, `Assert.Inconclusive()`, a `Skip=`/conditional `[Fact]`, an `#if`-out, or +an early `return` before the asserts). So before trusting a green leg as proof the target +passed, **confirm from the PR's OWN diff that the fix does NOT skip, ignore, disable, or +short-circuit the target test** (it adds no `[Ignore]`/`[Explicit]`, `Assert.Ignore`/ +`Assert.Inconclusive`, `Skip=`, category-exclusion, `#if`-out, or early `return` around the +target). If the fix could cause the target to be runtime-skipped rather than genuinely pass, +leg-level green is NOT sufficient evidence — record `skipped: target test may be +runtime-skipped by this fix (diff adds ); leg-green insufficient, deferring to human +on PR #` and stop this gate WITHOUT marking ready. Otherwise (the target genuinely runs +and asserts, as a de-flake or product fix does) a green category leg cannot hide a +target-test failure, so treat it as authoritative. + +Map the target test to the CI leg(s) that run it. A leg name encodes platform + test category +(e.g. `Android UITests SafeAreaEdges,Shadow`, `iOS UITests SafeAreaEdges,Shadow`, `macOS +UITests SafeAreaEdges,Shadow`, `Windows UITests SafeAreaEdges,Shadow`). Determine the target +test's UI-test category — its `[Category(UITestCategories.X)]` in the test/HostApp source, +visible in the PR diff or the test file — to know which leg-name substring identifies its legs; +for a device test, the per-platform device-test legs. + +Using the SAME build-discovery as Step 4 (filter AzDO builds by `branchName=refs/pull//merge` +or `sourceVersion == C.headSha`), read each build's **timeline** on `C.headSha` for the +pipeline(s) that RUN the target test — `maui-pr` (def 302) for unit/integration tests, +`maui-pr-uitests` (def 313) for Appium UI tests, `maui-pr-devicetests` (def 314) for device +tests: + +```bash +ORG=dnceng-public; PROJ=public; BUILD= +# Anonymous + Hard-Rule-8-compliant. NEVER call _apis/test/... (it 302-redirects to sign-in). +# Each type=="Job" record is one platform leg: result (succeeded/failed/canceled/null), +# state (completed/inProgress/pending), name (encodes " UITests "): +curl -s "https://dev.azure.com/$ORG/$PROJ/_apis/build/builds/$BUILD/timeline?api-version=7.1" \ + | tee /tmp/gh-aw/agent/timeline_${BUILD}.json \ + | jq -r '.records[] | select(.type=="Job") | "\(.result)\t\(.state)\t\(.name)"' +``` + +(The prefetch already exposes per-leg status in `C.failedLegs` / the checks context, and +`gh pr checks ` lists the same per-leg rows with platform+category in the name — use +whichever is handy; the build timeline is the authoritative cross-check on the exact build.) + +Treat a target test as **VALIDATED-GREEN** only if, on `C.headSha`, the leg that runs its +category is green on **every platform it runs on** — a fix that repairs one platform must not +silently regress the same test's leg on another, so a green leg on the originally-red platform +alone is NOT enough. Across the target pipeline's platform legs (for a UI test: the Android, +iOS, Windows, and macOS/MacCatalyst legs running the target's category; for a device test: each +device-test platform), require ALL of: +- the target's category leg is `result == "succeeded"` **and** `state == "completed"` on + **every** platform that runs it, AND +- that leg is `failed` / `canceled` / aborted on **NO** platform, AND +- **no platform is left unverified.** For each platform family the target pipeline covers, that + platform's category leg must have CONCLUDED (`state == "completed"`) on `C.headSha`. If a + platform simply has no leg for the target's category, the test does not run there — that is + fine, not a gap. But if a platform's category leg has **not concluded** (`state` is + `inProgress` / pending, or an `/azp`-gated `maui-pr-uitests` / `maui-pr-devicetests` leg that + has not been kicked), the test's status on that platform is UNKNOWN → the fix is NOT yet + validated across platforms: record `skipped: target test green on but + not yet verified on (leg(s) pending / need /azp run) on PR #` + and stop this gate WITHOUT marking ready. + +A target test whose category leg never concluded on ANY platform (**not executed** anywhere — +e.g. its `/azp`-gated pipeline has not been kicked) is likewise NOT validated: record `skipped: +target test not yet executed on PR # ( not run — needs /azp run)` and stop +this gate WITHOUT marking ready. Do NOT overclaim — a green *sibling* leg (a different category +on the same platform) is not the target's leg, and a green leg on one platform is not a pass on +the others. + +**T3 — Mark ready + report.** If EVERY target test is VALIDATED-GREEN. The 🎯 comment, +the mark-ready, and the `p/0` label are THREE SEPARATE safe-outputs — the comment +existing does NOT prove the mark-ready took effect, so they are tracked independently: + +- **Comment idempotency (dup-suppress only — never gates the mark-ready).** We only reach + T3 while `C.isDraft == true` (the precondition stops an already-ready PR before here). So + if a prior bot comment for THIS head that carries the leading `🎯` anchor AND the + contiguous phrase `validated green on ` already exists (BOTH T3 variants — + `🎯 Target test validated green on ` and the build-only `🎯 Build validated green on + ` — contain that exact contiguous phrase, so this recognizes both; keying on + `target test` alone would instead miss the build-only comment and re-post it every sweep. + **Require the `🎯` anchor** so the match can NEVER be loosened to a gapped + `validated…green…on` form that would false-positive on the Step 3 `✅ … validated — the + fix's CI is green on ` surface comment and wrongly suppress the audit 🎯), the comment + landed on an earlier sweep but the **mark-ready did NOT take + effect** (the PR is still a draft) — set `SUPPRESS_COMMENT = true` (do not re-post the + duplicate 🎯 comment) but STILL complete the mark-ready + label below. Never treat the + comment's existence as "already marked ready" while the PR is still a draft. Record this + case as `re-marking PR # (🎯 present; mark-ready did not take on a prior sweep)`. +- **Atomicity budget pre-check (Hard-Rule 7).** Determine the outputs this gate will emit: + `mark_pull_request_as_ready_for_review` + `add_labels` always, plus `add_comment` unless + `SUPPRESS_COMMENT`. Verify a free per-run slot remains in EACH bucket you are about to + use. If ANY required bucket is exhausted, emit NONE of them — record `skipped: per-run + cap reached; deferring mark-ready PR # to next cycle` and stop this gate. Never mark a + PR ready (or label it) unless its 🎯 audit comment is GUARANTEED to exist for this exact + head SHA — either posted in this same sweep, or (in the `SUPPRESS_COMMENT` re-marking case) + already present from a prior sweep. The outputs you DO emit either all land or all defer + together; the audit trail must never be absent, but it is NOT re-posted when it already exists. +- **Dry-run gate (Step 0):** if `dry_run == "true"`, emit NOTHING — print the intended + readiness comment and "would mark ready + add p/0" to the run log, tally `dry-run: + would-mark-ready PR #`, and stop. +- Otherwise emit for THIS PR number ``: + 1. `add_comment` (ONLY if not `SUPPRESS_COMMENT`): `🎯 Target test validated green on + — passed on ALL platforms it runs on (, + buildId ). — not + caused by this fix."> Transitioning this PR from draft to ready for review and adding + `p/0`; a maintainer still reviews and merges.` (For a **build-only fix** — the T1 + whole-build fallback, no single target test — phrase the first clause as `🎯 Build + validated green on — the maui-pr build passed on ALL platforms (buildId + ).` instead of naming a test.) + 2. `mark_pull_request_as_ready_for_review` with `reason:` a one-line justification + naming the validated test(s) and ``. + 3. `add_labels` with `labels: ["p/0"]` for PR # — put the now-review-ready fix into + the team's p/0 priority queue so it is triaged, not lost in the draft backlog. (If + the PR somehow already carries `p/0`, this is a harmless no-op.) +- Record `marked-ready PR # (target green on ALL platforms on , + labeled p/0)` and stop. + #### Step 3.5.R — Maintainer change-request response (Track C) Reached from Step 3.5 gate 0. Goal: when an **eligible human reviewer** (Hard-Rule @@ -1383,7 +1669,19 @@ Filed by [`ci-status-fix`](https://github.com/dotnet/maui/blob/main/.github/work ### Step 8 — Per-issue tally + end-of-run summary -Per issue, append one outcome line to `/tmp/gh-aw/agent/coverage.txt`: +Per issue, append **one** outcome line to `/tmp/gh-aw/agent/coverage.txt` — the +terminal outcome for the cycle. When one cycle produces a chained pair — a PR gets a +same-cycle precursor line and **then** a Step 3.6 readiness line (`marked-ready` on the +draft→ready flip, or `already-ready` on a later steady-state sweep of an already-ready +PR) — record ONLY the terminal Step 3.6 readiness line: it supersedes ANY same-cycle +non-readiness precursor for that PR, so an aggregator keying on one-line-per-issue never +double-counts. The precursor may be a Step 3 green line (`surfaced-green` on the first ✅, +or `already-surfaced` when it already exists) OR — when an unrelated-flake red and a +target-green coincide — a Step 4 `annotated-flake` line; either way the readiness line is +terminal, and the superseded precursor's signal survives in the PR's 🎯/♻️ comment so no +information is lost. This covers the flip cycle (`surfaced-green` → `marked-ready`), the +post-flip steady state (`already-surfaced` → `already-ready`), and the flake-coincident +flip (`annotated-flake` → `marked-ready`): ``` # main attempt- @@ -1394,8 +1692,14 @@ Per issue, append one outcome line to `/tmp/gh-aw/agent/coverage.txt`: `advance-PR # attempt /10` (ADVANCE mode — new commit pushed to the existing PR), `surfaced-green PR #` (fix's own CI went green; commented for review, did not advance), `annotated-flake PR #` (red was unrelated flake; -commented, attempt NOT burned), `waiting PR #` (CI not settled yet), -`dry-run: would-`, `skipped: `. (The +commented, attempt NOT burned), `marked-ready PR #` (the specific fixed test +was confirmed green in the PR's own CI; draft flipped to ready for review), +`already-surfaced PR #` (the fix's ✅ green comment already existed this sweep; +re-post suppressed — superseded by the Step 3.6 readiness line when the PR also flips +ready that cycle), `already-ready PR #` (terminal steady state — the PR was flipped +ready on a prior cycle and remains green on an unchanged head; no action taken, +supersedes `already-surfaced`), `waiting PR #` (CI not settled yet), +`dry-run: would-`, `skipped: `. (The `needs-human-PR` outcome is reserved for the deferred hand-off — Step 6 currently records a skip instead, so it is not emitted.)
(head )` and stop. *(Round 1: human re-runs; - Round 2: auto re-trigger.)* + **Comment idempotency + dry-run suppress the ♻️ comment ONLY — neither skips Step + 3.6.** Scan the PR's existing comments for a prior bot `♻️ … unrelated flake … on + ` note for THIS head SHA; if one exists, set `SKIP_FLAKE_COMMENT = true`. + Resolve the attempt number from `C.effectiveAttempt` (the authoritative max(marker, + bot-commit) counter), omitting the `Attempt /10:` prefix only if it is + somehow indeterminate. Then post the flake note UNLESS suppressed: + - if `dry_run == "true"`: do NOT emit — print the intended `♻️` unrelated-flake body + to the run log and tally `dry-run: would-annotate-flake PR #`; + - else if `SKIP_FLAKE_COMMENT`: do NOT re-post — record `already-annotated-flake PR + # (head )` (re-annotating the same flake every 12h sweep is noise and + burns the per-run comment budget other PRs need); + - else `add_comment` on PR #: `♻️ Attempt /10: red is unrelated flake on + leg(s) () on ; the fix itself is not implicated. A + maintainer re-run (/azp run ) should clear it.` record `annotated-flake + PR # (head )`. + Then — in ALL of the above cases — run **Step 3.6** (target-test readiness gate) for + this PR before stopping: its `/azp`-gated target legs may conclude green on this SAME + head SHA on a LATER sweep, and Step 3.6 is the ONLY place the draft→ready flip + happens, so it MUST re-evaluate every sweep — never `stop` here before it. *(Round 1: + human re-runs; Round 2: auto re-trigger.)* - **Caused by the fix** (a failed leg still matches the original target signature, or the fix introduced a NEW failure): advance an attempt. - **Attempt count.** `attempt = C.effectiveAttempt` — the authoritative @@ -762,6 +866,188 @@ Run these gates in order — the FIRST that fires decides this cycle's outcome: from every prior commit on this branch**, emitted via the Step 5.6 ADVANCE path (push onto the same PR). +#### Step 3.6 — Target-test verification & mark-ready gate + +Reached from the Step 3 **green-surface** branch, the Step 4 **unrelated-flake** branch +(AFTER that branch has posted its comment), and the Step 3.5 **gate-2 target-focused +fast-path** (2a — while unrelated legs are still pending). Purpose: confirm the SPECIFIC +test(s) this PR was opened to fix now PASS in the PR's own CI, and — only when they do +— transition the draft `[ci-fix-net11]` PR to **ready for review** so a maintainer sees +a validated fix instead of a draft. This is the ONLY place the loop flips draft→ready; +it is a state transition, **never** an approval or a merge (a human still reviews and +merges). Overall red on *unrelated* legs must NOT gate readiness — we validate the fix, +not the base branch's flakiness. + +**Preconditions** (ALL must hold; otherwise record `skipped: readiness N/A PR # +()` and stop this gate): +- `C.isDraft == true` — if the PR is already ready-for-review, the draft→ready + transition is already done; do NOT re-mark **and do NOT re-apply `p/0`**. `p/0` is applied + exactly ONCE, atomically with the draft→ready flip (T3 — all-or-nothing with the 🎯 audit + comment + mark-ready), so an already-ready loop PR that is MISSING `p/0` has almost + certainly had it **removed by a maintainer** de-prioritizing the PR — a pure triage action + the human-engagement guard (which inspects comments/reviews/commits, NOT label changes) + cannot see. Re-adding `p/0` every sweep would fight that maintainer indefinitely, violating + the loop's "never override a human" contract. So the loop does NOT reconcile the label: + record `already-ready PR #` and stop this gate. (Trade-off: on the rare occasion a + transient API error drops `p/0` at flip time *after* the T3 atomic pre-check passed, it is + not auto-re-added — but the PR is still ready-for-review with its 🎯 audit comment, and + re-adding `p/0` is a trivial manual action; that is strictly preferable to steam-rolling a + maintainer's deliberate de-prioritization.) +- The PR is unmistakably THIS workflow's own: `[ci-fix-net11] ` title prefix AND the + `agentic-workflows` label (mirrors the safe-output lock; the compensating scope + control if the v0.80.9 compiler drops the declarative required-*). +- You reached this gate from Step 3 (green), Step 4 (**unrelated-flake**), or the Step 3.5 + gate-2 **target-focused fast-path** (2a) — i.e. the fix is NOT implicated in any red that + has concluded. If Step 4 classified a red as **caused by the fix** (ADVANCE), do NOT run + this gate — advance the attempt instead. + +**T1 — Identify the target test(s).** From the `[ci-scan-net11]` issue signature the fix +addresses (and the PR's own diff), extract the fully-qualified test method name(s) the +fix targets — e.g. `SafeAreaShouldWorkOnAllShellTabs`. For a de-flake it is the +de-flaked test; for a product fix it is the originally-failing test(s). If NO specific +test can be identified (e.g. a product build-break rather than a test failure), this is a +**build-only fix** — there is no single test to validate cross-platform, so validate at +**whole-build** granularity rather than stopping (Step 3 only *comments*; Step 3.6 is the +sole draft→ready flip, so a build-only fix is undrafted HERE or not at all). We only reach +this gate from Step 3 (green) / Step 4 (unrelated-flake) / gate-2a, so the fix is not +implicated in any concluded red; additionally require, via the T2 timeline method on +`C.headSha`, that the primary build pipeline `maui-pr` (def 302) has CONCLUDED with EVERY +one of its platform build legs `succeeded`/`completed` and NONE failed/canceled or still +pending — the build is green on **every** platform, not just the originally-broken one (the +cross-platform guard, applied to the build instead of a test). A build-only fix is undrafted +ONLY on the strength of the auto-running `maui-pr` (def 302) whole-build green, which the +loop CAN observe: if the PR's `[ci-scan]` issue signature or its own diff indicates the +ORIGINATING failure was in a `/azp`-gated pipeline (`maui-pr-uitests` def 313 or +`maui-pr-devicetests` def 314) rather than `maui-pr` (def 302), do NOT undraft on +`maui-pr`-green alone — those pipelines do not auto-run on this PR (GITHUB_TOKEN cannot +trigger them), so a green `maui-pr` build is NOT evidence the gated build break is fixed; +record `skipped: build-only fix PR # targets gated pipeline () not run — +deferring to human` and stop this gate. Otherwise, set `TARGET := "the maui-pr build +(build-only fix — no single target test)"` and proceed to **T3** to mark ready. If any `maui-pr` build leg is still unconcluded, record `skipped: build-only fix PR +# not yet whole-build green (leg(s) pending)` and stop this gate WITHOUT marking +ready (a green subset is not enough — a still-pending build leg could yet fail). + +**T2 — Verify the target test's platform legs are green (anonymous, leg-level).** Per-test +outcomes come from the AzDO **test-results** API (`_apis/test/...`), which is **NOT reachable +anonymously — Hard-Rule 8**: those endpoints 302-redirect to a sign-in page on this runner, so +a `curl` returns an HTML redirect (not JSON) and every target test would look "not executed". +Validate instead at **leg granularity** using ONLY the anonymous `_apis/build/...` timeline +(Hard-Rule 8) — a `succeeded` category leg means every test in that category **that actually +ran** passed on that platform. This is a strong bar **only if the target test genuinely +executes**: a job can go green while one specific test is skipped at runtime (`[Ignore]`, +`Assert.Ignore()`, `Assert.Inconclusive()`, a `Skip=`/conditional `[Fact]`, an `#if`-out, or +an early `return` before the asserts). So before trusting a green leg as proof the target +passed, **confirm from the PR's OWN diff that the fix does NOT skip, ignore, disable, or +short-circuit the target test** (it adds no `[Ignore]`/`[Explicit]`, `Assert.Ignore`/ +`Assert.Inconclusive`, `Skip=`, category-exclusion, `#if`-out, or early `return` around the +target). If the fix could cause the target to be runtime-skipped rather than genuinely pass, +leg-level green is NOT sufficient evidence — record `skipped: target test may be +runtime-skipped by this fix (diff adds ); leg-green insufficient, deferring to human +on PR #` and stop this gate WITHOUT marking ready. Otherwise (the target genuinely runs +and asserts, as a de-flake or product fix does) a green category leg cannot hide a +target-test failure, so treat it as authoritative. + +Map the target test to the CI leg(s) that run it. A leg name encodes platform + test category +(e.g. `Android UITests SafeAreaEdges,Shadow`, `iOS UITests SafeAreaEdges,Shadow`, `macOS +UITests SafeAreaEdges,Shadow`, `Windows UITests SafeAreaEdges,Shadow`). Determine the target +test's UI-test category — its `[Category(UITestCategories.X)]` in the test/HostApp source, +visible in the PR diff or the test file — to know which leg-name substring identifies its legs; +for a device test, the per-platform device-test legs. + +Using the SAME build-discovery as Step 4 (filter AzDO builds by `branchName=refs/pull//merge` +or `sourceVersion == C.headSha`), read each build's **timeline** on `C.headSha` for the +pipeline(s) that RUN the target test — `maui-pr` (def 302) for unit/integration tests, +`maui-pr-uitests` (def 313) for Appium UI tests, `maui-pr-devicetests` (def 314) for device +tests: + +```bash +ORG=dnceng-public; PROJ=public; BUILD= +# Anonymous + Hard-Rule-8-compliant. NEVER call _apis/test/... (it 302-redirects to sign-in). +# Each type=="Job" record is one platform leg: result (succeeded/failed/canceled/null), +# state (completed/inProgress/pending), name (encodes " UITests "): +curl -s "https://dev.azure.com/$ORG/$PROJ/_apis/build/builds/$BUILD/timeline?api-version=7.1" \ + | tee /tmp/gh-aw/agent/timeline_${BUILD}.json \ + | jq -r '.records[] | select(.type=="Job") | "\(.result)\t\(.state)\t\(.name)"' +``` + +(The prefetch already exposes per-leg status in `C.failedLegs` / the checks context, and +`gh pr checks ` lists the same per-leg rows with platform+category in the name — use +whichever is handy; the build timeline is the authoritative cross-check on the exact build.) + +Treat a target test as **VALIDATED-GREEN** only if, on `C.headSha`, the leg that runs its +category is green on **every platform it runs on** — a fix that repairs one platform must not +silently regress the same test's leg on another, so a green leg on the originally-red platform +alone is NOT enough. Across the target pipeline's platform legs (for a UI test: the Android, +iOS, Windows, and macOS/MacCatalyst legs running the target's category; for a device test: each +device-test platform), require ALL of: +- the target's category leg is `result == "succeeded"` **and** `state == "completed"` on + **every** platform that runs it, AND +- that leg is `failed` / `canceled` / aborted on **NO** platform, AND +- **no platform is left unverified.** For each platform family the target pipeline covers, that + platform's category leg must have CONCLUDED (`state == "completed"`) on `C.headSha`. If a + platform simply has no leg for the target's category, the test does not run there — that is + fine, not a gap. But if a platform's category leg has **not concluded** (`state` is + `inProgress` / pending, or an `/azp`-gated `maui-pr-uitests` / `maui-pr-devicetests` leg that + has not been kicked), the test's status on that platform is UNKNOWN → the fix is NOT yet + validated across platforms: record `skipped: target test green on but + not yet verified on (leg(s) pending / need /azp run) on PR #` + and stop this gate WITHOUT marking ready. + +A target test whose category leg never concluded on ANY platform (**not executed** anywhere — +e.g. its `/azp`-gated pipeline has not been kicked) is likewise NOT validated: record `skipped: +target test not yet executed on PR # ( not run — needs /azp run)` and stop +this gate WITHOUT marking ready. Do NOT overclaim — a green *sibling* leg (a different category +on the same platform) is not the target's leg, and a green leg on one platform is not a pass on +the others. + +**T3 — Mark ready + report.** If EVERY target test is VALIDATED-GREEN. The 🎯 comment, +the mark-ready, and the `p/0` label are THREE SEPARATE safe-outputs — the comment +existing does NOT prove the mark-ready took effect, so they are tracked independently: + +- **Comment idempotency (dup-suppress only — never gates the mark-ready).** We only reach + T3 while `C.isDraft == true` (the precondition stops an already-ready PR before here). So + if a prior bot comment for THIS head that carries the leading `🎯` anchor AND the + contiguous phrase `validated green on ` already exists (BOTH T3 variants — + `🎯 Target test validated green on ` and the build-only `🎯 Build validated green on + ` — contain that exact contiguous phrase, so this recognizes both; keying on + `target test` alone would instead miss the build-only comment and re-post it every sweep. + **Require the `🎯` anchor** so the match can NEVER be loosened to a gapped + `validated…green…on` form that would false-positive on the Step 3 `✅ … validated — the + fix's CI is green on ` surface comment and wrongly suppress the audit 🎯), the comment + landed on an earlier sweep but the **mark-ready did NOT take + effect** (the PR is still a draft) — set `SUPPRESS_COMMENT = true` (do not re-post the + duplicate 🎯 comment) but STILL complete the mark-ready + label below. Never treat the + comment's existence as "already marked ready" while the PR is still a draft. Record this + case as `re-marking PR # (🎯 present; mark-ready did not take on a prior sweep)`. +- **Atomicity budget pre-check (Hard-Rule 7).** Determine the outputs this gate will emit: + `mark_pull_request_as_ready_for_review` + `add_labels` always, plus `add_comment` unless + `SUPPRESS_COMMENT`. Verify a free per-run slot remains in EACH bucket you are about to + use. If ANY required bucket is exhausted, emit NONE of them — record `skipped: per-run + cap reached; deferring mark-ready PR # to next cycle` and stop this gate. Never mark a + PR ready (or label it) unless its 🎯 audit comment is GUARANTEED to exist for this exact + head SHA — either posted in this same sweep, or (in the `SUPPRESS_COMMENT` re-marking case) + already present from a prior sweep. The outputs you DO emit either all land or all defer + together; the audit trail must never be absent, but it is NOT re-posted when it already exists. +- **Dry-run gate (Step 0):** if `dry_run == "true"`, emit NOTHING — print the intended + readiness comment and "would mark ready + add p/0" to the run log, tally `dry-run: + would-mark-ready PR #`, and stop. +- Otherwise emit for THIS PR number ``: + 1. `add_comment` (ONLY if not `SUPPRESS_COMMENT`): `🎯 Target test validated green on + — passed on ALL platforms it runs on (, + buildId ). — not + caused by this fix."> Transitioning this PR from draft to ready for review and adding + `p/0`; a maintainer still reviews and merges.` (For a **build-only fix** — the T1 + whole-build fallback, no single target test — phrase the first clause as `🎯 Build + validated green on — the maui-pr build passed on ALL platforms (buildId + ).` instead of naming a test.) + 2. `mark_pull_request_as_ready_for_review` with `reason:` a one-line justification + naming the validated test(s) and ``. + 3. `add_labels` with `labels: ["p/0"]` for PR # — put the now-review-ready fix into + the team's p/0 priority queue so it is triaged, not lost in the draft backlog. (If + the PR somehow already carries `p/0`, this is a harmless no-op.) +- Record `marked-ready PR # (target green on ALL platforms on , + labeled p/0)` and stop. + #### Step 3.5.R — Maintainer change-request response (Track C) Reached from Step 3.5 gate 0. Goal: when an **eligible human reviewer** (Hard-Rule @@ -1395,7 +1681,19 @@ Filed by [`ci-status-fix-net11`](https://github.com/dotnet/maui/blob/main/.githu ### Step 8 — Per-issue tally + end-of-run summary -Per issue, append one outcome line to `/tmp/gh-aw/agent/coverage.txt`: +Per issue, append **one** outcome line to `/tmp/gh-aw/agent/coverage.txt` — the +terminal outcome for the cycle. When one cycle produces a chained pair — a PR gets a +same-cycle precursor line and **then** a Step 3.6 readiness line (`marked-ready` on the +draft→ready flip, or `already-ready` on a later steady-state sweep of an already-ready +PR) — record ONLY the terminal Step 3.6 readiness line: it supersedes ANY same-cycle +non-readiness precursor for that PR, so an aggregator keying on one-line-per-issue never +double-counts. The precursor may be a Step 3 green line (`surfaced-green` on the first ✅, +or `already-surfaced` when it already exists) OR — when an unrelated-flake red and a +target-green coincide — a Step 4 `annotated-flake` line; either way the readiness line is +terminal, and the superseded precursor's signal survives in the PR's 🎯/♻️ comment so no +information is lost. This covers the flip cycle (`surfaced-green` → `marked-ready`), the +post-flip steady state (`already-surfaced` → `already-ready`), and the flake-coincident +flip (`annotated-flake` → `marked-ready`): ``` # net11.0 attempt- @@ -1406,8 +1704,14 @@ Per issue, append one outcome line to `/tmp/gh-aw/agent/coverage.txt`: `advance-PR # attempt /10` (ADVANCE mode — new commit pushed to the existing PR), `surfaced-green PR #` (fix's own CI went green; commented for review, did not advance), `annotated-flake PR #` (red was unrelated flake; -commented, attempt NOT burned), `waiting PR #` (CI not settled yet), -`dry-run: would-`, `skipped: `. (The +commented, attempt NOT burned), `marked-ready PR #` (the specific fixed test +was confirmed green in the PR's own CI; draft flipped to ready for review), +`already-surfaced PR #` (the fix's ✅ green comment already existed this sweep; +re-post suppressed — superseded by the Step 3.6 readiness line when the PR also flips +ready that cycle), `already-ready PR #` (terminal steady state — the PR was flipped +ready on a prior cycle and remains green on an unchanged head; no action taken, +supersedes `already-surfaced`), `waiting PR #` (CI not settled yet), +`dry-run: would-`, `skipped: `. (The `needs-human-PR` outcome is reserved for the deferred hand-off — Step 6 currently records a skip instead, so it is not emitted.) diff --git a/.github/workflows/ci-status-fix.lock.yml b/.github/workflows/ci-status-fix.lock.yml index 93dc5969cb0d..65511f2b3259 100644 --- a/.github/workflows/ci-status-fix.lock.yml +++ b/.github/workflows/ci-status-fix.lock.yml @@ -1,4 +1,4 @@ -# gh-aw-metadata: {"schema_version":"v4","frontmatter_hash":"f014161967589ebcaf1ac5ec6f3c88ee5a4388d28b55a07dc36c45b7b2aebab6","body_hash":"86c6b2d4c3df13da14c6a3c8b211381fcc9f97bb1951ff7b80588a2c5338e00c","compiler_version":"v0.80.9","strict":true,"agent_id":"copilot","agent_model":"claude-opus-4.8","engine_versions":{"copilot":"1.0.63"}} +# gh-aw-metadata: {"schema_version":"v4","frontmatter_hash":"b0ebf8a2f179900cfe3638e994b27a43cec52bdbdd2331dc1bd4d65762fa2d6b","body_hash":"1825e2b1f7c7bd88241bf37580655489360f9bebc806edd00837a52ce4ad83a0","compiler_version":"v0.80.9","strict":true,"agent_id":"copilot","agent_model":"claude-opus-4.8","engine_versions":{"copilot":"1.0.63"}} # gh-aw-manifest: {"version":1,"secrets":["COPILOT_PAT_0","COPILOT_PAT_1","COPILOT_PAT_2","COPILOT_PAT_3","COPILOT_PAT_4","COPILOT_PAT_5","COPILOT_PAT_6","COPILOT_PAT_7","COPILOT_PAT_8","COPILOT_PAT_9","GH_AW_CI_TRIGGER_TOKEN","GH_AW_GITHUB_MCP_SERVER_TOKEN","GH_AW_GITHUB_TOKEN","GITHUB_TOKEN"],"actions":[{"repo":"actions/checkout","sha":"34e114876b0b11c390a56381ad16ebd13914f8d5","version":"v4"},{"repo":"actions/checkout","sha":"9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0","version":"v7.0.0"},{"repo":"actions/checkout","sha":"de0fac2e4500dabe0009e67214ff5f5447ce83dd","version":"v6.0.2"},{"repo":"actions/download-artifact","sha":"3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c","version":"v8.0.1"},{"repo":"actions/github-script","sha":"3a2844b7e9c422d3c10d287c895573f7108da1b3","version":"v9.0.0"},{"repo":"actions/setup-node","sha":"48b55a011bda9f5d6aeb4c2d9c7362e8dae4041e","version":"v6.4.0"},{"repo":"actions/upload-artifact","sha":"043fb46d1a93c77aae656e7c1c64a875d1fc6a0a","version":"v7.0.1"},{"repo":"github/gh-aw-actions/setup","sha":"8c7d04ebf1ece56cd381446125da3e0f6896294a","version":"v0.80.9"}],"containers":[{"image":"ghcr.io/github/gh-aw-firewall/agent:0.27.7","digest":"sha256:aae231e4635c8999d039c132f1602d3df850fe9b84a00aa2b5ac981179b5661c","pinned_image":"ghcr.io/github/gh-aw-firewall/agent:0.27.7@sha256:aae231e4635c8999d039c132f1602d3df850fe9b84a00aa2b5ac981179b5661c"},{"image":"ghcr.io/github/gh-aw-firewall/api-proxy:0.27.7","digest":"sha256:009caf2e3d88fa77b64e9a03a95a228fc58db0f1701c6d324b29ba5a3c7c79b6","pinned_image":"ghcr.io/github/gh-aw-firewall/api-proxy:0.27.7@sha256:009caf2e3d88fa77b64e9a03a95a228fc58db0f1701c6d324b29ba5a3c7c79b6"},{"image":"ghcr.io/github/gh-aw-firewall/squid:0.27.7","digest":"sha256:deb1d4e19de62d51cee0508057a596a19315c3423ada4d675cad136dc8037c96","pinned_image":"ghcr.io/github/gh-aw-firewall/squid:0.27.7@sha256:deb1d4e19de62d51cee0508057a596a19315c3423ada4d675cad136dc8037c96"},{"image":"ghcr.io/github/gh-aw-mcpg:v0.3.27","digest":"sha256:fe984bddde4ec05d756d9043edb0a32912e6b7b72f6a121b1082f29221421cc7","pinned_image":"ghcr.io/github/gh-aw-mcpg:v0.3.27@sha256:fe984bddde4ec05d756d9043edb0a32912e6b7b72f6a121b1082f29221421cc7"},{"image":"ghcr.io/github/gh-aw-node","digest":"sha256:529d02eb970b1161aa25c593a9c3df57fdfad5a8add328cb3b6eccef66f3183b","pinned_image":"ghcr.io/github/gh-aw-node@sha256:529d02eb970b1161aa25c593a9c3df57fdfad5a8add328cb3b6eccef66f3183b"},{"image":"ghcr.io/github/github-mcp-server:v1.4.0","digest":"sha256:2afb26356481d1a350e14544a6e160f7f7ec1561a1ea309b823665abf0309036","pinned_image":"ghcr.io/github/github-mcp-server:v1.4.0@sha256:2afb26356481d1a350e14544a6e160f7f7ec1561a1ea309b823665abf0309036"}]} # This file was automatically generated by gh-aw (v0.80.9). DO NOT EDIT. To debug this workflow, load the skill at https://github.com/github/gh-aw/blob/main/debug.md # @@ -37,7 +37,9 @@ # humans (the open tracking issue is the hand-off surface; a dedicated # [ci-fix][needs-human] PR is planned but currently deferred — see Step 6). CI on # the PR is kicked by a human `/azp run` for now; the loop watches, classifies, -# and re-fixes autonomously between kicks. +# and re-fixes autonomously between kicks. When the SPECIFIC test a PR fixed is +# confirmed green in that PR's own CI, the loop marks the draft PR ready for review +# (a state transition only — it never approves or merges). # Never mutes tests, but # de-flakes genuinely flaky ones (deterministic synchronization, no retries / # timeout bumps). Always skips visual-regression / screenshot issues. @@ -273,24 +275,24 @@ jobs: run: | bash "${RUNNER_TEMP}/gh-aw/actions/create_prompt_first.sh" { - cat << 'GH_AW_PROMPT_25cf75111b670167_EOF' + cat << 'GH_AW_PROMPT_2ac1d0b4083428f8_EOF' - GH_AW_PROMPT_25cf75111b670167_EOF + GH_AW_PROMPT_2ac1d0b4083428f8_EOF cat "${RUNNER_TEMP}/gh-aw/prompts/xpia.md" cat "${RUNNER_TEMP}/gh-aw/prompts/temp_folder_prompt.md" cat "${RUNNER_TEMP}/gh-aw/prompts/markdown.md" cat "${RUNNER_TEMP}/gh-aw/prompts/safe_outputs_prompt.md" - cat << 'GH_AW_PROMPT_25cf75111b670167_EOF' + cat << 'GH_AW_PROMPT_2ac1d0b4083428f8_EOF' - Tools: add_comment(max:3), create_pull_request(max:3), update_pull_request(max:3), push_to_pull_request_branch(max:3), missing_tool, missing_data, noop - GH_AW_PROMPT_25cf75111b670167_EOF + Tools: add_comment(max:6), create_pull_request(max:3), update_pull_request(max:3), mark_pull_request_as_ready_for_review(max:3), add_labels(max:3), push_to_pull_request_branch(max:3), missing_tool, missing_data, noop + GH_AW_PROMPT_2ac1d0b4083428f8_EOF cat "${RUNNER_TEMP}/gh-aw/prompts/safe_outputs_create_pull_request.md" cat "${RUNNER_TEMP}/gh-aw/prompts/safe_outputs_push_to_pr_branch.md" - cat << 'GH_AW_PROMPT_25cf75111b670167_EOF' + cat << 'GH_AW_PROMPT_2ac1d0b4083428f8_EOF' - GH_AW_PROMPT_25cf75111b670167_EOF + GH_AW_PROMPT_2ac1d0b4083428f8_EOF cat "${RUNNER_TEMP}/gh-aw/prompts/mcp_cli_tools_prompt.md" - cat << 'GH_AW_PROMPT_25cf75111b670167_EOF' + cat << 'GH_AW_PROMPT_2ac1d0b4083428f8_EOF' The following GitHub context information is available for this workflow: {{#if github.actor}} @@ -332,12 +334,12 @@ jobs: stop immediately and report the limitation rather than spending turns trying to work around it. - GH_AW_PROMPT_25cf75111b670167_EOF + GH_AW_PROMPT_2ac1d0b4083428f8_EOF cat "${RUNNER_TEMP}/gh-aw/prompts/github_mcp_tools_with_safeoutputs_prompt.md" - cat << 'GH_AW_PROMPT_25cf75111b670167_EOF' + cat << 'GH_AW_PROMPT_2ac1d0b4083428f8_EOF' {{#runtime-import .github/workflows/ci-status-fix.md}} - GH_AW_PROMPT_25cf75111b670167_EOF + GH_AW_PROMPT_2ac1d0b4083428f8_EOF } > "$GH_AW_PROMPT" - name: Interpolate variables and render templates uses: actions/github-script@3a2844b7e9c422d3c10d287c895573f7108da1b3 # v9.0.0 @@ -554,16 +556,18 @@ jobs: mkdir -p "${RUNNER_TEMP}/gh-aw/safeoutputs" mkdir -p /tmp/gh-aw/safeoutputs mkdir -p /tmp/gh-aw/mcp-logs/safeoutputs - cat > "${RUNNER_TEMP}/gh-aw/safeoutputs/config.json" << 'GH_AW_SAFE_OUTPUTS_CONFIG_fa2a69fad5fd0485_EOF' - {"add_comment":{"discussions":false,"max":3,"required_labels":["agentic-workflows"],"required_title_prefix":"[ci-fix] ","target":"*"},"create_pull_request":{"allowed_base_branches":["main"],"allowed_branches":["ci-fix/**"],"allowed_files":["src/Core/**","src/Controls/**","src/Essentials/**","src/BlazorWebView/**","src/TestUtils/**","src/Templates/**","**/PublicAPI.Unshipped.txt"],"base_branch":"main","draft":true,"labels":["agentic-workflows"],"max":3,"max_patch_files":100,"max_patch_size":4096,"protect_top_level_dot_folders":true,"protected_files":["package.json","bun.lockb","bunfig.toml","deno.json","deno.jsonc","deno.lock","global.json","NuGet.Config","Directory.Packages.props","mix.exs","mix.lock","go.mod","go.sum","stack.yaml","stack.yaml.lock","pom.xml","build.gradle","build.gradle.kts","settings.gradle","settings.gradle.kts","gradle.properties","package-lock.json","yarn.lock","pnpm-lock.yaml","npm-shrinkwrap.json","requirements.txt","Pipfile","Pipfile.lock","pyproject.toml","setup.py","setup.cfg","Gemfile","Gemfile.lock","uv.lock","CODEOWNERS","DESIGN.md","README.md","CONTRIBUTING.md","CHANGELOG.md","SECURITY.md","CODE_OF_CONDUCT.md","AGENTS.md","CLAUDE.md","GEMINI.md"],"protected_files_policy":"request_review","title_prefix":"[ci-fix] "},"create_report_incomplete_issue":{},"missing_data":{},"missing_tool":{},"noop":{"max":1,"report-as-issue":"true"},"push_to_pull_request_branch":{"allowed_files":["src/Core/**","src/Controls/**","src/Essentials/**","src/BlazorWebView/**","src/TestUtils/**","src/Templates/**","**/PublicAPI.Unshipped.txt"],"if_no_changes":"warn","max":3,"max_patch_size":4096,"protect_top_level_dot_folders":true,"protected_files":["package.json","bun.lockb","bunfig.toml","deno.json","deno.jsonc","deno.lock","global.json","NuGet.Config","Directory.Packages.props","mix.exs","mix.lock","go.mod","go.sum","stack.yaml","stack.yaml.lock","pom.xml","build.gradle","build.gradle.kts","settings.gradle","settings.gradle.kts","gradle.properties","package-lock.json","yarn.lock","pnpm-lock.yaml","npm-shrinkwrap.json","requirements.txt","Pipfile","Pipfile.lock","pyproject.toml","setup.py","setup.cfg","Gemfile","Gemfile.lock","uv.lock","CODEOWNERS","DESIGN.md","README.md","CONTRIBUTING.md","CHANGELOG.md","SECURITY.md","CODE_OF_CONDUCT.md","AGENTS.md","CLAUDE.md","GEMINI.md"],"required_labels":["agentic-workflows"],"target":"*","title_prefix":"[ci-fix] "},"report_incomplete":{},"update_pull_request":{"allow_body":true,"allow_title":false,"max":3,"target":"*","update_branch":false}} - GH_AW_SAFE_OUTPUTS_CONFIG_fa2a69fad5fd0485_EOF + cat > "${RUNNER_TEMP}/gh-aw/safeoutputs/config.json" << 'GH_AW_SAFE_OUTPUTS_CONFIG_0644bf1434ca0071_EOF' + {"add_comment":{"discussions":false,"max":6,"required_labels":["agentic-workflows"],"required_title_prefix":"[ci-fix] ","target":"*"},"add_labels":{"allowed":["p/0"],"max":3,"required_labels":["agentic-workflows"],"required_title_prefix":"[ci-fix] ","target":"*"},"create_pull_request":{"allowed_base_branches":["main"],"allowed_branches":["ci-fix/**"],"allowed_files":["src/Core/**","src/Controls/**","src/Essentials/**","src/BlazorWebView/**","src/TestUtils/**","src/Templates/**","**/PublicAPI.Unshipped.txt"],"base_branch":"main","draft":true,"labels":["agentic-workflows"],"max":3,"max_patch_files":100,"max_patch_size":4096,"protect_top_level_dot_folders":true,"protected_files":["package.json","bun.lockb","bunfig.toml","deno.json","deno.jsonc","deno.lock","global.json","NuGet.Config","Directory.Packages.props","mix.exs","mix.lock","go.mod","go.sum","stack.yaml","stack.yaml.lock","pom.xml","build.gradle","build.gradle.kts","settings.gradle","settings.gradle.kts","gradle.properties","package-lock.json","yarn.lock","pnpm-lock.yaml","npm-shrinkwrap.json","requirements.txt","Pipfile","Pipfile.lock","pyproject.toml","setup.py","setup.cfg","Gemfile","Gemfile.lock","uv.lock","CODEOWNERS","DESIGN.md","README.md","CONTRIBUTING.md","CHANGELOG.md","SECURITY.md","CODE_OF_CONDUCT.md","AGENTS.md","CLAUDE.md","GEMINI.md"],"protected_files_policy":"request_review","title_prefix":"[ci-fix] "},"create_report_incomplete_issue":{},"mark_pull_request_as_ready_for_review":{"max":3,"required_labels":["agentic-workflows"],"required_title_prefix":"[ci-fix] ","target":"*"},"missing_data":{},"missing_tool":{},"noop":{"max":1,"report-as-issue":"true"},"push_to_pull_request_branch":{"allowed_files":["src/Core/**","src/Controls/**","src/Essentials/**","src/BlazorWebView/**","src/TestUtils/**","src/Templates/**","**/PublicAPI.Unshipped.txt"],"if_no_changes":"warn","max":3,"max_patch_size":4096,"protect_top_level_dot_folders":true,"protected_files":["package.json","bun.lockb","bunfig.toml","deno.json","deno.jsonc","deno.lock","global.json","NuGet.Config","Directory.Packages.props","mix.exs","mix.lock","go.mod","go.sum","stack.yaml","stack.yaml.lock","pom.xml","build.gradle","build.gradle.kts","settings.gradle","settings.gradle.kts","gradle.properties","package-lock.json","yarn.lock","pnpm-lock.yaml","npm-shrinkwrap.json","requirements.txt","Pipfile","Pipfile.lock","pyproject.toml","setup.py","setup.cfg","Gemfile","Gemfile.lock","uv.lock","CODEOWNERS","DESIGN.md","README.md","CONTRIBUTING.md","CHANGELOG.md","SECURITY.md","CODE_OF_CONDUCT.md","AGENTS.md","CLAUDE.md","GEMINI.md"],"required_labels":["agentic-workflows"],"target":"*","title_prefix":"[ci-fix] "},"report_incomplete":{},"update_pull_request":{"allow_body":true,"allow_title":false,"max":3,"target":"*","update_branch":false}} + GH_AW_SAFE_OUTPUTS_CONFIG_0644bf1434ca0071_EOF - name: Generate Safe Outputs Tools env: GH_AW_TOOLS_META_JSON: | { "description_suffixes": { - "add_comment": " CONSTRAINTS: Maximum 3 comment(s) can be added. Target: *. Supports reply_to_id for discussion threading.", + "add_comment": " CONSTRAINTS: Maximum 6 comment(s) can be added. Target: *. Supports reply_to_id for discussion threading.", + "add_labels": " CONSTRAINTS: Maximum 3 label(s) can be added. Only these labels are allowed: [\"p/0\"]. Target: *.", "create_pull_request": " CONSTRAINTS: Maximum 3 pull request(s) can be created. Title will be prefixed with \"[ci-fix] \". Labels [\"agentic-workflows\"] will be automatically added. Only these labels are allowed: [\"agentic-workflows\"]. PRs will be created as drafts.", + "mark_pull_request_as_ready_for_review": " CONSTRAINTS: Maximum 3 pull request(s) can be marked as ready for review.", "push_to_pull_request_branch": " CONSTRAINTS: Maximum 3 push(es) can be made. The target pull request title must start with \"[ci-fix] \".", "update_pull_request": " CONSTRAINTS: Maximum 3 pull request(s) can be updated. Target: *." }, @@ -594,6 +598,25 @@ jobs: } } }, + "add_labels": { + "defaultMax": 5, + "fields": { + "item_number": { + "issueNumberOrTemporaryId": true + }, + "labels": { + "required": true, + "type": "array", + "itemType": "string", + "itemSanitize": true, + "itemMaxLength": 128 + }, + "repo": { + "type": "string", + "maxLength": 256 + } + } + }, "create_pull_request": { "defaultMax": 1, "fields": { @@ -635,6 +658,24 @@ jobs: } } }, + "mark_pull_request_as_ready_for_review": { + "defaultMax": 1, + "fields": { + "pull_request_number": { + "issueOrPRNumber": true + }, + "reason": { + "required": true, + "type": "string", + "sanitize": true, + "maxLength": 65000 + }, + "repo": { + "type": "string", + "maxLength": 256 + } + } + }, "missing_data": { "defaultMax": 20, "fields": { @@ -1494,7 +1535,7 @@ jobs: uses: actions/github-script@3a2844b7e9c422d3c10d287c895573f7108da1b3 # v9.0.0 env: WORKFLOW_NAME: "CI Failure Fixer (main)" - WORKFLOW_DESCRIPTION: "Periodic pass over open ci-scan tracking issues filed by the main-branch CI\nfailure scanner (.github/workflows/ci-status-main.md). This workflow targets\nthe `main` branch EXCLUSIVELY: it processes only issues labelled ci-scan and\nopens every PR against main. (The net11.0 branch is handled by the parallel\n.github/workflows/ci-status-fix-net11.md — the two are split because gh-aw can\nonly transport a fix relative to ONE static base branch per workflow, and the\nmain↔net11.0 divergence exceeds gh-aw's 10 MB transport-patch cap.) The fixer\nopens ONE draft [ci-fix] PR per actionable issue against main, then WATCHES\nthat PR's own CI on later runs: when the fix's CI comes back red and the red is\ncaused by the fix itself, it pushes a fresh follow-up fix onto the SAME PR\nbranch (never a second PR) — up to 10 attempts — then stops and defers to\nhumans (the open tracking issue is the hand-off surface; a dedicated\n[ci-fix][needs-human] PR is planned but currently deferred — see Step 6). CI on\nthe PR is kicked by a human `/azp run` for now; the loop watches, classifies,\nand re-fixes autonomously between kicks.\nNever mutes tests, but\nde-flakes genuinely flaky ones (deterministic synchronization, no retries /\ntimeout bumps). Always skips visual-regression / screenshot issues." + WORKFLOW_DESCRIPTION: "Periodic pass over open ci-scan tracking issues filed by the main-branch CI\nfailure scanner (.github/workflows/ci-status-main.md). This workflow targets\nthe `main` branch EXCLUSIVELY: it processes only issues labelled ci-scan and\nopens every PR against main. (The net11.0 branch is handled by the parallel\n.github/workflows/ci-status-fix-net11.md — the two are split because gh-aw can\nonly transport a fix relative to ONE static base branch per workflow, and the\nmain↔net11.0 divergence exceeds gh-aw's 10 MB transport-patch cap.) The fixer\nopens ONE draft [ci-fix] PR per actionable issue against main, then WATCHES\nthat PR's own CI on later runs: when the fix's CI comes back red and the red is\ncaused by the fix itself, it pushes a fresh follow-up fix onto the SAME PR\nbranch (never a second PR) — up to 10 attempts — then stops and defers to\nhumans (the open tracking issue is the hand-off surface; a dedicated\n[ci-fix][needs-human] PR is planned but currently deferred — see Step 6). CI on\nthe PR is kicked by a human `/azp run` for now; the loop watches, classifies,\nand re-fixes autonomously between kicks. When the SPECIFIC test a PR fixed is\nconfirmed green in that PR's own CI, the loop marks the draft PR ready for review\n(a state transition only — it never approves or merges).\nNever mutes tests, but\nde-flakes genuinely flaky ones (deterministic synchronization, no retries /\ntimeout bumps). Always skips visual-regression / screenshot issues." HAS_PATCH: ${{ needs.agent.outputs.has_patch }} with: script: | @@ -1910,7 +1951,7 @@ jobs: GH_AW_ALLOWED_DOMAINS: "*.blob.core.windows.net,*.githubusercontent.com,api.business.githubcopilot.com,api.enterprise.githubcopilot.com,api.github.com,api.githubcopilot.com,api.individual.githubcopilot.com,api.snapcraft.io,archive.ubuntu.com,azure.archive.ubuntu.com,codeload.github.com,crl.geotrust.com,crl.globalsign.com,crl.identrust.com,crl.sectigo.com,crl.thawte.com,crl.usertrust.com,crl.verisign.com,crl3.digicert.com,crl4.digicert.com,crls.ssl.com,dev.azure.com,docs.github.com,github-cloud.githubusercontent.com,github-cloud.s3.amazonaws.com,github.blog,github.com,github.githubassets.com,helix.dot.net,host.docker.internal,json-schema.org,json.schemastore.org,keyserver.ubuntu.com,lfs.github.com,objects.githubusercontent.com,ocsp.digicert.com,ocsp.geotrust.com,ocsp.globalsign.com,ocsp.identrust.com,ocsp.sectigo.com,ocsp.ssl.com,ocsp.thawte.com,ocsp.usertrust.com,ocsp.verisign.com,packagecloud.io,packages.cloud.google.com,packages.microsoft.com,patch-diff.githubusercontent.com,ppa.launchpad.net,raw.githubusercontent.com,registry.npmjs.org,s.symcb.com,s.symcd.com,security.ubuntu.com,telemetry.enterprise.githubcopilot.com,ts-crl.ws.symantec.com,ts-ocsp.ws.symantec.com,www.googleapis.com" GITHUB_SERVER_URL: ${{ github.server_url }} GITHUB_API_URL: ${{ github.api_url }} - GH_AW_SAFE_OUTPUTS_HANDLER_CONFIG: "{\"add_comment\":{\"discussions\":false,\"max\":3,\"required_labels\":[\"agentic-workflows\"],\"required_title_prefix\":\"[ci-fix] \",\"target\":\"*\"},\"create_pull_request\":{\"allowed_base_branches\":[\"main\"],\"allowed_branches\":[\"ci-fix/**\"],\"allowed_files\":[\"src/Core/**\",\"src/Controls/**\",\"src/Essentials/**\",\"src/BlazorWebView/**\",\"src/TestUtils/**\",\"src/Templates/**\",\"**/PublicAPI.Unshipped.txt\"],\"base_branch\":\"main\",\"draft\":true,\"labels\":[\"agentic-workflows\"],\"max\":3,\"max_patch_files\":100,\"max_patch_size\":4096,\"protect_top_level_dot_folders\":true,\"protected_files\":[\"package.json\",\"bun.lockb\",\"bunfig.toml\",\"deno.json\",\"deno.jsonc\",\"deno.lock\",\"global.json\",\"NuGet.Config\",\"Directory.Packages.props\",\"mix.exs\",\"mix.lock\",\"go.mod\",\"go.sum\",\"stack.yaml\",\"stack.yaml.lock\",\"pom.xml\",\"build.gradle\",\"build.gradle.kts\",\"settings.gradle\",\"settings.gradle.kts\",\"gradle.properties\",\"package-lock.json\",\"yarn.lock\",\"pnpm-lock.yaml\",\"npm-shrinkwrap.json\",\"requirements.txt\",\"Pipfile\",\"Pipfile.lock\",\"pyproject.toml\",\"setup.py\",\"setup.cfg\",\"Gemfile\",\"Gemfile.lock\",\"uv.lock\",\"CODEOWNERS\",\"DESIGN.md\",\"README.md\",\"CONTRIBUTING.md\",\"CHANGELOG.md\",\"SECURITY.md\",\"CODE_OF_CONDUCT.md\",\"AGENTS.md\",\"CLAUDE.md\",\"GEMINI.md\"],\"protected_files_policy\":\"request_review\",\"title_prefix\":\"[ci-fix] \"},\"create_report_incomplete_issue\":{},\"missing_data\":{},\"missing_tool\":{},\"noop\":{\"max\":1,\"report-as-issue\":\"true\"},\"push_to_pull_request_branch\":{\"allowed_files\":[\"src/Core/**\",\"src/Controls/**\",\"src/Essentials/**\",\"src/BlazorWebView/**\",\"src/TestUtils/**\",\"src/Templates/**\",\"**/PublicAPI.Unshipped.txt\"],\"if_no_changes\":\"warn\",\"max\":3,\"max_patch_size\":4096,\"protect_top_level_dot_folders\":true,\"protected_files\":[\"package.json\",\"bun.lockb\",\"bunfig.toml\",\"deno.json\",\"deno.jsonc\",\"deno.lock\",\"global.json\",\"NuGet.Config\",\"Directory.Packages.props\",\"mix.exs\",\"mix.lock\",\"go.mod\",\"go.sum\",\"stack.yaml\",\"stack.yaml.lock\",\"pom.xml\",\"build.gradle\",\"build.gradle.kts\",\"settings.gradle\",\"settings.gradle.kts\",\"gradle.properties\",\"package-lock.json\",\"yarn.lock\",\"pnpm-lock.yaml\",\"npm-shrinkwrap.json\",\"requirements.txt\",\"Pipfile\",\"Pipfile.lock\",\"pyproject.toml\",\"setup.py\",\"setup.cfg\",\"Gemfile\",\"Gemfile.lock\",\"uv.lock\",\"CODEOWNERS\",\"DESIGN.md\",\"README.md\",\"CONTRIBUTING.md\",\"CHANGELOG.md\",\"SECURITY.md\",\"CODE_OF_CONDUCT.md\",\"AGENTS.md\",\"CLAUDE.md\",\"GEMINI.md\"],\"required_labels\":[\"agentic-workflows\"],\"target\":\"*\",\"title_prefix\":\"[ci-fix] \"},\"report_incomplete\":{},\"update_pull_request\":{\"allow_body\":true,\"allow_title\":false,\"max\":3,\"target\":\"*\",\"update_branch\":false}}" + GH_AW_SAFE_OUTPUTS_HANDLER_CONFIG: "{\"add_comment\":{\"discussions\":false,\"max\":6,\"required_labels\":[\"agentic-workflows\"],\"required_title_prefix\":\"[ci-fix] \",\"target\":\"*\"},\"add_labels\":{\"allowed\":[\"p/0\"],\"max\":3,\"required_labels\":[\"agentic-workflows\"],\"required_title_prefix\":\"[ci-fix] \",\"target\":\"*\"},\"create_pull_request\":{\"allowed_base_branches\":[\"main\"],\"allowed_branches\":[\"ci-fix/**\"],\"allowed_files\":[\"src/Core/**\",\"src/Controls/**\",\"src/Essentials/**\",\"src/BlazorWebView/**\",\"src/TestUtils/**\",\"src/Templates/**\",\"**/PublicAPI.Unshipped.txt\"],\"base_branch\":\"main\",\"draft\":true,\"labels\":[\"agentic-workflows\"],\"max\":3,\"max_patch_files\":100,\"max_patch_size\":4096,\"protect_top_level_dot_folders\":true,\"protected_files\":[\"package.json\",\"bun.lockb\",\"bunfig.toml\",\"deno.json\",\"deno.jsonc\",\"deno.lock\",\"global.json\",\"NuGet.Config\",\"Directory.Packages.props\",\"mix.exs\",\"mix.lock\",\"go.mod\",\"go.sum\",\"stack.yaml\",\"stack.yaml.lock\",\"pom.xml\",\"build.gradle\",\"build.gradle.kts\",\"settings.gradle\",\"settings.gradle.kts\",\"gradle.properties\",\"package-lock.json\",\"yarn.lock\",\"pnpm-lock.yaml\",\"npm-shrinkwrap.json\",\"requirements.txt\",\"Pipfile\",\"Pipfile.lock\",\"pyproject.toml\",\"setup.py\",\"setup.cfg\",\"Gemfile\",\"Gemfile.lock\",\"uv.lock\",\"CODEOWNERS\",\"DESIGN.md\",\"README.md\",\"CONTRIBUTING.md\",\"CHANGELOG.md\",\"SECURITY.md\",\"CODE_OF_CONDUCT.md\",\"AGENTS.md\",\"CLAUDE.md\",\"GEMINI.md\"],\"protected_files_policy\":\"request_review\",\"title_prefix\":\"[ci-fix] \"},\"create_report_incomplete_issue\":{},\"mark_pull_request_as_ready_for_review\":{\"max\":3,\"required_labels\":[\"agentic-workflows\"],\"required_title_prefix\":\"[ci-fix] \",\"target\":\"*\"},\"missing_data\":{},\"missing_tool\":{},\"noop\":{\"max\":1,\"report-as-issue\":\"true\"},\"push_to_pull_request_branch\":{\"allowed_files\":[\"src/Core/**\",\"src/Controls/**\",\"src/Essentials/**\",\"src/BlazorWebView/**\",\"src/TestUtils/**\",\"src/Templates/**\",\"**/PublicAPI.Unshipped.txt\"],\"if_no_changes\":\"warn\",\"max\":3,\"max_patch_size\":4096,\"protect_top_level_dot_folders\":true,\"protected_files\":[\"package.json\",\"bun.lockb\",\"bunfig.toml\",\"deno.json\",\"deno.jsonc\",\"deno.lock\",\"global.json\",\"NuGet.Config\",\"Directory.Packages.props\",\"mix.exs\",\"mix.lock\",\"go.mod\",\"go.sum\",\"stack.yaml\",\"stack.yaml.lock\",\"pom.xml\",\"build.gradle\",\"build.gradle.kts\",\"settings.gradle\",\"settings.gradle.kts\",\"gradle.properties\",\"package-lock.json\",\"yarn.lock\",\"pnpm-lock.yaml\",\"npm-shrinkwrap.json\",\"requirements.txt\",\"Pipfile\",\"Pipfile.lock\",\"pyproject.toml\",\"setup.py\",\"setup.cfg\",\"Gemfile\",\"Gemfile.lock\",\"uv.lock\",\"CODEOWNERS\",\"DESIGN.md\",\"README.md\",\"CONTRIBUTING.md\",\"CHANGELOG.md\",\"SECURITY.md\",\"CODE_OF_CONDUCT.md\",\"AGENTS.md\",\"CLAUDE.md\",\"GEMINI.md\"],\"required_labels\":[\"agentic-workflows\"],\"target\":\"*\",\"title_prefix\":\"[ci-fix] \"},\"report_incomplete\":{},\"update_pull_request\":{\"allow_body\":true,\"allow_title\":false,\"max\":3,\"target\":\"*\",\"update_branch\":false}}" GH_AW_CI_TRIGGER_TOKEN: ${{ secrets.GH_AW_CI_TRIGGER_TOKEN }} with: github-token: ${{ secrets.GH_AW_GITHUB_TOKEN || secrets.GITHUB_TOKEN }} diff --git a/.github/workflows/ci-status-fix.md b/.github/workflows/ci-status-fix.md index d3227e25170c..9c2f46054029 100644 --- a/.github/workflows/ci-status-fix.md +++ b/.github/workflows/ci-status-fix.md @@ -15,7 +15,9 @@ description: | humans (the open tracking issue is the hand-off surface; a dedicated [ci-fix][needs-human] PR is planned but currently deferred — see Step 6). CI on the PR is kicked by a human `/azp run` for now; the loop watches, classifies, - and re-fixes autonomously between kicks. + and re-fixes autonomously between kicks. When the SPECIFIC test a PR fixed is + confirmed green in that PR's own CI, the loop marks the draft PR ready for review + (a state transition only — it never approves or merges). Never mutes tests, but de-flakes genuinely flaky ones (deterministic synchronization, no retries / timeout bumps). Always skips visual-regression / screenshot issues. @@ -239,8 +241,15 @@ safe-outputs: # PER-RUN total (not per-PR): a sweep may surface several green PRs and/or # annotate several flaky reds in one run. At max:1 all but the first comment # were silently dropped, starving the workflow's #1 value (surfacing green PRs - # for review). Raised to 3 to match the per-run throughput of the other outputs. - max: 3 + # for review). Sized to 6 (not 3) because a single green DRAFT PR that gets + # flipped ready in one sweep spends TWO comment slots — the Step 3 ✅ surface + # comment (which still names the /azp-gated legs a human must kick, so it is NOT + # redundant with 🎯) AND the Step 3.6 T3 🎯 readiness comment. At max:3 the shared + # bucket drained after ~1 draft flip and T3's atomicity pre-check then deferred + # every further mark-ready even while the mark-ready/add_labels buckets (also 3) + # sat idle; 6 lets ~3 draft flips (2 comments each) land per sweep, so the + # mark-ready:3 / add_labels:3 caps are actually reachable. + max: 6 target: "*" # Hard constraint (defense-in-depth): only comment on THIS workflow's own # ci-fix PRs — [ci-fix] title prefix AND agentic-workflows label. Mirrors the @@ -261,13 +270,51 @@ safe-outputs: # never the title, so disable title rewrites — this compiles to allow_title:false # and removes any ability to retitle an arbitrary PR. title: false - # NOTE: gh-aw v0.79.8 does NOT emit required-title-prefix/required-labels into + # NOTE: gh-aw v0.80.9 (the pinned compiler) does NOT emit required-title-prefix/required-labels into # the compiled config for update-pull-request (verified against the lock — it # silently drops them, unlike add-comment / push-to-pull-request-branch which # honor them). So which-PR scoping here relies on prompt Hard-Rule 6 + # min-integrity:approved; the residual is body-marker edits only (no code, no # merge, no close), capped at max:3. Revisit required-* if a future gh-aw # compiler emits them for this output. + mark-pull-request-as-ready-for-review: + # Flip a validated draft [ci-fix] PR to ready-for-review once the SPECIFIC test + # this PR was opened to fix is confirmed green in the PR's OWN CI (Step 3.6) — + # even when unrelated legs are red. The workflow is schedule/dispatch-triggered + # (no triggering-PR context), so target "*" lets the agent name the PR number it + # validated. This is a state transition ONLY: it never approves and never merges — + # a human still reviews and merges. + # PER-RUN total (not per-PR): a sweep may validate several PRs' target tests green. + max: 3 + target: "*" + # Hard constraint (defense-in-depth): only ever un-draft THIS workflow's own + # ci-fix PRs — [ci-fix] title prefix AND agentic-workflows label — so a confused + # or prompt-injected agent cannot mark an arbitrary PR ready. (If the v0.80.9 + # compiler silently drops these — as documented for update-pull-request above — + # the Step 3.6 preconditions + min-integrity:approved are the compensating scope + # controls; verify against the lock after compiling.) + required-title-prefix: "[ci-fix] " + required-labels: [agentic-workflows] + add-labels: + # Apply the p/0 priority label to a [ci-fix] PR at the exact moment the loop flips it + # from draft to ready-for-review (Step 3.6 T3) — so a validated, review-ready fix lands + # in the team's p/0 triage queue instead of sitting unseen in the draft backlog. Paired + # 1:1 with mark-pull-request-as-ready-for-review; target "*" lets the agent name the PR + # it just validated. + # allowed = HARD allowlist: the agent may ONLY ever add p/0, nothing else. This caps the + # blast radius of a confused/prompt-injected agent to exactly one benign priority label — + # it can never apply a downstream-triggering or destructive label. + allowed: [p/0] + # PER-RUN total (not per-PR): a sweep may mark several PRs ready in one run; each adds + # one label, so this matches the mark-ready per-run cap (3). + max: 3 + target: "*" + # Hard constraint (defense-in-depth): only ever label THIS workflow's own ci-fix PRs — + # [ci-fix] title prefix AND agentic-workflows label. (If the v0.80.9 compiler drops these + # — as documented for update-pull-request above — the Step 3.6 preconditions + + # min-integrity:approved + the allowed:[p/0] allowlist are the compensating controls.) + required-title-prefix: "[ci-fix] " + required-labels: [agentic-workflows] timeout-minutes: 90 @@ -339,33 +386,48 @@ through `safe-outputs`. 5. **One issue = one outcome per run.** Exactly one of: respond to a maintainer's change-request on the open PR (Track C, Step 3.5.R); open the first fix/help/ de-flake PR; advance an existing PR by one attempt (push a follow-up fix); - surface a validated-green PR for review; annotate an unrelated-flake red; wait + surface a validated-green PR for review; mark a target-validated draft PR + ready-for-review (Step 3.6 T3 — the terminal outcome that supersedes the same + run's surface/annotate precursor line), or record its `already-ready` + steady-state no-op; annotate an unrelated-flake red; wait (CI not yet settled); or a recorded skip (the dedicated needs-human PR is deferred — the attempt cap records a skip instead; see Step 6). Always prefer advancing or opening a PR over a skip when a non-mute diff is producible. 6. **All writes via `safe-outputs`.** Allowed outputs: `create_pull_request` (first attempt only), `push_to_pull_request_branch` (advance an existing PR by one attempt), `update_pull_request` (bump the attempt marker / refresh the - prior-attempts table), and `add_comment` (progress notes on the `[ci-fix]` PR - ONLY). NEVER comment on the tracking issue (issues are locked by + prior-attempts table), `add_comment` (progress notes on the `[ci-fix]` PR + ONLY), `mark_pull_request_as_ready_for_review` (flip a target-validated draft + `[ci-fix]` PR from draft to ready — Step 3.6 T3 only), and `add_labels` (add + ONLY the `p/0` label, ONLY on that same draft→ready transition — Step 3.6 T3). + NEVER comment on the tracking issue (issues are locked by `.github/workflows/ci-scan-lock-issues.yml`) — `add_comment` targets the PR. No `gh pr create`, no manual `git push`. **Defense-in-depth:** `add_comment` and `update_pull_request` use `target: "*"` (the agent supplies the PR number). - `add_comment` is now config-locked to `required-title-prefix: "[ci-fix] "` + + `add_comment`, `mark_pull_request_as_ready_for_review`, and `add_labels` are + config-locked to `required-title-prefix: "[ci-fix] "` + `required-labels: [agentic-workflows]` (hard-enforced by the handler, same as - `push_to_pull_request_branch`), so a comment can only ever land on THIS - workflow's own PRs. `update_pull_request` CANNOT be config-locked to a - title/label in gh-aw v0.79.8 (the compiler silently drops `required-*` for that - output — verified against the lock), so it keeps `title: false` (no retitles; - body-marker edits only) plus this prompt-level guard. Before emitting either, - VERIFY the target PR carries BOTH the `[ci-fix]` title prefix AND the - `agentic-workflows` label; never comment on or edit the body of any PR that - lacks both. + `push_to_pull_request_branch`), and `add_labels` additionally has an + `allowed: [p/0]` allowlist so it can ONLY ever add `p/0` — so these can only + ever land on THIS workflow's own PRs. `update_pull_request` CANNOT be + config-locked to a title/label in gh-aw v0.80.9 (the compiler silently drops + `required-*` for that output — verified against the lock), so it keeps + `title: false` (no retitles; body-marker edits only) plus this prompt-level + guard. Before emitting ANY of these, VERIFY the target PR carries BOTH the + `[ci-fix]` title prefix AND the `agentic-workflows` label; never comment on, + edit, un-draft, or label any PR that lacks both. 7. **Per-run safe-output caps (per RUN, not per PR).** Each output type is capped per run: `create_pull_request` 3, `push_to_pull_request_branch` 3, - `add_comment` 3, `update_pull_request` 3. Note an ADVANCE spends one push **and** - one update **and** one comment, so ≤ 3 advances/run; surfacing a green or - annotating a flake spends one comment. When a bucket is exhausted, do NOT keep + `add_comment` 6, `update_pull_request` 3, `mark_pull_request_as_ready_for_review` + 3, `add_labels` 3 (counts LABELS, not calls). Note an ADVANCE spends one push + **and** one update **and** one comment, so ≤ 3 advances/run; surfacing a green or + annotating a flake spends one comment; a **mark-ready (Step 3.6 T3)** of a + still-draft PR spends TWO comments (the Step 3 ✅ surface **and** the T3 🎯 + readiness note) **and** one mark-ready **and** one label as an ALL-OR-NOTHING set + (T3 pre-checks those buckets and defers the whole PR if any is exhausted — never + mark a PR ready without its 🎯 audit comment). `add_comment` is therefore sized 6 + (not 3) so ~3 draft flips, at 2 comments each, can land in one sweep instead of the + shared comment bucket starving the otherwise-idle mark-ready/add_labels buckets. When a bucket is exhausted, do NOT keep emitting (extras are silently dropped) — record `skipped: per-run cap reached; deferring PR # to next cycle` for each remaining PR so the drop is a deliberate, logged decision. The continuous loop picks the deferred PRs up next @@ -409,8 +471,9 @@ For every open tracking issue in scope, converge on exactly one outcome: | Open first help-wanted draft `[ci-fix]` PR | No open PR yet; a plausible candidate exists but cannot be runner-validated (device/UI tests) (attempt 1) | | Open first de-flake draft `[ci-fix]` PR | No open PR yet; failure is intermittent (green-on-retry) from a genuine test-quality defect; a deterministic-synchronization fix is producible (attempt 1, Step 4.7 bucket b) | | Advance an existing PR (push attempt N+1) | Open PR's own CI settled red *because of the fix itself*, no human engaged, marker < 10 — push a NEW distinct fix onto the same branch (Step 3.5 → 5.6) | -| Surface a validated-green PR | Open PR's own CI settled green — comment "validated, attempt N/10; ready for review" and do NOT advance (Step 3.5) | -| Annotate an unrelated-flake red | Open PR is red only on baseline-flake / unrelated legs — comment which leg needs a re-run; do NOT burn an attempt (Step 3.5) | +| Surface a validated-green PR | Open PR's own CI settled green — comment "validated, attempt N/10; ready for review" and do NOT advance (Step 3.5), then mark the draft PR ready for review once the fixed test is confirmed green (Step 3.6) | +| Annotate an unrelated-flake red | Open PR is red only on baseline-flake / unrelated legs — comment which leg needs a re-run; do NOT burn an attempt (Step 3.5); if the SPECIFIC fixed test is confirmed green on that build, still mark the draft PR ready for review (Step 3.6) | +| Mark a validated PR ready for review | The specific test this PR fixed is confirmed green in the PR's own CI (even if unrelated legs are red) — comment the target-test result and transition the draft PR to ready for review; never approve or merge (Step 3.6) | | Wait | Open PR's CI is pending / not yet settled on the current head SHA — do nothing this cycle (Step 3.5) | | Hand-off skip (attempt cap) | Marker == 10 and the signature still reproduces — stop and defer to humans; the dedicated `[ci-fix][needs-human]` PR is planned but currently deferred (Step 6) | | Recorded skip | Visual-regression, human engaged, already-handled, fixed-in-latest-build, infra-flake, out-of-bounds, only-mute-available, or no novel approach producible | @@ -685,14 +748,34 @@ Run these gates in order — the FIRST that fires decides this cycle's outcome: 1. **Human engaged.** If `C.humanEngaged` → `skipped: human engaged on PR #; deferring` and stop. Never fight a human reviewer — the intentional hand-off boundary still holds the moment a person touches the PR. -2. **CI not settled / unknown / incomplete prefetch.** If `C.checksSettled == false` - OR `C.overallConclusion` is `pending`, `neutral`, or `unknown`, OR - `C.dataComplete == false` → the fix's CI has not finished on the current head SHA - (`C.headSha`), or the prefetch could not fully read this PR (a partial read can - understate `humanEngaged` / `attempt`). `skipped: PR # CI pending / prefetch - incomplete on ; waiting` and stop. *(Round 1: this is where a - maintainer `/azp run` is awaited — the `/azp`-gated uitests/devicetests legs will - not have run until a human kicks them.)* +2. **CI not settled — but validate the target test first (target-focused readiness).** + If `C.checksSettled == false` OR `C.overallConclusion` is `pending`, `neutral`, or + `unknown`, OR `C.dataComplete == false` → the *overall* build has not finished on the + current head SHA (`C.headSha`), or the prefetch could not fully read this PR (a partial + read can understate `humanEngaged` / `attempt`). Unrelated legs still draining must NOT + keep an already-proven fix parked as a draft, so before waiting, try the target-focused + fast-path: + - **2a — Target-green fast-path (draft `[ci-fix]` PRs only).** If `C.dataComplete == + true` AND the Step 3.6 preconditions hold (draft PR, `[ci-fix] ` title prefix AND the + `agentic-workflows` label), run Step 3.6 **T1–T2** against `C.headSha` now: identify + the target test(s) and read the PR's OWN build **timeline / per-leg status** (anonymous + `_apis/build` — never `_apis/test`, per Hard-Rule 8). If EVERY target test is + **VALIDATED-GREEN** (per T2's all-platform definition below — the target's category leg + is `succeeded` on every platform that runs it, failed on none, and NO target platform + family the pipeline covers is still pending/unconcluded, per T2's "no platform left + unverified" rule; a platform that simply has no leg for the target's category — the test + doesn't run there — is fine, not a gap) AND every leg in + `C.failedLegs` that has ALREADY concluded classifies as **unrelated flake** (Step 4 / + Step 4.7 method — the fix is implicated in NO completed red), then the fix is proven + regardless of unrelated *pending* legs → run **Step 3.6 T3** (mark ready + 🎯 comment) + and stop. This early-out NEVER advances an attempt and NEVER acts on a red that could + be the fix's fault. + - **2b — Otherwise WAIT.** If the target test has not yet executed (pending/absent on + its leg), or a completed red is (or may be) caused by the fix, or `C.dataComplete == + false`, or this PR has no identifiable target test → `skipped: PR # CI pending / + target not yet validated on ; waiting` and stop. *(Round 1: this is where a + maintainer `/azp run` is awaited — the `/azp`-gated uitests/devicetests legs will not + have run until a human kicks them.)* 3. **Green → surface for review.** If `C.overallConclusion == "success"`: the checks that RAN are green. **Primary-gate check first:** the deterministic `success` verdict only certifies "at least one green check, nothing failing or @@ -704,18 +787,31 @@ Run these gates in order — the FIRST that fires decides this cycle's outcome: # primary CI gate (maui-pr) not green on ; waiting` and stop — do NOT surface. (The `/azp`-gated `maui-pr-uitests` (def 313) / `maui-pr-devicetests` (def 314) legs MAY still be un-run — that is expected and is named below, not a - reason to withhold the surface.) **Idempotency:** scan the PR's existing comments - for a prior bot `✅ … validated … on ` note for THIS head SHA — if one - already exists, record `already-surfaced PR # (head )` and stop - WITHOUT re-commenting (re-surfacing the same green every tick is noise and burns - the per-run comment budget other PRs need). Otherwise resolve the attempt number from - `C.effectiveAttempt` (the authoritative max(marker, bot-commit) counter; omit the - number only if it is somehow indeterminate). **Dry-run gate (Step 0):** if `dry_run == "true"`, do NOT emit any comment — instead print the intended `✅` validated-green body to the run log, tally `dry-run: would-surface-green PR #`, and stop. Otherwise `add_comment` on PR #: `✅ Attempt /10 validated - — the fix's CI is green on . Ready for human review.` - Name any `/azp`-gated legs (uitests def 313 / devicetests def 314) that have not - run and still need a maintainer `/azp run`. Do NOT advance. Record - `surfaced-green PR # (attempt /10)` and stop. *(This directly - attacks the real bottleneck — no reviews — so it is the highest-value outcome.)* + reason to withhold the surface.) **Comment idempotency + dry-run suppress the ✅ + comment ONLY — neither skips Step 3.6.** Scan the PR's existing comments for a prior + bot `✅ … validated … on ` note for THIS head SHA; if one exists, set + `SKIP_SURFACE_COMMENT = true`. Resolve the attempt number from `C.effectiveAttempt` + (the authoritative max(marker, bot-commit) counter; omit the number only if it is + somehow indeterminate). Then post the surface comment UNLESS suppressed: + - if `dry_run == "true"`: do NOT emit — print the intended `✅` validated-green body to + the run log and tally `dry-run: would-surface-green PR #`; + - else if `SKIP_SURFACE_COMMENT`: do NOT re-post — record `already-surfaced PR # + (head )` (re-surfacing the same green every tick is noise and burns the + per-run comment budget other PRs need); + - else `add_comment` on PR #: `✅ Attempt /10 validated — the fix's CI is + green on .` naming any `/azp`-gated legs (uitests def 313 / devicetests def + 314) that have not run and still need a maintainer `/azp run`; record `surfaced-green + PR # (attempt /10)`. (Do NOT assert "ready for human review" on a PR that + is still a draft — Step 3.6, not this comment, owns the draft→ready flip and posts its + own 🎯 announcement when it fires.) + Do NOT advance. Then — in ALL of the above cases — run **Step 3.6** (target-test + readiness gate) for this PR before stopping: its `/azp`-gated target legs may conclude + green on this SAME head SHA on a LATER sweep (an `/azp run` adds no commit), and Step + 3.6 is the ONLY place the draft→ready flip happens, so it MUST re-evaluate every sweep — + never `stop` here before it. *(This directly attacks the real bottleneck — no reviews — + so it is the highest-value outcome.)* 4. **Red → classify caused-by-fix vs unrelated-flake.** If `C.overallConclusion == "failure"`, analyze the PR's OWN failing build (NOT `main`): find the AzDO `maui-pr` build for this PR (filter builds by `branchName=refs/pull//merge`, @@ -723,18 +819,26 @@ Run these gates in order — the FIRST that fires decides this cycle's outcome: method and Step 4.7 flake buckets to `C.failedLegs`. - **Unrelated flake only** (every failed leg is known-flaky / infra / a pre-existing baseline red NOT introduced by the fix): do NOT burn an attempt. - **Idempotency first:** scan the PR's existing comments for a prior bot - `♻️ … unrelated flake … on ` note for THIS head SHA — if one already - exists, record `already-annotated-flake PR # (head )` and stop - WITHOUT re-commenting (re-annotating the same flake every 12h sweep is noise and - burns the per-run comment budget other PRs need). Otherwise resolve the attempt - number from `C.effectiveAttempt` (the authoritative max(marker, bot-commit) - counter), omitting the `Attempt /10:` prefix only if it is somehow - indeterminate. **Dry-run gate (Step 0):** if `dry_run == "true"`, do NOT emit any comment — instead print the intended `♻️` unrelated-flake body to the run log, tally `dry-run: would-annotate-flake PR #`, and stop. Otherwise `add_comment` on PR #: `♻️ Attempt /10: red is - unrelated flake on leg(s) () on ; the fix itself is not - implicated. A maintainer re-run (/azp run ) should clear it.` Record - `annotated-flake PR # (head )` and stop. *(Round 1: human re-runs; - Round 2: auto re-trigger.)* + **Comment idempotency + dry-run suppress the ♻️ comment ONLY — neither skips Step + 3.6.** Scan the PR's existing comments for a prior bot `♻️ … unrelated flake … on + ` note for THIS head SHA; if one exists, set `SKIP_FLAKE_COMMENT = true`. + Resolve the attempt number from `C.effectiveAttempt` (the authoritative max(marker, + bot-commit) counter), omitting the `Attempt /10:` prefix only if it is + somehow indeterminate. Then post the flake note UNLESS suppressed: + - if `dry_run == "true"`: do NOT emit — print the intended `♻️` unrelated-flake body + to the run log and tally `dry-run: would-annotate-flake PR #`; + - else if `SKIP_FLAKE_COMMENT`: do NOT re-post — record `already-annotated-flake PR + # (head )` (re-annotating the same flake every 12h sweep is noise and + burns the per-run comment budget other PRs need); + - else `add_comment` on PR #: `♻️ Attempt /10: red is unrelated flake on + leg(s) () on ; the fix itself is not implicated. A + maintainer re-run (/azp run ) should clear it.` record `annotated-flake + PR # (head )`. + Then — in ALL of the above cases — run **Step 3.6** (target-test readiness gate) for + this PR before stopping: its `/azp`-gated target legs may conclude green on this SAME + head SHA on a LATER sweep, and Step 3.6 is the ONLY place the draft→ready flip + happens, so it MUST re-evaluate every sweep — never `stop` here before it. *(Round 1: + human re-runs; Round 2: auto re-trigger.)* - **Caused by the fix** (a failed leg still matches the original target signature, or the fix introduced a NEW failure): advance an attempt. - **Attempt count.** `attempt = C.effectiveAttempt` — the authoritative @@ -752,6 +856,188 @@ Run these gates in order — the FIRST that fires decides this cycle's outcome: from every prior commit on this branch**, emitted via the Step 5.6 ADVANCE path (push onto the same PR). +#### Step 3.6 — Target-test verification & mark-ready gate + +Reached from the Step 3 **green-surface** branch, the Step 4 **unrelated-flake** branch +(AFTER that branch has posted its comment), and the Step 3.5 **gate-2 target-focused +fast-path** (2a — while unrelated legs are still pending). Purpose: confirm the SPECIFIC +test(s) this PR was opened to fix now PASS in the PR's own CI, and — only when they do +— transition the draft `[ci-fix]` PR to **ready for review** so a maintainer sees a +validated fix instead of a draft. This is the ONLY place the loop flips draft→ready; it +is a state transition, **never** an approval or a merge (a human still reviews and +merges). Overall red on *unrelated* legs must NOT gate readiness — we validate the fix, +not the base branch's flakiness. + +**Preconditions** (ALL must hold; otherwise record `skipped: readiness N/A PR # +()` and stop this gate): +- `C.isDraft == true` — if the PR is already ready-for-review, the draft→ready + transition is already done; do NOT re-mark **and do NOT re-apply `p/0`**. `p/0` is applied + exactly ONCE, atomically with the draft→ready flip (T3 — all-or-nothing with the 🎯 audit + comment + mark-ready), so an already-ready loop PR that is MISSING `p/0` has almost + certainly had it **removed by a maintainer** de-prioritizing the PR — a pure triage action + the human-engagement guard (which inspects comments/reviews/commits, NOT label changes) + cannot see. Re-adding `p/0` every sweep would fight that maintainer indefinitely, violating + the loop's "never override a human" contract. So the loop does NOT reconcile the label: + record `already-ready PR #` and stop this gate. (Trade-off: on the rare occasion a + transient API error drops `p/0` at flip time *after* the T3 atomic pre-check passed, it is + not auto-re-added — but the PR is still ready-for-review with its 🎯 audit comment, and + re-adding `p/0` is a trivial manual action; that is strictly preferable to steam-rolling a + maintainer's deliberate de-prioritization.) +- The PR is unmistakably THIS workflow's own: `[ci-fix] ` title prefix AND the + `agentic-workflows` label (mirrors the safe-output lock; the compensating scope + control if the v0.80.9 compiler drops the declarative required-*). +- You reached this gate from Step 3 (green), Step 4 (**unrelated-flake**), or the Step 3.5 + gate-2 **target-focused fast-path** (2a) — i.e. the fix is NOT implicated in any red that + has concluded. If Step 4 classified a red as **caused by the fix** (ADVANCE), do NOT run + this gate — advance the attempt instead. + +**T1 — Identify the target test(s).** From the `[ci-scan]` issue signature the fix +addresses (and the PR's own diff), extract the fully-qualified test method name(s) the +fix targets — e.g. `SafeAreaShouldWorkOnAllShellTabs`. For a de-flake it is the +de-flaked test; for a product fix it is the originally-failing test(s). If NO specific +test can be identified (e.g. a product build-break rather than a test failure), this is a +**build-only fix** — there is no single test to validate cross-platform, so validate at +**whole-build** granularity rather than stopping (Step 3 only *comments*; Step 3.6 is the +sole draft→ready flip, so a build-only fix is undrafted HERE or not at all). We only reach +this gate from Step 3 (green) / Step 4 (unrelated-flake) / gate-2a, so the fix is not +implicated in any concluded red; additionally require, via the T2 timeline method on +`C.headSha`, that the primary build pipeline `maui-pr` (def 302) has CONCLUDED with EVERY +one of its platform build legs `succeeded`/`completed` and NONE failed/canceled or still +pending — the build is green on **every** platform, not just the originally-broken one (the +cross-platform guard, applied to the build instead of a test). A build-only fix is undrafted +ONLY on the strength of the auto-running `maui-pr` (def 302) whole-build green, which the +loop CAN observe: if the PR's `[ci-scan]` issue signature or its own diff indicates the +ORIGINATING failure was in a `/azp`-gated pipeline (`maui-pr-uitests` def 313 or +`maui-pr-devicetests` def 314) rather than `maui-pr` (def 302), do NOT undraft on +`maui-pr`-green alone — those pipelines do not auto-run on this PR (GITHUB_TOKEN cannot +trigger them), so a green `maui-pr` build is NOT evidence the gated build break is fixed; +record `skipped: build-only fix PR # targets gated pipeline () not run — +deferring to human` and stop this gate. Otherwise, set `TARGET := "the maui-pr build +(build-only fix — no single target test)"` and proceed to **T3** to mark ready. If any `maui-pr` build leg is still unconcluded, record `skipped: build-only fix PR +# not yet whole-build green (leg(s) pending)` and stop this gate WITHOUT marking +ready (a green subset is not enough — a still-pending build leg could yet fail). + +**T2 — Verify the target test's platform legs are green (anonymous, leg-level).** Per-test +outcomes come from the AzDO **test-results** API (`_apis/test/...`), which is **NOT reachable +anonymously — Hard-Rule 8**: those endpoints 302-redirect to a sign-in page on this runner, so +a `curl` returns an HTML redirect (not JSON) and every target test would look "not executed". +Validate instead at **leg granularity** using ONLY the anonymous `_apis/build/...` timeline +(Hard-Rule 8) — a `succeeded` category leg means every test in that category **that actually +ran** passed on that platform. This is a strong bar **only if the target test genuinely +executes**: a job can go green while one specific test is skipped at runtime (`[Ignore]`, +`Assert.Ignore()`, `Assert.Inconclusive()`, a `Skip=`/conditional `[Fact]`, an `#if`-out, or +an early `return` before the asserts). So before trusting a green leg as proof the target +passed, **confirm from the PR's OWN diff that the fix does NOT skip, ignore, disable, or +short-circuit the target test** (it adds no `[Ignore]`/`[Explicit]`, `Assert.Ignore`/ +`Assert.Inconclusive`, `Skip=`, category-exclusion, `#if`-out, or early `return` around the +target). If the fix could cause the target to be runtime-skipped rather than genuinely pass, +leg-level green is NOT sufficient evidence — record `skipped: target test may be +runtime-skipped by this fix (diff adds ); leg-green insufficient, deferring to human +on PR #` and stop this gate WITHOUT marking ready. Otherwise (the target genuinely runs +and asserts, as a de-flake or product fix does) a green category leg cannot hide a +target-test failure, so treat it as authoritative. + +Map the target test to the CI leg(s) that run it. A leg name encodes platform + test category +(e.g. `Android UITests SafeAreaEdges,Shadow`, `iOS UITests SafeAreaEdges,Shadow`, `macOS +UITests SafeAreaEdges,Shadow`, `Windows UITests SafeAreaEdges,Shadow`). Determine the target +test's UI-test category — its `[Category(UITestCategories.X)]` in the test/HostApp source, +visible in the PR diff or the test file — to know which leg-name substring identifies its legs; +for a device test, the per-platform device-test legs. + +Using the SAME build-discovery as Step 4 (filter AzDO builds by `branchName=refs/pull//merge` +or `sourceVersion == C.headSha`), read each build's **timeline** on `C.headSha` for the +pipeline(s) that RUN the target test — `maui-pr` (def 302) for unit/integration tests, +`maui-pr-uitests` (def 313) for Appium UI tests, `maui-pr-devicetests` (def 314) for device +tests: + +```bash +ORG=dnceng-public; PROJ=public; BUILD= +# Anonymous + Hard-Rule-8-compliant. NEVER call _apis/test/... (it 302-redirects to sign-in). +# Each type=="Job" record is one platform leg: result (succeeded/failed/canceled/null), +# state (completed/inProgress/pending), name (encodes " UITests "): +curl -s "https://dev.azure.com/$ORG/$PROJ/_apis/build/builds/$BUILD/timeline?api-version=7.1" \ + | tee /tmp/gh-aw/agent/timeline_${BUILD}.json \ + | jq -r '.records[] | select(.type=="Job") | "\(.result)\t\(.state)\t\(.name)"' +``` + +(The prefetch already exposes per-leg status in `C.failedLegs` / the checks context, and +`gh pr checks ` lists the same per-leg rows with platform+category in the name — use +whichever is handy; the build timeline is the authoritative cross-check on the exact build.) + +Treat a target test as **VALIDATED-GREEN** only if, on `C.headSha`, the leg that runs its +category is green on **every platform it runs on** — a fix that repairs one platform must not +silently regress the same test's leg on another, so a green leg on the originally-red platform +alone is NOT enough. Across the target pipeline's platform legs (for a UI test: the Android, +iOS, Windows, and macOS/MacCatalyst legs running the target's category; for a device test: each +device-test platform), require ALL of: +- the target's category leg is `result == "succeeded"` **and** `state == "completed"` on + **every** platform that runs it, AND +- that leg is `failed` / `canceled` / aborted on **NO** platform, AND +- **no platform is left unverified.** For each platform family the target pipeline covers, that + platform's category leg must have CONCLUDED (`state == "completed"`) on `C.headSha`. If a + platform simply has no leg for the target's category, the test does not run there — that is + fine, not a gap. But if a platform's category leg has **not concluded** (`state` is + `inProgress` / pending, or an `/azp`-gated `maui-pr-uitests` / `maui-pr-devicetests` leg that + has not been kicked), the test's status on that platform is UNKNOWN → the fix is NOT yet + validated across platforms: record `skipped: target test green on but + not yet verified on (leg(s) pending / need /azp run) on PR #` + and stop this gate WITHOUT marking ready. + +A target test whose category leg never concluded on ANY platform (**not executed** anywhere — +e.g. its `/azp`-gated pipeline has not been kicked) is likewise NOT validated: record `skipped: +target test not yet executed on PR # ( not run — needs /azp run)` and stop +this gate WITHOUT marking ready. Do NOT overclaim — a green *sibling* leg (a different category +on the same platform) is not the target's leg, and a green leg on one platform is not a pass on +the others. + +**T3 — Mark ready + report.** If EVERY target test is VALIDATED-GREEN. The 🎯 comment, +the mark-ready, and the `p/0` label are THREE SEPARATE safe-outputs — the comment +existing does NOT prove the mark-ready took effect, so they are tracked independently: + +- **Comment idempotency (dup-suppress only — never gates the mark-ready).** We only reach + T3 while `C.isDraft == true` (the precondition stops an already-ready PR before here). So + if a prior bot comment for THIS head that carries the leading `🎯` anchor AND the + contiguous phrase `validated green on ` already exists (BOTH T3 variants — + `🎯 Target test validated green on ` and the build-only `🎯 Build validated green on + ` — contain that exact contiguous phrase, so this recognizes both; keying on + `target test` alone would instead miss the build-only comment and re-post it every sweep. + **Require the `🎯` anchor** so the match can NEVER be loosened to a gapped + `validated…green…on` form that would false-positive on the Step 3 `✅ … validated — the + fix's CI is green on ` surface comment and wrongly suppress the audit 🎯), the comment + landed on an earlier sweep but the **mark-ready did NOT take + effect** (the PR is still a draft) — set `SUPPRESS_COMMENT = true` (do not re-post the + duplicate 🎯 comment) but STILL complete the mark-ready + label below. Never treat the + comment's existence as "already marked ready" while the PR is still a draft. Record this + case as `re-marking PR # (🎯 present; mark-ready did not take on a prior sweep)`. +- **Atomicity budget pre-check (Hard-Rule 7).** Determine the outputs this gate will emit: + `mark_pull_request_as_ready_for_review` + `add_labels` always, plus `add_comment` unless + `SUPPRESS_COMMENT`. Verify a free per-run slot remains in EACH bucket you are about to + use. If ANY required bucket is exhausted, emit NONE of them — record `skipped: per-run + cap reached; deferring mark-ready PR # to next cycle` and stop this gate. Never mark a + PR ready (or label it) unless its 🎯 audit comment is GUARANTEED to exist for this exact + head SHA — either posted in this same sweep, or (in the `SUPPRESS_COMMENT` re-marking case) + already present from a prior sweep. The outputs you DO emit either all land or all defer + together; the audit trail must never be absent, but it is NOT re-posted when it already exists. +- **Dry-run gate (Step 0):** if `dry_run == "true"`, emit NOTHING — print the intended + readiness comment and "would mark ready + add p/0" to the run log, tally `dry-run: + would-mark-ready PR #`, and stop. +- Otherwise emit for THIS PR number ``: + 1. `add_comment` (ONLY if not `SUPPRESS_COMMENT`): `🎯 Target test validated green on + — passed on ALL platforms it runs on (, + buildId ). — not + caused by this fix."> Transitioning this PR from draft to ready for review and adding + `p/0`; a maintainer still reviews and merges.` (For a **build-only fix** — the T1 + whole-build fallback, no single target test — phrase the first clause as `🎯 Build + validated green on — the maui-pr build passed on ALL platforms (buildId + ).` instead of naming a test.) + 2. `mark_pull_request_as_ready_for_review` with `reason:` a one-line justification + naming the validated test(s) and ``. + 3. `add_labels` with `labels: ["p/0"]` for PR # — put the now-review-ready fix into + the team's p/0 priority queue so it is triaged, not lost in the draft backlog. (If + the PR somehow already carries `p/0`, this is a harmless no-op.) +- Record `marked-ready PR # (target green on ALL platforms on , + labeled p/0)` and stop. + #### Step 3.5.R — Maintainer change-request response (Track C) Reached from Step 3.5 gate 0. Goal: when an **eligible human reviewer** (Hard-Rule @@ -1383,7 +1669,19 @@ Filed by [`ci-status-fix`](https://github.com/dotnet/maui/blob/main/.github/work ### Step 8 — Per-issue tally + end-of-run summary -Per issue, append one outcome line to `/tmp/gh-aw/agent/coverage.txt`: +Per issue, append **one** outcome line to `/tmp/gh-aw/agent/coverage.txt` — the +terminal outcome for the cycle. When one cycle produces a chained pair — a PR gets a +same-cycle precursor line and **then** a Step 3.6 readiness line (`marked-ready` on the +draft→ready flip, or `already-ready` on a later steady-state sweep of an already-ready +PR) — record ONLY the terminal Step 3.6 readiness line: it supersedes ANY same-cycle +non-readiness precursor for that PR, so an aggregator keying on one-line-per-issue never +double-counts. The precursor may be a Step 3 green line (`surfaced-green` on the first ✅, +or `already-surfaced` when it already exists) OR — when an unrelated-flake red and a +target-green coincide — a Step 4 `annotated-flake` line; either way the readiness line is +terminal, and the superseded precursor's signal survives in the PR's 🎯/♻️ comment so no +information is lost. This covers the flip cycle (`surfaced-green` → `marked-ready`), the +post-flip steady state (`already-surfaced` → `already-ready`), and the flake-coincident +flip (`annotated-flake` → `marked-ready`): ``` # main attempt- @@ -1394,8 +1692,14 @@ Per issue, append one outcome line to `/tmp/gh-aw/agent/coverage.txt`: `advance-PR # attempt /10` (ADVANCE mode — new commit pushed to the existing PR), `surfaced-green PR #` (fix's own CI went green; commented for review, did not advance), `annotated-flake PR #` (red was unrelated flake; -commented, attempt NOT burned), `waiting PR #` (CI not settled yet), -`dry-run: would-`, `skipped: `. (The +commented, attempt NOT burned), `marked-ready PR #` (the specific fixed test +was confirmed green in the PR's own CI; draft flipped to ready for review), +`already-surfaced PR #` (the fix's ✅ green comment already existed this sweep; +re-post suppressed — superseded by the Step 3.6 readiness line when the PR also flips +ready that cycle), `already-ready PR #` (terminal steady state — the PR was flipped +ready on a prior cycle and remains green on an unchanged head; no action taken, +supersedes `already-surfaced`), `waiting PR #` (CI not settled yet), +`dry-run: would-`, `skipped: `. (The `needs-human-PR` outcome is reserved for the deferred hand-off — Step 6 currently records a skip instead, so it is not emitted.)
`; + - else if `SKIP_FLAKE_COMMENT`: do NOT re-post — record `already-annotated-flake PR + #
(head )` (re-annotating the same flake every 12h sweep is noise and + burns the per-run comment budget other PRs need); + - else `add_comment` on PR #: `♻️ Attempt /10: red is unrelated flake on + leg(s) () on ; the fix itself is not implicated. A + maintainer re-run (/azp run ) should clear it.` record `annotated-flake + PR # (head )`. + Then — in ALL of the above cases — run **Step 3.6** (target-test readiness gate) for + this PR before stopping: its `/azp`-gated target legs may conclude green on this SAME + head SHA on a LATER sweep, and Step 3.6 is the ONLY place the draft→ready flip + happens, so it MUST re-evaluate every sweep — never `stop` here before it. *(Round 1: + human re-runs; Round 2: auto re-trigger.)* - **Caused by the fix** (a failed leg still matches the original target signature, or the fix introduced a NEW failure): advance an attempt. - **Attempt count.** `attempt = C.effectiveAttempt` — the authoritative @@ -762,6 +866,188 @@ Run these gates in order — the FIRST that fires decides this cycle's outcome: from every prior commit on this branch**, emitted via the Step 5.6 ADVANCE path (push onto the same PR). +#### Step 3.6 — Target-test verification & mark-ready gate + +Reached from the Step 3 **green-surface** branch, the Step 4 **unrelated-flake** branch +(AFTER that branch has posted its comment), and the Step 3.5 **gate-2 target-focused +fast-path** (2a — while unrelated legs are still pending). Purpose: confirm the SPECIFIC +test(s) this PR was opened to fix now PASS in the PR's own CI, and — only when they do +— transition the draft `[ci-fix-net11]` PR to **ready for review** so a maintainer sees +a validated fix instead of a draft. This is the ONLY place the loop flips draft→ready; +it is a state transition, **never** an approval or a merge (a human still reviews and +merges). Overall red on *unrelated* legs must NOT gate readiness — we validate the fix, +not the base branch's flakiness. + +**Preconditions** (ALL must hold; otherwise record `skipped: readiness N/A PR # +()` and stop this gate): +- `C.isDraft == true` — if the PR is already ready-for-review, the draft→ready + transition is already done; do NOT re-mark **and do NOT re-apply `p/0`**. `p/0` is applied + exactly ONCE, atomically with the draft→ready flip (T3 — all-or-nothing with the 🎯 audit + comment + mark-ready), so an already-ready loop PR that is MISSING `p/0` has almost + certainly had it **removed by a maintainer** de-prioritizing the PR — a pure triage action + the human-engagement guard (which inspects comments/reviews/commits, NOT label changes) + cannot see. Re-adding `p/0` every sweep would fight that maintainer indefinitely, violating + the loop's "never override a human" contract. So the loop does NOT reconcile the label: + record `already-ready PR #` and stop this gate. (Trade-off: on the rare occasion a + transient API error drops `p/0` at flip time *after* the T3 atomic pre-check passed, it is + not auto-re-added — but the PR is still ready-for-review with its 🎯 audit comment, and + re-adding `p/0` is a trivial manual action; that is strictly preferable to steam-rolling a + maintainer's deliberate de-prioritization.) +- The PR is unmistakably THIS workflow's own: `[ci-fix-net11] ` title prefix AND the + `agentic-workflows` label (mirrors the safe-output lock; the compensating scope + control if the v0.80.9 compiler drops the declarative required-*). +- You reached this gate from Step 3 (green), Step 4 (**unrelated-flake**), or the Step 3.5 + gate-2 **target-focused fast-path** (2a) — i.e. the fix is NOT implicated in any red that + has concluded. If Step 4 classified a red as **caused by the fix** (ADVANCE), do NOT run + this gate — advance the attempt instead. + +**T1 — Identify the target test(s).** From the `[ci-scan-net11]` issue signature the fix +addresses (and the PR's own diff), extract the fully-qualified test method name(s) the +fix targets — e.g. `SafeAreaShouldWorkOnAllShellTabs`. For a de-flake it is the +de-flaked test; for a product fix it is the originally-failing test(s). If NO specific +test can be identified (e.g. a product build-break rather than a test failure), this is a +**build-only fix** — there is no single test to validate cross-platform, so validate at +**whole-build** granularity rather than stopping (Step 3 only *comments*; Step 3.6 is the +sole draft→ready flip, so a build-only fix is undrafted HERE or not at all). We only reach +this gate from Step 3 (green) / Step 4 (unrelated-flake) / gate-2a, so the fix is not +implicated in any concluded red; additionally require, via the T2 timeline method on +`C.headSha`, that the primary build pipeline `maui-pr` (def 302) has CONCLUDED with EVERY +one of its platform build legs `succeeded`/`completed` and NONE failed/canceled or still +pending — the build is green on **every** platform, not just the originally-broken one (the +cross-platform guard, applied to the build instead of a test). A build-only fix is undrafted +ONLY on the strength of the auto-running `maui-pr` (def 302) whole-build green, which the +loop CAN observe: if the PR's `[ci-scan]` issue signature or its own diff indicates the +ORIGINATING failure was in a `/azp`-gated pipeline (`maui-pr-uitests` def 313 or +`maui-pr-devicetests` def 314) rather than `maui-pr` (def 302), do NOT undraft on +`maui-pr`-green alone — those pipelines do not auto-run on this PR (GITHUB_TOKEN cannot +trigger them), so a green `maui-pr` build is NOT evidence the gated build break is fixed; +record `skipped: build-only fix PR # targets gated pipeline () not run — +deferring to human` and stop this gate. Otherwise, set `TARGET := "the maui-pr build +(build-only fix — no single target test)"` and proceed to **T3** to mark ready. If any `maui-pr` build leg is still unconcluded, record `skipped: build-only fix PR +# not yet whole-build green (leg(s) pending)` and stop this gate WITHOUT marking +ready (a green subset is not enough — a still-pending build leg could yet fail). + +**T2 — Verify the target test's platform legs are green (anonymous, leg-level).** Per-test +outcomes come from the AzDO **test-results** API (`_apis/test/...`), which is **NOT reachable +anonymously — Hard-Rule 8**: those endpoints 302-redirect to a sign-in page on this runner, so +a `curl` returns an HTML redirect (not JSON) and every target test would look "not executed". +Validate instead at **leg granularity** using ONLY the anonymous `_apis/build/...` timeline +(Hard-Rule 8) — a `succeeded` category leg means every test in that category **that actually +ran** passed on that platform. This is a strong bar **only if the target test genuinely +executes**: a job can go green while one specific test is skipped at runtime (`[Ignore]`, +`Assert.Ignore()`, `Assert.Inconclusive()`, a `Skip=`/conditional `[Fact]`, an `#if`-out, or +an early `return` before the asserts). So before trusting a green leg as proof the target +passed, **confirm from the PR's OWN diff that the fix does NOT skip, ignore, disable, or +short-circuit the target test** (it adds no `[Ignore]`/`[Explicit]`, `Assert.Ignore`/ +`Assert.Inconclusive`, `Skip=`, category-exclusion, `#if`-out, or early `return` around the +target). If the fix could cause the target to be runtime-skipped rather than genuinely pass, +leg-level green is NOT sufficient evidence — record `skipped: target test may be +runtime-skipped by this fix (diff adds ); leg-green insufficient, deferring to human +on PR #` and stop this gate WITHOUT marking ready. Otherwise (the target genuinely runs +and asserts, as a de-flake or product fix does) a green category leg cannot hide a +target-test failure, so treat it as authoritative. + +Map the target test to the CI leg(s) that run it. A leg name encodes platform + test category +(e.g. `Android UITests SafeAreaEdges,Shadow`, `iOS UITests SafeAreaEdges,Shadow`, `macOS +UITests SafeAreaEdges,Shadow`, `Windows UITests SafeAreaEdges,Shadow`). Determine the target +test's UI-test category — its `[Category(UITestCategories.X)]` in the test/HostApp source, +visible in the PR diff or the test file — to know which leg-name substring identifies its legs; +for a device test, the per-platform device-test legs. + +Using the SAME build-discovery as Step 4 (filter AzDO builds by `branchName=refs/pull//merge` +or `sourceVersion == C.headSha`), read each build's **timeline** on `C.headSha` for the +pipeline(s) that RUN the target test — `maui-pr` (def 302) for unit/integration tests, +`maui-pr-uitests` (def 313) for Appium UI tests, `maui-pr-devicetests` (def 314) for device +tests: + +```bash +ORG=dnceng-public; PROJ=public; BUILD= +# Anonymous + Hard-Rule-8-compliant. NEVER call _apis/test/... (it 302-redirects to sign-in). +# Each type=="Job" record is one platform leg: result (succeeded/failed/canceled/null), +# state (completed/inProgress/pending), name (encodes " UITests "): +curl -s "https://dev.azure.com/$ORG/$PROJ/_apis/build/builds/$BUILD/timeline?api-version=7.1" \ + | tee /tmp/gh-aw/agent/timeline_${BUILD}.json \ + | jq -r '.records[] | select(.type=="Job") | "\(.result)\t\(.state)\t\(.name)"' +``` + +(The prefetch already exposes per-leg status in `C.failedLegs` / the checks context, and +`gh pr checks ` lists the same per-leg rows with platform+category in the name — use +whichever is handy; the build timeline is the authoritative cross-check on the exact build.) + +Treat a target test as **VALIDATED-GREEN** only if, on `C.headSha`, the leg that runs its +category is green on **every platform it runs on** — a fix that repairs one platform must not +silently regress the same test's leg on another, so a green leg on the originally-red platform +alone is NOT enough. Across the target pipeline's platform legs (for a UI test: the Android, +iOS, Windows, and macOS/MacCatalyst legs running the target's category; for a device test: each +device-test platform), require ALL of: +- the target's category leg is `result == "succeeded"` **and** `state == "completed"` on + **every** platform that runs it, AND +- that leg is `failed` / `canceled` / aborted on **NO** platform, AND +- **no platform is left unverified.** For each platform family the target pipeline covers, that + platform's category leg must have CONCLUDED (`state == "completed"`) on `C.headSha`. If a + platform simply has no leg for the target's category, the test does not run there — that is + fine, not a gap. But if a platform's category leg has **not concluded** (`state` is + `inProgress` / pending, or an `/azp`-gated `maui-pr-uitests` / `maui-pr-devicetests` leg that + has not been kicked), the test's status on that platform is UNKNOWN → the fix is NOT yet + validated across platforms: record `skipped: target test green on but + not yet verified on (leg(s) pending / need /azp run) on PR #` + and stop this gate WITHOUT marking ready. + +A target test whose category leg never concluded on ANY platform (**not executed** anywhere — +e.g. its `/azp`-gated pipeline has not been kicked) is likewise NOT validated: record `skipped: +target test not yet executed on PR # ( not run — needs /azp run)` and stop +this gate WITHOUT marking ready. Do NOT overclaim — a green *sibling* leg (a different category +on the same platform) is not the target's leg, and a green leg on one platform is not a pass on +the others. + +**T3 — Mark ready + report.** If EVERY target test is VALIDATED-GREEN. The 🎯 comment, +the mark-ready, and the `p/0` label are THREE SEPARATE safe-outputs — the comment +existing does NOT prove the mark-ready took effect, so they are tracked independently: + +- **Comment idempotency (dup-suppress only — never gates the mark-ready).** We only reach + T3 while `C.isDraft == true` (the precondition stops an already-ready PR before here). So + if a prior bot comment for THIS head that carries the leading `🎯` anchor AND the + contiguous phrase `validated green on ` already exists (BOTH T3 variants — + `🎯 Target test validated green on ` and the build-only `🎯 Build validated green on + ` — contain that exact contiguous phrase, so this recognizes both; keying on + `target test` alone would instead miss the build-only comment and re-post it every sweep. + **Require the `🎯` anchor** so the match can NEVER be loosened to a gapped + `validated…green…on` form that would false-positive on the Step 3 `✅ … validated — the + fix's CI is green on ` surface comment and wrongly suppress the audit 🎯), the comment + landed on an earlier sweep but the **mark-ready did NOT take + effect** (the PR is still a draft) — set `SUPPRESS_COMMENT = true` (do not re-post the + duplicate 🎯 comment) but STILL complete the mark-ready + label below. Never treat the + comment's existence as "already marked ready" while the PR is still a draft. Record this + case as `re-marking PR # (🎯 present; mark-ready did not take on a prior sweep)`. +- **Atomicity budget pre-check (Hard-Rule 7).** Determine the outputs this gate will emit: + `mark_pull_request_as_ready_for_review` + `add_labels` always, plus `add_comment` unless + `SUPPRESS_COMMENT`. Verify a free per-run slot remains in EACH bucket you are about to + use. If ANY required bucket is exhausted, emit NONE of them — record `skipped: per-run + cap reached; deferring mark-ready PR # to next cycle` and stop this gate. Never mark a + PR ready (or label it) unless its 🎯 audit comment is GUARANTEED to exist for this exact + head SHA — either posted in this same sweep, or (in the `SUPPRESS_COMMENT` re-marking case) + already present from a prior sweep. The outputs you DO emit either all land or all defer + together; the audit trail must never be absent, but it is NOT re-posted when it already exists. +- **Dry-run gate (Step 0):** if `dry_run == "true"`, emit NOTHING — print the intended + readiness comment and "would mark ready + add p/0" to the run log, tally `dry-run: + would-mark-ready PR #`, and stop. +- Otherwise emit for THIS PR number ``: + 1. `add_comment` (ONLY if not `SUPPRESS_COMMENT`): `🎯 Target test validated green on + — passed on ALL platforms it runs on (, + buildId ). — not + caused by this fix."> Transitioning this PR from draft to ready for review and adding + `p/0`; a maintainer still reviews and merges.` (For a **build-only fix** — the T1 + whole-build fallback, no single target test — phrase the first clause as `🎯 Build + validated green on — the maui-pr build passed on ALL platforms (buildId + ).` instead of naming a test.) + 2. `mark_pull_request_as_ready_for_review` with `reason:` a one-line justification + naming the validated test(s) and ``. + 3. `add_labels` with `labels: ["p/0"]` for PR # — put the now-review-ready fix into + the team's p/0 priority queue so it is triaged, not lost in the draft backlog. (If + the PR somehow already carries `p/0`, this is a harmless no-op.) +- Record `marked-ready PR # (target green on ALL platforms on , + labeled p/0)` and stop. + #### Step 3.5.R — Maintainer change-request response (Track C) Reached from Step 3.5 gate 0. Goal: when an **eligible human reviewer** (Hard-Rule @@ -1395,7 +1681,19 @@ Filed by [`ci-status-fix-net11`](https://github.com/dotnet/maui/blob/main/.githu ### Step 8 — Per-issue tally + end-of-run summary -Per issue, append one outcome line to `/tmp/gh-aw/agent/coverage.txt`: +Per issue, append **one** outcome line to `/tmp/gh-aw/agent/coverage.txt` — the +terminal outcome for the cycle. When one cycle produces a chained pair — a PR gets a +same-cycle precursor line and **then** a Step 3.6 readiness line (`marked-ready` on the +draft→ready flip, or `already-ready` on a later steady-state sweep of an already-ready +PR) — record ONLY the terminal Step 3.6 readiness line: it supersedes ANY same-cycle +non-readiness precursor for that PR, so an aggregator keying on one-line-per-issue never +double-counts. The precursor may be a Step 3 green line (`surfaced-green` on the first ✅, +or `already-surfaced` when it already exists) OR — when an unrelated-flake red and a +target-green coincide — a Step 4 `annotated-flake` line; either way the readiness line is +terminal, and the superseded precursor's signal survives in the PR's 🎯/♻️ comment so no +information is lost. This covers the flip cycle (`surfaced-green` → `marked-ready`), the +post-flip steady state (`already-surfaced` → `already-ready`), and the flake-coincident +flip (`annotated-flake` → `marked-ready`): ``` # net11.0 attempt- @@ -1406,8 +1704,14 @@ Per issue, append one outcome line to `/tmp/gh-aw/agent/coverage.txt`: `advance-PR # attempt /10` (ADVANCE mode — new commit pushed to the existing PR), `surfaced-green PR #` (fix's own CI went green; commented for review, did not advance), `annotated-flake PR #` (red was unrelated flake; -commented, attempt NOT burned), `waiting PR #` (CI not settled yet), -`dry-run: would-`, `skipped: `. (The +commented, attempt NOT burned), `marked-ready PR #` (the specific fixed test +was confirmed green in the PR's own CI; draft flipped to ready for review), +`already-surfaced PR #` (the fix's ✅ green comment already existed this sweep; +re-post suppressed — superseded by the Step 3.6 readiness line when the PR also flips +ready that cycle), `already-ready PR #` (terminal steady state — the PR was flipped +ready on a prior cycle and remains green on an unchanged head; no action taken, +supersedes `already-surfaced`), `waiting PR #` (CI not settled yet), +`dry-run: would-`, `skipped: `. (The `needs-human-PR` outcome is reserved for the deferred hand-off — Step 6 currently records a skip instead, so it is not emitted.) diff --git a/.github/workflows/ci-status-fix.lock.yml b/.github/workflows/ci-status-fix.lock.yml index 93dc5969cb0d..65511f2b3259 100644 --- a/.github/workflows/ci-status-fix.lock.yml +++ b/.github/workflows/ci-status-fix.lock.yml @@ -1,4 +1,4 @@ -# gh-aw-metadata: {"schema_version":"v4","frontmatter_hash":"f014161967589ebcaf1ac5ec6f3c88ee5a4388d28b55a07dc36c45b7b2aebab6","body_hash":"86c6b2d4c3df13da14c6a3c8b211381fcc9f97bb1951ff7b80588a2c5338e00c","compiler_version":"v0.80.9","strict":true,"agent_id":"copilot","agent_model":"claude-opus-4.8","engine_versions":{"copilot":"1.0.63"}} +# gh-aw-metadata: {"schema_version":"v4","frontmatter_hash":"b0ebf8a2f179900cfe3638e994b27a43cec52bdbdd2331dc1bd4d65762fa2d6b","body_hash":"1825e2b1f7c7bd88241bf37580655489360f9bebc806edd00837a52ce4ad83a0","compiler_version":"v0.80.9","strict":true,"agent_id":"copilot","agent_model":"claude-opus-4.8","engine_versions":{"copilot":"1.0.63"}} # gh-aw-manifest: {"version":1,"secrets":["COPILOT_PAT_0","COPILOT_PAT_1","COPILOT_PAT_2","COPILOT_PAT_3","COPILOT_PAT_4","COPILOT_PAT_5","COPILOT_PAT_6","COPILOT_PAT_7","COPILOT_PAT_8","COPILOT_PAT_9","GH_AW_CI_TRIGGER_TOKEN","GH_AW_GITHUB_MCP_SERVER_TOKEN","GH_AW_GITHUB_TOKEN","GITHUB_TOKEN"],"actions":[{"repo":"actions/checkout","sha":"34e114876b0b11c390a56381ad16ebd13914f8d5","version":"v4"},{"repo":"actions/checkout","sha":"9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0","version":"v7.0.0"},{"repo":"actions/checkout","sha":"de0fac2e4500dabe0009e67214ff5f5447ce83dd","version":"v6.0.2"},{"repo":"actions/download-artifact","sha":"3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c","version":"v8.0.1"},{"repo":"actions/github-script","sha":"3a2844b7e9c422d3c10d287c895573f7108da1b3","version":"v9.0.0"},{"repo":"actions/setup-node","sha":"48b55a011bda9f5d6aeb4c2d9c7362e8dae4041e","version":"v6.4.0"},{"repo":"actions/upload-artifact","sha":"043fb46d1a93c77aae656e7c1c64a875d1fc6a0a","version":"v7.0.1"},{"repo":"github/gh-aw-actions/setup","sha":"8c7d04ebf1ece56cd381446125da3e0f6896294a","version":"v0.80.9"}],"containers":[{"image":"ghcr.io/github/gh-aw-firewall/agent:0.27.7","digest":"sha256:aae231e4635c8999d039c132f1602d3df850fe9b84a00aa2b5ac981179b5661c","pinned_image":"ghcr.io/github/gh-aw-firewall/agent:0.27.7@sha256:aae231e4635c8999d039c132f1602d3df850fe9b84a00aa2b5ac981179b5661c"},{"image":"ghcr.io/github/gh-aw-firewall/api-proxy:0.27.7","digest":"sha256:009caf2e3d88fa77b64e9a03a95a228fc58db0f1701c6d324b29ba5a3c7c79b6","pinned_image":"ghcr.io/github/gh-aw-firewall/api-proxy:0.27.7@sha256:009caf2e3d88fa77b64e9a03a95a228fc58db0f1701c6d324b29ba5a3c7c79b6"},{"image":"ghcr.io/github/gh-aw-firewall/squid:0.27.7","digest":"sha256:deb1d4e19de62d51cee0508057a596a19315c3423ada4d675cad136dc8037c96","pinned_image":"ghcr.io/github/gh-aw-firewall/squid:0.27.7@sha256:deb1d4e19de62d51cee0508057a596a19315c3423ada4d675cad136dc8037c96"},{"image":"ghcr.io/github/gh-aw-mcpg:v0.3.27","digest":"sha256:fe984bddde4ec05d756d9043edb0a32912e6b7b72f6a121b1082f29221421cc7","pinned_image":"ghcr.io/github/gh-aw-mcpg:v0.3.27@sha256:fe984bddde4ec05d756d9043edb0a32912e6b7b72f6a121b1082f29221421cc7"},{"image":"ghcr.io/github/gh-aw-node","digest":"sha256:529d02eb970b1161aa25c593a9c3df57fdfad5a8add328cb3b6eccef66f3183b","pinned_image":"ghcr.io/github/gh-aw-node@sha256:529d02eb970b1161aa25c593a9c3df57fdfad5a8add328cb3b6eccef66f3183b"},{"image":"ghcr.io/github/github-mcp-server:v1.4.0","digest":"sha256:2afb26356481d1a350e14544a6e160f7f7ec1561a1ea309b823665abf0309036","pinned_image":"ghcr.io/github/github-mcp-server:v1.4.0@sha256:2afb26356481d1a350e14544a6e160f7f7ec1561a1ea309b823665abf0309036"}]} # This file was automatically generated by gh-aw (v0.80.9). DO NOT EDIT. To debug this workflow, load the skill at https://github.com/github/gh-aw/blob/main/debug.md # @@ -37,7 +37,9 @@ # humans (the open tracking issue is the hand-off surface; a dedicated # [ci-fix][needs-human] PR is planned but currently deferred — see Step 6). CI on # the PR is kicked by a human `/azp run` for now; the loop watches, classifies, -# and re-fixes autonomously between kicks. +# and re-fixes autonomously between kicks. When the SPECIFIC test a PR fixed is +# confirmed green in that PR's own CI, the loop marks the draft PR ready for review +# (a state transition only — it never approves or merges). # Never mutes tests, but # de-flakes genuinely flaky ones (deterministic synchronization, no retries / # timeout bumps). Always skips visual-regression / screenshot issues. @@ -273,24 +275,24 @@ jobs: run: | bash "${RUNNER_TEMP}/gh-aw/actions/create_prompt_first.sh" { - cat << 'GH_AW_PROMPT_25cf75111b670167_EOF' + cat << 'GH_AW_PROMPT_2ac1d0b4083428f8_EOF' - GH_AW_PROMPT_25cf75111b670167_EOF + GH_AW_PROMPT_2ac1d0b4083428f8_EOF cat "${RUNNER_TEMP}/gh-aw/prompts/xpia.md" cat "${RUNNER_TEMP}/gh-aw/prompts/temp_folder_prompt.md" cat "${RUNNER_TEMP}/gh-aw/prompts/markdown.md" cat "${RUNNER_TEMP}/gh-aw/prompts/safe_outputs_prompt.md" - cat << 'GH_AW_PROMPT_25cf75111b670167_EOF' + cat << 'GH_AW_PROMPT_2ac1d0b4083428f8_EOF' - Tools: add_comment(max:3), create_pull_request(max:3), update_pull_request(max:3), push_to_pull_request_branch(max:3), missing_tool, missing_data, noop - GH_AW_PROMPT_25cf75111b670167_EOF + Tools: add_comment(max:6), create_pull_request(max:3), update_pull_request(max:3), mark_pull_request_as_ready_for_review(max:3), add_labels(max:3), push_to_pull_request_branch(max:3), missing_tool, missing_data, noop + GH_AW_PROMPT_2ac1d0b4083428f8_EOF cat "${RUNNER_TEMP}/gh-aw/prompts/safe_outputs_create_pull_request.md" cat "${RUNNER_TEMP}/gh-aw/prompts/safe_outputs_push_to_pr_branch.md" - cat << 'GH_AW_PROMPT_25cf75111b670167_EOF' + cat << 'GH_AW_PROMPT_2ac1d0b4083428f8_EOF' - GH_AW_PROMPT_25cf75111b670167_EOF + GH_AW_PROMPT_2ac1d0b4083428f8_EOF cat "${RUNNER_TEMP}/gh-aw/prompts/mcp_cli_tools_prompt.md" - cat << 'GH_AW_PROMPT_25cf75111b670167_EOF' + cat << 'GH_AW_PROMPT_2ac1d0b4083428f8_EOF' The following GitHub context information is available for this workflow: {{#if github.actor}} @@ -332,12 +334,12 @@ jobs: stop immediately and report the limitation rather than spending turns trying to work around it. - GH_AW_PROMPT_25cf75111b670167_EOF + GH_AW_PROMPT_2ac1d0b4083428f8_EOF cat "${RUNNER_TEMP}/gh-aw/prompts/github_mcp_tools_with_safeoutputs_prompt.md" - cat << 'GH_AW_PROMPT_25cf75111b670167_EOF' + cat << 'GH_AW_PROMPT_2ac1d0b4083428f8_EOF' {{#runtime-import .github/workflows/ci-status-fix.md}} - GH_AW_PROMPT_25cf75111b670167_EOF + GH_AW_PROMPT_2ac1d0b4083428f8_EOF } > "$GH_AW_PROMPT" - name: Interpolate variables and render templates uses: actions/github-script@3a2844b7e9c422d3c10d287c895573f7108da1b3 # v9.0.0 @@ -554,16 +556,18 @@ jobs: mkdir -p "${RUNNER_TEMP}/gh-aw/safeoutputs" mkdir -p /tmp/gh-aw/safeoutputs mkdir -p /tmp/gh-aw/mcp-logs/safeoutputs - cat > "${RUNNER_TEMP}/gh-aw/safeoutputs/config.json" << 'GH_AW_SAFE_OUTPUTS_CONFIG_fa2a69fad5fd0485_EOF' - {"add_comment":{"discussions":false,"max":3,"required_labels":["agentic-workflows"],"required_title_prefix":"[ci-fix] ","target":"*"},"create_pull_request":{"allowed_base_branches":["main"],"allowed_branches":["ci-fix/**"],"allowed_files":["src/Core/**","src/Controls/**","src/Essentials/**","src/BlazorWebView/**","src/TestUtils/**","src/Templates/**","**/PublicAPI.Unshipped.txt"],"base_branch":"main","draft":true,"labels":["agentic-workflows"],"max":3,"max_patch_files":100,"max_patch_size":4096,"protect_top_level_dot_folders":true,"protected_files":["package.json","bun.lockb","bunfig.toml","deno.json","deno.jsonc","deno.lock","global.json","NuGet.Config","Directory.Packages.props","mix.exs","mix.lock","go.mod","go.sum","stack.yaml","stack.yaml.lock","pom.xml","build.gradle","build.gradle.kts","settings.gradle","settings.gradle.kts","gradle.properties","package-lock.json","yarn.lock","pnpm-lock.yaml","npm-shrinkwrap.json","requirements.txt","Pipfile","Pipfile.lock","pyproject.toml","setup.py","setup.cfg","Gemfile","Gemfile.lock","uv.lock","CODEOWNERS","DESIGN.md","README.md","CONTRIBUTING.md","CHANGELOG.md","SECURITY.md","CODE_OF_CONDUCT.md","AGENTS.md","CLAUDE.md","GEMINI.md"],"protected_files_policy":"request_review","title_prefix":"[ci-fix] "},"create_report_incomplete_issue":{},"missing_data":{},"missing_tool":{},"noop":{"max":1,"report-as-issue":"true"},"push_to_pull_request_branch":{"allowed_files":["src/Core/**","src/Controls/**","src/Essentials/**","src/BlazorWebView/**","src/TestUtils/**","src/Templates/**","**/PublicAPI.Unshipped.txt"],"if_no_changes":"warn","max":3,"max_patch_size":4096,"protect_top_level_dot_folders":true,"protected_files":["package.json","bun.lockb","bunfig.toml","deno.json","deno.jsonc","deno.lock","global.json","NuGet.Config","Directory.Packages.props","mix.exs","mix.lock","go.mod","go.sum","stack.yaml","stack.yaml.lock","pom.xml","build.gradle","build.gradle.kts","settings.gradle","settings.gradle.kts","gradle.properties","package-lock.json","yarn.lock","pnpm-lock.yaml","npm-shrinkwrap.json","requirements.txt","Pipfile","Pipfile.lock","pyproject.toml","setup.py","setup.cfg","Gemfile","Gemfile.lock","uv.lock","CODEOWNERS","DESIGN.md","README.md","CONTRIBUTING.md","CHANGELOG.md","SECURITY.md","CODE_OF_CONDUCT.md","AGENTS.md","CLAUDE.md","GEMINI.md"],"required_labels":["agentic-workflows"],"target":"*","title_prefix":"[ci-fix] "},"report_incomplete":{},"update_pull_request":{"allow_body":true,"allow_title":false,"max":3,"target":"*","update_branch":false}} - GH_AW_SAFE_OUTPUTS_CONFIG_fa2a69fad5fd0485_EOF + cat > "${RUNNER_TEMP}/gh-aw/safeoutputs/config.json" << 'GH_AW_SAFE_OUTPUTS_CONFIG_0644bf1434ca0071_EOF' + {"add_comment":{"discussions":false,"max":6,"required_labels":["agentic-workflows"],"required_title_prefix":"[ci-fix] ","target":"*"},"add_labels":{"allowed":["p/0"],"max":3,"required_labels":["agentic-workflows"],"required_title_prefix":"[ci-fix] ","target":"*"},"create_pull_request":{"allowed_base_branches":["main"],"allowed_branches":["ci-fix/**"],"allowed_files":["src/Core/**","src/Controls/**","src/Essentials/**","src/BlazorWebView/**","src/TestUtils/**","src/Templates/**","**/PublicAPI.Unshipped.txt"],"base_branch":"main","draft":true,"labels":["agentic-workflows"],"max":3,"max_patch_files":100,"max_patch_size":4096,"protect_top_level_dot_folders":true,"protected_files":["package.json","bun.lockb","bunfig.toml","deno.json","deno.jsonc","deno.lock","global.json","NuGet.Config","Directory.Packages.props","mix.exs","mix.lock","go.mod","go.sum","stack.yaml","stack.yaml.lock","pom.xml","build.gradle","build.gradle.kts","settings.gradle","settings.gradle.kts","gradle.properties","package-lock.json","yarn.lock","pnpm-lock.yaml","npm-shrinkwrap.json","requirements.txt","Pipfile","Pipfile.lock","pyproject.toml","setup.py","setup.cfg","Gemfile","Gemfile.lock","uv.lock","CODEOWNERS","DESIGN.md","README.md","CONTRIBUTING.md","CHANGELOG.md","SECURITY.md","CODE_OF_CONDUCT.md","AGENTS.md","CLAUDE.md","GEMINI.md"],"protected_files_policy":"request_review","title_prefix":"[ci-fix] "},"create_report_incomplete_issue":{},"mark_pull_request_as_ready_for_review":{"max":3,"required_labels":["agentic-workflows"],"required_title_prefix":"[ci-fix] ","target":"*"},"missing_data":{},"missing_tool":{},"noop":{"max":1,"report-as-issue":"true"},"push_to_pull_request_branch":{"allowed_files":["src/Core/**","src/Controls/**","src/Essentials/**","src/BlazorWebView/**","src/TestUtils/**","src/Templates/**","**/PublicAPI.Unshipped.txt"],"if_no_changes":"warn","max":3,"max_patch_size":4096,"protect_top_level_dot_folders":true,"protected_files":["package.json","bun.lockb","bunfig.toml","deno.json","deno.jsonc","deno.lock","global.json","NuGet.Config","Directory.Packages.props","mix.exs","mix.lock","go.mod","go.sum","stack.yaml","stack.yaml.lock","pom.xml","build.gradle","build.gradle.kts","settings.gradle","settings.gradle.kts","gradle.properties","package-lock.json","yarn.lock","pnpm-lock.yaml","npm-shrinkwrap.json","requirements.txt","Pipfile","Pipfile.lock","pyproject.toml","setup.py","setup.cfg","Gemfile","Gemfile.lock","uv.lock","CODEOWNERS","DESIGN.md","README.md","CONTRIBUTING.md","CHANGELOG.md","SECURITY.md","CODE_OF_CONDUCT.md","AGENTS.md","CLAUDE.md","GEMINI.md"],"required_labels":["agentic-workflows"],"target":"*","title_prefix":"[ci-fix] "},"report_incomplete":{},"update_pull_request":{"allow_body":true,"allow_title":false,"max":3,"target":"*","update_branch":false}} + GH_AW_SAFE_OUTPUTS_CONFIG_0644bf1434ca0071_EOF - name: Generate Safe Outputs Tools env: GH_AW_TOOLS_META_JSON: | { "description_suffixes": { - "add_comment": " CONSTRAINTS: Maximum 3 comment(s) can be added. Target: *. Supports reply_to_id for discussion threading.", + "add_comment": " CONSTRAINTS: Maximum 6 comment(s) can be added. Target: *. Supports reply_to_id for discussion threading.", + "add_labels": " CONSTRAINTS: Maximum 3 label(s) can be added. Only these labels are allowed: [\"p/0\"]. Target: *.", "create_pull_request": " CONSTRAINTS: Maximum 3 pull request(s) can be created. Title will be prefixed with \"[ci-fix] \". Labels [\"agentic-workflows\"] will be automatically added. Only these labels are allowed: [\"agentic-workflows\"]. PRs will be created as drafts.", + "mark_pull_request_as_ready_for_review": " CONSTRAINTS: Maximum 3 pull request(s) can be marked as ready for review.", "push_to_pull_request_branch": " CONSTRAINTS: Maximum 3 push(es) can be made. The target pull request title must start with \"[ci-fix] \".", "update_pull_request": " CONSTRAINTS: Maximum 3 pull request(s) can be updated. Target: *." }, @@ -594,6 +598,25 @@ jobs: } } }, + "add_labels": { + "defaultMax": 5, + "fields": { + "item_number": { + "issueNumberOrTemporaryId": true + }, + "labels": { + "required": true, + "type": "array", + "itemType": "string", + "itemSanitize": true, + "itemMaxLength": 128 + }, + "repo": { + "type": "string", + "maxLength": 256 + } + } + }, "create_pull_request": { "defaultMax": 1, "fields": { @@ -635,6 +658,24 @@ jobs: } } }, + "mark_pull_request_as_ready_for_review": { + "defaultMax": 1, + "fields": { + "pull_request_number": { + "issueOrPRNumber": true + }, + "reason": { + "required": true, + "type": "string", + "sanitize": true, + "maxLength": 65000 + }, + "repo": { + "type": "string", + "maxLength": 256 + } + } + }, "missing_data": { "defaultMax": 20, "fields": { @@ -1494,7 +1535,7 @@ jobs: uses: actions/github-script@3a2844b7e9c422d3c10d287c895573f7108da1b3 # v9.0.0 env: WORKFLOW_NAME: "CI Failure Fixer (main)" - WORKFLOW_DESCRIPTION: "Periodic pass over open ci-scan tracking issues filed by the main-branch CI\nfailure scanner (.github/workflows/ci-status-main.md). This workflow targets\nthe `main` branch EXCLUSIVELY: it processes only issues labelled ci-scan and\nopens every PR against main. (The net11.0 branch is handled by the parallel\n.github/workflows/ci-status-fix-net11.md — the two are split because gh-aw can\nonly transport a fix relative to ONE static base branch per workflow, and the\nmain↔net11.0 divergence exceeds gh-aw's 10 MB transport-patch cap.) The fixer\nopens ONE draft [ci-fix] PR per actionable issue against main, then WATCHES\nthat PR's own CI on later runs: when the fix's CI comes back red and the red is\ncaused by the fix itself, it pushes a fresh follow-up fix onto the SAME PR\nbranch (never a second PR) — up to 10 attempts — then stops and defers to\nhumans (the open tracking issue is the hand-off surface; a dedicated\n[ci-fix][needs-human] PR is planned but currently deferred — see Step 6). CI on\nthe PR is kicked by a human `/azp run` for now; the loop watches, classifies,\nand re-fixes autonomously between kicks.\nNever mutes tests, but\nde-flakes genuinely flaky ones (deterministic synchronization, no retries /\ntimeout bumps). Always skips visual-regression / screenshot issues." + WORKFLOW_DESCRIPTION: "Periodic pass over open ci-scan tracking issues filed by the main-branch CI\nfailure scanner (.github/workflows/ci-status-main.md). This workflow targets\nthe `main` branch EXCLUSIVELY: it processes only issues labelled ci-scan and\nopens every PR against main. (The net11.0 branch is handled by the parallel\n.github/workflows/ci-status-fix-net11.md — the two are split because gh-aw can\nonly transport a fix relative to ONE static base branch per workflow, and the\nmain↔net11.0 divergence exceeds gh-aw's 10 MB transport-patch cap.) The fixer\nopens ONE draft [ci-fix] PR per actionable issue against main, then WATCHES\nthat PR's own CI on later runs: when the fix's CI comes back red and the red is\ncaused by the fix itself, it pushes a fresh follow-up fix onto the SAME PR\nbranch (never a second PR) — up to 10 attempts — then stops and defers to\nhumans (the open tracking issue is the hand-off surface; a dedicated\n[ci-fix][needs-human] PR is planned but currently deferred — see Step 6). CI on\nthe PR is kicked by a human `/azp run` for now; the loop watches, classifies,\nand re-fixes autonomously between kicks. When the SPECIFIC test a PR fixed is\nconfirmed green in that PR's own CI, the loop marks the draft PR ready for review\n(a state transition only — it never approves or merges).\nNever mutes tests, but\nde-flakes genuinely flaky ones (deterministic synchronization, no retries /\ntimeout bumps). Always skips visual-regression / screenshot issues." HAS_PATCH: ${{ needs.agent.outputs.has_patch }} with: script: | @@ -1910,7 +1951,7 @@ jobs: GH_AW_ALLOWED_DOMAINS: "*.blob.core.windows.net,*.githubusercontent.com,api.business.githubcopilot.com,api.enterprise.githubcopilot.com,api.github.com,api.githubcopilot.com,api.individual.githubcopilot.com,api.snapcraft.io,archive.ubuntu.com,azure.archive.ubuntu.com,codeload.github.com,crl.geotrust.com,crl.globalsign.com,crl.identrust.com,crl.sectigo.com,crl.thawte.com,crl.usertrust.com,crl.verisign.com,crl3.digicert.com,crl4.digicert.com,crls.ssl.com,dev.azure.com,docs.github.com,github-cloud.githubusercontent.com,github-cloud.s3.amazonaws.com,github.blog,github.com,github.githubassets.com,helix.dot.net,host.docker.internal,json-schema.org,json.schemastore.org,keyserver.ubuntu.com,lfs.github.com,objects.githubusercontent.com,ocsp.digicert.com,ocsp.geotrust.com,ocsp.globalsign.com,ocsp.identrust.com,ocsp.sectigo.com,ocsp.ssl.com,ocsp.thawte.com,ocsp.usertrust.com,ocsp.verisign.com,packagecloud.io,packages.cloud.google.com,packages.microsoft.com,patch-diff.githubusercontent.com,ppa.launchpad.net,raw.githubusercontent.com,registry.npmjs.org,s.symcb.com,s.symcd.com,security.ubuntu.com,telemetry.enterprise.githubcopilot.com,ts-crl.ws.symantec.com,ts-ocsp.ws.symantec.com,www.googleapis.com" GITHUB_SERVER_URL: ${{ github.server_url }} GITHUB_API_URL: ${{ github.api_url }} - GH_AW_SAFE_OUTPUTS_HANDLER_CONFIG: "{\"add_comment\":{\"discussions\":false,\"max\":3,\"required_labels\":[\"agentic-workflows\"],\"required_title_prefix\":\"[ci-fix] \",\"target\":\"*\"},\"create_pull_request\":{\"allowed_base_branches\":[\"main\"],\"allowed_branches\":[\"ci-fix/**\"],\"allowed_files\":[\"src/Core/**\",\"src/Controls/**\",\"src/Essentials/**\",\"src/BlazorWebView/**\",\"src/TestUtils/**\",\"src/Templates/**\",\"**/PublicAPI.Unshipped.txt\"],\"base_branch\":\"main\",\"draft\":true,\"labels\":[\"agentic-workflows\"],\"max\":3,\"max_patch_files\":100,\"max_patch_size\":4096,\"protect_top_level_dot_folders\":true,\"protected_files\":[\"package.json\",\"bun.lockb\",\"bunfig.toml\",\"deno.json\",\"deno.jsonc\",\"deno.lock\",\"global.json\",\"NuGet.Config\",\"Directory.Packages.props\",\"mix.exs\",\"mix.lock\",\"go.mod\",\"go.sum\",\"stack.yaml\",\"stack.yaml.lock\",\"pom.xml\",\"build.gradle\",\"build.gradle.kts\",\"settings.gradle\",\"settings.gradle.kts\",\"gradle.properties\",\"package-lock.json\",\"yarn.lock\",\"pnpm-lock.yaml\",\"npm-shrinkwrap.json\",\"requirements.txt\",\"Pipfile\",\"Pipfile.lock\",\"pyproject.toml\",\"setup.py\",\"setup.cfg\",\"Gemfile\",\"Gemfile.lock\",\"uv.lock\",\"CODEOWNERS\",\"DESIGN.md\",\"README.md\",\"CONTRIBUTING.md\",\"CHANGELOG.md\",\"SECURITY.md\",\"CODE_OF_CONDUCT.md\",\"AGENTS.md\",\"CLAUDE.md\",\"GEMINI.md\"],\"protected_files_policy\":\"request_review\",\"title_prefix\":\"[ci-fix] \"},\"create_report_incomplete_issue\":{},\"missing_data\":{},\"missing_tool\":{},\"noop\":{\"max\":1,\"report-as-issue\":\"true\"},\"push_to_pull_request_branch\":{\"allowed_files\":[\"src/Core/**\",\"src/Controls/**\",\"src/Essentials/**\",\"src/BlazorWebView/**\",\"src/TestUtils/**\",\"src/Templates/**\",\"**/PublicAPI.Unshipped.txt\"],\"if_no_changes\":\"warn\",\"max\":3,\"max_patch_size\":4096,\"protect_top_level_dot_folders\":true,\"protected_files\":[\"package.json\",\"bun.lockb\",\"bunfig.toml\",\"deno.json\",\"deno.jsonc\",\"deno.lock\",\"global.json\",\"NuGet.Config\",\"Directory.Packages.props\",\"mix.exs\",\"mix.lock\",\"go.mod\",\"go.sum\",\"stack.yaml\",\"stack.yaml.lock\",\"pom.xml\",\"build.gradle\",\"build.gradle.kts\",\"settings.gradle\",\"settings.gradle.kts\",\"gradle.properties\",\"package-lock.json\",\"yarn.lock\",\"pnpm-lock.yaml\",\"npm-shrinkwrap.json\",\"requirements.txt\",\"Pipfile\",\"Pipfile.lock\",\"pyproject.toml\",\"setup.py\",\"setup.cfg\",\"Gemfile\",\"Gemfile.lock\",\"uv.lock\",\"CODEOWNERS\",\"DESIGN.md\",\"README.md\",\"CONTRIBUTING.md\",\"CHANGELOG.md\",\"SECURITY.md\",\"CODE_OF_CONDUCT.md\",\"AGENTS.md\",\"CLAUDE.md\",\"GEMINI.md\"],\"required_labels\":[\"agentic-workflows\"],\"target\":\"*\",\"title_prefix\":\"[ci-fix] \"},\"report_incomplete\":{},\"update_pull_request\":{\"allow_body\":true,\"allow_title\":false,\"max\":3,\"target\":\"*\",\"update_branch\":false}}" + GH_AW_SAFE_OUTPUTS_HANDLER_CONFIG: "{\"add_comment\":{\"discussions\":false,\"max\":6,\"required_labels\":[\"agentic-workflows\"],\"required_title_prefix\":\"[ci-fix] \",\"target\":\"*\"},\"add_labels\":{\"allowed\":[\"p/0\"],\"max\":3,\"required_labels\":[\"agentic-workflows\"],\"required_title_prefix\":\"[ci-fix] \",\"target\":\"*\"},\"create_pull_request\":{\"allowed_base_branches\":[\"main\"],\"allowed_branches\":[\"ci-fix/**\"],\"allowed_files\":[\"src/Core/**\",\"src/Controls/**\",\"src/Essentials/**\",\"src/BlazorWebView/**\",\"src/TestUtils/**\",\"src/Templates/**\",\"**/PublicAPI.Unshipped.txt\"],\"base_branch\":\"main\",\"draft\":true,\"labels\":[\"agentic-workflows\"],\"max\":3,\"max_patch_files\":100,\"max_patch_size\":4096,\"protect_top_level_dot_folders\":true,\"protected_files\":[\"package.json\",\"bun.lockb\",\"bunfig.toml\",\"deno.json\",\"deno.jsonc\",\"deno.lock\",\"global.json\",\"NuGet.Config\",\"Directory.Packages.props\",\"mix.exs\",\"mix.lock\",\"go.mod\",\"go.sum\",\"stack.yaml\",\"stack.yaml.lock\",\"pom.xml\",\"build.gradle\",\"build.gradle.kts\",\"settings.gradle\",\"settings.gradle.kts\",\"gradle.properties\",\"package-lock.json\",\"yarn.lock\",\"pnpm-lock.yaml\",\"npm-shrinkwrap.json\",\"requirements.txt\",\"Pipfile\",\"Pipfile.lock\",\"pyproject.toml\",\"setup.py\",\"setup.cfg\",\"Gemfile\",\"Gemfile.lock\",\"uv.lock\",\"CODEOWNERS\",\"DESIGN.md\",\"README.md\",\"CONTRIBUTING.md\",\"CHANGELOG.md\",\"SECURITY.md\",\"CODE_OF_CONDUCT.md\",\"AGENTS.md\",\"CLAUDE.md\",\"GEMINI.md\"],\"protected_files_policy\":\"request_review\",\"title_prefix\":\"[ci-fix] \"},\"create_report_incomplete_issue\":{},\"mark_pull_request_as_ready_for_review\":{\"max\":3,\"required_labels\":[\"agentic-workflows\"],\"required_title_prefix\":\"[ci-fix] \",\"target\":\"*\"},\"missing_data\":{},\"missing_tool\":{},\"noop\":{\"max\":1,\"report-as-issue\":\"true\"},\"push_to_pull_request_branch\":{\"allowed_files\":[\"src/Core/**\",\"src/Controls/**\",\"src/Essentials/**\",\"src/BlazorWebView/**\",\"src/TestUtils/**\",\"src/Templates/**\",\"**/PublicAPI.Unshipped.txt\"],\"if_no_changes\":\"warn\",\"max\":3,\"max_patch_size\":4096,\"protect_top_level_dot_folders\":true,\"protected_files\":[\"package.json\",\"bun.lockb\",\"bunfig.toml\",\"deno.json\",\"deno.jsonc\",\"deno.lock\",\"global.json\",\"NuGet.Config\",\"Directory.Packages.props\",\"mix.exs\",\"mix.lock\",\"go.mod\",\"go.sum\",\"stack.yaml\",\"stack.yaml.lock\",\"pom.xml\",\"build.gradle\",\"build.gradle.kts\",\"settings.gradle\",\"settings.gradle.kts\",\"gradle.properties\",\"package-lock.json\",\"yarn.lock\",\"pnpm-lock.yaml\",\"npm-shrinkwrap.json\",\"requirements.txt\",\"Pipfile\",\"Pipfile.lock\",\"pyproject.toml\",\"setup.py\",\"setup.cfg\",\"Gemfile\",\"Gemfile.lock\",\"uv.lock\",\"CODEOWNERS\",\"DESIGN.md\",\"README.md\",\"CONTRIBUTING.md\",\"CHANGELOG.md\",\"SECURITY.md\",\"CODE_OF_CONDUCT.md\",\"AGENTS.md\",\"CLAUDE.md\",\"GEMINI.md\"],\"required_labels\":[\"agentic-workflows\"],\"target\":\"*\",\"title_prefix\":\"[ci-fix] \"},\"report_incomplete\":{},\"update_pull_request\":{\"allow_body\":true,\"allow_title\":false,\"max\":3,\"target\":\"*\",\"update_branch\":false}}" GH_AW_CI_TRIGGER_TOKEN: ${{ secrets.GH_AW_CI_TRIGGER_TOKEN }} with: github-token: ${{ secrets.GH_AW_GITHUB_TOKEN || secrets.GITHUB_TOKEN }} diff --git a/.github/workflows/ci-status-fix.md b/.github/workflows/ci-status-fix.md index d3227e25170c..9c2f46054029 100644 --- a/.github/workflows/ci-status-fix.md +++ b/.github/workflows/ci-status-fix.md @@ -15,7 +15,9 @@ description: | humans (the open tracking issue is the hand-off surface; a dedicated [ci-fix][needs-human] PR is planned but currently deferred — see Step 6). CI on the PR is kicked by a human `/azp run` for now; the loop watches, classifies, - and re-fixes autonomously between kicks. + and re-fixes autonomously between kicks. When the SPECIFIC test a PR fixed is + confirmed green in that PR's own CI, the loop marks the draft PR ready for review + (a state transition only — it never approves or merges). Never mutes tests, but de-flakes genuinely flaky ones (deterministic synchronization, no retries / timeout bumps). Always skips visual-regression / screenshot issues. @@ -239,8 +241,15 @@ safe-outputs: # PER-RUN total (not per-PR): a sweep may surface several green PRs and/or # annotate several flaky reds in one run. At max:1 all but the first comment # were silently dropped, starving the workflow's #1 value (surfacing green PRs - # for review). Raised to 3 to match the per-run throughput of the other outputs. - max: 3 + # for review). Sized to 6 (not 3) because a single green DRAFT PR that gets + # flipped ready in one sweep spends TWO comment slots — the Step 3 ✅ surface + # comment (which still names the /azp-gated legs a human must kick, so it is NOT + # redundant with 🎯) AND the Step 3.6 T3 🎯 readiness comment. At max:3 the shared + # bucket drained after ~1 draft flip and T3's atomicity pre-check then deferred + # every further mark-ready even while the mark-ready/add_labels buckets (also 3) + # sat idle; 6 lets ~3 draft flips (2 comments each) land per sweep, so the + # mark-ready:3 / add_labels:3 caps are actually reachable. + max: 6 target: "*" # Hard constraint (defense-in-depth): only comment on THIS workflow's own # ci-fix PRs — [ci-fix] title prefix AND agentic-workflows label. Mirrors the @@ -261,13 +270,51 @@ safe-outputs: # never the title, so disable title rewrites — this compiles to allow_title:false # and removes any ability to retitle an arbitrary PR. title: false - # NOTE: gh-aw v0.79.8 does NOT emit required-title-prefix/required-labels into + # NOTE: gh-aw v0.80.9 (the pinned compiler) does NOT emit required-title-prefix/required-labels into # the compiled config for update-pull-request (verified against the lock — it # silently drops them, unlike add-comment / push-to-pull-request-branch which # honor them). So which-PR scoping here relies on prompt Hard-Rule 6 + # min-integrity:approved; the residual is body-marker edits only (no code, no # merge, no close), capped at max:3. Revisit required-* if a future gh-aw # compiler emits them for this output. + mark-pull-request-as-ready-for-review: + # Flip a validated draft [ci-fix] PR to ready-for-review once the SPECIFIC test + # this PR was opened to fix is confirmed green in the PR's OWN CI (Step 3.6) — + # even when unrelated legs are red. The workflow is schedule/dispatch-triggered + # (no triggering-PR context), so target "*" lets the agent name the PR number it + # validated. This is a state transition ONLY: it never approves and never merges — + # a human still reviews and merges. + # PER-RUN total (not per-PR): a sweep may validate several PRs' target tests green. + max: 3 + target: "*" + # Hard constraint (defense-in-depth): only ever un-draft THIS workflow's own + # ci-fix PRs — [ci-fix] title prefix AND agentic-workflows label — so a confused + # or prompt-injected agent cannot mark an arbitrary PR ready. (If the v0.80.9 + # compiler silently drops these — as documented for update-pull-request above — + # the Step 3.6 preconditions + min-integrity:approved are the compensating scope + # controls; verify against the lock after compiling.) + required-title-prefix: "[ci-fix] " + required-labels: [agentic-workflows] + add-labels: + # Apply the p/0 priority label to a [ci-fix] PR at the exact moment the loop flips it + # from draft to ready-for-review (Step 3.6 T3) — so a validated, review-ready fix lands + # in the team's p/0 triage queue instead of sitting unseen in the draft backlog. Paired + # 1:1 with mark-pull-request-as-ready-for-review; target "*" lets the agent name the PR + # it just validated. + # allowed = HARD allowlist: the agent may ONLY ever add p/0, nothing else. This caps the + # blast radius of a confused/prompt-injected agent to exactly one benign priority label — + # it can never apply a downstream-triggering or destructive label. + allowed: [p/0] + # PER-RUN total (not per-PR): a sweep may mark several PRs ready in one run; each adds + # one label, so this matches the mark-ready per-run cap (3). + max: 3 + target: "*" + # Hard constraint (defense-in-depth): only ever label THIS workflow's own ci-fix PRs — + # [ci-fix] title prefix AND agentic-workflows label. (If the v0.80.9 compiler drops these + # — as documented for update-pull-request above — the Step 3.6 preconditions + + # min-integrity:approved + the allowed:[p/0] allowlist are the compensating controls.) + required-title-prefix: "[ci-fix] " + required-labels: [agentic-workflows] timeout-minutes: 90 @@ -339,33 +386,48 @@ through `safe-outputs`. 5. **One issue = one outcome per run.** Exactly one of: respond to a maintainer's change-request on the open PR (Track C, Step 3.5.R); open the first fix/help/ de-flake PR; advance an existing PR by one attempt (push a follow-up fix); - surface a validated-green PR for review; annotate an unrelated-flake red; wait + surface a validated-green PR for review; mark a target-validated draft PR + ready-for-review (Step 3.6 T3 — the terminal outcome that supersedes the same + run's surface/annotate precursor line), or record its `already-ready` + steady-state no-op; annotate an unrelated-flake red; wait (CI not yet settled); or a recorded skip (the dedicated needs-human PR is deferred — the attempt cap records a skip instead; see Step 6). Always prefer advancing or opening a PR over a skip when a non-mute diff is producible. 6. **All writes via `safe-outputs`.** Allowed outputs: `create_pull_request` (first attempt only), `push_to_pull_request_branch` (advance an existing PR by one attempt), `update_pull_request` (bump the attempt marker / refresh the - prior-attempts table), and `add_comment` (progress notes on the `[ci-fix]` PR - ONLY). NEVER comment on the tracking issue (issues are locked by + prior-attempts table), `add_comment` (progress notes on the `[ci-fix]` PR + ONLY), `mark_pull_request_as_ready_for_review` (flip a target-validated draft + `[ci-fix]` PR from draft to ready — Step 3.6 T3 only), and `add_labels` (add + ONLY the `p/0` label, ONLY on that same draft→ready transition — Step 3.6 T3). + NEVER comment on the tracking issue (issues are locked by `.github/workflows/ci-scan-lock-issues.yml`) — `add_comment` targets the PR. No `gh pr create`, no manual `git push`. **Defense-in-depth:** `add_comment` and `update_pull_request` use `target: "*"` (the agent supplies the PR number). - `add_comment` is now config-locked to `required-title-prefix: "[ci-fix] "` + + `add_comment`, `mark_pull_request_as_ready_for_review`, and `add_labels` are + config-locked to `required-title-prefix: "[ci-fix] "` + `required-labels: [agentic-workflows]` (hard-enforced by the handler, same as - `push_to_pull_request_branch`), so a comment can only ever land on THIS - workflow's own PRs. `update_pull_request` CANNOT be config-locked to a - title/label in gh-aw v0.79.8 (the compiler silently drops `required-*` for that - output — verified against the lock), so it keeps `title: false` (no retitles; - body-marker edits only) plus this prompt-level guard. Before emitting either, - VERIFY the target PR carries BOTH the `[ci-fix]` title prefix AND the - `agentic-workflows` label; never comment on or edit the body of any PR that - lacks both. + `push_to_pull_request_branch`), and `add_labels` additionally has an + `allowed: [p/0]` allowlist so it can ONLY ever add `p/0` — so these can only + ever land on THIS workflow's own PRs. `update_pull_request` CANNOT be + config-locked to a title/label in gh-aw v0.80.9 (the compiler silently drops + `required-*` for that output — verified against the lock), so it keeps + `title: false` (no retitles; body-marker edits only) plus this prompt-level + guard. Before emitting ANY of these, VERIFY the target PR carries BOTH the + `[ci-fix]` title prefix AND the `agentic-workflows` label; never comment on, + edit, un-draft, or label any PR that lacks both. 7. **Per-run safe-output caps (per RUN, not per PR).** Each output type is capped per run: `create_pull_request` 3, `push_to_pull_request_branch` 3, - `add_comment` 3, `update_pull_request` 3. Note an ADVANCE spends one push **and** - one update **and** one comment, so ≤ 3 advances/run; surfacing a green or - annotating a flake spends one comment. When a bucket is exhausted, do NOT keep + `add_comment` 6, `update_pull_request` 3, `mark_pull_request_as_ready_for_review` + 3, `add_labels` 3 (counts LABELS, not calls). Note an ADVANCE spends one push + **and** one update **and** one comment, so ≤ 3 advances/run; surfacing a green or + annotating a flake spends one comment; a **mark-ready (Step 3.6 T3)** of a + still-draft PR spends TWO comments (the Step 3 ✅ surface **and** the T3 🎯 + readiness note) **and** one mark-ready **and** one label as an ALL-OR-NOTHING set + (T3 pre-checks those buckets and defers the whole PR if any is exhausted — never + mark a PR ready without its 🎯 audit comment). `add_comment` is therefore sized 6 + (not 3) so ~3 draft flips, at 2 comments each, can land in one sweep instead of the + shared comment bucket starving the otherwise-idle mark-ready/add_labels buckets. When a bucket is exhausted, do NOT keep emitting (extras are silently dropped) — record `skipped: per-run cap reached; deferring PR # to next cycle` for each remaining PR so the drop is a deliberate, logged decision. The continuous loop picks the deferred PRs up next @@ -409,8 +471,9 @@ For every open tracking issue in scope, converge on exactly one outcome: | Open first help-wanted draft `[ci-fix]` PR | No open PR yet; a plausible candidate exists but cannot be runner-validated (device/UI tests) (attempt 1) | | Open first de-flake draft `[ci-fix]` PR | No open PR yet; failure is intermittent (green-on-retry) from a genuine test-quality defect; a deterministic-synchronization fix is producible (attempt 1, Step 4.7 bucket b) | | Advance an existing PR (push attempt N+1) | Open PR's own CI settled red *because of the fix itself*, no human engaged, marker < 10 — push a NEW distinct fix onto the same branch (Step 3.5 → 5.6) | -| Surface a validated-green PR | Open PR's own CI settled green — comment "validated, attempt N/10; ready for review" and do NOT advance (Step 3.5) | -| Annotate an unrelated-flake red | Open PR is red only on baseline-flake / unrelated legs — comment which leg needs a re-run; do NOT burn an attempt (Step 3.5) | +| Surface a validated-green PR | Open PR's own CI settled green — comment "validated, attempt N/10; ready for review" and do NOT advance (Step 3.5), then mark the draft PR ready for review once the fixed test is confirmed green (Step 3.6) | +| Annotate an unrelated-flake red | Open PR is red only on baseline-flake / unrelated legs — comment which leg needs a re-run; do NOT burn an attempt (Step 3.5); if the SPECIFIC fixed test is confirmed green on that build, still mark the draft PR ready for review (Step 3.6) | +| Mark a validated PR ready for review | The specific test this PR fixed is confirmed green in the PR's own CI (even if unrelated legs are red) — comment the target-test result and transition the draft PR to ready for review; never approve or merge (Step 3.6) | | Wait | Open PR's CI is pending / not yet settled on the current head SHA — do nothing this cycle (Step 3.5) | | Hand-off skip (attempt cap) | Marker == 10 and the signature still reproduces — stop and defer to humans; the dedicated `[ci-fix][needs-human]` PR is planned but currently deferred (Step 6) | | Recorded skip | Visual-regression, human engaged, already-handled, fixed-in-latest-build, infra-flake, out-of-bounds, only-mute-available, or no novel approach producible | @@ -685,14 +748,34 @@ Run these gates in order — the FIRST that fires decides this cycle's outcome: 1. **Human engaged.** If `C.humanEngaged` → `skipped: human engaged on PR #; deferring` and stop. Never fight a human reviewer — the intentional hand-off boundary still holds the moment a person touches the PR. -2. **CI not settled / unknown / incomplete prefetch.** If `C.checksSettled == false` - OR `C.overallConclusion` is `pending`, `neutral`, or `unknown`, OR - `C.dataComplete == false` → the fix's CI has not finished on the current head SHA - (`C.headSha`), or the prefetch could not fully read this PR (a partial read can - understate `humanEngaged` / `attempt`). `skipped: PR # CI pending / prefetch - incomplete on ; waiting` and stop. *(Round 1: this is where a - maintainer `/azp run` is awaited — the `/azp`-gated uitests/devicetests legs will - not have run until a human kicks them.)* +2. **CI not settled — but validate the target test first (target-focused readiness).** + If `C.checksSettled == false` OR `C.overallConclusion` is `pending`, `neutral`, or + `unknown`, OR `C.dataComplete == false` → the *overall* build has not finished on the + current head SHA (`C.headSha`), or the prefetch could not fully read this PR (a partial + read can understate `humanEngaged` / `attempt`). Unrelated legs still draining must NOT + keep an already-proven fix parked as a draft, so before waiting, try the target-focused + fast-path: + - **2a — Target-green fast-path (draft `[ci-fix]` PRs only).** If `C.dataComplete == + true` AND the Step 3.6 preconditions hold (draft PR, `[ci-fix] ` title prefix AND the + `agentic-workflows` label), run Step 3.6 **T1–T2** against `C.headSha` now: identify + the target test(s) and read the PR's OWN build **timeline / per-leg status** (anonymous + `_apis/build` — never `_apis/test`, per Hard-Rule 8). If EVERY target test is + **VALIDATED-GREEN** (per T2's all-platform definition below — the target's category leg + is `succeeded` on every platform that runs it, failed on none, and NO target platform + family the pipeline covers is still pending/unconcluded, per T2's "no platform left + unverified" rule; a platform that simply has no leg for the target's category — the test + doesn't run there — is fine, not a gap) AND every leg in + `C.failedLegs` that has ALREADY concluded classifies as **unrelated flake** (Step 4 / + Step 4.7 method — the fix is implicated in NO completed red), then the fix is proven + regardless of unrelated *pending* legs → run **Step 3.6 T3** (mark ready + 🎯 comment) + and stop. This early-out NEVER advances an attempt and NEVER acts on a red that could + be the fix's fault. + - **2b — Otherwise WAIT.** If the target test has not yet executed (pending/absent on + its leg), or a completed red is (or may be) caused by the fix, or `C.dataComplete == + false`, or this PR has no identifiable target test → `skipped: PR # CI pending / + target not yet validated on ; waiting` and stop. *(Round 1: this is where a + maintainer `/azp run` is awaited — the `/azp`-gated uitests/devicetests legs will not + have run until a human kicks them.)* 3. **Green → surface for review.** If `C.overallConclusion == "success"`: the checks that RAN are green. **Primary-gate check first:** the deterministic `success` verdict only certifies "at least one green check, nothing failing or @@ -704,18 +787,31 @@ Run these gates in order — the FIRST that fires decides this cycle's outcome: # primary CI gate (maui-pr) not green on ; waiting` and stop — do NOT surface. (The `/azp`-gated `maui-pr-uitests` (def 313) / `maui-pr-devicetests` (def 314) legs MAY still be un-run — that is expected and is named below, not a - reason to withhold the surface.) **Idempotency:** scan the PR's existing comments - for a prior bot `✅ … validated … on ` note for THIS head SHA — if one - already exists, record `already-surfaced PR # (head )` and stop - WITHOUT re-commenting (re-surfacing the same green every tick is noise and burns - the per-run comment budget other PRs need). Otherwise resolve the attempt number from - `C.effectiveAttempt` (the authoritative max(marker, bot-commit) counter; omit the - number only if it is somehow indeterminate). **Dry-run gate (Step 0):** if `dry_run == "true"`, do NOT emit any comment — instead print the intended `✅` validated-green body to the run log, tally `dry-run: would-surface-green PR #`, and stop. Otherwise `add_comment` on PR #: `✅ Attempt /10 validated - — the fix's CI is green on . Ready for human review.` - Name any `/azp`-gated legs (uitests def 313 / devicetests def 314) that have not - run and still need a maintainer `/azp run`. Do NOT advance. Record - `surfaced-green PR # (attempt /10)` and stop. *(This directly - attacks the real bottleneck — no reviews — so it is the highest-value outcome.)* + reason to withhold the surface.) **Comment idempotency + dry-run suppress the ✅ + comment ONLY — neither skips Step 3.6.** Scan the PR's existing comments for a prior + bot `✅ … validated … on ` note for THIS head SHA; if one exists, set + `SKIP_SURFACE_COMMENT = true`. Resolve the attempt number from `C.effectiveAttempt` + (the authoritative max(marker, bot-commit) counter; omit the number only if it is + somehow indeterminate). Then post the surface comment UNLESS suppressed: + - if `dry_run == "true"`: do NOT emit — print the intended `✅` validated-green body to + the run log and tally `dry-run: would-surface-green PR #`; + - else if `SKIP_SURFACE_COMMENT`: do NOT re-post — record `already-surfaced PR # + (head )` (re-surfacing the same green every tick is noise and burns the + per-run comment budget other PRs need); + - else `add_comment` on PR #: `✅ Attempt /10 validated — the fix's CI is + green on .` naming any `/azp`-gated legs (uitests def 313 / devicetests def + 314) that have not run and still need a maintainer `/azp run`; record `surfaced-green + PR # (attempt /10)`. (Do NOT assert "ready for human review" on a PR that + is still a draft — Step 3.6, not this comment, owns the draft→ready flip and posts its + own 🎯 announcement when it fires.) + Do NOT advance. Then — in ALL of the above cases — run **Step 3.6** (target-test + readiness gate) for this PR before stopping: its `/azp`-gated target legs may conclude + green on this SAME head SHA on a LATER sweep (an `/azp run` adds no commit), and Step + 3.6 is the ONLY place the draft→ready flip happens, so it MUST re-evaluate every sweep — + never `stop` here before it. *(This directly attacks the real bottleneck — no reviews — + so it is the highest-value outcome.)* 4. **Red → classify caused-by-fix vs unrelated-flake.** If `C.overallConclusion == "failure"`, analyze the PR's OWN failing build (NOT `main`): find the AzDO `maui-pr` build for this PR (filter builds by `branchName=refs/pull//merge`, @@ -723,18 +819,26 @@ Run these gates in order — the FIRST that fires decides this cycle's outcome: method and Step 4.7 flake buckets to `C.failedLegs`. - **Unrelated flake only** (every failed leg is known-flaky / infra / a pre-existing baseline red NOT introduced by the fix): do NOT burn an attempt. - **Idempotency first:** scan the PR's existing comments for a prior bot - `♻️ … unrelated flake … on ` note for THIS head SHA — if one already - exists, record `already-annotated-flake PR # (head )` and stop - WITHOUT re-commenting (re-annotating the same flake every 12h sweep is noise and - burns the per-run comment budget other PRs need). Otherwise resolve the attempt - number from `C.effectiveAttempt` (the authoritative max(marker, bot-commit) - counter), omitting the `Attempt /10:` prefix only if it is somehow - indeterminate. **Dry-run gate (Step 0):** if `dry_run == "true"`, do NOT emit any comment — instead print the intended `♻️` unrelated-flake body to the run log, tally `dry-run: would-annotate-flake PR #`, and stop. Otherwise `add_comment` on PR #: `♻️ Attempt /10: red is - unrelated flake on leg(s) () on ; the fix itself is not - implicated. A maintainer re-run (/azp run ) should clear it.` Record - `annotated-flake PR # (head )` and stop. *(Round 1: human re-runs; - Round 2: auto re-trigger.)* + **Comment idempotency + dry-run suppress the ♻️ comment ONLY — neither skips Step + 3.6.** Scan the PR's existing comments for a prior bot `♻️ … unrelated flake … on + ` note for THIS head SHA; if one exists, set `SKIP_FLAKE_COMMENT = true`. + Resolve the attempt number from `C.effectiveAttempt` (the authoritative max(marker, + bot-commit) counter), omitting the `Attempt /10:` prefix only if it is + somehow indeterminate. Then post the flake note UNLESS suppressed: + - if `dry_run == "true"`: do NOT emit — print the intended `♻️` unrelated-flake body + to the run log and tally `dry-run: would-annotate-flake PR #`; + - else if `SKIP_FLAKE_COMMENT`: do NOT re-post — record `already-annotated-flake PR + # (head )` (re-annotating the same flake every 12h sweep is noise and + burns the per-run comment budget other PRs need); + - else `add_comment` on PR #: `♻️ Attempt /10: red is unrelated flake on + leg(s) () on ; the fix itself is not implicated. A + maintainer re-run (/azp run ) should clear it.` record `annotated-flake + PR # (head )`. + Then — in ALL of the above cases — run **Step 3.6** (target-test readiness gate) for + this PR before stopping: its `/azp`-gated target legs may conclude green on this SAME + head SHA on a LATER sweep, and Step 3.6 is the ONLY place the draft→ready flip + happens, so it MUST re-evaluate every sweep — never `stop` here before it. *(Round 1: + human re-runs; Round 2: auto re-trigger.)* - **Caused by the fix** (a failed leg still matches the original target signature, or the fix introduced a NEW failure): advance an attempt. - **Attempt count.** `attempt = C.effectiveAttempt` — the authoritative @@ -752,6 +856,188 @@ Run these gates in order — the FIRST that fires decides this cycle's outcome: from every prior commit on this branch**, emitted via the Step 5.6 ADVANCE path (push onto the same PR). +#### Step 3.6 — Target-test verification & mark-ready gate + +Reached from the Step 3 **green-surface** branch, the Step 4 **unrelated-flake** branch +(AFTER that branch has posted its comment), and the Step 3.5 **gate-2 target-focused +fast-path** (2a — while unrelated legs are still pending). Purpose: confirm the SPECIFIC +test(s) this PR was opened to fix now PASS in the PR's own CI, and — only when they do +— transition the draft `[ci-fix]` PR to **ready for review** so a maintainer sees a +validated fix instead of a draft. This is the ONLY place the loop flips draft→ready; it +is a state transition, **never** an approval or a merge (a human still reviews and +merges). Overall red on *unrelated* legs must NOT gate readiness — we validate the fix, +not the base branch's flakiness. + +**Preconditions** (ALL must hold; otherwise record `skipped: readiness N/A PR # +()` and stop this gate): +- `C.isDraft == true` — if the PR is already ready-for-review, the draft→ready + transition is already done; do NOT re-mark **and do NOT re-apply `p/0`**. `p/0` is applied + exactly ONCE, atomically with the draft→ready flip (T3 — all-or-nothing with the 🎯 audit + comment + mark-ready), so an already-ready loop PR that is MISSING `p/0` has almost + certainly had it **removed by a maintainer** de-prioritizing the PR — a pure triage action + the human-engagement guard (which inspects comments/reviews/commits, NOT label changes) + cannot see. Re-adding `p/0` every sweep would fight that maintainer indefinitely, violating + the loop's "never override a human" contract. So the loop does NOT reconcile the label: + record `already-ready PR #` and stop this gate. (Trade-off: on the rare occasion a + transient API error drops `p/0` at flip time *after* the T3 atomic pre-check passed, it is + not auto-re-added — but the PR is still ready-for-review with its 🎯 audit comment, and + re-adding `p/0` is a trivial manual action; that is strictly preferable to steam-rolling a + maintainer's deliberate de-prioritization.) +- The PR is unmistakably THIS workflow's own: `[ci-fix] ` title prefix AND the + `agentic-workflows` label (mirrors the safe-output lock; the compensating scope + control if the v0.80.9 compiler drops the declarative required-*). +- You reached this gate from Step 3 (green), Step 4 (**unrelated-flake**), or the Step 3.5 + gate-2 **target-focused fast-path** (2a) — i.e. the fix is NOT implicated in any red that + has concluded. If Step 4 classified a red as **caused by the fix** (ADVANCE), do NOT run + this gate — advance the attempt instead. + +**T1 — Identify the target test(s).** From the `[ci-scan]` issue signature the fix +addresses (and the PR's own diff), extract the fully-qualified test method name(s) the +fix targets — e.g. `SafeAreaShouldWorkOnAllShellTabs`. For a de-flake it is the +de-flaked test; for a product fix it is the originally-failing test(s). If NO specific +test can be identified (e.g. a product build-break rather than a test failure), this is a +**build-only fix** — there is no single test to validate cross-platform, so validate at +**whole-build** granularity rather than stopping (Step 3 only *comments*; Step 3.6 is the +sole draft→ready flip, so a build-only fix is undrafted HERE or not at all). We only reach +this gate from Step 3 (green) / Step 4 (unrelated-flake) / gate-2a, so the fix is not +implicated in any concluded red; additionally require, via the T2 timeline method on +`C.headSha`, that the primary build pipeline `maui-pr` (def 302) has CONCLUDED with EVERY +one of its platform build legs `succeeded`/`completed` and NONE failed/canceled or still +pending — the build is green on **every** platform, not just the originally-broken one (the +cross-platform guard, applied to the build instead of a test). A build-only fix is undrafted +ONLY on the strength of the auto-running `maui-pr` (def 302) whole-build green, which the +loop CAN observe: if the PR's `[ci-scan]` issue signature or its own diff indicates the +ORIGINATING failure was in a `/azp`-gated pipeline (`maui-pr-uitests` def 313 or +`maui-pr-devicetests` def 314) rather than `maui-pr` (def 302), do NOT undraft on +`maui-pr`-green alone — those pipelines do not auto-run on this PR (GITHUB_TOKEN cannot +trigger them), so a green `maui-pr` build is NOT evidence the gated build break is fixed; +record `skipped: build-only fix PR # targets gated pipeline () not run — +deferring to human` and stop this gate. Otherwise, set `TARGET := "the maui-pr build +(build-only fix — no single target test)"` and proceed to **T3** to mark ready. If any `maui-pr` build leg is still unconcluded, record `skipped: build-only fix PR +# not yet whole-build green (leg(s) pending)` and stop this gate WITHOUT marking +ready (a green subset is not enough — a still-pending build leg could yet fail). + +**T2 — Verify the target test's platform legs are green (anonymous, leg-level).** Per-test +outcomes come from the AzDO **test-results** API (`_apis/test/...`), which is **NOT reachable +anonymously — Hard-Rule 8**: those endpoints 302-redirect to a sign-in page on this runner, so +a `curl` returns an HTML redirect (not JSON) and every target test would look "not executed". +Validate instead at **leg granularity** using ONLY the anonymous `_apis/build/...` timeline +(Hard-Rule 8) — a `succeeded` category leg means every test in that category **that actually +ran** passed on that platform. This is a strong bar **only if the target test genuinely +executes**: a job can go green while one specific test is skipped at runtime (`[Ignore]`, +`Assert.Ignore()`, `Assert.Inconclusive()`, a `Skip=`/conditional `[Fact]`, an `#if`-out, or +an early `return` before the asserts). So before trusting a green leg as proof the target +passed, **confirm from the PR's OWN diff that the fix does NOT skip, ignore, disable, or +short-circuit the target test** (it adds no `[Ignore]`/`[Explicit]`, `Assert.Ignore`/ +`Assert.Inconclusive`, `Skip=`, category-exclusion, `#if`-out, or early `return` around the +target). If the fix could cause the target to be runtime-skipped rather than genuinely pass, +leg-level green is NOT sufficient evidence — record `skipped: target test may be +runtime-skipped by this fix (diff adds ); leg-green insufficient, deferring to human +on PR #` and stop this gate WITHOUT marking ready. Otherwise (the target genuinely runs +and asserts, as a de-flake or product fix does) a green category leg cannot hide a +target-test failure, so treat it as authoritative. + +Map the target test to the CI leg(s) that run it. A leg name encodes platform + test category +(e.g. `Android UITests SafeAreaEdges,Shadow`, `iOS UITests SafeAreaEdges,Shadow`, `macOS +UITests SafeAreaEdges,Shadow`, `Windows UITests SafeAreaEdges,Shadow`). Determine the target +test's UI-test category — its `[Category(UITestCategories.X)]` in the test/HostApp source, +visible in the PR diff or the test file — to know which leg-name substring identifies its legs; +for a device test, the per-platform device-test legs. + +Using the SAME build-discovery as Step 4 (filter AzDO builds by `branchName=refs/pull//merge` +or `sourceVersion == C.headSha`), read each build's **timeline** on `C.headSha` for the +pipeline(s) that RUN the target test — `maui-pr` (def 302) for unit/integration tests, +`maui-pr-uitests` (def 313) for Appium UI tests, `maui-pr-devicetests` (def 314) for device +tests: + +```bash +ORG=dnceng-public; PROJ=public; BUILD= +# Anonymous + Hard-Rule-8-compliant. NEVER call _apis/test/... (it 302-redirects to sign-in). +# Each type=="Job" record is one platform leg: result (succeeded/failed/canceled/null), +# state (completed/inProgress/pending), name (encodes " UITests "): +curl -s "https://dev.azure.com/$ORG/$PROJ/_apis/build/builds/$BUILD/timeline?api-version=7.1" \ + | tee /tmp/gh-aw/agent/timeline_${BUILD}.json \ + | jq -r '.records[] | select(.type=="Job") | "\(.result)\t\(.state)\t\(.name)"' +``` + +(The prefetch already exposes per-leg status in `C.failedLegs` / the checks context, and +`gh pr checks ` lists the same per-leg rows with platform+category in the name — use +whichever is handy; the build timeline is the authoritative cross-check on the exact build.) + +Treat a target test as **VALIDATED-GREEN** only if, on `C.headSha`, the leg that runs its +category is green on **every platform it runs on** — a fix that repairs one platform must not +silently regress the same test's leg on another, so a green leg on the originally-red platform +alone is NOT enough. Across the target pipeline's platform legs (for a UI test: the Android, +iOS, Windows, and macOS/MacCatalyst legs running the target's category; for a device test: each +device-test platform), require ALL of: +- the target's category leg is `result == "succeeded"` **and** `state == "completed"` on + **every** platform that runs it, AND +- that leg is `failed` / `canceled` / aborted on **NO** platform, AND +- **no platform is left unverified.** For each platform family the target pipeline covers, that + platform's category leg must have CONCLUDED (`state == "completed"`) on `C.headSha`. If a + platform simply has no leg for the target's category, the test does not run there — that is + fine, not a gap. But if a platform's category leg has **not concluded** (`state` is + `inProgress` / pending, or an `/azp`-gated `maui-pr-uitests` / `maui-pr-devicetests` leg that + has not been kicked), the test's status on that platform is UNKNOWN → the fix is NOT yet + validated across platforms: record `skipped: target test green on but + not yet verified on (leg(s) pending / need /azp run) on PR #` + and stop this gate WITHOUT marking ready. + +A target test whose category leg never concluded on ANY platform (**not executed** anywhere — +e.g. its `/azp`-gated pipeline has not been kicked) is likewise NOT validated: record `skipped: +target test not yet executed on PR # ( not run — needs /azp run)` and stop +this gate WITHOUT marking ready. Do NOT overclaim — a green *sibling* leg (a different category +on the same platform) is not the target's leg, and a green leg on one platform is not a pass on +the others. + +**T3 — Mark ready + report.** If EVERY target test is VALIDATED-GREEN. The 🎯 comment, +the mark-ready, and the `p/0` label are THREE SEPARATE safe-outputs — the comment +existing does NOT prove the mark-ready took effect, so they are tracked independently: + +- **Comment idempotency (dup-suppress only — never gates the mark-ready).** We only reach + T3 while `C.isDraft == true` (the precondition stops an already-ready PR before here). So + if a prior bot comment for THIS head that carries the leading `🎯` anchor AND the + contiguous phrase `validated green on ` already exists (BOTH T3 variants — + `🎯 Target test validated green on ` and the build-only `🎯 Build validated green on + ` — contain that exact contiguous phrase, so this recognizes both; keying on + `target test` alone would instead miss the build-only comment and re-post it every sweep. + **Require the `🎯` anchor** so the match can NEVER be loosened to a gapped + `validated…green…on` form that would false-positive on the Step 3 `✅ … validated — the + fix's CI is green on ` surface comment and wrongly suppress the audit 🎯), the comment + landed on an earlier sweep but the **mark-ready did NOT take + effect** (the PR is still a draft) — set `SUPPRESS_COMMENT = true` (do not re-post the + duplicate 🎯 comment) but STILL complete the mark-ready + label below. Never treat the + comment's existence as "already marked ready" while the PR is still a draft. Record this + case as `re-marking PR # (🎯 present; mark-ready did not take on a prior sweep)`. +- **Atomicity budget pre-check (Hard-Rule 7).** Determine the outputs this gate will emit: + `mark_pull_request_as_ready_for_review` + `add_labels` always, plus `add_comment` unless + `SUPPRESS_COMMENT`. Verify a free per-run slot remains in EACH bucket you are about to + use. If ANY required bucket is exhausted, emit NONE of them — record `skipped: per-run + cap reached; deferring mark-ready PR # to next cycle` and stop this gate. Never mark a + PR ready (or label it) unless its 🎯 audit comment is GUARANTEED to exist for this exact + head SHA — either posted in this same sweep, or (in the `SUPPRESS_COMMENT` re-marking case) + already present from a prior sweep. The outputs you DO emit either all land or all defer + together; the audit trail must never be absent, but it is NOT re-posted when it already exists. +- **Dry-run gate (Step 0):** if `dry_run == "true"`, emit NOTHING — print the intended + readiness comment and "would mark ready + add p/0" to the run log, tally `dry-run: + would-mark-ready PR #`, and stop. +- Otherwise emit for THIS PR number ``: + 1. `add_comment` (ONLY if not `SUPPRESS_COMMENT`): `🎯 Target test validated green on + — passed on ALL platforms it runs on (, + buildId ). — not + caused by this fix."> Transitioning this PR from draft to ready for review and adding + `p/0`; a maintainer still reviews and merges.` (For a **build-only fix** — the T1 + whole-build fallback, no single target test — phrase the first clause as `🎯 Build + validated green on — the maui-pr build passed on ALL platforms (buildId + ).` instead of naming a test.) + 2. `mark_pull_request_as_ready_for_review` with `reason:` a one-line justification + naming the validated test(s) and ``. + 3. `add_labels` with `labels: ["p/0"]` for PR # — put the now-review-ready fix into + the team's p/0 priority queue so it is triaged, not lost in the draft backlog. (If + the PR somehow already carries `p/0`, this is a harmless no-op.) +- Record `marked-ready PR # (target green on ALL platforms on , + labeled p/0)` and stop. + #### Step 3.5.R — Maintainer change-request response (Track C) Reached from Step 3.5 gate 0. Goal: when an **eligible human reviewer** (Hard-Rule @@ -1383,7 +1669,19 @@ Filed by [`ci-status-fix`](https://github.com/dotnet/maui/blob/main/.github/work ### Step 8 — Per-issue tally + end-of-run summary -Per issue, append one outcome line to `/tmp/gh-aw/agent/coverage.txt`: +Per issue, append **one** outcome line to `/tmp/gh-aw/agent/coverage.txt` — the +terminal outcome for the cycle. When one cycle produces a chained pair — a PR gets a +same-cycle precursor line and **then** a Step 3.6 readiness line (`marked-ready` on the +draft→ready flip, or `already-ready` on a later steady-state sweep of an already-ready +PR) — record ONLY the terminal Step 3.6 readiness line: it supersedes ANY same-cycle +non-readiness precursor for that PR, so an aggregator keying on one-line-per-issue never +double-counts. The precursor may be a Step 3 green line (`surfaced-green` on the first ✅, +or `already-surfaced` when it already exists) OR — when an unrelated-flake red and a +target-green coincide — a Step 4 `annotated-flake` line; either way the readiness line is +terminal, and the superseded precursor's signal survives in the PR's 🎯/♻️ comment so no +information is lost. This covers the flip cycle (`surfaced-green` → `marked-ready`), the +post-flip steady state (`already-surfaced` → `already-ready`), and the flake-coincident +flip (`annotated-flake` → `marked-ready`): ``` # main attempt- @@ -1394,8 +1692,14 @@ Per issue, append one outcome line to `/tmp/gh-aw/agent/coverage.txt`: `advance-PR # attempt /10` (ADVANCE mode — new commit pushed to the existing PR), `surfaced-green PR #` (fix's own CI went green; commented for review, did not advance), `annotated-flake PR #` (red was unrelated flake; -commented, attempt NOT burned), `waiting PR #` (CI not settled yet), -`dry-run: would-`, `skipped: `. (The +commented, attempt NOT burned), `marked-ready PR #` (the specific fixed test +was confirmed green in the PR's own CI; draft flipped to ready for review), +`already-surfaced PR #` (the fix's ✅ green comment already existed this sweep; +re-post suppressed — superseded by the Step 3.6 readiness line when the PR also flips +ready that cycle), `already-ready PR #` (terminal steady state — the PR was flipped +ready on a prior cycle and remains green on an unchanged head; no action taken, +supersedes `already-surfaced`), `waiting PR #` (CI not settled yet), +`dry-run: would-`, `skipped: `. (The `needs-human-PR` outcome is reserved for the deferred hand-off — Step 6 currently records a skip instead, so it is not emitted.)
: `♻️ Attempt /10: red is unrelated flake on + leg(s) () on ; the fix itself is not implicated. A + maintainer re-run (/azp run ) should clear it.` record `annotated-flake + PR # (head )`. + Then — in ALL of the above cases — run **Step 3.6** (target-test readiness gate) for + this PR before stopping: its `/azp`-gated target legs may conclude green on this SAME + head SHA on a LATER sweep, and Step 3.6 is the ONLY place the draft→ready flip + happens, so it MUST re-evaluate every sweep — never `stop` here before it. *(Round 1: + human re-runs; Round 2: auto re-trigger.)* - **Caused by the fix** (a failed leg still matches the original target signature, or the fix introduced a NEW failure): advance an attempt. - **Attempt count.** `attempt = C.effectiveAttempt` — the authoritative @@ -762,6 +866,188 @@ Run these gates in order — the FIRST that fires decides this cycle's outcome: from every prior commit on this branch**, emitted via the Step 5.6 ADVANCE path (push onto the same PR). +#### Step 3.6 — Target-test verification & mark-ready gate + +Reached from the Step 3 **green-surface** branch, the Step 4 **unrelated-flake** branch +(AFTER that branch has posted its comment), and the Step 3.5 **gate-2 target-focused +fast-path** (2a — while unrelated legs are still pending). Purpose: confirm the SPECIFIC +test(s) this PR was opened to fix now PASS in the PR's own CI, and — only when they do +— transition the draft `[ci-fix-net11]` PR to **ready for review** so a maintainer sees +a validated fix instead of a draft. This is the ONLY place the loop flips draft→ready; +it is a state transition, **never** an approval or a merge (a human still reviews and +merges). Overall red on *unrelated* legs must NOT gate readiness — we validate the fix, +not the base branch's flakiness. + +**Preconditions** (ALL must hold; otherwise record `skipped: readiness N/A PR # +()` and stop this gate): +- `C.isDraft == true` — if the PR is already ready-for-review, the draft→ready + transition is already done; do NOT re-mark **and do NOT re-apply `p/0`**. `p/0` is applied + exactly ONCE, atomically with the draft→ready flip (T3 — all-or-nothing with the 🎯 audit + comment + mark-ready), so an already-ready loop PR that is MISSING `p/0` has almost + certainly had it **removed by a maintainer** de-prioritizing the PR — a pure triage action + the human-engagement guard (which inspects comments/reviews/commits, NOT label changes) + cannot see. Re-adding `p/0` every sweep would fight that maintainer indefinitely, violating + the loop's "never override a human" contract. So the loop does NOT reconcile the label: + record `already-ready PR #` and stop this gate. (Trade-off: on the rare occasion a + transient API error drops `p/0` at flip time *after* the T3 atomic pre-check passed, it is + not auto-re-added — but the PR is still ready-for-review with its 🎯 audit comment, and + re-adding `p/0` is a trivial manual action; that is strictly preferable to steam-rolling a + maintainer's deliberate de-prioritization.) +- The PR is unmistakably THIS workflow's own: `[ci-fix-net11] ` title prefix AND the + `agentic-workflows` label (mirrors the safe-output lock; the compensating scope + control if the v0.80.9 compiler drops the declarative required-*). +- You reached this gate from Step 3 (green), Step 4 (**unrelated-flake**), or the Step 3.5 + gate-2 **target-focused fast-path** (2a) — i.e. the fix is NOT implicated in any red that + has concluded. If Step 4 classified a red as **caused by the fix** (ADVANCE), do NOT run + this gate — advance the attempt instead. + +**T1 — Identify the target test(s).** From the `[ci-scan-net11]` issue signature the fix +addresses (and the PR's own diff), extract the fully-qualified test method name(s) the +fix targets — e.g. `SafeAreaShouldWorkOnAllShellTabs`. For a de-flake it is the +de-flaked test; for a product fix it is the originally-failing test(s). If NO specific +test can be identified (e.g. a product build-break rather than a test failure), this is a +**build-only fix** — there is no single test to validate cross-platform, so validate at +**whole-build** granularity rather than stopping (Step 3 only *comments*; Step 3.6 is the +sole draft→ready flip, so a build-only fix is undrafted HERE or not at all). We only reach +this gate from Step 3 (green) / Step 4 (unrelated-flake) / gate-2a, so the fix is not +implicated in any concluded red; additionally require, via the T2 timeline method on +`C.headSha`, that the primary build pipeline `maui-pr` (def 302) has CONCLUDED with EVERY +one of its platform build legs `succeeded`/`completed` and NONE failed/canceled or still +pending — the build is green on **every** platform, not just the originally-broken one (the +cross-platform guard, applied to the build instead of a test). A build-only fix is undrafted +ONLY on the strength of the auto-running `maui-pr` (def 302) whole-build green, which the +loop CAN observe: if the PR's `[ci-scan]` issue signature or its own diff indicates the +ORIGINATING failure was in a `/azp`-gated pipeline (`maui-pr-uitests` def 313 or +`maui-pr-devicetests` def 314) rather than `maui-pr` (def 302), do NOT undraft on +`maui-pr`-green alone — those pipelines do not auto-run on this PR (GITHUB_TOKEN cannot +trigger them), so a green `maui-pr` build is NOT evidence the gated build break is fixed; +record `skipped: build-only fix PR # targets gated pipeline () not run — +deferring to human` and stop this gate. Otherwise, set `TARGET := "the maui-pr build +(build-only fix — no single target test)"` and proceed to **T3** to mark ready. If any `maui-pr` build leg is still unconcluded, record `skipped: build-only fix PR +# not yet whole-build green (leg(s) pending)` and stop this gate WITHOUT marking +ready (a green subset is not enough — a still-pending build leg could yet fail). + +**T2 — Verify the target test's platform legs are green (anonymous, leg-level).** Per-test +outcomes come from the AzDO **test-results** API (`_apis/test/...`), which is **NOT reachable +anonymously — Hard-Rule 8**: those endpoints 302-redirect to a sign-in page on this runner, so +a `curl` returns an HTML redirect (not JSON) and every target test would look "not executed". +Validate instead at **leg granularity** using ONLY the anonymous `_apis/build/...` timeline +(Hard-Rule 8) — a `succeeded` category leg means every test in that category **that actually +ran** passed on that platform. This is a strong bar **only if the target test genuinely +executes**: a job can go green while one specific test is skipped at runtime (`[Ignore]`, +`Assert.Ignore()`, `Assert.Inconclusive()`, a `Skip=`/conditional `[Fact]`, an `#if`-out, or +an early `return` before the asserts). So before trusting a green leg as proof the target +passed, **confirm from the PR's OWN diff that the fix does NOT skip, ignore, disable, or +short-circuit the target test** (it adds no `[Ignore]`/`[Explicit]`, `Assert.Ignore`/ +`Assert.Inconclusive`, `Skip=`, category-exclusion, `#if`-out, or early `return` around the +target). If the fix could cause the target to be runtime-skipped rather than genuinely pass, +leg-level green is NOT sufficient evidence — record `skipped: target test may be +runtime-skipped by this fix (diff adds ); leg-green insufficient, deferring to human +on PR #` and stop this gate WITHOUT marking ready. Otherwise (the target genuinely runs +and asserts, as a de-flake or product fix does) a green category leg cannot hide a +target-test failure, so treat it as authoritative. + +Map the target test to the CI leg(s) that run it. A leg name encodes platform + test category +(e.g. `Android UITests SafeAreaEdges,Shadow`, `iOS UITests SafeAreaEdges,Shadow`, `macOS +UITests SafeAreaEdges,Shadow`, `Windows UITests SafeAreaEdges,Shadow`). Determine the target +test's UI-test category — its `[Category(UITestCategories.X)]` in the test/HostApp source, +visible in the PR diff or the test file — to know which leg-name substring identifies its legs; +for a device test, the per-platform device-test legs. + +Using the SAME build-discovery as Step 4 (filter AzDO builds by `branchName=refs/pull//merge` +or `sourceVersion == C.headSha`), read each build's **timeline** on `C.headSha` for the +pipeline(s) that RUN the target test — `maui-pr` (def 302) for unit/integration tests, +`maui-pr-uitests` (def 313) for Appium UI tests, `maui-pr-devicetests` (def 314) for device +tests: + +```bash +ORG=dnceng-public; PROJ=public; BUILD= +# Anonymous + Hard-Rule-8-compliant. NEVER call _apis/test/... (it 302-redirects to sign-in). +# Each type=="Job" record is one platform leg: result (succeeded/failed/canceled/null), +# state (completed/inProgress/pending), name (encodes " UITests "): +curl -s "https://dev.azure.com/$ORG/$PROJ/_apis/build/builds/$BUILD/timeline?api-version=7.1" \ + | tee /tmp/gh-aw/agent/timeline_${BUILD}.json \ + | jq -r '.records[] | select(.type=="Job") | "\(.result)\t\(.state)\t\(.name)"' +``` + +(The prefetch already exposes per-leg status in `C.failedLegs` / the checks context, and +`gh pr checks ` lists the same per-leg rows with platform+category in the name — use +whichever is handy; the build timeline is the authoritative cross-check on the exact build.) + +Treat a target test as **VALIDATED-GREEN** only if, on `C.headSha`, the leg that runs its +category is green on **every platform it runs on** — a fix that repairs one platform must not +silently regress the same test's leg on another, so a green leg on the originally-red platform +alone is NOT enough. Across the target pipeline's platform legs (for a UI test: the Android, +iOS, Windows, and macOS/MacCatalyst legs running the target's category; for a device test: each +device-test platform), require ALL of: +- the target's category leg is `result == "succeeded"` **and** `state == "completed"` on + **every** platform that runs it, AND +- that leg is `failed` / `canceled` / aborted on **NO** platform, AND +- **no platform is left unverified.** For each platform family the target pipeline covers, that + platform's category leg must have CONCLUDED (`state == "completed"`) on `C.headSha`. If a + platform simply has no leg for the target's category, the test does not run there — that is + fine, not a gap. But if a platform's category leg has **not concluded** (`state` is + `inProgress` / pending, or an `/azp`-gated `maui-pr-uitests` / `maui-pr-devicetests` leg that + has not been kicked), the test's status on that platform is UNKNOWN → the fix is NOT yet + validated across platforms: record `skipped: target test green on but + not yet verified on (leg(s) pending / need /azp run) on PR #` + and stop this gate WITHOUT marking ready. + +A target test whose category leg never concluded on ANY platform (**not executed** anywhere — +e.g. its `/azp`-gated pipeline has not been kicked) is likewise NOT validated: record `skipped: +target test not yet executed on PR # ( not run — needs /azp run)` and stop +this gate WITHOUT marking ready. Do NOT overclaim — a green *sibling* leg (a different category +on the same platform) is not the target's leg, and a green leg on one platform is not a pass on +the others. + +**T3 — Mark ready + report.** If EVERY target test is VALIDATED-GREEN. The 🎯 comment, +the mark-ready, and the `p/0` label are THREE SEPARATE safe-outputs — the comment +existing does NOT prove the mark-ready took effect, so they are tracked independently: + +- **Comment idempotency (dup-suppress only — never gates the mark-ready).** We only reach + T3 while `C.isDraft == true` (the precondition stops an already-ready PR before here). So + if a prior bot comment for THIS head that carries the leading `🎯` anchor AND the + contiguous phrase `validated green on ` already exists (BOTH T3 variants — + `🎯 Target test validated green on ` and the build-only `🎯 Build validated green on + ` — contain that exact contiguous phrase, so this recognizes both; keying on + `target test` alone would instead miss the build-only comment and re-post it every sweep. + **Require the `🎯` anchor** so the match can NEVER be loosened to a gapped + `validated…green…on` form that would false-positive on the Step 3 `✅ … validated — the + fix's CI is green on ` surface comment and wrongly suppress the audit 🎯), the comment + landed on an earlier sweep but the **mark-ready did NOT take + effect** (the PR is still a draft) — set `SUPPRESS_COMMENT = true` (do not re-post the + duplicate 🎯 comment) but STILL complete the mark-ready + label below. Never treat the + comment's existence as "already marked ready" while the PR is still a draft. Record this + case as `re-marking PR # (🎯 present; mark-ready did not take on a prior sweep)`. +- **Atomicity budget pre-check (Hard-Rule 7).** Determine the outputs this gate will emit: + `mark_pull_request_as_ready_for_review` + `add_labels` always, plus `add_comment` unless + `SUPPRESS_COMMENT`. Verify a free per-run slot remains in EACH bucket you are about to + use. If ANY required bucket is exhausted, emit NONE of them — record `skipped: per-run + cap reached; deferring mark-ready PR # to next cycle` and stop this gate. Never mark a + PR ready (or label it) unless its 🎯 audit comment is GUARANTEED to exist for this exact + head SHA — either posted in this same sweep, or (in the `SUPPRESS_COMMENT` re-marking case) + already present from a prior sweep. The outputs you DO emit either all land or all defer + together; the audit trail must never be absent, but it is NOT re-posted when it already exists. +- **Dry-run gate (Step 0):** if `dry_run == "true"`, emit NOTHING — print the intended + readiness comment and "would mark ready + add p/0" to the run log, tally `dry-run: + would-mark-ready PR #`, and stop. +- Otherwise emit for THIS PR number ``: + 1. `add_comment` (ONLY if not `SUPPRESS_COMMENT`): `🎯 Target test validated green on + — passed on ALL platforms it runs on (, + buildId ). — not + caused by this fix."> Transitioning this PR from draft to ready for review and adding + `p/0`; a maintainer still reviews and merges.` (For a **build-only fix** — the T1 + whole-build fallback, no single target test — phrase the first clause as `🎯 Build + validated green on — the maui-pr build passed on ALL platforms (buildId + ).` instead of naming a test.) + 2. `mark_pull_request_as_ready_for_review` with `reason:` a one-line justification + naming the validated test(s) and ``. + 3. `add_labels` with `labels: ["p/0"]` for PR # — put the now-review-ready fix into + the team's p/0 priority queue so it is triaged, not lost in the draft backlog. (If + the PR somehow already carries `p/0`, this is a harmless no-op.) +- Record `marked-ready PR # (target green on ALL platforms on , + labeled p/0)` and stop. + #### Step 3.5.R — Maintainer change-request response (Track C) Reached from Step 3.5 gate 0. Goal: when an **eligible human reviewer** (Hard-Rule @@ -1395,7 +1681,19 @@ Filed by [`ci-status-fix-net11`](https://github.com/dotnet/maui/blob/main/.githu ### Step 8 — Per-issue tally + end-of-run summary -Per issue, append one outcome line to `/tmp/gh-aw/agent/coverage.txt`: +Per issue, append **one** outcome line to `/tmp/gh-aw/agent/coverage.txt` — the +terminal outcome for the cycle. When one cycle produces a chained pair — a PR gets a +same-cycle precursor line and **then** a Step 3.6 readiness line (`marked-ready` on the +draft→ready flip, or `already-ready` on a later steady-state sweep of an already-ready +PR) — record ONLY the terminal Step 3.6 readiness line: it supersedes ANY same-cycle +non-readiness precursor for that PR, so an aggregator keying on one-line-per-issue never +double-counts. The precursor may be a Step 3 green line (`surfaced-green` on the first ✅, +or `already-surfaced` when it already exists) OR — when an unrelated-flake red and a +target-green coincide — a Step 4 `annotated-flake` line; either way the readiness line is +terminal, and the superseded precursor's signal survives in the PR's 🎯/♻️ comment so no +information is lost. This covers the flip cycle (`surfaced-green` → `marked-ready`), the +post-flip steady state (`already-surfaced` → `already-ready`), and the flake-coincident +flip (`annotated-flake` → `marked-ready`): ``` # net11.0 attempt- @@ -1406,8 +1704,14 @@ Per issue, append one outcome line to `/tmp/gh-aw/agent/coverage.txt`: `advance-PR # attempt /10` (ADVANCE mode — new commit pushed to the existing PR), `surfaced-green PR #` (fix's own CI went green; commented for review, did not advance), `annotated-flake PR #` (red was unrelated flake; -commented, attempt NOT burned), `waiting PR #` (CI not settled yet), -`dry-run: would-`, `skipped: `. (The +commented, attempt NOT burned), `marked-ready PR #` (the specific fixed test +was confirmed green in the PR's own CI; draft flipped to ready for review), +`already-surfaced PR #` (the fix's ✅ green comment already existed this sweep; +re-post suppressed — superseded by the Step 3.6 readiness line when the PR also flips +ready that cycle), `already-ready PR #` (terminal steady state — the PR was flipped +ready on a prior cycle and remains green on an unchanged head; no action taken, +supersedes `already-surfaced`), `waiting PR #` (CI not settled yet), +`dry-run: would-`, `skipped: `. (The `needs-human-PR` outcome is reserved for the deferred hand-off — Step 6 currently records a skip instead, so it is not emitted.) diff --git a/.github/workflows/ci-status-fix.lock.yml b/.github/workflows/ci-status-fix.lock.yml index 93dc5969cb0d..65511f2b3259 100644 --- a/.github/workflows/ci-status-fix.lock.yml +++ b/.github/workflows/ci-status-fix.lock.yml @@ -1,4 +1,4 @@ -# gh-aw-metadata: {"schema_version":"v4","frontmatter_hash":"f014161967589ebcaf1ac5ec6f3c88ee5a4388d28b55a07dc36c45b7b2aebab6","body_hash":"86c6b2d4c3df13da14c6a3c8b211381fcc9f97bb1951ff7b80588a2c5338e00c","compiler_version":"v0.80.9","strict":true,"agent_id":"copilot","agent_model":"claude-opus-4.8","engine_versions":{"copilot":"1.0.63"}} +# gh-aw-metadata: {"schema_version":"v4","frontmatter_hash":"b0ebf8a2f179900cfe3638e994b27a43cec52bdbdd2331dc1bd4d65762fa2d6b","body_hash":"1825e2b1f7c7bd88241bf37580655489360f9bebc806edd00837a52ce4ad83a0","compiler_version":"v0.80.9","strict":true,"agent_id":"copilot","agent_model":"claude-opus-4.8","engine_versions":{"copilot":"1.0.63"}} # gh-aw-manifest: {"version":1,"secrets":["COPILOT_PAT_0","COPILOT_PAT_1","COPILOT_PAT_2","COPILOT_PAT_3","COPILOT_PAT_4","COPILOT_PAT_5","COPILOT_PAT_6","COPILOT_PAT_7","COPILOT_PAT_8","COPILOT_PAT_9","GH_AW_CI_TRIGGER_TOKEN","GH_AW_GITHUB_MCP_SERVER_TOKEN","GH_AW_GITHUB_TOKEN","GITHUB_TOKEN"],"actions":[{"repo":"actions/checkout","sha":"34e114876b0b11c390a56381ad16ebd13914f8d5","version":"v4"},{"repo":"actions/checkout","sha":"9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0","version":"v7.0.0"},{"repo":"actions/checkout","sha":"de0fac2e4500dabe0009e67214ff5f5447ce83dd","version":"v6.0.2"},{"repo":"actions/download-artifact","sha":"3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c","version":"v8.0.1"},{"repo":"actions/github-script","sha":"3a2844b7e9c422d3c10d287c895573f7108da1b3","version":"v9.0.0"},{"repo":"actions/setup-node","sha":"48b55a011bda9f5d6aeb4c2d9c7362e8dae4041e","version":"v6.4.0"},{"repo":"actions/upload-artifact","sha":"043fb46d1a93c77aae656e7c1c64a875d1fc6a0a","version":"v7.0.1"},{"repo":"github/gh-aw-actions/setup","sha":"8c7d04ebf1ece56cd381446125da3e0f6896294a","version":"v0.80.9"}],"containers":[{"image":"ghcr.io/github/gh-aw-firewall/agent:0.27.7","digest":"sha256:aae231e4635c8999d039c132f1602d3df850fe9b84a00aa2b5ac981179b5661c","pinned_image":"ghcr.io/github/gh-aw-firewall/agent:0.27.7@sha256:aae231e4635c8999d039c132f1602d3df850fe9b84a00aa2b5ac981179b5661c"},{"image":"ghcr.io/github/gh-aw-firewall/api-proxy:0.27.7","digest":"sha256:009caf2e3d88fa77b64e9a03a95a228fc58db0f1701c6d324b29ba5a3c7c79b6","pinned_image":"ghcr.io/github/gh-aw-firewall/api-proxy:0.27.7@sha256:009caf2e3d88fa77b64e9a03a95a228fc58db0f1701c6d324b29ba5a3c7c79b6"},{"image":"ghcr.io/github/gh-aw-firewall/squid:0.27.7","digest":"sha256:deb1d4e19de62d51cee0508057a596a19315c3423ada4d675cad136dc8037c96","pinned_image":"ghcr.io/github/gh-aw-firewall/squid:0.27.7@sha256:deb1d4e19de62d51cee0508057a596a19315c3423ada4d675cad136dc8037c96"},{"image":"ghcr.io/github/gh-aw-mcpg:v0.3.27","digest":"sha256:fe984bddde4ec05d756d9043edb0a32912e6b7b72f6a121b1082f29221421cc7","pinned_image":"ghcr.io/github/gh-aw-mcpg:v0.3.27@sha256:fe984bddde4ec05d756d9043edb0a32912e6b7b72f6a121b1082f29221421cc7"},{"image":"ghcr.io/github/gh-aw-node","digest":"sha256:529d02eb970b1161aa25c593a9c3df57fdfad5a8add328cb3b6eccef66f3183b","pinned_image":"ghcr.io/github/gh-aw-node@sha256:529d02eb970b1161aa25c593a9c3df57fdfad5a8add328cb3b6eccef66f3183b"},{"image":"ghcr.io/github/github-mcp-server:v1.4.0","digest":"sha256:2afb26356481d1a350e14544a6e160f7f7ec1561a1ea309b823665abf0309036","pinned_image":"ghcr.io/github/github-mcp-server:v1.4.0@sha256:2afb26356481d1a350e14544a6e160f7f7ec1561a1ea309b823665abf0309036"}]} # This file was automatically generated by gh-aw (v0.80.9). DO NOT EDIT. To debug this workflow, load the skill at https://github.com/github/gh-aw/blob/main/debug.md # @@ -37,7 +37,9 @@ # humans (the open tracking issue is the hand-off surface; a dedicated # [ci-fix][needs-human] PR is planned but currently deferred — see Step 6). CI on # the PR is kicked by a human `/azp run` for now; the loop watches, classifies, -# and re-fixes autonomously between kicks. +# and re-fixes autonomously between kicks. When the SPECIFIC test a PR fixed is +# confirmed green in that PR's own CI, the loop marks the draft PR ready for review +# (a state transition only — it never approves or merges). # Never mutes tests, but # de-flakes genuinely flaky ones (deterministic synchronization, no retries / # timeout bumps). Always skips visual-regression / screenshot issues. @@ -273,24 +275,24 @@ jobs: run: | bash "${RUNNER_TEMP}/gh-aw/actions/create_prompt_first.sh" { - cat << 'GH_AW_PROMPT_25cf75111b670167_EOF' + cat << 'GH_AW_PROMPT_2ac1d0b4083428f8_EOF' - GH_AW_PROMPT_25cf75111b670167_EOF + GH_AW_PROMPT_2ac1d0b4083428f8_EOF cat "${RUNNER_TEMP}/gh-aw/prompts/xpia.md" cat "${RUNNER_TEMP}/gh-aw/prompts/temp_folder_prompt.md" cat "${RUNNER_TEMP}/gh-aw/prompts/markdown.md" cat "${RUNNER_TEMP}/gh-aw/prompts/safe_outputs_prompt.md" - cat << 'GH_AW_PROMPT_25cf75111b670167_EOF' + cat << 'GH_AW_PROMPT_2ac1d0b4083428f8_EOF' - Tools: add_comment(max:3), create_pull_request(max:3), update_pull_request(max:3), push_to_pull_request_branch(max:3), missing_tool, missing_data, noop - GH_AW_PROMPT_25cf75111b670167_EOF + Tools: add_comment(max:6), create_pull_request(max:3), update_pull_request(max:3), mark_pull_request_as_ready_for_review(max:3), add_labels(max:3), push_to_pull_request_branch(max:3), missing_tool, missing_data, noop + GH_AW_PROMPT_2ac1d0b4083428f8_EOF cat "${RUNNER_TEMP}/gh-aw/prompts/safe_outputs_create_pull_request.md" cat "${RUNNER_TEMP}/gh-aw/prompts/safe_outputs_push_to_pr_branch.md" - cat << 'GH_AW_PROMPT_25cf75111b670167_EOF' + cat << 'GH_AW_PROMPT_2ac1d0b4083428f8_EOF' - GH_AW_PROMPT_25cf75111b670167_EOF + GH_AW_PROMPT_2ac1d0b4083428f8_EOF cat "${RUNNER_TEMP}/gh-aw/prompts/mcp_cli_tools_prompt.md" - cat << 'GH_AW_PROMPT_25cf75111b670167_EOF' + cat << 'GH_AW_PROMPT_2ac1d0b4083428f8_EOF' The following GitHub context information is available for this workflow: {{#if github.actor}} @@ -332,12 +334,12 @@ jobs: stop immediately and report the limitation rather than spending turns trying to work around it. - GH_AW_PROMPT_25cf75111b670167_EOF + GH_AW_PROMPT_2ac1d0b4083428f8_EOF cat "${RUNNER_TEMP}/gh-aw/prompts/github_mcp_tools_with_safeoutputs_prompt.md" - cat << 'GH_AW_PROMPT_25cf75111b670167_EOF' + cat << 'GH_AW_PROMPT_2ac1d0b4083428f8_EOF' {{#runtime-import .github/workflows/ci-status-fix.md}} - GH_AW_PROMPT_25cf75111b670167_EOF + GH_AW_PROMPT_2ac1d0b4083428f8_EOF } > "$GH_AW_PROMPT" - name: Interpolate variables and render templates uses: actions/github-script@3a2844b7e9c422d3c10d287c895573f7108da1b3 # v9.0.0 @@ -554,16 +556,18 @@ jobs: mkdir -p "${RUNNER_TEMP}/gh-aw/safeoutputs" mkdir -p /tmp/gh-aw/safeoutputs mkdir -p /tmp/gh-aw/mcp-logs/safeoutputs - cat > "${RUNNER_TEMP}/gh-aw/safeoutputs/config.json" << 'GH_AW_SAFE_OUTPUTS_CONFIG_fa2a69fad5fd0485_EOF' - {"add_comment":{"discussions":false,"max":3,"required_labels":["agentic-workflows"],"required_title_prefix":"[ci-fix] ","target":"*"},"create_pull_request":{"allowed_base_branches":["main"],"allowed_branches":["ci-fix/**"],"allowed_files":["src/Core/**","src/Controls/**","src/Essentials/**","src/BlazorWebView/**","src/TestUtils/**","src/Templates/**","**/PublicAPI.Unshipped.txt"],"base_branch":"main","draft":true,"labels":["agentic-workflows"],"max":3,"max_patch_files":100,"max_patch_size":4096,"protect_top_level_dot_folders":true,"protected_files":["package.json","bun.lockb","bunfig.toml","deno.json","deno.jsonc","deno.lock","global.json","NuGet.Config","Directory.Packages.props","mix.exs","mix.lock","go.mod","go.sum","stack.yaml","stack.yaml.lock","pom.xml","build.gradle","build.gradle.kts","settings.gradle","settings.gradle.kts","gradle.properties","package-lock.json","yarn.lock","pnpm-lock.yaml","npm-shrinkwrap.json","requirements.txt","Pipfile","Pipfile.lock","pyproject.toml","setup.py","setup.cfg","Gemfile","Gemfile.lock","uv.lock","CODEOWNERS","DESIGN.md","README.md","CONTRIBUTING.md","CHANGELOG.md","SECURITY.md","CODE_OF_CONDUCT.md","AGENTS.md","CLAUDE.md","GEMINI.md"],"protected_files_policy":"request_review","title_prefix":"[ci-fix] "},"create_report_incomplete_issue":{},"missing_data":{},"missing_tool":{},"noop":{"max":1,"report-as-issue":"true"},"push_to_pull_request_branch":{"allowed_files":["src/Core/**","src/Controls/**","src/Essentials/**","src/BlazorWebView/**","src/TestUtils/**","src/Templates/**","**/PublicAPI.Unshipped.txt"],"if_no_changes":"warn","max":3,"max_patch_size":4096,"protect_top_level_dot_folders":true,"protected_files":["package.json","bun.lockb","bunfig.toml","deno.json","deno.jsonc","deno.lock","global.json","NuGet.Config","Directory.Packages.props","mix.exs","mix.lock","go.mod","go.sum","stack.yaml","stack.yaml.lock","pom.xml","build.gradle","build.gradle.kts","settings.gradle","settings.gradle.kts","gradle.properties","package-lock.json","yarn.lock","pnpm-lock.yaml","npm-shrinkwrap.json","requirements.txt","Pipfile","Pipfile.lock","pyproject.toml","setup.py","setup.cfg","Gemfile","Gemfile.lock","uv.lock","CODEOWNERS","DESIGN.md","README.md","CONTRIBUTING.md","CHANGELOG.md","SECURITY.md","CODE_OF_CONDUCT.md","AGENTS.md","CLAUDE.md","GEMINI.md"],"required_labels":["agentic-workflows"],"target":"*","title_prefix":"[ci-fix] "},"report_incomplete":{},"update_pull_request":{"allow_body":true,"allow_title":false,"max":3,"target":"*","update_branch":false}} - GH_AW_SAFE_OUTPUTS_CONFIG_fa2a69fad5fd0485_EOF + cat > "${RUNNER_TEMP}/gh-aw/safeoutputs/config.json" << 'GH_AW_SAFE_OUTPUTS_CONFIG_0644bf1434ca0071_EOF' + {"add_comment":{"discussions":false,"max":6,"required_labels":["agentic-workflows"],"required_title_prefix":"[ci-fix] ","target":"*"},"add_labels":{"allowed":["p/0"],"max":3,"required_labels":["agentic-workflows"],"required_title_prefix":"[ci-fix] ","target":"*"},"create_pull_request":{"allowed_base_branches":["main"],"allowed_branches":["ci-fix/**"],"allowed_files":["src/Core/**","src/Controls/**","src/Essentials/**","src/BlazorWebView/**","src/TestUtils/**","src/Templates/**","**/PublicAPI.Unshipped.txt"],"base_branch":"main","draft":true,"labels":["agentic-workflows"],"max":3,"max_patch_files":100,"max_patch_size":4096,"protect_top_level_dot_folders":true,"protected_files":["package.json","bun.lockb","bunfig.toml","deno.json","deno.jsonc","deno.lock","global.json","NuGet.Config","Directory.Packages.props","mix.exs","mix.lock","go.mod","go.sum","stack.yaml","stack.yaml.lock","pom.xml","build.gradle","build.gradle.kts","settings.gradle","settings.gradle.kts","gradle.properties","package-lock.json","yarn.lock","pnpm-lock.yaml","npm-shrinkwrap.json","requirements.txt","Pipfile","Pipfile.lock","pyproject.toml","setup.py","setup.cfg","Gemfile","Gemfile.lock","uv.lock","CODEOWNERS","DESIGN.md","README.md","CONTRIBUTING.md","CHANGELOG.md","SECURITY.md","CODE_OF_CONDUCT.md","AGENTS.md","CLAUDE.md","GEMINI.md"],"protected_files_policy":"request_review","title_prefix":"[ci-fix] "},"create_report_incomplete_issue":{},"mark_pull_request_as_ready_for_review":{"max":3,"required_labels":["agentic-workflows"],"required_title_prefix":"[ci-fix] ","target":"*"},"missing_data":{},"missing_tool":{},"noop":{"max":1,"report-as-issue":"true"},"push_to_pull_request_branch":{"allowed_files":["src/Core/**","src/Controls/**","src/Essentials/**","src/BlazorWebView/**","src/TestUtils/**","src/Templates/**","**/PublicAPI.Unshipped.txt"],"if_no_changes":"warn","max":3,"max_patch_size":4096,"protect_top_level_dot_folders":true,"protected_files":["package.json","bun.lockb","bunfig.toml","deno.json","deno.jsonc","deno.lock","global.json","NuGet.Config","Directory.Packages.props","mix.exs","mix.lock","go.mod","go.sum","stack.yaml","stack.yaml.lock","pom.xml","build.gradle","build.gradle.kts","settings.gradle","settings.gradle.kts","gradle.properties","package-lock.json","yarn.lock","pnpm-lock.yaml","npm-shrinkwrap.json","requirements.txt","Pipfile","Pipfile.lock","pyproject.toml","setup.py","setup.cfg","Gemfile","Gemfile.lock","uv.lock","CODEOWNERS","DESIGN.md","README.md","CONTRIBUTING.md","CHANGELOG.md","SECURITY.md","CODE_OF_CONDUCT.md","AGENTS.md","CLAUDE.md","GEMINI.md"],"required_labels":["agentic-workflows"],"target":"*","title_prefix":"[ci-fix] "},"report_incomplete":{},"update_pull_request":{"allow_body":true,"allow_title":false,"max":3,"target":"*","update_branch":false}} + GH_AW_SAFE_OUTPUTS_CONFIG_0644bf1434ca0071_EOF - name: Generate Safe Outputs Tools env: GH_AW_TOOLS_META_JSON: | { "description_suffixes": { - "add_comment": " CONSTRAINTS: Maximum 3 comment(s) can be added. Target: *. Supports reply_to_id for discussion threading.", + "add_comment": " CONSTRAINTS: Maximum 6 comment(s) can be added. Target: *. Supports reply_to_id for discussion threading.", + "add_labels": " CONSTRAINTS: Maximum 3 label(s) can be added. Only these labels are allowed: [\"p/0\"]. Target: *.", "create_pull_request": " CONSTRAINTS: Maximum 3 pull request(s) can be created. Title will be prefixed with \"[ci-fix] \". Labels [\"agentic-workflows\"] will be automatically added. Only these labels are allowed: [\"agentic-workflows\"]. PRs will be created as drafts.", + "mark_pull_request_as_ready_for_review": " CONSTRAINTS: Maximum 3 pull request(s) can be marked as ready for review.", "push_to_pull_request_branch": " CONSTRAINTS: Maximum 3 push(es) can be made. The target pull request title must start with \"[ci-fix] \".", "update_pull_request": " CONSTRAINTS: Maximum 3 pull request(s) can be updated. Target: *." }, @@ -594,6 +598,25 @@ jobs: } } }, + "add_labels": { + "defaultMax": 5, + "fields": { + "item_number": { + "issueNumberOrTemporaryId": true + }, + "labels": { + "required": true, + "type": "array", + "itemType": "string", + "itemSanitize": true, + "itemMaxLength": 128 + }, + "repo": { + "type": "string", + "maxLength": 256 + } + } + }, "create_pull_request": { "defaultMax": 1, "fields": { @@ -635,6 +658,24 @@ jobs: } } }, + "mark_pull_request_as_ready_for_review": { + "defaultMax": 1, + "fields": { + "pull_request_number": { + "issueOrPRNumber": true + }, + "reason": { + "required": true, + "type": "string", + "sanitize": true, + "maxLength": 65000 + }, + "repo": { + "type": "string", + "maxLength": 256 + } + } + }, "missing_data": { "defaultMax": 20, "fields": { @@ -1494,7 +1535,7 @@ jobs: uses: actions/github-script@3a2844b7e9c422d3c10d287c895573f7108da1b3 # v9.0.0 env: WORKFLOW_NAME: "CI Failure Fixer (main)" - WORKFLOW_DESCRIPTION: "Periodic pass over open ci-scan tracking issues filed by the main-branch CI\nfailure scanner (.github/workflows/ci-status-main.md). This workflow targets\nthe `main` branch EXCLUSIVELY: it processes only issues labelled ci-scan and\nopens every PR against main. (The net11.0 branch is handled by the parallel\n.github/workflows/ci-status-fix-net11.md — the two are split because gh-aw can\nonly transport a fix relative to ONE static base branch per workflow, and the\nmain↔net11.0 divergence exceeds gh-aw's 10 MB transport-patch cap.) The fixer\nopens ONE draft [ci-fix] PR per actionable issue against main, then WATCHES\nthat PR's own CI on later runs: when the fix's CI comes back red and the red is\ncaused by the fix itself, it pushes a fresh follow-up fix onto the SAME PR\nbranch (never a second PR) — up to 10 attempts — then stops and defers to\nhumans (the open tracking issue is the hand-off surface; a dedicated\n[ci-fix][needs-human] PR is planned but currently deferred — see Step 6). CI on\nthe PR is kicked by a human `/azp run` for now; the loop watches, classifies,\nand re-fixes autonomously between kicks.\nNever mutes tests, but\nde-flakes genuinely flaky ones (deterministic synchronization, no retries /\ntimeout bumps). Always skips visual-regression / screenshot issues." + WORKFLOW_DESCRIPTION: "Periodic pass over open ci-scan tracking issues filed by the main-branch CI\nfailure scanner (.github/workflows/ci-status-main.md). This workflow targets\nthe `main` branch EXCLUSIVELY: it processes only issues labelled ci-scan and\nopens every PR against main. (The net11.0 branch is handled by the parallel\n.github/workflows/ci-status-fix-net11.md — the two are split because gh-aw can\nonly transport a fix relative to ONE static base branch per workflow, and the\nmain↔net11.0 divergence exceeds gh-aw's 10 MB transport-patch cap.) The fixer\nopens ONE draft [ci-fix] PR per actionable issue against main, then WATCHES\nthat PR's own CI on later runs: when the fix's CI comes back red and the red is\ncaused by the fix itself, it pushes a fresh follow-up fix onto the SAME PR\nbranch (never a second PR) — up to 10 attempts — then stops and defers to\nhumans (the open tracking issue is the hand-off surface; a dedicated\n[ci-fix][needs-human] PR is planned but currently deferred — see Step 6). CI on\nthe PR is kicked by a human `/azp run` for now; the loop watches, classifies,\nand re-fixes autonomously between kicks. When the SPECIFIC test a PR fixed is\nconfirmed green in that PR's own CI, the loop marks the draft PR ready for review\n(a state transition only — it never approves or merges).\nNever mutes tests, but\nde-flakes genuinely flaky ones (deterministic synchronization, no retries /\ntimeout bumps). Always skips visual-regression / screenshot issues." HAS_PATCH: ${{ needs.agent.outputs.has_patch }} with: script: | @@ -1910,7 +1951,7 @@ jobs: GH_AW_ALLOWED_DOMAINS: "*.blob.core.windows.net,*.githubusercontent.com,api.business.githubcopilot.com,api.enterprise.githubcopilot.com,api.github.com,api.githubcopilot.com,api.individual.githubcopilot.com,api.snapcraft.io,archive.ubuntu.com,azure.archive.ubuntu.com,codeload.github.com,crl.geotrust.com,crl.globalsign.com,crl.identrust.com,crl.sectigo.com,crl.thawte.com,crl.usertrust.com,crl.verisign.com,crl3.digicert.com,crl4.digicert.com,crls.ssl.com,dev.azure.com,docs.github.com,github-cloud.githubusercontent.com,github-cloud.s3.amazonaws.com,github.blog,github.com,github.githubassets.com,helix.dot.net,host.docker.internal,json-schema.org,json.schemastore.org,keyserver.ubuntu.com,lfs.github.com,objects.githubusercontent.com,ocsp.digicert.com,ocsp.geotrust.com,ocsp.globalsign.com,ocsp.identrust.com,ocsp.sectigo.com,ocsp.ssl.com,ocsp.thawte.com,ocsp.usertrust.com,ocsp.verisign.com,packagecloud.io,packages.cloud.google.com,packages.microsoft.com,patch-diff.githubusercontent.com,ppa.launchpad.net,raw.githubusercontent.com,registry.npmjs.org,s.symcb.com,s.symcd.com,security.ubuntu.com,telemetry.enterprise.githubcopilot.com,ts-crl.ws.symantec.com,ts-ocsp.ws.symantec.com,www.googleapis.com" GITHUB_SERVER_URL: ${{ github.server_url }} GITHUB_API_URL: ${{ github.api_url }} - GH_AW_SAFE_OUTPUTS_HANDLER_CONFIG: "{\"add_comment\":{\"discussions\":false,\"max\":3,\"required_labels\":[\"agentic-workflows\"],\"required_title_prefix\":\"[ci-fix] \",\"target\":\"*\"},\"create_pull_request\":{\"allowed_base_branches\":[\"main\"],\"allowed_branches\":[\"ci-fix/**\"],\"allowed_files\":[\"src/Core/**\",\"src/Controls/**\",\"src/Essentials/**\",\"src/BlazorWebView/**\",\"src/TestUtils/**\",\"src/Templates/**\",\"**/PublicAPI.Unshipped.txt\"],\"base_branch\":\"main\",\"draft\":true,\"labels\":[\"agentic-workflows\"],\"max\":3,\"max_patch_files\":100,\"max_patch_size\":4096,\"protect_top_level_dot_folders\":true,\"protected_files\":[\"package.json\",\"bun.lockb\",\"bunfig.toml\",\"deno.json\",\"deno.jsonc\",\"deno.lock\",\"global.json\",\"NuGet.Config\",\"Directory.Packages.props\",\"mix.exs\",\"mix.lock\",\"go.mod\",\"go.sum\",\"stack.yaml\",\"stack.yaml.lock\",\"pom.xml\",\"build.gradle\",\"build.gradle.kts\",\"settings.gradle\",\"settings.gradle.kts\",\"gradle.properties\",\"package-lock.json\",\"yarn.lock\",\"pnpm-lock.yaml\",\"npm-shrinkwrap.json\",\"requirements.txt\",\"Pipfile\",\"Pipfile.lock\",\"pyproject.toml\",\"setup.py\",\"setup.cfg\",\"Gemfile\",\"Gemfile.lock\",\"uv.lock\",\"CODEOWNERS\",\"DESIGN.md\",\"README.md\",\"CONTRIBUTING.md\",\"CHANGELOG.md\",\"SECURITY.md\",\"CODE_OF_CONDUCT.md\",\"AGENTS.md\",\"CLAUDE.md\",\"GEMINI.md\"],\"protected_files_policy\":\"request_review\",\"title_prefix\":\"[ci-fix] \"},\"create_report_incomplete_issue\":{},\"missing_data\":{},\"missing_tool\":{},\"noop\":{\"max\":1,\"report-as-issue\":\"true\"},\"push_to_pull_request_branch\":{\"allowed_files\":[\"src/Core/**\",\"src/Controls/**\",\"src/Essentials/**\",\"src/BlazorWebView/**\",\"src/TestUtils/**\",\"src/Templates/**\",\"**/PublicAPI.Unshipped.txt\"],\"if_no_changes\":\"warn\",\"max\":3,\"max_patch_size\":4096,\"protect_top_level_dot_folders\":true,\"protected_files\":[\"package.json\",\"bun.lockb\",\"bunfig.toml\",\"deno.json\",\"deno.jsonc\",\"deno.lock\",\"global.json\",\"NuGet.Config\",\"Directory.Packages.props\",\"mix.exs\",\"mix.lock\",\"go.mod\",\"go.sum\",\"stack.yaml\",\"stack.yaml.lock\",\"pom.xml\",\"build.gradle\",\"build.gradle.kts\",\"settings.gradle\",\"settings.gradle.kts\",\"gradle.properties\",\"package-lock.json\",\"yarn.lock\",\"pnpm-lock.yaml\",\"npm-shrinkwrap.json\",\"requirements.txt\",\"Pipfile\",\"Pipfile.lock\",\"pyproject.toml\",\"setup.py\",\"setup.cfg\",\"Gemfile\",\"Gemfile.lock\",\"uv.lock\",\"CODEOWNERS\",\"DESIGN.md\",\"README.md\",\"CONTRIBUTING.md\",\"CHANGELOG.md\",\"SECURITY.md\",\"CODE_OF_CONDUCT.md\",\"AGENTS.md\",\"CLAUDE.md\",\"GEMINI.md\"],\"required_labels\":[\"agentic-workflows\"],\"target\":\"*\",\"title_prefix\":\"[ci-fix] \"},\"report_incomplete\":{},\"update_pull_request\":{\"allow_body\":true,\"allow_title\":false,\"max\":3,\"target\":\"*\",\"update_branch\":false}}" + GH_AW_SAFE_OUTPUTS_HANDLER_CONFIG: "{\"add_comment\":{\"discussions\":false,\"max\":6,\"required_labels\":[\"agentic-workflows\"],\"required_title_prefix\":\"[ci-fix] \",\"target\":\"*\"},\"add_labels\":{\"allowed\":[\"p/0\"],\"max\":3,\"required_labels\":[\"agentic-workflows\"],\"required_title_prefix\":\"[ci-fix] \",\"target\":\"*\"},\"create_pull_request\":{\"allowed_base_branches\":[\"main\"],\"allowed_branches\":[\"ci-fix/**\"],\"allowed_files\":[\"src/Core/**\",\"src/Controls/**\",\"src/Essentials/**\",\"src/BlazorWebView/**\",\"src/TestUtils/**\",\"src/Templates/**\",\"**/PublicAPI.Unshipped.txt\"],\"base_branch\":\"main\",\"draft\":true,\"labels\":[\"agentic-workflows\"],\"max\":3,\"max_patch_files\":100,\"max_patch_size\":4096,\"protect_top_level_dot_folders\":true,\"protected_files\":[\"package.json\",\"bun.lockb\",\"bunfig.toml\",\"deno.json\",\"deno.jsonc\",\"deno.lock\",\"global.json\",\"NuGet.Config\",\"Directory.Packages.props\",\"mix.exs\",\"mix.lock\",\"go.mod\",\"go.sum\",\"stack.yaml\",\"stack.yaml.lock\",\"pom.xml\",\"build.gradle\",\"build.gradle.kts\",\"settings.gradle\",\"settings.gradle.kts\",\"gradle.properties\",\"package-lock.json\",\"yarn.lock\",\"pnpm-lock.yaml\",\"npm-shrinkwrap.json\",\"requirements.txt\",\"Pipfile\",\"Pipfile.lock\",\"pyproject.toml\",\"setup.py\",\"setup.cfg\",\"Gemfile\",\"Gemfile.lock\",\"uv.lock\",\"CODEOWNERS\",\"DESIGN.md\",\"README.md\",\"CONTRIBUTING.md\",\"CHANGELOG.md\",\"SECURITY.md\",\"CODE_OF_CONDUCT.md\",\"AGENTS.md\",\"CLAUDE.md\",\"GEMINI.md\"],\"protected_files_policy\":\"request_review\",\"title_prefix\":\"[ci-fix] \"},\"create_report_incomplete_issue\":{},\"mark_pull_request_as_ready_for_review\":{\"max\":3,\"required_labels\":[\"agentic-workflows\"],\"required_title_prefix\":\"[ci-fix] \",\"target\":\"*\"},\"missing_data\":{},\"missing_tool\":{},\"noop\":{\"max\":1,\"report-as-issue\":\"true\"},\"push_to_pull_request_branch\":{\"allowed_files\":[\"src/Core/**\",\"src/Controls/**\",\"src/Essentials/**\",\"src/BlazorWebView/**\",\"src/TestUtils/**\",\"src/Templates/**\",\"**/PublicAPI.Unshipped.txt\"],\"if_no_changes\":\"warn\",\"max\":3,\"max_patch_size\":4096,\"protect_top_level_dot_folders\":true,\"protected_files\":[\"package.json\",\"bun.lockb\",\"bunfig.toml\",\"deno.json\",\"deno.jsonc\",\"deno.lock\",\"global.json\",\"NuGet.Config\",\"Directory.Packages.props\",\"mix.exs\",\"mix.lock\",\"go.mod\",\"go.sum\",\"stack.yaml\",\"stack.yaml.lock\",\"pom.xml\",\"build.gradle\",\"build.gradle.kts\",\"settings.gradle\",\"settings.gradle.kts\",\"gradle.properties\",\"package-lock.json\",\"yarn.lock\",\"pnpm-lock.yaml\",\"npm-shrinkwrap.json\",\"requirements.txt\",\"Pipfile\",\"Pipfile.lock\",\"pyproject.toml\",\"setup.py\",\"setup.cfg\",\"Gemfile\",\"Gemfile.lock\",\"uv.lock\",\"CODEOWNERS\",\"DESIGN.md\",\"README.md\",\"CONTRIBUTING.md\",\"CHANGELOG.md\",\"SECURITY.md\",\"CODE_OF_CONDUCT.md\",\"AGENTS.md\",\"CLAUDE.md\",\"GEMINI.md\"],\"required_labels\":[\"agentic-workflows\"],\"target\":\"*\",\"title_prefix\":\"[ci-fix] \"},\"report_incomplete\":{},\"update_pull_request\":{\"allow_body\":true,\"allow_title\":false,\"max\":3,\"target\":\"*\",\"update_branch\":false}}" GH_AW_CI_TRIGGER_TOKEN: ${{ secrets.GH_AW_CI_TRIGGER_TOKEN }} with: github-token: ${{ secrets.GH_AW_GITHUB_TOKEN || secrets.GITHUB_TOKEN }} diff --git a/.github/workflows/ci-status-fix.md b/.github/workflows/ci-status-fix.md index d3227e25170c..9c2f46054029 100644 --- a/.github/workflows/ci-status-fix.md +++ b/.github/workflows/ci-status-fix.md @@ -15,7 +15,9 @@ description: | humans (the open tracking issue is the hand-off surface; a dedicated [ci-fix][needs-human] PR is planned but currently deferred — see Step 6). CI on the PR is kicked by a human `/azp run` for now; the loop watches, classifies, - and re-fixes autonomously between kicks. + and re-fixes autonomously between kicks. When the SPECIFIC test a PR fixed is + confirmed green in that PR's own CI, the loop marks the draft PR ready for review + (a state transition only — it never approves or merges). Never mutes tests, but de-flakes genuinely flaky ones (deterministic synchronization, no retries / timeout bumps). Always skips visual-regression / screenshot issues. @@ -239,8 +241,15 @@ safe-outputs: # PER-RUN total (not per-PR): a sweep may surface several green PRs and/or # annotate several flaky reds in one run. At max:1 all but the first comment # were silently dropped, starving the workflow's #1 value (surfacing green PRs - # for review). Raised to 3 to match the per-run throughput of the other outputs. - max: 3 + # for review). Sized to 6 (not 3) because a single green DRAFT PR that gets + # flipped ready in one sweep spends TWO comment slots — the Step 3 ✅ surface + # comment (which still names the /azp-gated legs a human must kick, so it is NOT + # redundant with 🎯) AND the Step 3.6 T3 🎯 readiness comment. At max:3 the shared + # bucket drained after ~1 draft flip and T3's atomicity pre-check then deferred + # every further mark-ready even while the mark-ready/add_labels buckets (also 3) + # sat idle; 6 lets ~3 draft flips (2 comments each) land per sweep, so the + # mark-ready:3 / add_labels:3 caps are actually reachable. + max: 6 target: "*" # Hard constraint (defense-in-depth): only comment on THIS workflow's own # ci-fix PRs — [ci-fix] title prefix AND agentic-workflows label. Mirrors the @@ -261,13 +270,51 @@ safe-outputs: # never the title, so disable title rewrites — this compiles to allow_title:false # and removes any ability to retitle an arbitrary PR. title: false - # NOTE: gh-aw v0.79.8 does NOT emit required-title-prefix/required-labels into + # NOTE: gh-aw v0.80.9 (the pinned compiler) does NOT emit required-title-prefix/required-labels into # the compiled config for update-pull-request (verified against the lock — it # silently drops them, unlike add-comment / push-to-pull-request-branch which # honor them). So which-PR scoping here relies on prompt Hard-Rule 6 + # min-integrity:approved; the residual is body-marker edits only (no code, no # merge, no close), capped at max:3. Revisit required-* if a future gh-aw # compiler emits them for this output. + mark-pull-request-as-ready-for-review: + # Flip a validated draft [ci-fix] PR to ready-for-review once the SPECIFIC test + # this PR was opened to fix is confirmed green in the PR's OWN CI (Step 3.6) — + # even when unrelated legs are red. The workflow is schedule/dispatch-triggered + # (no triggering-PR context), so target "*" lets the agent name the PR number it + # validated. This is a state transition ONLY: it never approves and never merges — + # a human still reviews and merges. + # PER-RUN total (not per-PR): a sweep may validate several PRs' target tests green. + max: 3 + target: "*" + # Hard constraint (defense-in-depth): only ever un-draft THIS workflow's own + # ci-fix PRs — [ci-fix] title prefix AND agentic-workflows label — so a confused + # or prompt-injected agent cannot mark an arbitrary PR ready. (If the v0.80.9 + # compiler silently drops these — as documented for update-pull-request above — + # the Step 3.6 preconditions + min-integrity:approved are the compensating scope + # controls; verify against the lock after compiling.) + required-title-prefix: "[ci-fix] " + required-labels: [agentic-workflows] + add-labels: + # Apply the p/0 priority label to a [ci-fix] PR at the exact moment the loop flips it + # from draft to ready-for-review (Step 3.6 T3) — so a validated, review-ready fix lands + # in the team's p/0 triage queue instead of sitting unseen in the draft backlog. Paired + # 1:1 with mark-pull-request-as-ready-for-review; target "*" lets the agent name the PR + # it just validated. + # allowed = HARD allowlist: the agent may ONLY ever add p/0, nothing else. This caps the + # blast radius of a confused/prompt-injected agent to exactly one benign priority label — + # it can never apply a downstream-triggering or destructive label. + allowed: [p/0] + # PER-RUN total (not per-PR): a sweep may mark several PRs ready in one run; each adds + # one label, so this matches the mark-ready per-run cap (3). + max: 3 + target: "*" + # Hard constraint (defense-in-depth): only ever label THIS workflow's own ci-fix PRs — + # [ci-fix] title prefix AND agentic-workflows label. (If the v0.80.9 compiler drops these + # — as documented for update-pull-request above — the Step 3.6 preconditions + + # min-integrity:approved + the allowed:[p/0] allowlist are the compensating controls.) + required-title-prefix: "[ci-fix] " + required-labels: [agentic-workflows] timeout-minutes: 90 @@ -339,33 +386,48 @@ through `safe-outputs`. 5. **One issue = one outcome per run.** Exactly one of: respond to a maintainer's change-request on the open PR (Track C, Step 3.5.R); open the first fix/help/ de-flake PR; advance an existing PR by one attempt (push a follow-up fix); - surface a validated-green PR for review; annotate an unrelated-flake red; wait + surface a validated-green PR for review; mark a target-validated draft PR + ready-for-review (Step 3.6 T3 — the terminal outcome that supersedes the same + run's surface/annotate precursor line), or record its `already-ready` + steady-state no-op; annotate an unrelated-flake red; wait (CI not yet settled); or a recorded skip (the dedicated needs-human PR is deferred — the attempt cap records a skip instead; see Step 6). Always prefer advancing or opening a PR over a skip when a non-mute diff is producible. 6. **All writes via `safe-outputs`.** Allowed outputs: `create_pull_request` (first attempt only), `push_to_pull_request_branch` (advance an existing PR by one attempt), `update_pull_request` (bump the attempt marker / refresh the - prior-attempts table), and `add_comment` (progress notes on the `[ci-fix]` PR - ONLY). NEVER comment on the tracking issue (issues are locked by + prior-attempts table), `add_comment` (progress notes on the `[ci-fix]` PR + ONLY), `mark_pull_request_as_ready_for_review` (flip a target-validated draft + `[ci-fix]` PR from draft to ready — Step 3.6 T3 only), and `add_labels` (add + ONLY the `p/0` label, ONLY on that same draft→ready transition — Step 3.6 T3). + NEVER comment on the tracking issue (issues are locked by `.github/workflows/ci-scan-lock-issues.yml`) — `add_comment` targets the PR. No `gh pr create`, no manual `git push`. **Defense-in-depth:** `add_comment` and `update_pull_request` use `target: "*"` (the agent supplies the PR number). - `add_comment` is now config-locked to `required-title-prefix: "[ci-fix] "` + + `add_comment`, `mark_pull_request_as_ready_for_review`, and `add_labels` are + config-locked to `required-title-prefix: "[ci-fix] "` + `required-labels: [agentic-workflows]` (hard-enforced by the handler, same as - `push_to_pull_request_branch`), so a comment can only ever land on THIS - workflow's own PRs. `update_pull_request` CANNOT be config-locked to a - title/label in gh-aw v0.79.8 (the compiler silently drops `required-*` for that - output — verified against the lock), so it keeps `title: false` (no retitles; - body-marker edits only) plus this prompt-level guard. Before emitting either, - VERIFY the target PR carries BOTH the `[ci-fix]` title prefix AND the - `agentic-workflows` label; never comment on or edit the body of any PR that - lacks both. + `push_to_pull_request_branch`), and `add_labels` additionally has an + `allowed: [p/0]` allowlist so it can ONLY ever add `p/0` — so these can only + ever land on THIS workflow's own PRs. `update_pull_request` CANNOT be + config-locked to a title/label in gh-aw v0.80.9 (the compiler silently drops + `required-*` for that output — verified against the lock), so it keeps + `title: false` (no retitles; body-marker edits only) plus this prompt-level + guard. Before emitting ANY of these, VERIFY the target PR carries BOTH the + `[ci-fix]` title prefix AND the `agentic-workflows` label; never comment on, + edit, un-draft, or label any PR that lacks both. 7. **Per-run safe-output caps (per RUN, not per PR).** Each output type is capped per run: `create_pull_request` 3, `push_to_pull_request_branch` 3, - `add_comment` 3, `update_pull_request` 3. Note an ADVANCE spends one push **and** - one update **and** one comment, so ≤ 3 advances/run; surfacing a green or - annotating a flake spends one comment. When a bucket is exhausted, do NOT keep + `add_comment` 6, `update_pull_request` 3, `mark_pull_request_as_ready_for_review` + 3, `add_labels` 3 (counts LABELS, not calls). Note an ADVANCE spends one push + **and** one update **and** one comment, so ≤ 3 advances/run; surfacing a green or + annotating a flake spends one comment; a **mark-ready (Step 3.6 T3)** of a + still-draft PR spends TWO comments (the Step 3 ✅ surface **and** the T3 🎯 + readiness note) **and** one mark-ready **and** one label as an ALL-OR-NOTHING set + (T3 pre-checks those buckets and defers the whole PR if any is exhausted — never + mark a PR ready without its 🎯 audit comment). `add_comment` is therefore sized 6 + (not 3) so ~3 draft flips, at 2 comments each, can land in one sweep instead of the + shared comment bucket starving the otherwise-idle mark-ready/add_labels buckets. When a bucket is exhausted, do NOT keep emitting (extras are silently dropped) — record `skipped: per-run cap reached; deferring PR # to next cycle` for each remaining PR so the drop is a deliberate, logged decision. The continuous loop picks the deferred PRs up next @@ -409,8 +471,9 @@ For every open tracking issue in scope, converge on exactly one outcome: | Open first help-wanted draft `[ci-fix]` PR | No open PR yet; a plausible candidate exists but cannot be runner-validated (device/UI tests) (attempt 1) | | Open first de-flake draft `[ci-fix]` PR | No open PR yet; failure is intermittent (green-on-retry) from a genuine test-quality defect; a deterministic-synchronization fix is producible (attempt 1, Step 4.7 bucket b) | | Advance an existing PR (push attempt N+1) | Open PR's own CI settled red *because of the fix itself*, no human engaged, marker < 10 — push a NEW distinct fix onto the same branch (Step 3.5 → 5.6) | -| Surface a validated-green PR | Open PR's own CI settled green — comment "validated, attempt N/10; ready for review" and do NOT advance (Step 3.5) | -| Annotate an unrelated-flake red | Open PR is red only on baseline-flake / unrelated legs — comment which leg needs a re-run; do NOT burn an attempt (Step 3.5) | +| Surface a validated-green PR | Open PR's own CI settled green — comment "validated, attempt N/10; ready for review" and do NOT advance (Step 3.5), then mark the draft PR ready for review once the fixed test is confirmed green (Step 3.6) | +| Annotate an unrelated-flake red | Open PR is red only on baseline-flake / unrelated legs — comment which leg needs a re-run; do NOT burn an attempt (Step 3.5); if the SPECIFIC fixed test is confirmed green on that build, still mark the draft PR ready for review (Step 3.6) | +| Mark a validated PR ready for review | The specific test this PR fixed is confirmed green in the PR's own CI (even if unrelated legs are red) — comment the target-test result and transition the draft PR to ready for review; never approve or merge (Step 3.6) | | Wait | Open PR's CI is pending / not yet settled on the current head SHA — do nothing this cycle (Step 3.5) | | Hand-off skip (attempt cap) | Marker == 10 and the signature still reproduces — stop and defer to humans; the dedicated `[ci-fix][needs-human]` PR is planned but currently deferred (Step 6) | | Recorded skip | Visual-regression, human engaged, already-handled, fixed-in-latest-build, infra-flake, out-of-bounds, only-mute-available, or no novel approach producible | @@ -685,14 +748,34 @@ Run these gates in order — the FIRST that fires decides this cycle's outcome: 1. **Human engaged.** If `C.humanEngaged` → `skipped: human engaged on PR #; deferring` and stop. Never fight a human reviewer — the intentional hand-off boundary still holds the moment a person touches the PR. -2. **CI not settled / unknown / incomplete prefetch.** If `C.checksSettled == false` - OR `C.overallConclusion` is `pending`, `neutral`, or `unknown`, OR - `C.dataComplete == false` → the fix's CI has not finished on the current head SHA - (`C.headSha`), or the prefetch could not fully read this PR (a partial read can - understate `humanEngaged` / `attempt`). `skipped: PR # CI pending / prefetch - incomplete on ; waiting` and stop. *(Round 1: this is where a - maintainer `/azp run` is awaited — the `/azp`-gated uitests/devicetests legs will - not have run until a human kicks them.)* +2. **CI not settled — but validate the target test first (target-focused readiness).** + If `C.checksSettled == false` OR `C.overallConclusion` is `pending`, `neutral`, or + `unknown`, OR `C.dataComplete == false` → the *overall* build has not finished on the + current head SHA (`C.headSha`), or the prefetch could not fully read this PR (a partial + read can understate `humanEngaged` / `attempt`). Unrelated legs still draining must NOT + keep an already-proven fix parked as a draft, so before waiting, try the target-focused + fast-path: + - **2a — Target-green fast-path (draft `[ci-fix]` PRs only).** If `C.dataComplete == + true` AND the Step 3.6 preconditions hold (draft PR, `[ci-fix] ` title prefix AND the + `agentic-workflows` label), run Step 3.6 **T1–T2** against `C.headSha` now: identify + the target test(s) and read the PR's OWN build **timeline / per-leg status** (anonymous + `_apis/build` — never `_apis/test`, per Hard-Rule 8). If EVERY target test is + **VALIDATED-GREEN** (per T2's all-platform definition below — the target's category leg + is `succeeded` on every platform that runs it, failed on none, and NO target platform + family the pipeline covers is still pending/unconcluded, per T2's "no platform left + unverified" rule; a platform that simply has no leg for the target's category — the test + doesn't run there — is fine, not a gap) AND every leg in + `C.failedLegs` that has ALREADY concluded classifies as **unrelated flake** (Step 4 / + Step 4.7 method — the fix is implicated in NO completed red), then the fix is proven + regardless of unrelated *pending* legs → run **Step 3.6 T3** (mark ready + 🎯 comment) + and stop. This early-out NEVER advances an attempt and NEVER acts on a red that could + be the fix's fault. + - **2b — Otherwise WAIT.** If the target test has not yet executed (pending/absent on + its leg), or a completed red is (or may be) caused by the fix, or `C.dataComplete == + false`, or this PR has no identifiable target test → `skipped: PR # CI pending / + target not yet validated on ; waiting` and stop. *(Round 1: this is where a + maintainer `/azp run` is awaited — the `/azp`-gated uitests/devicetests legs will not + have run until a human kicks them.)* 3. **Green → surface for review.** If `C.overallConclusion == "success"`: the checks that RAN are green. **Primary-gate check first:** the deterministic `success` verdict only certifies "at least one green check, nothing failing or @@ -704,18 +787,31 @@ Run these gates in order — the FIRST that fires decides this cycle's outcome: # primary CI gate (maui-pr) not green on ; waiting` and stop — do NOT surface. (The `/azp`-gated `maui-pr-uitests` (def 313) / `maui-pr-devicetests` (def 314) legs MAY still be un-run — that is expected and is named below, not a - reason to withhold the surface.) **Idempotency:** scan the PR's existing comments - for a prior bot `✅ … validated … on ` note for THIS head SHA — if one - already exists, record `already-surfaced PR # (head )` and stop - WITHOUT re-commenting (re-surfacing the same green every tick is noise and burns - the per-run comment budget other PRs need). Otherwise resolve the attempt number from - `C.effectiveAttempt` (the authoritative max(marker, bot-commit) counter; omit the - number only if it is somehow indeterminate). **Dry-run gate (Step 0):** if `dry_run == "true"`, do NOT emit any comment — instead print the intended `✅` validated-green body to the run log, tally `dry-run: would-surface-green PR #`, and stop. Otherwise `add_comment` on PR #: `✅ Attempt /10 validated - — the fix's CI is green on . Ready for human review.` - Name any `/azp`-gated legs (uitests def 313 / devicetests def 314) that have not - run and still need a maintainer `/azp run`. Do NOT advance. Record - `surfaced-green PR # (attempt /10)` and stop. *(This directly - attacks the real bottleneck — no reviews — so it is the highest-value outcome.)* + reason to withhold the surface.) **Comment idempotency + dry-run suppress the ✅ + comment ONLY — neither skips Step 3.6.** Scan the PR's existing comments for a prior + bot `✅ … validated … on ` note for THIS head SHA; if one exists, set + `SKIP_SURFACE_COMMENT = true`. Resolve the attempt number from `C.effectiveAttempt` + (the authoritative max(marker, bot-commit) counter; omit the number only if it is + somehow indeterminate). Then post the surface comment UNLESS suppressed: + - if `dry_run == "true"`: do NOT emit — print the intended `✅` validated-green body to + the run log and tally `dry-run: would-surface-green PR #`; + - else if `SKIP_SURFACE_COMMENT`: do NOT re-post — record `already-surfaced PR # + (head )` (re-surfacing the same green every tick is noise and burns the + per-run comment budget other PRs need); + - else `add_comment` on PR #: `✅ Attempt /10 validated — the fix's CI is + green on .` naming any `/azp`-gated legs (uitests def 313 / devicetests def + 314) that have not run and still need a maintainer `/azp run`; record `surfaced-green + PR # (attempt /10)`. (Do NOT assert "ready for human review" on a PR that + is still a draft — Step 3.6, not this comment, owns the draft→ready flip and posts its + own 🎯 announcement when it fires.) + Do NOT advance. Then — in ALL of the above cases — run **Step 3.6** (target-test + readiness gate) for this PR before stopping: its `/azp`-gated target legs may conclude + green on this SAME head SHA on a LATER sweep (an `/azp run` adds no commit), and Step + 3.6 is the ONLY place the draft→ready flip happens, so it MUST re-evaluate every sweep — + never `stop` here before it. *(This directly attacks the real bottleneck — no reviews — + so it is the highest-value outcome.)* 4. **Red → classify caused-by-fix vs unrelated-flake.** If `C.overallConclusion == "failure"`, analyze the PR's OWN failing build (NOT `main`): find the AzDO `maui-pr` build for this PR (filter builds by `branchName=refs/pull//merge`, @@ -723,18 +819,26 @@ Run these gates in order — the FIRST that fires decides this cycle's outcome: method and Step 4.7 flake buckets to `C.failedLegs`. - **Unrelated flake only** (every failed leg is known-flaky / infra / a pre-existing baseline red NOT introduced by the fix): do NOT burn an attempt. - **Idempotency first:** scan the PR's existing comments for a prior bot - `♻️ … unrelated flake … on ` note for THIS head SHA — if one already - exists, record `already-annotated-flake PR # (head )` and stop - WITHOUT re-commenting (re-annotating the same flake every 12h sweep is noise and - burns the per-run comment budget other PRs need). Otherwise resolve the attempt - number from `C.effectiveAttempt` (the authoritative max(marker, bot-commit) - counter), omitting the `Attempt /10:` prefix only if it is somehow - indeterminate. **Dry-run gate (Step 0):** if `dry_run == "true"`, do NOT emit any comment — instead print the intended `♻️` unrelated-flake body to the run log, tally `dry-run: would-annotate-flake PR #`, and stop. Otherwise `add_comment` on PR #: `♻️ Attempt /10: red is - unrelated flake on leg(s) () on ; the fix itself is not - implicated. A maintainer re-run (/azp run ) should clear it.` Record - `annotated-flake PR # (head )` and stop. *(Round 1: human re-runs; - Round 2: auto re-trigger.)* + **Comment idempotency + dry-run suppress the ♻️ comment ONLY — neither skips Step + 3.6.** Scan the PR's existing comments for a prior bot `♻️ … unrelated flake … on + ` note for THIS head SHA; if one exists, set `SKIP_FLAKE_COMMENT = true`. + Resolve the attempt number from `C.effectiveAttempt` (the authoritative max(marker, + bot-commit) counter), omitting the `Attempt /10:` prefix only if it is + somehow indeterminate. Then post the flake note UNLESS suppressed: + - if `dry_run == "true"`: do NOT emit — print the intended `♻️` unrelated-flake body + to the run log and tally `dry-run: would-annotate-flake PR #`; + - else if `SKIP_FLAKE_COMMENT`: do NOT re-post — record `already-annotated-flake PR + # (head )` (re-annotating the same flake every 12h sweep is noise and + burns the per-run comment budget other PRs need); + - else `add_comment` on PR #: `♻️ Attempt /10: red is unrelated flake on + leg(s) () on ; the fix itself is not implicated. A + maintainer re-run (/azp run ) should clear it.` record `annotated-flake + PR # (head )`. + Then — in ALL of the above cases — run **Step 3.6** (target-test readiness gate) for + this PR before stopping: its `/azp`-gated target legs may conclude green on this SAME + head SHA on a LATER sweep, and Step 3.6 is the ONLY place the draft→ready flip + happens, so it MUST re-evaluate every sweep — never `stop` here before it. *(Round 1: + human re-runs; Round 2: auto re-trigger.)* - **Caused by the fix** (a failed leg still matches the original target signature, or the fix introduced a NEW failure): advance an attempt. - **Attempt count.** `attempt = C.effectiveAttempt` — the authoritative @@ -752,6 +856,188 @@ Run these gates in order — the FIRST that fires decides this cycle's outcome: from every prior commit on this branch**, emitted via the Step 5.6 ADVANCE path (push onto the same PR). +#### Step 3.6 — Target-test verification & mark-ready gate + +Reached from the Step 3 **green-surface** branch, the Step 4 **unrelated-flake** branch +(AFTER that branch has posted its comment), and the Step 3.5 **gate-2 target-focused +fast-path** (2a — while unrelated legs are still pending). Purpose: confirm the SPECIFIC +test(s) this PR was opened to fix now PASS in the PR's own CI, and — only when they do +— transition the draft `[ci-fix]` PR to **ready for review** so a maintainer sees a +validated fix instead of a draft. This is the ONLY place the loop flips draft→ready; it +is a state transition, **never** an approval or a merge (a human still reviews and +merges). Overall red on *unrelated* legs must NOT gate readiness — we validate the fix, +not the base branch's flakiness. + +**Preconditions** (ALL must hold; otherwise record `skipped: readiness N/A PR # +()` and stop this gate): +- `C.isDraft == true` — if the PR is already ready-for-review, the draft→ready + transition is already done; do NOT re-mark **and do NOT re-apply `p/0`**. `p/0` is applied + exactly ONCE, atomically with the draft→ready flip (T3 — all-or-nothing with the 🎯 audit + comment + mark-ready), so an already-ready loop PR that is MISSING `p/0` has almost + certainly had it **removed by a maintainer** de-prioritizing the PR — a pure triage action + the human-engagement guard (which inspects comments/reviews/commits, NOT label changes) + cannot see. Re-adding `p/0` every sweep would fight that maintainer indefinitely, violating + the loop's "never override a human" contract. So the loop does NOT reconcile the label: + record `already-ready PR #` and stop this gate. (Trade-off: on the rare occasion a + transient API error drops `p/0` at flip time *after* the T3 atomic pre-check passed, it is + not auto-re-added — but the PR is still ready-for-review with its 🎯 audit comment, and + re-adding `p/0` is a trivial manual action; that is strictly preferable to steam-rolling a + maintainer's deliberate de-prioritization.) +- The PR is unmistakably THIS workflow's own: `[ci-fix] ` title prefix AND the + `agentic-workflows` label (mirrors the safe-output lock; the compensating scope + control if the v0.80.9 compiler drops the declarative required-*). +- You reached this gate from Step 3 (green), Step 4 (**unrelated-flake**), or the Step 3.5 + gate-2 **target-focused fast-path** (2a) — i.e. the fix is NOT implicated in any red that + has concluded. If Step 4 classified a red as **caused by the fix** (ADVANCE), do NOT run + this gate — advance the attempt instead. + +**T1 — Identify the target test(s).** From the `[ci-scan]` issue signature the fix +addresses (and the PR's own diff), extract the fully-qualified test method name(s) the +fix targets — e.g. `SafeAreaShouldWorkOnAllShellTabs`. For a de-flake it is the +de-flaked test; for a product fix it is the originally-failing test(s). If NO specific +test can be identified (e.g. a product build-break rather than a test failure), this is a +**build-only fix** — there is no single test to validate cross-platform, so validate at +**whole-build** granularity rather than stopping (Step 3 only *comments*; Step 3.6 is the +sole draft→ready flip, so a build-only fix is undrafted HERE or not at all). We only reach +this gate from Step 3 (green) / Step 4 (unrelated-flake) / gate-2a, so the fix is not +implicated in any concluded red; additionally require, via the T2 timeline method on +`C.headSha`, that the primary build pipeline `maui-pr` (def 302) has CONCLUDED with EVERY +one of its platform build legs `succeeded`/`completed` and NONE failed/canceled or still +pending — the build is green on **every** platform, not just the originally-broken one (the +cross-platform guard, applied to the build instead of a test). A build-only fix is undrafted +ONLY on the strength of the auto-running `maui-pr` (def 302) whole-build green, which the +loop CAN observe: if the PR's `[ci-scan]` issue signature or its own diff indicates the +ORIGINATING failure was in a `/azp`-gated pipeline (`maui-pr-uitests` def 313 or +`maui-pr-devicetests` def 314) rather than `maui-pr` (def 302), do NOT undraft on +`maui-pr`-green alone — those pipelines do not auto-run on this PR (GITHUB_TOKEN cannot +trigger them), so a green `maui-pr` build is NOT evidence the gated build break is fixed; +record `skipped: build-only fix PR # targets gated pipeline () not run — +deferring to human` and stop this gate. Otherwise, set `TARGET := "the maui-pr build +(build-only fix — no single target test)"` and proceed to **T3** to mark ready. If any `maui-pr` build leg is still unconcluded, record `skipped: build-only fix PR +# not yet whole-build green (leg(s) pending)` and stop this gate WITHOUT marking +ready (a green subset is not enough — a still-pending build leg could yet fail). + +**T2 — Verify the target test's platform legs are green (anonymous, leg-level).** Per-test +outcomes come from the AzDO **test-results** API (`_apis/test/...`), which is **NOT reachable +anonymously — Hard-Rule 8**: those endpoints 302-redirect to a sign-in page on this runner, so +a `curl` returns an HTML redirect (not JSON) and every target test would look "not executed". +Validate instead at **leg granularity** using ONLY the anonymous `_apis/build/...` timeline +(Hard-Rule 8) — a `succeeded` category leg means every test in that category **that actually +ran** passed on that platform. This is a strong bar **only if the target test genuinely +executes**: a job can go green while one specific test is skipped at runtime (`[Ignore]`, +`Assert.Ignore()`, `Assert.Inconclusive()`, a `Skip=`/conditional `[Fact]`, an `#if`-out, or +an early `return` before the asserts). So before trusting a green leg as proof the target +passed, **confirm from the PR's OWN diff that the fix does NOT skip, ignore, disable, or +short-circuit the target test** (it adds no `[Ignore]`/`[Explicit]`, `Assert.Ignore`/ +`Assert.Inconclusive`, `Skip=`, category-exclusion, `#if`-out, or early `return` around the +target). If the fix could cause the target to be runtime-skipped rather than genuinely pass, +leg-level green is NOT sufficient evidence — record `skipped: target test may be +runtime-skipped by this fix (diff adds ); leg-green insufficient, deferring to human +on PR #` and stop this gate WITHOUT marking ready. Otherwise (the target genuinely runs +and asserts, as a de-flake or product fix does) a green category leg cannot hide a +target-test failure, so treat it as authoritative. + +Map the target test to the CI leg(s) that run it. A leg name encodes platform + test category +(e.g. `Android UITests SafeAreaEdges,Shadow`, `iOS UITests SafeAreaEdges,Shadow`, `macOS +UITests SafeAreaEdges,Shadow`, `Windows UITests SafeAreaEdges,Shadow`). Determine the target +test's UI-test category — its `[Category(UITestCategories.X)]` in the test/HostApp source, +visible in the PR diff or the test file — to know which leg-name substring identifies its legs; +for a device test, the per-platform device-test legs. + +Using the SAME build-discovery as Step 4 (filter AzDO builds by `branchName=refs/pull//merge` +or `sourceVersion == C.headSha`), read each build's **timeline** on `C.headSha` for the +pipeline(s) that RUN the target test — `maui-pr` (def 302) for unit/integration tests, +`maui-pr-uitests` (def 313) for Appium UI tests, `maui-pr-devicetests` (def 314) for device +tests: + +```bash +ORG=dnceng-public; PROJ=public; BUILD= +# Anonymous + Hard-Rule-8-compliant. NEVER call _apis/test/... (it 302-redirects to sign-in). +# Each type=="Job" record is one platform leg: result (succeeded/failed/canceled/null), +# state (completed/inProgress/pending), name (encodes " UITests "): +curl -s "https://dev.azure.com/$ORG/$PROJ/_apis/build/builds/$BUILD/timeline?api-version=7.1" \ + | tee /tmp/gh-aw/agent/timeline_${BUILD}.json \ + | jq -r '.records[] | select(.type=="Job") | "\(.result)\t\(.state)\t\(.name)"' +``` + +(The prefetch already exposes per-leg status in `C.failedLegs` / the checks context, and +`gh pr checks ` lists the same per-leg rows with platform+category in the name — use +whichever is handy; the build timeline is the authoritative cross-check on the exact build.) + +Treat a target test as **VALIDATED-GREEN** only if, on `C.headSha`, the leg that runs its +category is green on **every platform it runs on** — a fix that repairs one platform must not +silently regress the same test's leg on another, so a green leg on the originally-red platform +alone is NOT enough. Across the target pipeline's platform legs (for a UI test: the Android, +iOS, Windows, and macOS/MacCatalyst legs running the target's category; for a device test: each +device-test platform), require ALL of: +- the target's category leg is `result == "succeeded"` **and** `state == "completed"` on + **every** platform that runs it, AND +- that leg is `failed` / `canceled` / aborted on **NO** platform, AND +- **no platform is left unverified.** For each platform family the target pipeline covers, that + platform's category leg must have CONCLUDED (`state == "completed"`) on `C.headSha`. If a + platform simply has no leg for the target's category, the test does not run there — that is + fine, not a gap. But if a platform's category leg has **not concluded** (`state` is + `inProgress` / pending, or an `/azp`-gated `maui-pr-uitests` / `maui-pr-devicetests` leg that + has not been kicked), the test's status on that platform is UNKNOWN → the fix is NOT yet + validated across platforms: record `skipped: target test green on but + not yet verified on (leg(s) pending / need /azp run) on PR #` + and stop this gate WITHOUT marking ready. + +A target test whose category leg never concluded on ANY platform (**not executed** anywhere — +e.g. its `/azp`-gated pipeline has not been kicked) is likewise NOT validated: record `skipped: +target test not yet executed on PR # ( not run — needs /azp run)` and stop +this gate WITHOUT marking ready. Do NOT overclaim — a green *sibling* leg (a different category +on the same platform) is not the target's leg, and a green leg on one platform is not a pass on +the others. + +**T3 — Mark ready + report.** If EVERY target test is VALIDATED-GREEN. The 🎯 comment, +the mark-ready, and the `p/0` label are THREE SEPARATE safe-outputs — the comment +existing does NOT prove the mark-ready took effect, so they are tracked independently: + +- **Comment idempotency (dup-suppress only — never gates the mark-ready).** We only reach + T3 while `C.isDraft == true` (the precondition stops an already-ready PR before here). So + if a prior bot comment for THIS head that carries the leading `🎯` anchor AND the + contiguous phrase `validated green on ` already exists (BOTH T3 variants — + `🎯 Target test validated green on ` and the build-only `🎯 Build validated green on + ` — contain that exact contiguous phrase, so this recognizes both; keying on + `target test` alone would instead miss the build-only comment and re-post it every sweep. + **Require the `🎯` anchor** so the match can NEVER be loosened to a gapped + `validated…green…on` form that would false-positive on the Step 3 `✅ … validated — the + fix's CI is green on ` surface comment and wrongly suppress the audit 🎯), the comment + landed on an earlier sweep but the **mark-ready did NOT take + effect** (the PR is still a draft) — set `SUPPRESS_COMMENT = true` (do not re-post the + duplicate 🎯 comment) but STILL complete the mark-ready + label below. Never treat the + comment's existence as "already marked ready" while the PR is still a draft. Record this + case as `re-marking PR # (🎯 present; mark-ready did not take on a prior sweep)`. +- **Atomicity budget pre-check (Hard-Rule 7).** Determine the outputs this gate will emit: + `mark_pull_request_as_ready_for_review` + `add_labels` always, plus `add_comment` unless + `SUPPRESS_COMMENT`. Verify a free per-run slot remains in EACH bucket you are about to + use. If ANY required bucket is exhausted, emit NONE of them — record `skipped: per-run + cap reached; deferring mark-ready PR # to next cycle` and stop this gate. Never mark a + PR ready (or label it) unless its 🎯 audit comment is GUARANTEED to exist for this exact + head SHA — either posted in this same sweep, or (in the `SUPPRESS_COMMENT` re-marking case) + already present from a prior sweep. The outputs you DO emit either all land or all defer + together; the audit trail must never be absent, but it is NOT re-posted when it already exists. +- **Dry-run gate (Step 0):** if `dry_run == "true"`, emit NOTHING — print the intended + readiness comment and "would mark ready + add p/0" to the run log, tally `dry-run: + would-mark-ready PR #`, and stop. +- Otherwise emit for THIS PR number ``: + 1. `add_comment` (ONLY if not `SUPPRESS_COMMENT`): `🎯 Target test validated green on + — passed on ALL platforms it runs on (, + buildId ). — not + caused by this fix."> Transitioning this PR from draft to ready for review and adding + `p/0`; a maintainer still reviews and merges.` (For a **build-only fix** — the T1 + whole-build fallback, no single target test — phrase the first clause as `🎯 Build + validated green on — the maui-pr build passed on ALL platforms (buildId + ).` instead of naming a test.) + 2. `mark_pull_request_as_ready_for_review` with `reason:` a one-line justification + naming the validated test(s) and ``. + 3. `add_labels` with `labels: ["p/0"]` for PR # — put the now-review-ready fix into + the team's p/0 priority queue so it is triaged, not lost in the draft backlog. (If + the PR somehow already carries `p/0`, this is a harmless no-op.) +- Record `marked-ready PR # (target green on ALL platforms on , + labeled p/0)` and stop. + #### Step 3.5.R — Maintainer change-request response (Track C) Reached from Step 3.5 gate 0. Goal: when an **eligible human reviewer** (Hard-Rule @@ -1383,7 +1669,19 @@ Filed by [`ci-status-fix`](https://github.com/dotnet/maui/blob/main/.github/work ### Step 8 — Per-issue tally + end-of-run summary -Per issue, append one outcome line to `/tmp/gh-aw/agent/coverage.txt`: +Per issue, append **one** outcome line to `/tmp/gh-aw/agent/coverage.txt` — the +terminal outcome for the cycle. When one cycle produces a chained pair — a PR gets a +same-cycle precursor line and **then** a Step 3.6 readiness line (`marked-ready` on the +draft→ready flip, or `already-ready` on a later steady-state sweep of an already-ready +PR) — record ONLY the terminal Step 3.6 readiness line: it supersedes ANY same-cycle +non-readiness precursor for that PR, so an aggregator keying on one-line-per-issue never +double-counts. The precursor may be a Step 3 green line (`surfaced-green` on the first ✅, +or `already-surfaced` when it already exists) OR — when an unrelated-flake red and a +target-green coincide — a Step 4 `annotated-flake` line; either way the readiness line is +terminal, and the superseded precursor's signal survives in the PR's 🎯/♻️ comment so no +information is lost. This covers the flip cycle (`surfaced-green` → `marked-ready`), the +post-flip steady state (`already-surfaced` → `already-ready`), and the flake-coincident +flip (`annotated-flake` → `marked-ready`): ``` # main attempt- @@ -1394,8 +1692,14 @@ Per issue, append one outcome line to `/tmp/gh-aw/agent/coverage.txt`: `advance-PR # attempt /10` (ADVANCE mode — new commit pushed to the existing PR), `surfaced-green PR #` (fix's own CI went green; commented for review, did not advance), `annotated-flake PR #` (red was unrelated flake; -commented, attempt NOT burned), `waiting PR #` (CI not settled yet), -`dry-run: would-`, `skipped: `. (The +commented, attempt NOT burned), `marked-ready PR #` (the specific fixed test +was confirmed green in the PR's own CI; draft flipped to ready for review), +`already-surfaced PR #` (the fix's ✅ green comment already existed this sweep; +re-post suppressed — superseded by the Step 3.6 readiness line when the PR also flips +ready that cycle), `already-ready PR #` (terminal steady state — the PR was flipped +ready on a prior cycle and remains green on an unchanged head; no action taken, +supersedes `already-surfaced`), `waiting PR #` (CI not settled yet), +`dry-run: would-`, `skipped: `. (The `needs-human-PR` outcome is reserved for the deferred hand-off — Step 6 currently records a skip instead, so it is not emitted.)
(head )`. + Then — in ALL of the above cases — run **Step 3.6** (target-test readiness gate) for + this PR before stopping: its `/azp`-gated target legs may conclude green on this SAME + head SHA on a LATER sweep, and Step 3.6 is the ONLY place the draft→ready flip + happens, so it MUST re-evaluate every sweep — never `stop` here before it. *(Round 1: + human re-runs; Round 2: auto re-trigger.)* - **Caused by the fix** (a failed leg still matches the original target signature, or the fix introduced a NEW failure): advance an attempt. - **Attempt count.** `attempt = C.effectiveAttempt` — the authoritative @@ -762,6 +866,188 @@ Run these gates in order — the FIRST that fires decides this cycle's outcome: from every prior commit on this branch**, emitted via the Step 5.6 ADVANCE path (push onto the same PR). +#### Step 3.6 — Target-test verification & mark-ready gate + +Reached from the Step 3 **green-surface** branch, the Step 4 **unrelated-flake** branch +(AFTER that branch has posted its comment), and the Step 3.5 **gate-2 target-focused +fast-path** (2a — while unrelated legs are still pending). Purpose: confirm the SPECIFIC +test(s) this PR was opened to fix now PASS in the PR's own CI, and — only when they do +— transition the draft `[ci-fix-net11]` PR to **ready for review** so a maintainer sees +a validated fix instead of a draft. This is the ONLY place the loop flips draft→ready; +it is a state transition, **never** an approval or a merge (a human still reviews and +merges). Overall red on *unrelated* legs must NOT gate readiness — we validate the fix, +not the base branch's flakiness. + +**Preconditions** (ALL must hold; otherwise record `skipped: readiness N/A PR # +()` and stop this gate): +- `C.isDraft == true` — if the PR is already ready-for-review, the draft→ready + transition is already done; do NOT re-mark **and do NOT re-apply `p/0`**. `p/0` is applied + exactly ONCE, atomically with the draft→ready flip (T3 — all-or-nothing with the 🎯 audit + comment + mark-ready), so an already-ready loop PR that is MISSING `p/0` has almost + certainly had it **removed by a maintainer** de-prioritizing the PR — a pure triage action + the human-engagement guard (which inspects comments/reviews/commits, NOT label changes) + cannot see. Re-adding `p/0` every sweep would fight that maintainer indefinitely, violating + the loop's "never override a human" contract. So the loop does NOT reconcile the label: + record `already-ready PR #` and stop this gate. (Trade-off: on the rare occasion a + transient API error drops `p/0` at flip time *after* the T3 atomic pre-check passed, it is + not auto-re-added — but the PR is still ready-for-review with its 🎯 audit comment, and + re-adding `p/0` is a trivial manual action; that is strictly preferable to steam-rolling a + maintainer's deliberate de-prioritization.) +- The PR is unmistakably THIS workflow's own: `[ci-fix-net11] ` title prefix AND the + `agentic-workflows` label (mirrors the safe-output lock; the compensating scope + control if the v0.80.9 compiler drops the declarative required-*). +- You reached this gate from Step 3 (green), Step 4 (**unrelated-flake**), or the Step 3.5 + gate-2 **target-focused fast-path** (2a) — i.e. the fix is NOT implicated in any red that + has concluded. If Step 4 classified a red as **caused by the fix** (ADVANCE), do NOT run + this gate — advance the attempt instead. + +**T1 — Identify the target test(s).** From the `[ci-scan-net11]` issue signature the fix +addresses (and the PR's own diff), extract the fully-qualified test method name(s) the +fix targets — e.g. `SafeAreaShouldWorkOnAllShellTabs`. For a de-flake it is the +de-flaked test; for a product fix it is the originally-failing test(s). If NO specific +test can be identified (e.g. a product build-break rather than a test failure), this is a +**build-only fix** — there is no single test to validate cross-platform, so validate at +**whole-build** granularity rather than stopping (Step 3 only *comments*; Step 3.6 is the +sole draft→ready flip, so a build-only fix is undrafted HERE or not at all). We only reach +this gate from Step 3 (green) / Step 4 (unrelated-flake) / gate-2a, so the fix is not +implicated in any concluded red; additionally require, via the T2 timeline method on +`C.headSha`, that the primary build pipeline `maui-pr` (def 302) has CONCLUDED with EVERY +one of its platform build legs `succeeded`/`completed` and NONE failed/canceled or still +pending — the build is green on **every** platform, not just the originally-broken one (the +cross-platform guard, applied to the build instead of a test). A build-only fix is undrafted +ONLY on the strength of the auto-running `maui-pr` (def 302) whole-build green, which the +loop CAN observe: if the PR's `[ci-scan]` issue signature or its own diff indicates the +ORIGINATING failure was in a `/azp`-gated pipeline (`maui-pr-uitests` def 313 or +`maui-pr-devicetests` def 314) rather than `maui-pr` (def 302), do NOT undraft on +`maui-pr`-green alone — those pipelines do not auto-run on this PR (GITHUB_TOKEN cannot +trigger them), so a green `maui-pr` build is NOT evidence the gated build break is fixed; +record `skipped: build-only fix PR # targets gated pipeline () not run — +deferring to human` and stop this gate. Otherwise, set `TARGET := "the maui-pr build +(build-only fix — no single target test)"` and proceed to **T3** to mark ready. If any `maui-pr` build leg is still unconcluded, record `skipped: build-only fix PR +# not yet whole-build green (leg(s) pending)` and stop this gate WITHOUT marking +ready (a green subset is not enough — a still-pending build leg could yet fail). + +**T2 — Verify the target test's platform legs are green (anonymous, leg-level).** Per-test +outcomes come from the AzDO **test-results** API (`_apis/test/...`), which is **NOT reachable +anonymously — Hard-Rule 8**: those endpoints 302-redirect to a sign-in page on this runner, so +a `curl` returns an HTML redirect (not JSON) and every target test would look "not executed". +Validate instead at **leg granularity** using ONLY the anonymous `_apis/build/...` timeline +(Hard-Rule 8) — a `succeeded` category leg means every test in that category **that actually +ran** passed on that platform. This is a strong bar **only if the target test genuinely +executes**: a job can go green while one specific test is skipped at runtime (`[Ignore]`, +`Assert.Ignore()`, `Assert.Inconclusive()`, a `Skip=`/conditional `[Fact]`, an `#if`-out, or +an early `return` before the asserts). So before trusting a green leg as proof the target +passed, **confirm from the PR's OWN diff that the fix does NOT skip, ignore, disable, or +short-circuit the target test** (it adds no `[Ignore]`/`[Explicit]`, `Assert.Ignore`/ +`Assert.Inconclusive`, `Skip=`, category-exclusion, `#if`-out, or early `return` around the +target). If the fix could cause the target to be runtime-skipped rather than genuinely pass, +leg-level green is NOT sufficient evidence — record `skipped: target test may be +runtime-skipped by this fix (diff adds ); leg-green insufficient, deferring to human +on PR #` and stop this gate WITHOUT marking ready. Otherwise (the target genuinely runs +and asserts, as a de-flake or product fix does) a green category leg cannot hide a +target-test failure, so treat it as authoritative. + +Map the target test to the CI leg(s) that run it. A leg name encodes platform + test category +(e.g. `Android UITests SafeAreaEdges,Shadow`, `iOS UITests SafeAreaEdges,Shadow`, `macOS +UITests SafeAreaEdges,Shadow`, `Windows UITests SafeAreaEdges,Shadow`). Determine the target +test's UI-test category — its `[Category(UITestCategories.X)]` in the test/HostApp source, +visible in the PR diff or the test file — to know which leg-name substring identifies its legs; +for a device test, the per-platform device-test legs. + +Using the SAME build-discovery as Step 4 (filter AzDO builds by `branchName=refs/pull//merge` +or `sourceVersion == C.headSha`), read each build's **timeline** on `C.headSha` for the +pipeline(s) that RUN the target test — `maui-pr` (def 302) for unit/integration tests, +`maui-pr-uitests` (def 313) for Appium UI tests, `maui-pr-devicetests` (def 314) for device +tests: + +```bash +ORG=dnceng-public; PROJ=public; BUILD= +# Anonymous + Hard-Rule-8-compliant. NEVER call _apis/test/... (it 302-redirects to sign-in). +# Each type=="Job" record is one platform leg: result (succeeded/failed/canceled/null), +# state (completed/inProgress/pending), name (encodes " UITests "): +curl -s "https://dev.azure.com/$ORG/$PROJ/_apis/build/builds/$BUILD/timeline?api-version=7.1" \ + | tee /tmp/gh-aw/agent/timeline_${BUILD}.json \ + | jq -r '.records[] | select(.type=="Job") | "\(.result)\t\(.state)\t\(.name)"' +``` + +(The prefetch already exposes per-leg status in `C.failedLegs` / the checks context, and +`gh pr checks ` lists the same per-leg rows with platform+category in the name — use +whichever is handy; the build timeline is the authoritative cross-check on the exact build.) + +Treat a target test as **VALIDATED-GREEN** only if, on `C.headSha`, the leg that runs its +category is green on **every platform it runs on** — a fix that repairs one platform must not +silently regress the same test's leg on another, so a green leg on the originally-red platform +alone is NOT enough. Across the target pipeline's platform legs (for a UI test: the Android, +iOS, Windows, and macOS/MacCatalyst legs running the target's category; for a device test: each +device-test platform), require ALL of: +- the target's category leg is `result == "succeeded"` **and** `state == "completed"` on + **every** platform that runs it, AND +- that leg is `failed` / `canceled` / aborted on **NO** platform, AND +- **no platform is left unverified.** For each platform family the target pipeline covers, that + platform's category leg must have CONCLUDED (`state == "completed"`) on `C.headSha`. If a + platform simply has no leg for the target's category, the test does not run there — that is + fine, not a gap. But if a platform's category leg has **not concluded** (`state` is + `inProgress` / pending, or an `/azp`-gated `maui-pr-uitests` / `maui-pr-devicetests` leg that + has not been kicked), the test's status on that platform is UNKNOWN → the fix is NOT yet + validated across platforms: record `skipped: target test green on but + not yet verified on (leg(s) pending / need /azp run) on PR #` + and stop this gate WITHOUT marking ready. + +A target test whose category leg never concluded on ANY platform (**not executed** anywhere — +e.g. its `/azp`-gated pipeline has not been kicked) is likewise NOT validated: record `skipped: +target test not yet executed on PR # ( not run — needs /azp run)` and stop +this gate WITHOUT marking ready. Do NOT overclaim — a green *sibling* leg (a different category +on the same platform) is not the target's leg, and a green leg on one platform is not a pass on +the others. + +**T3 — Mark ready + report.** If EVERY target test is VALIDATED-GREEN. The 🎯 comment, +the mark-ready, and the `p/0` label are THREE SEPARATE safe-outputs — the comment +existing does NOT prove the mark-ready took effect, so they are tracked independently: + +- **Comment idempotency (dup-suppress only — never gates the mark-ready).** We only reach + T3 while `C.isDraft == true` (the precondition stops an already-ready PR before here). So + if a prior bot comment for THIS head that carries the leading `🎯` anchor AND the + contiguous phrase `validated green on ` already exists (BOTH T3 variants — + `🎯 Target test validated green on ` and the build-only `🎯 Build validated green on + ` — contain that exact contiguous phrase, so this recognizes both; keying on + `target test` alone would instead miss the build-only comment and re-post it every sweep. + **Require the `🎯` anchor** so the match can NEVER be loosened to a gapped + `validated…green…on` form that would false-positive on the Step 3 `✅ … validated — the + fix's CI is green on ` surface comment and wrongly suppress the audit 🎯), the comment + landed on an earlier sweep but the **mark-ready did NOT take + effect** (the PR is still a draft) — set `SUPPRESS_COMMENT = true` (do not re-post the + duplicate 🎯 comment) but STILL complete the mark-ready + label below. Never treat the + comment's existence as "already marked ready" while the PR is still a draft. Record this + case as `re-marking PR # (🎯 present; mark-ready did not take on a prior sweep)`. +- **Atomicity budget pre-check (Hard-Rule 7).** Determine the outputs this gate will emit: + `mark_pull_request_as_ready_for_review` + `add_labels` always, plus `add_comment` unless + `SUPPRESS_COMMENT`. Verify a free per-run slot remains in EACH bucket you are about to + use. If ANY required bucket is exhausted, emit NONE of them — record `skipped: per-run + cap reached; deferring mark-ready PR # to next cycle` and stop this gate. Never mark a + PR ready (or label it) unless its 🎯 audit comment is GUARANTEED to exist for this exact + head SHA — either posted in this same sweep, or (in the `SUPPRESS_COMMENT` re-marking case) + already present from a prior sweep. The outputs you DO emit either all land or all defer + together; the audit trail must never be absent, but it is NOT re-posted when it already exists. +- **Dry-run gate (Step 0):** if `dry_run == "true"`, emit NOTHING — print the intended + readiness comment and "would mark ready + add p/0" to the run log, tally `dry-run: + would-mark-ready PR #`, and stop. +- Otherwise emit for THIS PR number ``: + 1. `add_comment` (ONLY if not `SUPPRESS_COMMENT`): `🎯 Target test validated green on + — passed on ALL platforms it runs on (, + buildId ). — not + caused by this fix."> Transitioning this PR from draft to ready for review and adding + `p/0`; a maintainer still reviews and merges.` (For a **build-only fix** — the T1 + whole-build fallback, no single target test — phrase the first clause as `🎯 Build + validated green on — the maui-pr build passed on ALL platforms (buildId + ).` instead of naming a test.) + 2. `mark_pull_request_as_ready_for_review` with `reason:` a one-line justification + naming the validated test(s) and ``. + 3. `add_labels` with `labels: ["p/0"]` for PR # — put the now-review-ready fix into + the team's p/0 priority queue so it is triaged, not lost in the draft backlog. (If + the PR somehow already carries `p/0`, this is a harmless no-op.) +- Record `marked-ready PR # (target green on ALL platforms on , + labeled p/0)` and stop. + #### Step 3.5.R — Maintainer change-request response (Track C) Reached from Step 3.5 gate 0. Goal: when an **eligible human reviewer** (Hard-Rule @@ -1395,7 +1681,19 @@ Filed by [`ci-status-fix-net11`](https://github.com/dotnet/maui/blob/main/.githu ### Step 8 — Per-issue tally + end-of-run summary -Per issue, append one outcome line to `/tmp/gh-aw/agent/coverage.txt`: +Per issue, append **one** outcome line to `/tmp/gh-aw/agent/coverage.txt` — the +terminal outcome for the cycle. When one cycle produces a chained pair — a PR gets a +same-cycle precursor line and **then** a Step 3.6 readiness line (`marked-ready` on the +draft→ready flip, or `already-ready` on a later steady-state sweep of an already-ready +PR) — record ONLY the terminal Step 3.6 readiness line: it supersedes ANY same-cycle +non-readiness precursor for that PR, so an aggregator keying on one-line-per-issue never +double-counts. The precursor may be a Step 3 green line (`surfaced-green` on the first ✅, +or `already-surfaced` when it already exists) OR — when an unrelated-flake red and a +target-green coincide — a Step 4 `annotated-flake` line; either way the readiness line is +terminal, and the superseded precursor's signal survives in the PR's 🎯/♻️ comment so no +information is lost. This covers the flip cycle (`surfaced-green` → `marked-ready`), the +post-flip steady state (`already-surfaced` → `already-ready`), and the flake-coincident +flip (`annotated-flake` → `marked-ready`): ``` # net11.0 attempt- @@ -1406,8 +1704,14 @@ Per issue, append one outcome line to `/tmp/gh-aw/agent/coverage.txt`: `advance-PR # attempt /10` (ADVANCE mode — new commit pushed to the existing PR), `surfaced-green PR #` (fix's own CI went green; commented for review, did not advance), `annotated-flake PR #` (red was unrelated flake; -commented, attempt NOT burned), `waiting PR #` (CI not settled yet), -`dry-run: would-`, `skipped: `. (The +commented, attempt NOT burned), `marked-ready PR #` (the specific fixed test +was confirmed green in the PR's own CI; draft flipped to ready for review), +`already-surfaced PR #` (the fix's ✅ green comment already existed this sweep; +re-post suppressed — superseded by the Step 3.6 readiness line when the PR also flips +ready that cycle), `already-ready PR #` (terminal steady state — the PR was flipped +ready on a prior cycle and remains green on an unchanged head; no action taken, +supersedes `already-surfaced`), `waiting PR #` (CI not settled yet), +`dry-run: would-`, `skipped: `. (The `needs-human-PR` outcome is reserved for the deferred hand-off — Step 6 currently records a skip instead, so it is not emitted.) diff --git a/.github/workflows/ci-status-fix.lock.yml b/.github/workflows/ci-status-fix.lock.yml index 93dc5969cb0d..65511f2b3259 100644 --- a/.github/workflows/ci-status-fix.lock.yml +++ b/.github/workflows/ci-status-fix.lock.yml @@ -1,4 +1,4 @@ -# gh-aw-metadata: {"schema_version":"v4","frontmatter_hash":"f014161967589ebcaf1ac5ec6f3c88ee5a4388d28b55a07dc36c45b7b2aebab6","body_hash":"86c6b2d4c3df13da14c6a3c8b211381fcc9f97bb1951ff7b80588a2c5338e00c","compiler_version":"v0.80.9","strict":true,"agent_id":"copilot","agent_model":"claude-opus-4.8","engine_versions":{"copilot":"1.0.63"}} +# gh-aw-metadata: {"schema_version":"v4","frontmatter_hash":"b0ebf8a2f179900cfe3638e994b27a43cec52bdbdd2331dc1bd4d65762fa2d6b","body_hash":"1825e2b1f7c7bd88241bf37580655489360f9bebc806edd00837a52ce4ad83a0","compiler_version":"v0.80.9","strict":true,"agent_id":"copilot","agent_model":"claude-opus-4.8","engine_versions":{"copilot":"1.0.63"}} # gh-aw-manifest: {"version":1,"secrets":["COPILOT_PAT_0","COPILOT_PAT_1","COPILOT_PAT_2","COPILOT_PAT_3","COPILOT_PAT_4","COPILOT_PAT_5","COPILOT_PAT_6","COPILOT_PAT_7","COPILOT_PAT_8","COPILOT_PAT_9","GH_AW_CI_TRIGGER_TOKEN","GH_AW_GITHUB_MCP_SERVER_TOKEN","GH_AW_GITHUB_TOKEN","GITHUB_TOKEN"],"actions":[{"repo":"actions/checkout","sha":"34e114876b0b11c390a56381ad16ebd13914f8d5","version":"v4"},{"repo":"actions/checkout","sha":"9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0","version":"v7.0.0"},{"repo":"actions/checkout","sha":"de0fac2e4500dabe0009e67214ff5f5447ce83dd","version":"v6.0.2"},{"repo":"actions/download-artifact","sha":"3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c","version":"v8.0.1"},{"repo":"actions/github-script","sha":"3a2844b7e9c422d3c10d287c895573f7108da1b3","version":"v9.0.0"},{"repo":"actions/setup-node","sha":"48b55a011bda9f5d6aeb4c2d9c7362e8dae4041e","version":"v6.4.0"},{"repo":"actions/upload-artifact","sha":"043fb46d1a93c77aae656e7c1c64a875d1fc6a0a","version":"v7.0.1"},{"repo":"github/gh-aw-actions/setup","sha":"8c7d04ebf1ece56cd381446125da3e0f6896294a","version":"v0.80.9"}],"containers":[{"image":"ghcr.io/github/gh-aw-firewall/agent:0.27.7","digest":"sha256:aae231e4635c8999d039c132f1602d3df850fe9b84a00aa2b5ac981179b5661c","pinned_image":"ghcr.io/github/gh-aw-firewall/agent:0.27.7@sha256:aae231e4635c8999d039c132f1602d3df850fe9b84a00aa2b5ac981179b5661c"},{"image":"ghcr.io/github/gh-aw-firewall/api-proxy:0.27.7","digest":"sha256:009caf2e3d88fa77b64e9a03a95a228fc58db0f1701c6d324b29ba5a3c7c79b6","pinned_image":"ghcr.io/github/gh-aw-firewall/api-proxy:0.27.7@sha256:009caf2e3d88fa77b64e9a03a95a228fc58db0f1701c6d324b29ba5a3c7c79b6"},{"image":"ghcr.io/github/gh-aw-firewall/squid:0.27.7","digest":"sha256:deb1d4e19de62d51cee0508057a596a19315c3423ada4d675cad136dc8037c96","pinned_image":"ghcr.io/github/gh-aw-firewall/squid:0.27.7@sha256:deb1d4e19de62d51cee0508057a596a19315c3423ada4d675cad136dc8037c96"},{"image":"ghcr.io/github/gh-aw-mcpg:v0.3.27","digest":"sha256:fe984bddde4ec05d756d9043edb0a32912e6b7b72f6a121b1082f29221421cc7","pinned_image":"ghcr.io/github/gh-aw-mcpg:v0.3.27@sha256:fe984bddde4ec05d756d9043edb0a32912e6b7b72f6a121b1082f29221421cc7"},{"image":"ghcr.io/github/gh-aw-node","digest":"sha256:529d02eb970b1161aa25c593a9c3df57fdfad5a8add328cb3b6eccef66f3183b","pinned_image":"ghcr.io/github/gh-aw-node@sha256:529d02eb970b1161aa25c593a9c3df57fdfad5a8add328cb3b6eccef66f3183b"},{"image":"ghcr.io/github/github-mcp-server:v1.4.0","digest":"sha256:2afb26356481d1a350e14544a6e160f7f7ec1561a1ea309b823665abf0309036","pinned_image":"ghcr.io/github/github-mcp-server:v1.4.0@sha256:2afb26356481d1a350e14544a6e160f7f7ec1561a1ea309b823665abf0309036"}]} # This file was automatically generated by gh-aw (v0.80.9). DO NOT EDIT. To debug this workflow, load the skill at https://github.com/github/gh-aw/blob/main/debug.md # @@ -37,7 +37,9 @@ # humans (the open tracking issue is the hand-off surface; a dedicated # [ci-fix][needs-human] PR is planned but currently deferred — see Step 6). CI on # the PR is kicked by a human `/azp run` for now; the loop watches, classifies, -# and re-fixes autonomously between kicks. +# and re-fixes autonomously between kicks. When the SPECIFIC test a PR fixed is +# confirmed green in that PR's own CI, the loop marks the draft PR ready for review +# (a state transition only — it never approves or merges). # Never mutes tests, but # de-flakes genuinely flaky ones (deterministic synchronization, no retries / # timeout bumps). Always skips visual-regression / screenshot issues. @@ -273,24 +275,24 @@ jobs: run: | bash "${RUNNER_TEMP}/gh-aw/actions/create_prompt_first.sh" { - cat << 'GH_AW_PROMPT_25cf75111b670167_EOF' + cat << 'GH_AW_PROMPT_2ac1d0b4083428f8_EOF' - GH_AW_PROMPT_25cf75111b670167_EOF + GH_AW_PROMPT_2ac1d0b4083428f8_EOF cat "${RUNNER_TEMP}/gh-aw/prompts/xpia.md" cat "${RUNNER_TEMP}/gh-aw/prompts/temp_folder_prompt.md" cat "${RUNNER_TEMP}/gh-aw/prompts/markdown.md" cat "${RUNNER_TEMP}/gh-aw/prompts/safe_outputs_prompt.md" - cat << 'GH_AW_PROMPT_25cf75111b670167_EOF' + cat << 'GH_AW_PROMPT_2ac1d0b4083428f8_EOF' - Tools: add_comment(max:3), create_pull_request(max:3), update_pull_request(max:3), push_to_pull_request_branch(max:3), missing_tool, missing_data, noop - GH_AW_PROMPT_25cf75111b670167_EOF + Tools: add_comment(max:6), create_pull_request(max:3), update_pull_request(max:3), mark_pull_request_as_ready_for_review(max:3), add_labels(max:3), push_to_pull_request_branch(max:3), missing_tool, missing_data, noop + GH_AW_PROMPT_2ac1d0b4083428f8_EOF cat "${RUNNER_TEMP}/gh-aw/prompts/safe_outputs_create_pull_request.md" cat "${RUNNER_TEMP}/gh-aw/prompts/safe_outputs_push_to_pr_branch.md" - cat << 'GH_AW_PROMPT_25cf75111b670167_EOF' + cat << 'GH_AW_PROMPT_2ac1d0b4083428f8_EOF' - GH_AW_PROMPT_25cf75111b670167_EOF + GH_AW_PROMPT_2ac1d0b4083428f8_EOF cat "${RUNNER_TEMP}/gh-aw/prompts/mcp_cli_tools_prompt.md" - cat << 'GH_AW_PROMPT_25cf75111b670167_EOF' + cat << 'GH_AW_PROMPT_2ac1d0b4083428f8_EOF' The following GitHub context information is available for this workflow: {{#if github.actor}} @@ -332,12 +334,12 @@ jobs: stop immediately and report the limitation rather than spending turns trying to work around it. - GH_AW_PROMPT_25cf75111b670167_EOF + GH_AW_PROMPT_2ac1d0b4083428f8_EOF cat "${RUNNER_TEMP}/gh-aw/prompts/github_mcp_tools_with_safeoutputs_prompt.md" - cat << 'GH_AW_PROMPT_25cf75111b670167_EOF' + cat << 'GH_AW_PROMPT_2ac1d0b4083428f8_EOF' {{#runtime-import .github/workflows/ci-status-fix.md}} - GH_AW_PROMPT_25cf75111b670167_EOF + GH_AW_PROMPT_2ac1d0b4083428f8_EOF } > "$GH_AW_PROMPT" - name: Interpolate variables and render templates uses: actions/github-script@3a2844b7e9c422d3c10d287c895573f7108da1b3 # v9.0.0 @@ -554,16 +556,18 @@ jobs: mkdir -p "${RUNNER_TEMP}/gh-aw/safeoutputs" mkdir -p /tmp/gh-aw/safeoutputs mkdir -p /tmp/gh-aw/mcp-logs/safeoutputs - cat > "${RUNNER_TEMP}/gh-aw/safeoutputs/config.json" << 'GH_AW_SAFE_OUTPUTS_CONFIG_fa2a69fad5fd0485_EOF' - {"add_comment":{"discussions":false,"max":3,"required_labels":["agentic-workflows"],"required_title_prefix":"[ci-fix] ","target":"*"},"create_pull_request":{"allowed_base_branches":["main"],"allowed_branches":["ci-fix/**"],"allowed_files":["src/Core/**","src/Controls/**","src/Essentials/**","src/BlazorWebView/**","src/TestUtils/**","src/Templates/**","**/PublicAPI.Unshipped.txt"],"base_branch":"main","draft":true,"labels":["agentic-workflows"],"max":3,"max_patch_files":100,"max_patch_size":4096,"protect_top_level_dot_folders":true,"protected_files":["package.json","bun.lockb","bunfig.toml","deno.json","deno.jsonc","deno.lock","global.json","NuGet.Config","Directory.Packages.props","mix.exs","mix.lock","go.mod","go.sum","stack.yaml","stack.yaml.lock","pom.xml","build.gradle","build.gradle.kts","settings.gradle","settings.gradle.kts","gradle.properties","package-lock.json","yarn.lock","pnpm-lock.yaml","npm-shrinkwrap.json","requirements.txt","Pipfile","Pipfile.lock","pyproject.toml","setup.py","setup.cfg","Gemfile","Gemfile.lock","uv.lock","CODEOWNERS","DESIGN.md","README.md","CONTRIBUTING.md","CHANGELOG.md","SECURITY.md","CODE_OF_CONDUCT.md","AGENTS.md","CLAUDE.md","GEMINI.md"],"protected_files_policy":"request_review","title_prefix":"[ci-fix] "},"create_report_incomplete_issue":{},"missing_data":{},"missing_tool":{},"noop":{"max":1,"report-as-issue":"true"},"push_to_pull_request_branch":{"allowed_files":["src/Core/**","src/Controls/**","src/Essentials/**","src/BlazorWebView/**","src/TestUtils/**","src/Templates/**","**/PublicAPI.Unshipped.txt"],"if_no_changes":"warn","max":3,"max_patch_size":4096,"protect_top_level_dot_folders":true,"protected_files":["package.json","bun.lockb","bunfig.toml","deno.json","deno.jsonc","deno.lock","global.json","NuGet.Config","Directory.Packages.props","mix.exs","mix.lock","go.mod","go.sum","stack.yaml","stack.yaml.lock","pom.xml","build.gradle","build.gradle.kts","settings.gradle","settings.gradle.kts","gradle.properties","package-lock.json","yarn.lock","pnpm-lock.yaml","npm-shrinkwrap.json","requirements.txt","Pipfile","Pipfile.lock","pyproject.toml","setup.py","setup.cfg","Gemfile","Gemfile.lock","uv.lock","CODEOWNERS","DESIGN.md","README.md","CONTRIBUTING.md","CHANGELOG.md","SECURITY.md","CODE_OF_CONDUCT.md","AGENTS.md","CLAUDE.md","GEMINI.md"],"required_labels":["agentic-workflows"],"target":"*","title_prefix":"[ci-fix] "},"report_incomplete":{},"update_pull_request":{"allow_body":true,"allow_title":false,"max":3,"target":"*","update_branch":false}} - GH_AW_SAFE_OUTPUTS_CONFIG_fa2a69fad5fd0485_EOF + cat > "${RUNNER_TEMP}/gh-aw/safeoutputs/config.json" << 'GH_AW_SAFE_OUTPUTS_CONFIG_0644bf1434ca0071_EOF' + {"add_comment":{"discussions":false,"max":6,"required_labels":["agentic-workflows"],"required_title_prefix":"[ci-fix] ","target":"*"},"add_labels":{"allowed":["p/0"],"max":3,"required_labels":["agentic-workflows"],"required_title_prefix":"[ci-fix] ","target":"*"},"create_pull_request":{"allowed_base_branches":["main"],"allowed_branches":["ci-fix/**"],"allowed_files":["src/Core/**","src/Controls/**","src/Essentials/**","src/BlazorWebView/**","src/TestUtils/**","src/Templates/**","**/PublicAPI.Unshipped.txt"],"base_branch":"main","draft":true,"labels":["agentic-workflows"],"max":3,"max_patch_files":100,"max_patch_size":4096,"protect_top_level_dot_folders":true,"protected_files":["package.json","bun.lockb","bunfig.toml","deno.json","deno.jsonc","deno.lock","global.json","NuGet.Config","Directory.Packages.props","mix.exs","mix.lock","go.mod","go.sum","stack.yaml","stack.yaml.lock","pom.xml","build.gradle","build.gradle.kts","settings.gradle","settings.gradle.kts","gradle.properties","package-lock.json","yarn.lock","pnpm-lock.yaml","npm-shrinkwrap.json","requirements.txt","Pipfile","Pipfile.lock","pyproject.toml","setup.py","setup.cfg","Gemfile","Gemfile.lock","uv.lock","CODEOWNERS","DESIGN.md","README.md","CONTRIBUTING.md","CHANGELOG.md","SECURITY.md","CODE_OF_CONDUCT.md","AGENTS.md","CLAUDE.md","GEMINI.md"],"protected_files_policy":"request_review","title_prefix":"[ci-fix] "},"create_report_incomplete_issue":{},"mark_pull_request_as_ready_for_review":{"max":3,"required_labels":["agentic-workflows"],"required_title_prefix":"[ci-fix] ","target":"*"},"missing_data":{},"missing_tool":{},"noop":{"max":1,"report-as-issue":"true"},"push_to_pull_request_branch":{"allowed_files":["src/Core/**","src/Controls/**","src/Essentials/**","src/BlazorWebView/**","src/TestUtils/**","src/Templates/**","**/PublicAPI.Unshipped.txt"],"if_no_changes":"warn","max":3,"max_patch_size":4096,"protect_top_level_dot_folders":true,"protected_files":["package.json","bun.lockb","bunfig.toml","deno.json","deno.jsonc","deno.lock","global.json","NuGet.Config","Directory.Packages.props","mix.exs","mix.lock","go.mod","go.sum","stack.yaml","stack.yaml.lock","pom.xml","build.gradle","build.gradle.kts","settings.gradle","settings.gradle.kts","gradle.properties","package-lock.json","yarn.lock","pnpm-lock.yaml","npm-shrinkwrap.json","requirements.txt","Pipfile","Pipfile.lock","pyproject.toml","setup.py","setup.cfg","Gemfile","Gemfile.lock","uv.lock","CODEOWNERS","DESIGN.md","README.md","CONTRIBUTING.md","CHANGELOG.md","SECURITY.md","CODE_OF_CONDUCT.md","AGENTS.md","CLAUDE.md","GEMINI.md"],"required_labels":["agentic-workflows"],"target":"*","title_prefix":"[ci-fix] "},"report_incomplete":{},"update_pull_request":{"allow_body":true,"allow_title":false,"max":3,"target":"*","update_branch":false}} + GH_AW_SAFE_OUTPUTS_CONFIG_0644bf1434ca0071_EOF - name: Generate Safe Outputs Tools env: GH_AW_TOOLS_META_JSON: | { "description_suffixes": { - "add_comment": " CONSTRAINTS: Maximum 3 comment(s) can be added. Target: *. Supports reply_to_id for discussion threading.", + "add_comment": " CONSTRAINTS: Maximum 6 comment(s) can be added. Target: *. Supports reply_to_id for discussion threading.", + "add_labels": " CONSTRAINTS: Maximum 3 label(s) can be added. Only these labels are allowed: [\"p/0\"]. Target: *.", "create_pull_request": " CONSTRAINTS: Maximum 3 pull request(s) can be created. Title will be prefixed with \"[ci-fix] \". Labels [\"agentic-workflows\"] will be automatically added. Only these labels are allowed: [\"agentic-workflows\"]. PRs will be created as drafts.", + "mark_pull_request_as_ready_for_review": " CONSTRAINTS: Maximum 3 pull request(s) can be marked as ready for review.", "push_to_pull_request_branch": " CONSTRAINTS: Maximum 3 push(es) can be made. The target pull request title must start with \"[ci-fix] \".", "update_pull_request": " CONSTRAINTS: Maximum 3 pull request(s) can be updated. Target: *." }, @@ -594,6 +598,25 @@ jobs: } } }, + "add_labels": { + "defaultMax": 5, + "fields": { + "item_number": { + "issueNumberOrTemporaryId": true + }, + "labels": { + "required": true, + "type": "array", + "itemType": "string", + "itemSanitize": true, + "itemMaxLength": 128 + }, + "repo": { + "type": "string", + "maxLength": 256 + } + } + }, "create_pull_request": { "defaultMax": 1, "fields": { @@ -635,6 +658,24 @@ jobs: } } }, + "mark_pull_request_as_ready_for_review": { + "defaultMax": 1, + "fields": { + "pull_request_number": { + "issueOrPRNumber": true + }, + "reason": { + "required": true, + "type": "string", + "sanitize": true, + "maxLength": 65000 + }, + "repo": { + "type": "string", + "maxLength": 256 + } + } + }, "missing_data": { "defaultMax": 20, "fields": { @@ -1494,7 +1535,7 @@ jobs: uses: actions/github-script@3a2844b7e9c422d3c10d287c895573f7108da1b3 # v9.0.0 env: WORKFLOW_NAME: "CI Failure Fixer (main)" - WORKFLOW_DESCRIPTION: "Periodic pass over open ci-scan tracking issues filed by the main-branch CI\nfailure scanner (.github/workflows/ci-status-main.md). This workflow targets\nthe `main` branch EXCLUSIVELY: it processes only issues labelled ci-scan and\nopens every PR against main. (The net11.0 branch is handled by the parallel\n.github/workflows/ci-status-fix-net11.md — the two are split because gh-aw can\nonly transport a fix relative to ONE static base branch per workflow, and the\nmain↔net11.0 divergence exceeds gh-aw's 10 MB transport-patch cap.) The fixer\nopens ONE draft [ci-fix] PR per actionable issue against main, then WATCHES\nthat PR's own CI on later runs: when the fix's CI comes back red and the red is\ncaused by the fix itself, it pushes a fresh follow-up fix onto the SAME PR\nbranch (never a second PR) — up to 10 attempts — then stops and defers to\nhumans (the open tracking issue is the hand-off surface; a dedicated\n[ci-fix][needs-human] PR is planned but currently deferred — see Step 6). CI on\nthe PR is kicked by a human `/azp run` for now; the loop watches, classifies,\nand re-fixes autonomously between kicks.\nNever mutes tests, but\nde-flakes genuinely flaky ones (deterministic synchronization, no retries /\ntimeout bumps). Always skips visual-regression / screenshot issues." + WORKFLOW_DESCRIPTION: "Periodic pass over open ci-scan tracking issues filed by the main-branch CI\nfailure scanner (.github/workflows/ci-status-main.md). This workflow targets\nthe `main` branch EXCLUSIVELY: it processes only issues labelled ci-scan and\nopens every PR against main. (The net11.0 branch is handled by the parallel\n.github/workflows/ci-status-fix-net11.md — the two are split because gh-aw can\nonly transport a fix relative to ONE static base branch per workflow, and the\nmain↔net11.0 divergence exceeds gh-aw's 10 MB transport-patch cap.) The fixer\nopens ONE draft [ci-fix] PR per actionable issue against main, then WATCHES\nthat PR's own CI on later runs: when the fix's CI comes back red and the red is\ncaused by the fix itself, it pushes a fresh follow-up fix onto the SAME PR\nbranch (never a second PR) — up to 10 attempts — then stops and defers to\nhumans (the open tracking issue is the hand-off surface; a dedicated\n[ci-fix][needs-human] PR is planned but currently deferred — see Step 6). CI on\nthe PR is kicked by a human `/azp run` for now; the loop watches, classifies,\nand re-fixes autonomously between kicks. When the SPECIFIC test a PR fixed is\nconfirmed green in that PR's own CI, the loop marks the draft PR ready for review\n(a state transition only — it never approves or merges).\nNever mutes tests, but\nde-flakes genuinely flaky ones (deterministic synchronization, no retries /\ntimeout bumps). Always skips visual-regression / screenshot issues." HAS_PATCH: ${{ needs.agent.outputs.has_patch }} with: script: | @@ -1910,7 +1951,7 @@ jobs: GH_AW_ALLOWED_DOMAINS: "*.blob.core.windows.net,*.githubusercontent.com,api.business.githubcopilot.com,api.enterprise.githubcopilot.com,api.github.com,api.githubcopilot.com,api.individual.githubcopilot.com,api.snapcraft.io,archive.ubuntu.com,azure.archive.ubuntu.com,codeload.github.com,crl.geotrust.com,crl.globalsign.com,crl.identrust.com,crl.sectigo.com,crl.thawte.com,crl.usertrust.com,crl.verisign.com,crl3.digicert.com,crl4.digicert.com,crls.ssl.com,dev.azure.com,docs.github.com,github-cloud.githubusercontent.com,github-cloud.s3.amazonaws.com,github.blog,github.com,github.githubassets.com,helix.dot.net,host.docker.internal,json-schema.org,json.schemastore.org,keyserver.ubuntu.com,lfs.github.com,objects.githubusercontent.com,ocsp.digicert.com,ocsp.geotrust.com,ocsp.globalsign.com,ocsp.identrust.com,ocsp.sectigo.com,ocsp.ssl.com,ocsp.thawte.com,ocsp.usertrust.com,ocsp.verisign.com,packagecloud.io,packages.cloud.google.com,packages.microsoft.com,patch-diff.githubusercontent.com,ppa.launchpad.net,raw.githubusercontent.com,registry.npmjs.org,s.symcb.com,s.symcd.com,security.ubuntu.com,telemetry.enterprise.githubcopilot.com,ts-crl.ws.symantec.com,ts-ocsp.ws.symantec.com,www.googleapis.com" GITHUB_SERVER_URL: ${{ github.server_url }} GITHUB_API_URL: ${{ github.api_url }} - GH_AW_SAFE_OUTPUTS_HANDLER_CONFIG: "{\"add_comment\":{\"discussions\":false,\"max\":3,\"required_labels\":[\"agentic-workflows\"],\"required_title_prefix\":\"[ci-fix] \",\"target\":\"*\"},\"create_pull_request\":{\"allowed_base_branches\":[\"main\"],\"allowed_branches\":[\"ci-fix/**\"],\"allowed_files\":[\"src/Core/**\",\"src/Controls/**\",\"src/Essentials/**\",\"src/BlazorWebView/**\",\"src/TestUtils/**\",\"src/Templates/**\",\"**/PublicAPI.Unshipped.txt\"],\"base_branch\":\"main\",\"draft\":true,\"labels\":[\"agentic-workflows\"],\"max\":3,\"max_patch_files\":100,\"max_patch_size\":4096,\"protect_top_level_dot_folders\":true,\"protected_files\":[\"package.json\",\"bun.lockb\",\"bunfig.toml\",\"deno.json\",\"deno.jsonc\",\"deno.lock\",\"global.json\",\"NuGet.Config\",\"Directory.Packages.props\",\"mix.exs\",\"mix.lock\",\"go.mod\",\"go.sum\",\"stack.yaml\",\"stack.yaml.lock\",\"pom.xml\",\"build.gradle\",\"build.gradle.kts\",\"settings.gradle\",\"settings.gradle.kts\",\"gradle.properties\",\"package-lock.json\",\"yarn.lock\",\"pnpm-lock.yaml\",\"npm-shrinkwrap.json\",\"requirements.txt\",\"Pipfile\",\"Pipfile.lock\",\"pyproject.toml\",\"setup.py\",\"setup.cfg\",\"Gemfile\",\"Gemfile.lock\",\"uv.lock\",\"CODEOWNERS\",\"DESIGN.md\",\"README.md\",\"CONTRIBUTING.md\",\"CHANGELOG.md\",\"SECURITY.md\",\"CODE_OF_CONDUCT.md\",\"AGENTS.md\",\"CLAUDE.md\",\"GEMINI.md\"],\"protected_files_policy\":\"request_review\",\"title_prefix\":\"[ci-fix] \"},\"create_report_incomplete_issue\":{},\"missing_data\":{},\"missing_tool\":{},\"noop\":{\"max\":1,\"report-as-issue\":\"true\"},\"push_to_pull_request_branch\":{\"allowed_files\":[\"src/Core/**\",\"src/Controls/**\",\"src/Essentials/**\",\"src/BlazorWebView/**\",\"src/TestUtils/**\",\"src/Templates/**\",\"**/PublicAPI.Unshipped.txt\"],\"if_no_changes\":\"warn\",\"max\":3,\"max_patch_size\":4096,\"protect_top_level_dot_folders\":true,\"protected_files\":[\"package.json\",\"bun.lockb\",\"bunfig.toml\",\"deno.json\",\"deno.jsonc\",\"deno.lock\",\"global.json\",\"NuGet.Config\",\"Directory.Packages.props\",\"mix.exs\",\"mix.lock\",\"go.mod\",\"go.sum\",\"stack.yaml\",\"stack.yaml.lock\",\"pom.xml\",\"build.gradle\",\"build.gradle.kts\",\"settings.gradle\",\"settings.gradle.kts\",\"gradle.properties\",\"package-lock.json\",\"yarn.lock\",\"pnpm-lock.yaml\",\"npm-shrinkwrap.json\",\"requirements.txt\",\"Pipfile\",\"Pipfile.lock\",\"pyproject.toml\",\"setup.py\",\"setup.cfg\",\"Gemfile\",\"Gemfile.lock\",\"uv.lock\",\"CODEOWNERS\",\"DESIGN.md\",\"README.md\",\"CONTRIBUTING.md\",\"CHANGELOG.md\",\"SECURITY.md\",\"CODE_OF_CONDUCT.md\",\"AGENTS.md\",\"CLAUDE.md\",\"GEMINI.md\"],\"required_labels\":[\"agentic-workflows\"],\"target\":\"*\",\"title_prefix\":\"[ci-fix] \"},\"report_incomplete\":{},\"update_pull_request\":{\"allow_body\":true,\"allow_title\":false,\"max\":3,\"target\":\"*\",\"update_branch\":false}}" + GH_AW_SAFE_OUTPUTS_HANDLER_CONFIG: "{\"add_comment\":{\"discussions\":false,\"max\":6,\"required_labels\":[\"agentic-workflows\"],\"required_title_prefix\":\"[ci-fix] \",\"target\":\"*\"},\"add_labels\":{\"allowed\":[\"p/0\"],\"max\":3,\"required_labels\":[\"agentic-workflows\"],\"required_title_prefix\":\"[ci-fix] \",\"target\":\"*\"},\"create_pull_request\":{\"allowed_base_branches\":[\"main\"],\"allowed_branches\":[\"ci-fix/**\"],\"allowed_files\":[\"src/Core/**\",\"src/Controls/**\",\"src/Essentials/**\",\"src/BlazorWebView/**\",\"src/TestUtils/**\",\"src/Templates/**\",\"**/PublicAPI.Unshipped.txt\"],\"base_branch\":\"main\",\"draft\":true,\"labels\":[\"agentic-workflows\"],\"max\":3,\"max_patch_files\":100,\"max_patch_size\":4096,\"protect_top_level_dot_folders\":true,\"protected_files\":[\"package.json\",\"bun.lockb\",\"bunfig.toml\",\"deno.json\",\"deno.jsonc\",\"deno.lock\",\"global.json\",\"NuGet.Config\",\"Directory.Packages.props\",\"mix.exs\",\"mix.lock\",\"go.mod\",\"go.sum\",\"stack.yaml\",\"stack.yaml.lock\",\"pom.xml\",\"build.gradle\",\"build.gradle.kts\",\"settings.gradle\",\"settings.gradle.kts\",\"gradle.properties\",\"package-lock.json\",\"yarn.lock\",\"pnpm-lock.yaml\",\"npm-shrinkwrap.json\",\"requirements.txt\",\"Pipfile\",\"Pipfile.lock\",\"pyproject.toml\",\"setup.py\",\"setup.cfg\",\"Gemfile\",\"Gemfile.lock\",\"uv.lock\",\"CODEOWNERS\",\"DESIGN.md\",\"README.md\",\"CONTRIBUTING.md\",\"CHANGELOG.md\",\"SECURITY.md\",\"CODE_OF_CONDUCT.md\",\"AGENTS.md\",\"CLAUDE.md\",\"GEMINI.md\"],\"protected_files_policy\":\"request_review\",\"title_prefix\":\"[ci-fix] \"},\"create_report_incomplete_issue\":{},\"mark_pull_request_as_ready_for_review\":{\"max\":3,\"required_labels\":[\"agentic-workflows\"],\"required_title_prefix\":\"[ci-fix] \",\"target\":\"*\"},\"missing_data\":{},\"missing_tool\":{},\"noop\":{\"max\":1,\"report-as-issue\":\"true\"},\"push_to_pull_request_branch\":{\"allowed_files\":[\"src/Core/**\",\"src/Controls/**\",\"src/Essentials/**\",\"src/BlazorWebView/**\",\"src/TestUtils/**\",\"src/Templates/**\",\"**/PublicAPI.Unshipped.txt\"],\"if_no_changes\":\"warn\",\"max\":3,\"max_patch_size\":4096,\"protect_top_level_dot_folders\":true,\"protected_files\":[\"package.json\",\"bun.lockb\",\"bunfig.toml\",\"deno.json\",\"deno.jsonc\",\"deno.lock\",\"global.json\",\"NuGet.Config\",\"Directory.Packages.props\",\"mix.exs\",\"mix.lock\",\"go.mod\",\"go.sum\",\"stack.yaml\",\"stack.yaml.lock\",\"pom.xml\",\"build.gradle\",\"build.gradle.kts\",\"settings.gradle\",\"settings.gradle.kts\",\"gradle.properties\",\"package-lock.json\",\"yarn.lock\",\"pnpm-lock.yaml\",\"npm-shrinkwrap.json\",\"requirements.txt\",\"Pipfile\",\"Pipfile.lock\",\"pyproject.toml\",\"setup.py\",\"setup.cfg\",\"Gemfile\",\"Gemfile.lock\",\"uv.lock\",\"CODEOWNERS\",\"DESIGN.md\",\"README.md\",\"CONTRIBUTING.md\",\"CHANGELOG.md\",\"SECURITY.md\",\"CODE_OF_CONDUCT.md\",\"AGENTS.md\",\"CLAUDE.md\",\"GEMINI.md\"],\"required_labels\":[\"agentic-workflows\"],\"target\":\"*\",\"title_prefix\":\"[ci-fix] \"},\"report_incomplete\":{},\"update_pull_request\":{\"allow_body\":true,\"allow_title\":false,\"max\":3,\"target\":\"*\",\"update_branch\":false}}" GH_AW_CI_TRIGGER_TOKEN: ${{ secrets.GH_AW_CI_TRIGGER_TOKEN }} with: github-token: ${{ secrets.GH_AW_GITHUB_TOKEN || secrets.GITHUB_TOKEN }} diff --git a/.github/workflows/ci-status-fix.md b/.github/workflows/ci-status-fix.md index d3227e25170c..9c2f46054029 100644 --- a/.github/workflows/ci-status-fix.md +++ b/.github/workflows/ci-status-fix.md @@ -15,7 +15,9 @@ description: | humans (the open tracking issue is the hand-off surface; a dedicated [ci-fix][needs-human] PR is planned but currently deferred — see Step 6). CI on the PR is kicked by a human `/azp run` for now; the loop watches, classifies, - and re-fixes autonomously between kicks. + and re-fixes autonomously between kicks. When the SPECIFIC test a PR fixed is + confirmed green in that PR's own CI, the loop marks the draft PR ready for review + (a state transition only — it never approves or merges). Never mutes tests, but de-flakes genuinely flaky ones (deterministic synchronization, no retries / timeout bumps). Always skips visual-regression / screenshot issues. @@ -239,8 +241,15 @@ safe-outputs: # PER-RUN total (not per-PR): a sweep may surface several green PRs and/or # annotate several flaky reds in one run. At max:1 all but the first comment # were silently dropped, starving the workflow's #1 value (surfacing green PRs - # for review). Raised to 3 to match the per-run throughput of the other outputs. - max: 3 + # for review). Sized to 6 (not 3) because a single green DRAFT PR that gets + # flipped ready in one sweep spends TWO comment slots — the Step 3 ✅ surface + # comment (which still names the /azp-gated legs a human must kick, so it is NOT + # redundant with 🎯) AND the Step 3.6 T3 🎯 readiness comment. At max:3 the shared + # bucket drained after ~1 draft flip and T3's atomicity pre-check then deferred + # every further mark-ready even while the mark-ready/add_labels buckets (also 3) + # sat idle; 6 lets ~3 draft flips (2 comments each) land per sweep, so the + # mark-ready:3 / add_labels:3 caps are actually reachable. + max: 6 target: "*" # Hard constraint (defense-in-depth): only comment on THIS workflow's own # ci-fix PRs — [ci-fix] title prefix AND agentic-workflows label. Mirrors the @@ -261,13 +270,51 @@ safe-outputs: # never the title, so disable title rewrites — this compiles to allow_title:false # and removes any ability to retitle an arbitrary PR. title: false - # NOTE: gh-aw v0.79.8 does NOT emit required-title-prefix/required-labels into + # NOTE: gh-aw v0.80.9 (the pinned compiler) does NOT emit required-title-prefix/required-labels into # the compiled config for update-pull-request (verified against the lock — it # silently drops them, unlike add-comment / push-to-pull-request-branch which # honor them). So which-PR scoping here relies on prompt Hard-Rule 6 + # min-integrity:approved; the residual is body-marker edits only (no code, no # merge, no close), capped at max:3. Revisit required-* if a future gh-aw # compiler emits them for this output. + mark-pull-request-as-ready-for-review: + # Flip a validated draft [ci-fix] PR to ready-for-review once the SPECIFIC test + # this PR was opened to fix is confirmed green in the PR's OWN CI (Step 3.6) — + # even when unrelated legs are red. The workflow is schedule/dispatch-triggered + # (no triggering-PR context), so target "*" lets the agent name the PR number it + # validated. This is a state transition ONLY: it never approves and never merges — + # a human still reviews and merges. + # PER-RUN total (not per-PR): a sweep may validate several PRs' target tests green. + max: 3 + target: "*" + # Hard constraint (defense-in-depth): only ever un-draft THIS workflow's own + # ci-fix PRs — [ci-fix] title prefix AND agentic-workflows label — so a confused + # or prompt-injected agent cannot mark an arbitrary PR ready. (If the v0.80.9 + # compiler silently drops these — as documented for update-pull-request above — + # the Step 3.6 preconditions + min-integrity:approved are the compensating scope + # controls; verify against the lock after compiling.) + required-title-prefix: "[ci-fix] " + required-labels: [agentic-workflows] + add-labels: + # Apply the p/0 priority label to a [ci-fix] PR at the exact moment the loop flips it + # from draft to ready-for-review (Step 3.6 T3) — so a validated, review-ready fix lands + # in the team's p/0 triage queue instead of sitting unseen in the draft backlog. Paired + # 1:1 with mark-pull-request-as-ready-for-review; target "*" lets the agent name the PR + # it just validated. + # allowed = HARD allowlist: the agent may ONLY ever add p/0, nothing else. This caps the + # blast radius of a confused/prompt-injected agent to exactly one benign priority label — + # it can never apply a downstream-triggering or destructive label. + allowed: [p/0] + # PER-RUN total (not per-PR): a sweep may mark several PRs ready in one run; each adds + # one label, so this matches the mark-ready per-run cap (3). + max: 3 + target: "*" + # Hard constraint (defense-in-depth): only ever label THIS workflow's own ci-fix PRs — + # [ci-fix] title prefix AND agentic-workflows label. (If the v0.80.9 compiler drops these + # — as documented for update-pull-request above — the Step 3.6 preconditions + + # min-integrity:approved + the allowed:[p/0] allowlist are the compensating controls.) + required-title-prefix: "[ci-fix] " + required-labels: [agentic-workflows] timeout-minutes: 90 @@ -339,33 +386,48 @@ through `safe-outputs`. 5. **One issue = one outcome per run.** Exactly one of: respond to a maintainer's change-request on the open PR (Track C, Step 3.5.R); open the first fix/help/ de-flake PR; advance an existing PR by one attempt (push a follow-up fix); - surface a validated-green PR for review; annotate an unrelated-flake red; wait + surface a validated-green PR for review; mark a target-validated draft PR + ready-for-review (Step 3.6 T3 — the terminal outcome that supersedes the same + run's surface/annotate precursor line), or record its `already-ready` + steady-state no-op; annotate an unrelated-flake red; wait (CI not yet settled); or a recorded skip (the dedicated needs-human PR is deferred — the attempt cap records a skip instead; see Step 6). Always prefer advancing or opening a PR over a skip when a non-mute diff is producible. 6. **All writes via `safe-outputs`.** Allowed outputs: `create_pull_request` (first attempt only), `push_to_pull_request_branch` (advance an existing PR by one attempt), `update_pull_request` (bump the attempt marker / refresh the - prior-attempts table), and `add_comment` (progress notes on the `[ci-fix]` PR - ONLY). NEVER comment on the tracking issue (issues are locked by + prior-attempts table), `add_comment` (progress notes on the `[ci-fix]` PR + ONLY), `mark_pull_request_as_ready_for_review` (flip a target-validated draft + `[ci-fix]` PR from draft to ready — Step 3.6 T3 only), and `add_labels` (add + ONLY the `p/0` label, ONLY on that same draft→ready transition — Step 3.6 T3). + NEVER comment on the tracking issue (issues are locked by `.github/workflows/ci-scan-lock-issues.yml`) — `add_comment` targets the PR. No `gh pr create`, no manual `git push`. **Defense-in-depth:** `add_comment` and `update_pull_request` use `target: "*"` (the agent supplies the PR number). - `add_comment` is now config-locked to `required-title-prefix: "[ci-fix] "` + + `add_comment`, `mark_pull_request_as_ready_for_review`, and `add_labels` are + config-locked to `required-title-prefix: "[ci-fix] "` + `required-labels: [agentic-workflows]` (hard-enforced by the handler, same as - `push_to_pull_request_branch`), so a comment can only ever land on THIS - workflow's own PRs. `update_pull_request` CANNOT be config-locked to a - title/label in gh-aw v0.79.8 (the compiler silently drops `required-*` for that - output — verified against the lock), so it keeps `title: false` (no retitles; - body-marker edits only) plus this prompt-level guard. Before emitting either, - VERIFY the target PR carries BOTH the `[ci-fix]` title prefix AND the - `agentic-workflows` label; never comment on or edit the body of any PR that - lacks both. + `push_to_pull_request_branch`), and `add_labels` additionally has an + `allowed: [p/0]` allowlist so it can ONLY ever add `p/0` — so these can only + ever land on THIS workflow's own PRs. `update_pull_request` CANNOT be + config-locked to a title/label in gh-aw v0.80.9 (the compiler silently drops + `required-*` for that output — verified against the lock), so it keeps + `title: false` (no retitles; body-marker edits only) plus this prompt-level + guard. Before emitting ANY of these, VERIFY the target PR carries BOTH the + `[ci-fix]` title prefix AND the `agentic-workflows` label; never comment on, + edit, un-draft, or label any PR that lacks both. 7. **Per-run safe-output caps (per RUN, not per PR).** Each output type is capped per run: `create_pull_request` 3, `push_to_pull_request_branch` 3, - `add_comment` 3, `update_pull_request` 3. Note an ADVANCE spends one push **and** - one update **and** one comment, so ≤ 3 advances/run; surfacing a green or - annotating a flake spends one comment. When a bucket is exhausted, do NOT keep + `add_comment` 6, `update_pull_request` 3, `mark_pull_request_as_ready_for_review` + 3, `add_labels` 3 (counts LABELS, not calls). Note an ADVANCE spends one push + **and** one update **and** one comment, so ≤ 3 advances/run; surfacing a green or + annotating a flake spends one comment; a **mark-ready (Step 3.6 T3)** of a + still-draft PR spends TWO comments (the Step 3 ✅ surface **and** the T3 🎯 + readiness note) **and** one mark-ready **and** one label as an ALL-OR-NOTHING set + (T3 pre-checks those buckets and defers the whole PR if any is exhausted — never + mark a PR ready without its 🎯 audit comment). `add_comment` is therefore sized 6 + (not 3) so ~3 draft flips, at 2 comments each, can land in one sweep instead of the + shared comment bucket starving the otherwise-idle mark-ready/add_labels buckets. When a bucket is exhausted, do NOT keep emitting (extras are silently dropped) — record `skipped: per-run cap reached; deferring PR # to next cycle` for each remaining PR so the drop is a deliberate, logged decision. The continuous loop picks the deferred PRs up next @@ -409,8 +471,9 @@ For every open tracking issue in scope, converge on exactly one outcome: | Open first help-wanted draft `[ci-fix]` PR | No open PR yet; a plausible candidate exists but cannot be runner-validated (device/UI tests) (attempt 1) | | Open first de-flake draft `[ci-fix]` PR | No open PR yet; failure is intermittent (green-on-retry) from a genuine test-quality defect; a deterministic-synchronization fix is producible (attempt 1, Step 4.7 bucket b) | | Advance an existing PR (push attempt N+1) | Open PR's own CI settled red *because of the fix itself*, no human engaged, marker < 10 — push a NEW distinct fix onto the same branch (Step 3.5 → 5.6) | -| Surface a validated-green PR | Open PR's own CI settled green — comment "validated, attempt N/10; ready for review" and do NOT advance (Step 3.5) | -| Annotate an unrelated-flake red | Open PR is red only on baseline-flake / unrelated legs — comment which leg needs a re-run; do NOT burn an attempt (Step 3.5) | +| Surface a validated-green PR | Open PR's own CI settled green — comment "validated, attempt N/10; ready for review" and do NOT advance (Step 3.5), then mark the draft PR ready for review once the fixed test is confirmed green (Step 3.6) | +| Annotate an unrelated-flake red | Open PR is red only on baseline-flake / unrelated legs — comment which leg needs a re-run; do NOT burn an attempt (Step 3.5); if the SPECIFIC fixed test is confirmed green on that build, still mark the draft PR ready for review (Step 3.6) | +| Mark a validated PR ready for review | The specific test this PR fixed is confirmed green in the PR's own CI (even if unrelated legs are red) — comment the target-test result and transition the draft PR to ready for review; never approve or merge (Step 3.6) | | Wait | Open PR's CI is pending / not yet settled on the current head SHA — do nothing this cycle (Step 3.5) | | Hand-off skip (attempt cap) | Marker == 10 and the signature still reproduces — stop and defer to humans; the dedicated `[ci-fix][needs-human]` PR is planned but currently deferred (Step 6) | | Recorded skip | Visual-regression, human engaged, already-handled, fixed-in-latest-build, infra-flake, out-of-bounds, only-mute-available, or no novel approach producible | @@ -685,14 +748,34 @@ Run these gates in order — the FIRST that fires decides this cycle's outcome: 1. **Human engaged.** If `C.humanEngaged` → `skipped: human engaged on PR #; deferring` and stop. Never fight a human reviewer — the intentional hand-off boundary still holds the moment a person touches the PR. -2. **CI not settled / unknown / incomplete prefetch.** If `C.checksSettled == false` - OR `C.overallConclusion` is `pending`, `neutral`, or `unknown`, OR - `C.dataComplete == false` → the fix's CI has not finished on the current head SHA - (`C.headSha`), or the prefetch could not fully read this PR (a partial read can - understate `humanEngaged` / `attempt`). `skipped: PR # CI pending / prefetch - incomplete on ; waiting` and stop. *(Round 1: this is where a - maintainer `/azp run` is awaited — the `/azp`-gated uitests/devicetests legs will - not have run until a human kicks them.)* +2. **CI not settled — but validate the target test first (target-focused readiness).** + If `C.checksSettled == false` OR `C.overallConclusion` is `pending`, `neutral`, or + `unknown`, OR `C.dataComplete == false` → the *overall* build has not finished on the + current head SHA (`C.headSha`), or the prefetch could not fully read this PR (a partial + read can understate `humanEngaged` / `attempt`). Unrelated legs still draining must NOT + keep an already-proven fix parked as a draft, so before waiting, try the target-focused + fast-path: + - **2a — Target-green fast-path (draft `[ci-fix]` PRs only).** If `C.dataComplete == + true` AND the Step 3.6 preconditions hold (draft PR, `[ci-fix] ` title prefix AND the + `agentic-workflows` label), run Step 3.6 **T1–T2** against `C.headSha` now: identify + the target test(s) and read the PR's OWN build **timeline / per-leg status** (anonymous + `_apis/build` — never `_apis/test`, per Hard-Rule 8). If EVERY target test is + **VALIDATED-GREEN** (per T2's all-platform definition below — the target's category leg + is `succeeded` on every platform that runs it, failed on none, and NO target platform + family the pipeline covers is still pending/unconcluded, per T2's "no platform left + unverified" rule; a platform that simply has no leg for the target's category — the test + doesn't run there — is fine, not a gap) AND every leg in + `C.failedLegs` that has ALREADY concluded classifies as **unrelated flake** (Step 4 / + Step 4.7 method — the fix is implicated in NO completed red), then the fix is proven + regardless of unrelated *pending* legs → run **Step 3.6 T3** (mark ready + 🎯 comment) + and stop. This early-out NEVER advances an attempt and NEVER acts on a red that could + be the fix's fault. + - **2b — Otherwise WAIT.** If the target test has not yet executed (pending/absent on + its leg), or a completed red is (or may be) caused by the fix, or `C.dataComplete == + false`, or this PR has no identifiable target test → `skipped: PR # CI pending / + target not yet validated on ; waiting` and stop. *(Round 1: this is where a + maintainer `/azp run` is awaited — the `/azp`-gated uitests/devicetests legs will not + have run until a human kicks them.)* 3. **Green → surface for review.** If `C.overallConclusion == "success"`: the checks that RAN are green. **Primary-gate check first:** the deterministic `success` verdict only certifies "at least one green check, nothing failing or @@ -704,18 +787,31 @@ Run these gates in order — the FIRST that fires decides this cycle's outcome: # primary CI gate (maui-pr) not green on ; waiting` and stop — do NOT surface. (The `/azp`-gated `maui-pr-uitests` (def 313) / `maui-pr-devicetests` (def 314) legs MAY still be un-run — that is expected and is named below, not a - reason to withhold the surface.) **Idempotency:** scan the PR's existing comments - for a prior bot `✅ … validated … on ` note for THIS head SHA — if one - already exists, record `already-surfaced PR # (head )` and stop - WITHOUT re-commenting (re-surfacing the same green every tick is noise and burns - the per-run comment budget other PRs need). Otherwise resolve the attempt number from - `C.effectiveAttempt` (the authoritative max(marker, bot-commit) counter; omit the - number only if it is somehow indeterminate). **Dry-run gate (Step 0):** if `dry_run == "true"`, do NOT emit any comment — instead print the intended `✅` validated-green body to the run log, tally `dry-run: would-surface-green PR #`, and stop. Otherwise `add_comment` on PR #: `✅ Attempt /10 validated - — the fix's CI is green on . Ready for human review.` - Name any `/azp`-gated legs (uitests def 313 / devicetests def 314) that have not - run and still need a maintainer `/azp run`. Do NOT advance. Record - `surfaced-green PR # (attempt /10)` and stop. *(This directly - attacks the real bottleneck — no reviews — so it is the highest-value outcome.)* + reason to withhold the surface.) **Comment idempotency + dry-run suppress the ✅ + comment ONLY — neither skips Step 3.6.** Scan the PR's existing comments for a prior + bot `✅ … validated … on ` note for THIS head SHA; if one exists, set + `SKIP_SURFACE_COMMENT = true`. Resolve the attempt number from `C.effectiveAttempt` + (the authoritative max(marker, bot-commit) counter; omit the number only if it is + somehow indeterminate). Then post the surface comment UNLESS suppressed: + - if `dry_run == "true"`: do NOT emit — print the intended `✅` validated-green body to + the run log and tally `dry-run: would-surface-green PR #`; + - else if `SKIP_SURFACE_COMMENT`: do NOT re-post — record `already-surfaced PR # + (head )` (re-surfacing the same green every tick is noise and burns the + per-run comment budget other PRs need); + - else `add_comment` on PR #: `✅ Attempt /10 validated — the fix's CI is + green on .` naming any `/azp`-gated legs (uitests def 313 / devicetests def + 314) that have not run and still need a maintainer `/azp run`; record `surfaced-green + PR # (attempt /10)`. (Do NOT assert "ready for human review" on a PR that + is still a draft — Step 3.6, not this comment, owns the draft→ready flip and posts its + own 🎯 announcement when it fires.) + Do NOT advance. Then — in ALL of the above cases — run **Step 3.6** (target-test + readiness gate) for this PR before stopping: its `/azp`-gated target legs may conclude + green on this SAME head SHA on a LATER sweep (an `/azp run` adds no commit), and Step + 3.6 is the ONLY place the draft→ready flip happens, so it MUST re-evaluate every sweep — + never `stop` here before it. *(This directly attacks the real bottleneck — no reviews — + so it is the highest-value outcome.)* 4. **Red → classify caused-by-fix vs unrelated-flake.** If `C.overallConclusion == "failure"`, analyze the PR's OWN failing build (NOT `main`): find the AzDO `maui-pr` build for this PR (filter builds by `branchName=refs/pull//merge`, @@ -723,18 +819,26 @@ Run these gates in order — the FIRST that fires decides this cycle's outcome: method and Step 4.7 flake buckets to `C.failedLegs`. - **Unrelated flake only** (every failed leg is known-flaky / infra / a pre-existing baseline red NOT introduced by the fix): do NOT burn an attempt. - **Idempotency first:** scan the PR's existing comments for a prior bot - `♻️ … unrelated flake … on ` note for THIS head SHA — if one already - exists, record `already-annotated-flake PR # (head )` and stop - WITHOUT re-commenting (re-annotating the same flake every 12h sweep is noise and - burns the per-run comment budget other PRs need). Otherwise resolve the attempt - number from `C.effectiveAttempt` (the authoritative max(marker, bot-commit) - counter), omitting the `Attempt /10:` prefix only if it is somehow - indeterminate. **Dry-run gate (Step 0):** if `dry_run == "true"`, do NOT emit any comment — instead print the intended `♻️` unrelated-flake body to the run log, tally `dry-run: would-annotate-flake PR #`, and stop. Otherwise `add_comment` on PR #: `♻️ Attempt /10: red is - unrelated flake on leg(s) () on ; the fix itself is not - implicated. A maintainer re-run (/azp run ) should clear it.` Record - `annotated-flake PR # (head )` and stop. *(Round 1: human re-runs; - Round 2: auto re-trigger.)* + **Comment idempotency + dry-run suppress the ♻️ comment ONLY — neither skips Step + 3.6.** Scan the PR's existing comments for a prior bot `♻️ … unrelated flake … on + ` note for THIS head SHA; if one exists, set `SKIP_FLAKE_COMMENT = true`. + Resolve the attempt number from `C.effectiveAttempt` (the authoritative max(marker, + bot-commit) counter), omitting the `Attempt /10:` prefix only if it is + somehow indeterminate. Then post the flake note UNLESS suppressed: + - if `dry_run == "true"`: do NOT emit — print the intended `♻️` unrelated-flake body + to the run log and tally `dry-run: would-annotate-flake PR #`; + - else if `SKIP_FLAKE_COMMENT`: do NOT re-post — record `already-annotated-flake PR + # (head )` (re-annotating the same flake every 12h sweep is noise and + burns the per-run comment budget other PRs need); + - else `add_comment` on PR #: `♻️ Attempt /10: red is unrelated flake on + leg(s) () on ; the fix itself is not implicated. A + maintainer re-run (/azp run ) should clear it.` record `annotated-flake + PR # (head )`. + Then — in ALL of the above cases — run **Step 3.6** (target-test readiness gate) for + this PR before stopping: its `/azp`-gated target legs may conclude green on this SAME + head SHA on a LATER sweep, and Step 3.6 is the ONLY place the draft→ready flip + happens, so it MUST re-evaluate every sweep — never `stop` here before it. *(Round 1: + human re-runs; Round 2: auto re-trigger.)* - **Caused by the fix** (a failed leg still matches the original target signature, or the fix introduced a NEW failure): advance an attempt. - **Attempt count.** `attempt = C.effectiveAttempt` — the authoritative @@ -752,6 +856,188 @@ Run these gates in order — the FIRST that fires decides this cycle's outcome: from every prior commit on this branch**, emitted via the Step 5.6 ADVANCE path (push onto the same PR). +#### Step 3.6 — Target-test verification & mark-ready gate + +Reached from the Step 3 **green-surface** branch, the Step 4 **unrelated-flake** branch +(AFTER that branch has posted its comment), and the Step 3.5 **gate-2 target-focused +fast-path** (2a — while unrelated legs are still pending). Purpose: confirm the SPECIFIC +test(s) this PR was opened to fix now PASS in the PR's own CI, and — only when they do +— transition the draft `[ci-fix]` PR to **ready for review** so a maintainer sees a +validated fix instead of a draft. This is the ONLY place the loop flips draft→ready; it +is a state transition, **never** an approval or a merge (a human still reviews and +merges). Overall red on *unrelated* legs must NOT gate readiness — we validate the fix, +not the base branch's flakiness. + +**Preconditions** (ALL must hold; otherwise record `skipped: readiness N/A PR # +()` and stop this gate): +- `C.isDraft == true` — if the PR is already ready-for-review, the draft→ready + transition is already done; do NOT re-mark **and do NOT re-apply `p/0`**. `p/0` is applied + exactly ONCE, atomically with the draft→ready flip (T3 — all-or-nothing with the 🎯 audit + comment + mark-ready), so an already-ready loop PR that is MISSING `p/0` has almost + certainly had it **removed by a maintainer** de-prioritizing the PR — a pure triage action + the human-engagement guard (which inspects comments/reviews/commits, NOT label changes) + cannot see. Re-adding `p/0` every sweep would fight that maintainer indefinitely, violating + the loop's "never override a human" contract. So the loop does NOT reconcile the label: + record `already-ready PR #` and stop this gate. (Trade-off: on the rare occasion a + transient API error drops `p/0` at flip time *after* the T3 atomic pre-check passed, it is + not auto-re-added — but the PR is still ready-for-review with its 🎯 audit comment, and + re-adding `p/0` is a trivial manual action; that is strictly preferable to steam-rolling a + maintainer's deliberate de-prioritization.) +- The PR is unmistakably THIS workflow's own: `[ci-fix] ` title prefix AND the + `agentic-workflows` label (mirrors the safe-output lock; the compensating scope + control if the v0.80.9 compiler drops the declarative required-*). +- You reached this gate from Step 3 (green), Step 4 (**unrelated-flake**), or the Step 3.5 + gate-2 **target-focused fast-path** (2a) — i.e. the fix is NOT implicated in any red that + has concluded. If Step 4 classified a red as **caused by the fix** (ADVANCE), do NOT run + this gate — advance the attempt instead. + +**T1 — Identify the target test(s).** From the `[ci-scan]` issue signature the fix +addresses (and the PR's own diff), extract the fully-qualified test method name(s) the +fix targets — e.g. `SafeAreaShouldWorkOnAllShellTabs`. For a de-flake it is the +de-flaked test; for a product fix it is the originally-failing test(s). If NO specific +test can be identified (e.g. a product build-break rather than a test failure), this is a +**build-only fix** — there is no single test to validate cross-platform, so validate at +**whole-build** granularity rather than stopping (Step 3 only *comments*; Step 3.6 is the +sole draft→ready flip, so a build-only fix is undrafted HERE or not at all). We only reach +this gate from Step 3 (green) / Step 4 (unrelated-flake) / gate-2a, so the fix is not +implicated in any concluded red; additionally require, via the T2 timeline method on +`C.headSha`, that the primary build pipeline `maui-pr` (def 302) has CONCLUDED with EVERY +one of its platform build legs `succeeded`/`completed` and NONE failed/canceled or still +pending — the build is green on **every** platform, not just the originally-broken one (the +cross-platform guard, applied to the build instead of a test). A build-only fix is undrafted +ONLY on the strength of the auto-running `maui-pr` (def 302) whole-build green, which the +loop CAN observe: if the PR's `[ci-scan]` issue signature or its own diff indicates the +ORIGINATING failure was in a `/azp`-gated pipeline (`maui-pr-uitests` def 313 or +`maui-pr-devicetests` def 314) rather than `maui-pr` (def 302), do NOT undraft on +`maui-pr`-green alone — those pipelines do not auto-run on this PR (GITHUB_TOKEN cannot +trigger them), so a green `maui-pr` build is NOT evidence the gated build break is fixed; +record `skipped: build-only fix PR # targets gated pipeline () not run — +deferring to human` and stop this gate. Otherwise, set `TARGET := "the maui-pr build +(build-only fix — no single target test)"` and proceed to **T3** to mark ready. If any `maui-pr` build leg is still unconcluded, record `skipped: build-only fix PR +# not yet whole-build green (leg(s) pending)` and stop this gate WITHOUT marking +ready (a green subset is not enough — a still-pending build leg could yet fail). + +**T2 — Verify the target test's platform legs are green (anonymous, leg-level).** Per-test +outcomes come from the AzDO **test-results** API (`_apis/test/...`), which is **NOT reachable +anonymously — Hard-Rule 8**: those endpoints 302-redirect to a sign-in page on this runner, so +a `curl` returns an HTML redirect (not JSON) and every target test would look "not executed". +Validate instead at **leg granularity** using ONLY the anonymous `_apis/build/...` timeline +(Hard-Rule 8) — a `succeeded` category leg means every test in that category **that actually +ran** passed on that platform. This is a strong bar **only if the target test genuinely +executes**: a job can go green while one specific test is skipped at runtime (`[Ignore]`, +`Assert.Ignore()`, `Assert.Inconclusive()`, a `Skip=`/conditional `[Fact]`, an `#if`-out, or +an early `return` before the asserts). So before trusting a green leg as proof the target +passed, **confirm from the PR's OWN diff that the fix does NOT skip, ignore, disable, or +short-circuit the target test** (it adds no `[Ignore]`/`[Explicit]`, `Assert.Ignore`/ +`Assert.Inconclusive`, `Skip=`, category-exclusion, `#if`-out, or early `return` around the +target). If the fix could cause the target to be runtime-skipped rather than genuinely pass, +leg-level green is NOT sufficient evidence — record `skipped: target test may be +runtime-skipped by this fix (diff adds ); leg-green insufficient, deferring to human +on PR #` and stop this gate WITHOUT marking ready. Otherwise (the target genuinely runs +and asserts, as a de-flake or product fix does) a green category leg cannot hide a +target-test failure, so treat it as authoritative. + +Map the target test to the CI leg(s) that run it. A leg name encodes platform + test category +(e.g. `Android UITests SafeAreaEdges,Shadow`, `iOS UITests SafeAreaEdges,Shadow`, `macOS +UITests SafeAreaEdges,Shadow`, `Windows UITests SafeAreaEdges,Shadow`). Determine the target +test's UI-test category — its `[Category(UITestCategories.X)]` in the test/HostApp source, +visible in the PR diff or the test file — to know which leg-name substring identifies its legs; +for a device test, the per-platform device-test legs. + +Using the SAME build-discovery as Step 4 (filter AzDO builds by `branchName=refs/pull//merge` +or `sourceVersion == C.headSha`), read each build's **timeline** on `C.headSha` for the +pipeline(s) that RUN the target test — `maui-pr` (def 302) for unit/integration tests, +`maui-pr-uitests` (def 313) for Appium UI tests, `maui-pr-devicetests` (def 314) for device +tests: + +```bash +ORG=dnceng-public; PROJ=public; BUILD= +# Anonymous + Hard-Rule-8-compliant. NEVER call _apis/test/... (it 302-redirects to sign-in). +# Each type=="Job" record is one platform leg: result (succeeded/failed/canceled/null), +# state (completed/inProgress/pending), name (encodes " UITests "): +curl -s "https://dev.azure.com/$ORG/$PROJ/_apis/build/builds/$BUILD/timeline?api-version=7.1" \ + | tee /tmp/gh-aw/agent/timeline_${BUILD}.json \ + | jq -r '.records[] | select(.type=="Job") | "\(.result)\t\(.state)\t\(.name)"' +``` + +(The prefetch already exposes per-leg status in `C.failedLegs` / the checks context, and +`gh pr checks ` lists the same per-leg rows with platform+category in the name — use +whichever is handy; the build timeline is the authoritative cross-check on the exact build.) + +Treat a target test as **VALIDATED-GREEN** only if, on `C.headSha`, the leg that runs its +category is green on **every platform it runs on** — a fix that repairs one platform must not +silently regress the same test's leg on another, so a green leg on the originally-red platform +alone is NOT enough. Across the target pipeline's platform legs (for a UI test: the Android, +iOS, Windows, and macOS/MacCatalyst legs running the target's category; for a device test: each +device-test platform), require ALL of: +- the target's category leg is `result == "succeeded"` **and** `state == "completed"` on + **every** platform that runs it, AND +- that leg is `failed` / `canceled` / aborted on **NO** platform, AND +- **no platform is left unverified.** For each platform family the target pipeline covers, that + platform's category leg must have CONCLUDED (`state == "completed"`) on `C.headSha`. If a + platform simply has no leg for the target's category, the test does not run there — that is + fine, not a gap. But if a platform's category leg has **not concluded** (`state` is + `inProgress` / pending, or an `/azp`-gated `maui-pr-uitests` / `maui-pr-devicetests` leg that + has not been kicked), the test's status on that platform is UNKNOWN → the fix is NOT yet + validated across platforms: record `skipped: target test green on but + not yet verified on (leg(s) pending / need /azp run) on PR #` + and stop this gate WITHOUT marking ready. + +A target test whose category leg never concluded on ANY platform (**not executed** anywhere — +e.g. its `/azp`-gated pipeline has not been kicked) is likewise NOT validated: record `skipped: +target test not yet executed on PR # ( not run — needs /azp run)` and stop +this gate WITHOUT marking ready. Do NOT overclaim — a green *sibling* leg (a different category +on the same platform) is not the target's leg, and a green leg on one platform is not a pass on +the others. + +**T3 — Mark ready + report.** If EVERY target test is VALIDATED-GREEN. The 🎯 comment, +the mark-ready, and the `p/0` label are THREE SEPARATE safe-outputs — the comment +existing does NOT prove the mark-ready took effect, so they are tracked independently: + +- **Comment idempotency (dup-suppress only — never gates the mark-ready).** We only reach + T3 while `C.isDraft == true` (the precondition stops an already-ready PR before here). So + if a prior bot comment for THIS head that carries the leading `🎯` anchor AND the + contiguous phrase `validated green on ` already exists (BOTH T3 variants — + `🎯 Target test validated green on ` and the build-only `🎯 Build validated green on + ` — contain that exact contiguous phrase, so this recognizes both; keying on + `target test` alone would instead miss the build-only comment and re-post it every sweep. + **Require the `🎯` anchor** so the match can NEVER be loosened to a gapped + `validated…green…on` form that would false-positive on the Step 3 `✅ … validated — the + fix's CI is green on ` surface comment and wrongly suppress the audit 🎯), the comment + landed on an earlier sweep but the **mark-ready did NOT take + effect** (the PR is still a draft) — set `SUPPRESS_COMMENT = true` (do not re-post the + duplicate 🎯 comment) but STILL complete the mark-ready + label below. Never treat the + comment's existence as "already marked ready" while the PR is still a draft. Record this + case as `re-marking PR # (🎯 present; mark-ready did not take on a prior sweep)`. +- **Atomicity budget pre-check (Hard-Rule 7).** Determine the outputs this gate will emit: + `mark_pull_request_as_ready_for_review` + `add_labels` always, plus `add_comment` unless + `SUPPRESS_COMMENT`. Verify a free per-run slot remains in EACH bucket you are about to + use. If ANY required bucket is exhausted, emit NONE of them — record `skipped: per-run + cap reached; deferring mark-ready PR # to next cycle` and stop this gate. Never mark a + PR ready (or label it) unless its 🎯 audit comment is GUARANTEED to exist for this exact + head SHA — either posted in this same sweep, or (in the `SUPPRESS_COMMENT` re-marking case) + already present from a prior sweep. The outputs you DO emit either all land or all defer + together; the audit trail must never be absent, but it is NOT re-posted when it already exists. +- **Dry-run gate (Step 0):** if `dry_run == "true"`, emit NOTHING — print the intended + readiness comment and "would mark ready + add p/0" to the run log, tally `dry-run: + would-mark-ready PR #`, and stop. +- Otherwise emit for THIS PR number ``: + 1. `add_comment` (ONLY if not `SUPPRESS_COMMENT`): `🎯 Target test validated green on + — passed on ALL platforms it runs on (, + buildId ). — not + caused by this fix."> Transitioning this PR from draft to ready for review and adding + `p/0`; a maintainer still reviews and merges.` (For a **build-only fix** — the T1 + whole-build fallback, no single target test — phrase the first clause as `🎯 Build + validated green on — the maui-pr build passed on ALL platforms (buildId + ).` instead of naming a test.) + 2. `mark_pull_request_as_ready_for_review` with `reason:` a one-line justification + naming the validated test(s) and ``. + 3. `add_labels` with `labels: ["p/0"]` for PR # — put the now-review-ready fix into + the team's p/0 priority queue so it is triaged, not lost in the draft backlog. (If + the PR somehow already carries `p/0`, this is a harmless no-op.) +- Record `marked-ready PR # (target green on ALL platforms on , + labeled p/0)` and stop. + #### Step 3.5.R — Maintainer change-request response (Track C) Reached from Step 3.5 gate 0. Goal: when an **eligible human reviewer** (Hard-Rule @@ -1383,7 +1669,19 @@ Filed by [`ci-status-fix`](https://github.com/dotnet/maui/blob/main/.github/work ### Step 8 — Per-issue tally + end-of-run summary -Per issue, append one outcome line to `/tmp/gh-aw/agent/coverage.txt`: +Per issue, append **one** outcome line to `/tmp/gh-aw/agent/coverage.txt` — the +terminal outcome for the cycle. When one cycle produces a chained pair — a PR gets a +same-cycle precursor line and **then** a Step 3.6 readiness line (`marked-ready` on the +draft→ready flip, or `already-ready` on a later steady-state sweep of an already-ready +PR) — record ONLY the terminal Step 3.6 readiness line: it supersedes ANY same-cycle +non-readiness precursor for that PR, so an aggregator keying on one-line-per-issue never +double-counts. The precursor may be a Step 3 green line (`surfaced-green` on the first ✅, +or `already-surfaced` when it already exists) OR — when an unrelated-flake red and a +target-green coincide — a Step 4 `annotated-flake` line; either way the readiness line is +terminal, and the superseded precursor's signal survives in the PR's 🎯/♻️ comment so no +information is lost. This covers the flip cycle (`surfaced-green` → `marked-ready`), the +post-flip steady state (`already-surfaced` → `already-ready`), and the flake-coincident +flip (`annotated-flake` → `marked-ready`): ``` # main attempt- @@ -1394,8 +1692,14 @@ Per issue, append one outcome line to `/tmp/gh-aw/agent/coverage.txt`: `advance-PR # attempt /10` (ADVANCE mode — new commit pushed to the existing PR), `surfaced-green PR #` (fix's own CI went green; commented for review, did not advance), `annotated-flake PR #` (red was unrelated flake; -commented, attempt NOT burned), `waiting PR #` (CI not settled yet), -`dry-run: would-`, `skipped: `. (The +commented, attempt NOT burned), `marked-ready PR #` (the specific fixed test +was confirmed green in the PR's own CI; draft flipped to ready for review), +`already-surfaced PR #` (the fix's ✅ green comment already existed this sweep; +re-post suppressed — superseded by the Step 3.6 readiness line when the PR also flips +ready that cycle), `already-ready PR #` (terminal steady state — the PR was flipped +ready on a prior cycle and remains green on an unchanged head; no action taken, +supersedes `already-surfaced`), `waiting PR #` (CI not settled yet), +`dry-run: would-`, `skipped: `. (The `needs-human-PR` outcome is reserved for the deferred hand-off — Step 6 currently records a skip instead, so it is not emitted.)
+()` and stop this gate): +- `C.isDraft == true` — if the PR is already ready-for-review, the draft→ready + transition is already done; do NOT re-mark **and do NOT re-apply `p/0`**. `p/0` is applied + exactly ONCE, atomically with the draft→ready flip (T3 — all-or-nothing with the 🎯 audit + comment + mark-ready), so an already-ready loop PR that is MISSING `p/0` has almost + certainly had it **removed by a maintainer** de-prioritizing the PR — a pure triage action + the human-engagement guard (which inspects comments/reviews/commits, NOT label changes) + cannot see. Re-adding `p/0` every sweep would fight that maintainer indefinitely, violating + the loop's "never override a human" contract. So the loop does NOT reconcile the label: + record `already-ready PR #` and stop this gate. (Trade-off: on the rare occasion a + transient API error drops `p/0` at flip time *after* the T3 atomic pre-check passed, it is + not auto-re-added — but the PR is still ready-for-review with its 🎯 audit comment, and + re-adding `p/0` is a trivial manual action; that is strictly preferable to steam-rolling a + maintainer's deliberate de-prioritization.) +- The PR is unmistakably THIS workflow's own: `[ci-fix-net11] ` title prefix AND the + `agentic-workflows` label (mirrors the safe-output lock; the compensating scope + control if the v0.80.9 compiler drops the declarative required-*). +- You reached this gate from Step 3 (green), Step 4 (**unrelated-flake**), or the Step 3.5 + gate-2 **target-focused fast-path** (2a) — i.e. the fix is NOT implicated in any red that + has concluded. If Step 4 classified a red as **caused by the fix** (ADVANCE), do NOT run + this gate — advance the attempt instead. + +**T1 — Identify the target test(s).** From the `[ci-scan-net11]` issue signature the fix +addresses (and the PR's own diff), extract the fully-qualified test method name(s) the +fix targets — e.g. `SafeAreaShouldWorkOnAllShellTabs`. For a de-flake it is the +de-flaked test; for a product fix it is the originally-failing test(s). If NO specific +test can be identified (e.g. a product build-break rather than a test failure), this is a +**build-only fix** — there is no single test to validate cross-platform, so validate at +**whole-build** granularity rather than stopping (Step 3 only *comments*; Step 3.6 is the +sole draft→ready flip, so a build-only fix is undrafted HERE or not at all). We only reach +this gate from Step 3 (green) / Step 4 (unrelated-flake) / gate-2a, so the fix is not +implicated in any concluded red; additionally require, via the T2 timeline method on +`C.headSha`, that the primary build pipeline `maui-pr` (def 302) has CONCLUDED with EVERY +one of its platform build legs `succeeded`/`completed` and NONE failed/canceled or still +pending — the build is green on **every** platform, not just the originally-broken one (the +cross-platform guard, applied to the build instead of a test). A build-only fix is undrafted +ONLY on the strength of the auto-running `maui-pr` (def 302) whole-build green, which the +loop CAN observe: if the PR's `[ci-scan]` issue signature or its own diff indicates the +ORIGINATING failure was in a `/azp`-gated pipeline (`maui-pr-uitests` def 313 or +`maui-pr-devicetests` def 314) rather than `maui-pr` (def 302), do NOT undraft on +`maui-pr`-green alone — those pipelines do not auto-run on this PR (GITHUB_TOKEN cannot +trigger them), so a green `maui-pr` build is NOT evidence the gated build break is fixed; +record `skipped: build-only fix PR # targets gated pipeline () not run — +deferring to human` and stop this gate. Otherwise, set `TARGET := "the maui-pr build +(build-only fix — no single target test)"` and proceed to **T3** to mark ready. If any `maui-pr` build leg is still unconcluded, record `skipped: build-only fix PR +# not yet whole-build green (leg(s) pending)` and stop this gate WITHOUT marking +ready (a green subset is not enough — a still-pending build leg could yet fail). + +**T2 — Verify the target test's platform legs are green (anonymous, leg-level).** Per-test +outcomes come from the AzDO **test-results** API (`_apis/test/...`), which is **NOT reachable +anonymously — Hard-Rule 8**: those endpoints 302-redirect to a sign-in page on this runner, so +a `curl` returns an HTML redirect (not JSON) and every target test would look "not executed". +Validate instead at **leg granularity** using ONLY the anonymous `_apis/build/...` timeline +(Hard-Rule 8) — a `succeeded` category leg means every test in that category **that actually +ran** passed on that platform. This is a strong bar **only if the target test genuinely +executes**: a job can go green while one specific test is skipped at runtime (`[Ignore]`, +`Assert.Ignore()`, `Assert.Inconclusive()`, a `Skip=`/conditional `[Fact]`, an `#if`-out, or +an early `return` before the asserts). So before trusting a green leg as proof the target +passed, **confirm from the PR's OWN diff that the fix does NOT skip, ignore, disable, or +short-circuit the target test** (it adds no `[Ignore]`/`[Explicit]`, `Assert.Ignore`/ +`Assert.Inconclusive`, `Skip=`, category-exclusion, `#if`-out, or early `return` around the +target). If the fix could cause the target to be runtime-skipped rather than genuinely pass, +leg-level green is NOT sufficient evidence — record `skipped: target test may be +runtime-skipped by this fix (diff adds ); leg-green insufficient, deferring to human +on PR #` and stop this gate WITHOUT marking ready. Otherwise (the target genuinely runs +and asserts, as a de-flake or product fix does) a green category leg cannot hide a +target-test failure, so treat it as authoritative. + +Map the target test to the CI leg(s) that run it. A leg name encodes platform + test category +(e.g. `Android UITests SafeAreaEdges,Shadow`, `iOS UITests SafeAreaEdges,Shadow`, `macOS +UITests SafeAreaEdges,Shadow`, `Windows UITests SafeAreaEdges,Shadow`). Determine the target +test's UI-test category — its `[Category(UITestCategories.X)]` in the test/HostApp source, +visible in the PR diff or the test file — to know which leg-name substring identifies its legs; +for a device test, the per-platform device-test legs. + +Using the SAME build-discovery as Step 4 (filter AzDO builds by `branchName=refs/pull//merge` +or `sourceVersion == C.headSha`), read each build's **timeline** on `C.headSha` for the +pipeline(s) that RUN the target test — `maui-pr` (def 302) for unit/integration tests, +`maui-pr-uitests` (def 313) for Appium UI tests, `maui-pr-devicetests` (def 314) for device +tests: + +```bash +ORG=dnceng-public; PROJ=public; BUILD= +# Anonymous + Hard-Rule-8-compliant. NEVER call _apis/test/... (it 302-redirects to sign-in). +# Each type=="Job" record is one platform leg: result (succeeded/failed/canceled/null), +# state (completed/inProgress/pending), name (encodes " UITests "): +curl -s "https://dev.azure.com/$ORG/$PROJ/_apis/build/builds/$BUILD/timeline?api-version=7.1" \ + | tee /tmp/gh-aw/agent/timeline_${BUILD}.json \ + | jq -r '.records[] | select(.type=="Job") | "\(.result)\t\(.state)\t\(.name)"' +``` + +(The prefetch already exposes per-leg status in `C.failedLegs` / the checks context, and +`gh pr checks ` lists the same per-leg rows with platform+category in the name — use +whichever is handy; the build timeline is the authoritative cross-check on the exact build.) + +Treat a target test as **VALIDATED-GREEN** only if, on `C.headSha`, the leg that runs its +category is green on **every platform it runs on** — a fix that repairs one platform must not +silently regress the same test's leg on another, so a green leg on the originally-red platform +alone is NOT enough. Across the target pipeline's platform legs (for a UI test: the Android, +iOS, Windows, and macOS/MacCatalyst legs running the target's category; for a device test: each +device-test platform), require ALL of: +- the target's category leg is `result == "succeeded"` **and** `state == "completed"` on + **every** platform that runs it, AND +- that leg is `failed` / `canceled` / aborted on **NO** platform, AND +- **no platform is left unverified.** For each platform family the target pipeline covers, that + platform's category leg must have CONCLUDED (`state == "completed"`) on `C.headSha`. If a + platform simply has no leg for the target's category, the test does not run there — that is + fine, not a gap. But if a platform's category leg has **not concluded** (`state` is + `inProgress` / pending, or an `/azp`-gated `maui-pr-uitests` / `maui-pr-devicetests` leg that + has not been kicked), the test's status on that platform is UNKNOWN → the fix is NOT yet + validated across platforms: record `skipped: target test green on but + not yet verified on (leg(s) pending / need /azp run) on PR #` + and stop this gate WITHOUT marking ready. + +A target test whose category leg never concluded on ANY platform (**not executed** anywhere — +e.g. its `/azp`-gated pipeline has not been kicked) is likewise NOT validated: record `skipped: +target test not yet executed on PR # ( not run — needs /azp run)` and stop +this gate WITHOUT marking ready. Do NOT overclaim — a green *sibling* leg (a different category +on the same platform) is not the target's leg, and a green leg on one platform is not a pass on +the others. + +**T3 — Mark ready + report.** If EVERY target test is VALIDATED-GREEN. The 🎯 comment, +the mark-ready, and the `p/0` label are THREE SEPARATE safe-outputs — the comment +existing does NOT prove the mark-ready took effect, so they are tracked independently: + +- **Comment idempotency (dup-suppress only — never gates the mark-ready).** We only reach + T3 while `C.isDraft == true` (the precondition stops an already-ready PR before here). So + if a prior bot comment for THIS head that carries the leading `🎯` anchor AND the + contiguous phrase `validated green on ` already exists (BOTH T3 variants — + `🎯 Target test validated green on ` and the build-only `🎯 Build validated green on + ` — contain that exact contiguous phrase, so this recognizes both; keying on + `target test` alone would instead miss the build-only comment and re-post it every sweep. + **Require the `🎯` anchor** so the match can NEVER be loosened to a gapped + `validated…green…on` form that would false-positive on the Step 3 `✅ … validated — the + fix's CI is green on ` surface comment and wrongly suppress the audit 🎯), the comment + landed on an earlier sweep but the **mark-ready did NOT take + effect** (the PR is still a draft) — set `SUPPRESS_COMMENT = true` (do not re-post the + duplicate 🎯 comment) but STILL complete the mark-ready + label below. Never treat the + comment's existence as "already marked ready" while the PR is still a draft. Record this + case as `re-marking PR # (🎯 present; mark-ready did not take on a prior sweep)`. +- **Atomicity budget pre-check (Hard-Rule 7).** Determine the outputs this gate will emit: + `mark_pull_request_as_ready_for_review` + `add_labels` always, plus `add_comment` unless + `SUPPRESS_COMMENT`. Verify a free per-run slot remains in EACH bucket you are about to + use. If ANY required bucket is exhausted, emit NONE of them — record `skipped: per-run + cap reached; deferring mark-ready PR # to next cycle` and stop this gate. Never mark a + PR ready (or label it) unless its 🎯 audit comment is GUARANTEED to exist for this exact + head SHA — either posted in this same sweep, or (in the `SUPPRESS_COMMENT` re-marking case) + already present from a prior sweep. The outputs you DO emit either all land or all defer + together; the audit trail must never be absent, but it is NOT re-posted when it already exists. +- **Dry-run gate (Step 0):** if `dry_run == "true"`, emit NOTHING — print the intended + readiness comment and "would mark ready + add p/0" to the run log, tally `dry-run: + would-mark-ready PR #`, and stop. +- Otherwise emit for THIS PR number ``: + 1. `add_comment` (ONLY if not `SUPPRESS_COMMENT`): `🎯 Target test validated green on + — passed on ALL platforms it runs on (, + buildId ). — not + caused by this fix."> Transitioning this PR from draft to ready for review and adding + `p/0`; a maintainer still reviews and merges.` (For a **build-only fix** — the T1 + whole-build fallback, no single target test — phrase the first clause as `🎯 Build + validated green on — the maui-pr build passed on ALL platforms (buildId + ).` instead of naming a test.) + 2. `mark_pull_request_as_ready_for_review` with `reason:` a one-line justification + naming the validated test(s) and ``. + 3. `add_labels` with `labels: ["p/0"]` for PR # — put the now-review-ready fix into + the team's p/0 priority queue so it is triaged, not lost in the draft backlog. (If + the PR somehow already carries `p/0`, this is a harmless no-op.) +- Record `marked-ready PR # (target green on ALL platforms on , + labeled p/0)` and stop. + #### Step 3.5.R — Maintainer change-request response (Track C) Reached from Step 3.5 gate 0. Goal: when an **eligible human reviewer** (Hard-Rule @@ -1395,7 +1681,19 @@ Filed by [`ci-status-fix-net11`](https://github.com/dotnet/maui/blob/main/.githu ### Step 8 — Per-issue tally + end-of-run summary -Per issue, append one outcome line to `/tmp/gh-aw/agent/coverage.txt`: +Per issue, append **one** outcome line to `/tmp/gh-aw/agent/coverage.txt` — the +terminal outcome for the cycle. When one cycle produces a chained pair — a PR gets a +same-cycle precursor line and **then** a Step 3.6 readiness line (`marked-ready` on the +draft→ready flip, or `already-ready` on a later steady-state sweep of an already-ready +PR) — record ONLY the terminal Step 3.6 readiness line: it supersedes ANY same-cycle +non-readiness precursor for that PR, so an aggregator keying on one-line-per-issue never +double-counts. The precursor may be a Step 3 green line (`surfaced-green` on the first ✅, +or `already-surfaced` when it already exists) OR — when an unrelated-flake red and a +target-green coincide — a Step 4 `annotated-flake` line; either way the readiness line is +terminal, and the superseded precursor's signal survives in the PR's 🎯/♻️ comment so no +information is lost. This covers the flip cycle (`surfaced-green` → `marked-ready`), the +post-flip steady state (`already-surfaced` → `already-ready`), and the flake-coincident +flip (`annotated-flake` → `marked-ready`): ``` # net11.0 attempt- @@ -1406,8 +1704,14 @@ Per issue, append one outcome line to `/tmp/gh-aw/agent/coverage.txt`: `advance-PR # attempt /10` (ADVANCE mode — new commit pushed to the existing PR), `surfaced-green PR #` (fix's own CI went green; commented for review, did not advance), `annotated-flake PR #` (red was unrelated flake; -commented, attempt NOT burned), `waiting PR #` (CI not settled yet), -`dry-run: would-`, `skipped: `. (The +commented, attempt NOT burned), `marked-ready PR #` (the specific fixed test +was confirmed green in the PR's own CI; draft flipped to ready for review), +`already-surfaced PR #` (the fix's ✅ green comment already existed this sweep; +re-post suppressed — superseded by the Step 3.6 readiness line when the PR also flips +ready that cycle), `already-ready PR #` (terminal steady state — the PR was flipped +ready on a prior cycle and remains green on an unchanged head; no action taken, +supersedes `already-surfaced`), `waiting PR #` (CI not settled yet), +`dry-run: would-`, `skipped: `. (The `needs-human-PR` outcome is reserved for the deferred hand-off — Step 6 currently records a skip instead, so it is not emitted.) diff --git a/.github/workflows/ci-status-fix.lock.yml b/.github/workflows/ci-status-fix.lock.yml index 93dc5969cb0d..65511f2b3259 100644 --- a/.github/workflows/ci-status-fix.lock.yml +++ b/.github/workflows/ci-status-fix.lock.yml @@ -1,4 +1,4 @@ -# gh-aw-metadata: {"schema_version":"v4","frontmatter_hash":"f014161967589ebcaf1ac5ec6f3c88ee5a4388d28b55a07dc36c45b7b2aebab6","body_hash":"86c6b2d4c3df13da14c6a3c8b211381fcc9f97bb1951ff7b80588a2c5338e00c","compiler_version":"v0.80.9","strict":true,"agent_id":"copilot","agent_model":"claude-opus-4.8","engine_versions":{"copilot":"1.0.63"}} +# gh-aw-metadata: {"schema_version":"v4","frontmatter_hash":"b0ebf8a2f179900cfe3638e994b27a43cec52bdbdd2331dc1bd4d65762fa2d6b","body_hash":"1825e2b1f7c7bd88241bf37580655489360f9bebc806edd00837a52ce4ad83a0","compiler_version":"v0.80.9","strict":true,"agent_id":"copilot","agent_model":"claude-opus-4.8","engine_versions":{"copilot":"1.0.63"}} # gh-aw-manifest: {"version":1,"secrets":["COPILOT_PAT_0","COPILOT_PAT_1","COPILOT_PAT_2","COPILOT_PAT_3","COPILOT_PAT_4","COPILOT_PAT_5","COPILOT_PAT_6","COPILOT_PAT_7","COPILOT_PAT_8","COPILOT_PAT_9","GH_AW_CI_TRIGGER_TOKEN","GH_AW_GITHUB_MCP_SERVER_TOKEN","GH_AW_GITHUB_TOKEN","GITHUB_TOKEN"],"actions":[{"repo":"actions/checkout","sha":"34e114876b0b11c390a56381ad16ebd13914f8d5","version":"v4"},{"repo":"actions/checkout","sha":"9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0","version":"v7.0.0"},{"repo":"actions/checkout","sha":"de0fac2e4500dabe0009e67214ff5f5447ce83dd","version":"v6.0.2"},{"repo":"actions/download-artifact","sha":"3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c","version":"v8.0.1"},{"repo":"actions/github-script","sha":"3a2844b7e9c422d3c10d287c895573f7108da1b3","version":"v9.0.0"},{"repo":"actions/setup-node","sha":"48b55a011bda9f5d6aeb4c2d9c7362e8dae4041e","version":"v6.4.0"},{"repo":"actions/upload-artifact","sha":"043fb46d1a93c77aae656e7c1c64a875d1fc6a0a","version":"v7.0.1"},{"repo":"github/gh-aw-actions/setup","sha":"8c7d04ebf1ece56cd381446125da3e0f6896294a","version":"v0.80.9"}],"containers":[{"image":"ghcr.io/github/gh-aw-firewall/agent:0.27.7","digest":"sha256:aae231e4635c8999d039c132f1602d3df850fe9b84a00aa2b5ac981179b5661c","pinned_image":"ghcr.io/github/gh-aw-firewall/agent:0.27.7@sha256:aae231e4635c8999d039c132f1602d3df850fe9b84a00aa2b5ac981179b5661c"},{"image":"ghcr.io/github/gh-aw-firewall/api-proxy:0.27.7","digest":"sha256:009caf2e3d88fa77b64e9a03a95a228fc58db0f1701c6d324b29ba5a3c7c79b6","pinned_image":"ghcr.io/github/gh-aw-firewall/api-proxy:0.27.7@sha256:009caf2e3d88fa77b64e9a03a95a228fc58db0f1701c6d324b29ba5a3c7c79b6"},{"image":"ghcr.io/github/gh-aw-firewall/squid:0.27.7","digest":"sha256:deb1d4e19de62d51cee0508057a596a19315c3423ada4d675cad136dc8037c96","pinned_image":"ghcr.io/github/gh-aw-firewall/squid:0.27.7@sha256:deb1d4e19de62d51cee0508057a596a19315c3423ada4d675cad136dc8037c96"},{"image":"ghcr.io/github/gh-aw-mcpg:v0.3.27","digest":"sha256:fe984bddde4ec05d756d9043edb0a32912e6b7b72f6a121b1082f29221421cc7","pinned_image":"ghcr.io/github/gh-aw-mcpg:v0.3.27@sha256:fe984bddde4ec05d756d9043edb0a32912e6b7b72f6a121b1082f29221421cc7"},{"image":"ghcr.io/github/gh-aw-node","digest":"sha256:529d02eb970b1161aa25c593a9c3df57fdfad5a8add328cb3b6eccef66f3183b","pinned_image":"ghcr.io/github/gh-aw-node@sha256:529d02eb970b1161aa25c593a9c3df57fdfad5a8add328cb3b6eccef66f3183b"},{"image":"ghcr.io/github/github-mcp-server:v1.4.0","digest":"sha256:2afb26356481d1a350e14544a6e160f7f7ec1561a1ea309b823665abf0309036","pinned_image":"ghcr.io/github/github-mcp-server:v1.4.0@sha256:2afb26356481d1a350e14544a6e160f7f7ec1561a1ea309b823665abf0309036"}]} # This file was automatically generated by gh-aw (v0.80.9). DO NOT EDIT. To debug this workflow, load the skill at https://github.com/github/gh-aw/blob/main/debug.md # @@ -37,7 +37,9 @@ # humans (the open tracking issue is the hand-off surface; a dedicated # [ci-fix][needs-human] PR is planned but currently deferred — see Step 6). CI on # the PR is kicked by a human `/azp run` for now; the loop watches, classifies, -# and re-fixes autonomously between kicks. +# and re-fixes autonomously between kicks. When the SPECIFIC test a PR fixed is +# confirmed green in that PR's own CI, the loop marks the draft PR ready for review +# (a state transition only — it never approves or merges). # Never mutes tests, but # de-flakes genuinely flaky ones (deterministic synchronization, no retries / # timeout bumps). Always skips visual-regression / screenshot issues. @@ -273,24 +275,24 @@ jobs: run: | bash "${RUNNER_TEMP}/gh-aw/actions/create_prompt_first.sh" { - cat << 'GH_AW_PROMPT_25cf75111b670167_EOF' + cat << 'GH_AW_PROMPT_2ac1d0b4083428f8_EOF' - GH_AW_PROMPT_25cf75111b670167_EOF + GH_AW_PROMPT_2ac1d0b4083428f8_EOF cat "${RUNNER_TEMP}/gh-aw/prompts/xpia.md" cat "${RUNNER_TEMP}/gh-aw/prompts/temp_folder_prompt.md" cat "${RUNNER_TEMP}/gh-aw/prompts/markdown.md" cat "${RUNNER_TEMP}/gh-aw/prompts/safe_outputs_prompt.md" - cat << 'GH_AW_PROMPT_25cf75111b670167_EOF' + cat << 'GH_AW_PROMPT_2ac1d0b4083428f8_EOF' - Tools: add_comment(max:3), create_pull_request(max:3), update_pull_request(max:3), push_to_pull_request_branch(max:3), missing_tool, missing_data, noop - GH_AW_PROMPT_25cf75111b670167_EOF + Tools: add_comment(max:6), create_pull_request(max:3), update_pull_request(max:3), mark_pull_request_as_ready_for_review(max:3), add_labels(max:3), push_to_pull_request_branch(max:3), missing_tool, missing_data, noop + GH_AW_PROMPT_2ac1d0b4083428f8_EOF cat "${RUNNER_TEMP}/gh-aw/prompts/safe_outputs_create_pull_request.md" cat "${RUNNER_TEMP}/gh-aw/prompts/safe_outputs_push_to_pr_branch.md" - cat << 'GH_AW_PROMPT_25cf75111b670167_EOF' + cat << 'GH_AW_PROMPT_2ac1d0b4083428f8_EOF' - GH_AW_PROMPT_25cf75111b670167_EOF + GH_AW_PROMPT_2ac1d0b4083428f8_EOF cat "${RUNNER_TEMP}/gh-aw/prompts/mcp_cli_tools_prompt.md" - cat << 'GH_AW_PROMPT_25cf75111b670167_EOF' + cat << 'GH_AW_PROMPT_2ac1d0b4083428f8_EOF' The following GitHub context information is available for this workflow: {{#if github.actor}} @@ -332,12 +334,12 @@ jobs: stop immediately and report the limitation rather than spending turns trying to work around it. - GH_AW_PROMPT_25cf75111b670167_EOF + GH_AW_PROMPT_2ac1d0b4083428f8_EOF cat "${RUNNER_TEMP}/gh-aw/prompts/github_mcp_tools_with_safeoutputs_prompt.md" - cat << 'GH_AW_PROMPT_25cf75111b670167_EOF' + cat << 'GH_AW_PROMPT_2ac1d0b4083428f8_EOF' {{#runtime-import .github/workflows/ci-status-fix.md}} - GH_AW_PROMPT_25cf75111b670167_EOF + GH_AW_PROMPT_2ac1d0b4083428f8_EOF } > "$GH_AW_PROMPT" - name: Interpolate variables and render templates uses: actions/github-script@3a2844b7e9c422d3c10d287c895573f7108da1b3 # v9.0.0 @@ -554,16 +556,18 @@ jobs: mkdir -p "${RUNNER_TEMP}/gh-aw/safeoutputs" mkdir -p /tmp/gh-aw/safeoutputs mkdir -p /tmp/gh-aw/mcp-logs/safeoutputs - cat > "${RUNNER_TEMP}/gh-aw/safeoutputs/config.json" << 'GH_AW_SAFE_OUTPUTS_CONFIG_fa2a69fad5fd0485_EOF' - {"add_comment":{"discussions":false,"max":3,"required_labels":["agentic-workflows"],"required_title_prefix":"[ci-fix] ","target":"*"},"create_pull_request":{"allowed_base_branches":["main"],"allowed_branches":["ci-fix/**"],"allowed_files":["src/Core/**","src/Controls/**","src/Essentials/**","src/BlazorWebView/**","src/TestUtils/**","src/Templates/**","**/PublicAPI.Unshipped.txt"],"base_branch":"main","draft":true,"labels":["agentic-workflows"],"max":3,"max_patch_files":100,"max_patch_size":4096,"protect_top_level_dot_folders":true,"protected_files":["package.json","bun.lockb","bunfig.toml","deno.json","deno.jsonc","deno.lock","global.json","NuGet.Config","Directory.Packages.props","mix.exs","mix.lock","go.mod","go.sum","stack.yaml","stack.yaml.lock","pom.xml","build.gradle","build.gradle.kts","settings.gradle","settings.gradle.kts","gradle.properties","package-lock.json","yarn.lock","pnpm-lock.yaml","npm-shrinkwrap.json","requirements.txt","Pipfile","Pipfile.lock","pyproject.toml","setup.py","setup.cfg","Gemfile","Gemfile.lock","uv.lock","CODEOWNERS","DESIGN.md","README.md","CONTRIBUTING.md","CHANGELOG.md","SECURITY.md","CODE_OF_CONDUCT.md","AGENTS.md","CLAUDE.md","GEMINI.md"],"protected_files_policy":"request_review","title_prefix":"[ci-fix] "},"create_report_incomplete_issue":{},"missing_data":{},"missing_tool":{},"noop":{"max":1,"report-as-issue":"true"},"push_to_pull_request_branch":{"allowed_files":["src/Core/**","src/Controls/**","src/Essentials/**","src/BlazorWebView/**","src/TestUtils/**","src/Templates/**","**/PublicAPI.Unshipped.txt"],"if_no_changes":"warn","max":3,"max_patch_size":4096,"protect_top_level_dot_folders":true,"protected_files":["package.json","bun.lockb","bunfig.toml","deno.json","deno.jsonc","deno.lock","global.json","NuGet.Config","Directory.Packages.props","mix.exs","mix.lock","go.mod","go.sum","stack.yaml","stack.yaml.lock","pom.xml","build.gradle","build.gradle.kts","settings.gradle","settings.gradle.kts","gradle.properties","package-lock.json","yarn.lock","pnpm-lock.yaml","npm-shrinkwrap.json","requirements.txt","Pipfile","Pipfile.lock","pyproject.toml","setup.py","setup.cfg","Gemfile","Gemfile.lock","uv.lock","CODEOWNERS","DESIGN.md","README.md","CONTRIBUTING.md","CHANGELOG.md","SECURITY.md","CODE_OF_CONDUCT.md","AGENTS.md","CLAUDE.md","GEMINI.md"],"required_labels":["agentic-workflows"],"target":"*","title_prefix":"[ci-fix] "},"report_incomplete":{},"update_pull_request":{"allow_body":true,"allow_title":false,"max":3,"target":"*","update_branch":false}} - GH_AW_SAFE_OUTPUTS_CONFIG_fa2a69fad5fd0485_EOF + cat > "${RUNNER_TEMP}/gh-aw/safeoutputs/config.json" << 'GH_AW_SAFE_OUTPUTS_CONFIG_0644bf1434ca0071_EOF' + {"add_comment":{"discussions":false,"max":6,"required_labels":["agentic-workflows"],"required_title_prefix":"[ci-fix] ","target":"*"},"add_labels":{"allowed":["p/0"],"max":3,"required_labels":["agentic-workflows"],"required_title_prefix":"[ci-fix] ","target":"*"},"create_pull_request":{"allowed_base_branches":["main"],"allowed_branches":["ci-fix/**"],"allowed_files":["src/Core/**","src/Controls/**","src/Essentials/**","src/BlazorWebView/**","src/TestUtils/**","src/Templates/**","**/PublicAPI.Unshipped.txt"],"base_branch":"main","draft":true,"labels":["agentic-workflows"],"max":3,"max_patch_files":100,"max_patch_size":4096,"protect_top_level_dot_folders":true,"protected_files":["package.json","bun.lockb","bunfig.toml","deno.json","deno.jsonc","deno.lock","global.json","NuGet.Config","Directory.Packages.props","mix.exs","mix.lock","go.mod","go.sum","stack.yaml","stack.yaml.lock","pom.xml","build.gradle","build.gradle.kts","settings.gradle","settings.gradle.kts","gradle.properties","package-lock.json","yarn.lock","pnpm-lock.yaml","npm-shrinkwrap.json","requirements.txt","Pipfile","Pipfile.lock","pyproject.toml","setup.py","setup.cfg","Gemfile","Gemfile.lock","uv.lock","CODEOWNERS","DESIGN.md","README.md","CONTRIBUTING.md","CHANGELOG.md","SECURITY.md","CODE_OF_CONDUCT.md","AGENTS.md","CLAUDE.md","GEMINI.md"],"protected_files_policy":"request_review","title_prefix":"[ci-fix] "},"create_report_incomplete_issue":{},"mark_pull_request_as_ready_for_review":{"max":3,"required_labels":["agentic-workflows"],"required_title_prefix":"[ci-fix] ","target":"*"},"missing_data":{},"missing_tool":{},"noop":{"max":1,"report-as-issue":"true"},"push_to_pull_request_branch":{"allowed_files":["src/Core/**","src/Controls/**","src/Essentials/**","src/BlazorWebView/**","src/TestUtils/**","src/Templates/**","**/PublicAPI.Unshipped.txt"],"if_no_changes":"warn","max":3,"max_patch_size":4096,"protect_top_level_dot_folders":true,"protected_files":["package.json","bun.lockb","bunfig.toml","deno.json","deno.jsonc","deno.lock","global.json","NuGet.Config","Directory.Packages.props","mix.exs","mix.lock","go.mod","go.sum","stack.yaml","stack.yaml.lock","pom.xml","build.gradle","build.gradle.kts","settings.gradle","settings.gradle.kts","gradle.properties","package-lock.json","yarn.lock","pnpm-lock.yaml","npm-shrinkwrap.json","requirements.txt","Pipfile","Pipfile.lock","pyproject.toml","setup.py","setup.cfg","Gemfile","Gemfile.lock","uv.lock","CODEOWNERS","DESIGN.md","README.md","CONTRIBUTING.md","CHANGELOG.md","SECURITY.md","CODE_OF_CONDUCT.md","AGENTS.md","CLAUDE.md","GEMINI.md"],"required_labels":["agentic-workflows"],"target":"*","title_prefix":"[ci-fix] "},"report_incomplete":{},"update_pull_request":{"allow_body":true,"allow_title":false,"max":3,"target":"*","update_branch":false}} + GH_AW_SAFE_OUTPUTS_CONFIG_0644bf1434ca0071_EOF - name: Generate Safe Outputs Tools env: GH_AW_TOOLS_META_JSON: | { "description_suffixes": { - "add_comment": " CONSTRAINTS: Maximum 3 comment(s) can be added. Target: *. Supports reply_to_id for discussion threading.", + "add_comment": " CONSTRAINTS: Maximum 6 comment(s) can be added. Target: *. Supports reply_to_id for discussion threading.", + "add_labels": " CONSTRAINTS: Maximum 3 label(s) can be added. Only these labels are allowed: [\"p/0\"]. Target: *.", "create_pull_request": " CONSTRAINTS: Maximum 3 pull request(s) can be created. Title will be prefixed with \"[ci-fix] \". Labels [\"agentic-workflows\"] will be automatically added. Only these labels are allowed: [\"agentic-workflows\"]. PRs will be created as drafts.", + "mark_pull_request_as_ready_for_review": " CONSTRAINTS: Maximum 3 pull request(s) can be marked as ready for review.", "push_to_pull_request_branch": " CONSTRAINTS: Maximum 3 push(es) can be made. The target pull request title must start with \"[ci-fix] \".", "update_pull_request": " CONSTRAINTS: Maximum 3 pull request(s) can be updated. Target: *." }, @@ -594,6 +598,25 @@ jobs: } } }, + "add_labels": { + "defaultMax": 5, + "fields": { + "item_number": { + "issueNumberOrTemporaryId": true + }, + "labels": { + "required": true, + "type": "array", + "itemType": "string", + "itemSanitize": true, + "itemMaxLength": 128 + }, + "repo": { + "type": "string", + "maxLength": 256 + } + } + }, "create_pull_request": { "defaultMax": 1, "fields": { @@ -635,6 +658,24 @@ jobs: } } }, + "mark_pull_request_as_ready_for_review": { + "defaultMax": 1, + "fields": { + "pull_request_number": { + "issueOrPRNumber": true + }, + "reason": { + "required": true, + "type": "string", + "sanitize": true, + "maxLength": 65000 + }, + "repo": { + "type": "string", + "maxLength": 256 + } + } + }, "missing_data": { "defaultMax": 20, "fields": { @@ -1494,7 +1535,7 @@ jobs: uses: actions/github-script@3a2844b7e9c422d3c10d287c895573f7108da1b3 # v9.0.0 env: WORKFLOW_NAME: "CI Failure Fixer (main)" - WORKFLOW_DESCRIPTION: "Periodic pass over open ci-scan tracking issues filed by the main-branch CI\nfailure scanner (.github/workflows/ci-status-main.md). This workflow targets\nthe `main` branch EXCLUSIVELY: it processes only issues labelled ci-scan and\nopens every PR against main. (The net11.0 branch is handled by the parallel\n.github/workflows/ci-status-fix-net11.md — the two are split because gh-aw can\nonly transport a fix relative to ONE static base branch per workflow, and the\nmain↔net11.0 divergence exceeds gh-aw's 10 MB transport-patch cap.) The fixer\nopens ONE draft [ci-fix] PR per actionable issue against main, then WATCHES\nthat PR's own CI on later runs: when the fix's CI comes back red and the red is\ncaused by the fix itself, it pushes a fresh follow-up fix onto the SAME PR\nbranch (never a second PR) — up to 10 attempts — then stops and defers to\nhumans (the open tracking issue is the hand-off surface; a dedicated\n[ci-fix][needs-human] PR is planned but currently deferred — see Step 6). CI on\nthe PR is kicked by a human `/azp run` for now; the loop watches, classifies,\nand re-fixes autonomously between kicks.\nNever mutes tests, but\nde-flakes genuinely flaky ones (deterministic synchronization, no retries /\ntimeout bumps). Always skips visual-regression / screenshot issues." + WORKFLOW_DESCRIPTION: "Periodic pass over open ci-scan tracking issues filed by the main-branch CI\nfailure scanner (.github/workflows/ci-status-main.md). This workflow targets\nthe `main` branch EXCLUSIVELY: it processes only issues labelled ci-scan and\nopens every PR against main. (The net11.0 branch is handled by the parallel\n.github/workflows/ci-status-fix-net11.md — the two are split because gh-aw can\nonly transport a fix relative to ONE static base branch per workflow, and the\nmain↔net11.0 divergence exceeds gh-aw's 10 MB transport-patch cap.) The fixer\nopens ONE draft [ci-fix] PR per actionable issue against main, then WATCHES\nthat PR's own CI on later runs: when the fix's CI comes back red and the red is\ncaused by the fix itself, it pushes a fresh follow-up fix onto the SAME PR\nbranch (never a second PR) — up to 10 attempts — then stops and defers to\nhumans (the open tracking issue is the hand-off surface; a dedicated\n[ci-fix][needs-human] PR is planned but currently deferred — see Step 6). CI on\nthe PR is kicked by a human `/azp run` for now; the loop watches, classifies,\nand re-fixes autonomously between kicks. When the SPECIFIC test a PR fixed is\nconfirmed green in that PR's own CI, the loop marks the draft PR ready for review\n(a state transition only — it never approves or merges).\nNever mutes tests, but\nde-flakes genuinely flaky ones (deterministic synchronization, no retries /\ntimeout bumps). Always skips visual-regression / screenshot issues." HAS_PATCH: ${{ needs.agent.outputs.has_patch }} with: script: | @@ -1910,7 +1951,7 @@ jobs: GH_AW_ALLOWED_DOMAINS: "*.blob.core.windows.net,*.githubusercontent.com,api.business.githubcopilot.com,api.enterprise.githubcopilot.com,api.github.com,api.githubcopilot.com,api.individual.githubcopilot.com,api.snapcraft.io,archive.ubuntu.com,azure.archive.ubuntu.com,codeload.github.com,crl.geotrust.com,crl.globalsign.com,crl.identrust.com,crl.sectigo.com,crl.thawte.com,crl.usertrust.com,crl.verisign.com,crl3.digicert.com,crl4.digicert.com,crls.ssl.com,dev.azure.com,docs.github.com,github-cloud.githubusercontent.com,github-cloud.s3.amazonaws.com,github.blog,github.com,github.githubassets.com,helix.dot.net,host.docker.internal,json-schema.org,json.schemastore.org,keyserver.ubuntu.com,lfs.github.com,objects.githubusercontent.com,ocsp.digicert.com,ocsp.geotrust.com,ocsp.globalsign.com,ocsp.identrust.com,ocsp.sectigo.com,ocsp.ssl.com,ocsp.thawte.com,ocsp.usertrust.com,ocsp.verisign.com,packagecloud.io,packages.cloud.google.com,packages.microsoft.com,patch-diff.githubusercontent.com,ppa.launchpad.net,raw.githubusercontent.com,registry.npmjs.org,s.symcb.com,s.symcd.com,security.ubuntu.com,telemetry.enterprise.githubcopilot.com,ts-crl.ws.symantec.com,ts-ocsp.ws.symantec.com,www.googleapis.com" GITHUB_SERVER_URL: ${{ github.server_url }} GITHUB_API_URL: ${{ github.api_url }} - GH_AW_SAFE_OUTPUTS_HANDLER_CONFIG: "{\"add_comment\":{\"discussions\":false,\"max\":3,\"required_labels\":[\"agentic-workflows\"],\"required_title_prefix\":\"[ci-fix] \",\"target\":\"*\"},\"create_pull_request\":{\"allowed_base_branches\":[\"main\"],\"allowed_branches\":[\"ci-fix/**\"],\"allowed_files\":[\"src/Core/**\",\"src/Controls/**\",\"src/Essentials/**\",\"src/BlazorWebView/**\",\"src/TestUtils/**\",\"src/Templates/**\",\"**/PublicAPI.Unshipped.txt\"],\"base_branch\":\"main\",\"draft\":true,\"labels\":[\"agentic-workflows\"],\"max\":3,\"max_patch_files\":100,\"max_patch_size\":4096,\"protect_top_level_dot_folders\":true,\"protected_files\":[\"package.json\",\"bun.lockb\",\"bunfig.toml\",\"deno.json\",\"deno.jsonc\",\"deno.lock\",\"global.json\",\"NuGet.Config\",\"Directory.Packages.props\",\"mix.exs\",\"mix.lock\",\"go.mod\",\"go.sum\",\"stack.yaml\",\"stack.yaml.lock\",\"pom.xml\",\"build.gradle\",\"build.gradle.kts\",\"settings.gradle\",\"settings.gradle.kts\",\"gradle.properties\",\"package-lock.json\",\"yarn.lock\",\"pnpm-lock.yaml\",\"npm-shrinkwrap.json\",\"requirements.txt\",\"Pipfile\",\"Pipfile.lock\",\"pyproject.toml\",\"setup.py\",\"setup.cfg\",\"Gemfile\",\"Gemfile.lock\",\"uv.lock\",\"CODEOWNERS\",\"DESIGN.md\",\"README.md\",\"CONTRIBUTING.md\",\"CHANGELOG.md\",\"SECURITY.md\",\"CODE_OF_CONDUCT.md\",\"AGENTS.md\",\"CLAUDE.md\",\"GEMINI.md\"],\"protected_files_policy\":\"request_review\",\"title_prefix\":\"[ci-fix] \"},\"create_report_incomplete_issue\":{},\"missing_data\":{},\"missing_tool\":{},\"noop\":{\"max\":1,\"report-as-issue\":\"true\"},\"push_to_pull_request_branch\":{\"allowed_files\":[\"src/Core/**\",\"src/Controls/**\",\"src/Essentials/**\",\"src/BlazorWebView/**\",\"src/TestUtils/**\",\"src/Templates/**\",\"**/PublicAPI.Unshipped.txt\"],\"if_no_changes\":\"warn\",\"max\":3,\"max_patch_size\":4096,\"protect_top_level_dot_folders\":true,\"protected_files\":[\"package.json\",\"bun.lockb\",\"bunfig.toml\",\"deno.json\",\"deno.jsonc\",\"deno.lock\",\"global.json\",\"NuGet.Config\",\"Directory.Packages.props\",\"mix.exs\",\"mix.lock\",\"go.mod\",\"go.sum\",\"stack.yaml\",\"stack.yaml.lock\",\"pom.xml\",\"build.gradle\",\"build.gradle.kts\",\"settings.gradle\",\"settings.gradle.kts\",\"gradle.properties\",\"package-lock.json\",\"yarn.lock\",\"pnpm-lock.yaml\",\"npm-shrinkwrap.json\",\"requirements.txt\",\"Pipfile\",\"Pipfile.lock\",\"pyproject.toml\",\"setup.py\",\"setup.cfg\",\"Gemfile\",\"Gemfile.lock\",\"uv.lock\",\"CODEOWNERS\",\"DESIGN.md\",\"README.md\",\"CONTRIBUTING.md\",\"CHANGELOG.md\",\"SECURITY.md\",\"CODE_OF_CONDUCT.md\",\"AGENTS.md\",\"CLAUDE.md\",\"GEMINI.md\"],\"required_labels\":[\"agentic-workflows\"],\"target\":\"*\",\"title_prefix\":\"[ci-fix] \"},\"report_incomplete\":{},\"update_pull_request\":{\"allow_body\":true,\"allow_title\":false,\"max\":3,\"target\":\"*\",\"update_branch\":false}}" + GH_AW_SAFE_OUTPUTS_HANDLER_CONFIG: "{\"add_comment\":{\"discussions\":false,\"max\":6,\"required_labels\":[\"agentic-workflows\"],\"required_title_prefix\":\"[ci-fix] \",\"target\":\"*\"},\"add_labels\":{\"allowed\":[\"p/0\"],\"max\":3,\"required_labels\":[\"agentic-workflows\"],\"required_title_prefix\":\"[ci-fix] \",\"target\":\"*\"},\"create_pull_request\":{\"allowed_base_branches\":[\"main\"],\"allowed_branches\":[\"ci-fix/**\"],\"allowed_files\":[\"src/Core/**\",\"src/Controls/**\",\"src/Essentials/**\",\"src/BlazorWebView/**\",\"src/TestUtils/**\",\"src/Templates/**\",\"**/PublicAPI.Unshipped.txt\"],\"base_branch\":\"main\",\"draft\":true,\"labels\":[\"agentic-workflows\"],\"max\":3,\"max_patch_files\":100,\"max_patch_size\":4096,\"protect_top_level_dot_folders\":true,\"protected_files\":[\"package.json\",\"bun.lockb\",\"bunfig.toml\",\"deno.json\",\"deno.jsonc\",\"deno.lock\",\"global.json\",\"NuGet.Config\",\"Directory.Packages.props\",\"mix.exs\",\"mix.lock\",\"go.mod\",\"go.sum\",\"stack.yaml\",\"stack.yaml.lock\",\"pom.xml\",\"build.gradle\",\"build.gradle.kts\",\"settings.gradle\",\"settings.gradle.kts\",\"gradle.properties\",\"package-lock.json\",\"yarn.lock\",\"pnpm-lock.yaml\",\"npm-shrinkwrap.json\",\"requirements.txt\",\"Pipfile\",\"Pipfile.lock\",\"pyproject.toml\",\"setup.py\",\"setup.cfg\",\"Gemfile\",\"Gemfile.lock\",\"uv.lock\",\"CODEOWNERS\",\"DESIGN.md\",\"README.md\",\"CONTRIBUTING.md\",\"CHANGELOG.md\",\"SECURITY.md\",\"CODE_OF_CONDUCT.md\",\"AGENTS.md\",\"CLAUDE.md\",\"GEMINI.md\"],\"protected_files_policy\":\"request_review\",\"title_prefix\":\"[ci-fix] \"},\"create_report_incomplete_issue\":{},\"mark_pull_request_as_ready_for_review\":{\"max\":3,\"required_labels\":[\"agentic-workflows\"],\"required_title_prefix\":\"[ci-fix] \",\"target\":\"*\"},\"missing_data\":{},\"missing_tool\":{},\"noop\":{\"max\":1,\"report-as-issue\":\"true\"},\"push_to_pull_request_branch\":{\"allowed_files\":[\"src/Core/**\",\"src/Controls/**\",\"src/Essentials/**\",\"src/BlazorWebView/**\",\"src/TestUtils/**\",\"src/Templates/**\",\"**/PublicAPI.Unshipped.txt\"],\"if_no_changes\":\"warn\",\"max\":3,\"max_patch_size\":4096,\"protect_top_level_dot_folders\":true,\"protected_files\":[\"package.json\",\"bun.lockb\",\"bunfig.toml\",\"deno.json\",\"deno.jsonc\",\"deno.lock\",\"global.json\",\"NuGet.Config\",\"Directory.Packages.props\",\"mix.exs\",\"mix.lock\",\"go.mod\",\"go.sum\",\"stack.yaml\",\"stack.yaml.lock\",\"pom.xml\",\"build.gradle\",\"build.gradle.kts\",\"settings.gradle\",\"settings.gradle.kts\",\"gradle.properties\",\"package-lock.json\",\"yarn.lock\",\"pnpm-lock.yaml\",\"npm-shrinkwrap.json\",\"requirements.txt\",\"Pipfile\",\"Pipfile.lock\",\"pyproject.toml\",\"setup.py\",\"setup.cfg\",\"Gemfile\",\"Gemfile.lock\",\"uv.lock\",\"CODEOWNERS\",\"DESIGN.md\",\"README.md\",\"CONTRIBUTING.md\",\"CHANGELOG.md\",\"SECURITY.md\",\"CODE_OF_CONDUCT.md\",\"AGENTS.md\",\"CLAUDE.md\",\"GEMINI.md\"],\"required_labels\":[\"agentic-workflows\"],\"target\":\"*\",\"title_prefix\":\"[ci-fix] \"},\"report_incomplete\":{},\"update_pull_request\":{\"allow_body\":true,\"allow_title\":false,\"max\":3,\"target\":\"*\",\"update_branch\":false}}" GH_AW_CI_TRIGGER_TOKEN: ${{ secrets.GH_AW_CI_TRIGGER_TOKEN }} with: github-token: ${{ secrets.GH_AW_GITHUB_TOKEN || secrets.GITHUB_TOKEN }} diff --git a/.github/workflows/ci-status-fix.md b/.github/workflows/ci-status-fix.md index d3227e25170c..9c2f46054029 100644 --- a/.github/workflows/ci-status-fix.md +++ b/.github/workflows/ci-status-fix.md @@ -15,7 +15,9 @@ description: | humans (the open tracking issue is the hand-off surface; a dedicated [ci-fix][needs-human] PR is planned but currently deferred — see Step 6). CI on the PR is kicked by a human `/azp run` for now; the loop watches, classifies, - and re-fixes autonomously between kicks. + and re-fixes autonomously between kicks. When the SPECIFIC test a PR fixed is + confirmed green in that PR's own CI, the loop marks the draft PR ready for review + (a state transition only — it never approves or merges). Never mutes tests, but de-flakes genuinely flaky ones (deterministic synchronization, no retries / timeout bumps). Always skips visual-regression / screenshot issues. @@ -239,8 +241,15 @@ safe-outputs: # PER-RUN total (not per-PR): a sweep may surface several green PRs and/or # annotate several flaky reds in one run. At max:1 all but the first comment # were silently dropped, starving the workflow's #1 value (surfacing green PRs - # for review). Raised to 3 to match the per-run throughput of the other outputs. - max: 3 + # for review). Sized to 6 (not 3) because a single green DRAFT PR that gets + # flipped ready in one sweep spends TWO comment slots — the Step 3 ✅ surface + # comment (which still names the /azp-gated legs a human must kick, so it is NOT + # redundant with 🎯) AND the Step 3.6 T3 🎯 readiness comment. At max:3 the shared + # bucket drained after ~1 draft flip and T3's atomicity pre-check then deferred + # every further mark-ready even while the mark-ready/add_labels buckets (also 3) + # sat idle; 6 lets ~3 draft flips (2 comments each) land per sweep, so the + # mark-ready:3 / add_labels:3 caps are actually reachable. + max: 6 target: "*" # Hard constraint (defense-in-depth): only comment on THIS workflow's own # ci-fix PRs — [ci-fix] title prefix AND agentic-workflows label. Mirrors the @@ -261,13 +270,51 @@ safe-outputs: # never the title, so disable title rewrites — this compiles to allow_title:false # and removes any ability to retitle an arbitrary PR. title: false - # NOTE: gh-aw v0.79.8 does NOT emit required-title-prefix/required-labels into + # NOTE: gh-aw v0.80.9 (the pinned compiler) does NOT emit required-title-prefix/required-labels into # the compiled config for update-pull-request (verified against the lock — it # silently drops them, unlike add-comment / push-to-pull-request-branch which # honor them). So which-PR scoping here relies on prompt Hard-Rule 6 + # min-integrity:approved; the residual is body-marker edits only (no code, no # merge, no close), capped at max:3. Revisit required-* if a future gh-aw # compiler emits them for this output. + mark-pull-request-as-ready-for-review: + # Flip a validated draft [ci-fix] PR to ready-for-review once the SPECIFIC test + # this PR was opened to fix is confirmed green in the PR's OWN CI (Step 3.6) — + # even when unrelated legs are red. The workflow is schedule/dispatch-triggered + # (no triggering-PR context), so target "*" lets the agent name the PR number it + # validated. This is a state transition ONLY: it never approves and never merges — + # a human still reviews and merges. + # PER-RUN total (not per-PR): a sweep may validate several PRs' target tests green. + max: 3 + target: "*" + # Hard constraint (defense-in-depth): only ever un-draft THIS workflow's own + # ci-fix PRs — [ci-fix] title prefix AND agentic-workflows label — so a confused + # or prompt-injected agent cannot mark an arbitrary PR ready. (If the v0.80.9 + # compiler silently drops these — as documented for update-pull-request above — + # the Step 3.6 preconditions + min-integrity:approved are the compensating scope + # controls; verify against the lock after compiling.) + required-title-prefix: "[ci-fix] " + required-labels: [agentic-workflows] + add-labels: + # Apply the p/0 priority label to a [ci-fix] PR at the exact moment the loop flips it + # from draft to ready-for-review (Step 3.6 T3) — so a validated, review-ready fix lands + # in the team's p/0 triage queue instead of sitting unseen in the draft backlog. Paired + # 1:1 with mark-pull-request-as-ready-for-review; target "*" lets the agent name the PR + # it just validated. + # allowed = HARD allowlist: the agent may ONLY ever add p/0, nothing else. This caps the + # blast radius of a confused/prompt-injected agent to exactly one benign priority label — + # it can never apply a downstream-triggering or destructive label. + allowed: [p/0] + # PER-RUN total (not per-PR): a sweep may mark several PRs ready in one run; each adds + # one label, so this matches the mark-ready per-run cap (3). + max: 3 + target: "*" + # Hard constraint (defense-in-depth): only ever label THIS workflow's own ci-fix PRs — + # [ci-fix] title prefix AND agentic-workflows label. (If the v0.80.9 compiler drops these + # — as documented for update-pull-request above — the Step 3.6 preconditions + + # min-integrity:approved + the allowed:[p/0] allowlist are the compensating controls.) + required-title-prefix: "[ci-fix] " + required-labels: [agentic-workflows] timeout-minutes: 90 @@ -339,33 +386,48 @@ through `safe-outputs`. 5. **One issue = one outcome per run.** Exactly one of: respond to a maintainer's change-request on the open PR (Track C, Step 3.5.R); open the first fix/help/ de-flake PR; advance an existing PR by one attempt (push a follow-up fix); - surface a validated-green PR for review; annotate an unrelated-flake red; wait + surface a validated-green PR for review; mark a target-validated draft PR + ready-for-review (Step 3.6 T3 — the terminal outcome that supersedes the same + run's surface/annotate precursor line), or record its `already-ready` + steady-state no-op; annotate an unrelated-flake red; wait (CI not yet settled); or a recorded skip (the dedicated needs-human PR is deferred — the attempt cap records a skip instead; see Step 6). Always prefer advancing or opening a PR over a skip when a non-mute diff is producible. 6. **All writes via `safe-outputs`.** Allowed outputs: `create_pull_request` (first attempt only), `push_to_pull_request_branch` (advance an existing PR by one attempt), `update_pull_request` (bump the attempt marker / refresh the - prior-attempts table), and `add_comment` (progress notes on the `[ci-fix]` PR - ONLY). NEVER comment on the tracking issue (issues are locked by + prior-attempts table), `add_comment` (progress notes on the `[ci-fix]` PR + ONLY), `mark_pull_request_as_ready_for_review` (flip a target-validated draft + `[ci-fix]` PR from draft to ready — Step 3.6 T3 only), and `add_labels` (add + ONLY the `p/0` label, ONLY on that same draft→ready transition — Step 3.6 T3). + NEVER comment on the tracking issue (issues are locked by `.github/workflows/ci-scan-lock-issues.yml`) — `add_comment` targets the PR. No `gh pr create`, no manual `git push`. **Defense-in-depth:** `add_comment` and `update_pull_request` use `target: "*"` (the agent supplies the PR number). - `add_comment` is now config-locked to `required-title-prefix: "[ci-fix] "` + + `add_comment`, `mark_pull_request_as_ready_for_review`, and `add_labels` are + config-locked to `required-title-prefix: "[ci-fix] "` + `required-labels: [agentic-workflows]` (hard-enforced by the handler, same as - `push_to_pull_request_branch`), so a comment can only ever land on THIS - workflow's own PRs. `update_pull_request` CANNOT be config-locked to a - title/label in gh-aw v0.79.8 (the compiler silently drops `required-*` for that - output — verified against the lock), so it keeps `title: false` (no retitles; - body-marker edits only) plus this prompt-level guard. Before emitting either, - VERIFY the target PR carries BOTH the `[ci-fix]` title prefix AND the - `agentic-workflows` label; never comment on or edit the body of any PR that - lacks both. + `push_to_pull_request_branch`), and `add_labels` additionally has an + `allowed: [p/0]` allowlist so it can ONLY ever add `p/0` — so these can only + ever land on THIS workflow's own PRs. `update_pull_request` CANNOT be + config-locked to a title/label in gh-aw v0.80.9 (the compiler silently drops + `required-*` for that output — verified against the lock), so it keeps + `title: false` (no retitles; body-marker edits only) plus this prompt-level + guard. Before emitting ANY of these, VERIFY the target PR carries BOTH the + `[ci-fix]` title prefix AND the `agentic-workflows` label; never comment on, + edit, un-draft, or label any PR that lacks both. 7. **Per-run safe-output caps (per RUN, not per PR).** Each output type is capped per run: `create_pull_request` 3, `push_to_pull_request_branch` 3, - `add_comment` 3, `update_pull_request` 3. Note an ADVANCE spends one push **and** - one update **and** one comment, so ≤ 3 advances/run; surfacing a green or - annotating a flake spends one comment. When a bucket is exhausted, do NOT keep + `add_comment` 6, `update_pull_request` 3, `mark_pull_request_as_ready_for_review` + 3, `add_labels` 3 (counts LABELS, not calls). Note an ADVANCE spends one push + **and** one update **and** one comment, so ≤ 3 advances/run; surfacing a green or + annotating a flake spends one comment; a **mark-ready (Step 3.6 T3)** of a + still-draft PR spends TWO comments (the Step 3 ✅ surface **and** the T3 🎯 + readiness note) **and** one mark-ready **and** one label as an ALL-OR-NOTHING set + (T3 pre-checks those buckets and defers the whole PR if any is exhausted — never + mark a PR ready without its 🎯 audit comment). `add_comment` is therefore sized 6 + (not 3) so ~3 draft flips, at 2 comments each, can land in one sweep instead of the + shared comment bucket starving the otherwise-idle mark-ready/add_labels buckets. When a bucket is exhausted, do NOT keep emitting (extras are silently dropped) — record `skipped: per-run cap reached; deferring PR # to next cycle` for each remaining PR so the drop is a deliberate, logged decision. The continuous loop picks the deferred PRs up next @@ -409,8 +471,9 @@ For every open tracking issue in scope, converge on exactly one outcome: | Open first help-wanted draft `[ci-fix]` PR | No open PR yet; a plausible candidate exists but cannot be runner-validated (device/UI tests) (attempt 1) | | Open first de-flake draft `[ci-fix]` PR | No open PR yet; failure is intermittent (green-on-retry) from a genuine test-quality defect; a deterministic-synchronization fix is producible (attempt 1, Step 4.7 bucket b) | | Advance an existing PR (push attempt N+1) | Open PR's own CI settled red *because of the fix itself*, no human engaged, marker < 10 — push a NEW distinct fix onto the same branch (Step 3.5 → 5.6) | -| Surface a validated-green PR | Open PR's own CI settled green — comment "validated, attempt N/10; ready for review" and do NOT advance (Step 3.5) | -| Annotate an unrelated-flake red | Open PR is red only on baseline-flake / unrelated legs — comment which leg needs a re-run; do NOT burn an attempt (Step 3.5) | +| Surface a validated-green PR | Open PR's own CI settled green — comment "validated, attempt N/10; ready for review" and do NOT advance (Step 3.5), then mark the draft PR ready for review once the fixed test is confirmed green (Step 3.6) | +| Annotate an unrelated-flake red | Open PR is red only on baseline-flake / unrelated legs — comment which leg needs a re-run; do NOT burn an attempt (Step 3.5); if the SPECIFIC fixed test is confirmed green on that build, still mark the draft PR ready for review (Step 3.6) | +| Mark a validated PR ready for review | The specific test this PR fixed is confirmed green in the PR's own CI (even if unrelated legs are red) — comment the target-test result and transition the draft PR to ready for review; never approve or merge (Step 3.6) | | Wait | Open PR's CI is pending / not yet settled on the current head SHA — do nothing this cycle (Step 3.5) | | Hand-off skip (attempt cap) | Marker == 10 and the signature still reproduces — stop and defer to humans; the dedicated `[ci-fix][needs-human]` PR is planned but currently deferred (Step 6) | | Recorded skip | Visual-regression, human engaged, already-handled, fixed-in-latest-build, infra-flake, out-of-bounds, only-mute-available, or no novel approach producible | @@ -685,14 +748,34 @@ Run these gates in order — the FIRST that fires decides this cycle's outcome: 1. **Human engaged.** If `C.humanEngaged` → `skipped: human engaged on PR #; deferring` and stop. Never fight a human reviewer — the intentional hand-off boundary still holds the moment a person touches the PR. -2. **CI not settled / unknown / incomplete prefetch.** If `C.checksSettled == false` - OR `C.overallConclusion` is `pending`, `neutral`, or `unknown`, OR - `C.dataComplete == false` → the fix's CI has not finished on the current head SHA - (`C.headSha`), or the prefetch could not fully read this PR (a partial read can - understate `humanEngaged` / `attempt`). `skipped: PR # CI pending / prefetch - incomplete on ; waiting` and stop. *(Round 1: this is where a - maintainer `/azp run` is awaited — the `/azp`-gated uitests/devicetests legs will - not have run until a human kicks them.)* +2. **CI not settled — but validate the target test first (target-focused readiness).** + If `C.checksSettled == false` OR `C.overallConclusion` is `pending`, `neutral`, or + `unknown`, OR `C.dataComplete == false` → the *overall* build has not finished on the + current head SHA (`C.headSha`), or the prefetch could not fully read this PR (a partial + read can understate `humanEngaged` / `attempt`). Unrelated legs still draining must NOT + keep an already-proven fix parked as a draft, so before waiting, try the target-focused + fast-path: + - **2a — Target-green fast-path (draft `[ci-fix]` PRs only).** If `C.dataComplete == + true` AND the Step 3.6 preconditions hold (draft PR, `[ci-fix] ` title prefix AND the + `agentic-workflows` label), run Step 3.6 **T1–T2** against `C.headSha` now: identify + the target test(s) and read the PR's OWN build **timeline / per-leg status** (anonymous + `_apis/build` — never `_apis/test`, per Hard-Rule 8). If EVERY target test is + **VALIDATED-GREEN** (per T2's all-platform definition below — the target's category leg + is `succeeded` on every platform that runs it, failed on none, and NO target platform + family the pipeline covers is still pending/unconcluded, per T2's "no platform left + unverified" rule; a platform that simply has no leg for the target's category — the test + doesn't run there — is fine, not a gap) AND every leg in + `C.failedLegs` that has ALREADY concluded classifies as **unrelated flake** (Step 4 / + Step 4.7 method — the fix is implicated in NO completed red), then the fix is proven + regardless of unrelated *pending* legs → run **Step 3.6 T3** (mark ready + 🎯 comment) + and stop. This early-out NEVER advances an attempt and NEVER acts on a red that could + be the fix's fault. + - **2b — Otherwise WAIT.** If the target test has not yet executed (pending/absent on + its leg), or a completed red is (or may be) caused by the fix, or `C.dataComplete == + false`, or this PR has no identifiable target test → `skipped: PR # CI pending / + target not yet validated on ; waiting` and stop. *(Round 1: this is where a + maintainer `/azp run` is awaited — the `/azp`-gated uitests/devicetests legs will not + have run until a human kicks them.)* 3. **Green → surface for review.** If `C.overallConclusion == "success"`: the checks that RAN are green. **Primary-gate check first:** the deterministic `success` verdict only certifies "at least one green check, nothing failing or @@ -704,18 +787,31 @@ Run these gates in order — the FIRST that fires decides this cycle's outcome: # primary CI gate (maui-pr) not green on ; waiting` and stop — do NOT surface. (The `/azp`-gated `maui-pr-uitests` (def 313) / `maui-pr-devicetests` (def 314) legs MAY still be un-run — that is expected and is named below, not a - reason to withhold the surface.) **Idempotency:** scan the PR's existing comments - for a prior bot `✅ … validated … on ` note for THIS head SHA — if one - already exists, record `already-surfaced PR # (head )` and stop - WITHOUT re-commenting (re-surfacing the same green every tick is noise and burns - the per-run comment budget other PRs need). Otherwise resolve the attempt number from - `C.effectiveAttempt` (the authoritative max(marker, bot-commit) counter; omit the - number only if it is somehow indeterminate). **Dry-run gate (Step 0):** if `dry_run == "true"`, do NOT emit any comment — instead print the intended `✅` validated-green body to the run log, tally `dry-run: would-surface-green PR #`, and stop. Otherwise `add_comment` on PR #: `✅ Attempt /10 validated - — the fix's CI is green on . Ready for human review.` - Name any `/azp`-gated legs (uitests def 313 / devicetests def 314) that have not - run and still need a maintainer `/azp run`. Do NOT advance. Record - `surfaced-green PR # (attempt /10)` and stop. *(This directly - attacks the real bottleneck — no reviews — so it is the highest-value outcome.)* + reason to withhold the surface.) **Comment idempotency + dry-run suppress the ✅ + comment ONLY — neither skips Step 3.6.** Scan the PR's existing comments for a prior + bot `✅ … validated … on ` note for THIS head SHA; if one exists, set + `SKIP_SURFACE_COMMENT = true`. Resolve the attempt number from `C.effectiveAttempt` + (the authoritative max(marker, bot-commit) counter; omit the number only if it is + somehow indeterminate). Then post the surface comment UNLESS suppressed: + - if `dry_run == "true"`: do NOT emit — print the intended `✅` validated-green body to + the run log and tally `dry-run: would-surface-green PR #`; + - else if `SKIP_SURFACE_COMMENT`: do NOT re-post — record `already-surfaced PR # + (head )` (re-surfacing the same green every tick is noise and burns the + per-run comment budget other PRs need); + - else `add_comment` on PR #: `✅ Attempt /10 validated — the fix's CI is + green on .` naming any `/azp`-gated legs (uitests def 313 / devicetests def + 314) that have not run and still need a maintainer `/azp run`; record `surfaced-green + PR # (attempt /10)`. (Do NOT assert "ready for human review" on a PR that + is still a draft — Step 3.6, not this comment, owns the draft→ready flip and posts its + own 🎯 announcement when it fires.) + Do NOT advance. Then — in ALL of the above cases — run **Step 3.6** (target-test + readiness gate) for this PR before stopping: its `/azp`-gated target legs may conclude + green on this SAME head SHA on a LATER sweep (an `/azp run` adds no commit), and Step + 3.6 is the ONLY place the draft→ready flip happens, so it MUST re-evaluate every sweep — + never `stop` here before it. *(This directly attacks the real bottleneck — no reviews — + so it is the highest-value outcome.)* 4. **Red → classify caused-by-fix vs unrelated-flake.** If `C.overallConclusion == "failure"`, analyze the PR's OWN failing build (NOT `main`): find the AzDO `maui-pr` build for this PR (filter builds by `branchName=refs/pull//merge`, @@ -723,18 +819,26 @@ Run these gates in order — the FIRST that fires decides this cycle's outcome: method and Step 4.7 flake buckets to `C.failedLegs`. - **Unrelated flake only** (every failed leg is known-flaky / infra / a pre-existing baseline red NOT introduced by the fix): do NOT burn an attempt. - **Idempotency first:** scan the PR's existing comments for a prior bot - `♻️ … unrelated flake … on ` note for THIS head SHA — if one already - exists, record `already-annotated-flake PR # (head )` and stop - WITHOUT re-commenting (re-annotating the same flake every 12h sweep is noise and - burns the per-run comment budget other PRs need). Otherwise resolve the attempt - number from `C.effectiveAttempt` (the authoritative max(marker, bot-commit) - counter), omitting the `Attempt /10:` prefix only if it is somehow - indeterminate. **Dry-run gate (Step 0):** if `dry_run == "true"`, do NOT emit any comment — instead print the intended `♻️` unrelated-flake body to the run log, tally `dry-run: would-annotate-flake PR #`, and stop. Otherwise `add_comment` on PR #: `♻️ Attempt /10: red is - unrelated flake on leg(s) () on ; the fix itself is not - implicated. A maintainer re-run (/azp run ) should clear it.` Record - `annotated-flake PR # (head )` and stop. *(Round 1: human re-runs; - Round 2: auto re-trigger.)* + **Comment idempotency + dry-run suppress the ♻️ comment ONLY — neither skips Step + 3.6.** Scan the PR's existing comments for a prior bot `♻️ … unrelated flake … on + ` note for THIS head SHA; if one exists, set `SKIP_FLAKE_COMMENT = true`. + Resolve the attempt number from `C.effectiveAttempt` (the authoritative max(marker, + bot-commit) counter), omitting the `Attempt /10:` prefix only if it is + somehow indeterminate. Then post the flake note UNLESS suppressed: + - if `dry_run == "true"`: do NOT emit — print the intended `♻️` unrelated-flake body + to the run log and tally `dry-run: would-annotate-flake PR #`; + - else if `SKIP_FLAKE_COMMENT`: do NOT re-post — record `already-annotated-flake PR + # (head )` (re-annotating the same flake every 12h sweep is noise and + burns the per-run comment budget other PRs need); + - else `add_comment` on PR #: `♻️ Attempt /10: red is unrelated flake on + leg(s) () on ; the fix itself is not implicated. A + maintainer re-run (/azp run ) should clear it.` record `annotated-flake + PR # (head )`. + Then — in ALL of the above cases — run **Step 3.6** (target-test readiness gate) for + this PR before stopping: its `/azp`-gated target legs may conclude green on this SAME + head SHA on a LATER sweep, and Step 3.6 is the ONLY place the draft→ready flip + happens, so it MUST re-evaluate every sweep — never `stop` here before it. *(Round 1: + human re-runs; Round 2: auto re-trigger.)* - **Caused by the fix** (a failed leg still matches the original target signature, or the fix introduced a NEW failure): advance an attempt. - **Attempt count.** `attempt = C.effectiveAttempt` — the authoritative @@ -752,6 +856,188 @@ Run these gates in order — the FIRST that fires decides this cycle's outcome: from every prior commit on this branch**, emitted via the Step 5.6 ADVANCE path (push onto the same PR). +#### Step 3.6 — Target-test verification & mark-ready gate + +Reached from the Step 3 **green-surface** branch, the Step 4 **unrelated-flake** branch +(AFTER that branch has posted its comment), and the Step 3.5 **gate-2 target-focused +fast-path** (2a — while unrelated legs are still pending). Purpose: confirm the SPECIFIC +test(s) this PR was opened to fix now PASS in the PR's own CI, and — only when they do +— transition the draft `[ci-fix]` PR to **ready for review** so a maintainer sees a +validated fix instead of a draft. This is the ONLY place the loop flips draft→ready; it +is a state transition, **never** an approval or a merge (a human still reviews and +merges). Overall red on *unrelated* legs must NOT gate readiness — we validate the fix, +not the base branch's flakiness. + +**Preconditions** (ALL must hold; otherwise record `skipped: readiness N/A PR # +()` and stop this gate): +- `C.isDraft == true` — if the PR is already ready-for-review, the draft→ready + transition is already done; do NOT re-mark **and do NOT re-apply `p/0`**. `p/0` is applied + exactly ONCE, atomically with the draft→ready flip (T3 — all-or-nothing with the 🎯 audit + comment + mark-ready), so an already-ready loop PR that is MISSING `p/0` has almost + certainly had it **removed by a maintainer** de-prioritizing the PR — a pure triage action + the human-engagement guard (which inspects comments/reviews/commits, NOT label changes) + cannot see. Re-adding `p/0` every sweep would fight that maintainer indefinitely, violating + the loop's "never override a human" contract. So the loop does NOT reconcile the label: + record `already-ready PR #` and stop this gate. (Trade-off: on the rare occasion a + transient API error drops `p/0` at flip time *after* the T3 atomic pre-check passed, it is + not auto-re-added — but the PR is still ready-for-review with its 🎯 audit comment, and + re-adding `p/0` is a trivial manual action; that is strictly preferable to steam-rolling a + maintainer's deliberate de-prioritization.) +- The PR is unmistakably THIS workflow's own: `[ci-fix] ` title prefix AND the + `agentic-workflows` label (mirrors the safe-output lock; the compensating scope + control if the v0.80.9 compiler drops the declarative required-*). +- You reached this gate from Step 3 (green), Step 4 (**unrelated-flake**), or the Step 3.5 + gate-2 **target-focused fast-path** (2a) — i.e. the fix is NOT implicated in any red that + has concluded. If Step 4 classified a red as **caused by the fix** (ADVANCE), do NOT run + this gate — advance the attempt instead. + +**T1 — Identify the target test(s).** From the `[ci-scan]` issue signature the fix +addresses (and the PR's own diff), extract the fully-qualified test method name(s) the +fix targets — e.g. `SafeAreaShouldWorkOnAllShellTabs`. For a de-flake it is the +de-flaked test; for a product fix it is the originally-failing test(s). If NO specific +test can be identified (e.g. a product build-break rather than a test failure), this is a +**build-only fix** — there is no single test to validate cross-platform, so validate at +**whole-build** granularity rather than stopping (Step 3 only *comments*; Step 3.6 is the +sole draft→ready flip, so a build-only fix is undrafted HERE or not at all). We only reach +this gate from Step 3 (green) / Step 4 (unrelated-flake) / gate-2a, so the fix is not +implicated in any concluded red; additionally require, via the T2 timeline method on +`C.headSha`, that the primary build pipeline `maui-pr` (def 302) has CONCLUDED with EVERY +one of its platform build legs `succeeded`/`completed` and NONE failed/canceled or still +pending — the build is green on **every** platform, not just the originally-broken one (the +cross-platform guard, applied to the build instead of a test). A build-only fix is undrafted +ONLY on the strength of the auto-running `maui-pr` (def 302) whole-build green, which the +loop CAN observe: if the PR's `[ci-scan]` issue signature or its own diff indicates the +ORIGINATING failure was in a `/azp`-gated pipeline (`maui-pr-uitests` def 313 or +`maui-pr-devicetests` def 314) rather than `maui-pr` (def 302), do NOT undraft on +`maui-pr`-green alone — those pipelines do not auto-run on this PR (GITHUB_TOKEN cannot +trigger them), so a green `maui-pr` build is NOT evidence the gated build break is fixed; +record `skipped: build-only fix PR # targets gated pipeline () not run — +deferring to human` and stop this gate. Otherwise, set `TARGET := "the maui-pr build +(build-only fix — no single target test)"` and proceed to **T3** to mark ready. If any `maui-pr` build leg is still unconcluded, record `skipped: build-only fix PR +# not yet whole-build green (leg(s) pending)` and stop this gate WITHOUT marking +ready (a green subset is not enough — a still-pending build leg could yet fail). + +**T2 — Verify the target test's platform legs are green (anonymous, leg-level).** Per-test +outcomes come from the AzDO **test-results** API (`_apis/test/...`), which is **NOT reachable +anonymously — Hard-Rule 8**: those endpoints 302-redirect to a sign-in page on this runner, so +a `curl` returns an HTML redirect (not JSON) and every target test would look "not executed". +Validate instead at **leg granularity** using ONLY the anonymous `_apis/build/...` timeline +(Hard-Rule 8) — a `succeeded` category leg means every test in that category **that actually +ran** passed on that platform. This is a strong bar **only if the target test genuinely +executes**: a job can go green while one specific test is skipped at runtime (`[Ignore]`, +`Assert.Ignore()`, `Assert.Inconclusive()`, a `Skip=`/conditional `[Fact]`, an `#if`-out, or +an early `return` before the asserts). So before trusting a green leg as proof the target +passed, **confirm from the PR's OWN diff that the fix does NOT skip, ignore, disable, or +short-circuit the target test** (it adds no `[Ignore]`/`[Explicit]`, `Assert.Ignore`/ +`Assert.Inconclusive`, `Skip=`, category-exclusion, `#if`-out, or early `return` around the +target). If the fix could cause the target to be runtime-skipped rather than genuinely pass, +leg-level green is NOT sufficient evidence — record `skipped: target test may be +runtime-skipped by this fix (diff adds ); leg-green insufficient, deferring to human +on PR #` and stop this gate WITHOUT marking ready. Otherwise (the target genuinely runs +and asserts, as a de-flake or product fix does) a green category leg cannot hide a +target-test failure, so treat it as authoritative. + +Map the target test to the CI leg(s) that run it. A leg name encodes platform + test category +(e.g. `Android UITests SafeAreaEdges,Shadow`, `iOS UITests SafeAreaEdges,Shadow`, `macOS +UITests SafeAreaEdges,Shadow`, `Windows UITests SafeAreaEdges,Shadow`). Determine the target +test's UI-test category — its `[Category(UITestCategories.X)]` in the test/HostApp source, +visible in the PR diff or the test file — to know which leg-name substring identifies its legs; +for a device test, the per-platform device-test legs. + +Using the SAME build-discovery as Step 4 (filter AzDO builds by `branchName=refs/pull//merge` +or `sourceVersion == C.headSha`), read each build's **timeline** on `C.headSha` for the +pipeline(s) that RUN the target test — `maui-pr` (def 302) for unit/integration tests, +`maui-pr-uitests` (def 313) for Appium UI tests, `maui-pr-devicetests` (def 314) for device +tests: + +```bash +ORG=dnceng-public; PROJ=public; BUILD= +# Anonymous + Hard-Rule-8-compliant. NEVER call _apis/test/... (it 302-redirects to sign-in). +# Each type=="Job" record is one platform leg: result (succeeded/failed/canceled/null), +# state (completed/inProgress/pending), name (encodes " UITests "): +curl -s "https://dev.azure.com/$ORG/$PROJ/_apis/build/builds/$BUILD/timeline?api-version=7.1" \ + | tee /tmp/gh-aw/agent/timeline_${BUILD}.json \ + | jq -r '.records[] | select(.type=="Job") | "\(.result)\t\(.state)\t\(.name)"' +``` + +(The prefetch already exposes per-leg status in `C.failedLegs` / the checks context, and +`gh pr checks ` lists the same per-leg rows with platform+category in the name — use +whichever is handy; the build timeline is the authoritative cross-check on the exact build.) + +Treat a target test as **VALIDATED-GREEN** only if, on `C.headSha`, the leg that runs its +category is green on **every platform it runs on** — a fix that repairs one platform must not +silently regress the same test's leg on another, so a green leg on the originally-red platform +alone is NOT enough. Across the target pipeline's platform legs (for a UI test: the Android, +iOS, Windows, and macOS/MacCatalyst legs running the target's category; for a device test: each +device-test platform), require ALL of: +- the target's category leg is `result == "succeeded"` **and** `state == "completed"` on + **every** platform that runs it, AND +- that leg is `failed` / `canceled` / aborted on **NO** platform, AND +- **no platform is left unverified.** For each platform family the target pipeline covers, that + platform's category leg must have CONCLUDED (`state == "completed"`) on `C.headSha`. If a + platform simply has no leg for the target's category, the test does not run there — that is + fine, not a gap. But if a platform's category leg has **not concluded** (`state` is + `inProgress` / pending, or an `/azp`-gated `maui-pr-uitests` / `maui-pr-devicetests` leg that + has not been kicked), the test's status on that platform is UNKNOWN → the fix is NOT yet + validated across platforms: record `skipped: target test green on but + not yet verified on (leg(s) pending / need /azp run) on PR #` + and stop this gate WITHOUT marking ready. + +A target test whose category leg never concluded on ANY platform (**not executed** anywhere — +e.g. its `/azp`-gated pipeline has not been kicked) is likewise NOT validated: record `skipped: +target test not yet executed on PR # ( not run — needs /azp run)` and stop +this gate WITHOUT marking ready. Do NOT overclaim — a green *sibling* leg (a different category +on the same platform) is not the target's leg, and a green leg on one platform is not a pass on +the others. + +**T3 — Mark ready + report.** If EVERY target test is VALIDATED-GREEN. The 🎯 comment, +the mark-ready, and the `p/0` label are THREE SEPARATE safe-outputs — the comment +existing does NOT prove the mark-ready took effect, so they are tracked independently: + +- **Comment idempotency (dup-suppress only — never gates the mark-ready).** We only reach + T3 while `C.isDraft == true` (the precondition stops an already-ready PR before here). So + if a prior bot comment for THIS head that carries the leading `🎯` anchor AND the + contiguous phrase `validated green on ` already exists (BOTH T3 variants — + `🎯 Target test validated green on ` and the build-only `🎯 Build validated green on + ` — contain that exact contiguous phrase, so this recognizes both; keying on + `target test` alone would instead miss the build-only comment and re-post it every sweep. + **Require the `🎯` anchor** so the match can NEVER be loosened to a gapped + `validated…green…on` form that would false-positive on the Step 3 `✅ … validated — the + fix's CI is green on ` surface comment and wrongly suppress the audit 🎯), the comment + landed on an earlier sweep but the **mark-ready did NOT take + effect** (the PR is still a draft) — set `SUPPRESS_COMMENT = true` (do not re-post the + duplicate 🎯 comment) but STILL complete the mark-ready + label below. Never treat the + comment's existence as "already marked ready" while the PR is still a draft. Record this + case as `re-marking PR # (🎯 present; mark-ready did not take on a prior sweep)`. +- **Atomicity budget pre-check (Hard-Rule 7).** Determine the outputs this gate will emit: + `mark_pull_request_as_ready_for_review` + `add_labels` always, plus `add_comment` unless + `SUPPRESS_COMMENT`. Verify a free per-run slot remains in EACH bucket you are about to + use. If ANY required bucket is exhausted, emit NONE of them — record `skipped: per-run + cap reached; deferring mark-ready PR # to next cycle` and stop this gate. Never mark a + PR ready (or label it) unless its 🎯 audit comment is GUARANTEED to exist for this exact + head SHA — either posted in this same sweep, or (in the `SUPPRESS_COMMENT` re-marking case) + already present from a prior sweep. The outputs you DO emit either all land or all defer + together; the audit trail must never be absent, but it is NOT re-posted when it already exists. +- **Dry-run gate (Step 0):** if `dry_run == "true"`, emit NOTHING — print the intended + readiness comment and "would mark ready + add p/0" to the run log, tally `dry-run: + would-mark-ready PR #`, and stop. +- Otherwise emit for THIS PR number ``: + 1. `add_comment` (ONLY if not `SUPPRESS_COMMENT`): `🎯 Target test validated green on + — passed on ALL platforms it runs on (, + buildId ). — not + caused by this fix."> Transitioning this PR from draft to ready for review and adding + `p/0`; a maintainer still reviews and merges.` (For a **build-only fix** — the T1 + whole-build fallback, no single target test — phrase the first clause as `🎯 Build + validated green on — the maui-pr build passed on ALL platforms (buildId + ).` instead of naming a test.) + 2. `mark_pull_request_as_ready_for_review` with `reason:` a one-line justification + naming the validated test(s) and ``. + 3. `add_labels` with `labels: ["p/0"]` for PR # — put the now-review-ready fix into + the team's p/0 priority queue so it is triaged, not lost in the draft backlog. (If + the PR somehow already carries `p/0`, this is a harmless no-op.) +- Record `marked-ready PR # (target green on ALL platforms on , + labeled p/0)` and stop. + #### Step 3.5.R — Maintainer change-request response (Track C) Reached from Step 3.5 gate 0. Goal: when an **eligible human reviewer** (Hard-Rule @@ -1383,7 +1669,19 @@ Filed by [`ci-status-fix`](https://github.com/dotnet/maui/blob/main/.github/work ### Step 8 — Per-issue tally + end-of-run summary -Per issue, append one outcome line to `/tmp/gh-aw/agent/coverage.txt`: +Per issue, append **one** outcome line to `/tmp/gh-aw/agent/coverage.txt` — the +terminal outcome for the cycle. When one cycle produces a chained pair — a PR gets a +same-cycle precursor line and **then** a Step 3.6 readiness line (`marked-ready` on the +draft→ready flip, or `already-ready` on a later steady-state sweep of an already-ready +PR) — record ONLY the terminal Step 3.6 readiness line: it supersedes ANY same-cycle +non-readiness precursor for that PR, so an aggregator keying on one-line-per-issue never +double-counts. The precursor may be a Step 3 green line (`surfaced-green` on the first ✅, +or `already-surfaced` when it already exists) OR — when an unrelated-flake red and a +target-green coincide — a Step 4 `annotated-flake` line; either way the readiness line is +terminal, and the superseded precursor's signal survives in the PR's 🎯/♻️ comment so no +information is lost. This covers the flip cycle (`surfaced-green` → `marked-ready`), the +post-flip steady state (`already-surfaced` → `already-ready`), and the flake-coincident +flip (`annotated-flake` → `marked-ready`): ``` # main attempt- @@ -1394,8 +1692,14 @@ Per issue, append one outcome line to `/tmp/gh-aw/agent/coverage.txt`: `advance-PR # attempt /10` (ADVANCE mode — new commit pushed to the existing PR), `surfaced-green PR #` (fix's own CI went green; commented for review, did not advance), `annotated-flake PR #` (red was unrelated flake; -commented, attempt NOT burned), `waiting PR #` (CI not settled yet), -`dry-run: would-`, `skipped: `. (The +commented, attempt NOT burned), `marked-ready PR #` (the specific fixed test +was confirmed green in the PR's own CI; draft flipped to ready for review), +`already-surfaced PR #` (the fix's ✅ green comment already existed this sweep; +re-post suppressed — superseded by the Step 3.6 readiness line when the PR also flips +ready that cycle), `already-ready PR #` (terminal steady state — the PR was flipped +ready on a prior cycle and remains green on an unchanged head; no action taken, +supersedes `already-surfaced`), `waiting PR #` (CI not settled yet), +`dry-run: would-`, `skipped: `. (The `needs-human-PR` outcome is reserved for the deferred hand-off — Step 6 currently records a skip instead, so it is not emitted.)
` and stop this gate. (Trade-off: on the rare occasion a + transient API error drops `p/0` at flip time *after* the T3 atomic pre-check passed, it is + not auto-re-added — but the PR is still ready-for-review with its 🎯 audit comment, and + re-adding `p/0` is a trivial manual action; that is strictly preferable to steam-rolling a + maintainer's deliberate de-prioritization.) +- The PR is unmistakably THIS workflow's own: `[ci-fix-net11] ` title prefix AND the + `agentic-workflows` label (mirrors the safe-output lock; the compensating scope + control if the v0.80.9 compiler drops the declarative required-*). +- You reached this gate from Step 3 (green), Step 4 (**unrelated-flake**), or the Step 3.5 + gate-2 **target-focused fast-path** (2a) — i.e. the fix is NOT implicated in any red that + has concluded. If Step 4 classified a red as **caused by the fix** (ADVANCE), do NOT run + this gate — advance the attempt instead. + +**T1 — Identify the target test(s).** From the `[ci-scan-net11]` issue signature the fix +addresses (and the PR's own diff), extract the fully-qualified test method name(s) the +fix targets — e.g. `SafeAreaShouldWorkOnAllShellTabs`. For a de-flake it is the +de-flaked test; for a product fix it is the originally-failing test(s). If NO specific +test can be identified (e.g. a product build-break rather than a test failure), this is a +**build-only fix** — there is no single test to validate cross-platform, so validate at +**whole-build** granularity rather than stopping (Step 3 only *comments*; Step 3.6 is the +sole draft→ready flip, so a build-only fix is undrafted HERE or not at all). We only reach +this gate from Step 3 (green) / Step 4 (unrelated-flake) / gate-2a, so the fix is not +implicated in any concluded red; additionally require, via the T2 timeline method on +`C.headSha`, that the primary build pipeline `maui-pr` (def 302) has CONCLUDED with EVERY +one of its platform build legs `succeeded`/`completed` and NONE failed/canceled or still +pending — the build is green on **every** platform, not just the originally-broken one (the +cross-platform guard, applied to the build instead of a test). A build-only fix is undrafted +ONLY on the strength of the auto-running `maui-pr` (def 302) whole-build green, which the +loop CAN observe: if the PR's `[ci-scan]` issue signature or its own diff indicates the +ORIGINATING failure was in a `/azp`-gated pipeline (`maui-pr-uitests` def 313 or +`maui-pr-devicetests` def 314) rather than `maui-pr` (def 302), do NOT undraft on +`maui-pr`-green alone — those pipelines do not auto-run on this PR (GITHUB_TOKEN cannot +trigger them), so a green `maui-pr` build is NOT evidence the gated build break is fixed; +record `skipped: build-only fix PR #
targets gated pipeline () not run — +deferring to human` and stop this gate. Otherwise, set `TARGET := "the maui-pr build +(build-only fix — no single target test)"` and proceed to **T3** to mark ready. If any `maui-pr` build leg is still unconcluded, record `skipped: build-only fix PR +# not yet whole-build green (leg(s) pending)` and stop this gate WITHOUT marking +ready (a green subset is not enough — a still-pending build leg could yet fail). + +**T2 — Verify the target test's platform legs are green (anonymous, leg-level).** Per-test +outcomes come from the AzDO **test-results** API (`_apis/test/...`), which is **NOT reachable +anonymously — Hard-Rule 8**: those endpoints 302-redirect to a sign-in page on this runner, so +a `curl` returns an HTML redirect (not JSON) and every target test would look "not executed". +Validate instead at **leg granularity** using ONLY the anonymous `_apis/build/...` timeline +(Hard-Rule 8) — a `succeeded` category leg means every test in that category **that actually +ran** passed on that platform. This is a strong bar **only if the target test genuinely +executes**: a job can go green while one specific test is skipped at runtime (`[Ignore]`, +`Assert.Ignore()`, `Assert.Inconclusive()`, a `Skip=`/conditional `[Fact]`, an `#if`-out, or +an early `return` before the asserts). So before trusting a green leg as proof the target +passed, **confirm from the PR's OWN diff that the fix does NOT skip, ignore, disable, or +short-circuit the target test** (it adds no `[Ignore]`/`[Explicit]`, `Assert.Ignore`/ +`Assert.Inconclusive`, `Skip=`, category-exclusion, `#if`-out, or early `return` around the +target). If the fix could cause the target to be runtime-skipped rather than genuinely pass, +leg-level green is NOT sufficient evidence — record `skipped: target test may be +runtime-skipped by this fix (diff adds ); leg-green insufficient, deferring to human +on PR #` and stop this gate WITHOUT marking ready. Otherwise (the target genuinely runs +and asserts, as a de-flake or product fix does) a green category leg cannot hide a +target-test failure, so treat it as authoritative. + +Map the target test to the CI leg(s) that run it. A leg name encodes platform + test category +(e.g. `Android UITests SafeAreaEdges,Shadow`, `iOS UITests SafeAreaEdges,Shadow`, `macOS +UITests SafeAreaEdges,Shadow`, `Windows UITests SafeAreaEdges,Shadow`). Determine the target +test's UI-test category — its `[Category(UITestCategories.X)]` in the test/HostApp source, +visible in the PR diff or the test file — to know which leg-name substring identifies its legs; +for a device test, the per-platform device-test legs. + +Using the SAME build-discovery as Step 4 (filter AzDO builds by `branchName=refs/pull//merge` +or `sourceVersion == C.headSha`), read each build's **timeline** on `C.headSha` for the +pipeline(s) that RUN the target test — `maui-pr` (def 302) for unit/integration tests, +`maui-pr-uitests` (def 313) for Appium UI tests, `maui-pr-devicetests` (def 314) for device +tests: + +```bash +ORG=dnceng-public; PROJ=public; BUILD= +# Anonymous + Hard-Rule-8-compliant. NEVER call _apis/test/... (it 302-redirects to sign-in). +# Each type=="Job" record is one platform leg: result (succeeded/failed/canceled/null), +# state (completed/inProgress/pending), name (encodes " UITests "): +curl -s "https://dev.azure.com/$ORG/$PROJ/_apis/build/builds/$BUILD/timeline?api-version=7.1" \ + | tee /tmp/gh-aw/agent/timeline_${BUILD}.json \ + | jq -r '.records[] | select(.type=="Job") | "\(.result)\t\(.state)\t\(.name)"' +``` + +(The prefetch already exposes per-leg status in `C.failedLegs` / the checks context, and +`gh pr checks ` lists the same per-leg rows with platform+category in the name — use +whichever is handy; the build timeline is the authoritative cross-check on the exact build.) + +Treat a target test as **VALIDATED-GREEN** only if, on `C.headSha`, the leg that runs its +category is green on **every platform it runs on** — a fix that repairs one platform must not +silently regress the same test's leg on another, so a green leg on the originally-red platform +alone is NOT enough. Across the target pipeline's platform legs (for a UI test: the Android, +iOS, Windows, and macOS/MacCatalyst legs running the target's category; for a device test: each +device-test platform), require ALL of: +- the target's category leg is `result == "succeeded"` **and** `state == "completed"` on + **every** platform that runs it, AND +- that leg is `failed` / `canceled` / aborted on **NO** platform, AND +- **no platform is left unverified.** For each platform family the target pipeline covers, that + platform's category leg must have CONCLUDED (`state == "completed"`) on `C.headSha`. If a + platform simply has no leg for the target's category, the test does not run there — that is + fine, not a gap. But if a platform's category leg has **not concluded** (`state` is + `inProgress` / pending, or an `/azp`-gated `maui-pr-uitests` / `maui-pr-devicetests` leg that + has not been kicked), the test's status on that platform is UNKNOWN → the fix is NOT yet + validated across platforms: record `skipped: target test green on but + not yet verified on (leg(s) pending / need /azp run) on PR #` + and stop this gate WITHOUT marking ready. + +A target test whose category leg never concluded on ANY platform (**not executed** anywhere — +e.g. its `/azp`-gated pipeline has not been kicked) is likewise NOT validated: record `skipped: +target test not yet executed on PR # ( not run — needs /azp run)` and stop +this gate WITHOUT marking ready. Do NOT overclaim — a green *sibling* leg (a different category +on the same platform) is not the target's leg, and a green leg on one platform is not a pass on +the others. + +**T3 — Mark ready + report.** If EVERY target test is VALIDATED-GREEN. The 🎯 comment, +the mark-ready, and the `p/0` label are THREE SEPARATE safe-outputs — the comment +existing does NOT prove the mark-ready took effect, so they are tracked independently: + +- **Comment idempotency (dup-suppress only — never gates the mark-ready).** We only reach + T3 while `C.isDraft == true` (the precondition stops an already-ready PR before here). So + if a prior bot comment for THIS head that carries the leading `🎯` anchor AND the + contiguous phrase `validated green on ` already exists (BOTH T3 variants — + `🎯 Target test validated green on ` and the build-only `🎯 Build validated green on + ` — contain that exact contiguous phrase, so this recognizes both; keying on + `target test` alone would instead miss the build-only comment and re-post it every sweep. + **Require the `🎯` anchor** so the match can NEVER be loosened to a gapped + `validated…green…on` form that would false-positive on the Step 3 `✅ … validated — the + fix's CI is green on ` surface comment and wrongly suppress the audit 🎯), the comment + landed on an earlier sweep but the **mark-ready did NOT take + effect** (the PR is still a draft) — set `SUPPRESS_COMMENT = true` (do not re-post the + duplicate 🎯 comment) but STILL complete the mark-ready + label below. Never treat the + comment's existence as "already marked ready" while the PR is still a draft. Record this + case as `re-marking PR # (🎯 present; mark-ready did not take on a prior sweep)`. +- **Atomicity budget pre-check (Hard-Rule 7).** Determine the outputs this gate will emit: + `mark_pull_request_as_ready_for_review` + `add_labels` always, plus `add_comment` unless + `SUPPRESS_COMMENT`. Verify a free per-run slot remains in EACH bucket you are about to + use. If ANY required bucket is exhausted, emit NONE of them — record `skipped: per-run + cap reached; deferring mark-ready PR # to next cycle` and stop this gate. Never mark a + PR ready (or label it) unless its 🎯 audit comment is GUARANTEED to exist for this exact + head SHA — either posted in this same sweep, or (in the `SUPPRESS_COMMENT` re-marking case) + already present from a prior sweep. The outputs you DO emit either all land or all defer + together; the audit trail must never be absent, but it is NOT re-posted when it already exists. +- **Dry-run gate (Step 0):** if `dry_run == "true"`, emit NOTHING — print the intended + readiness comment and "would mark ready + add p/0" to the run log, tally `dry-run: + would-mark-ready PR #`, and stop. +- Otherwise emit for THIS PR number ``: + 1. `add_comment` (ONLY if not `SUPPRESS_COMMENT`): `🎯 Target test validated green on + — passed on ALL platforms it runs on (, + buildId ). — not + caused by this fix."> Transitioning this PR from draft to ready for review and adding + `p/0`; a maintainer still reviews and merges.` (For a **build-only fix** — the T1 + whole-build fallback, no single target test — phrase the first clause as `🎯 Build + validated green on — the maui-pr build passed on ALL platforms (buildId + ).` instead of naming a test.) + 2. `mark_pull_request_as_ready_for_review` with `reason:` a one-line justification + naming the validated test(s) and ``. + 3. `add_labels` with `labels: ["p/0"]` for PR # — put the now-review-ready fix into + the team's p/0 priority queue so it is triaged, not lost in the draft backlog. (If + the PR somehow already carries `p/0`, this is a harmless no-op.) +- Record `marked-ready PR # (target green on ALL platforms on , + labeled p/0)` and stop. + #### Step 3.5.R — Maintainer change-request response (Track C) Reached from Step 3.5 gate 0. Goal: when an **eligible human reviewer** (Hard-Rule @@ -1395,7 +1681,19 @@ Filed by [`ci-status-fix-net11`](https://github.com/dotnet/maui/blob/main/.githu ### Step 8 — Per-issue tally + end-of-run summary -Per issue, append one outcome line to `/tmp/gh-aw/agent/coverage.txt`: +Per issue, append **one** outcome line to `/tmp/gh-aw/agent/coverage.txt` — the +terminal outcome for the cycle. When one cycle produces a chained pair — a PR gets a +same-cycle precursor line and **then** a Step 3.6 readiness line (`marked-ready` on the +draft→ready flip, or `already-ready` on a later steady-state sweep of an already-ready +PR) — record ONLY the terminal Step 3.6 readiness line: it supersedes ANY same-cycle +non-readiness precursor for that PR, so an aggregator keying on one-line-per-issue never +double-counts. The precursor may be a Step 3 green line (`surfaced-green` on the first ✅, +or `already-surfaced` when it already exists) OR — when an unrelated-flake red and a +target-green coincide — a Step 4 `annotated-flake` line; either way the readiness line is +terminal, and the superseded precursor's signal survives in the PR's 🎯/♻️ comment so no +information is lost. This covers the flip cycle (`surfaced-green` → `marked-ready`), the +post-flip steady state (`already-surfaced` → `already-ready`), and the flake-coincident +flip (`annotated-flake` → `marked-ready`): ``` # net11.0 attempt- @@ -1406,8 +1704,14 @@ Per issue, append one outcome line to `/tmp/gh-aw/agent/coverage.txt`: `advance-PR # attempt /10` (ADVANCE mode — new commit pushed to the existing PR), `surfaced-green PR #` (fix's own CI went green; commented for review, did not advance), `annotated-flake PR #` (red was unrelated flake; -commented, attempt NOT burned), `waiting PR #` (CI not settled yet), -`dry-run: would-`, `skipped: `. (The +commented, attempt NOT burned), `marked-ready PR #` (the specific fixed test +was confirmed green in the PR's own CI; draft flipped to ready for review), +`already-surfaced PR #` (the fix's ✅ green comment already existed this sweep; +re-post suppressed — superseded by the Step 3.6 readiness line when the PR also flips +ready that cycle), `already-ready PR #` (terminal steady state — the PR was flipped +ready on a prior cycle and remains green on an unchanged head; no action taken, +supersedes `already-surfaced`), `waiting PR #` (CI not settled yet), +`dry-run: would-`, `skipped: `. (The `needs-human-PR` outcome is reserved for the deferred hand-off — Step 6 currently records a skip instead, so it is not emitted.) diff --git a/.github/workflows/ci-status-fix.lock.yml b/.github/workflows/ci-status-fix.lock.yml index 93dc5969cb0d..65511f2b3259 100644 --- a/.github/workflows/ci-status-fix.lock.yml +++ b/.github/workflows/ci-status-fix.lock.yml @@ -1,4 +1,4 @@ -# gh-aw-metadata: {"schema_version":"v4","frontmatter_hash":"f014161967589ebcaf1ac5ec6f3c88ee5a4388d28b55a07dc36c45b7b2aebab6","body_hash":"86c6b2d4c3df13da14c6a3c8b211381fcc9f97bb1951ff7b80588a2c5338e00c","compiler_version":"v0.80.9","strict":true,"agent_id":"copilot","agent_model":"claude-opus-4.8","engine_versions":{"copilot":"1.0.63"}} +# gh-aw-metadata: {"schema_version":"v4","frontmatter_hash":"b0ebf8a2f179900cfe3638e994b27a43cec52bdbdd2331dc1bd4d65762fa2d6b","body_hash":"1825e2b1f7c7bd88241bf37580655489360f9bebc806edd00837a52ce4ad83a0","compiler_version":"v0.80.9","strict":true,"agent_id":"copilot","agent_model":"claude-opus-4.8","engine_versions":{"copilot":"1.0.63"}} # gh-aw-manifest: {"version":1,"secrets":["COPILOT_PAT_0","COPILOT_PAT_1","COPILOT_PAT_2","COPILOT_PAT_3","COPILOT_PAT_4","COPILOT_PAT_5","COPILOT_PAT_6","COPILOT_PAT_7","COPILOT_PAT_8","COPILOT_PAT_9","GH_AW_CI_TRIGGER_TOKEN","GH_AW_GITHUB_MCP_SERVER_TOKEN","GH_AW_GITHUB_TOKEN","GITHUB_TOKEN"],"actions":[{"repo":"actions/checkout","sha":"34e114876b0b11c390a56381ad16ebd13914f8d5","version":"v4"},{"repo":"actions/checkout","sha":"9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0","version":"v7.0.0"},{"repo":"actions/checkout","sha":"de0fac2e4500dabe0009e67214ff5f5447ce83dd","version":"v6.0.2"},{"repo":"actions/download-artifact","sha":"3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c","version":"v8.0.1"},{"repo":"actions/github-script","sha":"3a2844b7e9c422d3c10d287c895573f7108da1b3","version":"v9.0.0"},{"repo":"actions/setup-node","sha":"48b55a011bda9f5d6aeb4c2d9c7362e8dae4041e","version":"v6.4.0"},{"repo":"actions/upload-artifact","sha":"043fb46d1a93c77aae656e7c1c64a875d1fc6a0a","version":"v7.0.1"},{"repo":"github/gh-aw-actions/setup","sha":"8c7d04ebf1ece56cd381446125da3e0f6896294a","version":"v0.80.9"}],"containers":[{"image":"ghcr.io/github/gh-aw-firewall/agent:0.27.7","digest":"sha256:aae231e4635c8999d039c132f1602d3df850fe9b84a00aa2b5ac981179b5661c","pinned_image":"ghcr.io/github/gh-aw-firewall/agent:0.27.7@sha256:aae231e4635c8999d039c132f1602d3df850fe9b84a00aa2b5ac981179b5661c"},{"image":"ghcr.io/github/gh-aw-firewall/api-proxy:0.27.7","digest":"sha256:009caf2e3d88fa77b64e9a03a95a228fc58db0f1701c6d324b29ba5a3c7c79b6","pinned_image":"ghcr.io/github/gh-aw-firewall/api-proxy:0.27.7@sha256:009caf2e3d88fa77b64e9a03a95a228fc58db0f1701c6d324b29ba5a3c7c79b6"},{"image":"ghcr.io/github/gh-aw-firewall/squid:0.27.7","digest":"sha256:deb1d4e19de62d51cee0508057a596a19315c3423ada4d675cad136dc8037c96","pinned_image":"ghcr.io/github/gh-aw-firewall/squid:0.27.7@sha256:deb1d4e19de62d51cee0508057a596a19315c3423ada4d675cad136dc8037c96"},{"image":"ghcr.io/github/gh-aw-mcpg:v0.3.27","digest":"sha256:fe984bddde4ec05d756d9043edb0a32912e6b7b72f6a121b1082f29221421cc7","pinned_image":"ghcr.io/github/gh-aw-mcpg:v0.3.27@sha256:fe984bddde4ec05d756d9043edb0a32912e6b7b72f6a121b1082f29221421cc7"},{"image":"ghcr.io/github/gh-aw-node","digest":"sha256:529d02eb970b1161aa25c593a9c3df57fdfad5a8add328cb3b6eccef66f3183b","pinned_image":"ghcr.io/github/gh-aw-node@sha256:529d02eb970b1161aa25c593a9c3df57fdfad5a8add328cb3b6eccef66f3183b"},{"image":"ghcr.io/github/github-mcp-server:v1.4.0","digest":"sha256:2afb26356481d1a350e14544a6e160f7f7ec1561a1ea309b823665abf0309036","pinned_image":"ghcr.io/github/github-mcp-server:v1.4.0@sha256:2afb26356481d1a350e14544a6e160f7f7ec1561a1ea309b823665abf0309036"}]} # This file was automatically generated by gh-aw (v0.80.9). DO NOT EDIT. To debug this workflow, load the skill at https://github.com/github/gh-aw/blob/main/debug.md # @@ -37,7 +37,9 @@ # humans (the open tracking issue is the hand-off surface; a dedicated # [ci-fix][needs-human] PR is planned but currently deferred — see Step 6). CI on # the PR is kicked by a human `/azp run` for now; the loop watches, classifies, -# and re-fixes autonomously between kicks. +# and re-fixes autonomously between kicks. When the SPECIFIC test a PR fixed is +# confirmed green in that PR's own CI, the loop marks the draft PR ready for review +# (a state transition only — it never approves or merges). # Never mutes tests, but # de-flakes genuinely flaky ones (deterministic synchronization, no retries / # timeout bumps). Always skips visual-regression / screenshot issues. @@ -273,24 +275,24 @@ jobs: run: | bash "${RUNNER_TEMP}/gh-aw/actions/create_prompt_first.sh" { - cat << 'GH_AW_PROMPT_25cf75111b670167_EOF' + cat << 'GH_AW_PROMPT_2ac1d0b4083428f8_EOF' - GH_AW_PROMPT_25cf75111b670167_EOF + GH_AW_PROMPT_2ac1d0b4083428f8_EOF cat "${RUNNER_TEMP}/gh-aw/prompts/xpia.md" cat "${RUNNER_TEMP}/gh-aw/prompts/temp_folder_prompt.md" cat "${RUNNER_TEMP}/gh-aw/prompts/markdown.md" cat "${RUNNER_TEMP}/gh-aw/prompts/safe_outputs_prompt.md" - cat << 'GH_AW_PROMPT_25cf75111b670167_EOF' + cat << 'GH_AW_PROMPT_2ac1d0b4083428f8_EOF' - Tools: add_comment(max:3), create_pull_request(max:3), update_pull_request(max:3), push_to_pull_request_branch(max:3), missing_tool, missing_data, noop - GH_AW_PROMPT_25cf75111b670167_EOF + Tools: add_comment(max:6), create_pull_request(max:3), update_pull_request(max:3), mark_pull_request_as_ready_for_review(max:3), add_labels(max:3), push_to_pull_request_branch(max:3), missing_tool, missing_data, noop + GH_AW_PROMPT_2ac1d0b4083428f8_EOF cat "${RUNNER_TEMP}/gh-aw/prompts/safe_outputs_create_pull_request.md" cat "${RUNNER_TEMP}/gh-aw/prompts/safe_outputs_push_to_pr_branch.md" - cat << 'GH_AW_PROMPT_25cf75111b670167_EOF' + cat << 'GH_AW_PROMPT_2ac1d0b4083428f8_EOF' - GH_AW_PROMPT_25cf75111b670167_EOF + GH_AW_PROMPT_2ac1d0b4083428f8_EOF cat "${RUNNER_TEMP}/gh-aw/prompts/mcp_cli_tools_prompt.md" - cat << 'GH_AW_PROMPT_25cf75111b670167_EOF' + cat << 'GH_AW_PROMPT_2ac1d0b4083428f8_EOF' The following GitHub context information is available for this workflow: {{#if github.actor}} @@ -332,12 +334,12 @@ jobs: stop immediately and report the limitation rather than spending turns trying to work around it. - GH_AW_PROMPT_25cf75111b670167_EOF + GH_AW_PROMPT_2ac1d0b4083428f8_EOF cat "${RUNNER_TEMP}/gh-aw/prompts/github_mcp_tools_with_safeoutputs_prompt.md" - cat << 'GH_AW_PROMPT_25cf75111b670167_EOF' + cat << 'GH_AW_PROMPT_2ac1d0b4083428f8_EOF' {{#runtime-import .github/workflows/ci-status-fix.md}} - GH_AW_PROMPT_25cf75111b670167_EOF + GH_AW_PROMPT_2ac1d0b4083428f8_EOF } > "$GH_AW_PROMPT" - name: Interpolate variables and render templates uses: actions/github-script@3a2844b7e9c422d3c10d287c895573f7108da1b3 # v9.0.0 @@ -554,16 +556,18 @@ jobs: mkdir -p "${RUNNER_TEMP}/gh-aw/safeoutputs" mkdir -p /tmp/gh-aw/safeoutputs mkdir -p /tmp/gh-aw/mcp-logs/safeoutputs - cat > "${RUNNER_TEMP}/gh-aw/safeoutputs/config.json" << 'GH_AW_SAFE_OUTPUTS_CONFIG_fa2a69fad5fd0485_EOF' - {"add_comment":{"discussions":false,"max":3,"required_labels":["agentic-workflows"],"required_title_prefix":"[ci-fix] ","target":"*"},"create_pull_request":{"allowed_base_branches":["main"],"allowed_branches":["ci-fix/**"],"allowed_files":["src/Core/**","src/Controls/**","src/Essentials/**","src/BlazorWebView/**","src/TestUtils/**","src/Templates/**","**/PublicAPI.Unshipped.txt"],"base_branch":"main","draft":true,"labels":["agentic-workflows"],"max":3,"max_patch_files":100,"max_patch_size":4096,"protect_top_level_dot_folders":true,"protected_files":["package.json","bun.lockb","bunfig.toml","deno.json","deno.jsonc","deno.lock","global.json","NuGet.Config","Directory.Packages.props","mix.exs","mix.lock","go.mod","go.sum","stack.yaml","stack.yaml.lock","pom.xml","build.gradle","build.gradle.kts","settings.gradle","settings.gradle.kts","gradle.properties","package-lock.json","yarn.lock","pnpm-lock.yaml","npm-shrinkwrap.json","requirements.txt","Pipfile","Pipfile.lock","pyproject.toml","setup.py","setup.cfg","Gemfile","Gemfile.lock","uv.lock","CODEOWNERS","DESIGN.md","README.md","CONTRIBUTING.md","CHANGELOG.md","SECURITY.md","CODE_OF_CONDUCT.md","AGENTS.md","CLAUDE.md","GEMINI.md"],"protected_files_policy":"request_review","title_prefix":"[ci-fix] "},"create_report_incomplete_issue":{},"missing_data":{},"missing_tool":{},"noop":{"max":1,"report-as-issue":"true"},"push_to_pull_request_branch":{"allowed_files":["src/Core/**","src/Controls/**","src/Essentials/**","src/BlazorWebView/**","src/TestUtils/**","src/Templates/**","**/PublicAPI.Unshipped.txt"],"if_no_changes":"warn","max":3,"max_patch_size":4096,"protect_top_level_dot_folders":true,"protected_files":["package.json","bun.lockb","bunfig.toml","deno.json","deno.jsonc","deno.lock","global.json","NuGet.Config","Directory.Packages.props","mix.exs","mix.lock","go.mod","go.sum","stack.yaml","stack.yaml.lock","pom.xml","build.gradle","build.gradle.kts","settings.gradle","settings.gradle.kts","gradle.properties","package-lock.json","yarn.lock","pnpm-lock.yaml","npm-shrinkwrap.json","requirements.txt","Pipfile","Pipfile.lock","pyproject.toml","setup.py","setup.cfg","Gemfile","Gemfile.lock","uv.lock","CODEOWNERS","DESIGN.md","README.md","CONTRIBUTING.md","CHANGELOG.md","SECURITY.md","CODE_OF_CONDUCT.md","AGENTS.md","CLAUDE.md","GEMINI.md"],"required_labels":["agentic-workflows"],"target":"*","title_prefix":"[ci-fix] "},"report_incomplete":{},"update_pull_request":{"allow_body":true,"allow_title":false,"max":3,"target":"*","update_branch":false}} - GH_AW_SAFE_OUTPUTS_CONFIG_fa2a69fad5fd0485_EOF + cat > "${RUNNER_TEMP}/gh-aw/safeoutputs/config.json" << 'GH_AW_SAFE_OUTPUTS_CONFIG_0644bf1434ca0071_EOF' + {"add_comment":{"discussions":false,"max":6,"required_labels":["agentic-workflows"],"required_title_prefix":"[ci-fix] ","target":"*"},"add_labels":{"allowed":["p/0"],"max":3,"required_labels":["agentic-workflows"],"required_title_prefix":"[ci-fix] ","target":"*"},"create_pull_request":{"allowed_base_branches":["main"],"allowed_branches":["ci-fix/**"],"allowed_files":["src/Core/**","src/Controls/**","src/Essentials/**","src/BlazorWebView/**","src/TestUtils/**","src/Templates/**","**/PublicAPI.Unshipped.txt"],"base_branch":"main","draft":true,"labels":["agentic-workflows"],"max":3,"max_patch_files":100,"max_patch_size":4096,"protect_top_level_dot_folders":true,"protected_files":["package.json","bun.lockb","bunfig.toml","deno.json","deno.jsonc","deno.lock","global.json","NuGet.Config","Directory.Packages.props","mix.exs","mix.lock","go.mod","go.sum","stack.yaml","stack.yaml.lock","pom.xml","build.gradle","build.gradle.kts","settings.gradle","settings.gradle.kts","gradle.properties","package-lock.json","yarn.lock","pnpm-lock.yaml","npm-shrinkwrap.json","requirements.txt","Pipfile","Pipfile.lock","pyproject.toml","setup.py","setup.cfg","Gemfile","Gemfile.lock","uv.lock","CODEOWNERS","DESIGN.md","README.md","CONTRIBUTING.md","CHANGELOG.md","SECURITY.md","CODE_OF_CONDUCT.md","AGENTS.md","CLAUDE.md","GEMINI.md"],"protected_files_policy":"request_review","title_prefix":"[ci-fix] "},"create_report_incomplete_issue":{},"mark_pull_request_as_ready_for_review":{"max":3,"required_labels":["agentic-workflows"],"required_title_prefix":"[ci-fix] ","target":"*"},"missing_data":{},"missing_tool":{},"noop":{"max":1,"report-as-issue":"true"},"push_to_pull_request_branch":{"allowed_files":["src/Core/**","src/Controls/**","src/Essentials/**","src/BlazorWebView/**","src/TestUtils/**","src/Templates/**","**/PublicAPI.Unshipped.txt"],"if_no_changes":"warn","max":3,"max_patch_size":4096,"protect_top_level_dot_folders":true,"protected_files":["package.json","bun.lockb","bunfig.toml","deno.json","deno.jsonc","deno.lock","global.json","NuGet.Config","Directory.Packages.props","mix.exs","mix.lock","go.mod","go.sum","stack.yaml","stack.yaml.lock","pom.xml","build.gradle","build.gradle.kts","settings.gradle","settings.gradle.kts","gradle.properties","package-lock.json","yarn.lock","pnpm-lock.yaml","npm-shrinkwrap.json","requirements.txt","Pipfile","Pipfile.lock","pyproject.toml","setup.py","setup.cfg","Gemfile","Gemfile.lock","uv.lock","CODEOWNERS","DESIGN.md","README.md","CONTRIBUTING.md","CHANGELOG.md","SECURITY.md","CODE_OF_CONDUCT.md","AGENTS.md","CLAUDE.md","GEMINI.md"],"required_labels":["agentic-workflows"],"target":"*","title_prefix":"[ci-fix] "},"report_incomplete":{},"update_pull_request":{"allow_body":true,"allow_title":false,"max":3,"target":"*","update_branch":false}} + GH_AW_SAFE_OUTPUTS_CONFIG_0644bf1434ca0071_EOF - name: Generate Safe Outputs Tools env: GH_AW_TOOLS_META_JSON: | { "description_suffixes": { - "add_comment": " CONSTRAINTS: Maximum 3 comment(s) can be added. Target: *. Supports reply_to_id for discussion threading.", + "add_comment": " CONSTRAINTS: Maximum 6 comment(s) can be added. Target: *. Supports reply_to_id for discussion threading.", + "add_labels": " CONSTRAINTS: Maximum 3 label(s) can be added. Only these labels are allowed: [\"p/0\"]. Target: *.", "create_pull_request": " CONSTRAINTS: Maximum 3 pull request(s) can be created. Title will be prefixed with \"[ci-fix] \". Labels [\"agentic-workflows\"] will be automatically added. Only these labels are allowed: [\"agentic-workflows\"]. PRs will be created as drafts.", + "mark_pull_request_as_ready_for_review": " CONSTRAINTS: Maximum 3 pull request(s) can be marked as ready for review.", "push_to_pull_request_branch": " CONSTRAINTS: Maximum 3 push(es) can be made. The target pull request title must start with \"[ci-fix] \".", "update_pull_request": " CONSTRAINTS: Maximum 3 pull request(s) can be updated. Target: *." }, @@ -594,6 +598,25 @@ jobs: } } }, + "add_labels": { + "defaultMax": 5, + "fields": { + "item_number": { + "issueNumberOrTemporaryId": true + }, + "labels": { + "required": true, + "type": "array", + "itemType": "string", + "itemSanitize": true, + "itemMaxLength": 128 + }, + "repo": { + "type": "string", + "maxLength": 256 + } + } + }, "create_pull_request": { "defaultMax": 1, "fields": { @@ -635,6 +658,24 @@ jobs: } } }, + "mark_pull_request_as_ready_for_review": { + "defaultMax": 1, + "fields": { + "pull_request_number": { + "issueOrPRNumber": true + }, + "reason": { + "required": true, + "type": "string", + "sanitize": true, + "maxLength": 65000 + }, + "repo": { + "type": "string", + "maxLength": 256 + } + } + }, "missing_data": { "defaultMax": 20, "fields": { @@ -1494,7 +1535,7 @@ jobs: uses: actions/github-script@3a2844b7e9c422d3c10d287c895573f7108da1b3 # v9.0.0 env: WORKFLOW_NAME: "CI Failure Fixer (main)" - WORKFLOW_DESCRIPTION: "Periodic pass over open ci-scan tracking issues filed by the main-branch CI\nfailure scanner (.github/workflows/ci-status-main.md). This workflow targets\nthe `main` branch EXCLUSIVELY: it processes only issues labelled ci-scan and\nopens every PR against main. (The net11.0 branch is handled by the parallel\n.github/workflows/ci-status-fix-net11.md — the two are split because gh-aw can\nonly transport a fix relative to ONE static base branch per workflow, and the\nmain↔net11.0 divergence exceeds gh-aw's 10 MB transport-patch cap.) The fixer\nopens ONE draft [ci-fix] PR per actionable issue against main, then WATCHES\nthat PR's own CI on later runs: when the fix's CI comes back red and the red is\ncaused by the fix itself, it pushes a fresh follow-up fix onto the SAME PR\nbranch (never a second PR) — up to 10 attempts — then stops and defers to\nhumans (the open tracking issue is the hand-off surface; a dedicated\n[ci-fix][needs-human] PR is planned but currently deferred — see Step 6). CI on\nthe PR is kicked by a human `/azp run` for now; the loop watches, classifies,\nand re-fixes autonomously between kicks.\nNever mutes tests, but\nde-flakes genuinely flaky ones (deterministic synchronization, no retries /\ntimeout bumps). Always skips visual-regression / screenshot issues." + WORKFLOW_DESCRIPTION: "Periodic pass over open ci-scan tracking issues filed by the main-branch CI\nfailure scanner (.github/workflows/ci-status-main.md). This workflow targets\nthe `main` branch EXCLUSIVELY: it processes only issues labelled ci-scan and\nopens every PR against main. (The net11.0 branch is handled by the parallel\n.github/workflows/ci-status-fix-net11.md — the two are split because gh-aw can\nonly transport a fix relative to ONE static base branch per workflow, and the\nmain↔net11.0 divergence exceeds gh-aw's 10 MB transport-patch cap.) The fixer\nopens ONE draft [ci-fix] PR per actionable issue against main, then WATCHES\nthat PR's own CI on later runs: when the fix's CI comes back red and the red is\ncaused by the fix itself, it pushes a fresh follow-up fix onto the SAME PR\nbranch (never a second PR) — up to 10 attempts — then stops and defers to\nhumans (the open tracking issue is the hand-off surface; a dedicated\n[ci-fix][needs-human] PR is planned but currently deferred — see Step 6). CI on\nthe PR is kicked by a human `/azp run` for now; the loop watches, classifies,\nand re-fixes autonomously between kicks. When the SPECIFIC test a PR fixed is\nconfirmed green in that PR's own CI, the loop marks the draft PR ready for review\n(a state transition only — it never approves or merges).\nNever mutes tests, but\nde-flakes genuinely flaky ones (deterministic synchronization, no retries /\ntimeout bumps). Always skips visual-regression / screenshot issues." HAS_PATCH: ${{ needs.agent.outputs.has_patch }} with: script: | @@ -1910,7 +1951,7 @@ jobs: GH_AW_ALLOWED_DOMAINS: "*.blob.core.windows.net,*.githubusercontent.com,api.business.githubcopilot.com,api.enterprise.githubcopilot.com,api.github.com,api.githubcopilot.com,api.individual.githubcopilot.com,api.snapcraft.io,archive.ubuntu.com,azure.archive.ubuntu.com,codeload.github.com,crl.geotrust.com,crl.globalsign.com,crl.identrust.com,crl.sectigo.com,crl.thawte.com,crl.usertrust.com,crl.verisign.com,crl3.digicert.com,crl4.digicert.com,crls.ssl.com,dev.azure.com,docs.github.com,github-cloud.githubusercontent.com,github-cloud.s3.amazonaws.com,github.blog,github.com,github.githubassets.com,helix.dot.net,host.docker.internal,json-schema.org,json.schemastore.org,keyserver.ubuntu.com,lfs.github.com,objects.githubusercontent.com,ocsp.digicert.com,ocsp.geotrust.com,ocsp.globalsign.com,ocsp.identrust.com,ocsp.sectigo.com,ocsp.ssl.com,ocsp.thawte.com,ocsp.usertrust.com,ocsp.verisign.com,packagecloud.io,packages.cloud.google.com,packages.microsoft.com,patch-diff.githubusercontent.com,ppa.launchpad.net,raw.githubusercontent.com,registry.npmjs.org,s.symcb.com,s.symcd.com,security.ubuntu.com,telemetry.enterprise.githubcopilot.com,ts-crl.ws.symantec.com,ts-ocsp.ws.symantec.com,www.googleapis.com" GITHUB_SERVER_URL: ${{ github.server_url }} GITHUB_API_URL: ${{ github.api_url }} - GH_AW_SAFE_OUTPUTS_HANDLER_CONFIG: "{\"add_comment\":{\"discussions\":false,\"max\":3,\"required_labels\":[\"agentic-workflows\"],\"required_title_prefix\":\"[ci-fix] \",\"target\":\"*\"},\"create_pull_request\":{\"allowed_base_branches\":[\"main\"],\"allowed_branches\":[\"ci-fix/**\"],\"allowed_files\":[\"src/Core/**\",\"src/Controls/**\",\"src/Essentials/**\",\"src/BlazorWebView/**\",\"src/TestUtils/**\",\"src/Templates/**\",\"**/PublicAPI.Unshipped.txt\"],\"base_branch\":\"main\",\"draft\":true,\"labels\":[\"agentic-workflows\"],\"max\":3,\"max_patch_files\":100,\"max_patch_size\":4096,\"protect_top_level_dot_folders\":true,\"protected_files\":[\"package.json\",\"bun.lockb\",\"bunfig.toml\",\"deno.json\",\"deno.jsonc\",\"deno.lock\",\"global.json\",\"NuGet.Config\",\"Directory.Packages.props\",\"mix.exs\",\"mix.lock\",\"go.mod\",\"go.sum\",\"stack.yaml\",\"stack.yaml.lock\",\"pom.xml\",\"build.gradle\",\"build.gradle.kts\",\"settings.gradle\",\"settings.gradle.kts\",\"gradle.properties\",\"package-lock.json\",\"yarn.lock\",\"pnpm-lock.yaml\",\"npm-shrinkwrap.json\",\"requirements.txt\",\"Pipfile\",\"Pipfile.lock\",\"pyproject.toml\",\"setup.py\",\"setup.cfg\",\"Gemfile\",\"Gemfile.lock\",\"uv.lock\",\"CODEOWNERS\",\"DESIGN.md\",\"README.md\",\"CONTRIBUTING.md\",\"CHANGELOG.md\",\"SECURITY.md\",\"CODE_OF_CONDUCT.md\",\"AGENTS.md\",\"CLAUDE.md\",\"GEMINI.md\"],\"protected_files_policy\":\"request_review\",\"title_prefix\":\"[ci-fix] \"},\"create_report_incomplete_issue\":{},\"missing_data\":{},\"missing_tool\":{},\"noop\":{\"max\":1,\"report-as-issue\":\"true\"},\"push_to_pull_request_branch\":{\"allowed_files\":[\"src/Core/**\",\"src/Controls/**\",\"src/Essentials/**\",\"src/BlazorWebView/**\",\"src/TestUtils/**\",\"src/Templates/**\",\"**/PublicAPI.Unshipped.txt\"],\"if_no_changes\":\"warn\",\"max\":3,\"max_patch_size\":4096,\"protect_top_level_dot_folders\":true,\"protected_files\":[\"package.json\",\"bun.lockb\",\"bunfig.toml\",\"deno.json\",\"deno.jsonc\",\"deno.lock\",\"global.json\",\"NuGet.Config\",\"Directory.Packages.props\",\"mix.exs\",\"mix.lock\",\"go.mod\",\"go.sum\",\"stack.yaml\",\"stack.yaml.lock\",\"pom.xml\",\"build.gradle\",\"build.gradle.kts\",\"settings.gradle\",\"settings.gradle.kts\",\"gradle.properties\",\"package-lock.json\",\"yarn.lock\",\"pnpm-lock.yaml\",\"npm-shrinkwrap.json\",\"requirements.txt\",\"Pipfile\",\"Pipfile.lock\",\"pyproject.toml\",\"setup.py\",\"setup.cfg\",\"Gemfile\",\"Gemfile.lock\",\"uv.lock\",\"CODEOWNERS\",\"DESIGN.md\",\"README.md\",\"CONTRIBUTING.md\",\"CHANGELOG.md\",\"SECURITY.md\",\"CODE_OF_CONDUCT.md\",\"AGENTS.md\",\"CLAUDE.md\",\"GEMINI.md\"],\"required_labels\":[\"agentic-workflows\"],\"target\":\"*\",\"title_prefix\":\"[ci-fix] \"},\"report_incomplete\":{},\"update_pull_request\":{\"allow_body\":true,\"allow_title\":false,\"max\":3,\"target\":\"*\",\"update_branch\":false}}" + GH_AW_SAFE_OUTPUTS_HANDLER_CONFIG: "{\"add_comment\":{\"discussions\":false,\"max\":6,\"required_labels\":[\"agentic-workflows\"],\"required_title_prefix\":\"[ci-fix] \",\"target\":\"*\"},\"add_labels\":{\"allowed\":[\"p/0\"],\"max\":3,\"required_labels\":[\"agentic-workflows\"],\"required_title_prefix\":\"[ci-fix] \",\"target\":\"*\"},\"create_pull_request\":{\"allowed_base_branches\":[\"main\"],\"allowed_branches\":[\"ci-fix/**\"],\"allowed_files\":[\"src/Core/**\",\"src/Controls/**\",\"src/Essentials/**\",\"src/BlazorWebView/**\",\"src/TestUtils/**\",\"src/Templates/**\",\"**/PublicAPI.Unshipped.txt\"],\"base_branch\":\"main\",\"draft\":true,\"labels\":[\"agentic-workflows\"],\"max\":3,\"max_patch_files\":100,\"max_patch_size\":4096,\"protect_top_level_dot_folders\":true,\"protected_files\":[\"package.json\",\"bun.lockb\",\"bunfig.toml\",\"deno.json\",\"deno.jsonc\",\"deno.lock\",\"global.json\",\"NuGet.Config\",\"Directory.Packages.props\",\"mix.exs\",\"mix.lock\",\"go.mod\",\"go.sum\",\"stack.yaml\",\"stack.yaml.lock\",\"pom.xml\",\"build.gradle\",\"build.gradle.kts\",\"settings.gradle\",\"settings.gradle.kts\",\"gradle.properties\",\"package-lock.json\",\"yarn.lock\",\"pnpm-lock.yaml\",\"npm-shrinkwrap.json\",\"requirements.txt\",\"Pipfile\",\"Pipfile.lock\",\"pyproject.toml\",\"setup.py\",\"setup.cfg\",\"Gemfile\",\"Gemfile.lock\",\"uv.lock\",\"CODEOWNERS\",\"DESIGN.md\",\"README.md\",\"CONTRIBUTING.md\",\"CHANGELOG.md\",\"SECURITY.md\",\"CODE_OF_CONDUCT.md\",\"AGENTS.md\",\"CLAUDE.md\",\"GEMINI.md\"],\"protected_files_policy\":\"request_review\",\"title_prefix\":\"[ci-fix] \"},\"create_report_incomplete_issue\":{},\"mark_pull_request_as_ready_for_review\":{\"max\":3,\"required_labels\":[\"agentic-workflows\"],\"required_title_prefix\":\"[ci-fix] \",\"target\":\"*\"},\"missing_data\":{},\"missing_tool\":{},\"noop\":{\"max\":1,\"report-as-issue\":\"true\"},\"push_to_pull_request_branch\":{\"allowed_files\":[\"src/Core/**\",\"src/Controls/**\",\"src/Essentials/**\",\"src/BlazorWebView/**\",\"src/TestUtils/**\",\"src/Templates/**\",\"**/PublicAPI.Unshipped.txt\"],\"if_no_changes\":\"warn\",\"max\":3,\"max_patch_size\":4096,\"protect_top_level_dot_folders\":true,\"protected_files\":[\"package.json\",\"bun.lockb\",\"bunfig.toml\",\"deno.json\",\"deno.jsonc\",\"deno.lock\",\"global.json\",\"NuGet.Config\",\"Directory.Packages.props\",\"mix.exs\",\"mix.lock\",\"go.mod\",\"go.sum\",\"stack.yaml\",\"stack.yaml.lock\",\"pom.xml\",\"build.gradle\",\"build.gradle.kts\",\"settings.gradle\",\"settings.gradle.kts\",\"gradle.properties\",\"package-lock.json\",\"yarn.lock\",\"pnpm-lock.yaml\",\"npm-shrinkwrap.json\",\"requirements.txt\",\"Pipfile\",\"Pipfile.lock\",\"pyproject.toml\",\"setup.py\",\"setup.cfg\",\"Gemfile\",\"Gemfile.lock\",\"uv.lock\",\"CODEOWNERS\",\"DESIGN.md\",\"README.md\",\"CONTRIBUTING.md\",\"CHANGELOG.md\",\"SECURITY.md\",\"CODE_OF_CONDUCT.md\",\"AGENTS.md\",\"CLAUDE.md\",\"GEMINI.md\"],\"required_labels\":[\"agentic-workflows\"],\"target\":\"*\",\"title_prefix\":\"[ci-fix] \"},\"report_incomplete\":{},\"update_pull_request\":{\"allow_body\":true,\"allow_title\":false,\"max\":3,\"target\":\"*\",\"update_branch\":false}}" GH_AW_CI_TRIGGER_TOKEN: ${{ secrets.GH_AW_CI_TRIGGER_TOKEN }} with: github-token: ${{ secrets.GH_AW_GITHUB_TOKEN || secrets.GITHUB_TOKEN }} diff --git a/.github/workflows/ci-status-fix.md b/.github/workflows/ci-status-fix.md index d3227e25170c..9c2f46054029 100644 --- a/.github/workflows/ci-status-fix.md +++ b/.github/workflows/ci-status-fix.md @@ -15,7 +15,9 @@ description: | humans (the open tracking issue is the hand-off surface; a dedicated [ci-fix][needs-human] PR is planned but currently deferred — see Step 6). CI on the PR is kicked by a human `/azp run` for now; the loop watches, classifies, - and re-fixes autonomously between kicks. + and re-fixes autonomously between kicks. When the SPECIFIC test a PR fixed is + confirmed green in that PR's own CI, the loop marks the draft PR ready for review + (a state transition only — it never approves or merges). Never mutes tests, but de-flakes genuinely flaky ones (deterministic synchronization, no retries / timeout bumps). Always skips visual-regression / screenshot issues. @@ -239,8 +241,15 @@ safe-outputs: # PER-RUN total (not per-PR): a sweep may surface several green PRs and/or # annotate several flaky reds in one run. At max:1 all but the first comment # were silently dropped, starving the workflow's #1 value (surfacing green PRs - # for review). Raised to 3 to match the per-run throughput of the other outputs. - max: 3 + # for review). Sized to 6 (not 3) because a single green DRAFT PR that gets + # flipped ready in one sweep spends TWO comment slots — the Step 3 ✅ surface + # comment (which still names the /azp-gated legs a human must kick, so it is NOT + # redundant with 🎯) AND the Step 3.6 T3 🎯 readiness comment. At max:3 the shared + # bucket drained after ~1 draft flip and T3's atomicity pre-check then deferred + # every further mark-ready even while the mark-ready/add_labels buckets (also 3) + # sat idle; 6 lets ~3 draft flips (2 comments each) land per sweep, so the + # mark-ready:3 / add_labels:3 caps are actually reachable. + max: 6 target: "*" # Hard constraint (defense-in-depth): only comment on THIS workflow's own # ci-fix PRs — [ci-fix] title prefix AND agentic-workflows label. Mirrors the @@ -261,13 +270,51 @@ safe-outputs: # never the title, so disable title rewrites — this compiles to allow_title:false # and removes any ability to retitle an arbitrary PR. title: false - # NOTE: gh-aw v0.79.8 does NOT emit required-title-prefix/required-labels into + # NOTE: gh-aw v0.80.9 (the pinned compiler) does NOT emit required-title-prefix/required-labels into # the compiled config for update-pull-request (verified against the lock — it # silently drops them, unlike add-comment / push-to-pull-request-branch which # honor them). So which-PR scoping here relies on prompt Hard-Rule 6 + # min-integrity:approved; the residual is body-marker edits only (no code, no # merge, no close), capped at max:3. Revisit required-* if a future gh-aw # compiler emits them for this output. + mark-pull-request-as-ready-for-review: + # Flip a validated draft [ci-fix] PR to ready-for-review once the SPECIFIC test + # this PR was opened to fix is confirmed green in the PR's OWN CI (Step 3.6) — + # even when unrelated legs are red. The workflow is schedule/dispatch-triggered + # (no triggering-PR context), so target "*" lets the agent name the PR number it + # validated. This is a state transition ONLY: it never approves and never merges — + # a human still reviews and merges. + # PER-RUN total (not per-PR): a sweep may validate several PRs' target tests green. + max: 3 + target: "*" + # Hard constraint (defense-in-depth): only ever un-draft THIS workflow's own + # ci-fix PRs — [ci-fix] title prefix AND agentic-workflows label — so a confused + # or prompt-injected agent cannot mark an arbitrary PR ready. (If the v0.80.9 + # compiler silently drops these — as documented for update-pull-request above — + # the Step 3.6 preconditions + min-integrity:approved are the compensating scope + # controls; verify against the lock after compiling.) + required-title-prefix: "[ci-fix] " + required-labels: [agentic-workflows] + add-labels: + # Apply the p/0 priority label to a [ci-fix] PR at the exact moment the loop flips it + # from draft to ready-for-review (Step 3.6 T3) — so a validated, review-ready fix lands + # in the team's p/0 triage queue instead of sitting unseen in the draft backlog. Paired + # 1:1 with mark-pull-request-as-ready-for-review; target "*" lets the agent name the PR + # it just validated. + # allowed = HARD allowlist: the agent may ONLY ever add p/0, nothing else. This caps the + # blast radius of a confused/prompt-injected agent to exactly one benign priority label — + # it can never apply a downstream-triggering or destructive label. + allowed: [p/0] + # PER-RUN total (not per-PR): a sweep may mark several PRs ready in one run; each adds + # one label, so this matches the mark-ready per-run cap (3). + max: 3 + target: "*" + # Hard constraint (defense-in-depth): only ever label THIS workflow's own ci-fix PRs — + # [ci-fix] title prefix AND agentic-workflows label. (If the v0.80.9 compiler drops these + # — as documented for update-pull-request above — the Step 3.6 preconditions + + # min-integrity:approved + the allowed:[p/0] allowlist are the compensating controls.) + required-title-prefix: "[ci-fix] " + required-labels: [agentic-workflows] timeout-minutes: 90 @@ -339,33 +386,48 @@ through `safe-outputs`. 5. **One issue = one outcome per run.** Exactly one of: respond to a maintainer's change-request on the open PR (Track C, Step 3.5.R); open the first fix/help/ de-flake PR; advance an existing PR by one attempt (push a follow-up fix); - surface a validated-green PR for review; annotate an unrelated-flake red; wait + surface a validated-green PR for review; mark a target-validated draft PR + ready-for-review (Step 3.6 T3 — the terminal outcome that supersedes the same + run's surface/annotate precursor line), or record its `already-ready` + steady-state no-op; annotate an unrelated-flake red; wait (CI not yet settled); or a recorded skip (the dedicated needs-human PR is deferred — the attempt cap records a skip instead; see Step 6). Always prefer advancing or opening a PR over a skip when a non-mute diff is producible. 6. **All writes via `safe-outputs`.** Allowed outputs: `create_pull_request` (first attempt only), `push_to_pull_request_branch` (advance an existing PR by one attempt), `update_pull_request` (bump the attempt marker / refresh the - prior-attempts table), and `add_comment` (progress notes on the `[ci-fix]` PR - ONLY). NEVER comment on the tracking issue (issues are locked by + prior-attempts table), `add_comment` (progress notes on the `[ci-fix]` PR + ONLY), `mark_pull_request_as_ready_for_review` (flip a target-validated draft + `[ci-fix]` PR from draft to ready — Step 3.6 T3 only), and `add_labels` (add + ONLY the `p/0` label, ONLY on that same draft→ready transition — Step 3.6 T3). + NEVER comment on the tracking issue (issues are locked by `.github/workflows/ci-scan-lock-issues.yml`) — `add_comment` targets the PR. No `gh pr create`, no manual `git push`. **Defense-in-depth:** `add_comment` and `update_pull_request` use `target: "*"` (the agent supplies the PR number). - `add_comment` is now config-locked to `required-title-prefix: "[ci-fix] "` + + `add_comment`, `mark_pull_request_as_ready_for_review`, and `add_labels` are + config-locked to `required-title-prefix: "[ci-fix] "` + `required-labels: [agentic-workflows]` (hard-enforced by the handler, same as - `push_to_pull_request_branch`), so a comment can only ever land on THIS - workflow's own PRs. `update_pull_request` CANNOT be config-locked to a - title/label in gh-aw v0.79.8 (the compiler silently drops `required-*` for that - output — verified against the lock), so it keeps `title: false` (no retitles; - body-marker edits only) plus this prompt-level guard. Before emitting either, - VERIFY the target PR carries BOTH the `[ci-fix]` title prefix AND the - `agentic-workflows` label; never comment on or edit the body of any PR that - lacks both. + `push_to_pull_request_branch`), and `add_labels` additionally has an + `allowed: [p/0]` allowlist so it can ONLY ever add `p/0` — so these can only + ever land on THIS workflow's own PRs. `update_pull_request` CANNOT be + config-locked to a title/label in gh-aw v0.80.9 (the compiler silently drops + `required-*` for that output — verified against the lock), so it keeps + `title: false` (no retitles; body-marker edits only) plus this prompt-level + guard. Before emitting ANY of these, VERIFY the target PR carries BOTH the + `[ci-fix]` title prefix AND the `agentic-workflows` label; never comment on, + edit, un-draft, or label any PR that lacks both. 7. **Per-run safe-output caps (per RUN, not per PR).** Each output type is capped per run: `create_pull_request` 3, `push_to_pull_request_branch` 3, - `add_comment` 3, `update_pull_request` 3. Note an ADVANCE spends one push **and** - one update **and** one comment, so ≤ 3 advances/run; surfacing a green or - annotating a flake spends one comment. When a bucket is exhausted, do NOT keep + `add_comment` 6, `update_pull_request` 3, `mark_pull_request_as_ready_for_review` + 3, `add_labels` 3 (counts LABELS, not calls). Note an ADVANCE spends one push + **and** one update **and** one comment, so ≤ 3 advances/run; surfacing a green or + annotating a flake spends one comment; a **mark-ready (Step 3.6 T3)** of a + still-draft PR spends TWO comments (the Step 3 ✅ surface **and** the T3 🎯 + readiness note) **and** one mark-ready **and** one label as an ALL-OR-NOTHING set + (T3 pre-checks those buckets and defers the whole PR if any is exhausted — never + mark a PR ready without its 🎯 audit comment). `add_comment` is therefore sized 6 + (not 3) so ~3 draft flips, at 2 comments each, can land in one sweep instead of the + shared comment bucket starving the otherwise-idle mark-ready/add_labels buckets. When a bucket is exhausted, do NOT keep emitting (extras are silently dropped) — record `skipped: per-run cap reached; deferring PR # to next cycle` for each remaining PR so the drop is a deliberate, logged decision. The continuous loop picks the deferred PRs up next @@ -409,8 +471,9 @@ For every open tracking issue in scope, converge on exactly one outcome: | Open first help-wanted draft `[ci-fix]` PR | No open PR yet; a plausible candidate exists but cannot be runner-validated (device/UI tests) (attempt 1) | | Open first de-flake draft `[ci-fix]` PR | No open PR yet; failure is intermittent (green-on-retry) from a genuine test-quality defect; a deterministic-synchronization fix is producible (attempt 1, Step 4.7 bucket b) | | Advance an existing PR (push attempt N+1) | Open PR's own CI settled red *because of the fix itself*, no human engaged, marker < 10 — push a NEW distinct fix onto the same branch (Step 3.5 → 5.6) | -| Surface a validated-green PR | Open PR's own CI settled green — comment "validated, attempt N/10; ready for review" and do NOT advance (Step 3.5) | -| Annotate an unrelated-flake red | Open PR is red only on baseline-flake / unrelated legs — comment which leg needs a re-run; do NOT burn an attempt (Step 3.5) | +| Surface a validated-green PR | Open PR's own CI settled green — comment "validated, attempt N/10; ready for review" and do NOT advance (Step 3.5), then mark the draft PR ready for review once the fixed test is confirmed green (Step 3.6) | +| Annotate an unrelated-flake red | Open PR is red only on baseline-flake / unrelated legs — comment which leg needs a re-run; do NOT burn an attempt (Step 3.5); if the SPECIFIC fixed test is confirmed green on that build, still mark the draft PR ready for review (Step 3.6) | +| Mark a validated PR ready for review | The specific test this PR fixed is confirmed green in the PR's own CI (even if unrelated legs are red) — comment the target-test result and transition the draft PR to ready for review; never approve or merge (Step 3.6) | | Wait | Open PR's CI is pending / not yet settled on the current head SHA — do nothing this cycle (Step 3.5) | | Hand-off skip (attempt cap) | Marker == 10 and the signature still reproduces — stop and defer to humans; the dedicated `[ci-fix][needs-human]` PR is planned but currently deferred (Step 6) | | Recorded skip | Visual-regression, human engaged, already-handled, fixed-in-latest-build, infra-flake, out-of-bounds, only-mute-available, or no novel approach producible | @@ -685,14 +748,34 @@ Run these gates in order — the FIRST that fires decides this cycle's outcome: 1. **Human engaged.** If `C.humanEngaged` → `skipped: human engaged on PR #; deferring` and stop. Never fight a human reviewer — the intentional hand-off boundary still holds the moment a person touches the PR. -2. **CI not settled / unknown / incomplete prefetch.** If `C.checksSettled == false` - OR `C.overallConclusion` is `pending`, `neutral`, or `unknown`, OR - `C.dataComplete == false` → the fix's CI has not finished on the current head SHA - (`C.headSha`), or the prefetch could not fully read this PR (a partial read can - understate `humanEngaged` / `attempt`). `skipped: PR # CI pending / prefetch - incomplete on ; waiting` and stop. *(Round 1: this is where a - maintainer `/azp run` is awaited — the `/azp`-gated uitests/devicetests legs will - not have run until a human kicks them.)* +2. **CI not settled — but validate the target test first (target-focused readiness).** + If `C.checksSettled == false` OR `C.overallConclusion` is `pending`, `neutral`, or + `unknown`, OR `C.dataComplete == false` → the *overall* build has not finished on the + current head SHA (`C.headSha`), or the prefetch could not fully read this PR (a partial + read can understate `humanEngaged` / `attempt`). Unrelated legs still draining must NOT + keep an already-proven fix parked as a draft, so before waiting, try the target-focused + fast-path: + - **2a — Target-green fast-path (draft `[ci-fix]` PRs only).** If `C.dataComplete == + true` AND the Step 3.6 preconditions hold (draft PR, `[ci-fix] ` title prefix AND the + `agentic-workflows` label), run Step 3.6 **T1–T2** against `C.headSha` now: identify + the target test(s) and read the PR's OWN build **timeline / per-leg status** (anonymous + `_apis/build` — never `_apis/test`, per Hard-Rule 8). If EVERY target test is + **VALIDATED-GREEN** (per T2's all-platform definition below — the target's category leg + is `succeeded` on every platform that runs it, failed on none, and NO target platform + family the pipeline covers is still pending/unconcluded, per T2's "no platform left + unverified" rule; a platform that simply has no leg for the target's category — the test + doesn't run there — is fine, not a gap) AND every leg in + `C.failedLegs` that has ALREADY concluded classifies as **unrelated flake** (Step 4 / + Step 4.7 method — the fix is implicated in NO completed red), then the fix is proven + regardless of unrelated *pending* legs → run **Step 3.6 T3** (mark ready + 🎯 comment) + and stop. This early-out NEVER advances an attempt and NEVER acts on a red that could + be the fix's fault. + - **2b — Otherwise WAIT.** If the target test has not yet executed (pending/absent on + its leg), or a completed red is (or may be) caused by the fix, or `C.dataComplete == + false`, or this PR has no identifiable target test → `skipped: PR # CI pending / + target not yet validated on ; waiting` and stop. *(Round 1: this is where a + maintainer `/azp run` is awaited — the `/azp`-gated uitests/devicetests legs will not + have run until a human kicks them.)* 3. **Green → surface for review.** If `C.overallConclusion == "success"`: the checks that RAN are green. **Primary-gate check first:** the deterministic `success` verdict only certifies "at least one green check, nothing failing or @@ -704,18 +787,31 @@ Run these gates in order — the FIRST that fires decides this cycle's outcome: # primary CI gate (maui-pr) not green on ; waiting` and stop — do NOT surface. (The `/azp`-gated `maui-pr-uitests` (def 313) / `maui-pr-devicetests` (def 314) legs MAY still be un-run — that is expected and is named below, not a - reason to withhold the surface.) **Idempotency:** scan the PR's existing comments - for a prior bot `✅ … validated … on ` note for THIS head SHA — if one - already exists, record `already-surfaced PR # (head )` and stop - WITHOUT re-commenting (re-surfacing the same green every tick is noise and burns - the per-run comment budget other PRs need). Otherwise resolve the attempt number from - `C.effectiveAttempt` (the authoritative max(marker, bot-commit) counter; omit the - number only if it is somehow indeterminate). **Dry-run gate (Step 0):** if `dry_run == "true"`, do NOT emit any comment — instead print the intended `✅` validated-green body to the run log, tally `dry-run: would-surface-green PR #`, and stop. Otherwise `add_comment` on PR #: `✅ Attempt /10 validated - — the fix's CI is green on . Ready for human review.` - Name any `/azp`-gated legs (uitests def 313 / devicetests def 314) that have not - run and still need a maintainer `/azp run`. Do NOT advance. Record - `surfaced-green PR # (attempt /10)` and stop. *(This directly - attacks the real bottleneck — no reviews — so it is the highest-value outcome.)* + reason to withhold the surface.) **Comment idempotency + dry-run suppress the ✅ + comment ONLY — neither skips Step 3.6.** Scan the PR's existing comments for a prior + bot `✅ … validated … on ` note for THIS head SHA; if one exists, set + `SKIP_SURFACE_COMMENT = true`. Resolve the attempt number from `C.effectiveAttempt` + (the authoritative max(marker, bot-commit) counter; omit the number only if it is + somehow indeterminate). Then post the surface comment UNLESS suppressed: + - if `dry_run == "true"`: do NOT emit — print the intended `✅` validated-green body to + the run log and tally `dry-run: would-surface-green PR #`; + - else if `SKIP_SURFACE_COMMENT`: do NOT re-post — record `already-surfaced PR # + (head )` (re-surfacing the same green every tick is noise and burns the + per-run comment budget other PRs need); + - else `add_comment` on PR #: `✅ Attempt /10 validated — the fix's CI is + green on .` naming any `/azp`-gated legs (uitests def 313 / devicetests def + 314) that have not run and still need a maintainer `/azp run`; record `surfaced-green + PR # (attempt /10)`. (Do NOT assert "ready for human review" on a PR that + is still a draft — Step 3.6, not this comment, owns the draft→ready flip and posts its + own 🎯 announcement when it fires.) + Do NOT advance. Then — in ALL of the above cases — run **Step 3.6** (target-test + readiness gate) for this PR before stopping: its `/azp`-gated target legs may conclude + green on this SAME head SHA on a LATER sweep (an `/azp run` adds no commit), and Step + 3.6 is the ONLY place the draft→ready flip happens, so it MUST re-evaluate every sweep — + never `stop` here before it. *(This directly attacks the real bottleneck — no reviews — + so it is the highest-value outcome.)* 4. **Red → classify caused-by-fix vs unrelated-flake.** If `C.overallConclusion == "failure"`, analyze the PR's OWN failing build (NOT `main`): find the AzDO `maui-pr` build for this PR (filter builds by `branchName=refs/pull//merge`, @@ -723,18 +819,26 @@ Run these gates in order — the FIRST that fires decides this cycle's outcome: method and Step 4.7 flake buckets to `C.failedLegs`. - **Unrelated flake only** (every failed leg is known-flaky / infra / a pre-existing baseline red NOT introduced by the fix): do NOT burn an attempt. - **Idempotency first:** scan the PR's existing comments for a prior bot - `♻️ … unrelated flake … on ` note for THIS head SHA — if one already - exists, record `already-annotated-flake PR # (head )` and stop - WITHOUT re-commenting (re-annotating the same flake every 12h sweep is noise and - burns the per-run comment budget other PRs need). Otherwise resolve the attempt - number from `C.effectiveAttempt` (the authoritative max(marker, bot-commit) - counter), omitting the `Attempt /10:` prefix only if it is somehow - indeterminate. **Dry-run gate (Step 0):** if `dry_run == "true"`, do NOT emit any comment — instead print the intended `♻️` unrelated-flake body to the run log, tally `dry-run: would-annotate-flake PR #`, and stop. Otherwise `add_comment` on PR #: `♻️ Attempt /10: red is - unrelated flake on leg(s) () on ; the fix itself is not - implicated. A maintainer re-run (/azp run ) should clear it.` Record - `annotated-flake PR # (head )` and stop. *(Round 1: human re-runs; - Round 2: auto re-trigger.)* + **Comment idempotency + dry-run suppress the ♻️ comment ONLY — neither skips Step + 3.6.** Scan the PR's existing comments for a prior bot `♻️ … unrelated flake … on + ` note for THIS head SHA; if one exists, set `SKIP_FLAKE_COMMENT = true`. + Resolve the attempt number from `C.effectiveAttempt` (the authoritative max(marker, + bot-commit) counter), omitting the `Attempt /10:` prefix only if it is + somehow indeterminate. Then post the flake note UNLESS suppressed: + - if `dry_run == "true"`: do NOT emit — print the intended `♻️` unrelated-flake body + to the run log and tally `dry-run: would-annotate-flake PR #`; + - else if `SKIP_FLAKE_COMMENT`: do NOT re-post — record `already-annotated-flake PR + # (head )` (re-annotating the same flake every 12h sweep is noise and + burns the per-run comment budget other PRs need); + - else `add_comment` on PR #: `♻️ Attempt /10: red is unrelated flake on + leg(s) () on ; the fix itself is not implicated. A + maintainer re-run (/azp run ) should clear it.` record `annotated-flake + PR # (head )`. + Then — in ALL of the above cases — run **Step 3.6** (target-test readiness gate) for + this PR before stopping: its `/azp`-gated target legs may conclude green on this SAME + head SHA on a LATER sweep, and Step 3.6 is the ONLY place the draft→ready flip + happens, so it MUST re-evaluate every sweep — never `stop` here before it. *(Round 1: + human re-runs; Round 2: auto re-trigger.)* - **Caused by the fix** (a failed leg still matches the original target signature, or the fix introduced a NEW failure): advance an attempt. - **Attempt count.** `attempt = C.effectiveAttempt` — the authoritative @@ -752,6 +856,188 @@ Run these gates in order — the FIRST that fires decides this cycle's outcome: from every prior commit on this branch**, emitted via the Step 5.6 ADVANCE path (push onto the same PR). +#### Step 3.6 — Target-test verification & mark-ready gate + +Reached from the Step 3 **green-surface** branch, the Step 4 **unrelated-flake** branch +(AFTER that branch has posted its comment), and the Step 3.5 **gate-2 target-focused +fast-path** (2a — while unrelated legs are still pending). Purpose: confirm the SPECIFIC +test(s) this PR was opened to fix now PASS in the PR's own CI, and — only when they do +— transition the draft `[ci-fix]` PR to **ready for review** so a maintainer sees a +validated fix instead of a draft. This is the ONLY place the loop flips draft→ready; it +is a state transition, **never** an approval or a merge (a human still reviews and +merges). Overall red on *unrelated* legs must NOT gate readiness — we validate the fix, +not the base branch's flakiness. + +**Preconditions** (ALL must hold; otherwise record `skipped: readiness N/A PR # +()` and stop this gate): +- `C.isDraft == true` — if the PR is already ready-for-review, the draft→ready + transition is already done; do NOT re-mark **and do NOT re-apply `p/0`**. `p/0` is applied + exactly ONCE, atomically with the draft→ready flip (T3 — all-or-nothing with the 🎯 audit + comment + mark-ready), so an already-ready loop PR that is MISSING `p/0` has almost + certainly had it **removed by a maintainer** de-prioritizing the PR — a pure triage action + the human-engagement guard (which inspects comments/reviews/commits, NOT label changes) + cannot see. Re-adding `p/0` every sweep would fight that maintainer indefinitely, violating + the loop's "never override a human" contract. So the loop does NOT reconcile the label: + record `already-ready PR #` and stop this gate. (Trade-off: on the rare occasion a + transient API error drops `p/0` at flip time *after* the T3 atomic pre-check passed, it is + not auto-re-added — but the PR is still ready-for-review with its 🎯 audit comment, and + re-adding `p/0` is a trivial manual action; that is strictly preferable to steam-rolling a + maintainer's deliberate de-prioritization.) +- The PR is unmistakably THIS workflow's own: `[ci-fix] ` title prefix AND the + `agentic-workflows` label (mirrors the safe-output lock; the compensating scope + control if the v0.80.9 compiler drops the declarative required-*). +- You reached this gate from Step 3 (green), Step 4 (**unrelated-flake**), or the Step 3.5 + gate-2 **target-focused fast-path** (2a) — i.e. the fix is NOT implicated in any red that + has concluded. If Step 4 classified a red as **caused by the fix** (ADVANCE), do NOT run + this gate — advance the attempt instead. + +**T1 — Identify the target test(s).** From the `[ci-scan]` issue signature the fix +addresses (and the PR's own diff), extract the fully-qualified test method name(s) the +fix targets — e.g. `SafeAreaShouldWorkOnAllShellTabs`. For a de-flake it is the +de-flaked test; for a product fix it is the originally-failing test(s). If NO specific +test can be identified (e.g. a product build-break rather than a test failure), this is a +**build-only fix** — there is no single test to validate cross-platform, so validate at +**whole-build** granularity rather than stopping (Step 3 only *comments*; Step 3.6 is the +sole draft→ready flip, so a build-only fix is undrafted HERE or not at all). We only reach +this gate from Step 3 (green) / Step 4 (unrelated-flake) / gate-2a, so the fix is not +implicated in any concluded red; additionally require, via the T2 timeline method on +`C.headSha`, that the primary build pipeline `maui-pr` (def 302) has CONCLUDED with EVERY +one of its platform build legs `succeeded`/`completed` and NONE failed/canceled or still +pending — the build is green on **every** platform, not just the originally-broken one (the +cross-platform guard, applied to the build instead of a test). A build-only fix is undrafted +ONLY on the strength of the auto-running `maui-pr` (def 302) whole-build green, which the +loop CAN observe: if the PR's `[ci-scan]` issue signature or its own diff indicates the +ORIGINATING failure was in a `/azp`-gated pipeline (`maui-pr-uitests` def 313 or +`maui-pr-devicetests` def 314) rather than `maui-pr` (def 302), do NOT undraft on +`maui-pr`-green alone — those pipelines do not auto-run on this PR (GITHUB_TOKEN cannot +trigger them), so a green `maui-pr` build is NOT evidence the gated build break is fixed; +record `skipped: build-only fix PR # targets gated pipeline () not run — +deferring to human` and stop this gate. Otherwise, set `TARGET := "the maui-pr build +(build-only fix — no single target test)"` and proceed to **T3** to mark ready. If any `maui-pr` build leg is still unconcluded, record `skipped: build-only fix PR +# not yet whole-build green (leg(s) pending)` and stop this gate WITHOUT marking +ready (a green subset is not enough — a still-pending build leg could yet fail). + +**T2 — Verify the target test's platform legs are green (anonymous, leg-level).** Per-test +outcomes come from the AzDO **test-results** API (`_apis/test/...`), which is **NOT reachable +anonymously — Hard-Rule 8**: those endpoints 302-redirect to a sign-in page on this runner, so +a `curl` returns an HTML redirect (not JSON) and every target test would look "not executed". +Validate instead at **leg granularity** using ONLY the anonymous `_apis/build/...` timeline +(Hard-Rule 8) — a `succeeded` category leg means every test in that category **that actually +ran** passed on that platform. This is a strong bar **only if the target test genuinely +executes**: a job can go green while one specific test is skipped at runtime (`[Ignore]`, +`Assert.Ignore()`, `Assert.Inconclusive()`, a `Skip=`/conditional `[Fact]`, an `#if`-out, or +an early `return` before the asserts). So before trusting a green leg as proof the target +passed, **confirm from the PR's OWN diff that the fix does NOT skip, ignore, disable, or +short-circuit the target test** (it adds no `[Ignore]`/`[Explicit]`, `Assert.Ignore`/ +`Assert.Inconclusive`, `Skip=`, category-exclusion, `#if`-out, or early `return` around the +target). If the fix could cause the target to be runtime-skipped rather than genuinely pass, +leg-level green is NOT sufficient evidence — record `skipped: target test may be +runtime-skipped by this fix (diff adds ); leg-green insufficient, deferring to human +on PR #` and stop this gate WITHOUT marking ready. Otherwise (the target genuinely runs +and asserts, as a de-flake or product fix does) a green category leg cannot hide a +target-test failure, so treat it as authoritative. + +Map the target test to the CI leg(s) that run it. A leg name encodes platform + test category +(e.g. `Android UITests SafeAreaEdges,Shadow`, `iOS UITests SafeAreaEdges,Shadow`, `macOS +UITests SafeAreaEdges,Shadow`, `Windows UITests SafeAreaEdges,Shadow`). Determine the target +test's UI-test category — its `[Category(UITestCategories.X)]` in the test/HostApp source, +visible in the PR diff or the test file — to know which leg-name substring identifies its legs; +for a device test, the per-platform device-test legs. + +Using the SAME build-discovery as Step 4 (filter AzDO builds by `branchName=refs/pull//merge` +or `sourceVersion == C.headSha`), read each build's **timeline** on `C.headSha` for the +pipeline(s) that RUN the target test — `maui-pr` (def 302) for unit/integration tests, +`maui-pr-uitests` (def 313) for Appium UI tests, `maui-pr-devicetests` (def 314) for device +tests: + +```bash +ORG=dnceng-public; PROJ=public; BUILD= +# Anonymous + Hard-Rule-8-compliant. NEVER call _apis/test/... (it 302-redirects to sign-in). +# Each type=="Job" record is one platform leg: result (succeeded/failed/canceled/null), +# state (completed/inProgress/pending), name (encodes " UITests "): +curl -s "https://dev.azure.com/$ORG/$PROJ/_apis/build/builds/$BUILD/timeline?api-version=7.1" \ + | tee /tmp/gh-aw/agent/timeline_${BUILD}.json \ + | jq -r '.records[] | select(.type=="Job") | "\(.result)\t\(.state)\t\(.name)"' +``` + +(The prefetch already exposes per-leg status in `C.failedLegs` / the checks context, and +`gh pr checks ` lists the same per-leg rows with platform+category in the name — use +whichever is handy; the build timeline is the authoritative cross-check on the exact build.) + +Treat a target test as **VALIDATED-GREEN** only if, on `C.headSha`, the leg that runs its +category is green on **every platform it runs on** — a fix that repairs one platform must not +silently regress the same test's leg on another, so a green leg on the originally-red platform +alone is NOT enough. Across the target pipeline's platform legs (for a UI test: the Android, +iOS, Windows, and macOS/MacCatalyst legs running the target's category; for a device test: each +device-test platform), require ALL of: +- the target's category leg is `result == "succeeded"` **and** `state == "completed"` on + **every** platform that runs it, AND +- that leg is `failed` / `canceled` / aborted on **NO** platform, AND +- **no platform is left unverified.** For each platform family the target pipeline covers, that + platform's category leg must have CONCLUDED (`state == "completed"`) on `C.headSha`. If a + platform simply has no leg for the target's category, the test does not run there — that is + fine, not a gap. But if a platform's category leg has **not concluded** (`state` is + `inProgress` / pending, or an `/azp`-gated `maui-pr-uitests` / `maui-pr-devicetests` leg that + has not been kicked), the test's status on that platform is UNKNOWN → the fix is NOT yet + validated across platforms: record `skipped: target test green on but + not yet verified on (leg(s) pending / need /azp run) on PR #` + and stop this gate WITHOUT marking ready. + +A target test whose category leg never concluded on ANY platform (**not executed** anywhere — +e.g. its `/azp`-gated pipeline has not been kicked) is likewise NOT validated: record `skipped: +target test not yet executed on PR # ( not run — needs /azp run)` and stop +this gate WITHOUT marking ready. Do NOT overclaim — a green *sibling* leg (a different category +on the same platform) is not the target's leg, and a green leg on one platform is not a pass on +the others. + +**T3 — Mark ready + report.** If EVERY target test is VALIDATED-GREEN. The 🎯 comment, +the mark-ready, and the `p/0` label are THREE SEPARATE safe-outputs — the comment +existing does NOT prove the mark-ready took effect, so they are tracked independently: + +- **Comment idempotency (dup-suppress only — never gates the mark-ready).** We only reach + T3 while `C.isDraft == true` (the precondition stops an already-ready PR before here). So + if a prior bot comment for THIS head that carries the leading `🎯` anchor AND the + contiguous phrase `validated green on ` already exists (BOTH T3 variants — + `🎯 Target test validated green on ` and the build-only `🎯 Build validated green on + ` — contain that exact contiguous phrase, so this recognizes both; keying on + `target test` alone would instead miss the build-only comment and re-post it every sweep. + **Require the `🎯` anchor** so the match can NEVER be loosened to a gapped + `validated…green…on` form that would false-positive on the Step 3 `✅ … validated — the + fix's CI is green on ` surface comment and wrongly suppress the audit 🎯), the comment + landed on an earlier sweep but the **mark-ready did NOT take + effect** (the PR is still a draft) — set `SUPPRESS_COMMENT = true` (do not re-post the + duplicate 🎯 comment) but STILL complete the mark-ready + label below. Never treat the + comment's existence as "already marked ready" while the PR is still a draft. Record this + case as `re-marking PR # (🎯 present; mark-ready did not take on a prior sweep)`. +- **Atomicity budget pre-check (Hard-Rule 7).** Determine the outputs this gate will emit: + `mark_pull_request_as_ready_for_review` + `add_labels` always, plus `add_comment` unless + `SUPPRESS_COMMENT`. Verify a free per-run slot remains in EACH bucket you are about to + use. If ANY required bucket is exhausted, emit NONE of them — record `skipped: per-run + cap reached; deferring mark-ready PR # to next cycle` and stop this gate. Never mark a + PR ready (or label it) unless its 🎯 audit comment is GUARANTEED to exist for this exact + head SHA — either posted in this same sweep, or (in the `SUPPRESS_COMMENT` re-marking case) + already present from a prior sweep. The outputs you DO emit either all land or all defer + together; the audit trail must never be absent, but it is NOT re-posted when it already exists. +- **Dry-run gate (Step 0):** if `dry_run == "true"`, emit NOTHING — print the intended + readiness comment and "would mark ready + add p/0" to the run log, tally `dry-run: + would-mark-ready PR #`, and stop. +- Otherwise emit for THIS PR number ``: + 1. `add_comment` (ONLY if not `SUPPRESS_COMMENT`): `🎯 Target test validated green on + — passed on ALL platforms it runs on (, + buildId ). — not + caused by this fix."> Transitioning this PR from draft to ready for review and adding + `p/0`; a maintainer still reviews and merges.` (For a **build-only fix** — the T1 + whole-build fallback, no single target test — phrase the first clause as `🎯 Build + validated green on — the maui-pr build passed on ALL platforms (buildId + ).` instead of naming a test.) + 2. `mark_pull_request_as_ready_for_review` with `reason:` a one-line justification + naming the validated test(s) and ``. + 3. `add_labels` with `labels: ["p/0"]` for PR # — put the now-review-ready fix into + the team's p/0 priority queue so it is triaged, not lost in the draft backlog. (If + the PR somehow already carries `p/0`, this is a harmless no-op.) +- Record `marked-ready PR # (target green on ALL platforms on , + labeled p/0)` and stop. + #### Step 3.5.R — Maintainer change-request response (Track C) Reached from Step 3.5 gate 0. Goal: when an **eligible human reviewer** (Hard-Rule @@ -1383,7 +1669,19 @@ Filed by [`ci-status-fix`](https://github.com/dotnet/maui/blob/main/.github/work ### Step 8 — Per-issue tally + end-of-run summary -Per issue, append one outcome line to `/tmp/gh-aw/agent/coverage.txt`: +Per issue, append **one** outcome line to `/tmp/gh-aw/agent/coverage.txt` — the +terminal outcome for the cycle. When one cycle produces a chained pair — a PR gets a +same-cycle precursor line and **then** a Step 3.6 readiness line (`marked-ready` on the +draft→ready flip, or `already-ready` on a later steady-state sweep of an already-ready +PR) — record ONLY the terminal Step 3.6 readiness line: it supersedes ANY same-cycle +non-readiness precursor for that PR, so an aggregator keying on one-line-per-issue never +double-counts. The precursor may be a Step 3 green line (`surfaced-green` on the first ✅, +or `already-surfaced` when it already exists) OR — when an unrelated-flake red and a +target-green coincide — a Step 4 `annotated-flake` line; either way the readiness line is +terminal, and the superseded precursor's signal survives in the PR's 🎯/♻️ comment so no +information is lost. This covers the flip cycle (`surfaced-green` → `marked-ready`), the +post-flip steady state (`already-surfaced` → `already-ready`), and the flake-coincident +flip (`annotated-flake` → `marked-ready`): ``` # main attempt- @@ -1394,8 +1692,14 @@ Per issue, append one outcome line to `/tmp/gh-aw/agent/coverage.txt`: `advance-PR # attempt /10` (ADVANCE mode — new commit pushed to the existing PR), `surfaced-green PR #` (fix's own CI went green; commented for review, did not advance), `annotated-flake PR #` (red was unrelated flake; -commented, attempt NOT burned), `waiting PR #` (CI not settled yet), -`dry-run: would-`, `skipped: `. (The +commented, attempt NOT burned), `marked-ready PR #` (the specific fixed test +was confirmed green in the PR's own CI; draft flipped to ready for review), +`already-surfaced PR #` (the fix's ✅ green comment already existed this sweep; +re-post suppressed — superseded by the Step 3.6 readiness line when the PR also flips +ready that cycle), `already-ready PR #` (terminal steady state — the PR was flipped +ready on a prior cycle and remains green on an unchanged head; no action taken, +supersedes `already-surfaced`), `waiting PR #` (CI not settled yet), +`dry-run: would-`, `skipped: `. (The `needs-human-PR` outcome is reserved for the deferred hand-off — Step 6 currently records a skip instead, so it is not emitted.)
not yet whole-build green (leg(s) pending)` and stop this gate WITHOUT marking +ready (a green subset is not enough — a still-pending build leg could yet fail). + +**T2 — Verify the target test's platform legs are green (anonymous, leg-level).** Per-test +outcomes come from the AzDO **test-results** API (`_apis/test/...`), which is **NOT reachable +anonymously — Hard-Rule 8**: those endpoints 302-redirect to a sign-in page on this runner, so +a `curl` returns an HTML redirect (not JSON) and every target test would look "not executed". +Validate instead at **leg granularity** using ONLY the anonymous `_apis/build/...` timeline +(Hard-Rule 8) — a `succeeded` category leg means every test in that category **that actually +ran** passed on that platform. This is a strong bar **only if the target test genuinely +executes**: a job can go green while one specific test is skipped at runtime (`[Ignore]`, +`Assert.Ignore()`, `Assert.Inconclusive()`, a `Skip=`/conditional `[Fact]`, an `#if`-out, or +an early `return` before the asserts). So before trusting a green leg as proof the target +passed, **confirm from the PR's OWN diff that the fix does NOT skip, ignore, disable, or +short-circuit the target test** (it adds no `[Ignore]`/`[Explicit]`, `Assert.Ignore`/ +`Assert.Inconclusive`, `Skip=`, category-exclusion, `#if`-out, or early `return` around the +target). If the fix could cause the target to be runtime-skipped rather than genuinely pass, +leg-level green is NOT sufficient evidence — record `skipped: target test may be +runtime-skipped by this fix (diff adds ); leg-green insufficient, deferring to human +on PR #` and stop this gate WITHOUT marking ready. Otherwise (the target genuinely runs +and asserts, as a de-flake or product fix does) a green category leg cannot hide a +target-test failure, so treat it as authoritative. + +Map the target test to the CI leg(s) that run it. A leg name encodes platform + test category +(e.g. `Android UITests SafeAreaEdges,Shadow`, `iOS UITests SafeAreaEdges,Shadow`, `macOS +UITests SafeAreaEdges,Shadow`, `Windows UITests SafeAreaEdges,Shadow`). Determine the target +test's UI-test category — its `[Category(UITestCategories.X)]` in the test/HostApp source, +visible in the PR diff or the test file — to know which leg-name substring identifies its legs; +for a device test, the per-platform device-test legs. + +Using the SAME build-discovery as Step 4 (filter AzDO builds by `branchName=refs/pull//merge` +or `sourceVersion == C.headSha`), read each build's **timeline** on `C.headSha` for the +pipeline(s) that RUN the target test — `maui-pr` (def 302) for unit/integration tests, +`maui-pr-uitests` (def 313) for Appium UI tests, `maui-pr-devicetests` (def 314) for device +tests: + +```bash +ORG=dnceng-public; PROJ=public; BUILD= +# Anonymous + Hard-Rule-8-compliant. NEVER call _apis/test/... (it 302-redirects to sign-in). +# Each type=="Job" record is one platform leg: result (succeeded/failed/canceled/null), +# state (completed/inProgress/pending), name (encodes " UITests "): +curl -s "https://dev.azure.com/$ORG/$PROJ/_apis/build/builds/$BUILD/timeline?api-version=7.1" \ + | tee /tmp/gh-aw/agent/timeline_${BUILD}.json \ + | jq -r '.records[] | select(.type=="Job") | "\(.result)\t\(.state)\t\(.name)"' +``` + +(The prefetch already exposes per-leg status in `C.failedLegs` / the checks context, and +`gh pr checks ` lists the same per-leg rows with platform+category in the name — use +whichever is handy; the build timeline is the authoritative cross-check on the exact build.) + +Treat a target test as **VALIDATED-GREEN** only if, on `C.headSha`, the leg that runs its +category is green on **every platform it runs on** — a fix that repairs one platform must not +silently regress the same test's leg on another, so a green leg on the originally-red platform +alone is NOT enough. Across the target pipeline's platform legs (for a UI test: the Android, +iOS, Windows, and macOS/MacCatalyst legs running the target's category; for a device test: each +device-test platform), require ALL of: +- the target's category leg is `result == "succeeded"` **and** `state == "completed"` on + **every** platform that runs it, AND +- that leg is `failed` / `canceled` / aborted on **NO** platform, AND +- **no platform is left unverified.** For each platform family the target pipeline covers, that + platform's category leg must have CONCLUDED (`state == "completed"`) on `C.headSha`. If a + platform simply has no leg for the target's category, the test does not run there — that is + fine, not a gap. But if a platform's category leg has **not concluded** (`state` is + `inProgress` / pending, or an `/azp`-gated `maui-pr-uitests` / `maui-pr-devicetests` leg that + has not been kicked), the test's status on that platform is UNKNOWN → the fix is NOT yet + validated across platforms: record `skipped: target test green on but + not yet verified on (leg(s) pending / need /azp run) on PR #` + and stop this gate WITHOUT marking ready. + +A target test whose category leg never concluded on ANY platform (**not executed** anywhere — +e.g. its `/azp`-gated pipeline has not been kicked) is likewise NOT validated: record `skipped: +target test not yet executed on PR # ( not run — needs /azp run)` and stop +this gate WITHOUT marking ready. Do NOT overclaim — a green *sibling* leg (a different category +on the same platform) is not the target's leg, and a green leg on one platform is not a pass on +the others. + +**T3 — Mark ready + report.** If EVERY target test is VALIDATED-GREEN. The 🎯 comment, +the mark-ready, and the `p/0` label are THREE SEPARATE safe-outputs — the comment +existing does NOT prove the mark-ready took effect, so they are tracked independently: + +- **Comment idempotency (dup-suppress only — never gates the mark-ready).** We only reach + T3 while `C.isDraft == true` (the precondition stops an already-ready PR before here). So + if a prior bot comment for THIS head that carries the leading `🎯` anchor AND the + contiguous phrase `validated green on ` already exists (BOTH T3 variants — + `🎯 Target test validated green on ` and the build-only `🎯 Build validated green on + ` — contain that exact contiguous phrase, so this recognizes both; keying on + `target test` alone would instead miss the build-only comment and re-post it every sweep. + **Require the `🎯` anchor** so the match can NEVER be loosened to a gapped + `validated…green…on` form that would false-positive on the Step 3 `✅ … validated — the + fix's CI is green on ` surface comment and wrongly suppress the audit 🎯), the comment + landed on an earlier sweep but the **mark-ready did NOT take + effect** (the PR is still a draft) — set `SUPPRESS_COMMENT = true` (do not re-post the + duplicate 🎯 comment) but STILL complete the mark-ready + label below. Never treat the + comment's existence as "already marked ready" while the PR is still a draft. Record this + case as `re-marking PR # (🎯 present; mark-ready did not take on a prior sweep)`. +- **Atomicity budget pre-check (Hard-Rule 7).** Determine the outputs this gate will emit: + `mark_pull_request_as_ready_for_review` + `add_labels` always, plus `add_comment` unless + `SUPPRESS_COMMENT`. Verify a free per-run slot remains in EACH bucket you are about to + use. If ANY required bucket is exhausted, emit NONE of them — record `skipped: per-run + cap reached; deferring mark-ready PR # to next cycle` and stop this gate. Never mark a + PR ready (or label it) unless its 🎯 audit comment is GUARANTEED to exist for this exact + head SHA — either posted in this same sweep, or (in the `SUPPRESS_COMMENT` re-marking case) + already present from a prior sweep. The outputs you DO emit either all land or all defer + together; the audit trail must never be absent, but it is NOT re-posted when it already exists. +- **Dry-run gate (Step 0):** if `dry_run == "true"`, emit NOTHING — print the intended + readiness comment and "would mark ready + add p/0" to the run log, tally `dry-run: + would-mark-ready PR #`, and stop. +- Otherwise emit for THIS PR number ``: + 1. `add_comment` (ONLY if not `SUPPRESS_COMMENT`): `🎯 Target test validated green on + — passed on ALL platforms it runs on (, + buildId ). — not + caused by this fix."> Transitioning this PR from draft to ready for review and adding + `p/0`; a maintainer still reviews and merges.` (For a **build-only fix** — the T1 + whole-build fallback, no single target test — phrase the first clause as `🎯 Build + validated green on — the maui-pr build passed on ALL platforms (buildId + ).` instead of naming a test.) + 2. `mark_pull_request_as_ready_for_review` with `reason:` a one-line justification + naming the validated test(s) and ``. + 3. `add_labels` with `labels: ["p/0"]` for PR # — put the now-review-ready fix into + the team's p/0 priority queue so it is triaged, not lost in the draft backlog. (If + the PR somehow already carries `p/0`, this is a harmless no-op.) +- Record `marked-ready PR # (target green on ALL platforms on , + labeled p/0)` and stop. + #### Step 3.5.R — Maintainer change-request response (Track C) Reached from Step 3.5 gate 0. Goal: when an **eligible human reviewer** (Hard-Rule @@ -1395,7 +1681,19 @@ Filed by [`ci-status-fix-net11`](https://github.com/dotnet/maui/blob/main/.githu ### Step 8 — Per-issue tally + end-of-run summary -Per issue, append one outcome line to `/tmp/gh-aw/agent/coverage.txt`: +Per issue, append **one** outcome line to `/tmp/gh-aw/agent/coverage.txt` — the +terminal outcome for the cycle. When one cycle produces a chained pair — a PR gets a +same-cycle precursor line and **then** a Step 3.6 readiness line (`marked-ready` on the +draft→ready flip, or `already-ready` on a later steady-state sweep of an already-ready +PR) — record ONLY the terminal Step 3.6 readiness line: it supersedes ANY same-cycle +non-readiness precursor for that PR, so an aggregator keying on one-line-per-issue never +double-counts. The precursor may be a Step 3 green line (`surfaced-green` on the first ✅, +or `already-surfaced` when it already exists) OR — when an unrelated-flake red and a +target-green coincide — a Step 4 `annotated-flake` line; either way the readiness line is +terminal, and the superseded precursor's signal survives in the PR's 🎯/♻️ comment so no +information is lost. This covers the flip cycle (`surfaced-green` → `marked-ready`), the +post-flip steady state (`already-surfaced` → `already-ready`), and the flake-coincident +flip (`annotated-flake` → `marked-ready`): ``` # net11.0 attempt- @@ -1406,8 +1704,14 @@ Per issue, append one outcome line to `/tmp/gh-aw/agent/coverage.txt`: `advance-PR # attempt /10` (ADVANCE mode — new commit pushed to the existing PR), `surfaced-green PR #` (fix's own CI went green; commented for review, did not advance), `annotated-flake PR #` (red was unrelated flake; -commented, attempt NOT burned), `waiting PR #` (CI not settled yet), -`dry-run: would-`, `skipped: `. (The +commented, attempt NOT burned), `marked-ready PR #` (the specific fixed test +was confirmed green in the PR's own CI; draft flipped to ready for review), +`already-surfaced PR #` (the fix's ✅ green comment already existed this sweep; +re-post suppressed — superseded by the Step 3.6 readiness line when the PR also flips +ready that cycle), `already-ready PR #` (terminal steady state — the PR was flipped +ready on a prior cycle and remains green on an unchanged head; no action taken, +supersedes `already-surfaced`), `waiting PR #` (CI not settled yet), +`dry-run: would-`, `skipped: `. (The `needs-human-PR` outcome is reserved for the deferred hand-off — Step 6 currently records a skip instead, so it is not emitted.) diff --git a/.github/workflows/ci-status-fix.lock.yml b/.github/workflows/ci-status-fix.lock.yml index 93dc5969cb0d..65511f2b3259 100644 --- a/.github/workflows/ci-status-fix.lock.yml +++ b/.github/workflows/ci-status-fix.lock.yml @@ -1,4 +1,4 @@ -# gh-aw-metadata: {"schema_version":"v4","frontmatter_hash":"f014161967589ebcaf1ac5ec6f3c88ee5a4388d28b55a07dc36c45b7b2aebab6","body_hash":"86c6b2d4c3df13da14c6a3c8b211381fcc9f97bb1951ff7b80588a2c5338e00c","compiler_version":"v0.80.9","strict":true,"agent_id":"copilot","agent_model":"claude-opus-4.8","engine_versions":{"copilot":"1.0.63"}} +# gh-aw-metadata: {"schema_version":"v4","frontmatter_hash":"b0ebf8a2f179900cfe3638e994b27a43cec52bdbdd2331dc1bd4d65762fa2d6b","body_hash":"1825e2b1f7c7bd88241bf37580655489360f9bebc806edd00837a52ce4ad83a0","compiler_version":"v0.80.9","strict":true,"agent_id":"copilot","agent_model":"claude-opus-4.8","engine_versions":{"copilot":"1.0.63"}} # gh-aw-manifest: {"version":1,"secrets":["COPILOT_PAT_0","COPILOT_PAT_1","COPILOT_PAT_2","COPILOT_PAT_3","COPILOT_PAT_4","COPILOT_PAT_5","COPILOT_PAT_6","COPILOT_PAT_7","COPILOT_PAT_8","COPILOT_PAT_9","GH_AW_CI_TRIGGER_TOKEN","GH_AW_GITHUB_MCP_SERVER_TOKEN","GH_AW_GITHUB_TOKEN","GITHUB_TOKEN"],"actions":[{"repo":"actions/checkout","sha":"34e114876b0b11c390a56381ad16ebd13914f8d5","version":"v4"},{"repo":"actions/checkout","sha":"9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0","version":"v7.0.0"},{"repo":"actions/checkout","sha":"de0fac2e4500dabe0009e67214ff5f5447ce83dd","version":"v6.0.2"},{"repo":"actions/download-artifact","sha":"3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c","version":"v8.0.1"},{"repo":"actions/github-script","sha":"3a2844b7e9c422d3c10d287c895573f7108da1b3","version":"v9.0.0"},{"repo":"actions/setup-node","sha":"48b55a011bda9f5d6aeb4c2d9c7362e8dae4041e","version":"v6.4.0"},{"repo":"actions/upload-artifact","sha":"043fb46d1a93c77aae656e7c1c64a875d1fc6a0a","version":"v7.0.1"},{"repo":"github/gh-aw-actions/setup","sha":"8c7d04ebf1ece56cd381446125da3e0f6896294a","version":"v0.80.9"}],"containers":[{"image":"ghcr.io/github/gh-aw-firewall/agent:0.27.7","digest":"sha256:aae231e4635c8999d039c132f1602d3df850fe9b84a00aa2b5ac981179b5661c","pinned_image":"ghcr.io/github/gh-aw-firewall/agent:0.27.7@sha256:aae231e4635c8999d039c132f1602d3df850fe9b84a00aa2b5ac981179b5661c"},{"image":"ghcr.io/github/gh-aw-firewall/api-proxy:0.27.7","digest":"sha256:009caf2e3d88fa77b64e9a03a95a228fc58db0f1701c6d324b29ba5a3c7c79b6","pinned_image":"ghcr.io/github/gh-aw-firewall/api-proxy:0.27.7@sha256:009caf2e3d88fa77b64e9a03a95a228fc58db0f1701c6d324b29ba5a3c7c79b6"},{"image":"ghcr.io/github/gh-aw-firewall/squid:0.27.7","digest":"sha256:deb1d4e19de62d51cee0508057a596a19315c3423ada4d675cad136dc8037c96","pinned_image":"ghcr.io/github/gh-aw-firewall/squid:0.27.7@sha256:deb1d4e19de62d51cee0508057a596a19315c3423ada4d675cad136dc8037c96"},{"image":"ghcr.io/github/gh-aw-mcpg:v0.3.27","digest":"sha256:fe984bddde4ec05d756d9043edb0a32912e6b7b72f6a121b1082f29221421cc7","pinned_image":"ghcr.io/github/gh-aw-mcpg:v0.3.27@sha256:fe984bddde4ec05d756d9043edb0a32912e6b7b72f6a121b1082f29221421cc7"},{"image":"ghcr.io/github/gh-aw-node","digest":"sha256:529d02eb970b1161aa25c593a9c3df57fdfad5a8add328cb3b6eccef66f3183b","pinned_image":"ghcr.io/github/gh-aw-node@sha256:529d02eb970b1161aa25c593a9c3df57fdfad5a8add328cb3b6eccef66f3183b"},{"image":"ghcr.io/github/github-mcp-server:v1.4.0","digest":"sha256:2afb26356481d1a350e14544a6e160f7f7ec1561a1ea309b823665abf0309036","pinned_image":"ghcr.io/github/github-mcp-server:v1.4.0@sha256:2afb26356481d1a350e14544a6e160f7f7ec1561a1ea309b823665abf0309036"}]} # This file was automatically generated by gh-aw (v0.80.9). DO NOT EDIT. To debug this workflow, load the skill at https://github.com/github/gh-aw/blob/main/debug.md # @@ -37,7 +37,9 @@ # humans (the open tracking issue is the hand-off surface; a dedicated # [ci-fix][needs-human] PR is planned but currently deferred — see Step 6). CI on # the PR is kicked by a human `/azp run` for now; the loop watches, classifies, -# and re-fixes autonomously between kicks. +# and re-fixes autonomously between kicks. When the SPECIFIC test a PR fixed is +# confirmed green in that PR's own CI, the loop marks the draft PR ready for review +# (a state transition only — it never approves or merges). # Never mutes tests, but # de-flakes genuinely flaky ones (deterministic synchronization, no retries / # timeout bumps). Always skips visual-regression / screenshot issues. @@ -273,24 +275,24 @@ jobs: run: | bash "${RUNNER_TEMP}/gh-aw/actions/create_prompt_first.sh" { - cat << 'GH_AW_PROMPT_25cf75111b670167_EOF' + cat << 'GH_AW_PROMPT_2ac1d0b4083428f8_EOF' - GH_AW_PROMPT_25cf75111b670167_EOF + GH_AW_PROMPT_2ac1d0b4083428f8_EOF cat "${RUNNER_TEMP}/gh-aw/prompts/xpia.md" cat "${RUNNER_TEMP}/gh-aw/prompts/temp_folder_prompt.md" cat "${RUNNER_TEMP}/gh-aw/prompts/markdown.md" cat "${RUNNER_TEMP}/gh-aw/prompts/safe_outputs_prompt.md" - cat << 'GH_AW_PROMPT_25cf75111b670167_EOF' + cat << 'GH_AW_PROMPT_2ac1d0b4083428f8_EOF' - Tools: add_comment(max:3), create_pull_request(max:3), update_pull_request(max:3), push_to_pull_request_branch(max:3), missing_tool, missing_data, noop - GH_AW_PROMPT_25cf75111b670167_EOF + Tools: add_comment(max:6), create_pull_request(max:3), update_pull_request(max:3), mark_pull_request_as_ready_for_review(max:3), add_labels(max:3), push_to_pull_request_branch(max:3), missing_tool, missing_data, noop + GH_AW_PROMPT_2ac1d0b4083428f8_EOF cat "${RUNNER_TEMP}/gh-aw/prompts/safe_outputs_create_pull_request.md" cat "${RUNNER_TEMP}/gh-aw/prompts/safe_outputs_push_to_pr_branch.md" - cat << 'GH_AW_PROMPT_25cf75111b670167_EOF' + cat << 'GH_AW_PROMPT_2ac1d0b4083428f8_EOF' - GH_AW_PROMPT_25cf75111b670167_EOF + GH_AW_PROMPT_2ac1d0b4083428f8_EOF cat "${RUNNER_TEMP}/gh-aw/prompts/mcp_cli_tools_prompt.md" - cat << 'GH_AW_PROMPT_25cf75111b670167_EOF' + cat << 'GH_AW_PROMPT_2ac1d0b4083428f8_EOF' The following GitHub context information is available for this workflow: {{#if github.actor}} @@ -332,12 +334,12 @@ jobs: stop immediately and report the limitation rather than spending turns trying to work around it. - GH_AW_PROMPT_25cf75111b670167_EOF + GH_AW_PROMPT_2ac1d0b4083428f8_EOF cat "${RUNNER_TEMP}/gh-aw/prompts/github_mcp_tools_with_safeoutputs_prompt.md" - cat << 'GH_AW_PROMPT_25cf75111b670167_EOF' + cat << 'GH_AW_PROMPT_2ac1d0b4083428f8_EOF' {{#runtime-import .github/workflows/ci-status-fix.md}} - GH_AW_PROMPT_25cf75111b670167_EOF + GH_AW_PROMPT_2ac1d0b4083428f8_EOF } > "$GH_AW_PROMPT" - name: Interpolate variables and render templates uses: actions/github-script@3a2844b7e9c422d3c10d287c895573f7108da1b3 # v9.0.0 @@ -554,16 +556,18 @@ jobs: mkdir -p "${RUNNER_TEMP}/gh-aw/safeoutputs" mkdir -p /tmp/gh-aw/safeoutputs mkdir -p /tmp/gh-aw/mcp-logs/safeoutputs - cat > "${RUNNER_TEMP}/gh-aw/safeoutputs/config.json" << 'GH_AW_SAFE_OUTPUTS_CONFIG_fa2a69fad5fd0485_EOF' - {"add_comment":{"discussions":false,"max":3,"required_labels":["agentic-workflows"],"required_title_prefix":"[ci-fix] ","target":"*"},"create_pull_request":{"allowed_base_branches":["main"],"allowed_branches":["ci-fix/**"],"allowed_files":["src/Core/**","src/Controls/**","src/Essentials/**","src/BlazorWebView/**","src/TestUtils/**","src/Templates/**","**/PublicAPI.Unshipped.txt"],"base_branch":"main","draft":true,"labels":["agentic-workflows"],"max":3,"max_patch_files":100,"max_patch_size":4096,"protect_top_level_dot_folders":true,"protected_files":["package.json","bun.lockb","bunfig.toml","deno.json","deno.jsonc","deno.lock","global.json","NuGet.Config","Directory.Packages.props","mix.exs","mix.lock","go.mod","go.sum","stack.yaml","stack.yaml.lock","pom.xml","build.gradle","build.gradle.kts","settings.gradle","settings.gradle.kts","gradle.properties","package-lock.json","yarn.lock","pnpm-lock.yaml","npm-shrinkwrap.json","requirements.txt","Pipfile","Pipfile.lock","pyproject.toml","setup.py","setup.cfg","Gemfile","Gemfile.lock","uv.lock","CODEOWNERS","DESIGN.md","README.md","CONTRIBUTING.md","CHANGELOG.md","SECURITY.md","CODE_OF_CONDUCT.md","AGENTS.md","CLAUDE.md","GEMINI.md"],"protected_files_policy":"request_review","title_prefix":"[ci-fix] "},"create_report_incomplete_issue":{},"missing_data":{},"missing_tool":{},"noop":{"max":1,"report-as-issue":"true"},"push_to_pull_request_branch":{"allowed_files":["src/Core/**","src/Controls/**","src/Essentials/**","src/BlazorWebView/**","src/TestUtils/**","src/Templates/**","**/PublicAPI.Unshipped.txt"],"if_no_changes":"warn","max":3,"max_patch_size":4096,"protect_top_level_dot_folders":true,"protected_files":["package.json","bun.lockb","bunfig.toml","deno.json","deno.jsonc","deno.lock","global.json","NuGet.Config","Directory.Packages.props","mix.exs","mix.lock","go.mod","go.sum","stack.yaml","stack.yaml.lock","pom.xml","build.gradle","build.gradle.kts","settings.gradle","settings.gradle.kts","gradle.properties","package-lock.json","yarn.lock","pnpm-lock.yaml","npm-shrinkwrap.json","requirements.txt","Pipfile","Pipfile.lock","pyproject.toml","setup.py","setup.cfg","Gemfile","Gemfile.lock","uv.lock","CODEOWNERS","DESIGN.md","README.md","CONTRIBUTING.md","CHANGELOG.md","SECURITY.md","CODE_OF_CONDUCT.md","AGENTS.md","CLAUDE.md","GEMINI.md"],"required_labels":["agentic-workflows"],"target":"*","title_prefix":"[ci-fix] "},"report_incomplete":{},"update_pull_request":{"allow_body":true,"allow_title":false,"max":3,"target":"*","update_branch":false}} - GH_AW_SAFE_OUTPUTS_CONFIG_fa2a69fad5fd0485_EOF + cat > "${RUNNER_TEMP}/gh-aw/safeoutputs/config.json" << 'GH_AW_SAFE_OUTPUTS_CONFIG_0644bf1434ca0071_EOF' + {"add_comment":{"discussions":false,"max":6,"required_labels":["agentic-workflows"],"required_title_prefix":"[ci-fix] ","target":"*"},"add_labels":{"allowed":["p/0"],"max":3,"required_labels":["agentic-workflows"],"required_title_prefix":"[ci-fix] ","target":"*"},"create_pull_request":{"allowed_base_branches":["main"],"allowed_branches":["ci-fix/**"],"allowed_files":["src/Core/**","src/Controls/**","src/Essentials/**","src/BlazorWebView/**","src/TestUtils/**","src/Templates/**","**/PublicAPI.Unshipped.txt"],"base_branch":"main","draft":true,"labels":["agentic-workflows"],"max":3,"max_patch_files":100,"max_patch_size":4096,"protect_top_level_dot_folders":true,"protected_files":["package.json","bun.lockb","bunfig.toml","deno.json","deno.jsonc","deno.lock","global.json","NuGet.Config","Directory.Packages.props","mix.exs","mix.lock","go.mod","go.sum","stack.yaml","stack.yaml.lock","pom.xml","build.gradle","build.gradle.kts","settings.gradle","settings.gradle.kts","gradle.properties","package-lock.json","yarn.lock","pnpm-lock.yaml","npm-shrinkwrap.json","requirements.txt","Pipfile","Pipfile.lock","pyproject.toml","setup.py","setup.cfg","Gemfile","Gemfile.lock","uv.lock","CODEOWNERS","DESIGN.md","README.md","CONTRIBUTING.md","CHANGELOG.md","SECURITY.md","CODE_OF_CONDUCT.md","AGENTS.md","CLAUDE.md","GEMINI.md"],"protected_files_policy":"request_review","title_prefix":"[ci-fix] "},"create_report_incomplete_issue":{},"mark_pull_request_as_ready_for_review":{"max":3,"required_labels":["agentic-workflows"],"required_title_prefix":"[ci-fix] ","target":"*"},"missing_data":{},"missing_tool":{},"noop":{"max":1,"report-as-issue":"true"},"push_to_pull_request_branch":{"allowed_files":["src/Core/**","src/Controls/**","src/Essentials/**","src/BlazorWebView/**","src/TestUtils/**","src/Templates/**","**/PublicAPI.Unshipped.txt"],"if_no_changes":"warn","max":3,"max_patch_size":4096,"protect_top_level_dot_folders":true,"protected_files":["package.json","bun.lockb","bunfig.toml","deno.json","deno.jsonc","deno.lock","global.json","NuGet.Config","Directory.Packages.props","mix.exs","mix.lock","go.mod","go.sum","stack.yaml","stack.yaml.lock","pom.xml","build.gradle","build.gradle.kts","settings.gradle","settings.gradle.kts","gradle.properties","package-lock.json","yarn.lock","pnpm-lock.yaml","npm-shrinkwrap.json","requirements.txt","Pipfile","Pipfile.lock","pyproject.toml","setup.py","setup.cfg","Gemfile","Gemfile.lock","uv.lock","CODEOWNERS","DESIGN.md","README.md","CONTRIBUTING.md","CHANGELOG.md","SECURITY.md","CODE_OF_CONDUCT.md","AGENTS.md","CLAUDE.md","GEMINI.md"],"required_labels":["agentic-workflows"],"target":"*","title_prefix":"[ci-fix] "},"report_incomplete":{},"update_pull_request":{"allow_body":true,"allow_title":false,"max":3,"target":"*","update_branch":false}} + GH_AW_SAFE_OUTPUTS_CONFIG_0644bf1434ca0071_EOF - name: Generate Safe Outputs Tools env: GH_AW_TOOLS_META_JSON: | { "description_suffixes": { - "add_comment": " CONSTRAINTS: Maximum 3 comment(s) can be added. Target: *. Supports reply_to_id for discussion threading.", + "add_comment": " CONSTRAINTS: Maximum 6 comment(s) can be added. Target: *. Supports reply_to_id for discussion threading.", + "add_labels": " CONSTRAINTS: Maximum 3 label(s) can be added. Only these labels are allowed: [\"p/0\"]. Target: *.", "create_pull_request": " CONSTRAINTS: Maximum 3 pull request(s) can be created. Title will be prefixed with \"[ci-fix] \". Labels [\"agentic-workflows\"] will be automatically added. Only these labels are allowed: [\"agentic-workflows\"]. PRs will be created as drafts.", + "mark_pull_request_as_ready_for_review": " CONSTRAINTS: Maximum 3 pull request(s) can be marked as ready for review.", "push_to_pull_request_branch": " CONSTRAINTS: Maximum 3 push(es) can be made. The target pull request title must start with \"[ci-fix] \".", "update_pull_request": " CONSTRAINTS: Maximum 3 pull request(s) can be updated. Target: *." }, @@ -594,6 +598,25 @@ jobs: } } }, + "add_labels": { + "defaultMax": 5, + "fields": { + "item_number": { + "issueNumberOrTemporaryId": true + }, + "labels": { + "required": true, + "type": "array", + "itemType": "string", + "itemSanitize": true, + "itemMaxLength": 128 + }, + "repo": { + "type": "string", + "maxLength": 256 + } + } + }, "create_pull_request": { "defaultMax": 1, "fields": { @@ -635,6 +658,24 @@ jobs: } } }, + "mark_pull_request_as_ready_for_review": { + "defaultMax": 1, + "fields": { + "pull_request_number": { + "issueOrPRNumber": true + }, + "reason": { + "required": true, + "type": "string", + "sanitize": true, + "maxLength": 65000 + }, + "repo": { + "type": "string", + "maxLength": 256 + } + } + }, "missing_data": { "defaultMax": 20, "fields": { @@ -1494,7 +1535,7 @@ jobs: uses: actions/github-script@3a2844b7e9c422d3c10d287c895573f7108da1b3 # v9.0.0 env: WORKFLOW_NAME: "CI Failure Fixer (main)" - WORKFLOW_DESCRIPTION: "Periodic pass over open ci-scan tracking issues filed by the main-branch CI\nfailure scanner (.github/workflows/ci-status-main.md). This workflow targets\nthe `main` branch EXCLUSIVELY: it processes only issues labelled ci-scan and\nopens every PR against main. (The net11.0 branch is handled by the parallel\n.github/workflows/ci-status-fix-net11.md — the two are split because gh-aw can\nonly transport a fix relative to ONE static base branch per workflow, and the\nmain↔net11.0 divergence exceeds gh-aw's 10 MB transport-patch cap.) The fixer\nopens ONE draft [ci-fix] PR per actionable issue against main, then WATCHES\nthat PR's own CI on later runs: when the fix's CI comes back red and the red is\ncaused by the fix itself, it pushes a fresh follow-up fix onto the SAME PR\nbranch (never a second PR) — up to 10 attempts — then stops and defers to\nhumans (the open tracking issue is the hand-off surface; a dedicated\n[ci-fix][needs-human] PR is planned but currently deferred — see Step 6). CI on\nthe PR is kicked by a human `/azp run` for now; the loop watches, classifies,\nand re-fixes autonomously between kicks.\nNever mutes tests, but\nde-flakes genuinely flaky ones (deterministic synchronization, no retries /\ntimeout bumps). Always skips visual-regression / screenshot issues." + WORKFLOW_DESCRIPTION: "Periodic pass over open ci-scan tracking issues filed by the main-branch CI\nfailure scanner (.github/workflows/ci-status-main.md). This workflow targets\nthe `main` branch EXCLUSIVELY: it processes only issues labelled ci-scan and\nopens every PR against main. (The net11.0 branch is handled by the parallel\n.github/workflows/ci-status-fix-net11.md — the two are split because gh-aw can\nonly transport a fix relative to ONE static base branch per workflow, and the\nmain↔net11.0 divergence exceeds gh-aw's 10 MB transport-patch cap.) The fixer\nopens ONE draft [ci-fix] PR per actionable issue against main, then WATCHES\nthat PR's own CI on later runs: when the fix's CI comes back red and the red is\ncaused by the fix itself, it pushes a fresh follow-up fix onto the SAME PR\nbranch (never a second PR) — up to 10 attempts — then stops and defers to\nhumans (the open tracking issue is the hand-off surface; a dedicated\n[ci-fix][needs-human] PR is planned but currently deferred — see Step 6). CI on\nthe PR is kicked by a human `/azp run` for now; the loop watches, classifies,\nand re-fixes autonomously between kicks. When the SPECIFIC test a PR fixed is\nconfirmed green in that PR's own CI, the loop marks the draft PR ready for review\n(a state transition only — it never approves or merges).\nNever mutes tests, but\nde-flakes genuinely flaky ones (deterministic synchronization, no retries /\ntimeout bumps). Always skips visual-regression / screenshot issues." HAS_PATCH: ${{ needs.agent.outputs.has_patch }} with: script: | @@ -1910,7 +1951,7 @@ jobs: GH_AW_ALLOWED_DOMAINS: "*.blob.core.windows.net,*.githubusercontent.com,api.business.githubcopilot.com,api.enterprise.githubcopilot.com,api.github.com,api.githubcopilot.com,api.individual.githubcopilot.com,api.snapcraft.io,archive.ubuntu.com,azure.archive.ubuntu.com,codeload.github.com,crl.geotrust.com,crl.globalsign.com,crl.identrust.com,crl.sectigo.com,crl.thawte.com,crl.usertrust.com,crl.verisign.com,crl3.digicert.com,crl4.digicert.com,crls.ssl.com,dev.azure.com,docs.github.com,github-cloud.githubusercontent.com,github-cloud.s3.amazonaws.com,github.blog,github.com,github.githubassets.com,helix.dot.net,host.docker.internal,json-schema.org,json.schemastore.org,keyserver.ubuntu.com,lfs.github.com,objects.githubusercontent.com,ocsp.digicert.com,ocsp.geotrust.com,ocsp.globalsign.com,ocsp.identrust.com,ocsp.sectigo.com,ocsp.ssl.com,ocsp.thawte.com,ocsp.usertrust.com,ocsp.verisign.com,packagecloud.io,packages.cloud.google.com,packages.microsoft.com,patch-diff.githubusercontent.com,ppa.launchpad.net,raw.githubusercontent.com,registry.npmjs.org,s.symcb.com,s.symcd.com,security.ubuntu.com,telemetry.enterprise.githubcopilot.com,ts-crl.ws.symantec.com,ts-ocsp.ws.symantec.com,www.googleapis.com" GITHUB_SERVER_URL: ${{ github.server_url }} GITHUB_API_URL: ${{ github.api_url }} - GH_AW_SAFE_OUTPUTS_HANDLER_CONFIG: "{\"add_comment\":{\"discussions\":false,\"max\":3,\"required_labels\":[\"agentic-workflows\"],\"required_title_prefix\":\"[ci-fix] \",\"target\":\"*\"},\"create_pull_request\":{\"allowed_base_branches\":[\"main\"],\"allowed_branches\":[\"ci-fix/**\"],\"allowed_files\":[\"src/Core/**\",\"src/Controls/**\",\"src/Essentials/**\",\"src/BlazorWebView/**\",\"src/TestUtils/**\",\"src/Templates/**\",\"**/PublicAPI.Unshipped.txt\"],\"base_branch\":\"main\",\"draft\":true,\"labels\":[\"agentic-workflows\"],\"max\":3,\"max_patch_files\":100,\"max_patch_size\":4096,\"protect_top_level_dot_folders\":true,\"protected_files\":[\"package.json\",\"bun.lockb\",\"bunfig.toml\",\"deno.json\",\"deno.jsonc\",\"deno.lock\",\"global.json\",\"NuGet.Config\",\"Directory.Packages.props\",\"mix.exs\",\"mix.lock\",\"go.mod\",\"go.sum\",\"stack.yaml\",\"stack.yaml.lock\",\"pom.xml\",\"build.gradle\",\"build.gradle.kts\",\"settings.gradle\",\"settings.gradle.kts\",\"gradle.properties\",\"package-lock.json\",\"yarn.lock\",\"pnpm-lock.yaml\",\"npm-shrinkwrap.json\",\"requirements.txt\",\"Pipfile\",\"Pipfile.lock\",\"pyproject.toml\",\"setup.py\",\"setup.cfg\",\"Gemfile\",\"Gemfile.lock\",\"uv.lock\",\"CODEOWNERS\",\"DESIGN.md\",\"README.md\",\"CONTRIBUTING.md\",\"CHANGELOG.md\",\"SECURITY.md\",\"CODE_OF_CONDUCT.md\",\"AGENTS.md\",\"CLAUDE.md\",\"GEMINI.md\"],\"protected_files_policy\":\"request_review\",\"title_prefix\":\"[ci-fix] \"},\"create_report_incomplete_issue\":{},\"missing_data\":{},\"missing_tool\":{},\"noop\":{\"max\":1,\"report-as-issue\":\"true\"},\"push_to_pull_request_branch\":{\"allowed_files\":[\"src/Core/**\",\"src/Controls/**\",\"src/Essentials/**\",\"src/BlazorWebView/**\",\"src/TestUtils/**\",\"src/Templates/**\",\"**/PublicAPI.Unshipped.txt\"],\"if_no_changes\":\"warn\",\"max\":3,\"max_patch_size\":4096,\"protect_top_level_dot_folders\":true,\"protected_files\":[\"package.json\",\"bun.lockb\",\"bunfig.toml\",\"deno.json\",\"deno.jsonc\",\"deno.lock\",\"global.json\",\"NuGet.Config\",\"Directory.Packages.props\",\"mix.exs\",\"mix.lock\",\"go.mod\",\"go.sum\",\"stack.yaml\",\"stack.yaml.lock\",\"pom.xml\",\"build.gradle\",\"build.gradle.kts\",\"settings.gradle\",\"settings.gradle.kts\",\"gradle.properties\",\"package-lock.json\",\"yarn.lock\",\"pnpm-lock.yaml\",\"npm-shrinkwrap.json\",\"requirements.txt\",\"Pipfile\",\"Pipfile.lock\",\"pyproject.toml\",\"setup.py\",\"setup.cfg\",\"Gemfile\",\"Gemfile.lock\",\"uv.lock\",\"CODEOWNERS\",\"DESIGN.md\",\"README.md\",\"CONTRIBUTING.md\",\"CHANGELOG.md\",\"SECURITY.md\",\"CODE_OF_CONDUCT.md\",\"AGENTS.md\",\"CLAUDE.md\",\"GEMINI.md\"],\"required_labels\":[\"agentic-workflows\"],\"target\":\"*\",\"title_prefix\":\"[ci-fix] \"},\"report_incomplete\":{},\"update_pull_request\":{\"allow_body\":true,\"allow_title\":false,\"max\":3,\"target\":\"*\",\"update_branch\":false}}" + GH_AW_SAFE_OUTPUTS_HANDLER_CONFIG: "{\"add_comment\":{\"discussions\":false,\"max\":6,\"required_labels\":[\"agentic-workflows\"],\"required_title_prefix\":\"[ci-fix] \",\"target\":\"*\"},\"add_labels\":{\"allowed\":[\"p/0\"],\"max\":3,\"required_labels\":[\"agentic-workflows\"],\"required_title_prefix\":\"[ci-fix] \",\"target\":\"*\"},\"create_pull_request\":{\"allowed_base_branches\":[\"main\"],\"allowed_branches\":[\"ci-fix/**\"],\"allowed_files\":[\"src/Core/**\",\"src/Controls/**\",\"src/Essentials/**\",\"src/BlazorWebView/**\",\"src/TestUtils/**\",\"src/Templates/**\",\"**/PublicAPI.Unshipped.txt\"],\"base_branch\":\"main\",\"draft\":true,\"labels\":[\"agentic-workflows\"],\"max\":3,\"max_patch_files\":100,\"max_patch_size\":4096,\"protect_top_level_dot_folders\":true,\"protected_files\":[\"package.json\",\"bun.lockb\",\"bunfig.toml\",\"deno.json\",\"deno.jsonc\",\"deno.lock\",\"global.json\",\"NuGet.Config\",\"Directory.Packages.props\",\"mix.exs\",\"mix.lock\",\"go.mod\",\"go.sum\",\"stack.yaml\",\"stack.yaml.lock\",\"pom.xml\",\"build.gradle\",\"build.gradle.kts\",\"settings.gradle\",\"settings.gradle.kts\",\"gradle.properties\",\"package-lock.json\",\"yarn.lock\",\"pnpm-lock.yaml\",\"npm-shrinkwrap.json\",\"requirements.txt\",\"Pipfile\",\"Pipfile.lock\",\"pyproject.toml\",\"setup.py\",\"setup.cfg\",\"Gemfile\",\"Gemfile.lock\",\"uv.lock\",\"CODEOWNERS\",\"DESIGN.md\",\"README.md\",\"CONTRIBUTING.md\",\"CHANGELOG.md\",\"SECURITY.md\",\"CODE_OF_CONDUCT.md\",\"AGENTS.md\",\"CLAUDE.md\",\"GEMINI.md\"],\"protected_files_policy\":\"request_review\",\"title_prefix\":\"[ci-fix] \"},\"create_report_incomplete_issue\":{},\"mark_pull_request_as_ready_for_review\":{\"max\":3,\"required_labels\":[\"agentic-workflows\"],\"required_title_prefix\":\"[ci-fix] \",\"target\":\"*\"},\"missing_data\":{},\"missing_tool\":{},\"noop\":{\"max\":1,\"report-as-issue\":\"true\"},\"push_to_pull_request_branch\":{\"allowed_files\":[\"src/Core/**\",\"src/Controls/**\",\"src/Essentials/**\",\"src/BlazorWebView/**\",\"src/TestUtils/**\",\"src/Templates/**\",\"**/PublicAPI.Unshipped.txt\"],\"if_no_changes\":\"warn\",\"max\":3,\"max_patch_size\":4096,\"protect_top_level_dot_folders\":true,\"protected_files\":[\"package.json\",\"bun.lockb\",\"bunfig.toml\",\"deno.json\",\"deno.jsonc\",\"deno.lock\",\"global.json\",\"NuGet.Config\",\"Directory.Packages.props\",\"mix.exs\",\"mix.lock\",\"go.mod\",\"go.sum\",\"stack.yaml\",\"stack.yaml.lock\",\"pom.xml\",\"build.gradle\",\"build.gradle.kts\",\"settings.gradle\",\"settings.gradle.kts\",\"gradle.properties\",\"package-lock.json\",\"yarn.lock\",\"pnpm-lock.yaml\",\"npm-shrinkwrap.json\",\"requirements.txt\",\"Pipfile\",\"Pipfile.lock\",\"pyproject.toml\",\"setup.py\",\"setup.cfg\",\"Gemfile\",\"Gemfile.lock\",\"uv.lock\",\"CODEOWNERS\",\"DESIGN.md\",\"README.md\",\"CONTRIBUTING.md\",\"CHANGELOG.md\",\"SECURITY.md\",\"CODE_OF_CONDUCT.md\",\"AGENTS.md\",\"CLAUDE.md\",\"GEMINI.md\"],\"required_labels\":[\"agentic-workflows\"],\"target\":\"*\",\"title_prefix\":\"[ci-fix] \"},\"report_incomplete\":{},\"update_pull_request\":{\"allow_body\":true,\"allow_title\":false,\"max\":3,\"target\":\"*\",\"update_branch\":false}}" GH_AW_CI_TRIGGER_TOKEN: ${{ secrets.GH_AW_CI_TRIGGER_TOKEN }} with: github-token: ${{ secrets.GH_AW_GITHUB_TOKEN || secrets.GITHUB_TOKEN }} diff --git a/.github/workflows/ci-status-fix.md b/.github/workflows/ci-status-fix.md index d3227e25170c..9c2f46054029 100644 --- a/.github/workflows/ci-status-fix.md +++ b/.github/workflows/ci-status-fix.md @@ -15,7 +15,9 @@ description: | humans (the open tracking issue is the hand-off surface; a dedicated [ci-fix][needs-human] PR is planned but currently deferred — see Step 6). CI on the PR is kicked by a human `/azp run` for now; the loop watches, classifies, - and re-fixes autonomously between kicks. + and re-fixes autonomously between kicks. When the SPECIFIC test a PR fixed is + confirmed green in that PR's own CI, the loop marks the draft PR ready for review + (a state transition only — it never approves or merges). Never mutes tests, but de-flakes genuinely flaky ones (deterministic synchronization, no retries / timeout bumps). Always skips visual-regression / screenshot issues. @@ -239,8 +241,15 @@ safe-outputs: # PER-RUN total (not per-PR): a sweep may surface several green PRs and/or # annotate several flaky reds in one run. At max:1 all but the first comment # were silently dropped, starving the workflow's #1 value (surfacing green PRs - # for review). Raised to 3 to match the per-run throughput of the other outputs. - max: 3 + # for review). Sized to 6 (not 3) because a single green DRAFT PR that gets + # flipped ready in one sweep spends TWO comment slots — the Step 3 ✅ surface + # comment (which still names the /azp-gated legs a human must kick, so it is NOT + # redundant with 🎯) AND the Step 3.6 T3 🎯 readiness comment. At max:3 the shared + # bucket drained after ~1 draft flip and T3's atomicity pre-check then deferred + # every further mark-ready even while the mark-ready/add_labels buckets (also 3) + # sat idle; 6 lets ~3 draft flips (2 comments each) land per sweep, so the + # mark-ready:3 / add_labels:3 caps are actually reachable. + max: 6 target: "*" # Hard constraint (defense-in-depth): only comment on THIS workflow's own # ci-fix PRs — [ci-fix] title prefix AND agentic-workflows label. Mirrors the @@ -261,13 +270,51 @@ safe-outputs: # never the title, so disable title rewrites — this compiles to allow_title:false # and removes any ability to retitle an arbitrary PR. title: false - # NOTE: gh-aw v0.79.8 does NOT emit required-title-prefix/required-labels into + # NOTE: gh-aw v0.80.9 (the pinned compiler) does NOT emit required-title-prefix/required-labels into # the compiled config for update-pull-request (verified against the lock — it # silently drops them, unlike add-comment / push-to-pull-request-branch which # honor them). So which-PR scoping here relies on prompt Hard-Rule 6 + # min-integrity:approved; the residual is body-marker edits only (no code, no # merge, no close), capped at max:3. Revisit required-* if a future gh-aw # compiler emits them for this output. + mark-pull-request-as-ready-for-review: + # Flip a validated draft [ci-fix] PR to ready-for-review once the SPECIFIC test + # this PR was opened to fix is confirmed green in the PR's OWN CI (Step 3.6) — + # even when unrelated legs are red. The workflow is schedule/dispatch-triggered + # (no triggering-PR context), so target "*" lets the agent name the PR number it + # validated. This is a state transition ONLY: it never approves and never merges — + # a human still reviews and merges. + # PER-RUN total (not per-PR): a sweep may validate several PRs' target tests green. + max: 3 + target: "*" + # Hard constraint (defense-in-depth): only ever un-draft THIS workflow's own + # ci-fix PRs — [ci-fix] title prefix AND agentic-workflows label — so a confused + # or prompt-injected agent cannot mark an arbitrary PR ready. (If the v0.80.9 + # compiler silently drops these — as documented for update-pull-request above — + # the Step 3.6 preconditions + min-integrity:approved are the compensating scope + # controls; verify against the lock after compiling.) + required-title-prefix: "[ci-fix] " + required-labels: [agentic-workflows] + add-labels: + # Apply the p/0 priority label to a [ci-fix] PR at the exact moment the loop flips it + # from draft to ready-for-review (Step 3.6 T3) — so a validated, review-ready fix lands + # in the team's p/0 triage queue instead of sitting unseen in the draft backlog. Paired + # 1:1 with mark-pull-request-as-ready-for-review; target "*" lets the agent name the PR + # it just validated. + # allowed = HARD allowlist: the agent may ONLY ever add p/0, nothing else. This caps the + # blast radius of a confused/prompt-injected agent to exactly one benign priority label — + # it can never apply a downstream-triggering or destructive label. + allowed: [p/0] + # PER-RUN total (not per-PR): a sweep may mark several PRs ready in one run; each adds + # one label, so this matches the mark-ready per-run cap (3). + max: 3 + target: "*" + # Hard constraint (defense-in-depth): only ever label THIS workflow's own ci-fix PRs — + # [ci-fix] title prefix AND agentic-workflows label. (If the v0.80.9 compiler drops these + # — as documented for update-pull-request above — the Step 3.6 preconditions + + # min-integrity:approved + the allowed:[p/0] allowlist are the compensating controls.) + required-title-prefix: "[ci-fix] " + required-labels: [agentic-workflows] timeout-minutes: 90 @@ -339,33 +386,48 @@ through `safe-outputs`. 5. **One issue = one outcome per run.** Exactly one of: respond to a maintainer's change-request on the open PR (Track C, Step 3.5.R); open the first fix/help/ de-flake PR; advance an existing PR by one attempt (push a follow-up fix); - surface a validated-green PR for review; annotate an unrelated-flake red; wait + surface a validated-green PR for review; mark a target-validated draft PR + ready-for-review (Step 3.6 T3 — the terminal outcome that supersedes the same + run's surface/annotate precursor line), or record its `already-ready` + steady-state no-op; annotate an unrelated-flake red; wait (CI not yet settled); or a recorded skip (the dedicated needs-human PR is deferred — the attempt cap records a skip instead; see Step 6). Always prefer advancing or opening a PR over a skip when a non-mute diff is producible. 6. **All writes via `safe-outputs`.** Allowed outputs: `create_pull_request` (first attempt only), `push_to_pull_request_branch` (advance an existing PR by one attempt), `update_pull_request` (bump the attempt marker / refresh the - prior-attempts table), and `add_comment` (progress notes on the `[ci-fix]` PR - ONLY). NEVER comment on the tracking issue (issues are locked by + prior-attempts table), `add_comment` (progress notes on the `[ci-fix]` PR + ONLY), `mark_pull_request_as_ready_for_review` (flip a target-validated draft + `[ci-fix]` PR from draft to ready — Step 3.6 T3 only), and `add_labels` (add + ONLY the `p/0` label, ONLY on that same draft→ready transition — Step 3.6 T3). + NEVER comment on the tracking issue (issues are locked by `.github/workflows/ci-scan-lock-issues.yml`) — `add_comment` targets the PR. No `gh pr create`, no manual `git push`. **Defense-in-depth:** `add_comment` and `update_pull_request` use `target: "*"` (the agent supplies the PR number). - `add_comment` is now config-locked to `required-title-prefix: "[ci-fix] "` + + `add_comment`, `mark_pull_request_as_ready_for_review`, and `add_labels` are + config-locked to `required-title-prefix: "[ci-fix] "` + `required-labels: [agentic-workflows]` (hard-enforced by the handler, same as - `push_to_pull_request_branch`), so a comment can only ever land on THIS - workflow's own PRs. `update_pull_request` CANNOT be config-locked to a - title/label in gh-aw v0.79.8 (the compiler silently drops `required-*` for that - output — verified against the lock), so it keeps `title: false` (no retitles; - body-marker edits only) plus this prompt-level guard. Before emitting either, - VERIFY the target PR carries BOTH the `[ci-fix]` title prefix AND the - `agentic-workflows` label; never comment on or edit the body of any PR that - lacks both. + `push_to_pull_request_branch`), and `add_labels` additionally has an + `allowed: [p/0]` allowlist so it can ONLY ever add `p/0` — so these can only + ever land on THIS workflow's own PRs. `update_pull_request` CANNOT be + config-locked to a title/label in gh-aw v0.80.9 (the compiler silently drops + `required-*` for that output — verified against the lock), so it keeps + `title: false` (no retitles; body-marker edits only) plus this prompt-level + guard. Before emitting ANY of these, VERIFY the target PR carries BOTH the + `[ci-fix]` title prefix AND the `agentic-workflows` label; never comment on, + edit, un-draft, or label any PR that lacks both. 7. **Per-run safe-output caps (per RUN, not per PR).** Each output type is capped per run: `create_pull_request` 3, `push_to_pull_request_branch` 3, - `add_comment` 3, `update_pull_request` 3. Note an ADVANCE spends one push **and** - one update **and** one comment, so ≤ 3 advances/run; surfacing a green or - annotating a flake spends one comment. When a bucket is exhausted, do NOT keep + `add_comment` 6, `update_pull_request` 3, `mark_pull_request_as_ready_for_review` + 3, `add_labels` 3 (counts LABELS, not calls). Note an ADVANCE spends one push + **and** one update **and** one comment, so ≤ 3 advances/run; surfacing a green or + annotating a flake spends one comment; a **mark-ready (Step 3.6 T3)** of a + still-draft PR spends TWO comments (the Step 3 ✅ surface **and** the T3 🎯 + readiness note) **and** one mark-ready **and** one label as an ALL-OR-NOTHING set + (T3 pre-checks those buckets and defers the whole PR if any is exhausted — never + mark a PR ready without its 🎯 audit comment). `add_comment` is therefore sized 6 + (not 3) so ~3 draft flips, at 2 comments each, can land in one sweep instead of the + shared comment bucket starving the otherwise-idle mark-ready/add_labels buckets. When a bucket is exhausted, do NOT keep emitting (extras are silently dropped) — record `skipped: per-run cap reached; deferring PR # to next cycle` for each remaining PR so the drop is a deliberate, logged decision. The continuous loop picks the deferred PRs up next @@ -409,8 +471,9 @@ For every open tracking issue in scope, converge on exactly one outcome: | Open first help-wanted draft `[ci-fix]` PR | No open PR yet; a plausible candidate exists but cannot be runner-validated (device/UI tests) (attempt 1) | | Open first de-flake draft `[ci-fix]` PR | No open PR yet; failure is intermittent (green-on-retry) from a genuine test-quality defect; a deterministic-synchronization fix is producible (attempt 1, Step 4.7 bucket b) | | Advance an existing PR (push attempt N+1) | Open PR's own CI settled red *because of the fix itself*, no human engaged, marker < 10 — push a NEW distinct fix onto the same branch (Step 3.5 → 5.6) | -| Surface a validated-green PR | Open PR's own CI settled green — comment "validated, attempt N/10; ready for review" and do NOT advance (Step 3.5) | -| Annotate an unrelated-flake red | Open PR is red only on baseline-flake / unrelated legs — comment which leg needs a re-run; do NOT burn an attempt (Step 3.5) | +| Surface a validated-green PR | Open PR's own CI settled green — comment "validated, attempt N/10; ready for review" and do NOT advance (Step 3.5), then mark the draft PR ready for review once the fixed test is confirmed green (Step 3.6) | +| Annotate an unrelated-flake red | Open PR is red only on baseline-flake / unrelated legs — comment which leg needs a re-run; do NOT burn an attempt (Step 3.5); if the SPECIFIC fixed test is confirmed green on that build, still mark the draft PR ready for review (Step 3.6) | +| Mark a validated PR ready for review | The specific test this PR fixed is confirmed green in the PR's own CI (even if unrelated legs are red) — comment the target-test result and transition the draft PR to ready for review; never approve or merge (Step 3.6) | | Wait | Open PR's CI is pending / not yet settled on the current head SHA — do nothing this cycle (Step 3.5) | | Hand-off skip (attempt cap) | Marker == 10 and the signature still reproduces — stop and defer to humans; the dedicated `[ci-fix][needs-human]` PR is planned but currently deferred (Step 6) | | Recorded skip | Visual-regression, human engaged, already-handled, fixed-in-latest-build, infra-flake, out-of-bounds, only-mute-available, or no novel approach producible | @@ -685,14 +748,34 @@ Run these gates in order — the FIRST that fires decides this cycle's outcome: 1. **Human engaged.** If `C.humanEngaged` → `skipped: human engaged on PR #; deferring` and stop. Never fight a human reviewer — the intentional hand-off boundary still holds the moment a person touches the PR. -2. **CI not settled / unknown / incomplete prefetch.** If `C.checksSettled == false` - OR `C.overallConclusion` is `pending`, `neutral`, or `unknown`, OR - `C.dataComplete == false` → the fix's CI has not finished on the current head SHA - (`C.headSha`), or the prefetch could not fully read this PR (a partial read can - understate `humanEngaged` / `attempt`). `skipped: PR # CI pending / prefetch - incomplete on ; waiting` and stop. *(Round 1: this is where a - maintainer `/azp run` is awaited — the `/azp`-gated uitests/devicetests legs will - not have run until a human kicks them.)* +2. **CI not settled — but validate the target test first (target-focused readiness).** + If `C.checksSettled == false` OR `C.overallConclusion` is `pending`, `neutral`, or + `unknown`, OR `C.dataComplete == false` → the *overall* build has not finished on the + current head SHA (`C.headSha`), or the prefetch could not fully read this PR (a partial + read can understate `humanEngaged` / `attempt`). Unrelated legs still draining must NOT + keep an already-proven fix parked as a draft, so before waiting, try the target-focused + fast-path: + - **2a — Target-green fast-path (draft `[ci-fix]` PRs only).** If `C.dataComplete == + true` AND the Step 3.6 preconditions hold (draft PR, `[ci-fix] ` title prefix AND the + `agentic-workflows` label), run Step 3.6 **T1–T2** against `C.headSha` now: identify + the target test(s) and read the PR's OWN build **timeline / per-leg status** (anonymous + `_apis/build` — never `_apis/test`, per Hard-Rule 8). If EVERY target test is + **VALIDATED-GREEN** (per T2's all-platform definition below — the target's category leg + is `succeeded` on every platform that runs it, failed on none, and NO target platform + family the pipeline covers is still pending/unconcluded, per T2's "no platform left + unverified" rule; a platform that simply has no leg for the target's category — the test + doesn't run there — is fine, not a gap) AND every leg in + `C.failedLegs` that has ALREADY concluded classifies as **unrelated flake** (Step 4 / + Step 4.7 method — the fix is implicated in NO completed red), then the fix is proven + regardless of unrelated *pending* legs → run **Step 3.6 T3** (mark ready + 🎯 comment) + and stop. This early-out NEVER advances an attempt and NEVER acts on a red that could + be the fix's fault. + - **2b — Otherwise WAIT.** If the target test has not yet executed (pending/absent on + its leg), or a completed red is (or may be) caused by the fix, or `C.dataComplete == + false`, or this PR has no identifiable target test → `skipped: PR # CI pending / + target not yet validated on ; waiting` and stop. *(Round 1: this is where a + maintainer `/azp run` is awaited — the `/azp`-gated uitests/devicetests legs will not + have run until a human kicks them.)* 3. **Green → surface for review.** If `C.overallConclusion == "success"`: the checks that RAN are green. **Primary-gate check first:** the deterministic `success` verdict only certifies "at least one green check, nothing failing or @@ -704,18 +787,31 @@ Run these gates in order — the FIRST that fires decides this cycle's outcome: # primary CI gate (maui-pr) not green on ; waiting` and stop — do NOT surface. (The `/azp`-gated `maui-pr-uitests` (def 313) / `maui-pr-devicetests` (def 314) legs MAY still be un-run — that is expected and is named below, not a - reason to withhold the surface.) **Idempotency:** scan the PR's existing comments - for a prior bot `✅ … validated … on ` note for THIS head SHA — if one - already exists, record `already-surfaced PR # (head )` and stop - WITHOUT re-commenting (re-surfacing the same green every tick is noise and burns - the per-run comment budget other PRs need). Otherwise resolve the attempt number from - `C.effectiveAttempt` (the authoritative max(marker, bot-commit) counter; omit the - number only if it is somehow indeterminate). **Dry-run gate (Step 0):** if `dry_run == "true"`, do NOT emit any comment — instead print the intended `✅` validated-green body to the run log, tally `dry-run: would-surface-green PR #`, and stop. Otherwise `add_comment` on PR #: `✅ Attempt /10 validated - — the fix's CI is green on . Ready for human review.` - Name any `/azp`-gated legs (uitests def 313 / devicetests def 314) that have not - run and still need a maintainer `/azp run`. Do NOT advance. Record - `surfaced-green PR # (attempt /10)` and stop. *(This directly - attacks the real bottleneck — no reviews — so it is the highest-value outcome.)* + reason to withhold the surface.) **Comment idempotency + dry-run suppress the ✅ + comment ONLY — neither skips Step 3.6.** Scan the PR's existing comments for a prior + bot `✅ … validated … on ` note for THIS head SHA; if one exists, set + `SKIP_SURFACE_COMMENT = true`. Resolve the attempt number from `C.effectiveAttempt` + (the authoritative max(marker, bot-commit) counter; omit the number only if it is + somehow indeterminate). Then post the surface comment UNLESS suppressed: + - if `dry_run == "true"`: do NOT emit — print the intended `✅` validated-green body to + the run log and tally `dry-run: would-surface-green PR #`; + - else if `SKIP_SURFACE_COMMENT`: do NOT re-post — record `already-surfaced PR # + (head )` (re-surfacing the same green every tick is noise and burns the + per-run comment budget other PRs need); + - else `add_comment` on PR #: `✅ Attempt /10 validated — the fix's CI is + green on .` naming any `/azp`-gated legs (uitests def 313 / devicetests def + 314) that have not run and still need a maintainer `/azp run`; record `surfaced-green + PR # (attempt /10)`. (Do NOT assert "ready for human review" on a PR that + is still a draft — Step 3.6, not this comment, owns the draft→ready flip and posts its + own 🎯 announcement when it fires.) + Do NOT advance. Then — in ALL of the above cases — run **Step 3.6** (target-test + readiness gate) for this PR before stopping: its `/azp`-gated target legs may conclude + green on this SAME head SHA on a LATER sweep (an `/azp run` adds no commit), and Step + 3.6 is the ONLY place the draft→ready flip happens, so it MUST re-evaluate every sweep — + never `stop` here before it. *(This directly attacks the real bottleneck — no reviews — + so it is the highest-value outcome.)* 4. **Red → classify caused-by-fix vs unrelated-flake.** If `C.overallConclusion == "failure"`, analyze the PR's OWN failing build (NOT `main`): find the AzDO `maui-pr` build for this PR (filter builds by `branchName=refs/pull//merge`, @@ -723,18 +819,26 @@ Run these gates in order — the FIRST that fires decides this cycle's outcome: method and Step 4.7 flake buckets to `C.failedLegs`. - **Unrelated flake only** (every failed leg is known-flaky / infra / a pre-existing baseline red NOT introduced by the fix): do NOT burn an attempt. - **Idempotency first:** scan the PR's existing comments for a prior bot - `♻️ … unrelated flake … on ` note for THIS head SHA — if one already - exists, record `already-annotated-flake PR # (head )` and stop - WITHOUT re-commenting (re-annotating the same flake every 12h sweep is noise and - burns the per-run comment budget other PRs need). Otherwise resolve the attempt - number from `C.effectiveAttempt` (the authoritative max(marker, bot-commit) - counter), omitting the `Attempt /10:` prefix only if it is somehow - indeterminate. **Dry-run gate (Step 0):** if `dry_run == "true"`, do NOT emit any comment — instead print the intended `♻️` unrelated-flake body to the run log, tally `dry-run: would-annotate-flake PR #`, and stop. Otherwise `add_comment` on PR #: `♻️ Attempt /10: red is - unrelated flake on leg(s) () on ; the fix itself is not - implicated. A maintainer re-run (/azp run ) should clear it.` Record - `annotated-flake PR # (head )` and stop. *(Round 1: human re-runs; - Round 2: auto re-trigger.)* + **Comment idempotency + dry-run suppress the ♻️ comment ONLY — neither skips Step + 3.6.** Scan the PR's existing comments for a prior bot `♻️ … unrelated flake … on + ` note for THIS head SHA; if one exists, set `SKIP_FLAKE_COMMENT = true`. + Resolve the attempt number from `C.effectiveAttempt` (the authoritative max(marker, + bot-commit) counter), omitting the `Attempt /10:` prefix only if it is + somehow indeterminate. Then post the flake note UNLESS suppressed: + - if `dry_run == "true"`: do NOT emit — print the intended `♻️` unrelated-flake body + to the run log and tally `dry-run: would-annotate-flake PR #`; + - else if `SKIP_FLAKE_COMMENT`: do NOT re-post — record `already-annotated-flake PR + # (head )` (re-annotating the same flake every 12h sweep is noise and + burns the per-run comment budget other PRs need); + - else `add_comment` on PR #: `♻️ Attempt /10: red is unrelated flake on + leg(s) () on ; the fix itself is not implicated. A + maintainer re-run (/azp run ) should clear it.` record `annotated-flake + PR # (head )`. + Then — in ALL of the above cases — run **Step 3.6** (target-test readiness gate) for + this PR before stopping: its `/azp`-gated target legs may conclude green on this SAME + head SHA on a LATER sweep, and Step 3.6 is the ONLY place the draft→ready flip + happens, so it MUST re-evaluate every sweep — never `stop` here before it. *(Round 1: + human re-runs; Round 2: auto re-trigger.)* - **Caused by the fix** (a failed leg still matches the original target signature, or the fix introduced a NEW failure): advance an attempt. - **Attempt count.** `attempt = C.effectiveAttempt` — the authoritative @@ -752,6 +856,188 @@ Run these gates in order — the FIRST that fires decides this cycle's outcome: from every prior commit on this branch**, emitted via the Step 5.6 ADVANCE path (push onto the same PR). +#### Step 3.6 — Target-test verification & mark-ready gate + +Reached from the Step 3 **green-surface** branch, the Step 4 **unrelated-flake** branch +(AFTER that branch has posted its comment), and the Step 3.5 **gate-2 target-focused +fast-path** (2a — while unrelated legs are still pending). Purpose: confirm the SPECIFIC +test(s) this PR was opened to fix now PASS in the PR's own CI, and — only when they do +— transition the draft `[ci-fix]` PR to **ready for review** so a maintainer sees a +validated fix instead of a draft. This is the ONLY place the loop flips draft→ready; it +is a state transition, **never** an approval or a merge (a human still reviews and +merges). Overall red on *unrelated* legs must NOT gate readiness — we validate the fix, +not the base branch's flakiness. + +**Preconditions** (ALL must hold; otherwise record `skipped: readiness N/A PR # +()` and stop this gate): +- `C.isDraft == true` — if the PR is already ready-for-review, the draft→ready + transition is already done; do NOT re-mark **and do NOT re-apply `p/0`**. `p/0` is applied + exactly ONCE, atomically with the draft→ready flip (T3 — all-or-nothing with the 🎯 audit + comment + mark-ready), so an already-ready loop PR that is MISSING `p/0` has almost + certainly had it **removed by a maintainer** de-prioritizing the PR — a pure triage action + the human-engagement guard (which inspects comments/reviews/commits, NOT label changes) + cannot see. Re-adding `p/0` every sweep would fight that maintainer indefinitely, violating + the loop's "never override a human" contract. So the loop does NOT reconcile the label: + record `already-ready PR #` and stop this gate. (Trade-off: on the rare occasion a + transient API error drops `p/0` at flip time *after* the T3 atomic pre-check passed, it is + not auto-re-added — but the PR is still ready-for-review with its 🎯 audit comment, and + re-adding `p/0` is a trivial manual action; that is strictly preferable to steam-rolling a + maintainer's deliberate de-prioritization.) +- The PR is unmistakably THIS workflow's own: `[ci-fix] ` title prefix AND the + `agentic-workflows` label (mirrors the safe-output lock; the compensating scope + control if the v0.80.9 compiler drops the declarative required-*). +- You reached this gate from Step 3 (green), Step 4 (**unrelated-flake**), or the Step 3.5 + gate-2 **target-focused fast-path** (2a) — i.e. the fix is NOT implicated in any red that + has concluded. If Step 4 classified a red as **caused by the fix** (ADVANCE), do NOT run + this gate — advance the attempt instead. + +**T1 — Identify the target test(s).** From the `[ci-scan]` issue signature the fix +addresses (and the PR's own diff), extract the fully-qualified test method name(s) the +fix targets — e.g. `SafeAreaShouldWorkOnAllShellTabs`. For a de-flake it is the +de-flaked test; for a product fix it is the originally-failing test(s). If NO specific +test can be identified (e.g. a product build-break rather than a test failure), this is a +**build-only fix** — there is no single test to validate cross-platform, so validate at +**whole-build** granularity rather than stopping (Step 3 only *comments*; Step 3.6 is the +sole draft→ready flip, so a build-only fix is undrafted HERE or not at all). We only reach +this gate from Step 3 (green) / Step 4 (unrelated-flake) / gate-2a, so the fix is not +implicated in any concluded red; additionally require, via the T2 timeline method on +`C.headSha`, that the primary build pipeline `maui-pr` (def 302) has CONCLUDED with EVERY +one of its platform build legs `succeeded`/`completed` and NONE failed/canceled or still +pending — the build is green on **every** platform, not just the originally-broken one (the +cross-platform guard, applied to the build instead of a test). A build-only fix is undrafted +ONLY on the strength of the auto-running `maui-pr` (def 302) whole-build green, which the +loop CAN observe: if the PR's `[ci-scan]` issue signature or its own diff indicates the +ORIGINATING failure was in a `/azp`-gated pipeline (`maui-pr-uitests` def 313 or +`maui-pr-devicetests` def 314) rather than `maui-pr` (def 302), do NOT undraft on +`maui-pr`-green alone — those pipelines do not auto-run on this PR (GITHUB_TOKEN cannot +trigger them), so a green `maui-pr` build is NOT evidence the gated build break is fixed; +record `skipped: build-only fix PR # targets gated pipeline () not run — +deferring to human` and stop this gate. Otherwise, set `TARGET := "the maui-pr build +(build-only fix — no single target test)"` and proceed to **T3** to mark ready. If any `maui-pr` build leg is still unconcluded, record `skipped: build-only fix PR +# not yet whole-build green (leg(s) pending)` and stop this gate WITHOUT marking +ready (a green subset is not enough — a still-pending build leg could yet fail). + +**T2 — Verify the target test's platform legs are green (anonymous, leg-level).** Per-test +outcomes come from the AzDO **test-results** API (`_apis/test/...`), which is **NOT reachable +anonymously — Hard-Rule 8**: those endpoints 302-redirect to a sign-in page on this runner, so +a `curl` returns an HTML redirect (not JSON) and every target test would look "not executed". +Validate instead at **leg granularity** using ONLY the anonymous `_apis/build/...` timeline +(Hard-Rule 8) — a `succeeded` category leg means every test in that category **that actually +ran** passed on that platform. This is a strong bar **only if the target test genuinely +executes**: a job can go green while one specific test is skipped at runtime (`[Ignore]`, +`Assert.Ignore()`, `Assert.Inconclusive()`, a `Skip=`/conditional `[Fact]`, an `#if`-out, or +an early `return` before the asserts). So before trusting a green leg as proof the target +passed, **confirm from the PR's OWN diff that the fix does NOT skip, ignore, disable, or +short-circuit the target test** (it adds no `[Ignore]`/`[Explicit]`, `Assert.Ignore`/ +`Assert.Inconclusive`, `Skip=`, category-exclusion, `#if`-out, or early `return` around the +target). If the fix could cause the target to be runtime-skipped rather than genuinely pass, +leg-level green is NOT sufficient evidence — record `skipped: target test may be +runtime-skipped by this fix (diff adds ); leg-green insufficient, deferring to human +on PR #` and stop this gate WITHOUT marking ready. Otherwise (the target genuinely runs +and asserts, as a de-flake or product fix does) a green category leg cannot hide a +target-test failure, so treat it as authoritative. + +Map the target test to the CI leg(s) that run it. A leg name encodes platform + test category +(e.g. `Android UITests SafeAreaEdges,Shadow`, `iOS UITests SafeAreaEdges,Shadow`, `macOS +UITests SafeAreaEdges,Shadow`, `Windows UITests SafeAreaEdges,Shadow`). Determine the target +test's UI-test category — its `[Category(UITestCategories.X)]` in the test/HostApp source, +visible in the PR diff or the test file — to know which leg-name substring identifies its legs; +for a device test, the per-platform device-test legs. + +Using the SAME build-discovery as Step 4 (filter AzDO builds by `branchName=refs/pull//merge` +or `sourceVersion == C.headSha`), read each build's **timeline** on `C.headSha` for the +pipeline(s) that RUN the target test — `maui-pr` (def 302) for unit/integration tests, +`maui-pr-uitests` (def 313) for Appium UI tests, `maui-pr-devicetests` (def 314) for device +tests: + +```bash +ORG=dnceng-public; PROJ=public; BUILD= +# Anonymous + Hard-Rule-8-compliant. NEVER call _apis/test/... (it 302-redirects to sign-in). +# Each type=="Job" record is one platform leg: result (succeeded/failed/canceled/null), +# state (completed/inProgress/pending), name (encodes " UITests "): +curl -s "https://dev.azure.com/$ORG/$PROJ/_apis/build/builds/$BUILD/timeline?api-version=7.1" \ + | tee /tmp/gh-aw/agent/timeline_${BUILD}.json \ + | jq -r '.records[] | select(.type=="Job") | "\(.result)\t\(.state)\t\(.name)"' +``` + +(The prefetch already exposes per-leg status in `C.failedLegs` / the checks context, and +`gh pr checks ` lists the same per-leg rows with platform+category in the name — use +whichever is handy; the build timeline is the authoritative cross-check on the exact build.) + +Treat a target test as **VALIDATED-GREEN** only if, on `C.headSha`, the leg that runs its +category is green on **every platform it runs on** — a fix that repairs one platform must not +silently regress the same test's leg on another, so a green leg on the originally-red platform +alone is NOT enough. Across the target pipeline's platform legs (for a UI test: the Android, +iOS, Windows, and macOS/MacCatalyst legs running the target's category; for a device test: each +device-test platform), require ALL of: +- the target's category leg is `result == "succeeded"` **and** `state == "completed"` on + **every** platform that runs it, AND +- that leg is `failed` / `canceled` / aborted on **NO** platform, AND +- **no platform is left unverified.** For each platform family the target pipeline covers, that + platform's category leg must have CONCLUDED (`state == "completed"`) on `C.headSha`. If a + platform simply has no leg for the target's category, the test does not run there — that is + fine, not a gap. But if a platform's category leg has **not concluded** (`state` is + `inProgress` / pending, or an `/azp`-gated `maui-pr-uitests` / `maui-pr-devicetests` leg that + has not been kicked), the test's status on that platform is UNKNOWN → the fix is NOT yet + validated across platforms: record `skipped: target test green on but + not yet verified on (leg(s) pending / need /azp run) on PR #` + and stop this gate WITHOUT marking ready. + +A target test whose category leg never concluded on ANY platform (**not executed** anywhere — +e.g. its `/azp`-gated pipeline has not been kicked) is likewise NOT validated: record `skipped: +target test not yet executed on PR # ( not run — needs /azp run)` and stop +this gate WITHOUT marking ready. Do NOT overclaim — a green *sibling* leg (a different category +on the same platform) is not the target's leg, and a green leg on one platform is not a pass on +the others. + +**T3 — Mark ready + report.** If EVERY target test is VALIDATED-GREEN. The 🎯 comment, +the mark-ready, and the `p/0` label are THREE SEPARATE safe-outputs — the comment +existing does NOT prove the mark-ready took effect, so they are tracked independently: + +- **Comment idempotency (dup-suppress only — never gates the mark-ready).** We only reach + T3 while `C.isDraft == true` (the precondition stops an already-ready PR before here). So + if a prior bot comment for THIS head that carries the leading `🎯` anchor AND the + contiguous phrase `validated green on ` already exists (BOTH T3 variants — + `🎯 Target test validated green on ` and the build-only `🎯 Build validated green on + ` — contain that exact contiguous phrase, so this recognizes both; keying on + `target test` alone would instead miss the build-only comment and re-post it every sweep. + **Require the `🎯` anchor** so the match can NEVER be loosened to a gapped + `validated…green…on` form that would false-positive on the Step 3 `✅ … validated — the + fix's CI is green on ` surface comment and wrongly suppress the audit 🎯), the comment + landed on an earlier sweep but the **mark-ready did NOT take + effect** (the PR is still a draft) — set `SUPPRESS_COMMENT = true` (do not re-post the + duplicate 🎯 comment) but STILL complete the mark-ready + label below. Never treat the + comment's existence as "already marked ready" while the PR is still a draft. Record this + case as `re-marking PR # (🎯 present; mark-ready did not take on a prior sweep)`. +- **Atomicity budget pre-check (Hard-Rule 7).** Determine the outputs this gate will emit: + `mark_pull_request_as_ready_for_review` + `add_labels` always, plus `add_comment` unless + `SUPPRESS_COMMENT`. Verify a free per-run slot remains in EACH bucket you are about to + use. If ANY required bucket is exhausted, emit NONE of them — record `skipped: per-run + cap reached; deferring mark-ready PR # to next cycle` and stop this gate. Never mark a + PR ready (or label it) unless its 🎯 audit comment is GUARANTEED to exist for this exact + head SHA — either posted in this same sweep, or (in the `SUPPRESS_COMMENT` re-marking case) + already present from a prior sweep. The outputs you DO emit either all land or all defer + together; the audit trail must never be absent, but it is NOT re-posted when it already exists. +- **Dry-run gate (Step 0):** if `dry_run == "true"`, emit NOTHING — print the intended + readiness comment and "would mark ready + add p/0" to the run log, tally `dry-run: + would-mark-ready PR #`, and stop. +- Otherwise emit for THIS PR number ``: + 1. `add_comment` (ONLY if not `SUPPRESS_COMMENT`): `🎯 Target test validated green on + — passed on ALL platforms it runs on (, + buildId ). — not + caused by this fix."> Transitioning this PR from draft to ready for review and adding + `p/0`; a maintainer still reviews and merges.` (For a **build-only fix** — the T1 + whole-build fallback, no single target test — phrase the first clause as `🎯 Build + validated green on — the maui-pr build passed on ALL platforms (buildId + ).` instead of naming a test.) + 2. `mark_pull_request_as_ready_for_review` with `reason:` a one-line justification + naming the validated test(s) and ``. + 3. `add_labels` with `labels: ["p/0"]` for PR # — put the now-review-ready fix into + the team's p/0 priority queue so it is triaged, not lost in the draft backlog. (If + the PR somehow already carries `p/0`, this is a harmless no-op.) +- Record `marked-ready PR # (target green on ALL platforms on , + labeled p/0)` and stop. + #### Step 3.5.R — Maintainer change-request response (Track C) Reached from Step 3.5 gate 0. Goal: when an **eligible human reviewer** (Hard-Rule @@ -1383,7 +1669,19 @@ Filed by [`ci-status-fix`](https://github.com/dotnet/maui/blob/main/.github/work ### Step 8 — Per-issue tally + end-of-run summary -Per issue, append one outcome line to `/tmp/gh-aw/agent/coverage.txt`: +Per issue, append **one** outcome line to `/tmp/gh-aw/agent/coverage.txt` — the +terminal outcome for the cycle. When one cycle produces a chained pair — a PR gets a +same-cycle precursor line and **then** a Step 3.6 readiness line (`marked-ready` on the +draft→ready flip, or `already-ready` on a later steady-state sweep of an already-ready +PR) — record ONLY the terminal Step 3.6 readiness line: it supersedes ANY same-cycle +non-readiness precursor for that PR, so an aggregator keying on one-line-per-issue never +double-counts. The precursor may be a Step 3 green line (`surfaced-green` on the first ✅, +or `already-surfaced` when it already exists) OR — when an unrelated-flake red and a +target-green coincide — a Step 4 `annotated-flake` line; either way the readiness line is +terminal, and the superseded precursor's signal survives in the PR's 🎯/♻️ comment so no +information is lost. This covers the flip cycle (`surfaced-green` → `marked-ready`), the +post-flip steady state (`already-surfaced` → `already-ready`), and the flake-coincident +flip (`annotated-flake` → `marked-ready`): ``` # main attempt- @@ -1394,8 +1692,14 @@ Per issue, append one outcome line to `/tmp/gh-aw/agent/coverage.txt`: `advance-PR # attempt /10` (ADVANCE mode — new commit pushed to the existing PR), `surfaced-green PR #` (fix's own CI went green; commented for review, did not advance), `annotated-flake PR #` (red was unrelated flake; -commented, attempt NOT burned), `waiting PR #` (CI not settled yet), -`dry-run: would-`, `skipped: `. (The +commented, attempt NOT burned), `marked-ready PR #` (the specific fixed test +was confirmed green in the PR's own CI; draft flipped to ready for review), +`already-surfaced PR #` (the fix's ✅ green comment already existed this sweep; +re-post suppressed — superseded by the Step 3.6 readiness line when the PR also flips +ready that cycle), `already-ready PR #` (terminal steady state — the PR was flipped +ready on a prior cycle and remains green on an unchanged head; no action taken, +supersedes `already-surfaced`), `waiting PR #` (CI not settled yet), +`dry-run: would-`, `skipped: `. (The `needs-human-PR` outcome is reserved for the deferred hand-off — Step 6 currently records a skip instead, so it is not emitted.)
` and stop this gate WITHOUT marking ready. Otherwise (the target genuinely runs +and asserts, as a de-flake or product fix does) a green category leg cannot hide a +target-test failure, so treat it as authoritative. + +Map the target test to the CI leg(s) that run it. A leg name encodes platform + test category +(e.g. `Android UITests SafeAreaEdges,Shadow`, `iOS UITests SafeAreaEdges,Shadow`, `macOS +UITests SafeAreaEdges,Shadow`, `Windows UITests SafeAreaEdges,Shadow`). Determine the target +test's UI-test category — its `[Category(UITestCategories.X)]` in the test/HostApp source, +visible in the PR diff or the test file — to know which leg-name substring identifies its legs; +for a device test, the per-platform device-test legs. + +Using the SAME build-discovery as Step 4 (filter AzDO builds by `branchName=refs/pull/
/merge` +or `sourceVersion == C.headSha`), read each build's **timeline** on `C.headSha` for the +pipeline(s) that RUN the target test — `maui-pr` (def 302) for unit/integration tests, +`maui-pr-uitests` (def 313) for Appium UI tests, `maui-pr-devicetests` (def 314) for device +tests: + +```bash +ORG=dnceng-public; PROJ=public; BUILD= +# Anonymous + Hard-Rule-8-compliant. NEVER call _apis/test/... (it 302-redirects to sign-in). +# Each type=="Job" record is one platform leg: result (succeeded/failed/canceled/null), +# state (completed/inProgress/pending), name (encodes " UITests "): +curl -s "https://dev.azure.com/$ORG/$PROJ/_apis/build/builds/$BUILD/timeline?api-version=7.1" \ + | tee /tmp/gh-aw/agent/timeline_${BUILD}.json \ + | jq -r '.records[] | select(.type=="Job") | "\(.result)\t\(.state)\t\(.name)"' +``` + +(The prefetch already exposes per-leg status in `C.failedLegs` / the checks context, and +`gh pr checks ` lists the same per-leg rows with platform+category in the name — use +whichever is handy; the build timeline is the authoritative cross-check on the exact build.) + +Treat a target test as **VALIDATED-GREEN** only if, on `C.headSha`, the leg that runs its +category is green on **every platform it runs on** — a fix that repairs one platform must not +silently regress the same test's leg on another, so a green leg on the originally-red platform +alone is NOT enough. Across the target pipeline's platform legs (for a UI test: the Android, +iOS, Windows, and macOS/MacCatalyst legs running the target's category; for a device test: each +device-test platform), require ALL of: +- the target's category leg is `result == "succeeded"` **and** `state == "completed"` on + **every** platform that runs it, AND +- that leg is `failed` / `canceled` / aborted on **NO** platform, AND +- **no platform is left unverified.** For each platform family the target pipeline covers, that + platform's category leg must have CONCLUDED (`state == "completed"`) on `C.headSha`. If a + platform simply has no leg for the target's category, the test does not run there — that is + fine, not a gap. But if a platform's category leg has **not concluded** (`state` is + `inProgress` / pending, or an `/azp`-gated `maui-pr-uitests` / `maui-pr-devicetests` leg that + has not been kicked), the test's status on that platform is UNKNOWN → the fix is NOT yet + validated across platforms: record `skipped: target test green on but + not yet verified on (leg(s) pending / need /azp run) on PR #` + and stop this gate WITHOUT marking ready. + +A target test whose category leg never concluded on ANY platform (**not executed** anywhere — +e.g. its `/azp`-gated pipeline has not been kicked) is likewise NOT validated: record `skipped: +target test not yet executed on PR # ( not run — needs /azp run)` and stop +this gate WITHOUT marking ready. Do NOT overclaim — a green *sibling* leg (a different category +on the same platform) is not the target's leg, and a green leg on one platform is not a pass on +the others. + +**T3 — Mark ready + report.** If EVERY target test is VALIDATED-GREEN. The 🎯 comment, +the mark-ready, and the `p/0` label are THREE SEPARATE safe-outputs — the comment +existing does NOT prove the mark-ready took effect, so they are tracked independently: + +- **Comment idempotency (dup-suppress only — never gates the mark-ready).** We only reach + T3 while `C.isDraft == true` (the precondition stops an already-ready PR before here). So + if a prior bot comment for THIS head that carries the leading `🎯` anchor AND the + contiguous phrase `validated green on ` already exists (BOTH T3 variants — + `🎯 Target test validated green on ` and the build-only `🎯 Build validated green on + ` — contain that exact contiguous phrase, so this recognizes both; keying on + `target test` alone would instead miss the build-only comment and re-post it every sweep. + **Require the `🎯` anchor** so the match can NEVER be loosened to a gapped + `validated…green…on` form that would false-positive on the Step 3 `✅ … validated — the + fix's CI is green on ` surface comment and wrongly suppress the audit 🎯), the comment + landed on an earlier sweep but the **mark-ready did NOT take + effect** (the PR is still a draft) — set `SUPPRESS_COMMENT = true` (do not re-post the + duplicate 🎯 comment) but STILL complete the mark-ready + label below. Never treat the + comment's existence as "already marked ready" while the PR is still a draft. Record this + case as `re-marking PR # (🎯 present; mark-ready did not take on a prior sweep)`. +- **Atomicity budget pre-check (Hard-Rule 7).** Determine the outputs this gate will emit: + `mark_pull_request_as_ready_for_review` + `add_labels` always, plus `add_comment` unless + `SUPPRESS_COMMENT`. Verify a free per-run slot remains in EACH bucket you are about to + use. If ANY required bucket is exhausted, emit NONE of them — record `skipped: per-run + cap reached; deferring mark-ready PR # to next cycle` and stop this gate. Never mark a + PR ready (or label it) unless its 🎯 audit comment is GUARANTEED to exist for this exact + head SHA — either posted in this same sweep, or (in the `SUPPRESS_COMMENT` re-marking case) + already present from a prior sweep. The outputs you DO emit either all land or all defer + together; the audit trail must never be absent, but it is NOT re-posted when it already exists. +- **Dry-run gate (Step 0):** if `dry_run == "true"`, emit NOTHING — print the intended + readiness comment and "would mark ready + add p/0" to the run log, tally `dry-run: + would-mark-ready PR #`, and stop. +- Otherwise emit for THIS PR number ``: + 1. `add_comment` (ONLY if not `SUPPRESS_COMMENT`): `🎯 Target test validated green on + — passed on ALL platforms it runs on (, + buildId ). — not + caused by this fix."> Transitioning this PR from draft to ready for review and adding + `p/0`; a maintainer still reviews and merges.` (For a **build-only fix** — the T1 + whole-build fallback, no single target test — phrase the first clause as `🎯 Build + validated green on — the maui-pr build passed on ALL platforms (buildId + ).` instead of naming a test.) + 2. `mark_pull_request_as_ready_for_review` with `reason:` a one-line justification + naming the validated test(s) and ``. + 3. `add_labels` with `labels: ["p/0"]` for PR # — put the now-review-ready fix into + the team's p/0 priority queue so it is triaged, not lost in the draft backlog. (If + the PR somehow already carries `p/0`, this is a harmless no-op.) +- Record `marked-ready PR # (target green on ALL platforms on , + labeled p/0)` and stop. + #### Step 3.5.R — Maintainer change-request response (Track C) Reached from Step 3.5 gate 0. Goal: when an **eligible human reviewer** (Hard-Rule @@ -1395,7 +1681,19 @@ Filed by [`ci-status-fix-net11`](https://github.com/dotnet/maui/blob/main/.githu ### Step 8 — Per-issue tally + end-of-run summary -Per issue, append one outcome line to `/tmp/gh-aw/agent/coverage.txt`: +Per issue, append **one** outcome line to `/tmp/gh-aw/agent/coverage.txt` — the +terminal outcome for the cycle. When one cycle produces a chained pair — a PR gets a +same-cycle precursor line and **then** a Step 3.6 readiness line (`marked-ready` on the +draft→ready flip, or `already-ready` on a later steady-state sweep of an already-ready +PR) — record ONLY the terminal Step 3.6 readiness line: it supersedes ANY same-cycle +non-readiness precursor for that PR, so an aggregator keying on one-line-per-issue never +double-counts. The precursor may be a Step 3 green line (`surfaced-green` on the first ✅, +or `already-surfaced` when it already exists) OR — when an unrelated-flake red and a +target-green coincide — a Step 4 `annotated-flake` line; either way the readiness line is +terminal, and the superseded precursor's signal survives in the PR's 🎯/♻️ comment so no +information is lost. This covers the flip cycle (`surfaced-green` → `marked-ready`), the +post-flip steady state (`already-surfaced` → `already-ready`), and the flake-coincident +flip (`annotated-flake` → `marked-ready`): ``` # net11.0 attempt- @@ -1406,8 +1704,14 @@ Per issue, append one outcome line to `/tmp/gh-aw/agent/coverage.txt`: `advance-PR # attempt /10` (ADVANCE mode — new commit pushed to the existing PR), `surfaced-green PR #` (fix's own CI went green; commented for review, did not advance), `annotated-flake PR #` (red was unrelated flake; -commented, attempt NOT burned), `waiting PR #` (CI not settled yet), -`dry-run: would-`, `skipped: `. (The +commented, attempt NOT burned), `marked-ready PR #` (the specific fixed test +was confirmed green in the PR's own CI; draft flipped to ready for review), +`already-surfaced PR #` (the fix's ✅ green comment already existed this sweep; +re-post suppressed — superseded by the Step 3.6 readiness line when the PR also flips +ready that cycle), `already-ready PR #` (terminal steady state — the PR was flipped +ready on a prior cycle and remains green on an unchanged head; no action taken, +supersedes `already-surfaced`), `waiting PR #` (CI not settled yet), +`dry-run: would-`, `skipped: `. (The `needs-human-PR` outcome is reserved for the deferred hand-off — Step 6 currently records a skip instead, so it is not emitted.) diff --git a/.github/workflows/ci-status-fix.lock.yml b/.github/workflows/ci-status-fix.lock.yml index 93dc5969cb0d..65511f2b3259 100644 --- a/.github/workflows/ci-status-fix.lock.yml +++ b/.github/workflows/ci-status-fix.lock.yml @@ -1,4 +1,4 @@ -# gh-aw-metadata: {"schema_version":"v4","frontmatter_hash":"f014161967589ebcaf1ac5ec6f3c88ee5a4388d28b55a07dc36c45b7b2aebab6","body_hash":"86c6b2d4c3df13da14c6a3c8b211381fcc9f97bb1951ff7b80588a2c5338e00c","compiler_version":"v0.80.9","strict":true,"agent_id":"copilot","agent_model":"claude-opus-4.8","engine_versions":{"copilot":"1.0.63"}} +# gh-aw-metadata: {"schema_version":"v4","frontmatter_hash":"b0ebf8a2f179900cfe3638e994b27a43cec52bdbdd2331dc1bd4d65762fa2d6b","body_hash":"1825e2b1f7c7bd88241bf37580655489360f9bebc806edd00837a52ce4ad83a0","compiler_version":"v0.80.9","strict":true,"agent_id":"copilot","agent_model":"claude-opus-4.8","engine_versions":{"copilot":"1.0.63"}} # gh-aw-manifest: {"version":1,"secrets":["COPILOT_PAT_0","COPILOT_PAT_1","COPILOT_PAT_2","COPILOT_PAT_3","COPILOT_PAT_4","COPILOT_PAT_5","COPILOT_PAT_6","COPILOT_PAT_7","COPILOT_PAT_8","COPILOT_PAT_9","GH_AW_CI_TRIGGER_TOKEN","GH_AW_GITHUB_MCP_SERVER_TOKEN","GH_AW_GITHUB_TOKEN","GITHUB_TOKEN"],"actions":[{"repo":"actions/checkout","sha":"34e114876b0b11c390a56381ad16ebd13914f8d5","version":"v4"},{"repo":"actions/checkout","sha":"9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0","version":"v7.0.0"},{"repo":"actions/checkout","sha":"de0fac2e4500dabe0009e67214ff5f5447ce83dd","version":"v6.0.2"},{"repo":"actions/download-artifact","sha":"3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c","version":"v8.0.1"},{"repo":"actions/github-script","sha":"3a2844b7e9c422d3c10d287c895573f7108da1b3","version":"v9.0.0"},{"repo":"actions/setup-node","sha":"48b55a011bda9f5d6aeb4c2d9c7362e8dae4041e","version":"v6.4.0"},{"repo":"actions/upload-artifact","sha":"043fb46d1a93c77aae656e7c1c64a875d1fc6a0a","version":"v7.0.1"},{"repo":"github/gh-aw-actions/setup","sha":"8c7d04ebf1ece56cd381446125da3e0f6896294a","version":"v0.80.9"}],"containers":[{"image":"ghcr.io/github/gh-aw-firewall/agent:0.27.7","digest":"sha256:aae231e4635c8999d039c132f1602d3df850fe9b84a00aa2b5ac981179b5661c","pinned_image":"ghcr.io/github/gh-aw-firewall/agent:0.27.7@sha256:aae231e4635c8999d039c132f1602d3df850fe9b84a00aa2b5ac981179b5661c"},{"image":"ghcr.io/github/gh-aw-firewall/api-proxy:0.27.7","digest":"sha256:009caf2e3d88fa77b64e9a03a95a228fc58db0f1701c6d324b29ba5a3c7c79b6","pinned_image":"ghcr.io/github/gh-aw-firewall/api-proxy:0.27.7@sha256:009caf2e3d88fa77b64e9a03a95a228fc58db0f1701c6d324b29ba5a3c7c79b6"},{"image":"ghcr.io/github/gh-aw-firewall/squid:0.27.7","digest":"sha256:deb1d4e19de62d51cee0508057a596a19315c3423ada4d675cad136dc8037c96","pinned_image":"ghcr.io/github/gh-aw-firewall/squid:0.27.7@sha256:deb1d4e19de62d51cee0508057a596a19315c3423ada4d675cad136dc8037c96"},{"image":"ghcr.io/github/gh-aw-mcpg:v0.3.27","digest":"sha256:fe984bddde4ec05d756d9043edb0a32912e6b7b72f6a121b1082f29221421cc7","pinned_image":"ghcr.io/github/gh-aw-mcpg:v0.3.27@sha256:fe984bddde4ec05d756d9043edb0a32912e6b7b72f6a121b1082f29221421cc7"},{"image":"ghcr.io/github/gh-aw-node","digest":"sha256:529d02eb970b1161aa25c593a9c3df57fdfad5a8add328cb3b6eccef66f3183b","pinned_image":"ghcr.io/github/gh-aw-node@sha256:529d02eb970b1161aa25c593a9c3df57fdfad5a8add328cb3b6eccef66f3183b"},{"image":"ghcr.io/github/github-mcp-server:v1.4.0","digest":"sha256:2afb26356481d1a350e14544a6e160f7f7ec1561a1ea309b823665abf0309036","pinned_image":"ghcr.io/github/github-mcp-server:v1.4.0@sha256:2afb26356481d1a350e14544a6e160f7f7ec1561a1ea309b823665abf0309036"}]} # This file was automatically generated by gh-aw (v0.80.9). DO NOT EDIT. To debug this workflow, load the skill at https://github.com/github/gh-aw/blob/main/debug.md # @@ -37,7 +37,9 @@ # humans (the open tracking issue is the hand-off surface; a dedicated # [ci-fix][needs-human] PR is planned but currently deferred — see Step 6). CI on # the PR is kicked by a human `/azp run` for now; the loop watches, classifies, -# and re-fixes autonomously between kicks. +# and re-fixes autonomously between kicks. When the SPECIFIC test a PR fixed is +# confirmed green in that PR's own CI, the loop marks the draft PR ready for review +# (a state transition only — it never approves or merges). # Never mutes tests, but # de-flakes genuinely flaky ones (deterministic synchronization, no retries / # timeout bumps). Always skips visual-regression / screenshot issues. @@ -273,24 +275,24 @@ jobs: run: | bash "${RUNNER_TEMP}/gh-aw/actions/create_prompt_first.sh" { - cat << 'GH_AW_PROMPT_25cf75111b670167_EOF' + cat << 'GH_AW_PROMPT_2ac1d0b4083428f8_EOF' - GH_AW_PROMPT_25cf75111b670167_EOF + GH_AW_PROMPT_2ac1d0b4083428f8_EOF cat "${RUNNER_TEMP}/gh-aw/prompts/xpia.md" cat "${RUNNER_TEMP}/gh-aw/prompts/temp_folder_prompt.md" cat "${RUNNER_TEMP}/gh-aw/prompts/markdown.md" cat "${RUNNER_TEMP}/gh-aw/prompts/safe_outputs_prompt.md" - cat << 'GH_AW_PROMPT_25cf75111b670167_EOF' + cat << 'GH_AW_PROMPT_2ac1d0b4083428f8_EOF' - Tools: add_comment(max:3), create_pull_request(max:3), update_pull_request(max:3), push_to_pull_request_branch(max:3), missing_tool, missing_data, noop - GH_AW_PROMPT_25cf75111b670167_EOF + Tools: add_comment(max:6), create_pull_request(max:3), update_pull_request(max:3), mark_pull_request_as_ready_for_review(max:3), add_labels(max:3), push_to_pull_request_branch(max:3), missing_tool, missing_data, noop + GH_AW_PROMPT_2ac1d0b4083428f8_EOF cat "${RUNNER_TEMP}/gh-aw/prompts/safe_outputs_create_pull_request.md" cat "${RUNNER_TEMP}/gh-aw/prompts/safe_outputs_push_to_pr_branch.md" - cat << 'GH_AW_PROMPT_25cf75111b670167_EOF' + cat << 'GH_AW_PROMPT_2ac1d0b4083428f8_EOF' - GH_AW_PROMPT_25cf75111b670167_EOF + GH_AW_PROMPT_2ac1d0b4083428f8_EOF cat "${RUNNER_TEMP}/gh-aw/prompts/mcp_cli_tools_prompt.md" - cat << 'GH_AW_PROMPT_25cf75111b670167_EOF' + cat << 'GH_AW_PROMPT_2ac1d0b4083428f8_EOF' The following GitHub context information is available for this workflow: {{#if github.actor}} @@ -332,12 +334,12 @@ jobs: stop immediately and report the limitation rather than spending turns trying to work around it. - GH_AW_PROMPT_25cf75111b670167_EOF + GH_AW_PROMPT_2ac1d0b4083428f8_EOF cat "${RUNNER_TEMP}/gh-aw/prompts/github_mcp_tools_with_safeoutputs_prompt.md" - cat << 'GH_AW_PROMPT_25cf75111b670167_EOF' + cat << 'GH_AW_PROMPT_2ac1d0b4083428f8_EOF' {{#runtime-import .github/workflows/ci-status-fix.md}} - GH_AW_PROMPT_25cf75111b670167_EOF + GH_AW_PROMPT_2ac1d0b4083428f8_EOF } > "$GH_AW_PROMPT" - name: Interpolate variables and render templates uses: actions/github-script@3a2844b7e9c422d3c10d287c895573f7108da1b3 # v9.0.0 @@ -554,16 +556,18 @@ jobs: mkdir -p "${RUNNER_TEMP}/gh-aw/safeoutputs" mkdir -p /tmp/gh-aw/safeoutputs mkdir -p /tmp/gh-aw/mcp-logs/safeoutputs - cat > "${RUNNER_TEMP}/gh-aw/safeoutputs/config.json" << 'GH_AW_SAFE_OUTPUTS_CONFIG_fa2a69fad5fd0485_EOF' - {"add_comment":{"discussions":false,"max":3,"required_labels":["agentic-workflows"],"required_title_prefix":"[ci-fix] ","target":"*"},"create_pull_request":{"allowed_base_branches":["main"],"allowed_branches":["ci-fix/**"],"allowed_files":["src/Core/**","src/Controls/**","src/Essentials/**","src/BlazorWebView/**","src/TestUtils/**","src/Templates/**","**/PublicAPI.Unshipped.txt"],"base_branch":"main","draft":true,"labels":["agentic-workflows"],"max":3,"max_patch_files":100,"max_patch_size":4096,"protect_top_level_dot_folders":true,"protected_files":["package.json","bun.lockb","bunfig.toml","deno.json","deno.jsonc","deno.lock","global.json","NuGet.Config","Directory.Packages.props","mix.exs","mix.lock","go.mod","go.sum","stack.yaml","stack.yaml.lock","pom.xml","build.gradle","build.gradle.kts","settings.gradle","settings.gradle.kts","gradle.properties","package-lock.json","yarn.lock","pnpm-lock.yaml","npm-shrinkwrap.json","requirements.txt","Pipfile","Pipfile.lock","pyproject.toml","setup.py","setup.cfg","Gemfile","Gemfile.lock","uv.lock","CODEOWNERS","DESIGN.md","README.md","CONTRIBUTING.md","CHANGELOG.md","SECURITY.md","CODE_OF_CONDUCT.md","AGENTS.md","CLAUDE.md","GEMINI.md"],"protected_files_policy":"request_review","title_prefix":"[ci-fix] "},"create_report_incomplete_issue":{},"missing_data":{},"missing_tool":{},"noop":{"max":1,"report-as-issue":"true"},"push_to_pull_request_branch":{"allowed_files":["src/Core/**","src/Controls/**","src/Essentials/**","src/BlazorWebView/**","src/TestUtils/**","src/Templates/**","**/PublicAPI.Unshipped.txt"],"if_no_changes":"warn","max":3,"max_patch_size":4096,"protect_top_level_dot_folders":true,"protected_files":["package.json","bun.lockb","bunfig.toml","deno.json","deno.jsonc","deno.lock","global.json","NuGet.Config","Directory.Packages.props","mix.exs","mix.lock","go.mod","go.sum","stack.yaml","stack.yaml.lock","pom.xml","build.gradle","build.gradle.kts","settings.gradle","settings.gradle.kts","gradle.properties","package-lock.json","yarn.lock","pnpm-lock.yaml","npm-shrinkwrap.json","requirements.txt","Pipfile","Pipfile.lock","pyproject.toml","setup.py","setup.cfg","Gemfile","Gemfile.lock","uv.lock","CODEOWNERS","DESIGN.md","README.md","CONTRIBUTING.md","CHANGELOG.md","SECURITY.md","CODE_OF_CONDUCT.md","AGENTS.md","CLAUDE.md","GEMINI.md"],"required_labels":["agentic-workflows"],"target":"*","title_prefix":"[ci-fix] "},"report_incomplete":{},"update_pull_request":{"allow_body":true,"allow_title":false,"max":3,"target":"*","update_branch":false}} - GH_AW_SAFE_OUTPUTS_CONFIG_fa2a69fad5fd0485_EOF + cat > "${RUNNER_TEMP}/gh-aw/safeoutputs/config.json" << 'GH_AW_SAFE_OUTPUTS_CONFIG_0644bf1434ca0071_EOF' + {"add_comment":{"discussions":false,"max":6,"required_labels":["agentic-workflows"],"required_title_prefix":"[ci-fix] ","target":"*"},"add_labels":{"allowed":["p/0"],"max":3,"required_labels":["agentic-workflows"],"required_title_prefix":"[ci-fix] ","target":"*"},"create_pull_request":{"allowed_base_branches":["main"],"allowed_branches":["ci-fix/**"],"allowed_files":["src/Core/**","src/Controls/**","src/Essentials/**","src/BlazorWebView/**","src/TestUtils/**","src/Templates/**","**/PublicAPI.Unshipped.txt"],"base_branch":"main","draft":true,"labels":["agentic-workflows"],"max":3,"max_patch_files":100,"max_patch_size":4096,"protect_top_level_dot_folders":true,"protected_files":["package.json","bun.lockb","bunfig.toml","deno.json","deno.jsonc","deno.lock","global.json","NuGet.Config","Directory.Packages.props","mix.exs","mix.lock","go.mod","go.sum","stack.yaml","stack.yaml.lock","pom.xml","build.gradle","build.gradle.kts","settings.gradle","settings.gradle.kts","gradle.properties","package-lock.json","yarn.lock","pnpm-lock.yaml","npm-shrinkwrap.json","requirements.txt","Pipfile","Pipfile.lock","pyproject.toml","setup.py","setup.cfg","Gemfile","Gemfile.lock","uv.lock","CODEOWNERS","DESIGN.md","README.md","CONTRIBUTING.md","CHANGELOG.md","SECURITY.md","CODE_OF_CONDUCT.md","AGENTS.md","CLAUDE.md","GEMINI.md"],"protected_files_policy":"request_review","title_prefix":"[ci-fix] "},"create_report_incomplete_issue":{},"mark_pull_request_as_ready_for_review":{"max":3,"required_labels":["agentic-workflows"],"required_title_prefix":"[ci-fix] ","target":"*"},"missing_data":{},"missing_tool":{},"noop":{"max":1,"report-as-issue":"true"},"push_to_pull_request_branch":{"allowed_files":["src/Core/**","src/Controls/**","src/Essentials/**","src/BlazorWebView/**","src/TestUtils/**","src/Templates/**","**/PublicAPI.Unshipped.txt"],"if_no_changes":"warn","max":3,"max_patch_size":4096,"protect_top_level_dot_folders":true,"protected_files":["package.json","bun.lockb","bunfig.toml","deno.json","deno.jsonc","deno.lock","global.json","NuGet.Config","Directory.Packages.props","mix.exs","mix.lock","go.mod","go.sum","stack.yaml","stack.yaml.lock","pom.xml","build.gradle","build.gradle.kts","settings.gradle","settings.gradle.kts","gradle.properties","package-lock.json","yarn.lock","pnpm-lock.yaml","npm-shrinkwrap.json","requirements.txt","Pipfile","Pipfile.lock","pyproject.toml","setup.py","setup.cfg","Gemfile","Gemfile.lock","uv.lock","CODEOWNERS","DESIGN.md","README.md","CONTRIBUTING.md","CHANGELOG.md","SECURITY.md","CODE_OF_CONDUCT.md","AGENTS.md","CLAUDE.md","GEMINI.md"],"required_labels":["agentic-workflows"],"target":"*","title_prefix":"[ci-fix] "},"report_incomplete":{},"update_pull_request":{"allow_body":true,"allow_title":false,"max":3,"target":"*","update_branch":false}} + GH_AW_SAFE_OUTPUTS_CONFIG_0644bf1434ca0071_EOF - name: Generate Safe Outputs Tools env: GH_AW_TOOLS_META_JSON: | { "description_suffixes": { - "add_comment": " CONSTRAINTS: Maximum 3 comment(s) can be added. Target: *. Supports reply_to_id for discussion threading.", + "add_comment": " CONSTRAINTS: Maximum 6 comment(s) can be added. Target: *. Supports reply_to_id for discussion threading.", + "add_labels": " CONSTRAINTS: Maximum 3 label(s) can be added. Only these labels are allowed: [\"p/0\"]. Target: *.", "create_pull_request": " CONSTRAINTS: Maximum 3 pull request(s) can be created. Title will be prefixed with \"[ci-fix] \". Labels [\"agentic-workflows\"] will be automatically added. Only these labels are allowed: [\"agentic-workflows\"]. PRs will be created as drafts.", + "mark_pull_request_as_ready_for_review": " CONSTRAINTS: Maximum 3 pull request(s) can be marked as ready for review.", "push_to_pull_request_branch": " CONSTRAINTS: Maximum 3 push(es) can be made. The target pull request title must start with \"[ci-fix] \".", "update_pull_request": " CONSTRAINTS: Maximum 3 pull request(s) can be updated. Target: *." }, @@ -594,6 +598,25 @@ jobs: } } }, + "add_labels": { + "defaultMax": 5, + "fields": { + "item_number": { + "issueNumberOrTemporaryId": true + }, + "labels": { + "required": true, + "type": "array", + "itemType": "string", + "itemSanitize": true, + "itemMaxLength": 128 + }, + "repo": { + "type": "string", + "maxLength": 256 + } + } + }, "create_pull_request": { "defaultMax": 1, "fields": { @@ -635,6 +658,24 @@ jobs: } } }, + "mark_pull_request_as_ready_for_review": { + "defaultMax": 1, + "fields": { + "pull_request_number": { + "issueOrPRNumber": true + }, + "reason": { + "required": true, + "type": "string", + "sanitize": true, + "maxLength": 65000 + }, + "repo": { + "type": "string", + "maxLength": 256 + } + } + }, "missing_data": { "defaultMax": 20, "fields": { @@ -1494,7 +1535,7 @@ jobs: uses: actions/github-script@3a2844b7e9c422d3c10d287c895573f7108da1b3 # v9.0.0 env: WORKFLOW_NAME: "CI Failure Fixer (main)" - WORKFLOW_DESCRIPTION: "Periodic pass over open ci-scan tracking issues filed by the main-branch CI\nfailure scanner (.github/workflows/ci-status-main.md). This workflow targets\nthe `main` branch EXCLUSIVELY: it processes only issues labelled ci-scan and\nopens every PR against main. (The net11.0 branch is handled by the parallel\n.github/workflows/ci-status-fix-net11.md — the two are split because gh-aw can\nonly transport a fix relative to ONE static base branch per workflow, and the\nmain↔net11.0 divergence exceeds gh-aw's 10 MB transport-patch cap.) The fixer\nopens ONE draft [ci-fix] PR per actionable issue against main, then WATCHES\nthat PR's own CI on later runs: when the fix's CI comes back red and the red is\ncaused by the fix itself, it pushes a fresh follow-up fix onto the SAME PR\nbranch (never a second PR) — up to 10 attempts — then stops and defers to\nhumans (the open tracking issue is the hand-off surface; a dedicated\n[ci-fix][needs-human] PR is planned but currently deferred — see Step 6). CI on\nthe PR is kicked by a human `/azp run` for now; the loop watches, classifies,\nand re-fixes autonomously between kicks.\nNever mutes tests, but\nde-flakes genuinely flaky ones (deterministic synchronization, no retries /\ntimeout bumps). Always skips visual-regression / screenshot issues." + WORKFLOW_DESCRIPTION: "Periodic pass over open ci-scan tracking issues filed by the main-branch CI\nfailure scanner (.github/workflows/ci-status-main.md). This workflow targets\nthe `main` branch EXCLUSIVELY: it processes only issues labelled ci-scan and\nopens every PR against main. (The net11.0 branch is handled by the parallel\n.github/workflows/ci-status-fix-net11.md — the two are split because gh-aw can\nonly transport a fix relative to ONE static base branch per workflow, and the\nmain↔net11.0 divergence exceeds gh-aw's 10 MB transport-patch cap.) The fixer\nopens ONE draft [ci-fix] PR per actionable issue against main, then WATCHES\nthat PR's own CI on later runs: when the fix's CI comes back red and the red is\ncaused by the fix itself, it pushes a fresh follow-up fix onto the SAME PR\nbranch (never a second PR) — up to 10 attempts — then stops and defers to\nhumans (the open tracking issue is the hand-off surface; a dedicated\n[ci-fix][needs-human] PR is planned but currently deferred — see Step 6). CI on\nthe PR is kicked by a human `/azp run` for now; the loop watches, classifies,\nand re-fixes autonomously between kicks. When the SPECIFIC test a PR fixed is\nconfirmed green in that PR's own CI, the loop marks the draft PR ready for review\n(a state transition only — it never approves or merges).\nNever mutes tests, but\nde-flakes genuinely flaky ones (deterministic synchronization, no retries /\ntimeout bumps). Always skips visual-regression / screenshot issues." HAS_PATCH: ${{ needs.agent.outputs.has_patch }} with: script: | @@ -1910,7 +1951,7 @@ jobs: GH_AW_ALLOWED_DOMAINS: "*.blob.core.windows.net,*.githubusercontent.com,api.business.githubcopilot.com,api.enterprise.githubcopilot.com,api.github.com,api.githubcopilot.com,api.individual.githubcopilot.com,api.snapcraft.io,archive.ubuntu.com,azure.archive.ubuntu.com,codeload.github.com,crl.geotrust.com,crl.globalsign.com,crl.identrust.com,crl.sectigo.com,crl.thawte.com,crl.usertrust.com,crl.verisign.com,crl3.digicert.com,crl4.digicert.com,crls.ssl.com,dev.azure.com,docs.github.com,github-cloud.githubusercontent.com,github-cloud.s3.amazonaws.com,github.blog,github.com,github.githubassets.com,helix.dot.net,host.docker.internal,json-schema.org,json.schemastore.org,keyserver.ubuntu.com,lfs.github.com,objects.githubusercontent.com,ocsp.digicert.com,ocsp.geotrust.com,ocsp.globalsign.com,ocsp.identrust.com,ocsp.sectigo.com,ocsp.ssl.com,ocsp.thawte.com,ocsp.usertrust.com,ocsp.verisign.com,packagecloud.io,packages.cloud.google.com,packages.microsoft.com,patch-diff.githubusercontent.com,ppa.launchpad.net,raw.githubusercontent.com,registry.npmjs.org,s.symcb.com,s.symcd.com,security.ubuntu.com,telemetry.enterprise.githubcopilot.com,ts-crl.ws.symantec.com,ts-ocsp.ws.symantec.com,www.googleapis.com" GITHUB_SERVER_URL: ${{ github.server_url }} GITHUB_API_URL: ${{ github.api_url }} - GH_AW_SAFE_OUTPUTS_HANDLER_CONFIG: "{\"add_comment\":{\"discussions\":false,\"max\":3,\"required_labels\":[\"agentic-workflows\"],\"required_title_prefix\":\"[ci-fix] \",\"target\":\"*\"},\"create_pull_request\":{\"allowed_base_branches\":[\"main\"],\"allowed_branches\":[\"ci-fix/**\"],\"allowed_files\":[\"src/Core/**\",\"src/Controls/**\",\"src/Essentials/**\",\"src/BlazorWebView/**\",\"src/TestUtils/**\",\"src/Templates/**\",\"**/PublicAPI.Unshipped.txt\"],\"base_branch\":\"main\",\"draft\":true,\"labels\":[\"agentic-workflows\"],\"max\":3,\"max_patch_files\":100,\"max_patch_size\":4096,\"protect_top_level_dot_folders\":true,\"protected_files\":[\"package.json\",\"bun.lockb\",\"bunfig.toml\",\"deno.json\",\"deno.jsonc\",\"deno.lock\",\"global.json\",\"NuGet.Config\",\"Directory.Packages.props\",\"mix.exs\",\"mix.lock\",\"go.mod\",\"go.sum\",\"stack.yaml\",\"stack.yaml.lock\",\"pom.xml\",\"build.gradle\",\"build.gradle.kts\",\"settings.gradle\",\"settings.gradle.kts\",\"gradle.properties\",\"package-lock.json\",\"yarn.lock\",\"pnpm-lock.yaml\",\"npm-shrinkwrap.json\",\"requirements.txt\",\"Pipfile\",\"Pipfile.lock\",\"pyproject.toml\",\"setup.py\",\"setup.cfg\",\"Gemfile\",\"Gemfile.lock\",\"uv.lock\",\"CODEOWNERS\",\"DESIGN.md\",\"README.md\",\"CONTRIBUTING.md\",\"CHANGELOG.md\",\"SECURITY.md\",\"CODE_OF_CONDUCT.md\",\"AGENTS.md\",\"CLAUDE.md\",\"GEMINI.md\"],\"protected_files_policy\":\"request_review\",\"title_prefix\":\"[ci-fix] \"},\"create_report_incomplete_issue\":{},\"missing_data\":{},\"missing_tool\":{},\"noop\":{\"max\":1,\"report-as-issue\":\"true\"},\"push_to_pull_request_branch\":{\"allowed_files\":[\"src/Core/**\",\"src/Controls/**\",\"src/Essentials/**\",\"src/BlazorWebView/**\",\"src/TestUtils/**\",\"src/Templates/**\",\"**/PublicAPI.Unshipped.txt\"],\"if_no_changes\":\"warn\",\"max\":3,\"max_patch_size\":4096,\"protect_top_level_dot_folders\":true,\"protected_files\":[\"package.json\",\"bun.lockb\",\"bunfig.toml\",\"deno.json\",\"deno.jsonc\",\"deno.lock\",\"global.json\",\"NuGet.Config\",\"Directory.Packages.props\",\"mix.exs\",\"mix.lock\",\"go.mod\",\"go.sum\",\"stack.yaml\",\"stack.yaml.lock\",\"pom.xml\",\"build.gradle\",\"build.gradle.kts\",\"settings.gradle\",\"settings.gradle.kts\",\"gradle.properties\",\"package-lock.json\",\"yarn.lock\",\"pnpm-lock.yaml\",\"npm-shrinkwrap.json\",\"requirements.txt\",\"Pipfile\",\"Pipfile.lock\",\"pyproject.toml\",\"setup.py\",\"setup.cfg\",\"Gemfile\",\"Gemfile.lock\",\"uv.lock\",\"CODEOWNERS\",\"DESIGN.md\",\"README.md\",\"CONTRIBUTING.md\",\"CHANGELOG.md\",\"SECURITY.md\",\"CODE_OF_CONDUCT.md\",\"AGENTS.md\",\"CLAUDE.md\",\"GEMINI.md\"],\"required_labels\":[\"agentic-workflows\"],\"target\":\"*\",\"title_prefix\":\"[ci-fix] \"},\"report_incomplete\":{},\"update_pull_request\":{\"allow_body\":true,\"allow_title\":false,\"max\":3,\"target\":\"*\",\"update_branch\":false}}" + GH_AW_SAFE_OUTPUTS_HANDLER_CONFIG: "{\"add_comment\":{\"discussions\":false,\"max\":6,\"required_labels\":[\"agentic-workflows\"],\"required_title_prefix\":\"[ci-fix] \",\"target\":\"*\"},\"add_labels\":{\"allowed\":[\"p/0\"],\"max\":3,\"required_labels\":[\"agentic-workflows\"],\"required_title_prefix\":\"[ci-fix] \",\"target\":\"*\"},\"create_pull_request\":{\"allowed_base_branches\":[\"main\"],\"allowed_branches\":[\"ci-fix/**\"],\"allowed_files\":[\"src/Core/**\",\"src/Controls/**\",\"src/Essentials/**\",\"src/BlazorWebView/**\",\"src/TestUtils/**\",\"src/Templates/**\",\"**/PublicAPI.Unshipped.txt\"],\"base_branch\":\"main\",\"draft\":true,\"labels\":[\"agentic-workflows\"],\"max\":3,\"max_patch_files\":100,\"max_patch_size\":4096,\"protect_top_level_dot_folders\":true,\"protected_files\":[\"package.json\",\"bun.lockb\",\"bunfig.toml\",\"deno.json\",\"deno.jsonc\",\"deno.lock\",\"global.json\",\"NuGet.Config\",\"Directory.Packages.props\",\"mix.exs\",\"mix.lock\",\"go.mod\",\"go.sum\",\"stack.yaml\",\"stack.yaml.lock\",\"pom.xml\",\"build.gradle\",\"build.gradle.kts\",\"settings.gradle\",\"settings.gradle.kts\",\"gradle.properties\",\"package-lock.json\",\"yarn.lock\",\"pnpm-lock.yaml\",\"npm-shrinkwrap.json\",\"requirements.txt\",\"Pipfile\",\"Pipfile.lock\",\"pyproject.toml\",\"setup.py\",\"setup.cfg\",\"Gemfile\",\"Gemfile.lock\",\"uv.lock\",\"CODEOWNERS\",\"DESIGN.md\",\"README.md\",\"CONTRIBUTING.md\",\"CHANGELOG.md\",\"SECURITY.md\",\"CODE_OF_CONDUCT.md\",\"AGENTS.md\",\"CLAUDE.md\",\"GEMINI.md\"],\"protected_files_policy\":\"request_review\",\"title_prefix\":\"[ci-fix] \"},\"create_report_incomplete_issue\":{},\"mark_pull_request_as_ready_for_review\":{\"max\":3,\"required_labels\":[\"agentic-workflows\"],\"required_title_prefix\":\"[ci-fix] \",\"target\":\"*\"},\"missing_data\":{},\"missing_tool\":{},\"noop\":{\"max\":1,\"report-as-issue\":\"true\"},\"push_to_pull_request_branch\":{\"allowed_files\":[\"src/Core/**\",\"src/Controls/**\",\"src/Essentials/**\",\"src/BlazorWebView/**\",\"src/TestUtils/**\",\"src/Templates/**\",\"**/PublicAPI.Unshipped.txt\"],\"if_no_changes\":\"warn\",\"max\":3,\"max_patch_size\":4096,\"protect_top_level_dot_folders\":true,\"protected_files\":[\"package.json\",\"bun.lockb\",\"bunfig.toml\",\"deno.json\",\"deno.jsonc\",\"deno.lock\",\"global.json\",\"NuGet.Config\",\"Directory.Packages.props\",\"mix.exs\",\"mix.lock\",\"go.mod\",\"go.sum\",\"stack.yaml\",\"stack.yaml.lock\",\"pom.xml\",\"build.gradle\",\"build.gradle.kts\",\"settings.gradle\",\"settings.gradle.kts\",\"gradle.properties\",\"package-lock.json\",\"yarn.lock\",\"pnpm-lock.yaml\",\"npm-shrinkwrap.json\",\"requirements.txt\",\"Pipfile\",\"Pipfile.lock\",\"pyproject.toml\",\"setup.py\",\"setup.cfg\",\"Gemfile\",\"Gemfile.lock\",\"uv.lock\",\"CODEOWNERS\",\"DESIGN.md\",\"README.md\",\"CONTRIBUTING.md\",\"CHANGELOG.md\",\"SECURITY.md\",\"CODE_OF_CONDUCT.md\",\"AGENTS.md\",\"CLAUDE.md\",\"GEMINI.md\"],\"required_labels\":[\"agentic-workflows\"],\"target\":\"*\",\"title_prefix\":\"[ci-fix] \"},\"report_incomplete\":{},\"update_pull_request\":{\"allow_body\":true,\"allow_title\":false,\"max\":3,\"target\":\"*\",\"update_branch\":false}}" GH_AW_CI_TRIGGER_TOKEN: ${{ secrets.GH_AW_CI_TRIGGER_TOKEN }} with: github-token: ${{ secrets.GH_AW_GITHUB_TOKEN || secrets.GITHUB_TOKEN }} diff --git a/.github/workflows/ci-status-fix.md b/.github/workflows/ci-status-fix.md index d3227e25170c..9c2f46054029 100644 --- a/.github/workflows/ci-status-fix.md +++ b/.github/workflows/ci-status-fix.md @@ -15,7 +15,9 @@ description: | humans (the open tracking issue is the hand-off surface; a dedicated [ci-fix][needs-human] PR is planned but currently deferred — see Step 6). CI on the PR is kicked by a human `/azp run` for now; the loop watches, classifies, - and re-fixes autonomously between kicks. + and re-fixes autonomously between kicks. When the SPECIFIC test a PR fixed is + confirmed green in that PR's own CI, the loop marks the draft PR ready for review + (a state transition only — it never approves or merges). Never mutes tests, but de-flakes genuinely flaky ones (deterministic synchronization, no retries / timeout bumps). Always skips visual-regression / screenshot issues. @@ -239,8 +241,15 @@ safe-outputs: # PER-RUN total (not per-PR): a sweep may surface several green PRs and/or # annotate several flaky reds in one run. At max:1 all but the first comment # were silently dropped, starving the workflow's #1 value (surfacing green PRs - # for review). Raised to 3 to match the per-run throughput of the other outputs. - max: 3 + # for review). Sized to 6 (not 3) because a single green DRAFT PR that gets + # flipped ready in one sweep spends TWO comment slots — the Step 3 ✅ surface + # comment (which still names the /azp-gated legs a human must kick, so it is NOT + # redundant with 🎯) AND the Step 3.6 T3 🎯 readiness comment. At max:3 the shared + # bucket drained after ~1 draft flip and T3's atomicity pre-check then deferred + # every further mark-ready even while the mark-ready/add_labels buckets (also 3) + # sat idle; 6 lets ~3 draft flips (2 comments each) land per sweep, so the + # mark-ready:3 / add_labels:3 caps are actually reachable. + max: 6 target: "*" # Hard constraint (defense-in-depth): only comment on THIS workflow's own # ci-fix PRs — [ci-fix] title prefix AND agentic-workflows label. Mirrors the @@ -261,13 +270,51 @@ safe-outputs: # never the title, so disable title rewrites — this compiles to allow_title:false # and removes any ability to retitle an arbitrary PR. title: false - # NOTE: gh-aw v0.79.8 does NOT emit required-title-prefix/required-labels into + # NOTE: gh-aw v0.80.9 (the pinned compiler) does NOT emit required-title-prefix/required-labels into # the compiled config for update-pull-request (verified against the lock — it # silently drops them, unlike add-comment / push-to-pull-request-branch which # honor them). So which-PR scoping here relies on prompt Hard-Rule 6 + # min-integrity:approved; the residual is body-marker edits only (no code, no # merge, no close), capped at max:3. Revisit required-* if a future gh-aw # compiler emits them for this output. + mark-pull-request-as-ready-for-review: + # Flip a validated draft [ci-fix] PR to ready-for-review once the SPECIFIC test + # this PR was opened to fix is confirmed green in the PR's OWN CI (Step 3.6) — + # even when unrelated legs are red. The workflow is schedule/dispatch-triggered + # (no triggering-PR context), so target "*" lets the agent name the PR number it + # validated. This is a state transition ONLY: it never approves and never merges — + # a human still reviews and merges. + # PER-RUN total (not per-PR): a sweep may validate several PRs' target tests green. + max: 3 + target: "*" + # Hard constraint (defense-in-depth): only ever un-draft THIS workflow's own + # ci-fix PRs — [ci-fix] title prefix AND agentic-workflows label — so a confused + # or prompt-injected agent cannot mark an arbitrary PR ready. (If the v0.80.9 + # compiler silently drops these — as documented for update-pull-request above — + # the Step 3.6 preconditions + min-integrity:approved are the compensating scope + # controls; verify against the lock after compiling.) + required-title-prefix: "[ci-fix] " + required-labels: [agentic-workflows] + add-labels: + # Apply the p/0 priority label to a [ci-fix] PR at the exact moment the loop flips it + # from draft to ready-for-review (Step 3.6 T3) — so a validated, review-ready fix lands + # in the team's p/0 triage queue instead of sitting unseen in the draft backlog. Paired + # 1:1 with mark-pull-request-as-ready-for-review; target "*" lets the agent name the PR + # it just validated. + # allowed = HARD allowlist: the agent may ONLY ever add p/0, nothing else. This caps the + # blast radius of a confused/prompt-injected agent to exactly one benign priority label — + # it can never apply a downstream-triggering or destructive label. + allowed: [p/0] + # PER-RUN total (not per-PR): a sweep may mark several PRs ready in one run; each adds + # one label, so this matches the mark-ready per-run cap (3). + max: 3 + target: "*" + # Hard constraint (defense-in-depth): only ever label THIS workflow's own ci-fix PRs — + # [ci-fix] title prefix AND agentic-workflows label. (If the v0.80.9 compiler drops these + # — as documented for update-pull-request above — the Step 3.6 preconditions + + # min-integrity:approved + the allowed:[p/0] allowlist are the compensating controls.) + required-title-prefix: "[ci-fix] " + required-labels: [agentic-workflows] timeout-minutes: 90 @@ -339,33 +386,48 @@ through `safe-outputs`. 5. **One issue = one outcome per run.** Exactly one of: respond to a maintainer's change-request on the open PR (Track C, Step 3.5.R); open the first fix/help/ de-flake PR; advance an existing PR by one attempt (push a follow-up fix); - surface a validated-green PR for review; annotate an unrelated-flake red; wait + surface a validated-green PR for review; mark a target-validated draft PR + ready-for-review (Step 3.6 T3 — the terminal outcome that supersedes the same + run's surface/annotate precursor line), or record its `already-ready` + steady-state no-op; annotate an unrelated-flake red; wait (CI not yet settled); or a recorded skip (the dedicated needs-human PR is deferred — the attempt cap records a skip instead; see Step 6). Always prefer advancing or opening a PR over a skip when a non-mute diff is producible. 6. **All writes via `safe-outputs`.** Allowed outputs: `create_pull_request` (first attempt only), `push_to_pull_request_branch` (advance an existing PR by one attempt), `update_pull_request` (bump the attempt marker / refresh the - prior-attempts table), and `add_comment` (progress notes on the `[ci-fix]` PR - ONLY). NEVER comment on the tracking issue (issues are locked by + prior-attempts table), `add_comment` (progress notes on the `[ci-fix]` PR + ONLY), `mark_pull_request_as_ready_for_review` (flip a target-validated draft + `[ci-fix]` PR from draft to ready — Step 3.6 T3 only), and `add_labels` (add + ONLY the `p/0` label, ONLY on that same draft→ready transition — Step 3.6 T3). + NEVER comment on the tracking issue (issues are locked by `.github/workflows/ci-scan-lock-issues.yml`) — `add_comment` targets the PR. No `gh pr create`, no manual `git push`. **Defense-in-depth:** `add_comment` and `update_pull_request` use `target: "*"` (the agent supplies the PR number). - `add_comment` is now config-locked to `required-title-prefix: "[ci-fix] "` + + `add_comment`, `mark_pull_request_as_ready_for_review`, and `add_labels` are + config-locked to `required-title-prefix: "[ci-fix] "` + `required-labels: [agentic-workflows]` (hard-enforced by the handler, same as - `push_to_pull_request_branch`), so a comment can only ever land on THIS - workflow's own PRs. `update_pull_request` CANNOT be config-locked to a - title/label in gh-aw v0.79.8 (the compiler silently drops `required-*` for that - output — verified against the lock), so it keeps `title: false` (no retitles; - body-marker edits only) plus this prompt-level guard. Before emitting either, - VERIFY the target PR carries BOTH the `[ci-fix]` title prefix AND the - `agentic-workflows` label; never comment on or edit the body of any PR that - lacks both. + `push_to_pull_request_branch`), and `add_labels` additionally has an + `allowed: [p/0]` allowlist so it can ONLY ever add `p/0` — so these can only + ever land on THIS workflow's own PRs. `update_pull_request` CANNOT be + config-locked to a title/label in gh-aw v0.80.9 (the compiler silently drops + `required-*` for that output — verified against the lock), so it keeps + `title: false` (no retitles; body-marker edits only) plus this prompt-level + guard. Before emitting ANY of these, VERIFY the target PR carries BOTH the + `[ci-fix]` title prefix AND the `agentic-workflows` label; never comment on, + edit, un-draft, or label any PR that lacks both. 7. **Per-run safe-output caps (per RUN, not per PR).** Each output type is capped per run: `create_pull_request` 3, `push_to_pull_request_branch` 3, - `add_comment` 3, `update_pull_request` 3. Note an ADVANCE spends one push **and** - one update **and** one comment, so ≤ 3 advances/run; surfacing a green or - annotating a flake spends one comment. When a bucket is exhausted, do NOT keep + `add_comment` 6, `update_pull_request` 3, `mark_pull_request_as_ready_for_review` + 3, `add_labels` 3 (counts LABELS, not calls). Note an ADVANCE spends one push + **and** one update **and** one comment, so ≤ 3 advances/run; surfacing a green or + annotating a flake spends one comment; a **mark-ready (Step 3.6 T3)** of a + still-draft PR spends TWO comments (the Step 3 ✅ surface **and** the T3 🎯 + readiness note) **and** one mark-ready **and** one label as an ALL-OR-NOTHING set + (T3 pre-checks those buckets and defers the whole PR if any is exhausted — never + mark a PR ready without its 🎯 audit comment). `add_comment` is therefore sized 6 + (not 3) so ~3 draft flips, at 2 comments each, can land in one sweep instead of the + shared comment bucket starving the otherwise-idle mark-ready/add_labels buckets. When a bucket is exhausted, do NOT keep emitting (extras are silently dropped) — record `skipped: per-run cap reached; deferring PR # to next cycle` for each remaining PR so the drop is a deliberate, logged decision. The continuous loop picks the deferred PRs up next @@ -409,8 +471,9 @@ For every open tracking issue in scope, converge on exactly one outcome: | Open first help-wanted draft `[ci-fix]` PR | No open PR yet; a plausible candidate exists but cannot be runner-validated (device/UI tests) (attempt 1) | | Open first de-flake draft `[ci-fix]` PR | No open PR yet; failure is intermittent (green-on-retry) from a genuine test-quality defect; a deterministic-synchronization fix is producible (attempt 1, Step 4.7 bucket b) | | Advance an existing PR (push attempt N+1) | Open PR's own CI settled red *because of the fix itself*, no human engaged, marker < 10 — push a NEW distinct fix onto the same branch (Step 3.5 → 5.6) | -| Surface a validated-green PR | Open PR's own CI settled green — comment "validated, attempt N/10; ready for review" and do NOT advance (Step 3.5) | -| Annotate an unrelated-flake red | Open PR is red only on baseline-flake / unrelated legs — comment which leg needs a re-run; do NOT burn an attempt (Step 3.5) | +| Surface a validated-green PR | Open PR's own CI settled green — comment "validated, attempt N/10; ready for review" and do NOT advance (Step 3.5), then mark the draft PR ready for review once the fixed test is confirmed green (Step 3.6) | +| Annotate an unrelated-flake red | Open PR is red only on baseline-flake / unrelated legs — comment which leg needs a re-run; do NOT burn an attempt (Step 3.5); if the SPECIFIC fixed test is confirmed green on that build, still mark the draft PR ready for review (Step 3.6) | +| Mark a validated PR ready for review | The specific test this PR fixed is confirmed green in the PR's own CI (even if unrelated legs are red) — comment the target-test result and transition the draft PR to ready for review; never approve or merge (Step 3.6) | | Wait | Open PR's CI is pending / not yet settled on the current head SHA — do nothing this cycle (Step 3.5) | | Hand-off skip (attempt cap) | Marker == 10 and the signature still reproduces — stop and defer to humans; the dedicated `[ci-fix][needs-human]` PR is planned but currently deferred (Step 6) | | Recorded skip | Visual-regression, human engaged, already-handled, fixed-in-latest-build, infra-flake, out-of-bounds, only-mute-available, or no novel approach producible | @@ -685,14 +748,34 @@ Run these gates in order — the FIRST that fires decides this cycle's outcome: 1. **Human engaged.** If `C.humanEngaged` → `skipped: human engaged on PR #; deferring` and stop. Never fight a human reviewer — the intentional hand-off boundary still holds the moment a person touches the PR. -2. **CI not settled / unknown / incomplete prefetch.** If `C.checksSettled == false` - OR `C.overallConclusion` is `pending`, `neutral`, or `unknown`, OR - `C.dataComplete == false` → the fix's CI has not finished on the current head SHA - (`C.headSha`), or the prefetch could not fully read this PR (a partial read can - understate `humanEngaged` / `attempt`). `skipped: PR # CI pending / prefetch - incomplete on ; waiting` and stop. *(Round 1: this is where a - maintainer `/azp run` is awaited — the `/azp`-gated uitests/devicetests legs will - not have run until a human kicks them.)* +2. **CI not settled — but validate the target test first (target-focused readiness).** + If `C.checksSettled == false` OR `C.overallConclusion` is `pending`, `neutral`, or + `unknown`, OR `C.dataComplete == false` → the *overall* build has not finished on the + current head SHA (`C.headSha`), or the prefetch could not fully read this PR (a partial + read can understate `humanEngaged` / `attempt`). Unrelated legs still draining must NOT + keep an already-proven fix parked as a draft, so before waiting, try the target-focused + fast-path: + - **2a — Target-green fast-path (draft `[ci-fix]` PRs only).** If `C.dataComplete == + true` AND the Step 3.6 preconditions hold (draft PR, `[ci-fix] ` title prefix AND the + `agentic-workflows` label), run Step 3.6 **T1–T2** against `C.headSha` now: identify + the target test(s) and read the PR's OWN build **timeline / per-leg status** (anonymous + `_apis/build` — never `_apis/test`, per Hard-Rule 8). If EVERY target test is + **VALIDATED-GREEN** (per T2's all-platform definition below — the target's category leg + is `succeeded` on every platform that runs it, failed on none, and NO target platform + family the pipeline covers is still pending/unconcluded, per T2's "no platform left + unverified" rule; a platform that simply has no leg for the target's category — the test + doesn't run there — is fine, not a gap) AND every leg in + `C.failedLegs` that has ALREADY concluded classifies as **unrelated flake** (Step 4 / + Step 4.7 method — the fix is implicated in NO completed red), then the fix is proven + regardless of unrelated *pending* legs → run **Step 3.6 T3** (mark ready + 🎯 comment) + and stop. This early-out NEVER advances an attempt and NEVER acts on a red that could + be the fix's fault. + - **2b — Otherwise WAIT.** If the target test has not yet executed (pending/absent on + its leg), or a completed red is (or may be) caused by the fix, or `C.dataComplete == + false`, or this PR has no identifiable target test → `skipped: PR # CI pending / + target not yet validated on ; waiting` and stop. *(Round 1: this is where a + maintainer `/azp run` is awaited — the `/azp`-gated uitests/devicetests legs will not + have run until a human kicks them.)* 3. **Green → surface for review.** If `C.overallConclusion == "success"`: the checks that RAN are green. **Primary-gate check first:** the deterministic `success` verdict only certifies "at least one green check, nothing failing or @@ -704,18 +787,31 @@ Run these gates in order — the FIRST that fires decides this cycle's outcome: # primary CI gate (maui-pr) not green on ; waiting` and stop — do NOT surface. (The `/azp`-gated `maui-pr-uitests` (def 313) / `maui-pr-devicetests` (def 314) legs MAY still be un-run — that is expected and is named below, not a - reason to withhold the surface.) **Idempotency:** scan the PR's existing comments - for a prior bot `✅ … validated … on ` note for THIS head SHA — if one - already exists, record `already-surfaced PR # (head )` and stop - WITHOUT re-commenting (re-surfacing the same green every tick is noise and burns - the per-run comment budget other PRs need). Otherwise resolve the attempt number from - `C.effectiveAttempt` (the authoritative max(marker, bot-commit) counter; omit the - number only if it is somehow indeterminate). **Dry-run gate (Step 0):** if `dry_run == "true"`, do NOT emit any comment — instead print the intended `✅` validated-green body to the run log, tally `dry-run: would-surface-green PR #`, and stop. Otherwise `add_comment` on PR #: `✅ Attempt /10 validated - — the fix's CI is green on . Ready for human review.` - Name any `/azp`-gated legs (uitests def 313 / devicetests def 314) that have not - run and still need a maintainer `/azp run`. Do NOT advance. Record - `surfaced-green PR # (attempt /10)` and stop. *(This directly - attacks the real bottleneck — no reviews — so it is the highest-value outcome.)* + reason to withhold the surface.) **Comment idempotency + dry-run suppress the ✅ + comment ONLY — neither skips Step 3.6.** Scan the PR's existing comments for a prior + bot `✅ … validated … on ` note for THIS head SHA; if one exists, set + `SKIP_SURFACE_COMMENT = true`. Resolve the attempt number from `C.effectiveAttempt` + (the authoritative max(marker, bot-commit) counter; omit the number only if it is + somehow indeterminate). Then post the surface comment UNLESS suppressed: + - if `dry_run == "true"`: do NOT emit — print the intended `✅` validated-green body to + the run log and tally `dry-run: would-surface-green PR #`; + - else if `SKIP_SURFACE_COMMENT`: do NOT re-post — record `already-surfaced PR # + (head )` (re-surfacing the same green every tick is noise and burns the + per-run comment budget other PRs need); + - else `add_comment` on PR #: `✅ Attempt /10 validated — the fix's CI is + green on .` naming any `/azp`-gated legs (uitests def 313 / devicetests def + 314) that have not run and still need a maintainer `/azp run`; record `surfaced-green + PR # (attempt /10)`. (Do NOT assert "ready for human review" on a PR that + is still a draft — Step 3.6, not this comment, owns the draft→ready flip and posts its + own 🎯 announcement when it fires.) + Do NOT advance. Then — in ALL of the above cases — run **Step 3.6** (target-test + readiness gate) for this PR before stopping: its `/azp`-gated target legs may conclude + green on this SAME head SHA on a LATER sweep (an `/azp run` adds no commit), and Step + 3.6 is the ONLY place the draft→ready flip happens, so it MUST re-evaluate every sweep — + never `stop` here before it. *(This directly attacks the real bottleneck — no reviews — + so it is the highest-value outcome.)* 4. **Red → classify caused-by-fix vs unrelated-flake.** If `C.overallConclusion == "failure"`, analyze the PR's OWN failing build (NOT `main`): find the AzDO `maui-pr` build for this PR (filter builds by `branchName=refs/pull//merge`, @@ -723,18 +819,26 @@ Run these gates in order — the FIRST that fires decides this cycle's outcome: method and Step 4.7 flake buckets to `C.failedLegs`. - **Unrelated flake only** (every failed leg is known-flaky / infra / a pre-existing baseline red NOT introduced by the fix): do NOT burn an attempt. - **Idempotency first:** scan the PR's existing comments for a prior bot - `♻️ … unrelated flake … on ` note for THIS head SHA — if one already - exists, record `already-annotated-flake PR # (head )` and stop - WITHOUT re-commenting (re-annotating the same flake every 12h sweep is noise and - burns the per-run comment budget other PRs need). Otherwise resolve the attempt - number from `C.effectiveAttempt` (the authoritative max(marker, bot-commit) - counter), omitting the `Attempt /10:` prefix only if it is somehow - indeterminate. **Dry-run gate (Step 0):** if `dry_run == "true"`, do NOT emit any comment — instead print the intended `♻️` unrelated-flake body to the run log, tally `dry-run: would-annotate-flake PR #`, and stop. Otherwise `add_comment` on PR #: `♻️ Attempt /10: red is - unrelated flake on leg(s) () on ; the fix itself is not - implicated. A maintainer re-run (/azp run ) should clear it.` Record - `annotated-flake PR # (head )` and stop. *(Round 1: human re-runs; - Round 2: auto re-trigger.)* + **Comment idempotency + dry-run suppress the ♻️ comment ONLY — neither skips Step + 3.6.** Scan the PR's existing comments for a prior bot `♻️ … unrelated flake … on + ` note for THIS head SHA; if one exists, set `SKIP_FLAKE_COMMENT = true`. + Resolve the attempt number from `C.effectiveAttempt` (the authoritative max(marker, + bot-commit) counter), omitting the `Attempt /10:` prefix only if it is + somehow indeterminate. Then post the flake note UNLESS suppressed: + - if `dry_run == "true"`: do NOT emit — print the intended `♻️` unrelated-flake body + to the run log and tally `dry-run: would-annotate-flake PR #`; + - else if `SKIP_FLAKE_COMMENT`: do NOT re-post — record `already-annotated-flake PR + # (head )` (re-annotating the same flake every 12h sweep is noise and + burns the per-run comment budget other PRs need); + - else `add_comment` on PR #: `♻️ Attempt /10: red is unrelated flake on + leg(s) () on ; the fix itself is not implicated. A + maintainer re-run (/azp run ) should clear it.` record `annotated-flake + PR # (head )`. + Then — in ALL of the above cases — run **Step 3.6** (target-test readiness gate) for + this PR before stopping: its `/azp`-gated target legs may conclude green on this SAME + head SHA on a LATER sweep, and Step 3.6 is the ONLY place the draft→ready flip + happens, so it MUST re-evaluate every sweep — never `stop` here before it. *(Round 1: + human re-runs; Round 2: auto re-trigger.)* - **Caused by the fix** (a failed leg still matches the original target signature, or the fix introduced a NEW failure): advance an attempt. - **Attempt count.** `attempt = C.effectiveAttempt` — the authoritative @@ -752,6 +856,188 @@ Run these gates in order — the FIRST that fires decides this cycle's outcome: from every prior commit on this branch**, emitted via the Step 5.6 ADVANCE path (push onto the same PR). +#### Step 3.6 — Target-test verification & mark-ready gate + +Reached from the Step 3 **green-surface** branch, the Step 4 **unrelated-flake** branch +(AFTER that branch has posted its comment), and the Step 3.5 **gate-2 target-focused +fast-path** (2a — while unrelated legs are still pending). Purpose: confirm the SPECIFIC +test(s) this PR was opened to fix now PASS in the PR's own CI, and — only when they do +— transition the draft `[ci-fix]` PR to **ready for review** so a maintainer sees a +validated fix instead of a draft. This is the ONLY place the loop flips draft→ready; it +is a state transition, **never** an approval or a merge (a human still reviews and +merges). Overall red on *unrelated* legs must NOT gate readiness — we validate the fix, +not the base branch's flakiness. + +**Preconditions** (ALL must hold; otherwise record `skipped: readiness N/A PR # +()` and stop this gate): +- `C.isDraft == true` — if the PR is already ready-for-review, the draft→ready + transition is already done; do NOT re-mark **and do NOT re-apply `p/0`**. `p/0` is applied + exactly ONCE, atomically with the draft→ready flip (T3 — all-or-nothing with the 🎯 audit + comment + mark-ready), so an already-ready loop PR that is MISSING `p/0` has almost + certainly had it **removed by a maintainer** de-prioritizing the PR — a pure triage action + the human-engagement guard (which inspects comments/reviews/commits, NOT label changes) + cannot see. Re-adding `p/0` every sweep would fight that maintainer indefinitely, violating + the loop's "never override a human" contract. So the loop does NOT reconcile the label: + record `already-ready PR #` and stop this gate. (Trade-off: on the rare occasion a + transient API error drops `p/0` at flip time *after* the T3 atomic pre-check passed, it is + not auto-re-added — but the PR is still ready-for-review with its 🎯 audit comment, and + re-adding `p/0` is a trivial manual action; that is strictly preferable to steam-rolling a + maintainer's deliberate de-prioritization.) +- The PR is unmistakably THIS workflow's own: `[ci-fix] ` title prefix AND the + `agentic-workflows` label (mirrors the safe-output lock; the compensating scope + control if the v0.80.9 compiler drops the declarative required-*). +- You reached this gate from Step 3 (green), Step 4 (**unrelated-flake**), or the Step 3.5 + gate-2 **target-focused fast-path** (2a) — i.e. the fix is NOT implicated in any red that + has concluded. If Step 4 classified a red as **caused by the fix** (ADVANCE), do NOT run + this gate — advance the attempt instead. + +**T1 — Identify the target test(s).** From the `[ci-scan]` issue signature the fix +addresses (and the PR's own diff), extract the fully-qualified test method name(s) the +fix targets — e.g. `SafeAreaShouldWorkOnAllShellTabs`. For a de-flake it is the +de-flaked test; for a product fix it is the originally-failing test(s). If NO specific +test can be identified (e.g. a product build-break rather than a test failure), this is a +**build-only fix** — there is no single test to validate cross-platform, so validate at +**whole-build** granularity rather than stopping (Step 3 only *comments*; Step 3.6 is the +sole draft→ready flip, so a build-only fix is undrafted HERE or not at all). We only reach +this gate from Step 3 (green) / Step 4 (unrelated-flake) / gate-2a, so the fix is not +implicated in any concluded red; additionally require, via the T2 timeline method on +`C.headSha`, that the primary build pipeline `maui-pr` (def 302) has CONCLUDED with EVERY +one of its platform build legs `succeeded`/`completed` and NONE failed/canceled or still +pending — the build is green on **every** platform, not just the originally-broken one (the +cross-platform guard, applied to the build instead of a test). A build-only fix is undrafted +ONLY on the strength of the auto-running `maui-pr` (def 302) whole-build green, which the +loop CAN observe: if the PR's `[ci-scan]` issue signature or its own diff indicates the +ORIGINATING failure was in a `/azp`-gated pipeline (`maui-pr-uitests` def 313 or +`maui-pr-devicetests` def 314) rather than `maui-pr` (def 302), do NOT undraft on +`maui-pr`-green alone — those pipelines do not auto-run on this PR (GITHUB_TOKEN cannot +trigger them), so a green `maui-pr` build is NOT evidence the gated build break is fixed; +record `skipped: build-only fix PR # targets gated pipeline () not run — +deferring to human` and stop this gate. Otherwise, set `TARGET := "the maui-pr build +(build-only fix — no single target test)"` and proceed to **T3** to mark ready. If any `maui-pr` build leg is still unconcluded, record `skipped: build-only fix PR +# not yet whole-build green (leg(s) pending)` and stop this gate WITHOUT marking +ready (a green subset is not enough — a still-pending build leg could yet fail). + +**T2 — Verify the target test's platform legs are green (anonymous, leg-level).** Per-test +outcomes come from the AzDO **test-results** API (`_apis/test/...`), which is **NOT reachable +anonymously — Hard-Rule 8**: those endpoints 302-redirect to a sign-in page on this runner, so +a `curl` returns an HTML redirect (not JSON) and every target test would look "not executed". +Validate instead at **leg granularity** using ONLY the anonymous `_apis/build/...` timeline +(Hard-Rule 8) — a `succeeded` category leg means every test in that category **that actually +ran** passed on that platform. This is a strong bar **only if the target test genuinely +executes**: a job can go green while one specific test is skipped at runtime (`[Ignore]`, +`Assert.Ignore()`, `Assert.Inconclusive()`, a `Skip=`/conditional `[Fact]`, an `#if`-out, or +an early `return` before the asserts). So before trusting a green leg as proof the target +passed, **confirm from the PR's OWN diff that the fix does NOT skip, ignore, disable, or +short-circuit the target test** (it adds no `[Ignore]`/`[Explicit]`, `Assert.Ignore`/ +`Assert.Inconclusive`, `Skip=`, category-exclusion, `#if`-out, or early `return` around the +target). If the fix could cause the target to be runtime-skipped rather than genuinely pass, +leg-level green is NOT sufficient evidence — record `skipped: target test may be +runtime-skipped by this fix (diff adds ); leg-green insufficient, deferring to human +on PR #` and stop this gate WITHOUT marking ready. Otherwise (the target genuinely runs +and asserts, as a de-flake or product fix does) a green category leg cannot hide a +target-test failure, so treat it as authoritative. + +Map the target test to the CI leg(s) that run it. A leg name encodes platform + test category +(e.g. `Android UITests SafeAreaEdges,Shadow`, `iOS UITests SafeAreaEdges,Shadow`, `macOS +UITests SafeAreaEdges,Shadow`, `Windows UITests SafeAreaEdges,Shadow`). Determine the target +test's UI-test category — its `[Category(UITestCategories.X)]` in the test/HostApp source, +visible in the PR diff or the test file — to know which leg-name substring identifies its legs; +for a device test, the per-platform device-test legs. + +Using the SAME build-discovery as Step 4 (filter AzDO builds by `branchName=refs/pull//merge` +or `sourceVersion == C.headSha`), read each build's **timeline** on `C.headSha` for the +pipeline(s) that RUN the target test — `maui-pr` (def 302) for unit/integration tests, +`maui-pr-uitests` (def 313) for Appium UI tests, `maui-pr-devicetests` (def 314) for device +tests: + +```bash +ORG=dnceng-public; PROJ=public; BUILD= +# Anonymous + Hard-Rule-8-compliant. NEVER call _apis/test/... (it 302-redirects to sign-in). +# Each type=="Job" record is one platform leg: result (succeeded/failed/canceled/null), +# state (completed/inProgress/pending), name (encodes " UITests "): +curl -s "https://dev.azure.com/$ORG/$PROJ/_apis/build/builds/$BUILD/timeline?api-version=7.1" \ + | tee /tmp/gh-aw/agent/timeline_${BUILD}.json \ + | jq -r '.records[] | select(.type=="Job") | "\(.result)\t\(.state)\t\(.name)"' +``` + +(The prefetch already exposes per-leg status in `C.failedLegs` / the checks context, and +`gh pr checks ` lists the same per-leg rows with platform+category in the name — use +whichever is handy; the build timeline is the authoritative cross-check on the exact build.) + +Treat a target test as **VALIDATED-GREEN** only if, on `C.headSha`, the leg that runs its +category is green on **every platform it runs on** — a fix that repairs one platform must not +silently regress the same test's leg on another, so a green leg on the originally-red platform +alone is NOT enough. Across the target pipeline's platform legs (for a UI test: the Android, +iOS, Windows, and macOS/MacCatalyst legs running the target's category; for a device test: each +device-test platform), require ALL of: +- the target's category leg is `result == "succeeded"` **and** `state == "completed"` on + **every** platform that runs it, AND +- that leg is `failed` / `canceled` / aborted on **NO** platform, AND +- **no platform is left unverified.** For each platform family the target pipeline covers, that + platform's category leg must have CONCLUDED (`state == "completed"`) on `C.headSha`. If a + platform simply has no leg for the target's category, the test does not run there — that is + fine, not a gap. But if a platform's category leg has **not concluded** (`state` is + `inProgress` / pending, or an `/azp`-gated `maui-pr-uitests` / `maui-pr-devicetests` leg that + has not been kicked), the test's status on that platform is UNKNOWN → the fix is NOT yet + validated across platforms: record `skipped: target test green on but + not yet verified on (leg(s) pending / need /azp run) on PR #` + and stop this gate WITHOUT marking ready. + +A target test whose category leg never concluded on ANY platform (**not executed** anywhere — +e.g. its `/azp`-gated pipeline has not been kicked) is likewise NOT validated: record `skipped: +target test not yet executed on PR # ( not run — needs /azp run)` and stop +this gate WITHOUT marking ready. Do NOT overclaim — a green *sibling* leg (a different category +on the same platform) is not the target's leg, and a green leg on one platform is not a pass on +the others. + +**T3 — Mark ready + report.** If EVERY target test is VALIDATED-GREEN. The 🎯 comment, +the mark-ready, and the `p/0` label are THREE SEPARATE safe-outputs — the comment +existing does NOT prove the mark-ready took effect, so they are tracked independently: + +- **Comment idempotency (dup-suppress only — never gates the mark-ready).** We only reach + T3 while `C.isDraft == true` (the precondition stops an already-ready PR before here). So + if a prior bot comment for THIS head that carries the leading `🎯` anchor AND the + contiguous phrase `validated green on ` already exists (BOTH T3 variants — + `🎯 Target test validated green on ` and the build-only `🎯 Build validated green on + ` — contain that exact contiguous phrase, so this recognizes both; keying on + `target test` alone would instead miss the build-only comment and re-post it every sweep. + **Require the `🎯` anchor** so the match can NEVER be loosened to a gapped + `validated…green…on` form that would false-positive on the Step 3 `✅ … validated — the + fix's CI is green on ` surface comment and wrongly suppress the audit 🎯), the comment + landed on an earlier sweep but the **mark-ready did NOT take + effect** (the PR is still a draft) — set `SUPPRESS_COMMENT = true` (do not re-post the + duplicate 🎯 comment) but STILL complete the mark-ready + label below. Never treat the + comment's existence as "already marked ready" while the PR is still a draft. Record this + case as `re-marking PR # (🎯 present; mark-ready did not take on a prior sweep)`. +- **Atomicity budget pre-check (Hard-Rule 7).** Determine the outputs this gate will emit: + `mark_pull_request_as_ready_for_review` + `add_labels` always, plus `add_comment` unless + `SUPPRESS_COMMENT`. Verify a free per-run slot remains in EACH bucket you are about to + use. If ANY required bucket is exhausted, emit NONE of them — record `skipped: per-run + cap reached; deferring mark-ready PR # to next cycle` and stop this gate. Never mark a + PR ready (or label it) unless its 🎯 audit comment is GUARANTEED to exist for this exact + head SHA — either posted in this same sweep, or (in the `SUPPRESS_COMMENT` re-marking case) + already present from a prior sweep. The outputs you DO emit either all land or all defer + together; the audit trail must never be absent, but it is NOT re-posted when it already exists. +- **Dry-run gate (Step 0):** if `dry_run == "true"`, emit NOTHING — print the intended + readiness comment and "would mark ready + add p/0" to the run log, tally `dry-run: + would-mark-ready PR #`, and stop. +- Otherwise emit for THIS PR number ``: + 1. `add_comment` (ONLY if not `SUPPRESS_COMMENT`): `🎯 Target test validated green on + — passed on ALL platforms it runs on (, + buildId ). — not + caused by this fix."> Transitioning this PR from draft to ready for review and adding + `p/0`; a maintainer still reviews and merges.` (For a **build-only fix** — the T1 + whole-build fallback, no single target test — phrase the first clause as `🎯 Build + validated green on — the maui-pr build passed on ALL platforms (buildId + ).` instead of naming a test.) + 2. `mark_pull_request_as_ready_for_review` with `reason:` a one-line justification + naming the validated test(s) and ``. + 3. `add_labels` with `labels: ["p/0"]` for PR # — put the now-review-ready fix into + the team's p/0 priority queue so it is triaged, not lost in the draft backlog. (If + the PR somehow already carries `p/0`, this is a harmless no-op.) +- Record `marked-ready PR # (target green on ALL platforms on , + labeled p/0)` and stop. + #### Step 3.5.R — Maintainer change-request response (Track C) Reached from Step 3.5 gate 0. Goal: when an **eligible human reviewer** (Hard-Rule @@ -1383,7 +1669,19 @@ Filed by [`ci-status-fix`](https://github.com/dotnet/maui/blob/main/.github/work ### Step 8 — Per-issue tally + end-of-run summary -Per issue, append one outcome line to `/tmp/gh-aw/agent/coverage.txt`: +Per issue, append **one** outcome line to `/tmp/gh-aw/agent/coverage.txt` — the +terminal outcome for the cycle. When one cycle produces a chained pair — a PR gets a +same-cycle precursor line and **then** a Step 3.6 readiness line (`marked-ready` on the +draft→ready flip, or `already-ready` on a later steady-state sweep of an already-ready +PR) — record ONLY the terminal Step 3.6 readiness line: it supersedes ANY same-cycle +non-readiness precursor for that PR, so an aggregator keying on one-line-per-issue never +double-counts. The precursor may be a Step 3 green line (`surfaced-green` on the first ✅, +or `already-surfaced` when it already exists) OR — when an unrelated-flake red and a +target-green coincide — a Step 4 `annotated-flake` line; either way the readiness line is +terminal, and the superseded precursor's signal survives in the PR's 🎯/♻️ comment so no +information is lost. This covers the flip cycle (`surfaced-green` → `marked-ready`), the +post-flip steady state (`already-surfaced` → `already-ready`), and the flake-coincident +flip (`annotated-flake` → `marked-ready`): ``` # main attempt- @@ -1394,8 +1692,14 @@ Per issue, append one outcome line to `/tmp/gh-aw/agent/coverage.txt`: `advance-PR # attempt /10` (ADVANCE mode — new commit pushed to the existing PR), `surfaced-green PR #` (fix's own CI went green; commented for review, did not advance), `annotated-flake PR #` (red was unrelated flake; -commented, attempt NOT burned), `waiting PR #` (CI not settled yet), -`dry-run: would-`, `skipped: `. (The +commented, attempt NOT burned), `marked-ready PR #` (the specific fixed test +was confirmed green in the PR's own CI; draft flipped to ready for review), +`already-surfaced PR #` (the fix's ✅ green comment already existed this sweep; +re-post suppressed — superseded by the Step 3.6 readiness line when the PR also flips +ready that cycle), `already-ready PR #` (terminal steady state — the PR was flipped +ready on a prior cycle and remains green on an unchanged head; no action taken, +supersedes `already-surfaced`), `waiting PR #` (CI not settled yet), +`dry-run: would-`, `skipped: `. (The `needs-human-PR` outcome is reserved for the deferred hand-off — Step 6 currently records a skip instead, so it is not emitted.)
` lists the same per-leg rows with platform+category in the name — use +whichever is handy; the build timeline is the authoritative cross-check on the exact build.) + +Treat a target test as **VALIDATED-GREEN** only if, on `C.headSha`, the leg that runs its +category is green on **every platform it runs on** — a fix that repairs one platform must not +silently regress the same test's leg on another, so a green leg on the originally-red platform +alone is NOT enough. Across the target pipeline's platform legs (for a UI test: the Android, +iOS, Windows, and macOS/MacCatalyst legs running the target's category; for a device test: each +device-test platform), require ALL of: +- the target's category leg is `result == "succeeded"` **and** `state == "completed"` on + **every** platform that runs it, AND +- that leg is `failed` / `canceled` / aborted on **NO** platform, AND +- **no platform is left unverified.** For each platform family the target pipeline covers, that + platform's category leg must have CONCLUDED (`state == "completed"`) on `C.headSha`. If a + platform simply has no leg for the target's category, the test does not run there — that is + fine, not a gap. But if a platform's category leg has **not concluded** (`state` is + `inProgress` / pending, or an `/azp`-gated `maui-pr-uitests` / `maui-pr-devicetests` leg that + has not been kicked), the test's status on that platform is UNKNOWN → the fix is NOT yet + validated across platforms: record `skipped: target test green on but + not yet verified on (leg(s) pending / need /azp run) on PR #` + and stop this gate WITHOUT marking ready. + +A target test whose category leg never concluded on ANY platform (**not executed** anywhere — +e.g. its `/azp`-gated pipeline has not been kicked) is likewise NOT validated: record `skipped: +target test not yet executed on PR # ( not run — needs /azp run)` and stop +this gate WITHOUT marking ready. Do NOT overclaim — a green *sibling* leg (a different category +on the same platform) is not the target's leg, and a green leg on one platform is not a pass on +the others. + +**T3 — Mark ready + report.** If EVERY target test is VALIDATED-GREEN. The 🎯 comment, +the mark-ready, and the `p/0` label are THREE SEPARATE safe-outputs — the comment +existing does NOT prove the mark-ready took effect, so they are tracked independently: + +- **Comment idempotency (dup-suppress only — never gates the mark-ready).** We only reach + T3 while `C.isDraft == true` (the precondition stops an already-ready PR before here). So + if a prior bot comment for THIS head that carries the leading `🎯` anchor AND the + contiguous phrase `validated green on ` already exists (BOTH T3 variants — + `🎯 Target test validated green on ` and the build-only `🎯 Build validated green on + ` — contain that exact contiguous phrase, so this recognizes both; keying on + `target test` alone would instead miss the build-only comment and re-post it every sweep. + **Require the `🎯` anchor** so the match can NEVER be loosened to a gapped + `validated…green…on` form that would false-positive on the Step 3 `✅ … validated — the + fix's CI is green on ` surface comment and wrongly suppress the audit 🎯), the comment + landed on an earlier sweep but the **mark-ready did NOT take + effect** (the PR is still a draft) — set `SUPPRESS_COMMENT = true` (do not re-post the + duplicate 🎯 comment) but STILL complete the mark-ready + label below. Never treat the + comment's existence as "already marked ready" while the PR is still a draft. Record this + case as `re-marking PR # (🎯 present; mark-ready did not take on a prior sweep)`. +- **Atomicity budget pre-check (Hard-Rule 7).** Determine the outputs this gate will emit: + `mark_pull_request_as_ready_for_review` + `add_labels` always, plus `add_comment` unless + `SUPPRESS_COMMENT`. Verify a free per-run slot remains in EACH bucket you are about to + use. If ANY required bucket is exhausted, emit NONE of them — record `skipped: per-run + cap reached; deferring mark-ready PR # to next cycle` and stop this gate. Never mark a + PR ready (or label it) unless its 🎯 audit comment is GUARANTEED to exist for this exact + head SHA — either posted in this same sweep, or (in the `SUPPRESS_COMMENT` re-marking case) + already present from a prior sweep. The outputs you DO emit either all land or all defer + together; the audit trail must never be absent, but it is NOT re-posted when it already exists. +- **Dry-run gate (Step 0):** if `dry_run == "true"`, emit NOTHING — print the intended + readiness comment and "would mark ready + add p/0" to the run log, tally `dry-run: + would-mark-ready PR #`, and stop. +- Otherwise emit for THIS PR number ``: + 1. `add_comment` (ONLY if not `SUPPRESS_COMMENT`): `🎯 Target test validated green on + — passed on ALL platforms it runs on (, + buildId ). — not + caused by this fix."> Transitioning this PR from draft to ready for review and adding + `p/0`; a maintainer still reviews and merges.` (For a **build-only fix** — the T1 + whole-build fallback, no single target test — phrase the first clause as `🎯 Build + validated green on — the maui-pr build passed on ALL platforms (buildId + ).` instead of naming a test.) + 2. `mark_pull_request_as_ready_for_review` with `reason:` a one-line justification + naming the validated test(s) and ``. + 3. `add_labels` with `labels: ["p/0"]` for PR # — put the now-review-ready fix into + the team's p/0 priority queue so it is triaged, not lost in the draft backlog. (If + the PR somehow already carries `p/0`, this is a harmless no-op.) +- Record `marked-ready PR # (target green on ALL platforms on , + labeled p/0)` and stop. + #### Step 3.5.R — Maintainer change-request response (Track C) Reached from Step 3.5 gate 0. Goal: when an **eligible human reviewer** (Hard-Rule @@ -1395,7 +1681,19 @@ Filed by [`ci-status-fix-net11`](https://github.com/dotnet/maui/blob/main/.githu ### Step 8 — Per-issue tally + end-of-run summary -Per issue, append one outcome line to `/tmp/gh-aw/agent/coverage.txt`: +Per issue, append **one** outcome line to `/tmp/gh-aw/agent/coverage.txt` — the +terminal outcome for the cycle. When one cycle produces a chained pair — a PR gets a +same-cycle precursor line and **then** a Step 3.6 readiness line (`marked-ready` on the +draft→ready flip, or `already-ready` on a later steady-state sweep of an already-ready +PR) — record ONLY the terminal Step 3.6 readiness line: it supersedes ANY same-cycle +non-readiness precursor for that PR, so an aggregator keying on one-line-per-issue never +double-counts. The precursor may be a Step 3 green line (`surfaced-green` on the first ✅, +or `already-surfaced` when it already exists) OR — when an unrelated-flake red and a +target-green coincide — a Step 4 `annotated-flake` line; either way the readiness line is +terminal, and the superseded precursor's signal survives in the PR's 🎯/♻️ comment so no +information is lost. This covers the flip cycle (`surfaced-green` → `marked-ready`), the +post-flip steady state (`already-surfaced` → `already-ready`), and the flake-coincident +flip (`annotated-flake` → `marked-ready`): ``` # net11.0 attempt- @@ -1406,8 +1704,14 @@ Per issue, append one outcome line to `/tmp/gh-aw/agent/coverage.txt`: `advance-PR # attempt /10` (ADVANCE mode — new commit pushed to the existing PR), `surfaced-green PR #` (fix's own CI went green; commented for review, did not advance), `annotated-flake PR #` (red was unrelated flake; -commented, attempt NOT burned), `waiting PR #` (CI not settled yet), -`dry-run: would-`, `skipped: `. (The +commented, attempt NOT burned), `marked-ready PR #` (the specific fixed test +was confirmed green in the PR's own CI; draft flipped to ready for review), +`already-surfaced PR #` (the fix's ✅ green comment already existed this sweep; +re-post suppressed — superseded by the Step 3.6 readiness line when the PR also flips +ready that cycle), `already-ready PR #` (terminal steady state — the PR was flipped +ready on a prior cycle and remains green on an unchanged head; no action taken, +supersedes `already-surfaced`), `waiting PR #` (CI not settled yet), +`dry-run: would-`, `skipped: `. (The `needs-human-PR` outcome is reserved for the deferred hand-off — Step 6 currently records a skip instead, so it is not emitted.) diff --git a/.github/workflows/ci-status-fix.lock.yml b/.github/workflows/ci-status-fix.lock.yml index 93dc5969cb0d..65511f2b3259 100644 --- a/.github/workflows/ci-status-fix.lock.yml +++ b/.github/workflows/ci-status-fix.lock.yml @@ -1,4 +1,4 @@ -# gh-aw-metadata: {"schema_version":"v4","frontmatter_hash":"f014161967589ebcaf1ac5ec6f3c88ee5a4388d28b55a07dc36c45b7b2aebab6","body_hash":"86c6b2d4c3df13da14c6a3c8b211381fcc9f97bb1951ff7b80588a2c5338e00c","compiler_version":"v0.80.9","strict":true,"agent_id":"copilot","agent_model":"claude-opus-4.8","engine_versions":{"copilot":"1.0.63"}} +# gh-aw-metadata: {"schema_version":"v4","frontmatter_hash":"b0ebf8a2f179900cfe3638e994b27a43cec52bdbdd2331dc1bd4d65762fa2d6b","body_hash":"1825e2b1f7c7bd88241bf37580655489360f9bebc806edd00837a52ce4ad83a0","compiler_version":"v0.80.9","strict":true,"agent_id":"copilot","agent_model":"claude-opus-4.8","engine_versions":{"copilot":"1.0.63"}} # gh-aw-manifest: {"version":1,"secrets":["COPILOT_PAT_0","COPILOT_PAT_1","COPILOT_PAT_2","COPILOT_PAT_3","COPILOT_PAT_4","COPILOT_PAT_5","COPILOT_PAT_6","COPILOT_PAT_7","COPILOT_PAT_8","COPILOT_PAT_9","GH_AW_CI_TRIGGER_TOKEN","GH_AW_GITHUB_MCP_SERVER_TOKEN","GH_AW_GITHUB_TOKEN","GITHUB_TOKEN"],"actions":[{"repo":"actions/checkout","sha":"34e114876b0b11c390a56381ad16ebd13914f8d5","version":"v4"},{"repo":"actions/checkout","sha":"9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0","version":"v7.0.0"},{"repo":"actions/checkout","sha":"de0fac2e4500dabe0009e67214ff5f5447ce83dd","version":"v6.0.2"},{"repo":"actions/download-artifact","sha":"3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c","version":"v8.0.1"},{"repo":"actions/github-script","sha":"3a2844b7e9c422d3c10d287c895573f7108da1b3","version":"v9.0.0"},{"repo":"actions/setup-node","sha":"48b55a011bda9f5d6aeb4c2d9c7362e8dae4041e","version":"v6.4.0"},{"repo":"actions/upload-artifact","sha":"043fb46d1a93c77aae656e7c1c64a875d1fc6a0a","version":"v7.0.1"},{"repo":"github/gh-aw-actions/setup","sha":"8c7d04ebf1ece56cd381446125da3e0f6896294a","version":"v0.80.9"}],"containers":[{"image":"ghcr.io/github/gh-aw-firewall/agent:0.27.7","digest":"sha256:aae231e4635c8999d039c132f1602d3df850fe9b84a00aa2b5ac981179b5661c","pinned_image":"ghcr.io/github/gh-aw-firewall/agent:0.27.7@sha256:aae231e4635c8999d039c132f1602d3df850fe9b84a00aa2b5ac981179b5661c"},{"image":"ghcr.io/github/gh-aw-firewall/api-proxy:0.27.7","digest":"sha256:009caf2e3d88fa77b64e9a03a95a228fc58db0f1701c6d324b29ba5a3c7c79b6","pinned_image":"ghcr.io/github/gh-aw-firewall/api-proxy:0.27.7@sha256:009caf2e3d88fa77b64e9a03a95a228fc58db0f1701c6d324b29ba5a3c7c79b6"},{"image":"ghcr.io/github/gh-aw-firewall/squid:0.27.7","digest":"sha256:deb1d4e19de62d51cee0508057a596a19315c3423ada4d675cad136dc8037c96","pinned_image":"ghcr.io/github/gh-aw-firewall/squid:0.27.7@sha256:deb1d4e19de62d51cee0508057a596a19315c3423ada4d675cad136dc8037c96"},{"image":"ghcr.io/github/gh-aw-mcpg:v0.3.27","digest":"sha256:fe984bddde4ec05d756d9043edb0a32912e6b7b72f6a121b1082f29221421cc7","pinned_image":"ghcr.io/github/gh-aw-mcpg:v0.3.27@sha256:fe984bddde4ec05d756d9043edb0a32912e6b7b72f6a121b1082f29221421cc7"},{"image":"ghcr.io/github/gh-aw-node","digest":"sha256:529d02eb970b1161aa25c593a9c3df57fdfad5a8add328cb3b6eccef66f3183b","pinned_image":"ghcr.io/github/gh-aw-node@sha256:529d02eb970b1161aa25c593a9c3df57fdfad5a8add328cb3b6eccef66f3183b"},{"image":"ghcr.io/github/github-mcp-server:v1.4.0","digest":"sha256:2afb26356481d1a350e14544a6e160f7f7ec1561a1ea309b823665abf0309036","pinned_image":"ghcr.io/github/github-mcp-server:v1.4.0@sha256:2afb26356481d1a350e14544a6e160f7f7ec1561a1ea309b823665abf0309036"}]} # This file was automatically generated by gh-aw (v0.80.9). DO NOT EDIT. To debug this workflow, load the skill at https://github.com/github/gh-aw/blob/main/debug.md # @@ -37,7 +37,9 @@ # humans (the open tracking issue is the hand-off surface; a dedicated # [ci-fix][needs-human] PR is planned but currently deferred — see Step 6). CI on # the PR is kicked by a human `/azp run` for now; the loop watches, classifies, -# and re-fixes autonomously between kicks. +# and re-fixes autonomously between kicks. When the SPECIFIC test a PR fixed is +# confirmed green in that PR's own CI, the loop marks the draft PR ready for review +# (a state transition only — it never approves or merges). # Never mutes tests, but # de-flakes genuinely flaky ones (deterministic synchronization, no retries / # timeout bumps). Always skips visual-regression / screenshot issues. @@ -273,24 +275,24 @@ jobs: run: | bash "${RUNNER_TEMP}/gh-aw/actions/create_prompt_first.sh" { - cat << 'GH_AW_PROMPT_25cf75111b670167_EOF' + cat << 'GH_AW_PROMPT_2ac1d0b4083428f8_EOF' - GH_AW_PROMPT_25cf75111b670167_EOF + GH_AW_PROMPT_2ac1d0b4083428f8_EOF cat "${RUNNER_TEMP}/gh-aw/prompts/xpia.md" cat "${RUNNER_TEMP}/gh-aw/prompts/temp_folder_prompt.md" cat "${RUNNER_TEMP}/gh-aw/prompts/markdown.md" cat "${RUNNER_TEMP}/gh-aw/prompts/safe_outputs_prompt.md" - cat << 'GH_AW_PROMPT_25cf75111b670167_EOF' + cat << 'GH_AW_PROMPT_2ac1d0b4083428f8_EOF' - Tools: add_comment(max:3), create_pull_request(max:3), update_pull_request(max:3), push_to_pull_request_branch(max:3), missing_tool, missing_data, noop - GH_AW_PROMPT_25cf75111b670167_EOF + Tools: add_comment(max:6), create_pull_request(max:3), update_pull_request(max:3), mark_pull_request_as_ready_for_review(max:3), add_labels(max:3), push_to_pull_request_branch(max:3), missing_tool, missing_data, noop + GH_AW_PROMPT_2ac1d0b4083428f8_EOF cat "${RUNNER_TEMP}/gh-aw/prompts/safe_outputs_create_pull_request.md" cat "${RUNNER_TEMP}/gh-aw/prompts/safe_outputs_push_to_pr_branch.md" - cat << 'GH_AW_PROMPT_25cf75111b670167_EOF' + cat << 'GH_AW_PROMPT_2ac1d0b4083428f8_EOF' - GH_AW_PROMPT_25cf75111b670167_EOF + GH_AW_PROMPT_2ac1d0b4083428f8_EOF cat "${RUNNER_TEMP}/gh-aw/prompts/mcp_cli_tools_prompt.md" - cat << 'GH_AW_PROMPT_25cf75111b670167_EOF' + cat << 'GH_AW_PROMPT_2ac1d0b4083428f8_EOF' The following GitHub context information is available for this workflow: {{#if github.actor}} @@ -332,12 +334,12 @@ jobs: stop immediately and report the limitation rather than spending turns trying to work around it. - GH_AW_PROMPT_25cf75111b670167_EOF + GH_AW_PROMPT_2ac1d0b4083428f8_EOF cat "${RUNNER_TEMP}/gh-aw/prompts/github_mcp_tools_with_safeoutputs_prompt.md" - cat << 'GH_AW_PROMPT_25cf75111b670167_EOF' + cat << 'GH_AW_PROMPT_2ac1d0b4083428f8_EOF' {{#runtime-import .github/workflows/ci-status-fix.md}} - GH_AW_PROMPT_25cf75111b670167_EOF + GH_AW_PROMPT_2ac1d0b4083428f8_EOF } > "$GH_AW_PROMPT" - name: Interpolate variables and render templates uses: actions/github-script@3a2844b7e9c422d3c10d287c895573f7108da1b3 # v9.0.0 @@ -554,16 +556,18 @@ jobs: mkdir -p "${RUNNER_TEMP}/gh-aw/safeoutputs" mkdir -p /tmp/gh-aw/safeoutputs mkdir -p /tmp/gh-aw/mcp-logs/safeoutputs - cat > "${RUNNER_TEMP}/gh-aw/safeoutputs/config.json" << 'GH_AW_SAFE_OUTPUTS_CONFIG_fa2a69fad5fd0485_EOF' - {"add_comment":{"discussions":false,"max":3,"required_labels":["agentic-workflows"],"required_title_prefix":"[ci-fix] ","target":"*"},"create_pull_request":{"allowed_base_branches":["main"],"allowed_branches":["ci-fix/**"],"allowed_files":["src/Core/**","src/Controls/**","src/Essentials/**","src/BlazorWebView/**","src/TestUtils/**","src/Templates/**","**/PublicAPI.Unshipped.txt"],"base_branch":"main","draft":true,"labels":["agentic-workflows"],"max":3,"max_patch_files":100,"max_patch_size":4096,"protect_top_level_dot_folders":true,"protected_files":["package.json","bun.lockb","bunfig.toml","deno.json","deno.jsonc","deno.lock","global.json","NuGet.Config","Directory.Packages.props","mix.exs","mix.lock","go.mod","go.sum","stack.yaml","stack.yaml.lock","pom.xml","build.gradle","build.gradle.kts","settings.gradle","settings.gradle.kts","gradle.properties","package-lock.json","yarn.lock","pnpm-lock.yaml","npm-shrinkwrap.json","requirements.txt","Pipfile","Pipfile.lock","pyproject.toml","setup.py","setup.cfg","Gemfile","Gemfile.lock","uv.lock","CODEOWNERS","DESIGN.md","README.md","CONTRIBUTING.md","CHANGELOG.md","SECURITY.md","CODE_OF_CONDUCT.md","AGENTS.md","CLAUDE.md","GEMINI.md"],"protected_files_policy":"request_review","title_prefix":"[ci-fix] "},"create_report_incomplete_issue":{},"missing_data":{},"missing_tool":{},"noop":{"max":1,"report-as-issue":"true"},"push_to_pull_request_branch":{"allowed_files":["src/Core/**","src/Controls/**","src/Essentials/**","src/BlazorWebView/**","src/TestUtils/**","src/Templates/**","**/PublicAPI.Unshipped.txt"],"if_no_changes":"warn","max":3,"max_patch_size":4096,"protect_top_level_dot_folders":true,"protected_files":["package.json","bun.lockb","bunfig.toml","deno.json","deno.jsonc","deno.lock","global.json","NuGet.Config","Directory.Packages.props","mix.exs","mix.lock","go.mod","go.sum","stack.yaml","stack.yaml.lock","pom.xml","build.gradle","build.gradle.kts","settings.gradle","settings.gradle.kts","gradle.properties","package-lock.json","yarn.lock","pnpm-lock.yaml","npm-shrinkwrap.json","requirements.txt","Pipfile","Pipfile.lock","pyproject.toml","setup.py","setup.cfg","Gemfile","Gemfile.lock","uv.lock","CODEOWNERS","DESIGN.md","README.md","CONTRIBUTING.md","CHANGELOG.md","SECURITY.md","CODE_OF_CONDUCT.md","AGENTS.md","CLAUDE.md","GEMINI.md"],"required_labels":["agentic-workflows"],"target":"*","title_prefix":"[ci-fix] "},"report_incomplete":{},"update_pull_request":{"allow_body":true,"allow_title":false,"max":3,"target":"*","update_branch":false}} - GH_AW_SAFE_OUTPUTS_CONFIG_fa2a69fad5fd0485_EOF + cat > "${RUNNER_TEMP}/gh-aw/safeoutputs/config.json" << 'GH_AW_SAFE_OUTPUTS_CONFIG_0644bf1434ca0071_EOF' + {"add_comment":{"discussions":false,"max":6,"required_labels":["agentic-workflows"],"required_title_prefix":"[ci-fix] ","target":"*"},"add_labels":{"allowed":["p/0"],"max":3,"required_labels":["agentic-workflows"],"required_title_prefix":"[ci-fix] ","target":"*"},"create_pull_request":{"allowed_base_branches":["main"],"allowed_branches":["ci-fix/**"],"allowed_files":["src/Core/**","src/Controls/**","src/Essentials/**","src/BlazorWebView/**","src/TestUtils/**","src/Templates/**","**/PublicAPI.Unshipped.txt"],"base_branch":"main","draft":true,"labels":["agentic-workflows"],"max":3,"max_patch_files":100,"max_patch_size":4096,"protect_top_level_dot_folders":true,"protected_files":["package.json","bun.lockb","bunfig.toml","deno.json","deno.jsonc","deno.lock","global.json","NuGet.Config","Directory.Packages.props","mix.exs","mix.lock","go.mod","go.sum","stack.yaml","stack.yaml.lock","pom.xml","build.gradle","build.gradle.kts","settings.gradle","settings.gradle.kts","gradle.properties","package-lock.json","yarn.lock","pnpm-lock.yaml","npm-shrinkwrap.json","requirements.txt","Pipfile","Pipfile.lock","pyproject.toml","setup.py","setup.cfg","Gemfile","Gemfile.lock","uv.lock","CODEOWNERS","DESIGN.md","README.md","CONTRIBUTING.md","CHANGELOG.md","SECURITY.md","CODE_OF_CONDUCT.md","AGENTS.md","CLAUDE.md","GEMINI.md"],"protected_files_policy":"request_review","title_prefix":"[ci-fix] "},"create_report_incomplete_issue":{},"mark_pull_request_as_ready_for_review":{"max":3,"required_labels":["agentic-workflows"],"required_title_prefix":"[ci-fix] ","target":"*"},"missing_data":{},"missing_tool":{},"noop":{"max":1,"report-as-issue":"true"},"push_to_pull_request_branch":{"allowed_files":["src/Core/**","src/Controls/**","src/Essentials/**","src/BlazorWebView/**","src/TestUtils/**","src/Templates/**","**/PublicAPI.Unshipped.txt"],"if_no_changes":"warn","max":3,"max_patch_size":4096,"protect_top_level_dot_folders":true,"protected_files":["package.json","bun.lockb","bunfig.toml","deno.json","deno.jsonc","deno.lock","global.json","NuGet.Config","Directory.Packages.props","mix.exs","mix.lock","go.mod","go.sum","stack.yaml","stack.yaml.lock","pom.xml","build.gradle","build.gradle.kts","settings.gradle","settings.gradle.kts","gradle.properties","package-lock.json","yarn.lock","pnpm-lock.yaml","npm-shrinkwrap.json","requirements.txt","Pipfile","Pipfile.lock","pyproject.toml","setup.py","setup.cfg","Gemfile","Gemfile.lock","uv.lock","CODEOWNERS","DESIGN.md","README.md","CONTRIBUTING.md","CHANGELOG.md","SECURITY.md","CODE_OF_CONDUCT.md","AGENTS.md","CLAUDE.md","GEMINI.md"],"required_labels":["agentic-workflows"],"target":"*","title_prefix":"[ci-fix] "},"report_incomplete":{},"update_pull_request":{"allow_body":true,"allow_title":false,"max":3,"target":"*","update_branch":false}} + GH_AW_SAFE_OUTPUTS_CONFIG_0644bf1434ca0071_EOF - name: Generate Safe Outputs Tools env: GH_AW_TOOLS_META_JSON: | { "description_suffixes": { - "add_comment": " CONSTRAINTS: Maximum 3 comment(s) can be added. Target: *. Supports reply_to_id for discussion threading.", + "add_comment": " CONSTRAINTS: Maximum 6 comment(s) can be added. Target: *. Supports reply_to_id for discussion threading.", + "add_labels": " CONSTRAINTS: Maximum 3 label(s) can be added. Only these labels are allowed: [\"p/0\"]. Target: *.", "create_pull_request": " CONSTRAINTS: Maximum 3 pull request(s) can be created. Title will be prefixed with \"[ci-fix] \". Labels [\"agentic-workflows\"] will be automatically added. Only these labels are allowed: [\"agentic-workflows\"]. PRs will be created as drafts.", + "mark_pull_request_as_ready_for_review": " CONSTRAINTS: Maximum 3 pull request(s) can be marked as ready for review.", "push_to_pull_request_branch": " CONSTRAINTS: Maximum 3 push(es) can be made. The target pull request title must start with \"[ci-fix] \".", "update_pull_request": " CONSTRAINTS: Maximum 3 pull request(s) can be updated. Target: *." }, @@ -594,6 +598,25 @@ jobs: } } }, + "add_labels": { + "defaultMax": 5, + "fields": { + "item_number": { + "issueNumberOrTemporaryId": true + }, + "labels": { + "required": true, + "type": "array", + "itemType": "string", + "itemSanitize": true, + "itemMaxLength": 128 + }, + "repo": { + "type": "string", + "maxLength": 256 + } + } + }, "create_pull_request": { "defaultMax": 1, "fields": { @@ -635,6 +658,24 @@ jobs: } } }, + "mark_pull_request_as_ready_for_review": { + "defaultMax": 1, + "fields": { + "pull_request_number": { + "issueOrPRNumber": true + }, + "reason": { + "required": true, + "type": "string", + "sanitize": true, + "maxLength": 65000 + }, + "repo": { + "type": "string", + "maxLength": 256 + } + } + }, "missing_data": { "defaultMax": 20, "fields": { @@ -1494,7 +1535,7 @@ jobs: uses: actions/github-script@3a2844b7e9c422d3c10d287c895573f7108da1b3 # v9.0.0 env: WORKFLOW_NAME: "CI Failure Fixer (main)" - WORKFLOW_DESCRIPTION: "Periodic pass over open ci-scan tracking issues filed by the main-branch CI\nfailure scanner (.github/workflows/ci-status-main.md). This workflow targets\nthe `main` branch EXCLUSIVELY: it processes only issues labelled ci-scan and\nopens every PR against main. (The net11.0 branch is handled by the parallel\n.github/workflows/ci-status-fix-net11.md — the two are split because gh-aw can\nonly transport a fix relative to ONE static base branch per workflow, and the\nmain↔net11.0 divergence exceeds gh-aw's 10 MB transport-patch cap.) The fixer\nopens ONE draft [ci-fix] PR per actionable issue against main, then WATCHES\nthat PR's own CI on later runs: when the fix's CI comes back red and the red is\ncaused by the fix itself, it pushes a fresh follow-up fix onto the SAME PR\nbranch (never a second PR) — up to 10 attempts — then stops and defers to\nhumans (the open tracking issue is the hand-off surface; a dedicated\n[ci-fix][needs-human] PR is planned but currently deferred — see Step 6). CI on\nthe PR is kicked by a human `/azp run` for now; the loop watches, classifies,\nand re-fixes autonomously between kicks.\nNever mutes tests, but\nde-flakes genuinely flaky ones (deterministic synchronization, no retries /\ntimeout bumps). Always skips visual-regression / screenshot issues." + WORKFLOW_DESCRIPTION: "Periodic pass over open ci-scan tracking issues filed by the main-branch CI\nfailure scanner (.github/workflows/ci-status-main.md). This workflow targets\nthe `main` branch EXCLUSIVELY: it processes only issues labelled ci-scan and\nopens every PR against main. (The net11.0 branch is handled by the parallel\n.github/workflows/ci-status-fix-net11.md — the two are split because gh-aw can\nonly transport a fix relative to ONE static base branch per workflow, and the\nmain↔net11.0 divergence exceeds gh-aw's 10 MB transport-patch cap.) The fixer\nopens ONE draft [ci-fix] PR per actionable issue against main, then WATCHES\nthat PR's own CI on later runs: when the fix's CI comes back red and the red is\ncaused by the fix itself, it pushes a fresh follow-up fix onto the SAME PR\nbranch (never a second PR) — up to 10 attempts — then stops and defers to\nhumans (the open tracking issue is the hand-off surface; a dedicated\n[ci-fix][needs-human] PR is planned but currently deferred — see Step 6). CI on\nthe PR is kicked by a human `/azp run` for now; the loop watches, classifies,\nand re-fixes autonomously between kicks. When the SPECIFIC test a PR fixed is\nconfirmed green in that PR's own CI, the loop marks the draft PR ready for review\n(a state transition only — it never approves or merges).\nNever mutes tests, but\nde-flakes genuinely flaky ones (deterministic synchronization, no retries /\ntimeout bumps). Always skips visual-regression / screenshot issues." HAS_PATCH: ${{ needs.agent.outputs.has_patch }} with: script: | @@ -1910,7 +1951,7 @@ jobs: GH_AW_ALLOWED_DOMAINS: "*.blob.core.windows.net,*.githubusercontent.com,api.business.githubcopilot.com,api.enterprise.githubcopilot.com,api.github.com,api.githubcopilot.com,api.individual.githubcopilot.com,api.snapcraft.io,archive.ubuntu.com,azure.archive.ubuntu.com,codeload.github.com,crl.geotrust.com,crl.globalsign.com,crl.identrust.com,crl.sectigo.com,crl.thawte.com,crl.usertrust.com,crl.verisign.com,crl3.digicert.com,crl4.digicert.com,crls.ssl.com,dev.azure.com,docs.github.com,github-cloud.githubusercontent.com,github-cloud.s3.amazonaws.com,github.blog,github.com,github.githubassets.com,helix.dot.net,host.docker.internal,json-schema.org,json.schemastore.org,keyserver.ubuntu.com,lfs.github.com,objects.githubusercontent.com,ocsp.digicert.com,ocsp.geotrust.com,ocsp.globalsign.com,ocsp.identrust.com,ocsp.sectigo.com,ocsp.ssl.com,ocsp.thawte.com,ocsp.usertrust.com,ocsp.verisign.com,packagecloud.io,packages.cloud.google.com,packages.microsoft.com,patch-diff.githubusercontent.com,ppa.launchpad.net,raw.githubusercontent.com,registry.npmjs.org,s.symcb.com,s.symcd.com,security.ubuntu.com,telemetry.enterprise.githubcopilot.com,ts-crl.ws.symantec.com,ts-ocsp.ws.symantec.com,www.googleapis.com" GITHUB_SERVER_URL: ${{ github.server_url }} GITHUB_API_URL: ${{ github.api_url }} - GH_AW_SAFE_OUTPUTS_HANDLER_CONFIG: "{\"add_comment\":{\"discussions\":false,\"max\":3,\"required_labels\":[\"agentic-workflows\"],\"required_title_prefix\":\"[ci-fix] \",\"target\":\"*\"},\"create_pull_request\":{\"allowed_base_branches\":[\"main\"],\"allowed_branches\":[\"ci-fix/**\"],\"allowed_files\":[\"src/Core/**\",\"src/Controls/**\",\"src/Essentials/**\",\"src/BlazorWebView/**\",\"src/TestUtils/**\",\"src/Templates/**\",\"**/PublicAPI.Unshipped.txt\"],\"base_branch\":\"main\",\"draft\":true,\"labels\":[\"agentic-workflows\"],\"max\":3,\"max_patch_files\":100,\"max_patch_size\":4096,\"protect_top_level_dot_folders\":true,\"protected_files\":[\"package.json\",\"bun.lockb\",\"bunfig.toml\",\"deno.json\",\"deno.jsonc\",\"deno.lock\",\"global.json\",\"NuGet.Config\",\"Directory.Packages.props\",\"mix.exs\",\"mix.lock\",\"go.mod\",\"go.sum\",\"stack.yaml\",\"stack.yaml.lock\",\"pom.xml\",\"build.gradle\",\"build.gradle.kts\",\"settings.gradle\",\"settings.gradle.kts\",\"gradle.properties\",\"package-lock.json\",\"yarn.lock\",\"pnpm-lock.yaml\",\"npm-shrinkwrap.json\",\"requirements.txt\",\"Pipfile\",\"Pipfile.lock\",\"pyproject.toml\",\"setup.py\",\"setup.cfg\",\"Gemfile\",\"Gemfile.lock\",\"uv.lock\",\"CODEOWNERS\",\"DESIGN.md\",\"README.md\",\"CONTRIBUTING.md\",\"CHANGELOG.md\",\"SECURITY.md\",\"CODE_OF_CONDUCT.md\",\"AGENTS.md\",\"CLAUDE.md\",\"GEMINI.md\"],\"protected_files_policy\":\"request_review\",\"title_prefix\":\"[ci-fix] \"},\"create_report_incomplete_issue\":{},\"missing_data\":{},\"missing_tool\":{},\"noop\":{\"max\":1,\"report-as-issue\":\"true\"},\"push_to_pull_request_branch\":{\"allowed_files\":[\"src/Core/**\",\"src/Controls/**\",\"src/Essentials/**\",\"src/BlazorWebView/**\",\"src/TestUtils/**\",\"src/Templates/**\",\"**/PublicAPI.Unshipped.txt\"],\"if_no_changes\":\"warn\",\"max\":3,\"max_patch_size\":4096,\"protect_top_level_dot_folders\":true,\"protected_files\":[\"package.json\",\"bun.lockb\",\"bunfig.toml\",\"deno.json\",\"deno.jsonc\",\"deno.lock\",\"global.json\",\"NuGet.Config\",\"Directory.Packages.props\",\"mix.exs\",\"mix.lock\",\"go.mod\",\"go.sum\",\"stack.yaml\",\"stack.yaml.lock\",\"pom.xml\",\"build.gradle\",\"build.gradle.kts\",\"settings.gradle\",\"settings.gradle.kts\",\"gradle.properties\",\"package-lock.json\",\"yarn.lock\",\"pnpm-lock.yaml\",\"npm-shrinkwrap.json\",\"requirements.txt\",\"Pipfile\",\"Pipfile.lock\",\"pyproject.toml\",\"setup.py\",\"setup.cfg\",\"Gemfile\",\"Gemfile.lock\",\"uv.lock\",\"CODEOWNERS\",\"DESIGN.md\",\"README.md\",\"CONTRIBUTING.md\",\"CHANGELOG.md\",\"SECURITY.md\",\"CODE_OF_CONDUCT.md\",\"AGENTS.md\",\"CLAUDE.md\",\"GEMINI.md\"],\"required_labels\":[\"agentic-workflows\"],\"target\":\"*\",\"title_prefix\":\"[ci-fix] \"},\"report_incomplete\":{},\"update_pull_request\":{\"allow_body\":true,\"allow_title\":false,\"max\":3,\"target\":\"*\",\"update_branch\":false}}" + GH_AW_SAFE_OUTPUTS_HANDLER_CONFIG: "{\"add_comment\":{\"discussions\":false,\"max\":6,\"required_labels\":[\"agentic-workflows\"],\"required_title_prefix\":\"[ci-fix] \",\"target\":\"*\"},\"add_labels\":{\"allowed\":[\"p/0\"],\"max\":3,\"required_labels\":[\"agentic-workflows\"],\"required_title_prefix\":\"[ci-fix] \",\"target\":\"*\"},\"create_pull_request\":{\"allowed_base_branches\":[\"main\"],\"allowed_branches\":[\"ci-fix/**\"],\"allowed_files\":[\"src/Core/**\",\"src/Controls/**\",\"src/Essentials/**\",\"src/BlazorWebView/**\",\"src/TestUtils/**\",\"src/Templates/**\",\"**/PublicAPI.Unshipped.txt\"],\"base_branch\":\"main\",\"draft\":true,\"labels\":[\"agentic-workflows\"],\"max\":3,\"max_patch_files\":100,\"max_patch_size\":4096,\"protect_top_level_dot_folders\":true,\"protected_files\":[\"package.json\",\"bun.lockb\",\"bunfig.toml\",\"deno.json\",\"deno.jsonc\",\"deno.lock\",\"global.json\",\"NuGet.Config\",\"Directory.Packages.props\",\"mix.exs\",\"mix.lock\",\"go.mod\",\"go.sum\",\"stack.yaml\",\"stack.yaml.lock\",\"pom.xml\",\"build.gradle\",\"build.gradle.kts\",\"settings.gradle\",\"settings.gradle.kts\",\"gradle.properties\",\"package-lock.json\",\"yarn.lock\",\"pnpm-lock.yaml\",\"npm-shrinkwrap.json\",\"requirements.txt\",\"Pipfile\",\"Pipfile.lock\",\"pyproject.toml\",\"setup.py\",\"setup.cfg\",\"Gemfile\",\"Gemfile.lock\",\"uv.lock\",\"CODEOWNERS\",\"DESIGN.md\",\"README.md\",\"CONTRIBUTING.md\",\"CHANGELOG.md\",\"SECURITY.md\",\"CODE_OF_CONDUCT.md\",\"AGENTS.md\",\"CLAUDE.md\",\"GEMINI.md\"],\"protected_files_policy\":\"request_review\",\"title_prefix\":\"[ci-fix] \"},\"create_report_incomplete_issue\":{},\"mark_pull_request_as_ready_for_review\":{\"max\":3,\"required_labels\":[\"agentic-workflows\"],\"required_title_prefix\":\"[ci-fix] \",\"target\":\"*\"},\"missing_data\":{},\"missing_tool\":{},\"noop\":{\"max\":1,\"report-as-issue\":\"true\"},\"push_to_pull_request_branch\":{\"allowed_files\":[\"src/Core/**\",\"src/Controls/**\",\"src/Essentials/**\",\"src/BlazorWebView/**\",\"src/TestUtils/**\",\"src/Templates/**\",\"**/PublicAPI.Unshipped.txt\"],\"if_no_changes\":\"warn\",\"max\":3,\"max_patch_size\":4096,\"protect_top_level_dot_folders\":true,\"protected_files\":[\"package.json\",\"bun.lockb\",\"bunfig.toml\",\"deno.json\",\"deno.jsonc\",\"deno.lock\",\"global.json\",\"NuGet.Config\",\"Directory.Packages.props\",\"mix.exs\",\"mix.lock\",\"go.mod\",\"go.sum\",\"stack.yaml\",\"stack.yaml.lock\",\"pom.xml\",\"build.gradle\",\"build.gradle.kts\",\"settings.gradle\",\"settings.gradle.kts\",\"gradle.properties\",\"package-lock.json\",\"yarn.lock\",\"pnpm-lock.yaml\",\"npm-shrinkwrap.json\",\"requirements.txt\",\"Pipfile\",\"Pipfile.lock\",\"pyproject.toml\",\"setup.py\",\"setup.cfg\",\"Gemfile\",\"Gemfile.lock\",\"uv.lock\",\"CODEOWNERS\",\"DESIGN.md\",\"README.md\",\"CONTRIBUTING.md\",\"CHANGELOG.md\",\"SECURITY.md\",\"CODE_OF_CONDUCT.md\",\"AGENTS.md\",\"CLAUDE.md\",\"GEMINI.md\"],\"required_labels\":[\"agentic-workflows\"],\"target\":\"*\",\"title_prefix\":\"[ci-fix] \"},\"report_incomplete\":{},\"update_pull_request\":{\"allow_body\":true,\"allow_title\":false,\"max\":3,\"target\":\"*\",\"update_branch\":false}}" GH_AW_CI_TRIGGER_TOKEN: ${{ secrets.GH_AW_CI_TRIGGER_TOKEN }} with: github-token: ${{ secrets.GH_AW_GITHUB_TOKEN || secrets.GITHUB_TOKEN }} diff --git a/.github/workflows/ci-status-fix.md b/.github/workflows/ci-status-fix.md index d3227e25170c..9c2f46054029 100644 --- a/.github/workflows/ci-status-fix.md +++ b/.github/workflows/ci-status-fix.md @@ -15,7 +15,9 @@ description: | humans (the open tracking issue is the hand-off surface; a dedicated [ci-fix][needs-human] PR is planned but currently deferred — see Step 6). CI on the PR is kicked by a human `/azp run` for now; the loop watches, classifies, - and re-fixes autonomously between kicks. + and re-fixes autonomously between kicks. When the SPECIFIC test a PR fixed is + confirmed green in that PR's own CI, the loop marks the draft PR ready for review + (a state transition only — it never approves or merges). Never mutes tests, but de-flakes genuinely flaky ones (deterministic synchronization, no retries / timeout bumps). Always skips visual-regression / screenshot issues. @@ -239,8 +241,15 @@ safe-outputs: # PER-RUN total (not per-PR): a sweep may surface several green PRs and/or # annotate several flaky reds in one run. At max:1 all but the first comment # were silently dropped, starving the workflow's #1 value (surfacing green PRs - # for review). Raised to 3 to match the per-run throughput of the other outputs. - max: 3 + # for review). Sized to 6 (not 3) because a single green DRAFT PR that gets + # flipped ready in one sweep spends TWO comment slots — the Step 3 ✅ surface + # comment (which still names the /azp-gated legs a human must kick, so it is NOT + # redundant with 🎯) AND the Step 3.6 T3 🎯 readiness comment. At max:3 the shared + # bucket drained after ~1 draft flip and T3's atomicity pre-check then deferred + # every further mark-ready even while the mark-ready/add_labels buckets (also 3) + # sat idle; 6 lets ~3 draft flips (2 comments each) land per sweep, so the + # mark-ready:3 / add_labels:3 caps are actually reachable. + max: 6 target: "*" # Hard constraint (defense-in-depth): only comment on THIS workflow's own # ci-fix PRs — [ci-fix] title prefix AND agentic-workflows label. Mirrors the @@ -261,13 +270,51 @@ safe-outputs: # never the title, so disable title rewrites — this compiles to allow_title:false # and removes any ability to retitle an arbitrary PR. title: false - # NOTE: gh-aw v0.79.8 does NOT emit required-title-prefix/required-labels into + # NOTE: gh-aw v0.80.9 (the pinned compiler) does NOT emit required-title-prefix/required-labels into # the compiled config for update-pull-request (verified against the lock — it # silently drops them, unlike add-comment / push-to-pull-request-branch which # honor them). So which-PR scoping here relies on prompt Hard-Rule 6 + # min-integrity:approved; the residual is body-marker edits only (no code, no # merge, no close), capped at max:3. Revisit required-* if a future gh-aw # compiler emits them for this output. + mark-pull-request-as-ready-for-review: + # Flip a validated draft [ci-fix] PR to ready-for-review once the SPECIFIC test + # this PR was opened to fix is confirmed green in the PR's OWN CI (Step 3.6) — + # even when unrelated legs are red. The workflow is schedule/dispatch-triggered + # (no triggering-PR context), so target "*" lets the agent name the PR number it + # validated. This is a state transition ONLY: it never approves and never merges — + # a human still reviews and merges. + # PER-RUN total (not per-PR): a sweep may validate several PRs' target tests green. + max: 3 + target: "*" + # Hard constraint (defense-in-depth): only ever un-draft THIS workflow's own + # ci-fix PRs — [ci-fix] title prefix AND agentic-workflows label — so a confused + # or prompt-injected agent cannot mark an arbitrary PR ready. (If the v0.80.9 + # compiler silently drops these — as documented for update-pull-request above — + # the Step 3.6 preconditions + min-integrity:approved are the compensating scope + # controls; verify against the lock after compiling.) + required-title-prefix: "[ci-fix] " + required-labels: [agentic-workflows] + add-labels: + # Apply the p/0 priority label to a [ci-fix] PR at the exact moment the loop flips it + # from draft to ready-for-review (Step 3.6 T3) — so a validated, review-ready fix lands + # in the team's p/0 triage queue instead of sitting unseen in the draft backlog. Paired + # 1:1 with mark-pull-request-as-ready-for-review; target "*" lets the agent name the PR + # it just validated. + # allowed = HARD allowlist: the agent may ONLY ever add p/0, nothing else. This caps the + # blast radius of a confused/prompt-injected agent to exactly one benign priority label — + # it can never apply a downstream-triggering or destructive label. + allowed: [p/0] + # PER-RUN total (not per-PR): a sweep may mark several PRs ready in one run; each adds + # one label, so this matches the mark-ready per-run cap (3). + max: 3 + target: "*" + # Hard constraint (defense-in-depth): only ever label THIS workflow's own ci-fix PRs — + # [ci-fix] title prefix AND agentic-workflows label. (If the v0.80.9 compiler drops these + # — as documented for update-pull-request above — the Step 3.6 preconditions + + # min-integrity:approved + the allowed:[p/0] allowlist are the compensating controls.) + required-title-prefix: "[ci-fix] " + required-labels: [agentic-workflows] timeout-minutes: 90 @@ -339,33 +386,48 @@ through `safe-outputs`. 5. **One issue = one outcome per run.** Exactly one of: respond to a maintainer's change-request on the open PR (Track C, Step 3.5.R); open the first fix/help/ de-flake PR; advance an existing PR by one attempt (push a follow-up fix); - surface a validated-green PR for review; annotate an unrelated-flake red; wait + surface a validated-green PR for review; mark a target-validated draft PR + ready-for-review (Step 3.6 T3 — the terminal outcome that supersedes the same + run's surface/annotate precursor line), or record its `already-ready` + steady-state no-op; annotate an unrelated-flake red; wait (CI not yet settled); or a recorded skip (the dedicated needs-human PR is deferred — the attempt cap records a skip instead; see Step 6). Always prefer advancing or opening a PR over a skip when a non-mute diff is producible. 6. **All writes via `safe-outputs`.** Allowed outputs: `create_pull_request` (first attempt only), `push_to_pull_request_branch` (advance an existing PR by one attempt), `update_pull_request` (bump the attempt marker / refresh the - prior-attempts table), and `add_comment` (progress notes on the `[ci-fix]` PR - ONLY). NEVER comment on the tracking issue (issues are locked by + prior-attempts table), `add_comment` (progress notes on the `[ci-fix]` PR + ONLY), `mark_pull_request_as_ready_for_review` (flip a target-validated draft + `[ci-fix]` PR from draft to ready — Step 3.6 T3 only), and `add_labels` (add + ONLY the `p/0` label, ONLY on that same draft→ready transition — Step 3.6 T3). + NEVER comment on the tracking issue (issues are locked by `.github/workflows/ci-scan-lock-issues.yml`) — `add_comment` targets the PR. No `gh pr create`, no manual `git push`. **Defense-in-depth:** `add_comment` and `update_pull_request` use `target: "*"` (the agent supplies the PR number). - `add_comment` is now config-locked to `required-title-prefix: "[ci-fix] "` + + `add_comment`, `mark_pull_request_as_ready_for_review`, and `add_labels` are + config-locked to `required-title-prefix: "[ci-fix] "` + `required-labels: [agentic-workflows]` (hard-enforced by the handler, same as - `push_to_pull_request_branch`), so a comment can only ever land on THIS - workflow's own PRs. `update_pull_request` CANNOT be config-locked to a - title/label in gh-aw v0.79.8 (the compiler silently drops `required-*` for that - output — verified against the lock), so it keeps `title: false` (no retitles; - body-marker edits only) plus this prompt-level guard. Before emitting either, - VERIFY the target PR carries BOTH the `[ci-fix]` title prefix AND the - `agentic-workflows` label; never comment on or edit the body of any PR that - lacks both. + `push_to_pull_request_branch`), and `add_labels` additionally has an + `allowed: [p/0]` allowlist so it can ONLY ever add `p/0` — so these can only + ever land on THIS workflow's own PRs. `update_pull_request` CANNOT be + config-locked to a title/label in gh-aw v0.80.9 (the compiler silently drops + `required-*` for that output — verified against the lock), so it keeps + `title: false` (no retitles; body-marker edits only) plus this prompt-level + guard. Before emitting ANY of these, VERIFY the target PR carries BOTH the + `[ci-fix]` title prefix AND the `agentic-workflows` label; never comment on, + edit, un-draft, or label any PR that lacks both. 7. **Per-run safe-output caps (per RUN, not per PR).** Each output type is capped per run: `create_pull_request` 3, `push_to_pull_request_branch` 3, - `add_comment` 3, `update_pull_request` 3. Note an ADVANCE spends one push **and** - one update **and** one comment, so ≤ 3 advances/run; surfacing a green or - annotating a flake spends one comment. When a bucket is exhausted, do NOT keep + `add_comment` 6, `update_pull_request` 3, `mark_pull_request_as_ready_for_review` + 3, `add_labels` 3 (counts LABELS, not calls). Note an ADVANCE spends one push + **and** one update **and** one comment, so ≤ 3 advances/run; surfacing a green or + annotating a flake spends one comment; a **mark-ready (Step 3.6 T3)** of a + still-draft PR spends TWO comments (the Step 3 ✅ surface **and** the T3 🎯 + readiness note) **and** one mark-ready **and** one label as an ALL-OR-NOTHING set + (T3 pre-checks those buckets and defers the whole PR if any is exhausted — never + mark a PR ready without its 🎯 audit comment). `add_comment` is therefore sized 6 + (not 3) so ~3 draft flips, at 2 comments each, can land in one sweep instead of the + shared comment bucket starving the otherwise-idle mark-ready/add_labels buckets. When a bucket is exhausted, do NOT keep emitting (extras are silently dropped) — record `skipped: per-run cap reached; deferring PR # to next cycle` for each remaining PR so the drop is a deliberate, logged decision. The continuous loop picks the deferred PRs up next @@ -409,8 +471,9 @@ For every open tracking issue in scope, converge on exactly one outcome: | Open first help-wanted draft `[ci-fix]` PR | No open PR yet; a plausible candidate exists but cannot be runner-validated (device/UI tests) (attempt 1) | | Open first de-flake draft `[ci-fix]` PR | No open PR yet; failure is intermittent (green-on-retry) from a genuine test-quality defect; a deterministic-synchronization fix is producible (attempt 1, Step 4.7 bucket b) | | Advance an existing PR (push attempt N+1) | Open PR's own CI settled red *because of the fix itself*, no human engaged, marker < 10 — push a NEW distinct fix onto the same branch (Step 3.5 → 5.6) | -| Surface a validated-green PR | Open PR's own CI settled green — comment "validated, attempt N/10; ready for review" and do NOT advance (Step 3.5) | -| Annotate an unrelated-flake red | Open PR is red only on baseline-flake / unrelated legs — comment which leg needs a re-run; do NOT burn an attempt (Step 3.5) | +| Surface a validated-green PR | Open PR's own CI settled green — comment "validated, attempt N/10; ready for review" and do NOT advance (Step 3.5), then mark the draft PR ready for review once the fixed test is confirmed green (Step 3.6) | +| Annotate an unrelated-flake red | Open PR is red only on baseline-flake / unrelated legs — comment which leg needs a re-run; do NOT burn an attempt (Step 3.5); if the SPECIFIC fixed test is confirmed green on that build, still mark the draft PR ready for review (Step 3.6) | +| Mark a validated PR ready for review | The specific test this PR fixed is confirmed green in the PR's own CI (even if unrelated legs are red) — comment the target-test result and transition the draft PR to ready for review; never approve or merge (Step 3.6) | | Wait | Open PR's CI is pending / not yet settled on the current head SHA — do nothing this cycle (Step 3.5) | | Hand-off skip (attempt cap) | Marker == 10 and the signature still reproduces — stop and defer to humans; the dedicated `[ci-fix][needs-human]` PR is planned but currently deferred (Step 6) | | Recorded skip | Visual-regression, human engaged, already-handled, fixed-in-latest-build, infra-flake, out-of-bounds, only-mute-available, or no novel approach producible | @@ -685,14 +748,34 @@ Run these gates in order — the FIRST that fires decides this cycle's outcome: 1. **Human engaged.** If `C.humanEngaged` → `skipped: human engaged on PR #; deferring` and stop. Never fight a human reviewer — the intentional hand-off boundary still holds the moment a person touches the PR. -2. **CI not settled / unknown / incomplete prefetch.** If `C.checksSettled == false` - OR `C.overallConclusion` is `pending`, `neutral`, or `unknown`, OR - `C.dataComplete == false` → the fix's CI has not finished on the current head SHA - (`C.headSha`), or the prefetch could not fully read this PR (a partial read can - understate `humanEngaged` / `attempt`). `skipped: PR # CI pending / prefetch - incomplete on ; waiting` and stop. *(Round 1: this is where a - maintainer `/azp run` is awaited — the `/azp`-gated uitests/devicetests legs will - not have run until a human kicks them.)* +2. **CI not settled — but validate the target test first (target-focused readiness).** + If `C.checksSettled == false` OR `C.overallConclusion` is `pending`, `neutral`, or + `unknown`, OR `C.dataComplete == false` → the *overall* build has not finished on the + current head SHA (`C.headSha`), or the prefetch could not fully read this PR (a partial + read can understate `humanEngaged` / `attempt`). Unrelated legs still draining must NOT + keep an already-proven fix parked as a draft, so before waiting, try the target-focused + fast-path: + - **2a — Target-green fast-path (draft `[ci-fix]` PRs only).** If `C.dataComplete == + true` AND the Step 3.6 preconditions hold (draft PR, `[ci-fix] ` title prefix AND the + `agentic-workflows` label), run Step 3.6 **T1–T2** against `C.headSha` now: identify + the target test(s) and read the PR's OWN build **timeline / per-leg status** (anonymous + `_apis/build` — never `_apis/test`, per Hard-Rule 8). If EVERY target test is + **VALIDATED-GREEN** (per T2's all-platform definition below — the target's category leg + is `succeeded` on every platform that runs it, failed on none, and NO target platform + family the pipeline covers is still pending/unconcluded, per T2's "no platform left + unverified" rule; a platform that simply has no leg for the target's category — the test + doesn't run there — is fine, not a gap) AND every leg in + `C.failedLegs` that has ALREADY concluded classifies as **unrelated flake** (Step 4 / + Step 4.7 method — the fix is implicated in NO completed red), then the fix is proven + regardless of unrelated *pending* legs → run **Step 3.6 T3** (mark ready + 🎯 comment) + and stop. This early-out NEVER advances an attempt and NEVER acts on a red that could + be the fix's fault. + - **2b — Otherwise WAIT.** If the target test has not yet executed (pending/absent on + its leg), or a completed red is (or may be) caused by the fix, or `C.dataComplete == + false`, or this PR has no identifiable target test → `skipped: PR # CI pending / + target not yet validated on ; waiting` and stop. *(Round 1: this is where a + maintainer `/azp run` is awaited — the `/azp`-gated uitests/devicetests legs will not + have run until a human kicks them.)* 3. **Green → surface for review.** If `C.overallConclusion == "success"`: the checks that RAN are green. **Primary-gate check first:** the deterministic `success` verdict only certifies "at least one green check, nothing failing or @@ -704,18 +787,31 @@ Run these gates in order — the FIRST that fires decides this cycle's outcome: # primary CI gate (maui-pr) not green on ; waiting` and stop — do NOT surface. (The `/azp`-gated `maui-pr-uitests` (def 313) / `maui-pr-devicetests` (def 314) legs MAY still be un-run — that is expected and is named below, not a - reason to withhold the surface.) **Idempotency:** scan the PR's existing comments - for a prior bot `✅ … validated … on ` note for THIS head SHA — if one - already exists, record `already-surfaced PR # (head )` and stop - WITHOUT re-commenting (re-surfacing the same green every tick is noise and burns - the per-run comment budget other PRs need). Otherwise resolve the attempt number from - `C.effectiveAttempt` (the authoritative max(marker, bot-commit) counter; omit the - number only if it is somehow indeterminate). **Dry-run gate (Step 0):** if `dry_run == "true"`, do NOT emit any comment — instead print the intended `✅` validated-green body to the run log, tally `dry-run: would-surface-green PR #`, and stop. Otherwise `add_comment` on PR #: `✅ Attempt /10 validated - — the fix's CI is green on . Ready for human review.` - Name any `/azp`-gated legs (uitests def 313 / devicetests def 314) that have not - run and still need a maintainer `/azp run`. Do NOT advance. Record - `surfaced-green PR # (attempt /10)` and stop. *(This directly - attacks the real bottleneck — no reviews — so it is the highest-value outcome.)* + reason to withhold the surface.) **Comment idempotency + dry-run suppress the ✅ + comment ONLY — neither skips Step 3.6.** Scan the PR's existing comments for a prior + bot `✅ … validated … on ` note for THIS head SHA; if one exists, set + `SKIP_SURFACE_COMMENT = true`. Resolve the attempt number from `C.effectiveAttempt` + (the authoritative max(marker, bot-commit) counter; omit the number only if it is + somehow indeterminate). Then post the surface comment UNLESS suppressed: + - if `dry_run == "true"`: do NOT emit — print the intended `✅` validated-green body to + the run log and tally `dry-run: would-surface-green PR #`; + - else if `SKIP_SURFACE_COMMENT`: do NOT re-post — record `already-surfaced PR # + (head )` (re-surfacing the same green every tick is noise and burns the + per-run comment budget other PRs need); + - else `add_comment` on PR #: `✅ Attempt /10 validated — the fix's CI is + green on .` naming any `/azp`-gated legs (uitests def 313 / devicetests def + 314) that have not run and still need a maintainer `/azp run`; record `surfaced-green + PR # (attempt /10)`. (Do NOT assert "ready for human review" on a PR that + is still a draft — Step 3.6, not this comment, owns the draft→ready flip and posts its + own 🎯 announcement when it fires.) + Do NOT advance. Then — in ALL of the above cases — run **Step 3.6** (target-test + readiness gate) for this PR before stopping: its `/azp`-gated target legs may conclude + green on this SAME head SHA on a LATER sweep (an `/azp run` adds no commit), and Step + 3.6 is the ONLY place the draft→ready flip happens, so it MUST re-evaluate every sweep — + never `stop` here before it. *(This directly attacks the real bottleneck — no reviews — + so it is the highest-value outcome.)* 4. **Red → classify caused-by-fix vs unrelated-flake.** If `C.overallConclusion == "failure"`, analyze the PR's OWN failing build (NOT `main`): find the AzDO `maui-pr` build for this PR (filter builds by `branchName=refs/pull//merge`, @@ -723,18 +819,26 @@ Run these gates in order — the FIRST that fires decides this cycle's outcome: method and Step 4.7 flake buckets to `C.failedLegs`. - **Unrelated flake only** (every failed leg is known-flaky / infra / a pre-existing baseline red NOT introduced by the fix): do NOT burn an attempt. - **Idempotency first:** scan the PR's existing comments for a prior bot - `♻️ … unrelated flake … on ` note for THIS head SHA — if one already - exists, record `already-annotated-flake PR # (head )` and stop - WITHOUT re-commenting (re-annotating the same flake every 12h sweep is noise and - burns the per-run comment budget other PRs need). Otherwise resolve the attempt - number from `C.effectiveAttempt` (the authoritative max(marker, bot-commit) - counter), omitting the `Attempt /10:` prefix only if it is somehow - indeterminate. **Dry-run gate (Step 0):** if `dry_run == "true"`, do NOT emit any comment — instead print the intended `♻️` unrelated-flake body to the run log, tally `dry-run: would-annotate-flake PR #`, and stop. Otherwise `add_comment` on PR #: `♻️ Attempt /10: red is - unrelated flake on leg(s) () on ; the fix itself is not - implicated. A maintainer re-run (/azp run ) should clear it.` Record - `annotated-flake PR # (head )` and stop. *(Round 1: human re-runs; - Round 2: auto re-trigger.)* + **Comment idempotency + dry-run suppress the ♻️ comment ONLY — neither skips Step + 3.6.** Scan the PR's existing comments for a prior bot `♻️ … unrelated flake … on + ` note for THIS head SHA; if one exists, set `SKIP_FLAKE_COMMENT = true`. + Resolve the attempt number from `C.effectiveAttempt` (the authoritative max(marker, + bot-commit) counter), omitting the `Attempt /10:` prefix only if it is + somehow indeterminate. Then post the flake note UNLESS suppressed: + - if `dry_run == "true"`: do NOT emit — print the intended `♻️` unrelated-flake body + to the run log and tally `dry-run: would-annotate-flake PR #`; + - else if `SKIP_FLAKE_COMMENT`: do NOT re-post — record `already-annotated-flake PR + # (head )` (re-annotating the same flake every 12h sweep is noise and + burns the per-run comment budget other PRs need); + - else `add_comment` on PR #: `♻️ Attempt /10: red is unrelated flake on + leg(s) () on ; the fix itself is not implicated. A + maintainer re-run (/azp run ) should clear it.` record `annotated-flake + PR # (head )`. + Then — in ALL of the above cases — run **Step 3.6** (target-test readiness gate) for + this PR before stopping: its `/azp`-gated target legs may conclude green on this SAME + head SHA on a LATER sweep, and Step 3.6 is the ONLY place the draft→ready flip + happens, so it MUST re-evaluate every sweep — never `stop` here before it. *(Round 1: + human re-runs; Round 2: auto re-trigger.)* - **Caused by the fix** (a failed leg still matches the original target signature, or the fix introduced a NEW failure): advance an attempt. - **Attempt count.** `attempt = C.effectiveAttempt` — the authoritative @@ -752,6 +856,188 @@ Run these gates in order — the FIRST that fires decides this cycle's outcome: from every prior commit on this branch**, emitted via the Step 5.6 ADVANCE path (push onto the same PR). +#### Step 3.6 — Target-test verification & mark-ready gate + +Reached from the Step 3 **green-surface** branch, the Step 4 **unrelated-flake** branch +(AFTER that branch has posted its comment), and the Step 3.5 **gate-2 target-focused +fast-path** (2a — while unrelated legs are still pending). Purpose: confirm the SPECIFIC +test(s) this PR was opened to fix now PASS in the PR's own CI, and — only when they do +— transition the draft `[ci-fix]` PR to **ready for review** so a maintainer sees a +validated fix instead of a draft. This is the ONLY place the loop flips draft→ready; it +is a state transition, **never** an approval or a merge (a human still reviews and +merges). Overall red on *unrelated* legs must NOT gate readiness — we validate the fix, +not the base branch's flakiness. + +**Preconditions** (ALL must hold; otherwise record `skipped: readiness N/A PR # +()` and stop this gate): +- `C.isDraft == true` — if the PR is already ready-for-review, the draft→ready + transition is already done; do NOT re-mark **and do NOT re-apply `p/0`**. `p/0` is applied + exactly ONCE, atomically with the draft→ready flip (T3 — all-or-nothing with the 🎯 audit + comment + mark-ready), so an already-ready loop PR that is MISSING `p/0` has almost + certainly had it **removed by a maintainer** de-prioritizing the PR — a pure triage action + the human-engagement guard (which inspects comments/reviews/commits, NOT label changes) + cannot see. Re-adding `p/0` every sweep would fight that maintainer indefinitely, violating + the loop's "never override a human" contract. So the loop does NOT reconcile the label: + record `already-ready PR #` and stop this gate. (Trade-off: on the rare occasion a + transient API error drops `p/0` at flip time *after* the T3 atomic pre-check passed, it is + not auto-re-added — but the PR is still ready-for-review with its 🎯 audit comment, and + re-adding `p/0` is a trivial manual action; that is strictly preferable to steam-rolling a + maintainer's deliberate de-prioritization.) +- The PR is unmistakably THIS workflow's own: `[ci-fix] ` title prefix AND the + `agentic-workflows` label (mirrors the safe-output lock; the compensating scope + control if the v0.80.9 compiler drops the declarative required-*). +- You reached this gate from Step 3 (green), Step 4 (**unrelated-flake**), or the Step 3.5 + gate-2 **target-focused fast-path** (2a) — i.e. the fix is NOT implicated in any red that + has concluded. If Step 4 classified a red as **caused by the fix** (ADVANCE), do NOT run + this gate — advance the attempt instead. + +**T1 — Identify the target test(s).** From the `[ci-scan]` issue signature the fix +addresses (and the PR's own diff), extract the fully-qualified test method name(s) the +fix targets — e.g. `SafeAreaShouldWorkOnAllShellTabs`. For a de-flake it is the +de-flaked test; for a product fix it is the originally-failing test(s). If NO specific +test can be identified (e.g. a product build-break rather than a test failure), this is a +**build-only fix** — there is no single test to validate cross-platform, so validate at +**whole-build** granularity rather than stopping (Step 3 only *comments*; Step 3.6 is the +sole draft→ready flip, so a build-only fix is undrafted HERE or not at all). We only reach +this gate from Step 3 (green) / Step 4 (unrelated-flake) / gate-2a, so the fix is not +implicated in any concluded red; additionally require, via the T2 timeline method on +`C.headSha`, that the primary build pipeline `maui-pr` (def 302) has CONCLUDED with EVERY +one of its platform build legs `succeeded`/`completed` and NONE failed/canceled or still +pending — the build is green on **every** platform, not just the originally-broken one (the +cross-platform guard, applied to the build instead of a test). A build-only fix is undrafted +ONLY on the strength of the auto-running `maui-pr` (def 302) whole-build green, which the +loop CAN observe: if the PR's `[ci-scan]` issue signature or its own diff indicates the +ORIGINATING failure was in a `/azp`-gated pipeline (`maui-pr-uitests` def 313 or +`maui-pr-devicetests` def 314) rather than `maui-pr` (def 302), do NOT undraft on +`maui-pr`-green alone — those pipelines do not auto-run on this PR (GITHUB_TOKEN cannot +trigger them), so a green `maui-pr` build is NOT evidence the gated build break is fixed; +record `skipped: build-only fix PR # targets gated pipeline () not run — +deferring to human` and stop this gate. Otherwise, set `TARGET := "the maui-pr build +(build-only fix — no single target test)"` and proceed to **T3** to mark ready. If any `maui-pr` build leg is still unconcluded, record `skipped: build-only fix PR +# not yet whole-build green (leg(s) pending)` and stop this gate WITHOUT marking +ready (a green subset is not enough — a still-pending build leg could yet fail). + +**T2 — Verify the target test's platform legs are green (anonymous, leg-level).** Per-test +outcomes come from the AzDO **test-results** API (`_apis/test/...`), which is **NOT reachable +anonymously — Hard-Rule 8**: those endpoints 302-redirect to a sign-in page on this runner, so +a `curl` returns an HTML redirect (not JSON) and every target test would look "not executed". +Validate instead at **leg granularity** using ONLY the anonymous `_apis/build/...` timeline +(Hard-Rule 8) — a `succeeded` category leg means every test in that category **that actually +ran** passed on that platform. This is a strong bar **only if the target test genuinely +executes**: a job can go green while one specific test is skipped at runtime (`[Ignore]`, +`Assert.Ignore()`, `Assert.Inconclusive()`, a `Skip=`/conditional `[Fact]`, an `#if`-out, or +an early `return` before the asserts). So before trusting a green leg as proof the target +passed, **confirm from the PR's OWN diff that the fix does NOT skip, ignore, disable, or +short-circuit the target test** (it adds no `[Ignore]`/`[Explicit]`, `Assert.Ignore`/ +`Assert.Inconclusive`, `Skip=`, category-exclusion, `#if`-out, or early `return` around the +target). If the fix could cause the target to be runtime-skipped rather than genuinely pass, +leg-level green is NOT sufficient evidence — record `skipped: target test may be +runtime-skipped by this fix (diff adds ); leg-green insufficient, deferring to human +on PR #` and stop this gate WITHOUT marking ready. Otherwise (the target genuinely runs +and asserts, as a de-flake or product fix does) a green category leg cannot hide a +target-test failure, so treat it as authoritative. + +Map the target test to the CI leg(s) that run it. A leg name encodes platform + test category +(e.g. `Android UITests SafeAreaEdges,Shadow`, `iOS UITests SafeAreaEdges,Shadow`, `macOS +UITests SafeAreaEdges,Shadow`, `Windows UITests SafeAreaEdges,Shadow`). Determine the target +test's UI-test category — its `[Category(UITestCategories.X)]` in the test/HostApp source, +visible in the PR diff or the test file — to know which leg-name substring identifies its legs; +for a device test, the per-platform device-test legs. + +Using the SAME build-discovery as Step 4 (filter AzDO builds by `branchName=refs/pull//merge` +or `sourceVersion == C.headSha`), read each build's **timeline** on `C.headSha` for the +pipeline(s) that RUN the target test — `maui-pr` (def 302) for unit/integration tests, +`maui-pr-uitests` (def 313) for Appium UI tests, `maui-pr-devicetests` (def 314) for device +tests: + +```bash +ORG=dnceng-public; PROJ=public; BUILD= +# Anonymous + Hard-Rule-8-compliant. NEVER call _apis/test/... (it 302-redirects to sign-in). +# Each type=="Job" record is one platform leg: result (succeeded/failed/canceled/null), +# state (completed/inProgress/pending), name (encodes " UITests "): +curl -s "https://dev.azure.com/$ORG/$PROJ/_apis/build/builds/$BUILD/timeline?api-version=7.1" \ + | tee /tmp/gh-aw/agent/timeline_${BUILD}.json \ + | jq -r '.records[] | select(.type=="Job") | "\(.result)\t\(.state)\t\(.name)"' +``` + +(The prefetch already exposes per-leg status in `C.failedLegs` / the checks context, and +`gh pr checks ` lists the same per-leg rows with platform+category in the name — use +whichever is handy; the build timeline is the authoritative cross-check on the exact build.) + +Treat a target test as **VALIDATED-GREEN** only if, on `C.headSha`, the leg that runs its +category is green on **every platform it runs on** — a fix that repairs one platform must not +silently regress the same test's leg on another, so a green leg on the originally-red platform +alone is NOT enough. Across the target pipeline's platform legs (for a UI test: the Android, +iOS, Windows, and macOS/MacCatalyst legs running the target's category; for a device test: each +device-test platform), require ALL of: +- the target's category leg is `result == "succeeded"` **and** `state == "completed"` on + **every** platform that runs it, AND +- that leg is `failed` / `canceled` / aborted on **NO** platform, AND +- **no platform is left unverified.** For each platform family the target pipeline covers, that + platform's category leg must have CONCLUDED (`state == "completed"`) on `C.headSha`. If a + platform simply has no leg for the target's category, the test does not run there — that is + fine, not a gap. But if a platform's category leg has **not concluded** (`state` is + `inProgress` / pending, or an `/azp`-gated `maui-pr-uitests` / `maui-pr-devicetests` leg that + has not been kicked), the test's status on that platform is UNKNOWN → the fix is NOT yet + validated across platforms: record `skipped: target test green on but + not yet verified on (leg(s) pending / need /azp run) on PR #` + and stop this gate WITHOUT marking ready. + +A target test whose category leg never concluded on ANY platform (**not executed** anywhere — +e.g. its `/azp`-gated pipeline has not been kicked) is likewise NOT validated: record `skipped: +target test not yet executed on PR # ( not run — needs /azp run)` and stop +this gate WITHOUT marking ready. Do NOT overclaim — a green *sibling* leg (a different category +on the same platform) is not the target's leg, and a green leg on one platform is not a pass on +the others. + +**T3 — Mark ready + report.** If EVERY target test is VALIDATED-GREEN. The 🎯 comment, +the mark-ready, and the `p/0` label are THREE SEPARATE safe-outputs — the comment +existing does NOT prove the mark-ready took effect, so they are tracked independently: + +- **Comment idempotency (dup-suppress only — never gates the mark-ready).** We only reach + T3 while `C.isDraft == true` (the precondition stops an already-ready PR before here). So + if a prior bot comment for THIS head that carries the leading `🎯` anchor AND the + contiguous phrase `validated green on ` already exists (BOTH T3 variants — + `🎯 Target test validated green on ` and the build-only `🎯 Build validated green on + ` — contain that exact contiguous phrase, so this recognizes both; keying on + `target test` alone would instead miss the build-only comment and re-post it every sweep. + **Require the `🎯` anchor** so the match can NEVER be loosened to a gapped + `validated…green…on` form that would false-positive on the Step 3 `✅ … validated — the + fix's CI is green on ` surface comment and wrongly suppress the audit 🎯), the comment + landed on an earlier sweep but the **mark-ready did NOT take + effect** (the PR is still a draft) — set `SUPPRESS_COMMENT = true` (do not re-post the + duplicate 🎯 comment) but STILL complete the mark-ready + label below. Never treat the + comment's existence as "already marked ready" while the PR is still a draft. Record this + case as `re-marking PR # (🎯 present; mark-ready did not take on a prior sweep)`. +- **Atomicity budget pre-check (Hard-Rule 7).** Determine the outputs this gate will emit: + `mark_pull_request_as_ready_for_review` + `add_labels` always, plus `add_comment` unless + `SUPPRESS_COMMENT`. Verify a free per-run slot remains in EACH bucket you are about to + use. If ANY required bucket is exhausted, emit NONE of them — record `skipped: per-run + cap reached; deferring mark-ready PR # to next cycle` and stop this gate. Never mark a + PR ready (or label it) unless its 🎯 audit comment is GUARANTEED to exist for this exact + head SHA — either posted in this same sweep, or (in the `SUPPRESS_COMMENT` re-marking case) + already present from a prior sweep. The outputs you DO emit either all land or all defer + together; the audit trail must never be absent, but it is NOT re-posted when it already exists. +- **Dry-run gate (Step 0):** if `dry_run == "true"`, emit NOTHING — print the intended + readiness comment and "would mark ready + add p/0" to the run log, tally `dry-run: + would-mark-ready PR #`, and stop. +- Otherwise emit for THIS PR number ``: + 1. `add_comment` (ONLY if not `SUPPRESS_COMMENT`): `🎯 Target test validated green on + — passed on ALL platforms it runs on (, + buildId ). — not + caused by this fix."> Transitioning this PR from draft to ready for review and adding + `p/0`; a maintainer still reviews and merges.` (For a **build-only fix** — the T1 + whole-build fallback, no single target test — phrase the first clause as `🎯 Build + validated green on — the maui-pr build passed on ALL platforms (buildId + ).` instead of naming a test.) + 2. `mark_pull_request_as_ready_for_review` with `reason:` a one-line justification + naming the validated test(s) and ``. + 3. `add_labels` with `labels: ["p/0"]` for PR # — put the now-review-ready fix into + the team's p/0 priority queue so it is triaged, not lost in the draft backlog. (If + the PR somehow already carries `p/0`, this is a harmless no-op.) +- Record `marked-ready PR # (target green on ALL platforms on , + labeled p/0)` and stop. + #### Step 3.5.R — Maintainer change-request response (Track C) Reached from Step 3.5 gate 0. Goal: when an **eligible human reviewer** (Hard-Rule @@ -1383,7 +1669,19 @@ Filed by [`ci-status-fix`](https://github.com/dotnet/maui/blob/main/.github/work ### Step 8 — Per-issue tally + end-of-run summary -Per issue, append one outcome line to `/tmp/gh-aw/agent/coverage.txt`: +Per issue, append **one** outcome line to `/tmp/gh-aw/agent/coverage.txt` — the +terminal outcome for the cycle. When one cycle produces a chained pair — a PR gets a +same-cycle precursor line and **then** a Step 3.6 readiness line (`marked-ready` on the +draft→ready flip, or `already-ready` on a later steady-state sweep of an already-ready +PR) — record ONLY the terminal Step 3.6 readiness line: it supersedes ANY same-cycle +non-readiness precursor for that PR, so an aggregator keying on one-line-per-issue never +double-counts. The precursor may be a Step 3 green line (`surfaced-green` on the first ✅, +or `already-surfaced` when it already exists) OR — when an unrelated-flake red and a +target-green coincide — a Step 4 `annotated-flake` line; either way the readiness line is +terminal, and the superseded precursor's signal survives in the PR's 🎯/♻️ comment so no +information is lost. This covers the flip cycle (`surfaced-green` → `marked-ready`), the +post-flip steady state (`already-surfaced` → `already-ready`), and the flake-coincident +flip (`annotated-flake` → `marked-ready`): ``` # main attempt- @@ -1394,8 +1692,14 @@ Per issue, append one outcome line to `/tmp/gh-aw/agent/coverage.txt`: `advance-PR # attempt /10` (ADVANCE mode — new commit pushed to the existing PR), `surfaced-green PR #` (fix's own CI went green; commented for review, did not advance), `annotated-flake PR #` (red was unrelated flake; -commented, attempt NOT burned), `waiting PR #` (CI not settled yet), -`dry-run: would-`, `skipped: `. (The +commented, attempt NOT burned), `marked-ready PR #` (the specific fixed test +was confirmed green in the PR's own CI; draft flipped to ready for review), +`already-surfaced PR #` (the fix's ✅ green comment already existed this sweep; +re-post suppressed — superseded by the Step 3.6 readiness line when the PR also flips +ready that cycle), `already-ready PR #` (terminal steady state — the PR was flipped +ready on a prior cycle and remains green on an unchanged head; no action taken, +supersedes `already-surfaced`), `waiting PR #` (CI not settled yet), +`dry-run: would-`, `skipped: `. (The `needs-human-PR` outcome is reserved for the deferred hand-off — Step 6 currently records a skip instead, so it is not emitted.)
` + and stop this gate WITHOUT marking ready. + +A target test whose category leg never concluded on ANY platform (**not executed** anywhere — +e.g. its `/azp`-gated pipeline has not been kicked) is likewise NOT validated: record `skipped: +target test not yet executed on PR # ( not run — needs /azp run)` and stop +this gate WITHOUT marking ready. Do NOT overclaim — a green *sibling* leg (a different category +on the same platform) is not the target's leg, and a green leg on one platform is not a pass on +the others. + +**T3 — Mark ready + report.** If EVERY target test is VALIDATED-GREEN. The 🎯 comment, +the mark-ready, and the `p/0` label are THREE SEPARATE safe-outputs — the comment +existing does NOT prove the mark-ready took effect, so they are tracked independently: + +- **Comment idempotency (dup-suppress only — never gates the mark-ready).** We only reach + T3 while `C.isDraft == true` (the precondition stops an already-ready PR before here). So + if a prior bot comment for THIS head that carries the leading `🎯` anchor AND the + contiguous phrase `validated green on ` already exists (BOTH T3 variants — + `🎯 Target test validated green on ` and the build-only `🎯 Build validated green on + ` — contain that exact contiguous phrase, so this recognizes both; keying on + `target test` alone would instead miss the build-only comment and re-post it every sweep. + **Require the `🎯` anchor** so the match can NEVER be loosened to a gapped + `validated…green…on` form that would false-positive on the Step 3 `✅ … validated — the + fix's CI is green on ` surface comment and wrongly suppress the audit 🎯), the comment + landed on an earlier sweep but the **mark-ready did NOT take + effect** (the PR is still a draft) — set `SUPPRESS_COMMENT = true` (do not re-post the + duplicate 🎯 comment) but STILL complete the mark-ready + label below. Never treat the + comment's existence as "already marked ready" while the PR is still a draft. Record this + case as `re-marking PR # (🎯 present; mark-ready did not take on a prior sweep)`. +- **Atomicity budget pre-check (Hard-Rule 7).** Determine the outputs this gate will emit: + `mark_pull_request_as_ready_for_review` + `add_labels` always, plus `add_comment` unless + `SUPPRESS_COMMENT`. Verify a free per-run slot remains in EACH bucket you are about to + use. If ANY required bucket is exhausted, emit NONE of them — record `skipped: per-run + cap reached; deferring mark-ready PR # to next cycle` and stop this gate. Never mark a + PR ready (or label it) unless its 🎯 audit comment is GUARANTEED to exist for this exact + head SHA — either posted in this same sweep, or (in the `SUPPRESS_COMMENT` re-marking case) + already present from a prior sweep. The outputs you DO emit either all land or all defer + together; the audit trail must never be absent, but it is NOT re-posted when it already exists. +- **Dry-run gate (Step 0):** if `dry_run == "true"`, emit NOTHING — print the intended + readiness comment and "would mark ready + add p/0" to the run log, tally `dry-run: + would-mark-ready PR #`, and stop. +- Otherwise emit for THIS PR number ``: + 1. `add_comment` (ONLY if not `SUPPRESS_COMMENT`): `🎯 Target test validated green on + — passed on ALL platforms it runs on (, + buildId ). — not + caused by this fix."> Transitioning this PR from draft to ready for review and adding + `p/0`; a maintainer still reviews and merges.` (For a **build-only fix** — the T1 + whole-build fallback, no single target test — phrase the first clause as `🎯 Build + validated green on — the maui-pr build passed on ALL platforms (buildId + ).` instead of naming a test.) + 2. `mark_pull_request_as_ready_for_review` with `reason:` a one-line justification + naming the validated test(s) and ``. + 3. `add_labels` with `labels: ["p/0"]` for PR # — put the now-review-ready fix into + the team's p/0 priority queue so it is triaged, not lost in the draft backlog. (If + the PR somehow already carries `p/0`, this is a harmless no-op.) +- Record `marked-ready PR # (target green on ALL platforms on , + labeled p/0)` and stop. + #### Step 3.5.R — Maintainer change-request response (Track C) Reached from Step 3.5 gate 0. Goal: when an **eligible human reviewer** (Hard-Rule @@ -1395,7 +1681,19 @@ Filed by [`ci-status-fix-net11`](https://github.com/dotnet/maui/blob/main/.githu ### Step 8 — Per-issue tally + end-of-run summary -Per issue, append one outcome line to `/tmp/gh-aw/agent/coverage.txt`: +Per issue, append **one** outcome line to `/tmp/gh-aw/agent/coverage.txt` — the +terminal outcome for the cycle. When one cycle produces a chained pair — a PR gets a +same-cycle precursor line and **then** a Step 3.6 readiness line (`marked-ready` on the +draft→ready flip, or `already-ready` on a later steady-state sweep of an already-ready +PR) — record ONLY the terminal Step 3.6 readiness line: it supersedes ANY same-cycle +non-readiness precursor for that PR, so an aggregator keying on one-line-per-issue never +double-counts. The precursor may be a Step 3 green line (`surfaced-green` on the first ✅, +or `already-surfaced` when it already exists) OR — when an unrelated-flake red and a +target-green coincide — a Step 4 `annotated-flake` line; either way the readiness line is +terminal, and the superseded precursor's signal survives in the PR's 🎯/♻️ comment so no +information is lost. This covers the flip cycle (`surfaced-green` → `marked-ready`), the +post-flip steady state (`already-surfaced` → `already-ready`), and the flake-coincident +flip (`annotated-flake` → `marked-ready`): ``` # net11.0 attempt- @@ -1406,8 +1704,14 @@ Per issue, append one outcome line to `/tmp/gh-aw/agent/coverage.txt`: `advance-PR # attempt /10` (ADVANCE mode — new commit pushed to the existing PR), `surfaced-green PR #` (fix's own CI went green; commented for review, did not advance), `annotated-flake PR #` (red was unrelated flake; -commented, attempt NOT burned), `waiting PR #` (CI not settled yet), -`dry-run: would-`, `skipped: `. (The +commented, attempt NOT burned), `marked-ready PR #` (the specific fixed test +was confirmed green in the PR's own CI; draft flipped to ready for review), +`already-surfaced PR #` (the fix's ✅ green comment already existed this sweep; +re-post suppressed — superseded by the Step 3.6 readiness line when the PR also flips +ready that cycle), `already-ready PR #` (terminal steady state — the PR was flipped +ready on a prior cycle and remains green on an unchanged head; no action taken, +supersedes `already-surfaced`), `waiting PR #` (CI not settled yet), +`dry-run: would-`, `skipped: `. (The `needs-human-PR` outcome is reserved for the deferred hand-off — Step 6 currently records a skip instead, so it is not emitted.) diff --git a/.github/workflows/ci-status-fix.lock.yml b/.github/workflows/ci-status-fix.lock.yml index 93dc5969cb0d..65511f2b3259 100644 --- a/.github/workflows/ci-status-fix.lock.yml +++ b/.github/workflows/ci-status-fix.lock.yml @@ -1,4 +1,4 @@ -# gh-aw-metadata: {"schema_version":"v4","frontmatter_hash":"f014161967589ebcaf1ac5ec6f3c88ee5a4388d28b55a07dc36c45b7b2aebab6","body_hash":"86c6b2d4c3df13da14c6a3c8b211381fcc9f97bb1951ff7b80588a2c5338e00c","compiler_version":"v0.80.9","strict":true,"agent_id":"copilot","agent_model":"claude-opus-4.8","engine_versions":{"copilot":"1.0.63"}} +# gh-aw-metadata: {"schema_version":"v4","frontmatter_hash":"b0ebf8a2f179900cfe3638e994b27a43cec52bdbdd2331dc1bd4d65762fa2d6b","body_hash":"1825e2b1f7c7bd88241bf37580655489360f9bebc806edd00837a52ce4ad83a0","compiler_version":"v0.80.9","strict":true,"agent_id":"copilot","agent_model":"claude-opus-4.8","engine_versions":{"copilot":"1.0.63"}} # gh-aw-manifest: {"version":1,"secrets":["COPILOT_PAT_0","COPILOT_PAT_1","COPILOT_PAT_2","COPILOT_PAT_3","COPILOT_PAT_4","COPILOT_PAT_5","COPILOT_PAT_6","COPILOT_PAT_7","COPILOT_PAT_8","COPILOT_PAT_9","GH_AW_CI_TRIGGER_TOKEN","GH_AW_GITHUB_MCP_SERVER_TOKEN","GH_AW_GITHUB_TOKEN","GITHUB_TOKEN"],"actions":[{"repo":"actions/checkout","sha":"34e114876b0b11c390a56381ad16ebd13914f8d5","version":"v4"},{"repo":"actions/checkout","sha":"9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0","version":"v7.0.0"},{"repo":"actions/checkout","sha":"de0fac2e4500dabe0009e67214ff5f5447ce83dd","version":"v6.0.2"},{"repo":"actions/download-artifact","sha":"3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c","version":"v8.0.1"},{"repo":"actions/github-script","sha":"3a2844b7e9c422d3c10d287c895573f7108da1b3","version":"v9.0.0"},{"repo":"actions/setup-node","sha":"48b55a011bda9f5d6aeb4c2d9c7362e8dae4041e","version":"v6.4.0"},{"repo":"actions/upload-artifact","sha":"043fb46d1a93c77aae656e7c1c64a875d1fc6a0a","version":"v7.0.1"},{"repo":"github/gh-aw-actions/setup","sha":"8c7d04ebf1ece56cd381446125da3e0f6896294a","version":"v0.80.9"}],"containers":[{"image":"ghcr.io/github/gh-aw-firewall/agent:0.27.7","digest":"sha256:aae231e4635c8999d039c132f1602d3df850fe9b84a00aa2b5ac981179b5661c","pinned_image":"ghcr.io/github/gh-aw-firewall/agent:0.27.7@sha256:aae231e4635c8999d039c132f1602d3df850fe9b84a00aa2b5ac981179b5661c"},{"image":"ghcr.io/github/gh-aw-firewall/api-proxy:0.27.7","digest":"sha256:009caf2e3d88fa77b64e9a03a95a228fc58db0f1701c6d324b29ba5a3c7c79b6","pinned_image":"ghcr.io/github/gh-aw-firewall/api-proxy:0.27.7@sha256:009caf2e3d88fa77b64e9a03a95a228fc58db0f1701c6d324b29ba5a3c7c79b6"},{"image":"ghcr.io/github/gh-aw-firewall/squid:0.27.7","digest":"sha256:deb1d4e19de62d51cee0508057a596a19315c3423ada4d675cad136dc8037c96","pinned_image":"ghcr.io/github/gh-aw-firewall/squid:0.27.7@sha256:deb1d4e19de62d51cee0508057a596a19315c3423ada4d675cad136dc8037c96"},{"image":"ghcr.io/github/gh-aw-mcpg:v0.3.27","digest":"sha256:fe984bddde4ec05d756d9043edb0a32912e6b7b72f6a121b1082f29221421cc7","pinned_image":"ghcr.io/github/gh-aw-mcpg:v0.3.27@sha256:fe984bddde4ec05d756d9043edb0a32912e6b7b72f6a121b1082f29221421cc7"},{"image":"ghcr.io/github/gh-aw-node","digest":"sha256:529d02eb970b1161aa25c593a9c3df57fdfad5a8add328cb3b6eccef66f3183b","pinned_image":"ghcr.io/github/gh-aw-node@sha256:529d02eb970b1161aa25c593a9c3df57fdfad5a8add328cb3b6eccef66f3183b"},{"image":"ghcr.io/github/github-mcp-server:v1.4.0","digest":"sha256:2afb26356481d1a350e14544a6e160f7f7ec1561a1ea309b823665abf0309036","pinned_image":"ghcr.io/github/github-mcp-server:v1.4.0@sha256:2afb26356481d1a350e14544a6e160f7f7ec1561a1ea309b823665abf0309036"}]} # This file was automatically generated by gh-aw (v0.80.9). DO NOT EDIT. To debug this workflow, load the skill at https://github.com/github/gh-aw/blob/main/debug.md # @@ -37,7 +37,9 @@ # humans (the open tracking issue is the hand-off surface; a dedicated # [ci-fix][needs-human] PR is planned but currently deferred — see Step 6). CI on # the PR is kicked by a human `/azp run` for now; the loop watches, classifies, -# and re-fixes autonomously between kicks. +# and re-fixes autonomously between kicks. When the SPECIFIC test a PR fixed is +# confirmed green in that PR's own CI, the loop marks the draft PR ready for review +# (a state transition only — it never approves or merges). # Never mutes tests, but # de-flakes genuinely flaky ones (deterministic synchronization, no retries / # timeout bumps). Always skips visual-regression / screenshot issues. @@ -273,24 +275,24 @@ jobs: run: | bash "${RUNNER_TEMP}/gh-aw/actions/create_prompt_first.sh" { - cat << 'GH_AW_PROMPT_25cf75111b670167_EOF' + cat << 'GH_AW_PROMPT_2ac1d0b4083428f8_EOF' - GH_AW_PROMPT_25cf75111b670167_EOF + GH_AW_PROMPT_2ac1d0b4083428f8_EOF cat "${RUNNER_TEMP}/gh-aw/prompts/xpia.md" cat "${RUNNER_TEMP}/gh-aw/prompts/temp_folder_prompt.md" cat "${RUNNER_TEMP}/gh-aw/prompts/markdown.md" cat "${RUNNER_TEMP}/gh-aw/prompts/safe_outputs_prompt.md" - cat << 'GH_AW_PROMPT_25cf75111b670167_EOF' + cat << 'GH_AW_PROMPT_2ac1d0b4083428f8_EOF' - Tools: add_comment(max:3), create_pull_request(max:3), update_pull_request(max:3), push_to_pull_request_branch(max:3), missing_tool, missing_data, noop - GH_AW_PROMPT_25cf75111b670167_EOF + Tools: add_comment(max:6), create_pull_request(max:3), update_pull_request(max:3), mark_pull_request_as_ready_for_review(max:3), add_labels(max:3), push_to_pull_request_branch(max:3), missing_tool, missing_data, noop + GH_AW_PROMPT_2ac1d0b4083428f8_EOF cat "${RUNNER_TEMP}/gh-aw/prompts/safe_outputs_create_pull_request.md" cat "${RUNNER_TEMP}/gh-aw/prompts/safe_outputs_push_to_pr_branch.md" - cat << 'GH_AW_PROMPT_25cf75111b670167_EOF' + cat << 'GH_AW_PROMPT_2ac1d0b4083428f8_EOF' - GH_AW_PROMPT_25cf75111b670167_EOF + GH_AW_PROMPT_2ac1d0b4083428f8_EOF cat "${RUNNER_TEMP}/gh-aw/prompts/mcp_cli_tools_prompt.md" - cat << 'GH_AW_PROMPT_25cf75111b670167_EOF' + cat << 'GH_AW_PROMPT_2ac1d0b4083428f8_EOF' The following GitHub context information is available for this workflow: {{#if github.actor}} @@ -332,12 +334,12 @@ jobs: stop immediately and report the limitation rather than spending turns trying to work around it. - GH_AW_PROMPT_25cf75111b670167_EOF + GH_AW_PROMPT_2ac1d0b4083428f8_EOF cat "${RUNNER_TEMP}/gh-aw/prompts/github_mcp_tools_with_safeoutputs_prompt.md" - cat << 'GH_AW_PROMPT_25cf75111b670167_EOF' + cat << 'GH_AW_PROMPT_2ac1d0b4083428f8_EOF' {{#runtime-import .github/workflows/ci-status-fix.md}} - GH_AW_PROMPT_25cf75111b670167_EOF + GH_AW_PROMPT_2ac1d0b4083428f8_EOF } > "$GH_AW_PROMPT" - name: Interpolate variables and render templates uses: actions/github-script@3a2844b7e9c422d3c10d287c895573f7108da1b3 # v9.0.0 @@ -554,16 +556,18 @@ jobs: mkdir -p "${RUNNER_TEMP}/gh-aw/safeoutputs" mkdir -p /tmp/gh-aw/safeoutputs mkdir -p /tmp/gh-aw/mcp-logs/safeoutputs - cat > "${RUNNER_TEMP}/gh-aw/safeoutputs/config.json" << 'GH_AW_SAFE_OUTPUTS_CONFIG_fa2a69fad5fd0485_EOF' - {"add_comment":{"discussions":false,"max":3,"required_labels":["agentic-workflows"],"required_title_prefix":"[ci-fix] ","target":"*"},"create_pull_request":{"allowed_base_branches":["main"],"allowed_branches":["ci-fix/**"],"allowed_files":["src/Core/**","src/Controls/**","src/Essentials/**","src/BlazorWebView/**","src/TestUtils/**","src/Templates/**","**/PublicAPI.Unshipped.txt"],"base_branch":"main","draft":true,"labels":["agentic-workflows"],"max":3,"max_patch_files":100,"max_patch_size":4096,"protect_top_level_dot_folders":true,"protected_files":["package.json","bun.lockb","bunfig.toml","deno.json","deno.jsonc","deno.lock","global.json","NuGet.Config","Directory.Packages.props","mix.exs","mix.lock","go.mod","go.sum","stack.yaml","stack.yaml.lock","pom.xml","build.gradle","build.gradle.kts","settings.gradle","settings.gradle.kts","gradle.properties","package-lock.json","yarn.lock","pnpm-lock.yaml","npm-shrinkwrap.json","requirements.txt","Pipfile","Pipfile.lock","pyproject.toml","setup.py","setup.cfg","Gemfile","Gemfile.lock","uv.lock","CODEOWNERS","DESIGN.md","README.md","CONTRIBUTING.md","CHANGELOG.md","SECURITY.md","CODE_OF_CONDUCT.md","AGENTS.md","CLAUDE.md","GEMINI.md"],"protected_files_policy":"request_review","title_prefix":"[ci-fix] "},"create_report_incomplete_issue":{},"missing_data":{},"missing_tool":{},"noop":{"max":1,"report-as-issue":"true"},"push_to_pull_request_branch":{"allowed_files":["src/Core/**","src/Controls/**","src/Essentials/**","src/BlazorWebView/**","src/TestUtils/**","src/Templates/**","**/PublicAPI.Unshipped.txt"],"if_no_changes":"warn","max":3,"max_patch_size":4096,"protect_top_level_dot_folders":true,"protected_files":["package.json","bun.lockb","bunfig.toml","deno.json","deno.jsonc","deno.lock","global.json","NuGet.Config","Directory.Packages.props","mix.exs","mix.lock","go.mod","go.sum","stack.yaml","stack.yaml.lock","pom.xml","build.gradle","build.gradle.kts","settings.gradle","settings.gradle.kts","gradle.properties","package-lock.json","yarn.lock","pnpm-lock.yaml","npm-shrinkwrap.json","requirements.txt","Pipfile","Pipfile.lock","pyproject.toml","setup.py","setup.cfg","Gemfile","Gemfile.lock","uv.lock","CODEOWNERS","DESIGN.md","README.md","CONTRIBUTING.md","CHANGELOG.md","SECURITY.md","CODE_OF_CONDUCT.md","AGENTS.md","CLAUDE.md","GEMINI.md"],"required_labels":["agentic-workflows"],"target":"*","title_prefix":"[ci-fix] "},"report_incomplete":{},"update_pull_request":{"allow_body":true,"allow_title":false,"max":3,"target":"*","update_branch":false}} - GH_AW_SAFE_OUTPUTS_CONFIG_fa2a69fad5fd0485_EOF + cat > "${RUNNER_TEMP}/gh-aw/safeoutputs/config.json" << 'GH_AW_SAFE_OUTPUTS_CONFIG_0644bf1434ca0071_EOF' + {"add_comment":{"discussions":false,"max":6,"required_labels":["agentic-workflows"],"required_title_prefix":"[ci-fix] ","target":"*"},"add_labels":{"allowed":["p/0"],"max":3,"required_labels":["agentic-workflows"],"required_title_prefix":"[ci-fix] ","target":"*"},"create_pull_request":{"allowed_base_branches":["main"],"allowed_branches":["ci-fix/**"],"allowed_files":["src/Core/**","src/Controls/**","src/Essentials/**","src/BlazorWebView/**","src/TestUtils/**","src/Templates/**","**/PublicAPI.Unshipped.txt"],"base_branch":"main","draft":true,"labels":["agentic-workflows"],"max":3,"max_patch_files":100,"max_patch_size":4096,"protect_top_level_dot_folders":true,"protected_files":["package.json","bun.lockb","bunfig.toml","deno.json","deno.jsonc","deno.lock","global.json","NuGet.Config","Directory.Packages.props","mix.exs","mix.lock","go.mod","go.sum","stack.yaml","stack.yaml.lock","pom.xml","build.gradle","build.gradle.kts","settings.gradle","settings.gradle.kts","gradle.properties","package-lock.json","yarn.lock","pnpm-lock.yaml","npm-shrinkwrap.json","requirements.txt","Pipfile","Pipfile.lock","pyproject.toml","setup.py","setup.cfg","Gemfile","Gemfile.lock","uv.lock","CODEOWNERS","DESIGN.md","README.md","CONTRIBUTING.md","CHANGELOG.md","SECURITY.md","CODE_OF_CONDUCT.md","AGENTS.md","CLAUDE.md","GEMINI.md"],"protected_files_policy":"request_review","title_prefix":"[ci-fix] "},"create_report_incomplete_issue":{},"mark_pull_request_as_ready_for_review":{"max":3,"required_labels":["agentic-workflows"],"required_title_prefix":"[ci-fix] ","target":"*"},"missing_data":{},"missing_tool":{},"noop":{"max":1,"report-as-issue":"true"},"push_to_pull_request_branch":{"allowed_files":["src/Core/**","src/Controls/**","src/Essentials/**","src/BlazorWebView/**","src/TestUtils/**","src/Templates/**","**/PublicAPI.Unshipped.txt"],"if_no_changes":"warn","max":3,"max_patch_size":4096,"protect_top_level_dot_folders":true,"protected_files":["package.json","bun.lockb","bunfig.toml","deno.json","deno.jsonc","deno.lock","global.json","NuGet.Config","Directory.Packages.props","mix.exs","mix.lock","go.mod","go.sum","stack.yaml","stack.yaml.lock","pom.xml","build.gradle","build.gradle.kts","settings.gradle","settings.gradle.kts","gradle.properties","package-lock.json","yarn.lock","pnpm-lock.yaml","npm-shrinkwrap.json","requirements.txt","Pipfile","Pipfile.lock","pyproject.toml","setup.py","setup.cfg","Gemfile","Gemfile.lock","uv.lock","CODEOWNERS","DESIGN.md","README.md","CONTRIBUTING.md","CHANGELOG.md","SECURITY.md","CODE_OF_CONDUCT.md","AGENTS.md","CLAUDE.md","GEMINI.md"],"required_labels":["agentic-workflows"],"target":"*","title_prefix":"[ci-fix] "},"report_incomplete":{},"update_pull_request":{"allow_body":true,"allow_title":false,"max":3,"target":"*","update_branch":false}} + GH_AW_SAFE_OUTPUTS_CONFIG_0644bf1434ca0071_EOF - name: Generate Safe Outputs Tools env: GH_AW_TOOLS_META_JSON: | { "description_suffixes": { - "add_comment": " CONSTRAINTS: Maximum 3 comment(s) can be added. Target: *. Supports reply_to_id for discussion threading.", + "add_comment": " CONSTRAINTS: Maximum 6 comment(s) can be added. Target: *. Supports reply_to_id for discussion threading.", + "add_labels": " CONSTRAINTS: Maximum 3 label(s) can be added. Only these labels are allowed: [\"p/0\"]. Target: *.", "create_pull_request": " CONSTRAINTS: Maximum 3 pull request(s) can be created. Title will be prefixed with \"[ci-fix] \". Labels [\"agentic-workflows\"] will be automatically added. Only these labels are allowed: [\"agentic-workflows\"]. PRs will be created as drafts.", + "mark_pull_request_as_ready_for_review": " CONSTRAINTS: Maximum 3 pull request(s) can be marked as ready for review.", "push_to_pull_request_branch": " CONSTRAINTS: Maximum 3 push(es) can be made. The target pull request title must start with \"[ci-fix] \".", "update_pull_request": " CONSTRAINTS: Maximum 3 pull request(s) can be updated. Target: *." }, @@ -594,6 +598,25 @@ jobs: } } }, + "add_labels": { + "defaultMax": 5, + "fields": { + "item_number": { + "issueNumberOrTemporaryId": true + }, + "labels": { + "required": true, + "type": "array", + "itemType": "string", + "itemSanitize": true, + "itemMaxLength": 128 + }, + "repo": { + "type": "string", + "maxLength": 256 + } + } + }, "create_pull_request": { "defaultMax": 1, "fields": { @@ -635,6 +658,24 @@ jobs: } } }, + "mark_pull_request_as_ready_for_review": { + "defaultMax": 1, + "fields": { + "pull_request_number": { + "issueOrPRNumber": true + }, + "reason": { + "required": true, + "type": "string", + "sanitize": true, + "maxLength": 65000 + }, + "repo": { + "type": "string", + "maxLength": 256 + } + } + }, "missing_data": { "defaultMax": 20, "fields": { @@ -1494,7 +1535,7 @@ jobs: uses: actions/github-script@3a2844b7e9c422d3c10d287c895573f7108da1b3 # v9.0.0 env: WORKFLOW_NAME: "CI Failure Fixer (main)" - WORKFLOW_DESCRIPTION: "Periodic pass over open ci-scan tracking issues filed by the main-branch CI\nfailure scanner (.github/workflows/ci-status-main.md). This workflow targets\nthe `main` branch EXCLUSIVELY: it processes only issues labelled ci-scan and\nopens every PR against main. (The net11.0 branch is handled by the parallel\n.github/workflows/ci-status-fix-net11.md — the two are split because gh-aw can\nonly transport a fix relative to ONE static base branch per workflow, and the\nmain↔net11.0 divergence exceeds gh-aw's 10 MB transport-patch cap.) The fixer\nopens ONE draft [ci-fix] PR per actionable issue against main, then WATCHES\nthat PR's own CI on later runs: when the fix's CI comes back red and the red is\ncaused by the fix itself, it pushes a fresh follow-up fix onto the SAME PR\nbranch (never a second PR) — up to 10 attempts — then stops and defers to\nhumans (the open tracking issue is the hand-off surface; a dedicated\n[ci-fix][needs-human] PR is planned but currently deferred — see Step 6). CI on\nthe PR is kicked by a human `/azp run` for now; the loop watches, classifies,\nand re-fixes autonomously between kicks.\nNever mutes tests, but\nde-flakes genuinely flaky ones (deterministic synchronization, no retries /\ntimeout bumps). Always skips visual-regression / screenshot issues." + WORKFLOW_DESCRIPTION: "Periodic pass over open ci-scan tracking issues filed by the main-branch CI\nfailure scanner (.github/workflows/ci-status-main.md). This workflow targets\nthe `main` branch EXCLUSIVELY: it processes only issues labelled ci-scan and\nopens every PR against main. (The net11.0 branch is handled by the parallel\n.github/workflows/ci-status-fix-net11.md — the two are split because gh-aw can\nonly transport a fix relative to ONE static base branch per workflow, and the\nmain↔net11.0 divergence exceeds gh-aw's 10 MB transport-patch cap.) The fixer\nopens ONE draft [ci-fix] PR per actionable issue against main, then WATCHES\nthat PR's own CI on later runs: when the fix's CI comes back red and the red is\ncaused by the fix itself, it pushes a fresh follow-up fix onto the SAME PR\nbranch (never a second PR) — up to 10 attempts — then stops and defers to\nhumans (the open tracking issue is the hand-off surface; a dedicated\n[ci-fix][needs-human] PR is planned but currently deferred — see Step 6). CI on\nthe PR is kicked by a human `/azp run` for now; the loop watches, classifies,\nand re-fixes autonomously between kicks. When the SPECIFIC test a PR fixed is\nconfirmed green in that PR's own CI, the loop marks the draft PR ready for review\n(a state transition only — it never approves or merges).\nNever mutes tests, but\nde-flakes genuinely flaky ones (deterministic synchronization, no retries /\ntimeout bumps). Always skips visual-regression / screenshot issues." HAS_PATCH: ${{ needs.agent.outputs.has_patch }} with: script: | @@ -1910,7 +1951,7 @@ jobs: GH_AW_ALLOWED_DOMAINS: "*.blob.core.windows.net,*.githubusercontent.com,api.business.githubcopilot.com,api.enterprise.githubcopilot.com,api.github.com,api.githubcopilot.com,api.individual.githubcopilot.com,api.snapcraft.io,archive.ubuntu.com,azure.archive.ubuntu.com,codeload.github.com,crl.geotrust.com,crl.globalsign.com,crl.identrust.com,crl.sectigo.com,crl.thawte.com,crl.usertrust.com,crl.verisign.com,crl3.digicert.com,crl4.digicert.com,crls.ssl.com,dev.azure.com,docs.github.com,github-cloud.githubusercontent.com,github-cloud.s3.amazonaws.com,github.blog,github.com,github.githubassets.com,helix.dot.net,host.docker.internal,json-schema.org,json.schemastore.org,keyserver.ubuntu.com,lfs.github.com,objects.githubusercontent.com,ocsp.digicert.com,ocsp.geotrust.com,ocsp.globalsign.com,ocsp.identrust.com,ocsp.sectigo.com,ocsp.ssl.com,ocsp.thawte.com,ocsp.usertrust.com,ocsp.verisign.com,packagecloud.io,packages.cloud.google.com,packages.microsoft.com,patch-diff.githubusercontent.com,ppa.launchpad.net,raw.githubusercontent.com,registry.npmjs.org,s.symcb.com,s.symcd.com,security.ubuntu.com,telemetry.enterprise.githubcopilot.com,ts-crl.ws.symantec.com,ts-ocsp.ws.symantec.com,www.googleapis.com" GITHUB_SERVER_URL: ${{ github.server_url }} GITHUB_API_URL: ${{ github.api_url }} - GH_AW_SAFE_OUTPUTS_HANDLER_CONFIG: "{\"add_comment\":{\"discussions\":false,\"max\":3,\"required_labels\":[\"agentic-workflows\"],\"required_title_prefix\":\"[ci-fix] \",\"target\":\"*\"},\"create_pull_request\":{\"allowed_base_branches\":[\"main\"],\"allowed_branches\":[\"ci-fix/**\"],\"allowed_files\":[\"src/Core/**\",\"src/Controls/**\",\"src/Essentials/**\",\"src/BlazorWebView/**\",\"src/TestUtils/**\",\"src/Templates/**\",\"**/PublicAPI.Unshipped.txt\"],\"base_branch\":\"main\",\"draft\":true,\"labels\":[\"agentic-workflows\"],\"max\":3,\"max_patch_files\":100,\"max_patch_size\":4096,\"protect_top_level_dot_folders\":true,\"protected_files\":[\"package.json\",\"bun.lockb\",\"bunfig.toml\",\"deno.json\",\"deno.jsonc\",\"deno.lock\",\"global.json\",\"NuGet.Config\",\"Directory.Packages.props\",\"mix.exs\",\"mix.lock\",\"go.mod\",\"go.sum\",\"stack.yaml\",\"stack.yaml.lock\",\"pom.xml\",\"build.gradle\",\"build.gradle.kts\",\"settings.gradle\",\"settings.gradle.kts\",\"gradle.properties\",\"package-lock.json\",\"yarn.lock\",\"pnpm-lock.yaml\",\"npm-shrinkwrap.json\",\"requirements.txt\",\"Pipfile\",\"Pipfile.lock\",\"pyproject.toml\",\"setup.py\",\"setup.cfg\",\"Gemfile\",\"Gemfile.lock\",\"uv.lock\",\"CODEOWNERS\",\"DESIGN.md\",\"README.md\",\"CONTRIBUTING.md\",\"CHANGELOG.md\",\"SECURITY.md\",\"CODE_OF_CONDUCT.md\",\"AGENTS.md\",\"CLAUDE.md\",\"GEMINI.md\"],\"protected_files_policy\":\"request_review\",\"title_prefix\":\"[ci-fix] \"},\"create_report_incomplete_issue\":{},\"missing_data\":{},\"missing_tool\":{},\"noop\":{\"max\":1,\"report-as-issue\":\"true\"},\"push_to_pull_request_branch\":{\"allowed_files\":[\"src/Core/**\",\"src/Controls/**\",\"src/Essentials/**\",\"src/BlazorWebView/**\",\"src/TestUtils/**\",\"src/Templates/**\",\"**/PublicAPI.Unshipped.txt\"],\"if_no_changes\":\"warn\",\"max\":3,\"max_patch_size\":4096,\"protect_top_level_dot_folders\":true,\"protected_files\":[\"package.json\",\"bun.lockb\",\"bunfig.toml\",\"deno.json\",\"deno.jsonc\",\"deno.lock\",\"global.json\",\"NuGet.Config\",\"Directory.Packages.props\",\"mix.exs\",\"mix.lock\",\"go.mod\",\"go.sum\",\"stack.yaml\",\"stack.yaml.lock\",\"pom.xml\",\"build.gradle\",\"build.gradle.kts\",\"settings.gradle\",\"settings.gradle.kts\",\"gradle.properties\",\"package-lock.json\",\"yarn.lock\",\"pnpm-lock.yaml\",\"npm-shrinkwrap.json\",\"requirements.txt\",\"Pipfile\",\"Pipfile.lock\",\"pyproject.toml\",\"setup.py\",\"setup.cfg\",\"Gemfile\",\"Gemfile.lock\",\"uv.lock\",\"CODEOWNERS\",\"DESIGN.md\",\"README.md\",\"CONTRIBUTING.md\",\"CHANGELOG.md\",\"SECURITY.md\",\"CODE_OF_CONDUCT.md\",\"AGENTS.md\",\"CLAUDE.md\",\"GEMINI.md\"],\"required_labels\":[\"agentic-workflows\"],\"target\":\"*\",\"title_prefix\":\"[ci-fix] \"},\"report_incomplete\":{},\"update_pull_request\":{\"allow_body\":true,\"allow_title\":false,\"max\":3,\"target\":\"*\",\"update_branch\":false}}" + GH_AW_SAFE_OUTPUTS_HANDLER_CONFIG: "{\"add_comment\":{\"discussions\":false,\"max\":6,\"required_labels\":[\"agentic-workflows\"],\"required_title_prefix\":\"[ci-fix] \",\"target\":\"*\"},\"add_labels\":{\"allowed\":[\"p/0\"],\"max\":3,\"required_labels\":[\"agentic-workflows\"],\"required_title_prefix\":\"[ci-fix] \",\"target\":\"*\"},\"create_pull_request\":{\"allowed_base_branches\":[\"main\"],\"allowed_branches\":[\"ci-fix/**\"],\"allowed_files\":[\"src/Core/**\",\"src/Controls/**\",\"src/Essentials/**\",\"src/BlazorWebView/**\",\"src/TestUtils/**\",\"src/Templates/**\",\"**/PublicAPI.Unshipped.txt\"],\"base_branch\":\"main\",\"draft\":true,\"labels\":[\"agentic-workflows\"],\"max\":3,\"max_patch_files\":100,\"max_patch_size\":4096,\"protect_top_level_dot_folders\":true,\"protected_files\":[\"package.json\",\"bun.lockb\",\"bunfig.toml\",\"deno.json\",\"deno.jsonc\",\"deno.lock\",\"global.json\",\"NuGet.Config\",\"Directory.Packages.props\",\"mix.exs\",\"mix.lock\",\"go.mod\",\"go.sum\",\"stack.yaml\",\"stack.yaml.lock\",\"pom.xml\",\"build.gradle\",\"build.gradle.kts\",\"settings.gradle\",\"settings.gradle.kts\",\"gradle.properties\",\"package-lock.json\",\"yarn.lock\",\"pnpm-lock.yaml\",\"npm-shrinkwrap.json\",\"requirements.txt\",\"Pipfile\",\"Pipfile.lock\",\"pyproject.toml\",\"setup.py\",\"setup.cfg\",\"Gemfile\",\"Gemfile.lock\",\"uv.lock\",\"CODEOWNERS\",\"DESIGN.md\",\"README.md\",\"CONTRIBUTING.md\",\"CHANGELOG.md\",\"SECURITY.md\",\"CODE_OF_CONDUCT.md\",\"AGENTS.md\",\"CLAUDE.md\",\"GEMINI.md\"],\"protected_files_policy\":\"request_review\",\"title_prefix\":\"[ci-fix] \"},\"create_report_incomplete_issue\":{},\"mark_pull_request_as_ready_for_review\":{\"max\":3,\"required_labels\":[\"agentic-workflows\"],\"required_title_prefix\":\"[ci-fix] \",\"target\":\"*\"},\"missing_data\":{},\"missing_tool\":{},\"noop\":{\"max\":1,\"report-as-issue\":\"true\"},\"push_to_pull_request_branch\":{\"allowed_files\":[\"src/Core/**\",\"src/Controls/**\",\"src/Essentials/**\",\"src/BlazorWebView/**\",\"src/TestUtils/**\",\"src/Templates/**\",\"**/PublicAPI.Unshipped.txt\"],\"if_no_changes\":\"warn\",\"max\":3,\"max_patch_size\":4096,\"protect_top_level_dot_folders\":true,\"protected_files\":[\"package.json\",\"bun.lockb\",\"bunfig.toml\",\"deno.json\",\"deno.jsonc\",\"deno.lock\",\"global.json\",\"NuGet.Config\",\"Directory.Packages.props\",\"mix.exs\",\"mix.lock\",\"go.mod\",\"go.sum\",\"stack.yaml\",\"stack.yaml.lock\",\"pom.xml\",\"build.gradle\",\"build.gradle.kts\",\"settings.gradle\",\"settings.gradle.kts\",\"gradle.properties\",\"package-lock.json\",\"yarn.lock\",\"pnpm-lock.yaml\",\"npm-shrinkwrap.json\",\"requirements.txt\",\"Pipfile\",\"Pipfile.lock\",\"pyproject.toml\",\"setup.py\",\"setup.cfg\",\"Gemfile\",\"Gemfile.lock\",\"uv.lock\",\"CODEOWNERS\",\"DESIGN.md\",\"README.md\",\"CONTRIBUTING.md\",\"CHANGELOG.md\",\"SECURITY.md\",\"CODE_OF_CONDUCT.md\",\"AGENTS.md\",\"CLAUDE.md\",\"GEMINI.md\"],\"required_labels\":[\"agentic-workflows\"],\"target\":\"*\",\"title_prefix\":\"[ci-fix] \"},\"report_incomplete\":{},\"update_pull_request\":{\"allow_body\":true,\"allow_title\":false,\"max\":3,\"target\":\"*\",\"update_branch\":false}}" GH_AW_CI_TRIGGER_TOKEN: ${{ secrets.GH_AW_CI_TRIGGER_TOKEN }} with: github-token: ${{ secrets.GH_AW_GITHUB_TOKEN || secrets.GITHUB_TOKEN }} diff --git a/.github/workflows/ci-status-fix.md b/.github/workflows/ci-status-fix.md index d3227e25170c..9c2f46054029 100644 --- a/.github/workflows/ci-status-fix.md +++ b/.github/workflows/ci-status-fix.md @@ -15,7 +15,9 @@ description: | humans (the open tracking issue is the hand-off surface; a dedicated [ci-fix][needs-human] PR is planned but currently deferred — see Step 6). CI on the PR is kicked by a human `/azp run` for now; the loop watches, classifies, - and re-fixes autonomously between kicks. + and re-fixes autonomously between kicks. When the SPECIFIC test a PR fixed is + confirmed green in that PR's own CI, the loop marks the draft PR ready for review + (a state transition only — it never approves or merges). Never mutes tests, but de-flakes genuinely flaky ones (deterministic synchronization, no retries / timeout bumps). Always skips visual-regression / screenshot issues. @@ -239,8 +241,15 @@ safe-outputs: # PER-RUN total (not per-PR): a sweep may surface several green PRs and/or # annotate several flaky reds in one run. At max:1 all but the first comment # were silently dropped, starving the workflow's #1 value (surfacing green PRs - # for review). Raised to 3 to match the per-run throughput of the other outputs. - max: 3 + # for review). Sized to 6 (not 3) because a single green DRAFT PR that gets + # flipped ready in one sweep spends TWO comment slots — the Step 3 ✅ surface + # comment (which still names the /azp-gated legs a human must kick, so it is NOT + # redundant with 🎯) AND the Step 3.6 T3 🎯 readiness comment. At max:3 the shared + # bucket drained after ~1 draft flip and T3's atomicity pre-check then deferred + # every further mark-ready even while the mark-ready/add_labels buckets (also 3) + # sat idle; 6 lets ~3 draft flips (2 comments each) land per sweep, so the + # mark-ready:3 / add_labels:3 caps are actually reachable. + max: 6 target: "*" # Hard constraint (defense-in-depth): only comment on THIS workflow's own # ci-fix PRs — [ci-fix] title prefix AND agentic-workflows label. Mirrors the @@ -261,13 +270,51 @@ safe-outputs: # never the title, so disable title rewrites — this compiles to allow_title:false # and removes any ability to retitle an arbitrary PR. title: false - # NOTE: gh-aw v0.79.8 does NOT emit required-title-prefix/required-labels into + # NOTE: gh-aw v0.80.9 (the pinned compiler) does NOT emit required-title-prefix/required-labels into # the compiled config for update-pull-request (verified against the lock — it # silently drops them, unlike add-comment / push-to-pull-request-branch which # honor them). So which-PR scoping here relies on prompt Hard-Rule 6 + # min-integrity:approved; the residual is body-marker edits only (no code, no # merge, no close), capped at max:3. Revisit required-* if a future gh-aw # compiler emits them for this output. + mark-pull-request-as-ready-for-review: + # Flip a validated draft [ci-fix] PR to ready-for-review once the SPECIFIC test + # this PR was opened to fix is confirmed green in the PR's OWN CI (Step 3.6) — + # even when unrelated legs are red. The workflow is schedule/dispatch-triggered + # (no triggering-PR context), so target "*" lets the agent name the PR number it + # validated. This is a state transition ONLY: it never approves and never merges — + # a human still reviews and merges. + # PER-RUN total (not per-PR): a sweep may validate several PRs' target tests green. + max: 3 + target: "*" + # Hard constraint (defense-in-depth): only ever un-draft THIS workflow's own + # ci-fix PRs — [ci-fix] title prefix AND agentic-workflows label — so a confused + # or prompt-injected agent cannot mark an arbitrary PR ready. (If the v0.80.9 + # compiler silently drops these — as documented for update-pull-request above — + # the Step 3.6 preconditions + min-integrity:approved are the compensating scope + # controls; verify against the lock after compiling.) + required-title-prefix: "[ci-fix] " + required-labels: [agentic-workflows] + add-labels: + # Apply the p/0 priority label to a [ci-fix] PR at the exact moment the loop flips it + # from draft to ready-for-review (Step 3.6 T3) — so a validated, review-ready fix lands + # in the team's p/0 triage queue instead of sitting unseen in the draft backlog. Paired + # 1:1 with mark-pull-request-as-ready-for-review; target "*" lets the agent name the PR + # it just validated. + # allowed = HARD allowlist: the agent may ONLY ever add p/0, nothing else. This caps the + # blast radius of a confused/prompt-injected agent to exactly one benign priority label — + # it can never apply a downstream-triggering or destructive label. + allowed: [p/0] + # PER-RUN total (not per-PR): a sweep may mark several PRs ready in one run; each adds + # one label, so this matches the mark-ready per-run cap (3). + max: 3 + target: "*" + # Hard constraint (defense-in-depth): only ever label THIS workflow's own ci-fix PRs — + # [ci-fix] title prefix AND agentic-workflows label. (If the v0.80.9 compiler drops these + # — as documented for update-pull-request above — the Step 3.6 preconditions + + # min-integrity:approved + the allowed:[p/0] allowlist are the compensating controls.) + required-title-prefix: "[ci-fix] " + required-labels: [agentic-workflows] timeout-minutes: 90 @@ -339,33 +386,48 @@ through `safe-outputs`. 5. **One issue = one outcome per run.** Exactly one of: respond to a maintainer's change-request on the open PR (Track C, Step 3.5.R); open the first fix/help/ de-flake PR; advance an existing PR by one attempt (push a follow-up fix); - surface a validated-green PR for review; annotate an unrelated-flake red; wait + surface a validated-green PR for review; mark a target-validated draft PR + ready-for-review (Step 3.6 T3 — the terminal outcome that supersedes the same + run's surface/annotate precursor line), or record its `already-ready` + steady-state no-op; annotate an unrelated-flake red; wait (CI not yet settled); or a recorded skip (the dedicated needs-human PR is deferred — the attempt cap records a skip instead; see Step 6). Always prefer advancing or opening a PR over a skip when a non-mute diff is producible. 6. **All writes via `safe-outputs`.** Allowed outputs: `create_pull_request` (first attempt only), `push_to_pull_request_branch` (advance an existing PR by one attempt), `update_pull_request` (bump the attempt marker / refresh the - prior-attempts table), and `add_comment` (progress notes on the `[ci-fix]` PR - ONLY). NEVER comment on the tracking issue (issues are locked by + prior-attempts table), `add_comment` (progress notes on the `[ci-fix]` PR + ONLY), `mark_pull_request_as_ready_for_review` (flip a target-validated draft + `[ci-fix]` PR from draft to ready — Step 3.6 T3 only), and `add_labels` (add + ONLY the `p/0` label, ONLY on that same draft→ready transition — Step 3.6 T3). + NEVER comment on the tracking issue (issues are locked by `.github/workflows/ci-scan-lock-issues.yml`) — `add_comment` targets the PR. No `gh pr create`, no manual `git push`. **Defense-in-depth:** `add_comment` and `update_pull_request` use `target: "*"` (the agent supplies the PR number). - `add_comment` is now config-locked to `required-title-prefix: "[ci-fix] "` + + `add_comment`, `mark_pull_request_as_ready_for_review`, and `add_labels` are + config-locked to `required-title-prefix: "[ci-fix] "` + `required-labels: [agentic-workflows]` (hard-enforced by the handler, same as - `push_to_pull_request_branch`), so a comment can only ever land on THIS - workflow's own PRs. `update_pull_request` CANNOT be config-locked to a - title/label in gh-aw v0.79.8 (the compiler silently drops `required-*` for that - output — verified against the lock), so it keeps `title: false` (no retitles; - body-marker edits only) plus this prompt-level guard. Before emitting either, - VERIFY the target PR carries BOTH the `[ci-fix]` title prefix AND the - `agentic-workflows` label; never comment on or edit the body of any PR that - lacks both. + `push_to_pull_request_branch`), and `add_labels` additionally has an + `allowed: [p/0]` allowlist so it can ONLY ever add `p/0` — so these can only + ever land on THIS workflow's own PRs. `update_pull_request` CANNOT be + config-locked to a title/label in gh-aw v0.80.9 (the compiler silently drops + `required-*` for that output — verified against the lock), so it keeps + `title: false` (no retitles; body-marker edits only) plus this prompt-level + guard. Before emitting ANY of these, VERIFY the target PR carries BOTH the + `[ci-fix]` title prefix AND the `agentic-workflows` label; never comment on, + edit, un-draft, or label any PR that lacks both. 7. **Per-run safe-output caps (per RUN, not per PR).** Each output type is capped per run: `create_pull_request` 3, `push_to_pull_request_branch` 3, - `add_comment` 3, `update_pull_request` 3. Note an ADVANCE spends one push **and** - one update **and** one comment, so ≤ 3 advances/run; surfacing a green or - annotating a flake spends one comment. When a bucket is exhausted, do NOT keep + `add_comment` 6, `update_pull_request` 3, `mark_pull_request_as_ready_for_review` + 3, `add_labels` 3 (counts LABELS, not calls). Note an ADVANCE spends one push + **and** one update **and** one comment, so ≤ 3 advances/run; surfacing a green or + annotating a flake spends one comment; a **mark-ready (Step 3.6 T3)** of a + still-draft PR spends TWO comments (the Step 3 ✅ surface **and** the T3 🎯 + readiness note) **and** one mark-ready **and** one label as an ALL-OR-NOTHING set + (T3 pre-checks those buckets and defers the whole PR if any is exhausted — never + mark a PR ready without its 🎯 audit comment). `add_comment` is therefore sized 6 + (not 3) so ~3 draft flips, at 2 comments each, can land in one sweep instead of the + shared comment bucket starving the otherwise-idle mark-ready/add_labels buckets. When a bucket is exhausted, do NOT keep emitting (extras are silently dropped) — record `skipped: per-run cap reached; deferring PR # to next cycle` for each remaining PR so the drop is a deliberate, logged decision. The continuous loop picks the deferred PRs up next @@ -409,8 +471,9 @@ For every open tracking issue in scope, converge on exactly one outcome: | Open first help-wanted draft `[ci-fix]` PR | No open PR yet; a plausible candidate exists but cannot be runner-validated (device/UI tests) (attempt 1) | | Open first de-flake draft `[ci-fix]` PR | No open PR yet; failure is intermittent (green-on-retry) from a genuine test-quality defect; a deterministic-synchronization fix is producible (attempt 1, Step 4.7 bucket b) | | Advance an existing PR (push attempt N+1) | Open PR's own CI settled red *because of the fix itself*, no human engaged, marker < 10 — push a NEW distinct fix onto the same branch (Step 3.5 → 5.6) | -| Surface a validated-green PR | Open PR's own CI settled green — comment "validated, attempt N/10; ready for review" and do NOT advance (Step 3.5) | -| Annotate an unrelated-flake red | Open PR is red only on baseline-flake / unrelated legs — comment which leg needs a re-run; do NOT burn an attempt (Step 3.5) | +| Surface a validated-green PR | Open PR's own CI settled green — comment "validated, attempt N/10; ready for review" and do NOT advance (Step 3.5), then mark the draft PR ready for review once the fixed test is confirmed green (Step 3.6) | +| Annotate an unrelated-flake red | Open PR is red only on baseline-flake / unrelated legs — comment which leg needs a re-run; do NOT burn an attempt (Step 3.5); if the SPECIFIC fixed test is confirmed green on that build, still mark the draft PR ready for review (Step 3.6) | +| Mark a validated PR ready for review | The specific test this PR fixed is confirmed green in the PR's own CI (even if unrelated legs are red) — comment the target-test result and transition the draft PR to ready for review; never approve or merge (Step 3.6) | | Wait | Open PR's CI is pending / not yet settled on the current head SHA — do nothing this cycle (Step 3.5) | | Hand-off skip (attempt cap) | Marker == 10 and the signature still reproduces — stop and defer to humans; the dedicated `[ci-fix][needs-human]` PR is planned but currently deferred (Step 6) | | Recorded skip | Visual-regression, human engaged, already-handled, fixed-in-latest-build, infra-flake, out-of-bounds, only-mute-available, or no novel approach producible | @@ -685,14 +748,34 @@ Run these gates in order — the FIRST that fires decides this cycle's outcome: 1. **Human engaged.** If `C.humanEngaged` → `skipped: human engaged on PR #; deferring` and stop. Never fight a human reviewer — the intentional hand-off boundary still holds the moment a person touches the PR. -2. **CI not settled / unknown / incomplete prefetch.** If `C.checksSettled == false` - OR `C.overallConclusion` is `pending`, `neutral`, or `unknown`, OR - `C.dataComplete == false` → the fix's CI has not finished on the current head SHA - (`C.headSha`), or the prefetch could not fully read this PR (a partial read can - understate `humanEngaged` / `attempt`). `skipped: PR # CI pending / prefetch - incomplete on ; waiting` and stop. *(Round 1: this is where a - maintainer `/azp run` is awaited — the `/azp`-gated uitests/devicetests legs will - not have run until a human kicks them.)* +2. **CI not settled — but validate the target test first (target-focused readiness).** + If `C.checksSettled == false` OR `C.overallConclusion` is `pending`, `neutral`, or + `unknown`, OR `C.dataComplete == false` → the *overall* build has not finished on the + current head SHA (`C.headSha`), or the prefetch could not fully read this PR (a partial + read can understate `humanEngaged` / `attempt`). Unrelated legs still draining must NOT + keep an already-proven fix parked as a draft, so before waiting, try the target-focused + fast-path: + - **2a — Target-green fast-path (draft `[ci-fix]` PRs only).** If `C.dataComplete == + true` AND the Step 3.6 preconditions hold (draft PR, `[ci-fix] ` title prefix AND the + `agentic-workflows` label), run Step 3.6 **T1–T2** against `C.headSha` now: identify + the target test(s) and read the PR's OWN build **timeline / per-leg status** (anonymous + `_apis/build` — never `_apis/test`, per Hard-Rule 8). If EVERY target test is + **VALIDATED-GREEN** (per T2's all-platform definition below — the target's category leg + is `succeeded` on every platform that runs it, failed on none, and NO target platform + family the pipeline covers is still pending/unconcluded, per T2's "no platform left + unverified" rule; a platform that simply has no leg for the target's category — the test + doesn't run there — is fine, not a gap) AND every leg in + `C.failedLegs` that has ALREADY concluded classifies as **unrelated flake** (Step 4 / + Step 4.7 method — the fix is implicated in NO completed red), then the fix is proven + regardless of unrelated *pending* legs → run **Step 3.6 T3** (mark ready + 🎯 comment) + and stop. This early-out NEVER advances an attempt and NEVER acts on a red that could + be the fix's fault. + - **2b — Otherwise WAIT.** If the target test has not yet executed (pending/absent on + its leg), or a completed red is (or may be) caused by the fix, or `C.dataComplete == + false`, or this PR has no identifiable target test → `skipped: PR # CI pending / + target not yet validated on ; waiting` and stop. *(Round 1: this is where a + maintainer `/azp run` is awaited — the `/azp`-gated uitests/devicetests legs will not + have run until a human kicks them.)* 3. **Green → surface for review.** If `C.overallConclusion == "success"`: the checks that RAN are green. **Primary-gate check first:** the deterministic `success` verdict only certifies "at least one green check, nothing failing or @@ -704,18 +787,31 @@ Run these gates in order — the FIRST that fires decides this cycle's outcome: # primary CI gate (maui-pr) not green on ; waiting` and stop — do NOT surface. (The `/azp`-gated `maui-pr-uitests` (def 313) / `maui-pr-devicetests` (def 314) legs MAY still be un-run — that is expected and is named below, not a - reason to withhold the surface.) **Idempotency:** scan the PR's existing comments - for a prior bot `✅ … validated … on ` note for THIS head SHA — if one - already exists, record `already-surfaced PR # (head )` and stop - WITHOUT re-commenting (re-surfacing the same green every tick is noise and burns - the per-run comment budget other PRs need). Otherwise resolve the attempt number from - `C.effectiveAttempt` (the authoritative max(marker, bot-commit) counter; omit the - number only if it is somehow indeterminate). **Dry-run gate (Step 0):** if `dry_run == "true"`, do NOT emit any comment — instead print the intended `✅` validated-green body to the run log, tally `dry-run: would-surface-green PR #`, and stop. Otherwise `add_comment` on PR #: `✅ Attempt /10 validated - — the fix's CI is green on . Ready for human review.` - Name any `/azp`-gated legs (uitests def 313 / devicetests def 314) that have not - run and still need a maintainer `/azp run`. Do NOT advance. Record - `surfaced-green PR # (attempt /10)` and stop. *(This directly - attacks the real bottleneck — no reviews — so it is the highest-value outcome.)* + reason to withhold the surface.) **Comment idempotency + dry-run suppress the ✅ + comment ONLY — neither skips Step 3.6.** Scan the PR's existing comments for a prior + bot `✅ … validated … on ` note for THIS head SHA; if one exists, set + `SKIP_SURFACE_COMMENT = true`. Resolve the attempt number from `C.effectiveAttempt` + (the authoritative max(marker, bot-commit) counter; omit the number only if it is + somehow indeterminate). Then post the surface comment UNLESS suppressed: + - if `dry_run == "true"`: do NOT emit — print the intended `✅` validated-green body to + the run log and tally `dry-run: would-surface-green PR #`; + - else if `SKIP_SURFACE_COMMENT`: do NOT re-post — record `already-surfaced PR # + (head )` (re-surfacing the same green every tick is noise and burns the + per-run comment budget other PRs need); + - else `add_comment` on PR #: `✅ Attempt /10 validated — the fix's CI is + green on .` naming any `/azp`-gated legs (uitests def 313 / devicetests def + 314) that have not run and still need a maintainer `/azp run`; record `surfaced-green + PR # (attempt /10)`. (Do NOT assert "ready for human review" on a PR that + is still a draft — Step 3.6, not this comment, owns the draft→ready flip and posts its + own 🎯 announcement when it fires.) + Do NOT advance. Then — in ALL of the above cases — run **Step 3.6** (target-test + readiness gate) for this PR before stopping: its `/azp`-gated target legs may conclude + green on this SAME head SHA on a LATER sweep (an `/azp run` adds no commit), and Step + 3.6 is the ONLY place the draft→ready flip happens, so it MUST re-evaluate every sweep — + never `stop` here before it. *(This directly attacks the real bottleneck — no reviews — + so it is the highest-value outcome.)* 4. **Red → classify caused-by-fix vs unrelated-flake.** If `C.overallConclusion == "failure"`, analyze the PR's OWN failing build (NOT `main`): find the AzDO `maui-pr` build for this PR (filter builds by `branchName=refs/pull//merge`, @@ -723,18 +819,26 @@ Run these gates in order — the FIRST that fires decides this cycle's outcome: method and Step 4.7 flake buckets to `C.failedLegs`. - **Unrelated flake only** (every failed leg is known-flaky / infra / a pre-existing baseline red NOT introduced by the fix): do NOT burn an attempt. - **Idempotency first:** scan the PR's existing comments for a prior bot - `♻️ … unrelated flake … on ` note for THIS head SHA — if one already - exists, record `already-annotated-flake PR # (head )` and stop - WITHOUT re-commenting (re-annotating the same flake every 12h sweep is noise and - burns the per-run comment budget other PRs need). Otherwise resolve the attempt - number from `C.effectiveAttempt` (the authoritative max(marker, bot-commit) - counter), omitting the `Attempt /10:` prefix only if it is somehow - indeterminate. **Dry-run gate (Step 0):** if `dry_run == "true"`, do NOT emit any comment — instead print the intended `♻️` unrelated-flake body to the run log, tally `dry-run: would-annotate-flake PR #`, and stop. Otherwise `add_comment` on PR #: `♻️ Attempt /10: red is - unrelated flake on leg(s) () on ; the fix itself is not - implicated. A maintainer re-run (/azp run ) should clear it.` Record - `annotated-flake PR # (head )` and stop. *(Round 1: human re-runs; - Round 2: auto re-trigger.)* + **Comment idempotency + dry-run suppress the ♻️ comment ONLY — neither skips Step + 3.6.** Scan the PR's existing comments for a prior bot `♻️ … unrelated flake … on + ` note for THIS head SHA; if one exists, set `SKIP_FLAKE_COMMENT = true`. + Resolve the attempt number from `C.effectiveAttempt` (the authoritative max(marker, + bot-commit) counter), omitting the `Attempt /10:` prefix only if it is + somehow indeterminate. Then post the flake note UNLESS suppressed: + - if `dry_run == "true"`: do NOT emit — print the intended `♻️` unrelated-flake body + to the run log and tally `dry-run: would-annotate-flake PR #`; + - else if `SKIP_FLAKE_COMMENT`: do NOT re-post — record `already-annotated-flake PR + # (head )` (re-annotating the same flake every 12h sweep is noise and + burns the per-run comment budget other PRs need); + - else `add_comment` on PR #: `♻️ Attempt /10: red is unrelated flake on + leg(s) () on ; the fix itself is not implicated. A + maintainer re-run (/azp run ) should clear it.` record `annotated-flake + PR # (head )`. + Then — in ALL of the above cases — run **Step 3.6** (target-test readiness gate) for + this PR before stopping: its `/azp`-gated target legs may conclude green on this SAME + head SHA on a LATER sweep, and Step 3.6 is the ONLY place the draft→ready flip + happens, so it MUST re-evaluate every sweep — never `stop` here before it. *(Round 1: + human re-runs; Round 2: auto re-trigger.)* - **Caused by the fix** (a failed leg still matches the original target signature, or the fix introduced a NEW failure): advance an attempt. - **Attempt count.** `attempt = C.effectiveAttempt` — the authoritative @@ -752,6 +856,188 @@ Run these gates in order — the FIRST that fires decides this cycle's outcome: from every prior commit on this branch**, emitted via the Step 5.6 ADVANCE path (push onto the same PR). +#### Step 3.6 — Target-test verification & mark-ready gate + +Reached from the Step 3 **green-surface** branch, the Step 4 **unrelated-flake** branch +(AFTER that branch has posted its comment), and the Step 3.5 **gate-2 target-focused +fast-path** (2a — while unrelated legs are still pending). Purpose: confirm the SPECIFIC +test(s) this PR was opened to fix now PASS in the PR's own CI, and — only when they do +— transition the draft `[ci-fix]` PR to **ready for review** so a maintainer sees a +validated fix instead of a draft. This is the ONLY place the loop flips draft→ready; it +is a state transition, **never** an approval or a merge (a human still reviews and +merges). Overall red on *unrelated* legs must NOT gate readiness — we validate the fix, +not the base branch's flakiness. + +**Preconditions** (ALL must hold; otherwise record `skipped: readiness N/A PR # +()` and stop this gate): +- `C.isDraft == true` — if the PR is already ready-for-review, the draft→ready + transition is already done; do NOT re-mark **and do NOT re-apply `p/0`**. `p/0` is applied + exactly ONCE, atomically with the draft→ready flip (T3 — all-or-nothing with the 🎯 audit + comment + mark-ready), so an already-ready loop PR that is MISSING `p/0` has almost + certainly had it **removed by a maintainer** de-prioritizing the PR — a pure triage action + the human-engagement guard (which inspects comments/reviews/commits, NOT label changes) + cannot see. Re-adding `p/0` every sweep would fight that maintainer indefinitely, violating + the loop's "never override a human" contract. So the loop does NOT reconcile the label: + record `already-ready PR #` and stop this gate. (Trade-off: on the rare occasion a + transient API error drops `p/0` at flip time *after* the T3 atomic pre-check passed, it is + not auto-re-added — but the PR is still ready-for-review with its 🎯 audit comment, and + re-adding `p/0` is a trivial manual action; that is strictly preferable to steam-rolling a + maintainer's deliberate de-prioritization.) +- The PR is unmistakably THIS workflow's own: `[ci-fix] ` title prefix AND the + `agentic-workflows` label (mirrors the safe-output lock; the compensating scope + control if the v0.80.9 compiler drops the declarative required-*). +- You reached this gate from Step 3 (green), Step 4 (**unrelated-flake**), or the Step 3.5 + gate-2 **target-focused fast-path** (2a) — i.e. the fix is NOT implicated in any red that + has concluded. If Step 4 classified a red as **caused by the fix** (ADVANCE), do NOT run + this gate — advance the attempt instead. + +**T1 — Identify the target test(s).** From the `[ci-scan]` issue signature the fix +addresses (and the PR's own diff), extract the fully-qualified test method name(s) the +fix targets — e.g. `SafeAreaShouldWorkOnAllShellTabs`. For a de-flake it is the +de-flaked test; for a product fix it is the originally-failing test(s). If NO specific +test can be identified (e.g. a product build-break rather than a test failure), this is a +**build-only fix** — there is no single test to validate cross-platform, so validate at +**whole-build** granularity rather than stopping (Step 3 only *comments*; Step 3.6 is the +sole draft→ready flip, so a build-only fix is undrafted HERE or not at all). We only reach +this gate from Step 3 (green) / Step 4 (unrelated-flake) / gate-2a, so the fix is not +implicated in any concluded red; additionally require, via the T2 timeline method on +`C.headSha`, that the primary build pipeline `maui-pr` (def 302) has CONCLUDED with EVERY +one of its platform build legs `succeeded`/`completed` and NONE failed/canceled or still +pending — the build is green on **every** platform, not just the originally-broken one (the +cross-platform guard, applied to the build instead of a test). A build-only fix is undrafted +ONLY on the strength of the auto-running `maui-pr` (def 302) whole-build green, which the +loop CAN observe: if the PR's `[ci-scan]` issue signature or its own diff indicates the +ORIGINATING failure was in a `/azp`-gated pipeline (`maui-pr-uitests` def 313 or +`maui-pr-devicetests` def 314) rather than `maui-pr` (def 302), do NOT undraft on +`maui-pr`-green alone — those pipelines do not auto-run on this PR (GITHUB_TOKEN cannot +trigger them), so a green `maui-pr` build is NOT evidence the gated build break is fixed; +record `skipped: build-only fix PR # targets gated pipeline () not run — +deferring to human` and stop this gate. Otherwise, set `TARGET := "the maui-pr build +(build-only fix — no single target test)"` and proceed to **T3** to mark ready. If any `maui-pr` build leg is still unconcluded, record `skipped: build-only fix PR +# not yet whole-build green (leg(s) pending)` and stop this gate WITHOUT marking +ready (a green subset is not enough — a still-pending build leg could yet fail). + +**T2 — Verify the target test's platform legs are green (anonymous, leg-level).** Per-test +outcomes come from the AzDO **test-results** API (`_apis/test/...`), which is **NOT reachable +anonymously — Hard-Rule 8**: those endpoints 302-redirect to a sign-in page on this runner, so +a `curl` returns an HTML redirect (not JSON) and every target test would look "not executed". +Validate instead at **leg granularity** using ONLY the anonymous `_apis/build/...` timeline +(Hard-Rule 8) — a `succeeded` category leg means every test in that category **that actually +ran** passed on that platform. This is a strong bar **only if the target test genuinely +executes**: a job can go green while one specific test is skipped at runtime (`[Ignore]`, +`Assert.Ignore()`, `Assert.Inconclusive()`, a `Skip=`/conditional `[Fact]`, an `#if`-out, or +an early `return` before the asserts). So before trusting a green leg as proof the target +passed, **confirm from the PR's OWN diff that the fix does NOT skip, ignore, disable, or +short-circuit the target test** (it adds no `[Ignore]`/`[Explicit]`, `Assert.Ignore`/ +`Assert.Inconclusive`, `Skip=`, category-exclusion, `#if`-out, or early `return` around the +target). If the fix could cause the target to be runtime-skipped rather than genuinely pass, +leg-level green is NOT sufficient evidence — record `skipped: target test may be +runtime-skipped by this fix (diff adds ); leg-green insufficient, deferring to human +on PR #` and stop this gate WITHOUT marking ready. Otherwise (the target genuinely runs +and asserts, as a de-flake or product fix does) a green category leg cannot hide a +target-test failure, so treat it as authoritative. + +Map the target test to the CI leg(s) that run it. A leg name encodes platform + test category +(e.g. `Android UITests SafeAreaEdges,Shadow`, `iOS UITests SafeAreaEdges,Shadow`, `macOS +UITests SafeAreaEdges,Shadow`, `Windows UITests SafeAreaEdges,Shadow`). Determine the target +test's UI-test category — its `[Category(UITestCategories.X)]` in the test/HostApp source, +visible in the PR diff or the test file — to know which leg-name substring identifies its legs; +for a device test, the per-platform device-test legs. + +Using the SAME build-discovery as Step 4 (filter AzDO builds by `branchName=refs/pull//merge` +or `sourceVersion == C.headSha`), read each build's **timeline** on `C.headSha` for the +pipeline(s) that RUN the target test — `maui-pr` (def 302) for unit/integration tests, +`maui-pr-uitests` (def 313) for Appium UI tests, `maui-pr-devicetests` (def 314) for device +tests: + +```bash +ORG=dnceng-public; PROJ=public; BUILD= +# Anonymous + Hard-Rule-8-compliant. NEVER call _apis/test/... (it 302-redirects to sign-in). +# Each type=="Job" record is one platform leg: result (succeeded/failed/canceled/null), +# state (completed/inProgress/pending), name (encodes " UITests "): +curl -s "https://dev.azure.com/$ORG/$PROJ/_apis/build/builds/$BUILD/timeline?api-version=7.1" \ + | tee /tmp/gh-aw/agent/timeline_${BUILD}.json \ + | jq -r '.records[] | select(.type=="Job") | "\(.result)\t\(.state)\t\(.name)"' +``` + +(The prefetch already exposes per-leg status in `C.failedLegs` / the checks context, and +`gh pr checks ` lists the same per-leg rows with platform+category in the name — use +whichever is handy; the build timeline is the authoritative cross-check on the exact build.) + +Treat a target test as **VALIDATED-GREEN** only if, on `C.headSha`, the leg that runs its +category is green on **every platform it runs on** — a fix that repairs one platform must not +silently regress the same test's leg on another, so a green leg on the originally-red platform +alone is NOT enough. Across the target pipeline's platform legs (for a UI test: the Android, +iOS, Windows, and macOS/MacCatalyst legs running the target's category; for a device test: each +device-test platform), require ALL of: +- the target's category leg is `result == "succeeded"` **and** `state == "completed"` on + **every** platform that runs it, AND +- that leg is `failed` / `canceled` / aborted on **NO** platform, AND +- **no platform is left unverified.** For each platform family the target pipeline covers, that + platform's category leg must have CONCLUDED (`state == "completed"`) on `C.headSha`. If a + platform simply has no leg for the target's category, the test does not run there — that is + fine, not a gap. But if a platform's category leg has **not concluded** (`state` is + `inProgress` / pending, or an `/azp`-gated `maui-pr-uitests` / `maui-pr-devicetests` leg that + has not been kicked), the test's status on that platform is UNKNOWN → the fix is NOT yet + validated across platforms: record `skipped: target test green on but + not yet verified on (leg(s) pending / need /azp run) on PR #` + and stop this gate WITHOUT marking ready. + +A target test whose category leg never concluded on ANY platform (**not executed** anywhere — +e.g. its `/azp`-gated pipeline has not been kicked) is likewise NOT validated: record `skipped: +target test not yet executed on PR # ( not run — needs /azp run)` and stop +this gate WITHOUT marking ready. Do NOT overclaim — a green *sibling* leg (a different category +on the same platform) is not the target's leg, and a green leg on one platform is not a pass on +the others. + +**T3 — Mark ready + report.** If EVERY target test is VALIDATED-GREEN. The 🎯 comment, +the mark-ready, and the `p/0` label are THREE SEPARATE safe-outputs — the comment +existing does NOT prove the mark-ready took effect, so they are tracked independently: + +- **Comment idempotency (dup-suppress only — never gates the mark-ready).** We only reach + T3 while `C.isDraft == true` (the precondition stops an already-ready PR before here). So + if a prior bot comment for THIS head that carries the leading `🎯` anchor AND the + contiguous phrase `validated green on ` already exists (BOTH T3 variants — + `🎯 Target test validated green on ` and the build-only `🎯 Build validated green on + ` — contain that exact contiguous phrase, so this recognizes both; keying on + `target test` alone would instead miss the build-only comment and re-post it every sweep. + **Require the `🎯` anchor** so the match can NEVER be loosened to a gapped + `validated…green…on` form that would false-positive on the Step 3 `✅ … validated — the + fix's CI is green on ` surface comment and wrongly suppress the audit 🎯), the comment + landed on an earlier sweep but the **mark-ready did NOT take + effect** (the PR is still a draft) — set `SUPPRESS_COMMENT = true` (do not re-post the + duplicate 🎯 comment) but STILL complete the mark-ready + label below. Never treat the + comment's existence as "already marked ready" while the PR is still a draft. Record this + case as `re-marking PR # (🎯 present; mark-ready did not take on a prior sweep)`. +- **Atomicity budget pre-check (Hard-Rule 7).** Determine the outputs this gate will emit: + `mark_pull_request_as_ready_for_review` + `add_labels` always, plus `add_comment` unless + `SUPPRESS_COMMENT`. Verify a free per-run slot remains in EACH bucket you are about to + use. If ANY required bucket is exhausted, emit NONE of them — record `skipped: per-run + cap reached; deferring mark-ready PR # to next cycle` and stop this gate. Never mark a + PR ready (or label it) unless its 🎯 audit comment is GUARANTEED to exist for this exact + head SHA — either posted in this same sweep, or (in the `SUPPRESS_COMMENT` re-marking case) + already present from a prior sweep. The outputs you DO emit either all land or all defer + together; the audit trail must never be absent, but it is NOT re-posted when it already exists. +- **Dry-run gate (Step 0):** if `dry_run == "true"`, emit NOTHING — print the intended + readiness comment and "would mark ready + add p/0" to the run log, tally `dry-run: + would-mark-ready PR #`, and stop. +- Otherwise emit for THIS PR number ``: + 1. `add_comment` (ONLY if not `SUPPRESS_COMMENT`): `🎯 Target test validated green on + — passed on ALL platforms it runs on (, + buildId ). — not + caused by this fix."> Transitioning this PR from draft to ready for review and adding + `p/0`; a maintainer still reviews and merges.` (For a **build-only fix** — the T1 + whole-build fallback, no single target test — phrase the first clause as `🎯 Build + validated green on — the maui-pr build passed on ALL platforms (buildId + ).` instead of naming a test.) + 2. `mark_pull_request_as_ready_for_review` with `reason:` a one-line justification + naming the validated test(s) and ``. + 3. `add_labels` with `labels: ["p/0"]` for PR # — put the now-review-ready fix into + the team's p/0 priority queue so it is triaged, not lost in the draft backlog. (If + the PR somehow already carries `p/0`, this is a harmless no-op.) +- Record `marked-ready PR # (target green on ALL platforms on , + labeled p/0)` and stop. + #### Step 3.5.R — Maintainer change-request response (Track C) Reached from Step 3.5 gate 0. Goal: when an **eligible human reviewer** (Hard-Rule @@ -1383,7 +1669,19 @@ Filed by [`ci-status-fix`](https://github.com/dotnet/maui/blob/main/.github/work ### Step 8 — Per-issue tally + end-of-run summary -Per issue, append one outcome line to `/tmp/gh-aw/agent/coverage.txt`: +Per issue, append **one** outcome line to `/tmp/gh-aw/agent/coverage.txt` — the +terminal outcome for the cycle. When one cycle produces a chained pair — a PR gets a +same-cycle precursor line and **then** a Step 3.6 readiness line (`marked-ready` on the +draft→ready flip, or `already-ready` on a later steady-state sweep of an already-ready +PR) — record ONLY the terminal Step 3.6 readiness line: it supersedes ANY same-cycle +non-readiness precursor for that PR, so an aggregator keying on one-line-per-issue never +double-counts. The precursor may be a Step 3 green line (`surfaced-green` on the first ✅, +or `already-surfaced` when it already exists) OR — when an unrelated-flake red and a +target-green coincide — a Step 4 `annotated-flake` line; either way the readiness line is +terminal, and the superseded precursor's signal survives in the PR's 🎯/♻️ comment so no +information is lost. This covers the flip cycle (`surfaced-green` → `marked-ready`), the +post-flip steady state (`already-surfaced` → `already-ready`), and the flake-coincident +flip (`annotated-flake` → `marked-ready`): ``` # main attempt- @@ -1394,8 +1692,14 @@ Per issue, append one outcome line to `/tmp/gh-aw/agent/coverage.txt`: `advance-PR # attempt /10` (ADVANCE mode — new commit pushed to the existing PR), `surfaced-green PR #` (fix's own CI went green; commented for review, did not advance), `annotated-flake PR #` (red was unrelated flake; -commented, attempt NOT burned), `waiting PR #` (CI not settled yet), -`dry-run: would-`, `skipped: `. (The +commented, attempt NOT burned), `marked-ready PR #` (the specific fixed test +was confirmed green in the PR's own CI; draft flipped to ready for review), +`already-surfaced PR #` (the fix's ✅ green comment already existed this sweep; +re-post suppressed — superseded by the Step 3.6 readiness line when the PR also flips +ready that cycle), `already-ready PR #` (terminal steady state — the PR was flipped +ready on a prior cycle and remains green on an unchanged head; no action taken, +supersedes `already-surfaced`), `waiting PR #` (CI not settled yet), +`dry-run: would-`, `skipped: `. (The `needs-human-PR` outcome is reserved for the deferred hand-off — Step 6 currently records a skip instead, so it is not emitted.)
( not run — needs /azp run)` and stop +this gate WITHOUT marking ready. Do NOT overclaim — a green *sibling* leg (a different category +on the same platform) is not the target's leg, and a green leg on one platform is not a pass on +the others. + +**T3 — Mark ready + report.** If EVERY target test is VALIDATED-GREEN. The 🎯 comment, +the mark-ready, and the `p/0` label are THREE SEPARATE safe-outputs — the comment +existing does NOT prove the mark-ready took effect, so they are tracked independently: + +- **Comment idempotency (dup-suppress only — never gates the mark-ready).** We only reach + T3 while `C.isDraft == true` (the precondition stops an already-ready PR before here). So + if a prior bot comment for THIS head that carries the leading `🎯` anchor AND the + contiguous phrase `validated green on ` already exists (BOTH T3 variants — + `🎯 Target test validated green on ` and the build-only `🎯 Build validated green on + ` — contain that exact contiguous phrase, so this recognizes both; keying on + `target test` alone would instead miss the build-only comment and re-post it every sweep. + **Require the `🎯` anchor** so the match can NEVER be loosened to a gapped + `validated…green…on` form that would false-positive on the Step 3 `✅ … validated — the + fix's CI is green on ` surface comment and wrongly suppress the audit 🎯), the comment + landed on an earlier sweep but the **mark-ready did NOT take + effect** (the PR is still a draft) — set `SUPPRESS_COMMENT = true` (do not re-post the + duplicate 🎯 comment) but STILL complete the mark-ready + label below. Never treat the + comment's existence as "already marked ready" while the PR is still a draft. Record this + case as `re-marking PR # (🎯 present; mark-ready did not take on a prior sweep)`. +- **Atomicity budget pre-check (Hard-Rule 7).** Determine the outputs this gate will emit: + `mark_pull_request_as_ready_for_review` + `add_labels` always, plus `add_comment` unless + `SUPPRESS_COMMENT`. Verify a free per-run slot remains in EACH bucket you are about to + use. If ANY required bucket is exhausted, emit NONE of them — record `skipped: per-run + cap reached; deferring mark-ready PR # to next cycle` and stop this gate. Never mark a + PR ready (or label it) unless its 🎯 audit comment is GUARANTEED to exist for this exact + head SHA — either posted in this same sweep, or (in the `SUPPRESS_COMMENT` re-marking case) + already present from a prior sweep. The outputs you DO emit either all land or all defer + together; the audit trail must never be absent, but it is NOT re-posted when it already exists. +- **Dry-run gate (Step 0):** if `dry_run == "true"`, emit NOTHING — print the intended + readiness comment and "would mark ready + add p/0" to the run log, tally `dry-run: + would-mark-ready PR #`, and stop. +- Otherwise emit for THIS PR number ``: + 1. `add_comment` (ONLY if not `SUPPRESS_COMMENT`): `🎯 Target test validated green on + — passed on ALL platforms it runs on (, + buildId ). — not + caused by this fix."> Transitioning this PR from draft to ready for review and adding + `p/0`; a maintainer still reviews and merges.` (For a **build-only fix** — the T1 + whole-build fallback, no single target test — phrase the first clause as `🎯 Build + validated green on — the maui-pr build passed on ALL platforms (buildId + ).` instead of naming a test.) + 2. `mark_pull_request_as_ready_for_review` with `reason:` a one-line justification + naming the validated test(s) and ``. + 3. `add_labels` with `labels: ["p/0"]` for PR # — put the now-review-ready fix into + the team's p/0 priority queue so it is triaged, not lost in the draft backlog. (If + the PR somehow already carries `p/0`, this is a harmless no-op.) +- Record `marked-ready PR # (target green on ALL platforms on , + labeled p/0)` and stop. + #### Step 3.5.R — Maintainer change-request response (Track C) Reached from Step 3.5 gate 0. Goal: when an **eligible human reviewer** (Hard-Rule @@ -1395,7 +1681,19 @@ Filed by [`ci-status-fix-net11`](https://github.com/dotnet/maui/blob/main/.githu ### Step 8 — Per-issue tally + end-of-run summary -Per issue, append one outcome line to `/tmp/gh-aw/agent/coverage.txt`: +Per issue, append **one** outcome line to `/tmp/gh-aw/agent/coverage.txt` — the +terminal outcome for the cycle. When one cycle produces a chained pair — a PR gets a +same-cycle precursor line and **then** a Step 3.6 readiness line (`marked-ready` on the +draft→ready flip, or `already-ready` on a later steady-state sweep of an already-ready +PR) — record ONLY the terminal Step 3.6 readiness line: it supersedes ANY same-cycle +non-readiness precursor for that PR, so an aggregator keying on one-line-per-issue never +double-counts. The precursor may be a Step 3 green line (`surfaced-green` on the first ✅, +or `already-surfaced` when it already exists) OR — when an unrelated-flake red and a +target-green coincide — a Step 4 `annotated-flake` line; either way the readiness line is +terminal, and the superseded precursor's signal survives in the PR's 🎯/♻️ comment so no +information is lost. This covers the flip cycle (`surfaced-green` → `marked-ready`), the +post-flip steady state (`already-surfaced` → `already-ready`), and the flake-coincident +flip (`annotated-flake` → `marked-ready`): ``` # net11.0 attempt- @@ -1406,8 +1704,14 @@ Per issue, append one outcome line to `/tmp/gh-aw/agent/coverage.txt`: `advance-PR # attempt /10` (ADVANCE mode — new commit pushed to the existing PR), `surfaced-green PR #` (fix's own CI went green; commented for review, did not advance), `annotated-flake PR #` (red was unrelated flake; -commented, attempt NOT burned), `waiting PR #` (CI not settled yet), -`dry-run: would-`, `skipped: `. (The +commented, attempt NOT burned), `marked-ready PR #` (the specific fixed test +was confirmed green in the PR's own CI; draft flipped to ready for review), +`already-surfaced PR #` (the fix's ✅ green comment already existed this sweep; +re-post suppressed — superseded by the Step 3.6 readiness line when the PR also flips +ready that cycle), `already-ready PR #` (terminal steady state — the PR was flipped +ready on a prior cycle and remains green on an unchanged head; no action taken, +supersedes `already-surfaced`), `waiting PR #` (CI not settled yet), +`dry-run: would-`, `skipped: `. (The `needs-human-PR` outcome is reserved for the deferred hand-off — Step 6 currently records a skip instead, so it is not emitted.) diff --git a/.github/workflows/ci-status-fix.lock.yml b/.github/workflows/ci-status-fix.lock.yml index 93dc5969cb0d..65511f2b3259 100644 --- a/.github/workflows/ci-status-fix.lock.yml +++ b/.github/workflows/ci-status-fix.lock.yml @@ -1,4 +1,4 @@ -# gh-aw-metadata: {"schema_version":"v4","frontmatter_hash":"f014161967589ebcaf1ac5ec6f3c88ee5a4388d28b55a07dc36c45b7b2aebab6","body_hash":"86c6b2d4c3df13da14c6a3c8b211381fcc9f97bb1951ff7b80588a2c5338e00c","compiler_version":"v0.80.9","strict":true,"agent_id":"copilot","agent_model":"claude-opus-4.8","engine_versions":{"copilot":"1.0.63"}} +# gh-aw-metadata: {"schema_version":"v4","frontmatter_hash":"b0ebf8a2f179900cfe3638e994b27a43cec52bdbdd2331dc1bd4d65762fa2d6b","body_hash":"1825e2b1f7c7bd88241bf37580655489360f9bebc806edd00837a52ce4ad83a0","compiler_version":"v0.80.9","strict":true,"agent_id":"copilot","agent_model":"claude-opus-4.8","engine_versions":{"copilot":"1.0.63"}} # gh-aw-manifest: {"version":1,"secrets":["COPILOT_PAT_0","COPILOT_PAT_1","COPILOT_PAT_2","COPILOT_PAT_3","COPILOT_PAT_4","COPILOT_PAT_5","COPILOT_PAT_6","COPILOT_PAT_7","COPILOT_PAT_8","COPILOT_PAT_9","GH_AW_CI_TRIGGER_TOKEN","GH_AW_GITHUB_MCP_SERVER_TOKEN","GH_AW_GITHUB_TOKEN","GITHUB_TOKEN"],"actions":[{"repo":"actions/checkout","sha":"34e114876b0b11c390a56381ad16ebd13914f8d5","version":"v4"},{"repo":"actions/checkout","sha":"9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0","version":"v7.0.0"},{"repo":"actions/checkout","sha":"de0fac2e4500dabe0009e67214ff5f5447ce83dd","version":"v6.0.2"},{"repo":"actions/download-artifact","sha":"3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c","version":"v8.0.1"},{"repo":"actions/github-script","sha":"3a2844b7e9c422d3c10d287c895573f7108da1b3","version":"v9.0.0"},{"repo":"actions/setup-node","sha":"48b55a011bda9f5d6aeb4c2d9c7362e8dae4041e","version":"v6.4.0"},{"repo":"actions/upload-artifact","sha":"043fb46d1a93c77aae656e7c1c64a875d1fc6a0a","version":"v7.0.1"},{"repo":"github/gh-aw-actions/setup","sha":"8c7d04ebf1ece56cd381446125da3e0f6896294a","version":"v0.80.9"}],"containers":[{"image":"ghcr.io/github/gh-aw-firewall/agent:0.27.7","digest":"sha256:aae231e4635c8999d039c132f1602d3df850fe9b84a00aa2b5ac981179b5661c","pinned_image":"ghcr.io/github/gh-aw-firewall/agent:0.27.7@sha256:aae231e4635c8999d039c132f1602d3df850fe9b84a00aa2b5ac981179b5661c"},{"image":"ghcr.io/github/gh-aw-firewall/api-proxy:0.27.7","digest":"sha256:009caf2e3d88fa77b64e9a03a95a228fc58db0f1701c6d324b29ba5a3c7c79b6","pinned_image":"ghcr.io/github/gh-aw-firewall/api-proxy:0.27.7@sha256:009caf2e3d88fa77b64e9a03a95a228fc58db0f1701c6d324b29ba5a3c7c79b6"},{"image":"ghcr.io/github/gh-aw-firewall/squid:0.27.7","digest":"sha256:deb1d4e19de62d51cee0508057a596a19315c3423ada4d675cad136dc8037c96","pinned_image":"ghcr.io/github/gh-aw-firewall/squid:0.27.7@sha256:deb1d4e19de62d51cee0508057a596a19315c3423ada4d675cad136dc8037c96"},{"image":"ghcr.io/github/gh-aw-mcpg:v0.3.27","digest":"sha256:fe984bddde4ec05d756d9043edb0a32912e6b7b72f6a121b1082f29221421cc7","pinned_image":"ghcr.io/github/gh-aw-mcpg:v0.3.27@sha256:fe984bddde4ec05d756d9043edb0a32912e6b7b72f6a121b1082f29221421cc7"},{"image":"ghcr.io/github/gh-aw-node","digest":"sha256:529d02eb970b1161aa25c593a9c3df57fdfad5a8add328cb3b6eccef66f3183b","pinned_image":"ghcr.io/github/gh-aw-node@sha256:529d02eb970b1161aa25c593a9c3df57fdfad5a8add328cb3b6eccef66f3183b"},{"image":"ghcr.io/github/github-mcp-server:v1.4.0","digest":"sha256:2afb26356481d1a350e14544a6e160f7f7ec1561a1ea309b823665abf0309036","pinned_image":"ghcr.io/github/github-mcp-server:v1.4.0@sha256:2afb26356481d1a350e14544a6e160f7f7ec1561a1ea309b823665abf0309036"}]} # This file was automatically generated by gh-aw (v0.80.9). DO NOT EDIT. To debug this workflow, load the skill at https://github.com/github/gh-aw/blob/main/debug.md # @@ -37,7 +37,9 @@ # humans (the open tracking issue is the hand-off surface; a dedicated # [ci-fix][needs-human] PR is planned but currently deferred — see Step 6). CI on # the PR is kicked by a human `/azp run` for now; the loop watches, classifies, -# and re-fixes autonomously between kicks. +# and re-fixes autonomously between kicks. When the SPECIFIC test a PR fixed is +# confirmed green in that PR's own CI, the loop marks the draft PR ready for review +# (a state transition only — it never approves or merges). # Never mutes tests, but # de-flakes genuinely flaky ones (deterministic synchronization, no retries / # timeout bumps). Always skips visual-regression / screenshot issues. @@ -273,24 +275,24 @@ jobs: run: | bash "${RUNNER_TEMP}/gh-aw/actions/create_prompt_first.sh" { - cat << 'GH_AW_PROMPT_25cf75111b670167_EOF' + cat << 'GH_AW_PROMPT_2ac1d0b4083428f8_EOF' - GH_AW_PROMPT_25cf75111b670167_EOF + GH_AW_PROMPT_2ac1d0b4083428f8_EOF cat "${RUNNER_TEMP}/gh-aw/prompts/xpia.md" cat "${RUNNER_TEMP}/gh-aw/prompts/temp_folder_prompt.md" cat "${RUNNER_TEMP}/gh-aw/prompts/markdown.md" cat "${RUNNER_TEMP}/gh-aw/prompts/safe_outputs_prompt.md" - cat << 'GH_AW_PROMPT_25cf75111b670167_EOF' + cat << 'GH_AW_PROMPT_2ac1d0b4083428f8_EOF' - Tools: add_comment(max:3), create_pull_request(max:3), update_pull_request(max:3), push_to_pull_request_branch(max:3), missing_tool, missing_data, noop - GH_AW_PROMPT_25cf75111b670167_EOF + Tools: add_comment(max:6), create_pull_request(max:3), update_pull_request(max:3), mark_pull_request_as_ready_for_review(max:3), add_labels(max:3), push_to_pull_request_branch(max:3), missing_tool, missing_data, noop + GH_AW_PROMPT_2ac1d0b4083428f8_EOF cat "${RUNNER_TEMP}/gh-aw/prompts/safe_outputs_create_pull_request.md" cat "${RUNNER_TEMP}/gh-aw/prompts/safe_outputs_push_to_pr_branch.md" - cat << 'GH_AW_PROMPT_25cf75111b670167_EOF' + cat << 'GH_AW_PROMPT_2ac1d0b4083428f8_EOF' - GH_AW_PROMPT_25cf75111b670167_EOF + GH_AW_PROMPT_2ac1d0b4083428f8_EOF cat "${RUNNER_TEMP}/gh-aw/prompts/mcp_cli_tools_prompt.md" - cat << 'GH_AW_PROMPT_25cf75111b670167_EOF' + cat << 'GH_AW_PROMPT_2ac1d0b4083428f8_EOF' The following GitHub context information is available for this workflow: {{#if github.actor}} @@ -332,12 +334,12 @@ jobs: stop immediately and report the limitation rather than spending turns trying to work around it. - GH_AW_PROMPT_25cf75111b670167_EOF + GH_AW_PROMPT_2ac1d0b4083428f8_EOF cat "${RUNNER_TEMP}/gh-aw/prompts/github_mcp_tools_with_safeoutputs_prompt.md" - cat << 'GH_AW_PROMPT_25cf75111b670167_EOF' + cat << 'GH_AW_PROMPT_2ac1d0b4083428f8_EOF' {{#runtime-import .github/workflows/ci-status-fix.md}} - GH_AW_PROMPT_25cf75111b670167_EOF + GH_AW_PROMPT_2ac1d0b4083428f8_EOF } > "$GH_AW_PROMPT" - name: Interpolate variables and render templates uses: actions/github-script@3a2844b7e9c422d3c10d287c895573f7108da1b3 # v9.0.0 @@ -554,16 +556,18 @@ jobs: mkdir -p "${RUNNER_TEMP}/gh-aw/safeoutputs" mkdir -p /tmp/gh-aw/safeoutputs mkdir -p /tmp/gh-aw/mcp-logs/safeoutputs - cat > "${RUNNER_TEMP}/gh-aw/safeoutputs/config.json" << 'GH_AW_SAFE_OUTPUTS_CONFIG_fa2a69fad5fd0485_EOF' - {"add_comment":{"discussions":false,"max":3,"required_labels":["agentic-workflows"],"required_title_prefix":"[ci-fix] ","target":"*"},"create_pull_request":{"allowed_base_branches":["main"],"allowed_branches":["ci-fix/**"],"allowed_files":["src/Core/**","src/Controls/**","src/Essentials/**","src/BlazorWebView/**","src/TestUtils/**","src/Templates/**","**/PublicAPI.Unshipped.txt"],"base_branch":"main","draft":true,"labels":["agentic-workflows"],"max":3,"max_patch_files":100,"max_patch_size":4096,"protect_top_level_dot_folders":true,"protected_files":["package.json","bun.lockb","bunfig.toml","deno.json","deno.jsonc","deno.lock","global.json","NuGet.Config","Directory.Packages.props","mix.exs","mix.lock","go.mod","go.sum","stack.yaml","stack.yaml.lock","pom.xml","build.gradle","build.gradle.kts","settings.gradle","settings.gradle.kts","gradle.properties","package-lock.json","yarn.lock","pnpm-lock.yaml","npm-shrinkwrap.json","requirements.txt","Pipfile","Pipfile.lock","pyproject.toml","setup.py","setup.cfg","Gemfile","Gemfile.lock","uv.lock","CODEOWNERS","DESIGN.md","README.md","CONTRIBUTING.md","CHANGELOG.md","SECURITY.md","CODE_OF_CONDUCT.md","AGENTS.md","CLAUDE.md","GEMINI.md"],"protected_files_policy":"request_review","title_prefix":"[ci-fix] "},"create_report_incomplete_issue":{},"missing_data":{},"missing_tool":{},"noop":{"max":1,"report-as-issue":"true"},"push_to_pull_request_branch":{"allowed_files":["src/Core/**","src/Controls/**","src/Essentials/**","src/BlazorWebView/**","src/TestUtils/**","src/Templates/**","**/PublicAPI.Unshipped.txt"],"if_no_changes":"warn","max":3,"max_patch_size":4096,"protect_top_level_dot_folders":true,"protected_files":["package.json","bun.lockb","bunfig.toml","deno.json","deno.jsonc","deno.lock","global.json","NuGet.Config","Directory.Packages.props","mix.exs","mix.lock","go.mod","go.sum","stack.yaml","stack.yaml.lock","pom.xml","build.gradle","build.gradle.kts","settings.gradle","settings.gradle.kts","gradle.properties","package-lock.json","yarn.lock","pnpm-lock.yaml","npm-shrinkwrap.json","requirements.txt","Pipfile","Pipfile.lock","pyproject.toml","setup.py","setup.cfg","Gemfile","Gemfile.lock","uv.lock","CODEOWNERS","DESIGN.md","README.md","CONTRIBUTING.md","CHANGELOG.md","SECURITY.md","CODE_OF_CONDUCT.md","AGENTS.md","CLAUDE.md","GEMINI.md"],"required_labels":["agentic-workflows"],"target":"*","title_prefix":"[ci-fix] "},"report_incomplete":{},"update_pull_request":{"allow_body":true,"allow_title":false,"max":3,"target":"*","update_branch":false}} - GH_AW_SAFE_OUTPUTS_CONFIG_fa2a69fad5fd0485_EOF + cat > "${RUNNER_TEMP}/gh-aw/safeoutputs/config.json" << 'GH_AW_SAFE_OUTPUTS_CONFIG_0644bf1434ca0071_EOF' + {"add_comment":{"discussions":false,"max":6,"required_labels":["agentic-workflows"],"required_title_prefix":"[ci-fix] ","target":"*"},"add_labels":{"allowed":["p/0"],"max":3,"required_labels":["agentic-workflows"],"required_title_prefix":"[ci-fix] ","target":"*"},"create_pull_request":{"allowed_base_branches":["main"],"allowed_branches":["ci-fix/**"],"allowed_files":["src/Core/**","src/Controls/**","src/Essentials/**","src/BlazorWebView/**","src/TestUtils/**","src/Templates/**","**/PublicAPI.Unshipped.txt"],"base_branch":"main","draft":true,"labels":["agentic-workflows"],"max":3,"max_patch_files":100,"max_patch_size":4096,"protect_top_level_dot_folders":true,"protected_files":["package.json","bun.lockb","bunfig.toml","deno.json","deno.jsonc","deno.lock","global.json","NuGet.Config","Directory.Packages.props","mix.exs","mix.lock","go.mod","go.sum","stack.yaml","stack.yaml.lock","pom.xml","build.gradle","build.gradle.kts","settings.gradle","settings.gradle.kts","gradle.properties","package-lock.json","yarn.lock","pnpm-lock.yaml","npm-shrinkwrap.json","requirements.txt","Pipfile","Pipfile.lock","pyproject.toml","setup.py","setup.cfg","Gemfile","Gemfile.lock","uv.lock","CODEOWNERS","DESIGN.md","README.md","CONTRIBUTING.md","CHANGELOG.md","SECURITY.md","CODE_OF_CONDUCT.md","AGENTS.md","CLAUDE.md","GEMINI.md"],"protected_files_policy":"request_review","title_prefix":"[ci-fix] "},"create_report_incomplete_issue":{},"mark_pull_request_as_ready_for_review":{"max":3,"required_labels":["agentic-workflows"],"required_title_prefix":"[ci-fix] ","target":"*"},"missing_data":{},"missing_tool":{},"noop":{"max":1,"report-as-issue":"true"},"push_to_pull_request_branch":{"allowed_files":["src/Core/**","src/Controls/**","src/Essentials/**","src/BlazorWebView/**","src/TestUtils/**","src/Templates/**","**/PublicAPI.Unshipped.txt"],"if_no_changes":"warn","max":3,"max_patch_size":4096,"protect_top_level_dot_folders":true,"protected_files":["package.json","bun.lockb","bunfig.toml","deno.json","deno.jsonc","deno.lock","global.json","NuGet.Config","Directory.Packages.props","mix.exs","mix.lock","go.mod","go.sum","stack.yaml","stack.yaml.lock","pom.xml","build.gradle","build.gradle.kts","settings.gradle","settings.gradle.kts","gradle.properties","package-lock.json","yarn.lock","pnpm-lock.yaml","npm-shrinkwrap.json","requirements.txt","Pipfile","Pipfile.lock","pyproject.toml","setup.py","setup.cfg","Gemfile","Gemfile.lock","uv.lock","CODEOWNERS","DESIGN.md","README.md","CONTRIBUTING.md","CHANGELOG.md","SECURITY.md","CODE_OF_CONDUCT.md","AGENTS.md","CLAUDE.md","GEMINI.md"],"required_labels":["agentic-workflows"],"target":"*","title_prefix":"[ci-fix] "},"report_incomplete":{},"update_pull_request":{"allow_body":true,"allow_title":false,"max":3,"target":"*","update_branch":false}} + GH_AW_SAFE_OUTPUTS_CONFIG_0644bf1434ca0071_EOF - name: Generate Safe Outputs Tools env: GH_AW_TOOLS_META_JSON: | { "description_suffixes": { - "add_comment": " CONSTRAINTS: Maximum 3 comment(s) can be added. Target: *. Supports reply_to_id for discussion threading.", + "add_comment": " CONSTRAINTS: Maximum 6 comment(s) can be added. Target: *. Supports reply_to_id for discussion threading.", + "add_labels": " CONSTRAINTS: Maximum 3 label(s) can be added. Only these labels are allowed: [\"p/0\"]. Target: *.", "create_pull_request": " CONSTRAINTS: Maximum 3 pull request(s) can be created. Title will be prefixed with \"[ci-fix] \". Labels [\"agentic-workflows\"] will be automatically added. Only these labels are allowed: [\"agentic-workflows\"]. PRs will be created as drafts.", + "mark_pull_request_as_ready_for_review": " CONSTRAINTS: Maximum 3 pull request(s) can be marked as ready for review.", "push_to_pull_request_branch": " CONSTRAINTS: Maximum 3 push(es) can be made. The target pull request title must start with \"[ci-fix] \".", "update_pull_request": " CONSTRAINTS: Maximum 3 pull request(s) can be updated. Target: *." }, @@ -594,6 +598,25 @@ jobs: } } }, + "add_labels": { + "defaultMax": 5, + "fields": { + "item_number": { + "issueNumberOrTemporaryId": true + }, + "labels": { + "required": true, + "type": "array", + "itemType": "string", + "itemSanitize": true, + "itemMaxLength": 128 + }, + "repo": { + "type": "string", + "maxLength": 256 + } + } + }, "create_pull_request": { "defaultMax": 1, "fields": { @@ -635,6 +658,24 @@ jobs: } } }, + "mark_pull_request_as_ready_for_review": { + "defaultMax": 1, + "fields": { + "pull_request_number": { + "issueOrPRNumber": true + }, + "reason": { + "required": true, + "type": "string", + "sanitize": true, + "maxLength": 65000 + }, + "repo": { + "type": "string", + "maxLength": 256 + } + } + }, "missing_data": { "defaultMax": 20, "fields": { @@ -1494,7 +1535,7 @@ jobs: uses: actions/github-script@3a2844b7e9c422d3c10d287c895573f7108da1b3 # v9.0.0 env: WORKFLOW_NAME: "CI Failure Fixer (main)" - WORKFLOW_DESCRIPTION: "Periodic pass over open ci-scan tracking issues filed by the main-branch CI\nfailure scanner (.github/workflows/ci-status-main.md). This workflow targets\nthe `main` branch EXCLUSIVELY: it processes only issues labelled ci-scan and\nopens every PR against main. (The net11.0 branch is handled by the parallel\n.github/workflows/ci-status-fix-net11.md — the two are split because gh-aw can\nonly transport a fix relative to ONE static base branch per workflow, and the\nmain↔net11.0 divergence exceeds gh-aw's 10 MB transport-patch cap.) The fixer\nopens ONE draft [ci-fix] PR per actionable issue against main, then WATCHES\nthat PR's own CI on later runs: when the fix's CI comes back red and the red is\ncaused by the fix itself, it pushes a fresh follow-up fix onto the SAME PR\nbranch (never a second PR) — up to 10 attempts — then stops and defers to\nhumans (the open tracking issue is the hand-off surface; a dedicated\n[ci-fix][needs-human] PR is planned but currently deferred — see Step 6). CI on\nthe PR is kicked by a human `/azp run` for now; the loop watches, classifies,\nand re-fixes autonomously between kicks.\nNever mutes tests, but\nde-flakes genuinely flaky ones (deterministic synchronization, no retries /\ntimeout bumps). Always skips visual-regression / screenshot issues." + WORKFLOW_DESCRIPTION: "Periodic pass over open ci-scan tracking issues filed by the main-branch CI\nfailure scanner (.github/workflows/ci-status-main.md). This workflow targets\nthe `main` branch EXCLUSIVELY: it processes only issues labelled ci-scan and\nopens every PR against main. (The net11.0 branch is handled by the parallel\n.github/workflows/ci-status-fix-net11.md — the two are split because gh-aw can\nonly transport a fix relative to ONE static base branch per workflow, and the\nmain↔net11.0 divergence exceeds gh-aw's 10 MB transport-patch cap.) The fixer\nopens ONE draft [ci-fix] PR per actionable issue against main, then WATCHES\nthat PR's own CI on later runs: when the fix's CI comes back red and the red is\ncaused by the fix itself, it pushes a fresh follow-up fix onto the SAME PR\nbranch (never a second PR) — up to 10 attempts — then stops and defers to\nhumans (the open tracking issue is the hand-off surface; a dedicated\n[ci-fix][needs-human] PR is planned but currently deferred — see Step 6). CI on\nthe PR is kicked by a human `/azp run` for now; the loop watches, classifies,\nand re-fixes autonomously between kicks. When the SPECIFIC test a PR fixed is\nconfirmed green in that PR's own CI, the loop marks the draft PR ready for review\n(a state transition only — it never approves or merges).\nNever mutes tests, but\nde-flakes genuinely flaky ones (deterministic synchronization, no retries /\ntimeout bumps). Always skips visual-regression / screenshot issues." HAS_PATCH: ${{ needs.agent.outputs.has_patch }} with: script: | @@ -1910,7 +1951,7 @@ jobs: GH_AW_ALLOWED_DOMAINS: "*.blob.core.windows.net,*.githubusercontent.com,api.business.githubcopilot.com,api.enterprise.githubcopilot.com,api.github.com,api.githubcopilot.com,api.individual.githubcopilot.com,api.snapcraft.io,archive.ubuntu.com,azure.archive.ubuntu.com,codeload.github.com,crl.geotrust.com,crl.globalsign.com,crl.identrust.com,crl.sectigo.com,crl.thawte.com,crl.usertrust.com,crl.verisign.com,crl3.digicert.com,crl4.digicert.com,crls.ssl.com,dev.azure.com,docs.github.com,github-cloud.githubusercontent.com,github-cloud.s3.amazonaws.com,github.blog,github.com,github.githubassets.com,helix.dot.net,host.docker.internal,json-schema.org,json.schemastore.org,keyserver.ubuntu.com,lfs.github.com,objects.githubusercontent.com,ocsp.digicert.com,ocsp.geotrust.com,ocsp.globalsign.com,ocsp.identrust.com,ocsp.sectigo.com,ocsp.ssl.com,ocsp.thawte.com,ocsp.usertrust.com,ocsp.verisign.com,packagecloud.io,packages.cloud.google.com,packages.microsoft.com,patch-diff.githubusercontent.com,ppa.launchpad.net,raw.githubusercontent.com,registry.npmjs.org,s.symcb.com,s.symcd.com,security.ubuntu.com,telemetry.enterprise.githubcopilot.com,ts-crl.ws.symantec.com,ts-ocsp.ws.symantec.com,www.googleapis.com" GITHUB_SERVER_URL: ${{ github.server_url }} GITHUB_API_URL: ${{ github.api_url }} - GH_AW_SAFE_OUTPUTS_HANDLER_CONFIG: "{\"add_comment\":{\"discussions\":false,\"max\":3,\"required_labels\":[\"agentic-workflows\"],\"required_title_prefix\":\"[ci-fix] \",\"target\":\"*\"},\"create_pull_request\":{\"allowed_base_branches\":[\"main\"],\"allowed_branches\":[\"ci-fix/**\"],\"allowed_files\":[\"src/Core/**\",\"src/Controls/**\",\"src/Essentials/**\",\"src/BlazorWebView/**\",\"src/TestUtils/**\",\"src/Templates/**\",\"**/PublicAPI.Unshipped.txt\"],\"base_branch\":\"main\",\"draft\":true,\"labels\":[\"agentic-workflows\"],\"max\":3,\"max_patch_files\":100,\"max_patch_size\":4096,\"protect_top_level_dot_folders\":true,\"protected_files\":[\"package.json\",\"bun.lockb\",\"bunfig.toml\",\"deno.json\",\"deno.jsonc\",\"deno.lock\",\"global.json\",\"NuGet.Config\",\"Directory.Packages.props\",\"mix.exs\",\"mix.lock\",\"go.mod\",\"go.sum\",\"stack.yaml\",\"stack.yaml.lock\",\"pom.xml\",\"build.gradle\",\"build.gradle.kts\",\"settings.gradle\",\"settings.gradle.kts\",\"gradle.properties\",\"package-lock.json\",\"yarn.lock\",\"pnpm-lock.yaml\",\"npm-shrinkwrap.json\",\"requirements.txt\",\"Pipfile\",\"Pipfile.lock\",\"pyproject.toml\",\"setup.py\",\"setup.cfg\",\"Gemfile\",\"Gemfile.lock\",\"uv.lock\",\"CODEOWNERS\",\"DESIGN.md\",\"README.md\",\"CONTRIBUTING.md\",\"CHANGELOG.md\",\"SECURITY.md\",\"CODE_OF_CONDUCT.md\",\"AGENTS.md\",\"CLAUDE.md\",\"GEMINI.md\"],\"protected_files_policy\":\"request_review\",\"title_prefix\":\"[ci-fix] \"},\"create_report_incomplete_issue\":{},\"missing_data\":{},\"missing_tool\":{},\"noop\":{\"max\":1,\"report-as-issue\":\"true\"},\"push_to_pull_request_branch\":{\"allowed_files\":[\"src/Core/**\",\"src/Controls/**\",\"src/Essentials/**\",\"src/BlazorWebView/**\",\"src/TestUtils/**\",\"src/Templates/**\",\"**/PublicAPI.Unshipped.txt\"],\"if_no_changes\":\"warn\",\"max\":3,\"max_patch_size\":4096,\"protect_top_level_dot_folders\":true,\"protected_files\":[\"package.json\",\"bun.lockb\",\"bunfig.toml\",\"deno.json\",\"deno.jsonc\",\"deno.lock\",\"global.json\",\"NuGet.Config\",\"Directory.Packages.props\",\"mix.exs\",\"mix.lock\",\"go.mod\",\"go.sum\",\"stack.yaml\",\"stack.yaml.lock\",\"pom.xml\",\"build.gradle\",\"build.gradle.kts\",\"settings.gradle\",\"settings.gradle.kts\",\"gradle.properties\",\"package-lock.json\",\"yarn.lock\",\"pnpm-lock.yaml\",\"npm-shrinkwrap.json\",\"requirements.txt\",\"Pipfile\",\"Pipfile.lock\",\"pyproject.toml\",\"setup.py\",\"setup.cfg\",\"Gemfile\",\"Gemfile.lock\",\"uv.lock\",\"CODEOWNERS\",\"DESIGN.md\",\"README.md\",\"CONTRIBUTING.md\",\"CHANGELOG.md\",\"SECURITY.md\",\"CODE_OF_CONDUCT.md\",\"AGENTS.md\",\"CLAUDE.md\",\"GEMINI.md\"],\"required_labels\":[\"agentic-workflows\"],\"target\":\"*\",\"title_prefix\":\"[ci-fix] \"},\"report_incomplete\":{},\"update_pull_request\":{\"allow_body\":true,\"allow_title\":false,\"max\":3,\"target\":\"*\",\"update_branch\":false}}" + GH_AW_SAFE_OUTPUTS_HANDLER_CONFIG: "{\"add_comment\":{\"discussions\":false,\"max\":6,\"required_labels\":[\"agentic-workflows\"],\"required_title_prefix\":\"[ci-fix] \",\"target\":\"*\"},\"add_labels\":{\"allowed\":[\"p/0\"],\"max\":3,\"required_labels\":[\"agentic-workflows\"],\"required_title_prefix\":\"[ci-fix] \",\"target\":\"*\"},\"create_pull_request\":{\"allowed_base_branches\":[\"main\"],\"allowed_branches\":[\"ci-fix/**\"],\"allowed_files\":[\"src/Core/**\",\"src/Controls/**\",\"src/Essentials/**\",\"src/BlazorWebView/**\",\"src/TestUtils/**\",\"src/Templates/**\",\"**/PublicAPI.Unshipped.txt\"],\"base_branch\":\"main\",\"draft\":true,\"labels\":[\"agentic-workflows\"],\"max\":3,\"max_patch_files\":100,\"max_patch_size\":4096,\"protect_top_level_dot_folders\":true,\"protected_files\":[\"package.json\",\"bun.lockb\",\"bunfig.toml\",\"deno.json\",\"deno.jsonc\",\"deno.lock\",\"global.json\",\"NuGet.Config\",\"Directory.Packages.props\",\"mix.exs\",\"mix.lock\",\"go.mod\",\"go.sum\",\"stack.yaml\",\"stack.yaml.lock\",\"pom.xml\",\"build.gradle\",\"build.gradle.kts\",\"settings.gradle\",\"settings.gradle.kts\",\"gradle.properties\",\"package-lock.json\",\"yarn.lock\",\"pnpm-lock.yaml\",\"npm-shrinkwrap.json\",\"requirements.txt\",\"Pipfile\",\"Pipfile.lock\",\"pyproject.toml\",\"setup.py\",\"setup.cfg\",\"Gemfile\",\"Gemfile.lock\",\"uv.lock\",\"CODEOWNERS\",\"DESIGN.md\",\"README.md\",\"CONTRIBUTING.md\",\"CHANGELOG.md\",\"SECURITY.md\",\"CODE_OF_CONDUCT.md\",\"AGENTS.md\",\"CLAUDE.md\",\"GEMINI.md\"],\"protected_files_policy\":\"request_review\",\"title_prefix\":\"[ci-fix] \"},\"create_report_incomplete_issue\":{},\"mark_pull_request_as_ready_for_review\":{\"max\":3,\"required_labels\":[\"agentic-workflows\"],\"required_title_prefix\":\"[ci-fix] \",\"target\":\"*\"},\"missing_data\":{},\"missing_tool\":{},\"noop\":{\"max\":1,\"report-as-issue\":\"true\"},\"push_to_pull_request_branch\":{\"allowed_files\":[\"src/Core/**\",\"src/Controls/**\",\"src/Essentials/**\",\"src/BlazorWebView/**\",\"src/TestUtils/**\",\"src/Templates/**\",\"**/PublicAPI.Unshipped.txt\"],\"if_no_changes\":\"warn\",\"max\":3,\"max_patch_size\":4096,\"protect_top_level_dot_folders\":true,\"protected_files\":[\"package.json\",\"bun.lockb\",\"bunfig.toml\",\"deno.json\",\"deno.jsonc\",\"deno.lock\",\"global.json\",\"NuGet.Config\",\"Directory.Packages.props\",\"mix.exs\",\"mix.lock\",\"go.mod\",\"go.sum\",\"stack.yaml\",\"stack.yaml.lock\",\"pom.xml\",\"build.gradle\",\"build.gradle.kts\",\"settings.gradle\",\"settings.gradle.kts\",\"gradle.properties\",\"package-lock.json\",\"yarn.lock\",\"pnpm-lock.yaml\",\"npm-shrinkwrap.json\",\"requirements.txt\",\"Pipfile\",\"Pipfile.lock\",\"pyproject.toml\",\"setup.py\",\"setup.cfg\",\"Gemfile\",\"Gemfile.lock\",\"uv.lock\",\"CODEOWNERS\",\"DESIGN.md\",\"README.md\",\"CONTRIBUTING.md\",\"CHANGELOG.md\",\"SECURITY.md\",\"CODE_OF_CONDUCT.md\",\"AGENTS.md\",\"CLAUDE.md\",\"GEMINI.md\"],\"required_labels\":[\"agentic-workflows\"],\"target\":\"*\",\"title_prefix\":\"[ci-fix] \"},\"report_incomplete\":{},\"update_pull_request\":{\"allow_body\":true,\"allow_title\":false,\"max\":3,\"target\":\"*\",\"update_branch\":false}}" GH_AW_CI_TRIGGER_TOKEN: ${{ secrets.GH_AW_CI_TRIGGER_TOKEN }} with: github-token: ${{ secrets.GH_AW_GITHUB_TOKEN || secrets.GITHUB_TOKEN }} diff --git a/.github/workflows/ci-status-fix.md b/.github/workflows/ci-status-fix.md index d3227e25170c..9c2f46054029 100644 --- a/.github/workflows/ci-status-fix.md +++ b/.github/workflows/ci-status-fix.md @@ -15,7 +15,9 @@ description: | humans (the open tracking issue is the hand-off surface; a dedicated [ci-fix][needs-human] PR is planned but currently deferred — see Step 6). CI on the PR is kicked by a human `/azp run` for now; the loop watches, classifies, - and re-fixes autonomously between kicks. + and re-fixes autonomously between kicks. When the SPECIFIC test a PR fixed is + confirmed green in that PR's own CI, the loop marks the draft PR ready for review + (a state transition only — it never approves or merges). Never mutes tests, but de-flakes genuinely flaky ones (deterministic synchronization, no retries / timeout bumps). Always skips visual-regression / screenshot issues. @@ -239,8 +241,15 @@ safe-outputs: # PER-RUN total (not per-PR): a sweep may surface several green PRs and/or # annotate several flaky reds in one run. At max:1 all but the first comment # were silently dropped, starving the workflow's #1 value (surfacing green PRs - # for review). Raised to 3 to match the per-run throughput of the other outputs. - max: 3 + # for review). Sized to 6 (not 3) because a single green DRAFT PR that gets + # flipped ready in one sweep spends TWO comment slots — the Step 3 ✅ surface + # comment (which still names the /azp-gated legs a human must kick, so it is NOT + # redundant with 🎯) AND the Step 3.6 T3 🎯 readiness comment. At max:3 the shared + # bucket drained after ~1 draft flip and T3's atomicity pre-check then deferred + # every further mark-ready even while the mark-ready/add_labels buckets (also 3) + # sat idle; 6 lets ~3 draft flips (2 comments each) land per sweep, so the + # mark-ready:3 / add_labels:3 caps are actually reachable. + max: 6 target: "*" # Hard constraint (defense-in-depth): only comment on THIS workflow's own # ci-fix PRs — [ci-fix] title prefix AND agentic-workflows label. Mirrors the @@ -261,13 +270,51 @@ safe-outputs: # never the title, so disable title rewrites — this compiles to allow_title:false # and removes any ability to retitle an arbitrary PR. title: false - # NOTE: gh-aw v0.79.8 does NOT emit required-title-prefix/required-labels into + # NOTE: gh-aw v0.80.9 (the pinned compiler) does NOT emit required-title-prefix/required-labels into # the compiled config for update-pull-request (verified against the lock — it # silently drops them, unlike add-comment / push-to-pull-request-branch which # honor them). So which-PR scoping here relies on prompt Hard-Rule 6 + # min-integrity:approved; the residual is body-marker edits only (no code, no # merge, no close), capped at max:3. Revisit required-* if a future gh-aw # compiler emits them for this output. + mark-pull-request-as-ready-for-review: + # Flip a validated draft [ci-fix] PR to ready-for-review once the SPECIFIC test + # this PR was opened to fix is confirmed green in the PR's OWN CI (Step 3.6) — + # even when unrelated legs are red. The workflow is schedule/dispatch-triggered + # (no triggering-PR context), so target "*" lets the agent name the PR number it + # validated. This is a state transition ONLY: it never approves and never merges — + # a human still reviews and merges. + # PER-RUN total (not per-PR): a sweep may validate several PRs' target tests green. + max: 3 + target: "*" + # Hard constraint (defense-in-depth): only ever un-draft THIS workflow's own + # ci-fix PRs — [ci-fix] title prefix AND agentic-workflows label — so a confused + # or prompt-injected agent cannot mark an arbitrary PR ready. (If the v0.80.9 + # compiler silently drops these — as documented for update-pull-request above — + # the Step 3.6 preconditions + min-integrity:approved are the compensating scope + # controls; verify against the lock after compiling.) + required-title-prefix: "[ci-fix] " + required-labels: [agentic-workflows] + add-labels: + # Apply the p/0 priority label to a [ci-fix] PR at the exact moment the loop flips it + # from draft to ready-for-review (Step 3.6 T3) — so a validated, review-ready fix lands + # in the team's p/0 triage queue instead of sitting unseen in the draft backlog. Paired + # 1:1 with mark-pull-request-as-ready-for-review; target "*" lets the agent name the PR + # it just validated. + # allowed = HARD allowlist: the agent may ONLY ever add p/0, nothing else. This caps the + # blast radius of a confused/prompt-injected agent to exactly one benign priority label — + # it can never apply a downstream-triggering or destructive label. + allowed: [p/0] + # PER-RUN total (not per-PR): a sweep may mark several PRs ready in one run; each adds + # one label, so this matches the mark-ready per-run cap (3). + max: 3 + target: "*" + # Hard constraint (defense-in-depth): only ever label THIS workflow's own ci-fix PRs — + # [ci-fix] title prefix AND agentic-workflows label. (If the v0.80.9 compiler drops these + # — as documented for update-pull-request above — the Step 3.6 preconditions + + # min-integrity:approved + the allowed:[p/0] allowlist are the compensating controls.) + required-title-prefix: "[ci-fix] " + required-labels: [agentic-workflows] timeout-minutes: 90 @@ -339,33 +386,48 @@ through `safe-outputs`. 5. **One issue = one outcome per run.** Exactly one of: respond to a maintainer's change-request on the open PR (Track C, Step 3.5.R); open the first fix/help/ de-flake PR; advance an existing PR by one attempt (push a follow-up fix); - surface a validated-green PR for review; annotate an unrelated-flake red; wait + surface a validated-green PR for review; mark a target-validated draft PR + ready-for-review (Step 3.6 T3 — the terminal outcome that supersedes the same + run's surface/annotate precursor line), or record its `already-ready` + steady-state no-op; annotate an unrelated-flake red; wait (CI not yet settled); or a recorded skip (the dedicated needs-human PR is deferred — the attempt cap records a skip instead; see Step 6). Always prefer advancing or opening a PR over a skip when a non-mute diff is producible. 6. **All writes via `safe-outputs`.** Allowed outputs: `create_pull_request` (first attempt only), `push_to_pull_request_branch` (advance an existing PR by one attempt), `update_pull_request` (bump the attempt marker / refresh the - prior-attempts table), and `add_comment` (progress notes on the `[ci-fix]` PR - ONLY). NEVER comment on the tracking issue (issues are locked by + prior-attempts table), `add_comment` (progress notes on the `[ci-fix]` PR + ONLY), `mark_pull_request_as_ready_for_review` (flip a target-validated draft + `[ci-fix]` PR from draft to ready — Step 3.6 T3 only), and `add_labels` (add + ONLY the `p/0` label, ONLY on that same draft→ready transition — Step 3.6 T3). + NEVER comment on the tracking issue (issues are locked by `.github/workflows/ci-scan-lock-issues.yml`) — `add_comment` targets the PR. No `gh pr create`, no manual `git push`. **Defense-in-depth:** `add_comment` and `update_pull_request` use `target: "*"` (the agent supplies the PR number). - `add_comment` is now config-locked to `required-title-prefix: "[ci-fix] "` + + `add_comment`, `mark_pull_request_as_ready_for_review`, and `add_labels` are + config-locked to `required-title-prefix: "[ci-fix] "` + `required-labels: [agentic-workflows]` (hard-enforced by the handler, same as - `push_to_pull_request_branch`), so a comment can only ever land on THIS - workflow's own PRs. `update_pull_request` CANNOT be config-locked to a - title/label in gh-aw v0.79.8 (the compiler silently drops `required-*` for that - output — verified against the lock), so it keeps `title: false` (no retitles; - body-marker edits only) plus this prompt-level guard. Before emitting either, - VERIFY the target PR carries BOTH the `[ci-fix]` title prefix AND the - `agentic-workflows` label; never comment on or edit the body of any PR that - lacks both. + `push_to_pull_request_branch`), and `add_labels` additionally has an + `allowed: [p/0]` allowlist so it can ONLY ever add `p/0` — so these can only + ever land on THIS workflow's own PRs. `update_pull_request` CANNOT be + config-locked to a title/label in gh-aw v0.80.9 (the compiler silently drops + `required-*` for that output — verified against the lock), so it keeps + `title: false` (no retitles; body-marker edits only) plus this prompt-level + guard. Before emitting ANY of these, VERIFY the target PR carries BOTH the + `[ci-fix]` title prefix AND the `agentic-workflows` label; never comment on, + edit, un-draft, or label any PR that lacks both. 7. **Per-run safe-output caps (per RUN, not per PR).** Each output type is capped per run: `create_pull_request` 3, `push_to_pull_request_branch` 3, - `add_comment` 3, `update_pull_request` 3. Note an ADVANCE spends one push **and** - one update **and** one comment, so ≤ 3 advances/run; surfacing a green or - annotating a flake spends one comment. When a bucket is exhausted, do NOT keep + `add_comment` 6, `update_pull_request` 3, `mark_pull_request_as_ready_for_review` + 3, `add_labels` 3 (counts LABELS, not calls). Note an ADVANCE spends one push + **and** one update **and** one comment, so ≤ 3 advances/run; surfacing a green or + annotating a flake spends one comment; a **mark-ready (Step 3.6 T3)** of a + still-draft PR spends TWO comments (the Step 3 ✅ surface **and** the T3 🎯 + readiness note) **and** one mark-ready **and** one label as an ALL-OR-NOTHING set + (T3 pre-checks those buckets and defers the whole PR if any is exhausted — never + mark a PR ready without its 🎯 audit comment). `add_comment` is therefore sized 6 + (not 3) so ~3 draft flips, at 2 comments each, can land in one sweep instead of the + shared comment bucket starving the otherwise-idle mark-ready/add_labels buckets. When a bucket is exhausted, do NOT keep emitting (extras are silently dropped) — record `skipped: per-run cap reached; deferring PR # to next cycle` for each remaining PR so the drop is a deliberate, logged decision. The continuous loop picks the deferred PRs up next @@ -409,8 +471,9 @@ For every open tracking issue in scope, converge on exactly one outcome: | Open first help-wanted draft `[ci-fix]` PR | No open PR yet; a plausible candidate exists but cannot be runner-validated (device/UI tests) (attempt 1) | | Open first de-flake draft `[ci-fix]` PR | No open PR yet; failure is intermittent (green-on-retry) from a genuine test-quality defect; a deterministic-synchronization fix is producible (attempt 1, Step 4.7 bucket b) | | Advance an existing PR (push attempt N+1) | Open PR's own CI settled red *because of the fix itself*, no human engaged, marker < 10 — push a NEW distinct fix onto the same branch (Step 3.5 → 5.6) | -| Surface a validated-green PR | Open PR's own CI settled green — comment "validated, attempt N/10; ready for review" and do NOT advance (Step 3.5) | -| Annotate an unrelated-flake red | Open PR is red only on baseline-flake / unrelated legs — comment which leg needs a re-run; do NOT burn an attempt (Step 3.5) | +| Surface a validated-green PR | Open PR's own CI settled green — comment "validated, attempt N/10; ready for review" and do NOT advance (Step 3.5), then mark the draft PR ready for review once the fixed test is confirmed green (Step 3.6) | +| Annotate an unrelated-flake red | Open PR is red only on baseline-flake / unrelated legs — comment which leg needs a re-run; do NOT burn an attempt (Step 3.5); if the SPECIFIC fixed test is confirmed green on that build, still mark the draft PR ready for review (Step 3.6) | +| Mark a validated PR ready for review | The specific test this PR fixed is confirmed green in the PR's own CI (even if unrelated legs are red) — comment the target-test result and transition the draft PR to ready for review; never approve or merge (Step 3.6) | | Wait | Open PR's CI is pending / not yet settled on the current head SHA — do nothing this cycle (Step 3.5) | | Hand-off skip (attempt cap) | Marker == 10 and the signature still reproduces — stop and defer to humans; the dedicated `[ci-fix][needs-human]` PR is planned but currently deferred (Step 6) | | Recorded skip | Visual-regression, human engaged, already-handled, fixed-in-latest-build, infra-flake, out-of-bounds, only-mute-available, or no novel approach producible | @@ -685,14 +748,34 @@ Run these gates in order — the FIRST that fires decides this cycle's outcome: 1. **Human engaged.** If `C.humanEngaged` → `skipped: human engaged on PR #; deferring` and stop. Never fight a human reviewer — the intentional hand-off boundary still holds the moment a person touches the PR. -2. **CI not settled / unknown / incomplete prefetch.** If `C.checksSettled == false` - OR `C.overallConclusion` is `pending`, `neutral`, or `unknown`, OR - `C.dataComplete == false` → the fix's CI has not finished on the current head SHA - (`C.headSha`), or the prefetch could not fully read this PR (a partial read can - understate `humanEngaged` / `attempt`). `skipped: PR # CI pending / prefetch - incomplete on ; waiting` and stop. *(Round 1: this is where a - maintainer `/azp run` is awaited — the `/azp`-gated uitests/devicetests legs will - not have run until a human kicks them.)* +2. **CI not settled — but validate the target test first (target-focused readiness).** + If `C.checksSettled == false` OR `C.overallConclusion` is `pending`, `neutral`, or + `unknown`, OR `C.dataComplete == false` → the *overall* build has not finished on the + current head SHA (`C.headSha`), or the prefetch could not fully read this PR (a partial + read can understate `humanEngaged` / `attempt`). Unrelated legs still draining must NOT + keep an already-proven fix parked as a draft, so before waiting, try the target-focused + fast-path: + - **2a — Target-green fast-path (draft `[ci-fix]` PRs only).** If `C.dataComplete == + true` AND the Step 3.6 preconditions hold (draft PR, `[ci-fix] ` title prefix AND the + `agentic-workflows` label), run Step 3.6 **T1–T2** against `C.headSha` now: identify + the target test(s) and read the PR's OWN build **timeline / per-leg status** (anonymous + `_apis/build` — never `_apis/test`, per Hard-Rule 8). If EVERY target test is + **VALIDATED-GREEN** (per T2's all-platform definition below — the target's category leg + is `succeeded` on every platform that runs it, failed on none, and NO target platform + family the pipeline covers is still pending/unconcluded, per T2's "no platform left + unverified" rule; a platform that simply has no leg for the target's category — the test + doesn't run there — is fine, not a gap) AND every leg in + `C.failedLegs` that has ALREADY concluded classifies as **unrelated flake** (Step 4 / + Step 4.7 method — the fix is implicated in NO completed red), then the fix is proven + regardless of unrelated *pending* legs → run **Step 3.6 T3** (mark ready + 🎯 comment) + and stop. This early-out NEVER advances an attempt and NEVER acts on a red that could + be the fix's fault. + - **2b — Otherwise WAIT.** If the target test has not yet executed (pending/absent on + its leg), or a completed red is (or may be) caused by the fix, or `C.dataComplete == + false`, or this PR has no identifiable target test → `skipped: PR # CI pending / + target not yet validated on ; waiting` and stop. *(Round 1: this is where a + maintainer `/azp run` is awaited — the `/azp`-gated uitests/devicetests legs will not + have run until a human kicks them.)* 3. **Green → surface for review.** If `C.overallConclusion == "success"`: the checks that RAN are green. **Primary-gate check first:** the deterministic `success` verdict only certifies "at least one green check, nothing failing or @@ -704,18 +787,31 @@ Run these gates in order — the FIRST that fires decides this cycle's outcome: # primary CI gate (maui-pr) not green on ; waiting` and stop — do NOT surface. (The `/azp`-gated `maui-pr-uitests` (def 313) / `maui-pr-devicetests` (def 314) legs MAY still be un-run — that is expected and is named below, not a - reason to withhold the surface.) **Idempotency:** scan the PR's existing comments - for a prior bot `✅ … validated … on ` note for THIS head SHA — if one - already exists, record `already-surfaced PR # (head )` and stop - WITHOUT re-commenting (re-surfacing the same green every tick is noise and burns - the per-run comment budget other PRs need). Otherwise resolve the attempt number from - `C.effectiveAttempt` (the authoritative max(marker, bot-commit) counter; omit the - number only if it is somehow indeterminate). **Dry-run gate (Step 0):** if `dry_run == "true"`, do NOT emit any comment — instead print the intended `✅` validated-green body to the run log, tally `dry-run: would-surface-green PR #`, and stop. Otherwise `add_comment` on PR #: `✅ Attempt /10 validated - — the fix's CI is green on . Ready for human review.` - Name any `/azp`-gated legs (uitests def 313 / devicetests def 314) that have not - run and still need a maintainer `/azp run`. Do NOT advance. Record - `surfaced-green PR # (attempt /10)` and stop. *(This directly - attacks the real bottleneck — no reviews — so it is the highest-value outcome.)* + reason to withhold the surface.) **Comment idempotency + dry-run suppress the ✅ + comment ONLY — neither skips Step 3.6.** Scan the PR's existing comments for a prior + bot `✅ … validated … on ` note for THIS head SHA; if one exists, set + `SKIP_SURFACE_COMMENT = true`. Resolve the attempt number from `C.effectiveAttempt` + (the authoritative max(marker, bot-commit) counter; omit the number only if it is + somehow indeterminate). Then post the surface comment UNLESS suppressed: + - if `dry_run == "true"`: do NOT emit — print the intended `✅` validated-green body to + the run log and tally `dry-run: would-surface-green PR #`; + - else if `SKIP_SURFACE_COMMENT`: do NOT re-post — record `already-surfaced PR # + (head )` (re-surfacing the same green every tick is noise and burns the + per-run comment budget other PRs need); + - else `add_comment` on PR #: `✅ Attempt /10 validated — the fix's CI is + green on .` naming any `/azp`-gated legs (uitests def 313 / devicetests def + 314) that have not run and still need a maintainer `/azp run`; record `surfaced-green + PR # (attempt /10)`. (Do NOT assert "ready for human review" on a PR that + is still a draft — Step 3.6, not this comment, owns the draft→ready flip and posts its + own 🎯 announcement when it fires.) + Do NOT advance. Then — in ALL of the above cases — run **Step 3.6** (target-test + readiness gate) for this PR before stopping: its `/azp`-gated target legs may conclude + green on this SAME head SHA on a LATER sweep (an `/azp run` adds no commit), and Step + 3.6 is the ONLY place the draft→ready flip happens, so it MUST re-evaluate every sweep — + never `stop` here before it. *(This directly attacks the real bottleneck — no reviews — + so it is the highest-value outcome.)* 4. **Red → classify caused-by-fix vs unrelated-flake.** If `C.overallConclusion == "failure"`, analyze the PR's OWN failing build (NOT `main`): find the AzDO `maui-pr` build for this PR (filter builds by `branchName=refs/pull//merge`, @@ -723,18 +819,26 @@ Run these gates in order — the FIRST that fires decides this cycle's outcome: method and Step 4.7 flake buckets to `C.failedLegs`. - **Unrelated flake only** (every failed leg is known-flaky / infra / a pre-existing baseline red NOT introduced by the fix): do NOT burn an attempt. - **Idempotency first:** scan the PR's existing comments for a prior bot - `♻️ … unrelated flake … on ` note for THIS head SHA — if one already - exists, record `already-annotated-flake PR # (head )` and stop - WITHOUT re-commenting (re-annotating the same flake every 12h sweep is noise and - burns the per-run comment budget other PRs need). Otherwise resolve the attempt - number from `C.effectiveAttempt` (the authoritative max(marker, bot-commit) - counter), omitting the `Attempt /10:` prefix only if it is somehow - indeterminate. **Dry-run gate (Step 0):** if `dry_run == "true"`, do NOT emit any comment — instead print the intended `♻️` unrelated-flake body to the run log, tally `dry-run: would-annotate-flake PR #`, and stop. Otherwise `add_comment` on PR #: `♻️ Attempt /10: red is - unrelated flake on leg(s) () on ; the fix itself is not - implicated. A maintainer re-run (/azp run ) should clear it.` Record - `annotated-flake PR # (head )` and stop. *(Round 1: human re-runs; - Round 2: auto re-trigger.)* + **Comment idempotency + dry-run suppress the ♻️ comment ONLY — neither skips Step + 3.6.** Scan the PR's existing comments for a prior bot `♻️ … unrelated flake … on + ` note for THIS head SHA; if one exists, set `SKIP_FLAKE_COMMENT = true`. + Resolve the attempt number from `C.effectiveAttempt` (the authoritative max(marker, + bot-commit) counter), omitting the `Attempt /10:` prefix only if it is + somehow indeterminate. Then post the flake note UNLESS suppressed: + - if `dry_run == "true"`: do NOT emit — print the intended `♻️` unrelated-flake body + to the run log and tally `dry-run: would-annotate-flake PR #`; + - else if `SKIP_FLAKE_COMMENT`: do NOT re-post — record `already-annotated-flake PR + # (head )` (re-annotating the same flake every 12h sweep is noise and + burns the per-run comment budget other PRs need); + - else `add_comment` on PR #: `♻️ Attempt /10: red is unrelated flake on + leg(s) () on ; the fix itself is not implicated. A + maintainer re-run (/azp run ) should clear it.` record `annotated-flake + PR # (head )`. + Then — in ALL of the above cases — run **Step 3.6** (target-test readiness gate) for + this PR before stopping: its `/azp`-gated target legs may conclude green on this SAME + head SHA on a LATER sweep, and Step 3.6 is the ONLY place the draft→ready flip + happens, so it MUST re-evaluate every sweep — never `stop` here before it. *(Round 1: + human re-runs; Round 2: auto re-trigger.)* - **Caused by the fix** (a failed leg still matches the original target signature, or the fix introduced a NEW failure): advance an attempt. - **Attempt count.** `attempt = C.effectiveAttempt` — the authoritative @@ -752,6 +856,188 @@ Run these gates in order — the FIRST that fires decides this cycle's outcome: from every prior commit on this branch**, emitted via the Step 5.6 ADVANCE path (push onto the same PR). +#### Step 3.6 — Target-test verification & mark-ready gate + +Reached from the Step 3 **green-surface** branch, the Step 4 **unrelated-flake** branch +(AFTER that branch has posted its comment), and the Step 3.5 **gate-2 target-focused +fast-path** (2a — while unrelated legs are still pending). Purpose: confirm the SPECIFIC +test(s) this PR was opened to fix now PASS in the PR's own CI, and — only when they do +— transition the draft `[ci-fix]` PR to **ready for review** so a maintainer sees a +validated fix instead of a draft. This is the ONLY place the loop flips draft→ready; it +is a state transition, **never** an approval or a merge (a human still reviews and +merges). Overall red on *unrelated* legs must NOT gate readiness — we validate the fix, +not the base branch's flakiness. + +**Preconditions** (ALL must hold; otherwise record `skipped: readiness N/A PR # +()` and stop this gate): +- `C.isDraft == true` — if the PR is already ready-for-review, the draft→ready + transition is already done; do NOT re-mark **and do NOT re-apply `p/0`**. `p/0` is applied + exactly ONCE, atomically with the draft→ready flip (T3 — all-or-nothing with the 🎯 audit + comment + mark-ready), so an already-ready loop PR that is MISSING `p/0` has almost + certainly had it **removed by a maintainer** de-prioritizing the PR — a pure triage action + the human-engagement guard (which inspects comments/reviews/commits, NOT label changes) + cannot see. Re-adding `p/0` every sweep would fight that maintainer indefinitely, violating + the loop's "never override a human" contract. So the loop does NOT reconcile the label: + record `already-ready PR #` and stop this gate. (Trade-off: on the rare occasion a + transient API error drops `p/0` at flip time *after* the T3 atomic pre-check passed, it is + not auto-re-added — but the PR is still ready-for-review with its 🎯 audit comment, and + re-adding `p/0` is a trivial manual action; that is strictly preferable to steam-rolling a + maintainer's deliberate de-prioritization.) +- The PR is unmistakably THIS workflow's own: `[ci-fix] ` title prefix AND the + `agentic-workflows` label (mirrors the safe-output lock; the compensating scope + control if the v0.80.9 compiler drops the declarative required-*). +- You reached this gate from Step 3 (green), Step 4 (**unrelated-flake**), or the Step 3.5 + gate-2 **target-focused fast-path** (2a) — i.e. the fix is NOT implicated in any red that + has concluded. If Step 4 classified a red as **caused by the fix** (ADVANCE), do NOT run + this gate — advance the attempt instead. + +**T1 — Identify the target test(s).** From the `[ci-scan]` issue signature the fix +addresses (and the PR's own diff), extract the fully-qualified test method name(s) the +fix targets — e.g. `SafeAreaShouldWorkOnAllShellTabs`. For a de-flake it is the +de-flaked test; for a product fix it is the originally-failing test(s). If NO specific +test can be identified (e.g. a product build-break rather than a test failure), this is a +**build-only fix** — there is no single test to validate cross-platform, so validate at +**whole-build** granularity rather than stopping (Step 3 only *comments*; Step 3.6 is the +sole draft→ready flip, so a build-only fix is undrafted HERE or not at all). We only reach +this gate from Step 3 (green) / Step 4 (unrelated-flake) / gate-2a, so the fix is not +implicated in any concluded red; additionally require, via the T2 timeline method on +`C.headSha`, that the primary build pipeline `maui-pr` (def 302) has CONCLUDED with EVERY +one of its platform build legs `succeeded`/`completed` and NONE failed/canceled or still +pending — the build is green on **every** platform, not just the originally-broken one (the +cross-platform guard, applied to the build instead of a test). A build-only fix is undrafted +ONLY on the strength of the auto-running `maui-pr` (def 302) whole-build green, which the +loop CAN observe: if the PR's `[ci-scan]` issue signature or its own diff indicates the +ORIGINATING failure was in a `/azp`-gated pipeline (`maui-pr-uitests` def 313 or +`maui-pr-devicetests` def 314) rather than `maui-pr` (def 302), do NOT undraft on +`maui-pr`-green alone — those pipelines do not auto-run on this PR (GITHUB_TOKEN cannot +trigger them), so a green `maui-pr` build is NOT evidence the gated build break is fixed; +record `skipped: build-only fix PR # targets gated pipeline () not run — +deferring to human` and stop this gate. Otherwise, set `TARGET := "the maui-pr build +(build-only fix — no single target test)"` and proceed to **T3** to mark ready. If any `maui-pr` build leg is still unconcluded, record `skipped: build-only fix PR +# not yet whole-build green (leg(s) pending)` and stop this gate WITHOUT marking +ready (a green subset is not enough — a still-pending build leg could yet fail). + +**T2 — Verify the target test's platform legs are green (anonymous, leg-level).** Per-test +outcomes come from the AzDO **test-results** API (`_apis/test/...`), which is **NOT reachable +anonymously — Hard-Rule 8**: those endpoints 302-redirect to a sign-in page on this runner, so +a `curl` returns an HTML redirect (not JSON) and every target test would look "not executed". +Validate instead at **leg granularity** using ONLY the anonymous `_apis/build/...` timeline +(Hard-Rule 8) — a `succeeded` category leg means every test in that category **that actually +ran** passed on that platform. This is a strong bar **only if the target test genuinely +executes**: a job can go green while one specific test is skipped at runtime (`[Ignore]`, +`Assert.Ignore()`, `Assert.Inconclusive()`, a `Skip=`/conditional `[Fact]`, an `#if`-out, or +an early `return` before the asserts). So before trusting a green leg as proof the target +passed, **confirm from the PR's OWN diff that the fix does NOT skip, ignore, disable, or +short-circuit the target test** (it adds no `[Ignore]`/`[Explicit]`, `Assert.Ignore`/ +`Assert.Inconclusive`, `Skip=`, category-exclusion, `#if`-out, or early `return` around the +target). If the fix could cause the target to be runtime-skipped rather than genuinely pass, +leg-level green is NOT sufficient evidence — record `skipped: target test may be +runtime-skipped by this fix (diff adds ); leg-green insufficient, deferring to human +on PR #` and stop this gate WITHOUT marking ready. Otherwise (the target genuinely runs +and asserts, as a de-flake or product fix does) a green category leg cannot hide a +target-test failure, so treat it as authoritative. + +Map the target test to the CI leg(s) that run it. A leg name encodes platform + test category +(e.g. `Android UITests SafeAreaEdges,Shadow`, `iOS UITests SafeAreaEdges,Shadow`, `macOS +UITests SafeAreaEdges,Shadow`, `Windows UITests SafeAreaEdges,Shadow`). Determine the target +test's UI-test category — its `[Category(UITestCategories.X)]` in the test/HostApp source, +visible in the PR diff or the test file — to know which leg-name substring identifies its legs; +for a device test, the per-platform device-test legs. + +Using the SAME build-discovery as Step 4 (filter AzDO builds by `branchName=refs/pull//merge` +or `sourceVersion == C.headSha`), read each build's **timeline** on `C.headSha` for the +pipeline(s) that RUN the target test — `maui-pr` (def 302) for unit/integration tests, +`maui-pr-uitests` (def 313) for Appium UI tests, `maui-pr-devicetests` (def 314) for device +tests: + +```bash +ORG=dnceng-public; PROJ=public; BUILD= +# Anonymous + Hard-Rule-8-compliant. NEVER call _apis/test/... (it 302-redirects to sign-in). +# Each type=="Job" record is one platform leg: result (succeeded/failed/canceled/null), +# state (completed/inProgress/pending), name (encodes " UITests "): +curl -s "https://dev.azure.com/$ORG/$PROJ/_apis/build/builds/$BUILD/timeline?api-version=7.1" \ + | tee /tmp/gh-aw/agent/timeline_${BUILD}.json \ + | jq -r '.records[] | select(.type=="Job") | "\(.result)\t\(.state)\t\(.name)"' +``` + +(The prefetch already exposes per-leg status in `C.failedLegs` / the checks context, and +`gh pr checks ` lists the same per-leg rows with platform+category in the name — use +whichever is handy; the build timeline is the authoritative cross-check on the exact build.) + +Treat a target test as **VALIDATED-GREEN** only if, on `C.headSha`, the leg that runs its +category is green on **every platform it runs on** — a fix that repairs one platform must not +silently regress the same test's leg on another, so a green leg on the originally-red platform +alone is NOT enough. Across the target pipeline's platform legs (for a UI test: the Android, +iOS, Windows, and macOS/MacCatalyst legs running the target's category; for a device test: each +device-test platform), require ALL of: +- the target's category leg is `result == "succeeded"` **and** `state == "completed"` on + **every** platform that runs it, AND +- that leg is `failed` / `canceled` / aborted on **NO** platform, AND +- **no platform is left unverified.** For each platform family the target pipeline covers, that + platform's category leg must have CONCLUDED (`state == "completed"`) on `C.headSha`. If a + platform simply has no leg for the target's category, the test does not run there — that is + fine, not a gap. But if a platform's category leg has **not concluded** (`state` is + `inProgress` / pending, or an `/azp`-gated `maui-pr-uitests` / `maui-pr-devicetests` leg that + has not been kicked), the test's status on that platform is UNKNOWN → the fix is NOT yet + validated across platforms: record `skipped: target test green on but + not yet verified on (leg(s) pending / need /azp run) on PR #` + and stop this gate WITHOUT marking ready. + +A target test whose category leg never concluded on ANY platform (**not executed** anywhere — +e.g. its `/azp`-gated pipeline has not been kicked) is likewise NOT validated: record `skipped: +target test not yet executed on PR # ( not run — needs /azp run)` and stop +this gate WITHOUT marking ready. Do NOT overclaim — a green *sibling* leg (a different category +on the same platform) is not the target's leg, and a green leg on one platform is not a pass on +the others. + +**T3 — Mark ready + report.** If EVERY target test is VALIDATED-GREEN. The 🎯 comment, +the mark-ready, and the `p/0` label are THREE SEPARATE safe-outputs — the comment +existing does NOT prove the mark-ready took effect, so they are tracked independently: + +- **Comment idempotency (dup-suppress only — never gates the mark-ready).** We only reach + T3 while `C.isDraft == true` (the precondition stops an already-ready PR before here). So + if a prior bot comment for THIS head that carries the leading `🎯` anchor AND the + contiguous phrase `validated green on ` already exists (BOTH T3 variants — + `🎯 Target test validated green on ` and the build-only `🎯 Build validated green on + ` — contain that exact contiguous phrase, so this recognizes both; keying on + `target test` alone would instead miss the build-only comment and re-post it every sweep. + **Require the `🎯` anchor** so the match can NEVER be loosened to a gapped + `validated…green…on` form that would false-positive on the Step 3 `✅ … validated — the + fix's CI is green on ` surface comment and wrongly suppress the audit 🎯), the comment + landed on an earlier sweep but the **mark-ready did NOT take + effect** (the PR is still a draft) — set `SUPPRESS_COMMENT = true` (do not re-post the + duplicate 🎯 comment) but STILL complete the mark-ready + label below. Never treat the + comment's existence as "already marked ready" while the PR is still a draft. Record this + case as `re-marking PR # (🎯 present; mark-ready did not take on a prior sweep)`. +- **Atomicity budget pre-check (Hard-Rule 7).** Determine the outputs this gate will emit: + `mark_pull_request_as_ready_for_review` + `add_labels` always, plus `add_comment` unless + `SUPPRESS_COMMENT`. Verify a free per-run slot remains in EACH bucket you are about to + use. If ANY required bucket is exhausted, emit NONE of them — record `skipped: per-run + cap reached; deferring mark-ready PR # to next cycle` and stop this gate. Never mark a + PR ready (or label it) unless its 🎯 audit comment is GUARANTEED to exist for this exact + head SHA — either posted in this same sweep, or (in the `SUPPRESS_COMMENT` re-marking case) + already present from a prior sweep. The outputs you DO emit either all land or all defer + together; the audit trail must never be absent, but it is NOT re-posted when it already exists. +- **Dry-run gate (Step 0):** if `dry_run == "true"`, emit NOTHING — print the intended + readiness comment and "would mark ready + add p/0" to the run log, tally `dry-run: + would-mark-ready PR #`, and stop. +- Otherwise emit for THIS PR number ``: + 1. `add_comment` (ONLY if not `SUPPRESS_COMMENT`): `🎯 Target test validated green on + — passed on ALL platforms it runs on (, + buildId ). — not + caused by this fix."> Transitioning this PR from draft to ready for review and adding + `p/0`; a maintainer still reviews and merges.` (For a **build-only fix** — the T1 + whole-build fallback, no single target test — phrase the first clause as `🎯 Build + validated green on — the maui-pr build passed on ALL platforms (buildId + ).` instead of naming a test.) + 2. `mark_pull_request_as_ready_for_review` with `reason:` a one-line justification + naming the validated test(s) and ``. + 3. `add_labels` with `labels: ["p/0"]` for PR # — put the now-review-ready fix into + the team's p/0 priority queue so it is triaged, not lost in the draft backlog. (If + the PR somehow already carries `p/0`, this is a harmless no-op.) +- Record `marked-ready PR # (target green on ALL platforms on , + labeled p/0)` and stop. + #### Step 3.5.R — Maintainer change-request response (Track C) Reached from Step 3.5 gate 0. Goal: when an **eligible human reviewer** (Hard-Rule @@ -1383,7 +1669,19 @@ Filed by [`ci-status-fix`](https://github.com/dotnet/maui/blob/main/.github/work ### Step 8 — Per-issue tally + end-of-run summary -Per issue, append one outcome line to `/tmp/gh-aw/agent/coverage.txt`: +Per issue, append **one** outcome line to `/tmp/gh-aw/agent/coverage.txt` — the +terminal outcome for the cycle. When one cycle produces a chained pair — a PR gets a +same-cycle precursor line and **then** a Step 3.6 readiness line (`marked-ready` on the +draft→ready flip, or `already-ready` on a later steady-state sweep of an already-ready +PR) — record ONLY the terminal Step 3.6 readiness line: it supersedes ANY same-cycle +non-readiness precursor for that PR, so an aggregator keying on one-line-per-issue never +double-counts. The precursor may be a Step 3 green line (`surfaced-green` on the first ✅, +or `already-surfaced` when it already exists) OR — when an unrelated-flake red and a +target-green coincide — a Step 4 `annotated-flake` line; either way the readiness line is +terminal, and the superseded precursor's signal survives in the PR's 🎯/♻️ comment so no +information is lost. This covers the flip cycle (`surfaced-green` → `marked-ready`), the +post-flip steady state (`already-surfaced` → `already-ready`), and the flake-coincident +flip (`annotated-flake` → `marked-ready`): ``` # main attempt- @@ -1394,8 +1692,14 @@ Per issue, append one outcome line to `/tmp/gh-aw/agent/coverage.txt`: `advance-PR # attempt /10` (ADVANCE mode — new commit pushed to the existing PR), `surfaced-green PR #` (fix's own CI went green; commented for review, did not advance), `annotated-flake PR #` (red was unrelated flake; -commented, attempt NOT burned), `waiting PR #` (CI not settled yet), -`dry-run: would-`, `skipped: `. (The +commented, attempt NOT burned), `marked-ready PR #` (the specific fixed test +was confirmed green in the PR's own CI; draft flipped to ready for review), +`already-surfaced PR #` (the fix's ✅ green comment already existed this sweep; +re-post suppressed — superseded by the Step 3.6 readiness line when the PR also flips +ready that cycle), `already-ready PR #` (terminal steady state — the PR was flipped +ready on a prior cycle and remains green on an unchanged head; no action taken, +supersedes `already-surfaced`), `waiting PR #` (CI not settled yet), +`dry-run: would-`, `skipped: `. (The `needs-human-PR` outcome is reserved for the deferred hand-off — Step 6 currently records a skip instead, so it is not emitted.)
(🎯 present; mark-ready did not take on a prior sweep)`. +- **Atomicity budget pre-check (Hard-Rule 7).** Determine the outputs this gate will emit: + `mark_pull_request_as_ready_for_review` + `add_labels` always, plus `add_comment` unless + `SUPPRESS_COMMENT`. Verify a free per-run slot remains in EACH bucket you are about to + use. If ANY required bucket is exhausted, emit NONE of them — record `skipped: per-run + cap reached; deferring mark-ready PR #
to next cycle` and stop this gate. Never mark a + PR ready (or label it) unless its 🎯 audit comment is GUARANTEED to exist for this exact + head SHA — either posted in this same sweep, or (in the `SUPPRESS_COMMENT` re-marking case) + already present from a prior sweep. The outputs you DO emit either all land or all defer + together; the audit trail must never be absent, but it is NOT re-posted when it already exists. +- **Dry-run gate (Step 0):** if `dry_run == "true"`, emit NOTHING — print the intended + readiness comment and "would mark ready + add p/0" to the run log, tally `dry-run: + would-mark-ready PR #
`, and stop. +- Otherwise emit for THIS PR number `
`: + 1. `add_comment` (ONLY if not `SUPPRESS_COMMENT`): `🎯 Target test validated green on + — passed on ALL platforms it runs on (, + buildId ). — not + caused by this fix."> Transitioning this PR from draft to ready for review and adding + `p/0`; a maintainer still reviews and merges.` (For a **build-only fix** — the T1 + whole-build fallback, no single target test — phrase the first clause as `🎯 Build + validated green on — the maui-pr build passed on ALL platforms (buildId + ).` instead of naming a test.) + 2. `mark_pull_request_as_ready_for_review` with `reason:` a one-line justification + naming the validated test(s) and ``. + 3. `add_labels` with `labels: ["p/0"]` for PR # — put the now-review-ready fix into + the team's p/0 priority queue so it is triaged, not lost in the draft backlog. (If + the PR somehow already carries `p/0`, this is a harmless no-op.) +- Record `marked-ready PR # (target green on ALL platforms on , + labeled p/0)` and stop. + #### Step 3.5.R — Maintainer change-request response (Track C) Reached from Step 3.5 gate 0. Goal: when an **eligible human reviewer** (Hard-Rule @@ -1395,7 +1681,19 @@ Filed by [`ci-status-fix-net11`](https://github.com/dotnet/maui/blob/main/.githu ### Step 8 — Per-issue tally + end-of-run summary -Per issue, append one outcome line to `/tmp/gh-aw/agent/coverage.txt`: +Per issue, append **one** outcome line to `/tmp/gh-aw/agent/coverage.txt` — the +terminal outcome for the cycle. When one cycle produces a chained pair — a PR gets a +same-cycle precursor line and **then** a Step 3.6 readiness line (`marked-ready` on the +draft→ready flip, or `already-ready` on a later steady-state sweep of an already-ready +PR) — record ONLY the terminal Step 3.6 readiness line: it supersedes ANY same-cycle +non-readiness precursor for that PR, so an aggregator keying on one-line-per-issue never +double-counts. The precursor may be a Step 3 green line (`surfaced-green` on the first ✅, +or `already-surfaced` when it already exists) OR — when an unrelated-flake red and a +target-green coincide — a Step 4 `annotated-flake` line; either way the readiness line is +terminal, and the superseded precursor's signal survives in the PR's 🎯/♻️ comment so no +information is lost. This covers the flip cycle (`surfaced-green` → `marked-ready`), the +post-flip steady state (`already-surfaced` → `already-ready`), and the flake-coincident +flip (`annotated-flake` → `marked-ready`): ``` # net11.0 attempt- @@ -1406,8 +1704,14 @@ Per issue, append one outcome line to `/tmp/gh-aw/agent/coverage.txt`: `advance-PR # attempt /10` (ADVANCE mode — new commit pushed to the existing PR), `surfaced-green PR #` (fix's own CI went green; commented for review, did not advance), `annotated-flake PR #` (red was unrelated flake; -commented, attempt NOT burned), `waiting PR #` (CI not settled yet), -`dry-run: would-`, `skipped: `. (The +commented, attempt NOT burned), `marked-ready PR #` (the specific fixed test +was confirmed green in the PR's own CI; draft flipped to ready for review), +`already-surfaced PR #` (the fix's ✅ green comment already existed this sweep; +re-post suppressed — superseded by the Step 3.6 readiness line when the PR also flips +ready that cycle), `already-ready PR #` (terminal steady state — the PR was flipped +ready on a prior cycle and remains green on an unchanged head; no action taken, +supersedes `already-surfaced`), `waiting PR #` (CI not settled yet), +`dry-run: would-`, `skipped: `. (The `needs-human-PR` outcome is reserved for the deferred hand-off — Step 6 currently records a skip instead, so it is not emitted.) diff --git a/.github/workflows/ci-status-fix.lock.yml b/.github/workflows/ci-status-fix.lock.yml index 93dc5969cb0d..65511f2b3259 100644 --- a/.github/workflows/ci-status-fix.lock.yml +++ b/.github/workflows/ci-status-fix.lock.yml @@ -1,4 +1,4 @@ -# gh-aw-metadata: {"schema_version":"v4","frontmatter_hash":"f014161967589ebcaf1ac5ec6f3c88ee5a4388d28b55a07dc36c45b7b2aebab6","body_hash":"86c6b2d4c3df13da14c6a3c8b211381fcc9f97bb1951ff7b80588a2c5338e00c","compiler_version":"v0.80.9","strict":true,"agent_id":"copilot","agent_model":"claude-opus-4.8","engine_versions":{"copilot":"1.0.63"}} +# gh-aw-metadata: {"schema_version":"v4","frontmatter_hash":"b0ebf8a2f179900cfe3638e994b27a43cec52bdbdd2331dc1bd4d65762fa2d6b","body_hash":"1825e2b1f7c7bd88241bf37580655489360f9bebc806edd00837a52ce4ad83a0","compiler_version":"v0.80.9","strict":true,"agent_id":"copilot","agent_model":"claude-opus-4.8","engine_versions":{"copilot":"1.0.63"}} # gh-aw-manifest: {"version":1,"secrets":["COPILOT_PAT_0","COPILOT_PAT_1","COPILOT_PAT_2","COPILOT_PAT_3","COPILOT_PAT_4","COPILOT_PAT_5","COPILOT_PAT_6","COPILOT_PAT_7","COPILOT_PAT_8","COPILOT_PAT_9","GH_AW_CI_TRIGGER_TOKEN","GH_AW_GITHUB_MCP_SERVER_TOKEN","GH_AW_GITHUB_TOKEN","GITHUB_TOKEN"],"actions":[{"repo":"actions/checkout","sha":"34e114876b0b11c390a56381ad16ebd13914f8d5","version":"v4"},{"repo":"actions/checkout","sha":"9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0","version":"v7.0.0"},{"repo":"actions/checkout","sha":"de0fac2e4500dabe0009e67214ff5f5447ce83dd","version":"v6.0.2"},{"repo":"actions/download-artifact","sha":"3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c","version":"v8.0.1"},{"repo":"actions/github-script","sha":"3a2844b7e9c422d3c10d287c895573f7108da1b3","version":"v9.0.0"},{"repo":"actions/setup-node","sha":"48b55a011bda9f5d6aeb4c2d9c7362e8dae4041e","version":"v6.4.0"},{"repo":"actions/upload-artifact","sha":"043fb46d1a93c77aae656e7c1c64a875d1fc6a0a","version":"v7.0.1"},{"repo":"github/gh-aw-actions/setup","sha":"8c7d04ebf1ece56cd381446125da3e0f6896294a","version":"v0.80.9"}],"containers":[{"image":"ghcr.io/github/gh-aw-firewall/agent:0.27.7","digest":"sha256:aae231e4635c8999d039c132f1602d3df850fe9b84a00aa2b5ac981179b5661c","pinned_image":"ghcr.io/github/gh-aw-firewall/agent:0.27.7@sha256:aae231e4635c8999d039c132f1602d3df850fe9b84a00aa2b5ac981179b5661c"},{"image":"ghcr.io/github/gh-aw-firewall/api-proxy:0.27.7","digest":"sha256:009caf2e3d88fa77b64e9a03a95a228fc58db0f1701c6d324b29ba5a3c7c79b6","pinned_image":"ghcr.io/github/gh-aw-firewall/api-proxy:0.27.7@sha256:009caf2e3d88fa77b64e9a03a95a228fc58db0f1701c6d324b29ba5a3c7c79b6"},{"image":"ghcr.io/github/gh-aw-firewall/squid:0.27.7","digest":"sha256:deb1d4e19de62d51cee0508057a596a19315c3423ada4d675cad136dc8037c96","pinned_image":"ghcr.io/github/gh-aw-firewall/squid:0.27.7@sha256:deb1d4e19de62d51cee0508057a596a19315c3423ada4d675cad136dc8037c96"},{"image":"ghcr.io/github/gh-aw-mcpg:v0.3.27","digest":"sha256:fe984bddde4ec05d756d9043edb0a32912e6b7b72f6a121b1082f29221421cc7","pinned_image":"ghcr.io/github/gh-aw-mcpg:v0.3.27@sha256:fe984bddde4ec05d756d9043edb0a32912e6b7b72f6a121b1082f29221421cc7"},{"image":"ghcr.io/github/gh-aw-node","digest":"sha256:529d02eb970b1161aa25c593a9c3df57fdfad5a8add328cb3b6eccef66f3183b","pinned_image":"ghcr.io/github/gh-aw-node@sha256:529d02eb970b1161aa25c593a9c3df57fdfad5a8add328cb3b6eccef66f3183b"},{"image":"ghcr.io/github/github-mcp-server:v1.4.0","digest":"sha256:2afb26356481d1a350e14544a6e160f7f7ec1561a1ea309b823665abf0309036","pinned_image":"ghcr.io/github/github-mcp-server:v1.4.0@sha256:2afb26356481d1a350e14544a6e160f7f7ec1561a1ea309b823665abf0309036"}]} # This file was automatically generated by gh-aw (v0.80.9). DO NOT EDIT. To debug this workflow, load the skill at https://github.com/github/gh-aw/blob/main/debug.md # @@ -37,7 +37,9 @@ # humans (the open tracking issue is the hand-off surface; a dedicated # [ci-fix][needs-human] PR is planned but currently deferred — see Step 6). CI on # the PR is kicked by a human `/azp run` for now; the loop watches, classifies, -# and re-fixes autonomously between kicks. +# and re-fixes autonomously between kicks. When the SPECIFIC test a PR fixed is +# confirmed green in that PR's own CI, the loop marks the draft PR ready for review +# (a state transition only — it never approves or merges). # Never mutes tests, but # de-flakes genuinely flaky ones (deterministic synchronization, no retries / # timeout bumps). Always skips visual-regression / screenshot issues. @@ -273,24 +275,24 @@ jobs: run: | bash "${RUNNER_TEMP}/gh-aw/actions/create_prompt_first.sh" { - cat << 'GH_AW_PROMPT_25cf75111b670167_EOF' + cat << 'GH_AW_PROMPT_2ac1d0b4083428f8_EOF' - GH_AW_PROMPT_25cf75111b670167_EOF + GH_AW_PROMPT_2ac1d0b4083428f8_EOF cat "${RUNNER_TEMP}/gh-aw/prompts/xpia.md" cat "${RUNNER_TEMP}/gh-aw/prompts/temp_folder_prompt.md" cat "${RUNNER_TEMP}/gh-aw/prompts/markdown.md" cat "${RUNNER_TEMP}/gh-aw/prompts/safe_outputs_prompt.md" - cat << 'GH_AW_PROMPT_25cf75111b670167_EOF' + cat << 'GH_AW_PROMPT_2ac1d0b4083428f8_EOF' - Tools: add_comment(max:3), create_pull_request(max:3), update_pull_request(max:3), push_to_pull_request_branch(max:3), missing_tool, missing_data, noop - GH_AW_PROMPT_25cf75111b670167_EOF + Tools: add_comment(max:6), create_pull_request(max:3), update_pull_request(max:3), mark_pull_request_as_ready_for_review(max:3), add_labels(max:3), push_to_pull_request_branch(max:3), missing_tool, missing_data, noop + GH_AW_PROMPT_2ac1d0b4083428f8_EOF cat "${RUNNER_TEMP}/gh-aw/prompts/safe_outputs_create_pull_request.md" cat "${RUNNER_TEMP}/gh-aw/prompts/safe_outputs_push_to_pr_branch.md" - cat << 'GH_AW_PROMPT_25cf75111b670167_EOF' + cat << 'GH_AW_PROMPT_2ac1d0b4083428f8_EOF' - GH_AW_PROMPT_25cf75111b670167_EOF + GH_AW_PROMPT_2ac1d0b4083428f8_EOF cat "${RUNNER_TEMP}/gh-aw/prompts/mcp_cli_tools_prompt.md" - cat << 'GH_AW_PROMPT_25cf75111b670167_EOF' + cat << 'GH_AW_PROMPT_2ac1d0b4083428f8_EOF' The following GitHub context information is available for this workflow: {{#if github.actor}} @@ -332,12 +334,12 @@ jobs: stop immediately and report the limitation rather than spending turns trying to work around it. - GH_AW_PROMPT_25cf75111b670167_EOF + GH_AW_PROMPT_2ac1d0b4083428f8_EOF cat "${RUNNER_TEMP}/gh-aw/prompts/github_mcp_tools_with_safeoutputs_prompt.md" - cat << 'GH_AW_PROMPT_25cf75111b670167_EOF' + cat << 'GH_AW_PROMPT_2ac1d0b4083428f8_EOF' {{#runtime-import .github/workflows/ci-status-fix.md}} - GH_AW_PROMPT_25cf75111b670167_EOF + GH_AW_PROMPT_2ac1d0b4083428f8_EOF } > "$GH_AW_PROMPT" - name: Interpolate variables and render templates uses: actions/github-script@3a2844b7e9c422d3c10d287c895573f7108da1b3 # v9.0.0 @@ -554,16 +556,18 @@ jobs: mkdir -p "${RUNNER_TEMP}/gh-aw/safeoutputs" mkdir -p /tmp/gh-aw/safeoutputs mkdir -p /tmp/gh-aw/mcp-logs/safeoutputs - cat > "${RUNNER_TEMP}/gh-aw/safeoutputs/config.json" << 'GH_AW_SAFE_OUTPUTS_CONFIG_fa2a69fad5fd0485_EOF' - {"add_comment":{"discussions":false,"max":3,"required_labels":["agentic-workflows"],"required_title_prefix":"[ci-fix] ","target":"*"},"create_pull_request":{"allowed_base_branches":["main"],"allowed_branches":["ci-fix/**"],"allowed_files":["src/Core/**","src/Controls/**","src/Essentials/**","src/BlazorWebView/**","src/TestUtils/**","src/Templates/**","**/PublicAPI.Unshipped.txt"],"base_branch":"main","draft":true,"labels":["agentic-workflows"],"max":3,"max_patch_files":100,"max_patch_size":4096,"protect_top_level_dot_folders":true,"protected_files":["package.json","bun.lockb","bunfig.toml","deno.json","deno.jsonc","deno.lock","global.json","NuGet.Config","Directory.Packages.props","mix.exs","mix.lock","go.mod","go.sum","stack.yaml","stack.yaml.lock","pom.xml","build.gradle","build.gradle.kts","settings.gradle","settings.gradle.kts","gradle.properties","package-lock.json","yarn.lock","pnpm-lock.yaml","npm-shrinkwrap.json","requirements.txt","Pipfile","Pipfile.lock","pyproject.toml","setup.py","setup.cfg","Gemfile","Gemfile.lock","uv.lock","CODEOWNERS","DESIGN.md","README.md","CONTRIBUTING.md","CHANGELOG.md","SECURITY.md","CODE_OF_CONDUCT.md","AGENTS.md","CLAUDE.md","GEMINI.md"],"protected_files_policy":"request_review","title_prefix":"[ci-fix] "},"create_report_incomplete_issue":{},"missing_data":{},"missing_tool":{},"noop":{"max":1,"report-as-issue":"true"},"push_to_pull_request_branch":{"allowed_files":["src/Core/**","src/Controls/**","src/Essentials/**","src/BlazorWebView/**","src/TestUtils/**","src/Templates/**","**/PublicAPI.Unshipped.txt"],"if_no_changes":"warn","max":3,"max_patch_size":4096,"protect_top_level_dot_folders":true,"protected_files":["package.json","bun.lockb","bunfig.toml","deno.json","deno.jsonc","deno.lock","global.json","NuGet.Config","Directory.Packages.props","mix.exs","mix.lock","go.mod","go.sum","stack.yaml","stack.yaml.lock","pom.xml","build.gradle","build.gradle.kts","settings.gradle","settings.gradle.kts","gradle.properties","package-lock.json","yarn.lock","pnpm-lock.yaml","npm-shrinkwrap.json","requirements.txt","Pipfile","Pipfile.lock","pyproject.toml","setup.py","setup.cfg","Gemfile","Gemfile.lock","uv.lock","CODEOWNERS","DESIGN.md","README.md","CONTRIBUTING.md","CHANGELOG.md","SECURITY.md","CODE_OF_CONDUCT.md","AGENTS.md","CLAUDE.md","GEMINI.md"],"required_labels":["agentic-workflows"],"target":"*","title_prefix":"[ci-fix] "},"report_incomplete":{},"update_pull_request":{"allow_body":true,"allow_title":false,"max":3,"target":"*","update_branch":false}} - GH_AW_SAFE_OUTPUTS_CONFIG_fa2a69fad5fd0485_EOF + cat > "${RUNNER_TEMP}/gh-aw/safeoutputs/config.json" << 'GH_AW_SAFE_OUTPUTS_CONFIG_0644bf1434ca0071_EOF' + {"add_comment":{"discussions":false,"max":6,"required_labels":["agentic-workflows"],"required_title_prefix":"[ci-fix] ","target":"*"},"add_labels":{"allowed":["p/0"],"max":3,"required_labels":["agentic-workflows"],"required_title_prefix":"[ci-fix] ","target":"*"},"create_pull_request":{"allowed_base_branches":["main"],"allowed_branches":["ci-fix/**"],"allowed_files":["src/Core/**","src/Controls/**","src/Essentials/**","src/BlazorWebView/**","src/TestUtils/**","src/Templates/**","**/PublicAPI.Unshipped.txt"],"base_branch":"main","draft":true,"labels":["agentic-workflows"],"max":3,"max_patch_files":100,"max_patch_size":4096,"protect_top_level_dot_folders":true,"protected_files":["package.json","bun.lockb","bunfig.toml","deno.json","deno.jsonc","deno.lock","global.json","NuGet.Config","Directory.Packages.props","mix.exs","mix.lock","go.mod","go.sum","stack.yaml","stack.yaml.lock","pom.xml","build.gradle","build.gradle.kts","settings.gradle","settings.gradle.kts","gradle.properties","package-lock.json","yarn.lock","pnpm-lock.yaml","npm-shrinkwrap.json","requirements.txt","Pipfile","Pipfile.lock","pyproject.toml","setup.py","setup.cfg","Gemfile","Gemfile.lock","uv.lock","CODEOWNERS","DESIGN.md","README.md","CONTRIBUTING.md","CHANGELOG.md","SECURITY.md","CODE_OF_CONDUCT.md","AGENTS.md","CLAUDE.md","GEMINI.md"],"protected_files_policy":"request_review","title_prefix":"[ci-fix] "},"create_report_incomplete_issue":{},"mark_pull_request_as_ready_for_review":{"max":3,"required_labels":["agentic-workflows"],"required_title_prefix":"[ci-fix] ","target":"*"},"missing_data":{},"missing_tool":{},"noop":{"max":1,"report-as-issue":"true"},"push_to_pull_request_branch":{"allowed_files":["src/Core/**","src/Controls/**","src/Essentials/**","src/BlazorWebView/**","src/TestUtils/**","src/Templates/**","**/PublicAPI.Unshipped.txt"],"if_no_changes":"warn","max":3,"max_patch_size":4096,"protect_top_level_dot_folders":true,"protected_files":["package.json","bun.lockb","bunfig.toml","deno.json","deno.jsonc","deno.lock","global.json","NuGet.Config","Directory.Packages.props","mix.exs","mix.lock","go.mod","go.sum","stack.yaml","stack.yaml.lock","pom.xml","build.gradle","build.gradle.kts","settings.gradle","settings.gradle.kts","gradle.properties","package-lock.json","yarn.lock","pnpm-lock.yaml","npm-shrinkwrap.json","requirements.txt","Pipfile","Pipfile.lock","pyproject.toml","setup.py","setup.cfg","Gemfile","Gemfile.lock","uv.lock","CODEOWNERS","DESIGN.md","README.md","CONTRIBUTING.md","CHANGELOG.md","SECURITY.md","CODE_OF_CONDUCT.md","AGENTS.md","CLAUDE.md","GEMINI.md"],"required_labels":["agentic-workflows"],"target":"*","title_prefix":"[ci-fix] "},"report_incomplete":{},"update_pull_request":{"allow_body":true,"allow_title":false,"max":3,"target":"*","update_branch":false}} + GH_AW_SAFE_OUTPUTS_CONFIG_0644bf1434ca0071_EOF - name: Generate Safe Outputs Tools env: GH_AW_TOOLS_META_JSON: | { "description_suffixes": { - "add_comment": " CONSTRAINTS: Maximum 3 comment(s) can be added. Target: *. Supports reply_to_id for discussion threading.", + "add_comment": " CONSTRAINTS: Maximum 6 comment(s) can be added. Target: *. Supports reply_to_id for discussion threading.", + "add_labels": " CONSTRAINTS: Maximum 3 label(s) can be added. Only these labels are allowed: [\"p/0\"]. Target: *.", "create_pull_request": " CONSTRAINTS: Maximum 3 pull request(s) can be created. Title will be prefixed with \"[ci-fix] \". Labels [\"agentic-workflows\"] will be automatically added. Only these labels are allowed: [\"agentic-workflows\"]. PRs will be created as drafts.", + "mark_pull_request_as_ready_for_review": " CONSTRAINTS: Maximum 3 pull request(s) can be marked as ready for review.", "push_to_pull_request_branch": " CONSTRAINTS: Maximum 3 push(es) can be made. The target pull request title must start with \"[ci-fix] \".", "update_pull_request": " CONSTRAINTS: Maximum 3 pull request(s) can be updated. Target: *." }, @@ -594,6 +598,25 @@ jobs: } } }, + "add_labels": { + "defaultMax": 5, + "fields": { + "item_number": { + "issueNumberOrTemporaryId": true + }, + "labels": { + "required": true, + "type": "array", + "itemType": "string", + "itemSanitize": true, + "itemMaxLength": 128 + }, + "repo": { + "type": "string", + "maxLength": 256 + } + } + }, "create_pull_request": { "defaultMax": 1, "fields": { @@ -635,6 +658,24 @@ jobs: } } }, + "mark_pull_request_as_ready_for_review": { + "defaultMax": 1, + "fields": { + "pull_request_number": { + "issueOrPRNumber": true + }, + "reason": { + "required": true, + "type": "string", + "sanitize": true, + "maxLength": 65000 + }, + "repo": { + "type": "string", + "maxLength": 256 + } + } + }, "missing_data": { "defaultMax": 20, "fields": { @@ -1494,7 +1535,7 @@ jobs: uses: actions/github-script@3a2844b7e9c422d3c10d287c895573f7108da1b3 # v9.0.0 env: WORKFLOW_NAME: "CI Failure Fixer (main)" - WORKFLOW_DESCRIPTION: "Periodic pass over open ci-scan tracking issues filed by the main-branch CI\nfailure scanner (.github/workflows/ci-status-main.md). This workflow targets\nthe `main` branch EXCLUSIVELY: it processes only issues labelled ci-scan and\nopens every PR against main. (The net11.0 branch is handled by the parallel\n.github/workflows/ci-status-fix-net11.md — the two are split because gh-aw can\nonly transport a fix relative to ONE static base branch per workflow, and the\nmain↔net11.0 divergence exceeds gh-aw's 10 MB transport-patch cap.) The fixer\nopens ONE draft [ci-fix] PR per actionable issue against main, then WATCHES\nthat PR's own CI on later runs: when the fix's CI comes back red and the red is\ncaused by the fix itself, it pushes a fresh follow-up fix onto the SAME PR\nbranch (never a second PR) — up to 10 attempts — then stops and defers to\nhumans (the open tracking issue is the hand-off surface; a dedicated\n[ci-fix][needs-human] PR is planned but currently deferred — see Step 6). CI on\nthe PR is kicked by a human `/azp run` for now; the loop watches, classifies,\nand re-fixes autonomously between kicks.\nNever mutes tests, but\nde-flakes genuinely flaky ones (deterministic synchronization, no retries /\ntimeout bumps). Always skips visual-regression / screenshot issues." + WORKFLOW_DESCRIPTION: "Periodic pass over open ci-scan tracking issues filed by the main-branch CI\nfailure scanner (.github/workflows/ci-status-main.md). This workflow targets\nthe `main` branch EXCLUSIVELY: it processes only issues labelled ci-scan and\nopens every PR against main. (The net11.0 branch is handled by the parallel\n.github/workflows/ci-status-fix-net11.md — the two are split because gh-aw can\nonly transport a fix relative to ONE static base branch per workflow, and the\nmain↔net11.0 divergence exceeds gh-aw's 10 MB transport-patch cap.) The fixer\nopens ONE draft [ci-fix] PR per actionable issue against main, then WATCHES\nthat PR's own CI on later runs: when the fix's CI comes back red and the red is\ncaused by the fix itself, it pushes a fresh follow-up fix onto the SAME PR\nbranch (never a second PR) — up to 10 attempts — then stops and defers to\nhumans (the open tracking issue is the hand-off surface; a dedicated\n[ci-fix][needs-human] PR is planned but currently deferred — see Step 6). CI on\nthe PR is kicked by a human `/azp run` for now; the loop watches, classifies,\nand re-fixes autonomously between kicks. When the SPECIFIC test a PR fixed is\nconfirmed green in that PR's own CI, the loop marks the draft PR ready for review\n(a state transition only — it never approves or merges).\nNever mutes tests, but\nde-flakes genuinely flaky ones (deterministic synchronization, no retries /\ntimeout bumps). Always skips visual-regression / screenshot issues." HAS_PATCH: ${{ needs.agent.outputs.has_patch }} with: script: | @@ -1910,7 +1951,7 @@ jobs: GH_AW_ALLOWED_DOMAINS: "*.blob.core.windows.net,*.githubusercontent.com,api.business.githubcopilot.com,api.enterprise.githubcopilot.com,api.github.com,api.githubcopilot.com,api.individual.githubcopilot.com,api.snapcraft.io,archive.ubuntu.com,azure.archive.ubuntu.com,codeload.github.com,crl.geotrust.com,crl.globalsign.com,crl.identrust.com,crl.sectigo.com,crl.thawte.com,crl.usertrust.com,crl.verisign.com,crl3.digicert.com,crl4.digicert.com,crls.ssl.com,dev.azure.com,docs.github.com,github-cloud.githubusercontent.com,github-cloud.s3.amazonaws.com,github.blog,github.com,github.githubassets.com,helix.dot.net,host.docker.internal,json-schema.org,json.schemastore.org,keyserver.ubuntu.com,lfs.github.com,objects.githubusercontent.com,ocsp.digicert.com,ocsp.geotrust.com,ocsp.globalsign.com,ocsp.identrust.com,ocsp.sectigo.com,ocsp.ssl.com,ocsp.thawte.com,ocsp.usertrust.com,ocsp.verisign.com,packagecloud.io,packages.cloud.google.com,packages.microsoft.com,patch-diff.githubusercontent.com,ppa.launchpad.net,raw.githubusercontent.com,registry.npmjs.org,s.symcb.com,s.symcd.com,security.ubuntu.com,telemetry.enterprise.githubcopilot.com,ts-crl.ws.symantec.com,ts-ocsp.ws.symantec.com,www.googleapis.com" GITHUB_SERVER_URL: ${{ github.server_url }} GITHUB_API_URL: ${{ github.api_url }} - GH_AW_SAFE_OUTPUTS_HANDLER_CONFIG: "{\"add_comment\":{\"discussions\":false,\"max\":3,\"required_labels\":[\"agentic-workflows\"],\"required_title_prefix\":\"[ci-fix] \",\"target\":\"*\"},\"create_pull_request\":{\"allowed_base_branches\":[\"main\"],\"allowed_branches\":[\"ci-fix/**\"],\"allowed_files\":[\"src/Core/**\",\"src/Controls/**\",\"src/Essentials/**\",\"src/BlazorWebView/**\",\"src/TestUtils/**\",\"src/Templates/**\",\"**/PublicAPI.Unshipped.txt\"],\"base_branch\":\"main\",\"draft\":true,\"labels\":[\"agentic-workflows\"],\"max\":3,\"max_patch_files\":100,\"max_patch_size\":4096,\"protect_top_level_dot_folders\":true,\"protected_files\":[\"package.json\",\"bun.lockb\",\"bunfig.toml\",\"deno.json\",\"deno.jsonc\",\"deno.lock\",\"global.json\",\"NuGet.Config\",\"Directory.Packages.props\",\"mix.exs\",\"mix.lock\",\"go.mod\",\"go.sum\",\"stack.yaml\",\"stack.yaml.lock\",\"pom.xml\",\"build.gradle\",\"build.gradle.kts\",\"settings.gradle\",\"settings.gradle.kts\",\"gradle.properties\",\"package-lock.json\",\"yarn.lock\",\"pnpm-lock.yaml\",\"npm-shrinkwrap.json\",\"requirements.txt\",\"Pipfile\",\"Pipfile.lock\",\"pyproject.toml\",\"setup.py\",\"setup.cfg\",\"Gemfile\",\"Gemfile.lock\",\"uv.lock\",\"CODEOWNERS\",\"DESIGN.md\",\"README.md\",\"CONTRIBUTING.md\",\"CHANGELOG.md\",\"SECURITY.md\",\"CODE_OF_CONDUCT.md\",\"AGENTS.md\",\"CLAUDE.md\",\"GEMINI.md\"],\"protected_files_policy\":\"request_review\",\"title_prefix\":\"[ci-fix] \"},\"create_report_incomplete_issue\":{},\"missing_data\":{},\"missing_tool\":{},\"noop\":{\"max\":1,\"report-as-issue\":\"true\"},\"push_to_pull_request_branch\":{\"allowed_files\":[\"src/Core/**\",\"src/Controls/**\",\"src/Essentials/**\",\"src/BlazorWebView/**\",\"src/TestUtils/**\",\"src/Templates/**\",\"**/PublicAPI.Unshipped.txt\"],\"if_no_changes\":\"warn\",\"max\":3,\"max_patch_size\":4096,\"protect_top_level_dot_folders\":true,\"protected_files\":[\"package.json\",\"bun.lockb\",\"bunfig.toml\",\"deno.json\",\"deno.jsonc\",\"deno.lock\",\"global.json\",\"NuGet.Config\",\"Directory.Packages.props\",\"mix.exs\",\"mix.lock\",\"go.mod\",\"go.sum\",\"stack.yaml\",\"stack.yaml.lock\",\"pom.xml\",\"build.gradle\",\"build.gradle.kts\",\"settings.gradle\",\"settings.gradle.kts\",\"gradle.properties\",\"package-lock.json\",\"yarn.lock\",\"pnpm-lock.yaml\",\"npm-shrinkwrap.json\",\"requirements.txt\",\"Pipfile\",\"Pipfile.lock\",\"pyproject.toml\",\"setup.py\",\"setup.cfg\",\"Gemfile\",\"Gemfile.lock\",\"uv.lock\",\"CODEOWNERS\",\"DESIGN.md\",\"README.md\",\"CONTRIBUTING.md\",\"CHANGELOG.md\",\"SECURITY.md\",\"CODE_OF_CONDUCT.md\",\"AGENTS.md\",\"CLAUDE.md\",\"GEMINI.md\"],\"required_labels\":[\"agentic-workflows\"],\"target\":\"*\",\"title_prefix\":\"[ci-fix] \"},\"report_incomplete\":{},\"update_pull_request\":{\"allow_body\":true,\"allow_title\":false,\"max\":3,\"target\":\"*\",\"update_branch\":false}}" + GH_AW_SAFE_OUTPUTS_HANDLER_CONFIG: "{\"add_comment\":{\"discussions\":false,\"max\":6,\"required_labels\":[\"agentic-workflows\"],\"required_title_prefix\":\"[ci-fix] \",\"target\":\"*\"},\"add_labels\":{\"allowed\":[\"p/0\"],\"max\":3,\"required_labels\":[\"agentic-workflows\"],\"required_title_prefix\":\"[ci-fix] \",\"target\":\"*\"},\"create_pull_request\":{\"allowed_base_branches\":[\"main\"],\"allowed_branches\":[\"ci-fix/**\"],\"allowed_files\":[\"src/Core/**\",\"src/Controls/**\",\"src/Essentials/**\",\"src/BlazorWebView/**\",\"src/TestUtils/**\",\"src/Templates/**\",\"**/PublicAPI.Unshipped.txt\"],\"base_branch\":\"main\",\"draft\":true,\"labels\":[\"agentic-workflows\"],\"max\":3,\"max_patch_files\":100,\"max_patch_size\":4096,\"protect_top_level_dot_folders\":true,\"protected_files\":[\"package.json\",\"bun.lockb\",\"bunfig.toml\",\"deno.json\",\"deno.jsonc\",\"deno.lock\",\"global.json\",\"NuGet.Config\",\"Directory.Packages.props\",\"mix.exs\",\"mix.lock\",\"go.mod\",\"go.sum\",\"stack.yaml\",\"stack.yaml.lock\",\"pom.xml\",\"build.gradle\",\"build.gradle.kts\",\"settings.gradle\",\"settings.gradle.kts\",\"gradle.properties\",\"package-lock.json\",\"yarn.lock\",\"pnpm-lock.yaml\",\"npm-shrinkwrap.json\",\"requirements.txt\",\"Pipfile\",\"Pipfile.lock\",\"pyproject.toml\",\"setup.py\",\"setup.cfg\",\"Gemfile\",\"Gemfile.lock\",\"uv.lock\",\"CODEOWNERS\",\"DESIGN.md\",\"README.md\",\"CONTRIBUTING.md\",\"CHANGELOG.md\",\"SECURITY.md\",\"CODE_OF_CONDUCT.md\",\"AGENTS.md\",\"CLAUDE.md\",\"GEMINI.md\"],\"protected_files_policy\":\"request_review\",\"title_prefix\":\"[ci-fix] \"},\"create_report_incomplete_issue\":{},\"mark_pull_request_as_ready_for_review\":{\"max\":3,\"required_labels\":[\"agentic-workflows\"],\"required_title_prefix\":\"[ci-fix] \",\"target\":\"*\"},\"missing_data\":{},\"missing_tool\":{},\"noop\":{\"max\":1,\"report-as-issue\":\"true\"},\"push_to_pull_request_branch\":{\"allowed_files\":[\"src/Core/**\",\"src/Controls/**\",\"src/Essentials/**\",\"src/BlazorWebView/**\",\"src/TestUtils/**\",\"src/Templates/**\",\"**/PublicAPI.Unshipped.txt\"],\"if_no_changes\":\"warn\",\"max\":3,\"max_patch_size\":4096,\"protect_top_level_dot_folders\":true,\"protected_files\":[\"package.json\",\"bun.lockb\",\"bunfig.toml\",\"deno.json\",\"deno.jsonc\",\"deno.lock\",\"global.json\",\"NuGet.Config\",\"Directory.Packages.props\",\"mix.exs\",\"mix.lock\",\"go.mod\",\"go.sum\",\"stack.yaml\",\"stack.yaml.lock\",\"pom.xml\",\"build.gradle\",\"build.gradle.kts\",\"settings.gradle\",\"settings.gradle.kts\",\"gradle.properties\",\"package-lock.json\",\"yarn.lock\",\"pnpm-lock.yaml\",\"npm-shrinkwrap.json\",\"requirements.txt\",\"Pipfile\",\"Pipfile.lock\",\"pyproject.toml\",\"setup.py\",\"setup.cfg\",\"Gemfile\",\"Gemfile.lock\",\"uv.lock\",\"CODEOWNERS\",\"DESIGN.md\",\"README.md\",\"CONTRIBUTING.md\",\"CHANGELOG.md\",\"SECURITY.md\",\"CODE_OF_CONDUCT.md\",\"AGENTS.md\",\"CLAUDE.md\",\"GEMINI.md\"],\"required_labels\":[\"agentic-workflows\"],\"target\":\"*\",\"title_prefix\":\"[ci-fix] \"},\"report_incomplete\":{},\"update_pull_request\":{\"allow_body\":true,\"allow_title\":false,\"max\":3,\"target\":\"*\",\"update_branch\":false}}" GH_AW_CI_TRIGGER_TOKEN: ${{ secrets.GH_AW_CI_TRIGGER_TOKEN }} with: github-token: ${{ secrets.GH_AW_GITHUB_TOKEN || secrets.GITHUB_TOKEN }} diff --git a/.github/workflows/ci-status-fix.md b/.github/workflows/ci-status-fix.md index d3227e25170c..9c2f46054029 100644 --- a/.github/workflows/ci-status-fix.md +++ b/.github/workflows/ci-status-fix.md @@ -15,7 +15,9 @@ description: | humans (the open tracking issue is the hand-off surface; a dedicated [ci-fix][needs-human] PR is planned but currently deferred — see Step 6). CI on the PR is kicked by a human `/azp run` for now; the loop watches, classifies, - and re-fixes autonomously between kicks. + and re-fixes autonomously between kicks. When the SPECIFIC test a PR fixed is + confirmed green in that PR's own CI, the loop marks the draft PR ready for review + (a state transition only — it never approves or merges). Never mutes tests, but de-flakes genuinely flaky ones (deterministic synchronization, no retries / timeout bumps). Always skips visual-regression / screenshot issues. @@ -239,8 +241,15 @@ safe-outputs: # PER-RUN total (not per-PR): a sweep may surface several green PRs and/or # annotate several flaky reds in one run. At max:1 all but the first comment # were silently dropped, starving the workflow's #1 value (surfacing green PRs - # for review). Raised to 3 to match the per-run throughput of the other outputs. - max: 3 + # for review). Sized to 6 (not 3) because a single green DRAFT PR that gets + # flipped ready in one sweep spends TWO comment slots — the Step 3 ✅ surface + # comment (which still names the /azp-gated legs a human must kick, so it is NOT + # redundant with 🎯) AND the Step 3.6 T3 🎯 readiness comment. At max:3 the shared + # bucket drained after ~1 draft flip and T3's atomicity pre-check then deferred + # every further mark-ready even while the mark-ready/add_labels buckets (also 3) + # sat idle; 6 lets ~3 draft flips (2 comments each) land per sweep, so the + # mark-ready:3 / add_labels:3 caps are actually reachable. + max: 6 target: "*" # Hard constraint (defense-in-depth): only comment on THIS workflow's own # ci-fix PRs — [ci-fix] title prefix AND agentic-workflows label. Mirrors the @@ -261,13 +270,51 @@ safe-outputs: # never the title, so disable title rewrites — this compiles to allow_title:false # and removes any ability to retitle an arbitrary PR. title: false - # NOTE: gh-aw v0.79.8 does NOT emit required-title-prefix/required-labels into + # NOTE: gh-aw v0.80.9 (the pinned compiler) does NOT emit required-title-prefix/required-labels into # the compiled config for update-pull-request (verified against the lock — it # silently drops them, unlike add-comment / push-to-pull-request-branch which # honor them). So which-PR scoping here relies on prompt Hard-Rule 6 + # min-integrity:approved; the residual is body-marker edits only (no code, no # merge, no close), capped at max:3. Revisit required-* if a future gh-aw # compiler emits them for this output. + mark-pull-request-as-ready-for-review: + # Flip a validated draft [ci-fix] PR to ready-for-review once the SPECIFIC test + # this PR was opened to fix is confirmed green in the PR's OWN CI (Step 3.6) — + # even when unrelated legs are red. The workflow is schedule/dispatch-triggered + # (no triggering-PR context), so target "*" lets the agent name the PR number it + # validated. This is a state transition ONLY: it never approves and never merges — + # a human still reviews and merges. + # PER-RUN total (not per-PR): a sweep may validate several PRs' target tests green. + max: 3 + target: "*" + # Hard constraint (defense-in-depth): only ever un-draft THIS workflow's own + # ci-fix PRs — [ci-fix] title prefix AND agentic-workflows label — so a confused + # or prompt-injected agent cannot mark an arbitrary PR ready. (If the v0.80.9 + # compiler silently drops these — as documented for update-pull-request above — + # the Step 3.6 preconditions + min-integrity:approved are the compensating scope + # controls; verify against the lock after compiling.) + required-title-prefix: "[ci-fix] " + required-labels: [agentic-workflows] + add-labels: + # Apply the p/0 priority label to a [ci-fix] PR at the exact moment the loop flips it + # from draft to ready-for-review (Step 3.6 T3) — so a validated, review-ready fix lands + # in the team's p/0 triage queue instead of sitting unseen in the draft backlog. Paired + # 1:1 with mark-pull-request-as-ready-for-review; target "*" lets the agent name the PR + # it just validated. + # allowed = HARD allowlist: the agent may ONLY ever add p/0, nothing else. This caps the + # blast radius of a confused/prompt-injected agent to exactly one benign priority label — + # it can never apply a downstream-triggering or destructive label. + allowed: [p/0] + # PER-RUN total (not per-PR): a sweep may mark several PRs ready in one run; each adds + # one label, so this matches the mark-ready per-run cap (3). + max: 3 + target: "*" + # Hard constraint (defense-in-depth): only ever label THIS workflow's own ci-fix PRs — + # [ci-fix] title prefix AND agentic-workflows label. (If the v0.80.9 compiler drops these + # — as documented for update-pull-request above — the Step 3.6 preconditions + + # min-integrity:approved + the allowed:[p/0] allowlist are the compensating controls.) + required-title-prefix: "[ci-fix] " + required-labels: [agentic-workflows] timeout-minutes: 90 @@ -339,33 +386,48 @@ through `safe-outputs`. 5. **One issue = one outcome per run.** Exactly one of: respond to a maintainer's change-request on the open PR (Track C, Step 3.5.R); open the first fix/help/ de-flake PR; advance an existing PR by one attempt (push a follow-up fix); - surface a validated-green PR for review; annotate an unrelated-flake red; wait + surface a validated-green PR for review; mark a target-validated draft PR + ready-for-review (Step 3.6 T3 — the terminal outcome that supersedes the same + run's surface/annotate precursor line), or record its `already-ready` + steady-state no-op; annotate an unrelated-flake red; wait (CI not yet settled); or a recorded skip (the dedicated needs-human PR is deferred — the attempt cap records a skip instead; see Step 6). Always prefer advancing or opening a PR over a skip when a non-mute diff is producible. 6. **All writes via `safe-outputs`.** Allowed outputs: `create_pull_request` (first attempt only), `push_to_pull_request_branch` (advance an existing PR by one attempt), `update_pull_request` (bump the attempt marker / refresh the - prior-attempts table), and `add_comment` (progress notes on the `[ci-fix]` PR - ONLY). NEVER comment on the tracking issue (issues are locked by + prior-attempts table), `add_comment` (progress notes on the `[ci-fix]` PR + ONLY), `mark_pull_request_as_ready_for_review` (flip a target-validated draft + `[ci-fix]` PR from draft to ready — Step 3.6 T3 only), and `add_labels` (add + ONLY the `p/0` label, ONLY on that same draft→ready transition — Step 3.6 T3). + NEVER comment on the tracking issue (issues are locked by `.github/workflows/ci-scan-lock-issues.yml`) — `add_comment` targets the PR. No `gh pr create`, no manual `git push`. **Defense-in-depth:** `add_comment` and `update_pull_request` use `target: "*"` (the agent supplies the PR number). - `add_comment` is now config-locked to `required-title-prefix: "[ci-fix] "` + + `add_comment`, `mark_pull_request_as_ready_for_review`, and `add_labels` are + config-locked to `required-title-prefix: "[ci-fix] "` + `required-labels: [agentic-workflows]` (hard-enforced by the handler, same as - `push_to_pull_request_branch`), so a comment can only ever land on THIS - workflow's own PRs. `update_pull_request` CANNOT be config-locked to a - title/label in gh-aw v0.79.8 (the compiler silently drops `required-*` for that - output — verified against the lock), so it keeps `title: false` (no retitles; - body-marker edits only) plus this prompt-level guard. Before emitting either, - VERIFY the target PR carries BOTH the `[ci-fix]` title prefix AND the - `agentic-workflows` label; never comment on or edit the body of any PR that - lacks both. + `push_to_pull_request_branch`), and `add_labels` additionally has an + `allowed: [p/0]` allowlist so it can ONLY ever add `p/0` — so these can only + ever land on THIS workflow's own PRs. `update_pull_request` CANNOT be + config-locked to a title/label in gh-aw v0.80.9 (the compiler silently drops + `required-*` for that output — verified against the lock), so it keeps + `title: false` (no retitles; body-marker edits only) plus this prompt-level + guard. Before emitting ANY of these, VERIFY the target PR carries BOTH the + `[ci-fix]` title prefix AND the `agentic-workflows` label; never comment on, + edit, un-draft, or label any PR that lacks both. 7. **Per-run safe-output caps (per RUN, not per PR).** Each output type is capped per run: `create_pull_request` 3, `push_to_pull_request_branch` 3, - `add_comment` 3, `update_pull_request` 3. Note an ADVANCE spends one push **and** - one update **and** one comment, so ≤ 3 advances/run; surfacing a green or - annotating a flake spends one comment. When a bucket is exhausted, do NOT keep + `add_comment` 6, `update_pull_request` 3, `mark_pull_request_as_ready_for_review` + 3, `add_labels` 3 (counts LABELS, not calls). Note an ADVANCE spends one push + **and** one update **and** one comment, so ≤ 3 advances/run; surfacing a green or + annotating a flake spends one comment; a **mark-ready (Step 3.6 T3)** of a + still-draft PR spends TWO comments (the Step 3 ✅ surface **and** the T3 🎯 + readiness note) **and** one mark-ready **and** one label as an ALL-OR-NOTHING set + (T3 pre-checks those buckets and defers the whole PR if any is exhausted — never + mark a PR ready without its 🎯 audit comment). `add_comment` is therefore sized 6 + (not 3) so ~3 draft flips, at 2 comments each, can land in one sweep instead of the + shared comment bucket starving the otherwise-idle mark-ready/add_labels buckets. When a bucket is exhausted, do NOT keep emitting (extras are silently dropped) — record `skipped: per-run cap reached; deferring PR # to next cycle` for each remaining PR so the drop is a deliberate, logged decision. The continuous loop picks the deferred PRs up next @@ -409,8 +471,9 @@ For every open tracking issue in scope, converge on exactly one outcome: | Open first help-wanted draft `[ci-fix]` PR | No open PR yet; a plausible candidate exists but cannot be runner-validated (device/UI tests) (attempt 1) | | Open first de-flake draft `[ci-fix]` PR | No open PR yet; failure is intermittent (green-on-retry) from a genuine test-quality defect; a deterministic-synchronization fix is producible (attempt 1, Step 4.7 bucket b) | | Advance an existing PR (push attempt N+1) | Open PR's own CI settled red *because of the fix itself*, no human engaged, marker < 10 — push a NEW distinct fix onto the same branch (Step 3.5 → 5.6) | -| Surface a validated-green PR | Open PR's own CI settled green — comment "validated, attempt N/10; ready for review" and do NOT advance (Step 3.5) | -| Annotate an unrelated-flake red | Open PR is red only on baseline-flake / unrelated legs — comment which leg needs a re-run; do NOT burn an attempt (Step 3.5) | +| Surface a validated-green PR | Open PR's own CI settled green — comment "validated, attempt N/10; ready for review" and do NOT advance (Step 3.5), then mark the draft PR ready for review once the fixed test is confirmed green (Step 3.6) | +| Annotate an unrelated-flake red | Open PR is red only on baseline-flake / unrelated legs — comment which leg needs a re-run; do NOT burn an attempt (Step 3.5); if the SPECIFIC fixed test is confirmed green on that build, still mark the draft PR ready for review (Step 3.6) | +| Mark a validated PR ready for review | The specific test this PR fixed is confirmed green in the PR's own CI (even if unrelated legs are red) — comment the target-test result and transition the draft PR to ready for review; never approve or merge (Step 3.6) | | Wait | Open PR's CI is pending / not yet settled on the current head SHA — do nothing this cycle (Step 3.5) | | Hand-off skip (attempt cap) | Marker == 10 and the signature still reproduces — stop and defer to humans; the dedicated `[ci-fix][needs-human]` PR is planned but currently deferred (Step 6) | | Recorded skip | Visual-regression, human engaged, already-handled, fixed-in-latest-build, infra-flake, out-of-bounds, only-mute-available, or no novel approach producible | @@ -685,14 +748,34 @@ Run these gates in order — the FIRST that fires decides this cycle's outcome: 1. **Human engaged.** If `C.humanEngaged` → `skipped: human engaged on PR #; deferring` and stop. Never fight a human reviewer — the intentional hand-off boundary still holds the moment a person touches the PR. -2. **CI not settled / unknown / incomplete prefetch.** If `C.checksSettled == false` - OR `C.overallConclusion` is `pending`, `neutral`, or `unknown`, OR - `C.dataComplete == false` → the fix's CI has not finished on the current head SHA - (`C.headSha`), or the prefetch could not fully read this PR (a partial read can - understate `humanEngaged` / `attempt`). `skipped: PR # CI pending / prefetch - incomplete on ; waiting` and stop. *(Round 1: this is where a - maintainer `/azp run` is awaited — the `/azp`-gated uitests/devicetests legs will - not have run until a human kicks them.)* +2. **CI not settled — but validate the target test first (target-focused readiness).** + If `C.checksSettled == false` OR `C.overallConclusion` is `pending`, `neutral`, or + `unknown`, OR `C.dataComplete == false` → the *overall* build has not finished on the + current head SHA (`C.headSha`), or the prefetch could not fully read this PR (a partial + read can understate `humanEngaged` / `attempt`). Unrelated legs still draining must NOT + keep an already-proven fix parked as a draft, so before waiting, try the target-focused + fast-path: + - **2a — Target-green fast-path (draft `[ci-fix]` PRs only).** If `C.dataComplete == + true` AND the Step 3.6 preconditions hold (draft PR, `[ci-fix] ` title prefix AND the + `agentic-workflows` label), run Step 3.6 **T1–T2** against `C.headSha` now: identify + the target test(s) and read the PR's OWN build **timeline / per-leg status** (anonymous + `_apis/build` — never `_apis/test`, per Hard-Rule 8). If EVERY target test is + **VALIDATED-GREEN** (per T2's all-platform definition below — the target's category leg + is `succeeded` on every platform that runs it, failed on none, and NO target platform + family the pipeline covers is still pending/unconcluded, per T2's "no platform left + unverified" rule; a platform that simply has no leg for the target's category — the test + doesn't run there — is fine, not a gap) AND every leg in + `C.failedLegs` that has ALREADY concluded classifies as **unrelated flake** (Step 4 / + Step 4.7 method — the fix is implicated in NO completed red), then the fix is proven + regardless of unrelated *pending* legs → run **Step 3.6 T3** (mark ready + 🎯 comment) + and stop. This early-out NEVER advances an attempt and NEVER acts on a red that could + be the fix's fault. + - **2b — Otherwise WAIT.** If the target test has not yet executed (pending/absent on + its leg), or a completed red is (or may be) caused by the fix, or `C.dataComplete == + false`, or this PR has no identifiable target test → `skipped: PR # CI pending / + target not yet validated on ; waiting` and stop. *(Round 1: this is where a + maintainer `/azp run` is awaited — the `/azp`-gated uitests/devicetests legs will not + have run until a human kicks them.)* 3. **Green → surface for review.** If `C.overallConclusion == "success"`: the checks that RAN are green. **Primary-gate check first:** the deterministic `success` verdict only certifies "at least one green check, nothing failing or @@ -704,18 +787,31 @@ Run these gates in order — the FIRST that fires decides this cycle's outcome: # primary CI gate (maui-pr) not green on ; waiting` and stop — do NOT surface. (The `/azp`-gated `maui-pr-uitests` (def 313) / `maui-pr-devicetests` (def 314) legs MAY still be un-run — that is expected and is named below, not a - reason to withhold the surface.) **Idempotency:** scan the PR's existing comments - for a prior bot `✅ … validated … on ` note for THIS head SHA — if one - already exists, record `already-surfaced PR # (head )` and stop - WITHOUT re-commenting (re-surfacing the same green every tick is noise and burns - the per-run comment budget other PRs need). Otherwise resolve the attempt number from - `C.effectiveAttempt` (the authoritative max(marker, bot-commit) counter; omit the - number only if it is somehow indeterminate). **Dry-run gate (Step 0):** if `dry_run == "true"`, do NOT emit any comment — instead print the intended `✅` validated-green body to the run log, tally `dry-run: would-surface-green PR #`, and stop. Otherwise `add_comment` on PR #: `✅ Attempt /10 validated - — the fix's CI is green on . Ready for human review.` - Name any `/azp`-gated legs (uitests def 313 / devicetests def 314) that have not - run and still need a maintainer `/azp run`. Do NOT advance. Record - `surfaced-green PR # (attempt /10)` and stop. *(This directly - attacks the real bottleneck — no reviews — so it is the highest-value outcome.)* + reason to withhold the surface.) **Comment idempotency + dry-run suppress the ✅ + comment ONLY — neither skips Step 3.6.** Scan the PR's existing comments for a prior + bot `✅ … validated … on ` note for THIS head SHA; if one exists, set + `SKIP_SURFACE_COMMENT = true`. Resolve the attempt number from `C.effectiveAttempt` + (the authoritative max(marker, bot-commit) counter; omit the number only if it is + somehow indeterminate). Then post the surface comment UNLESS suppressed: + - if `dry_run == "true"`: do NOT emit — print the intended `✅` validated-green body to + the run log and tally `dry-run: would-surface-green PR #`; + - else if `SKIP_SURFACE_COMMENT`: do NOT re-post — record `already-surfaced PR # + (head )` (re-surfacing the same green every tick is noise and burns the + per-run comment budget other PRs need); + - else `add_comment` on PR #: `✅ Attempt /10 validated — the fix's CI is + green on .` naming any `/azp`-gated legs (uitests def 313 / devicetests def + 314) that have not run and still need a maintainer `/azp run`; record `surfaced-green + PR # (attempt /10)`. (Do NOT assert "ready for human review" on a PR that + is still a draft — Step 3.6, not this comment, owns the draft→ready flip and posts its + own 🎯 announcement when it fires.) + Do NOT advance. Then — in ALL of the above cases — run **Step 3.6** (target-test + readiness gate) for this PR before stopping: its `/azp`-gated target legs may conclude + green on this SAME head SHA on a LATER sweep (an `/azp run` adds no commit), and Step + 3.6 is the ONLY place the draft→ready flip happens, so it MUST re-evaluate every sweep — + never `stop` here before it. *(This directly attacks the real bottleneck — no reviews — + so it is the highest-value outcome.)* 4. **Red → classify caused-by-fix vs unrelated-flake.** If `C.overallConclusion == "failure"`, analyze the PR's OWN failing build (NOT `main`): find the AzDO `maui-pr` build for this PR (filter builds by `branchName=refs/pull//merge`, @@ -723,18 +819,26 @@ Run these gates in order — the FIRST that fires decides this cycle's outcome: method and Step 4.7 flake buckets to `C.failedLegs`. - **Unrelated flake only** (every failed leg is known-flaky / infra / a pre-existing baseline red NOT introduced by the fix): do NOT burn an attempt. - **Idempotency first:** scan the PR's existing comments for a prior bot - `♻️ … unrelated flake … on ` note for THIS head SHA — if one already - exists, record `already-annotated-flake PR # (head )` and stop - WITHOUT re-commenting (re-annotating the same flake every 12h sweep is noise and - burns the per-run comment budget other PRs need). Otherwise resolve the attempt - number from `C.effectiveAttempt` (the authoritative max(marker, bot-commit) - counter), omitting the `Attempt /10:` prefix only if it is somehow - indeterminate. **Dry-run gate (Step 0):** if `dry_run == "true"`, do NOT emit any comment — instead print the intended `♻️` unrelated-flake body to the run log, tally `dry-run: would-annotate-flake PR #`, and stop. Otherwise `add_comment` on PR #: `♻️ Attempt /10: red is - unrelated flake on leg(s) () on ; the fix itself is not - implicated. A maintainer re-run (/azp run ) should clear it.` Record - `annotated-flake PR # (head )` and stop. *(Round 1: human re-runs; - Round 2: auto re-trigger.)* + **Comment idempotency + dry-run suppress the ♻️ comment ONLY — neither skips Step + 3.6.** Scan the PR's existing comments for a prior bot `♻️ … unrelated flake … on + ` note for THIS head SHA; if one exists, set `SKIP_FLAKE_COMMENT = true`. + Resolve the attempt number from `C.effectiveAttempt` (the authoritative max(marker, + bot-commit) counter), omitting the `Attempt /10:` prefix only if it is + somehow indeterminate. Then post the flake note UNLESS suppressed: + - if `dry_run == "true"`: do NOT emit — print the intended `♻️` unrelated-flake body + to the run log and tally `dry-run: would-annotate-flake PR #`; + - else if `SKIP_FLAKE_COMMENT`: do NOT re-post — record `already-annotated-flake PR + # (head )` (re-annotating the same flake every 12h sweep is noise and + burns the per-run comment budget other PRs need); + - else `add_comment` on PR #: `♻️ Attempt /10: red is unrelated flake on + leg(s) () on ; the fix itself is not implicated. A + maintainer re-run (/azp run ) should clear it.` record `annotated-flake + PR # (head )`. + Then — in ALL of the above cases — run **Step 3.6** (target-test readiness gate) for + this PR before stopping: its `/azp`-gated target legs may conclude green on this SAME + head SHA on a LATER sweep, and Step 3.6 is the ONLY place the draft→ready flip + happens, so it MUST re-evaluate every sweep — never `stop` here before it. *(Round 1: + human re-runs; Round 2: auto re-trigger.)* - **Caused by the fix** (a failed leg still matches the original target signature, or the fix introduced a NEW failure): advance an attempt. - **Attempt count.** `attempt = C.effectiveAttempt` — the authoritative @@ -752,6 +856,188 @@ Run these gates in order — the FIRST that fires decides this cycle's outcome: from every prior commit on this branch**, emitted via the Step 5.6 ADVANCE path (push onto the same PR). +#### Step 3.6 — Target-test verification & mark-ready gate + +Reached from the Step 3 **green-surface** branch, the Step 4 **unrelated-flake** branch +(AFTER that branch has posted its comment), and the Step 3.5 **gate-2 target-focused +fast-path** (2a — while unrelated legs are still pending). Purpose: confirm the SPECIFIC +test(s) this PR was opened to fix now PASS in the PR's own CI, and — only when they do +— transition the draft `[ci-fix]` PR to **ready for review** so a maintainer sees a +validated fix instead of a draft. This is the ONLY place the loop flips draft→ready; it +is a state transition, **never** an approval or a merge (a human still reviews and +merges). Overall red on *unrelated* legs must NOT gate readiness — we validate the fix, +not the base branch's flakiness. + +**Preconditions** (ALL must hold; otherwise record `skipped: readiness N/A PR # +()` and stop this gate): +- `C.isDraft == true` — if the PR is already ready-for-review, the draft→ready + transition is already done; do NOT re-mark **and do NOT re-apply `p/0`**. `p/0` is applied + exactly ONCE, atomically with the draft→ready flip (T3 — all-or-nothing with the 🎯 audit + comment + mark-ready), so an already-ready loop PR that is MISSING `p/0` has almost + certainly had it **removed by a maintainer** de-prioritizing the PR — a pure triage action + the human-engagement guard (which inspects comments/reviews/commits, NOT label changes) + cannot see. Re-adding `p/0` every sweep would fight that maintainer indefinitely, violating + the loop's "never override a human" contract. So the loop does NOT reconcile the label: + record `already-ready PR #` and stop this gate. (Trade-off: on the rare occasion a + transient API error drops `p/0` at flip time *after* the T3 atomic pre-check passed, it is + not auto-re-added — but the PR is still ready-for-review with its 🎯 audit comment, and + re-adding `p/0` is a trivial manual action; that is strictly preferable to steam-rolling a + maintainer's deliberate de-prioritization.) +- The PR is unmistakably THIS workflow's own: `[ci-fix] ` title prefix AND the + `agentic-workflows` label (mirrors the safe-output lock; the compensating scope + control if the v0.80.9 compiler drops the declarative required-*). +- You reached this gate from Step 3 (green), Step 4 (**unrelated-flake**), or the Step 3.5 + gate-2 **target-focused fast-path** (2a) — i.e. the fix is NOT implicated in any red that + has concluded. If Step 4 classified a red as **caused by the fix** (ADVANCE), do NOT run + this gate — advance the attempt instead. + +**T1 — Identify the target test(s).** From the `[ci-scan]` issue signature the fix +addresses (and the PR's own diff), extract the fully-qualified test method name(s) the +fix targets — e.g. `SafeAreaShouldWorkOnAllShellTabs`. For a de-flake it is the +de-flaked test; for a product fix it is the originally-failing test(s). If NO specific +test can be identified (e.g. a product build-break rather than a test failure), this is a +**build-only fix** — there is no single test to validate cross-platform, so validate at +**whole-build** granularity rather than stopping (Step 3 only *comments*; Step 3.6 is the +sole draft→ready flip, so a build-only fix is undrafted HERE or not at all). We only reach +this gate from Step 3 (green) / Step 4 (unrelated-flake) / gate-2a, so the fix is not +implicated in any concluded red; additionally require, via the T2 timeline method on +`C.headSha`, that the primary build pipeline `maui-pr` (def 302) has CONCLUDED with EVERY +one of its platform build legs `succeeded`/`completed` and NONE failed/canceled or still +pending — the build is green on **every** platform, not just the originally-broken one (the +cross-platform guard, applied to the build instead of a test). A build-only fix is undrafted +ONLY on the strength of the auto-running `maui-pr` (def 302) whole-build green, which the +loop CAN observe: if the PR's `[ci-scan]` issue signature or its own diff indicates the +ORIGINATING failure was in a `/azp`-gated pipeline (`maui-pr-uitests` def 313 or +`maui-pr-devicetests` def 314) rather than `maui-pr` (def 302), do NOT undraft on +`maui-pr`-green alone — those pipelines do not auto-run on this PR (GITHUB_TOKEN cannot +trigger them), so a green `maui-pr` build is NOT evidence the gated build break is fixed; +record `skipped: build-only fix PR # targets gated pipeline () not run — +deferring to human` and stop this gate. Otherwise, set `TARGET := "the maui-pr build +(build-only fix — no single target test)"` and proceed to **T3** to mark ready. If any `maui-pr` build leg is still unconcluded, record `skipped: build-only fix PR +# not yet whole-build green (leg(s) pending)` and stop this gate WITHOUT marking +ready (a green subset is not enough — a still-pending build leg could yet fail). + +**T2 — Verify the target test's platform legs are green (anonymous, leg-level).** Per-test +outcomes come from the AzDO **test-results** API (`_apis/test/...`), which is **NOT reachable +anonymously — Hard-Rule 8**: those endpoints 302-redirect to a sign-in page on this runner, so +a `curl` returns an HTML redirect (not JSON) and every target test would look "not executed". +Validate instead at **leg granularity** using ONLY the anonymous `_apis/build/...` timeline +(Hard-Rule 8) — a `succeeded` category leg means every test in that category **that actually +ran** passed on that platform. This is a strong bar **only if the target test genuinely +executes**: a job can go green while one specific test is skipped at runtime (`[Ignore]`, +`Assert.Ignore()`, `Assert.Inconclusive()`, a `Skip=`/conditional `[Fact]`, an `#if`-out, or +an early `return` before the asserts). So before trusting a green leg as proof the target +passed, **confirm from the PR's OWN diff that the fix does NOT skip, ignore, disable, or +short-circuit the target test** (it adds no `[Ignore]`/`[Explicit]`, `Assert.Ignore`/ +`Assert.Inconclusive`, `Skip=`, category-exclusion, `#if`-out, or early `return` around the +target). If the fix could cause the target to be runtime-skipped rather than genuinely pass, +leg-level green is NOT sufficient evidence — record `skipped: target test may be +runtime-skipped by this fix (diff adds ); leg-green insufficient, deferring to human +on PR #` and stop this gate WITHOUT marking ready. Otherwise (the target genuinely runs +and asserts, as a de-flake or product fix does) a green category leg cannot hide a +target-test failure, so treat it as authoritative. + +Map the target test to the CI leg(s) that run it. A leg name encodes platform + test category +(e.g. `Android UITests SafeAreaEdges,Shadow`, `iOS UITests SafeAreaEdges,Shadow`, `macOS +UITests SafeAreaEdges,Shadow`, `Windows UITests SafeAreaEdges,Shadow`). Determine the target +test's UI-test category — its `[Category(UITestCategories.X)]` in the test/HostApp source, +visible in the PR diff or the test file — to know which leg-name substring identifies its legs; +for a device test, the per-platform device-test legs. + +Using the SAME build-discovery as Step 4 (filter AzDO builds by `branchName=refs/pull//merge` +or `sourceVersion == C.headSha`), read each build's **timeline** on `C.headSha` for the +pipeline(s) that RUN the target test — `maui-pr` (def 302) for unit/integration tests, +`maui-pr-uitests` (def 313) for Appium UI tests, `maui-pr-devicetests` (def 314) for device +tests: + +```bash +ORG=dnceng-public; PROJ=public; BUILD= +# Anonymous + Hard-Rule-8-compliant. NEVER call _apis/test/... (it 302-redirects to sign-in). +# Each type=="Job" record is one platform leg: result (succeeded/failed/canceled/null), +# state (completed/inProgress/pending), name (encodes " UITests "): +curl -s "https://dev.azure.com/$ORG/$PROJ/_apis/build/builds/$BUILD/timeline?api-version=7.1" \ + | tee /tmp/gh-aw/agent/timeline_${BUILD}.json \ + | jq -r '.records[] | select(.type=="Job") | "\(.result)\t\(.state)\t\(.name)"' +``` + +(The prefetch already exposes per-leg status in `C.failedLegs` / the checks context, and +`gh pr checks ` lists the same per-leg rows with platform+category in the name — use +whichever is handy; the build timeline is the authoritative cross-check on the exact build.) + +Treat a target test as **VALIDATED-GREEN** only if, on `C.headSha`, the leg that runs its +category is green on **every platform it runs on** — a fix that repairs one platform must not +silently regress the same test's leg on another, so a green leg on the originally-red platform +alone is NOT enough. Across the target pipeline's platform legs (for a UI test: the Android, +iOS, Windows, and macOS/MacCatalyst legs running the target's category; for a device test: each +device-test platform), require ALL of: +- the target's category leg is `result == "succeeded"` **and** `state == "completed"` on + **every** platform that runs it, AND +- that leg is `failed` / `canceled` / aborted on **NO** platform, AND +- **no platform is left unverified.** For each platform family the target pipeline covers, that + platform's category leg must have CONCLUDED (`state == "completed"`) on `C.headSha`. If a + platform simply has no leg for the target's category, the test does not run there — that is + fine, not a gap. But if a platform's category leg has **not concluded** (`state` is + `inProgress` / pending, or an `/azp`-gated `maui-pr-uitests` / `maui-pr-devicetests` leg that + has not been kicked), the test's status on that platform is UNKNOWN → the fix is NOT yet + validated across platforms: record `skipped: target test green on but + not yet verified on (leg(s) pending / need /azp run) on PR #` + and stop this gate WITHOUT marking ready. + +A target test whose category leg never concluded on ANY platform (**not executed** anywhere — +e.g. its `/azp`-gated pipeline has not been kicked) is likewise NOT validated: record `skipped: +target test not yet executed on PR # ( not run — needs /azp run)` and stop +this gate WITHOUT marking ready. Do NOT overclaim — a green *sibling* leg (a different category +on the same platform) is not the target's leg, and a green leg on one platform is not a pass on +the others. + +**T3 — Mark ready + report.** If EVERY target test is VALIDATED-GREEN. The 🎯 comment, +the mark-ready, and the `p/0` label are THREE SEPARATE safe-outputs — the comment +existing does NOT prove the mark-ready took effect, so they are tracked independently: + +- **Comment idempotency (dup-suppress only — never gates the mark-ready).** We only reach + T3 while `C.isDraft == true` (the precondition stops an already-ready PR before here). So + if a prior bot comment for THIS head that carries the leading `🎯` anchor AND the + contiguous phrase `validated green on ` already exists (BOTH T3 variants — + `🎯 Target test validated green on ` and the build-only `🎯 Build validated green on + ` — contain that exact contiguous phrase, so this recognizes both; keying on + `target test` alone would instead miss the build-only comment and re-post it every sweep. + **Require the `🎯` anchor** so the match can NEVER be loosened to a gapped + `validated…green…on` form that would false-positive on the Step 3 `✅ … validated — the + fix's CI is green on ` surface comment and wrongly suppress the audit 🎯), the comment + landed on an earlier sweep but the **mark-ready did NOT take + effect** (the PR is still a draft) — set `SUPPRESS_COMMENT = true` (do not re-post the + duplicate 🎯 comment) but STILL complete the mark-ready + label below. Never treat the + comment's existence as "already marked ready" while the PR is still a draft. Record this + case as `re-marking PR # (🎯 present; mark-ready did not take on a prior sweep)`. +- **Atomicity budget pre-check (Hard-Rule 7).** Determine the outputs this gate will emit: + `mark_pull_request_as_ready_for_review` + `add_labels` always, plus `add_comment` unless + `SUPPRESS_COMMENT`. Verify a free per-run slot remains in EACH bucket you are about to + use. If ANY required bucket is exhausted, emit NONE of them — record `skipped: per-run + cap reached; deferring mark-ready PR # to next cycle` and stop this gate. Never mark a + PR ready (or label it) unless its 🎯 audit comment is GUARANTEED to exist for this exact + head SHA — either posted in this same sweep, or (in the `SUPPRESS_COMMENT` re-marking case) + already present from a prior sweep. The outputs you DO emit either all land or all defer + together; the audit trail must never be absent, but it is NOT re-posted when it already exists. +- **Dry-run gate (Step 0):** if `dry_run == "true"`, emit NOTHING — print the intended + readiness comment and "would mark ready + add p/0" to the run log, tally `dry-run: + would-mark-ready PR #`, and stop. +- Otherwise emit for THIS PR number ``: + 1. `add_comment` (ONLY if not `SUPPRESS_COMMENT`): `🎯 Target test validated green on + — passed on ALL platforms it runs on (, + buildId ). — not + caused by this fix."> Transitioning this PR from draft to ready for review and adding + `p/0`; a maintainer still reviews and merges.` (For a **build-only fix** — the T1 + whole-build fallback, no single target test — phrase the first clause as `🎯 Build + validated green on — the maui-pr build passed on ALL platforms (buildId + ).` instead of naming a test.) + 2. `mark_pull_request_as_ready_for_review` with `reason:` a one-line justification + naming the validated test(s) and ``. + 3. `add_labels` with `labels: ["p/0"]` for PR # — put the now-review-ready fix into + the team's p/0 priority queue so it is triaged, not lost in the draft backlog. (If + the PR somehow already carries `p/0`, this is a harmless no-op.) +- Record `marked-ready PR # (target green on ALL platforms on , + labeled p/0)` and stop. + #### Step 3.5.R — Maintainer change-request response (Track C) Reached from Step 3.5 gate 0. Goal: when an **eligible human reviewer** (Hard-Rule @@ -1383,7 +1669,19 @@ Filed by [`ci-status-fix`](https://github.com/dotnet/maui/blob/main/.github/work ### Step 8 — Per-issue tally + end-of-run summary -Per issue, append one outcome line to `/tmp/gh-aw/agent/coverage.txt`: +Per issue, append **one** outcome line to `/tmp/gh-aw/agent/coverage.txt` — the +terminal outcome for the cycle. When one cycle produces a chained pair — a PR gets a +same-cycle precursor line and **then** a Step 3.6 readiness line (`marked-ready` on the +draft→ready flip, or `already-ready` on a later steady-state sweep of an already-ready +PR) — record ONLY the terminal Step 3.6 readiness line: it supersedes ANY same-cycle +non-readiness precursor for that PR, so an aggregator keying on one-line-per-issue never +double-counts. The precursor may be a Step 3 green line (`surfaced-green` on the first ✅, +or `already-surfaced` when it already exists) OR — when an unrelated-flake red and a +target-green coincide — a Step 4 `annotated-flake` line; either way the readiness line is +terminal, and the superseded precursor's signal survives in the PR's 🎯/♻️ comment so no +information is lost. This covers the flip cycle (`surfaced-green` → `marked-ready`), the +post-flip steady state (`already-surfaced` → `already-ready`), and the flake-coincident +flip (`annotated-flake` → `marked-ready`): ``` # main attempt- @@ -1394,8 +1692,14 @@ Per issue, append one outcome line to `/tmp/gh-aw/agent/coverage.txt`: `advance-PR # attempt /10` (ADVANCE mode — new commit pushed to the existing PR), `surfaced-green PR #` (fix's own CI went green; commented for review, did not advance), `annotated-flake PR #` (red was unrelated flake; -commented, attempt NOT burned), `waiting PR #` (CI not settled yet), -`dry-run: would-`, `skipped: `. (The +commented, attempt NOT burned), `marked-ready PR #` (the specific fixed test +was confirmed green in the PR's own CI; draft flipped to ready for review), +`already-surfaced PR #` (the fix's ✅ green comment already existed this sweep; +re-post suppressed — superseded by the Step 3.6 readiness line when the PR also flips +ready that cycle), `already-ready PR #` (terminal steady state — the PR was flipped +ready on a prior cycle and remains green on an unchanged head; no action taken, +supersedes `already-surfaced`), `waiting PR #` (CI not settled yet), +`dry-run: would-`, `skipped: `. (The `needs-human-PR` outcome is reserved for the deferred hand-off — Step 6 currently records a skip instead, so it is not emitted.)
— put the now-review-ready fix into + the team's p/0 priority queue so it is triaged, not lost in the draft backlog. (If + the PR somehow already carries `p/0`, this is a harmless no-op.) +- Record `marked-ready PR #
(target green on ALL platforms on , + labeled p/0)` and stop. + #### Step 3.5.R — Maintainer change-request response (Track C) Reached from Step 3.5 gate 0. Goal: when an **eligible human reviewer** (Hard-Rule @@ -1395,7 +1681,19 @@ Filed by [`ci-status-fix-net11`](https://github.com/dotnet/maui/blob/main/.githu ### Step 8 — Per-issue tally + end-of-run summary -Per issue, append one outcome line to `/tmp/gh-aw/agent/coverage.txt`: +Per issue, append **one** outcome line to `/tmp/gh-aw/agent/coverage.txt` — the +terminal outcome for the cycle. When one cycle produces a chained pair — a PR gets a +same-cycle precursor line and **then** a Step 3.6 readiness line (`marked-ready` on the +draft→ready flip, or `already-ready` on a later steady-state sweep of an already-ready +PR) — record ONLY the terminal Step 3.6 readiness line: it supersedes ANY same-cycle +non-readiness precursor for that PR, so an aggregator keying on one-line-per-issue never +double-counts. The precursor may be a Step 3 green line (`surfaced-green` on the first ✅, +or `already-surfaced` when it already exists) OR — when an unrelated-flake red and a +target-green coincide — a Step 4 `annotated-flake` line; either way the readiness line is +terminal, and the superseded precursor's signal survives in the PR's 🎯/♻️ comment so no +information is lost. This covers the flip cycle (`surfaced-green` → `marked-ready`), the +post-flip steady state (`already-surfaced` → `already-ready`), and the flake-coincident +flip (`annotated-flake` → `marked-ready`): ``` # net11.0 attempt- @@ -1406,8 +1704,14 @@ Per issue, append one outcome line to `/tmp/gh-aw/agent/coverage.txt`: `advance-PR # attempt /10` (ADVANCE mode — new commit pushed to the existing PR), `surfaced-green PR #` (fix's own CI went green; commented for review, did not advance), `annotated-flake PR #` (red was unrelated flake; -commented, attempt NOT burned), `waiting PR #` (CI not settled yet), -`dry-run: would-`, `skipped: `. (The +commented, attempt NOT burned), `marked-ready PR #` (the specific fixed test +was confirmed green in the PR's own CI; draft flipped to ready for review), +`already-surfaced PR #` (the fix's ✅ green comment already existed this sweep; +re-post suppressed — superseded by the Step 3.6 readiness line when the PR also flips +ready that cycle), `already-ready PR #` (terminal steady state — the PR was flipped +ready on a prior cycle and remains green on an unchanged head; no action taken, +supersedes `already-surfaced`), `waiting PR #` (CI not settled yet), +`dry-run: would-`, `skipped: `. (The `needs-human-PR` outcome is reserved for the deferred hand-off — Step 6 currently records a skip instead, so it is not emitted.) diff --git a/.github/workflows/ci-status-fix.lock.yml b/.github/workflows/ci-status-fix.lock.yml index 93dc5969cb0d..65511f2b3259 100644 --- a/.github/workflows/ci-status-fix.lock.yml +++ b/.github/workflows/ci-status-fix.lock.yml @@ -1,4 +1,4 @@ -# gh-aw-metadata: {"schema_version":"v4","frontmatter_hash":"f014161967589ebcaf1ac5ec6f3c88ee5a4388d28b55a07dc36c45b7b2aebab6","body_hash":"86c6b2d4c3df13da14c6a3c8b211381fcc9f97bb1951ff7b80588a2c5338e00c","compiler_version":"v0.80.9","strict":true,"agent_id":"copilot","agent_model":"claude-opus-4.8","engine_versions":{"copilot":"1.0.63"}} +# gh-aw-metadata: {"schema_version":"v4","frontmatter_hash":"b0ebf8a2f179900cfe3638e994b27a43cec52bdbdd2331dc1bd4d65762fa2d6b","body_hash":"1825e2b1f7c7bd88241bf37580655489360f9bebc806edd00837a52ce4ad83a0","compiler_version":"v0.80.9","strict":true,"agent_id":"copilot","agent_model":"claude-opus-4.8","engine_versions":{"copilot":"1.0.63"}} # gh-aw-manifest: {"version":1,"secrets":["COPILOT_PAT_0","COPILOT_PAT_1","COPILOT_PAT_2","COPILOT_PAT_3","COPILOT_PAT_4","COPILOT_PAT_5","COPILOT_PAT_6","COPILOT_PAT_7","COPILOT_PAT_8","COPILOT_PAT_9","GH_AW_CI_TRIGGER_TOKEN","GH_AW_GITHUB_MCP_SERVER_TOKEN","GH_AW_GITHUB_TOKEN","GITHUB_TOKEN"],"actions":[{"repo":"actions/checkout","sha":"34e114876b0b11c390a56381ad16ebd13914f8d5","version":"v4"},{"repo":"actions/checkout","sha":"9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0","version":"v7.0.0"},{"repo":"actions/checkout","sha":"de0fac2e4500dabe0009e67214ff5f5447ce83dd","version":"v6.0.2"},{"repo":"actions/download-artifact","sha":"3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c","version":"v8.0.1"},{"repo":"actions/github-script","sha":"3a2844b7e9c422d3c10d287c895573f7108da1b3","version":"v9.0.0"},{"repo":"actions/setup-node","sha":"48b55a011bda9f5d6aeb4c2d9c7362e8dae4041e","version":"v6.4.0"},{"repo":"actions/upload-artifact","sha":"043fb46d1a93c77aae656e7c1c64a875d1fc6a0a","version":"v7.0.1"},{"repo":"github/gh-aw-actions/setup","sha":"8c7d04ebf1ece56cd381446125da3e0f6896294a","version":"v0.80.9"}],"containers":[{"image":"ghcr.io/github/gh-aw-firewall/agent:0.27.7","digest":"sha256:aae231e4635c8999d039c132f1602d3df850fe9b84a00aa2b5ac981179b5661c","pinned_image":"ghcr.io/github/gh-aw-firewall/agent:0.27.7@sha256:aae231e4635c8999d039c132f1602d3df850fe9b84a00aa2b5ac981179b5661c"},{"image":"ghcr.io/github/gh-aw-firewall/api-proxy:0.27.7","digest":"sha256:009caf2e3d88fa77b64e9a03a95a228fc58db0f1701c6d324b29ba5a3c7c79b6","pinned_image":"ghcr.io/github/gh-aw-firewall/api-proxy:0.27.7@sha256:009caf2e3d88fa77b64e9a03a95a228fc58db0f1701c6d324b29ba5a3c7c79b6"},{"image":"ghcr.io/github/gh-aw-firewall/squid:0.27.7","digest":"sha256:deb1d4e19de62d51cee0508057a596a19315c3423ada4d675cad136dc8037c96","pinned_image":"ghcr.io/github/gh-aw-firewall/squid:0.27.7@sha256:deb1d4e19de62d51cee0508057a596a19315c3423ada4d675cad136dc8037c96"},{"image":"ghcr.io/github/gh-aw-mcpg:v0.3.27","digest":"sha256:fe984bddde4ec05d756d9043edb0a32912e6b7b72f6a121b1082f29221421cc7","pinned_image":"ghcr.io/github/gh-aw-mcpg:v0.3.27@sha256:fe984bddde4ec05d756d9043edb0a32912e6b7b72f6a121b1082f29221421cc7"},{"image":"ghcr.io/github/gh-aw-node","digest":"sha256:529d02eb970b1161aa25c593a9c3df57fdfad5a8add328cb3b6eccef66f3183b","pinned_image":"ghcr.io/github/gh-aw-node@sha256:529d02eb970b1161aa25c593a9c3df57fdfad5a8add328cb3b6eccef66f3183b"},{"image":"ghcr.io/github/github-mcp-server:v1.4.0","digest":"sha256:2afb26356481d1a350e14544a6e160f7f7ec1561a1ea309b823665abf0309036","pinned_image":"ghcr.io/github/github-mcp-server:v1.4.0@sha256:2afb26356481d1a350e14544a6e160f7f7ec1561a1ea309b823665abf0309036"}]} # This file was automatically generated by gh-aw (v0.80.9). DO NOT EDIT. To debug this workflow, load the skill at https://github.com/github/gh-aw/blob/main/debug.md # @@ -37,7 +37,9 @@ # humans (the open tracking issue is the hand-off surface; a dedicated # [ci-fix][needs-human] PR is planned but currently deferred — see Step 6). CI on # the PR is kicked by a human `/azp run` for now; the loop watches, classifies, -# and re-fixes autonomously between kicks. +# and re-fixes autonomously between kicks. When the SPECIFIC test a PR fixed is +# confirmed green in that PR's own CI, the loop marks the draft PR ready for review +# (a state transition only — it never approves or merges). # Never mutes tests, but # de-flakes genuinely flaky ones (deterministic synchronization, no retries / # timeout bumps). Always skips visual-regression / screenshot issues. @@ -273,24 +275,24 @@ jobs: run: | bash "${RUNNER_TEMP}/gh-aw/actions/create_prompt_first.sh" { - cat << 'GH_AW_PROMPT_25cf75111b670167_EOF' + cat << 'GH_AW_PROMPT_2ac1d0b4083428f8_EOF' - GH_AW_PROMPT_25cf75111b670167_EOF + GH_AW_PROMPT_2ac1d0b4083428f8_EOF cat "${RUNNER_TEMP}/gh-aw/prompts/xpia.md" cat "${RUNNER_TEMP}/gh-aw/prompts/temp_folder_prompt.md" cat "${RUNNER_TEMP}/gh-aw/prompts/markdown.md" cat "${RUNNER_TEMP}/gh-aw/prompts/safe_outputs_prompt.md" - cat << 'GH_AW_PROMPT_25cf75111b670167_EOF' + cat << 'GH_AW_PROMPT_2ac1d0b4083428f8_EOF' - Tools: add_comment(max:3), create_pull_request(max:3), update_pull_request(max:3), push_to_pull_request_branch(max:3), missing_tool, missing_data, noop - GH_AW_PROMPT_25cf75111b670167_EOF + Tools: add_comment(max:6), create_pull_request(max:3), update_pull_request(max:3), mark_pull_request_as_ready_for_review(max:3), add_labels(max:3), push_to_pull_request_branch(max:3), missing_tool, missing_data, noop + GH_AW_PROMPT_2ac1d0b4083428f8_EOF cat "${RUNNER_TEMP}/gh-aw/prompts/safe_outputs_create_pull_request.md" cat "${RUNNER_TEMP}/gh-aw/prompts/safe_outputs_push_to_pr_branch.md" - cat << 'GH_AW_PROMPT_25cf75111b670167_EOF' + cat << 'GH_AW_PROMPT_2ac1d0b4083428f8_EOF' - GH_AW_PROMPT_25cf75111b670167_EOF + GH_AW_PROMPT_2ac1d0b4083428f8_EOF cat "${RUNNER_TEMP}/gh-aw/prompts/mcp_cli_tools_prompt.md" - cat << 'GH_AW_PROMPT_25cf75111b670167_EOF' + cat << 'GH_AW_PROMPT_2ac1d0b4083428f8_EOF' The following GitHub context information is available for this workflow: {{#if github.actor}} @@ -332,12 +334,12 @@ jobs: stop immediately and report the limitation rather than spending turns trying to work around it. - GH_AW_PROMPT_25cf75111b670167_EOF + GH_AW_PROMPT_2ac1d0b4083428f8_EOF cat "${RUNNER_TEMP}/gh-aw/prompts/github_mcp_tools_with_safeoutputs_prompt.md" - cat << 'GH_AW_PROMPT_25cf75111b670167_EOF' + cat << 'GH_AW_PROMPT_2ac1d0b4083428f8_EOF' {{#runtime-import .github/workflows/ci-status-fix.md}} - GH_AW_PROMPT_25cf75111b670167_EOF + GH_AW_PROMPT_2ac1d0b4083428f8_EOF } > "$GH_AW_PROMPT" - name: Interpolate variables and render templates uses: actions/github-script@3a2844b7e9c422d3c10d287c895573f7108da1b3 # v9.0.0 @@ -554,16 +556,18 @@ jobs: mkdir -p "${RUNNER_TEMP}/gh-aw/safeoutputs" mkdir -p /tmp/gh-aw/safeoutputs mkdir -p /tmp/gh-aw/mcp-logs/safeoutputs - cat > "${RUNNER_TEMP}/gh-aw/safeoutputs/config.json" << 'GH_AW_SAFE_OUTPUTS_CONFIG_fa2a69fad5fd0485_EOF' - {"add_comment":{"discussions":false,"max":3,"required_labels":["agentic-workflows"],"required_title_prefix":"[ci-fix] ","target":"*"},"create_pull_request":{"allowed_base_branches":["main"],"allowed_branches":["ci-fix/**"],"allowed_files":["src/Core/**","src/Controls/**","src/Essentials/**","src/BlazorWebView/**","src/TestUtils/**","src/Templates/**","**/PublicAPI.Unshipped.txt"],"base_branch":"main","draft":true,"labels":["agentic-workflows"],"max":3,"max_patch_files":100,"max_patch_size":4096,"protect_top_level_dot_folders":true,"protected_files":["package.json","bun.lockb","bunfig.toml","deno.json","deno.jsonc","deno.lock","global.json","NuGet.Config","Directory.Packages.props","mix.exs","mix.lock","go.mod","go.sum","stack.yaml","stack.yaml.lock","pom.xml","build.gradle","build.gradle.kts","settings.gradle","settings.gradle.kts","gradle.properties","package-lock.json","yarn.lock","pnpm-lock.yaml","npm-shrinkwrap.json","requirements.txt","Pipfile","Pipfile.lock","pyproject.toml","setup.py","setup.cfg","Gemfile","Gemfile.lock","uv.lock","CODEOWNERS","DESIGN.md","README.md","CONTRIBUTING.md","CHANGELOG.md","SECURITY.md","CODE_OF_CONDUCT.md","AGENTS.md","CLAUDE.md","GEMINI.md"],"protected_files_policy":"request_review","title_prefix":"[ci-fix] "},"create_report_incomplete_issue":{},"missing_data":{},"missing_tool":{},"noop":{"max":1,"report-as-issue":"true"},"push_to_pull_request_branch":{"allowed_files":["src/Core/**","src/Controls/**","src/Essentials/**","src/BlazorWebView/**","src/TestUtils/**","src/Templates/**","**/PublicAPI.Unshipped.txt"],"if_no_changes":"warn","max":3,"max_patch_size":4096,"protect_top_level_dot_folders":true,"protected_files":["package.json","bun.lockb","bunfig.toml","deno.json","deno.jsonc","deno.lock","global.json","NuGet.Config","Directory.Packages.props","mix.exs","mix.lock","go.mod","go.sum","stack.yaml","stack.yaml.lock","pom.xml","build.gradle","build.gradle.kts","settings.gradle","settings.gradle.kts","gradle.properties","package-lock.json","yarn.lock","pnpm-lock.yaml","npm-shrinkwrap.json","requirements.txt","Pipfile","Pipfile.lock","pyproject.toml","setup.py","setup.cfg","Gemfile","Gemfile.lock","uv.lock","CODEOWNERS","DESIGN.md","README.md","CONTRIBUTING.md","CHANGELOG.md","SECURITY.md","CODE_OF_CONDUCT.md","AGENTS.md","CLAUDE.md","GEMINI.md"],"required_labels":["agentic-workflows"],"target":"*","title_prefix":"[ci-fix] "},"report_incomplete":{},"update_pull_request":{"allow_body":true,"allow_title":false,"max":3,"target":"*","update_branch":false}} - GH_AW_SAFE_OUTPUTS_CONFIG_fa2a69fad5fd0485_EOF + cat > "${RUNNER_TEMP}/gh-aw/safeoutputs/config.json" << 'GH_AW_SAFE_OUTPUTS_CONFIG_0644bf1434ca0071_EOF' + {"add_comment":{"discussions":false,"max":6,"required_labels":["agentic-workflows"],"required_title_prefix":"[ci-fix] ","target":"*"},"add_labels":{"allowed":["p/0"],"max":3,"required_labels":["agentic-workflows"],"required_title_prefix":"[ci-fix] ","target":"*"},"create_pull_request":{"allowed_base_branches":["main"],"allowed_branches":["ci-fix/**"],"allowed_files":["src/Core/**","src/Controls/**","src/Essentials/**","src/BlazorWebView/**","src/TestUtils/**","src/Templates/**","**/PublicAPI.Unshipped.txt"],"base_branch":"main","draft":true,"labels":["agentic-workflows"],"max":3,"max_patch_files":100,"max_patch_size":4096,"protect_top_level_dot_folders":true,"protected_files":["package.json","bun.lockb","bunfig.toml","deno.json","deno.jsonc","deno.lock","global.json","NuGet.Config","Directory.Packages.props","mix.exs","mix.lock","go.mod","go.sum","stack.yaml","stack.yaml.lock","pom.xml","build.gradle","build.gradle.kts","settings.gradle","settings.gradle.kts","gradle.properties","package-lock.json","yarn.lock","pnpm-lock.yaml","npm-shrinkwrap.json","requirements.txt","Pipfile","Pipfile.lock","pyproject.toml","setup.py","setup.cfg","Gemfile","Gemfile.lock","uv.lock","CODEOWNERS","DESIGN.md","README.md","CONTRIBUTING.md","CHANGELOG.md","SECURITY.md","CODE_OF_CONDUCT.md","AGENTS.md","CLAUDE.md","GEMINI.md"],"protected_files_policy":"request_review","title_prefix":"[ci-fix] "},"create_report_incomplete_issue":{},"mark_pull_request_as_ready_for_review":{"max":3,"required_labels":["agentic-workflows"],"required_title_prefix":"[ci-fix] ","target":"*"},"missing_data":{},"missing_tool":{},"noop":{"max":1,"report-as-issue":"true"},"push_to_pull_request_branch":{"allowed_files":["src/Core/**","src/Controls/**","src/Essentials/**","src/BlazorWebView/**","src/TestUtils/**","src/Templates/**","**/PublicAPI.Unshipped.txt"],"if_no_changes":"warn","max":3,"max_patch_size":4096,"protect_top_level_dot_folders":true,"protected_files":["package.json","bun.lockb","bunfig.toml","deno.json","deno.jsonc","deno.lock","global.json","NuGet.Config","Directory.Packages.props","mix.exs","mix.lock","go.mod","go.sum","stack.yaml","stack.yaml.lock","pom.xml","build.gradle","build.gradle.kts","settings.gradle","settings.gradle.kts","gradle.properties","package-lock.json","yarn.lock","pnpm-lock.yaml","npm-shrinkwrap.json","requirements.txt","Pipfile","Pipfile.lock","pyproject.toml","setup.py","setup.cfg","Gemfile","Gemfile.lock","uv.lock","CODEOWNERS","DESIGN.md","README.md","CONTRIBUTING.md","CHANGELOG.md","SECURITY.md","CODE_OF_CONDUCT.md","AGENTS.md","CLAUDE.md","GEMINI.md"],"required_labels":["agentic-workflows"],"target":"*","title_prefix":"[ci-fix] "},"report_incomplete":{},"update_pull_request":{"allow_body":true,"allow_title":false,"max":3,"target":"*","update_branch":false}} + GH_AW_SAFE_OUTPUTS_CONFIG_0644bf1434ca0071_EOF - name: Generate Safe Outputs Tools env: GH_AW_TOOLS_META_JSON: | { "description_suffixes": { - "add_comment": " CONSTRAINTS: Maximum 3 comment(s) can be added. Target: *. Supports reply_to_id for discussion threading.", + "add_comment": " CONSTRAINTS: Maximum 6 comment(s) can be added. Target: *. Supports reply_to_id for discussion threading.", + "add_labels": " CONSTRAINTS: Maximum 3 label(s) can be added. Only these labels are allowed: [\"p/0\"]. Target: *.", "create_pull_request": " CONSTRAINTS: Maximum 3 pull request(s) can be created. Title will be prefixed with \"[ci-fix] \". Labels [\"agentic-workflows\"] will be automatically added. Only these labels are allowed: [\"agentic-workflows\"]. PRs will be created as drafts.", + "mark_pull_request_as_ready_for_review": " CONSTRAINTS: Maximum 3 pull request(s) can be marked as ready for review.", "push_to_pull_request_branch": " CONSTRAINTS: Maximum 3 push(es) can be made. The target pull request title must start with \"[ci-fix] \".", "update_pull_request": " CONSTRAINTS: Maximum 3 pull request(s) can be updated. Target: *." }, @@ -594,6 +598,25 @@ jobs: } } }, + "add_labels": { + "defaultMax": 5, + "fields": { + "item_number": { + "issueNumberOrTemporaryId": true + }, + "labels": { + "required": true, + "type": "array", + "itemType": "string", + "itemSanitize": true, + "itemMaxLength": 128 + }, + "repo": { + "type": "string", + "maxLength": 256 + } + } + }, "create_pull_request": { "defaultMax": 1, "fields": { @@ -635,6 +658,24 @@ jobs: } } }, + "mark_pull_request_as_ready_for_review": { + "defaultMax": 1, + "fields": { + "pull_request_number": { + "issueOrPRNumber": true + }, + "reason": { + "required": true, + "type": "string", + "sanitize": true, + "maxLength": 65000 + }, + "repo": { + "type": "string", + "maxLength": 256 + } + } + }, "missing_data": { "defaultMax": 20, "fields": { @@ -1494,7 +1535,7 @@ jobs: uses: actions/github-script@3a2844b7e9c422d3c10d287c895573f7108da1b3 # v9.0.0 env: WORKFLOW_NAME: "CI Failure Fixer (main)" - WORKFLOW_DESCRIPTION: "Periodic pass over open ci-scan tracking issues filed by the main-branch CI\nfailure scanner (.github/workflows/ci-status-main.md). This workflow targets\nthe `main` branch EXCLUSIVELY: it processes only issues labelled ci-scan and\nopens every PR against main. (The net11.0 branch is handled by the parallel\n.github/workflows/ci-status-fix-net11.md — the two are split because gh-aw can\nonly transport a fix relative to ONE static base branch per workflow, and the\nmain↔net11.0 divergence exceeds gh-aw's 10 MB transport-patch cap.) The fixer\nopens ONE draft [ci-fix] PR per actionable issue against main, then WATCHES\nthat PR's own CI on later runs: when the fix's CI comes back red and the red is\ncaused by the fix itself, it pushes a fresh follow-up fix onto the SAME PR\nbranch (never a second PR) — up to 10 attempts — then stops and defers to\nhumans (the open tracking issue is the hand-off surface; a dedicated\n[ci-fix][needs-human] PR is planned but currently deferred — see Step 6). CI on\nthe PR is kicked by a human `/azp run` for now; the loop watches, classifies,\nand re-fixes autonomously between kicks.\nNever mutes tests, but\nde-flakes genuinely flaky ones (deterministic synchronization, no retries /\ntimeout bumps). Always skips visual-regression / screenshot issues." + WORKFLOW_DESCRIPTION: "Periodic pass over open ci-scan tracking issues filed by the main-branch CI\nfailure scanner (.github/workflows/ci-status-main.md). This workflow targets\nthe `main` branch EXCLUSIVELY: it processes only issues labelled ci-scan and\nopens every PR against main. (The net11.0 branch is handled by the parallel\n.github/workflows/ci-status-fix-net11.md — the two are split because gh-aw can\nonly transport a fix relative to ONE static base branch per workflow, and the\nmain↔net11.0 divergence exceeds gh-aw's 10 MB transport-patch cap.) The fixer\nopens ONE draft [ci-fix] PR per actionable issue against main, then WATCHES\nthat PR's own CI on later runs: when the fix's CI comes back red and the red is\ncaused by the fix itself, it pushes a fresh follow-up fix onto the SAME PR\nbranch (never a second PR) — up to 10 attempts — then stops and defers to\nhumans (the open tracking issue is the hand-off surface; a dedicated\n[ci-fix][needs-human] PR is planned but currently deferred — see Step 6). CI on\nthe PR is kicked by a human `/azp run` for now; the loop watches, classifies,\nand re-fixes autonomously between kicks. When the SPECIFIC test a PR fixed is\nconfirmed green in that PR's own CI, the loop marks the draft PR ready for review\n(a state transition only — it never approves or merges).\nNever mutes tests, but\nde-flakes genuinely flaky ones (deterministic synchronization, no retries /\ntimeout bumps). Always skips visual-regression / screenshot issues." HAS_PATCH: ${{ needs.agent.outputs.has_patch }} with: script: | @@ -1910,7 +1951,7 @@ jobs: GH_AW_ALLOWED_DOMAINS: "*.blob.core.windows.net,*.githubusercontent.com,api.business.githubcopilot.com,api.enterprise.githubcopilot.com,api.github.com,api.githubcopilot.com,api.individual.githubcopilot.com,api.snapcraft.io,archive.ubuntu.com,azure.archive.ubuntu.com,codeload.github.com,crl.geotrust.com,crl.globalsign.com,crl.identrust.com,crl.sectigo.com,crl.thawte.com,crl.usertrust.com,crl.verisign.com,crl3.digicert.com,crl4.digicert.com,crls.ssl.com,dev.azure.com,docs.github.com,github-cloud.githubusercontent.com,github-cloud.s3.amazonaws.com,github.blog,github.com,github.githubassets.com,helix.dot.net,host.docker.internal,json-schema.org,json.schemastore.org,keyserver.ubuntu.com,lfs.github.com,objects.githubusercontent.com,ocsp.digicert.com,ocsp.geotrust.com,ocsp.globalsign.com,ocsp.identrust.com,ocsp.sectigo.com,ocsp.ssl.com,ocsp.thawte.com,ocsp.usertrust.com,ocsp.verisign.com,packagecloud.io,packages.cloud.google.com,packages.microsoft.com,patch-diff.githubusercontent.com,ppa.launchpad.net,raw.githubusercontent.com,registry.npmjs.org,s.symcb.com,s.symcd.com,security.ubuntu.com,telemetry.enterprise.githubcopilot.com,ts-crl.ws.symantec.com,ts-ocsp.ws.symantec.com,www.googleapis.com" GITHUB_SERVER_URL: ${{ github.server_url }} GITHUB_API_URL: ${{ github.api_url }} - GH_AW_SAFE_OUTPUTS_HANDLER_CONFIG: "{\"add_comment\":{\"discussions\":false,\"max\":3,\"required_labels\":[\"agentic-workflows\"],\"required_title_prefix\":\"[ci-fix] \",\"target\":\"*\"},\"create_pull_request\":{\"allowed_base_branches\":[\"main\"],\"allowed_branches\":[\"ci-fix/**\"],\"allowed_files\":[\"src/Core/**\",\"src/Controls/**\",\"src/Essentials/**\",\"src/BlazorWebView/**\",\"src/TestUtils/**\",\"src/Templates/**\",\"**/PublicAPI.Unshipped.txt\"],\"base_branch\":\"main\",\"draft\":true,\"labels\":[\"agentic-workflows\"],\"max\":3,\"max_patch_files\":100,\"max_patch_size\":4096,\"protect_top_level_dot_folders\":true,\"protected_files\":[\"package.json\",\"bun.lockb\",\"bunfig.toml\",\"deno.json\",\"deno.jsonc\",\"deno.lock\",\"global.json\",\"NuGet.Config\",\"Directory.Packages.props\",\"mix.exs\",\"mix.lock\",\"go.mod\",\"go.sum\",\"stack.yaml\",\"stack.yaml.lock\",\"pom.xml\",\"build.gradle\",\"build.gradle.kts\",\"settings.gradle\",\"settings.gradle.kts\",\"gradle.properties\",\"package-lock.json\",\"yarn.lock\",\"pnpm-lock.yaml\",\"npm-shrinkwrap.json\",\"requirements.txt\",\"Pipfile\",\"Pipfile.lock\",\"pyproject.toml\",\"setup.py\",\"setup.cfg\",\"Gemfile\",\"Gemfile.lock\",\"uv.lock\",\"CODEOWNERS\",\"DESIGN.md\",\"README.md\",\"CONTRIBUTING.md\",\"CHANGELOG.md\",\"SECURITY.md\",\"CODE_OF_CONDUCT.md\",\"AGENTS.md\",\"CLAUDE.md\",\"GEMINI.md\"],\"protected_files_policy\":\"request_review\",\"title_prefix\":\"[ci-fix] \"},\"create_report_incomplete_issue\":{},\"missing_data\":{},\"missing_tool\":{},\"noop\":{\"max\":1,\"report-as-issue\":\"true\"},\"push_to_pull_request_branch\":{\"allowed_files\":[\"src/Core/**\",\"src/Controls/**\",\"src/Essentials/**\",\"src/BlazorWebView/**\",\"src/TestUtils/**\",\"src/Templates/**\",\"**/PublicAPI.Unshipped.txt\"],\"if_no_changes\":\"warn\",\"max\":3,\"max_patch_size\":4096,\"protect_top_level_dot_folders\":true,\"protected_files\":[\"package.json\",\"bun.lockb\",\"bunfig.toml\",\"deno.json\",\"deno.jsonc\",\"deno.lock\",\"global.json\",\"NuGet.Config\",\"Directory.Packages.props\",\"mix.exs\",\"mix.lock\",\"go.mod\",\"go.sum\",\"stack.yaml\",\"stack.yaml.lock\",\"pom.xml\",\"build.gradle\",\"build.gradle.kts\",\"settings.gradle\",\"settings.gradle.kts\",\"gradle.properties\",\"package-lock.json\",\"yarn.lock\",\"pnpm-lock.yaml\",\"npm-shrinkwrap.json\",\"requirements.txt\",\"Pipfile\",\"Pipfile.lock\",\"pyproject.toml\",\"setup.py\",\"setup.cfg\",\"Gemfile\",\"Gemfile.lock\",\"uv.lock\",\"CODEOWNERS\",\"DESIGN.md\",\"README.md\",\"CONTRIBUTING.md\",\"CHANGELOG.md\",\"SECURITY.md\",\"CODE_OF_CONDUCT.md\",\"AGENTS.md\",\"CLAUDE.md\",\"GEMINI.md\"],\"required_labels\":[\"agentic-workflows\"],\"target\":\"*\",\"title_prefix\":\"[ci-fix] \"},\"report_incomplete\":{},\"update_pull_request\":{\"allow_body\":true,\"allow_title\":false,\"max\":3,\"target\":\"*\",\"update_branch\":false}}" + GH_AW_SAFE_OUTPUTS_HANDLER_CONFIG: "{\"add_comment\":{\"discussions\":false,\"max\":6,\"required_labels\":[\"agentic-workflows\"],\"required_title_prefix\":\"[ci-fix] \",\"target\":\"*\"},\"add_labels\":{\"allowed\":[\"p/0\"],\"max\":3,\"required_labels\":[\"agentic-workflows\"],\"required_title_prefix\":\"[ci-fix] \",\"target\":\"*\"},\"create_pull_request\":{\"allowed_base_branches\":[\"main\"],\"allowed_branches\":[\"ci-fix/**\"],\"allowed_files\":[\"src/Core/**\",\"src/Controls/**\",\"src/Essentials/**\",\"src/BlazorWebView/**\",\"src/TestUtils/**\",\"src/Templates/**\",\"**/PublicAPI.Unshipped.txt\"],\"base_branch\":\"main\",\"draft\":true,\"labels\":[\"agentic-workflows\"],\"max\":3,\"max_patch_files\":100,\"max_patch_size\":4096,\"protect_top_level_dot_folders\":true,\"protected_files\":[\"package.json\",\"bun.lockb\",\"bunfig.toml\",\"deno.json\",\"deno.jsonc\",\"deno.lock\",\"global.json\",\"NuGet.Config\",\"Directory.Packages.props\",\"mix.exs\",\"mix.lock\",\"go.mod\",\"go.sum\",\"stack.yaml\",\"stack.yaml.lock\",\"pom.xml\",\"build.gradle\",\"build.gradle.kts\",\"settings.gradle\",\"settings.gradle.kts\",\"gradle.properties\",\"package-lock.json\",\"yarn.lock\",\"pnpm-lock.yaml\",\"npm-shrinkwrap.json\",\"requirements.txt\",\"Pipfile\",\"Pipfile.lock\",\"pyproject.toml\",\"setup.py\",\"setup.cfg\",\"Gemfile\",\"Gemfile.lock\",\"uv.lock\",\"CODEOWNERS\",\"DESIGN.md\",\"README.md\",\"CONTRIBUTING.md\",\"CHANGELOG.md\",\"SECURITY.md\",\"CODE_OF_CONDUCT.md\",\"AGENTS.md\",\"CLAUDE.md\",\"GEMINI.md\"],\"protected_files_policy\":\"request_review\",\"title_prefix\":\"[ci-fix] \"},\"create_report_incomplete_issue\":{},\"mark_pull_request_as_ready_for_review\":{\"max\":3,\"required_labels\":[\"agentic-workflows\"],\"required_title_prefix\":\"[ci-fix] \",\"target\":\"*\"},\"missing_data\":{},\"missing_tool\":{},\"noop\":{\"max\":1,\"report-as-issue\":\"true\"},\"push_to_pull_request_branch\":{\"allowed_files\":[\"src/Core/**\",\"src/Controls/**\",\"src/Essentials/**\",\"src/BlazorWebView/**\",\"src/TestUtils/**\",\"src/Templates/**\",\"**/PublicAPI.Unshipped.txt\"],\"if_no_changes\":\"warn\",\"max\":3,\"max_patch_size\":4096,\"protect_top_level_dot_folders\":true,\"protected_files\":[\"package.json\",\"bun.lockb\",\"bunfig.toml\",\"deno.json\",\"deno.jsonc\",\"deno.lock\",\"global.json\",\"NuGet.Config\",\"Directory.Packages.props\",\"mix.exs\",\"mix.lock\",\"go.mod\",\"go.sum\",\"stack.yaml\",\"stack.yaml.lock\",\"pom.xml\",\"build.gradle\",\"build.gradle.kts\",\"settings.gradle\",\"settings.gradle.kts\",\"gradle.properties\",\"package-lock.json\",\"yarn.lock\",\"pnpm-lock.yaml\",\"npm-shrinkwrap.json\",\"requirements.txt\",\"Pipfile\",\"Pipfile.lock\",\"pyproject.toml\",\"setup.py\",\"setup.cfg\",\"Gemfile\",\"Gemfile.lock\",\"uv.lock\",\"CODEOWNERS\",\"DESIGN.md\",\"README.md\",\"CONTRIBUTING.md\",\"CHANGELOG.md\",\"SECURITY.md\",\"CODE_OF_CONDUCT.md\",\"AGENTS.md\",\"CLAUDE.md\",\"GEMINI.md\"],\"required_labels\":[\"agentic-workflows\"],\"target\":\"*\",\"title_prefix\":\"[ci-fix] \"},\"report_incomplete\":{},\"update_pull_request\":{\"allow_body\":true,\"allow_title\":false,\"max\":3,\"target\":\"*\",\"update_branch\":false}}" GH_AW_CI_TRIGGER_TOKEN: ${{ secrets.GH_AW_CI_TRIGGER_TOKEN }} with: github-token: ${{ secrets.GH_AW_GITHUB_TOKEN || secrets.GITHUB_TOKEN }} diff --git a/.github/workflows/ci-status-fix.md b/.github/workflows/ci-status-fix.md index d3227e25170c..9c2f46054029 100644 --- a/.github/workflows/ci-status-fix.md +++ b/.github/workflows/ci-status-fix.md @@ -15,7 +15,9 @@ description: | humans (the open tracking issue is the hand-off surface; a dedicated [ci-fix][needs-human] PR is planned but currently deferred — see Step 6). CI on the PR is kicked by a human `/azp run` for now; the loop watches, classifies, - and re-fixes autonomously between kicks. + and re-fixes autonomously between kicks. When the SPECIFIC test a PR fixed is + confirmed green in that PR's own CI, the loop marks the draft PR ready for review + (a state transition only — it never approves or merges). Never mutes tests, but de-flakes genuinely flaky ones (deterministic synchronization, no retries / timeout bumps). Always skips visual-regression / screenshot issues. @@ -239,8 +241,15 @@ safe-outputs: # PER-RUN total (not per-PR): a sweep may surface several green PRs and/or # annotate several flaky reds in one run. At max:1 all but the first comment # were silently dropped, starving the workflow's #1 value (surfacing green PRs - # for review). Raised to 3 to match the per-run throughput of the other outputs. - max: 3 + # for review). Sized to 6 (not 3) because a single green DRAFT PR that gets + # flipped ready in one sweep spends TWO comment slots — the Step 3 ✅ surface + # comment (which still names the /azp-gated legs a human must kick, so it is NOT + # redundant with 🎯) AND the Step 3.6 T3 🎯 readiness comment. At max:3 the shared + # bucket drained after ~1 draft flip and T3's atomicity pre-check then deferred + # every further mark-ready even while the mark-ready/add_labels buckets (also 3) + # sat idle; 6 lets ~3 draft flips (2 comments each) land per sweep, so the + # mark-ready:3 / add_labels:3 caps are actually reachable. + max: 6 target: "*" # Hard constraint (defense-in-depth): only comment on THIS workflow's own # ci-fix PRs — [ci-fix] title prefix AND agentic-workflows label. Mirrors the @@ -261,13 +270,51 @@ safe-outputs: # never the title, so disable title rewrites — this compiles to allow_title:false # and removes any ability to retitle an arbitrary PR. title: false - # NOTE: gh-aw v0.79.8 does NOT emit required-title-prefix/required-labels into + # NOTE: gh-aw v0.80.9 (the pinned compiler) does NOT emit required-title-prefix/required-labels into # the compiled config for update-pull-request (verified against the lock — it # silently drops them, unlike add-comment / push-to-pull-request-branch which # honor them). So which-PR scoping here relies on prompt Hard-Rule 6 + # min-integrity:approved; the residual is body-marker edits only (no code, no # merge, no close), capped at max:3. Revisit required-* if a future gh-aw # compiler emits them for this output. + mark-pull-request-as-ready-for-review: + # Flip a validated draft [ci-fix] PR to ready-for-review once the SPECIFIC test + # this PR was opened to fix is confirmed green in the PR's OWN CI (Step 3.6) — + # even when unrelated legs are red. The workflow is schedule/dispatch-triggered + # (no triggering-PR context), so target "*" lets the agent name the PR number it + # validated. This is a state transition ONLY: it never approves and never merges — + # a human still reviews and merges. + # PER-RUN total (not per-PR): a sweep may validate several PRs' target tests green. + max: 3 + target: "*" + # Hard constraint (defense-in-depth): only ever un-draft THIS workflow's own + # ci-fix PRs — [ci-fix] title prefix AND agentic-workflows label — so a confused + # or prompt-injected agent cannot mark an arbitrary PR ready. (If the v0.80.9 + # compiler silently drops these — as documented for update-pull-request above — + # the Step 3.6 preconditions + min-integrity:approved are the compensating scope + # controls; verify against the lock after compiling.) + required-title-prefix: "[ci-fix] " + required-labels: [agentic-workflows] + add-labels: + # Apply the p/0 priority label to a [ci-fix] PR at the exact moment the loop flips it + # from draft to ready-for-review (Step 3.6 T3) — so a validated, review-ready fix lands + # in the team's p/0 triage queue instead of sitting unseen in the draft backlog. Paired + # 1:1 with mark-pull-request-as-ready-for-review; target "*" lets the agent name the PR + # it just validated. + # allowed = HARD allowlist: the agent may ONLY ever add p/0, nothing else. This caps the + # blast radius of a confused/prompt-injected agent to exactly one benign priority label — + # it can never apply a downstream-triggering or destructive label. + allowed: [p/0] + # PER-RUN total (not per-PR): a sweep may mark several PRs ready in one run; each adds + # one label, so this matches the mark-ready per-run cap (3). + max: 3 + target: "*" + # Hard constraint (defense-in-depth): only ever label THIS workflow's own ci-fix PRs — + # [ci-fix] title prefix AND agentic-workflows label. (If the v0.80.9 compiler drops these + # — as documented for update-pull-request above — the Step 3.6 preconditions + + # min-integrity:approved + the allowed:[p/0] allowlist are the compensating controls.) + required-title-prefix: "[ci-fix] " + required-labels: [agentic-workflows] timeout-minutes: 90 @@ -339,33 +386,48 @@ through `safe-outputs`. 5. **One issue = one outcome per run.** Exactly one of: respond to a maintainer's change-request on the open PR (Track C, Step 3.5.R); open the first fix/help/ de-flake PR; advance an existing PR by one attempt (push a follow-up fix); - surface a validated-green PR for review; annotate an unrelated-flake red; wait + surface a validated-green PR for review; mark a target-validated draft PR + ready-for-review (Step 3.6 T3 — the terminal outcome that supersedes the same + run's surface/annotate precursor line), or record its `already-ready` + steady-state no-op; annotate an unrelated-flake red; wait (CI not yet settled); or a recorded skip (the dedicated needs-human PR is deferred — the attempt cap records a skip instead; see Step 6). Always prefer advancing or opening a PR over a skip when a non-mute diff is producible. 6. **All writes via `safe-outputs`.** Allowed outputs: `create_pull_request` (first attempt only), `push_to_pull_request_branch` (advance an existing PR by one attempt), `update_pull_request` (bump the attempt marker / refresh the - prior-attempts table), and `add_comment` (progress notes on the `[ci-fix]` PR - ONLY). NEVER comment on the tracking issue (issues are locked by + prior-attempts table), `add_comment` (progress notes on the `[ci-fix]` PR + ONLY), `mark_pull_request_as_ready_for_review` (flip a target-validated draft + `[ci-fix]` PR from draft to ready — Step 3.6 T3 only), and `add_labels` (add + ONLY the `p/0` label, ONLY on that same draft→ready transition — Step 3.6 T3). + NEVER comment on the tracking issue (issues are locked by `.github/workflows/ci-scan-lock-issues.yml`) — `add_comment` targets the PR. No `gh pr create`, no manual `git push`. **Defense-in-depth:** `add_comment` and `update_pull_request` use `target: "*"` (the agent supplies the PR number). - `add_comment` is now config-locked to `required-title-prefix: "[ci-fix] "` + + `add_comment`, `mark_pull_request_as_ready_for_review`, and `add_labels` are + config-locked to `required-title-prefix: "[ci-fix] "` + `required-labels: [agentic-workflows]` (hard-enforced by the handler, same as - `push_to_pull_request_branch`), so a comment can only ever land on THIS - workflow's own PRs. `update_pull_request` CANNOT be config-locked to a - title/label in gh-aw v0.79.8 (the compiler silently drops `required-*` for that - output — verified against the lock), so it keeps `title: false` (no retitles; - body-marker edits only) plus this prompt-level guard. Before emitting either, - VERIFY the target PR carries BOTH the `[ci-fix]` title prefix AND the - `agentic-workflows` label; never comment on or edit the body of any PR that - lacks both. + `push_to_pull_request_branch`), and `add_labels` additionally has an + `allowed: [p/0]` allowlist so it can ONLY ever add `p/0` — so these can only + ever land on THIS workflow's own PRs. `update_pull_request` CANNOT be + config-locked to a title/label in gh-aw v0.80.9 (the compiler silently drops + `required-*` for that output — verified against the lock), so it keeps + `title: false` (no retitles; body-marker edits only) plus this prompt-level + guard. Before emitting ANY of these, VERIFY the target PR carries BOTH the + `[ci-fix]` title prefix AND the `agentic-workflows` label; never comment on, + edit, un-draft, or label any PR that lacks both. 7. **Per-run safe-output caps (per RUN, not per PR).** Each output type is capped per run: `create_pull_request` 3, `push_to_pull_request_branch` 3, - `add_comment` 3, `update_pull_request` 3. Note an ADVANCE spends one push **and** - one update **and** one comment, so ≤ 3 advances/run; surfacing a green or - annotating a flake spends one comment. When a bucket is exhausted, do NOT keep + `add_comment` 6, `update_pull_request` 3, `mark_pull_request_as_ready_for_review` + 3, `add_labels` 3 (counts LABELS, not calls). Note an ADVANCE spends one push + **and** one update **and** one comment, so ≤ 3 advances/run; surfacing a green or + annotating a flake spends one comment; a **mark-ready (Step 3.6 T3)** of a + still-draft PR spends TWO comments (the Step 3 ✅ surface **and** the T3 🎯 + readiness note) **and** one mark-ready **and** one label as an ALL-OR-NOTHING set + (T3 pre-checks those buckets and defers the whole PR if any is exhausted — never + mark a PR ready without its 🎯 audit comment). `add_comment` is therefore sized 6 + (not 3) so ~3 draft flips, at 2 comments each, can land in one sweep instead of the + shared comment bucket starving the otherwise-idle mark-ready/add_labels buckets. When a bucket is exhausted, do NOT keep emitting (extras are silently dropped) — record `skipped: per-run cap reached; deferring PR # to next cycle` for each remaining PR so the drop is a deliberate, logged decision. The continuous loop picks the deferred PRs up next @@ -409,8 +471,9 @@ For every open tracking issue in scope, converge on exactly one outcome: | Open first help-wanted draft `[ci-fix]` PR | No open PR yet; a plausible candidate exists but cannot be runner-validated (device/UI tests) (attempt 1) | | Open first de-flake draft `[ci-fix]` PR | No open PR yet; failure is intermittent (green-on-retry) from a genuine test-quality defect; a deterministic-synchronization fix is producible (attempt 1, Step 4.7 bucket b) | | Advance an existing PR (push attempt N+1) | Open PR's own CI settled red *because of the fix itself*, no human engaged, marker < 10 — push a NEW distinct fix onto the same branch (Step 3.5 → 5.6) | -| Surface a validated-green PR | Open PR's own CI settled green — comment "validated, attempt N/10; ready for review" and do NOT advance (Step 3.5) | -| Annotate an unrelated-flake red | Open PR is red only on baseline-flake / unrelated legs — comment which leg needs a re-run; do NOT burn an attempt (Step 3.5) | +| Surface a validated-green PR | Open PR's own CI settled green — comment "validated, attempt N/10; ready for review" and do NOT advance (Step 3.5), then mark the draft PR ready for review once the fixed test is confirmed green (Step 3.6) | +| Annotate an unrelated-flake red | Open PR is red only on baseline-flake / unrelated legs — comment which leg needs a re-run; do NOT burn an attempt (Step 3.5); if the SPECIFIC fixed test is confirmed green on that build, still mark the draft PR ready for review (Step 3.6) | +| Mark a validated PR ready for review | The specific test this PR fixed is confirmed green in the PR's own CI (even if unrelated legs are red) — comment the target-test result and transition the draft PR to ready for review; never approve or merge (Step 3.6) | | Wait | Open PR's CI is pending / not yet settled on the current head SHA — do nothing this cycle (Step 3.5) | | Hand-off skip (attempt cap) | Marker == 10 and the signature still reproduces — stop and defer to humans; the dedicated `[ci-fix][needs-human]` PR is planned but currently deferred (Step 6) | | Recorded skip | Visual-regression, human engaged, already-handled, fixed-in-latest-build, infra-flake, out-of-bounds, only-mute-available, or no novel approach producible | @@ -685,14 +748,34 @@ Run these gates in order — the FIRST that fires decides this cycle's outcome: 1. **Human engaged.** If `C.humanEngaged` → `skipped: human engaged on PR #; deferring` and stop. Never fight a human reviewer — the intentional hand-off boundary still holds the moment a person touches the PR. -2. **CI not settled / unknown / incomplete prefetch.** If `C.checksSettled == false` - OR `C.overallConclusion` is `pending`, `neutral`, or `unknown`, OR - `C.dataComplete == false` → the fix's CI has not finished on the current head SHA - (`C.headSha`), or the prefetch could not fully read this PR (a partial read can - understate `humanEngaged` / `attempt`). `skipped: PR # CI pending / prefetch - incomplete on ; waiting` and stop. *(Round 1: this is where a - maintainer `/azp run` is awaited — the `/azp`-gated uitests/devicetests legs will - not have run until a human kicks them.)* +2. **CI not settled — but validate the target test first (target-focused readiness).** + If `C.checksSettled == false` OR `C.overallConclusion` is `pending`, `neutral`, or + `unknown`, OR `C.dataComplete == false` → the *overall* build has not finished on the + current head SHA (`C.headSha`), or the prefetch could not fully read this PR (a partial + read can understate `humanEngaged` / `attempt`). Unrelated legs still draining must NOT + keep an already-proven fix parked as a draft, so before waiting, try the target-focused + fast-path: + - **2a — Target-green fast-path (draft `[ci-fix]` PRs only).** If `C.dataComplete == + true` AND the Step 3.6 preconditions hold (draft PR, `[ci-fix] ` title prefix AND the + `agentic-workflows` label), run Step 3.6 **T1–T2** against `C.headSha` now: identify + the target test(s) and read the PR's OWN build **timeline / per-leg status** (anonymous + `_apis/build` — never `_apis/test`, per Hard-Rule 8). If EVERY target test is + **VALIDATED-GREEN** (per T2's all-platform definition below — the target's category leg + is `succeeded` on every platform that runs it, failed on none, and NO target platform + family the pipeline covers is still pending/unconcluded, per T2's "no platform left + unverified" rule; a platform that simply has no leg for the target's category — the test + doesn't run there — is fine, not a gap) AND every leg in + `C.failedLegs` that has ALREADY concluded classifies as **unrelated flake** (Step 4 / + Step 4.7 method — the fix is implicated in NO completed red), then the fix is proven + regardless of unrelated *pending* legs → run **Step 3.6 T3** (mark ready + 🎯 comment) + and stop. This early-out NEVER advances an attempt and NEVER acts on a red that could + be the fix's fault. + - **2b — Otherwise WAIT.** If the target test has not yet executed (pending/absent on + its leg), or a completed red is (or may be) caused by the fix, or `C.dataComplete == + false`, or this PR has no identifiable target test → `skipped: PR # CI pending / + target not yet validated on ; waiting` and stop. *(Round 1: this is where a + maintainer `/azp run` is awaited — the `/azp`-gated uitests/devicetests legs will not + have run until a human kicks them.)* 3. **Green → surface for review.** If `C.overallConclusion == "success"`: the checks that RAN are green. **Primary-gate check first:** the deterministic `success` verdict only certifies "at least one green check, nothing failing or @@ -704,18 +787,31 @@ Run these gates in order — the FIRST that fires decides this cycle's outcome: # primary CI gate (maui-pr) not green on ; waiting` and stop — do NOT surface. (The `/azp`-gated `maui-pr-uitests` (def 313) / `maui-pr-devicetests` (def 314) legs MAY still be un-run — that is expected and is named below, not a - reason to withhold the surface.) **Idempotency:** scan the PR's existing comments - for a prior bot `✅ … validated … on ` note for THIS head SHA — if one - already exists, record `already-surfaced PR # (head )` and stop - WITHOUT re-commenting (re-surfacing the same green every tick is noise and burns - the per-run comment budget other PRs need). Otherwise resolve the attempt number from - `C.effectiveAttempt` (the authoritative max(marker, bot-commit) counter; omit the - number only if it is somehow indeterminate). **Dry-run gate (Step 0):** if `dry_run == "true"`, do NOT emit any comment — instead print the intended `✅` validated-green body to the run log, tally `dry-run: would-surface-green PR #`, and stop. Otherwise `add_comment` on PR #: `✅ Attempt /10 validated - — the fix's CI is green on . Ready for human review.` - Name any `/azp`-gated legs (uitests def 313 / devicetests def 314) that have not - run and still need a maintainer `/azp run`. Do NOT advance. Record - `surfaced-green PR # (attempt /10)` and stop. *(This directly - attacks the real bottleneck — no reviews — so it is the highest-value outcome.)* + reason to withhold the surface.) **Comment idempotency + dry-run suppress the ✅ + comment ONLY — neither skips Step 3.6.** Scan the PR's existing comments for a prior + bot `✅ … validated … on ` note for THIS head SHA; if one exists, set + `SKIP_SURFACE_COMMENT = true`. Resolve the attempt number from `C.effectiveAttempt` + (the authoritative max(marker, bot-commit) counter; omit the number only if it is + somehow indeterminate). Then post the surface comment UNLESS suppressed: + - if `dry_run == "true"`: do NOT emit — print the intended `✅` validated-green body to + the run log and tally `dry-run: would-surface-green PR #`; + - else if `SKIP_SURFACE_COMMENT`: do NOT re-post — record `already-surfaced PR # + (head )` (re-surfacing the same green every tick is noise and burns the + per-run comment budget other PRs need); + - else `add_comment` on PR #: `✅ Attempt /10 validated — the fix's CI is + green on .` naming any `/azp`-gated legs (uitests def 313 / devicetests def + 314) that have not run and still need a maintainer `/azp run`; record `surfaced-green + PR # (attempt /10)`. (Do NOT assert "ready for human review" on a PR that + is still a draft — Step 3.6, not this comment, owns the draft→ready flip and posts its + own 🎯 announcement when it fires.) + Do NOT advance. Then — in ALL of the above cases — run **Step 3.6** (target-test + readiness gate) for this PR before stopping: its `/azp`-gated target legs may conclude + green on this SAME head SHA on a LATER sweep (an `/azp run` adds no commit), and Step + 3.6 is the ONLY place the draft→ready flip happens, so it MUST re-evaluate every sweep — + never `stop` here before it. *(This directly attacks the real bottleneck — no reviews — + so it is the highest-value outcome.)* 4. **Red → classify caused-by-fix vs unrelated-flake.** If `C.overallConclusion == "failure"`, analyze the PR's OWN failing build (NOT `main`): find the AzDO `maui-pr` build for this PR (filter builds by `branchName=refs/pull//merge`, @@ -723,18 +819,26 @@ Run these gates in order — the FIRST that fires decides this cycle's outcome: method and Step 4.7 flake buckets to `C.failedLegs`. - **Unrelated flake only** (every failed leg is known-flaky / infra / a pre-existing baseline red NOT introduced by the fix): do NOT burn an attempt. - **Idempotency first:** scan the PR's existing comments for a prior bot - `♻️ … unrelated flake … on ` note for THIS head SHA — if one already - exists, record `already-annotated-flake PR # (head )` and stop - WITHOUT re-commenting (re-annotating the same flake every 12h sweep is noise and - burns the per-run comment budget other PRs need). Otherwise resolve the attempt - number from `C.effectiveAttempt` (the authoritative max(marker, bot-commit) - counter), omitting the `Attempt /10:` prefix only if it is somehow - indeterminate. **Dry-run gate (Step 0):** if `dry_run == "true"`, do NOT emit any comment — instead print the intended `♻️` unrelated-flake body to the run log, tally `dry-run: would-annotate-flake PR #`, and stop. Otherwise `add_comment` on PR #: `♻️ Attempt /10: red is - unrelated flake on leg(s) () on ; the fix itself is not - implicated. A maintainer re-run (/azp run ) should clear it.` Record - `annotated-flake PR # (head )` and stop. *(Round 1: human re-runs; - Round 2: auto re-trigger.)* + **Comment idempotency + dry-run suppress the ♻️ comment ONLY — neither skips Step + 3.6.** Scan the PR's existing comments for a prior bot `♻️ … unrelated flake … on + ` note for THIS head SHA; if one exists, set `SKIP_FLAKE_COMMENT = true`. + Resolve the attempt number from `C.effectiveAttempt` (the authoritative max(marker, + bot-commit) counter), omitting the `Attempt /10:` prefix only if it is + somehow indeterminate. Then post the flake note UNLESS suppressed: + - if `dry_run == "true"`: do NOT emit — print the intended `♻️` unrelated-flake body + to the run log and tally `dry-run: would-annotate-flake PR #`; + - else if `SKIP_FLAKE_COMMENT`: do NOT re-post — record `already-annotated-flake PR + # (head )` (re-annotating the same flake every 12h sweep is noise and + burns the per-run comment budget other PRs need); + - else `add_comment` on PR #: `♻️ Attempt /10: red is unrelated flake on + leg(s) () on ; the fix itself is not implicated. A + maintainer re-run (/azp run ) should clear it.` record `annotated-flake + PR # (head )`. + Then — in ALL of the above cases — run **Step 3.6** (target-test readiness gate) for + this PR before stopping: its `/azp`-gated target legs may conclude green on this SAME + head SHA on a LATER sweep, and Step 3.6 is the ONLY place the draft→ready flip + happens, so it MUST re-evaluate every sweep — never `stop` here before it. *(Round 1: + human re-runs; Round 2: auto re-trigger.)* - **Caused by the fix** (a failed leg still matches the original target signature, or the fix introduced a NEW failure): advance an attempt. - **Attempt count.** `attempt = C.effectiveAttempt` — the authoritative @@ -752,6 +856,188 @@ Run these gates in order — the FIRST that fires decides this cycle's outcome: from every prior commit on this branch**, emitted via the Step 5.6 ADVANCE path (push onto the same PR). +#### Step 3.6 — Target-test verification & mark-ready gate + +Reached from the Step 3 **green-surface** branch, the Step 4 **unrelated-flake** branch +(AFTER that branch has posted its comment), and the Step 3.5 **gate-2 target-focused +fast-path** (2a — while unrelated legs are still pending). Purpose: confirm the SPECIFIC +test(s) this PR was opened to fix now PASS in the PR's own CI, and — only when they do +— transition the draft `[ci-fix]` PR to **ready for review** so a maintainer sees a +validated fix instead of a draft. This is the ONLY place the loop flips draft→ready; it +is a state transition, **never** an approval or a merge (a human still reviews and +merges). Overall red on *unrelated* legs must NOT gate readiness — we validate the fix, +not the base branch's flakiness. + +**Preconditions** (ALL must hold; otherwise record `skipped: readiness N/A PR # +()` and stop this gate): +- `C.isDraft == true` — if the PR is already ready-for-review, the draft→ready + transition is already done; do NOT re-mark **and do NOT re-apply `p/0`**. `p/0` is applied + exactly ONCE, atomically with the draft→ready flip (T3 — all-or-nothing with the 🎯 audit + comment + mark-ready), so an already-ready loop PR that is MISSING `p/0` has almost + certainly had it **removed by a maintainer** de-prioritizing the PR — a pure triage action + the human-engagement guard (which inspects comments/reviews/commits, NOT label changes) + cannot see. Re-adding `p/0` every sweep would fight that maintainer indefinitely, violating + the loop's "never override a human" contract. So the loop does NOT reconcile the label: + record `already-ready PR #` and stop this gate. (Trade-off: on the rare occasion a + transient API error drops `p/0` at flip time *after* the T3 atomic pre-check passed, it is + not auto-re-added — but the PR is still ready-for-review with its 🎯 audit comment, and + re-adding `p/0` is a trivial manual action; that is strictly preferable to steam-rolling a + maintainer's deliberate de-prioritization.) +- The PR is unmistakably THIS workflow's own: `[ci-fix] ` title prefix AND the + `agentic-workflows` label (mirrors the safe-output lock; the compensating scope + control if the v0.80.9 compiler drops the declarative required-*). +- You reached this gate from Step 3 (green), Step 4 (**unrelated-flake**), or the Step 3.5 + gate-2 **target-focused fast-path** (2a) — i.e. the fix is NOT implicated in any red that + has concluded. If Step 4 classified a red as **caused by the fix** (ADVANCE), do NOT run + this gate — advance the attempt instead. + +**T1 — Identify the target test(s).** From the `[ci-scan]` issue signature the fix +addresses (and the PR's own diff), extract the fully-qualified test method name(s) the +fix targets — e.g. `SafeAreaShouldWorkOnAllShellTabs`. For a de-flake it is the +de-flaked test; for a product fix it is the originally-failing test(s). If NO specific +test can be identified (e.g. a product build-break rather than a test failure), this is a +**build-only fix** — there is no single test to validate cross-platform, so validate at +**whole-build** granularity rather than stopping (Step 3 only *comments*; Step 3.6 is the +sole draft→ready flip, so a build-only fix is undrafted HERE or not at all). We only reach +this gate from Step 3 (green) / Step 4 (unrelated-flake) / gate-2a, so the fix is not +implicated in any concluded red; additionally require, via the T2 timeline method on +`C.headSha`, that the primary build pipeline `maui-pr` (def 302) has CONCLUDED with EVERY +one of its platform build legs `succeeded`/`completed` and NONE failed/canceled or still +pending — the build is green on **every** platform, not just the originally-broken one (the +cross-platform guard, applied to the build instead of a test). A build-only fix is undrafted +ONLY on the strength of the auto-running `maui-pr` (def 302) whole-build green, which the +loop CAN observe: if the PR's `[ci-scan]` issue signature or its own diff indicates the +ORIGINATING failure was in a `/azp`-gated pipeline (`maui-pr-uitests` def 313 or +`maui-pr-devicetests` def 314) rather than `maui-pr` (def 302), do NOT undraft on +`maui-pr`-green alone — those pipelines do not auto-run on this PR (GITHUB_TOKEN cannot +trigger them), so a green `maui-pr` build is NOT evidence the gated build break is fixed; +record `skipped: build-only fix PR # targets gated pipeline () not run — +deferring to human` and stop this gate. Otherwise, set `TARGET := "the maui-pr build +(build-only fix — no single target test)"` and proceed to **T3** to mark ready. If any `maui-pr` build leg is still unconcluded, record `skipped: build-only fix PR +# not yet whole-build green (leg(s) pending)` and stop this gate WITHOUT marking +ready (a green subset is not enough — a still-pending build leg could yet fail). + +**T2 — Verify the target test's platform legs are green (anonymous, leg-level).** Per-test +outcomes come from the AzDO **test-results** API (`_apis/test/...`), which is **NOT reachable +anonymously — Hard-Rule 8**: those endpoints 302-redirect to a sign-in page on this runner, so +a `curl` returns an HTML redirect (not JSON) and every target test would look "not executed". +Validate instead at **leg granularity** using ONLY the anonymous `_apis/build/...` timeline +(Hard-Rule 8) — a `succeeded` category leg means every test in that category **that actually +ran** passed on that platform. This is a strong bar **only if the target test genuinely +executes**: a job can go green while one specific test is skipped at runtime (`[Ignore]`, +`Assert.Ignore()`, `Assert.Inconclusive()`, a `Skip=`/conditional `[Fact]`, an `#if`-out, or +an early `return` before the asserts). So before trusting a green leg as proof the target +passed, **confirm from the PR's OWN diff that the fix does NOT skip, ignore, disable, or +short-circuit the target test** (it adds no `[Ignore]`/`[Explicit]`, `Assert.Ignore`/ +`Assert.Inconclusive`, `Skip=`, category-exclusion, `#if`-out, or early `return` around the +target). If the fix could cause the target to be runtime-skipped rather than genuinely pass, +leg-level green is NOT sufficient evidence — record `skipped: target test may be +runtime-skipped by this fix (diff adds ); leg-green insufficient, deferring to human +on PR #` and stop this gate WITHOUT marking ready. Otherwise (the target genuinely runs +and asserts, as a de-flake or product fix does) a green category leg cannot hide a +target-test failure, so treat it as authoritative. + +Map the target test to the CI leg(s) that run it. A leg name encodes platform + test category +(e.g. `Android UITests SafeAreaEdges,Shadow`, `iOS UITests SafeAreaEdges,Shadow`, `macOS +UITests SafeAreaEdges,Shadow`, `Windows UITests SafeAreaEdges,Shadow`). Determine the target +test's UI-test category — its `[Category(UITestCategories.X)]` in the test/HostApp source, +visible in the PR diff or the test file — to know which leg-name substring identifies its legs; +for a device test, the per-platform device-test legs. + +Using the SAME build-discovery as Step 4 (filter AzDO builds by `branchName=refs/pull//merge` +or `sourceVersion == C.headSha`), read each build's **timeline** on `C.headSha` for the +pipeline(s) that RUN the target test — `maui-pr` (def 302) for unit/integration tests, +`maui-pr-uitests` (def 313) for Appium UI tests, `maui-pr-devicetests` (def 314) for device +tests: + +```bash +ORG=dnceng-public; PROJ=public; BUILD= +# Anonymous + Hard-Rule-8-compliant. NEVER call _apis/test/... (it 302-redirects to sign-in). +# Each type=="Job" record is one platform leg: result (succeeded/failed/canceled/null), +# state (completed/inProgress/pending), name (encodes " UITests "): +curl -s "https://dev.azure.com/$ORG/$PROJ/_apis/build/builds/$BUILD/timeline?api-version=7.1" \ + | tee /tmp/gh-aw/agent/timeline_${BUILD}.json \ + | jq -r '.records[] | select(.type=="Job") | "\(.result)\t\(.state)\t\(.name)"' +``` + +(The prefetch already exposes per-leg status in `C.failedLegs` / the checks context, and +`gh pr checks ` lists the same per-leg rows with platform+category in the name — use +whichever is handy; the build timeline is the authoritative cross-check on the exact build.) + +Treat a target test as **VALIDATED-GREEN** only if, on `C.headSha`, the leg that runs its +category is green on **every platform it runs on** — a fix that repairs one platform must not +silently regress the same test's leg on another, so a green leg on the originally-red platform +alone is NOT enough. Across the target pipeline's platform legs (for a UI test: the Android, +iOS, Windows, and macOS/MacCatalyst legs running the target's category; for a device test: each +device-test platform), require ALL of: +- the target's category leg is `result == "succeeded"` **and** `state == "completed"` on + **every** platform that runs it, AND +- that leg is `failed` / `canceled` / aborted on **NO** platform, AND +- **no platform is left unverified.** For each platform family the target pipeline covers, that + platform's category leg must have CONCLUDED (`state == "completed"`) on `C.headSha`. If a + platform simply has no leg for the target's category, the test does not run there — that is + fine, not a gap. But if a platform's category leg has **not concluded** (`state` is + `inProgress` / pending, or an `/azp`-gated `maui-pr-uitests` / `maui-pr-devicetests` leg that + has not been kicked), the test's status on that platform is UNKNOWN → the fix is NOT yet + validated across platforms: record `skipped: target test green on but + not yet verified on (leg(s) pending / need /azp run) on PR #` + and stop this gate WITHOUT marking ready. + +A target test whose category leg never concluded on ANY platform (**not executed** anywhere — +e.g. its `/azp`-gated pipeline has not been kicked) is likewise NOT validated: record `skipped: +target test not yet executed on PR # ( not run — needs /azp run)` and stop +this gate WITHOUT marking ready. Do NOT overclaim — a green *sibling* leg (a different category +on the same platform) is not the target's leg, and a green leg on one platform is not a pass on +the others. + +**T3 — Mark ready + report.** If EVERY target test is VALIDATED-GREEN. The 🎯 comment, +the mark-ready, and the `p/0` label are THREE SEPARATE safe-outputs — the comment +existing does NOT prove the mark-ready took effect, so they are tracked independently: + +- **Comment idempotency (dup-suppress only — never gates the mark-ready).** We only reach + T3 while `C.isDraft == true` (the precondition stops an already-ready PR before here). So + if a prior bot comment for THIS head that carries the leading `🎯` anchor AND the + contiguous phrase `validated green on ` already exists (BOTH T3 variants — + `🎯 Target test validated green on ` and the build-only `🎯 Build validated green on + ` — contain that exact contiguous phrase, so this recognizes both; keying on + `target test` alone would instead miss the build-only comment and re-post it every sweep. + **Require the `🎯` anchor** so the match can NEVER be loosened to a gapped + `validated…green…on` form that would false-positive on the Step 3 `✅ … validated — the + fix's CI is green on ` surface comment and wrongly suppress the audit 🎯), the comment + landed on an earlier sweep but the **mark-ready did NOT take + effect** (the PR is still a draft) — set `SUPPRESS_COMMENT = true` (do not re-post the + duplicate 🎯 comment) but STILL complete the mark-ready + label below. Never treat the + comment's existence as "already marked ready" while the PR is still a draft. Record this + case as `re-marking PR # (🎯 present; mark-ready did not take on a prior sweep)`. +- **Atomicity budget pre-check (Hard-Rule 7).** Determine the outputs this gate will emit: + `mark_pull_request_as_ready_for_review` + `add_labels` always, plus `add_comment` unless + `SUPPRESS_COMMENT`. Verify a free per-run slot remains in EACH bucket you are about to + use. If ANY required bucket is exhausted, emit NONE of them — record `skipped: per-run + cap reached; deferring mark-ready PR # to next cycle` and stop this gate. Never mark a + PR ready (or label it) unless its 🎯 audit comment is GUARANTEED to exist for this exact + head SHA — either posted in this same sweep, or (in the `SUPPRESS_COMMENT` re-marking case) + already present from a prior sweep. The outputs you DO emit either all land or all defer + together; the audit trail must never be absent, but it is NOT re-posted when it already exists. +- **Dry-run gate (Step 0):** if `dry_run == "true"`, emit NOTHING — print the intended + readiness comment and "would mark ready + add p/0" to the run log, tally `dry-run: + would-mark-ready PR #`, and stop. +- Otherwise emit for THIS PR number ``: + 1. `add_comment` (ONLY if not `SUPPRESS_COMMENT`): `🎯 Target test validated green on + — passed on ALL platforms it runs on (, + buildId ). — not + caused by this fix."> Transitioning this PR from draft to ready for review and adding + `p/0`; a maintainer still reviews and merges.` (For a **build-only fix** — the T1 + whole-build fallback, no single target test — phrase the first clause as `🎯 Build + validated green on — the maui-pr build passed on ALL platforms (buildId + ).` instead of naming a test.) + 2. `mark_pull_request_as_ready_for_review` with `reason:` a one-line justification + naming the validated test(s) and ``. + 3. `add_labels` with `labels: ["p/0"]` for PR # — put the now-review-ready fix into + the team's p/0 priority queue so it is triaged, not lost in the draft backlog. (If + the PR somehow already carries `p/0`, this is a harmless no-op.) +- Record `marked-ready PR # (target green on ALL platforms on , + labeled p/0)` and stop. + #### Step 3.5.R — Maintainer change-request response (Track C) Reached from Step 3.5 gate 0. Goal: when an **eligible human reviewer** (Hard-Rule @@ -1383,7 +1669,19 @@ Filed by [`ci-status-fix`](https://github.com/dotnet/maui/blob/main/.github/work ### Step 8 — Per-issue tally + end-of-run summary -Per issue, append one outcome line to `/tmp/gh-aw/agent/coverage.txt`: +Per issue, append **one** outcome line to `/tmp/gh-aw/agent/coverage.txt` — the +terminal outcome for the cycle. When one cycle produces a chained pair — a PR gets a +same-cycle precursor line and **then** a Step 3.6 readiness line (`marked-ready` on the +draft→ready flip, or `already-ready` on a later steady-state sweep of an already-ready +PR) — record ONLY the terminal Step 3.6 readiness line: it supersedes ANY same-cycle +non-readiness precursor for that PR, so an aggregator keying on one-line-per-issue never +double-counts. The precursor may be a Step 3 green line (`surfaced-green` on the first ✅, +or `already-surfaced` when it already exists) OR — when an unrelated-flake red and a +target-green coincide — a Step 4 `annotated-flake` line; either way the readiness line is +terminal, and the superseded precursor's signal survives in the PR's 🎯/♻️ comment so no +information is lost. This covers the flip cycle (`surfaced-green` → `marked-ready`), the +post-flip steady state (`already-surfaced` → `already-ready`), and the flake-coincident +flip (`annotated-flake` → `marked-ready`): ``` # main attempt- @@ -1394,8 +1692,14 @@ Per issue, append one outcome line to `/tmp/gh-aw/agent/coverage.txt`: `advance-PR # attempt /10` (ADVANCE mode — new commit pushed to the existing PR), `surfaced-green PR #` (fix's own CI went green; commented for review, did not advance), `annotated-flake PR #` (red was unrelated flake; -commented, attempt NOT burned), `waiting PR #` (CI not settled yet), -`dry-run: would-`, `skipped: `. (The +commented, attempt NOT burned), `marked-ready PR #` (the specific fixed test +was confirmed green in the PR's own CI; draft flipped to ready for review), +`already-surfaced PR #` (the fix's ✅ green comment already existed this sweep; +re-post suppressed — superseded by the Step 3.6 readiness line when the PR also flips +ready that cycle), `already-ready PR #` (terminal steady state — the PR was flipped +ready on a prior cycle and remains green on an unchanged head; no action taken, +supersedes `already-surfaced`), `waiting PR #` (CI not settled yet), +`dry-run: would-`, `skipped: `. (The `needs-human-PR` outcome is reserved for the deferred hand-off — Step 6 currently records a skip instead, so it is not emitted.)
attempt /10` (ADVANCE mode — new commit pushed to the existing PR), `surfaced-green PR #` (fix's own CI went green; commented for review, did not advance), `annotated-flake PR #` (red was unrelated flake; -commented, attempt NOT burned), `waiting PR #` (CI not settled yet), -`dry-run: would-`, `skipped: `. (The +commented, attempt NOT burned), `marked-ready PR #` (the specific fixed test +was confirmed green in the PR's own CI; draft flipped to ready for review), +`already-surfaced PR #` (the fix's ✅ green comment already existed this sweep; +re-post suppressed — superseded by the Step 3.6 readiness line when the PR also flips +ready that cycle), `already-ready PR #` (terminal steady state — the PR was flipped +ready on a prior cycle and remains green on an unchanged head; no action taken, +supersedes `already-surfaced`), `waiting PR #` (CI not settled yet), +`dry-run: would-`, `skipped: `. (The `needs-human-PR` outcome is reserved for the deferred hand-off — Step 6 currently records a skip instead, so it is not emitted.) diff --git a/.github/workflows/ci-status-fix.lock.yml b/.github/workflows/ci-status-fix.lock.yml index 93dc5969cb0d..65511f2b3259 100644 --- a/.github/workflows/ci-status-fix.lock.yml +++ b/.github/workflows/ci-status-fix.lock.yml @@ -1,4 +1,4 @@ -# gh-aw-metadata: {"schema_version":"v4","frontmatter_hash":"f014161967589ebcaf1ac5ec6f3c88ee5a4388d28b55a07dc36c45b7b2aebab6","body_hash":"86c6b2d4c3df13da14c6a3c8b211381fcc9f97bb1951ff7b80588a2c5338e00c","compiler_version":"v0.80.9","strict":true,"agent_id":"copilot","agent_model":"claude-opus-4.8","engine_versions":{"copilot":"1.0.63"}} +# gh-aw-metadata: {"schema_version":"v4","frontmatter_hash":"b0ebf8a2f179900cfe3638e994b27a43cec52bdbdd2331dc1bd4d65762fa2d6b","body_hash":"1825e2b1f7c7bd88241bf37580655489360f9bebc806edd00837a52ce4ad83a0","compiler_version":"v0.80.9","strict":true,"agent_id":"copilot","agent_model":"claude-opus-4.8","engine_versions":{"copilot":"1.0.63"}} # gh-aw-manifest: {"version":1,"secrets":["COPILOT_PAT_0","COPILOT_PAT_1","COPILOT_PAT_2","COPILOT_PAT_3","COPILOT_PAT_4","COPILOT_PAT_5","COPILOT_PAT_6","COPILOT_PAT_7","COPILOT_PAT_8","COPILOT_PAT_9","GH_AW_CI_TRIGGER_TOKEN","GH_AW_GITHUB_MCP_SERVER_TOKEN","GH_AW_GITHUB_TOKEN","GITHUB_TOKEN"],"actions":[{"repo":"actions/checkout","sha":"34e114876b0b11c390a56381ad16ebd13914f8d5","version":"v4"},{"repo":"actions/checkout","sha":"9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0","version":"v7.0.0"},{"repo":"actions/checkout","sha":"de0fac2e4500dabe0009e67214ff5f5447ce83dd","version":"v6.0.2"},{"repo":"actions/download-artifact","sha":"3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c","version":"v8.0.1"},{"repo":"actions/github-script","sha":"3a2844b7e9c422d3c10d287c895573f7108da1b3","version":"v9.0.0"},{"repo":"actions/setup-node","sha":"48b55a011bda9f5d6aeb4c2d9c7362e8dae4041e","version":"v6.4.0"},{"repo":"actions/upload-artifact","sha":"043fb46d1a93c77aae656e7c1c64a875d1fc6a0a","version":"v7.0.1"},{"repo":"github/gh-aw-actions/setup","sha":"8c7d04ebf1ece56cd381446125da3e0f6896294a","version":"v0.80.9"}],"containers":[{"image":"ghcr.io/github/gh-aw-firewall/agent:0.27.7","digest":"sha256:aae231e4635c8999d039c132f1602d3df850fe9b84a00aa2b5ac981179b5661c","pinned_image":"ghcr.io/github/gh-aw-firewall/agent:0.27.7@sha256:aae231e4635c8999d039c132f1602d3df850fe9b84a00aa2b5ac981179b5661c"},{"image":"ghcr.io/github/gh-aw-firewall/api-proxy:0.27.7","digest":"sha256:009caf2e3d88fa77b64e9a03a95a228fc58db0f1701c6d324b29ba5a3c7c79b6","pinned_image":"ghcr.io/github/gh-aw-firewall/api-proxy:0.27.7@sha256:009caf2e3d88fa77b64e9a03a95a228fc58db0f1701c6d324b29ba5a3c7c79b6"},{"image":"ghcr.io/github/gh-aw-firewall/squid:0.27.7","digest":"sha256:deb1d4e19de62d51cee0508057a596a19315c3423ada4d675cad136dc8037c96","pinned_image":"ghcr.io/github/gh-aw-firewall/squid:0.27.7@sha256:deb1d4e19de62d51cee0508057a596a19315c3423ada4d675cad136dc8037c96"},{"image":"ghcr.io/github/gh-aw-mcpg:v0.3.27","digest":"sha256:fe984bddde4ec05d756d9043edb0a32912e6b7b72f6a121b1082f29221421cc7","pinned_image":"ghcr.io/github/gh-aw-mcpg:v0.3.27@sha256:fe984bddde4ec05d756d9043edb0a32912e6b7b72f6a121b1082f29221421cc7"},{"image":"ghcr.io/github/gh-aw-node","digest":"sha256:529d02eb970b1161aa25c593a9c3df57fdfad5a8add328cb3b6eccef66f3183b","pinned_image":"ghcr.io/github/gh-aw-node@sha256:529d02eb970b1161aa25c593a9c3df57fdfad5a8add328cb3b6eccef66f3183b"},{"image":"ghcr.io/github/github-mcp-server:v1.4.0","digest":"sha256:2afb26356481d1a350e14544a6e160f7f7ec1561a1ea309b823665abf0309036","pinned_image":"ghcr.io/github/github-mcp-server:v1.4.0@sha256:2afb26356481d1a350e14544a6e160f7f7ec1561a1ea309b823665abf0309036"}]} # This file was automatically generated by gh-aw (v0.80.9). DO NOT EDIT. To debug this workflow, load the skill at https://github.com/github/gh-aw/blob/main/debug.md # @@ -37,7 +37,9 @@ # humans (the open tracking issue is the hand-off surface; a dedicated # [ci-fix][needs-human] PR is planned but currently deferred — see Step 6). CI on # the PR is kicked by a human `/azp run` for now; the loop watches, classifies, -# and re-fixes autonomously between kicks. +# and re-fixes autonomously between kicks. When the SPECIFIC test a PR fixed is +# confirmed green in that PR's own CI, the loop marks the draft PR ready for review +# (a state transition only — it never approves or merges). # Never mutes tests, but # de-flakes genuinely flaky ones (deterministic synchronization, no retries / # timeout bumps). Always skips visual-regression / screenshot issues. @@ -273,24 +275,24 @@ jobs: run: | bash "${RUNNER_TEMP}/gh-aw/actions/create_prompt_first.sh" { - cat << 'GH_AW_PROMPT_25cf75111b670167_EOF' + cat << 'GH_AW_PROMPT_2ac1d0b4083428f8_EOF' - GH_AW_PROMPT_25cf75111b670167_EOF + GH_AW_PROMPT_2ac1d0b4083428f8_EOF cat "${RUNNER_TEMP}/gh-aw/prompts/xpia.md" cat "${RUNNER_TEMP}/gh-aw/prompts/temp_folder_prompt.md" cat "${RUNNER_TEMP}/gh-aw/prompts/markdown.md" cat "${RUNNER_TEMP}/gh-aw/prompts/safe_outputs_prompt.md" - cat << 'GH_AW_PROMPT_25cf75111b670167_EOF' + cat << 'GH_AW_PROMPT_2ac1d0b4083428f8_EOF' - Tools: add_comment(max:3), create_pull_request(max:3), update_pull_request(max:3), push_to_pull_request_branch(max:3), missing_tool, missing_data, noop - GH_AW_PROMPT_25cf75111b670167_EOF + Tools: add_comment(max:6), create_pull_request(max:3), update_pull_request(max:3), mark_pull_request_as_ready_for_review(max:3), add_labels(max:3), push_to_pull_request_branch(max:3), missing_tool, missing_data, noop + GH_AW_PROMPT_2ac1d0b4083428f8_EOF cat "${RUNNER_TEMP}/gh-aw/prompts/safe_outputs_create_pull_request.md" cat "${RUNNER_TEMP}/gh-aw/prompts/safe_outputs_push_to_pr_branch.md" - cat << 'GH_AW_PROMPT_25cf75111b670167_EOF' + cat << 'GH_AW_PROMPT_2ac1d0b4083428f8_EOF' - GH_AW_PROMPT_25cf75111b670167_EOF + GH_AW_PROMPT_2ac1d0b4083428f8_EOF cat "${RUNNER_TEMP}/gh-aw/prompts/mcp_cli_tools_prompt.md" - cat << 'GH_AW_PROMPT_25cf75111b670167_EOF' + cat << 'GH_AW_PROMPT_2ac1d0b4083428f8_EOF' The following GitHub context information is available for this workflow: {{#if github.actor}} @@ -332,12 +334,12 @@ jobs: stop immediately and report the limitation rather than spending turns trying to work around it. - GH_AW_PROMPT_25cf75111b670167_EOF + GH_AW_PROMPT_2ac1d0b4083428f8_EOF cat "${RUNNER_TEMP}/gh-aw/prompts/github_mcp_tools_with_safeoutputs_prompt.md" - cat << 'GH_AW_PROMPT_25cf75111b670167_EOF' + cat << 'GH_AW_PROMPT_2ac1d0b4083428f8_EOF' {{#runtime-import .github/workflows/ci-status-fix.md}} - GH_AW_PROMPT_25cf75111b670167_EOF + GH_AW_PROMPT_2ac1d0b4083428f8_EOF } > "$GH_AW_PROMPT" - name: Interpolate variables and render templates uses: actions/github-script@3a2844b7e9c422d3c10d287c895573f7108da1b3 # v9.0.0 @@ -554,16 +556,18 @@ jobs: mkdir -p "${RUNNER_TEMP}/gh-aw/safeoutputs" mkdir -p /tmp/gh-aw/safeoutputs mkdir -p /tmp/gh-aw/mcp-logs/safeoutputs - cat > "${RUNNER_TEMP}/gh-aw/safeoutputs/config.json" << 'GH_AW_SAFE_OUTPUTS_CONFIG_fa2a69fad5fd0485_EOF' - {"add_comment":{"discussions":false,"max":3,"required_labels":["agentic-workflows"],"required_title_prefix":"[ci-fix] ","target":"*"},"create_pull_request":{"allowed_base_branches":["main"],"allowed_branches":["ci-fix/**"],"allowed_files":["src/Core/**","src/Controls/**","src/Essentials/**","src/BlazorWebView/**","src/TestUtils/**","src/Templates/**","**/PublicAPI.Unshipped.txt"],"base_branch":"main","draft":true,"labels":["agentic-workflows"],"max":3,"max_patch_files":100,"max_patch_size":4096,"protect_top_level_dot_folders":true,"protected_files":["package.json","bun.lockb","bunfig.toml","deno.json","deno.jsonc","deno.lock","global.json","NuGet.Config","Directory.Packages.props","mix.exs","mix.lock","go.mod","go.sum","stack.yaml","stack.yaml.lock","pom.xml","build.gradle","build.gradle.kts","settings.gradle","settings.gradle.kts","gradle.properties","package-lock.json","yarn.lock","pnpm-lock.yaml","npm-shrinkwrap.json","requirements.txt","Pipfile","Pipfile.lock","pyproject.toml","setup.py","setup.cfg","Gemfile","Gemfile.lock","uv.lock","CODEOWNERS","DESIGN.md","README.md","CONTRIBUTING.md","CHANGELOG.md","SECURITY.md","CODE_OF_CONDUCT.md","AGENTS.md","CLAUDE.md","GEMINI.md"],"protected_files_policy":"request_review","title_prefix":"[ci-fix] "},"create_report_incomplete_issue":{},"missing_data":{},"missing_tool":{},"noop":{"max":1,"report-as-issue":"true"},"push_to_pull_request_branch":{"allowed_files":["src/Core/**","src/Controls/**","src/Essentials/**","src/BlazorWebView/**","src/TestUtils/**","src/Templates/**","**/PublicAPI.Unshipped.txt"],"if_no_changes":"warn","max":3,"max_patch_size":4096,"protect_top_level_dot_folders":true,"protected_files":["package.json","bun.lockb","bunfig.toml","deno.json","deno.jsonc","deno.lock","global.json","NuGet.Config","Directory.Packages.props","mix.exs","mix.lock","go.mod","go.sum","stack.yaml","stack.yaml.lock","pom.xml","build.gradle","build.gradle.kts","settings.gradle","settings.gradle.kts","gradle.properties","package-lock.json","yarn.lock","pnpm-lock.yaml","npm-shrinkwrap.json","requirements.txt","Pipfile","Pipfile.lock","pyproject.toml","setup.py","setup.cfg","Gemfile","Gemfile.lock","uv.lock","CODEOWNERS","DESIGN.md","README.md","CONTRIBUTING.md","CHANGELOG.md","SECURITY.md","CODE_OF_CONDUCT.md","AGENTS.md","CLAUDE.md","GEMINI.md"],"required_labels":["agentic-workflows"],"target":"*","title_prefix":"[ci-fix] "},"report_incomplete":{},"update_pull_request":{"allow_body":true,"allow_title":false,"max":3,"target":"*","update_branch":false}} - GH_AW_SAFE_OUTPUTS_CONFIG_fa2a69fad5fd0485_EOF + cat > "${RUNNER_TEMP}/gh-aw/safeoutputs/config.json" << 'GH_AW_SAFE_OUTPUTS_CONFIG_0644bf1434ca0071_EOF' + {"add_comment":{"discussions":false,"max":6,"required_labels":["agentic-workflows"],"required_title_prefix":"[ci-fix] ","target":"*"},"add_labels":{"allowed":["p/0"],"max":3,"required_labels":["agentic-workflows"],"required_title_prefix":"[ci-fix] ","target":"*"},"create_pull_request":{"allowed_base_branches":["main"],"allowed_branches":["ci-fix/**"],"allowed_files":["src/Core/**","src/Controls/**","src/Essentials/**","src/BlazorWebView/**","src/TestUtils/**","src/Templates/**","**/PublicAPI.Unshipped.txt"],"base_branch":"main","draft":true,"labels":["agentic-workflows"],"max":3,"max_patch_files":100,"max_patch_size":4096,"protect_top_level_dot_folders":true,"protected_files":["package.json","bun.lockb","bunfig.toml","deno.json","deno.jsonc","deno.lock","global.json","NuGet.Config","Directory.Packages.props","mix.exs","mix.lock","go.mod","go.sum","stack.yaml","stack.yaml.lock","pom.xml","build.gradle","build.gradle.kts","settings.gradle","settings.gradle.kts","gradle.properties","package-lock.json","yarn.lock","pnpm-lock.yaml","npm-shrinkwrap.json","requirements.txt","Pipfile","Pipfile.lock","pyproject.toml","setup.py","setup.cfg","Gemfile","Gemfile.lock","uv.lock","CODEOWNERS","DESIGN.md","README.md","CONTRIBUTING.md","CHANGELOG.md","SECURITY.md","CODE_OF_CONDUCT.md","AGENTS.md","CLAUDE.md","GEMINI.md"],"protected_files_policy":"request_review","title_prefix":"[ci-fix] "},"create_report_incomplete_issue":{},"mark_pull_request_as_ready_for_review":{"max":3,"required_labels":["agentic-workflows"],"required_title_prefix":"[ci-fix] ","target":"*"},"missing_data":{},"missing_tool":{},"noop":{"max":1,"report-as-issue":"true"},"push_to_pull_request_branch":{"allowed_files":["src/Core/**","src/Controls/**","src/Essentials/**","src/BlazorWebView/**","src/TestUtils/**","src/Templates/**","**/PublicAPI.Unshipped.txt"],"if_no_changes":"warn","max":3,"max_patch_size":4096,"protect_top_level_dot_folders":true,"protected_files":["package.json","bun.lockb","bunfig.toml","deno.json","deno.jsonc","deno.lock","global.json","NuGet.Config","Directory.Packages.props","mix.exs","mix.lock","go.mod","go.sum","stack.yaml","stack.yaml.lock","pom.xml","build.gradle","build.gradle.kts","settings.gradle","settings.gradle.kts","gradle.properties","package-lock.json","yarn.lock","pnpm-lock.yaml","npm-shrinkwrap.json","requirements.txt","Pipfile","Pipfile.lock","pyproject.toml","setup.py","setup.cfg","Gemfile","Gemfile.lock","uv.lock","CODEOWNERS","DESIGN.md","README.md","CONTRIBUTING.md","CHANGELOG.md","SECURITY.md","CODE_OF_CONDUCT.md","AGENTS.md","CLAUDE.md","GEMINI.md"],"required_labels":["agentic-workflows"],"target":"*","title_prefix":"[ci-fix] "},"report_incomplete":{},"update_pull_request":{"allow_body":true,"allow_title":false,"max":3,"target":"*","update_branch":false}} + GH_AW_SAFE_OUTPUTS_CONFIG_0644bf1434ca0071_EOF - name: Generate Safe Outputs Tools env: GH_AW_TOOLS_META_JSON: | { "description_suffixes": { - "add_comment": " CONSTRAINTS: Maximum 3 comment(s) can be added. Target: *. Supports reply_to_id for discussion threading.", + "add_comment": " CONSTRAINTS: Maximum 6 comment(s) can be added. Target: *. Supports reply_to_id for discussion threading.", + "add_labels": " CONSTRAINTS: Maximum 3 label(s) can be added. Only these labels are allowed: [\"p/0\"]. Target: *.", "create_pull_request": " CONSTRAINTS: Maximum 3 pull request(s) can be created. Title will be prefixed with \"[ci-fix] \". Labels [\"agentic-workflows\"] will be automatically added. Only these labels are allowed: [\"agentic-workflows\"]. PRs will be created as drafts.", + "mark_pull_request_as_ready_for_review": " CONSTRAINTS: Maximum 3 pull request(s) can be marked as ready for review.", "push_to_pull_request_branch": " CONSTRAINTS: Maximum 3 push(es) can be made. The target pull request title must start with \"[ci-fix] \".", "update_pull_request": " CONSTRAINTS: Maximum 3 pull request(s) can be updated. Target: *." }, @@ -594,6 +598,25 @@ jobs: } } }, + "add_labels": { + "defaultMax": 5, + "fields": { + "item_number": { + "issueNumberOrTemporaryId": true + }, + "labels": { + "required": true, + "type": "array", + "itemType": "string", + "itemSanitize": true, + "itemMaxLength": 128 + }, + "repo": { + "type": "string", + "maxLength": 256 + } + } + }, "create_pull_request": { "defaultMax": 1, "fields": { @@ -635,6 +658,24 @@ jobs: } } }, + "mark_pull_request_as_ready_for_review": { + "defaultMax": 1, + "fields": { + "pull_request_number": { + "issueOrPRNumber": true + }, + "reason": { + "required": true, + "type": "string", + "sanitize": true, + "maxLength": 65000 + }, + "repo": { + "type": "string", + "maxLength": 256 + } + } + }, "missing_data": { "defaultMax": 20, "fields": { @@ -1494,7 +1535,7 @@ jobs: uses: actions/github-script@3a2844b7e9c422d3c10d287c895573f7108da1b3 # v9.0.0 env: WORKFLOW_NAME: "CI Failure Fixer (main)" - WORKFLOW_DESCRIPTION: "Periodic pass over open ci-scan tracking issues filed by the main-branch CI\nfailure scanner (.github/workflows/ci-status-main.md). This workflow targets\nthe `main` branch EXCLUSIVELY: it processes only issues labelled ci-scan and\nopens every PR against main. (The net11.0 branch is handled by the parallel\n.github/workflows/ci-status-fix-net11.md — the two are split because gh-aw can\nonly transport a fix relative to ONE static base branch per workflow, and the\nmain↔net11.0 divergence exceeds gh-aw's 10 MB transport-patch cap.) The fixer\nopens ONE draft [ci-fix] PR per actionable issue against main, then WATCHES\nthat PR's own CI on later runs: when the fix's CI comes back red and the red is\ncaused by the fix itself, it pushes a fresh follow-up fix onto the SAME PR\nbranch (never a second PR) — up to 10 attempts — then stops and defers to\nhumans (the open tracking issue is the hand-off surface; a dedicated\n[ci-fix][needs-human] PR is planned but currently deferred — see Step 6). CI on\nthe PR is kicked by a human `/azp run` for now; the loop watches, classifies,\nand re-fixes autonomously between kicks.\nNever mutes tests, but\nde-flakes genuinely flaky ones (deterministic synchronization, no retries /\ntimeout bumps). Always skips visual-regression / screenshot issues." + WORKFLOW_DESCRIPTION: "Periodic pass over open ci-scan tracking issues filed by the main-branch CI\nfailure scanner (.github/workflows/ci-status-main.md). This workflow targets\nthe `main` branch EXCLUSIVELY: it processes only issues labelled ci-scan and\nopens every PR against main. (The net11.0 branch is handled by the parallel\n.github/workflows/ci-status-fix-net11.md — the two are split because gh-aw can\nonly transport a fix relative to ONE static base branch per workflow, and the\nmain↔net11.0 divergence exceeds gh-aw's 10 MB transport-patch cap.) The fixer\nopens ONE draft [ci-fix] PR per actionable issue against main, then WATCHES\nthat PR's own CI on later runs: when the fix's CI comes back red and the red is\ncaused by the fix itself, it pushes a fresh follow-up fix onto the SAME PR\nbranch (never a second PR) — up to 10 attempts — then stops and defers to\nhumans (the open tracking issue is the hand-off surface; a dedicated\n[ci-fix][needs-human] PR is planned but currently deferred — see Step 6). CI on\nthe PR is kicked by a human `/azp run` for now; the loop watches, classifies,\nand re-fixes autonomously between kicks. When the SPECIFIC test a PR fixed is\nconfirmed green in that PR's own CI, the loop marks the draft PR ready for review\n(a state transition only — it never approves or merges).\nNever mutes tests, but\nde-flakes genuinely flaky ones (deterministic synchronization, no retries /\ntimeout bumps). Always skips visual-regression / screenshot issues." HAS_PATCH: ${{ needs.agent.outputs.has_patch }} with: script: | @@ -1910,7 +1951,7 @@ jobs: GH_AW_ALLOWED_DOMAINS: "*.blob.core.windows.net,*.githubusercontent.com,api.business.githubcopilot.com,api.enterprise.githubcopilot.com,api.github.com,api.githubcopilot.com,api.individual.githubcopilot.com,api.snapcraft.io,archive.ubuntu.com,azure.archive.ubuntu.com,codeload.github.com,crl.geotrust.com,crl.globalsign.com,crl.identrust.com,crl.sectigo.com,crl.thawte.com,crl.usertrust.com,crl.verisign.com,crl3.digicert.com,crl4.digicert.com,crls.ssl.com,dev.azure.com,docs.github.com,github-cloud.githubusercontent.com,github-cloud.s3.amazonaws.com,github.blog,github.com,github.githubassets.com,helix.dot.net,host.docker.internal,json-schema.org,json.schemastore.org,keyserver.ubuntu.com,lfs.github.com,objects.githubusercontent.com,ocsp.digicert.com,ocsp.geotrust.com,ocsp.globalsign.com,ocsp.identrust.com,ocsp.sectigo.com,ocsp.ssl.com,ocsp.thawte.com,ocsp.usertrust.com,ocsp.verisign.com,packagecloud.io,packages.cloud.google.com,packages.microsoft.com,patch-diff.githubusercontent.com,ppa.launchpad.net,raw.githubusercontent.com,registry.npmjs.org,s.symcb.com,s.symcd.com,security.ubuntu.com,telemetry.enterprise.githubcopilot.com,ts-crl.ws.symantec.com,ts-ocsp.ws.symantec.com,www.googleapis.com" GITHUB_SERVER_URL: ${{ github.server_url }} GITHUB_API_URL: ${{ github.api_url }} - GH_AW_SAFE_OUTPUTS_HANDLER_CONFIG: "{\"add_comment\":{\"discussions\":false,\"max\":3,\"required_labels\":[\"agentic-workflows\"],\"required_title_prefix\":\"[ci-fix] \",\"target\":\"*\"},\"create_pull_request\":{\"allowed_base_branches\":[\"main\"],\"allowed_branches\":[\"ci-fix/**\"],\"allowed_files\":[\"src/Core/**\",\"src/Controls/**\",\"src/Essentials/**\",\"src/BlazorWebView/**\",\"src/TestUtils/**\",\"src/Templates/**\",\"**/PublicAPI.Unshipped.txt\"],\"base_branch\":\"main\",\"draft\":true,\"labels\":[\"agentic-workflows\"],\"max\":3,\"max_patch_files\":100,\"max_patch_size\":4096,\"protect_top_level_dot_folders\":true,\"protected_files\":[\"package.json\",\"bun.lockb\",\"bunfig.toml\",\"deno.json\",\"deno.jsonc\",\"deno.lock\",\"global.json\",\"NuGet.Config\",\"Directory.Packages.props\",\"mix.exs\",\"mix.lock\",\"go.mod\",\"go.sum\",\"stack.yaml\",\"stack.yaml.lock\",\"pom.xml\",\"build.gradle\",\"build.gradle.kts\",\"settings.gradle\",\"settings.gradle.kts\",\"gradle.properties\",\"package-lock.json\",\"yarn.lock\",\"pnpm-lock.yaml\",\"npm-shrinkwrap.json\",\"requirements.txt\",\"Pipfile\",\"Pipfile.lock\",\"pyproject.toml\",\"setup.py\",\"setup.cfg\",\"Gemfile\",\"Gemfile.lock\",\"uv.lock\",\"CODEOWNERS\",\"DESIGN.md\",\"README.md\",\"CONTRIBUTING.md\",\"CHANGELOG.md\",\"SECURITY.md\",\"CODE_OF_CONDUCT.md\",\"AGENTS.md\",\"CLAUDE.md\",\"GEMINI.md\"],\"protected_files_policy\":\"request_review\",\"title_prefix\":\"[ci-fix] \"},\"create_report_incomplete_issue\":{},\"missing_data\":{},\"missing_tool\":{},\"noop\":{\"max\":1,\"report-as-issue\":\"true\"},\"push_to_pull_request_branch\":{\"allowed_files\":[\"src/Core/**\",\"src/Controls/**\",\"src/Essentials/**\",\"src/BlazorWebView/**\",\"src/TestUtils/**\",\"src/Templates/**\",\"**/PublicAPI.Unshipped.txt\"],\"if_no_changes\":\"warn\",\"max\":3,\"max_patch_size\":4096,\"protect_top_level_dot_folders\":true,\"protected_files\":[\"package.json\",\"bun.lockb\",\"bunfig.toml\",\"deno.json\",\"deno.jsonc\",\"deno.lock\",\"global.json\",\"NuGet.Config\",\"Directory.Packages.props\",\"mix.exs\",\"mix.lock\",\"go.mod\",\"go.sum\",\"stack.yaml\",\"stack.yaml.lock\",\"pom.xml\",\"build.gradle\",\"build.gradle.kts\",\"settings.gradle\",\"settings.gradle.kts\",\"gradle.properties\",\"package-lock.json\",\"yarn.lock\",\"pnpm-lock.yaml\",\"npm-shrinkwrap.json\",\"requirements.txt\",\"Pipfile\",\"Pipfile.lock\",\"pyproject.toml\",\"setup.py\",\"setup.cfg\",\"Gemfile\",\"Gemfile.lock\",\"uv.lock\",\"CODEOWNERS\",\"DESIGN.md\",\"README.md\",\"CONTRIBUTING.md\",\"CHANGELOG.md\",\"SECURITY.md\",\"CODE_OF_CONDUCT.md\",\"AGENTS.md\",\"CLAUDE.md\",\"GEMINI.md\"],\"required_labels\":[\"agentic-workflows\"],\"target\":\"*\",\"title_prefix\":\"[ci-fix] \"},\"report_incomplete\":{},\"update_pull_request\":{\"allow_body\":true,\"allow_title\":false,\"max\":3,\"target\":\"*\",\"update_branch\":false}}" + GH_AW_SAFE_OUTPUTS_HANDLER_CONFIG: "{\"add_comment\":{\"discussions\":false,\"max\":6,\"required_labels\":[\"agentic-workflows\"],\"required_title_prefix\":\"[ci-fix] \",\"target\":\"*\"},\"add_labels\":{\"allowed\":[\"p/0\"],\"max\":3,\"required_labels\":[\"agentic-workflows\"],\"required_title_prefix\":\"[ci-fix] \",\"target\":\"*\"},\"create_pull_request\":{\"allowed_base_branches\":[\"main\"],\"allowed_branches\":[\"ci-fix/**\"],\"allowed_files\":[\"src/Core/**\",\"src/Controls/**\",\"src/Essentials/**\",\"src/BlazorWebView/**\",\"src/TestUtils/**\",\"src/Templates/**\",\"**/PublicAPI.Unshipped.txt\"],\"base_branch\":\"main\",\"draft\":true,\"labels\":[\"agentic-workflows\"],\"max\":3,\"max_patch_files\":100,\"max_patch_size\":4096,\"protect_top_level_dot_folders\":true,\"protected_files\":[\"package.json\",\"bun.lockb\",\"bunfig.toml\",\"deno.json\",\"deno.jsonc\",\"deno.lock\",\"global.json\",\"NuGet.Config\",\"Directory.Packages.props\",\"mix.exs\",\"mix.lock\",\"go.mod\",\"go.sum\",\"stack.yaml\",\"stack.yaml.lock\",\"pom.xml\",\"build.gradle\",\"build.gradle.kts\",\"settings.gradle\",\"settings.gradle.kts\",\"gradle.properties\",\"package-lock.json\",\"yarn.lock\",\"pnpm-lock.yaml\",\"npm-shrinkwrap.json\",\"requirements.txt\",\"Pipfile\",\"Pipfile.lock\",\"pyproject.toml\",\"setup.py\",\"setup.cfg\",\"Gemfile\",\"Gemfile.lock\",\"uv.lock\",\"CODEOWNERS\",\"DESIGN.md\",\"README.md\",\"CONTRIBUTING.md\",\"CHANGELOG.md\",\"SECURITY.md\",\"CODE_OF_CONDUCT.md\",\"AGENTS.md\",\"CLAUDE.md\",\"GEMINI.md\"],\"protected_files_policy\":\"request_review\",\"title_prefix\":\"[ci-fix] \"},\"create_report_incomplete_issue\":{},\"mark_pull_request_as_ready_for_review\":{\"max\":3,\"required_labels\":[\"agentic-workflows\"],\"required_title_prefix\":\"[ci-fix] \",\"target\":\"*\"},\"missing_data\":{},\"missing_tool\":{},\"noop\":{\"max\":1,\"report-as-issue\":\"true\"},\"push_to_pull_request_branch\":{\"allowed_files\":[\"src/Core/**\",\"src/Controls/**\",\"src/Essentials/**\",\"src/BlazorWebView/**\",\"src/TestUtils/**\",\"src/Templates/**\",\"**/PublicAPI.Unshipped.txt\"],\"if_no_changes\":\"warn\",\"max\":3,\"max_patch_size\":4096,\"protect_top_level_dot_folders\":true,\"protected_files\":[\"package.json\",\"bun.lockb\",\"bunfig.toml\",\"deno.json\",\"deno.jsonc\",\"deno.lock\",\"global.json\",\"NuGet.Config\",\"Directory.Packages.props\",\"mix.exs\",\"mix.lock\",\"go.mod\",\"go.sum\",\"stack.yaml\",\"stack.yaml.lock\",\"pom.xml\",\"build.gradle\",\"build.gradle.kts\",\"settings.gradle\",\"settings.gradle.kts\",\"gradle.properties\",\"package-lock.json\",\"yarn.lock\",\"pnpm-lock.yaml\",\"npm-shrinkwrap.json\",\"requirements.txt\",\"Pipfile\",\"Pipfile.lock\",\"pyproject.toml\",\"setup.py\",\"setup.cfg\",\"Gemfile\",\"Gemfile.lock\",\"uv.lock\",\"CODEOWNERS\",\"DESIGN.md\",\"README.md\",\"CONTRIBUTING.md\",\"CHANGELOG.md\",\"SECURITY.md\",\"CODE_OF_CONDUCT.md\",\"AGENTS.md\",\"CLAUDE.md\",\"GEMINI.md\"],\"required_labels\":[\"agentic-workflows\"],\"target\":\"*\",\"title_prefix\":\"[ci-fix] \"},\"report_incomplete\":{},\"update_pull_request\":{\"allow_body\":true,\"allow_title\":false,\"max\":3,\"target\":\"*\",\"update_branch\":false}}" GH_AW_CI_TRIGGER_TOKEN: ${{ secrets.GH_AW_CI_TRIGGER_TOKEN }} with: github-token: ${{ secrets.GH_AW_GITHUB_TOKEN || secrets.GITHUB_TOKEN }} diff --git a/.github/workflows/ci-status-fix.md b/.github/workflows/ci-status-fix.md index d3227e25170c..9c2f46054029 100644 --- a/.github/workflows/ci-status-fix.md +++ b/.github/workflows/ci-status-fix.md @@ -15,7 +15,9 @@ description: | humans (the open tracking issue is the hand-off surface; a dedicated [ci-fix][needs-human] PR is planned but currently deferred — see Step 6). CI on the PR is kicked by a human `/azp run` for now; the loop watches, classifies, - and re-fixes autonomously between kicks. + and re-fixes autonomously between kicks. When the SPECIFIC test a PR fixed is + confirmed green in that PR's own CI, the loop marks the draft PR ready for review + (a state transition only — it never approves or merges). Never mutes tests, but de-flakes genuinely flaky ones (deterministic synchronization, no retries / timeout bumps). Always skips visual-regression / screenshot issues. @@ -239,8 +241,15 @@ safe-outputs: # PER-RUN total (not per-PR): a sweep may surface several green PRs and/or # annotate several flaky reds in one run. At max:1 all but the first comment # were silently dropped, starving the workflow's #1 value (surfacing green PRs - # for review). Raised to 3 to match the per-run throughput of the other outputs. - max: 3 + # for review). Sized to 6 (not 3) because a single green DRAFT PR that gets + # flipped ready in one sweep spends TWO comment slots — the Step 3 ✅ surface + # comment (which still names the /azp-gated legs a human must kick, so it is NOT + # redundant with 🎯) AND the Step 3.6 T3 🎯 readiness comment. At max:3 the shared + # bucket drained after ~1 draft flip and T3's atomicity pre-check then deferred + # every further mark-ready even while the mark-ready/add_labels buckets (also 3) + # sat idle; 6 lets ~3 draft flips (2 comments each) land per sweep, so the + # mark-ready:3 / add_labels:3 caps are actually reachable. + max: 6 target: "*" # Hard constraint (defense-in-depth): only comment on THIS workflow's own # ci-fix PRs — [ci-fix] title prefix AND agentic-workflows label. Mirrors the @@ -261,13 +270,51 @@ safe-outputs: # never the title, so disable title rewrites — this compiles to allow_title:false # and removes any ability to retitle an arbitrary PR. title: false - # NOTE: gh-aw v0.79.8 does NOT emit required-title-prefix/required-labels into + # NOTE: gh-aw v0.80.9 (the pinned compiler) does NOT emit required-title-prefix/required-labels into # the compiled config for update-pull-request (verified against the lock — it # silently drops them, unlike add-comment / push-to-pull-request-branch which # honor them). So which-PR scoping here relies on prompt Hard-Rule 6 + # min-integrity:approved; the residual is body-marker edits only (no code, no # merge, no close), capped at max:3. Revisit required-* if a future gh-aw # compiler emits them for this output. + mark-pull-request-as-ready-for-review: + # Flip a validated draft [ci-fix] PR to ready-for-review once the SPECIFIC test + # this PR was opened to fix is confirmed green in the PR's OWN CI (Step 3.6) — + # even when unrelated legs are red. The workflow is schedule/dispatch-triggered + # (no triggering-PR context), so target "*" lets the agent name the PR number it + # validated. This is a state transition ONLY: it never approves and never merges — + # a human still reviews and merges. + # PER-RUN total (not per-PR): a sweep may validate several PRs' target tests green. + max: 3 + target: "*" + # Hard constraint (defense-in-depth): only ever un-draft THIS workflow's own + # ci-fix PRs — [ci-fix] title prefix AND agentic-workflows label — so a confused + # or prompt-injected agent cannot mark an arbitrary PR ready. (If the v0.80.9 + # compiler silently drops these — as documented for update-pull-request above — + # the Step 3.6 preconditions + min-integrity:approved are the compensating scope + # controls; verify against the lock after compiling.) + required-title-prefix: "[ci-fix] " + required-labels: [agentic-workflows] + add-labels: + # Apply the p/0 priority label to a [ci-fix] PR at the exact moment the loop flips it + # from draft to ready-for-review (Step 3.6 T3) — so a validated, review-ready fix lands + # in the team's p/0 triage queue instead of sitting unseen in the draft backlog. Paired + # 1:1 with mark-pull-request-as-ready-for-review; target "*" lets the agent name the PR + # it just validated. + # allowed = HARD allowlist: the agent may ONLY ever add p/0, nothing else. This caps the + # blast radius of a confused/prompt-injected agent to exactly one benign priority label — + # it can never apply a downstream-triggering or destructive label. + allowed: [p/0] + # PER-RUN total (not per-PR): a sweep may mark several PRs ready in one run; each adds + # one label, so this matches the mark-ready per-run cap (3). + max: 3 + target: "*" + # Hard constraint (defense-in-depth): only ever label THIS workflow's own ci-fix PRs — + # [ci-fix] title prefix AND agentic-workflows label. (If the v0.80.9 compiler drops these + # — as documented for update-pull-request above — the Step 3.6 preconditions + + # min-integrity:approved + the allowed:[p/0] allowlist are the compensating controls.) + required-title-prefix: "[ci-fix] " + required-labels: [agentic-workflows] timeout-minutes: 90 @@ -339,33 +386,48 @@ through `safe-outputs`. 5. **One issue = one outcome per run.** Exactly one of: respond to a maintainer's change-request on the open PR (Track C, Step 3.5.R); open the first fix/help/ de-flake PR; advance an existing PR by one attempt (push a follow-up fix); - surface a validated-green PR for review; annotate an unrelated-flake red; wait + surface a validated-green PR for review; mark a target-validated draft PR + ready-for-review (Step 3.6 T3 — the terminal outcome that supersedes the same + run's surface/annotate precursor line), or record its `already-ready` + steady-state no-op; annotate an unrelated-flake red; wait (CI not yet settled); or a recorded skip (the dedicated needs-human PR is deferred — the attempt cap records a skip instead; see Step 6). Always prefer advancing or opening a PR over a skip when a non-mute diff is producible. 6. **All writes via `safe-outputs`.** Allowed outputs: `create_pull_request` (first attempt only), `push_to_pull_request_branch` (advance an existing PR by one attempt), `update_pull_request` (bump the attempt marker / refresh the - prior-attempts table), and `add_comment` (progress notes on the `[ci-fix]` PR - ONLY). NEVER comment on the tracking issue (issues are locked by + prior-attempts table), `add_comment` (progress notes on the `[ci-fix]` PR + ONLY), `mark_pull_request_as_ready_for_review` (flip a target-validated draft + `[ci-fix]` PR from draft to ready — Step 3.6 T3 only), and `add_labels` (add + ONLY the `p/0` label, ONLY on that same draft→ready transition — Step 3.6 T3). + NEVER comment on the tracking issue (issues are locked by `.github/workflows/ci-scan-lock-issues.yml`) — `add_comment` targets the PR. No `gh pr create`, no manual `git push`. **Defense-in-depth:** `add_comment` and `update_pull_request` use `target: "*"` (the agent supplies the PR number). - `add_comment` is now config-locked to `required-title-prefix: "[ci-fix] "` + + `add_comment`, `mark_pull_request_as_ready_for_review`, and `add_labels` are + config-locked to `required-title-prefix: "[ci-fix] "` + `required-labels: [agentic-workflows]` (hard-enforced by the handler, same as - `push_to_pull_request_branch`), so a comment can only ever land on THIS - workflow's own PRs. `update_pull_request` CANNOT be config-locked to a - title/label in gh-aw v0.79.8 (the compiler silently drops `required-*` for that - output — verified against the lock), so it keeps `title: false` (no retitles; - body-marker edits only) plus this prompt-level guard. Before emitting either, - VERIFY the target PR carries BOTH the `[ci-fix]` title prefix AND the - `agentic-workflows` label; never comment on or edit the body of any PR that - lacks both. + `push_to_pull_request_branch`), and `add_labels` additionally has an + `allowed: [p/0]` allowlist so it can ONLY ever add `p/0` — so these can only + ever land on THIS workflow's own PRs. `update_pull_request` CANNOT be + config-locked to a title/label in gh-aw v0.80.9 (the compiler silently drops + `required-*` for that output — verified against the lock), so it keeps + `title: false` (no retitles; body-marker edits only) plus this prompt-level + guard. Before emitting ANY of these, VERIFY the target PR carries BOTH the + `[ci-fix]` title prefix AND the `agentic-workflows` label; never comment on, + edit, un-draft, or label any PR that lacks both. 7. **Per-run safe-output caps (per RUN, not per PR).** Each output type is capped per run: `create_pull_request` 3, `push_to_pull_request_branch` 3, - `add_comment` 3, `update_pull_request` 3. Note an ADVANCE spends one push **and** - one update **and** one comment, so ≤ 3 advances/run; surfacing a green or - annotating a flake spends one comment. When a bucket is exhausted, do NOT keep + `add_comment` 6, `update_pull_request` 3, `mark_pull_request_as_ready_for_review` + 3, `add_labels` 3 (counts LABELS, not calls). Note an ADVANCE spends one push + **and** one update **and** one comment, so ≤ 3 advances/run; surfacing a green or + annotating a flake spends one comment; a **mark-ready (Step 3.6 T3)** of a + still-draft PR spends TWO comments (the Step 3 ✅ surface **and** the T3 🎯 + readiness note) **and** one mark-ready **and** one label as an ALL-OR-NOTHING set + (T3 pre-checks those buckets and defers the whole PR if any is exhausted — never + mark a PR ready without its 🎯 audit comment). `add_comment` is therefore sized 6 + (not 3) so ~3 draft flips, at 2 comments each, can land in one sweep instead of the + shared comment bucket starving the otherwise-idle mark-ready/add_labels buckets. When a bucket is exhausted, do NOT keep emitting (extras are silently dropped) — record `skipped: per-run cap reached; deferring PR # to next cycle` for each remaining PR so the drop is a deliberate, logged decision. The continuous loop picks the deferred PRs up next @@ -409,8 +471,9 @@ For every open tracking issue in scope, converge on exactly one outcome: | Open first help-wanted draft `[ci-fix]` PR | No open PR yet; a plausible candidate exists but cannot be runner-validated (device/UI tests) (attempt 1) | | Open first de-flake draft `[ci-fix]` PR | No open PR yet; failure is intermittent (green-on-retry) from a genuine test-quality defect; a deterministic-synchronization fix is producible (attempt 1, Step 4.7 bucket b) | | Advance an existing PR (push attempt N+1) | Open PR's own CI settled red *because of the fix itself*, no human engaged, marker < 10 — push a NEW distinct fix onto the same branch (Step 3.5 → 5.6) | -| Surface a validated-green PR | Open PR's own CI settled green — comment "validated, attempt N/10; ready for review" and do NOT advance (Step 3.5) | -| Annotate an unrelated-flake red | Open PR is red only on baseline-flake / unrelated legs — comment which leg needs a re-run; do NOT burn an attempt (Step 3.5) | +| Surface a validated-green PR | Open PR's own CI settled green — comment "validated, attempt N/10; ready for review" and do NOT advance (Step 3.5), then mark the draft PR ready for review once the fixed test is confirmed green (Step 3.6) | +| Annotate an unrelated-flake red | Open PR is red only on baseline-flake / unrelated legs — comment which leg needs a re-run; do NOT burn an attempt (Step 3.5); if the SPECIFIC fixed test is confirmed green on that build, still mark the draft PR ready for review (Step 3.6) | +| Mark a validated PR ready for review | The specific test this PR fixed is confirmed green in the PR's own CI (even if unrelated legs are red) — comment the target-test result and transition the draft PR to ready for review; never approve or merge (Step 3.6) | | Wait | Open PR's CI is pending / not yet settled on the current head SHA — do nothing this cycle (Step 3.5) | | Hand-off skip (attempt cap) | Marker == 10 and the signature still reproduces — stop and defer to humans; the dedicated `[ci-fix][needs-human]` PR is planned but currently deferred (Step 6) | | Recorded skip | Visual-regression, human engaged, already-handled, fixed-in-latest-build, infra-flake, out-of-bounds, only-mute-available, or no novel approach producible | @@ -685,14 +748,34 @@ Run these gates in order — the FIRST that fires decides this cycle's outcome: 1. **Human engaged.** If `C.humanEngaged` → `skipped: human engaged on PR #; deferring` and stop. Never fight a human reviewer — the intentional hand-off boundary still holds the moment a person touches the PR. -2. **CI not settled / unknown / incomplete prefetch.** If `C.checksSettled == false` - OR `C.overallConclusion` is `pending`, `neutral`, or `unknown`, OR - `C.dataComplete == false` → the fix's CI has not finished on the current head SHA - (`C.headSha`), or the prefetch could not fully read this PR (a partial read can - understate `humanEngaged` / `attempt`). `skipped: PR # CI pending / prefetch - incomplete on ; waiting` and stop. *(Round 1: this is where a - maintainer `/azp run` is awaited — the `/azp`-gated uitests/devicetests legs will - not have run until a human kicks them.)* +2. **CI not settled — but validate the target test first (target-focused readiness).** + If `C.checksSettled == false` OR `C.overallConclusion` is `pending`, `neutral`, or + `unknown`, OR `C.dataComplete == false` → the *overall* build has not finished on the + current head SHA (`C.headSha`), or the prefetch could not fully read this PR (a partial + read can understate `humanEngaged` / `attempt`). Unrelated legs still draining must NOT + keep an already-proven fix parked as a draft, so before waiting, try the target-focused + fast-path: + - **2a — Target-green fast-path (draft `[ci-fix]` PRs only).** If `C.dataComplete == + true` AND the Step 3.6 preconditions hold (draft PR, `[ci-fix] ` title prefix AND the + `agentic-workflows` label), run Step 3.6 **T1–T2** against `C.headSha` now: identify + the target test(s) and read the PR's OWN build **timeline / per-leg status** (anonymous + `_apis/build` — never `_apis/test`, per Hard-Rule 8). If EVERY target test is + **VALIDATED-GREEN** (per T2's all-platform definition below — the target's category leg + is `succeeded` on every platform that runs it, failed on none, and NO target platform + family the pipeline covers is still pending/unconcluded, per T2's "no platform left + unverified" rule; a platform that simply has no leg for the target's category — the test + doesn't run there — is fine, not a gap) AND every leg in + `C.failedLegs` that has ALREADY concluded classifies as **unrelated flake** (Step 4 / + Step 4.7 method — the fix is implicated in NO completed red), then the fix is proven + regardless of unrelated *pending* legs → run **Step 3.6 T3** (mark ready + 🎯 comment) + and stop. This early-out NEVER advances an attempt and NEVER acts on a red that could + be the fix's fault. + - **2b — Otherwise WAIT.** If the target test has not yet executed (pending/absent on + its leg), or a completed red is (or may be) caused by the fix, or `C.dataComplete == + false`, or this PR has no identifiable target test → `skipped: PR # CI pending / + target not yet validated on ; waiting` and stop. *(Round 1: this is where a + maintainer `/azp run` is awaited — the `/azp`-gated uitests/devicetests legs will not + have run until a human kicks them.)* 3. **Green → surface for review.** If `C.overallConclusion == "success"`: the checks that RAN are green. **Primary-gate check first:** the deterministic `success` verdict only certifies "at least one green check, nothing failing or @@ -704,18 +787,31 @@ Run these gates in order — the FIRST that fires decides this cycle's outcome: # primary CI gate (maui-pr) not green on ; waiting` and stop — do NOT surface. (The `/azp`-gated `maui-pr-uitests` (def 313) / `maui-pr-devicetests` (def 314) legs MAY still be un-run — that is expected and is named below, not a - reason to withhold the surface.) **Idempotency:** scan the PR's existing comments - for a prior bot `✅ … validated … on ` note for THIS head SHA — if one - already exists, record `already-surfaced PR # (head )` and stop - WITHOUT re-commenting (re-surfacing the same green every tick is noise and burns - the per-run comment budget other PRs need). Otherwise resolve the attempt number from - `C.effectiveAttempt` (the authoritative max(marker, bot-commit) counter; omit the - number only if it is somehow indeterminate). **Dry-run gate (Step 0):** if `dry_run == "true"`, do NOT emit any comment — instead print the intended `✅` validated-green body to the run log, tally `dry-run: would-surface-green PR #`, and stop. Otherwise `add_comment` on PR #: `✅ Attempt /10 validated - — the fix's CI is green on . Ready for human review.` - Name any `/azp`-gated legs (uitests def 313 / devicetests def 314) that have not - run and still need a maintainer `/azp run`. Do NOT advance. Record - `surfaced-green PR # (attempt /10)` and stop. *(This directly - attacks the real bottleneck — no reviews — so it is the highest-value outcome.)* + reason to withhold the surface.) **Comment idempotency + dry-run suppress the ✅ + comment ONLY — neither skips Step 3.6.** Scan the PR's existing comments for a prior + bot `✅ … validated … on ` note for THIS head SHA; if one exists, set + `SKIP_SURFACE_COMMENT = true`. Resolve the attempt number from `C.effectiveAttempt` + (the authoritative max(marker, bot-commit) counter; omit the number only if it is + somehow indeterminate). Then post the surface comment UNLESS suppressed: + - if `dry_run == "true"`: do NOT emit — print the intended `✅` validated-green body to + the run log and tally `dry-run: would-surface-green PR #`; + - else if `SKIP_SURFACE_COMMENT`: do NOT re-post — record `already-surfaced PR # + (head )` (re-surfacing the same green every tick is noise and burns the + per-run comment budget other PRs need); + - else `add_comment` on PR #: `✅ Attempt /10 validated — the fix's CI is + green on .` naming any `/azp`-gated legs (uitests def 313 / devicetests def + 314) that have not run and still need a maintainer `/azp run`; record `surfaced-green + PR # (attempt /10)`. (Do NOT assert "ready for human review" on a PR that + is still a draft — Step 3.6, not this comment, owns the draft→ready flip and posts its + own 🎯 announcement when it fires.) + Do NOT advance. Then — in ALL of the above cases — run **Step 3.6** (target-test + readiness gate) for this PR before stopping: its `/azp`-gated target legs may conclude + green on this SAME head SHA on a LATER sweep (an `/azp run` adds no commit), and Step + 3.6 is the ONLY place the draft→ready flip happens, so it MUST re-evaluate every sweep — + never `stop` here before it. *(This directly attacks the real bottleneck — no reviews — + so it is the highest-value outcome.)* 4. **Red → classify caused-by-fix vs unrelated-flake.** If `C.overallConclusion == "failure"`, analyze the PR's OWN failing build (NOT `main`): find the AzDO `maui-pr` build for this PR (filter builds by `branchName=refs/pull//merge`, @@ -723,18 +819,26 @@ Run these gates in order — the FIRST that fires decides this cycle's outcome: method and Step 4.7 flake buckets to `C.failedLegs`. - **Unrelated flake only** (every failed leg is known-flaky / infra / a pre-existing baseline red NOT introduced by the fix): do NOT burn an attempt. - **Idempotency first:** scan the PR's existing comments for a prior bot - `♻️ … unrelated flake … on ` note for THIS head SHA — if one already - exists, record `already-annotated-flake PR # (head )` and stop - WITHOUT re-commenting (re-annotating the same flake every 12h sweep is noise and - burns the per-run comment budget other PRs need). Otherwise resolve the attempt - number from `C.effectiveAttempt` (the authoritative max(marker, bot-commit) - counter), omitting the `Attempt /10:` prefix only if it is somehow - indeterminate. **Dry-run gate (Step 0):** if `dry_run == "true"`, do NOT emit any comment — instead print the intended `♻️` unrelated-flake body to the run log, tally `dry-run: would-annotate-flake PR #`, and stop. Otherwise `add_comment` on PR #: `♻️ Attempt /10: red is - unrelated flake on leg(s) () on ; the fix itself is not - implicated. A maintainer re-run (/azp run ) should clear it.` Record - `annotated-flake PR # (head )` and stop. *(Round 1: human re-runs; - Round 2: auto re-trigger.)* + **Comment idempotency + dry-run suppress the ♻️ comment ONLY — neither skips Step + 3.6.** Scan the PR's existing comments for a prior bot `♻️ … unrelated flake … on + ` note for THIS head SHA; if one exists, set `SKIP_FLAKE_COMMENT = true`. + Resolve the attempt number from `C.effectiveAttempt` (the authoritative max(marker, + bot-commit) counter), omitting the `Attempt /10:` prefix only if it is + somehow indeterminate. Then post the flake note UNLESS suppressed: + - if `dry_run == "true"`: do NOT emit — print the intended `♻️` unrelated-flake body + to the run log and tally `dry-run: would-annotate-flake PR #`; + - else if `SKIP_FLAKE_COMMENT`: do NOT re-post — record `already-annotated-flake PR + # (head )` (re-annotating the same flake every 12h sweep is noise and + burns the per-run comment budget other PRs need); + - else `add_comment` on PR #: `♻️ Attempt /10: red is unrelated flake on + leg(s) () on ; the fix itself is not implicated. A + maintainer re-run (/azp run ) should clear it.` record `annotated-flake + PR # (head )`. + Then — in ALL of the above cases — run **Step 3.6** (target-test readiness gate) for + this PR before stopping: its `/azp`-gated target legs may conclude green on this SAME + head SHA on a LATER sweep, and Step 3.6 is the ONLY place the draft→ready flip + happens, so it MUST re-evaluate every sweep — never `stop` here before it. *(Round 1: + human re-runs; Round 2: auto re-trigger.)* - **Caused by the fix** (a failed leg still matches the original target signature, or the fix introduced a NEW failure): advance an attempt. - **Attempt count.** `attempt = C.effectiveAttempt` — the authoritative @@ -752,6 +856,188 @@ Run these gates in order — the FIRST that fires decides this cycle's outcome: from every prior commit on this branch**, emitted via the Step 5.6 ADVANCE path (push onto the same PR). +#### Step 3.6 — Target-test verification & mark-ready gate + +Reached from the Step 3 **green-surface** branch, the Step 4 **unrelated-flake** branch +(AFTER that branch has posted its comment), and the Step 3.5 **gate-2 target-focused +fast-path** (2a — while unrelated legs are still pending). Purpose: confirm the SPECIFIC +test(s) this PR was opened to fix now PASS in the PR's own CI, and — only when they do +— transition the draft `[ci-fix]` PR to **ready for review** so a maintainer sees a +validated fix instead of a draft. This is the ONLY place the loop flips draft→ready; it +is a state transition, **never** an approval or a merge (a human still reviews and +merges). Overall red on *unrelated* legs must NOT gate readiness — we validate the fix, +not the base branch's flakiness. + +**Preconditions** (ALL must hold; otherwise record `skipped: readiness N/A PR # +()` and stop this gate): +- `C.isDraft == true` — if the PR is already ready-for-review, the draft→ready + transition is already done; do NOT re-mark **and do NOT re-apply `p/0`**. `p/0` is applied + exactly ONCE, atomically with the draft→ready flip (T3 — all-or-nothing with the 🎯 audit + comment + mark-ready), so an already-ready loop PR that is MISSING `p/0` has almost + certainly had it **removed by a maintainer** de-prioritizing the PR — a pure triage action + the human-engagement guard (which inspects comments/reviews/commits, NOT label changes) + cannot see. Re-adding `p/0` every sweep would fight that maintainer indefinitely, violating + the loop's "never override a human" contract. So the loop does NOT reconcile the label: + record `already-ready PR #` and stop this gate. (Trade-off: on the rare occasion a + transient API error drops `p/0` at flip time *after* the T3 atomic pre-check passed, it is + not auto-re-added — but the PR is still ready-for-review with its 🎯 audit comment, and + re-adding `p/0` is a trivial manual action; that is strictly preferable to steam-rolling a + maintainer's deliberate de-prioritization.) +- The PR is unmistakably THIS workflow's own: `[ci-fix] ` title prefix AND the + `agentic-workflows` label (mirrors the safe-output lock; the compensating scope + control if the v0.80.9 compiler drops the declarative required-*). +- You reached this gate from Step 3 (green), Step 4 (**unrelated-flake**), or the Step 3.5 + gate-2 **target-focused fast-path** (2a) — i.e. the fix is NOT implicated in any red that + has concluded. If Step 4 classified a red as **caused by the fix** (ADVANCE), do NOT run + this gate — advance the attempt instead. + +**T1 — Identify the target test(s).** From the `[ci-scan]` issue signature the fix +addresses (and the PR's own diff), extract the fully-qualified test method name(s) the +fix targets — e.g. `SafeAreaShouldWorkOnAllShellTabs`. For a de-flake it is the +de-flaked test; for a product fix it is the originally-failing test(s). If NO specific +test can be identified (e.g. a product build-break rather than a test failure), this is a +**build-only fix** — there is no single test to validate cross-platform, so validate at +**whole-build** granularity rather than stopping (Step 3 only *comments*; Step 3.6 is the +sole draft→ready flip, so a build-only fix is undrafted HERE or not at all). We only reach +this gate from Step 3 (green) / Step 4 (unrelated-flake) / gate-2a, so the fix is not +implicated in any concluded red; additionally require, via the T2 timeline method on +`C.headSha`, that the primary build pipeline `maui-pr` (def 302) has CONCLUDED with EVERY +one of its platform build legs `succeeded`/`completed` and NONE failed/canceled or still +pending — the build is green on **every** platform, not just the originally-broken one (the +cross-platform guard, applied to the build instead of a test). A build-only fix is undrafted +ONLY on the strength of the auto-running `maui-pr` (def 302) whole-build green, which the +loop CAN observe: if the PR's `[ci-scan]` issue signature or its own diff indicates the +ORIGINATING failure was in a `/azp`-gated pipeline (`maui-pr-uitests` def 313 or +`maui-pr-devicetests` def 314) rather than `maui-pr` (def 302), do NOT undraft on +`maui-pr`-green alone — those pipelines do not auto-run on this PR (GITHUB_TOKEN cannot +trigger them), so a green `maui-pr` build is NOT evidence the gated build break is fixed; +record `skipped: build-only fix PR # targets gated pipeline () not run — +deferring to human` and stop this gate. Otherwise, set `TARGET := "the maui-pr build +(build-only fix — no single target test)"` and proceed to **T3** to mark ready. If any `maui-pr` build leg is still unconcluded, record `skipped: build-only fix PR +# not yet whole-build green (leg(s) pending)` and stop this gate WITHOUT marking +ready (a green subset is not enough — a still-pending build leg could yet fail). + +**T2 — Verify the target test's platform legs are green (anonymous, leg-level).** Per-test +outcomes come from the AzDO **test-results** API (`_apis/test/...`), which is **NOT reachable +anonymously — Hard-Rule 8**: those endpoints 302-redirect to a sign-in page on this runner, so +a `curl` returns an HTML redirect (not JSON) and every target test would look "not executed". +Validate instead at **leg granularity** using ONLY the anonymous `_apis/build/...` timeline +(Hard-Rule 8) — a `succeeded` category leg means every test in that category **that actually +ran** passed on that platform. This is a strong bar **only if the target test genuinely +executes**: a job can go green while one specific test is skipped at runtime (`[Ignore]`, +`Assert.Ignore()`, `Assert.Inconclusive()`, a `Skip=`/conditional `[Fact]`, an `#if`-out, or +an early `return` before the asserts). So before trusting a green leg as proof the target +passed, **confirm from the PR's OWN diff that the fix does NOT skip, ignore, disable, or +short-circuit the target test** (it adds no `[Ignore]`/`[Explicit]`, `Assert.Ignore`/ +`Assert.Inconclusive`, `Skip=`, category-exclusion, `#if`-out, or early `return` around the +target). If the fix could cause the target to be runtime-skipped rather than genuinely pass, +leg-level green is NOT sufficient evidence — record `skipped: target test may be +runtime-skipped by this fix (diff adds ); leg-green insufficient, deferring to human +on PR #` and stop this gate WITHOUT marking ready. Otherwise (the target genuinely runs +and asserts, as a de-flake or product fix does) a green category leg cannot hide a +target-test failure, so treat it as authoritative. + +Map the target test to the CI leg(s) that run it. A leg name encodes platform + test category +(e.g. `Android UITests SafeAreaEdges,Shadow`, `iOS UITests SafeAreaEdges,Shadow`, `macOS +UITests SafeAreaEdges,Shadow`, `Windows UITests SafeAreaEdges,Shadow`). Determine the target +test's UI-test category — its `[Category(UITestCategories.X)]` in the test/HostApp source, +visible in the PR diff or the test file — to know which leg-name substring identifies its legs; +for a device test, the per-platform device-test legs. + +Using the SAME build-discovery as Step 4 (filter AzDO builds by `branchName=refs/pull//merge` +or `sourceVersion == C.headSha`), read each build's **timeline** on `C.headSha` for the +pipeline(s) that RUN the target test — `maui-pr` (def 302) for unit/integration tests, +`maui-pr-uitests` (def 313) for Appium UI tests, `maui-pr-devicetests` (def 314) for device +tests: + +```bash +ORG=dnceng-public; PROJ=public; BUILD= +# Anonymous + Hard-Rule-8-compliant. NEVER call _apis/test/... (it 302-redirects to sign-in). +# Each type=="Job" record is one platform leg: result (succeeded/failed/canceled/null), +# state (completed/inProgress/pending), name (encodes " UITests "): +curl -s "https://dev.azure.com/$ORG/$PROJ/_apis/build/builds/$BUILD/timeline?api-version=7.1" \ + | tee /tmp/gh-aw/agent/timeline_${BUILD}.json \ + | jq -r '.records[] | select(.type=="Job") | "\(.result)\t\(.state)\t\(.name)"' +``` + +(The prefetch already exposes per-leg status in `C.failedLegs` / the checks context, and +`gh pr checks ` lists the same per-leg rows with platform+category in the name — use +whichever is handy; the build timeline is the authoritative cross-check on the exact build.) + +Treat a target test as **VALIDATED-GREEN** only if, on `C.headSha`, the leg that runs its +category is green on **every platform it runs on** — a fix that repairs one platform must not +silently regress the same test's leg on another, so a green leg on the originally-red platform +alone is NOT enough. Across the target pipeline's platform legs (for a UI test: the Android, +iOS, Windows, and macOS/MacCatalyst legs running the target's category; for a device test: each +device-test platform), require ALL of: +- the target's category leg is `result == "succeeded"` **and** `state == "completed"` on + **every** platform that runs it, AND +- that leg is `failed` / `canceled` / aborted on **NO** platform, AND +- **no platform is left unverified.** For each platform family the target pipeline covers, that + platform's category leg must have CONCLUDED (`state == "completed"`) on `C.headSha`. If a + platform simply has no leg for the target's category, the test does not run there — that is + fine, not a gap. But if a platform's category leg has **not concluded** (`state` is + `inProgress` / pending, or an `/azp`-gated `maui-pr-uitests` / `maui-pr-devicetests` leg that + has not been kicked), the test's status on that platform is UNKNOWN → the fix is NOT yet + validated across platforms: record `skipped: target test green on but + not yet verified on (leg(s) pending / need /azp run) on PR #` + and stop this gate WITHOUT marking ready. + +A target test whose category leg never concluded on ANY platform (**not executed** anywhere — +e.g. its `/azp`-gated pipeline has not been kicked) is likewise NOT validated: record `skipped: +target test not yet executed on PR # ( not run — needs /azp run)` and stop +this gate WITHOUT marking ready. Do NOT overclaim — a green *sibling* leg (a different category +on the same platform) is not the target's leg, and a green leg on one platform is not a pass on +the others. + +**T3 — Mark ready + report.** If EVERY target test is VALIDATED-GREEN. The 🎯 comment, +the mark-ready, and the `p/0` label are THREE SEPARATE safe-outputs — the comment +existing does NOT prove the mark-ready took effect, so they are tracked independently: + +- **Comment idempotency (dup-suppress only — never gates the mark-ready).** We only reach + T3 while `C.isDraft == true` (the precondition stops an already-ready PR before here). So + if a prior bot comment for THIS head that carries the leading `🎯` anchor AND the + contiguous phrase `validated green on ` already exists (BOTH T3 variants — + `🎯 Target test validated green on ` and the build-only `🎯 Build validated green on + ` — contain that exact contiguous phrase, so this recognizes both; keying on + `target test` alone would instead miss the build-only comment and re-post it every sweep. + **Require the `🎯` anchor** so the match can NEVER be loosened to a gapped + `validated…green…on` form that would false-positive on the Step 3 `✅ … validated — the + fix's CI is green on ` surface comment and wrongly suppress the audit 🎯), the comment + landed on an earlier sweep but the **mark-ready did NOT take + effect** (the PR is still a draft) — set `SUPPRESS_COMMENT = true` (do not re-post the + duplicate 🎯 comment) but STILL complete the mark-ready + label below. Never treat the + comment's existence as "already marked ready" while the PR is still a draft. Record this + case as `re-marking PR # (🎯 present; mark-ready did not take on a prior sweep)`. +- **Atomicity budget pre-check (Hard-Rule 7).** Determine the outputs this gate will emit: + `mark_pull_request_as_ready_for_review` + `add_labels` always, plus `add_comment` unless + `SUPPRESS_COMMENT`. Verify a free per-run slot remains in EACH bucket you are about to + use. If ANY required bucket is exhausted, emit NONE of them — record `skipped: per-run + cap reached; deferring mark-ready PR # to next cycle` and stop this gate. Never mark a + PR ready (or label it) unless its 🎯 audit comment is GUARANTEED to exist for this exact + head SHA — either posted in this same sweep, or (in the `SUPPRESS_COMMENT` re-marking case) + already present from a prior sweep. The outputs you DO emit either all land or all defer + together; the audit trail must never be absent, but it is NOT re-posted when it already exists. +- **Dry-run gate (Step 0):** if `dry_run == "true"`, emit NOTHING — print the intended + readiness comment and "would mark ready + add p/0" to the run log, tally `dry-run: + would-mark-ready PR #`, and stop. +- Otherwise emit for THIS PR number ``: + 1. `add_comment` (ONLY if not `SUPPRESS_COMMENT`): `🎯 Target test validated green on + — passed on ALL platforms it runs on (, + buildId ). — not + caused by this fix."> Transitioning this PR from draft to ready for review and adding + `p/0`; a maintainer still reviews and merges.` (For a **build-only fix** — the T1 + whole-build fallback, no single target test — phrase the first clause as `🎯 Build + validated green on — the maui-pr build passed on ALL platforms (buildId + ).` instead of naming a test.) + 2. `mark_pull_request_as_ready_for_review` with `reason:` a one-line justification + naming the validated test(s) and ``. + 3. `add_labels` with `labels: ["p/0"]` for PR # — put the now-review-ready fix into + the team's p/0 priority queue so it is triaged, not lost in the draft backlog. (If + the PR somehow already carries `p/0`, this is a harmless no-op.) +- Record `marked-ready PR # (target green on ALL platforms on , + labeled p/0)` and stop. + #### Step 3.5.R — Maintainer change-request response (Track C) Reached from Step 3.5 gate 0. Goal: when an **eligible human reviewer** (Hard-Rule @@ -1383,7 +1669,19 @@ Filed by [`ci-status-fix`](https://github.com/dotnet/maui/blob/main/.github/work ### Step 8 — Per-issue tally + end-of-run summary -Per issue, append one outcome line to `/tmp/gh-aw/agent/coverage.txt`: +Per issue, append **one** outcome line to `/tmp/gh-aw/agent/coverage.txt` — the +terminal outcome for the cycle. When one cycle produces a chained pair — a PR gets a +same-cycle precursor line and **then** a Step 3.6 readiness line (`marked-ready` on the +draft→ready flip, or `already-ready` on a later steady-state sweep of an already-ready +PR) — record ONLY the terminal Step 3.6 readiness line: it supersedes ANY same-cycle +non-readiness precursor for that PR, so an aggregator keying on one-line-per-issue never +double-counts. The precursor may be a Step 3 green line (`surfaced-green` on the first ✅, +or `already-surfaced` when it already exists) OR — when an unrelated-flake red and a +target-green coincide — a Step 4 `annotated-flake` line; either way the readiness line is +terminal, and the superseded precursor's signal survives in the PR's 🎯/♻️ comment so no +information is lost. This covers the flip cycle (`surfaced-green` → `marked-ready`), the +post-flip steady state (`already-surfaced` → `already-ready`), and the flake-coincident +flip (`annotated-flake` → `marked-ready`): ``` # main attempt- @@ -1394,8 +1692,14 @@ Per issue, append one outcome line to `/tmp/gh-aw/agent/coverage.txt`: `advance-PR # attempt /10` (ADVANCE mode — new commit pushed to the existing PR), `surfaced-green PR #` (fix's own CI went green; commented for review, did not advance), `annotated-flake PR #` (red was unrelated flake; -commented, attempt NOT burned), `waiting PR #` (CI not settled yet), -`dry-run: would-`, `skipped: `. (The +commented, attempt NOT burned), `marked-ready PR #` (the specific fixed test +was confirmed green in the PR's own CI; draft flipped to ready for review), +`already-surfaced PR #` (the fix's ✅ green comment already existed this sweep; +re-post suppressed — superseded by the Step 3.6 readiness line when the PR also flips +ready that cycle), `already-ready PR #` (terminal steady state — the PR was flipped +ready on a prior cycle and remains green on an unchanged head; no action taken, +supersedes `already-surfaced`), `waiting PR #` (CI not settled yet), +`dry-run: would-`, `skipped: `. (The `needs-human-PR` outcome is reserved for the deferred hand-off — Step 6 currently records a skip instead, so it is not emitted.)
` (fix's own CI went green; commented for review, did not advance), `annotated-flake PR #
` (red was unrelated flake; -commented, attempt NOT burned), `waiting PR #
` (CI not settled yet), -`dry-run: would-`, `skipped: `. (The +commented, attempt NOT burned), `marked-ready PR #` (the specific fixed test +was confirmed green in the PR's own CI; draft flipped to ready for review), +`already-surfaced PR #` (the fix's ✅ green comment already existed this sweep; +re-post suppressed — superseded by the Step 3.6 readiness line when the PR also flips +ready that cycle), `already-ready PR #` (terminal steady state — the PR was flipped +ready on a prior cycle and remains green on an unchanged head; no action taken, +supersedes `already-surfaced`), `waiting PR #` (CI not settled yet), +`dry-run: would-`, `skipped: `. (The `needs-human-PR` outcome is reserved for the deferred hand-off — Step 6 currently records a skip instead, so it is not emitted.) diff --git a/.github/workflows/ci-status-fix.lock.yml b/.github/workflows/ci-status-fix.lock.yml index 93dc5969cb0d..65511f2b3259 100644 --- a/.github/workflows/ci-status-fix.lock.yml +++ b/.github/workflows/ci-status-fix.lock.yml @@ -1,4 +1,4 @@ -# gh-aw-metadata: {"schema_version":"v4","frontmatter_hash":"f014161967589ebcaf1ac5ec6f3c88ee5a4388d28b55a07dc36c45b7b2aebab6","body_hash":"86c6b2d4c3df13da14c6a3c8b211381fcc9f97bb1951ff7b80588a2c5338e00c","compiler_version":"v0.80.9","strict":true,"agent_id":"copilot","agent_model":"claude-opus-4.8","engine_versions":{"copilot":"1.0.63"}} +# gh-aw-metadata: {"schema_version":"v4","frontmatter_hash":"b0ebf8a2f179900cfe3638e994b27a43cec52bdbdd2331dc1bd4d65762fa2d6b","body_hash":"1825e2b1f7c7bd88241bf37580655489360f9bebc806edd00837a52ce4ad83a0","compiler_version":"v0.80.9","strict":true,"agent_id":"copilot","agent_model":"claude-opus-4.8","engine_versions":{"copilot":"1.0.63"}} # gh-aw-manifest: {"version":1,"secrets":["COPILOT_PAT_0","COPILOT_PAT_1","COPILOT_PAT_2","COPILOT_PAT_3","COPILOT_PAT_4","COPILOT_PAT_5","COPILOT_PAT_6","COPILOT_PAT_7","COPILOT_PAT_8","COPILOT_PAT_9","GH_AW_CI_TRIGGER_TOKEN","GH_AW_GITHUB_MCP_SERVER_TOKEN","GH_AW_GITHUB_TOKEN","GITHUB_TOKEN"],"actions":[{"repo":"actions/checkout","sha":"34e114876b0b11c390a56381ad16ebd13914f8d5","version":"v4"},{"repo":"actions/checkout","sha":"9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0","version":"v7.0.0"},{"repo":"actions/checkout","sha":"de0fac2e4500dabe0009e67214ff5f5447ce83dd","version":"v6.0.2"},{"repo":"actions/download-artifact","sha":"3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c","version":"v8.0.1"},{"repo":"actions/github-script","sha":"3a2844b7e9c422d3c10d287c895573f7108da1b3","version":"v9.0.0"},{"repo":"actions/setup-node","sha":"48b55a011bda9f5d6aeb4c2d9c7362e8dae4041e","version":"v6.4.0"},{"repo":"actions/upload-artifact","sha":"043fb46d1a93c77aae656e7c1c64a875d1fc6a0a","version":"v7.0.1"},{"repo":"github/gh-aw-actions/setup","sha":"8c7d04ebf1ece56cd381446125da3e0f6896294a","version":"v0.80.9"}],"containers":[{"image":"ghcr.io/github/gh-aw-firewall/agent:0.27.7","digest":"sha256:aae231e4635c8999d039c132f1602d3df850fe9b84a00aa2b5ac981179b5661c","pinned_image":"ghcr.io/github/gh-aw-firewall/agent:0.27.7@sha256:aae231e4635c8999d039c132f1602d3df850fe9b84a00aa2b5ac981179b5661c"},{"image":"ghcr.io/github/gh-aw-firewall/api-proxy:0.27.7","digest":"sha256:009caf2e3d88fa77b64e9a03a95a228fc58db0f1701c6d324b29ba5a3c7c79b6","pinned_image":"ghcr.io/github/gh-aw-firewall/api-proxy:0.27.7@sha256:009caf2e3d88fa77b64e9a03a95a228fc58db0f1701c6d324b29ba5a3c7c79b6"},{"image":"ghcr.io/github/gh-aw-firewall/squid:0.27.7","digest":"sha256:deb1d4e19de62d51cee0508057a596a19315c3423ada4d675cad136dc8037c96","pinned_image":"ghcr.io/github/gh-aw-firewall/squid:0.27.7@sha256:deb1d4e19de62d51cee0508057a596a19315c3423ada4d675cad136dc8037c96"},{"image":"ghcr.io/github/gh-aw-mcpg:v0.3.27","digest":"sha256:fe984bddde4ec05d756d9043edb0a32912e6b7b72f6a121b1082f29221421cc7","pinned_image":"ghcr.io/github/gh-aw-mcpg:v0.3.27@sha256:fe984bddde4ec05d756d9043edb0a32912e6b7b72f6a121b1082f29221421cc7"},{"image":"ghcr.io/github/gh-aw-node","digest":"sha256:529d02eb970b1161aa25c593a9c3df57fdfad5a8add328cb3b6eccef66f3183b","pinned_image":"ghcr.io/github/gh-aw-node@sha256:529d02eb970b1161aa25c593a9c3df57fdfad5a8add328cb3b6eccef66f3183b"},{"image":"ghcr.io/github/github-mcp-server:v1.4.0","digest":"sha256:2afb26356481d1a350e14544a6e160f7f7ec1561a1ea309b823665abf0309036","pinned_image":"ghcr.io/github/github-mcp-server:v1.4.0@sha256:2afb26356481d1a350e14544a6e160f7f7ec1561a1ea309b823665abf0309036"}]} # This file was automatically generated by gh-aw (v0.80.9). DO NOT EDIT. To debug this workflow, load the skill at https://github.com/github/gh-aw/blob/main/debug.md # @@ -37,7 +37,9 @@ # humans (the open tracking issue is the hand-off surface; a dedicated # [ci-fix][needs-human] PR is planned but currently deferred — see Step 6). CI on # the PR is kicked by a human `/azp run` for now; the loop watches, classifies, -# and re-fixes autonomously between kicks. +# and re-fixes autonomously between kicks. When the SPECIFIC test a PR fixed is +# confirmed green in that PR's own CI, the loop marks the draft PR ready for review +# (a state transition only — it never approves or merges). # Never mutes tests, but # de-flakes genuinely flaky ones (deterministic synchronization, no retries / # timeout bumps). Always skips visual-regression / screenshot issues. @@ -273,24 +275,24 @@ jobs: run: | bash "${RUNNER_TEMP}/gh-aw/actions/create_prompt_first.sh" { - cat << 'GH_AW_PROMPT_25cf75111b670167_EOF' + cat << 'GH_AW_PROMPT_2ac1d0b4083428f8_EOF' - GH_AW_PROMPT_25cf75111b670167_EOF + GH_AW_PROMPT_2ac1d0b4083428f8_EOF cat "${RUNNER_TEMP}/gh-aw/prompts/xpia.md" cat "${RUNNER_TEMP}/gh-aw/prompts/temp_folder_prompt.md" cat "${RUNNER_TEMP}/gh-aw/prompts/markdown.md" cat "${RUNNER_TEMP}/gh-aw/prompts/safe_outputs_prompt.md" - cat << 'GH_AW_PROMPT_25cf75111b670167_EOF' + cat << 'GH_AW_PROMPT_2ac1d0b4083428f8_EOF' - Tools: add_comment(max:3), create_pull_request(max:3), update_pull_request(max:3), push_to_pull_request_branch(max:3), missing_tool, missing_data, noop - GH_AW_PROMPT_25cf75111b670167_EOF + Tools: add_comment(max:6), create_pull_request(max:3), update_pull_request(max:3), mark_pull_request_as_ready_for_review(max:3), add_labels(max:3), push_to_pull_request_branch(max:3), missing_tool, missing_data, noop + GH_AW_PROMPT_2ac1d0b4083428f8_EOF cat "${RUNNER_TEMP}/gh-aw/prompts/safe_outputs_create_pull_request.md" cat "${RUNNER_TEMP}/gh-aw/prompts/safe_outputs_push_to_pr_branch.md" - cat << 'GH_AW_PROMPT_25cf75111b670167_EOF' + cat << 'GH_AW_PROMPT_2ac1d0b4083428f8_EOF' - GH_AW_PROMPT_25cf75111b670167_EOF + GH_AW_PROMPT_2ac1d0b4083428f8_EOF cat "${RUNNER_TEMP}/gh-aw/prompts/mcp_cli_tools_prompt.md" - cat << 'GH_AW_PROMPT_25cf75111b670167_EOF' + cat << 'GH_AW_PROMPT_2ac1d0b4083428f8_EOF' The following GitHub context information is available for this workflow: {{#if github.actor}} @@ -332,12 +334,12 @@ jobs: stop immediately and report the limitation rather than spending turns trying to work around it. - GH_AW_PROMPT_25cf75111b670167_EOF + GH_AW_PROMPT_2ac1d0b4083428f8_EOF cat "${RUNNER_TEMP}/gh-aw/prompts/github_mcp_tools_with_safeoutputs_prompt.md" - cat << 'GH_AW_PROMPT_25cf75111b670167_EOF' + cat << 'GH_AW_PROMPT_2ac1d0b4083428f8_EOF' {{#runtime-import .github/workflows/ci-status-fix.md}} - GH_AW_PROMPT_25cf75111b670167_EOF + GH_AW_PROMPT_2ac1d0b4083428f8_EOF } > "$GH_AW_PROMPT" - name: Interpolate variables and render templates uses: actions/github-script@3a2844b7e9c422d3c10d287c895573f7108da1b3 # v9.0.0 @@ -554,16 +556,18 @@ jobs: mkdir -p "${RUNNER_TEMP}/gh-aw/safeoutputs" mkdir -p /tmp/gh-aw/safeoutputs mkdir -p /tmp/gh-aw/mcp-logs/safeoutputs - cat > "${RUNNER_TEMP}/gh-aw/safeoutputs/config.json" << 'GH_AW_SAFE_OUTPUTS_CONFIG_fa2a69fad5fd0485_EOF' - {"add_comment":{"discussions":false,"max":3,"required_labels":["agentic-workflows"],"required_title_prefix":"[ci-fix] ","target":"*"},"create_pull_request":{"allowed_base_branches":["main"],"allowed_branches":["ci-fix/**"],"allowed_files":["src/Core/**","src/Controls/**","src/Essentials/**","src/BlazorWebView/**","src/TestUtils/**","src/Templates/**","**/PublicAPI.Unshipped.txt"],"base_branch":"main","draft":true,"labels":["agentic-workflows"],"max":3,"max_patch_files":100,"max_patch_size":4096,"protect_top_level_dot_folders":true,"protected_files":["package.json","bun.lockb","bunfig.toml","deno.json","deno.jsonc","deno.lock","global.json","NuGet.Config","Directory.Packages.props","mix.exs","mix.lock","go.mod","go.sum","stack.yaml","stack.yaml.lock","pom.xml","build.gradle","build.gradle.kts","settings.gradle","settings.gradle.kts","gradle.properties","package-lock.json","yarn.lock","pnpm-lock.yaml","npm-shrinkwrap.json","requirements.txt","Pipfile","Pipfile.lock","pyproject.toml","setup.py","setup.cfg","Gemfile","Gemfile.lock","uv.lock","CODEOWNERS","DESIGN.md","README.md","CONTRIBUTING.md","CHANGELOG.md","SECURITY.md","CODE_OF_CONDUCT.md","AGENTS.md","CLAUDE.md","GEMINI.md"],"protected_files_policy":"request_review","title_prefix":"[ci-fix] "},"create_report_incomplete_issue":{},"missing_data":{},"missing_tool":{},"noop":{"max":1,"report-as-issue":"true"},"push_to_pull_request_branch":{"allowed_files":["src/Core/**","src/Controls/**","src/Essentials/**","src/BlazorWebView/**","src/TestUtils/**","src/Templates/**","**/PublicAPI.Unshipped.txt"],"if_no_changes":"warn","max":3,"max_patch_size":4096,"protect_top_level_dot_folders":true,"protected_files":["package.json","bun.lockb","bunfig.toml","deno.json","deno.jsonc","deno.lock","global.json","NuGet.Config","Directory.Packages.props","mix.exs","mix.lock","go.mod","go.sum","stack.yaml","stack.yaml.lock","pom.xml","build.gradle","build.gradle.kts","settings.gradle","settings.gradle.kts","gradle.properties","package-lock.json","yarn.lock","pnpm-lock.yaml","npm-shrinkwrap.json","requirements.txt","Pipfile","Pipfile.lock","pyproject.toml","setup.py","setup.cfg","Gemfile","Gemfile.lock","uv.lock","CODEOWNERS","DESIGN.md","README.md","CONTRIBUTING.md","CHANGELOG.md","SECURITY.md","CODE_OF_CONDUCT.md","AGENTS.md","CLAUDE.md","GEMINI.md"],"required_labels":["agentic-workflows"],"target":"*","title_prefix":"[ci-fix] "},"report_incomplete":{},"update_pull_request":{"allow_body":true,"allow_title":false,"max":3,"target":"*","update_branch":false}} - GH_AW_SAFE_OUTPUTS_CONFIG_fa2a69fad5fd0485_EOF + cat > "${RUNNER_TEMP}/gh-aw/safeoutputs/config.json" << 'GH_AW_SAFE_OUTPUTS_CONFIG_0644bf1434ca0071_EOF' + {"add_comment":{"discussions":false,"max":6,"required_labels":["agentic-workflows"],"required_title_prefix":"[ci-fix] ","target":"*"},"add_labels":{"allowed":["p/0"],"max":3,"required_labels":["agentic-workflows"],"required_title_prefix":"[ci-fix] ","target":"*"},"create_pull_request":{"allowed_base_branches":["main"],"allowed_branches":["ci-fix/**"],"allowed_files":["src/Core/**","src/Controls/**","src/Essentials/**","src/BlazorWebView/**","src/TestUtils/**","src/Templates/**","**/PublicAPI.Unshipped.txt"],"base_branch":"main","draft":true,"labels":["agentic-workflows"],"max":3,"max_patch_files":100,"max_patch_size":4096,"protect_top_level_dot_folders":true,"protected_files":["package.json","bun.lockb","bunfig.toml","deno.json","deno.jsonc","deno.lock","global.json","NuGet.Config","Directory.Packages.props","mix.exs","mix.lock","go.mod","go.sum","stack.yaml","stack.yaml.lock","pom.xml","build.gradle","build.gradle.kts","settings.gradle","settings.gradle.kts","gradle.properties","package-lock.json","yarn.lock","pnpm-lock.yaml","npm-shrinkwrap.json","requirements.txt","Pipfile","Pipfile.lock","pyproject.toml","setup.py","setup.cfg","Gemfile","Gemfile.lock","uv.lock","CODEOWNERS","DESIGN.md","README.md","CONTRIBUTING.md","CHANGELOG.md","SECURITY.md","CODE_OF_CONDUCT.md","AGENTS.md","CLAUDE.md","GEMINI.md"],"protected_files_policy":"request_review","title_prefix":"[ci-fix] "},"create_report_incomplete_issue":{},"mark_pull_request_as_ready_for_review":{"max":3,"required_labels":["agentic-workflows"],"required_title_prefix":"[ci-fix] ","target":"*"},"missing_data":{},"missing_tool":{},"noop":{"max":1,"report-as-issue":"true"},"push_to_pull_request_branch":{"allowed_files":["src/Core/**","src/Controls/**","src/Essentials/**","src/BlazorWebView/**","src/TestUtils/**","src/Templates/**","**/PublicAPI.Unshipped.txt"],"if_no_changes":"warn","max":3,"max_patch_size":4096,"protect_top_level_dot_folders":true,"protected_files":["package.json","bun.lockb","bunfig.toml","deno.json","deno.jsonc","deno.lock","global.json","NuGet.Config","Directory.Packages.props","mix.exs","mix.lock","go.mod","go.sum","stack.yaml","stack.yaml.lock","pom.xml","build.gradle","build.gradle.kts","settings.gradle","settings.gradle.kts","gradle.properties","package-lock.json","yarn.lock","pnpm-lock.yaml","npm-shrinkwrap.json","requirements.txt","Pipfile","Pipfile.lock","pyproject.toml","setup.py","setup.cfg","Gemfile","Gemfile.lock","uv.lock","CODEOWNERS","DESIGN.md","README.md","CONTRIBUTING.md","CHANGELOG.md","SECURITY.md","CODE_OF_CONDUCT.md","AGENTS.md","CLAUDE.md","GEMINI.md"],"required_labels":["agentic-workflows"],"target":"*","title_prefix":"[ci-fix] "},"report_incomplete":{},"update_pull_request":{"allow_body":true,"allow_title":false,"max":3,"target":"*","update_branch":false}} + GH_AW_SAFE_OUTPUTS_CONFIG_0644bf1434ca0071_EOF - name: Generate Safe Outputs Tools env: GH_AW_TOOLS_META_JSON: | { "description_suffixes": { - "add_comment": " CONSTRAINTS: Maximum 3 comment(s) can be added. Target: *. Supports reply_to_id for discussion threading.", + "add_comment": " CONSTRAINTS: Maximum 6 comment(s) can be added. Target: *. Supports reply_to_id for discussion threading.", + "add_labels": " CONSTRAINTS: Maximum 3 label(s) can be added. Only these labels are allowed: [\"p/0\"]. Target: *.", "create_pull_request": " CONSTRAINTS: Maximum 3 pull request(s) can be created. Title will be prefixed with \"[ci-fix] \". Labels [\"agentic-workflows\"] will be automatically added. Only these labels are allowed: [\"agentic-workflows\"]. PRs will be created as drafts.", + "mark_pull_request_as_ready_for_review": " CONSTRAINTS: Maximum 3 pull request(s) can be marked as ready for review.", "push_to_pull_request_branch": " CONSTRAINTS: Maximum 3 push(es) can be made. The target pull request title must start with \"[ci-fix] \".", "update_pull_request": " CONSTRAINTS: Maximum 3 pull request(s) can be updated. Target: *." }, @@ -594,6 +598,25 @@ jobs: } } }, + "add_labels": { + "defaultMax": 5, + "fields": { + "item_number": { + "issueNumberOrTemporaryId": true + }, + "labels": { + "required": true, + "type": "array", + "itemType": "string", + "itemSanitize": true, + "itemMaxLength": 128 + }, + "repo": { + "type": "string", + "maxLength": 256 + } + } + }, "create_pull_request": { "defaultMax": 1, "fields": { @@ -635,6 +658,24 @@ jobs: } } }, + "mark_pull_request_as_ready_for_review": { + "defaultMax": 1, + "fields": { + "pull_request_number": { + "issueOrPRNumber": true + }, + "reason": { + "required": true, + "type": "string", + "sanitize": true, + "maxLength": 65000 + }, + "repo": { + "type": "string", + "maxLength": 256 + } + } + }, "missing_data": { "defaultMax": 20, "fields": { @@ -1494,7 +1535,7 @@ jobs: uses: actions/github-script@3a2844b7e9c422d3c10d287c895573f7108da1b3 # v9.0.0 env: WORKFLOW_NAME: "CI Failure Fixer (main)" - WORKFLOW_DESCRIPTION: "Periodic pass over open ci-scan tracking issues filed by the main-branch CI\nfailure scanner (.github/workflows/ci-status-main.md). This workflow targets\nthe `main` branch EXCLUSIVELY: it processes only issues labelled ci-scan and\nopens every PR against main. (The net11.0 branch is handled by the parallel\n.github/workflows/ci-status-fix-net11.md — the two are split because gh-aw can\nonly transport a fix relative to ONE static base branch per workflow, and the\nmain↔net11.0 divergence exceeds gh-aw's 10 MB transport-patch cap.) The fixer\nopens ONE draft [ci-fix] PR per actionable issue against main, then WATCHES\nthat PR's own CI on later runs: when the fix's CI comes back red and the red is\ncaused by the fix itself, it pushes a fresh follow-up fix onto the SAME PR\nbranch (never a second PR) — up to 10 attempts — then stops and defers to\nhumans (the open tracking issue is the hand-off surface; a dedicated\n[ci-fix][needs-human] PR is planned but currently deferred — see Step 6). CI on\nthe PR is kicked by a human `/azp run` for now; the loop watches, classifies,\nand re-fixes autonomously between kicks.\nNever mutes tests, but\nde-flakes genuinely flaky ones (deterministic synchronization, no retries /\ntimeout bumps). Always skips visual-regression / screenshot issues." + WORKFLOW_DESCRIPTION: "Periodic pass over open ci-scan tracking issues filed by the main-branch CI\nfailure scanner (.github/workflows/ci-status-main.md). This workflow targets\nthe `main` branch EXCLUSIVELY: it processes only issues labelled ci-scan and\nopens every PR against main. (The net11.0 branch is handled by the parallel\n.github/workflows/ci-status-fix-net11.md — the two are split because gh-aw can\nonly transport a fix relative to ONE static base branch per workflow, and the\nmain↔net11.0 divergence exceeds gh-aw's 10 MB transport-patch cap.) The fixer\nopens ONE draft [ci-fix] PR per actionable issue against main, then WATCHES\nthat PR's own CI on later runs: when the fix's CI comes back red and the red is\ncaused by the fix itself, it pushes a fresh follow-up fix onto the SAME PR\nbranch (never a second PR) — up to 10 attempts — then stops and defers to\nhumans (the open tracking issue is the hand-off surface; a dedicated\n[ci-fix][needs-human] PR is planned but currently deferred — see Step 6). CI on\nthe PR is kicked by a human `/azp run` for now; the loop watches, classifies,\nand re-fixes autonomously between kicks. When the SPECIFIC test a PR fixed is\nconfirmed green in that PR's own CI, the loop marks the draft PR ready for review\n(a state transition only — it never approves or merges).\nNever mutes tests, but\nde-flakes genuinely flaky ones (deterministic synchronization, no retries /\ntimeout bumps). Always skips visual-regression / screenshot issues." HAS_PATCH: ${{ needs.agent.outputs.has_patch }} with: script: | @@ -1910,7 +1951,7 @@ jobs: GH_AW_ALLOWED_DOMAINS: "*.blob.core.windows.net,*.githubusercontent.com,api.business.githubcopilot.com,api.enterprise.githubcopilot.com,api.github.com,api.githubcopilot.com,api.individual.githubcopilot.com,api.snapcraft.io,archive.ubuntu.com,azure.archive.ubuntu.com,codeload.github.com,crl.geotrust.com,crl.globalsign.com,crl.identrust.com,crl.sectigo.com,crl.thawte.com,crl.usertrust.com,crl.verisign.com,crl3.digicert.com,crl4.digicert.com,crls.ssl.com,dev.azure.com,docs.github.com,github-cloud.githubusercontent.com,github-cloud.s3.amazonaws.com,github.blog,github.com,github.githubassets.com,helix.dot.net,host.docker.internal,json-schema.org,json.schemastore.org,keyserver.ubuntu.com,lfs.github.com,objects.githubusercontent.com,ocsp.digicert.com,ocsp.geotrust.com,ocsp.globalsign.com,ocsp.identrust.com,ocsp.sectigo.com,ocsp.ssl.com,ocsp.thawte.com,ocsp.usertrust.com,ocsp.verisign.com,packagecloud.io,packages.cloud.google.com,packages.microsoft.com,patch-diff.githubusercontent.com,ppa.launchpad.net,raw.githubusercontent.com,registry.npmjs.org,s.symcb.com,s.symcd.com,security.ubuntu.com,telemetry.enterprise.githubcopilot.com,ts-crl.ws.symantec.com,ts-ocsp.ws.symantec.com,www.googleapis.com" GITHUB_SERVER_URL: ${{ github.server_url }} GITHUB_API_URL: ${{ github.api_url }} - GH_AW_SAFE_OUTPUTS_HANDLER_CONFIG: "{\"add_comment\":{\"discussions\":false,\"max\":3,\"required_labels\":[\"agentic-workflows\"],\"required_title_prefix\":\"[ci-fix] \",\"target\":\"*\"},\"create_pull_request\":{\"allowed_base_branches\":[\"main\"],\"allowed_branches\":[\"ci-fix/**\"],\"allowed_files\":[\"src/Core/**\",\"src/Controls/**\",\"src/Essentials/**\",\"src/BlazorWebView/**\",\"src/TestUtils/**\",\"src/Templates/**\",\"**/PublicAPI.Unshipped.txt\"],\"base_branch\":\"main\",\"draft\":true,\"labels\":[\"agentic-workflows\"],\"max\":3,\"max_patch_files\":100,\"max_patch_size\":4096,\"protect_top_level_dot_folders\":true,\"protected_files\":[\"package.json\",\"bun.lockb\",\"bunfig.toml\",\"deno.json\",\"deno.jsonc\",\"deno.lock\",\"global.json\",\"NuGet.Config\",\"Directory.Packages.props\",\"mix.exs\",\"mix.lock\",\"go.mod\",\"go.sum\",\"stack.yaml\",\"stack.yaml.lock\",\"pom.xml\",\"build.gradle\",\"build.gradle.kts\",\"settings.gradle\",\"settings.gradle.kts\",\"gradle.properties\",\"package-lock.json\",\"yarn.lock\",\"pnpm-lock.yaml\",\"npm-shrinkwrap.json\",\"requirements.txt\",\"Pipfile\",\"Pipfile.lock\",\"pyproject.toml\",\"setup.py\",\"setup.cfg\",\"Gemfile\",\"Gemfile.lock\",\"uv.lock\",\"CODEOWNERS\",\"DESIGN.md\",\"README.md\",\"CONTRIBUTING.md\",\"CHANGELOG.md\",\"SECURITY.md\",\"CODE_OF_CONDUCT.md\",\"AGENTS.md\",\"CLAUDE.md\",\"GEMINI.md\"],\"protected_files_policy\":\"request_review\",\"title_prefix\":\"[ci-fix] \"},\"create_report_incomplete_issue\":{},\"missing_data\":{},\"missing_tool\":{},\"noop\":{\"max\":1,\"report-as-issue\":\"true\"},\"push_to_pull_request_branch\":{\"allowed_files\":[\"src/Core/**\",\"src/Controls/**\",\"src/Essentials/**\",\"src/BlazorWebView/**\",\"src/TestUtils/**\",\"src/Templates/**\",\"**/PublicAPI.Unshipped.txt\"],\"if_no_changes\":\"warn\",\"max\":3,\"max_patch_size\":4096,\"protect_top_level_dot_folders\":true,\"protected_files\":[\"package.json\",\"bun.lockb\",\"bunfig.toml\",\"deno.json\",\"deno.jsonc\",\"deno.lock\",\"global.json\",\"NuGet.Config\",\"Directory.Packages.props\",\"mix.exs\",\"mix.lock\",\"go.mod\",\"go.sum\",\"stack.yaml\",\"stack.yaml.lock\",\"pom.xml\",\"build.gradle\",\"build.gradle.kts\",\"settings.gradle\",\"settings.gradle.kts\",\"gradle.properties\",\"package-lock.json\",\"yarn.lock\",\"pnpm-lock.yaml\",\"npm-shrinkwrap.json\",\"requirements.txt\",\"Pipfile\",\"Pipfile.lock\",\"pyproject.toml\",\"setup.py\",\"setup.cfg\",\"Gemfile\",\"Gemfile.lock\",\"uv.lock\",\"CODEOWNERS\",\"DESIGN.md\",\"README.md\",\"CONTRIBUTING.md\",\"CHANGELOG.md\",\"SECURITY.md\",\"CODE_OF_CONDUCT.md\",\"AGENTS.md\",\"CLAUDE.md\",\"GEMINI.md\"],\"required_labels\":[\"agentic-workflows\"],\"target\":\"*\",\"title_prefix\":\"[ci-fix] \"},\"report_incomplete\":{},\"update_pull_request\":{\"allow_body\":true,\"allow_title\":false,\"max\":3,\"target\":\"*\",\"update_branch\":false}}" + GH_AW_SAFE_OUTPUTS_HANDLER_CONFIG: "{\"add_comment\":{\"discussions\":false,\"max\":6,\"required_labels\":[\"agentic-workflows\"],\"required_title_prefix\":\"[ci-fix] \",\"target\":\"*\"},\"add_labels\":{\"allowed\":[\"p/0\"],\"max\":3,\"required_labels\":[\"agentic-workflows\"],\"required_title_prefix\":\"[ci-fix] \",\"target\":\"*\"},\"create_pull_request\":{\"allowed_base_branches\":[\"main\"],\"allowed_branches\":[\"ci-fix/**\"],\"allowed_files\":[\"src/Core/**\",\"src/Controls/**\",\"src/Essentials/**\",\"src/BlazorWebView/**\",\"src/TestUtils/**\",\"src/Templates/**\",\"**/PublicAPI.Unshipped.txt\"],\"base_branch\":\"main\",\"draft\":true,\"labels\":[\"agentic-workflows\"],\"max\":3,\"max_patch_files\":100,\"max_patch_size\":4096,\"protect_top_level_dot_folders\":true,\"protected_files\":[\"package.json\",\"bun.lockb\",\"bunfig.toml\",\"deno.json\",\"deno.jsonc\",\"deno.lock\",\"global.json\",\"NuGet.Config\",\"Directory.Packages.props\",\"mix.exs\",\"mix.lock\",\"go.mod\",\"go.sum\",\"stack.yaml\",\"stack.yaml.lock\",\"pom.xml\",\"build.gradle\",\"build.gradle.kts\",\"settings.gradle\",\"settings.gradle.kts\",\"gradle.properties\",\"package-lock.json\",\"yarn.lock\",\"pnpm-lock.yaml\",\"npm-shrinkwrap.json\",\"requirements.txt\",\"Pipfile\",\"Pipfile.lock\",\"pyproject.toml\",\"setup.py\",\"setup.cfg\",\"Gemfile\",\"Gemfile.lock\",\"uv.lock\",\"CODEOWNERS\",\"DESIGN.md\",\"README.md\",\"CONTRIBUTING.md\",\"CHANGELOG.md\",\"SECURITY.md\",\"CODE_OF_CONDUCT.md\",\"AGENTS.md\",\"CLAUDE.md\",\"GEMINI.md\"],\"protected_files_policy\":\"request_review\",\"title_prefix\":\"[ci-fix] \"},\"create_report_incomplete_issue\":{},\"mark_pull_request_as_ready_for_review\":{\"max\":3,\"required_labels\":[\"agentic-workflows\"],\"required_title_prefix\":\"[ci-fix] \",\"target\":\"*\"},\"missing_data\":{},\"missing_tool\":{},\"noop\":{\"max\":1,\"report-as-issue\":\"true\"},\"push_to_pull_request_branch\":{\"allowed_files\":[\"src/Core/**\",\"src/Controls/**\",\"src/Essentials/**\",\"src/BlazorWebView/**\",\"src/TestUtils/**\",\"src/Templates/**\",\"**/PublicAPI.Unshipped.txt\"],\"if_no_changes\":\"warn\",\"max\":3,\"max_patch_size\":4096,\"protect_top_level_dot_folders\":true,\"protected_files\":[\"package.json\",\"bun.lockb\",\"bunfig.toml\",\"deno.json\",\"deno.jsonc\",\"deno.lock\",\"global.json\",\"NuGet.Config\",\"Directory.Packages.props\",\"mix.exs\",\"mix.lock\",\"go.mod\",\"go.sum\",\"stack.yaml\",\"stack.yaml.lock\",\"pom.xml\",\"build.gradle\",\"build.gradle.kts\",\"settings.gradle\",\"settings.gradle.kts\",\"gradle.properties\",\"package-lock.json\",\"yarn.lock\",\"pnpm-lock.yaml\",\"npm-shrinkwrap.json\",\"requirements.txt\",\"Pipfile\",\"Pipfile.lock\",\"pyproject.toml\",\"setup.py\",\"setup.cfg\",\"Gemfile\",\"Gemfile.lock\",\"uv.lock\",\"CODEOWNERS\",\"DESIGN.md\",\"README.md\",\"CONTRIBUTING.md\",\"CHANGELOG.md\",\"SECURITY.md\",\"CODE_OF_CONDUCT.md\",\"AGENTS.md\",\"CLAUDE.md\",\"GEMINI.md\"],\"required_labels\":[\"agentic-workflows\"],\"target\":\"*\",\"title_prefix\":\"[ci-fix] \"},\"report_incomplete\":{},\"update_pull_request\":{\"allow_body\":true,\"allow_title\":false,\"max\":3,\"target\":\"*\",\"update_branch\":false}}" GH_AW_CI_TRIGGER_TOKEN: ${{ secrets.GH_AW_CI_TRIGGER_TOKEN }} with: github-token: ${{ secrets.GH_AW_GITHUB_TOKEN || secrets.GITHUB_TOKEN }} diff --git a/.github/workflows/ci-status-fix.md b/.github/workflows/ci-status-fix.md index d3227e25170c..9c2f46054029 100644 --- a/.github/workflows/ci-status-fix.md +++ b/.github/workflows/ci-status-fix.md @@ -15,7 +15,9 @@ description: | humans (the open tracking issue is the hand-off surface; a dedicated [ci-fix][needs-human] PR is planned but currently deferred — see Step 6). CI on the PR is kicked by a human `/azp run` for now; the loop watches, classifies, - and re-fixes autonomously between kicks. + and re-fixes autonomously between kicks. When the SPECIFIC test a PR fixed is + confirmed green in that PR's own CI, the loop marks the draft PR ready for review + (a state transition only — it never approves or merges). Never mutes tests, but de-flakes genuinely flaky ones (deterministic synchronization, no retries / timeout bumps). Always skips visual-regression / screenshot issues. @@ -239,8 +241,15 @@ safe-outputs: # PER-RUN total (not per-PR): a sweep may surface several green PRs and/or # annotate several flaky reds in one run. At max:1 all but the first comment # were silently dropped, starving the workflow's #1 value (surfacing green PRs - # for review). Raised to 3 to match the per-run throughput of the other outputs. - max: 3 + # for review). Sized to 6 (not 3) because a single green DRAFT PR that gets + # flipped ready in one sweep spends TWO comment slots — the Step 3 ✅ surface + # comment (which still names the /azp-gated legs a human must kick, so it is NOT + # redundant with 🎯) AND the Step 3.6 T3 🎯 readiness comment. At max:3 the shared + # bucket drained after ~1 draft flip and T3's atomicity pre-check then deferred + # every further mark-ready even while the mark-ready/add_labels buckets (also 3) + # sat idle; 6 lets ~3 draft flips (2 comments each) land per sweep, so the + # mark-ready:3 / add_labels:3 caps are actually reachable. + max: 6 target: "*" # Hard constraint (defense-in-depth): only comment on THIS workflow's own # ci-fix PRs — [ci-fix] title prefix AND agentic-workflows label. Mirrors the @@ -261,13 +270,51 @@ safe-outputs: # never the title, so disable title rewrites — this compiles to allow_title:false # and removes any ability to retitle an arbitrary PR. title: false - # NOTE: gh-aw v0.79.8 does NOT emit required-title-prefix/required-labels into + # NOTE: gh-aw v0.80.9 (the pinned compiler) does NOT emit required-title-prefix/required-labels into # the compiled config for update-pull-request (verified against the lock — it # silently drops them, unlike add-comment / push-to-pull-request-branch which # honor them). So which-PR scoping here relies on prompt Hard-Rule 6 + # min-integrity:approved; the residual is body-marker edits only (no code, no # merge, no close), capped at max:3. Revisit required-* if a future gh-aw # compiler emits them for this output. + mark-pull-request-as-ready-for-review: + # Flip a validated draft [ci-fix] PR to ready-for-review once the SPECIFIC test + # this PR was opened to fix is confirmed green in the PR's OWN CI (Step 3.6) — + # even when unrelated legs are red. The workflow is schedule/dispatch-triggered + # (no triggering-PR context), so target "*" lets the agent name the PR number it + # validated. This is a state transition ONLY: it never approves and never merges — + # a human still reviews and merges. + # PER-RUN total (not per-PR): a sweep may validate several PRs' target tests green. + max: 3 + target: "*" + # Hard constraint (defense-in-depth): only ever un-draft THIS workflow's own + # ci-fix PRs — [ci-fix] title prefix AND agentic-workflows label — so a confused + # or prompt-injected agent cannot mark an arbitrary PR ready. (If the v0.80.9 + # compiler silently drops these — as documented for update-pull-request above — + # the Step 3.6 preconditions + min-integrity:approved are the compensating scope + # controls; verify against the lock after compiling.) + required-title-prefix: "[ci-fix] " + required-labels: [agentic-workflows] + add-labels: + # Apply the p/0 priority label to a [ci-fix] PR at the exact moment the loop flips it + # from draft to ready-for-review (Step 3.6 T3) — so a validated, review-ready fix lands + # in the team's p/0 triage queue instead of sitting unseen in the draft backlog. Paired + # 1:1 with mark-pull-request-as-ready-for-review; target "*" lets the agent name the PR + # it just validated. + # allowed = HARD allowlist: the agent may ONLY ever add p/0, nothing else. This caps the + # blast radius of a confused/prompt-injected agent to exactly one benign priority label — + # it can never apply a downstream-triggering or destructive label. + allowed: [p/0] + # PER-RUN total (not per-PR): a sweep may mark several PRs ready in one run; each adds + # one label, so this matches the mark-ready per-run cap (3). + max: 3 + target: "*" + # Hard constraint (defense-in-depth): only ever label THIS workflow's own ci-fix PRs — + # [ci-fix] title prefix AND agentic-workflows label. (If the v0.80.9 compiler drops these + # — as documented for update-pull-request above — the Step 3.6 preconditions + + # min-integrity:approved + the allowed:[p/0] allowlist are the compensating controls.) + required-title-prefix: "[ci-fix] " + required-labels: [agentic-workflows] timeout-minutes: 90 @@ -339,33 +386,48 @@ through `safe-outputs`. 5. **One issue = one outcome per run.** Exactly one of: respond to a maintainer's change-request on the open PR (Track C, Step 3.5.R); open the first fix/help/ de-flake PR; advance an existing PR by one attempt (push a follow-up fix); - surface a validated-green PR for review; annotate an unrelated-flake red; wait + surface a validated-green PR for review; mark a target-validated draft PR + ready-for-review (Step 3.6 T3 — the terminal outcome that supersedes the same + run's surface/annotate precursor line), or record its `already-ready` + steady-state no-op; annotate an unrelated-flake red; wait (CI not yet settled); or a recorded skip (the dedicated needs-human PR is deferred — the attempt cap records a skip instead; see Step 6). Always prefer advancing or opening a PR over a skip when a non-mute diff is producible. 6. **All writes via `safe-outputs`.** Allowed outputs: `create_pull_request` (first attempt only), `push_to_pull_request_branch` (advance an existing PR by one attempt), `update_pull_request` (bump the attempt marker / refresh the - prior-attempts table), and `add_comment` (progress notes on the `[ci-fix]` PR - ONLY). NEVER comment on the tracking issue (issues are locked by + prior-attempts table), `add_comment` (progress notes on the `[ci-fix]` PR + ONLY), `mark_pull_request_as_ready_for_review` (flip a target-validated draft + `[ci-fix]` PR from draft to ready — Step 3.6 T3 only), and `add_labels` (add + ONLY the `p/0` label, ONLY on that same draft→ready transition — Step 3.6 T3). + NEVER comment on the tracking issue (issues are locked by `.github/workflows/ci-scan-lock-issues.yml`) — `add_comment` targets the PR. No `gh pr create`, no manual `git push`. **Defense-in-depth:** `add_comment` and `update_pull_request` use `target: "*"` (the agent supplies the PR number). - `add_comment` is now config-locked to `required-title-prefix: "[ci-fix] "` + + `add_comment`, `mark_pull_request_as_ready_for_review`, and `add_labels` are + config-locked to `required-title-prefix: "[ci-fix] "` + `required-labels: [agentic-workflows]` (hard-enforced by the handler, same as - `push_to_pull_request_branch`), so a comment can only ever land on THIS - workflow's own PRs. `update_pull_request` CANNOT be config-locked to a - title/label in gh-aw v0.79.8 (the compiler silently drops `required-*` for that - output — verified against the lock), so it keeps `title: false` (no retitles; - body-marker edits only) plus this prompt-level guard. Before emitting either, - VERIFY the target PR carries BOTH the `[ci-fix]` title prefix AND the - `agentic-workflows` label; never comment on or edit the body of any PR that - lacks both. + `push_to_pull_request_branch`), and `add_labels` additionally has an + `allowed: [p/0]` allowlist so it can ONLY ever add `p/0` — so these can only + ever land on THIS workflow's own PRs. `update_pull_request` CANNOT be + config-locked to a title/label in gh-aw v0.80.9 (the compiler silently drops + `required-*` for that output — verified against the lock), so it keeps + `title: false` (no retitles; body-marker edits only) plus this prompt-level + guard. Before emitting ANY of these, VERIFY the target PR carries BOTH the + `[ci-fix]` title prefix AND the `agentic-workflows` label; never comment on, + edit, un-draft, or label any PR that lacks both. 7. **Per-run safe-output caps (per RUN, not per PR).** Each output type is capped per run: `create_pull_request` 3, `push_to_pull_request_branch` 3, - `add_comment` 3, `update_pull_request` 3. Note an ADVANCE spends one push **and** - one update **and** one comment, so ≤ 3 advances/run; surfacing a green or - annotating a flake spends one comment. When a bucket is exhausted, do NOT keep + `add_comment` 6, `update_pull_request` 3, `mark_pull_request_as_ready_for_review` + 3, `add_labels` 3 (counts LABELS, not calls). Note an ADVANCE spends one push + **and** one update **and** one comment, so ≤ 3 advances/run; surfacing a green or + annotating a flake spends one comment; a **mark-ready (Step 3.6 T3)** of a + still-draft PR spends TWO comments (the Step 3 ✅ surface **and** the T3 🎯 + readiness note) **and** one mark-ready **and** one label as an ALL-OR-NOTHING set + (T3 pre-checks those buckets and defers the whole PR if any is exhausted — never + mark a PR ready without its 🎯 audit comment). `add_comment` is therefore sized 6 + (not 3) so ~3 draft flips, at 2 comments each, can land in one sweep instead of the + shared comment bucket starving the otherwise-idle mark-ready/add_labels buckets. When a bucket is exhausted, do NOT keep emitting (extras are silently dropped) — record `skipped: per-run cap reached; deferring PR # to next cycle` for each remaining PR so the drop is a deliberate, logged decision. The continuous loop picks the deferred PRs up next @@ -409,8 +471,9 @@ For every open tracking issue in scope, converge on exactly one outcome: | Open first help-wanted draft `[ci-fix]` PR | No open PR yet; a plausible candidate exists but cannot be runner-validated (device/UI tests) (attempt 1) | | Open first de-flake draft `[ci-fix]` PR | No open PR yet; failure is intermittent (green-on-retry) from a genuine test-quality defect; a deterministic-synchronization fix is producible (attempt 1, Step 4.7 bucket b) | | Advance an existing PR (push attempt N+1) | Open PR's own CI settled red *because of the fix itself*, no human engaged, marker < 10 — push a NEW distinct fix onto the same branch (Step 3.5 → 5.6) | -| Surface a validated-green PR | Open PR's own CI settled green — comment "validated, attempt N/10; ready for review" and do NOT advance (Step 3.5) | -| Annotate an unrelated-flake red | Open PR is red only on baseline-flake / unrelated legs — comment which leg needs a re-run; do NOT burn an attempt (Step 3.5) | +| Surface a validated-green PR | Open PR's own CI settled green — comment "validated, attempt N/10; ready for review" and do NOT advance (Step 3.5), then mark the draft PR ready for review once the fixed test is confirmed green (Step 3.6) | +| Annotate an unrelated-flake red | Open PR is red only on baseline-flake / unrelated legs — comment which leg needs a re-run; do NOT burn an attempt (Step 3.5); if the SPECIFIC fixed test is confirmed green on that build, still mark the draft PR ready for review (Step 3.6) | +| Mark a validated PR ready for review | The specific test this PR fixed is confirmed green in the PR's own CI (even if unrelated legs are red) — comment the target-test result and transition the draft PR to ready for review; never approve or merge (Step 3.6) | | Wait | Open PR's CI is pending / not yet settled on the current head SHA — do nothing this cycle (Step 3.5) | | Hand-off skip (attempt cap) | Marker == 10 and the signature still reproduces — stop and defer to humans; the dedicated `[ci-fix][needs-human]` PR is planned but currently deferred (Step 6) | | Recorded skip | Visual-regression, human engaged, already-handled, fixed-in-latest-build, infra-flake, out-of-bounds, only-mute-available, or no novel approach producible | @@ -685,14 +748,34 @@ Run these gates in order — the FIRST that fires decides this cycle's outcome: 1. **Human engaged.** If `C.humanEngaged` → `skipped: human engaged on PR #; deferring` and stop. Never fight a human reviewer — the intentional hand-off boundary still holds the moment a person touches the PR. -2. **CI not settled / unknown / incomplete prefetch.** If `C.checksSettled == false` - OR `C.overallConclusion` is `pending`, `neutral`, or `unknown`, OR - `C.dataComplete == false` → the fix's CI has not finished on the current head SHA - (`C.headSha`), or the prefetch could not fully read this PR (a partial read can - understate `humanEngaged` / `attempt`). `skipped: PR # CI pending / prefetch - incomplete on ; waiting` and stop. *(Round 1: this is where a - maintainer `/azp run` is awaited — the `/azp`-gated uitests/devicetests legs will - not have run until a human kicks them.)* +2. **CI not settled — but validate the target test first (target-focused readiness).** + If `C.checksSettled == false` OR `C.overallConclusion` is `pending`, `neutral`, or + `unknown`, OR `C.dataComplete == false` → the *overall* build has not finished on the + current head SHA (`C.headSha`), or the prefetch could not fully read this PR (a partial + read can understate `humanEngaged` / `attempt`). Unrelated legs still draining must NOT + keep an already-proven fix parked as a draft, so before waiting, try the target-focused + fast-path: + - **2a — Target-green fast-path (draft `[ci-fix]` PRs only).** If `C.dataComplete == + true` AND the Step 3.6 preconditions hold (draft PR, `[ci-fix] ` title prefix AND the + `agentic-workflows` label), run Step 3.6 **T1–T2** against `C.headSha` now: identify + the target test(s) and read the PR's OWN build **timeline / per-leg status** (anonymous + `_apis/build` — never `_apis/test`, per Hard-Rule 8). If EVERY target test is + **VALIDATED-GREEN** (per T2's all-platform definition below — the target's category leg + is `succeeded` on every platform that runs it, failed on none, and NO target platform + family the pipeline covers is still pending/unconcluded, per T2's "no platform left + unverified" rule; a platform that simply has no leg for the target's category — the test + doesn't run there — is fine, not a gap) AND every leg in + `C.failedLegs` that has ALREADY concluded classifies as **unrelated flake** (Step 4 / + Step 4.7 method — the fix is implicated in NO completed red), then the fix is proven + regardless of unrelated *pending* legs → run **Step 3.6 T3** (mark ready + 🎯 comment) + and stop. This early-out NEVER advances an attempt and NEVER acts on a red that could + be the fix's fault. + - **2b — Otherwise WAIT.** If the target test has not yet executed (pending/absent on + its leg), or a completed red is (or may be) caused by the fix, or `C.dataComplete == + false`, or this PR has no identifiable target test → `skipped: PR # CI pending / + target not yet validated on ; waiting` and stop. *(Round 1: this is where a + maintainer `/azp run` is awaited — the `/azp`-gated uitests/devicetests legs will not + have run until a human kicks them.)* 3. **Green → surface for review.** If `C.overallConclusion == "success"`: the checks that RAN are green. **Primary-gate check first:** the deterministic `success` verdict only certifies "at least one green check, nothing failing or @@ -704,18 +787,31 @@ Run these gates in order — the FIRST that fires decides this cycle's outcome: # primary CI gate (maui-pr) not green on ; waiting` and stop — do NOT surface. (The `/azp`-gated `maui-pr-uitests` (def 313) / `maui-pr-devicetests` (def 314) legs MAY still be un-run — that is expected and is named below, not a - reason to withhold the surface.) **Idempotency:** scan the PR's existing comments - for a prior bot `✅ … validated … on ` note for THIS head SHA — if one - already exists, record `already-surfaced PR # (head )` and stop - WITHOUT re-commenting (re-surfacing the same green every tick is noise and burns - the per-run comment budget other PRs need). Otherwise resolve the attempt number from - `C.effectiveAttempt` (the authoritative max(marker, bot-commit) counter; omit the - number only if it is somehow indeterminate). **Dry-run gate (Step 0):** if `dry_run == "true"`, do NOT emit any comment — instead print the intended `✅` validated-green body to the run log, tally `dry-run: would-surface-green PR #`, and stop. Otherwise `add_comment` on PR #: `✅ Attempt /10 validated - — the fix's CI is green on . Ready for human review.` - Name any `/azp`-gated legs (uitests def 313 / devicetests def 314) that have not - run and still need a maintainer `/azp run`. Do NOT advance. Record - `surfaced-green PR # (attempt /10)` and stop. *(This directly - attacks the real bottleneck — no reviews — so it is the highest-value outcome.)* + reason to withhold the surface.) **Comment idempotency + dry-run suppress the ✅ + comment ONLY — neither skips Step 3.6.** Scan the PR's existing comments for a prior + bot `✅ … validated … on ` note for THIS head SHA; if one exists, set + `SKIP_SURFACE_COMMENT = true`. Resolve the attempt number from `C.effectiveAttempt` + (the authoritative max(marker, bot-commit) counter; omit the number only if it is + somehow indeterminate). Then post the surface comment UNLESS suppressed: + - if `dry_run == "true"`: do NOT emit — print the intended `✅` validated-green body to + the run log and tally `dry-run: would-surface-green PR #`; + - else if `SKIP_SURFACE_COMMENT`: do NOT re-post — record `already-surfaced PR # + (head )` (re-surfacing the same green every tick is noise and burns the + per-run comment budget other PRs need); + - else `add_comment` on PR #: `✅ Attempt /10 validated — the fix's CI is + green on .` naming any `/azp`-gated legs (uitests def 313 / devicetests def + 314) that have not run and still need a maintainer `/azp run`; record `surfaced-green + PR # (attempt /10)`. (Do NOT assert "ready for human review" on a PR that + is still a draft — Step 3.6, not this comment, owns the draft→ready flip and posts its + own 🎯 announcement when it fires.) + Do NOT advance. Then — in ALL of the above cases — run **Step 3.6** (target-test + readiness gate) for this PR before stopping: its `/azp`-gated target legs may conclude + green on this SAME head SHA on a LATER sweep (an `/azp run` adds no commit), and Step + 3.6 is the ONLY place the draft→ready flip happens, so it MUST re-evaluate every sweep — + never `stop` here before it. *(This directly attacks the real bottleneck — no reviews — + so it is the highest-value outcome.)* 4. **Red → classify caused-by-fix vs unrelated-flake.** If `C.overallConclusion == "failure"`, analyze the PR's OWN failing build (NOT `main`): find the AzDO `maui-pr` build for this PR (filter builds by `branchName=refs/pull//merge`, @@ -723,18 +819,26 @@ Run these gates in order — the FIRST that fires decides this cycle's outcome: method and Step 4.7 flake buckets to `C.failedLegs`. - **Unrelated flake only** (every failed leg is known-flaky / infra / a pre-existing baseline red NOT introduced by the fix): do NOT burn an attempt. - **Idempotency first:** scan the PR's existing comments for a prior bot - `♻️ … unrelated flake … on ` note for THIS head SHA — if one already - exists, record `already-annotated-flake PR # (head )` and stop - WITHOUT re-commenting (re-annotating the same flake every 12h sweep is noise and - burns the per-run comment budget other PRs need). Otherwise resolve the attempt - number from `C.effectiveAttempt` (the authoritative max(marker, bot-commit) - counter), omitting the `Attempt /10:` prefix only if it is somehow - indeterminate. **Dry-run gate (Step 0):** if `dry_run == "true"`, do NOT emit any comment — instead print the intended `♻️` unrelated-flake body to the run log, tally `dry-run: would-annotate-flake PR #`, and stop. Otherwise `add_comment` on PR #: `♻️ Attempt /10: red is - unrelated flake on leg(s) () on ; the fix itself is not - implicated. A maintainer re-run (/azp run ) should clear it.` Record - `annotated-flake PR # (head )` and stop. *(Round 1: human re-runs; - Round 2: auto re-trigger.)* + **Comment idempotency + dry-run suppress the ♻️ comment ONLY — neither skips Step + 3.6.** Scan the PR's existing comments for a prior bot `♻️ … unrelated flake … on + ` note for THIS head SHA; if one exists, set `SKIP_FLAKE_COMMENT = true`. + Resolve the attempt number from `C.effectiveAttempt` (the authoritative max(marker, + bot-commit) counter), omitting the `Attempt /10:` prefix only if it is + somehow indeterminate. Then post the flake note UNLESS suppressed: + - if `dry_run == "true"`: do NOT emit — print the intended `♻️` unrelated-flake body + to the run log and tally `dry-run: would-annotate-flake PR #`; + - else if `SKIP_FLAKE_COMMENT`: do NOT re-post — record `already-annotated-flake PR + # (head )` (re-annotating the same flake every 12h sweep is noise and + burns the per-run comment budget other PRs need); + - else `add_comment` on PR #: `♻️ Attempt /10: red is unrelated flake on + leg(s) () on ; the fix itself is not implicated. A + maintainer re-run (/azp run ) should clear it.` record `annotated-flake + PR # (head )`. + Then — in ALL of the above cases — run **Step 3.6** (target-test readiness gate) for + this PR before stopping: its `/azp`-gated target legs may conclude green on this SAME + head SHA on a LATER sweep, and Step 3.6 is the ONLY place the draft→ready flip + happens, so it MUST re-evaluate every sweep — never `stop` here before it. *(Round 1: + human re-runs; Round 2: auto re-trigger.)* - **Caused by the fix** (a failed leg still matches the original target signature, or the fix introduced a NEW failure): advance an attempt. - **Attempt count.** `attempt = C.effectiveAttempt` — the authoritative @@ -752,6 +856,188 @@ Run these gates in order — the FIRST that fires decides this cycle's outcome: from every prior commit on this branch**, emitted via the Step 5.6 ADVANCE path (push onto the same PR). +#### Step 3.6 — Target-test verification & mark-ready gate + +Reached from the Step 3 **green-surface** branch, the Step 4 **unrelated-flake** branch +(AFTER that branch has posted its comment), and the Step 3.5 **gate-2 target-focused +fast-path** (2a — while unrelated legs are still pending). Purpose: confirm the SPECIFIC +test(s) this PR was opened to fix now PASS in the PR's own CI, and — only when they do +— transition the draft `[ci-fix]` PR to **ready for review** so a maintainer sees a +validated fix instead of a draft. This is the ONLY place the loop flips draft→ready; it +is a state transition, **never** an approval or a merge (a human still reviews and +merges). Overall red on *unrelated* legs must NOT gate readiness — we validate the fix, +not the base branch's flakiness. + +**Preconditions** (ALL must hold; otherwise record `skipped: readiness N/A PR # +()` and stop this gate): +- `C.isDraft == true` — if the PR is already ready-for-review, the draft→ready + transition is already done; do NOT re-mark **and do NOT re-apply `p/0`**. `p/0` is applied + exactly ONCE, atomically with the draft→ready flip (T3 — all-or-nothing with the 🎯 audit + comment + mark-ready), so an already-ready loop PR that is MISSING `p/0` has almost + certainly had it **removed by a maintainer** de-prioritizing the PR — a pure triage action + the human-engagement guard (which inspects comments/reviews/commits, NOT label changes) + cannot see. Re-adding `p/0` every sweep would fight that maintainer indefinitely, violating + the loop's "never override a human" contract. So the loop does NOT reconcile the label: + record `already-ready PR #` and stop this gate. (Trade-off: on the rare occasion a + transient API error drops `p/0` at flip time *after* the T3 atomic pre-check passed, it is + not auto-re-added — but the PR is still ready-for-review with its 🎯 audit comment, and + re-adding `p/0` is a trivial manual action; that is strictly preferable to steam-rolling a + maintainer's deliberate de-prioritization.) +- The PR is unmistakably THIS workflow's own: `[ci-fix] ` title prefix AND the + `agentic-workflows` label (mirrors the safe-output lock; the compensating scope + control if the v0.80.9 compiler drops the declarative required-*). +- You reached this gate from Step 3 (green), Step 4 (**unrelated-flake**), or the Step 3.5 + gate-2 **target-focused fast-path** (2a) — i.e. the fix is NOT implicated in any red that + has concluded. If Step 4 classified a red as **caused by the fix** (ADVANCE), do NOT run + this gate — advance the attempt instead. + +**T1 — Identify the target test(s).** From the `[ci-scan]` issue signature the fix +addresses (and the PR's own diff), extract the fully-qualified test method name(s) the +fix targets — e.g. `SafeAreaShouldWorkOnAllShellTabs`. For a de-flake it is the +de-flaked test; for a product fix it is the originally-failing test(s). If NO specific +test can be identified (e.g. a product build-break rather than a test failure), this is a +**build-only fix** — there is no single test to validate cross-platform, so validate at +**whole-build** granularity rather than stopping (Step 3 only *comments*; Step 3.6 is the +sole draft→ready flip, so a build-only fix is undrafted HERE or not at all). We only reach +this gate from Step 3 (green) / Step 4 (unrelated-flake) / gate-2a, so the fix is not +implicated in any concluded red; additionally require, via the T2 timeline method on +`C.headSha`, that the primary build pipeline `maui-pr` (def 302) has CONCLUDED with EVERY +one of its platform build legs `succeeded`/`completed` and NONE failed/canceled or still +pending — the build is green on **every** platform, not just the originally-broken one (the +cross-platform guard, applied to the build instead of a test). A build-only fix is undrafted +ONLY on the strength of the auto-running `maui-pr` (def 302) whole-build green, which the +loop CAN observe: if the PR's `[ci-scan]` issue signature or its own diff indicates the +ORIGINATING failure was in a `/azp`-gated pipeline (`maui-pr-uitests` def 313 or +`maui-pr-devicetests` def 314) rather than `maui-pr` (def 302), do NOT undraft on +`maui-pr`-green alone — those pipelines do not auto-run on this PR (GITHUB_TOKEN cannot +trigger them), so a green `maui-pr` build is NOT evidence the gated build break is fixed; +record `skipped: build-only fix PR # targets gated pipeline () not run — +deferring to human` and stop this gate. Otherwise, set `TARGET := "the maui-pr build +(build-only fix — no single target test)"` and proceed to **T3** to mark ready. If any `maui-pr` build leg is still unconcluded, record `skipped: build-only fix PR +# not yet whole-build green (leg(s) pending)` and stop this gate WITHOUT marking +ready (a green subset is not enough — a still-pending build leg could yet fail). + +**T2 — Verify the target test's platform legs are green (anonymous, leg-level).** Per-test +outcomes come from the AzDO **test-results** API (`_apis/test/...`), which is **NOT reachable +anonymously — Hard-Rule 8**: those endpoints 302-redirect to a sign-in page on this runner, so +a `curl` returns an HTML redirect (not JSON) and every target test would look "not executed". +Validate instead at **leg granularity** using ONLY the anonymous `_apis/build/...` timeline +(Hard-Rule 8) — a `succeeded` category leg means every test in that category **that actually +ran** passed on that platform. This is a strong bar **only if the target test genuinely +executes**: a job can go green while one specific test is skipped at runtime (`[Ignore]`, +`Assert.Ignore()`, `Assert.Inconclusive()`, a `Skip=`/conditional `[Fact]`, an `#if`-out, or +an early `return` before the asserts). So before trusting a green leg as proof the target +passed, **confirm from the PR's OWN diff that the fix does NOT skip, ignore, disable, or +short-circuit the target test** (it adds no `[Ignore]`/`[Explicit]`, `Assert.Ignore`/ +`Assert.Inconclusive`, `Skip=`, category-exclusion, `#if`-out, or early `return` around the +target). If the fix could cause the target to be runtime-skipped rather than genuinely pass, +leg-level green is NOT sufficient evidence — record `skipped: target test may be +runtime-skipped by this fix (diff adds ); leg-green insufficient, deferring to human +on PR #` and stop this gate WITHOUT marking ready. Otherwise (the target genuinely runs +and asserts, as a de-flake or product fix does) a green category leg cannot hide a +target-test failure, so treat it as authoritative. + +Map the target test to the CI leg(s) that run it. A leg name encodes platform + test category +(e.g. `Android UITests SafeAreaEdges,Shadow`, `iOS UITests SafeAreaEdges,Shadow`, `macOS +UITests SafeAreaEdges,Shadow`, `Windows UITests SafeAreaEdges,Shadow`). Determine the target +test's UI-test category — its `[Category(UITestCategories.X)]` in the test/HostApp source, +visible in the PR diff or the test file — to know which leg-name substring identifies its legs; +for a device test, the per-platform device-test legs. + +Using the SAME build-discovery as Step 4 (filter AzDO builds by `branchName=refs/pull//merge` +or `sourceVersion == C.headSha`), read each build's **timeline** on `C.headSha` for the +pipeline(s) that RUN the target test — `maui-pr` (def 302) for unit/integration tests, +`maui-pr-uitests` (def 313) for Appium UI tests, `maui-pr-devicetests` (def 314) for device +tests: + +```bash +ORG=dnceng-public; PROJ=public; BUILD= +# Anonymous + Hard-Rule-8-compliant. NEVER call _apis/test/... (it 302-redirects to sign-in). +# Each type=="Job" record is one platform leg: result (succeeded/failed/canceled/null), +# state (completed/inProgress/pending), name (encodes " UITests "): +curl -s "https://dev.azure.com/$ORG/$PROJ/_apis/build/builds/$BUILD/timeline?api-version=7.1" \ + | tee /tmp/gh-aw/agent/timeline_${BUILD}.json \ + | jq -r '.records[] | select(.type=="Job") | "\(.result)\t\(.state)\t\(.name)"' +``` + +(The prefetch already exposes per-leg status in `C.failedLegs` / the checks context, and +`gh pr checks ` lists the same per-leg rows with platform+category in the name — use +whichever is handy; the build timeline is the authoritative cross-check on the exact build.) + +Treat a target test as **VALIDATED-GREEN** only if, on `C.headSha`, the leg that runs its +category is green on **every platform it runs on** — a fix that repairs one platform must not +silently regress the same test's leg on another, so a green leg on the originally-red platform +alone is NOT enough. Across the target pipeline's platform legs (for a UI test: the Android, +iOS, Windows, and macOS/MacCatalyst legs running the target's category; for a device test: each +device-test platform), require ALL of: +- the target's category leg is `result == "succeeded"` **and** `state == "completed"` on + **every** platform that runs it, AND +- that leg is `failed` / `canceled` / aborted on **NO** platform, AND +- **no platform is left unverified.** For each platform family the target pipeline covers, that + platform's category leg must have CONCLUDED (`state == "completed"`) on `C.headSha`. If a + platform simply has no leg for the target's category, the test does not run there — that is + fine, not a gap. But if a platform's category leg has **not concluded** (`state` is + `inProgress` / pending, or an `/azp`-gated `maui-pr-uitests` / `maui-pr-devicetests` leg that + has not been kicked), the test's status on that platform is UNKNOWN → the fix is NOT yet + validated across platforms: record `skipped: target test green on but + not yet verified on (leg(s) pending / need /azp run) on PR #` + and stop this gate WITHOUT marking ready. + +A target test whose category leg never concluded on ANY platform (**not executed** anywhere — +e.g. its `/azp`-gated pipeline has not been kicked) is likewise NOT validated: record `skipped: +target test not yet executed on PR # ( not run — needs /azp run)` and stop +this gate WITHOUT marking ready. Do NOT overclaim — a green *sibling* leg (a different category +on the same platform) is not the target's leg, and a green leg on one platform is not a pass on +the others. + +**T3 — Mark ready + report.** If EVERY target test is VALIDATED-GREEN. The 🎯 comment, +the mark-ready, and the `p/0` label are THREE SEPARATE safe-outputs — the comment +existing does NOT prove the mark-ready took effect, so they are tracked independently: + +- **Comment idempotency (dup-suppress only — never gates the mark-ready).** We only reach + T3 while `C.isDraft == true` (the precondition stops an already-ready PR before here). So + if a prior bot comment for THIS head that carries the leading `🎯` anchor AND the + contiguous phrase `validated green on ` already exists (BOTH T3 variants — + `🎯 Target test validated green on ` and the build-only `🎯 Build validated green on + ` — contain that exact contiguous phrase, so this recognizes both; keying on + `target test` alone would instead miss the build-only comment and re-post it every sweep. + **Require the `🎯` anchor** so the match can NEVER be loosened to a gapped + `validated…green…on` form that would false-positive on the Step 3 `✅ … validated — the + fix's CI is green on ` surface comment and wrongly suppress the audit 🎯), the comment + landed on an earlier sweep but the **mark-ready did NOT take + effect** (the PR is still a draft) — set `SUPPRESS_COMMENT = true` (do not re-post the + duplicate 🎯 comment) but STILL complete the mark-ready + label below. Never treat the + comment's existence as "already marked ready" while the PR is still a draft. Record this + case as `re-marking PR # (🎯 present; mark-ready did not take on a prior sweep)`. +- **Atomicity budget pre-check (Hard-Rule 7).** Determine the outputs this gate will emit: + `mark_pull_request_as_ready_for_review` + `add_labels` always, plus `add_comment` unless + `SUPPRESS_COMMENT`. Verify a free per-run slot remains in EACH bucket you are about to + use. If ANY required bucket is exhausted, emit NONE of them — record `skipped: per-run + cap reached; deferring mark-ready PR # to next cycle` and stop this gate. Never mark a + PR ready (or label it) unless its 🎯 audit comment is GUARANTEED to exist for this exact + head SHA — either posted in this same sweep, or (in the `SUPPRESS_COMMENT` re-marking case) + already present from a prior sweep. The outputs you DO emit either all land or all defer + together; the audit trail must never be absent, but it is NOT re-posted when it already exists. +- **Dry-run gate (Step 0):** if `dry_run == "true"`, emit NOTHING — print the intended + readiness comment and "would mark ready + add p/0" to the run log, tally `dry-run: + would-mark-ready PR #`, and stop. +- Otherwise emit for THIS PR number ``: + 1. `add_comment` (ONLY if not `SUPPRESS_COMMENT`): `🎯 Target test validated green on + — passed on ALL platforms it runs on (, + buildId ). — not + caused by this fix."> Transitioning this PR from draft to ready for review and adding + `p/0`; a maintainer still reviews and merges.` (For a **build-only fix** — the T1 + whole-build fallback, no single target test — phrase the first clause as `🎯 Build + validated green on — the maui-pr build passed on ALL platforms (buildId + ).` instead of naming a test.) + 2. `mark_pull_request_as_ready_for_review` with `reason:` a one-line justification + naming the validated test(s) and ``. + 3. `add_labels` with `labels: ["p/0"]` for PR # — put the now-review-ready fix into + the team's p/0 priority queue so it is triaged, not lost in the draft backlog. (If + the PR somehow already carries `p/0`, this is a harmless no-op.) +- Record `marked-ready PR # (target green on ALL platforms on , + labeled p/0)` and stop. + #### Step 3.5.R — Maintainer change-request response (Track C) Reached from Step 3.5 gate 0. Goal: when an **eligible human reviewer** (Hard-Rule @@ -1383,7 +1669,19 @@ Filed by [`ci-status-fix`](https://github.com/dotnet/maui/blob/main/.github/work ### Step 8 — Per-issue tally + end-of-run summary -Per issue, append one outcome line to `/tmp/gh-aw/agent/coverage.txt`: +Per issue, append **one** outcome line to `/tmp/gh-aw/agent/coverage.txt` — the +terminal outcome for the cycle. When one cycle produces a chained pair — a PR gets a +same-cycle precursor line and **then** a Step 3.6 readiness line (`marked-ready` on the +draft→ready flip, or `already-ready` on a later steady-state sweep of an already-ready +PR) — record ONLY the terminal Step 3.6 readiness line: it supersedes ANY same-cycle +non-readiness precursor for that PR, so an aggregator keying on one-line-per-issue never +double-counts. The precursor may be a Step 3 green line (`surfaced-green` on the first ✅, +or `already-surfaced` when it already exists) OR — when an unrelated-flake red and a +target-green coincide — a Step 4 `annotated-flake` line; either way the readiness line is +terminal, and the superseded precursor's signal survives in the PR's 🎯/♻️ comment so no +information is lost. This covers the flip cycle (`surfaced-green` → `marked-ready`), the +post-flip steady state (`already-surfaced` → `already-ready`), and the flake-coincident +flip (`annotated-flake` → `marked-ready`): ``` # main attempt- @@ -1394,8 +1692,14 @@ Per issue, append one outcome line to `/tmp/gh-aw/agent/coverage.txt`: `advance-PR # attempt /10` (ADVANCE mode — new commit pushed to the existing PR), `surfaced-green PR #` (fix's own CI went green; commented for review, did not advance), `annotated-flake PR #` (red was unrelated flake; -commented, attempt NOT burned), `waiting PR #` (CI not settled yet), -`dry-run: would-`, `skipped: `. (The +commented, attempt NOT burned), `marked-ready PR #` (the specific fixed test +was confirmed green in the PR's own CI; draft flipped to ready for review), +`already-surfaced PR #` (the fix's ✅ green comment already existed this sweep; +re-post suppressed — superseded by the Step 3.6 readiness line when the PR also flips +ready that cycle), `already-ready PR #` (terminal steady state — the PR was flipped +ready on a prior cycle and remains green on an unchanged head; no action taken, +supersedes `already-surfaced`), `waiting PR #` (CI not settled yet), +`dry-run: would-`, `skipped: `. (The `needs-human-PR` outcome is reserved for the deferred hand-off — Step 6 currently records a skip instead, so it is not emitted.)
` (the specific fixed test +was confirmed green in the PR's own CI; draft flipped to ready for review), +`already-surfaced PR #
` (the fix's ✅ green comment already existed this sweep; +re-post suppressed — superseded by the Step 3.6 readiness line when the PR also flips +ready that cycle), `already-ready PR #
` (terminal steady state — the PR was flipped +ready on a prior cycle and remains green on an unchanged head; no action taken, +supersedes `already-surfaced`), `waiting PR #
` (CI not settled yet), +`dry-run: would-`, `skipped: `. (The `needs-human-PR` outcome is reserved for the deferred hand-off — Step 6 currently records a skip instead, so it is not emitted.) diff --git a/.github/workflows/ci-status-fix.lock.yml b/.github/workflows/ci-status-fix.lock.yml index 93dc5969cb0d..65511f2b3259 100644 --- a/.github/workflows/ci-status-fix.lock.yml +++ b/.github/workflows/ci-status-fix.lock.yml @@ -1,4 +1,4 @@ -# gh-aw-metadata: {"schema_version":"v4","frontmatter_hash":"f014161967589ebcaf1ac5ec6f3c88ee5a4388d28b55a07dc36c45b7b2aebab6","body_hash":"86c6b2d4c3df13da14c6a3c8b211381fcc9f97bb1951ff7b80588a2c5338e00c","compiler_version":"v0.80.9","strict":true,"agent_id":"copilot","agent_model":"claude-opus-4.8","engine_versions":{"copilot":"1.0.63"}} +# gh-aw-metadata: {"schema_version":"v4","frontmatter_hash":"b0ebf8a2f179900cfe3638e994b27a43cec52bdbdd2331dc1bd4d65762fa2d6b","body_hash":"1825e2b1f7c7bd88241bf37580655489360f9bebc806edd00837a52ce4ad83a0","compiler_version":"v0.80.9","strict":true,"agent_id":"copilot","agent_model":"claude-opus-4.8","engine_versions":{"copilot":"1.0.63"}} # gh-aw-manifest: {"version":1,"secrets":["COPILOT_PAT_0","COPILOT_PAT_1","COPILOT_PAT_2","COPILOT_PAT_3","COPILOT_PAT_4","COPILOT_PAT_5","COPILOT_PAT_6","COPILOT_PAT_7","COPILOT_PAT_8","COPILOT_PAT_9","GH_AW_CI_TRIGGER_TOKEN","GH_AW_GITHUB_MCP_SERVER_TOKEN","GH_AW_GITHUB_TOKEN","GITHUB_TOKEN"],"actions":[{"repo":"actions/checkout","sha":"34e114876b0b11c390a56381ad16ebd13914f8d5","version":"v4"},{"repo":"actions/checkout","sha":"9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0","version":"v7.0.0"},{"repo":"actions/checkout","sha":"de0fac2e4500dabe0009e67214ff5f5447ce83dd","version":"v6.0.2"},{"repo":"actions/download-artifact","sha":"3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c","version":"v8.0.1"},{"repo":"actions/github-script","sha":"3a2844b7e9c422d3c10d287c895573f7108da1b3","version":"v9.0.0"},{"repo":"actions/setup-node","sha":"48b55a011bda9f5d6aeb4c2d9c7362e8dae4041e","version":"v6.4.0"},{"repo":"actions/upload-artifact","sha":"043fb46d1a93c77aae656e7c1c64a875d1fc6a0a","version":"v7.0.1"},{"repo":"github/gh-aw-actions/setup","sha":"8c7d04ebf1ece56cd381446125da3e0f6896294a","version":"v0.80.9"}],"containers":[{"image":"ghcr.io/github/gh-aw-firewall/agent:0.27.7","digest":"sha256:aae231e4635c8999d039c132f1602d3df850fe9b84a00aa2b5ac981179b5661c","pinned_image":"ghcr.io/github/gh-aw-firewall/agent:0.27.7@sha256:aae231e4635c8999d039c132f1602d3df850fe9b84a00aa2b5ac981179b5661c"},{"image":"ghcr.io/github/gh-aw-firewall/api-proxy:0.27.7","digest":"sha256:009caf2e3d88fa77b64e9a03a95a228fc58db0f1701c6d324b29ba5a3c7c79b6","pinned_image":"ghcr.io/github/gh-aw-firewall/api-proxy:0.27.7@sha256:009caf2e3d88fa77b64e9a03a95a228fc58db0f1701c6d324b29ba5a3c7c79b6"},{"image":"ghcr.io/github/gh-aw-firewall/squid:0.27.7","digest":"sha256:deb1d4e19de62d51cee0508057a596a19315c3423ada4d675cad136dc8037c96","pinned_image":"ghcr.io/github/gh-aw-firewall/squid:0.27.7@sha256:deb1d4e19de62d51cee0508057a596a19315c3423ada4d675cad136dc8037c96"},{"image":"ghcr.io/github/gh-aw-mcpg:v0.3.27","digest":"sha256:fe984bddde4ec05d756d9043edb0a32912e6b7b72f6a121b1082f29221421cc7","pinned_image":"ghcr.io/github/gh-aw-mcpg:v0.3.27@sha256:fe984bddde4ec05d756d9043edb0a32912e6b7b72f6a121b1082f29221421cc7"},{"image":"ghcr.io/github/gh-aw-node","digest":"sha256:529d02eb970b1161aa25c593a9c3df57fdfad5a8add328cb3b6eccef66f3183b","pinned_image":"ghcr.io/github/gh-aw-node@sha256:529d02eb970b1161aa25c593a9c3df57fdfad5a8add328cb3b6eccef66f3183b"},{"image":"ghcr.io/github/github-mcp-server:v1.4.0","digest":"sha256:2afb26356481d1a350e14544a6e160f7f7ec1561a1ea309b823665abf0309036","pinned_image":"ghcr.io/github/github-mcp-server:v1.4.0@sha256:2afb26356481d1a350e14544a6e160f7f7ec1561a1ea309b823665abf0309036"}]} # This file was automatically generated by gh-aw (v0.80.9). DO NOT EDIT. To debug this workflow, load the skill at https://github.com/github/gh-aw/blob/main/debug.md # @@ -37,7 +37,9 @@ # humans (the open tracking issue is the hand-off surface; a dedicated # [ci-fix][needs-human] PR is planned but currently deferred — see Step 6). CI on # the PR is kicked by a human `/azp run` for now; the loop watches, classifies, -# and re-fixes autonomously between kicks. +# and re-fixes autonomously between kicks. When the SPECIFIC test a PR fixed is +# confirmed green in that PR's own CI, the loop marks the draft PR ready for review +# (a state transition only — it never approves or merges). # Never mutes tests, but # de-flakes genuinely flaky ones (deterministic synchronization, no retries / # timeout bumps). Always skips visual-regression / screenshot issues. @@ -273,24 +275,24 @@ jobs: run: | bash "${RUNNER_TEMP}/gh-aw/actions/create_prompt_first.sh" { - cat << 'GH_AW_PROMPT_25cf75111b670167_EOF' + cat << 'GH_AW_PROMPT_2ac1d0b4083428f8_EOF' - GH_AW_PROMPT_25cf75111b670167_EOF + GH_AW_PROMPT_2ac1d0b4083428f8_EOF cat "${RUNNER_TEMP}/gh-aw/prompts/xpia.md" cat "${RUNNER_TEMP}/gh-aw/prompts/temp_folder_prompt.md" cat "${RUNNER_TEMP}/gh-aw/prompts/markdown.md" cat "${RUNNER_TEMP}/gh-aw/prompts/safe_outputs_prompt.md" - cat << 'GH_AW_PROMPT_25cf75111b670167_EOF' + cat << 'GH_AW_PROMPT_2ac1d0b4083428f8_EOF' - Tools: add_comment(max:3), create_pull_request(max:3), update_pull_request(max:3), push_to_pull_request_branch(max:3), missing_tool, missing_data, noop - GH_AW_PROMPT_25cf75111b670167_EOF + Tools: add_comment(max:6), create_pull_request(max:3), update_pull_request(max:3), mark_pull_request_as_ready_for_review(max:3), add_labels(max:3), push_to_pull_request_branch(max:3), missing_tool, missing_data, noop + GH_AW_PROMPT_2ac1d0b4083428f8_EOF cat "${RUNNER_TEMP}/gh-aw/prompts/safe_outputs_create_pull_request.md" cat "${RUNNER_TEMP}/gh-aw/prompts/safe_outputs_push_to_pr_branch.md" - cat << 'GH_AW_PROMPT_25cf75111b670167_EOF' + cat << 'GH_AW_PROMPT_2ac1d0b4083428f8_EOF' - GH_AW_PROMPT_25cf75111b670167_EOF + GH_AW_PROMPT_2ac1d0b4083428f8_EOF cat "${RUNNER_TEMP}/gh-aw/prompts/mcp_cli_tools_prompt.md" - cat << 'GH_AW_PROMPT_25cf75111b670167_EOF' + cat << 'GH_AW_PROMPT_2ac1d0b4083428f8_EOF' The following GitHub context information is available for this workflow: {{#if github.actor}} @@ -332,12 +334,12 @@ jobs: stop immediately and report the limitation rather than spending turns trying to work around it. - GH_AW_PROMPT_25cf75111b670167_EOF + GH_AW_PROMPT_2ac1d0b4083428f8_EOF cat "${RUNNER_TEMP}/gh-aw/prompts/github_mcp_tools_with_safeoutputs_prompt.md" - cat << 'GH_AW_PROMPT_25cf75111b670167_EOF' + cat << 'GH_AW_PROMPT_2ac1d0b4083428f8_EOF' {{#runtime-import .github/workflows/ci-status-fix.md}} - GH_AW_PROMPT_25cf75111b670167_EOF + GH_AW_PROMPT_2ac1d0b4083428f8_EOF } > "$GH_AW_PROMPT" - name: Interpolate variables and render templates uses: actions/github-script@3a2844b7e9c422d3c10d287c895573f7108da1b3 # v9.0.0 @@ -554,16 +556,18 @@ jobs: mkdir -p "${RUNNER_TEMP}/gh-aw/safeoutputs" mkdir -p /tmp/gh-aw/safeoutputs mkdir -p /tmp/gh-aw/mcp-logs/safeoutputs - cat > "${RUNNER_TEMP}/gh-aw/safeoutputs/config.json" << 'GH_AW_SAFE_OUTPUTS_CONFIG_fa2a69fad5fd0485_EOF' - {"add_comment":{"discussions":false,"max":3,"required_labels":["agentic-workflows"],"required_title_prefix":"[ci-fix] ","target":"*"},"create_pull_request":{"allowed_base_branches":["main"],"allowed_branches":["ci-fix/**"],"allowed_files":["src/Core/**","src/Controls/**","src/Essentials/**","src/BlazorWebView/**","src/TestUtils/**","src/Templates/**","**/PublicAPI.Unshipped.txt"],"base_branch":"main","draft":true,"labels":["agentic-workflows"],"max":3,"max_patch_files":100,"max_patch_size":4096,"protect_top_level_dot_folders":true,"protected_files":["package.json","bun.lockb","bunfig.toml","deno.json","deno.jsonc","deno.lock","global.json","NuGet.Config","Directory.Packages.props","mix.exs","mix.lock","go.mod","go.sum","stack.yaml","stack.yaml.lock","pom.xml","build.gradle","build.gradle.kts","settings.gradle","settings.gradle.kts","gradle.properties","package-lock.json","yarn.lock","pnpm-lock.yaml","npm-shrinkwrap.json","requirements.txt","Pipfile","Pipfile.lock","pyproject.toml","setup.py","setup.cfg","Gemfile","Gemfile.lock","uv.lock","CODEOWNERS","DESIGN.md","README.md","CONTRIBUTING.md","CHANGELOG.md","SECURITY.md","CODE_OF_CONDUCT.md","AGENTS.md","CLAUDE.md","GEMINI.md"],"protected_files_policy":"request_review","title_prefix":"[ci-fix] "},"create_report_incomplete_issue":{},"missing_data":{},"missing_tool":{},"noop":{"max":1,"report-as-issue":"true"},"push_to_pull_request_branch":{"allowed_files":["src/Core/**","src/Controls/**","src/Essentials/**","src/BlazorWebView/**","src/TestUtils/**","src/Templates/**","**/PublicAPI.Unshipped.txt"],"if_no_changes":"warn","max":3,"max_patch_size":4096,"protect_top_level_dot_folders":true,"protected_files":["package.json","bun.lockb","bunfig.toml","deno.json","deno.jsonc","deno.lock","global.json","NuGet.Config","Directory.Packages.props","mix.exs","mix.lock","go.mod","go.sum","stack.yaml","stack.yaml.lock","pom.xml","build.gradle","build.gradle.kts","settings.gradle","settings.gradle.kts","gradle.properties","package-lock.json","yarn.lock","pnpm-lock.yaml","npm-shrinkwrap.json","requirements.txt","Pipfile","Pipfile.lock","pyproject.toml","setup.py","setup.cfg","Gemfile","Gemfile.lock","uv.lock","CODEOWNERS","DESIGN.md","README.md","CONTRIBUTING.md","CHANGELOG.md","SECURITY.md","CODE_OF_CONDUCT.md","AGENTS.md","CLAUDE.md","GEMINI.md"],"required_labels":["agentic-workflows"],"target":"*","title_prefix":"[ci-fix] "},"report_incomplete":{},"update_pull_request":{"allow_body":true,"allow_title":false,"max":3,"target":"*","update_branch":false}} - GH_AW_SAFE_OUTPUTS_CONFIG_fa2a69fad5fd0485_EOF + cat > "${RUNNER_TEMP}/gh-aw/safeoutputs/config.json" << 'GH_AW_SAFE_OUTPUTS_CONFIG_0644bf1434ca0071_EOF' + {"add_comment":{"discussions":false,"max":6,"required_labels":["agentic-workflows"],"required_title_prefix":"[ci-fix] ","target":"*"},"add_labels":{"allowed":["p/0"],"max":3,"required_labels":["agentic-workflows"],"required_title_prefix":"[ci-fix] ","target":"*"},"create_pull_request":{"allowed_base_branches":["main"],"allowed_branches":["ci-fix/**"],"allowed_files":["src/Core/**","src/Controls/**","src/Essentials/**","src/BlazorWebView/**","src/TestUtils/**","src/Templates/**","**/PublicAPI.Unshipped.txt"],"base_branch":"main","draft":true,"labels":["agentic-workflows"],"max":3,"max_patch_files":100,"max_patch_size":4096,"protect_top_level_dot_folders":true,"protected_files":["package.json","bun.lockb","bunfig.toml","deno.json","deno.jsonc","deno.lock","global.json","NuGet.Config","Directory.Packages.props","mix.exs","mix.lock","go.mod","go.sum","stack.yaml","stack.yaml.lock","pom.xml","build.gradle","build.gradle.kts","settings.gradle","settings.gradle.kts","gradle.properties","package-lock.json","yarn.lock","pnpm-lock.yaml","npm-shrinkwrap.json","requirements.txt","Pipfile","Pipfile.lock","pyproject.toml","setup.py","setup.cfg","Gemfile","Gemfile.lock","uv.lock","CODEOWNERS","DESIGN.md","README.md","CONTRIBUTING.md","CHANGELOG.md","SECURITY.md","CODE_OF_CONDUCT.md","AGENTS.md","CLAUDE.md","GEMINI.md"],"protected_files_policy":"request_review","title_prefix":"[ci-fix] "},"create_report_incomplete_issue":{},"mark_pull_request_as_ready_for_review":{"max":3,"required_labels":["agentic-workflows"],"required_title_prefix":"[ci-fix] ","target":"*"},"missing_data":{},"missing_tool":{},"noop":{"max":1,"report-as-issue":"true"},"push_to_pull_request_branch":{"allowed_files":["src/Core/**","src/Controls/**","src/Essentials/**","src/BlazorWebView/**","src/TestUtils/**","src/Templates/**","**/PublicAPI.Unshipped.txt"],"if_no_changes":"warn","max":3,"max_patch_size":4096,"protect_top_level_dot_folders":true,"protected_files":["package.json","bun.lockb","bunfig.toml","deno.json","deno.jsonc","deno.lock","global.json","NuGet.Config","Directory.Packages.props","mix.exs","mix.lock","go.mod","go.sum","stack.yaml","stack.yaml.lock","pom.xml","build.gradle","build.gradle.kts","settings.gradle","settings.gradle.kts","gradle.properties","package-lock.json","yarn.lock","pnpm-lock.yaml","npm-shrinkwrap.json","requirements.txt","Pipfile","Pipfile.lock","pyproject.toml","setup.py","setup.cfg","Gemfile","Gemfile.lock","uv.lock","CODEOWNERS","DESIGN.md","README.md","CONTRIBUTING.md","CHANGELOG.md","SECURITY.md","CODE_OF_CONDUCT.md","AGENTS.md","CLAUDE.md","GEMINI.md"],"required_labels":["agentic-workflows"],"target":"*","title_prefix":"[ci-fix] "},"report_incomplete":{},"update_pull_request":{"allow_body":true,"allow_title":false,"max":3,"target":"*","update_branch":false}} + GH_AW_SAFE_OUTPUTS_CONFIG_0644bf1434ca0071_EOF - name: Generate Safe Outputs Tools env: GH_AW_TOOLS_META_JSON: | { "description_suffixes": { - "add_comment": " CONSTRAINTS: Maximum 3 comment(s) can be added. Target: *. Supports reply_to_id for discussion threading.", + "add_comment": " CONSTRAINTS: Maximum 6 comment(s) can be added. Target: *. Supports reply_to_id for discussion threading.", + "add_labels": " CONSTRAINTS: Maximum 3 label(s) can be added. Only these labels are allowed: [\"p/0\"]. Target: *.", "create_pull_request": " CONSTRAINTS: Maximum 3 pull request(s) can be created. Title will be prefixed with \"[ci-fix] \". Labels [\"agentic-workflows\"] will be automatically added. Only these labels are allowed: [\"agentic-workflows\"]. PRs will be created as drafts.", + "mark_pull_request_as_ready_for_review": " CONSTRAINTS: Maximum 3 pull request(s) can be marked as ready for review.", "push_to_pull_request_branch": " CONSTRAINTS: Maximum 3 push(es) can be made. The target pull request title must start with \"[ci-fix] \".", "update_pull_request": " CONSTRAINTS: Maximum 3 pull request(s) can be updated. Target: *." }, @@ -594,6 +598,25 @@ jobs: } } }, + "add_labels": { + "defaultMax": 5, + "fields": { + "item_number": { + "issueNumberOrTemporaryId": true + }, + "labels": { + "required": true, + "type": "array", + "itemType": "string", + "itemSanitize": true, + "itemMaxLength": 128 + }, + "repo": { + "type": "string", + "maxLength": 256 + } + } + }, "create_pull_request": { "defaultMax": 1, "fields": { @@ -635,6 +658,24 @@ jobs: } } }, + "mark_pull_request_as_ready_for_review": { + "defaultMax": 1, + "fields": { + "pull_request_number": { + "issueOrPRNumber": true + }, + "reason": { + "required": true, + "type": "string", + "sanitize": true, + "maxLength": 65000 + }, + "repo": { + "type": "string", + "maxLength": 256 + } + } + }, "missing_data": { "defaultMax": 20, "fields": { @@ -1494,7 +1535,7 @@ jobs: uses: actions/github-script@3a2844b7e9c422d3c10d287c895573f7108da1b3 # v9.0.0 env: WORKFLOW_NAME: "CI Failure Fixer (main)" - WORKFLOW_DESCRIPTION: "Periodic pass over open ci-scan tracking issues filed by the main-branch CI\nfailure scanner (.github/workflows/ci-status-main.md). This workflow targets\nthe `main` branch EXCLUSIVELY: it processes only issues labelled ci-scan and\nopens every PR against main. (The net11.0 branch is handled by the parallel\n.github/workflows/ci-status-fix-net11.md — the two are split because gh-aw can\nonly transport a fix relative to ONE static base branch per workflow, and the\nmain↔net11.0 divergence exceeds gh-aw's 10 MB transport-patch cap.) The fixer\nopens ONE draft [ci-fix] PR per actionable issue against main, then WATCHES\nthat PR's own CI on later runs: when the fix's CI comes back red and the red is\ncaused by the fix itself, it pushes a fresh follow-up fix onto the SAME PR\nbranch (never a second PR) — up to 10 attempts — then stops and defers to\nhumans (the open tracking issue is the hand-off surface; a dedicated\n[ci-fix][needs-human] PR is planned but currently deferred — see Step 6). CI on\nthe PR is kicked by a human `/azp run` for now; the loop watches, classifies,\nand re-fixes autonomously between kicks.\nNever mutes tests, but\nde-flakes genuinely flaky ones (deterministic synchronization, no retries /\ntimeout bumps). Always skips visual-regression / screenshot issues." + WORKFLOW_DESCRIPTION: "Periodic pass over open ci-scan tracking issues filed by the main-branch CI\nfailure scanner (.github/workflows/ci-status-main.md). This workflow targets\nthe `main` branch EXCLUSIVELY: it processes only issues labelled ci-scan and\nopens every PR against main. (The net11.0 branch is handled by the parallel\n.github/workflows/ci-status-fix-net11.md — the two are split because gh-aw can\nonly transport a fix relative to ONE static base branch per workflow, and the\nmain↔net11.0 divergence exceeds gh-aw's 10 MB transport-patch cap.) The fixer\nopens ONE draft [ci-fix] PR per actionable issue against main, then WATCHES\nthat PR's own CI on later runs: when the fix's CI comes back red and the red is\ncaused by the fix itself, it pushes a fresh follow-up fix onto the SAME PR\nbranch (never a second PR) — up to 10 attempts — then stops and defers to\nhumans (the open tracking issue is the hand-off surface; a dedicated\n[ci-fix][needs-human] PR is planned but currently deferred — see Step 6). CI on\nthe PR is kicked by a human `/azp run` for now; the loop watches, classifies,\nand re-fixes autonomously between kicks. When the SPECIFIC test a PR fixed is\nconfirmed green in that PR's own CI, the loop marks the draft PR ready for review\n(a state transition only — it never approves or merges).\nNever mutes tests, but\nde-flakes genuinely flaky ones (deterministic synchronization, no retries /\ntimeout bumps). Always skips visual-regression / screenshot issues." HAS_PATCH: ${{ needs.agent.outputs.has_patch }} with: script: | @@ -1910,7 +1951,7 @@ jobs: GH_AW_ALLOWED_DOMAINS: "*.blob.core.windows.net,*.githubusercontent.com,api.business.githubcopilot.com,api.enterprise.githubcopilot.com,api.github.com,api.githubcopilot.com,api.individual.githubcopilot.com,api.snapcraft.io,archive.ubuntu.com,azure.archive.ubuntu.com,codeload.github.com,crl.geotrust.com,crl.globalsign.com,crl.identrust.com,crl.sectigo.com,crl.thawte.com,crl.usertrust.com,crl.verisign.com,crl3.digicert.com,crl4.digicert.com,crls.ssl.com,dev.azure.com,docs.github.com,github-cloud.githubusercontent.com,github-cloud.s3.amazonaws.com,github.blog,github.com,github.githubassets.com,helix.dot.net,host.docker.internal,json-schema.org,json.schemastore.org,keyserver.ubuntu.com,lfs.github.com,objects.githubusercontent.com,ocsp.digicert.com,ocsp.geotrust.com,ocsp.globalsign.com,ocsp.identrust.com,ocsp.sectigo.com,ocsp.ssl.com,ocsp.thawte.com,ocsp.usertrust.com,ocsp.verisign.com,packagecloud.io,packages.cloud.google.com,packages.microsoft.com,patch-diff.githubusercontent.com,ppa.launchpad.net,raw.githubusercontent.com,registry.npmjs.org,s.symcb.com,s.symcd.com,security.ubuntu.com,telemetry.enterprise.githubcopilot.com,ts-crl.ws.symantec.com,ts-ocsp.ws.symantec.com,www.googleapis.com" GITHUB_SERVER_URL: ${{ github.server_url }} GITHUB_API_URL: ${{ github.api_url }} - GH_AW_SAFE_OUTPUTS_HANDLER_CONFIG: "{\"add_comment\":{\"discussions\":false,\"max\":3,\"required_labels\":[\"agentic-workflows\"],\"required_title_prefix\":\"[ci-fix] \",\"target\":\"*\"},\"create_pull_request\":{\"allowed_base_branches\":[\"main\"],\"allowed_branches\":[\"ci-fix/**\"],\"allowed_files\":[\"src/Core/**\",\"src/Controls/**\",\"src/Essentials/**\",\"src/BlazorWebView/**\",\"src/TestUtils/**\",\"src/Templates/**\",\"**/PublicAPI.Unshipped.txt\"],\"base_branch\":\"main\",\"draft\":true,\"labels\":[\"agentic-workflows\"],\"max\":3,\"max_patch_files\":100,\"max_patch_size\":4096,\"protect_top_level_dot_folders\":true,\"protected_files\":[\"package.json\",\"bun.lockb\",\"bunfig.toml\",\"deno.json\",\"deno.jsonc\",\"deno.lock\",\"global.json\",\"NuGet.Config\",\"Directory.Packages.props\",\"mix.exs\",\"mix.lock\",\"go.mod\",\"go.sum\",\"stack.yaml\",\"stack.yaml.lock\",\"pom.xml\",\"build.gradle\",\"build.gradle.kts\",\"settings.gradle\",\"settings.gradle.kts\",\"gradle.properties\",\"package-lock.json\",\"yarn.lock\",\"pnpm-lock.yaml\",\"npm-shrinkwrap.json\",\"requirements.txt\",\"Pipfile\",\"Pipfile.lock\",\"pyproject.toml\",\"setup.py\",\"setup.cfg\",\"Gemfile\",\"Gemfile.lock\",\"uv.lock\",\"CODEOWNERS\",\"DESIGN.md\",\"README.md\",\"CONTRIBUTING.md\",\"CHANGELOG.md\",\"SECURITY.md\",\"CODE_OF_CONDUCT.md\",\"AGENTS.md\",\"CLAUDE.md\",\"GEMINI.md\"],\"protected_files_policy\":\"request_review\",\"title_prefix\":\"[ci-fix] \"},\"create_report_incomplete_issue\":{},\"missing_data\":{},\"missing_tool\":{},\"noop\":{\"max\":1,\"report-as-issue\":\"true\"},\"push_to_pull_request_branch\":{\"allowed_files\":[\"src/Core/**\",\"src/Controls/**\",\"src/Essentials/**\",\"src/BlazorWebView/**\",\"src/TestUtils/**\",\"src/Templates/**\",\"**/PublicAPI.Unshipped.txt\"],\"if_no_changes\":\"warn\",\"max\":3,\"max_patch_size\":4096,\"protect_top_level_dot_folders\":true,\"protected_files\":[\"package.json\",\"bun.lockb\",\"bunfig.toml\",\"deno.json\",\"deno.jsonc\",\"deno.lock\",\"global.json\",\"NuGet.Config\",\"Directory.Packages.props\",\"mix.exs\",\"mix.lock\",\"go.mod\",\"go.sum\",\"stack.yaml\",\"stack.yaml.lock\",\"pom.xml\",\"build.gradle\",\"build.gradle.kts\",\"settings.gradle\",\"settings.gradle.kts\",\"gradle.properties\",\"package-lock.json\",\"yarn.lock\",\"pnpm-lock.yaml\",\"npm-shrinkwrap.json\",\"requirements.txt\",\"Pipfile\",\"Pipfile.lock\",\"pyproject.toml\",\"setup.py\",\"setup.cfg\",\"Gemfile\",\"Gemfile.lock\",\"uv.lock\",\"CODEOWNERS\",\"DESIGN.md\",\"README.md\",\"CONTRIBUTING.md\",\"CHANGELOG.md\",\"SECURITY.md\",\"CODE_OF_CONDUCT.md\",\"AGENTS.md\",\"CLAUDE.md\",\"GEMINI.md\"],\"required_labels\":[\"agentic-workflows\"],\"target\":\"*\",\"title_prefix\":\"[ci-fix] \"},\"report_incomplete\":{},\"update_pull_request\":{\"allow_body\":true,\"allow_title\":false,\"max\":3,\"target\":\"*\",\"update_branch\":false}}" + GH_AW_SAFE_OUTPUTS_HANDLER_CONFIG: "{\"add_comment\":{\"discussions\":false,\"max\":6,\"required_labels\":[\"agentic-workflows\"],\"required_title_prefix\":\"[ci-fix] \",\"target\":\"*\"},\"add_labels\":{\"allowed\":[\"p/0\"],\"max\":3,\"required_labels\":[\"agentic-workflows\"],\"required_title_prefix\":\"[ci-fix] \",\"target\":\"*\"},\"create_pull_request\":{\"allowed_base_branches\":[\"main\"],\"allowed_branches\":[\"ci-fix/**\"],\"allowed_files\":[\"src/Core/**\",\"src/Controls/**\",\"src/Essentials/**\",\"src/BlazorWebView/**\",\"src/TestUtils/**\",\"src/Templates/**\",\"**/PublicAPI.Unshipped.txt\"],\"base_branch\":\"main\",\"draft\":true,\"labels\":[\"agentic-workflows\"],\"max\":3,\"max_patch_files\":100,\"max_patch_size\":4096,\"protect_top_level_dot_folders\":true,\"protected_files\":[\"package.json\",\"bun.lockb\",\"bunfig.toml\",\"deno.json\",\"deno.jsonc\",\"deno.lock\",\"global.json\",\"NuGet.Config\",\"Directory.Packages.props\",\"mix.exs\",\"mix.lock\",\"go.mod\",\"go.sum\",\"stack.yaml\",\"stack.yaml.lock\",\"pom.xml\",\"build.gradle\",\"build.gradle.kts\",\"settings.gradle\",\"settings.gradle.kts\",\"gradle.properties\",\"package-lock.json\",\"yarn.lock\",\"pnpm-lock.yaml\",\"npm-shrinkwrap.json\",\"requirements.txt\",\"Pipfile\",\"Pipfile.lock\",\"pyproject.toml\",\"setup.py\",\"setup.cfg\",\"Gemfile\",\"Gemfile.lock\",\"uv.lock\",\"CODEOWNERS\",\"DESIGN.md\",\"README.md\",\"CONTRIBUTING.md\",\"CHANGELOG.md\",\"SECURITY.md\",\"CODE_OF_CONDUCT.md\",\"AGENTS.md\",\"CLAUDE.md\",\"GEMINI.md\"],\"protected_files_policy\":\"request_review\",\"title_prefix\":\"[ci-fix] \"},\"create_report_incomplete_issue\":{},\"mark_pull_request_as_ready_for_review\":{\"max\":3,\"required_labels\":[\"agentic-workflows\"],\"required_title_prefix\":\"[ci-fix] \",\"target\":\"*\"},\"missing_data\":{},\"missing_tool\":{},\"noop\":{\"max\":1,\"report-as-issue\":\"true\"},\"push_to_pull_request_branch\":{\"allowed_files\":[\"src/Core/**\",\"src/Controls/**\",\"src/Essentials/**\",\"src/BlazorWebView/**\",\"src/TestUtils/**\",\"src/Templates/**\",\"**/PublicAPI.Unshipped.txt\"],\"if_no_changes\":\"warn\",\"max\":3,\"max_patch_size\":4096,\"protect_top_level_dot_folders\":true,\"protected_files\":[\"package.json\",\"bun.lockb\",\"bunfig.toml\",\"deno.json\",\"deno.jsonc\",\"deno.lock\",\"global.json\",\"NuGet.Config\",\"Directory.Packages.props\",\"mix.exs\",\"mix.lock\",\"go.mod\",\"go.sum\",\"stack.yaml\",\"stack.yaml.lock\",\"pom.xml\",\"build.gradle\",\"build.gradle.kts\",\"settings.gradle\",\"settings.gradle.kts\",\"gradle.properties\",\"package-lock.json\",\"yarn.lock\",\"pnpm-lock.yaml\",\"npm-shrinkwrap.json\",\"requirements.txt\",\"Pipfile\",\"Pipfile.lock\",\"pyproject.toml\",\"setup.py\",\"setup.cfg\",\"Gemfile\",\"Gemfile.lock\",\"uv.lock\",\"CODEOWNERS\",\"DESIGN.md\",\"README.md\",\"CONTRIBUTING.md\",\"CHANGELOG.md\",\"SECURITY.md\",\"CODE_OF_CONDUCT.md\",\"AGENTS.md\",\"CLAUDE.md\",\"GEMINI.md\"],\"required_labels\":[\"agentic-workflows\"],\"target\":\"*\",\"title_prefix\":\"[ci-fix] \"},\"report_incomplete\":{},\"update_pull_request\":{\"allow_body\":true,\"allow_title\":false,\"max\":3,\"target\":\"*\",\"update_branch\":false}}" GH_AW_CI_TRIGGER_TOKEN: ${{ secrets.GH_AW_CI_TRIGGER_TOKEN }} with: github-token: ${{ secrets.GH_AW_GITHUB_TOKEN || secrets.GITHUB_TOKEN }} diff --git a/.github/workflows/ci-status-fix.md b/.github/workflows/ci-status-fix.md index d3227e25170c..9c2f46054029 100644 --- a/.github/workflows/ci-status-fix.md +++ b/.github/workflows/ci-status-fix.md @@ -15,7 +15,9 @@ description: | humans (the open tracking issue is the hand-off surface; a dedicated [ci-fix][needs-human] PR is planned but currently deferred — see Step 6). CI on the PR is kicked by a human `/azp run` for now; the loop watches, classifies, - and re-fixes autonomously between kicks. + and re-fixes autonomously between kicks. When the SPECIFIC test a PR fixed is + confirmed green in that PR's own CI, the loop marks the draft PR ready for review + (a state transition only — it never approves or merges). Never mutes tests, but de-flakes genuinely flaky ones (deterministic synchronization, no retries / timeout bumps). Always skips visual-regression / screenshot issues. @@ -239,8 +241,15 @@ safe-outputs: # PER-RUN total (not per-PR): a sweep may surface several green PRs and/or # annotate several flaky reds in one run. At max:1 all but the first comment # were silently dropped, starving the workflow's #1 value (surfacing green PRs - # for review). Raised to 3 to match the per-run throughput of the other outputs. - max: 3 + # for review). Sized to 6 (not 3) because a single green DRAFT PR that gets + # flipped ready in one sweep spends TWO comment slots — the Step 3 ✅ surface + # comment (which still names the /azp-gated legs a human must kick, so it is NOT + # redundant with 🎯) AND the Step 3.6 T3 🎯 readiness comment. At max:3 the shared + # bucket drained after ~1 draft flip and T3's atomicity pre-check then deferred + # every further mark-ready even while the mark-ready/add_labels buckets (also 3) + # sat idle; 6 lets ~3 draft flips (2 comments each) land per sweep, so the + # mark-ready:3 / add_labels:3 caps are actually reachable. + max: 6 target: "*" # Hard constraint (defense-in-depth): only comment on THIS workflow's own # ci-fix PRs — [ci-fix] title prefix AND agentic-workflows label. Mirrors the @@ -261,13 +270,51 @@ safe-outputs: # never the title, so disable title rewrites — this compiles to allow_title:false # and removes any ability to retitle an arbitrary PR. title: false - # NOTE: gh-aw v0.79.8 does NOT emit required-title-prefix/required-labels into + # NOTE: gh-aw v0.80.9 (the pinned compiler) does NOT emit required-title-prefix/required-labels into # the compiled config for update-pull-request (verified against the lock — it # silently drops them, unlike add-comment / push-to-pull-request-branch which # honor them). So which-PR scoping here relies on prompt Hard-Rule 6 + # min-integrity:approved; the residual is body-marker edits only (no code, no # merge, no close), capped at max:3. Revisit required-* if a future gh-aw # compiler emits them for this output. + mark-pull-request-as-ready-for-review: + # Flip a validated draft [ci-fix] PR to ready-for-review once the SPECIFIC test + # this PR was opened to fix is confirmed green in the PR's OWN CI (Step 3.6) — + # even when unrelated legs are red. The workflow is schedule/dispatch-triggered + # (no triggering-PR context), so target "*" lets the agent name the PR number it + # validated. This is a state transition ONLY: it never approves and never merges — + # a human still reviews and merges. + # PER-RUN total (not per-PR): a sweep may validate several PRs' target tests green. + max: 3 + target: "*" + # Hard constraint (defense-in-depth): only ever un-draft THIS workflow's own + # ci-fix PRs — [ci-fix] title prefix AND agentic-workflows label — so a confused + # or prompt-injected agent cannot mark an arbitrary PR ready. (If the v0.80.9 + # compiler silently drops these — as documented for update-pull-request above — + # the Step 3.6 preconditions + min-integrity:approved are the compensating scope + # controls; verify against the lock after compiling.) + required-title-prefix: "[ci-fix] " + required-labels: [agentic-workflows] + add-labels: + # Apply the p/0 priority label to a [ci-fix] PR at the exact moment the loop flips it + # from draft to ready-for-review (Step 3.6 T3) — so a validated, review-ready fix lands + # in the team's p/0 triage queue instead of sitting unseen in the draft backlog. Paired + # 1:1 with mark-pull-request-as-ready-for-review; target "*" lets the agent name the PR + # it just validated. + # allowed = HARD allowlist: the agent may ONLY ever add p/0, nothing else. This caps the + # blast radius of a confused/prompt-injected agent to exactly one benign priority label — + # it can never apply a downstream-triggering or destructive label. + allowed: [p/0] + # PER-RUN total (not per-PR): a sweep may mark several PRs ready in one run; each adds + # one label, so this matches the mark-ready per-run cap (3). + max: 3 + target: "*" + # Hard constraint (defense-in-depth): only ever label THIS workflow's own ci-fix PRs — + # [ci-fix] title prefix AND agentic-workflows label. (If the v0.80.9 compiler drops these + # — as documented for update-pull-request above — the Step 3.6 preconditions + + # min-integrity:approved + the allowed:[p/0] allowlist are the compensating controls.) + required-title-prefix: "[ci-fix] " + required-labels: [agentic-workflows] timeout-minutes: 90 @@ -339,33 +386,48 @@ through `safe-outputs`. 5. **One issue = one outcome per run.** Exactly one of: respond to a maintainer's change-request on the open PR (Track C, Step 3.5.R); open the first fix/help/ de-flake PR; advance an existing PR by one attempt (push a follow-up fix); - surface a validated-green PR for review; annotate an unrelated-flake red; wait + surface a validated-green PR for review; mark a target-validated draft PR + ready-for-review (Step 3.6 T3 — the terminal outcome that supersedes the same + run's surface/annotate precursor line), or record its `already-ready` + steady-state no-op; annotate an unrelated-flake red; wait (CI not yet settled); or a recorded skip (the dedicated needs-human PR is deferred — the attempt cap records a skip instead; see Step 6). Always prefer advancing or opening a PR over a skip when a non-mute diff is producible. 6. **All writes via `safe-outputs`.** Allowed outputs: `create_pull_request` (first attempt only), `push_to_pull_request_branch` (advance an existing PR by one attempt), `update_pull_request` (bump the attempt marker / refresh the - prior-attempts table), and `add_comment` (progress notes on the `[ci-fix]` PR - ONLY). NEVER comment on the tracking issue (issues are locked by + prior-attempts table), `add_comment` (progress notes on the `[ci-fix]` PR + ONLY), `mark_pull_request_as_ready_for_review` (flip a target-validated draft + `[ci-fix]` PR from draft to ready — Step 3.6 T3 only), and `add_labels` (add + ONLY the `p/0` label, ONLY on that same draft→ready transition — Step 3.6 T3). + NEVER comment on the tracking issue (issues are locked by `.github/workflows/ci-scan-lock-issues.yml`) — `add_comment` targets the PR. No `gh pr create`, no manual `git push`. **Defense-in-depth:** `add_comment` and `update_pull_request` use `target: "*"` (the agent supplies the PR number). - `add_comment` is now config-locked to `required-title-prefix: "[ci-fix] "` + + `add_comment`, `mark_pull_request_as_ready_for_review`, and `add_labels` are + config-locked to `required-title-prefix: "[ci-fix] "` + `required-labels: [agentic-workflows]` (hard-enforced by the handler, same as - `push_to_pull_request_branch`), so a comment can only ever land on THIS - workflow's own PRs. `update_pull_request` CANNOT be config-locked to a - title/label in gh-aw v0.79.8 (the compiler silently drops `required-*` for that - output — verified against the lock), so it keeps `title: false` (no retitles; - body-marker edits only) plus this prompt-level guard. Before emitting either, - VERIFY the target PR carries BOTH the `[ci-fix]` title prefix AND the - `agentic-workflows` label; never comment on or edit the body of any PR that - lacks both. + `push_to_pull_request_branch`), and `add_labels` additionally has an + `allowed: [p/0]` allowlist so it can ONLY ever add `p/0` — so these can only + ever land on THIS workflow's own PRs. `update_pull_request` CANNOT be + config-locked to a title/label in gh-aw v0.80.9 (the compiler silently drops + `required-*` for that output — verified against the lock), so it keeps + `title: false` (no retitles; body-marker edits only) plus this prompt-level + guard. Before emitting ANY of these, VERIFY the target PR carries BOTH the + `[ci-fix]` title prefix AND the `agentic-workflows` label; never comment on, + edit, un-draft, or label any PR that lacks both. 7. **Per-run safe-output caps (per RUN, not per PR).** Each output type is capped per run: `create_pull_request` 3, `push_to_pull_request_branch` 3, - `add_comment` 3, `update_pull_request` 3. Note an ADVANCE spends one push **and** - one update **and** one comment, so ≤ 3 advances/run; surfacing a green or - annotating a flake spends one comment. When a bucket is exhausted, do NOT keep + `add_comment` 6, `update_pull_request` 3, `mark_pull_request_as_ready_for_review` + 3, `add_labels` 3 (counts LABELS, not calls). Note an ADVANCE spends one push + **and** one update **and** one comment, so ≤ 3 advances/run; surfacing a green or + annotating a flake spends one comment; a **mark-ready (Step 3.6 T3)** of a + still-draft PR spends TWO comments (the Step 3 ✅ surface **and** the T3 🎯 + readiness note) **and** one mark-ready **and** one label as an ALL-OR-NOTHING set + (T3 pre-checks those buckets and defers the whole PR if any is exhausted — never + mark a PR ready without its 🎯 audit comment). `add_comment` is therefore sized 6 + (not 3) so ~3 draft flips, at 2 comments each, can land in one sweep instead of the + shared comment bucket starving the otherwise-idle mark-ready/add_labels buckets. When a bucket is exhausted, do NOT keep emitting (extras are silently dropped) — record `skipped: per-run cap reached; deferring PR # to next cycle` for each remaining PR so the drop is a deliberate, logged decision. The continuous loop picks the deferred PRs up next @@ -409,8 +471,9 @@ For every open tracking issue in scope, converge on exactly one outcome: | Open first help-wanted draft `[ci-fix]` PR | No open PR yet; a plausible candidate exists but cannot be runner-validated (device/UI tests) (attempt 1) | | Open first de-flake draft `[ci-fix]` PR | No open PR yet; failure is intermittent (green-on-retry) from a genuine test-quality defect; a deterministic-synchronization fix is producible (attempt 1, Step 4.7 bucket b) | | Advance an existing PR (push attempt N+1) | Open PR's own CI settled red *because of the fix itself*, no human engaged, marker < 10 — push a NEW distinct fix onto the same branch (Step 3.5 → 5.6) | -| Surface a validated-green PR | Open PR's own CI settled green — comment "validated, attempt N/10; ready for review" and do NOT advance (Step 3.5) | -| Annotate an unrelated-flake red | Open PR is red only on baseline-flake / unrelated legs — comment which leg needs a re-run; do NOT burn an attempt (Step 3.5) | +| Surface a validated-green PR | Open PR's own CI settled green — comment "validated, attempt N/10; ready for review" and do NOT advance (Step 3.5), then mark the draft PR ready for review once the fixed test is confirmed green (Step 3.6) | +| Annotate an unrelated-flake red | Open PR is red only on baseline-flake / unrelated legs — comment which leg needs a re-run; do NOT burn an attempt (Step 3.5); if the SPECIFIC fixed test is confirmed green on that build, still mark the draft PR ready for review (Step 3.6) | +| Mark a validated PR ready for review | The specific test this PR fixed is confirmed green in the PR's own CI (even if unrelated legs are red) — comment the target-test result and transition the draft PR to ready for review; never approve or merge (Step 3.6) | | Wait | Open PR's CI is pending / not yet settled on the current head SHA — do nothing this cycle (Step 3.5) | | Hand-off skip (attempt cap) | Marker == 10 and the signature still reproduces — stop and defer to humans; the dedicated `[ci-fix][needs-human]` PR is planned but currently deferred (Step 6) | | Recorded skip | Visual-regression, human engaged, already-handled, fixed-in-latest-build, infra-flake, out-of-bounds, only-mute-available, or no novel approach producible | @@ -685,14 +748,34 @@ Run these gates in order — the FIRST that fires decides this cycle's outcome: 1. **Human engaged.** If `C.humanEngaged` → `skipped: human engaged on PR #; deferring` and stop. Never fight a human reviewer — the intentional hand-off boundary still holds the moment a person touches the PR. -2. **CI not settled / unknown / incomplete prefetch.** If `C.checksSettled == false` - OR `C.overallConclusion` is `pending`, `neutral`, or `unknown`, OR - `C.dataComplete == false` → the fix's CI has not finished on the current head SHA - (`C.headSha`), or the prefetch could not fully read this PR (a partial read can - understate `humanEngaged` / `attempt`). `skipped: PR # CI pending / prefetch - incomplete on ; waiting` and stop. *(Round 1: this is where a - maintainer `/azp run` is awaited — the `/azp`-gated uitests/devicetests legs will - not have run until a human kicks them.)* +2. **CI not settled — but validate the target test first (target-focused readiness).** + If `C.checksSettled == false` OR `C.overallConclusion` is `pending`, `neutral`, or + `unknown`, OR `C.dataComplete == false` → the *overall* build has not finished on the + current head SHA (`C.headSha`), or the prefetch could not fully read this PR (a partial + read can understate `humanEngaged` / `attempt`). Unrelated legs still draining must NOT + keep an already-proven fix parked as a draft, so before waiting, try the target-focused + fast-path: + - **2a — Target-green fast-path (draft `[ci-fix]` PRs only).** If `C.dataComplete == + true` AND the Step 3.6 preconditions hold (draft PR, `[ci-fix] ` title prefix AND the + `agentic-workflows` label), run Step 3.6 **T1–T2** against `C.headSha` now: identify + the target test(s) and read the PR's OWN build **timeline / per-leg status** (anonymous + `_apis/build` — never `_apis/test`, per Hard-Rule 8). If EVERY target test is + **VALIDATED-GREEN** (per T2's all-platform definition below — the target's category leg + is `succeeded` on every platform that runs it, failed on none, and NO target platform + family the pipeline covers is still pending/unconcluded, per T2's "no platform left + unverified" rule; a platform that simply has no leg for the target's category — the test + doesn't run there — is fine, not a gap) AND every leg in + `C.failedLegs` that has ALREADY concluded classifies as **unrelated flake** (Step 4 / + Step 4.7 method — the fix is implicated in NO completed red), then the fix is proven + regardless of unrelated *pending* legs → run **Step 3.6 T3** (mark ready + 🎯 comment) + and stop. This early-out NEVER advances an attempt and NEVER acts on a red that could + be the fix's fault. + - **2b — Otherwise WAIT.** If the target test has not yet executed (pending/absent on + its leg), or a completed red is (or may be) caused by the fix, or `C.dataComplete == + false`, or this PR has no identifiable target test → `skipped: PR # CI pending / + target not yet validated on ; waiting` and stop. *(Round 1: this is where a + maintainer `/azp run` is awaited — the `/azp`-gated uitests/devicetests legs will not + have run until a human kicks them.)* 3. **Green → surface for review.** If `C.overallConclusion == "success"`: the checks that RAN are green. **Primary-gate check first:** the deterministic `success` verdict only certifies "at least one green check, nothing failing or @@ -704,18 +787,31 @@ Run these gates in order — the FIRST that fires decides this cycle's outcome: # primary CI gate (maui-pr) not green on ; waiting` and stop — do NOT surface. (The `/azp`-gated `maui-pr-uitests` (def 313) / `maui-pr-devicetests` (def 314) legs MAY still be un-run — that is expected and is named below, not a - reason to withhold the surface.) **Idempotency:** scan the PR's existing comments - for a prior bot `✅ … validated … on ` note for THIS head SHA — if one - already exists, record `already-surfaced PR # (head )` and stop - WITHOUT re-commenting (re-surfacing the same green every tick is noise and burns - the per-run comment budget other PRs need). Otherwise resolve the attempt number from - `C.effectiveAttempt` (the authoritative max(marker, bot-commit) counter; omit the - number only if it is somehow indeterminate). **Dry-run gate (Step 0):** if `dry_run == "true"`, do NOT emit any comment — instead print the intended `✅` validated-green body to the run log, tally `dry-run: would-surface-green PR #`, and stop. Otherwise `add_comment` on PR #: `✅ Attempt /10 validated - — the fix's CI is green on . Ready for human review.` - Name any `/azp`-gated legs (uitests def 313 / devicetests def 314) that have not - run and still need a maintainer `/azp run`. Do NOT advance. Record - `surfaced-green PR # (attempt /10)` and stop. *(This directly - attacks the real bottleneck — no reviews — so it is the highest-value outcome.)* + reason to withhold the surface.) **Comment idempotency + dry-run suppress the ✅ + comment ONLY — neither skips Step 3.6.** Scan the PR's existing comments for a prior + bot `✅ … validated … on ` note for THIS head SHA; if one exists, set + `SKIP_SURFACE_COMMENT = true`. Resolve the attempt number from `C.effectiveAttempt` + (the authoritative max(marker, bot-commit) counter; omit the number only if it is + somehow indeterminate). Then post the surface comment UNLESS suppressed: + - if `dry_run == "true"`: do NOT emit — print the intended `✅` validated-green body to + the run log and tally `dry-run: would-surface-green PR #`; + - else if `SKIP_SURFACE_COMMENT`: do NOT re-post — record `already-surfaced PR # + (head )` (re-surfacing the same green every tick is noise and burns the + per-run comment budget other PRs need); + - else `add_comment` on PR #: `✅ Attempt /10 validated — the fix's CI is + green on .` naming any `/azp`-gated legs (uitests def 313 / devicetests def + 314) that have not run and still need a maintainer `/azp run`; record `surfaced-green + PR # (attempt /10)`. (Do NOT assert "ready for human review" on a PR that + is still a draft — Step 3.6, not this comment, owns the draft→ready flip and posts its + own 🎯 announcement when it fires.) + Do NOT advance. Then — in ALL of the above cases — run **Step 3.6** (target-test + readiness gate) for this PR before stopping: its `/azp`-gated target legs may conclude + green on this SAME head SHA on a LATER sweep (an `/azp run` adds no commit), and Step + 3.6 is the ONLY place the draft→ready flip happens, so it MUST re-evaluate every sweep — + never `stop` here before it. *(This directly attacks the real bottleneck — no reviews — + so it is the highest-value outcome.)* 4. **Red → classify caused-by-fix vs unrelated-flake.** If `C.overallConclusion == "failure"`, analyze the PR's OWN failing build (NOT `main`): find the AzDO `maui-pr` build for this PR (filter builds by `branchName=refs/pull//merge`, @@ -723,18 +819,26 @@ Run these gates in order — the FIRST that fires decides this cycle's outcome: method and Step 4.7 flake buckets to `C.failedLegs`. - **Unrelated flake only** (every failed leg is known-flaky / infra / a pre-existing baseline red NOT introduced by the fix): do NOT burn an attempt. - **Idempotency first:** scan the PR's existing comments for a prior bot - `♻️ … unrelated flake … on ` note for THIS head SHA — if one already - exists, record `already-annotated-flake PR # (head )` and stop - WITHOUT re-commenting (re-annotating the same flake every 12h sweep is noise and - burns the per-run comment budget other PRs need). Otherwise resolve the attempt - number from `C.effectiveAttempt` (the authoritative max(marker, bot-commit) - counter), omitting the `Attempt /10:` prefix only if it is somehow - indeterminate. **Dry-run gate (Step 0):** if `dry_run == "true"`, do NOT emit any comment — instead print the intended `♻️` unrelated-flake body to the run log, tally `dry-run: would-annotate-flake PR #`, and stop. Otherwise `add_comment` on PR #: `♻️ Attempt /10: red is - unrelated flake on leg(s) () on ; the fix itself is not - implicated. A maintainer re-run (/azp run ) should clear it.` Record - `annotated-flake PR # (head )` and stop. *(Round 1: human re-runs; - Round 2: auto re-trigger.)* + **Comment idempotency + dry-run suppress the ♻️ comment ONLY — neither skips Step + 3.6.** Scan the PR's existing comments for a prior bot `♻️ … unrelated flake … on + ` note for THIS head SHA; if one exists, set `SKIP_FLAKE_COMMENT = true`. + Resolve the attempt number from `C.effectiveAttempt` (the authoritative max(marker, + bot-commit) counter), omitting the `Attempt /10:` prefix only if it is + somehow indeterminate. Then post the flake note UNLESS suppressed: + - if `dry_run == "true"`: do NOT emit — print the intended `♻️` unrelated-flake body + to the run log and tally `dry-run: would-annotate-flake PR #`; + - else if `SKIP_FLAKE_COMMENT`: do NOT re-post — record `already-annotated-flake PR + # (head )` (re-annotating the same flake every 12h sweep is noise and + burns the per-run comment budget other PRs need); + - else `add_comment` on PR #: `♻️ Attempt /10: red is unrelated flake on + leg(s) () on ; the fix itself is not implicated. A + maintainer re-run (/azp run ) should clear it.` record `annotated-flake + PR # (head )`. + Then — in ALL of the above cases — run **Step 3.6** (target-test readiness gate) for + this PR before stopping: its `/azp`-gated target legs may conclude green on this SAME + head SHA on a LATER sweep, and Step 3.6 is the ONLY place the draft→ready flip + happens, so it MUST re-evaluate every sweep — never `stop` here before it. *(Round 1: + human re-runs; Round 2: auto re-trigger.)* - **Caused by the fix** (a failed leg still matches the original target signature, or the fix introduced a NEW failure): advance an attempt. - **Attempt count.** `attempt = C.effectiveAttempt` — the authoritative @@ -752,6 +856,188 @@ Run these gates in order — the FIRST that fires decides this cycle's outcome: from every prior commit on this branch**, emitted via the Step 5.6 ADVANCE path (push onto the same PR). +#### Step 3.6 — Target-test verification & mark-ready gate + +Reached from the Step 3 **green-surface** branch, the Step 4 **unrelated-flake** branch +(AFTER that branch has posted its comment), and the Step 3.5 **gate-2 target-focused +fast-path** (2a — while unrelated legs are still pending). Purpose: confirm the SPECIFIC +test(s) this PR was opened to fix now PASS in the PR's own CI, and — only when they do +— transition the draft `[ci-fix]` PR to **ready for review** so a maintainer sees a +validated fix instead of a draft. This is the ONLY place the loop flips draft→ready; it +is a state transition, **never** an approval or a merge (a human still reviews and +merges). Overall red on *unrelated* legs must NOT gate readiness — we validate the fix, +not the base branch's flakiness. + +**Preconditions** (ALL must hold; otherwise record `skipped: readiness N/A PR # +()` and stop this gate): +- `C.isDraft == true` — if the PR is already ready-for-review, the draft→ready + transition is already done; do NOT re-mark **and do NOT re-apply `p/0`**. `p/0` is applied + exactly ONCE, atomically with the draft→ready flip (T3 — all-or-nothing with the 🎯 audit + comment + mark-ready), so an already-ready loop PR that is MISSING `p/0` has almost + certainly had it **removed by a maintainer** de-prioritizing the PR — a pure triage action + the human-engagement guard (which inspects comments/reviews/commits, NOT label changes) + cannot see. Re-adding `p/0` every sweep would fight that maintainer indefinitely, violating + the loop's "never override a human" contract. So the loop does NOT reconcile the label: + record `already-ready PR #` and stop this gate. (Trade-off: on the rare occasion a + transient API error drops `p/0` at flip time *after* the T3 atomic pre-check passed, it is + not auto-re-added — but the PR is still ready-for-review with its 🎯 audit comment, and + re-adding `p/0` is a trivial manual action; that is strictly preferable to steam-rolling a + maintainer's deliberate de-prioritization.) +- The PR is unmistakably THIS workflow's own: `[ci-fix] ` title prefix AND the + `agentic-workflows` label (mirrors the safe-output lock; the compensating scope + control if the v0.80.9 compiler drops the declarative required-*). +- You reached this gate from Step 3 (green), Step 4 (**unrelated-flake**), or the Step 3.5 + gate-2 **target-focused fast-path** (2a) — i.e. the fix is NOT implicated in any red that + has concluded. If Step 4 classified a red as **caused by the fix** (ADVANCE), do NOT run + this gate — advance the attempt instead. + +**T1 — Identify the target test(s).** From the `[ci-scan]` issue signature the fix +addresses (and the PR's own diff), extract the fully-qualified test method name(s) the +fix targets — e.g. `SafeAreaShouldWorkOnAllShellTabs`. For a de-flake it is the +de-flaked test; for a product fix it is the originally-failing test(s). If NO specific +test can be identified (e.g. a product build-break rather than a test failure), this is a +**build-only fix** — there is no single test to validate cross-platform, so validate at +**whole-build** granularity rather than stopping (Step 3 only *comments*; Step 3.6 is the +sole draft→ready flip, so a build-only fix is undrafted HERE or not at all). We only reach +this gate from Step 3 (green) / Step 4 (unrelated-flake) / gate-2a, so the fix is not +implicated in any concluded red; additionally require, via the T2 timeline method on +`C.headSha`, that the primary build pipeline `maui-pr` (def 302) has CONCLUDED with EVERY +one of its platform build legs `succeeded`/`completed` and NONE failed/canceled or still +pending — the build is green on **every** platform, not just the originally-broken one (the +cross-platform guard, applied to the build instead of a test). A build-only fix is undrafted +ONLY on the strength of the auto-running `maui-pr` (def 302) whole-build green, which the +loop CAN observe: if the PR's `[ci-scan]` issue signature or its own diff indicates the +ORIGINATING failure was in a `/azp`-gated pipeline (`maui-pr-uitests` def 313 or +`maui-pr-devicetests` def 314) rather than `maui-pr` (def 302), do NOT undraft on +`maui-pr`-green alone — those pipelines do not auto-run on this PR (GITHUB_TOKEN cannot +trigger them), so a green `maui-pr` build is NOT evidence the gated build break is fixed; +record `skipped: build-only fix PR # targets gated pipeline () not run — +deferring to human` and stop this gate. Otherwise, set `TARGET := "the maui-pr build +(build-only fix — no single target test)"` and proceed to **T3** to mark ready. If any `maui-pr` build leg is still unconcluded, record `skipped: build-only fix PR +# not yet whole-build green (leg(s) pending)` and stop this gate WITHOUT marking +ready (a green subset is not enough — a still-pending build leg could yet fail). + +**T2 — Verify the target test's platform legs are green (anonymous, leg-level).** Per-test +outcomes come from the AzDO **test-results** API (`_apis/test/...`), which is **NOT reachable +anonymously — Hard-Rule 8**: those endpoints 302-redirect to a sign-in page on this runner, so +a `curl` returns an HTML redirect (not JSON) and every target test would look "not executed". +Validate instead at **leg granularity** using ONLY the anonymous `_apis/build/...` timeline +(Hard-Rule 8) — a `succeeded` category leg means every test in that category **that actually +ran** passed on that platform. This is a strong bar **only if the target test genuinely +executes**: a job can go green while one specific test is skipped at runtime (`[Ignore]`, +`Assert.Ignore()`, `Assert.Inconclusive()`, a `Skip=`/conditional `[Fact]`, an `#if`-out, or +an early `return` before the asserts). So before trusting a green leg as proof the target +passed, **confirm from the PR's OWN diff that the fix does NOT skip, ignore, disable, or +short-circuit the target test** (it adds no `[Ignore]`/`[Explicit]`, `Assert.Ignore`/ +`Assert.Inconclusive`, `Skip=`, category-exclusion, `#if`-out, or early `return` around the +target). If the fix could cause the target to be runtime-skipped rather than genuinely pass, +leg-level green is NOT sufficient evidence — record `skipped: target test may be +runtime-skipped by this fix (diff adds ); leg-green insufficient, deferring to human +on PR #` and stop this gate WITHOUT marking ready. Otherwise (the target genuinely runs +and asserts, as a de-flake or product fix does) a green category leg cannot hide a +target-test failure, so treat it as authoritative. + +Map the target test to the CI leg(s) that run it. A leg name encodes platform + test category +(e.g. `Android UITests SafeAreaEdges,Shadow`, `iOS UITests SafeAreaEdges,Shadow`, `macOS +UITests SafeAreaEdges,Shadow`, `Windows UITests SafeAreaEdges,Shadow`). Determine the target +test's UI-test category — its `[Category(UITestCategories.X)]` in the test/HostApp source, +visible in the PR diff or the test file — to know which leg-name substring identifies its legs; +for a device test, the per-platform device-test legs. + +Using the SAME build-discovery as Step 4 (filter AzDO builds by `branchName=refs/pull//merge` +or `sourceVersion == C.headSha`), read each build's **timeline** on `C.headSha` for the +pipeline(s) that RUN the target test — `maui-pr` (def 302) for unit/integration tests, +`maui-pr-uitests` (def 313) for Appium UI tests, `maui-pr-devicetests` (def 314) for device +tests: + +```bash +ORG=dnceng-public; PROJ=public; BUILD= +# Anonymous + Hard-Rule-8-compliant. NEVER call _apis/test/... (it 302-redirects to sign-in). +# Each type=="Job" record is one platform leg: result (succeeded/failed/canceled/null), +# state (completed/inProgress/pending), name (encodes " UITests "): +curl -s "https://dev.azure.com/$ORG/$PROJ/_apis/build/builds/$BUILD/timeline?api-version=7.1" \ + | tee /tmp/gh-aw/agent/timeline_${BUILD}.json \ + | jq -r '.records[] | select(.type=="Job") | "\(.result)\t\(.state)\t\(.name)"' +``` + +(The prefetch already exposes per-leg status in `C.failedLegs` / the checks context, and +`gh pr checks ` lists the same per-leg rows with platform+category in the name — use +whichever is handy; the build timeline is the authoritative cross-check on the exact build.) + +Treat a target test as **VALIDATED-GREEN** only if, on `C.headSha`, the leg that runs its +category is green on **every platform it runs on** — a fix that repairs one platform must not +silently regress the same test's leg on another, so a green leg on the originally-red platform +alone is NOT enough. Across the target pipeline's platform legs (for a UI test: the Android, +iOS, Windows, and macOS/MacCatalyst legs running the target's category; for a device test: each +device-test platform), require ALL of: +- the target's category leg is `result == "succeeded"` **and** `state == "completed"` on + **every** platform that runs it, AND +- that leg is `failed` / `canceled` / aborted on **NO** platform, AND +- **no platform is left unverified.** For each platform family the target pipeline covers, that + platform's category leg must have CONCLUDED (`state == "completed"`) on `C.headSha`. If a + platform simply has no leg for the target's category, the test does not run there — that is + fine, not a gap. But if a platform's category leg has **not concluded** (`state` is + `inProgress` / pending, or an `/azp`-gated `maui-pr-uitests` / `maui-pr-devicetests` leg that + has not been kicked), the test's status on that platform is UNKNOWN → the fix is NOT yet + validated across platforms: record `skipped: target test green on but + not yet verified on (leg(s) pending / need /azp run) on PR #` + and stop this gate WITHOUT marking ready. + +A target test whose category leg never concluded on ANY platform (**not executed** anywhere — +e.g. its `/azp`-gated pipeline has not been kicked) is likewise NOT validated: record `skipped: +target test not yet executed on PR # ( not run — needs /azp run)` and stop +this gate WITHOUT marking ready. Do NOT overclaim — a green *sibling* leg (a different category +on the same platform) is not the target's leg, and a green leg on one platform is not a pass on +the others. + +**T3 — Mark ready + report.** If EVERY target test is VALIDATED-GREEN. The 🎯 comment, +the mark-ready, and the `p/0` label are THREE SEPARATE safe-outputs — the comment +existing does NOT prove the mark-ready took effect, so they are tracked independently: + +- **Comment idempotency (dup-suppress only — never gates the mark-ready).** We only reach + T3 while `C.isDraft == true` (the precondition stops an already-ready PR before here). So + if a prior bot comment for THIS head that carries the leading `🎯` anchor AND the + contiguous phrase `validated green on ` already exists (BOTH T3 variants — + `🎯 Target test validated green on ` and the build-only `🎯 Build validated green on + ` — contain that exact contiguous phrase, so this recognizes both; keying on + `target test` alone would instead miss the build-only comment and re-post it every sweep. + **Require the `🎯` anchor** so the match can NEVER be loosened to a gapped + `validated…green…on` form that would false-positive on the Step 3 `✅ … validated — the + fix's CI is green on ` surface comment and wrongly suppress the audit 🎯), the comment + landed on an earlier sweep but the **mark-ready did NOT take + effect** (the PR is still a draft) — set `SUPPRESS_COMMENT = true` (do not re-post the + duplicate 🎯 comment) but STILL complete the mark-ready + label below. Never treat the + comment's existence as "already marked ready" while the PR is still a draft. Record this + case as `re-marking PR # (🎯 present; mark-ready did not take on a prior sweep)`. +- **Atomicity budget pre-check (Hard-Rule 7).** Determine the outputs this gate will emit: + `mark_pull_request_as_ready_for_review` + `add_labels` always, plus `add_comment` unless + `SUPPRESS_COMMENT`. Verify a free per-run slot remains in EACH bucket you are about to + use. If ANY required bucket is exhausted, emit NONE of them — record `skipped: per-run + cap reached; deferring mark-ready PR # to next cycle` and stop this gate. Never mark a + PR ready (or label it) unless its 🎯 audit comment is GUARANTEED to exist for this exact + head SHA — either posted in this same sweep, or (in the `SUPPRESS_COMMENT` re-marking case) + already present from a prior sweep. The outputs you DO emit either all land or all defer + together; the audit trail must never be absent, but it is NOT re-posted when it already exists. +- **Dry-run gate (Step 0):** if `dry_run == "true"`, emit NOTHING — print the intended + readiness comment and "would mark ready + add p/0" to the run log, tally `dry-run: + would-mark-ready PR #`, and stop. +- Otherwise emit for THIS PR number ``: + 1. `add_comment` (ONLY if not `SUPPRESS_COMMENT`): `🎯 Target test validated green on + — passed on ALL platforms it runs on (, + buildId ). — not + caused by this fix."> Transitioning this PR from draft to ready for review and adding + `p/0`; a maintainer still reviews and merges.` (For a **build-only fix** — the T1 + whole-build fallback, no single target test — phrase the first clause as `🎯 Build + validated green on — the maui-pr build passed on ALL platforms (buildId + ).` instead of naming a test.) + 2. `mark_pull_request_as_ready_for_review` with `reason:` a one-line justification + naming the validated test(s) and ``. + 3. `add_labels` with `labels: ["p/0"]` for PR # — put the now-review-ready fix into + the team's p/0 priority queue so it is triaged, not lost in the draft backlog. (If + the PR somehow already carries `p/0`, this is a harmless no-op.) +- Record `marked-ready PR # (target green on ALL platforms on , + labeled p/0)` and stop. + #### Step 3.5.R — Maintainer change-request response (Track C) Reached from Step 3.5 gate 0. Goal: when an **eligible human reviewer** (Hard-Rule @@ -1383,7 +1669,19 @@ Filed by [`ci-status-fix`](https://github.com/dotnet/maui/blob/main/.github/work ### Step 8 — Per-issue tally + end-of-run summary -Per issue, append one outcome line to `/tmp/gh-aw/agent/coverage.txt`: +Per issue, append **one** outcome line to `/tmp/gh-aw/agent/coverage.txt` — the +terminal outcome for the cycle. When one cycle produces a chained pair — a PR gets a +same-cycle precursor line and **then** a Step 3.6 readiness line (`marked-ready` on the +draft→ready flip, or `already-ready` on a later steady-state sweep of an already-ready +PR) — record ONLY the terminal Step 3.6 readiness line: it supersedes ANY same-cycle +non-readiness precursor for that PR, so an aggregator keying on one-line-per-issue never +double-counts. The precursor may be a Step 3 green line (`surfaced-green` on the first ✅, +or `already-surfaced` when it already exists) OR — when an unrelated-flake red and a +target-green coincide — a Step 4 `annotated-flake` line; either way the readiness line is +terminal, and the superseded precursor's signal survives in the PR's 🎯/♻️ comment so no +information is lost. This covers the flip cycle (`surfaced-green` → `marked-ready`), the +post-flip steady state (`already-surfaced` → `already-ready`), and the flake-coincident +flip (`annotated-flake` → `marked-ready`): ``` # main attempt- @@ -1394,8 +1692,14 @@ Per issue, append one outcome line to `/tmp/gh-aw/agent/coverage.txt`: `advance-PR # attempt /10` (ADVANCE mode — new commit pushed to the existing PR), `surfaced-green PR #` (fix's own CI went green; commented for review, did not advance), `annotated-flake PR #` (red was unrelated flake; -commented, attempt NOT burned), `waiting PR #` (CI not settled yet), -`dry-run: would-`, `skipped: `. (The +commented, attempt NOT burned), `marked-ready PR #` (the specific fixed test +was confirmed green in the PR's own CI; draft flipped to ready for review), +`already-surfaced PR #` (the fix's ✅ green comment already existed this sweep; +re-post suppressed — superseded by the Step 3.6 readiness line when the PR also flips +ready that cycle), `already-ready PR #` (terminal steady state — the PR was flipped +ready on a prior cycle and remains green on an unchanged head; no action taken, +supersedes `already-surfaced`), `waiting PR #` (CI not settled yet), +`dry-run: would-`, `skipped: `. (The `needs-human-PR` outcome is reserved for the deferred hand-off — Step 6 currently records a skip instead, so it is not emitted.)
to next cycle` for each remaining PR so the drop is a deliberate, logged decision. The continuous loop picks the deferred PRs up next @@ -409,8 +471,9 @@ For every open tracking issue in scope, converge on exactly one outcome: | Open first help-wanted draft `[ci-fix]` PR | No open PR yet; a plausible candidate exists but cannot be runner-validated (device/UI tests) (attempt 1) | | Open first de-flake draft `[ci-fix]` PR | No open PR yet; failure is intermittent (green-on-retry) from a genuine test-quality defect; a deterministic-synchronization fix is producible (attempt 1, Step 4.7 bucket b) | | Advance an existing PR (push attempt N+1) | Open PR's own CI settled red *because of the fix itself*, no human engaged, marker < 10 — push a NEW distinct fix onto the same branch (Step 3.5 → 5.6) | -| Surface a validated-green PR | Open PR's own CI settled green — comment "validated, attempt N/10; ready for review" and do NOT advance (Step 3.5) | -| Annotate an unrelated-flake red | Open PR is red only on baseline-flake / unrelated legs — comment which leg needs a re-run; do NOT burn an attempt (Step 3.5) | +| Surface a validated-green PR | Open PR's own CI settled green — comment "validated, attempt N/10; ready for review" and do NOT advance (Step 3.5), then mark the draft PR ready for review once the fixed test is confirmed green (Step 3.6) | +| Annotate an unrelated-flake red | Open PR is red only on baseline-flake / unrelated legs — comment which leg needs a re-run; do NOT burn an attempt (Step 3.5); if the SPECIFIC fixed test is confirmed green on that build, still mark the draft PR ready for review (Step 3.6) | +| Mark a validated PR ready for review | The specific test this PR fixed is confirmed green in the PR's own CI (even if unrelated legs are red) — comment the target-test result and transition the draft PR to ready for review; never approve or merge (Step 3.6) | | Wait | Open PR's CI is pending / not yet settled on the current head SHA — do nothing this cycle (Step 3.5) | | Hand-off skip (attempt cap) | Marker == 10 and the signature still reproduces — stop and defer to humans; the dedicated `[ci-fix][needs-human]` PR is planned but currently deferred (Step 6) | | Recorded skip | Visual-regression, human engaged, already-handled, fixed-in-latest-build, infra-flake, out-of-bounds, only-mute-available, or no novel approach producible | @@ -685,14 +748,34 @@ Run these gates in order — the FIRST that fires decides this cycle's outcome: 1. **Human engaged.** If `C.humanEngaged` → `skipped: human engaged on PR #
CI pending / prefetch - incomplete on ; waiting` and stop. *(Round 1: this is where a - maintainer `/azp run` is awaited — the `/azp`-gated uitests/devicetests legs will - not have run until a human kicks them.)* +2. **CI not settled — but validate the target test first (target-focused readiness).** + If `C.checksSettled == false` OR `C.overallConclusion` is `pending`, `neutral`, or + `unknown`, OR `C.dataComplete == false` → the *overall* build has not finished on the + current head SHA (`C.headSha`), or the prefetch could not fully read this PR (a partial + read can understate `humanEngaged` / `attempt`). Unrelated legs still draining must NOT + keep an already-proven fix parked as a draft, so before waiting, try the target-focused + fast-path: + - **2a — Target-green fast-path (draft `[ci-fix]` PRs only).** If `C.dataComplete == + true` AND the Step 3.6 preconditions hold (draft PR, `[ci-fix] ` title prefix AND the + `agentic-workflows` label), run Step 3.6 **T1–T2** against `C.headSha` now: identify + the target test(s) and read the PR's OWN build **timeline / per-leg status** (anonymous + `_apis/build` — never `_apis/test`, per Hard-Rule 8). If EVERY target test is + **VALIDATED-GREEN** (per T2's all-platform definition below — the target's category leg + is `succeeded` on every platform that runs it, failed on none, and NO target platform + family the pipeline covers is still pending/unconcluded, per T2's "no platform left + unverified" rule; a platform that simply has no leg for the target's category — the test + doesn't run there — is fine, not a gap) AND every leg in + `C.failedLegs` that has ALREADY concluded classifies as **unrelated flake** (Step 4 / + Step 4.7 method — the fix is implicated in NO completed red), then the fix is proven + regardless of unrelated *pending* legs → run **Step 3.6 T3** (mark ready + 🎯 comment) + and stop. This early-out NEVER advances an attempt and NEVER acts on a red that could + be the fix's fault. + - **2b — Otherwise WAIT.** If the target test has not yet executed (pending/absent on + its leg), or a completed red is (or may be) caused by the fix, or `C.dataComplete == + false`, or this PR has no identifiable target test → `skipped: PR # CI pending / + target not yet validated on ; waiting` and stop. *(Round 1: this is where a + maintainer `/azp run` is awaited — the `/azp`-gated uitests/devicetests legs will not + have run until a human kicks them.)* 3. **Green → surface for review.** If `C.overallConclusion == "success"`: the checks that RAN are green. **Primary-gate check first:** the deterministic `success` verdict only certifies "at least one green check, nothing failing or @@ -704,18 +787,31 @@ Run these gates in order — the FIRST that fires decides this cycle's outcome: # primary CI gate (maui-pr) not green on ; waiting` and stop — do NOT surface. (The `/azp`-gated `maui-pr-uitests` (def 313) / `maui-pr-devicetests` (def 314) legs MAY still be un-run — that is expected and is named below, not a - reason to withhold the surface.) **Idempotency:** scan the PR's existing comments - for a prior bot `✅ … validated … on ` note for THIS head SHA — if one - already exists, record `already-surfaced PR # (head )` and stop - WITHOUT re-commenting (re-surfacing the same green every tick is noise and burns - the per-run comment budget other PRs need). Otherwise resolve the attempt number from - `C.effectiveAttempt` (the authoritative max(marker, bot-commit) counter; omit the - number only if it is somehow indeterminate). **Dry-run gate (Step 0):** if `dry_run == "true"`, do NOT emit any comment — instead print the intended `✅` validated-green body to the run log, tally `dry-run: would-surface-green PR #`, and stop. Otherwise `add_comment` on PR #: `✅ Attempt /10 validated - — the fix's CI is green on . Ready for human review.` - Name any `/azp`-gated legs (uitests def 313 / devicetests def 314) that have not - run and still need a maintainer `/azp run`. Do NOT advance. Record - `surfaced-green PR # (attempt /10)` and stop. *(This directly - attacks the real bottleneck — no reviews — so it is the highest-value outcome.)* + reason to withhold the surface.) **Comment idempotency + dry-run suppress the ✅ + comment ONLY — neither skips Step 3.6.** Scan the PR's existing comments for a prior + bot `✅ … validated … on ` note for THIS head SHA; if one exists, set + `SKIP_SURFACE_COMMENT = true`. Resolve the attempt number from `C.effectiveAttempt` + (the authoritative max(marker, bot-commit) counter; omit the number only if it is + somehow indeterminate). Then post the surface comment UNLESS suppressed: + - if `dry_run == "true"`: do NOT emit — print the intended `✅` validated-green body to + the run log and tally `dry-run: would-surface-green PR #`; + - else if `SKIP_SURFACE_COMMENT`: do NOT re-post — record `already-surfaced PR # + (head )` (re-surfacing the same green every tick is noise and burns the + per-run comment budget other PRs need); + - else `add_comment` on PR #: `✅ Attempt /10 validated — the fix's CI is + green on .` naming any `/azp`-gated legs (uitests def 313 / devicetests def + 314) that have not run and still need a maintainer `/azp run`; record `surfaced-green + PR # (attempt /10)`. (Do NOT assert "ready for human review" on a PR that + is still a draft — Step 3.6, not this comment, owns the draft→ready flip and posts its + own 🎯 announcement when it fires.) + Do NOT advance. Then — in ALL of the above cases — run **Step 3.6** (target-test + readiness gate) for this PR before stopping: its `/azp`-gated target legs may conclude + green on this SAME head SHA on a LATER sweep (an `/azp run` adds no commit), and Step + 3.6 is the ONLY place the draft→ready flip happens, so it MUST re-evaluate every sweep — + never `stop` here before it. *(This directly attacks the real bottleneck — no reviews — + so it is the highest-value outcome.)* 4. **Red → classify caused-by-fix vs unrelated-flake.** If `C.overallConclusion == "failure"`, analyze the PR's OWN failing build (NOT `main`): find the AzDO `maui-pr` build for this PR (filter builds by `branchName=refs/pull//merge`, @@ -723,18 +819,26 @@ Run these gates in order — the FIRST that fires decides this cycle's outcome: method and Step 4.7 flake buckets to `C.failedLegs`. - **Unrelated flake only** (every failed leg is known-flaky / infra / a pre-existing baseline red NOT introduced by the fix): do NOT burn an attempt. - **Idempotency first:** scan the PR's existing comments for a prior bot - `♻️ … unrelated flake … on ` note for THIS head SHA — if one already - exists, record `already-annotated-flake PR # (head )` and stop - WITHOUT re-commenting (re-annotating the same flake every 12h sweep is noise and - burns the per-run comment budget other PRs need). Otherwise resolve the attempt - number from `C.effectiveAttempt` (the authoritative max(marker, bot-commit) - counter), omitting the `Attempt /10:` prefix only if it is somehow - indeterminate. **Dry-run gate (Step 0):** if `dry_run == "true"`, do NOT emit any comment — instead print the intended `♻️` unrelated-flake body to the run log, tally `dry-run: would-annotate-flake PR #`, and stop. Otherwise `add_comment` on PR #: `♻️ Attempt /10: red is - unrelated flake on leg(s) () on ; the fix itself is not - implicated. A maintainer re-run (/azp run ) should clear it.` Record - `annotated-flake PR # (head )` and stop. *(Round 1: human re-runs; - Round 2: auto re-trigger.)* + **Comment idempotency + dry-run suppress the ♻️ comment ONLY — neither skips Step + 3.6.** Scan the PR's existing comments for a prior bot `♻️ … unrelated flake … on + ` note for THIS head SHA; if one exists, set `SKIP_FLAKE_COMMENT = true`. + Resolve the attempt number from `C.effectiveAttempt` (the authoritative max(marker, + bot-commit) counter), omitting the `Attempt /10:` prefix only if it is + somehow indeterminate. Then post the flake note UNLESS suppressed: + - if `dry_run == "true"`: do NOT emit — print the intended `♻️` unrelated-flake body + to the run log and tally `dry-run: would-annotate-flake PR #`; + - else if `SKIP_FLAKE_COMMENT`: do NOT re-post — record `already-annotated-flake PR + # (head )` (re-annotating the same flake every 12h sweep is noise and + burns the per-run comment budget other PRs need); + - else `add_comment` on PR #: `♻️ Attempt /10: red is unrelated flake on + leg(s) () on ; the fix itself is not implicated. A + maintainer re-run (/azp run ) should clear it.` record `annotated-flake + PR # (head )`. + Then — in ALL of the above cases — run **Step 3.6** (target-test readiness gate) for + this PR before stopping: its `/azp`-gated target legs may conclude green on this SAME + head SHA on a LATER sweep, and Step 3.6 is the ONLY place the draft→ready flip + happens, so it MUST re-evaluate every sweep — never `stop` here before it. *(Round 1: + human re-runs; Round 2: auto re-trigger.)* - **Caused by the fix** (a failed leg still matches the original target signature, or the fix introduced a NEW failure): advance an attempt. - **Attempt count.** `attempt = C.effectiveAttempt` — the authoritative @@ -752,6 +856,188 @@ Run these gates in order — the FIRST that fires decides this cycle's outcome: from every prior commit on this branch**, emitted via the Step 5.6 ADVANCE path (push onto the same PR). +#### Step 3.6 — Target-test verification & mark-ready gate + +Reached from the Step 3 **green-surface** branch, the Step 4 **unrelated-flake** branch +(AFTER that branch has posted its comment), and the Step 3.5 **gate-2 target-focused +fast-path** (2a — while unrelated legs are still pending). Purpose: confirm the SPECIFIC +test(s) this PR was opened to fix now PASS in the PR's own CI, and — only when they do +— transition the draft `[ci-fix]` PR to **ready for review** so a maintainer sees a +validated fix instead of a draft. This is the ONLY place the loop flips draft→ready; it +is a state transition, **never** an approval or a merge (a human still reviews and +merges). Overall red on *unrelated* legs must NOT gate readiness — we validate the fix, +not the base branch's flakiness. + +**Preconditions** (ALL must hold; otherwise record `skipped: readiness N/A PR # +()` and stop this gate): +- `C.isDraft == true` — if the PR is already ready-for-review, the draft→ready + transition is already done; do NOT re-mark **and do NOT re-apply `p/0`**. `p/0` is applied + exactly ONCE, atomically with the draft→ready flip (T3 — all-or-nothing with the 🎯 audit + comment + mark-ready), so an already-ready loop PR that is MISSING `p/0` has almost + certainly had it **removed by a maintainer** de-prioritizing the PR — a pure triage action + the human-engagement guard (which inspects comments/reviews/commits, NOT label changes) + cannot see. Re-adding `p/0` every sweep would fight that maintainer indefinitely, violating + the loop's "never override a human" contract. So the loop does NOT reconcile the label: + record `already-ready PR #` and stop this gate. (Trade-off: on the rare occasion a + transient API error drops `p/0` at flip time *after* the T3 atomic pre-check passed, it is + not auto-re-added — but the PR is still ready-for-review with its 🎯 audit comment, and + re-adding `p/0` is a trivial manual action; that is strictly preferable to steam-rolling a + maintainer's deliberate de-prioritization.) +- The PR is unmistakably THIS workflow's own: `[ci-fix] ` title prefix AND the + `agentic-workflows` label (mirrors the safe-output lock; the compensating scope + control if the v0.80.9 compiler drops the declarative required-*). +- You reached this gate from Step 3 (green), Step 4 (**unrelated-flake**), or the Step 3.5 + gate-2 **target-focused fast-path** (2a) — i.e. the fix is NOT implicated in any red that + has concluded. If Step 4 classified a red as **caused by the fix** (ADVANCE), do NOT run + this gate — advance the attempt instead. + +**T1 — Identify the target test(s).** From the `[ci-scan]` issue signature the fix +addresses (and the PR's own diff), extract the fully-qualified test method name(s) the +fix targets — e.g. `SafeAreaShouldWorkOnAllShellTabs`. For a de-flake it is the +de-flaked test; for a product fix it is the originally-failing test(s). If NO specific +test can be identified (e.g. a product build-break rather than a test failure), this is a +**build-only fix** — there is no single test to validate cross-platform, so validate at +**whole-build** granularity rather than stopping (Step 3 only *comments*; Step 3.6 is the +sole draft→ready flip, so a build-only fix is undrafted HERE or not at all). We only reach +this gate from Step 3 (green) / Step 4 (unrelated-flake) / gate-2a, so the fix is not +implicated in any concluded red; additionally require, via the T2 timeline method on +`C.headSha`, that the primary build pipeline `maui-pr` (def 302) has CONCLUDED with EVERY +one of its platform build legs `succeeded`/`completed` and NONE failed/canceled or still +pending — the build is green on **every** platform, not just the originally-broken one (the +cross-platform guard, applied to the build instead of a test). A build-only fix is undrafted +ONLY on the strength of the auto-running `maui-pr` (def 302) whole-build green, which the +loop CAN observe: if the PR's `[ci-scan]` issue signature or its own diff indicates the +ORIGINATING failure was in a `/azp`-gated pipeline (`maui-pr-uitests` def 313 or +`maui-pr-devicetests` def 314) rather than `maui-pr` (def 302), do NOT undraft on +`maui-pr`-green alone — those pipelines do not auto-run on this PR (GITHUB_TOKEN cannot +trigger them), so a green `maui-pr` build is NOT evidence the gated build break is fixed; +record `skipped: build-only fix PR # targets gated pipeline () not run — +deferring to human` and stop this gate. Otherwise, set `TARGET := "the maui-pr build +(build-only fix — no single target test)"` and proceed to **T3** to mark ready. If any `maui-pr` build leg is still unconcluded, record `skipped: build-only fix PR +# not yet whole-build green (leg(s) pending)` and stop this gate WITHOUT marking +ready (a green subset is not enough — a still-pending build leg could yet fail). + +**T2 — Verify the target test's platform legs are green (anonymous, leg-level).** Per-test +outcomes come from the AzDO **test-results** API (`_apis/test/...`), which is **NOT reachable +anonymously — Hard-Rule 8**: those endpoints 302-redirect to a sign-in page on this runner, so +a `curl` returns an HTML redirect (not JSON) and every target test would look "not executed". +Validate instead at **leg granularity** using ONLY the anonymous `_apis/build/...` timeline +(Hard-Rule 8) — a `succeeded` category leg means every test in that category **that actually +ran** passed on that platform. This is a strong bar **only if the target test genuinely +executes**: a job can go green while one specific test is skipped at runtime (`[Ignore]`, +`Assert.Ignore()`, `Assert.Inconclusive()`, a `Skip=`/conditional `[Fact]`, an `#if`-out, or +an early `return` before the asserts). So before trusting a green leg as proof the target +passed, **confirm from the PR's OWN diff that the fix does NOT skip, ignore, disable, or +short-circuit the target test** (it adds no `[Ignore]`/`[Explicit]`, `Assert.Ignore`/ +`Assert.Inconclusive`, `Skip=`, category-exclusion, `#if`-out, or early `return` around the +target). If the fix could cause the target to be runtime-skipped rather than genuinely pass, +leg-level green is NOT sufficient evidence — record `skipped: target test may be +runtime-skipped by this fix (diff adds ); leg-green insufficient, deferring to human +on PR #` and stop this gate WITHOUT marking ready. Otherwise (the target genuinely runs +and asserts, as a de-flake or product fix does) a green category leg cannot hide a +target-test failure, so treat it as authoritative. + +Map the target test to the CI leg(s) that run it. A leg name encodes platform + test category +(e.g. `Android UITests SafeAreaEdges,Shadow`, `iOS UITests SafeAreaEdges,Shadow`, `macOS +UITests SafeAreaEdges,Shadow`, `Windows UITests SafeAreaEdges,Shadow`). Determine the target +test's UI-test category — its `[Category(UITestCategories.X)]` in the test/HostApp source, +visible in the PR diff or the test file — to know which leg-name substring identifies its legs; +for a device test, the per-platform device-test legs. + +Using the SAME build-discovery as Step 4 (filter AzDO builds by `branchName=refs/pull//merge` +or `sourceVersion == C.headSha`), read each build's **timeline** on `C.headSha` for the +pipeline(s) that RUN the target test — `maui-pr` (def 302) for unit/integration tests, +`maui-pr-uitests` (def 313) for Appium UI tests, `maui-pr-devicetests` (def 314) for device +tests: + +```bash +ORG=dnceng-public; PROJ=public; BUILD= +# Anonymous + Hard-Rule-8-compliant. NEVER call _apis/test/... (it 302-redirects to sign-in). +# Each type=="Job" record is one platform leg: result (succeeded/failed/canceled/null), +# state (completed/inProgress/pending), name (encodes " UITests "): +curl -s "https://dev.azure.com/$ORG/$PROJ/_apis/build/builds/$BUILD/timeline?api-version=7.1" \ + | tee /tmp/gh-aw/agent/timeline_${BUILD}.json \ + | jq -r '.records[] | select(.type=="Job") | "\(.result)\t\(.state)\t\(.name)"' +``` + +(The prefetch already exposes per-leg status in `C.failedLegs` / the checks context, and +`gh pr checks ` lists the same per-leg rows with platform+category in the name — use +whichever is handy; the build timeline is the authoritative cross-check on the exact build.) + +Treat a target test as **VALIDATED-GREEN** only if, on `C.headSha`, the leg that runs its +category is green on **every platform it runs on** — a fix that repairs one platform must not +silently regress the same test's leg on another, so a green leg on the originally-red platform +alone is NOT enough. Across the target pipeline's platform legs (for a UI test: the Android, +iOS, Windows, and macOS/MacCatalyst legs running the target's category; for a device test: each +device-test platform), require ALL of: +- the target's category leg is `result == "succeeded"` **and** `state == "completed"` on + **every** platform that runs it, AND +- that leg is `failed` / `canceled` / aborted on **NO** platform, AND +- **no platform is left unverified.** For each platform family the target pipeline covers, that + platform's category leg must have CONCLUDED (`state == "completed"`) on `C.headSha`. If a + platform simply has no leg for the target's category, the test does not run there — that is + fine, not a gap. But if a platform's category leg has **not concluded** (`state` is + `inProgress` / pending, or an `/azp`-gated `maui-pr-uitests` / `maui-pr-devicetests` leg that + has not been kicked), the test's status on that platform is UNKNOWN → the fix is NOT yet + validated across platforms: record `skipped: target test green on but + not yet verified on (leg(s) pending / need /azp run) on PR #` + and stop this gate WITHOUT marking ready. + +A target test whose category leg never concluded on ANY platform (**not executed** anywhere — +e.g. its `/azp`-gated pipeline has not been kicked) is likewise NOT validated: record `skipped: +target test not yet executed on PR # ( not run — needs /azp run)` and stop +this gate WITHOUT marking ready. Do NOT overclaim — a green *sibling* leg (a different category +on the same platform) is not the target's leg, and a green leg on one platform is not a pass on +the others. + +**T3 — Mark ready + report.** If EVERY target test is VALIDATED-GREEN. The 🎯 comment, +the mark-ready, and the `p/0` label are THREE SEPARATE safe-outputs — the comment +existing does NOT prove the mark-ready took effect, so they are tracked independently: + +- **Comment idempotency (dup-suppress only — never gates the mark-ready).** We only reach + T3 while `C.isDraft == true` (the precondition stops an already-ready PR before here). So + if a prior bot comment for THIS head that carries the leading `🎯` anchor AND the + contiguous phrase `validated green on ` already exists (BOTH T3 variants — + `🎯 Target test validated green on ` and the build-only `🎯 Build validated green on + ` — contain that exact contiguous phrase, so this recognizes both; keying on + `target test` alone would instead miss the build-only comment and re-post it every sweep. + **Require the `🎯` anchor** so the match can NEVER be loosened to a gapped + `validated…green…on` form that would false-positive on the Step 3 `✅ … validated — the + fix's CI is green on ` surface comment and wrongly suppress the audit 🎯), the comment + landed on an earlier sweep but the **mark-ready did NOT take + effect** (the PR is still a draft) — set `SUPPRESS_COMMENT = true` (do not re-post the + duplicate 🎯 comment) but STILL complete the mark-ready + label below. Never treat the + comment's existence as "already marked ready" while the PR is still a draft. Record this + case as `re-marking PR # (🎯 present; mark-ready did not take on a prior sweep)`. +- **Atomicity budget pre-check (Hard-Rule 7).** Determine the outputs this gate will emit: + `mark_pull_request_as_ready_for_review` + `add_labels` always, plus `add_comment` unless + `SUPPRESS_COMMENT`. Verify a free per-run slot remains in EACH bucket you are about to + use. If ANY required bucket is exhausted, emit NONE of them — record `skipped: per-run + cap reached; deferring mark-ready PR # to next cycle` and stop this gate. Never mark a + PR ready (or label it) unless its 🎯 audit comment is GUARANTEED to exist for this exact + head SHA — either posted in this same sweep, or (in the `SUPPRESS_COMMENT` re-marking case) + already present from a prior sweep. The outputs you DO emit either all land or all defer + together; the audit trail must never be absent, but it is NOT re-posted when it already exists. +- **Dry-run gate (Step 0):** if `dry_run == "true"`, emit NOTHING — print the intended + readiness comment and "would mark ready + add p/0" to the run log, tally `dry-run: + would-mark-ready PR #`, and stop. +- Otherwise emit for THIS PR number ``: + 1. `add_comment` (ONLY if not `SUPPRESS_COMMENT`): `🎯 Target test validated green on + — passed on ALL platforms it runs on (, + buildId ). — not + caused by this fix."> Transitioning this PR from draft to ready for review and adding + `p/0`; a maintainer still reviews and merges.` (For a **build-only fix** — the T1 + whole-build fallback, no single target test — phrase the first clause as `🎯 Build + validated green on — the maui-pr build passed on ALL platforms (buildId + ).` instead of naming a test.) + 2. `mark_pull_request_as_ready_for_review` with `reason:` a one-line justification + naming the validated test(s) and ``. + 3. `add_labels` with `labels: ["p/0"]` for PR # — put the now-review-ready fix into + the team's p/0 priority queue so it is triaged, not lost in the draft backlog. (If + the PR somehow already carries `p/0`, this is a harmless no-op.) +- Record `marked-ready PR # (target green on ALL platforms on , + labeled p/0)` and stop. + #### Step 3.5.R — Maintainer change-request response (Track C) Reached from Step 3.5 gate 0. Goal: when an **eligible human reviewer** (Hard-Rule @@ -1383,7 +1669,19 @@ Filed by [`ci-status-fix`](https://github.com/dotnet/maui/blob/main/.github/work ### Step 8 — Per-issue tally + end-of-run summary -Per issue, append one outcome line to `/tmp/gh-aw/agent/coverage.txt`: +Per issue, append **one** outcome line to `/tmp/gh-aw/agent/coverage.txt` — the +terminal outcome for the cycle. When one cycle produces a chained pair — a PR gets a +same-cycle precursor line and **then** a Step 3.6 readiness line (`marked-ready` on the +draft→ready flip, or `already-ready` on a later steady-state sweep of an already-ready +PR) — record ONLY the terminal Step 3.6 readiness line: it supersedes ANY same-cycle +non-readiness precursor for that PR, so an aggregator keying on one-line-per-issue never +double-counts. The precursor may be a Step 3 green line (`surfaced-green` on the first ✅, +or `already-surfaced` when it already exists) OR — when an unrelated-flake red and a +target-green coincide — a Step 4 `annotated-flake` line; either way the readiness line is +terminal, and the superseded precursor's signal survives in the PR's 🎯/♻️ comment so no +information is lost. This covers the flip cycle (`surfaced-green` → `marked-ready`), the +post-flip steady state (`already-surfaced` → `already-ready`), and the flake-coincident +flip (`annotated-flake` → `marked-ready`): ``` # main attempt- @@ -1394,8 +1692,14 @@ Per issue, append one outcome line to `/tmp/gh-aw/agent/coverage.txt`: `advance-PR # attempt /10` (ADVANCE mode — new commit pushed to the existing PR), `surfaced-green PR #` (fix's own CI went green; commented for review, did not advance), `annotated-flake PR #` (red was unrelated flake; -commented, attempt NOT burned), `waiting PR #` (CI not settled yet), -`dry-run: would-`, `skipped: `. (The +commented, attempt NOT burned), `marked-ready PR #` (the specific fixed test +was confirmed green in the PR's own CI; draft flipped to ready for review), +`already-surfaced PR #` (the fix's ✅ green comment already existed this sweep; +re-post suppressed — superseded by the Step 3.6 readiness line when the PR also flips +ready that cycle), `already-ready PR #` (terminal steady state — the PR was flipped +ready on a prior cycle and remains green on an unchanged head; no action taken, +supersedes `already-surfaced`), `waiting PR #` (CI not settled yet), +`dry-run: would-`, `skipped: `. (The `needs-human-PR` outcome is reserved for the deferred hand-off — Step 6 currently records a skip instead, so it is not emitted.)
CI pending / + target not yet validated on ; waiting` and stop. *(Round 1: this is where a + maintainer `/azp run` is awaited — the `/azp`-gated uitests/devicetests legs will not + have run until a human kicks them.)* 3. **Green → surface for review.** If `C.overallConclusion == "success"`: the checks that RAN are green. **Primary-gate check first:** the deterministic `success` verdict only certifies "at least one green check, nothing failing or @@ -704,18 +787,31 @@ Run these gates in order — the FIRST that fires decides this cycle's outcome: # primary CI gate (maui-pr) not green on ; waiting` and stop — do NOT surface. (The `/azp`-gated `maui-pr-uitests` (def 313) / `maui-pr-devicetests` (def 314) legs MAY still be un-run — that is expected and is named below, not a - reason to withhold the surface.) **Idempotency:** scan the PR's existing comments - for a prior bot `✅ … validated … on ` note for THIS head SHA — if one - already exists, record `already-surfaced PR # (head )` and stop - WITHOUT re-commenting (re-surfacing the same green every tick is noise and burns - the per-run comment budget other PRs need). Otherwise resolve the attempt number from - `C.effectiveAttempt` (the authoritative max(marker, bot-commit) counter; omit the - number only if it is somehow indeterminate). **Dry-run gate (Step 0):** if `dry_run == "true"`, do NOT emit any comment — instead print the intended `✅` validated-green body to the run log, tally `dry-run: would-surface-green PR #`, and stop. Otherwise `add_comment` on PR #: `✅ Attempt /10 validated - — the fix's CI is green on . Ready for human review.` - Name any `/azp`-gated legs (uitests def 313 / devicetests def 314) that have not - run and still need a maintainer `/azp run`. Do NOT advance. Record - `surfaced-green PR # (attempt /10)` and stop. *(This directly - attacks the real bottleneck — no reviews — so it is the highest-value outcome.)* + reason to withhold the surface.) **Comment idempotency + dry-run suppress the ✅ + comment ONLY — neither skips Step 3.6.** Scan the PR's existing comments for a prior + bot `✅ … validated … on ` note for THIS head SHA; if one exists, set + `SKIP_SURFACE_COMMENT = true`. Resolve the attempt number from `C.effectiveAttempt` + (the authoritative max(marker, bot-commit) counter; omit the number only if it is + somehow indeterminate). Then post the surface comment UNLESS suppressed: + - if `dry_run == "true"`: do NOT emit — print the intended `✅` validated-green body to + the run log and tally `dry-run: would-surface-green PR #`; + - else if `SKIP_SURFACE_COMMENT`: do NOT re-post — record `already-surfaced PR # + (head )` (re-surfacing the same green every tick is noise and burns the + per-run comment budget other PRs need); + - else `add_comment` on PR #: `✅ Attempt /10 validated — the fix's CI is + green on .` naming any `/azp`-gated legs (uitests def 313 / devicetests def + 314) that have not run and still need a maintainer `/azp run`; record `surfaced-green + PR # (attempt /10)`. (Do NOT assert "ready for human review" on a PR that + is still a draft — Step 3.6, not this comment, owns the draft→ready flip and posts its + own 🎯 announcement when it fires.) + Do NOT advance. Then — in ALL of the above cases — run **Step 3.6** (target-test + readiness gate) for this PR before stopping: its `/azp`-gated target legs may conclude + green on this SAME head SHA on a LATER sweep (an `/azp run` adds no commit), and Step + 3.6 is the ONLY place the draft→ready flip happens, so it MUST re-evaluate every sweep — + never `stop` here before it. *(This directly attacks the real bottleneck — no reviews — + so it is the highest-value outcome.)* 4. **Red → classify caused-by-fix vs unrelated-flake.** If `C.overallConclusion == "failure"`, analyze the PR's OWN failing build (NOT `main`): find the AzDO `maui-pr` build for this PR (filter builds by `branchName=refs/pull//merge`, @@ -723,18 +819,26 @@ Run these gates in order — the FIRST that fires decides this cycle's outcome: method and Step 4.7 flake buckets to `C.failedLegs`. - **Unrelated flake only** (every failed leg is known-flaky / infra / a pre-existing baseline red NOT introduced by the fix): do NOT burn an attempt. - **Idempotency first:** scan the PR's existing comments for a prior bot - `♻️ … unrelated flake … on ` note for THIS head SHA — if one already - exists, record `already-annotated-flake PR # (head )` and stop - WITHOUT re-commenting (re-annotating the same flake every 12h sweep is noise and - burns the per-run comment budget other PRs need). Otherwise resolve the attempt - number from `C.effectiveAttempt` (the authoritative max(marker, bot-commit) - counter), omitting the `Attempt /10:` prefix only if it is somehow - indeterminate. **Dry-run gate (Step 0):** if `dry_run == "true"`, do NOT emit any comment — instead print the intended `♻️` unrelated-flake body to the run log, tally `dry-run: would-annotate-flake PR #`, and stop. Otherwise `add_comment` on PR #: `♻️ Attempt /10: red is - unrelated flake on leg(s) () on ; the fix itself is not - implicated. A maintainer re-run (/azp run ) should clear it.` Record - `annotated-flake PR # (head )` and stop. *(Round 1: human re-runs; - Round 2: auto re-trigger.)* + **Comment idempotency + dry-run suppress the ♻️ comment ONLY — neither skips Step + 3.6.** Scan the PR's existing comments for a prior bot `♻️ … unrelated flake … on + ` note for THIS head SHA; if one exists, set `SKIP_FLAKE_COMMENT = true`. + Resolve the attempt number from `C.effectiveAttempt` (the authoritative max(marker, + bot-commit) counter), omitting the `Attempt /10:` prefix only if it is + somehow indeterminate. Then post the flake note UNLESS suppressed: + - if `dry_run == "true"`: do NOT emit — print the intended `♻️` unrelated-flake body + to the run log and tally `dry-run: would-annotate-flake PR #`; + - else if `SKIP_FLAKE_COMMENT`: do NOT re-post — record `already-annotated-flake PR + # (head )` (re-annotating the same flake every 12h sweep is noise and + burns the per-run comment budget other PRs need); + - else `add_comment` on PR #: `♻️ Attempt /10: red is unrelated flake on + leg(s) () on ; the fix itself is not implicated. A + maintainer re-run (/azp run ) should clear it.` record `annotated-flake + PR # (head )`. + Then — in ALL of the above cases — run **Step 3.6** (target-test readiness gate) for + this PR before stopping: its `/azp`-gated target legs may conclude green on this SAME + head SHA on a LATER sweep, and Step 3.6 is the ONLY place the draft→ready flip + happens, so it MUST re-evaluate every sweep — never `stop` here before it. *(Round 1: + human re-runs; Round 2: auto re-trigger.)* - **Caused by the fix** (a failed leg still matches the original target signature, or the fix introduced a NEW failure): advance an attempt. - **Attempt count.** `attempt = C.effectiveAttempt` — the authoritative @@ -752,6 +856,188 @@ Run these gates in order — the FIRST that fires decides this cycle's outcome: from every prior commit on this branch**, emitted via the Step 5.6 ADVANCE path (push onto the same PR). +#### Step 3.6 — Target-test verification & mark-ready gate + +Reached from the Step 3 **green-surface** branch, the Step 4 **unrelated-flake** branch +(AFTER that branch has posted its comment), and the Step 3.5 **gate-2 target-focused +fast-path** (2a — while unrelated legs are still pending). Purpose: confirm the SPECIFIC +test(s) this PR was opened to fix now PASS in the PR's own CI, and — only when they do +— transition the draft `[ci-fix]` PR to **ready for review** so a maintainer sees a +validated fix instead of a draft. This is the ONLY place the loop flips draft→ready; it +is a state transition, **never** an approval or a merge (a human still reviews and +merges). Overall red on *unrelated* legs must NOT gate readiness — we validate the fix, +not the base branch's flakiness. + +**Preconditions** (ALL must hold; otherwise record `skipped: readiness N/A PR # +()` and stop this gate): +- `C.isDraft == true` — if the PR is already ready-for-review, the draft→ready + transition is already done; do NOT re-mark **and do NOT re-apply `p/0`**. `p/0` is applied + exactly ONCE, atomically with the draft→ready flip (T3 — all-or-nothing with the 🎯 audit + comment + mark-ready), so an already-ready loop PR that is MISSING `p/0` has almost + certainly had it **removed by a maintainer** de-prioritizing the PR — a pure triage action + the human-engagement guard (which inspects comments/reviews/commits, NOT label changes) + cannot see. Re-adding `p/0` every sweep would fight that maintainer indefinitely, violating + the loop's "never override a human" contract. So the loop does NOT reconcile the label: + record `already-ready PR #` and stop this gate. (Trade-off: on the rare occasion a + transient API error drops `p/0` at flip time *after* the T3 atomic pre-check passed, it is + not auto-re-added — but the PR is still ready-for-review with its 🎯 audit comment, and + re-adding `p/0` is a trivial manual action; that is strictly preferable to steam-rolling a + maintainer's deliberate de-prioritization.) +- The PR is unmistakably THIS workflow's own: `[ci-fix] ` title prefix AND the + `agentic-workflows` label (mirrors the safe-output lock; the compensating scope + control if the v0.80.9 compiler drops the declarative required-*). +- You reached this gate from Step 3 (green), Step 4 (**unrelated-flake**), or the Step 3.5 + gate-2 **target-focused fast-path** (2a) — i.e. the fix is NOT implicated in any red that + has concluded. If Step 4 classified a red as **caused by the fix** (ADVANCE), do NOT run + this gate — advance the attempt instead. + +**T1 — Identify the target test(s).** From the `[ci-scan]` issue signature the fix +addresses (and the PR's own diff), extract the fully-qualified test method name(s) the +fix targets — e.g. `SafeAreaShouldWorkOnAllShellTabs`. For a de-flake it is the +de-flaked test; for a product fix it is the originally-failing test(s). If NO specific +test can be identified (e.g. a product build-break rather than a test failure), this is a +**build-only fix** — there is no single test to validate cross-platform, so validate at +**whole-build** granularity rather than stopping (Step 3 only *comments*; Step 3.6 is the +sole draft→ready flip, so a build-only fix is undrafted HERE or not at all). We only reach +this gate from Step 3 (green) / Step 4 (unrelated-flake) / gate-2a, so the fix is not +implicated in any concluded red; additionally require, via the T2 timeline method on +`C.headSha`, that the primary build pipeline `maui-pr` (def 302) has CONCLUDED with EVERY +one of its platform build legs `succeeded`/`completed` and NONE failed/canceled or still +pending — the build is green on **every** platform, not just the originally-broken one (the +cross-platform guard, applied to the build instead of a test). A build-only fix is undrafted +ONLY on the strength of the auto-running `maui-pr` (def 302) whole-build green, which the +loop CAN observe: if the PR's `[ci-scan]` issue signature or its own diff indicates the +ORIGINATING failure was in a `/azp`-gated pipeline (`maui-pr-uitests` def 313 or +`maui-pr-devicetests` def 314) rather than `maui-pr` (def 302), do NOT undraft on +`maui-pr`-green alone — those pipelines do not auto-run on this PR (GITHUB_TOKEN cannot +trigger them), so a green `maui-pr` build is NOT evidence the gated build break is fixed; +record `skipped: build-only fix PR # targets gated pipeline () not run — +deferring to human` and stop this gate. Otherwise, set `TARGET := "the maui-pr build +(build-only fix — no single target test)"` and proceed to **T3** to mark ready. If any `maui-pr` build leg is still unconcluded, record `skipped: build-only fix PR +# not yet whole-build green (leg(s) pending)` and stop this gate WITHOUT marking +ready (a green subset is not enough — a still-pending build leg could yet fail). + +**T2 — Verify the target test's platform legs are green (anonymous, leg-level).** Per-test +outcomes come from the AzDO **test-results** API (`_apis/test/...`), which is **NOT reachable +anonymously — Hard-Rule 8**: those endpoints 302-redirect to a sign-in page on this runner, so +a `curl` returns an HTML redirect (not JSON) and every target test would look "not executed". +Validate instead at **leg granularity** using ONLY the anonymous `_apis/build/...` timeline +(Hard-Rule 8) — a `succeeded` category leg means every test in that category **that actually +ran** passed on that platform. This is a strong bar **only if the target test genuinely +executes**: a job can go green while one specific test is skipped at runtime (`[Ignore]`, +`Assert.Ignore()`, `Assert.Inconclusive()`, a `Skip=`/conditional `[Fact]`, an `#if`-out, or +an early `return` before the asserts). So before trusting a green leg as proof the target +passed, **confirm from the PR's OWN diff that the fix does NOT skip, ignore, disable, or +short-circuit the target test** (it adds no `[Ignore]`/`[Explicit]`, `Assert.Ignore`/ +`Assert.Inconclusive`, `Skip=`, category-exclusion, `#if`-out, or early `return` around the +target). If the fix could cause the target to be runtime-skipped rather than genuinely pass, +leg-level green is NOT sufficient evidence — record `skipped: target test may be +runtime-skipped by this fix (diff adds ); leg-green insufficient, deferring to human +on PR #` and stop this gate WITHOUT marking ready. Otherwise (the target genuinely runs +and asserts, as a de-flake or product fix does) a green category leg cannot hide a +target-test failure, so treat it as authoritative. + +Map the target test to the CI leg(s) that run it. A leg name encodes platform + test category +(e.g. `Android UITests SafeAreaEdges,Shadow`, `iOS UITests SafeAreaEdges,Shadow`, `macOS +UITests SafeAreaEdges,Shadow`, `Windows UITests SafeAreaEdges,Shadow`). Determine the target +test's UI-test category — its `[Category(UITestCategories.X)]` in the test/HostApp source, +visible in the PR diff or the test file — to know which leg-name substring identifies its legs; +for a device test, the per-platform device-test legs. + +Using the SAME build-discovery as Step 4 (filter AzDO builds by `branchName=refs/pull//merge` +or `sourceVersion == C.headSha`), read each build's **timeline** on `C.headSha` for the +pipeline(s) that RUN the target test — `maui-pr` (def 302) for unit/integration tests, +`maui-pr-uitests` (def 313) for Appium UI tests, `maui-pr-devicetests` (def 314) for device +tests: + +```bash +ORG=dnceng-public; PROJ=public; BUILD= +# Anonymous + Hard-Rule-8-compliant. NEVER call _apis/test/... (it 302-redirects to sign-in). +# Each type=="Job" record is one platform leg: result (succeeded/failed/canceled/null), +# state (completed/inProgress/pending), name (encodes " UITests "): +curl -s "https://dev.azure.com/$ORG/$PROJ/_apis/build/builds/$BUILD/timeline?api-version=7.1" \ + | tee /tmp/gh-aw/agent/timeline_${BUILD}.json \ + | jq -r '.records[] | select(.type=="Job") | "\(.result)\t\(.state)\t\(.name)"' +``` + +(The prefetch already exposes per-leg status in `C.failedLegs` / the checks context, and +`gh pr checks ` lists the same per-leg rows with platform+category in the name — use +whichever is handy; the build timeline is the authoritative cross-check on the exact build.) + +Treat a target test as **VALIDATED-GREEN** only if, on `C.headSha`, the leg that runs its +category is green on **every platform it runs on** — a fix that repairs one platform must not +silently regress the same test's leg on another, so a green leg on the originally-red platform +alone is NOT enough. Across the target pipeline's platform legs (for a UI test: the Android, +iOS, Windows, and macOS/MacCatalyst legs running the target's category; for a device test: each +device-test platform), require ALL of: +- the target's category leg is `result == "succeeded"` **and** `state == "completed"` on + **every** platform that runs it, AND +- that leg is `failed` / `canceled` / aborted on **NO** platform, AND +- **no platform is left unverified.** For each platform family the target pipeline covers, that + platform's category leg must have CONCLUDED (`state == "completed"`) on `C.headSha`. If a + platform simply has no leg for the target's category, the test does not run there — that is + fine, not a gap. But if a platform's category leg has **not concluded** (`state` is + `inProgress` / pending, or an `/azp`-gated `maui-pr-uitests` / `maui-pr-devicetests` leg that + has not been kicked), the test's status on that platform is UNKNOWN → the fix is NOT yet + validated across platforms: record `skipped: target test green on but + not yet verified on (leg(s) pending / need /azp run) on PR #` + and stop this gate WITHOUT marking ready. + +A target test whose category leg never concluded on ANY platform (**not executed** anywhere — +e.g. its `/azp`-gated pipeline has not been kicked) is likewise NOT validated: record `skipped: +target test not yet executed on PR # ( not run — needs /azp run)` and stop +this gate WITHOUT marking ready. Do NOT overclaim — a green *sibling* leg (a different category +on the same platform) is not the target's leg, and a green leg on one platform is not a pass on +the others. + +**T3 — Mark ready + report.** If EVERY target test is VALIDATED-GREEN. The 🎯 comment, +the mark-ready, and the `p/0` label are THREE SEPARATE safe-outputs — the comment +existing does NOT prove the mark-ready took effect, so they are tracked independently: + +- **Comment idempotency (dup-suppress only — never gates the mark-ready).** We only reach + T3 while `C.isDraft == true` (the precondition stops an already-ready PR before here). So + if a prior bot comment for THIS head that carries the leading `🎯` anchor AND the + contiguous phrase `validated green on ` already exists (BOTH T3 variants — + `🎯 Target test validated green on ` and the build-only `🎯 Build validated green on + ` — contain that exact contiguous phrase, so this recognizes both; keying on + `target test` alone would instead miss the build-only comment and re-post it every sweep. + **Require the `🎯` anchor** so the match can NEVER be loosened to a gapped + `validated…green…on` form that would false-positive on the Step 3 `✅ … validated — the + fix's CI is green on ` surface comment and wrongly suppress the audit 🎯), the comment + landed on an earlier sweep but the **mark-ready did NOT take + effect** (the PR is still a draft) — set `SUPPRESS_COMMENT = true` (do not re-post the + duplicate 🎯 comment) but STILL complete the mark-ready + label below. Never treat the + comment's existence as "already marked ready" while the PR is still a draft. Record this + case as `re-marking PR # (🎯 present; mark-ready did not take on a prior sweep)`. +- **Atomicity budget pre-check (Hard-Rule 7).** Determine the outputs this gate will emit: + `mark_pull_request_as_ready_for_review` + `add_labels` always, plus `add_comment` unless + `SUPPRESS_COMMENT`. Verify a free per-run slot remains in EACH bucket you are about to + use. If ANY required bucket is exhausted, emit NONE of them — record `skipped: per-run + cap reached; deferring mark-ready PR # to next cycle` and stop this gate. Never mark a + PR ready (or label it) unless its 🎯 audit comment is GUARANTEED to exist for this exact + head SHA — either posted in this same sweep, or (in the `SUPPRESS_COMMENT` re-marking case) + already present from a prior sweep. The outputs you DO emit either all land or all defer + together; the audit trail must never be absent, but it is NOT re-posted when it already exists. +- **Dry-run gate (Step 0):** if `dry_run == "true"`, emit NOTHING — print the intended + readiness comment and "would mark ready + add p/0" to the run log, tally `dry-run: + would-mark-ready PR #`, and stop. +- Otherwise emit for THIS PR number ``: + 1. `add_comment` (ONLY if not `SUPPRESS_COMMENT`): `🎯 Target test validated green on + — passed on ALL platforms it runs on (, + buildId ). — not + caused by this fix."> Transitioning this PR from draft to ready for review and adding + `p/0`; a maintainer still reviews and merges.` (For a **build-only fix** — the T1 + whole-build fallback, no single target test — phrase the first clause as `🎯 Build + validated green on — the maui-pr build passed on ALL platforms (buildId + ).` instead of naming a test.) + 2. `mark_pull_request_as_ready_for_review` with `reason:` a one-line justification + naming the validated test(s) and ``. + 3. `add_labels` with `labels: ["p/0"]` for PR # — put the now-review-ready fix into + the team's p/0 priority queue so it is triaged, not lost in the draft backlog. (If + the PR somehow already carries `p/0`, this is a harmless no-op.) +- Record `marked-ready PR # (target green on ALL platforms on , + labeled p/0)` and stop. + #### Step 3.5.R — Maintainer change-request response (Track C) Reached from Step 3.5 gate 0. Goal: when an **eligible human reviewer** (Hard-Rule @@ -1383,7 +1669,19 @@ Filed by [`ci-status-fix`](https://github.com/dotnet/maui/blob/main/.github/work ### Step 8 — Per-issue tally + end-of-run summary -Per issue, append one outcome line to `/tmp/gh-aw/agent/coverage.txt`: +Per issue, append **one** outcome line to `/tmp/gh-aw/agent/coverage.txt` — the +terminal outcome for the cycle. When one cycle produces a chained pair — a PR gets a +same-cycle precursor line and **then** a Step 3.6 readiness line (`marked-ready` on the +draft→ready flip, or `already-ready` on a later steady-state sweep of an already-ready +PR) — record ONLY the terminal Step 3.6 readiness line: it supersedes ANY same-cycle +non-readiness precursor for that PR, so an aggregator keying on one-line-per-issue never +double-counts. The precursor may be a Step 3 green line (`surfaced-green` on the first ✅, +or `already-surfaced` when it already exists) OR — when an unrelated-flake red and a +target-green coincide — a Step 4 `annotated-flake` line; either way the readiness line is +terminal, and the superseded precursor's signal survives in the PR's 🎯/♻️ comment so no +information is lost. This covers the flip cycle (`surfaced-green` → `marked-ready`), the +post-flip steady state (`already-surfaced` → `already-ready`), and the flake-coincident +flip (`annotated-flake` → `marked-ready`): ``` # main attempt- @@ -1394,8 +1692,14 @@ Per issue, append one outcome line to `/tmp/gh-aw/agent/coverage.txt`: `advance-PR # attempt /10` (ADVANCE mode — new commit pushed to the existing PR), `surfaced-green PR #` (fix's own CI went green; commented for review, did not advance), `annotated-flake PR #` (red was unrelated flake; -commented, attempt NOT burned), `waiting PR #` (CI not settled yet), -`dry-run: would-`, `skipped: `. (The +commented, attempt NOT burned), `marked-ready PR #` (the specific fixed test +was confirmed green in the PR's own CI; draft flipped to ready for review), +`already-surfaced PR #` (the fix's ✅ green comment already existed this sweep; +re-post suppressed — superseded by the Step 3.6 readiness line when the PR also flips +ready that cycle), `already-ready PR #` (terminal steady state — the PR was flipped +ready on a prior cycle and remains green on an unchanged head; no action taken, +supersedes `already-surfaced`), `waiting PR #` (CI not settled yet), +`dry-run: would-`, `skipped: `. (The `needs-human-PR` outcome is reserved for the deferred hand-off — Step 6 currently records a skip instead, so it is not emitted.)
primary CI gate (maui-pr) not green on ; waiting` and stop — do NOT surface. (The `/azp`-gated `maui-pr-uitests` (def 313) / `maui-pr-devicetests` (def 314) legs MAY still be un-run — that is expected and is named below, not a - reason to withhold the surface.) **Idempotency:** scan the PR's existing comments - for a prior bot `✅ … validated … on ` note for THIS head SHA — if one - already exists, record `already-surfaced PR # (head )` and stop - WITHOUT re-commenting (re-surfacing the same green every tick is noise and burns - the per-run comment budget other PRs need). Otherwise resolve the attempt number from - `C.effectiveAttempt` (the authoritative max(marker, bot-commit) counter; omit the - number only if it is somehow indeterminate). **Dry-run gate (Step 0):** if `dry_run == "true"`, do NOT emit any comment — instead print the intended `✅` validated-green body to the run log, tally `dry-run: would-surface-green PR #`, and stop. Otherwise `add_comment` on PR #: `✅ Attempt /10 validated - — the fix's CI is green on . Ready for human review.` - Name any `/azp`-gated legs (uitests def 313 / devicetests def 314) that have not - run and still need a maintainer `/azp run`. Do NOT advance. Record - `surfaced-green PR # (attempt /10)` and stop. *(This directly - attacks the real bottleneck — no reviews — so it is the highest-value outcome.)* + reason to withhold the surface.) **Comment idempotency + dry-run suppress the ✅ + comment ONLY — neither skips Step 3.6.** Scan the PR's existing comments for a prior + bot `✅ … validated … on ` note for THIS head SHA; if one exists, set + `SKIP_SURFACE_COMMENT = true`. Resolve the attempt number from `C.effectiveAttempt` + (the authoritative max(marker, bot-commit) counter; omit the number only if it is + somehow indeterminate). Then post the surface comment UNLESS suppressed: + - if `dry_run == "true"`: do NOT emit — print the intended `✅` validated-green body to + the run log and tally `dry-run: would-surface-green PR #`; + - else if `SKIP_SURFACE_COMMENT`: do NOT re-post — record `already-surfaced PR # + (head )` (re-surfacing the same green every tick is noise and burns the + per-run comment budget other PRs need); + - else `add_comment` on PR #: `✅ Attempt /10 validated — the fix's CI is + green on .` naming any `/azp`-gated legs (uitests def 313 / devicetests def + 314) that have not run and still need a maintainer `/azp run`; record `surfaced-green + PR # (attempt /10)`. (Do NOT assert "ready for human review" on a PR that + is still a draft — Step 3.6, not this comment, owns the draft→ready flip and posts its + own 🎯 announcement when it fires.) + Do NOT advance. Then — in ALL of the above cases — run **Step 3.6** (target-test + readiness gate) for this PR before stopping: its `/azp`-gated target legs may conclude + green on this SAME head SHA on a LATER sweep (an `/azp run` adds no commit), and Step + 3.6 is the ONLY place the draft→ready flip happens, so it MUST re-evaluate every sweep — + never `stop` here before it. *(This directly attacks the real bottleneck — no reviews — + so it is the highest-value outcome.)* 4. **Red → classify caused-by-fix vs unrelated-flake.** If `C.overallConclusion == "failure"`, analyze the PR's OWN failing build (NOT `main`): find the AzDO `maui-pr` build for this PR (filter builds by `branchName=refs/pull//merge`, @@ -723,18 +819,26 @@ Run these gates in order — the FIRST that fires decides this cycle's outcome: method and Step 4.7 flake buckets to `C.failedLegs`. - **Unrelated flake only** (every failed leg is known-flaky / infra / a pre-existing baseline red NOT introduced by the fix): do NOT burn an attempt. - **Idempotency first:** scan the PR's existing comments for a prior bot - `♻️ … unrelated flake … on ` note for THIS head SHA — if one already - exists, record `already-annotated-flake PR # (head )` and stop - WITHOUT re-commenting (re-annotating the same flake every 12h sweep is noise and - burns the per-run comment budget other PRs need). Otherwise resolve the attempt - number from `C.effectiveAttempt` (the authoritative max(marker, bot-commit) - counter), omitting the `Attempt /10:` prefix only if it is somehow - indeterminate. **Dry-run gate (Step 0):** if `dry_run == "true"`, do NOT emit any comment — instead print the intended `♻️` unrelated-flake body to the run log, tally `dry-run: would-annotate-flake PR #`, and stop. Otherwise `add_comment` on PR #: `♻️ Attempt /10: red is - unrelated flake on leg(s) () on ; the fix itself is not - implicated. A maintainer re-run (/azp run ) should clear it.` Record - `annotated-flake PR # (head )` and stop. *(Round 1: human re-runs; - Round 2: auto re-trigger.)* + **Comment idempotency + dry-run suppress the ♻️ comment ONLY — neither skips Step + 3.6.** Scan the PR's existing comments for a prior bot `♻️ … unrelated flake … on + ` note for THIS head SHA; if one exists, set `SKIP_FLAKE_COMMENT = true`. + Resolve the attempt number from `C.effectiveAttempt` (the authoritative max(marker, + bot-commit) counter), omitting the `Attempt /10:` prefix only if it is + somehow indeterminate. Then post the flake note UNLESS suppressed: + - if `dry_run == "true"`: do NOT emit — print the intended `♻️` unrelated-flake body + to the run log and tally `dry-run: would-annotate-flake PR #`; + - else if `SKIP_FLAKE_COMMENT`: do NOT re-post — record `already-annotated-flake PR + # (head )` (re-annotating the same flake every 12h sweep is noise and + burns the per-run comment budget other PRs need); + - else `add_comment` on PR #: `♻️ Attempt /10: red is unrelated flake on + leg(s) () on ; the fix itself is not implicated. A + maintainer re-run (/azp run ) should clear it.` record `annotated-flake + PR # (head )`. + Then — in ALL of the above cases — run **Step 3.6** (target-test readiness gate) for + this PR before stopping: its `/azp`-gated target legs may conclude green on this SAME + head SHA on a LATER sweep, and Step 3.6 is the ONLY place the draft→ready flip + happens, so it MUST re-evaluate every sweep — never `stop` here before it. *(Round 1: + human re-runs; Round 2: auto re-trigger.)* - **Caused by the fix** (a failed leg still matches the original target signature, or the fix introduced a NEW failure): advance an attempt. - **Attempt count.** `attempt = C.effectiveAttempt` — the authoritative @@ -752,6 +856,188 @@ Run these gates in order — the FIRST that fires decides this cycle's outcome: from every prior commit on this branch**, emitted via the Step 5.6 ADVANCE path (push onto the same PR). +#### Step 3.6 — Target-test verification & mark-ready gate + +Reached from the Step 3 **green-surface** branch, the Step 4 **unrelated-flake** branch +(AFTER that branch has posted its comment), and the Step 3.5 **gate-2 target-focused +fast-path** (2a — while unrelated legs are still pending). Purpose: confirm the SPECIFIC +test(s) this PR was opened to fix now PASS in the PR's own CI, and — only when they do +— transition the draft `[ci-fix]` PR to **ready for review** so a maintainer sees a +validated fix instead of a draft. This is the ONLY place the loop flips draft→ready; it +is a state transition, **never** an approval or a merge (a human still reviews and +merges). Overall red on *unrelated* legs must NOT gate readiness — we validate the fix, +not the base branch's flakiness. + +**Preconditions** (ALL must hold; otherwise record `skipped: readiness N/A PR # +()` and stop this gate): +- `C.isDraft == true` — if the PR is already ready-for-review, the draft→ready + transition is already done; do NOT re-mark **and do NOT re-apply `p/0`**. `p/0` is applied + exactly ONCE, atomically with the draft→ready flip (T3 — all-or-nothing with the 🎯 audit + comment + mark-ready), so an already-ready loop PR that is MISSING `p/0` has almost + certainly had it **removed by a maintainer** de-prioritizing the PR — a pure triage action + the human-engagement guard (which inspects comments/reviews/commits, NOT label changes) + cannot see. Re-adding `p/0` every sweep would fight that maintainer indefinitely, violating + the loop's "never override a human" contract. So the loop does NOT reconcile the label: + record `already-ready PR #` and stop this gate. (Trade-off: on the rare occasion a + transient API error drops `p/0` at flip time *after* the T3 atomic pre-check passed, it is + not auto-re-added — but the PR is still ready-for-review with its 🎯 audit comment, and + re-adding `p/0` is a trivial manual action; that is strictly preferable to steam-rolling a + maintainer's deliberate de-prioritization.) +- The PR is unmistakably THIS workflow's own: `[ci-fix] ` title prefix AND the + `agentic-workflows` label (mirrors the safe-output lock; the compensating scope + control if the v0.80.9 compiler drops the declarative required-*). +- You reached this gate from Step 3 (green), Step 4 (**unrelated-flake**), or the Step 3.5 + gate-2 **target-focused fast-path** (2a) — i.e. the fix is NOT implicated in any red that + has concluded. If Step 4 classified a red as **caused by the fix** (ADVANCE), do NOT run + this gate — advance the attempt instead. + +**T1 — Identify the target test(s).** From the `[ci-scan]` issue signature the fix +addresses (and the PR's own diff), extract the fully-qualified test method name(s) the +fix targets — e.g. `SafeAreaShouldWorkOnAllShellTabs`. For a de-flake it is the +de-flaked test; for a product fix it is the originally-failing test(s). If NO specific +test can be identified (e.g. a product build-break rather than a test failure), this is a +**build-only fix** — there is no single test to validate cross-platform, so validate at +**whole-build** granularity rather than stopping (Step 3 only *comments*; Step 3.6 is the +sole draft→ready flip, so a build-only fix is undrafted HERE or not at all). We only reach +this gate from Step 3 (green) / Step 4 (unrelated-flake) / gate-2a, so the fix is not +implicated in any concluded red; additionally require, via the T2 timeline method on +`C.headSha`, that the primary build pipeline `maui-pr` (def 302) has CONCLUDED with EVERY +one of its platform build legs `succeeded`/`completed` and NONE failed/canceled or still +pending — the build is green on **every** platform, not just the originally-broken one (the +cross-platform guard, applied to the build instead of a test). A build-only fix is undrafted +ONLY on the strength of the auto-running `maui-pr` (def 302) whole-build green, which the +loop CAN observe: if the PR's `[ci-scan]` issue signature or its own diff indicates the +ORIGINATING failure was in a `/azp`-gated pipeline (`maui-pr-uitests` def 313 or +`maui-pr-devicetests` def 314) rather than `maui-pr` (def 302), do NOT undraft on +`maui-pr`-green alone — those pipelines do not auto-run on this PR (GITHUB_TOKEN cannot +trigger them), so a green `maui-pr` build is NOT evidence the gated build break is fixed; +record `skipped: build-only fix PR # targets gated pipeline () not run — +deferring to human` and stop this gate. Otherwise, set `TARGET := "the maui-pr build +(build-only fix — no single target test)"` and proceed to **T3** to mark ready. If any `maui-pr` build leg is still unconcluded, record `skipped: build-only fix PR +# not yet whole-build green (leg(s) pending)` and stop this gate WITHOUT marking +ready (a green subset is not enough — a still-pending build leg could yet fail). + +**T2 — Verify the target test's platform legs are green (anonymous, leg-level).** Per-test +outcomes come from the AzDO **test-results** API (`_apis/test/...`), which is **NOT reachable +anonymously — Hard-Rule 8**: those endpoints 302-redirect to a sign-in page on this runner, so +a `curl` returns an HTML redirect (not JSON) and every target test would look "not executed". +Validate instead at **leg granularity** using ONLY the anonymous `_apis/build/...` timeline +(Hard-Rule 8) — a `succeeded` category leg means every test in that category **that actually +ran** passed on that platform. This is a strong bar **only if the target test genuinely +executes**: a job can go green while one specific test is skipped at runtime (`[Ignore]`, +`Assert.Ignore()`, `Assert.Inconclusive()`, a `Skip=`/conditional `[Fact]`, an `#if`-out, or +an early `return` before the asserts). So before trusting a green leg as proof the target +passed, **confirm from the PR's OWN diff that the fix does NOT skip, ignore, disable, or +short-circuit the target test** (it adds no `[Ignore]`/`[Explicit]`, `Assert.Ignore`/ +`Assert.Inconclusive`, `Skip=`, category-exclusion, `#if`-out, or early `return` around the +target). If the fix could cause the target to be runtime-skipped rather than genuinely pass, +leg-level green is NOT sufficient evidence — record `skipped: target test may be +runtime-skipped by this fix (diff adds ); leg-green insufficient, deferring to human +on PR #` and stop this gate WITHOUT marking ready. Otherwise (the target genuinely runs +and asserts, as a de-flake or product fix does) a green category leg cannot hide a +target-test failure, so treat it as authoritative. + +Map the target test to the CI leg(s) that run it. A leg name encodes platform + test category +(e.g. `Android UITests SafeAreaEdges,Shadow`, `iOS UITests SafeAreaEdges,Shadow`, `macOS +UITests SafeAreaEdges,Shadow`, `Windows UITests SafeAreaEdges,Shadow`). Determine the target +test's UI-test category — its `[Category(UITestCategories.X)]` in the test/HostApp source, +visible in the PR diff or the test file — to know which leg-name substring identifies its legs; +for a device test, the per-platform device-test legs. + +Using the SAME build-discovery as Step 4 (filter AzDO builds by `branchName=refs/pull//merge` +or `sourceVersion == C.headSha`), read each build's **timeline** on `C.headSha` for the +pipeline(s) that RUN the target test — `maui-pr` (def 302) for unit/integration tests, +`maui-pr-uitests` (def 313) for Appium UI tests, `maui-pr-devicetests` (def 314) for device +tests: + +```bash +ORG=dnceng-public; PROJ=public; BUILD= +# Anonymous + Hard-Rule-8-compliant. NEVER call _apis/test/... (it 302-redirects to sign-in). +# Each type=="Job" record is one platform leg: result (succeeded/failed/canceled/null), +# state (completed/inProgress/pending), name (encodes " UITests "): +curl -s "https://dev.azure.com/$ORG/$PROJ/_apis/build/builds/$BUILD/timeline?api-version=7.1" \ + | tee /tmp/gh-aw/agent/timeline_${BUILD}.json \ + | jq -r '.records[] | select(.type=="Job") | "\(.result)\t\(.state)\t\(.name)"' +``` + +(The prefetch already exposes per-leg status in `C.failedLegs` / the checks context, and +`gh pr checks ` lists the same per-leg rows with platform+category in the name — use +whichever is handy; the build timeline is the authoritative cross-check on the exact build.) + +Treat a target test as **VALIDATED-GREEN** only if, on `C.headSha`, the leg that runs its +category is green on **every platform it runs on** — a fix that repairs one platform must not +silently regress the same test's leg on another, so a green leg on the originally-red platform +alone is NOT enough. Across the target pipeline's platform legs (for a UI test: the Android, +iOS, Windows, and macOS/MacCatalyst legs running the target's category; for a device test: each +device-test platform), require ALL of: +- the target's category leg is `result == "succeeded"` **and** `state == "completed"` on + **every** platform that runs it, AND +- that leg is `failed` / `canceled` / aborted on **NO** platform, AND +- **no platform is left unverified.** For each platform family the target pipeline covers, that + platform's category leg must have CONCLUDED (`state == "completed"`) on `C.headSha`. If a + platform simply has no leg for the target's category, the test does not run there — that is + fine, not a gap. But if a platform's category leg has **not concluded** (`state` is + `inProgress` / pending, or an `/azp`-gated `maui-pr-uitests` / `maui-pr-devicetests` leg that + has not been kicked), the test's status on that platform is UNKNOWN → the fix is NOT yet + validated across platforms: record `skipped: target test green on but + not yet verified on (leg(s) pending / need /azp run) on PR #` + and stop this gate WITHOUT marking ready. + +A target test whose category leg never concluded on ANY platform (**not executed** anywhere — +e.g. its `/azp`-gated pipeline has not been kicked) is likewise NOT validated: record `skipped: +target test not yet executed on PR # ( not run — needs /azp run)` and stop +this gate WITHOUT marking ready. Do NOT overclaim — a green *sibling* leg (a different category +on the same platform) is not the target's leg, and a green leg on one platform is not a pass on +the others. + +**T3 — Mark ready + report.** If EVERY target test is VALIDATED-GREEN. The 🎯 comment, +the mark-ready, and the `p/0` label are THREE SEPARATE safe-outputs — the comment +existing does NOT prove the mark-ready took effect, so they are tracked independently: + +- **Comment idempotency (dup-suppress only — never gates the mark-ready).** We only reach + T3 while `C.isDraft == true` (the precondition stops an already-ready PR before here). So + if a prior bot comment for THIS head that carries the leading `🎯` anchor AND the + contiguous phrase `validated green on ` already exists (BOTH T3 variants — + `🎯 Target test validated green on ` and the build-only `🎯 Build validated green on + ` — contain that exact contiguous phrase, so this recognizes both; keying on + `target test` alone would instead miss the build-only comment and re-post it every sweep. + **Require the `🎯` anchor** so the match can NEVER be loosened to a gapped + `validated…green…on` form that would false-positive on the Step 3 `✅ … validated — the + fix's CI is green on ` surface comment and wrongly suppress the audit 🎯), the comment + landed on an earlier sweep but the **mark-ready did NOT take + effect** (the PR is still a draft) — set `SUPPRESS_COMMENT = true` (do not re-post the + duplicate 🎯 comment) but STILL complete the mark-ready + label below. Never treat the + comment's existence as "already marked ready" while the PR is still a draft. Record this + case as `re-marking PR # (🎯 present; mark-ready did not take on a prior sweep)`. +- **Atomicity budget pre-check (Hard-Rule 7).** Determine the outputs this gate will emit: + `mark_pull_request_as_ready_for_review` + `add_labels` always, plus `add_comment` unless + `SUPPRESS_COMMENT`. Verify a free per-run slot remains in EACH bucket you are about to + use. If ANY required bucket is exhausted, emit NONE of them — record `skipped: per-run + cap reached; deferring mark-ready PR # to next cycle` and stop this gate. Never mark a + PR ready (or label it) unless its 🎯 audit comment is GUARANTEED to exist for this exact + head SHA — either posted in this same sweep, or (in the `SUPPRESS_COMMENT` re-marking case) + already present from a prior sweep. The outputs you DO emit either all land or all defer + together; the audit trail must never be absent, but it is NOT re-posted when it already exists. +- **Dry-run gate (Step 0):** if `dry_run == "true"`, emit NOTHING — print the intended + readiness comment and "would mark ready + add p/0" to the run log, tally `dry-run: + would-mark-ready PR #`, and stop. +- Otherwise emit for THIS PR number ``: + 1. `add_comment` (ONLY if not `SUPPRESS_COMMENT`): `🎯 Target test validated green on + — passed on ALL platforms it runs on (, + buildId ). — not + caused by this fix."> Transitioning this PR from draft to ready for review and adding + `p/0`; a maintainer still reviews and merges.` (For a **build-only fix** — the T1 + whole-build fallback, no single target test — phrase the first clause as `🎯 Build + validated green on — the maui-pr build passed on ALL platforms (buildId + ).` instead of naming a test.) + 2. `mark_pull_request_as_ready_for_review` with `reason:` a one-line justification + naming the validated test(s) and ``. + 3. `add_labels` with `labels: ["p/0"]` for PR # — put the now-review-ready fix into + the team's p/0 priority queue so it is triaged, not lost in the draft backlog. (If + the PR somehow already carries `p/0`, this is a harmless no-op.) +- Record `marked-ready PR # (target green on ALL platforms on , + labeled p/0)` and stop. + #### Step 3.5.R — Maintainer change-request response (Track C) Reached from Step 3.5 gate 0. Goal: when an **eligible human reviewer** (Hard-Rule @@ -1383,7 +1669,19 @@ Filed by [`ci-status-fix`](https://github.com/dotnet/maui/blob/main/.github/work ### Step 8 — Per-issue tally + end-of-run summary -Per issue, append one outcome line to `/tmp/gh-aw/agent/coverage.txt`: +Per issue, append **one** outcome line to `/tmp/gh-aw/agent/coverage.txt` — the +terminal outcome for the cycle. When one cycle produces a chained pair — a PR gets a +same-cycle precursor line and **then** a Step 3.6 readiness line (`marked-ready` on the +draft→ready flip, or `already-ready` on a later steady-state sweep of an already-ready +PR) — record ONLY the terminal Step 3.6 readiness line: it supersedes ANY same-cycle +non-readiness precursor for that PR, so an aggregator keying on one-line-per-issue never +double-counts. The precursor may be a Step 3 green line (`surfaced-green` on the first ✅, +or `already-surfaced` when it already exists) OR — when an unrelated-flake red and a +target-green coincide — a Step 4 `annotated-flake` line; either way the readiness line is +terminal, and the superseded precursor's signal survives in the PR's 🎯/♻️ comment so no +information is lost. This covers the flip cycle (`surfaced-green` → `marked-ready`), the +post-flip steady state (`already-surfaced` → `already-ready`), and the flake-coincident +flip (`annotated-flake` → `marked-ready`): ``` # main attempt- @@ -1394,8 +1692,14 @@ Per issue, append one outcome line to `/tmp/gh-aw/agent/coverage.txt`: `advance-PR # attempt /10` (ADVANCE mode — new commit pushed to the existing PR), `surfaced-green PR #` (fix's own CI went green; commented for review, did not advance), `annotated-flake PR #` (red was unrelated flake; -commented, attempt NOT burned), `waiting PR #` (CI not settled yet), -`dry-run: would-`, `skipped: `. (The +commented, attempt NOT burned), `marked-ready PR #` (the specific fixed test +was confirmed green in the PR's own CI; draft flipped to ready for review), +`already-surfaced PR #` (the fix's ✅ green comment already existed this sweep; +re-post suppressed — superseded by the Step 3.6 readiness line when the PR also flips +ready that cycle), `already-ready PR #` (terminal steady state — the PR was flipped +ready on a prior cycle and remains green on an unchanged head; no action taken, +supersedes `already-surfaced`), `waiting PR #` (CI not settled yet), +`dry-run: would-`, `skipped: `. (The `needs-human-PR` outcome is reserved for the deferred hand-off — Step 6 currently records a skip instead, so it is not emitted.)
(head )` and stop - WITHOUT re-commenting (re-surfacing the same green every tick is noise and burns - the per-run comment budget other PRs need). Otherwise resolve the attempt number from - `C.effectiveAttempt` (the authoritative max(marker, bot-commit) counter; omit the - number only if it is somehow indeterminate). **Dry-run gate (Step 0):** if `dry_run == "true"`, do NOT emit any comment — instead print the intended `✅` validated-green body to the run log, tally `dry-run: would-surface-green PR #`, and stop. Otherwise `add_comment` on PR #: `✅ Attempt /10 validated - — the fix's CI is green on . Ready for human review.` - Name any `/azp`-gated legs (uitests def 313 / devicetests def 314) that have not - run and still need a maintainer `/azp run`. Do NOT advance. Record - `surfaced-green PR # (attempt /10)` and stop. *(This directly - attacks the real bottleneck — no reviews — so it is the highest-value outcome.)* + reason to withhold the surface.) **Comment idempotency + dry-run suppress the ✅ + comment ONLY — neither skips Step 3.6.** Scan the PR's existing comments for a prior + bot `✅ … validated … on ` note for THIS head SHA; if one exists, set + `SKIP_SURFACE_COMMENT = true`. Resolve the attempt number from `C.effectiveAttempt` + (the authoritative max(marker, bot-commit) counter; omit the number only if it is + somehow indeterminate). Then post the surface comment UNLESS suppressed: + - if `dry_run == "true"`: do NOT emit — print the intended `✅` validated-green body to + the run log and tally `dry-run: would-surface-green PR #`; + - else if `SKIP_SURFACE_COMMENT`: do NOT re-post — record `already-surfaced PR # + (head )` (re-surfacing the same green every tick is noise and burns the + per-run comment budget other PRs need); + - else `add_comment` on PR #: `✅ Attempt /10 validated — the fix's CI is + green on .` naming any `/azp`-gated legs (uitests def 313 / devicetests def + 314) that have not run and still need a maintainer `/azp run`; record `surfaced-green + PR # (attempt /10)`. (Do NOT assert "ready for human review" on a PR that + is still a draft — Step 3.6, not this comment, owns the draft→ready flip and posts its + own 🎯 announcement when it fires.) + Do NOT advance. Then — in ALL of the above cases — run **Step 3.6** (target-test + readiness gate) for this PR before stopping: its `/azp`-gated target legs may conclude + green on this SAME head SHA on a LATER sweep (an `/azp run` adds no commit), and Step + 3.6 is the ONLY place the draft→ready flip happens, so it MUST re-evaluate every sweep — + never `stop` here before it. *(This directly attacks the real bottleneck — no reviews — + so it is the highest-value outcome.)* 4. **Red → classify caused-by-fix vs unrelated-flake.** If `C.overallConclusion == "failure"`, analyze the PR's OWN failing build (NOT `main`): find the AzDO `maui-pr` build for this PR (filter builds by `branchName=refs/pull//merge`, @@ -723,18 +819,26 @@ Run these gates in order — the FIRST that fires decides this cycle's outcome: method and Step 4.7 flake buckets to `C.failedLegs`. - **Unrelated flake only** (every failed leg is known-flaky / infra / a pre-existing baseline red NOT introduced by the fix): do NOT burn an attempt. - **Idempotency first:** scan the PR's existing comments for a prior bot - `♻️ … unrelated flake … on ` note for THIS head SHA — if one already - exists, record `already-annotated-flake PR # (head )` and stop - WITHOUT re-commenting (re-annotating the same flake every 12h sweep is noise and - burns the per-run comment budget other PRs need). Otherwise resolve the attempt - number from `C.effectiveAttempt` (the authoritative max(marker, bot-commit) - counter), omitting the `Attempt /10:` prefix only if it is somehow - indeterminate. **Dry-run gate (Step 0):** if `dry_run == "true"`, do NOT emit any comment — instead print the intended `♻️` unrelated-flake body to the run log, tally `dry-run: would-annotate-flake PR #`, and stop. Otherwise `add_comment` on PR #: `♻️ Attempt /10: red is - unrelated flake on leg(s) () on ; the fix itself is not - implicated. A maintainer re-run (/azp run ) should clear it.` Record - `annotated-flake PR # (head )` and stop. *(Round 1: human re-runs; - Round 2: auto re-trigger.)* + **Comment idempotency + dry-run suppress the ♻️ comment ONLY — neither skips Step + 3.6.** Scan the PR's existing comments for a prior bot `♻️ … unrelated flake … on + ` note for THIS head SHA; if one exists, set `SKIP_FLAKE_COMMENT = true`. + Resolve the attempt number from `C.effectiveAttempt` (the authoritative max(marker, + bot-commit) counter), omitting the `Attempt /10:` prefix only if it is + somehow indeterminate. Then post the flake note UNLESS suppressed: + - if `dry_run == "true"`: do NOT emit — print the intended `♻️` unrelated-flake body + to the run log and tally `dry-run: would-annotate-flake PR #`; + - else if `SKIP_FLAKE_COMMENT`: do NOT re-post — record `already-annotated-flake PR + # (head )` (re-annotating the same flake every 12h sweep is noise and + burns the per-run comment budget other PRs need); + - else `add_comment` on PR #: `♻️ Attempt /10: red is unrelated flake on + leg(s) () on ; the fix itself is not implicated. A + maintainer re-run (/azp run ) should clear it.` record `annotated-flake + PR # (head )`. + Then — in ALL of the above cases — run **Step 3.6** (target-test readiness gate) for + this PR before stopping: its `/azp`-gated target legs may conclude green on this SAME + head SHA on a LATER sweep, and Step 3.6 is the ONLY place the draft→ready flip + happens, so it MUST re-evaluate every sweep — never `stop` here before it. *(Round 1: + human re-runs; Round 2: auto re-trigger.)* - **Caused by the fix** (a failed leg still matches the original target signature, or the fix introduced a NEW failure): advance an attempt. - **Attempt count.** `attempt = C.effectiveAttempt` — the authoritative @@ -752,6 +856,188 @@ Run these gates in order — the FIRST that fires decides this cycle's outcome: from every prior commit on this branch**, emitted via the Step 5.6 ADVANCE path (push onto the same PR). +#### Step 3.6 — Target-test verification & mark-ready gate + +Reached from the Step 3 **green-surface** branch, the Step 4 **unrelated-flake** branch +(AFTER that branch has posted its comment), and the Step 3.5 **gate-2 target-focused +fast-path** (2a — while unrelated legs are still pending). Purpose: confirm the SPECIFIC +test(s) this PR was opened to fix now PASS in the PR's own CI, and — only when they do +— transition the draft `[ci-fix]` PR to **ready for review** so a maintainer sees a +validated fix instead of a draft. This is the ONLY place the loop flips draft→ready; it +is a state transition, **never** an approval or a merge (a human still reviews and +merges). Overall red on *unrelated* legs must NOT gate readiness — we validate the fix, +not the base branch's flakiness. + +**Preconditions** (ALL must hold; otherwise record `skipped: readiness N/A PR # +()` and stop this gate): +- `C.isDraft == true` — if the PR is already ready-for-review, the draft→ready + transition is already done; do NOT re-mark **and do NOT re-apply `p/0`**. `p/0` is applied + exactly ONCE, atomically with the draft→ready flip (T3 — all-or-nothing with the 🎯 audit + comment + mark-ready), so an already-ready loop PR that is MISSING `p/0` has almost + certainly had it **removed by a maintainer** de-prioritizing the PR — a pure triage action + the human-engagement guard (which inspects comments/reviews/commits, NOT label changes) + cannot see. Re-adding `p/0` every sweep would fight that maintainer indefinitely, violating + the loop's "never override a human" contract. So the loop does NOT reconcile the label: + record `already-ready PR #` and stop this gate. (Trade-off: on the rare occasion a + transient API error drops `p/0` at flip time *after* the T3 atomic pre-check passed, it is + not auto-re-added — but the PR is still ready-for-review with its 🎯 audit comment, and + re-adding `p/0` is a trivial manual action; that is strictly preferable to steam-rolling a + maintainer's deliberate de-prioritization.) +- The PR is unmistakably THIS workflow's own: `[ci-fix] ` title prefix AND the + `agentic-workflows` label (mirrors the safe-output lock; the compensating scope + control if the v0.80.9 compiler drops the declarative required-*). +- You reached this gate from Step 3 (green), Step 4 (**unrelated-flake**), or the Step 3.5 + gate-2 **target-focused fast-path** (2a) — i.e. the fix is NOT implicated in any red that + has concluded. If Step 4 classified a red as **caused by the fix** (ADVANCE), do NOT run + this gate — advance the attempt instead. + +**T1 — Identify the target test(s).** From the `[ci-scan]` issue signature the fix +addresses (and the PR's own diff), extract the fully-qualified test method name(s) the +fix targets — e.g. `SafeAreaShouldWorkOnAllShellTabs`. For a de-flake it is the +de-flaked test; for a product fix it is the originally-failing test(s). If NO specific +test can be identified (e.g. a product build-break rather than a test failure), this is a +**build-only fix** — there is no single test to validate cross-platform, so validate at +**whole-build** granularity rather than stopping (Step 3 only *comments*; Step 3.6 is the +sole draft→ready flip, so a build-only fix is undrafted HERE or not at all). We only reach +this gate from Step 3 (green) / Step 4 (unrelated-flake) / gate-2a, so the fix is not +implicated in any concluded red; additionally require, via the T2 timeline method on +`C.headSha`, that the primary build pipeline `maui-pr` (def 302) has CONCLUDED with EVERY +one of its platform build legs `succeeded`/`completed` and NONE failed/canceled or still +pending — the build is green on **every** platform, not just the originally-broken one (the +cross-platform guard, applied to the build instead of a test). A build-only fix is undrafted +ONLY on the strength of the auto-running `maui-pr` (def 302) whole-build green, which the +loop CAN observe: if the PR's `[ci-scan]` issue signature or its own diff indicates the +ORIGINATING failure was in a `/azp`-gated pipeline (`maui-pr-uitests` def 313 or +`maui-pr-devicetests` def 314) rather than `maui-pr` (def 302), do NOT undraft on +`maui-pr`-green alone — those pipelines do not auto-run on this PR (GITHUB_TOKEN cannot +trigger them), so a green `maui-pr` build is NOT evidence the gated build break is fixed; +record `skipped: build-only fix PR # targets gated pipeline () not run — +deferring to human` and stop this gate. Otherwise, set `TARGET := "the maui-pr build +(build-only fix — no single target test)"` and proceed to **T3** to mark ready. If any `maui-pr` build leg is still unconcluded, record `skipped: build-only fix PR +# not yet whole-build green (leg(s) pending)` and stop this gate WITHOUT marking +ready (a green subset is not enough — a still-pending build leg could yet fail). + +**T2 — Verify the target test's platform legs are green (anonymous, leg-level).** Per-test +outcomes come from the AzDO **test-results** API (`_apis/test/...`), which is **NOT reachable +anonymously — Hard-Rule 8**: those endpoints 302-redirect to a sign-in page on this runner, so +a `curl` returns an HTML redirect (not JSON) and every target test would look "not executed". +Validate instead at **leg granularity** using ONLY the anonymous `_apis/build/...` timeline +(Hard-Rule 8) — a `succeeded` category leg means every test in that category **that actually +ran** passed on that platform. This is a strong bar **only if the target test genuinely +executes**: a job can go green while one specific test is skipped at runtime (`[Ignore]`, +`Assert.Ignore()`, `Assert.Inconclusive()`, a `Skip=`/conditional `[Fact]`, an `#if`-out, or +an early `return` before the asserts). So before trusting a green leg as proof the target +passed, **confirm from the PR's OWN diff that the fix does NOT skip, ignore, disable, or +short-circuit the target test** (it adds no `[Ignore]`/`[Explicit]`, `Assert.Ignore`/ +`Assert.Inconclusive`, `Skip=`, category-exclusion, `#if`-out, or early `return` around the +target). If the fix could cause the target to be runtime-skipped rather than genuinely pass, +leg-level green is NOT sufficient evidence — record `skipped: target test may be +runtime-skipped by this fix (diff adds ); leg-green insufficient, deferring to human +on PR #` and stop this gate WITHOUT marking ready. Otherwise (the target genuinely runs +and asserts, as a de-flake or product fix does) a green category leg cannot hide a +target-test failure, so treat it as authoritative. + +Map the target test to the CI leg(s) that run it. A leg name encodes platform + test category +(e.g. `Android UITests SafeAreaEdges,Shadow`, `iOS UITests SafeAreaEdges,Shadow`, `macOS +UITests SafeAreaEdges,Shadow`, `Windows UITests SafeAreaEdges,Shadow`). Determine the target +test's UI-test category — its `[Category(UITestCategories.X)]` in the test/HostApp source, +visible in the PR diff or the test file — to know which leg-name substring identifies its legs; +for a device test, the per-platform device-test legs. + +Using the SAME build-discovery as Step 4 (filter AzDO builds by `branchName=refs/pull//merge` +or `sourceVersion == C.headSha`), read each build's **timeline** on `C.headSha` for the +pipeline(s) that RUN the target test — `maui-pr` (def 302) for unit/integration tests, +`maui-pr-uitests` (def 313) for Appium UI tests, `maui-pr-devicetests` (def 314) for device +tests: + +```bash +ORG=dnceng-public; PROJ=public; BUILD= +# Anonymous + Hard-Rule-8-compliant. NEVER call _apis/test/... (it 302-redirects to sign-in). +# Each type=="Job" record is one platform leg: result (succeeded/failed/canceled/null), +# state (completed/inProgress/pending), name (encodes " UITests "): +curl -s "https://dev.azure.com/$ORG/$PROJ/_apis/build/builds/$BUILD/timeline?api-version=7.1" \ + | tee /tmp/gh-aw/agent/timeline_${BUILD}.json \ + | jq -r '.records[] | select(.type=="Job") | "\(.result)\t\(.state)\t\(.name)"' +``` + +(The prefetch already exposes per-leg status in `C.failedLegs` / the checks context, and +`gh pr checks ` lists the same per-leg rows with platform+category in the name — use +whichever is handy; the build timeline is the authoritative cross-check on the exact build.) + +Treat a target test as **VALIDATED-GREEN** only if, on `C.headSha`, the leg that runs its +category is green on **every platform it runs on** — a fix that repairs one platform must not +silently regress the same test's leg on another, so a green leg on the originally-red platform +alone is NOT enough. Across the target pipeline's platform legs (for a UI test: the Android, +iOS, Windows, and macOS/MacCatalyst legs running the target's category; for a device test: each +device-test platform), require ALL of: +- the target's category leg is `result == "succeeded"` **and** `state == "completed"` on + **every** platform that runs it, AND +- that leg is `failed` / `canceled` / aborted on **NO** platform, AND +- **no platform is left unverified.** For each platform family the target pipeline covers, that + platform's category leg must have CONCLUDED (`state == "completed"`) on `C.headSha`. If a + platform simply has no leg for the target's category, the test does not run there — that is + fine, not a gap. But if a platform's category leg has **not concluded** (`state` is + `inProgress` / pending, or an `/azp`-gated `maui-pr-uitests` / `maui-pr-devicetests` leg that + has not been kicked), the test's status on that platform is UNKNOWN → the fix is NOT yet + validated across platforms: record `skipped: target test green on but + not yet verified on (leg(s) pending / need /azp run) on PR #` + and stop this gate WITHOUT marking ready. + +A target test whose category leg never concluded on ANY platform (**not executed** anywhere — +e.g. its `/azp`-gated pipeline has not been kicked) is likewise NOT validated: record `skipped: +target test not yet executed on PR # ( not run — needs /azp run)` and stop +this gate WITHOUT marking ready. Do NOT overclaim — a green *sibling* leg (a different category +on the same platform) is not the target's leg, and a green leg on one platform is not a pass on +the others. + +**T3 — Mark ready + report.** If EVERY target test is VALIDATED-GREEN. The 🎯 comment, +the mark-ready, and the `p/0` label are THREE SEPARATE safe-outputs — the comment +existing does NOT prove the mark-ready took effect, so they are tracked independently: + +- **Comment idempotency (dup-suppress only — never gates the mark-ready).** We only reach + T3 while `C.isDraft == true` (the precondition stops an already-ready PR before here). So + if a prior bot comment for THIS head that carries the leading `🎯` anchor AND the + contiguous phrase `validated green on ` already exists (BOTH T3 variants — + `🎯 Target test validated green on ` and the build-only `🎯 Build validated green on + ` — contain that exact contiguous phrase, so this recognizes both; keying on + `target test` alone would instead miss the build-only comment and re-post it every sweep. + **Require the `🎯` anchor** so the match can NEVER be loosened to a gapped + `validated…green…on` form that would false-positive on the Step 3 `✅ … validated — the + fix's CI is green on ` surface comment and wrongly suppress the audit 🎯), the comment + landed on an earlier sweep but the **mark-ready did NOT take + effect** (the PR is still a draft) — set `SUPPRESS_COMMENT = true` (do not re-post the + duplicate 🎯 comment) but STILL complete the mark-ready + label below. Never treat the + comment's existence as "already marked ready" while the PR is still a draft. Record this + case as `re-marking PR # (🎯 present; mark-ready did not take on a prior sweep)`. +- **Atomicity budget pre-check (Hard-Rule 7).** Determine the outputs this gate will emit: + `mark_pull_request_as_ready_for_review` + `add_labels` always, plus `add_comment` unless + `SUPPRESS_COMMENT`. Verify a free per-run slot remains in EACH bucket you are about to + use. If ANY required bucket is exhausted, emit NONE of them — record `skipped: per-run + cap reached; deferring mark-ready PR # to next cycle` and stop this gate. Never mark a + PR ready (or label it) unless its 🎯 audit comment is GUARANTEED to exist for this exact + head SHA — either posted in this same sweep, or (in the `SUPPRESS_COMMENT` re-marking case) + already present from a prior sweep. The outputs you DO emit either all land or all defer + together; the audit trail must never be absent, but it is NOT re-posted when it already exists. +- **Dry-run gate (Step 0):** if `dry_run == "true"`, emit NOTHING — print the intended + readiness comment and "would mark ready + add p/0" to the run log, tally `dry-run: + would-mark-ready PR #`, and stop. +- Otherwise emit for THIS PR number ``: + 1. `add_comment` (ONLY if not `SUPPRESS_COMMENT`): `🎯 Target test validated green on + — passed on ALL platforms it runs on (, + buildId ). — not + caused by this fix."> Transitioning this PR from draft to ready for review and adding + `p/0`; a maintainer still reviews and merges.` (For a **build-only fix** — the T1 + whole-build fallback, no single target test — phrase the first clause as `🎯 Build + validated green on — the maui-pr build passed on ALL platforms (buildId + ).` instead of naming a test.) + 2. `mark_pull_request_as_ready_for_review` with `reason:` a one-line justification + naming the validated test(s) and ``. + 3. `add_labels` with `labels: ["p/0"]` for PR # — put the now-review-ready fix into + the team's p/0 priority queue so it is triaged, not lost in the draft backlog. (If + the PR somehow already carries `p/0`, this is a harmless no-op.) +- Record `marked-ready PR # (target green on ALL platforms on , + labeled p/0)` and stop. + #### Step 3.5.R — Maintainer change-request response (Track C) Reached from Step 3.5 gate 0. Goal: when an **eligible human reviewer** (Hard-Rule @@ -1383,7 +1669,19 @@ Filed by [`ci-status-fix`](https://github.com/dotnet/maui/blob/main/.github/work ### Step 8 — Per-issue tally + end-of-run summary -Per issue, append one outcome line to `/tmp/gh-aw/agent/coverage.txt`: +Per issue, append **one** outcome line to `/tmp/gh-aw/agent/coverage.txt` — the +terminal outcome for the cycle. When one cycle produces a chained pair — a PR gets a +same-cycle precursor line and **then** a Step 3.6 readiness line (`marked-ready` on the +draft→ready flip, or `already-ready` on a later steady-state sweep of an already-ready +PR) — record ONLY the terminal Step 3.6 readiness line: it supersedes ANY same-cycle +non-readiness precursor for that PR, so an aggregator keying on one-line-per-issue never +double-counts. The precursor may be a Step 3 green line (`surfaced-green` on the first ✅, +or `already-surfaced` when it already exists) OR — when an unrelated-flake red and a +target-green coincide — a Step 4 `annotated-flake` line; either way the readiness line is +terminal, and the superseded precursor's signal survives in the PR's 🎯/♻️ comment so no +information is lost. This covers the flip cycle (`surfaced-green` → `marked-ready`), the +post-flip steady state (`already-surfaced` → `already-ready`), and the flake-coincident +flip (`annotated-flake` → `marked-ready`): ``` # main attempt- @@ -1394,8 +1692,14 @@ Per issue, append one outcome line to `/tmp/gh-aw/agent/coverage.txt`: `advance-PR # attempt /10` (ADVANCE mode — new commit pushed to the existing PR), `surfaced-green PR #` (fix's own CI went green; commented for review, did not advance), `annotated-flake PR #` (red was unrelated flake; -commented, attempt NOT burned), `waiting PR #` (CI not settled yet), -`dry-run: would-`, `skipped: `. (The +commented, attempt NOT burned), `marked-ready PR #` (the specific fixed test +was confirmed green in the PR's own CI; draft flipped to ready for review), +`already-surfaced PR #` (the fix's ✅ green comment already existed this sweep; +re-post suppressed — superseded by the Step 3.6 readiness line when the PR also flips +ready that cycle), `already-ready PR #` (terminal steady state — the PR was flipped +ready on a prior cycle and remains green on an unchanged head; no action taken, +supersedes `already-surfaced`), `waiting PR #` (CI not settled yet), +`dry-run: would-`, `skipped: `. (The `needs-human-PR` outcome is reserved for the deferred hand-off — Step 6 currently records a skip instead, so it is not emitted.)
: `✅ Attempt /10 validated - — the fix's CI is green on . Ready for human review.` - Name any `/azp`-gated legs (uitests def 313 / devicetests def 314) that have not - run and still need a maintainer `/azp run`. Do NOT advance. Record - `surfaced-green PR # (attempt /10)` and stop. *(This directly - attacks the real bottleneck — no reviews — so it is the highest-value outcome.)* + reason to withhold the surface.) **Comment idempotency + dry-run suppress the ✅ + comment ONLY — neither skips Step 3.6.** Scan the PR's existing comments for a prior + bot `✅ … validated … on ` note for THIS head SHA; if one exists, set + `SKIP_SURFACE_COMMENT = true`. Resolve the attempt number from `C.effectiveAttempt` + (the authoritative max(marker, bot-commit) counter; omit the number only if it is + somehow indeterminate). Then post the surface comment UNLESS suppressed: + - if `dry_run == "true"`: do NOT emit — print the intended `✅` validated-green body to + the run log and tally `dry-run: would-surface-green PR #`; + - else if `SKIP_SURFACE_COMMENT`: do NOT re-post — record `already-surfaced PR # + (head )` (re-surfacing the same green every tick is noise and burns the + per-run comment budget other PRs need); + - else `add_comment` on PR #: `✅ Attempt /10 validated — the fix's CI is + green on .` naming any `/azp`-gated legs (uitests def 313 / devicetests def + 314) that have not run and still need a maintainer `/azp run`; record `surfaced-green + PR # (attempt /10)`. (Do NOT assert "ready for human review" on a PR that + is still a draft — Step 3.6, not this comment, owns the draft→ready flip and posts its + own 🎯 announcement when it fires.) + Do NOT advance. Then — in ALL of the above cases — run **Step 3.6** (target-test + readiness gate) for this PR before stopping: its `/azp`-gated target legs may conclude + green on this SAME head SHA on a LATER sweep (an `/azp run` adds no commit), and Step + 3.6 is the ONLY place the draft→ready flip happens, so it MUST re-evaluate every sweep — + never `stop` here before it. *(This directly attacks the real bottleneck — no reviews — + so it is the highest-value outcome.)* 4. **Red → classify caused-by-fix vs unrelated-flake.** If `C.overallConclusion == "failure"`, analyze the PR's OWN failing build (NOT `main`): find the AzDO `maui-pr` build for this PR (filter builds by `branchName=refs/pull//merge`, @@ -723,18 +819,26 @@ Run these gates in order — the FIRST that fires decides this cycle's outcome: method and Step 4.7 flake buckets to `C.failedLegs`. - **Unrelated flake only** (every failed leg is known-flaky / infra / a pre-existing baseline red NOT introduced by the fix): do NOT burn an attempt. - **Idempotency first:** scan the PR's existing comments for a prior bot - `♻️ … unrelated flake … on ` note for THIS head SHA — if one already - exists, record `already-annotated-flake PR # (head )` and stop - WITHOUT re-commenting (re-annotating the same flake every 12h sweep is noise and - burns the per-run comment budget other PRs need). Otherwise resolve the attempt - number from `C.effectiveAttempt` (the authoritative max(marker, bot-commit) - counter), omitting the `Attempt /10:` prefix only if it is somehow - indeterminate. **Dry-run gate (Step 0):** if `dry_run == "true"`, do NOT emit any comment — instead print the intended `♻️` unrelated-flake body to the run log, tally `dry-run: would-annotate-flake PR #`, and stop. Otherwise `add_comment` on PR #: `♻️ Attempt /10: red is - unrelated flake on leg(s) () on ; the fix itself is not - implicated. A maintainer re-run (/azp run ) should clear it.` Record - `annotated-flake PR # (head )` and stop. *(Round 1: human re-runs; - Round 2: auto re-trigger.)* + **Comment idempotency + dry-run suppress the ♻️ comment ONLY — neither skips Step + 3.6.** Scan the PR's existing comments for a prior bot `♻️ … unrelated flake … on + ` note for THIS head SHA; if one exists, set `SKIP_FLAKE_COMMENT = true`. + Resolve the attempt number from `C.effectiveAttempt` (the authoritative max(marker, + bot-commit) counter), omitting the `Attempt /10:` prefix only if it is + somehow indeterminate. Then post the flake note UNLESS suppressed: + - if `dry_run == "true"`: do NOT emit — print the intended `♻️` unrelated-flake body + to the run log and tally `dry-run: would-annotate-flake PR #`; + - else if `SKIP_FLAKE_COMMENT`: do NOT re-post — record `already-annotated-flake PR + # (head )` (re-annotating the same flake every 12h sweep is noise and + burns the per-run comment budget other PRs need); + - else `add_comment` on PR #: `♻️ Attempt /10: red is unrelated flake on + leg(s) () on ; the fix itself is not implicated. A + maintainer re-run (/azp run ) should clear it.` record `annotated-flake + PR # (head )`. + Then — in ALL of the above cases — run **Step 3.6** (target-test readiness gate) for + this PR before stopping: its `/azp`-gated target legs may conclude green on this SAME + head SHA on a LATER sweep, and Step 3.6 is the ONLY place the draft→ready flip + happens, so it MUST re-evaluate every sweep — never `stop` here before it. *(Round 1: + human re-runs; Round 2: auto re-trigger.)* - **Caused by the fix** (a failed leg still matches the original target signature, or the fix introduced a NEW failure): advance an attempt. - **Attempt count.** `attempt = C.effectiveAttempt` — the authoritative @@ -752,6 +856,188 @@ Run these gates in order — the FIRST that fires decides this cycle's outcome: from every prior commit on this branch**, emitted via the Step 5.6 ADVANCE path (push onto the same PR). +#### Step 3.6 — Target-test verification & mark-ready gate + +Reached from the Step 3 **green-surface** branch, the Step 4 **unrelated-flake** branch +(AFTER that branch has posted its comment), and the Step 3.5 **gate-2 target-focused +fast-path** (2a — while unrelated legs are still pending). Purpose: confirm the SPECIFIC +test(s) this PR was opened to fix now PASS in the PR's own CI, and — only when they do +— transition the draft `[ci-fix]` PR to **ready for review** so a maintainer sees a +validated fix instead of a draft. This is the ONLY place the loop flips draft→ready; it +is a state transition, **never** an approval or a merge (a human still reviews and +merges). Overall red on *unrelated* legs must NOT gate readiness — we validate the fix, +not the base branch's flakiness. + +**Preconditions** (ALL must hold; otherwise record `skipped: readiness N/A PR # +()` and stop this gate): +- `C.isDraft == true` — if the PR is already ready-for-review, the draft→ready + transition is already done; do NOT re-mark **and do NOT re-apply `p/0`**. `p/0` is applied + exactly ONCE, atomically with the draft→ready flip (T3 — all-or-nothing with the 🎯 audit + comment + mark-ready), so an already-ready loop PR that is MISSING `p/0` has almost + certainly had it **removed by a maintainer** de-prioritizing the PR — a pure triage action + the human-engagement guard (which inspects comments/reviews/commits, NOT label changes) + cannot see. Re-adding `p/0` every sweep would fight that maintainer indefinitely, violating + the loop's "never override a human" contract. So the loop does NOT reconcile the label: + record `already-ready PR #` and stop this gate. (Trade-off: on the rare occasion a + transient API error drops `p/0` at flip time *after* the T3 atomic pre-check passed, it is + not auto-re-added — but the PR is still ready-for-review with its 🎯 audit comment, and + re-adding `p/0` is a trivial manual action; that is strictly preferable to steam-rolling a + maintainer's deliberate de-prioritization.) +- The PR is unmistakably THIS workflow's own: `[ci-fix] ` title prefix AND the + `agentic-workflows` label (mirrors the safe-output lock; the compensating scope + control if the v0.80.9 compiler drops the declarative required-*). +- You reached this gate from Step 3 (green), Step 4 (**unrelated-flake**), or the Step 3.5 + gate-2 **target-focused fast-path** (2a) — i.e. the fix is NOT implicated in any red that + has concluded. If Step 4 classified a red as **caused by the fix** (ADVANCE), do NOT run + this gate — advance the attempt instead. + +**T1 — Identify the target test(s).** From the `[ci-scan]` issue signature the fix +addresses (and the PR's own diff), extract the fully-qualified test method name(s) the +fix targets — e.g. `SafeAreaShouldWorkOnAllShellTabs`. For a de-flake it is the +de-flaked test; for a product fix it is the originally-failing test(s). If NO specific +test can be identified (e.g. a product build-break rather than a test failure), this is a +**build-only fix** — there is no single test to validate cross-platform, so validate at +**whole-build** granularity rather than stopping (Step 3 only *comments*; Step 3.6 is the +sole draft→ready flip, so a build-only fix is undrafted HERE or not at all). We only reach +this gate from Step 3 (green) / Step 4 (unrelated-flake) / gate-2a, so the fix is not +implicated in any concluded red; additionally require, via the T2 timeline method on +`C.headSha`, that the primary build pipeline `maui-pr` (def 302) has CONCLUDED with EVERY +one of its platform build legs `succeeded`/`completed` and NONE failed/canceled or still +pending — the build is green on **every** platform, not just the originally-broken one (the +cross-platform guard, applied to the build instead of a test). A build-only fix is undrafted +ONLY on the strength of the auto-running `maui-pr` (def 302) whole-build green, which the +loop CAN observe: if the PR's `[ci-scan]` issue signature or its own diff indicates the +ORIGINATING failure was in a `/azp`-gated pipeline (`maui-pr-uitests` def 313 or +`maui-pr-devicetests` def 314) rather than `maui-pr` (def 302), do NOT undraft on +`maui-pr`-green alone — those pipelines do not auto-run on this PR (GITHUB_TOKEN cannot +trigger them), so a green `maui-pr` build is NOT evidence the gated build break is fixed; +record `skipped: build-only fix PR # targets gated pipeline () not run — +deferring to human` and stop this gate. Otherwise, set `TARGET := "the maui-pr build +(build-only fix — no single target test)"` and proceed to **T3** to mark ready. If any `maui-pr` build leg is still unconcluded, record `skipped: build-only fix PR +# not yet whole-build green (leg(s) pending)` and stop this gate WITHOUT marking +ready (a green subset is not enough — a still-pending build leg could yet fail). + +**T2 — Verify the target test's platform legs are green (anonymous, leg-level).** Per-test +outcomes come from the AzDO **test-results** API (`_apis/test/...`), which is **NOT reachable +anonymously — Hard-Rule 8**: those endpoints 302-redirect to a sign-in page on this runner, so +a `curl` returns an HTML redirect (not JSON) and every target test would look "not executed". +Validate instead at **leg granularity** using ONLY the anonymous `_apis/build/...` timeline +(Hard-Rule 8) — a `succeeded` category leg means every test in that category **that actually +ran** passed on that platform. This is a strong bar **only if the target test genuinely +executes**: a job can go green while one specific test is skipped at runtime (`[Ignore]`, +`Assert.Ignore()`, `Assert.Inconclusive()`, a `Skip=`/conditional `[Fact]`, an `#if`-out, or +an early `return` before the asserts). So before trusting a green leg as proof the target +passed, **confirm from the PR's OWN diff that the fix does NOT skip, ignore, disable, or +short-circuit the target test** (it adds no `[Ignore]`/`[Explicit]`, `Assert.Ignore`/ +`Assert.Inconclusive`, `Skip=`, category-exclusion, `#if`-out, or early `return` around the +target). If the fix could cause the target to be runtime-skipped rather than genuinely pass, +leg-level green is NOT sufficient evidence — record `skipped: target test may be +runtime-skipped by this fix (diff adds ); leg-green insufficient, deferring to human +on PR #` and stop this gate WITHOUT marking ready. Otherwise (the target genuinely runs +and asserts, as a de-flake or product fix does) a green category leg cannot hide a +target-test failure, so treat it as authoritative. + +Map the target test to the CI leg(s) that run it. A leg name encodes platform + test category +(e.g. `Android UITests SafeAreaEdges,Shadow`, `iOS UITests SafeAreaEdges,Shadow`, `macOS +UITests SafeAreaEdges,Shadow`, `Windows UITests SafeAreaEdges,Shadow`). Determine the target +test's UI-test category — its `[Category(UITestCategories.X)]` in the test/HostApp source, +visible in the PR diff or the test file — to know which leg-name substring identifies its legs; +for a device test, the per-platform device-test legs. + +Using the SAME build-discovery as Step 4 (filter AzDO builds by `branchName=refs/pull//merge` +or `sourceVersion == C.headSha`), read each build's **timeline** on `C.headSha` for the +pipeline(s) that RUN the target test — `maui-pr` (def 302) for unit/integration tests, +`maui-pr-uitests` (def 313) for Appium UI tests, `maui-pr-devicetests` (def 314) for device +tests: + +```bash +ORG=dnceng-public; PROJ=public; BUILD= +# Anonymous + Hard-Rule-8-compliant. NEVER call _apis/test/... (it 302-redirects to sign-in). +# Each type=="Job" record is one platform leg: result (succeeded/failed/canceled/null), +# state (completed/inProgress/pending), name (encodes " UITests "): +curl -s "https://dev.azure.com/$ORG/$PROJ/_apis/build/builds/$BUILD/timeline?api-version=7.1" \ + | tee /tmp/gh-aw/agent/timeline_${BUILD}.json \ + | jq -r '.records[] | select(.type=="Job") | "\(.result)\t\(.state)\t\(.name)"' +``` + +(The prefetch already exposes per-leg status in `C.failedLegs` / the checks context, and +`gh pr checks ` lists the same per-leg rows with platform+category in the name — use +whichever is handy; the build timeline is the authoritative cross-check on the exact build.) + +Treat a target test as **VALIDATED-GREEN** only if, on `C.headSha`, the leg that runs its +category is green on **every platform it runs on** — a fix that repairs one platform must not +silently regress the same test's leg on another, so a green leg on the originally-red platform +alone is NOT enough. Across the target pipeline's platform legs (for a UI test: the Android, +iOS, Windows, and macOS/MacCatalyst legs running the target's category; for a device test: each +device-test platform), require ALL of: +- the target's category leg is `result == "succeeded"` **and** `state == "completed"` on + **every** platform that runs it, AND +- that leg is `failed` / `canceled` / aborted on **NO** platform, AND +- **no platform is left unverified.** For each platform family the target pipeline covers, that + platform's category leg must have CONCLUDED (`state == "completed"`) on `C.headSha`. If a + platform simply has no leg for the target's category, the test does not run there — that is + fine, not a gap. But if a platform's category leg has **not concluded** (`state` is + `inProgress` / pending, or an `/azp`-gated `maui-pr-uitests` / `maui-pr-devicetests` leg that + has not been kicked), the test's status on that platform is UNKNOWN → the fix is NOT yet + validated across platforms: record `skipped: target test green on but + not yet verified on (leg(s) pending / need /azp run) on PR #` + and stop this gate WITHOUT marking ready. + +A target test whose category leg never concluded on ANY platform (**not executed** anywhere — +e.g. its `/azp`-gated pipeline has not been kicked) is likewise NOT validated: record `skipped: +target test not yet executed on PR # ( not run — needs /azp run)` and stop +this gate WITHOUT marking ready. Do NOT overclaim — a green *sibling* leg (a different category +on the same platform) is not the target's leg, and a green leg on one platform is not a pass on +the others. + +**T3 — Mark ready + report.** If EVERY target test is VALIDATED-GREEN. The 🎯 comment, +the mark-ready, and the `p/0` label are THREE SEPARATE safe-outputs — the comment +existing does NOT prove the mark-ready took effect, so they are tracked independently: + +- **Comment idempotency (dup-suppress only — never gates the mark-ready).** We only reach + T3 while `C.isDraft == true` (the precondition stops an already-ready PR before here). So + if a prior bot comment for THIS head that carries the leading `🎯` anchor AND the + contiguous phrase `validated green on ` already exists (BOTH T3 variants — + `🎯 Target test validated green on ` and the build-only `🎯 Build validated green on + ` — contain that exact contiguous phrase, so this recognizes both; keying on + `target test` alone would instead miss the build-only comment and re-post it every sweep. + **Require the `🎯` anchor** so the match can NEVER be loosened to a gapped + `validated…green…on` form that would false-positive on the Step 3 `✅ … validated — the + fix's CI is green on ` surface comment and wrongly suppress the audit 🎯), the comment + landed on an earlier sweep but the **mark-ready did NOT take + effect** (the PR is still a draft) — set `SUPPRESS_COMMENT = true` (do not re-post the + duplicate 🎯 comment) but STILL complete the mark-ready + label below. Never treat the + comment's existence as "already marked ready" while the PR is still a draft. Record this + case as `re-marking PR # (🎯 present; mark-ready did not take on a prior sweep)`. +- **Atomicity budget pre-check (Hard-Rule 7).** Determine the outputs this gate will emit: + `mark_pull_request_as_ready_for_review` + `add_labels` always, plus `add_comment` unless + `SUPPRESS_COMMENT`. Verify a free per-run slot remains in EACH bucket you are about to + use. If ANY required bucket is exhausted, emit NONE of them — record `skipped: per-run + cap reached; deferring mark-ready PR # to next cycle` and stop this gate. Never mark a + PR ready (or label it) unless its 🎯 audit comment is GUARANTEED to exist for this exact + head SHA — either posted in this same sweep, or (in the `SUPPRESS_COMMENT` re-marking case) + already present from a prior sweep. The outputs you DO emit either all land or all defer + together; the audit trail must never be absent, but it is NOT re-posted when it already exists. +- **Dry-run gate (Step 0):** if `dry_run == "true"`, emit NOTHING — print the intended + readiness comment and "would mark ready + add p/0" to the run log, tally `dry-run: + would-mark-ready PR #`, and stop. +- Otherwise emit for THIS PR number ``: + 1. `add_comment` (ONLY if not `SUPPRESS_COMMENT`): `🎯 Target test validated green on + — passed on ALL platforms it runs on (, + buildId ). — not + caused by this fix."> Transitioning this PR from draft to ready for review and adding + `p/0`; a maintainer still reviews and merges.` (For a **build-only fix** — the T1 + whole-build fallback, no single target test — phrase the first clause as `🎯 Build + validated green on — the maui-pr build passed on ALL platforms (buildId + ).` instead of naming a test.) + 2. `mark_pull_request_as_ready_for_review` with `reason:` a one-line justification + naming the validated test(s) and ``. + 3. `add_labels` with `labels: ["p/0"]` for PR # — put the now-review-ready fix into + the team's p/0 priority queue so it is triaged, not lost in the draft backlog. (If + the PR somehow already carries `p/0`, this is a harmless no-op.) +- Record `marked-ready PR # (target green on ALL platforms on , + labeled p/0)` and stop. + #### Step 3.5.R — Maintainer change-request response (Track C) Reached from Step 3.5 gate 0. Goal: when an **eligible human reviewer** (Hard-Rule @@ -1383,7 +1669,19 @@ Filed by [`ci-status-fix`](https://github.com/dotnet/maui/blob/main/.github/work ### Step 8 — Per-issue tally + end-of-run summary -Per issue, append one outcome line to `/tmp/gh-aw/agent/coverage.txt`: +Per issue, append **one** outcome line to `/tmp/gh-aw/agent/coverage.txt` — the +terminal outcome for the cycle. When one cycle produces a chained pair — a PR gets a +same-cycle precursor line and **then** a Step 3.6 readiness line (`marked-ready` on the +draft→ready flip, or `already-ready` on a later steady-state sweep of an already-ready +PR) — record ONLY the terminal Step 3.6 readiness line: it supersedes ANY same-cycle +non-readiness precursor for that PR, so an aggregator keying on one-line-per-issue never +double-counts. The precursor may be a Step 3 green line (`surfaced-green` on the first ✅, +or `already-surfaced` when it already exists) OR — when an unrelated-flake red and a +target-green coincide — a Step 4 `annotated-flake` line; either way the readiness line is +terminal, and the superseded precursor's signal survives in the PR's 🎯/♻️ comment so no +information is lost. This covers the flip cycle (`surfaced-green` → `marked-ready`), the +post-flip steady state (`already-surfaced` → `already-ready`), and the flake-coincident +flip (`annotated-flake` → `marked-ready`): ``` # main attempt- @@ -1394,8 +1692,14 @@ Per issue, append one outcome line to `/tmp/gh-aw/agent/coverage.txt`: `advance-PR # attempt /10` (ADVANCE mode — new commit pushed to the existing PR), `surfaced-green PR #` (fix's own CI went green; commented for review, did not advance), `annotated-flake PR #` (red was unrelated flake; -commented, attempt NOT burned), `waiting PR #` (CI not settled yet), -`dry-run: would-`, `skipped: `. (The +commented, attempt NOT burned), `marked-ready PR #` (the specific fixed test +was confirmed green in the PR's own CI; draft flipped to ready for review), +`already-surfaced PR #` (the fix's ✅ green comment already existed this sweep; +re-post suppressed — superseded by the Step 3.6 readiness line when the PR also flips +ready that cycle), `already-ready PR #` (terminal steady state — the PR was flipped +ready on a prior cycle and remains green on an unchanged head; no action taken, +supersedes `already-surfaced`), `waiting PR #` (CI not settled yet), +`dry-run: would-`, `skipped: `. (The `needs-human-PR` outcome is reserved for the deferred hand-off — Step 6 currently records a skip instead, so it is not emitted.)
(attempt /10)` and stop. *(This directly - attacks the real bottleneck — no reviews — so it is the highest-value outcome.)* + reason to withhold the surface.) **Comment idempotency + dry-run suppress the ✅ + comment ONLY — neither skips Step 3.6.** Scan the PR's existing comments for a prior + bot `✅ … validated … on ` note for THIS head SHA; if one exists, set + `SKIP_SURFACE_COMMENT = true`. Resolve the attempt number from `C.effectiveAttempt` + (the authoritative max(marker, bot-commit) counter; omit the number only if it is + somehow indeterminate). Then post the surface comment UNLESS suppressed: + - if `dry_run == "true"`: do NOT emit — print the intended `✅` validated-green body to + the run log and tally `dry-run: would-surface-green PR #`; + - else if `SKIP_SURFACE_COMMENT`: do NOT re-post — record `already-surfaced PR # + (head )` (re-surfacing the same green every tick is noise and burns the + per-run comment budget other PRs need); + - else `add_comment` on PR #: `✅ Attempt /10 validated — the fix's CI is + green on .` naming any `/azp`-gated legs (uitests def 313 / devicetests def + 314) that have not run and still need a maintainer `/azp run`; record `surfaced-green + PR # (attempt /10)`. (Do NOT assert "ready for human review" on a PR that + is still a draft — Step 3.6, not this comment, owns the draft→ready flip and posts its + own 🎯 announcement when it fires.) + Do NOT advance. Then — in ALL of the above cases — run **Step 3.6** (target-test + readiness gate) for this PR before stopping: its `/azp`-gated target legs may conclude + green on this SAME head SHA on a LATER sweep (an `/azp run` adds no commit), and Step + 3.6 is the ONLY place the draft→ready flip happens, so it MUST re-evaluate every sweep — + never `stop` here before it. *(This directly attacks the real bottleneck — no reviews — + so it is the highest-value outcome.)* 4. **Red → classify caused-by-fix vs unrelated-flake.** If `C.overallConclusion == "failure"`, analyze the PR's OWN failing build (NOT `main`): find the AzDO `maui-pr` build for this PR (filter builds by `branchName=refs/pull//merge`, @@ -723,18 +819,26 @@ Run these gates in order — the FIRST that fires decides this cycle's outcome: method and Step 4.7 flake buckets to `C.failedLegs`. - **Unrelated flake only** (every failed leg is known-flaky / infra / a pre-existing baseline red NOT introduced by the fix): do NOT burn an attempt. - **Idempotency first:** scan the PR's existing comments for a prior bot - `♻️ … unrelated flake … on ` note for THIS head SHA — if one already - exists, record `already-annotated-flake PR # (head )` and stop - WITHOUT re-commenting (re-annotating the same flake every 12h sweep is noise and - burns the per-run comment budget other PRs need). Otherwise resolve the attempt - number from `C.effectiveAttempt` (the authoritative max(marker, bot-commit) - counter), omitting the `Attempt /10:` prefix only if it is somehow - indeterminate. **Dry-run gate (Step 0):** if `dry_run == "true"`, do NOT emit any comment — instead print the intended `♻️` unrelated-flake body to the run log, tally `dry-run: would-annotate-flake PR #`, and stop. Otherwise `add_comment` on PR #: `♻️ Attempt /10: red is - unrelated flake on leg(s) () on ; the fix itself is not - implicated. A maintainer re-run (/azp run ) should clear it.` Record - `annotated-flake PR # (head )` and stop. *(Round 1: human re-runs; - Round 2: auto re-trigger.)* + **Comment idempotency + dry-run suppress the ♻️ comment ONLY — neither skips Step + 3.6.** Scan the PR's existing comments for a prior bot `♻️ … unrelated flake … on + ` note for THIS head SHA; if one exists, set `SKIP_FLAKE_COMMENT = true`. + Resolve the attempt number from `C.effectiveAttempt` (the authoritative max(marker, + bot-commit) counter), omitting the `Attempt /10:` prefix only if it is + somehow indeterminate. Then post the flake note UNLESS suppressed: + - if `dry_run == "true"`: do NOT emit — print the intended `♻️` unrelated-flake body + to the run log and tally `dry-run: would-annotate-flake PR #`; + - else if `SKIP_FLAKE_COMMENT`: do NOT re-post — record `already-annotated-flake PR + # (head )` (re-annotating the same flake every 12h sweep is noise and + burns the per-run comment budget other PRs need); + - else `add_comment` on PR #: `♻️ Attempt /10: red is unrelated flake on + leg(s) () on ; the fix itself is not implicated. A + maintainer re-run (/azp run ) should clear it.` record `annotated-flake + PR # (head )`. + Then — in ALL of the above cases — run **Step 3.6** (target-test readiness gate) for + this PR before stopping: its `/azp`-gated target legs may conclude green on this SAME + head SHA on a LATER sweep, and Step 3.6 is the ONLY place the draft→ready flip + happens, so it MUST re-evaluate every sweep — never `stop` here before it. *(Round 1: + human re-runs; Round 2: auto re-trigger.)* - **Caused by the fix** (a failed leg still matches the original target signature, or the fix introduced a NEW failure): advance an attempt. - **Attempt count.** `attempt = C.effectiveAttempt` — the authoritative @@ -752,6 +856,188 @@ Run these gates in order — the FIRST that fires decides this cycle's outcome: from every prior commit on this branch**, emitted via the Step 5.6 ADVANCE path (push onto the same PR). +#### Step 3.6 — Target-test verification & mark-ready gate + +Reached from the Step 3 **green-surface** branch, the Step 4 **unrelated-flake** branch +(AFTER that branch has posted its comment), and the Step 3.5 **gate-2 target-focused +fast-path** (2a — while unrelated legs are still pending). Purpose: confirm the SPECIFIC +test(s) this PR was opened to fix now PASS in the PR's own CI, and — only when they do +— transition the draft `[ci-fix]` PR to **ready for review** so a maintainer sees a +validated fix instead of a draft. This is the ONLY place the loop flips draft→ready; it +is a state transition, **never** an approval or a merge (a human still reviews and +merges). Overall red on *unrelated* legs must NOT gate readiness — we validate the fix, +not the base branch's flakiness. + +**Preconditions** (ALL must hold; otherwise record `skipped: readiness N/A PR # +()` and stop this gate): +- `C.isDraft == true` — if the PR is already ready-for-review, the draft→ready + transition is already done; do NOT re-mark **and do NOT re-apply `p/0`**. `p/0` is applied + exactly ONCE, atomically with the draft→ready flip (T3 — all-or-nothing with the 🎯 audit + comment + mark-ready), so an already-ready loop PR that is MISSING `p/0` has almost + certainly had it **removed by a maintainer** de-prioritizing the PR — a pure triage action + the human-engagement guard (which inspects comments/reviews/commits, NOT label changes) + cannot see. Re-adding `p/0` every sweep would fight that maintainer indefinitely, violating + the loop's "never override a human" contract. So the loop does NOT reconcile the label: + record `already-ready PR #` and stop this gate. (Trade-off: on the rare occasion a + transient API error drops `p/0` at flip time *after* the T3 atomic pre-check passed, it is + not auto-re-added — but the PR is still ready-for-review with its 🎯 audit comment, and + re-adding `p/0` is a trivial manual action; that is strictly preferable to steam-rolling a + maintainer's deliberate de-prioritization.) +- The PR is unmistakably THIS workflow's own: `[ci-fix] ` title prefix AND the + `agentic-workflows` label (mirrors the safe-output lock; the compensating scope + control if the v0.80.9 compiler drops the declarative required-*). +- You reached this gate from Step 3 (green), Step 4 (**unrelated-flake**), or the Step 3.5 + gate-2 **target-focused fast-path** (2a) — i.e. the fix is NOT implicated in any red that + has concluded. If Step 4 classified a red as **caused by the fix** (ADVANCE), do NOT run + this gate — advance the attempt instead. + +**T1 — Identify the target test(s).** From the `[ci-scan]` issue signature the fix +addresses (and the PR's own diff), extract the fully-qualified test method name(s) the +fix targets — e.g. `SafeAreaShouldWorkOnAllShellTabs`. For a de-flake it is the +de-flaked test; for a product fix it is the originally-failing test(s). If NO specific +test can be identified (e.g. a product build-break rather than a test failure), this is a +**build-only fix** — there is no single test to validate cross-platform, so validate at +**whole-build** granularity rather than stopping (Step 3 only *comments*; Step 3.6 is the +sole draft→ready flip, so a build-only fix is undrafted HERE or not at all). We only reach +this gate from Step 3 (green) / Step 4 (unrelated-flake) / gate-2a, so the fix is not +implicated in any concluded red; additionally require, via the T2 timeline method on +`C.headSha`, that the primary build pipeline `maui-pr` (def 302) has CONCLUDED with EVERY +one of its platform build legs `succeeded`/`completed` and NONE failed/canceled or still +pending — the build is green on **every** platform, not just the originally-broken one (the +cross-platform guard, applied to the build instead of a test). A build-only fix is undrafted +ONLY on the strength of the auto-running `maui-pr` (def 302) whole-build green, which the +loop CAN observe: if the PR's `[ci-scan]` issue signature or its own diff indicates the +ORIGINATING failure was in a `/azp`-gated pipeline (`maui-pr-uitests` def 313 or +`maui-pr-devicetests` def 314) rather than `maui-pr` (def 302), do NOT undraft on +`maui-pr`-green alone — those pipelines do not auto-run on this PR (GITHUB_TOKEN cannot +trigger them), so a green `maui-pr` build is NOT evidence the gated build break is fixed; +record `skipped: build-only fix PR # targets gated pipeline () not run — +deferring to human` and stop this gate. Otherwise, set `TARGET := "the maui-pr build +(build-only fix — no single target test)"` and proceed to **T3** to mark ready. If any `maui-pr` build leg is still unconcluded, record `skipped: build-only fix PR +# not yet whole-build green (leg(s) pending)` and stop this gate WITHOUT marking +ready (a green subset is not enough — a still-pending build leg could yet fail). + +**T2 — Verify the target test's platform legs are green (anonymous, leg-level).** Per-test +outcomes come from the AzDO **test-results** API (`_apis/test/...`), which is **NOT reachable +anonymously — Hard-Rule 8**: those endpoints 302-redirect to a sign-in page on this runner, so +a `curl` returns an HTML redirect (not JSON) and every target test would look "not executed". +Validate instead at **leg granularity** using ONLY the anonymous `_apis/build/...` timeline +(Hard-Rule 8) — a `succeeded` category leg means every test in that category **that actually +ran** passed on that platform. This is a strong bar **only if the target test genuinely +executes**: a job can go green while one specific test is skipped at runtime (`[Ignore]`, +`Assert.Ignore()`, `Assert.Inconclusive()`, a `Skip=`/conditional `[Fact]`, an `#if`-out, or +an early `return` before the asserts). So before trusting a green leg as proof the target +passed, **confirm from the PR's OWN diff that the fix does NOT skip, ignore, disable, or +short-circuit the target test** (it adds no `[Ignore]`/`[Explicit]`, `Assert.Ignore`/ +`Assert.Inconclusive`, `Skip=`, category-exclusion, `#if`-out, or early `return` around the +target). If the fix could cause the target to be runtime-skipped rather than genuinely pass, +leg-level green is NOT sufficient evidence — record `skipped: target test may be +runtime-skipped by this fix (diff adds ); leg-green insufficient, deferring to human +on PR #` and stop this gate WITHOUT marking ready. Otherwise (the target genuinely runs +and asserts, as a de-flake or product fix does) a green category leg cannot hide a +target-test failure, so treat it as authoritative. + +Map the target test to the CI leg(s) that run it. A leg name encodes platform + test category +(e.g. `Android UITests SafeAreaEdges,Shadow`, `iOS UITests SafeAreaEdges,Shadow`, `macOS +UITests SafeAreaEdges,Shadow`, `Windows UITests SafeAreaEdges,Shadow`). Determine the target +test's UI-test category — its `[Category(UITestCategories.X)]` in the test/HostApp source, +visible in the PR diff or the test file — to know which leg-name substring identifies its legs; +for a device test, the per-platform device-test legs. + +Using the SAME build-discovery as Step 4 (filter AzDO builds by `branchName=refs/pull//merge` +or `sourceVersion == C.headSha`), read each build's **timeline** on `C.headSha` for the +pipeline(s) that RUN the target test — `maui-pr` (def 302) for unit/integration tests, +`maui-pr-uitests` (def 313) for Appium UI tests, `maui-pr-devicetests` (def 314) for device +tests: + +```bash +ORG=dnceng-public; PROJ=public; BUILD= +# Anonymous + Hard-Rule-8-compliant. NEVER call _apis/test/... (it 302-redirects to sign-in). +# Each type=="Job" record is one platform leg: result (succeeded/failed/canceled/null), +# state (completed/inProgress/pending), name (encodes " UITests "): +curl -s "https://dev.azure.com/$ORG/$PROJ/_apis/build/builds/$BUILD/timeline?api-version=7.1" \ + | tee /tmp/gh-aw/agent/timeline_${BUILD}.json \ + | jq -r '.records[] | select(.type=="Job") | "\(.result)\t\(.state)\t\(.name)"' +``` + +(The prefetch already exposes per-leg status in `C.failedLegs` / the checks context, and +`gh pr checks ` lists the same per-leg rows with platform+category in the name — use +whichever is handy; the build timeline is the authoritative cross-check on the exact build.) + +Treat a target test as **VALIDATED-GREEN** only if, on `C.headSha`, the leg that runs its +category is green on **every platform it runs on** — a fix that repairs one platform must not +silently regress the same test's leg on another, so a green leg on the originally-red platform +alone is NOT enough. Across the target pipeline's platform legs (for a UI test: the Android, +iOS, Windows, and macOS/MacCatalyst legs running the target's category; for a device test: each +device-test platform), require ALL of: +- the target's category leg is `result == "succeeded"` **and** `state == "completed"` on + **every** platform that runs it, AND +- that leg is `failed` / `canceled` / aborted on **NO** platform, AND +- **no platform is left unverified.** For each platform family the target pipeline covers, that + platform's category leg must have CONCLUDED (`state == "completed"`) on `C.headSha`. If a + platform simply has no leg for the target's category, the test does not run there — that is + fine, not a gap. But if a platform's category leg has **not concluded** (`state` is + `inProgress` / pending, or an `/azp`-gated `maui-pr-uitests` / `maui-pr-devicetests` leg that + has not been kicked), the test's status on that platform is UNKNOWN → the fix is NOT yet + validated across platforms: record `skipped: target test green on but + not yet verified on (leg(s) pending / need /azp run) on PR #` + and stop this gate WITHOUT marking ready. + +A target test whose category leg never concluded on ANY platform (**not executed** anywhere — +e.g. its `/azp`-gated pipeline has not been kicked) is likewise NOT validated: record `skipped: +target test not yet executed on PR # ( not run — needs /azp run)` and stop +this gate WITHOUT marking ready. Do NOT overclaim — a green *sibling* leg (a different category +on the same platform) is not the target's leg, and a green leg on one platform is not a pass on +the others. + +**T3 — Mark ready + report.** If EVERY target test is VALIDATED-GREEN. The 🎯 comment, +the mark-ready, and the `p/0` label are THREE SEPARATE safe-outputs — the comment +existing does NOT prove the mark-ready took effect, so they are tracked independently: + +- **Comment idempotency (dup-suppress only — never gates the mark-ready).** We only reach + T3 while `C.isDraft == true` (the precondition stops an already-ready PR before here). So + if a prior bot comment for THIS head that carries the leading `🎯` anchor AND the + contiguous phrase `validated green on ` already exists (BOTH T3 variants — + `🎯 Target test validated green on ` and the build-only `🎯 Build validated green on + ` — contain that exact contiguous phrase, so this recognizes both; keying on + `target test` alone would instead miss the build-only comment and re-post it every sweep. + **Require the `🎯` anchor** so the match can NEVER be loosened to a gapped + `validated…green…on` form that would false-positive on the Step 3 `✅ … validated — the + fix's CI is green on ` surface comment and wrongly suppress the audit 🎯), the comment + landed on an earlier sweep but the **mark-ready did NOT take + effect** (the PR is still a draft) — set `SUPPRESS_COMMENT = true` (do not re-post the + duplicate 🎯 comment) but STILL complete the mark-ready + label below. Never treat the + comment's existence as "already marked ready" while the PR is still a draft. Record this + case as `re-marking PR # (🎯 present; mark-ready did not take on a prior sweep)`. +- **Atomicity budget pre-check (Hard-Rule 7).** Determine the outputs this gate will emit: + `mark_pull_request_as_ready_for_review` + `add_labels` always, plus `add_comment` unless + `SUPPRESS_COMMENT`. Verify a free per-run slot remains in EACH bucket you are about to + use. If ANY required bucket is exhausted, emit NONE of them — record `skipped: per-run + cap reached; deferring mark-ready PR # to next cycle` and stop this gate. Never mark a + PR ready (or label it) unless its 🎯 audit comment is GUARANTEED to exist for this exact + head SHA — either posted in this same sweep, or (in the `SUPPRESS_COMMENT` re-marking case) + already present from a prior sweep. The outputs you DO emit either all land or all defer + together; the audit trail must never be absent, but it is NOT re-posted when it already exists. +- **Dry-run gate (Step 0):** if `dry_run == "true"`, emit NOTHING — print the intended + readiness comment and "would mark ready + add p/0" to the run log, tally `dry-run: + would-mark-ready PR #`, and stop. +- Otherwise emit for THIS PR number ``: + 1. `add_comment` (ONLY if not `SUPPRESS_COMMENT`): `🎯 Target test validated green on + — passed on ALL platforms it runs on (, + buildId ). — not + caused by this fix."> Transitioning this PR from draft to ready for review and adding + `p/0`; a maintainer still reviews and merges.` (For a **build-only fix** — the T1 + whole-build fallback, no single target test — phrase the first clause as `🎯 Build + validated green on — the maui-pr build passed on ALL platforms (buildId + ).` instead of naming a test.) + 2. `mark_pull_request_as_ready_for_review` with `reason:` a one-line justification + naming the validated test(s) and ``. + 3. `add_labels` with `labels: ["p/0"]` for PR # — put the now-review-ready fix into + the team's p/0 priority queue so it is triaged, not lost in the draft backlog. (If + the PR somehow already carries `p/0`, this is a harmless no-op.) +- Record `marked-ready PR # (target green on ALL platforms on , + labeled p/0)` and stop. + #### Step 3.5.R — Maintainer change-request response (Track C) Reached from Step 3.5 gate 0. Goal: when an **eligible human reviewer** (Hard-Rule @@ -1383,7 +1669,19 @@ Filed by [`ci-status-fix`](https://github.com/dotnet/maui/blob/main/.github/work ### Step 8 — Per-issue tally + end-of-run summary -Per issue, append one outcome line to `/tmp/gh-aw/agent/coverage.txt`: +Per issue, append **one** outcome line to `/tmp/gh-aw/agent/coverage.txt` — the +terminal outcome for the cycle. When one cycle produces a chained pair — a PR gets a +same-cycle precursor line and **then** a Step 3.6 readiness line (`marked-ready` on the +draft→ready flip, or `already-ready` on a later steady-state sweep of an already-ready +PR) — record ONLY the terminal Step 3.6 readiness line: it supersedes ANY same-cycle +non-readiness precursor for that PR, so an aggregator keying on one-line-per-issue never +double-counts. The precursor may be a Step 3 green line (`surfaced-green` on the first ✅, +or `already-surfaced` when it already exists) OR — when an unrelated-flake red and a +target-green coincide — a Step 4 `annotated-flake` line; either way the readiness line is +terminal, and the superseded precursor's signal survives in the PR's 🎯/♻️ comment so no +information is lost. This covers the flip cycle (`surfaced-green` → `marked-ready`), the +post-flip steady state (`already-surfaced` → `already-ready`), and the flake-coincident +flip (`annotated-flake` → `marked-ready`): ``` # main attempt- @@ -1394,8 +1692,14 @@ Per issue, append one outcome line to `/tmp/gh-aw/agent/coverage.txt`: `advance-PR # attempt /10` (ADVANCE mode — new commit pushed to the existing PR), `surfaced-green PR #` (fix's own CI went green; commented for review, did not advance), `annotated-flake PR #` (red was unrelated flake; -commented, attempt NOT burned), `waiting PR #` (CI not settled yet), -`dry-run: would-`, `skipped: `. (The +commented, attempt NOT burned), `marked-ready PR #` (the specific fixed test +was confirmed green in the PR's own CI; draft flipped to ready for review), +`already-surfaced PR #` (the fix's ✅ green comment already existed this sweep; +re-post suppressed — superseded by the Step 3.6 readiness line when the PR also flips +ready that cycle), `already-ready PR #` (terminal steady state — the PR was flipped +ready on a prior cycle and remains green on an unchanged head; no action taken, +supersedes `already-surfaced`), `waiting PR #` (CI not settled yet), +`dry-run: would-`, `skipped: `. (The `needs-human-PR` outcome is reserved for the deferred hand-off — Step 6 currently records a skip instead, so it is not emitted.)
+ (head )` (re-surfacing the same green every tick is noise and burns the + per-run comment budget other PRs need); + - else `add_comment` on PR #: `✅ Attempt /10 validated — the fix's CI is + green on .` naming any `/azp`-gated legs (uitests def 313 / devicetests def + 314) that have not run and still need a maintainer `/azp run`; record `surfaced-green + PR # (attempt /10)`. (Do NOT assert "ready for human review" on a PR that + is still a draft — Step 3.6, not this comment, owns the draft→ready flip and posts its + own 🎯 announcement when it fires.) + Do NOT advance. Then — in ALL of the above cases — run **Step 3.6** (target-test + readiness gate) for this PR before stopping: its `/azp`-gated target legs may conclude + green on this SAME head SHA on a LATER sweep (an `/azp run` adds no commit), and Step + 3.6 is the ONLY place the draft→ready flip happens, so it MUST re-evaluate every sweep — + never `stop` here before it. *(This directly attacks the real bottleneck — no reviews — + so it is the highest-value outcome.)* 4. **Red → classify caused-by-fix vs unrelated-flake.** If `C.overallConclusion == "failure"`, analyze the PR's OWN failing build (NOT `main`): find the AzDO `maui-pr` build for this PR (filter builds by `branchName=refs/pull//merge`, @@ -723,18 +819,26 @@ Run these gates in order — the FIRST that fires decides this cycle's outcome: method and Step 4.7 flake buckets to `C.failedLegs`. - **Unrelated flake only** (every failed leg is known-flaky / infra / a pre-existing baseline red NOT introduced by the fix): do NOT burn an attempt. - **Idempotency first:** scan the PR's existing comments for a prior bot - `♻️ … unrelated flake … on ` note for THIS head SHA — if one already - exists, record `already-annotated-flake PR # (head )` and stop - WITHOUT re-commenting (re-annotating the same flake every 12h sweep is noise and - burns the per-run comment budget other PRs need). Otherwise resolve the attempt - number from `C.effectiveAttempt` (the authoritative max(marker, bot-commit) - counter), omitting the `Attempt /10:` prefix only if it is somehow - indeterminate. **Dry-run gate (Step 0):** if `dry_run == "true"`, do NOT emit any comment — instead print the intended `♻️` unrelated-flake body to the run log, tally `dry-run: would-annotate-flake PR #`, and stop. Otherwise `add_comment` on PR #: `♻️ Attempt /10: red is - unrelated flake on leg(s) () on ; the fix itself is not - implicated. A maintainer re-run (/azp run ) should clear it.` Record - `annotated-flake PR # (head )` and stop. *(Round 1: human re-runs; - Round 2: auto re-trigger.)* + **Comment idempotency + dry-run suppress the ♻️ comment ONLY — neither skips Step + 3.6.** Scan the PR's existing comments for a prior bot `♻️ … unrelated flake … on + ` note for THIS head SHA; if one exists, set `SKIP_FLAKE_COMMENT = true`. + Resolve the attempt number from `C.effectiveAttempt` (the authoritative max(marker, + bot-commit) counter), omitting the `Attempt /10:` prefix only if it is + somehow indeterminate. Then post the flake note UNLESS suppressed: + - if `dry_run == "true"`: do NOT emit — print the intended `♻️` unrelated-flake body + to the run log and tally `dry-run: would-annotate-flake PR #`; + - else if `SKIP_FLAKE_COMMENT`: do NOT re-post — record `already-annotated-flake PR + # (head )` (re-annotating the same flake every 12h sweep is noise and + burns the per-run comment budget other PRs need); + - else `add_comment` on PR #: `♻️ Attempt /10: red is unrelated flake on + leg(s) () on ; the fix itself is not implicated. A + maintainer re-run (/azp run ) should clear it.` record `annotated-flake + PR # (head )`. + Then — in ALL of the above cases — run **Step 3.6** (target-test readiness gate) for + this PR before stopping: its `/azp`-gated target legs may conclude green on this SAME + head SHA on a LATER sweep, and Step 3.6 is the ONLY place the draft→ready flip + happens, so it MUST re-evaluate every sweep — never `stop` here before it. *(Round 1: + human re-runs; Round 2: auto re-trigger.)* - **Caused by the fix** (a failed leg still matches the original target signature, or the fix introduced a NEW failure): advance an attempt. - **Attempt count.** `attempt = C.effectiveAttempt` — the authoritative @@ -752,6 +856,188 @@ Run these gates in order — the FIRST that fires decides this cycle's outcome: from every prior commit on this branch**, emitted via the Step 5.6 ADVANCE path (push onto the same PR). +#### Step 3.6 — Target-test verification & mark-ready gate + +Reached from the Step 3 **green-surface** branch, the Step 4 **unrelated-flake** branch +(AFTER that branch has posted its comment), and the Step 3.5 **gate-2 target-focused +fast-path** (2a — while unrelated legs are still pending). Purpose: confirm the SPECIFIC +test(s) this PR was opened to fix now PASS in the PR's own CI, and — only when they do +— transition the draft `[ci-fix]` PR to **ready for review** so a maintainer sees a +validated fix instead of a draft. This is the ONLY place the loop flips draft→ready; it +is a state transition, **never** an approval or a merge (a human still reviews and +merges). Overall red on *unrelated* legs must NOT gate readiness — we validate the fix, +not the base branch's flakiness. + +**Preconditions** (ALL must hold; otherwise record `skipped: readiness N/A PR # +()` and stop this gate): +- `C.isDraft == true` — if the PR is already ready-for-review, the draft→ready + transition is already done; do NOT re-mark **and do NOT re-apply `p/0`**. `p/0` is applied + exactly ONCE, atomically with the draft→ready flip (T3 — all-or-nothing with the 🎯 audit + comment + mark-ready), so an already-ready loop PR that is MISSING `p/0` has almost + certainly had it **removed by a maintainer** de-prioritizing the PR — a pure triage action + the human-engagement guard (which inspects comments/reviews/commits, NOT label changes) + cannot see. Re-adding `p/0` every sweep would fight that maintainer indefinitely, violating + the loop's "never override a human" contract. So the loop does NOT reconcile the label: + record `already-ready PR #` and stop this gate. (Trade-off: on the rare occasion a + transient API error drops `p/0` at flip time *after* the T3 atomic pre-check passed, it is + not auto-re-added — but the PR is still ready-for-review with its 🎯 audit comment, and + re-adding `p/0` is a trivial manual action; that is strictly preferable to steam-rolling a + maintainer's deliberate de-prioritization.) +- The PR is unmistakably THIS workflow's own: `[ci-fix] ` title prefix AND the + `agentic-workflows` label (mirrors the safe-output lock; the compensating scope + control if the v0.80.9 compiler drops the declarative required-*). +- You reached this gate from Step 3 (green), Step 4 (**unrelated-flake**), or the Step 3.5 + gate-2 **target-focused fast-path** (2a) — i.e. the fix is NOT implicated in any red that + has concluded. If Step 4 classified a red as **caused by the fix** (ADVANCE), do NOT run + this gate — advance the attempt instead. + +**T1 — Identify the target test(s).** From the `[ci-scan]` issue signature the fix +addresses (and the PR's own diff), extract the fully-qualified test method name(s) the +fix targets — e.g. `SafeAreaShouldWorkOnAllShellTabs`. For a de-flake it is the +de-flaked test; for a product fix it is the originally-failing test(s). If NO specific +test can be identified (e.g. a product build-break rather than a test failure), this is a +**build-only fix** — there is no single test to validate cross-platform, so validate at +**whole-build** granularity rather than stopping (Step 3 only *comments*; Step 3.6 is the +sole draft→ready flip, so a build-only fix is undrafted HERE or not at all). We only reach +this gate from Step 3 (green) / Step 4 (unrelated-flake) / gate-2a, so the fix is not +implicated in any concluded red; additionally require, via the T2 timeline method on +`C.headSha`, that the primary build pipeline `maui-pr` (def 302) has CONCLUDED with EVERY +one of its platform build legs `succeeded`/`completed` and NONE failed/canceled or still +pending — the build is green on **every** platform, not just the originally-broken one (the +cross-platform guard, applied to the build instead of a test). A build-only fix is undrafted +ONLY on the strength of the auto-running `maui-pr` (def 302) whole-build green, which the +loop CAN observe: if the PR's `[ci-scan]` issue signature or its own diff indicates the +ORIGINATING failure was in a `/azp`-gated pipeline (`maui-pr-uitests` def 313 or +`maui-pr-devicetests` def 314) rather than `maui-pr` (def 302), do NOT undraft on +`maui-pr`-green alone — those pipelines do not auto-run on this PR (GITHUB_TOKEN cannot +trigger them), so a green `maui-pr` build is NOT evidence the gated build break is fixed; +record `skipped: build-only fix PR # targets gated pipeline () not run — +deferring to human` and stop this gate. Otherwise, set `TARGET := "the maui-pr build +(build-only fix — no single target test)"` and proceed to **T3** to mark ready. If any `maui-pr` build leg is still unconcluded, record `skipped: build-only fix PR +# not yet whole-build green (leg(s) pending)` and stop this gate WITHOUT marking +ready (a green subset is not enough — a still-pending build leg could yet fail). + +**T2 — Verify the target test's platform legs are green (anonymous, leg-level).** Per-test +outcomes come from the AzDO **test-results** API (`_apis/test/...`), which is **NOT reachable +anonymously — Hard-Rule 8**: those endpoints 302-redirect to a sign-in page on this runner, so +a `curl` returns an HTML redirect (not JSON) and every target test would look "not executed". +Validate instead at **leg granularity** using ONLY the anonymous `_apis/build/...` timeline +(Hard-Rule 8) — a `succeeded` category leg means every test in that category **that actually +ran** passed on that platform. This is a strong bar **only if the target test genuinely +executes**: a job can go green while one specific test is skipped at runtime (`[Ignore]`, +`Assert.Ignore()`, `Assert.Inconclusive()`, a `Skip=`/conditional `[Fact]`, an `#if`-out, or +an early `return` before the asserts). So before trusting a green leg as proof the target +passed, **confirm from the PR's OWN diff that the fix does NOT skip, ignore, disable, or +short-circuit the target test** (it adds no `[Ignore]`/`[Explicit]`, `Assert.Ignore`/ +`Assert.Inconclusive`, `Skip=`, category-exclusion, `#if`-out, or early `return` around the +target). If the fix could cause the target to be runtime-skipped rather than genuinely pass, +leg-level green is NOT sufficient evidence — record `skipped: target test may be +runtime-skipped by this fix (diff adds ); leg-green insufficient, deferring to human +on PR #` and stop this gate WITHOUT marking ready. Otherwise (the target genuinely runs +and asserts, as a de-flake or product fix does) a green category leg cannot hide a +target-test failure, so treat it as authoritative. + +Map the target test to the CI leg(s) that run it. A leg name encodes platform + test category +(e.g. `Android UITests SafeAreaEdges,Shadow`, `iOS UITests SafeAreaEdges,Shadow`, `macOS +UITests SafeAreaEdges,Shadow`, `Windows UITests SafeAreaEdges,Shadow`). Determine the target +test's UI-test category — its `[Category(UITestCategories.X)]` in the test/HostApp source, +visible in the PR diff or the test file — to know which leg-name substring identifies its legs; +for a device test, the per-platform device-test legs. + +Using the SAME build-discovery as Step 4 (filter AzDO builds by `branchName=refs/pull//merge` +or `sourceVersion == C.headSha`), read each build's **timeline** on `C.headSha` for the +pipeline(s) that RUN the target test — `maui-pr` (def 302) for unit/integration tests, +`maui-pr-uitests` (def 313) for Appium UI tests, `maui-pr-devicetests` (def 314) for device +tests: + +```bash +ORG=dnceng-public; PROJ=public; BUILD= +# Anonymous + Hard-Rule-8-compliant. NEVER call _apis/test/... (it 302-redirects to sign-in). +# Each type=="Job" record is one platform leg: result (succeeded/failed/canceled/null), +# state (completed/inProgress/pending), name (encodes " UITests "): +curl -s "https://dev.azure.com/$ORG/$PROJ/_apis/build/builds/$BUILD/timeline?api-version=7.1" \ + | tee /tmp/gh-aw/agent/timeline_${BUILD}.json \ + | jq -r '.records[] | select(.type=="Job") | "\(.result)\t\(.state)\t\(.name)"' +``` + +(The prefetch already exposes per-leg status in `C.failedLegs` / the checks context, and +`gh pr checks ` lists the same per-leg rows with platform+category in the name — use +whichever is handy; the build timeline is the authoritative cross-check on the exact build.) + +Treat a target test as **VALIDATED-GREEN** only if, on `C.headSha`, the leg that runs its +category is green on **every platform it runs on** — a fix that repairs one platform must not +silently regress the same test's leg on another, so a green leg on the originally-red platform +alone is NOT enough. Across the target pipeline's platform legs (for a UI test: the Android, +iOS, Windows, and macOS/MacCatalyst legs running the target's category; for a device test: each +device-test platform), require ALL of: +- the target's category leg is `result == "succeeded"` **and** `state == "completed"` on + **every** platform that runs it, AND +- that leg is `failed` / `canceled` / aborted on **NO** platform, AND +- **no platform is left unverified.** For each platform family the target pipeline covers, that + platform's category leg must have CONCLUDED (`state == "completed"`) on `C.headSha`. If a + platform simply has no leg for the target's category, the test does not run there — that is + fine, not a gap. But if a platform's category leg has **not concluded** (`state` is + `inProgress` / pending, or an `/azp`-gated `maui-pr-uitests` / `maui-pr-devicetests` leg that + has not been kicked), the test's status on that platform is UNKNOWN → the fix is NOT yet + validated across platforms: record `skipped: target test green on but + not yet verified on (leg(s) pending / need /azp run) on PR #` + and stop this gate WITHOUT marking ready. + +A target test whose category leg never concluded on ANY platform (**not executed** anywhere — +e.g. its `/azp`-gated pipeline has not been kicked) is likewise NOT validated: record `skipped: +target test not yet executed on PR # ( not run — needs /azp run)` and stop +this gate WITHOUT marking ready. Do NOT overclaim — a green *sibling* leg (a different category +on the same platform) is not the target's leg, and a green leg on one platform is not a pass on +the others. + +**T3 — Mark ready + report.** If EVERY target test is VALIDATED-GREEN. The 🎯 comment, +the mark-ready, and the `p/0` label are THREE SEPARATE safe-outputs — the comment +existing does NOT prove the mark-ready took effect, so they are tracked independently: + +- **Comment idempotency (dup-suppress only — never gates the mark-ready).** We only reach + T3 while `C.isDraft == true` (the precondition stops an already-ready PR before here). So + if a prior bot comment for THIS head that carries the leading `🎯` anchor AND the + contiguous phrase `validated green on ` already exists (BOTH T3 variants — + `🎯 Target test validated green on ` and the build-only `🎯 Build validated green on + ` — contain that exact contiguous phrase, so this recognizes both; keying on + `target test` alone would instead miss the build-only comment and re-post it every sweep. + **Require the `🎯` anchor** so the match can NEVER be loosened to a gapped + `validated…green…on` form that would false-positive on the Step 3 `✅ … validated — the + fix's CI is green on ` surface comment and wrongly suppress the audit 🎯), the comment + landed on an earlier sweep but the **mark-ready did NOT take + effect** (the PR is still a draft) — set `SUPPRESS_COMMENT = true` (do not re-post the + duplicate 🎯 comment) but STILL complete the mark-ready + label below. Never treat the + comment's existence as "already marked ready" while the PR is still a draft. Record this + case as `re-marking PR # (🎯 present; mark-ready did not take on a prior sweep)`. +- **Atomicity budget pre-check (Hard-Rule 7).** Determine the outputs this gate will emit: + `mark_pull_request_as_ready_for_review` + `add_labels` always, plus `add_comment` unless + `SUPPRESS_COMMENT`. Verify a free per-run slot remains in EACH bucket you are about to + use. If ANY required bucket is exhausted, emit NONE of them — record `skipped: per-run + cap reached; deferring mark-ready PR # to next cycle` and stop this gate. Never mark a + PR ready (or label it) unless its 🎯 audit comment is GUARANTEED to exist for this exact + head SHA — either posted in this same sweep, or (in the `SUPPRESS_COMMENT` re-marking case) + already present from a prior sweep. The outputs you DO emit either all land or all defer + together; the audit trail must never be absent, but it is NOT re-posted when it already exists. +- **Dry-run gate (Step 0):** if `dry_run == "true"`, emit NOTHING — print the intended + readiness comment and "would mark ready + add p/0" to the run log, tally `dry-run: + would-mark-ready PR #`, and stop. +- Otherwise emit for THIS PR number ``: + 1. `add_comment` (ONLY if not `SUPPRESS_COMMENT`): `🎯 Target test validated green on + — passed on ALL platforms it runs on (, + buildId ). — not + caused by this fix."> Transitioning this PR from draft to ready for review and adding + `p/0`; a maintainer still reviews and merges.` (For a **build-only fix** — the T1 + whole-build fallback, no single target test — phrase the first clause as `🎯 Build + validated green on — the maui-pr build passed on ALL platforms (buildId + ).` instead of naming a test.) + 2. `mark_pull_request_as_ready_for_review` with `reason:` a one-line justification + naming the validated test(s) and ``. + 3. `add_labels` with `labels: ["p/0"]` for PR # — put the now-review-ready fix into + the team's p/0 priority queue so it is triaged, not lost in the draft backlog. (If + the PR somehow already carries `p/0`, this is a harmless no-op.) +- Record `marked-ready PR # (target green on ALL platforms on , + labeled p/0)` and stop. + #### Step 3.5.R — Maintainer change-request response (Track C) Reached from Step 3.5 gate 0. Goal: when an **eligible human reviewer** (Hard-Rule @@ -1383,7 +1669,19 @@ Filed by [`ci-status-fix`](https://github.com/dotnet/maui/blob/main/.github/work ### Step 8 — Per-issue tally + end-of-run summary -Per issue, append one outcome line to `/tmp/gh-aw/agent/coverage.txt`: +Per issue, append **one** outcome line to `/tmp/gh-aw/agent/coverage.txt` — the +terminal outcome for the cycle. When one cycle produces a chained pair — a PR gets a +same-cycle precursor line and **then** a Step 3.6 readiness line (`marked-ready` on the +draft→ready flip, or `already-ready` on a later steady-state sweep of an already-ready +PR) — record ONLY the terminal Step 3.6 readiness line: it supersedes ANY same-cycle +non-readiness precursor for that PR, so an aggregator keying on one-line-per-issue never +double-counts. The precursor may be a Step 3 green line (`surfaced-green` on the first ✅, +or `already-surfaced` when it already exists) OR — when an unrelated-flake red and a +target-green coincide — a Step 4 `annotated-flake` line; either way the readiness line is +terminal, and the superseded precursor's signal survives in the PR's 🎯/♻️ comment so no +information is lost. This covers the flip cycle (`surfaced-green` → `marked-ready`), the +post-flip steady state (`already-surfaced` → `already-ready`), and the flake-coincident +flip (`annotated-flake` → `marked-ready`): ``` # main attempt- @@ -1394,8 +1692,14 @@ Per issue, append one outcome line to `/tmp/gh-aw/agent/coverage.txt`: `advance-PR # attempt /10` (ADVANCE mode — new commit pushed to the existing PR), `surfaced-green PR #` (fix's own CI went green; commented for review, did not advance), `annotated-flake PR #` (red was unrelated flake; -commented, attempt NOT burned), `waiting PR #` (CI not settled yet), -`dry-run: would-`, `skipped: `. (The +commented, attempt NOT burned), `marked-ready PR #` (the specific fixed test +was confirmed green in the PR's own CI; draft flipped to ready for review), +`already-surfaced PR #` (the fix's ✅ green comment already existed this sweep; +re-post suppressed — superseded by the Step 3.6 readiness line when the PR also flips +ready that cycle), `already-ready PR #` (terminal steady state — the PR was flipped +ready on a prior cycle and remains green on an unchanged head; no action taken, +supersedes `already-surfaced`), `waiting PR #` (CI not settled yet), +`dry-run: would-`, `skipped: `. (The `needs-human-PR` outcome is reserved for the deferred hand-off — Step 6 currently records a skip instead, so it is not emitted.)
: `✅ Attempt /10 validated — the fix's CI is + green on .` naming any `/azp`-gated legs (uitests def 313 / devicetests def + 314) that have not run and still need a maintainer `/azp run`; record `surfaced-green + PR # (attempt /10)`. (Do NOT assert "ready for human review" on a PR that + is still a draft — Step 3.6, not this comment, owns the draft→ready flip and posts its + own 🎯 announcement when it fires.) + Do NOT advance. Then — in ALL of the above cases — run **Step 3.6** (target-test + readiness gate) for this PR before stopping: its `/azp`-gated target legs may conclude + green on this SAME head SHA on a LATER sweep (an `/azp run` adds no commit), and Step + 3.6 is the ONLY place the draft→ready flip happens, so it MUST re-evaluate every sweep — + never `stop` here before it. *(This directly attacks the real bottleneck — no reviews — + so it is the highest-value outcome.)* 4. **Red → classify caused-by-fix vs unrelated-flake.** If `C.overallConclusion == "failure"`, analyze the PR's OWN failing build (NOT `main`): find the AzDO `maui-pr` build for this PR (filter builds by `branchName=refs/pull//merge`, @@ -723,18 +819,26 @@ Run these gates in order — the FIRST that fires decides this cycle's outcome: method and Step 4.7 flake buckets to `C.failedLegs`. - **Unrelated flake only** (every failed leg is known-flaky / infra / a pre-existing baseline red NOT introduced by the fix): do NOT burn an attempt. - **Idempotency first:** scan the PR's existing comments for a prior bot - `♻️ … unrelated flake … on ` note for THIS head SHA — if one already - exists, record `already-annotated-flake PR # (head )` and stop - WITHOUT re-commenting (re-annotating the same flake every 12h sweep is noise and - burns the per-run comment budget other PRs need). Otherwise resolve the attempt - number from `C.effectiveAttempt` (the authoritative max(marker, bot-commit) - counter), omitting the `Attempt /10:` prefix only if it is somehow - indeterminate. **Dry-run gate (Step 0):** if `dry_run == "true"`, do NOT emit any comment — instead print the intended `♻️` unrelated-flake body to the run log, tally `dry-run: would-annotate-flake PR #`, and stop. Otherwise `add_comment` on PR #: `♻️ Attempt /10: red is - unrelated flake on leg(s) () on ; the fix itself is not - implicated. A maintainer re-run (/azp run ) should clear it.` Record - `annotated-flake PR # (head )` and stop. *(Round 1: human re-runs; - Round 2: auto re-trigger.)* + **Comment idempotency + dry-run suppress the ♻️ comment ONLY — neither skips Step + 3.6.** Scan the PR's existing comments for a prior bot `♻️ … unrelated flake … on + ` note for THIS head SHA; if one exists, set `SKIP_FLAKE_COMMENT = true`. + Resolve the attempt number from `C.effectiveAttempt` (the authoritative max(marker, + bot-commit) counter), omitting the `Attempt /10:` prefix only if it is + somehow indeterminate. Then post the flake note UNLESS suppressed: + - if `dry_run == "true"`: do NOT emit — print the intended `♻️` unrelated-flake body + to the run log and tally `dry-run: would-annotate-flake PR #`; + - else if `SKIP_FLAKE_COMMENT`: do NOT re-post — record `already-annotated-flake PR + # (head )` (re-annotating the same flake every 12h sweep is noise and + burns the per-run comment budget other PRs need); + - else `add_comment` on PR #: `♻️ Attempt /10: red is unrelated flake on + leg(s) () on ; the fix itself is not implicated. A + maintainer re-run (/azp run ) should clear it.` record `annotated-flake + PR # (head )`. + Then — in ALL of the above cases — run **Step 3.6** (target-test readiness gate) for + this PR before stopping: its `/azp`-gated target legs may conclude green on this SAME + head SHA on a LATER sweep, and Step 3.6 is the ONLY place the draft→ready flip + happens, so it MUST re-evaluate every sweep — never `stop` here before it. *(Round 1: + human re-runs; Round 2: auto re-trigger.)* - **Caused by the fix** (a failed leg still matches the original target signature, or the fix introduced a NEW failure): advance an attempt. - **Attempt count.** `attempt = C.effectiveAttempt` — the authoritative @@ -752,6 +856,188 @@ Run these gates in order — the FIRST that fires decides this cycle's outcome: from every prior commit on this branch**, emitted via the Step 5.6 ADVANCE path (push onto the same PR). +#### Step 3.6 — Target-test verification & mark-ready gate + +Reached from the Step 3 **green-surface** branch, the Step 4 **unrelated-flake** branch +(AFTER that branch has posted its comment), and the Step 3.5 **gate-2 target-focused +fast-path** (2a — while unrelated legs are still pending). Purpose: confirm the SPECIFIC +test(s) this PR was opened to fix now PASS in the PR's own CI, and — only when they do +— transition the draft `[ci-fix]` PR to **ready for review** so a maintainer sees a +validated fix instead of a draft. This is the ONLY place the loop flips draft→ready; it +is a state transition, **never** an approval or a merge (a human still reviews and +merges). Overall red on *unrelated* legs must NOT gate readiness — we validate the fix, +not the base branch's flakiness. + +**Preconditions** (ALL must hold; otherwise record `skipped: readiness N/A PR # +()` and stop this gate): +- `C.isDraft == true` — if the PR is already ready-for-review, the draft→ready + transition is already done; do NOT re-mark **and do NOT re-apply `p/0`**. `p/0` is applied + exactly ONCE, atomically with the draft→ready flip (T3 — all-or-nothing with the 🎯 audit + comment + mark-ready), so an already-ready loop PR that is MISSING `p/0` has almost + certainly had it **removed by a maintainer** de-prioritizing the PR — a pure triage action + the human-engagement guard (which inspects comments/reviews/commits, NOT label changes) + cannot see. Re-adding `p/0` every sweep would fight that maintainer indefinitely, violating + the loop's "never override a human" contract. So the loop does NOT reconcile the label: + record `already-ready PR #` and stop this gate. (Trade-off: on the rare occasion a + transient API error drops `p/0` at flip time *after* the T3 atomic pre-check passed, it is + not auto-re-added — but the PR is still ready-for-review with its 🎯 audit comment, and + re-adding `p/0` is a trivial manual action; that is strictly preferable to steam-rolling a + maintainer's deliberate de-prioritization.) +- The PR is unmistakably THIS workflow's own: `[ci-fix] ` title prefix AND the + `agentic-workflows` label (mirrors the safe-output lock; the compensating scope + control if the v0.80.9 compiler drops the declarative required-*). +- You reached this gate from Step 3 (green), Step 4 (**unrelated-flake**), or the Step 3.5 + gate-2 **target-focused fast-path** (2a) — i.e. the fix is NOT implicated in any red that + has concluded. If Step 4 classified a red as **caused by the fix** (ADVANCE), do NOT run + this gate — advance the attempt instead. + +**T1 — Identify the target test(s).** From the `[ci-scan]` issue signature the fix +addresses (and the PR's own diff), extract the fully-qualified test method name(s) the +fix targets — e.g. `SafeAreaShouldWorkOnAllShellTabs`. For a de-flake it is the +de-flaked test; for a product fix it is the originally-failing test(s). If NO specific +test can be identified (e.g. a product build-break rather than a test failure), this is a +**build-only fix** — there is no single test to validate cross-platform, so validate at +**whole-build** granularity rather than stopping (Step 3 only *comments*; Step 3.6 is the +sole draft→ready flip, so a build-only fix is undrafted HERE or not at all). We only reach +this gate from Step 3 (green) / Step 4 (unrelated-flake) / gate-2a, so the fix is not +implicated in any concluded red; additionally require, via the T2 timeline method on +`C.headSha`, that the primary build pipeline `maui-pr` (def 302) has CONCLUDED with EVERY +one of its platform build legs `succeeded`/`completed` and NONE failed/canceled or still +pending — the build is green on **every** platform, not just the originally-broken one (the +cross-platform guard, applied to the build instead of a test). A build-only fix is undrafted +ONLY on the strength of the auto-running `maui-pr` (def 302) whole-build green, which the +loop CAN observe: if the PR's `[ci-scan]` issue signature or its own diff indicates the +ORIGINATING failure was in a `/azp`-gated pipeline (`maui-pr-uitests` def 313 or +`maui-pr-devicetests` def 314) rather than `maui-pr` (def 302), do NOT undraft on +`maui-pr`-green alone — those pipelines do not auto-run on this PR (GITHUB_TOKEN cannot +trigger them), so a green `maui-pr` build is NOT evidence the gated build break is fixed; +record `skipped: build-only fix PR # targets gated pipeline () not run — +deferring to human` and stop this gate. Otherwise, set `TARGET := "the maui-pr build +(build-only fix — no single target test)"` and proceed to **T3** to mark ready. If any `maui-pr` build leg is still unconcluded, record `skipped: build-only fix PR +# not yet whole-build green (leg(s) pending)` and stop this gate WITHOUT marking +ready (a green subset is not enough — a still-pending build leg could yet fail). + +**T2 — Verify the target test's platform legs are green (anonymous, leg-level).** Per-test +outcomes come from the AzDO **test-results** API (`_apis/test/...`), which is **NOT reachable +anonymously — Hard-Rule 8**: those endpoints 302-redirect to a sign-in page on this runner, so +a `curl` returns an HTML redirect (not JSON) and every target test would look "not executed". +Validate instead at **leg granularity** using ONLY the anonymous `_apis/build/...` timeline +(Hard-Rule 8) — a `succeeded` category leg means every test in that category **that actually +ran** passed on that platform. This is a strong bar **only if the target test genuinely +executes**: a job can go green while one specific test is skipped at runtime (`[Ignore]`, +`Assert.Ignore()`, `Assert.Inconclusive()`, a `Skip=`/conditional `[Fact]`, an `#if`-out, or +an early `return` before the asserts). So before trusting a green leg as proof the target +passed, **confirm from the PR's OWN diff that the fix does NOT skip, ignore, disable, or +short-circuit the target test** (it adds no `[Ignore]`/`[Explicit]`, `Assert.Ignore`/ +`Assert.Inconclusive`, `Skip=`, category-exclusion, `#if`-out, or early `return` around the +target). If the fix could cause the target to be runtime-skipped rather than genuinely pass, +leg-level green is NOT sufficient evidence — record `skipped: target test may be +runtime-skipped by this fix (diff adds ); leg-green insufficient, deferring to human +on PR #` and stop this gate WITHOUT marking ready. Otherwise (the target genuinely runs +and asserts, as a de-flake or product fix does) a green category leg cannot hide a +target-test failure, so treat it as authoritative. + +Map the target test to the CI leg(s) that run it. A leg name encodes platform + test category +(e.g. `Android UITests SafeAreaEdges,Shadow`, `iOS UITests SafeAreaEdges,Shadow`, `macOS +UITests SafeAreaEdges,Shadow`, `Windows UITests SafeAreaEdges,Shadow`). Determine the target +test's UI-test category — its `[Category(UITestCategories.X)]` in the test/HostApp source, +visible in the PR diff or the test file — to know which leg-name substring identifies its legs; +for a device test, the per-platform device-test legs. + +Using the SAME build-discovery as Step 4 (filter AzDO builds by `branchName=refs/pull//merge` +or `sourceVersion == C.headSha`), read each build's **timeline** on `C.headSha` for the +pipeline(s) that RUN the target test — `maui-pr` (def 302) for unit/integration tests, +`maui-pr-uitests` (def 313) for Appium UI tests, `maui-pr-devicetests` (def 314) for device +tests: + +```bash +ORG=dnceng-public; PROJ=public; BUILD= +# Anonymous + Hard-Rule-8-compliant. NEVER call _apis/test/... (it 302-redirects to sign-in). +# Each type=="Job" record is one platform leg: result (succeeded/failed/canceled/null), +# state (completed/inProgress/pending), name (encodes " UITests "): +curl -s "https://dev.azure.com/$ORG/$PROJ/_apis/build/builds/$BUILD/timeline?api-version=7.1" \ + | tee /tmp/gh-aw/agent/timeline_${BUILD}.json \ + | jq -r '.records[] | select(.type=="Job") | "\(.result)\t\(.state)\t\(.name)"' +``` + +(The prefetch already exposes per-leg status in `C.failedLegs` / the checks context, and +`gh pr checks ` lists the same per-leg rows with platform+category in the name — use +whichever is handy; the build timeline is the authoritative cross-check on the exact build.) + +Treat a target test as **VALIDATED-GREEN** only if, on `C.headSha`, the leg that runs its +category is green on **every platform it runs on** — a fix that repairs one platform must not +silently regress the same test's leg on another, so a green leg on the originally-red platform +alone is NOT enough. Across the target pipeline's platform legs (for a UI test: the Android, +iOS, Windows, and macOS/MacCatalyst legs running the target's category; for a device test: each +device-test platform), require ALL of: +- the target's category leg is `result == "succeeded"` **and** `state == "completed"` on + **every** platform that runs it, AND +- that leg is `failed` / `canceled` / aborted on **NO** platform, AND +- **no platform is left unverified.** For each platform family the target pipeline covers, that + platform's category leg must have CONCLUDED (`state == "completed"`) on `C.headSha`. If a + platform simply has no leg for the target's category, the test does not run there — that is + fine, not a gap. But if a platform's category leg has **not concluded** (`state` is + `inProgress` / pending, or an `/azp`-gated `maui-pr-uitests` / `maui-pr-devicetests` leg that + has not been kicked), the test's status on that platform is UNKNOWN → the fix is NOT yet + validated across platforms: record `skipped: target test green on but + not yet verified on (leg(s) pending / need /azp run) on PR #` + and stop this gate WITHOUT marking ready. + +A target test whose category leg never concluded on ANY platform (**not executed** anywhere — +e.g. its `/azp`-gated pipeline has not been kicked) is likewise NOT validated: record `skipped: +target test not yet executed on PR # ( not run — needs /azp run)` and stop +this gate WITHOUT marking ready. Do NOT overclaim — a green *sibling* leg (a different category +on the same platform) is not the target's leg, and a green leg on one platform is not a pass on +the others. + +**T3 — Mark ready + report.** If EVERY target test is VALIDATED-GREEN. The 🎯 comment, +the mark-ready, and the `p/0` label are THREE SEPARATE safe-outputs — the comment +existing does NOT prove the mark-ready took effect, so they are tracked independently: + +- **Comment idempotency (dup-suppress only — never gates the mark-ready).** We only reach + T3 while `C.isDraft == true` (the precondition stops an already-ready PR before here). So + if a prior bot comment for THIS head that carries the leading `🎯` anchor AND the + contiguous phrase `validated green on ` already exists (BOTH T3 variants — + `🎯 Target test validated green on ` and the build-only `🎯 Build validated green on + ` — contain that exact contiguous phrase, so this recognizes both; keying on + `target test` alone would instead miss the build-only comment and re-post it every sweep. + **Require the `🎯` anchor** so the match can NEVER be loosened to a gapped + `validated…green…on` form that would false-positive on the Step 3 `✅ … validated — the + fix's CI is green on ` surface comment and wrongly suppress the audit 🎯), the comment + landed on an earlier sweep but the **mark-ready did NOT take + effect** (the PR is still a draft) — set `SUPPRESS_COMMENT = true` (do not re-post the + duplicate 🎯 comment) but STILL complete the mark-ready + label below. Never treat the + comment's existence as "already marked ready" while the PR is still a draft. Record this + case as `re-marking PR # (🎯 present; mark-ready did not take on a prior sweep)`. +- **Atomicity budget pre-check (Hard-Rule 7).** Determine the outputs this gate will emit: + `mark_pull_request_as_ready_for_review` + `add_labels` always, plus `add_comment` unless + `SUPPRESS_COMMENT`. Verify a free per-run slot remains in EACH bucket you are about to + use. If ANY required bucket is exhausted, emit NONE of them — record `skipped: per-run + cap reached; deferring mark-ready PR # to next cycle` and stop this gate. Never mark a + PR ready (or label it) unless its 🎯 audit comment is GUARANTEED to exist for this exact + head SHA — either posted in this same sweep, or (in the `SUPPRESS_COMMENT` re-marking case) + already present from a prior sweep. The outputs you DO emit either all land or all defer + together; the audit trail must never be absent, but it is NOT re-posted when it already exists. +- **Dry-run gate (Step 0):** if `dry_run == "true"`, emit NOTHING — print the intended + readiness comment and "would mark ready + add p/0" to the run log, tally `dry-run: + would-mark-ready PR #`, and stop. +- Otherwise emit for THIS PR number ``: + 1. `add_comment` (ONLY if not `SUPPRESS_COMMENT`): `🎯 Target test validated green on + — passed on ALL platforms it runs on (, + buildId ). — not + caused by this fix."> Transitioning this PR from draft to ready for review and adding + `p/0`; a maintainer still reviews and merges.` (For a **build-only fix** — the T1 + whole-build fallback, no single target test — phrase the first clause as `🎯 Build + validated green on — the maui-pr build passed on ALL platforms (buildId + ).` instead of naming a test.) + 2. `mark_pull_request_as_ready_for_review` with `reason:` a one-line justification + naming the validated test(s) and ``. + 3. `add_labels` with `labels: ["p/0"]` for PR # — put the now-review-ready fix into + the team's p/0 priority queue so it is triaged, not lost in the draft backlog. (If + the PR somehow already carries `p/0`, this is a harmless no-op.) +- Record `marked-ready PR # (target green on ALL platforms on , + labeled p/0)` and stop. + #### Step 3.5.R — Maintainer change-request response (Track C) Reached from Step 3.5 gate 0. Goal: when an **eligible human reviewer** (Hard-Rule @@ -1383,7 +1669,19 @@ Filed by [`ci-status-fix`](https://github.com/dotnet/maui/blob/main/.github/work ### Step 8 — Per-issue tally + end-of-run summary -Per issue, append one outcome line to `/tmp/gh-aw/agent/coverage.txt`: +Per issue, append **one** outcome line to `/tmp/gh-aw/agent/coverage.txt` — the +terminal outcome for the cycle. When one cycle produces a chained pair — a PR gets a +same-cycle precursor line and **then** a Step 3.6 readiness line (`marked-ready` on the +draft→ready flip, or `already-ready` on a later steady-state sweep of an already-ready +PR) — record ONLY the terminal Step 3.6 readiness line: it supersedes ANY same-cycle +non-readiness precursor for that PR, so an aggregator keying on one-line-per-issue never +double-counts. The precursor may be a Step 3 green line (`surfaced-green` on the first ✅, +or `already-surfaced` when it already exists) OR — when an unrelated-flake red and a +target-green coincide — a Step 4 `annotated-flake` line; either way the readiness line is +terminal, and the superseded precursor's signal survives in the PR's 🎯/♻️ comment so no +information is lost. This covers the flip cycle (`surfaced-green` → `marked-ready`), the +post-flip steady state (`already-surfaced` → `already-ready`), and the flake-coincident +flip (`annotated-flake` → `marked-ready`): ``` # main attempt- @@ -1394,8 +1692,14 @@ Per issue, append one outcome line to `/tmp/gh-aw/agent/coverage.txt`: `advance-PR # attempt /10` (ADVANCE mode — new commit pushed to the existing PR), `surfaced-green PR #` (fix's own CI went green; commented for review, did not advance), `annotated-flake PR #` (red was unrelated flake; -commented, attempt NOT burned), `waiting PR #` (CI not settled yet), -`dry-run: would-`, `skipped: `. (The +commented, attempt NOT burned), `marked-ready PR #` (the specific fixed test +was confirmed green in the PR's own CI; draft flipped to ready for review), +`already-surfaced PR #` (the fix's ✅ green comment already existed this sweep; +re-post suppressed — superseded by the Step 3.6 readiness line when the PR also flips +ready that cycle), `already-ready PR #` (terminal steady state — the PR was flipped +ready on a prior cycle and remains green on an unchanged head; no action taken, +supersedes `already-surfaced`), `waiting PR #` (CI not settled yet), +`dry-run: would-`, `skipped: `. (The `needs-human-PR` outcome is reserved for the deferred hand-off — Step 6 currently records a skip instead, so it is not emitted.)
(attempt /10)`. (Do NOT assert "ready for human review" on a PR that + is still a draft — Step 3.6, not this comment, owns the draft→ready flip and posts its + own 🎯 announcement when it fires.) + Do NOT advance. Then — in ALL of the above cases — run **Step 3.6** (target-test + readiness gate) for this PR before stopping: its `/azp`-gated target legs may conclude + green on this SAME head SHA on a LATER sweep (an `/azp run` adds no commit), and Step + 3.6 is the ONLY place the draft→ready flip happens, so it MUST re-evaluate every sweep — + never `stop` here before it. *(This directly attacks the real bottleneck — no reviews — + so it is the highest-value outcome.)* 4. **Red → classify caused-by-fix vs unrelated-flake.** If `C.overallConclusion == "failure"`, analyze the PR's OWN failing build (NOT `main`): find the AzDO `maui-pr` build for this PR (filter builds by `branchName=refs/pull//merge`, @@ -723,18 +819,26 @@ Run these gates in order — the FIRST that fires decides this cycle's outcome: method and Step 4.7 flake buckets to `C.failedLegs`. - **Unrelated flake only** (every failed leg is known-flaky / infra / a pre-existing baseline red NOT introduced by the fix): do NOT burn an attempt. - **Idempotency first:** scan the PR's existing comments for a prior bot - `♻️ … unrelated flake … on ` note for THIS head SHA — if one already - exists, record `already-annotated-flake PR # (head )` and stop - WITHOUT re-commenting (re-annotating the same flake every 12h sweep is noise and - burns the per-run comment budget other PRs need). Otherwise resolve the attempt - number from `C.effectiveAttempt` (the authoritative max(marker, bot-commit) - counter), omitting the `Attempt /10:` prefix only if it is somehow - indeterminate. **Dry-run gate (Step 0):** if `dry_run == "true"`, do NOT emit any comment — instead print the intended `♻️` unrelated-flake body to the run log, tally `dry-run: would-annotate-flake PR #`, and stop. Otherwise `add_comment` on PR #: `♻️ Attempt /10: red is - unrelated flake on leg(s) () on ; the fix itself is not - implicated. A maintainer re-run (/azp run ) should clear it.` Record - `annotated-flake PR # (head )` and stop. *(Round 1: human re-runs; - Round 2: auto re-trigger.)* + **Comment idempotency + dry-run suppress the ♻️ comment ONLY — neither skips Step + 3.6.** Scan the PR's existing comments for a prior bot `♻️ … unrelated flake … on + ` note for THIS head SHA; if one exists, set `SKIP_FLAKE_COMMENT = true`. + Resolve the attempt number from `C.effectiveAttempt` (the authoritative max(marker, + bot-commit) counter), omitting the `Attempt /10:` prefix only if it is + somehow indeterminate. Then post the flake note UNLESS suppressed: + - if `dry_run == "true"`: do NOT emit — print the intended `♻️` unrelated-flake body + to the run log and tally `dry-run: would-annotate-flake PR #`; + - else if `SKIP_FLAKE_COMMENT`: do NOT re-post — record `already-annotated-flake PR + # (head )` (re-annotating the same flake every 12h sweep is noise and + burns the per-run comment budget other PRs need); + - else `add_comment` on PR #: `♻️ Attempt /10: red is unrelated flake on + leg(s) () on ; the fix itself is not implicated. A + maintainer re-run (/azp run ) should clear it.` record `annotated-flake + PR # (head )`. + Then — in ALL of the above cases — run **Step 3.6** (target-test readiness gate) for + this PR before stopping: its `/azp`-gated target legs may conclude green on this SAME + head SHA on a LATER sweep, and Step 3.6 is the ONLY place the draft→ready flip + happens, so it MUST re-evaluate every sweep — never `stop` here before it. *(Round 1: + human re-runs; Round 2: auto re-trigger.)* - **Caused by the fix** (a failed leg still matches the original target signature, or the fix introduced a NEW failure): advance an attempt. - **Attempt count.** `attempt = C.effectiveAttempt` — the authoritative @@ -752,6 +856,188 @@ Run these gates in order — the FIRST that fires decides this cycle's outcome: from every prior commit on this branch**, emitted via the Step 5.6 ADVANCE path (push onto the same PR). +#### Step 3.6 — Target-test verification & mark-ready gate + +Reached from the Step 3 **green-surface** branch, the Step 4 **unrelated-flake** branch +(AFTER that branch has posted its comment), and the Step 3.5 **gate-2 target-focused +fast-path** (2a — while unrelated legs are still pending). Purpose: confirm the SPECIFIC +test(s) this PR was opened to fix now PASS in the PR's own CI, and — only when they do +— transition the draft `[ci-fix]` PR to **ready for review** so a maintainer sees a +validated fix instead of a draft. This is the ONLY place the loop flips draft→ready; it +is a state transition, **never** an approval or a merge (a human still reviews and +merges). Overall red on *unrelated* legs must NOT gate readiness — we validate the fix, +not the base branch's flakiness. + +**Preconditions** (ALL must hold; otherwise record `skipped: readiness N/A PR # +()` and stop this gate): +- `C.isDraft == true` — if the PR is already ready-for-review, the draft→ready + transition is already done; do NOT re-mark **and do NOT re-apply `p/0`**. `p/0` is applied + exactly ONCE, atomically with the draft→ready flip (T3 — all-or-nothing with the 🎯 audit + comment + mark-ready), so an already-ready loop PR that is MISSING `p/0` has almost + certainly had it **removed by a maintainer** de-prioritizing the PR — a pure triage action + the human-engagement guard (which inspects comments/reviews/commits, NOT label changes) + cannot see. Re-adding `p/0` every sweep would fight that maintainer indefinitely, violating + the loop's "never override a human" contract. So the loop does NOT reconcile the label: + record `already-ready PR #` and stop this gate. (Trade-off: on the rare occasion a + transient API error drops `p/0` at flip time *after* the T3 atomic pre-check passed, it is + not auto-re-added — but the PR is still ready-for-review with its 🎯 audit comment, and + re-adding `p/0` is a trivial manual action; that is strictly preferable to steam-rolling a + maintainer's deliberate de-prioritization.) +- The PR is unmistakably THIS workflow's own: `[ci-fix] ` title prefix AND the + `agentic-workflows` label (mirrors the safe-output lock; the compensating scope + control if the v0.80.9 compiler drops the declarative required-*). +- You reached this gate from Step 3 (green), Step 4 (**unrelated-flake**), or the Step 3.5 + gate-2 **target-focused fast-path** (2a) — i.e. the fix is NOT implicated in any red that + has concluded. If Step 4 classified a red as **caused by the fix** (ADVANCE), do NOT run + this gate — advance the attempt instead. + +**T1 — Identify the target test(s).** From the `[ci-scan]` issue signature the fix +addresses (and the PR's own diff), extract the fully-qualified test method name(s) the +fix targets — e.g. `SafeAreaShouldWorkOnAllShellTabs`. For a de-flake it is the +de-flaked test; for a product fix it is the originally-failing test(s). If NO specific +test can be identified (e.g. a product build-break rather than a test failure), this is a +**build-only fix** — there is no single test to validate cross-platform, so validate at +**whole-build** granularity rather than stopping (Step 3 only *comments*; Step 3.6 is the +sole draft→ready flip, so a build-only fix is undrafted HERE or not at all). We only reach +this gate from Step 3 (green) / Step 4 (unrelated-flake) / gate-2a, so the fix is not +implicated in any concluded red; additionally require, via the T2 timeline method on +`C.headSha`, that the primary build pipeline `maui-pr` (def 302) has CONCLUDED with EVERY +one of its platform build legs `succeeded`/`completed` and NONE failed/canceled or still +pending — the build is green on **every** platform, not just the originally-broken one (the +cross-platform guard, applied to the build instead of a test). A build-only fix is undrafted +ONLY on the strength of the auto-running `maui-pr` (def 302) whole-build green, which the +loop CAN observe: if the PR's `[ci-scan]` issue signature or its own diff indicates the +ORIGINATING failure was in a `/azp`-gated pipeline (`maui-pr-uitests` def 313 or +`maui-pr-devicetests` def 314) rather than `maui-pr` (def 302), do NOT undraft on +`maui-pr`-green alone — those pipelines do not auto-run on this PR (GITHUB_TOKEN cannot +trigger them), so a green `maui-pr` build is NOT evidence the gated build break is fixed; +record `skipped: build-only fix PR # targets gated pipeline () not run — +deferring to human` and stop this gate. Otherwise, set `TARGET := "the maui-pr build +(build-only fix — no single target test)"` and proceed to **T3** to mark ready. If any `maui-pr` build leg is still unconcluded, record `skipped: build-only fix PR +# not yet whole-build green (leg(s) pending)` and stop this gate WITHOUT marking +ready (a green subset is not enough — a still-pending build leg could yet fail). + +**T2 — Verify the target test's platform legs are green (anonymous, leg-level).** Per-test +outcomes come from the AzDO **test-results** API (`_apis/test/...`), which is **NOT reachable +anonymously — Hard-Rule 8**: those endpoints 302-redirect to a sign-in page on this runner, so +a `curl` returns an HTML redirect (not JSON) and every target test would look "not executed". +Validate instead at **leg granularity** using ONLY the anonymous `_apis/build/...` timeline +(Hard-Rule 8) — a `succeeded` category leg means every test in that category **that actually +ran** passed on that platform. This is a strong bar **only if the target test genuinely +executes**: a job can go green while one specific test is skipped at runtime (`[Ignore]`, +`Assert.Ignore()`, `Assert.Inconclusive()`, a `Skip=`/conditional `[Fact]`, an `#if`-out, or +an early `return` before the asserts). So before trusting a green leg as proof the target +passed, **confirm from the PR's OWN diff that the fix does NOT skip, ignore, disable, or +short-circuit the target test** (it adds no `[Ignore]`/`[Explicit]`, `Assert.Ignore`/ +`Assert.Inconclusive`, `Skip=`, category-exclusion, `#if`-out, or early `return` around the +target). If the fix could cause the target to be runtime-skipped rather than genuinely pass, +leg-level green is NOT sufficient evidence — record `skipped: target test may be +runtime-skipped by this fix (diff adds ); leg-green insufficient, deferring to human +on PR #` and stop this gate WITHOUT marking ready. Otherwise (the target genuinely runs +and asserts, as a de-flake or product fix does) a green category leg cannot hide a +target-test failure, so treat it as authoritative. + +Map the target test to the CI leg(s) that run it. A leg name encodes platform + test category +(e.g. `Android UITests SafeAreaEdges,Shadow`, `iOS UITests SafeAreaEdges,Shadow`, `macOS +UITests SafeAreaEdges,Shadow`, `Windows UITests SafeAreaEdges,Shadow`). Determine the target +test's UI-test category — its `[Category(UITestCategories.X)]` in the test/HostApp source, +visible in the PR diff or the test file — to know which leg-name substring identifies its legs; +for a device test, the per-platform device-test legs. + +Using the SAME build-discovery as Step 4 (filter AzDO builds by `branchName=refs/pull//merge` +or `sourceVersion == C.headSha`), read each build's **timeline** on `C.headSha` for the +pipeline(s) that RUN the target test — `maui-pr` (def 302) for unit/integration tests, +`maui-pr-uitests` (def 313) for Appium UI tests, `maui-pr-devicetests` (def 314) for device +tests: + +```bash +ORG=dnceng-public; PROJ=public; BUILD= +# Anonymous + Hard-Rule-8-compliant. NEVER call _apis/test/... (it 302-redirects to sign-in). +# Each type=="Job" record is one platform leg: result (succeeded/failed/canceled/null), +# state (completed/inProgress/pending), name (encodes " UITests "): +curl -s "https://dev.azure.com/$ORG/$PROJ/_apis/build/builds/$BUILD/timeline?api-version=7.1" \ + | tee /tmp/gh-aw/agent/timeline_${BUILD}.json \ + | jq -r '.records[] | select(.type=="Job") | "\(.result)\t\(.state)\t\(.name)"' +``` + +(The prefetch already exposes per-leg status in `C.failedLegs` / the checks context, and +`gh pr checks ` lists the same per-leg rows with platform+category in the name — use +whichever is handy; the build timeline is the authoritative cross-check on the exact build.) + +Treat a target test as **VALIDATED-GREEN** only if, on `C.headSha`, the leg that runs its +category is green on **every platform it runs on** — a fix that repairs one platform must not +silently regress the same test's leg on another, so a green leg on the originally-red platform +alone is NOT enough. Across the target pipeline's platform legs (for a UI test: the Android, +iOS, Windows, and macOS/MacCatalyst legs running the target's category; for a device test: each +device-test platform), require ALL of: +- the target's category leg is `result == "succeeded"` **and** `state == "completed"` on + **every** platform that runs it, AND +- that leg is `failed` / `canceled` / aborted on **NO** platform, AND +- **no platform is left unverified.** For each platform family the target pipeline covers, that + platform's category leg must have CONCLUDED (`state == "completed"`) on `C.headSha`. If a + platform simply has no leg for the target's category, the test does not run there — that is + fine, not a gap. But if a platform's category leg has **not concluded** (`state` is + `inProgress` / pending, or an `/azp`-gated `maui-pr-uitests` / `maui-pr-devicetests` leg that + has not been kicked), the test's status on that platform is UNKNOWN → the fix is NOT yet + validated across platforms: record `skipped: target test green on but + not yet verified on (leg(s) pending / need /azp run) on PR #` + and stop this gate WITHOUT marking ready. + +A target test whose category leg never concluded on ANY platform (**not executed** anywhere — +e.g. its `/azp`-gated pipeline has not been kicked) is likewise NOT validated: record `skipped: +target test not yet executed on PR # ( not run — needs /azp run)` and stop +this gate WITHOUT marking ready. Do NOT overclaim — a green *sibling* leg (a different category +on the same platform) is not the target's leg, and a green leg on one platform is not a pass on +the others. + +**T3 — Mark ready + report.** If EVERY target test is VALIDATED-GREEN. The 🎯 comment, +the mark-ready, and the `p/0` label are THREE SEPARATE safe-outputs — the comment +existing does NOT prove the mark-ready took effect, so they are tracked independently: + +- **Comment idempotency (dup-suppress only — never gates the mark-ready).** We only reach + T3 while `C.isDraft == true` (the precondition stops an already-ready PR before here). So + if a prior bot comment for THIS head that carries the leading `🎯` anchor AND the + contiguous phrase `validated green on ` already exists (BOTH T3 variants — + `🎯 Target test validated green on ` and the build-only `🎯 Build validated green on + ` — contain that exact contiguous phrase, so this recognizes both; keying on + `target test` alone would instead miss the build-only comment and re-post it every sweep. + **Require the `🎯` anchor** so the match can NEVER be loosened to a gapped + `validated…green…on` form that would false-positive on the Step 3 `✅ … validated — the + fix's CI is green on ` surface comment and wrongly suppress the audit 🎯), the comment + landed on an earlier sweep but the **mark-ready did NOT take + effect** (the PR is still a draft) — set `SUPPRESS_COMMENT = true` (do not re-post the + duplicate 🎯 comment) but STILL complete the mark-ready + label below. Never treat the + comment's existence as "already marked ready" while the PR is still a draft. Record this + case as `re-marking PR # (🎯 present; mark-ready did not take on a prior sweep)`. +- **Atomicity budget pre-check (Hard-Rule 7).** Determine the outputs this gate will emit: + `mark_pull_request_as_ready_for_review` + `add_labels` always, plus `add_comment` unless + `SUPPRESS_COMMENT`. Verify a free per-run slot remains in EACH bucket you are about to + use. If ANY required bucket is exhausted, emit NONE of them — record `skipped: per-run + cap reached; deferring mark-ready PR # to next cycle` and stop this gate. Never mark a + PR ready (or label it) unless its 🎯 audit comment is GUARANTEED to exist for this exact + head SHA — either posted in this same sweep, or (in the `SUPPRESS_COMMENT` re-marking case) + already present from a prior sweep. The outputs you DO emit either all land or all defer + together; the audit trail must never be absent, but it is NOT re-posted when it already exists. +- **Dry-run gate (Step 0):** if `dry_run == "true"`, emit NOTHING — print the intended + readiness comment and "would mark ready + add p/0" to the run log, tally `dry-run: + would-mark-ready PR #`, and stop. +- Otherwise emit for THIS PR number ``: + 1. `add_comment` (ONLY if not `SUPPRESS_COMMENT`): `🎯 Target test validated green on + — passed on ALL platforms it runs on (, + buildId ). — not + caused by this fix."> Transitioning this PR from draft to ready for review and adding + `p/0`; a maintainer still reviews and merges.` (For a **build-only fix** — the T1 + whole-build fallback, no single target test — phrase the first clause as `🎯 Build + validated green on — the maui-pr build passed on ALL platforms (buildId + ).` instead of naming a test.) + 2. `mark_pull_request_as_ready_for_review` with `reason:` a one-line justification + naming the validated test(s) and ``. + 3. `add_labels` with `labels: ["p/0"]` for PR # — put the now-review-ready fix into + the team's p/0 priority queue so it is triaged, not lost in the draft backlog. (If + the PR somehow already carries `p/0`, this is a harmless no-op.) +- Record `marked-ready PR # (target green on ALL platforms on , + labeled p/0)` and stop. + #### Step 3.5.R — Maintainer change-request response (Track C) Reached from Step 3.5 gate 0. Goal: when an **eligible human reviewer** (Hard-Rule @@ -1383,7 +1669,19 @@ Filed by [`ci-status-fix`](https://github.com/dotnet/maui/blob/main/.github/work ### Step 8 — Per-issue tally + end-of-run summary -Per issue, append one outcome line to `/tmp/gh-aw/agent/coverage.txt`: +Per issue, append **one** outcome line to `/tmp/gh-aw/agent/coverage.txt` — the +terminal outcome for the cycle. When one cycle produces a chained pair — a PR gets a +same-cycle precursor line and **then** a Step 3.6 readiness line (`marked-ready` on the +draft→ready flip, or `already-ready` on a later steady-state sweep of an already-ready +PR) — record ONLY the terminal Step 3.6 readiness line: it supersedes ANY same-cycle +non-readiness precursor for that PR, so an aggregator keying on one-line-per-issue never +double-counts. The precursor may be a Step 3 green line (`surfaced-green` on the first ✅, +or `already-surfaced` when it already exists) OR — when an unrelated-flake red and a +target-green coincide — a Step 4 `annotated-flake` line; either way the readiness line is +terminal, and the superseded precursor's signal survives in the PR's 🎯/♻️ comment so no +information is lost. This covers the flip cycle (`surfaced-green` → `marked-ready`), the +post-flip steady state (`already-surfaced` → `already-ready`), and the flake-coincident +flip (`annotated-flake` → `marked-ready`): ``` # main attempt- @@ -1394,8 +1692,14 @@ Per issue, append one outcome line to `/tmp/gh-aw/agent/coverage.txt`: `advance-PR # attempt /10` (ADVANCE mode — new commit pushed to the existing PR), `surfaced-green PR #` (fix's own CI went green; commented for review, did not advance), `annotated-flake PR #` (red was unrelated flake; -commented, attempt NOT burned), `waiting PR #` (CI not settled yet), -`dry-run: would-`, `skipped: `. (The +commented, attempt NOT burned), `marked-ready PR #` (the specific fixed test +was confirmed green in the PR's own CI; draft flipped to ready for review), +`already-surfaced PR #` (the fix's ✅ green comment already existed this sweep; +re-post suppressed — superseded by the Step 3.6 readiness line when the PR also flips +ready that cycle), `already-ready PR #` (terminal steady state — the PR was flipped +ready on a prior cycle and remains green on an unchanged head; no action taken, +supersedes `already-surfaced`), `waiting PR #` (CI not settled yet), +`dry-run: would-`, `skipped: `. (The `needs-human-PR` outcome is reserved for the deferred hand-off — Step 6 currently records a skip instead, so it is not emitted.)
/merge`, @@ -723,18 +819,26 @@ Run these gates in order — the FIRST that fires decides this cycle's outcome: method and Step 4.7 flake buckets to `C.failedLegs`. - **Unrelated flake only** (every failed leg is known-flaky / infra / a pre-existing baseline red NOT introduced by the fix): do NOT burn an attempt. - **Idempotency first:** scan the PR's existing comments for a prior bot - `♻️ … unrelated flake … on ` note for THIS head SHA — if one already - exists, record `already-annotated-flake PR # (head )` and stop - WITHOUT re-commenting (re-annotating the same flake every 12h sweep is noise and - burns the per-run comment budget other PRs need). Otherwise resolve the attempt - number from `C.effectiveAttempt` (the authoritative max(marker, bot-commit) - counter), omitting the `Attempt /10:` prefix only if it is somehow - indeterminate. **Dry-run gate (Step 0):** if `dry_run == "true"`, do NOT emit any comment — instead print the intended `♻️` unrelated-flake body to the run log, tally `dry-run: would-annotate-flake PR #`, and stop. Otherwise `add_comment` on PR #: `♻️ Attempt /10: red is - unrelated flake on leg(s) () on ; the fix itself is not - implicated. A maintainer re-run (/azp run ) should clear it.` Record - `annotated-flake PR # (head )` and stop. *(Round 1: human re-runs; - Round 2: auto re-trigger.)* + **Comment idempotency + dry-run suppress the ♻️ comment ONLY — neither skips Step + 3.6.** Scan the PR's existing comments for a prior bot `♻️ … unrelated flake … on + ` note for THIS head SHA; if one exists, set `SKIP_FLAKE_COMMENT = true`. + Resolve the attempt number from `C.effectiveAttempt` (the authoritative max(marker, + bot-commit) counter), omitting the `Attempt /10:` prefix only if it is + somehow indeterminate. Then post the flake note UNLESS suppressed: + - if `dry_run == "true"`: do NOT emit — print the intended `♻️` unrelated-flake body + to the run log and tally `dry-run: would-annotate-flake PR #`; + - else if `SKIP_FLAKE_COMMENT`: do NOT re-post — record `already-annotated-flake PR + # (head )` (re-annotating the same flake every 12h sweep is noise and + burns the per-run comment budget other PRs need); + - else `add_comment` on PR #: `♻️ Attempt /10: red is unrelated flake on + leg(s) () on ; the fix itself is not implicated. A + maintainer re-run (/azp run ) should clear it.` record `annotated-flake + PR # (head )`. + Then — in ALL of the above cases — run **Step 3.6** (target-test readiness gate) for + this PR before stopping: its `/azp`-gated target legs may conclude green on this SAME + head SHA on a LATER sweep, and Step 3.6 is the ONLY place the draft→ready flip + happens, so it MUST re-evaluate every sweep — never `stop` here before it. *(Round 1: + human re-runs; Round 2: auto re-trigger.)* - **Caused by the fix** (a failed leg still matches the original target signature, or the fix introduced a NEW failure): advance an attempt. - **Attempt count.** `attempt = C.effectiveAttempt` — the authoritative @@ -752,6 +856,188 @@ Run these gates in order — the FIRST that fires decides this cycle's outcome: from every prior commit on this branch**, emitted via the Step 5.6 ADVANCE path (push onto the same PR). +#### Step 3.6 — Target-test verification & mark-ready gate + +Reached from the Step 3 **green-surface** branch, the Step 4 **unrelated-flake** branch +(AFTER that branch has posted its comment), and the Step 3.5 **gate-2 target-focused +fast-path** (2a — while unrelated legs are still pending). Purpose: confirm the SPECIFIC +test(s) this PR was opened to fix now PASS in the PR's own CI, and — only when they do +— transition the draft `[ci-fix]` PR to **ready for review** so a maintainer sees a +validated fix instead of a draft. This is the ONLY place the loop flips draft→ready; it +is a state transition, **never** an approval or a merge (a human still reviews and +merges). Overall red on *unrelated* legs must NOT gate readiness — we validate the fix, +not the base branch's flakiness. + +**Preconditions** (ALL must hold; otherwise record `skipped: readiness N/A PR # +()` and stop this gate): +- `C.isDraft == true` — if the PR is already ready-for-review, the draft→ready + transition is already done; do NOT re-mark **and do NOT re-apply `p/0`**. `p/0` is applied + exactly ONCE, atomically with the draft→ready flip (T3 — all-or-nothing with the 🎯 audit + comment + mark-ready), so an already-ready loop PR that is MISSING `p/0` has almost + certainly had it **removed by a maintainer** de-prioritizing the PR — a pure triage action + the human-engagement guard (which inspects comments/reviews/commits, NOT label changes) + cannot see. Re-adding `p/0` every sweep would fight that maintainer indefinitely, violating + the loop's "never override a human" contract. So the loop does NOT reconcile the label: + record `already-ready PR #` and stop this gate. (Trade-off: on the rare occasion a + transient API error drops `p/0` at flip time *after* the T3 atomic pre-check passed, it is + not auto-re-added — but the PR is still ready-for-review with its 🎯 audit comment, and + re-adding `p/0` is a trivial manual action; that is strictly preferable to steam-rolling a + maintainer's deliberate de-prioritization.) +- The PR is unmistakably THIS workflow's own: `[ci-fix] ` title prefix AND the + `agentic-workflows` label (mirrors the safe-output lock; the compensating scope + control if the v0.80.9 compiler drops the declarative required-*). +- You reached this gate from Step 3 (green), Step 4 (**unrelated-flake**), or the Step 3.5 + gate-2 **target-focused fast-path** (2a) — i.e. the fix is NOT implicated in any red that + has concluded. If Step 4 classified a red as **caused by the fix** (ADVANCE), do NOT run + this gate — advance the attempt instead. + +**T1 — Identify the target test(s).** From the `[ci-scan]` issue signature the fix +addresses (and the PR's own diff), extract the fully-qualified test method name(s) the +fix targets — e.g. `SafeAreaShouldWorkOnAllShellTabs`. For a de-flake it is the +de-flaked test; for a product fix it is the originally-failing test(s). If NO specific +test can be identified (e.g. a product build-break rather than a test failure), this is a +**build-only fix** — there is no single test to validate cross-platform, so validate at +**whole-build** granularity rather than stopping (Step 3 only *comments*; Step 3.6 is the +sole draft→ready flip, so a build-only fix is undrafted HERE or not at all). We only reach +this gate from Step 3 (green) / Step 4 (unrelated-flake) / gate-2a, so the fix is not +implicated in any concluded red; additionally require, via the T2 timeline method on +`C.headSha`, that the primary build pipeline `maui-pr` (def 302) has CONCLUDED with EVERY +one of its platform build legs `succeeded`/`completed` and NONE failed/canceled or still +pending — the build is green on **every** platform, not just the originally-broken one (the +cross-platform guard, applied to the build instead of a test). A build-only fix is undrafted +ONLY on the strength of the auto-running `maui-pr` (def 302) whole-build green, which the +loop CAN observe: if the PR's `[ci-scan]` issue signature or its own diff indicates the +ORIGINATING failure was in a `/azp`-gated pipeline (`maui-pr-uitests` def 313 or +`maui-pr-devicetests` def 314) rather than `maui-pr` (def 302), do NOT undraft on +`maui-pr`-green alone — those pipelines do not auto-run on this PR (GITHUB_TOKEN cannot +trigger them), so a green `maui-pr` build is NOT evidence the gated build break is fixed; +record `skipped: build-only fix PR # targets gated pipeline () not run — +deferring to human` and stop this gate. Otherwise, set `TARGET := "the maui-pr build +(build-only fix — no single target test)"` and proceed to **T3** to mark ready. If any `maui-pr` build leg is still unconcluded, record `skipped: build-only fix PR +# not yet whole-build green (leg(s) pending)` and stop this gate WITHOUT marking +ready (a green subset is not enough — a still-pending build leg could yet fail). + +**T2 — Verify the target test's platform legs are green (anonymous, leg-level).** Per-test +outcomes come from the AzDO **test-results** API (`_apis/test/...`), which is **NOT reachable +anonymously — Hard-Rule 8**: those endpoints 302-redirect to a sign-in page on this runner, so +a `curl` returns an HTML redirect (not JSON) and every target test would look "not executed". +Validate instead at **leg granularity** using ONLY the anonymous `_apis/build/...` timeline +(Hard-Rule 8) — a `succeeded` category leg means every test in that category **that actually +ran** passed on that platform. This is a strong bar **only if the target test genuinely +executes**: a job can go green while one specific test is skipped at runtime (`[Ignore]`, +`Assert.Ignore()`, `Assert.Inconclusive()`, a `Skip=`/conditional `[Fact]`, an `#if`-out, or +an early `return` before the asserts). So before trusting a green leg as proof the target +passed, **confirm from the PR's OWN diff that the fix does NOT skip, ignore, disable, or +short-circuit the target test** (it adds no `[Ignore]`/`[Explicit]`, `Assert.Ignore`/ +`Assert.Inconclusive`, `Skip=`, category-exclusion, `#if`-out, or early `return` around the +target). If the fix could cause the target to be runtime-skipped rather than genuinely pass, +leg-level green is NOT sufficient evidence — record `skipped: target test may be +runtime-skipped by this fix (diff adds ); leg-green insufficient, deferring to human +on PR #` and stop this gate WITHOUT marking ready. Otherwise (the target genuinely runs +and asserts, as a de-flake or product fix does) a green category leg cannot hide a +target-test failure, so treat it as authoritative. + +Map the target test to the CI leg(s) that run it. A leg name encodes platform + test category +(e.g. `Android UITests SafeAreaEdges,Shadow`, `iOS UITests SafeAreaEdges,Shadow`, `macOS +UITests SafeAreaEdges,Shadow`, `Windows UITests SafeAreaEdges,Shadow`). Determine the target +test's UI-test category — its `[Category(UITestCategories.X)]` in the test/HostApp source, +visible in the PR diff or the test file — to know which leg-name substring identifies its legs; +for a device test, the per-platform device-test legs. + +Using the SAME build-discovery as Step 4 (filter AzDO builds by `branchName=refs/pull//merge` +or `sourceVersion == C.headSha`), read each build's **timeline** on `C.headSha` for the +pipeline(s) that RUN the target test — `maui-pr` (def 302) for unit/integration tests, +`maui-pr-uitests` (def 313) for Appium UI tests, `maui-pr-devicetests` (def 314) for device +tests: + +```bash +ORG=dnceng-public; PROJ=public; BUILD= +# Anonymous + Hard-Rule-8-compliant. NEVER call _apis/test/... (it 302-redirects to sign-in). +# Each type=="Job" record is one platform leg: result (succeeded/failed/canceled/null), +# state (completed/inProgress/pending), name (encodes " UITests "): +curl -s "https://dev.azure.com/$ORG/$PROJ/_apis/build/builds/$BUILD/timeline?api-version=7.1" \ + | tee /tmp/gh-aw/agent/timeline_${BUILD}.json \ + | jq -r '.records[] | select(.type=="Job") | "\(.result)\t\(.state)\t\(.name)"' +``` + +(The prefetch already exposes per-leg status in `C.failedLegs` / the checks context, and +`gh pr checks ` lists the same per-leg rows with platform+category in the name — use +whichever is handy; the build timeline is the authoritative cross-check on the exact build.) + +Treat a target test as **VALIDATED-GREEN** only if, on `C.headSha`, the leg that runs its +category is green on **every platform it runs on** — a fix that repairs one platform must not +silently regress the same test's leg on another, so a green leg on the originally-red platform +alone is NOT enough. Across the target pipeline's platform legs (for a UI test: the Android, +iOS, Windows, and macOS/MacCatalyst legs running the target's category; for a device test: each +device-test platform), require ALL of: +- the target's category leg is `result == "succeeded"` **and** `state == "completed"` on + **every** platform that runs it, AND +- that leg is `failed` / `canceled` / aborted on **NO** platform, AND +- **no platform is left unverified.** For each platform family the target pipeline covers, that + platform's category leg must have CONCLUDED (`state == "completed"`) on `C.headSha`. If a + platform simply has no leg for the target's category, the test does not run there — that is + fine, not a gap. But if a platform's category leg has **not concluded** (`state` is + `inProgress` / pending, or an `/azp`-gated `maui-pr-uitests` / `maui-pr-devicetests` leg that + has not been kicked), the test's status on that platform is UNKNOWN → the fix is NOT yet + validated across platforms: record `skipped: target test green on but + not yet verified on (leg(s) pending / need /azp run) on PR #` + and stop this gate WITHOUT marking ready. + +A target test whose category leg never concluded on ANY platform (**not executed** anywhere — +e.g. its `/azp`-gated pipeline has not been kicked) is likewise NOT validated: record `skipped: +target test not yet executed on PR # ( not run — needs /azp run)` and stop +this gate WITHOUT marking ready. Do NOT overclaim — a green *sibling* leg (a different category +on the same platform) is not the target's leg, and a green leg on one platform is not a pass on +the others. + +**T3 — Mark ready + report.** If EVERY target test is VALIDATED-GREEN. The 🎯 comment, +the mark-ready, and the `p/0` label are THREE SEPARATE safe-outputs — the comment +existing does NOT prove the mark-ready took effect, so they are tracked independently: + +- **Comment idempotency (dup-suppress only — never gates the mark-ready).** We only reach + T3 while `C.isDraft == true` (the precondition stops an already-ready PR before here). So + if a prior bot comment for THIS head that carries the leading `🎯` anchor AND the + contiguous phrase `validated green on ` already exists (BOTH T3 variants — + `🎯 Target test validated green on ` and the build-only `🎯 Build validated green on + ` — contain that exact contiguous phrase, so this recognizes both; keying on + `target test` alone would instead miss the build-only comment and re-post it every sweep. + **Require the `🎯` anchor** so the match can NEVER be loosened to a gapped + `validated…green…on` form that would false-positive on the Step 3 `✅ … validated — the + fix's CI is green on ` surface comment and wrongly suppress the audit 🎯), the comment + landed on an earlier sweep but the **mark-ready did NOT take + effect** (the PR is still a draft) — set `SUPPRESS_COMMENT = true` (do not re-post the + duplicate 🎯 comment) but STILL complete the mark-ready + label below. Never treat the + comment's existence as "already marked ready" while the PR is still a draft. Record this + case as `re-marking PR # (🎯 present; mark-ready did not take on a prior sweep)`. +- **Atomicity budget pre-check (Hard-Rule 7).** Determine the outputs this gate will emit: + `mark_pull_request_as_ready_for_review` + `add_labels` always, plus `add_comment` unless + `SUPPRESS_COMMENT`. Verify a free per-run slot remains in EACH bucket you are about to + use. If ANY required bucket is exhausted, emit NONE of them — record `skipped: per-run + cap reached; deferring mark-ready PR # to next cycle` and stop this gate. Never mark a + PR ready (or label it) unless its 🎯 audit comment is GUARANTEED to exist for this exact + head SHA — either posted in this same sweep, or (in the `SUPPRESS_COMMENT` re-marking case) + already present from a prior sweep. The outputs you DO emit either all land or all defer + together; the audit trail must never be absent, but it is NOT re-posted when it already exists. +- **Dry-run gate (Step 0):** if `dry_run == "true"`, emit NOTHING — print the intended + readiness comment and "would mark ready + add p/0" to the run log, tally `dry-run: + would-mark-ready PR #`, and stop. +- Otherwise emit for THIS PR number ``: + 1. `add_comment` (ONLY if not `SUPPRESS_COMMENT`): `🎯 Target test validated green on + — passed on ALL platforms it runs on (, + buildId ). — not + caused by this fix."> Transitioning this PR from draft to ready for review and adding + `p/0`; a maintainer still reviews and merges.` (For a **build-only fix** — the T1 + whole-build fallback, no single target test — phrase the first clause as `🎯 Build + validated green on — the maui-pr build passed on ALL platforms (buildId + ).` instead of naming a test.) + 2. `mark_pull_request_as_ready_for_review` with `reason:` a one-line justification + naming the validated test(s) and ``. + 3. `add_labels` with `labels: ["p/0"]` for PR # — put the now-review-ready fix into + the team's p/0 priority queue so it is triaged, not lost in the draft backlog. (If + the PR somehow already carries `p/0`, this is a harmless no-op.) +- Record `marked-ready PR # (target green on ALL platforms on , + labeled p/0)` and stop. + #### Step 3.5.R — Maintainer change-request response (Track C) Reached from Step 3.5 gate 0. Goal: when an **eligible human reviewer** (Hard-Rule @@ -1383,7 +1669,19 @@ Filed by [`ci-status-fix`](https://github.com/dotnet/maui/blob/main/.github/work ### Step 8 — Per-issue tally + end-of-run summary -Per issue, append one outcome line to `/tmp/gh-aw/agent/coverage.txt`: +Per issue, append **one** outcome line to `/tmp/gh-aw/agent/coverage.txt` — the +terminal outcome for the cycle. When one cycle produces a chained pair — a PR gets a +same-cycle precursor line and **then** a Step 3.6 readiness line (`marked-ready` on the +draft→ready flip, or `already-ready` on a later steady-state sweep of an already-ready +PR) — record ONLY the terminal Step 3.6 readiness line: it supersedes ANY same-cycle +non-readiness precursor for that PR, so an aggregator keying on one-line-per-issue never +double-counts. The precursor may be a Step 3 green line (`surfaced-green` on the first ✅, +or `already-surfaced` when it already exists) OR — when an unrelated-flake red and a +target-green coincide — a Step 4 `annotated-flake` line; either way the readiness line is +terminal, and the superseded precursor's signal survives in the PR's 🎯/♻️ comment so no +information is lost. This covers the flip cycle (`surfaced-green` → `marked-ready`), the +post-flip steady state (`already-surfaced` → `already-ready`), and the flake-coincident +flip (`annotated-flake` → `marked-ready`): ``` # main attempt- @@ -1394,8 +1692,14 @@ Per issue, append one outcome line to `/tmp/gh-aw/agent/coverage.txt`: `advance-PR # attempt /10` (ADVANCE mode — new commit pushed to the existing PR), `surfaced-green PR #` (fix's own CI went green; commented for review, did not advance), `annotated-flake PR #` (red was unrelated flake; -commented, attempt NOT burned), `waiting PR #` (CI not settled yet), -`dry-run: would-`, `skipped: `. (The +commented, attempt NOT burned), `marked-ready PR #` (the specific fixed test +was confirmed green in the PR's own CI; draft flipped to ready for review), +`already-surfaced PR #` (the fix's ✅ green comment already existed this sweep; +re-post suppressed — superseded by the Step 3.6 readiness line when the PR also flips +ready that cycle), `already-ready PR #` (terminal steady state — the PR was flipped +ready on a prior cycle and remains green on an unchanged head; no action taken, +supersedes `already-surfaced`), `waiting PR #` (CI not settled yet), +`dry-run: would-`, `skipped: `. (The `needs-human-PR` outcome is reserved for the deferred hand-off — Step 6 currently records a skip instead, so it is not emitted.)
(head )` and stop - WITHOUT re-commenting (re-annotating the same flake every 12h sweep is noise and - burns the per-run comment budget other PRs need). Otherwise resolve the attempt - number from `C.effectiveAttempt` (the authoritative max(marker, bot-commit) - counter), omitting the `Attempt /10:` prefix only if it is somehow - indeterminate. **Dry-run gate (Step 0):** if `dry_run == "true"`, do NOT emit any comment — instead print the intended `♻️` unrelated-flake body to the run log, tally `dry-run: would-annotate-flake PR #`, and stop. Otherwise `add_comment` on PR #: `♻️ Attempt /10: red is - unrelated flake on leg(s) () on ; the fix itself is not - implicated. A maintainer re-run (/azp run ) should clear it.` Record - `annotated-flake PR # (head )` and stop. *(Round 1: human re-runs; - Round 2: auto re-trigger.)* + **Comment idempotency + dry-run suppress the ♻️ comment ONLY — neither skips Step + 3.6.** Scan the PR's existing comments for a prior bot `♻️ … unrelated flake … on + ` note for THIS head SHA; if one exists, set `SKIP_FLAKE_COMMENT = true`. + Resolve the attempt number from `C.effectiveAttempt` (the authoritative max(marker, + bot-commit) counter), omitting the `Attempt /10:` prefix only if it is + somehow indeterminate. Then post the flake note UNLESS suppressed: + - if `dry_run == "true"`: do NOT emit — print the intended `♻️` unrelated-flake body + to the run log and tally `dry-run: would-annotate-flake PR #`; + - else if `SKIP_FLAKE_COMMENT`: do NOT re-post — record `already-annotated-flake PR + # (head )` (re-annotating the same flake every 12h sweep is noise and + burns the per-run comment budget other PRs need); + - else `add_comment` on PR #: `♻️ Attempt /10: red is unrelated flake on + leg(s) () on ; the fix itself is not implicated. A + maintainer re-run (/azp run ) should clear it.` record `annotated-flake + PR # (head )`. + Then — in ALL of the above cases — run **Step 3.6** (target-test readiness gate) for + this PR before stopping: its `/azp`-gated target legs may conclude green on this SAME + head SHA on a LATER sweep, and Step 3.6 is the ONLY place the draft→ready flip + happens, so it MUST re-evaluate every sweep — never `stop` here before it. *(Round 1: + human re-runs; Round 2: auto re-trigger.)* - **Caused by the fix** (a failed leg still matches the original target signature, or the fix introduced a NEW failure): advance an attempt. - **Attempt count.** `attempt = C.effectiveAttempt` — the authoritative @@ -752,6 +856,188 @@ Run these gates in order — the FIRST that fires decides this cycle's outcome: from every prior commit on this branch**, emitted via the Step 5.6 ADVANCE path (push onto the same PR). +#### Step 3.6 — Target-test verification & mark-ready gate + +Reached from the Step 3 **green-surface** branch, the Step 4 **unrelated-flake** branch +(AFTER that branch has posted its comment), and the Step 3.5 **gate-2 target-focused +fast-path** (2a — while unrelated legs are still pending). Purpose: confirm the SPECIFIC +test(s) this PR was opened to fix now PASS in the PR's own CI, and — only when they do +— transition the draft `[ci-fix]` PR to **ready for review** so a maintainer sees a +validated fix instead of a draft. This is the ONLY place the loop flips draft→ready; it +is a state transition, **never** an approval or a merge (a human still reviews and +merges). Overall red on *unrelated* legs must NOT gate readiness — we validate the fix, +not the base branch's flakiness. + +**Preconditions** (ALL must hold; otherwise record `skipped: readiness N/A PR # +()` and stop this gate): +- `C.isDraft == true` — if the PR is already ready-for-review, the draft→ready + transition is already done; do NOT re-mark **and do NOT re-apply `p/0`**. `p/0` is applied + exactly ONCE, atomically with the draft→ready flip (T3 — all-or-nothing with the 🎯 audit + comment + mark-ready), so an already-ready loop PR that is MISSING `p/0` has almost + certainly had it **removed by a maintainer** de-prioritizing the PR — a pure triage action + the human-engagement guard (which inspects comments/reviews/commits, NOT label changes) + cannot see. Re-adding `p/0` every sweep would fight that maintainer indefinitely, violating + the loop's "never override a human" contract. So the loop does NOT reconcile the label: + record `already-ready PR #` and stop this gate. (Trade-off: on the rare occasion a + transient API error drops `p/0` at flip time *after* the T3 atomic pre-check passed, it is + not auto-re-added — but the PR is still ready-for-review with its 🎯 audit comment, and + re-adding `p/0` is a trivial manual action; that is strictly preferable to steam-rolling a + maintainer's deliberate de-prioritization.) +- The PR is unmistakably THIS workflow's own: `[ci-fix] ` title prefix AND the + `agentic-workflows` label (mirrors the safe-output lock; the compensating scope + control if the v0.80.9 compiler drops the declarative required-*). +- You reached this gate from Step 3 (green), Step 4 (**unrelated-flake**), or the Step 3.5 + gate-2 **target-focused fast-path** (2a) — i.e. the fix is NOT implicated in any red that + has concluded. If Step 4 classified a red as **caused by the fix** (ADVANCE), do NOT run + this gate — advance the attempt instead. + +**T1 — Identify the target test(s).** From the `[ci-scan]` issue signature the fix +addresses (and the PR's own diff), extract the fully-qualified test method name(s) the +fix targets — e.g. `SafeAreaShouldWorkOnAllShellTabs`. For a de-flake it is the +de-flaked test; for a product fix it is the originally-failing test(s). If NO specific +test can be identified (e.g. a product build-break rather than a test failure), this is a +**build-only fix** — there is no single test to validate cross-platform, so validate at +**whole-build** granularity rather than stopping (Step 3 only *comments*; Step 3.6 is the +sole draft→ready flip, so a build-only fix is undrafted HERE or not at all). We only reach +this gate from Step 3 (green) / Step 4 (unrelated-flake) / gate-2a, so the fix is not +implicated in any concluded red; additionally require, via the T2 timeline method on +`C.headSha`, that the primary build pipeline `maui-pr` (def 302) has CONCLUDED with EVERY +one of its platform build legs `succeeded`/`completed` and NONE failed/canceled or still +pending — the build is green on **every** platform, not just the originally-broken one (the +cross-platform guard, applied to the build instead of a test). A build-only fix is undrafted +ONLY on the strength of the auto-running `maui-pr` (def 302) whole-build green, which the +loop CAN observe: if the PR's `[ci-scan]` issue signature or its own diff indicates the +ORIGINATING failure was in a `/azp`-gated pipeline (`maui-pr-uitests` def 313 or +`maui-pr-devicetests` def 314) rather than `maui-pr` (def 302), do NOT undraft on +`maui-pr`-green alone — those pipelines do not auto-run on this PR (GITHUB_TOKEN cannot +trigger them), so a green `maui-pr` build is NOT evidence the gated build break is fixed; +record `skipped: build-only fix PR # targets gated pipeline () not run — +deferring to human` and stop this gate. Otherwise, set `TARGET := "the maui-pr build +(build-only fix — no single target test)"` and proceed to **T3** to mark ready. If any `maui-pr` build leg is still unconcluded, record `skipped: build-only fix PR +# not yet whole-build green (leg(s) pending)` and stop this gate WITHOUT marking +ready (a green subset is not enough — a still-pending build leg could yet fail). + +**T2 — Verify the target test's platform legs are green (anonymous, leg-level).** Per-test +outcomes come from the AzDO **test-results** API (`_apis/test/...`), which is **NOT reachable +anonymously — Hard-Rule 8**: those endpoints 302-redirect to a sign-in page on this runner, so +a `curl` returns an HTML redirect (not JSON) and every target test would look "not executed". +Validate instead at **leg granularity** using ONLY the anonymous `_apis/build/...` timeline +(Hard-Rule 8) — a `succeeded` category leg means every test in that category **that actually +ran** passed on that platform. This is a strong bar **only if the target test genuinely +executes**: a job can go green while one specific test is skipped at runtime (`[Ignore]`, +`Assert.Ignore()`, `Assert.Inconclusive()`, a `Skip=`/conditional `[Fact]`, an `#if`-out, or +an early `return` before the asserts). So before trusting a green leg as proof the target +passed, **confirm from the PR's OWN diff that the fix does NOT skip, ignore, disable, or +short-circuit the target test** (it adds no `[Ignore]`/`[Explicit]`, `Assert.Ignore`/ +`Assert.Inconclusive`, `Skip=`, category-exclusion, `#if`-out, or early `return` around the +target). If the fix could cause the target to be runtime-skipped rather than genuinely pass, +leg-level green is NOT sufficient evidence — record `skipped: target test may be +runtime-skipped by this fix (diff adds ); leg-green insufficient, deferring to human +on PR #` and stop this gate WITHOUT marking ready. Otherwise (the target genuinely runs +and asserts, as a de-flake or product fix does) a green category leg cannot hide a +target-test failure, so treat it as authoritative. + +Map the target test to the CI leg(s) that run it. A leg name encodes platform + test category +(e.g. `Android UITests SafeAreaEdges,Shadow`, `iOS UITests SafeAreaEdges,Shadow`, `macOS +UITests SafeAreaEdges,Shadow`, `Windows UITests SafeAreaEdges,Shadow`). Determine the target +test's UI-test category — its `[Category(UITestCategories.X)]` in the test/HostApp source, +visible in the PR diff or the test file — to know which leg-name substring identifies its legs; +for a device test, the per-platform device-test legs. + +Using the SAME build-discovery as Step 4 (filter AzDO builds by `branchName=refs/pull//merge` +or `sourceVersion == C.headSha`), read each build's **timeline** on `C.headSha` for the +pipeline(s) that RUN the target test — `maui-pr` (def 302) for unit/integration tests, +`maui-pr-uitests` (def 313) for Appium UI tests, `maui-pr-devicetests` (def 314) for device +tests: + +```bash +ORG=dnceng-public; PROJ=public; BUILD= +# Anonymous + Hard-Rule-8-compliant. NEVER call _apis/test/... (it 302-redirects to sign-in). +# Each type=="Job" record is one platform leg: result (succeeded/failed/canceled/null), +# state (completed/inProgress/pending), name (encodes " UITests "): +curl -s "https://dev.azure.com/$ORG/$PROJ/_apis/build/builds/$BUILD/timeline?api-version=7.1" \ + | tee /tmp/gh-aw/agent/timeline_${BUILD}.json \ + | jq -r '.records[] | select(.type=="Job") | "\(.result)\t\(.state)\t\(.name)"' +``` + +(The prefetch already exposes per-leg status in `C.failedLegs` / the checks context, and +`gh pr checks ` lists the same per-leg rows with platform+category in the name — use +whichever is handy; the build timeline is the authoritative cross-check on the exact build.) + +Treat a target test as **VALIDATED-GREEN** only if, on `C.headSha`, the leg that runs its +category is green on **every platform it runs on** — a fix that repairs one platform must not +silently regress the same test's leg on another, so a green leg on the originally-red platform +alone is NOT enough. Across the target pipeline's platform legs (for a UI test: the Android, +iOS, Windows, and macOS/MacCatalyst legs running the target's category; for a device test: each +device-test platform), require ALL of: +- the target's category leg is `result == "succeeded"` **and** `state == "completed"` on + **every** platform that runs it, AND +- that leg is `failed` / `canceled` / aborted on **NO** platform, AND +- **no platform is left unverified.** For each platform family the target pipeline covers, that + platform's category leg must have CONCLUDED (`state == "completed"`) on `C.headSha`. If a + platform simply has no leg for the target's category, the test does not run there — that is + fine, not a gap. But if a platform's category leg has **not concluded** (`state` is + `inProgress` / pending, or an `/azp`-gated `maui-pr-uitests` / `maui-pr-devicetests` leg that + has not been kicked), the test's status on that platform is UNKNOWN → the fix is NOT yet + validated across platforms: record `skipped: target test green on but + not yet verified on (leg(s) pending / need /azp run) on PR #` + and stop this gate WITHOUT marking ready. + +A target test whose category leg never concluded on ANY platform (**not executed** anywhere — +e.g. its `/azp`-gated pipeline has not been kicked) is likewise NOT validated: record `skipped: +target test not yet executed on PR # ( not run — needs /azp run)` and stop +this gate WITHOUT marking ready. Do NOT overclaim — a green *sibling* leg (a different category +on the same platform) is not the target's leg, and a green leg on one platform is not a pass on +the others. + +**T3 — Mark ready + report.** If EVERY target test is VALIDATED-GREEN. The 🎯 comment, +the mark-ready, and the `p/0` label are THREE SEPARATE safe-outputs — the comment +existing does NOT prove the mark-ready took effect, so they are tracked independently: + +- **Comment idempotency (dup-suppress only — never gates the mark-ready).** We only reach + T3 while `C.isDraft == true` (the precondition stops an already-ready PR before here). So + if a prior bot comment for THIS head that carries the leading `🎯` anchor AND the + contiguous phrase `validated green on ` already exists (BOTH T3 variants — + `🎯 Target test validated green on ` and the build-only `🎯 Build validated green on + ` — contain that exact contiguous phrase, so this recognizes both; keying on + `target test` alone would instead miss the build-only comment and re-post it every sweep. + **Require the `🎯` anchor** so the match can NEVER be loosened to a gapped + `validated…green…on` form that would false-positive on the Step 3 `✅ … validated — the + fix's CI is green on ` surface comment and wrongly suppress the audit 🎯), the comment + landed on an earlier sweep but the **mark-ready did NOT take + effect** (the PR is still a draft) — set `SUPPRESS_COMMENT = true` (do not re-post the + duplicate 🎯 comment) but STILL complete the mark-ready + label below. Never treat the + comment's existence as "already marked ready" while the PR is still a draft. Record this + case as `re-marking PR # (🎯 present; mark-ready did not take on a prior sweep)`. +- **Atomicity budget pre-check (Hard-Rule 7).** Determine the outputs this gate will emit: + `mark_pull_request_as_ready_for_review` + `add_labels` always, plus `add_comment` unless + `SUPPRESS_COMMENT`. Verify a free per-run slot remains in EACH bucket you are about to + use. If ANY required bucket is exhausted, emit NONE of them — record `skipped: per-run + cap reached; deferring mark-ready PR # to next cycle` and stop this gate. Never mark a + PR ready (or label it) unless its 🎯 audit comment is GUARANTEED to exist for this exact + head SHA — either posted in this same sweep, or (in the `SUPPRESS_COMMENT` re-marking case) + already present from a prior sweep. The outputs you DO emit either all land or all defer + together; the audit trail must never be absent, but it is NOT re-posted when it already exists. +- **Dry-run gate (Step 0):** if `dry_run == "true"`, emit NOTHING — print the intended + readiness comment and "would mark ready + add p/0" to the run log, tally `dry-run: + would-mark-ready PR #`, and stop. +- Otherwise emit for THIS PR number ``: + 1. `add_comment` (ONLY if not `SUPPRESS_COMMENT`): `🎯 Target test validated green on + — passed on ALL platforms it runs on (, + buildId ). — not + caused by this fix."> Transitioning this PR from draft to ready for review and adding + `p/0`; a maintainer still reviews and merges.` (For a **build-only fix** — the T1 + whole-build fallback, no single target test — phrase the first clause as `🎯 Build + validated green on — the maui-pr build passed on ALL platforms (buildId + ).` instead of naming a test.) + 2. `mark_pull_request_as_ready_for_review` with `reason:` a one-line justification + naming the validated test(s) and ``. + 3. `add_labels` with `labels: ["p/0"]` for PR # — put the now-review-ready fix into + the team's p/0 priority queue so it is triaged, not lost in the draft backlog. (If + the PR somehow already carries `p/0`, this is a harmless no-op.) +- Record `marked-ready PR # (target green on ALL platforms on , + labeled p/0)` and stop. + #### Step 3.5.R — Maintainer change-request response (Track C) Reached from Step 3.5 gate 0. Goal: when an **eligible human reviewer** (Hard-Rule @@ -1383,7 +1669,19 @@ Filed by [`ci-status-fix`](https://github.com/dotnet/maui/blob/main/.github/work ### Step 8 — Per-issue tally + end-of-run summary -Per issue, append one outcome line to `/tmp/gh-aw/agent/coverage.txt`: +Per issue, append **one** outcome line to `/tmp/gh-aw/agent/coverage.txt` — the +terminal outcome for the cycle. When one cycle produces a chained pair — a PR gets a +same-cycle precursor line and **then** a Step 3.6 readiness line (`marked-ready` on the +draft→ready flip, or `already-ready` on a later steady-state sweep of an already-ready +PR) — record ONLY the terminal Step 3.6 readiness line: it supersedes ANY same-cycle +non-readiness precursor for that PR, so an aggregator keying on one-line-per-issue never +double-counts. The precursor may be a Step 3 green line (`surfaced-green` on the first ✅, +or `already-surfaced` when it already exists) OR — when an unrelated-flake red and a +target-green coincide — a Step 4 `annotated-flake` line; either way the readiness line is +terminal, and the superseded precursor's signal survives in the PR's 🎯/♻️ comment so no +information is lost. This covers the flip cycle (`surfaced-green` → `marked-ready`), the +post-flip steady state (`already-surfaced` → `already-ready`), and the flake-coincident +flip (`annotated-flake` → `marked-ready`): ``` # main attempt- @@ -1394,8 +1692,14 @@ Per issue, append one outcome line to `/tmp/gh-aw/agent/coverage.txt`: `advance-PR # attempt /10` (ADVANCE mode — new commit pushed to the existing PR), `surfaced-green PR #` (fix's own CI went green; commented for review, did not advance), `annotated-flake PR #` (red was unrelated flake; -commented, attempt NOT burned), `waiting PR #` (CI not settled yet), -`dry-run: would-`, `skipped: `. (The +commented, attempt NOT burned), `marked-ready PR #` (the specific fixed test +was confirmed green in the PR's own CI; draft flipped to ready for review), +`already-surfaced PR #` (the fix's ✅ green comment already existed this sweep; +re-post suppressed — superseded by the Step 3.6 readiness line when the PR also flips +ready that cycle), `already-ready PR #` (terminal steady state — the PR was flipped +ready on a prior cycle and remains green on an unchanged head; no action taken, +supersedes `already-surfaced`), `waiting PR #` (CI not settled yet), +`dry-run: would-`, `skipped: `. (The `needs-human-PR` outcome is reserved for the deferred hand-off — Step 6 currently records a skip instead, so it is not emitted.)
: `♻️ Attempt /10: red is - unrelated flake on leg(s) () on ; the fix itself is not - implicated. A maintainer re-run (/azp run ) should clear it.` Record - `annotated-flake PR # (head )` and stop. *(Round 1: human re-runs; - Round 2: auto re-trigger.)* + **Comment idempotency + dry-run suppress the ♻️ comment ONLY — neither skips Step + 3.6.** Scan the PR's existing comments for a prior bot `♻️ … unrelated flake … on + ` note for THIS head SHA; if one exists, set `SKIP_FLAKE_COMMENT = true`. + Resolve the attempt number from `C.effectiveAttempt` (the authoritative max(marker, + bot-commit) counter), omitting the `Attempt /10:` prefix only if it is + somehow indeterminate. Then post the flake note UNLESS suppressed: + - if `dry_run == "true"`: do NOT emit — print the intended `♻️` unrelated-flake body + to the run log and tally `dry-run: would-annotate-flake PR #`; + - else if `SKIP_FLAKE_COMMENT`: do NOT re-post — record `already-annotated-flake PR + # (head )` (re-annotating the same flake every 12h sweep is noise and + burns the per-run comment budget other PRs need); + - else `add_comment` on PR #: `♻️ Attempt /10: red is unrelated flake on + leg(s) () on ; the fix itself is not implicated. A + maintainer re-run (/azp run ) should clear it.` record `annotated-flake + PR # (head )`. + Then — in ALL of the above cases — run **Step 3.6** (target-test readiness gate) for + this PR before stopping: its `/azp`-gated target legs may conclude green on this SAME + head SHA on a LATER sweep, and Step 3.6 is the ONLY place the draft→ready flip + happens, so it MUST re-evaluate every sweep — never `stop` here before it. *(Round 1: + human re-runs; Round 2: auto re-trigger.)* - **Caused by the fix** (a failed leg still matches the original target signature, or the fix introduced a NEW failure): advance an attempt. - **Attempt count.** `attempt = C.effectiveAttempt` — the authoritative @@ -752,6 +856,188 @@ Run these gates in order — the FIRST that fires decides this cycle's outcome: from every prior commit on this branch**, emitted via the Step 5.6 ADVANCE path (push onto the same PR). +#### Step 3.6 — Target-test verification & mark-ready gate + +Reached from the Step 3 **green-surface** branch, the Step 4 **unrelated-flake** branch +(AFTER that branch has posted its comment), and the Step 3.5 **gate-2 target-focused +fast-path** (2a — while unrelated legs are still pending). Purpose: confirm the SPECIFIC +test(s) this PR was opened to fix now PASS in the PR's own CI, and — only when they do +— transition the draft `[ci-fix]` PR to **ready for review** so a maintainer sees a +validated fix instead of a draft. This is the ONLY place the loop flips draft→ready; it +is a state transition, **never** an approval or a merge (a human still reviews and +merges). Overall red on *unrelated* legs must NOT gate readiness — we validate the fix, +not the base branch's flakiness. + +**Preconditions** (ALL must hold; otherwise record `skipped: readiness N/A PR # +()` and stop this gate): +- `C.isDraft == true` — if the PR is already ready-for-review, the draft→ready + transition is already done; do NOT re-mark **and do NOT re-apply `p/0`**. `p/0` is applied + exactly ONCE, atomically with the draft→ready flip (T3 — all-or-nothing with the 🎯 audit + comment + mark-ready), so an already-ready loop PR that is MISSING `p/0` has almost + certainly had it **removed by a maintainer** de-prioritizing the PR — a pure triage action + the human-engagement guard (which inspects comments/reviews/commits, NOT label changes) + cannot see. Re-adding `p/0` every sweep would fight that maintainer indefinitely, violating + the loop's "never override a human" contract. So the loop does NOT reconcile the label: + record `already-ready PR #` and stop this gate. (Trade-off: on the rare occasion a + transient API error drops `p/0` at flip time *after* the T3 atomic pre-check passed, it is + not auto-re-added — but the PR is still ready-for-review with its 🎯 audit comment, and + re-adding `p/0` is a trivial manual action; that is strictly preferable to steam-rolling a + maintainer's deliberate de-prioritization.) +- The PR is unmistakably THIS workflow's own: `[ci-fix] ` title prefix AND the + `agentic-workflows` label (mirrors the safe-output lock; the compensating scope + control if the v0.80.9 compiler drops the declarative required-*). +- You reached this gate from Step 3 (green), Step 4 (**unrelated-flake**), or the Step 3.5 + gate-2 **target-focused fast-path** (2a) — i.e. the fix is NOT implicated in any red that + has concluded. If Step 4 classified a red as **caused by the fix** (ADVANCE), do NOT run + this gate — advance the attempt instead. + +**T1 — Identify the target test(s).** From the `[ci-scan]` issue signature the fix +addresses (and the PR's own diff), extract the fully-qualified test method name(s) the +fix targets — e.g. `SafeAreaShouldWorkOnAllShellTabs`. For a de-flake it is the +de-flaked test; for a product fix it is the originally-failing test(s). If NO specific +test can be identified (e.g. a product build-break rather than a test failure), this is a +**build-only fix** — there is no single test to validate cross-platform, so validate at +**whole-build** granularity rather than stopping (Step 3 only *comments*; Step 3.6 is the +sole draft→ready flip, so a build-only fix is undrafted HERE or not at all). We only reach +this gate from Step 3 (green) / Step 4 (unrelated-flake) / gate-2a, so the fix is not +implicated in any concluded red; additionally require, via the T2 timeline method on +`C.headSha`, that the primary build pipeline `maui-pr` (def 302) has CONCLUDED with EVERY +one of its platform build legs `succeeded`/`completed` and NONE failed/canceled or still +pending — the build is green on **every** platform, not just the originally-broken one (the +cross-platform guard, applied to the build instead of a test). A build-only fix is undrafted +ONLY on the strength of the auto-running `maui-pr` (def 302) whole-build green, which the +loop CAN observe: if the PR's `[ci-scan]` issue signature or its own diff indicates the +ORIGINATING failure was in a `/azp`-gated pipeline (`maui-pr-uitests` def 313 or +`maui-pr-devicetests` def 314) rather than `maui-pr` (def 302), do NOT undraft on +`maui-pr`-green alone — those pipelines do not auto-run on this PR (GITHUB_TOKEN cannot +trigger them), so a green `maui-pr` build is NOT evidence the gated build break is fixed; +record `skipped: build-only fix PR # targets gated pipeline () not run — +deferring to human` and stop this gate. Otherwise, set `TARGET := "the maui-pr build +(build-only fix — no single target test)"` and proceed to **T3** to mark ready. If any `maui-pr` build leg is still unconcluded, record `skipped: build-only fix PR +# not yet whole-build green (leg(s) pending)` and stop this gate WITHOUT marking +ready (a green subset is not enough — a still-pending build leg could yet fail). + +**T2 — Verify the target test's platform legs are green (anonymous, leg-level).** Per-test +outcomes come from the AzDO **test-results** API (`_apis/test/...`), which is **NOT reachable +anonymously — Hard-Rule 8**: those endpoints 302-redirect to a sign-in page on this runner, so +a `curl` returns an HTML redirect (not JSON) and every target test would look "not executed". +Validate instead at **leg granularity** using ONLY the anonymous `_apis/build/...` timeline +(Hard-Rule 8) — a `succeeded` category leg means every test in that category **that actually +ran** passed on that platform. This is a strong bar **only if the target test genuinely +executes**: a job can go green while one specific test is skipped at runtime (`[Ignore]`, +`Assert.Ignore()`, `Assert.Inconclusive()`, a `Skip=`/conditional `[Fact]`, an `#if`-out, or +an early `return` before the asserts). So before trusting a green leg as proof the target +passed, **confirm from the PR's OWN diff that the fix does NOT skip, ignore, disable, or +short-circuit the target test** (it adds no `[Ignore]`/`[Explicit]`, `Assert.Ignore`/ +`Assert.Inconclusive`, `Skip=`, category-exclusion, `#if`-out, or early `return` around the +target). If the fix could cause the target to be runtime-skipped rather than genuinely pass, +leg-level green is NOT sufficient evidence — record `skipped: target test may be +runtime-skipped by this fix (diff adds ); leg-green insufficient, deferring to human +on PR #` and stop this gate WITHOUT marking ready. Otherwise (the target genuinely runs +and asserts, as a de-flake or product fix does) a green category leg cannot hide a +target-test failure, so treat it as authoritative. + +Map the target test to the CI leg(s) that run it. A leg name encodes platform + test category +(e.g. `Android UITests SafeAreaEdges,Shadow`, `iOS UITests SafeAreaEdges,Shadow`, `macOS +UITests SafeAreaEdges,Shadow`, `Windows UITests SafeAreaEdges,Shadow`). Determine the target +test's UI-test category — its `[Category(UITestCategories.X)]` in the test/HostApp source, +visible in the PR diff or the test file — to know which leg-name substring identifies its legs; +for a device test, the per-platform device-test legs. + +Using the SAME build-discovery as Step 4 (filter AzDO builds by `branchName=refs/pull//merge` +or `sourceVersion == C.headSha`), read each build's **timeline** on `C.headSha` for the +pipeline(s) that RUN the target test — `maui-pr` (def 302) for unit/integration tests, +`maui-pr-uitests` (def 313) for Appium UI tests, `maui-pr-devicetests` (def 314) for device +tests: + +```bash +ORG=dnceng-public; PROJ=public; BUILD= +# Anonymous + Hard-Rule-8-compliant. NEVER call _apis/test/... (it 302-redirects to sign-in). +# Each type=="Job" record is one platform leg: result (succeeded/failed/canceled/null), +# state (completed/inProgress/pending), name (encodes " UITests "): +curl -s "https://dev.azure.com/$ORG/$PROJ/_apis/build/builds/$BUILD/timeline?api-version=7.1" \ + | tee /tmp/gh-aw/agent/timeline_${BUILD}.json \ + | jq -r '.records[] | select(.type=="Job") | "\(.result)\t\(.state)\t\(.name)"' +``` + +(The prefetch already exposes per-leg status in `C.failedLegs` / the checks context, and +`gh pr checks ` lists the same per-leg rows with platform+category in the name — use +whichever is handy; the build timeline is the authoritative cross-check on the exact build.) + +Treat a target test as **VALIDATED-GREEN** only if, on `C.headSha`, the leg that runs its +category is green on **every platform it runs on** — a fix that repairs one platform must not +silently regress the same test's leg on another, so a green leg on the originally-red platform +alone is NOT enough. Across the target pipeline's platform legs (for a UI test: the Android, +iOS, Windows, and macOS/MacCatalyst legs running the target's category; for a device test: each +device-test platform), require ALL of: +- the target's category leg is `result == "succeeded"` **and** `state == "completed"` on + **every** platform that runs it, AND +- that leg is `failed` / `canceled` / aborted on **NO** platform, AND +- **no platform is left unverified.** For each platform family the target pipeline covers, that + platform's category leg must have CONCLUDED (`state == "completed"`) on `C.headSha`. If a + platform simply has no leg for the target's category, the test does not run there — that is + fine, not a gap. But if a platform's category leg has **not concluded** (`state` is + `inProgress` / pending, or an `/azp`-gated `maui-pr-uitests` / `maui-pr-devicetests` leg that + has not been kicked), the test's status on that platform is UNKNOWN → the fix is NOT yet + validated across platforms: record `skipped: target test green on but + not yet verified on (leg(s) pending / need /azp run) on PR #` + and stop this gate WITHOUT marking ready. + +A target test whose category leg never concluded on ANY platform (**not executed** anywhere — +e.g. its `/azp`-gated pipeline has not been kicked) is likewise NOT validated: record `skipped: +target test not yet executed on PR # ( not run — needs /azp run)` and stop +this gate WITHOUT marking ready. Do NOT overclaim — a green *sibling* leg (a different category +on the same platform) is not the target's leg, and a green leg on one platform is not a pass on +the others. + +**T3 — Mark ready + report.** If EVERY target test is VALIDATED-GREEN. The 🎯 comment, +the mark-ready, and the `p/0` label are THREE SEPARATE safe-outputs — the comment +existing does NOT prove the mark-ready took effect, so they are tracked independently: + +- **Comment idempotency (dup-suppress only — never gates the mark-ready).** We only reach + T3 while `C.isDraft == true` (the precondition stops an already-ready PR before here). So + if a prior bot comment for THIS head that carries the leading `🎯` anchor AND the + contiguous phrase `validated green on ` already exists (BOTH T3 variants — + `🎯 Target test validated green on ` and the build-only `🎯 Build validated green on + ` — contain that exact contiguous phrase, so this recognizes both; keying on + `target test` alone would instead miss the build-only comment and re-post it every sweep. + **Require the `🎯` anchor** so the match can NEVER be loosened to a gapped + `validated…green…on` form that would false-positive on the Step 3 `✅ … validated — the + fix's CI is green on ` surface comment and wrongly suppress the audit 🎯), the comment + landed on an earlier sweep but the **mark-ready did NOT take + effect** (the PR is still a draft) — set `SUPPRESS_COMMENT = true` (do not re-post the + duplicate 🎯 comment) but STILL complete the mark-ready + label below. Never treat the + comment's existence as "already marked ready" while the PR is still a draft. Record this + case as `re-marking PR # (🎯 present; mark-ready did not take on a prior sweep)`. +- **Atomicity budget pre-check (Hard-Rule 7).** Determine the outputs this gate will emit: + `mark_pull_request_as_ready_for_review` + `add_labels` always, plus `add_comment` unless + `SUPPRESS_COMMENT`. Verify a free per-run slot remains in EACH bucket you are about to + use. If ANY required bucket is exhausted, emit NONE of them — record `skipped: per-run + cap reached; deferring mark-ready PR # to next cycle` and stop this gate. Never mark a + PR ready (or label it) unless its 🎯 audit comment is GUARANTEED to exist for this exact + head SHA — either posted in this same sweep, or (in the `SUPPRESS_COMMENT` re-marking case) + already present from a prior sweep. The outputs you DO emit either all land or all defer + together; the audit trail must never be absent, but it is NOT re-posted when it already exists. +- **Dry-run gate (Step 0):** if `dry_run == "true"`, emit NOTHING — print the intended + readiness comment and "would mark ready + add p/0" to the run log, tally `dry-run: + would-mark-ready PR #`, and stop. +- Otherwise emit for THIS PR number ``: + 1. `add_comment` (ONLY if not `SUPPRESS_COMMENT`): `🎯 Target test validated green on + — passed on ALL platforms it runs on (, + buildId ). — not + caused by this fix."> Transitioning this PR from draft to ready for review and adding + `p/0`; a maintainer still reviews and merges.` (For a **build-only fix** — the T1 + whole-build fallback, no single target test — phrase the first clause as `🎯 Build + validated green on — the maui-pr build passed on ALL platforms (buildId + ).` instead of naming a test.) + 2. `mark_pull_request_as_ready_for_review` with `reason:` a one-line justification + naming the validated test(s) and ``. + 3. `add_labels` with `labels: ["p/0"]` for PR # — put the now-review-ready fix into + the team's p/0 priority queue so it is triaged, not lost in the draft backlog. (If + the PR somehow already carries `p/0`, this is a harmless no-op.) +- Record `marked-ready PR # (target green on ALL platforms on , + labeled p/0)` and stop. + #### Step 3.5.R — Maintainer change-request response (Track C) Reached from Step 3.5 gate 0. Goal: when an **eligible human reviewer** (Hard-Rule @@ -1383,7 +1669,19 @@ Filed by [`ci-status-fix`](https://github.com/dotnet/maui/blob/main/.github/work ### Step 8 — Per-issue tally + end-of-run summary -Per issue, append one outcome line to `/tmp/gh-aw/agent/coverage.txt`: +Per issue, append **one** outcome line to `/tmp/gh-aw/agent/coverage.txt` — the +terminal outcome for the cycle. When one cycle produces a chained pair — a PR gets a +same-cycle precursor line and **then** a Step 3.6 readiness line (`marked-ready` on the +draft→ready flip, or `already-ready` on a later steady-state sweep of an already-ready +PR) — record ONLY the terminal Step 3.6 readiness line: it supersedes ANY same-cycle +non-readiness precursor for that PR, so an aggregator keying on one-line-per-issue never +double-counts. The precursor may be a Step 3 green line (`surfaced-green` on the first ✅, +or `already-surfaced` when it already exists) OR — when an unrelated-flake red and a +target-green coincide — a Step 4 `annotated-flake` line; either way the readiness line is +terminal, and the superseded precursor's signal survives in the PR's 🎯/♻️ comment so no +information is lost. This covers the flip cycle (`surfaced-green` → `marked-ready`), the +post-flip steady state (`already-surfaced` → `already-ready`), and the flake-coincident +flip (`annotated-flake` → `marked-ready`): ``` # main attempt- @@ -1394,8 +1692,14 @@ Per issue, append one outcome line to `/tmp/gh-aw/agent/coverage.txt`: `advance-PR # attempt /10` (ADVANCE mode — new commit pushed to the existing PR), `surfaced-green PR #` (fix's own CI went green; commented for review, did not advance), `annotated-flake PR #` (red was unrelated flake; -commented, attempt NOT burned), `waiting PR #` (CI not settled yet), -`dry-run: would-`, `skipped: `. (The +commented, attempt NOT burned), `marked-ready PR #` (the specific fixed test +was confirmed green in the PR's own CI; draft flipped to ready for review), +`already-surfaced PR #` (the fix's ✅ green comment already existed this sweep; +re-post suppressed — superseded by the Step 3.6 readiness line when the PR also flips +ready that cycle), `already-ready PR #` (terminal steady state — the PR was flipped +ready on a prior cycle and remains green on an unchanged head; no action taken, +supersedes `already-surfaced`), `waiting PR #` (CI not settled yet), +`dry-run: would-`, `skipped: `. (The `needs-human-PR` outcome is reserved for the deferred hand-off — Step 6 currently records a skip instead, so it is not emitted.)
(head )` and stop. *(Round 1: human re-runs; - Round 2: auto re-trigger.)* + **Comment idempotency + dry-run suppress the ♻️ comment ONLY — neither skips Step + 3.6.** Scan the PR's existing comments for a prior bot `♻️ … unrelated flake … on + ` note for THIS head SHA; if one exists, set `SKIP_FLAKE_COMMENT = true`. + Resolve the attempt number from `C.effectiveAttempt` (the authoritative max(marker, + bot-commit) counter), omitting the `Attempt /10:` prefix only if it is + somehow indeterminate. Then post the flake note UNLESS suppressed: + - if `dry_run == "true"`: do NOT emit — print the intended `♻️` unrelated-flake body + to the run log and tally `dry-run: would-annotate-flake PR #`; + - else if `SKIP_FLAKE_COMMENT`: do NOT re-post — record `already-annotated-flake PR + # (head )` (re-annotating the same flake every 12h sweep is noise and + burns the per-run comment budget other PRs need); + - else `add_comment` on PR #: `♻️ Attempt /10: red is unrelated flake on + leg(s) () on ; the fix itself is not implicated. A + maintainer re-run (/azp run ) should clear it.` record `annotated-flake + PR # (head )`. + Then — in ALL of the above cases — run **Step 3.6** (target-test readiness gate) for + this PR before stopping: its `/azp`-gated target legs may conclude green on this SAME + head SHA on a LATER sweep, and Step 3.6 is the ONLY place the draft→ready flip + happens, so it MUST re-evaluate every sweep — never `stop` here before it. *(Round 1: + human re-runs; Round 2: auto re-trigger.)* - **Caused by the fix** (a failed leg still matches the original target signature, or the fix introduced a NEW failure): advance an attempt. - **Attempt count.** `attempt = C.effectiveAttempt` — the authoritative @@ -752,6 +856,188 @@ Run these gates in order — the FIRST that fires decides this cycle's outcome: from every prior commit on this branch**, emitted via the Step 5.6 ADVANCE path (push onto the same PR). +#### Step 3.6 — Target-test verification & mark-ready gate + +Reached from the Step 3 **green-surface** branch, the Step 4 **unrelated-flake** branch +(AFTER that branch has posted its comment), and the Step 3.5 **gate-2 target-focused +fast-path** (2a — while unrelated legs are still pending). Purpose: confirm the SPECIFIC +test(s) this PR was opened to fix now PASS in the PR's own CI, and — only when they do +— transition the draft `[ci-fix]` PR to **ready for review** so a maintainer sees a +validated fix instead of a draft. This is the ONLY place the loop flips draft→ready; it +is a state transition, **never** an approval or a merge (a human still reviews and +merges). Overall red on *unrelated* legs must NOT gate readiness — we validate the fix, +not the base branch's flakiness. + +**Preconditions** (ALL must hold; otherwise record `skipped: readiness N/A PR # +()` and stop this gate): +- `C.isDraft == true` — if the PR is already ready-for-review, the draft→ready + transition is already done; do NOT re-mark **and do NOT re-apply `p/0`**. `p/0` is applied + exactly ONCE, atomically with the draft→ready flip (T3 — all-or-nothing with the 🎯 audit + comment + mark-ready), so an already-ready loop PR that is MISSING `p/0` has almost + certainly had it **removed by a maintainer** de-prioritizing the PR — a pure triage action + the human-engagement guard (which inspects comments/reviews/commits, NOT label changes) + cannot see. Re-adding `p/0` every sweep would fight that maintainer indefinitely, violating + the loop's "never override a human" contract. So the loop does NOT reconcile the label: + record `already-ready PR #` and stop this gate. (Trade-off: on the rare occasion a + transient API error drops `p/0` at flip time *after* the T3 atomic pre-check passed, it is + not auto-re-added — but the PR is still ready-for-review with its 🎯 audit comment, and + re-adding `p/0` is a trivial manual action; that is strictly preferable to steam-rolling a + maintainer's deliberate de-prioritization.) +- The PR is unmistakably THIS workflow's own: `[ci-fix] ` title prefix AND the + `agentic-workflows` label (mirrors the safe-output lock; the compensating scope + control if the v0.80.9 compiler drops the declarative required-*). +- You reached this gate from Step 3 (green), Step 4 (**unrelated-flake**), or the Step 3.5 + gate-2 **target-focused fast-path** (2a) — i.e. the fix is NOT implicated in any red that + has concluded. If Step 4 classified a red as **caused by the fix** (ADVANCE), do NOT run + this gate — advance the attempt instead. + +**T1 — Identify the target test(s).** From the `[ci-scan]` issue signature the fix +addresses (and the PR's own diff), extract the fully-qualified test method name(s) the +fix targets — e.g. `SafeAreaShouldWorkOnAllShellTabs`. For a de-flake it is the +de-flaked test; for a product fix it is the originally-failing test(s). If NO specific +test can be identified (e.g. a product build-break rather than a test failure), this is a +**build-only fix** — there is no single test to validate cross-platform, so validate at +**whole-build** granularity rather than stopping (Step 3 only *comments*; Step 3.6 is the +sole draft→ready flip, so a build-only fix is undrafted HERE or not at all). We only reach +this gate from Step 3 (green) / Step 4 (unrelated-flake) / gate-2a, so the fix is not +implicated in any concluded red; additionally require, via the T2 timeline method on +`C.headSha`, that the primary build pipeline `maui-pr` (def 302) has CONCLUDED with EVERY +one of its platform build legs `succeeded`/`completed` and NONE failed/canceled or still +pending — the build is green on **every** platform, not just the originally-broken one (the +cross-platform guard, applied to the build instead of a test). A build-only fix is undrafted +ONLY on the strength of the auto-running `maui-pr` (def 302) whole-build green, which the +loop CAN observe: if the PR's `[ci-scan]` issue signature or its own diff indicates the +ORIGINATING failure was in a `/azp`-gated pipeline (`maui-pr-uitests` def 313 or +`maui-pr-devicetests` def 314) rather than `maui-pr` (def 302), do NOT undraft on +`maui-pr`-green alone — those pipelines do not auto-run on this PR (GITHUB_TOKEN cannot +trigger them), so a green `maui-pr` build is NOT evidence the gated build break is fixed; +record `skipped: build-only fix PR # targets gated pipeline () not run — +deferring to human` and stop this gate. Otherwise, set `TARGET := "the maui-pr build +(build-only fix — no single target test)"` and proceed to **T3** to mark ready. If any `maui-pr` build leg is still unconcluded, record `skipped: build-only fix PR +# not yet whole-build green (leg(s) pending)` and stop this gate WITHOUT marking +ready (a green subset is not enough — a still-pending build leg could yet fail). + +**T2 — Verify the target test's platform legs are green (anonymous, leg-level).** Per-test +outcomes come from the AzDO **test-results** API (`_apis/test/...`), which is **NOT reachable +anonymously — Hard-Rule 8**: those endpoints 302-redirect to a sign-in page on this runner, so +a `curl` returns an HTML redirect (not JSON) and every target test would look "not executed". +Validate instead at **leg granularity** using ONLY the anonymous `_apis/build/...` timeline +(Hard-Rule 8) — a `succeeded` category leg means every test in that category **that actually +ran** passed on that platform. This is a strong bar **only if the target test genuinely +executes**: a job can go green while one specific test is skipped at runtime (`[Ignore]`, +`Assert.Ignore()`, `Assert.Inconclusive()`, a `Skip=`/conditional `[Fact]`, an `#if`-out, or +an early `return` before the asserts). So before trusting a green leg as proof the target +passed, **confirm from the PR's OWN diff that the fix does NOT skip, ignore, disable, or +short-circuit the target test** (it adds no `[Ignore]`/`[Explicit]`, `Assert.Ignore`/ +`Assert.Inconclusive`, `Skip=`, category-exclusion, `#if`-out, or early `return` around the +target). If the fix could cause the target to be runtime-skipped rather than genuinely pass, +leg-level green is NOT sufficient evidence — record `skipped: target test may be +runtime-skipped by this fix (diff adds ); leg-green insufficient, deferring to human +on PR #` and stop this gate WITHOUT marking ready. Otherwise (the target genuinely runs +and asserts, as a de-flake or product fix does) a green category leg cannot hide a +target-test failure, so treat it as authoritative. + +Map the target test to the CI leg(s) that run it. A leg name encodes platform + test category +(e.g. `Android UITests SafeAreaEdges,Shadow`, `iOS UITests SafeAreaEdges,Shadow`, `macOS +UITests SafeAreaEdges,Shadow`, `Windows UITests SafeAreaEdges,Shadow`). Determine the target +test's UI-test category — its `[Category(UITestCategories.X)]` in the test/HostApp source, +visible in the PR diff or the test file — to know which leg-name substring identifies its legs; +for a device test, the per-platform device-test legs. + +Using the SAME build-discovery as Step 4 (filter AzDO builds by `branchName=refs/pull//merge` +or `sourceVersion == C.headSha`), read each build's **timeline** on `C.headSha` for the +pipeline(s) that RUN the target test — `maui-pr` (def 302) for unit/integration tests, +`maui-pr-uitests` (def 313) for Appium UI tests, `maui-pr-devicetests` (def 314) for device +tests: + +```bash +ORG=dnceng-public; PROJ=public; BUILD= +# Anonymous + Hard-Rule-8-compliant. NEVER call _apis/test/... (it 302-redirects to sign-in). +# Each type=="Job" record is one platform leg: result (succeeded/failed/canceled/null), +# state (completed/inProgress/pending), name (encodes " UITests "): +curl -s "https://dev.azure.com/$ORG/$PROJ/_apis/build/builds/$BUILD/timeline?api-version=7.1" \ + | tee /tmp/gh-aw/agent/timeline_${BUILD}.json \ + | jq -r '.records[] | select(.type=="Job") | "\(.result)\t\(.state)\t\(.name)"' +``` + +(The prefetch already exposes per-leg status in `C.failedLegs` / the checks context, and +`gh pr checks ` lists the same per-leg rows with platform+category in the name — use +whichever is handy; the build timeline is the authoritative cross-check on the exact build.) + +Treat a target test as **VALIDATED-GREEN** only if, on `C.headSha`, the leg that runs its +category is green on **every platform it runs on** — a fix that repairs one platform must not +silently regress the same test's leg on another, so a green leg on the originally-red platform +alone is NOT enough. Across the target pipeline's platform legs (for a UI test: the Android, +iOS, Windows, and macOS/MacCatalyst legs running the target's category; for a device test: each +device-test platform), require ALL of: +- the target's category leg is `result == "succeeded"` **and** `state == "completed"` on + **every** platform that runs it, AND +- that leg is `failed` / `canceled` / aborted on **NO** platform, AND +- **no platform is left unverified.** For each platform family the target pipeline covers, that + platform's category leg must have CONCLUDED (`state == "completed"`) on `C.headSha`. If a + platform simply has no leg for the target's category, the test does not run there — that is + fine, not a gap. But if a platform's category leg has **not concluded** (`state` is + `inProgress` / pending, or an `/azp`-gated `maui-pr-uitests` / `maui-pr-devicetests` leg that + has not been kicked), the test's status on that platform is UNKNOWN → the fix is NOT yet + validated across platforms: record `skipped: target test green on but + not yet verified on (leg(s) pending / need /azp run) on PR #` + and stop this gate WITHOUT marking ready. + +A target test whose category leg never concluded on ANY platform (**not executed** anywhere — +e.g. its `/azp`-gated pipeline has not been kicked) is likewise NOT validated: record `skipped: +target test not yet executed on PR # ( not run — needs /azp run)` and stop +this gate WITHOUT marking ready. Do NOT overclaim — a green *sibling* leg (a different category +on the same platform) is not the target's leg, and a green leg on one platform is not a pass on +the others. + +**T3 — Mark ready + report.** If EVERY target test is VALIDATED-GREEN. The 🎯 comment, +the mark-ready, and the `p/0` label are THREE SEPARATE safe-outputs — the comment +existing does NOT prove the mark-ready took effect, so they are tracked independently: + +- **Comment idempotency (dup-suppress only — never gates the mark-ready).** We only reach + T3 while `C.isDraft == true` (the precondition stops an already-ready PR before here). So + if a prior bot comment for THIS head that carries the leading `🎯` anchor AND the + contiguous phrase `validated green on ` already exists (BOTH T3 variants — + `🎯 Target test validated green on ` and the build-only `🎯 Build validated green on + ` — contain that exact contiguous phrase, so this recognizes both; keying on + `target test` alone would instead miss the build-only comment and re-post it every sweep. + **Require the `🎯` anchor** so the match can NEVER be loosened to a gapped + `validated…green…on` form that would false-positive on the Step 3 `✅ … validated — the + fix's CI is green on ` surface comment and wrongly suppress the audit 🎯), the comment + landed on an earlier sweep but the **mark-ready did NOT take + effect** (the PR is still a draft) — set `SUPPRESS_COMMENT = true` (do not re-post the + duplicate 🎯 comment) but STILL complete the mark-ready + label below. Never treat the + comment's existence as "already marked ready" while the PR is still a draft. Record this + case as `re-marking PR # (🎯 present; mark-ready did not take on a prior sweep)`. +- **Atomicity budget pre-check (Hard-Rule 7).** Determine the outputs this gate will emit: + `mark_pull_request_as_ready_for_review` + `add_labels` always, plus `add_comment` unless + `SUPPRESS_COMMENT`. Verify a free per-run slot remains in EACH bucket you are about to + use. If ANY required bucket is exhausted, emit NONE of them — record `skipped: per-run + cap reached; deferring mark-ready PR # to next cycle` and stop this gate. Never mark a + PR ready (or label it) unless its 🎯 audit comment is GUARANTEED to exist for this exact + head SHA — either posted in this same sweep, or (in the `SUPPRESS_COMMENT` re-marking case) + already present from a prior sweep. The outputs you DO emit either all land or all defer + together; the audit trail must never be absent, but it is NOT re-posted when it already exists. +- **Dry-run gate (Step 0):** if `dry_run == "true"`, emit NOTHING — print the intended + readiness comment and "would mark ready + add p/0" to the run log, tally `dry-run: + would-mark-ready PR #`, and stop. +- Otherwise emit for THIS PR number ``: + 1. `add_comment` (ONLY if not `SUPPRESS_COMMENT`): `🎯 Target test validated green on + — passed on ALL platforms it runs on (, + buildId ). — not + caused by this fix."> Transitioning this PR from draft to ready for review and adding + `p/0`; a maintainer still reviews and merges.` (For a **build-only fix** — the T1 + whole-build fallback, no single target test — phrase the first clause as `🎯 Build + validated green on — the maui-pr build passed on ALL platforms (buildId + ).` instead of naming a test.) + 2. `mark_pull_request_as_ready_for_review` with `reason:` a one-line justification + naming the validated test(s) and ``. + 3. `add_labels` with `labels: ["p/0"]` for PR # — put the now-review-ready fix into + the team's p/0 priority queue so it is triaged, not lost in the draft backlog. (If + the PR somehow already carries `p/0`, this is a harmless no-op.) +- Record `marked-ready PR # (target green on ALL platforms on , + labeled p/0)` and stop. + #### Step 3.5.R — Maintainer change-request response (Track C) Reached from Step 3.5 gate 0. Goal: when an **eligible human reviewer** (Hard-Rule @@ -1383,7 +1669,19 @@ Filed by [`ci-status-fix`](https://github.com/dotnet/maui/blob/main/.github/work ### Step 8 — Per-issue tally + end-of-run summary -Per issue, append one outcome line to `/tmp/gh-aw/agent/coverage.txt`: +Per issue, append **one** outcome line to `/tmp/gh-aw/agent/coverage.txt` — the +terminal outcome for the cycle. When one cycle produces a chained pair — a PR gets a +same-cycle precursor line and **then** a Step 3.6 readiness line (`marked-ready` on the +draft→ready flip, or `already-ready` on a later steady-state sweep of an already-ready +PR) — record ONLY the terminal Step 3.6 readiness line: it supersedes ANY same-cycle +non-readiness precursor for that PR, so an aggregator keying on one-line-per-issue never +double-counts. The precursor may be a Step 3 green line (`surfaced-green` on the first ✅, +or `already-surfaced` when it already exists) OR — when an unrelated-flake red and a +target-green coincide — a Step 4 `annotated-flake` line; either way the readiness line is +terminal, and the superseded precursor's signal survives in the PR's 🎯/♻️ comment so no +information is lost. This covers the flip cycle (`surfaced-green` → `marked-ready`), the +post-flip steady state (`already-surfaced` → `already-ready`), and the flake-coincident +flip (`annotated-flake` → `marked-ready`): ``` # main attempt- @@ -1394,8 +1692,14 @@ Per issue, append one outcome line to `/tmp/gh-aw/agent/coverage.txt`: `advance-PR # attempt /10` (ADVANCE mode — new commit pushed to the existing PR), `surfaced-green PR #` (fix's own CI went green; commented for review, did not advance), `annotated-flake PR #` (red was unrelated flake; -commented, attempt NOT burned), `waiting PR #` (CI not settled yet), -`dry-run: would-`, `skipped: `. (The +commented, attempt NOT burned), `marked-ready PR #` (the specific fixed test +was confirmed green in the PR's own CI; draft flipped to ready for review), +`already-surfaced PR #` (the fix's ✅ green comment already existed this sweep; +re-post suppressed — superseded by the Step 3.6 readiness line when the PR also flips +ready that cycle), `already-ready PR #` (terminal steady state — the PR was flipped +ready on a prior cycle and remains green on an unchanged head; no action taken, +supersedes `already-surfaced`), `waiting PR #` (CI not settled yet), +`dry-run: would-`, `skipped: `. (The `needs-human-PR` outcome is reserved for the deferred hand-off — Step 6 currently records a skip instead, so it is not emitted.)
(head )` (re-annotating the same flake every 12h sweep is noise and + burns the per-run comment budget other PRs need); + - else `add_comment` on PR #: `♻️ Attempt /10: red is unrelated flake on + leg(s) () on ; the fix itself is not implicated. A + maintainer re-run (/azp run ) should clear it.` record `annotated-flake + PR # (head )`. + Then — in ALL of the above cases — run **Step 3.6** (target-test readiness gate) for + this PR before stopping: its `/azp`-gated target legs may conclude green on this SAME + head SHA on a LATER sweep, and Step 3.6 is the ONLY place the draft→ready flip + happens, so it MUST re-evaluate every sweep — never `stop` here before it. *(Round 1: + human re-runs; Round 2: auto re-trigger.)* - **Caused by the fix** (a failed leg still matches the original target signature, or the fix introduced a NEW failure): advance an attempt. - **Attempt count.** `attempt = C.effectiveAttempt` — the authoritative @@ -752,6 +856,188 @@ Run these gates in order — the FIRST that fires decides this cycle's outcome: from every prior commit on this branch**, emitted via the Step 5.6 ADVANCE path (push onto the same PR). +#### Step 3.6 — Target-test verification & mark-ready gate + +Reached from the Step 3 **green-surface** branch, the Step 4 **unrelated-flake** branch +(AFTER that branch has posted its comment), and the Step 3.5 **gate-2 target-focused +fast-path** (2a — while unrelated legs are still pending). Purpose: confirm the SPECIFIC +test(s) this PR was opened to fix now PASS in the PR's own CI, and — only when they do +— transition the draft `[ci-fix]` PR to **ready for review** so a maintainer sees a +validated fix instead of a draft. This is the ONLY place the loop flips draft→ready; it +is a state transition, **never** an approval or a merge (a human still reviews and +merges). Overall red on *unrelated* legs must NOT gate readiness — we validate the fix, +not the base branch's flakiness. + +**Preconditions** (ALL must hold; otherwise record `skipped: readiness N/A PR # +()` and stop this gate): +- `C.isDraft == true` — if the PR is already ready-for-review, the draft→ready + transition is already done; do NOT re-mark **and do NOT re-apply `p/0`**. `p/0` is applied + exactly ONCE, atomically with the draft→ready flip (T3 — all-or-nothing with the 🎯 audit + comment + mark-ready), so an already-ready loop PR that is MISSING `p/0` has almost + certainly had it **removed by a maintainer** de-prioritizing the PR — a pure triage action + the human-engagement guard (which inspects comments/reviews/commits, NOT label changes) + cannot see. Re-adding `p/0` every sweep would fight that maintainer indefinitely, violating + the loop's "never override a human" contract. So the loop does NOT reconcile the label: + record `already-ready PR #` and stop this gate. (Trade-off: on the rare occasion a + transient API error drops `p/0` at flip time *after* the T3 atomic pre-check passed, it is + not auto-re-added — but the PR is still ready-for-review with its 🎯 audit comment, and + re-adding `p/0` is a trivial manual action; that is strictly preferable to steam-rolling a + maintainer's deliberate de-prioritization.) +- The PR is unmistakably THIS workflow's own: `[ci-fix] ` title prefix AND the + `agentic-workflows` label (mirrors the safe-output lock; the compensating scope + control if the v0.80.9 compiler drops the declarative required-*). +- You reached this gate from Step 3 (green), Step 4 (**unrelated-flake**), or the Step 3.5 + gate-2 **target-focused fast-path** (2a) — i.e. the fix is NOT implicated in any red that + has concluded. If Step 4 classified a red as **caused by the fix** (ADVANCE), do NOT run + this gate — advance the attempt instead. + +**T1 — Identify the target test(s).** From the `[ci-scan]` issue signature the fix +addresses (and the PR's own diff), extract the fully-qualified test method name(s) the +fix targets — e.g. `SafeAreaShouldWorkOnAllShellTabs`. For a de-flake it is the +de-flaked test; for a product fix it is the originally-failing test(s). If NO specific +test can be identified (e.g. a product build-break rather than a test failure), this is a +**build-only fix** — there is no single test to validate cross-platform, so validate at +**whole-build** granularity rather than stopping (Step 3 only *comments*; Step 3.6 is the +sole draft→ready flip, so a build-only fix is undrafted HERE or not at all). We only reach +this gate from Step 3 (green) / Step 4 (unrelated-flake) / gate-2a, so the fix is not +implicated in any concluded red; additionally require, via the T2 timeline method on +`C.headSha`, that the primary build pipeline `maui-pr` (def 302) has CONCLUDED with EVERY +one of its platform build legs `succeeded`/`completed` and NONE failed/canceled or still +pending — the build is green on **every** platform, not just the originally-broken one (the +cross-platform guard, applied to the build instead of a test). A build-only fix is undrafted +ONLY on the strength of the auto-running `maui-pr` (def 302) whole-build green, which the +loop CAN observe: if the PR's `[ci-scan]` issue signature or its own diff indicates the +ORIGINATING failure was in a `/azp`-gated pipeline (`maui-pr-uitests` def 313 or +`maui-pr-devicetests` def 314) rather than `maui-pr` (def 302), do NOT undraft on +`maui-pr`-green alone — those pipelines do not auto-run on this PR (GITHUB_TOKEN cannot +trigger them), so a green `maui-pr` build is NOT evidence the gated build break is fixed; +record `skipped: build-only fix PR # targets gated pipeline () not run — +deferring to human` and stop this gate. Otherwise, set `TARGET := "the maui-pr build +(build-only fix — no single target test)"` and proceed to **T3** to mark ready. If any `maui-pr` build leg is still unconcluded, record `skipped: build-only fix PR +# not yet whole-build green (leg(s) pending)` and stop this gate WITHOUT marking +ready (a green subset is not enough — a still-pending build leg could yet fail). + +**T2 — Verify the target test's platform legs are green (anonymous, leg-level).** Per-test +outcomes come from the AzDO **test-results** API (`_apis/test/...`), which is **NOT reachable +anonymously — Hard-Rule 8**: those endpoints 302-redirect to a sign-in page on this runner, so +a `curl` returns an HTML redirect (not JSON) and every target test would look "not executed". +Validate instead at **leg granularity** using ONLY the anonymous `_apis/build/...` timeline +(Hard-Rule 8) — a `succeeded` category leg means every test in that category **that actually +ran** passed on that platform. This is a strong bar **only if the target test genuinely +executes**: a job can go green while one specific test is skipped at runtime (`[Ignore]`, +`Assert.Ignore()`, `Assert.Inconclusive()`, a `Skip=`/conditional `[Fact]`, an `#if`-out, or +an early `return` before the asserts). So before trusting a green leg as proof the target +passed, **confirm from the PR's OWN diff that the fix does NOT skip, ignore, disable, or +short-circuit the target test** (it adds no `[Ignore]`/`[Explicit]`, `Assert.Ignore`/ +`Assert.Inconclusive`, `Skip=`, category-exclusion, `#if`-out, or early `return` around the +target). If the fix could cause the target to be runtime-skipped rather than genuinely pass, +leg-level green is NOT sufficient evidence — record `skipped: target test may be +runtime-skipped by this fix (diff adds ); leg-green insufficient, deferring to human +on PR #` and stop this gate WITHOUT marking ready. Otherwise (the target genuinely runs +and asserts, as a de-flake or product fix does) a green category leg cannot hide a +target-test failure, so treat it as authoritative. + +Map the target test to the CI leg(s) that run it. A leg name encodes platform + test category +(e.g. `Android UITests SafeAreaEdges,Shadow`, `iOS UITests SafeAreaEdges,Shadow`, `macOS +UITests SafeAreaEdges,Shadow`, `Windows UITests SafeAreaEdges,Shadow`). Determine the target +test's UI-test category — its `[Category(UITestCategories.X)]` in the test/HostApp source, +visible in the PR diff or the test file — to know which leg-name substring identifies its legs; +for a device test, the per-platform device-test legs. + +Using the SAME build-discovery as Step 4 (filter AzDO builds by `branchName=refs/pull//merge` +or `sourceVersion == C.headSha`), read each build's **timeline** on `C.headSha` for the +pipeline(s) that RUN the target test — `maui-pr` (def 302) for unit/integration tests, +`maui-pr-uitests` (def 313) for Appium UI tests, `maui-pr-devicetests` (def 314) for device +tests: + +```bash +ORG=dnceng-public; PROJ=public; BUILD= +# Anonymous + Hard-Rule-8-compliant. NEVER call _apis/test/... (it 302-redirects to sign-in). +# Each type=="Job" record is one platform leg: result (succeeded/failed/canceled/null), +# state (completed/inProgress/pending), name (encodes " UITests "): +curl -s "https://dev.azure.com/$ORG/$PROJ/_apis/build/builds/$BUILD/timeline?api-version=7.1" \ + | tee /tmp/gh-aw/agent/timeline_${BUILD}.json \ + | jq -r '.records[] | select(.type=="Job") | "\(.result)\t\(.state)\t\(.name)"' +``` + +(The prefetch already exposes per-leg status in `C.failedLegs` / the checks context, and +`gh pr checks ` lists the same per-leg rows with platform+category in the name — use +whichever is handy; the build timeline is the authoritative cross-check on the exact build.) + +Treat a target test as **VALIDATED-GREEN** only if, on `C.headSha`, the leg that runs its +category is green on **every platform it runs on** — a fix that repairs one platform must not +silently regress the same test's leg on another, so a green leg on the originally-red platform +alone is NOT enough. Across the target pipeline's platform legs (for a UI test: the Android, +iOS, Windows, and macOS/MacCatalyst legs running the target's category; for a device test: each +device-test platform), require ALL of: +- the target's category leg is `result == "succeeded"` **and** `state == "completed"` on + **every** platform that runs it, AND +- that leg is `failed` / `canceled` / aborted on **NO** platform, AND +- **no platform is left unverified.** For each platform family the target pipeline covers, that + platform's category leg must have CONCLUDED (`state == "completed"`) on `C.headSha`. If a + platform simply has no leg for the target's category, the test does not run there — that is + fine, not a gap. But if a platform's category leg has **not concluded** (`state` is + `inProgress` / pending, or an `/azp`-gated `maui-pr-uitests` / `maui-pr-devicetests` leg that + has not been kicked), the test's status on that platform is UNKNOWN → the fix is NOT yet + validated across platforms: record `skipped: target test green on but + not yet verified on (leg(s) pending / need /azp run) on PR #` + and stop this gate WITHOUT marking ready. + +A target test whose category leg never concluded on ANY platform (**not executed** anywhere — +e.g. its `/azp`-gated pipeline has not been kicked) is likewise NOT validated: record `skipped: +target test not yet executed on PR # ( not run — needs /azp run)` and stop +this gate WITHOUT marking ready. Do NOT overclaim — a green *sibling* leg (a different category +on the same platform) is not the target's leg, and a green leg on one platform is not a pass on +the others. + +**T3 — Mark ready + report.** If EVERY target test is VALIDATED-GREEN. The 🎯 comment, +the mark-ready, and the `p/0` label are THREE SEPARATE safe-outputs — the comment +existing does NOT prove the mark-ready took effect, so they are tracked independently: + +- **Comment idempotency (dup-suppress only — never gates the mark-ready).** We only reach + T3 while `C.isDraft == true` (the precondition stops an already-ready PR before here). So + if a prior bot comment for THIS head that carries the leading `🎯` anchor AND the + contiguous phrase `validated green on ` already exists (BOTH T3 variants — + `🎯 Target test validated green on ` and the build-only `🎯 Build validated green on + ` — contain that exact contiguous phrase, so this recognizes both; keying on + `target test` alone would instead miss the build-only comment and re-post it every sweep. + **Require the `🎯` anchor** so the match can NEVER be loosened to a gapped + `validated…green…on` form that would false-positive on the Step 3 `✅ … validated — the + fix's CI is green on ` surface comment and wrongly suppress the audit 🎯), the comment + landed on an earlier sweep but the **mark-ready did NOT take + effect** (the PR is still a draft) — set `SUPPRESS_COMMENT = true` (do not re-post the + duplicate 🎯 comment) but STILL complete the mark-ready + label below. Never treat the + comment's existence as "already marked ready" while the PR is still a draft. Record this + case as `re-marking PR # (🎯 present; mark-ready did not take on a prior sweep)`. +- **Atomicity budget pre-check (Hard-Rule 7).** Determine the outputs this gate will emit: + `mark_pull_request_as_ready_for_review` + `add_labels` always, plus `add_comment` unless + `SUPPRESS_COMMENT`. Verify a free per-run slot remains in EACH bucket you are about to + use. If ANY required bucket is exhausted, emit NONE of them — record `skipped: per-run + cap reached; deferring mark-ready PR # to next cycle` and stop this gate. Never mark a + PR ready (or label it) unless its 🎯 audit comment is GUARANTEED to exist for this exact + head SHA — either posted in this same sweep, or (in the `SUPPRESS_COMMENT` re-marking case) + already present from a prior sweep. The outputs you DO emit either all land or all defer + together; the audit trail must never be absent, but it is NOT re-posted when it already exists. +- **Dry-run gate (Step 0):** if `dry_run == "true"`, emit NOTHING — print the intended + readiness comment and "would mark ready + add p/0" to the run log, tally `dry-run: + would-mark-ready PR #`, and stop. +- Otherwise emit for THIS PR number ``: + 1. `add_comment` (ONLY if not `SUPPRESS_COMMENT`): `🎯 Target test validated green on + — passed on ALL platforms it runs on (, + buildId ). — not + caused by this fix."> Transitioning this PR from draft to ready for review and adding + `p/0`; a maintainer still reviews and merges.` (For a **build-only fix** — the T1 + whole-build fallback, no single target test — phrase the first clause as `🎯 Build + validated green on — the maui-pr build passed on ALL platforms (buildId + ).` instead of naming a test.) + 2. `mark_pull_request_as_ready_for_review` with `reason:` a one-line justification + naming the validated test(s) and ``. + 3. `add_labels` with `labels: ["p/0"]` for PR # — put the now-review-ready fix into + the team's p/0 priority queue so it is triaged, not lost in the draft backlog. (If + the PR somehow already carries `p/0`, this is a harmless no-op.) +- Record `marked-ready PR # (target green on ALL platforms on , + labeled p/0)` and stop. + #### Step 3.5.R — Maintainer change-request response (Track C) Reached from Step 3.5 gate 0. Goal: when an **eligible human reviewer** (Hard-Rule @@ -1383,7 +1669,19 @@ Filed by [`ci-status-fix`](https://github.com/dotnet/maui/blob/main/.github/work ### Step 8 — Per-issue tally + end-of-run summary -Per issue, append one outcome line to `/tmp/gh-aw/agent/coverage.txt`: +Per issue, append **one** outcome line to `/tmp/gh-aw/agent/coverage.txt` — the +terminal outcome for the cycle. When one cycle produces a chained pair — a PR gets a +same-cycle precursor line and **then** a Step 3.6 readiness line (`marked-ready` on the +draft→ready flip, or `already-ready` on a later steady-state sweep of an already-ready +PR) — record ONLY the terminal Step 3.6 readiness line: it supersedes ANY same-cycle +non-readiness precursor for that PR, so an aggregator keying on one-line-per-issue never +double-counts. The precursor may be a Step 3 green line (`surfaced-green` on the first ✅, +or `already-surfaced` when it already exists) OR — when an unrelated-flake red and a +target-green coincide — a Step 4 `annotated-flake` line; either way the readiness line is +terminal, and the superseded precursor's signal survives in the PR's 🎯/♻️ comment so no +information is lost. This covers the flip cycle (`surfaced-green` → `marked-ready`), the +post-flip steady state (`already-surfaced` → `already-ready`), and the flake-coincident +flip (`annotated-flake` → `marked-ready`): ``` # main attempt- @@ -1394,8 +1692,14 @@ Per issue, append one outcome line to `/tmp/gh-aw/agent/coverage.txt`: `advance-PR # attempt /10` (ADVANCE mode — new commit pushed to the existing PR), `surfaced-green PR #` (fix's own CI went green; commented for review, did not advance), `annotated-flake PR #` (red was unrelated flake; -commented, attempt NOT burned), `waiting PR #` (CI not settled yet), -`dry-run: would-`, `skipped: `. (The +commented, attempt NOT burned), `marked-ready PR #` (the specific fixed test +was confirmed green in the PR's own CI; draft flipped to ready for review), +`already-surfaced PR #` (the fix's ✅ green comment already existed this sweep; +re-post suppressed — superseded by the Step 3.6 readiness line when the PR also flips +ready that cycle), `already-ready PR #` (terminal steady state — the PR was flipped +ready on a prior cycle and remains green on an unchanged head; no action taken, +supersedes `already-surfaced`), `waiting PR #` (CI not settled yet), +`dry-run: would-`, `skipped: `. (The `needs-human-PR` outcome is reserved for the deferred hand-off — Step 6 currently records a skip instead, so it is not emitted.)
: `♻️ Attempt /10: red is unrelated flake on + leg(s) () on ; the fix itself is not implicated. A + maintainer re-run (/azp run ) should clear it.` record `annotated-flake + PR # (head )`. + Then — in ALL of the above cases — run **Step 3.6** (target-test readiness gate) for + this PR before stopping: its `/azp`-gated target legs may conclude green on this SAME + head SHA on a LATER sweep, and Step 3.6 is the ONLY place the draft→ready flip + happens, so it MUST re-evaluate every sweep — never `stop` here before it. *(Round 1: + human re-runs; Round 2: auto re-trigger.)* - **Caused by the fix** (a failed leg still matches the original target signature, or the fix introduced a NEW failure): advance an attempt. - **Attempt count.** `attempt = C.effectiveAttempt` — the authoritative @@ -752,6 +856,188 @@ Run these gates in order — the FIRST that fires decides this cycle's outcome: from every prior commit on this branch**, emitted via the Step 5.6 ADVANCE path (push onto the same PR). +#### Step 3.6 — Target-test verification & mark-ready gate + +Reached from the Step 3 **green-surface** branch, the Step 4 **unrelated-flake** branch +(AFTER that branch has posted its comment), and the Step 3.5 **gate-2 target-focused +fast-path** (2a — while unrelated legs are still pending). Purpose: confirm the SPECIFIC +test(s) this PR was opened to fix now PASS in the PR's own CI, and — only when they do +— transition the draft `[ci-fix]` PR to **ready for review** so a maintainer sees a +validated fix instead of a draft. This is the ONLY place the loop flips draft→ready; it +is a state transition, **never** an approval or a merge (a human still reviews and +merges). Overall red on *unrelated* legs must NOT gate readiness — we validate the fix, +not the base branch's flakiness. + +**Preconditions** (ALL must hold; otherwise record `skipped: readiness N/A PR # +()` and stop this gate): +- `C.isDraft == true` — if the PR is already ready-for-review, the draft→ready + transition is already done; do NOT re-mark **and do NOT re-apply `p/0`**. `p/0` is applied + exactly ONCE, atomically with the draft→ready flip (T3 — all-or-nothing with the 🎯 audit + comment + mark-ready), so an already-ready loop PR that is MISSING `p/0` has almost + certainly had it **removed by a maintainer** de-prioritizing the PR — a pure triage action + the human-engagement guard (which inspects comments/reviews/commits, NOT label changes) + cannot see. Re-adding `p/0` every sweep would fight that maintainer indefinitely, violating + the loop's "never override a human" contract. So the loop does NOT reconcile the label: + record `already-ready PR #` and stop this gate. (Trade-off: on the rare occasion a + transient API error drops `p/0` at flip time *after* the T3 atomic pre-check passed, it is + not auto-re-added — but the PR is still ready-for-review with its 🎯 audit comment, and + re-adding `p/0` is a trivial manual action; that is strictly preferable to steam-rolling a + maintainer's deliberate de-prioritization.) +- The PR is unmistakably THIS workflow's own: `[ci-fix] ` title prefix AND the + `agentic-workflows` label (mirrors the safe-output lock; the compensating scope + control if the v0.80.9 compiler drops the declarative required-*). +- You reached this gate from Step 3 (green), Step 4 (**unrelated-flake**), or the Step 3.5 + gate-2 **target-focused fast-path** (2a) — i.e. the fix is NOT implicated in any red that + has concluded. If Step 4 classified a red as **caused by the fix** (ADVANCE), do NOT run + this gate — advance the attempt instead. + +**T1 — Identify the target test(s).** From the `[ci-scan]` issue signature the fix +addresses (and the PR's own diff), extract the fully-qualified test method name(s) the +fix targets — e.g. `SafeAreaShouldWorkOnAllShellTabs`. For a de-flake it is the +de-flaked test; for a product fix it is the originally-failing test(s). If NO specific +test can be identified (e.g. a product build-break rather than a test failure), this is a +**build-only fix** — there is no single test to validate cross-platform, so validate at +**whole-build** granularity rather than stopping (Step 3 only *comments*; Step 3.6 is the +sole draft→ready flip, so a build-only fix is undrafted HERE or not at all). We only reach +this gate from Step 3 (green) / Step 4 (unrelated-flake) / gate-2a, so the fix is not +implicated in any concluded red; additionally require, via the T2 timeline method on +`C.headSha`, that the primary build pipeline `maui-pr` (def 302) has CONCLUDED with EVERY +one of its platform build legs `succeeded`/`completed` and NONE failed/canceled or still +pending — the build is green on **every** platform, not just the originally-broken one (the +cross-platform guard, applied to the build instead of a test). A build-only fix is undrafted +ONLY on the strength of the auto-running `maui-pr` (def 302) whole-build green, which the +loop CAN observe: if the PR's `[ci-scan]` issue signature or its own diff indicates the +ORIGINATING failure was in a `/azp`-gated pipeline (`maui-pr-uitests` def 313 or +`maui-pr-devicetests` def 314) rather than `maui-pr` (def 302), do NOT undraft on +`maui-pr`-green alone — those pipelines do not auto-run on this PR (GITHUB_TOKEN cannot +trigger them), so a green `maui-pr` build is NOT evidence the gated build break is fixed; +record `skipped: build-only fix PR # targets gated pipeline () not run — +deferring to human` and stop this gate. Otherwise, set `TARGET := "the maui-pr build +(build-only fix — no single target test)"` and proceed to **T3** to mark ready. If any `maui-pr` build leg is still unconcluded, record `skipped: build-only fix PR +# not yet whole-build green (leg(s) pending)` and stop this gate WITHOUT marking +ready (a green subset is not enough — a still-pending build leg could yet fail). + +**T2 — Verify the target test's platform legs are green (anonymous, leg-level).** Per-test +outcomes come from the AzDO **test-results** API (`_apis/test/...`), which is **NOT reachable +anonymously — Hard-Rule 8**: those endpoints 302-redirect to a sign-in page on this runner, so +a `curl` returns an HTML redirect (not JSON) and every target test would look "not executed". +Validate instead at **leg granularity** using ONLY the anonymous `_apis/build/...` timeline +(Hard-Rule 8) — a `succeeded` category leg means every test in that category **that actually +ran** passed on that platform. This is a strong bar **only if the target test genuinely +executes**: a job can go green while one specific test is skipped at runtime (`[Ignore]`, +`Assert.Ignore()`, `Assert.Inconclusive()`, a `Skip=`/conditional `[Fact]`, an `#if`-out, or +an early `return` before the asserts). So before trusting a green leg as proof the target +passed, **confirm from the PR's OWN diff that the fix does NOT skip, ignore, disable, or +short-circuit the target test** (it adds no `[Ignore]`/`[Explicit]`, `Assert.Ignore`/ +`Assert.Inconclusive`, `Skip=`, category-exclusion, `#if`-out, or early `return` around the +target). If the fix could cause the target to be runtime-skipped rather than genuinely pass, +leg-level green is NOT sufficient evidence — record `skipped: target test may be +runtime-skipped by this fix (diff adds ); leg-green insufficient, deferring to human +on PR #` and stop this gate WITHOUT marking ready. Otherwise (the target genuinely runs +and asserts, as a de-flake or product fix does) a green category leg cannot hide a +target-test failure, so treat it as authoritative. + +Map the target test to the CI leg(s) that run it. A leg name encodes platform + test category +(e.g. `Android UITests SafeAreaEdges,Shadow`, `iOS UITests SafeAreaEdges,Shadow`, `macOS +UITests SafeAreaEdges,Shadow`, `Windows UITests SafeAreaEdges,Shadow`). Determine the target +test's UI-test category — its `[Category(UITestCategories.X)]` in the test/HostApp source, +visible in the PR diff or the test file — to know which leg-name substring identifies its legs; +for a device test, the per-platform device-test legs. + +Using the SAME build-discovery as Step 4 (filter AzDO builds by `branchName=refs/pull//merge` +or `sourceVersion == C.headSha`), read each build's **timeline** on `C.headSha` for the +pipeline(s) that RUN the target test — `maui-pr` (def 302) for unit/integration tests, +`maui-pr-uitests` (def 313) for Appium UI tests, `maui-pr-devicetests` (def 314) for device +tests: + +```bash +ORG=dnceng-public; PROJ=public; BUILD= +# Anonymous + Hard-Rule-8-compliant. NEVER call _apis/test/... (it 302-redirects to sign-in). +# Each type=="Job" record is one platform leg: result (succeeded/failed/canceled/null), +# state (completed/inProgress/pending), name (encodes " UITests "): +curl -s "https://dev.azure.com/$ORG/$PROJ/_apis/build/builds/$BUILD/timeline?api-version=7.1" \ + | tee /tmp/gh-aw/agent/timeline_${BUILD}.json \ + | jq -r '.records[] | select(.type=="Job") | "\(.result)\t\(.state)\t\(.name)"' +``` + +(The prefetch already exposes per-leg status in `C.failedLegs` / the checks context, and +`gh pr checks ` lists the same per-leg rows with platform+category in the name — use +whichever is handy; the build timeline is the authoritative cross-check on the exact build.) + +Treat a target test as **VALIDATED-GREEN** only if, on `C.headSha`, the leg that runs its +category is green on **every platform it runs on** — a fix that repairs one platform must not +silently regress the same test's leg on another, so a green leg on the originally-red platform +alone is NOT enough. Across the target pipeline's platform legs (for a UI test: the Android, +iOS, Windows, and macOS/MacCatalyst legs running the target's category; for a device test: each +device-test platform), require ALL of: +- the target's category leg is `result == "succeeded"` **and** `state == "completed"` on + **every** platform that runs it, AND +- that leg is `failed` / `canceled` / aborted on **NO** platform, AND +- **no platform is left unverified.** For each platform family the target pipeline covers, that + platform's category leg must have CONCLUDED (`state == "completed"`) on `C.headSha`. If a + platform simply has no leg for the target's category, the test does not run there — that is + fine, not a gap. But if a platform's category leg has **not concluded** (`state` is + `inProgress` / pending, or an `/azp`-gated `maui-pr-uitests` / `maui-pr-devicetests` leg that + has not been kicked), the test's status on that platform is UNKNOWN → the fix is NOT yet + validated across platforms: record `skipped: target test green on but + not yet verified on (leg(s) pending / need /azp run) on PR #` + and stop this gate WITHOUT marking ready. + +A target test whose category leg never concluded on ANY platform (**not executed** anywhere — +e.g. its `/azp`-gated pipeline has not been kicked) is likewise NOT validated: record `skipped: +target test not yet executed on PR # ( not run — needs /azp run)` and stop +this gate WITHOUT marking ready. Do NOT overclaim — a green *sibling* leg (a different category +on the same platform) is not the target's leg, and a green leg on one platform is not a pass on +the others. + +**T3 — Mark ready + report.** If EVERY target test is VALIDATED-GREEN. The 🎯 comment, +the mark-ready, and the `p/0` label are THREE SEPARATE safe-outputs — the comment +existing does NOT prove the mark-ready took effect, so they are tracked independently: + +- **Comment idempotency (dup-suppress only — never gates the mark-ready).** We only reach + T3 while `C.isDraft == true` (the precondition stops an already-ready PR before here). So + if a prior bot comment for THIS head that carries the leading `🎯` anchor AND the + contiguous phrase `validated green on ` already exists (BOTH T3 variants — + `🎯 Target test validated green on ` and the build-only `🎯 Build validated green on + ` — contain that exact contiguous phrase, so this recognizes both; keying on + `target test` alone would instead miss the build-only comment and re-post it every sweep. + **Require the `🎯` anchor** so the match can NEVER be loosened to a gapped + `validated…green…on` form that would false-positive on the Step 3 `✅ … validated — the + fix's CI is green on ` surface comment and wrongly suppress the audit 🎯), the comment + landed on an earlier sweep but the **mark-ready did NOT take + effect** (the PR is still a draft) — set `SUPPRESS_COMMENT = true` (do not re-post the + duplicate 🎯 comment) but STILL complete the mark-ready + label below. Never treat the + comment's existence as "already marked ready" while the PR is still a draft. Record this + case as `re-marking PR # (🎯 present; mark-ready did not take on a prior sweep)`. +- **Atomicity budget pre-check (Hard-Rule 7).** Determine the outputs this gate will emit: + `mark_pull_request_as_ready_for_review` + `add_labels` always, plus `add_comment` unless + `SUPPRESS_COMMENT`. Verify a free per-run slot remains in EACH bucket you are about to + use. If ANY required bucket is exhausted, emit NONE of them — record `skipped: per-run + cap reached; deferring mark-ready PR # to next cycle` and stop this gate. Never mark a + PR ready (or label it) unless its 🎯 audit comment is GUARANTEED to exist for this exact + head SHA — either posted in this same sweep, or (in the `SUPPRESS_COMMENT` re-marking case) + already present from a prior sweep. The outputs you DO emit either all land or all defer + together; the audit trail must never be absent, but it is NOT re-posted when it already exists. +- **Dry-run gate (Step 0):** if `dry_run == "true"`, emit NOTHING — print the intended + readiness comment and "would mark ready + add p/0" to the run log, tally `dry-run: + would-mark-ready PR #`, and stop. +- Otherwise emit for THIS PR number ``: + 1. `add_comment` (ONLY if not `SUPPRESS_COMMENT`): `🎯 Target test validated green on + — passed on ALL platforms it runs on (, + buildId ). — not + caused by this fix."> Transitioning this PR from draft to ready for review and adding + `p/0`; a maintainer still reviews and merges.` (For a **build-only fix** — the T1 + whole-build fallback, no single target test — phrase the first clause as `🎯 Build + validated green on — the maui-pr build passed on ALL platforms (buildId + ).` instead of naming a test.) + 2. `mark_pull_request_as_ready_for_review` with `reason:` a one-line justification + naming the validated test(s) and ``. + 3. `add_labels` with `labels: ["p/0"]` for PR # — put the now-review-ready fix into + the team's p/0 priority queue so it is triaged, not lost in the draft backlog. (If + the PR somehow already carries `p/0`, this is a harmless no-op.) +- Record `marked-ready PR # (target green on ALL platforms on , + labeled p/0)` and stop. + #### Step 3.5.R — Maintainer change-request response (Track C) Reached from Step 3.5 gate 0. Goal: when an **eligible human reviewer** (Hard-Rule @@ -1383,7 +1669,19 @@ Filed by [`ci-status-fix`](https://github.com/dotnet/maui/blob/main/.github/work ### Step 8 — Per-issue tally + end-of-run summary -Per issue, append one outcome line to `/tmp/gh-aw/agent/coverage.txt`: +Per issue, append **one** outcome line to `/tmp/gh-aw/agent/coverage.txt` — the +terminal outcome for the cycle. When one cycle produces a chained pair — a PR gets a +same-cycle precursor line and **then** a Step 3.6 readiness line (`marked-ready` on the +draft→ready flip, or `already-ready` on a later steady-state sweep of an already-ready +PR) — record ONLY the terminal Step 3.6 readiness line: it supersedes ANY same-cycle +non-readiness precursor for that PR, so an aggregator keying on one-line-per-issue never +double-counts. The precursor may be a Step 3 green line (`surfaced-green` on the first ✅, +or `already-surfaced` when it already exists) OR — when an unrelated-flake red and a +target-green coincide — a Step 4 `annotated-flake` line; either way the readiness line is +terminal, and the superseded precursor's signal survives in the PR's 🎯/♻️ comment so no +information is lost. This covers the flip cycle (`surfaced-green` → `marked-ready`), the +post-flip steady state (`already-surfaced` → `already-ready`), and the flake-coincident +flip (`annotated-flake` → `marked-ready`): ``` # main attempt- @@ -1394,8 +1692,14 @@ Per issue, append one outcome line to `/tmp/gh-aw/agent/coverage.txt`: `advance-PR # attempt /10` (ADVANCE mode — new commit pushed to the existing PR), `surfaced-green PR #` (fix's own CI went green; commented for review, did not advance), `annotated-flake PR #` (red was unrelated flake; -commented, attempt NOT burned), `waiting PR #` (CI not settled yet), -`dry-run: would-`, `skipped: `. (The +commented, attempt NOT burned), `marked-ready PR #` (the specific fixed test +was confirmed green in the PR's own CI; draft flipped to ready for review), +`already-surfaced PR #` (the fix's ✅ green comment already existed this sweep; +re-post suppressed — superseded by the Step 3.6 readiness line when the PR also flips +ready that cycle), `already-ready PR #` (terminal steady state — the PR was flipped +ready on a prior cycle and remains green on an unchanged head; no action taken, +supersedes `already-surfaced`), `waiting PR #` (CI not settled yet), +`dry-run: would-`, `skipped: `. (The `needs-human-PR` outcome is reserved for the deferred hand-off — Step 6 currently records a skip instead, so it is not emitted.)
(head )`. + Then — in ALL of the above cases — run **Step 3.6** (target-test readiness gate) for + this PR before stopping: its `/azp`-gated target legs may conclude green on this SAME + head SHA on a LATER sweep, and Step 3.6 is the ONLY place the draft→ready flip + happens, so it MUST re-evaluate every sweep — never `stop` here before it. *(Round 1: + human re-runs; Round 2: auto re-trigger.)* - **Caused by the fix** (a failed leg still matches the original target signature, or the fix introduced a NEW failure): advance an attempt. - **Attempt count.** `attempt = C.effectiveAttempt` — the authoritative @@ -752,6 +856,188 @@ Run these gates in order — the FIRST that fires decides this cycle's outcome: from every prior commit on this branch**, emitted via the Step 5.6 ADVANCE path (push onto the same PR). +#### Step 3.6 — Target-test verification & mark-ready gate + +Reached from the Step 3 **green-surface** branch, the Step 4 **unrelated-flake** branch +(AFTER that branch has posted its comment), and the Step 3.5 **gate-2 target-focused +fast-path** (2a — while unrelated legs are still pending). Purpose: confirm the SPECIFIC +test(s) this PR was opened to fix now PASS in the PR's own CI, and — only when they do +— transition the draft `[ci-fix]` PR to **ready for review** so a maintainer sees a +validated fix instead of a draft. This is the ONLY place the loop flips draft→ready; it +is a state transition, **never** an approval or a merge (a human still reviews and +merges). Overall red on *unrelated* legs must NOT gate readiness — we validate the fix, +not the base branch's flakiness. + +**Preconditions** (ALL must hold; otherwise record `skipped: readiness N/A PR # +()` and stop this gate): +- `C.isDraft == true` — if the PR is already ready-for-review, the draft→ready + transition is already done; do NOT re-mark **and do NOT re-apply `p/0`**. `p/0` is applied + exactly ONCE, atomically with the draft→ready flip (T3 — all-or-nothing with the 🎯 audit + comment + mark-ready), so an already-ready loop PR that is MISSING `p/0` has almost + certainly had it **removed by a maintainer** de-prioritizing the PR — a pure triage action + the human-engagement guard (which inspects comments/reviews/commits, NOT label changes) + cannot see. Re-adding `p/0` every sweep would fight that maintainer indefinitely, violating + the loop's "never override a human" contract. So the loop does NOT reconcile the label: + record `already-ready PR #` and stop this gate. (Trade-off: on the rare occasion a + transient API error drops `p/0` at flip time *after* the T3 atomic pre-check passed, it is + not auto-re-added — but the PR is still ready-for-review with its 🎯 audit comment, and + re-adding `p/0` is a trivial manual action; that is strictly preferable to steam-rolling a + maintainer's deliberate de-prioritization.) +- The PR is unmistakably THIS workflow's own: `[ci-fix] ` title prefix AND the + `agentic-workflows` label (mirrors the safe-output lock; the compensating scope + control if the v0.80.9 compiler drops the declarative required-*). +- You reached this gate from Step 3 (green), Step 4 (**unrelated-flake**), or the Step 3.5 + gate-2 **target-focused fast-path** (2a) — i.e. the fix is NOT implicated in any red that + has concluded. If Step 4 classified a red as **caused by the fix** (ADVANCE), do NOT run + this gate — advance the attempt instead. + +**T1 — Identify the target test(s).** From the `[ci-scan]` issue signature the fix +addresses (and the PR's own diff), extract the fully-qualified test method name(s) the +fix targets — e.g. `SafeAreaShouldWorkOnAllShellTabs`. For a de-flake it is the +de-flaked test; for a product fix it is the originally-failing test(s). If NO specific +test can be identified (e.g. a product build-break rather than a test failure), this is a +**build-only fix** — there is no single test to validate cross-platform, so validate at +**whole-build** granularity rather than stopping (Step 3 only *comments*; Step 3.6 is the +sole draft→ready flip, so a build-only fix is undrafted HERE or not at all). We only reach +this gate from Step 3 (green) / Step 4 (unrelated-flake) / gate-2a, so the fix is not +implicated in any concluded red; additionally require, via the T2 timeline method on +`C.headSha`, that the primary build pipeline `maui-pr` (def 302) has CONCLUDED with EVERY +one of its platform build legs `succeeded`/`completed` and NONE failed/canceled or still +pending — the build is green on **every** platform, not just the originally-broken one (the +cross-platform guard, applied to the build instead of a test). A build-only fix is undrafted +ONLY on the strength of the auto-running `maui-pr` (def 302) whole-build green, which the +loop CAN observe: if the PR's `[ci-scan]` issue signature or its own diff indicates the +ORIGINATING failure was in a `/azp`-gated pipeline (`maui-pr-uitests` def 313 or +`maui-pr-devicetests` def 314) rather than `maui-pr` (def 302), do NOT undraft on +`maui-pr`-green alone — those pipelines do not auto-run on this PR (GITHUB_TOKEN cannot +trigger them), so a green `maui-pr` build is NOT evidence the gated build break is fixed; +record `skipped: build-only fix PR # targets gated pipeline () not run — +deferring to human` and stop this gate. Otherwise, set `TARGET := "the maui-pr build +(build-only fix — no single target test)"` and proceed to **T3** to mark ready. If any `maui-pr` build leg is still unconcluded, record `skipped: build-only fix PR +# not yet whole-build green (leg(s) pending)` and stop this gate WITHOUT marking +ready (a green subset is not enough — a still-pending build leg could yet fail). + +**T2 — Verify the target test's platform legs are green (anonymous, leg-level).** Per-test +outcomes come from the AzDO **test-results** API (`_apis/test/...`), which is **NOT reachable +anonymously — Hard-Rule 8**: those endpoints 302-redirect to a sign-in page on this runner, so +a `curl` returns an HTML redirect (not JSON) and every target test would look "not executed". +Validate instead at **leg granularity** using ONLY the anonymous `_apis/build/...` timeline +(Hard-Rule 8) — a `succeeded` category leg means every test in that category **that actually +ran** passed on that platform. This is a strong bar **only if the target test genuinely +executes**: a job can go green while one specific test is skipped at runtime (`[Ignore]`, +`Assert.Ignore()`, `Assert.Inconclusive()`, a `Skip=`/conditional `[Fact]`, an `#if`-out, or +an early `return` before the asserts). So before trusting a green leg as proof the target +passed, **confirm from the PR's OWN diff that the fix does NOT skip, ignore, disable, or +short-circuit the target test** (it adds no `[Ignore]`/`[Explicit]`, `Assert.Ignore`/ +`Assert.Inconclusive`, `Skip=`, category-exclusion, `#if`-out, or early `return` around the +target). If the fix could cause the target to be runtime-skipped rather than genuinely pass, +leg-level green is NOT sufficient evidence — record `skipped: target test may be +runtime-skipped by this fix (diff adds ); leg-green insufficient, deferring to human +on PR #` and stop this gate WITHOUT marking ready. Otherwise (the target genuinely runs +and asserts, as a de-flake or product fix does) a green category leg cannot hide a +target-test failure, so treat it as authoritative. + +Map the target test to the CI leg(s) that run it. A leg name encodes platform + test category +(e.g. `Android UITests SafeAreaEdges,Shadow`, `iOS UITests SafeAreaEdges,Shadow`, `macOS +UITests SafeAreaEdges,Shadow`, `Windows UITests SafeAreaEdges,Shadow`). Determine the target +test's UI-test category — its `[Category(UITestCategories.X)]` in the test/HostApp source, +visible in the PR diff or the test file — to know which leg-name substring identifies its legs; +for a device test, the per-platform device-test legs. + +Using the SAME build-discovery as Step 4 (filter AzDO builds by `branchName=refs/pull//merge` +or `sourceVersion == C.headSha`), read each build's **timeline** on `C.headSha` for the +pipeline(s) that RUN the target test — `maui-pr` (def 302) for unit/integration tests, +`maui-pr-uitests` (def 313) for Appium UI tests, `maui-pr-devicetests` (def 314) for device +tests: + +```bash +ORG=dnceng-public; PROJ=public; BUILD= +# Anonymous + Hard-Rule-8-compliant. NEVER call _apis/test/... (it 302-redirects to sign-in). +# Each type=="Job" record is one platform leg: result (succeeded/failed/canceled/null), +# state (completed/inProgress/pending), name (encodes " UITests "): +curl -s "https://dev.azure.com/$ORG/$PROJ/_apis/build/builds/$BUILD/timeline?api-version=7.1" \ + | tee /tmp/gh-aw/agent/timeline_${BUILD}.json \ + | jq -r '.records[] | select(.type=="Job") | "\(.result)\t\(.state)\t\(.name)"' +``` + +(The prefetch already exposes per-leg status in `C.failedLegs` / the checks context, and +`gh pr checks ` lists the same per-leg rows with platform+category in the name — use +whichever is handy; the build timeline is the authoritative cross-check on the exact build.) + +Treat a target test as **VALIDATED-GREEN** only if, on `C.headSha`, the leg that runs its +category is green on **every platform it runs on** — a fix that repairs one platform must not +silently regress the same test's leg on another, so a green leg on the originally-red platform +alone is NOT enough. Across the target pipeline's platform legs (for a UI test: the Android, +iOS, Windows, and macOS/MacCatalyst legs running the target's category; for a device test: each +device-test platform), require ALL of: +- the target's category leg is `result == "succeeded"` **and** `state == "completed"` on + **every** platform that runs it, AND +- that leg is `failed` / `canceled` / aborted on **NO** platform, AND +- **no platform is left unverified.** For each platform family the target pipeline covers, that + platform's category leg must have CONCLUDED (`state == "completed"`) on `C.headSha`. If a + platform simply has no leg for the target's category, the test does not run there — that is + fine, not a gap. But if a platform's category leg has **not concluded** (`state` is + `inProgress` / pending, or an `/azp`-gated `maui-pr-uitests` / `maui-pr-devicetests` leg that + has not been kicked), the test's status on that platform is UNKNOWN → the fix is NOT yet + validated across platforms: record `skipped: target test green on but + not yet verified on (leg(s) pending / need /azp run) on PR #` + and stop this gate WITHOUT marking ready. + +A target test whose category leg never concluded on ANY platform (**not executed** anywhere — +e.g. its `/azp`-gated pipeline has not been kicked) is likewise NOT validated: record `skipped: +target test not yet executed on PR # ( not run — needs /azp run)` and stop +this gate WITHOUT marking ready. Do NOT overclaim — a green *sibling* leg (a different category +on the same platform) is not the target's leg, and a green leg on one platform is not a pass on +the others. + +**T3 — Mark ready + report.** If EVERY target test is VALIDATED-GREEN. The 🎯 comment, +the mark-ready, and the `p/0` label are THREE SEPARATE safe-outputs — the comment +existing does NOT prove the mark-ready took effect, so they are tracked independently: + +- **Comment idempotency (dup-suppress only — never gates the mark-ready).** We only reach + T3 while `C.isDraft == true` (the precondition stops an already-ready PR before here). So + if a prior bot comment for THIS head that carries the leading `🎯` anchor AND the + contiguous phrase `validated green on ` already exists (BOTH T3 variants — + `🎯 Target test validated green on ` and the build-only `🎯 Build validated green on + ` — contain that exact contiguous phrase, so this recognizes both; keying on + `target test` alone would instead miss the build-only comment and re-post it every sweep. + **Require the `🎯` anchor** so the match can NEVER be loosened to a gapped + `validated…green…on` form that would false-positive on the Step 3 `✅ … validated — the + fix's CI is green on ` surface comment and wrongly suppress the audit 🎯), the comment + landed on an earlier sweep but the **mark-ready did NOT take + effect** (the PR is still a draft) — set `SUPPRESS_COMMENT = true` (do not re-post the + duplicate 🎯 comment) but STILL complete the mark-ready + label below. Never treat the + comment's existence as "already marked ready" while the PR is still a draft. Record this + case as `re-marking PR # (🎯 present; mark-ready did not take on a prior sweep)`. +- **Atomicity budget pre-check (Hard-Rule 7).** Determine the outputs this gate will emit: + `mark_pull_request_as_ready_for_review` + `add_labels` always, plus `add_comment` unless + `SUPPRESS_COMMENT`. Verify a free per-run slot remains in EACH bucket you are about to + use. If ANY required bucket is exhausted, emit NONE of them — record `skipped: per-run + cap reached; deferring mark-ready PR # to next cycle` and stop this gate. Never mark a + PR ready (or label it) unless its 🎯 audit comment is GUARANTEED to exist for this exact + head SHA — either posted in this same sweep, or (in the `SUPPRESS_COMMENT` re-marking case) + already present from a prior sweep. The outputs you DO emit either all land or all defer + together; the audit trail must never be absent, but it is NOT re-posted when it already exists. +- **Dry-run gate (Step 0):** if `dry_run == "true"`, emit NOTHING — print the intended + readiness comment and "would mark ready + add p/0" to the run log, tally `dry-run: + would-mark-ready PR #`, and stop. +- Otherwise emit for THIS PR number ``: + 1. `add_comment` (ONLY if not `SUPPRESS_COMMENT`): `🎯 Target test validated green on + — passed on ALL platforms it runs on (, + buildId ). — not + caused by this fix."> Transitioning this PR from draft to ready for review and adding + `p/0`; a maintainer still reviews and merges.` (For a **build-only fix** — the T1 + whole-build fallback, no single target test — phrase the first clause as `🎯 Build + validated green on — the maui-pr build passed on ALL platforms (buildId + ).` instead of naming a test.) + 2. `mark_pull_request_as_ready_for_review` with `reason:` a one-line justification + naming the validated test(s) and ``. + 3. `add_labels` with `labels: ["p/0"]` for PR # — put the now-review-ready fix into + the team's p/0 priority queue so it is triaged, not lost in the draft backlog. (If + the PR somehow already carries `p/0`, this is a harmless no-op.) +- Record `marked-ready PR # (target green on ALL platforms on , + labeled p/0)` and stop. + #### Step 3.5.R — Maintainer change-request response (Track C) Reached from Step 3.5 gate 0. Goal: when an **eligible human reviewer** (Hard-Rule @@ -1383,7 +1669,19 @@ Filed by [`ci-status-fix`](https://github.com/dotnet/maui/blob/main/.github/work ### Step 8 — Per-issue tally + end-of-run summary -Per issue, append one outcome line to `/tmp/gh-aw/agent/coverage.txt`: +Per issue, append **one** outcome line to `/tmp/gh-aw/agent/coverage.txt` — the +terminal outcome for the cycle. When one cycle produces a chained pair — a PR gets a +same-cycle precursor line and **then** a Step 3.6 readiness line (`marked-ready` on the +draft→ready flip, or `already-ready` on a later steady-state sweep of an already-ready +PR) — record ONLY the terminal Step 3.6 readiness line: it supersedes ANY same-cycle +non-readiness precursor for that PR, so an aggregator keying on one-line-per-issue never +double-counts. The precursor may be a Step 3 green line (`surfaced-green` on the first ✅, +or `already-surfaced` when it already exists) OR — when an unrelated-flake red and a +target-green coincide — a Step 4 `annotated-flake` line; either way the readiness line is +terminal, and the superseded precursor's signal survives in the PR's 🎯/♻️ comment so no +information is lost. This covers the flip cycle (`surfaced-green` → `marked-ready`), the +post-flip steady state (`already-surfaced` → `already-ready`), and the flake-coincident +flip (`annotated-flake` → `marked-ready`): ``` # main attempt- @@ -1394,8 +1692,14 @@ Per issue, append one outcome line to `/tmp/gh-aw/agent/coverage.txt`: `advance-PR # attempt /10` (ADVANCE mode — new commit pushed to the existing PR), `surfaced-green PR #` (fix's own CI went green; commented for review, did not advance), `annotated-flake PR #` (red was unrelated flake; -commented, attempt NOT burned), `waiting PR #` (CI not settled yet), -`dry-run: would-`, `skipped: `. (The +commented, attempt NOT burned), `marked-ready PR #` (the specific fixed test +was confirmed green in the PR's own CI; draft flipped to ready for review), +`already-surfaced PR #` (the fix's ✅ green comment already existed this sweep; +re-post suppressed — superseded by the Step 3.6 readiness line when the PR also flips +ready that cycle), `already-ready PR #` (terminal steady state — the PR was flipped +ready on a prior cycle and remains green on an unchanged head; no action taken, +supersedes `already-surfaced`), `waiting PR #` (CI not settled yet), +`dry-run: would-`, `skipped: `. (The `needs-human-PR` outcome is reserved for the deferred hand-off — Step 6 currently records a skip instead, so it is not emitted.)
+()` and stop this gate): +- `C.isDraft == true` — if the PR is already ready-for-review, the draft→ready + transition is already done; do NOT re-mark **and do NOT re-apply `p/0`**. `p/0` is applied + exactly ONCE, atomically with the draft→ready flip (T3 — all-or-nothing with the 🎯 audit + comment + mark-ready), so an already-ready loop PR that is MISSING `p/0` has almost + certainly had it **removed by a maintainer** de-prioritizing the PR — a pure triage action + the human-engagement guard (which inspects comments/reviews/commits, NOT label changes) + cannot see. Re-adding `p/0` every sweep would fight that maintainer indefinitely, violating + the loop's "never override a human" contract. So the loop does NOT reconcile the label: + record `already-ready PR #` and stop this gate. (Trade-off: on the rare occasion a + transient API error drops `p/0` at flip time *after* the T3 atomic pre-check passed, it is + not auto-re-added — but the PR is still ready-for-review with its 🎯 audit comment, and + re-adding `p/0` is a trivial manual action; that is strictly preferable to steam-rolling a + maintainer's deliberate de-prioritization.) +- The PR is unmistakably THIS workflow's own: `[ci-fix] ` title prefix AND the + `agentic-workflows` label (mirrors the safe-output lock; the compensating scope + control if the v0.80.9 compiler drops the declarative required-*). +- You reached this gate from Step 3 (green), Step 4 (**unrelated-flake**), or the Step 3.5 + gate-2 **target-focused fast-path** (2a) — i.e. the fix is NOT implicated in any red that + has concluded. If Step 4 classified a red as **caused by the fix** (ADVANCE), do NOT run + this gate — advance the attempt instead. + +**T1 — Identify the target test(s).** From the `[ci-scan]` issue signature the fix +addresses (and the PR's own diff), extract the fully-qualified test method name(s) the +fix targets — e.g. `SafeAreaShouldWorkOnAllShellTabs`. For a de-flake it is the +de-flaked test; for a product fix it is the originally-failing test(s). If NO specific +test can be identified (e.g. a product build-break rather than a test failure), this is a +**build-only fix** — there is no single test to validate cross-platform, so validate at +**whole-build** granularity rather than stopping (Step 3 only *comments*; Step 3.6 is the +sole draft→ready flip, so a build-only fix is undrafted HERE or not at all). We only reach +this gate from Step 3 (green) / Step 4 (unrelated-flake) / gate-2a, so the fix is not +implicated in any concluded red; additionally require, via the T2 timeline method on +`C.headSha`, that the primary build pipeline `maui-pr` (def 302) has CONCLUDED with EVERY +one of its platform build legs `succeeded`/`completed` and NONE failed/canceled or still +pending — the build is green on **every** platform, not just the originally-broken one (the +cross-platform guard, applied to the build instead of a test). A build-only fix is undrafted +ONLY on the strength of the auto-running `maui-pr` (def 302) whole-build green, which the +loop CAN observe: if the PR's `[ci-scan]` issue signature or its own diff indicates the +ORIGINATING failure was in a `/azp`-gated pipeline (`maui-pr-uitests` def 313 or +`maui-pr-devicetests` def 314) rather than `maui-pr` (def 302), do NOT undraft on +`maui-pr`-green alone — those pipelines do not auto-run on this PR (GITHUB_TOKEN cannot +trigger them), so a green `maui-pr` build is NOT evidence the gated build break is fixed; +record `skipped: build-only fix PR # targets gated pipeline () not run — +deferring to human` and stop this gate. Otherwise, set `TARGET := "the maui-pr build +(build-only fix — no single target test)"` and proceed to **T3** to mark ready. If any `maui-pr` build leg is still unconcluded, record `skipped: build-only fix PR +# not yet whole-build green (leg(s) pending)` and stop this gate WITHOUT marking +ready (a green subset is not enough — a still-pending build leg could yet fail). + +**T2 — Verify the target test's platform legs are green (anonymous, leg-level).** Per-test +outcomes come from the AzDO **test-results** API (`_apis/test/...`), which is **NOT reachable +anonymously — Hard-Rule 8**: those endpoints 302-redirect to a sign-in page on this runner, so +a `curl` returns an HTML redirect (not JSON) and every target test would look "not executed". +Validate instead at **leg granularity** using ONLY the anonymous `_apis/build/...` timeline +(Hard-Rule 8) — a `succeeded` category leg means every test in that category **that actually +ran** passed on that platform. This is a strong bar **only if the target test genuinely +executes**: a job can go green while one specific test is skipped at runtime (`[Ignore]`, +`Assert.Ignore()`, `Assert.Inconclusive()`, a `Skip=`/conditional `[Fact]`, an `#if`-out, or +an early `return` before the asserts). So before trusting a green leg as proof the target +passed, **confirm from the PR's OWN diff that the fix does NOT skip, ignore, disable, or +short-circuit the target test** (it adds no `[Ignore]`/`[Explicit]`, `Assert.Ignore`/ +`Assert.Inconclusive`, `Skip=`, category-exclusion, `#if`-out, or early `return` around the +target). If the fix could cause the target to be runtime-skipped rather than genuinely pass, +leg-level green is NOT sufficient evidence — record `skipped: target test may be +runtime-skipped by this fix (diff adds ); leg-green insufficient, deferring to human +on PR #` and stop this gate WITHOUT marking ready. Otherwise (the target genuinely runs +and asserts, as a de-flake or product fix does) a green category leg cannot hide a +target-test failure, so treat it as authoritative. + +Map the target test to the CI leg(s) that run it. A leg name encodes platform + test category +(e.g. `Android UITests SafeAreaEdges,Shadow`, `iOS UITests SafeAreaEdges,Shadow`, `macOS +UITests SafeAreaEdges,Shadow`, `Windows UITests SafeAreaEdges,Shadow`). Determine the target +test's UI-test category — its `[Category(UITestCategories.X)]` in the test/HostApp source, +visible in the PR diff or the test file — to know which leg-name substring identifies its legs; +for a device test, the per-platform device-test legs. + +Using the SAME build-discovery as Step 4 (filter AzDO builds by `branchName=refs/pull//merge` +or `sourceVersion == C.headSha`), read each build's **timeline** on `C.headSha` for the +pipeline(s) that RUN the target test — `maui-pr` (def 302) for unit/integration tests, +`maui-pr-uitests` (def 313) for Appium UI tests, `maui-pr-devicetests` (def 314) for device +tests: + +```bash +ORG=dnceng-public; PROJ=public; BUILD= +# Anonymous + Hard-Rule-8-compliant. NEVER call _apis/test/... (it 302-redirects to sign-in). +# Each type=="Job" record is one platform leg: result (succeeded/failed/canceled/null), +# state (completed/inProgress/pending), name (encodes " UITests "): +curl -s "https://dev.azure.com/$ORG/$PROJ/_apis/build/builds/$BUILD/timeline?api-version=7.1" \ + | tee /tmp/gh-aw/agent/timeline_${BUILD}.json \ + | jq -r '.records[] | select(.type=="Job") | "\(.result)\t\(.state)\t\(.name)"' +``` + +(The prefetch already exposes per-leg status in `C.failedLegs` / the checks context, and +`gh pr checks ` lists the same per-leg rows with platform+category in the name — use +whichever is handy; the build timeline is the authoritative cross-check on the exact build.) + +Treat a target test as **VALIDATED-GREEN** only if, on `C.headSha`, the leg that runs its +category is green on **every platform it runs on** — a fix that repairs one platform must not +silently regress the same test's leg on another, so a green leg on the originally-red platform +alone is NOT enough. Across the target pipeline's platform legs (for a UI test: the Android, +iOS, Windows, and macOS/MacCatalyst legs running the target's category; for a device test: each +device-test platform), require ALL of: +- the target's category leg is `result == "succeeded"` **and** `state == "completed"` on + **every** platform that runs it, AND +- that leg is `failed` / `canceled` / aborted on **NO** platform, AND +- **no platform is left unverified.** For each platform family the target pipeline covers, that + platform's category leg must have CONCLUDED (`state == "completed"`) on `C.headSha`. If a + platform simply has no leg for the target's category, the test does not run there — that is + fine, not a gap. But if a platform's category leg has **not concluded** (`state` is + `inProgress` / pending, or an `/azp`-gated `maui-pr-uitests` / `maui-pr-devicetests` leg that + has not been kicked), the test's status on that platform is UNKNOWN → the fix is NOT yet + validated across platforms: record `skipped: target test green on but + not yet verified on (leg(s) pending / need /azp run) on PR #` + and stop this gate WITHOUT marking ready. + +A target test whose category leg never concluded on ANY platform (**not executed** anywhere — +e.g. its `/azp`-gated pipeline has not been kicked) is likewise NOT validated: record `skipped: +target test not yet executed on PR # ( not run — needs /azp run)` and stop +this gate WITHOUT marking ready. Do NOT overclaim — a green *sibling* leg (a different category +on the same platform) is not the target's leg, and a green leg on one platform is not a pass on +the others. + +**T3 — Mark ready + report.** If EVERY target test is VALIDATED-GREEN. The 🎯 comment, +the mark-ready, and the `p/0` label are THREE SEPARATE safe-outputs — the comment +existing does NOT prove the mark-ready took effect, so they are tracked independently: + +- **Comment idempotency (dup-suppress only — never gates the mark-ready).** We only reach + T3 while `C.isDraft == true` (the precondition stops an already-ready PR before here). So + if a prior bot comment for THIS head that carries the leading `🎯` anchor AND the + contiguous phrase `validated green on ` already exists (BOTH T3 variants — + `🎯 Target test validated green on ` and the build-only `🎯 Build validated green on + ` — contain that exact contiguous phrase, so this recognizes both; keying on + `target test` alone would instead miss the build-only comment and re-post it every sweep. + **Require the `🎯` anchor** so the match can NEVER be loosened to a gapped + `validated…green…on` form that would false-positive on the Step 3 `✅ … validated — the + fix's CI is green on ` surface comment and wrongly suppress the audit 🎯), the comment + landed on an earlier sweep but the **mark-ready did NOT take + effect** (the PR is still a draft) — set `SUPPRESS_COMMENT = true` (do not re-post the + duplicate 🎯 comment) but STILL complete the mark-ready + label below. Never treat the + comment's existence as "already marked ready" while the PR is still a draft. Record this + case as `re-marking PR # (🎯 present; mark-ready did not take on a prior sweep)`. +- **Atomicity budget pre-check (Hard-Rule 7).** Determine the outputs this gate will emit: + `mark_pull_request_as_ready_for_review` + `add_labels` always, plus `add_comment` unless + `SUPPRESS_COMMENT`. Verify a free per-run slot remains in EACH bucket you are about to + use. If ANY required bucket is exhausted, emit NONE of them — record `skipped: per-run + cap reached; deferring mark-ready PR # to next cycle` and stop this gate. Never mark a + PR ready (or label it) unless its 🎯 audit comment is GUARANTEED to exist for this exact + head SHA — either posted in this same sweep, or (in the `SUPPRESS_COMMENT` re-marking case) + already present from a prior sweep. The outputs you DO emit either all land or all defer + together; the audit trail must never be absent, but it is NOT re-posted when it already exists. +- **Dry-run gate (Step 0):** if `dry_run == "true"`, emit NOTHING — print the intended + readiness comment and "would mark ready + add p/0" to the run log, tally `dry-run: + would-mark-ready PR #`, and stop. +- Otherwise emit for THIS PR number ``: + 1. `add_comment` (ONLY if not `SUPPRESS_COMMENT`): `🎯 Target test validated green on + — passed on ALL platforms it runs on (, + buildId ). — not + caused by this fix."> Transitioning this PR from draft to ready for review and adding + `p/0`; a maintainer still reviews and merges.` (For a **build-only fix** — the T1 + whole-build fallback, no single target test — phrase the first clause as `🎯 Build + validated green on — the maui-pr build passed on ALL platforms (buildId + ).` instead of naming a test.) + 2. `mark_pull_request_as_ready_for_review` with `reason:` a one-line justification + naming the validated test(s) and ``. + 3. `add_labels` with `labels: ["p/0"]` for PR # — put the now-review-ready fix into + the team's p/0 priority queue so it is triaged, not lost in the draft backlog. (If + the PR somehow already carries `p/0`, this is a harmless no-op.) +- Record `marked-ready PR # (target green on ALL platforms on , + labeled p/0)` and stop. + #### Step 3.5.R — Maintainer change-request response (Track C) Reached from Step 3.5 gate 0. Goal: when an **eligible human reviewer** (Hard-Rule @@ -1383,7 +1669,19 @@ Filed by [`ci-status-fix`](https://github.com/dotnet/maui/blob/main/.github/work ### Step 8 — Per-issue tally + end-of-run summary -Per issue, append one outcome line to `/tmp/gh-aw/agent/coverage.txt`: +Per issue, append **one** outcome line to `/tmp/gh-aw/agent/coverage.txt` — the +terminal outcome for the cycle. When one cycle produces a chained pair — a PR gets a +same-cycle precursor line and **then** a Step 3.6 readiness line (`marked-ready` on the +draft→ready flip, or `already-ready` on a later steady-state sweep of an already-ready +PR) — record ONLY the terminal Step 3.6 readiness line: it supersedes ANY same-cycle +non-readiness precursor for that PR, so an aggregator keying on one-line-per-issue never +double-counts. The precursor may be a Step 3 green line (`surfaced-green` on the first ✅, +or `already-surfaced` when it already exists) OR — when an unrelated-flake red and a +target-green coincide — a Step 4 `annotated-flake` line; either way the readiness line is +terminal, and the superseded precursor's signal survives in the PR's 🎯/♻️ comment so no +information is lost. This covers the flip cycle (`surfaced-green` → `marked-ready`), the +post-flip steady state (`already-surfaced` → `already-ready`), and the flake-coincident +flip (`annotated-flake` → `marked-ready`): ``` # main attempt- @@ -1394,8 +1692,14 @@ Per issue, append one outcome line to `/tmp/gh-aw/agent/coverage.txt`: `advance-PR # attempt /10` (ADVANCE mode — new commit pushed to the existing PR), `surfaced-green PR #` (fix's own CI went green; commented for review, did not advance), `annotated-flake PR #` (red was unrelated flake; -commented, attempt NOT burned), `waiting PR #` (CI not settled yet), -`dry-run: would-`, `skipped: `. (The +commented, attempt NOT burned), `marked-ready PR #` (the specific fixed test +was confirmed green in the PR's own CI; draft flipped to ready for review), +`already-surfaced PR #` (the fix's ✅ green comment already existed this sweep; +re-post suppressed — superseded by the Step 3.6 readiness line when the PR also flips +ready that cycle), `already-ready PR #` (terminal steady state — the PR was flipped +ready on a prior cycle and remains green on an unchanged head; no action taken, +supersedes `already-surfaced`), `waiting PR #` (CI not settled yet), +`dry-run: would-`, `skipped: `. (The `needs-human-PR` outcome is reserved for the deferred hand-off — Step 6 currently records a skip instead, so it is not emitted.)
` and stop this gate. (Trade-off: on the rare occasion a + transient API error drops `p/0` at flip time *after* the T3 atomic pre-check passed, it is + not auto-re-added — but the PR is still ready-for-review with its 🎯 audit comment, and + re-adding `p/0` is a trivial manual action; that is strictly preferable to steam-rolling a + maintainer's deliberate de-prioritization.) +- The PR is unmistakably THIS workflow's own: `[ci-fix] ` title prefix AND the + `agentic-workflows` label (mirrors the safe-output lock; the compensating scope + control if the v0.80.9 compiler drops the declarative required-*). +- You reached this gate from Step 3 (green), Step 4 (**unrelated-flake**), or the Step 3.5 + gate-2 **target-focused fast-path** (2a) — i.e. the fix is NOT implicated in any red that + has concluded. If Step 4 classified a red as **caused by the fix** (ADVANCE), do NOT run + this gate — advance the attempt instead. + +**T1 — Identify the target test(s).** From the `[ci-scan]` issue signature the fix +addresses (and the PR's own diff), extract the fully-qualified test method name(s) the +fix targets — e.g. `SafeAreaShouldWorkOnAllShellTabs`. For a de-flake it is the +de-flaked test; for a product fix it is the originally-failing test(s). If NO specific +test can be identified (e.g. a product build-break rather than a test failure), this is a +**build-only fix** — there is no single test to validate cross-platform, so validate at +**whole-build** granularity rather than stopping (Step 3 only *comments*; Step 3.6 is the +sole draft→ready flip, so a build-only fix is undrafted HERE or not at all). We only reach +this gate from Step 3 (green) / Step 4 (unrelated-flake) / gate-2a, so the fix is not +implicated in any concluded red; additionally require, via the T2 timeline method on +`C.headSha`, that the primary build pipeline `maui-pr` (def 302) has CONCLUDED with EVERY +one of its platform build legs `succeeded`/`completed` and NONE failed/canceled or still +pending — the build is green on **every** platform, not just the originally-broken one (the +cross-platform guard, applied to the build instead of a test). A build-only fix is undrafted +ONLY on the strength of the auto-running `maui-pr` (def 302) whole-build green, which the +loop CAN observe: if the PR's `[ci-scan]` issue signature or its own diff indicates the +ORIGINATING failure was in a `/azp`-gated pipeline (`maui-pr-uitests` def 313 or +`maui-pr-devicetests` def 314) rather than `maui-pr` (def 302), do NOT undraft on +`maui-pr`-green alone — those pipelines do not auto-run on this PR (GITHUB_TOKEN cannot +trigger them), so a green `maui-pr` build is NOT evidence the gated build break is fixed; +record `skipped: build-only fix PR #
targets gated pipeline () not run — +deferring to human` and stop this gate. Otherwise, set `TARGET := "the maui-pr build +(build-only fix — no single target test)"` and proceed to **T3** to mark ready. If any `maui-pr` build leg is still unconcluded, record `skipped: build-only fix PR +# not yet whole-build green (leg(s) pending)` and stop this gate WITHOUT marking +ready (a green subset is not enough — a still-pending build leg could yet fail). + +**T2 — Verify the target test's platform legs are green (anonymous, leg-level).** Per-test +outcomes come from the AzDO **test-results** API (`_apis/test/...`), which is **NOT reachable +anonymously — Hard-Rule 8**: those endpoints 302-redirect to a sign-in page on this runner, so +a `curl` returns an HTML redirect (not JSON) and every target test would look "not executed". +Validate instead at **leg granularity** using ONLY the anonymous `_apis/build/...` timeline +(Hard-Rule 8) — a `succeeded` category leg means every test in that category **that actually +ran** passed on that platform. This is a strong bar **only if the target test genuinely +executes**: a job can go green while one specific test is skipped at runtime (`[Ignore]`, +`Assert.Ignore()`, `Assert.Inconclusive()`, a `Skip=`/conditional `[Fact]`, an `#if`-out, or +an early `return` before the asserts). So before trusting a green leg as proof the target +passed, **confirm from the PR's OWN diff that the fix does NOT skip, ignore, disable, or +short-circuit the target test** (it adds no `[Ignore]`/`[Explicit]`, `Assert.Ignore`/ +`Assert.Inconclusive`, `Skip=`, category-exclusion, `#if`-out, or early `return` around the +target). If the fix could cause the target to be runtime-skipped rather than genuinely pass, +leg-level green is NOT sufficient evidence — record `skipped: target test may be +runtime-skipped by this fix (diff adds ); leg-green insufficient, deferring to human +on PR #` and stop this gate WITHOUT marking ready. Otherwise (the target genuinely runs +and asserts, as a de-flake or product fix does) a green category leg cannot hide a +target-test failure, so treat it as authoritative. + +Map the target test to the CI leg(s) that run it. A leg name encodes platform + test category +(e.g. `Android UITests SafeAreaEdges,Shadow`, `iOS UITests SafeAreaEdges,Shadow`, `macOS +UITests SafeAreaEdges,Shadow`, `Windows UITests SafeAreaEdges,Shadow`). Determine the target +test's UI-test category — its `[Category(UITestCategories.X)]` in the test/HostApp source, +visible in the PR diff or the test file — to know which leg-name substring identifies its legs; +for a device test, the per-platform device-test legs. + +Using the SAME build-discovery as Step 4 (filter AzDO builds by `branchName=refs/pull//merge` +or `sourceVersion == C.headSha`), read each build's **timeline** on `C.headSha` for the +pipeline(s) that RUN the target test — `maui-pr` (def 302) for unit/integration tests, +`maui-pr-uitests` (def 313) for Appium UI tests, `maui-pr-devicetests` (def 314) for device +tests: + +```bash +ORG=dnceng-public; PROJ=public; BUILD= +# Anonymous + Hard-Rule-8-compliant. NEVER call _apis/test/... (it 302-redirects to sign-in). +# Each type=="Job" record is one platform leg: result (succeeded/failed/canceled/null), +# state (completed/inProgress/pending), name (encodes " UITests "): +curl -s "https://dev.azure.com/$ORG/$PROJ/_apis/build/builds/$BUILD/timeline?api-version=7.1" \ + | tee /tmp/gh-aw/agent/timeline_${BUILD}.json \ + | jq -r '.records[] | select(.type=="Job") | "\(.result)\t\(.state)\t\(.name)"' +``` + +(The prefetch already exposes per-leg status in `C.failedLegs` / the checks context, and +`gh pr checks ` lists the same per-leg rows with platform+category in the name — use +whichever is handy; the build timeline is the authoritative cross-check on the exact build.) + +Treat a target test as **VALIDATED-GREEN** only if, on `C.headSha`, the leg that runs its +category is green on **every platform it runs on** — a fix that repairs one platform must not +silently regress the same test's leg on another, so a green leg on the originally-red platform +alone is NOT enough. Across the target pipeline's platform legs (for a UI test: the Android, +iOS, Windows, and macOS/MacCatalyst legs running the target's category; for a device test: each +device-test platform), require ALL of: +- the target's category leg is `result == "succeeded"` **and** `state == "completed"` on + **every** platform that runs it, AND +- that leg is `failed` / `canceled` / aborted on **NO** platform, AND +- **no platform is left unverified.** For each platform family the target pipeline covers, that + platform's category leg must have CONCLUDED (`state == "completed"`) on `C.headSha`. If a + platform simply has no leg for the target's category, the test does not run there — that is + fine, not a gap. But if a platform's category leg has **not concluded** (`state` is + `inProgress` / pending, or an `/azp`-gated `maui-pr-uitests` / `maui-pr-devicetests` leg that + has not been kicked), the test's status on that platform is UNKNOWN → the fix is NOT yet + validated across platforms: record `skipped: target test green on but + not yet verified on (leg(s) pending / need /azp run) on PR #` + and stop this gate WITHOUT marking ready. + +A target test whose category leg never concluded on ANY platform (**not executed** anywhere — +e.g. its `/azp`-gated pipeline has not been kicked) is likewise NOT validated: record `skipped: +target test not yet executed on PR # ( not run — needs /azp run)` and stop +this gate WITHOUT marking ready. Do NOT overclaim — a green *sibling* leg (a different category +on the same platform) is not the target's leg, and a green leg on one platform is not a pass on +the others. + +**T3 — Mark ready + report.** If EVERY target test is VALIDATED-GREEN. The 🎯 comment, +the mark-ready, and the `p/0` label are THREE SEPARATE safe-outputs — the comment +existing does NOT prove the mark-ready took effect, so they are tracked independently: + +- **Comment idempotency (dup-suppress only — never gates the mark-ready).** We only reach + T3 while `C.isDraft == true` (the precondition stops an already-ready PR before here). So + if a prior bot comment for THIS head that carries the leading `🎯` anchor AND the + contiguous phrase `validated green on ` already exists (BOTH T3 variants — + `🎯 Target test validated green on ` and the build-only `🎯 Build validated green on + ` — contain that exact contiguous phrase, so this recognizes both; keying on + `target test` alone would instead miss the build-only comment and re-post it every sweep. + **Require the `🎯` anchor** so the match can NEVER be loosened to a gapped + `validated…green…on` form that would false-positive on the Step 3 `✅ … validated — the + fix's CI is green on ` surface comment and wrongly suppress the audit 🎯), the comment + landed on an earlier sweep but the **mark-ready did NOT take + effect** (the PR is still a draft) — set `SUPPRESS_COMMENT = true` (do not re-post the + duplicate 🎯 comment) but STILL complete the mark-ready + label below. Never treat the + comment's existence as "already marked ready" while the PR is still a draft. Record this + case as `re-marking PR # (🎯 present; mark-ready did not take on a prior sweep)`. +- **Atomicity budget pre-check (Hard-Rule 7).** Determine the outputs this gate will emit: + `mark_pull_request_as_ready_for_review` + `add_labels` always, plus `add_comment` unless + `SUPPRESS_COMMENT`. Verify a free per-run slot remains in EACH bucket you are about to + use. If ANY required bucket is exhausted, emit NONE of them — record `skipped: per-run + cap reached; deferring mark-ready PR # to next cycle` and stop this gate. Never mark a + PR ready (or label it) unless its 🎯 audit comment is GUARANTEED to exist for this exact + head SHA — either posted in this same sweep, or (in the `SUPPRESS_COMMENT` re-marking case) + already present from a prior sweep. The outputs you DO emit either all land or all defer + together; the audit trail must never be absent, but it is NOT re-posted when it already exists. +- **Dry-run gate (Step 0):** if `dry_run == "true"`, emit NOTHING — print the intended + readiness comment and "would mark ready + add p/0" to the run log, tally `dry-run: + would-mark-ready PR #`, and stop. +- Otherwise emit for THIS PR number ``: + 1. `add_comment` (ONLY if not `SUPPRESS_COMMENT`): `🎯 Target test validated green on + — passed on ALL platforms it runs on (, + buildId ). — not + caused by this fix."> Transitioning this PR from draft to ready for review and adding + `p/0`; a maintainer still reviews and merges.` (For a **build-only fix** — the T1 + whole-build fallback, no single target test — phrase the first clause as `🎯 Build + validated green on — the maui-pr build passed on ALL platforms (buildId + ).` instead of naming a test.) + 2. `mark_pull_request_as_ready_for_review` with `reason:` a one-line justification + naming the validated test(s) and ``. + 3. `add_labels` with `labels: ["p/0"]` for PR # — put the now-review-ready fix into + the team's p/0 priority queue so it is triaged, not lost in the draft backlog. (If + the PR somehow already carries `p/0`, this is a harmless no-op.) +- Record `marked-ready PR # (target green on ALL platforms on , + labeled p/0)` and stop. + #### Step 3.5.R — Maintainer change-request response (Track C) Reached from Step 3.5 gate 0. Goal: when an **eligible human reviewer** (Hard-Rule @@ -1383,7 +1669,19 @@ Filed by [`ci-status-fix`](https://github.com/dotnet/maui/blob/main/.github/work ### Step 8 — Per-issue tally + end-of-run summary -Per issue, append one outcome line to `/tmp/gh-aw/agent/coverage.txt`: +Per issue, append **one** outcome line to `/tmp/gh-aw/agent/coverage.txt` — the +terminal outcome for the cycle. When one cycle produces a chained pair — a PR gets a +same-cycle precursor line and **then** a Step 3.6 readiness line (`marked-ready` on the +draft→ready flip, or `already-ready` on a later steady-state sweep of an already-ready +PR) — record ONLY the terminal Step 3.6 readiness line: it supersedes ANY same-cycle +non-readiness precursor for that PR, so an aggregator keying on one-line-per-issue never +double-counts. The precursor may be a Step 3 green line (`surfaced-green` on the first ✅, +or `already-surfaced` when it already exists) OR — when an unrelated-flake red and a +target-green coincide — a Step 4 `annotated-flake` line; either way the readiness line is +terminal, and the superseded precursor's signal survives in the PR's 🎯/♻️ comment so no +information is lost. This covers the flip cycle (`surfaced-green` → `marked-ready`), the +post-flip steady state (`already-surfaced` → `already-ready`), and the flake-coincident +flip (`annotated-flake` → `marked-ready`): ``` # main attempt- @@ -1394,8 +1692,14 @@ Per issue, append one outcome line to `/tmp/gh-aw/agent/coverage.txt`: `advance-PR # attempt /10` (ADVANCE mode — new commit pushed to the existing PR), `surfaced-green PR #` (fix's own CI went green; commented for review, did not advance), `annotated-flake PR #` (red was unrelated flake; -commented, attempt NOT burned), `waiting PR #` (CI not settled yet), -`dry-run: would-`, `skipped: `. (The +commented, attempt NOT burned), `marked-ready PR #` (the specific fixed test +was confirmed green in the PR's own CI; draft flipped to ready for review), +`already-surfaced PR #` (the fix's ✅ green comment already existed this sweep; +re-post suppressed — superseded by the Step 3.6 readiness line when the PR also flips +ready that cycle), `already-ready PR #` (terminal steady state — the PR was flipped +ready on a prior cycle and remains green on an unchanged head; no action taken, +supersedes `already-surfaced`), `waiting PR #` (CI not settled yet), +`dry-run: would-`, `skipped: `. (The `needs-human-PR` outcome is reserved for the deferred hand-off — Step 6 currently records a skip instead, so it is not emitted.)
not yet whole-build green (leg(s) pending)` and stop this gate WITHOUT marking +ready (a green subset is not enough — a still-pending build leg could yet fail). + +**T2 — Verify the target test's platform legs are green (anonymous, leg-level).** Per-test +outcomes come from the AzDO **test-results** API (`_apis/test/...`), which is **NOT reachable +anonymously — Hard-Rule 8**: those endpoints 302-redirect to a sign-in page on this runner, so +a `curl` returns an HTML redirect (not JSON) and every target test would look "not executed". +Validate instead at **leg granularity** using ONLY the anonymous `_apis/build/...` timeline +(Hard-Rule 8) — a `succeeded` category leg means every test in that category **that actually +ran** passed on that platform. This is a strong bar **only if the target test genuinely +executes**: a job can go green while one specific test is skipped at runtime (`[Ignore]`, +`Assert.Ignore()`, `Assert.Inconclusive()`, a `Skip=`/conditional `[Fact]`, an `#if`-out, or +an early `return` before the asserts). So before trusting a green leg as proof the target +passed, **confirm from the PR's OWN diff that the fix does NOT skip, ignore, disable, or +short-circuit the target test** (it adds no `[Ignore]`/`[Explicit]`, `Assert.Ignore`/ +`Assert.Inconclusive`, `Skip=`, category-exclusion, `#if`-out, or early `return` around the +target). If the fix could cause the target to be runtime-skipped rather than genuinely pass, +leg-level green is NOT sufficient evidence — record `skipped: target test may be +runtime-skipped by this fix (diff adds ); leg-green insufficient, deferring to human +on PR #` and stop this gate WITHOUT marking ready. Otherwise (the target genuinely runs +and asserts, as a de-flake or product fix does) a green category leg cannot hide a +target-test failure, so treat it as authoritative. + +Map the target test to the CI leg(s) that run it. A leg name encodes platform + test category +(e.g. `Android UITests SafeAreaEdges,Shadow`, `iOS UITests SafeAreaEdges,Shadow`, `macOS +UITests SafeAreaEdges,Shadow`, `Windows UITests SafeAreaEdges,Shadow`). Determine the target +test's UI-test category — its `[Category(UITestCategories.X)]` in the test/HostApp source, +visible in the PR diff or the test file — to know which leg-name substring identifies its legs; +for a device test, the per-platform device-test legs. + +Using the SAME build-discovery as Step 4 (filter AzDO builds by `branchName=refs/pull//merge` +or `sourceVersion == C.headSha`), read each build's **timeline** on `C.headSha` for the +pipeline(s) that RUN the target test — `maui-pr` (def 302) for unit/integration tests, +`maui-pr-uitests` (def 313) for Appium UI tests, `maui-pr-devicetests` (def 314) for device +tests: + +```bash +ORG=dnceng-public; PROJ=public; BUILD= +# Anonymous + Hard-Rule-8-compliant. NEVER call _apis/test/... (it 302-redirects to sign-in). +# Each type=="Job" record is one platform leg: result (succeeded/failed/canceled/null), +# state (completed/inProgress/pending), name (encodes " UITests "): +curl -s "https://dev.azure.com/$ORG/$PROJ/_apis/build/builds/$BUILD/timeline?api-version=7.1" \ + | tee /tmp/gh-aw/agent/timeline_${BUILD}.json \ + | jq -r '.records[] | select(.type=="Job") | "\(.result)\t\(.state)\t\(.name)"' +``` + +(The prefetch already exposes per-leg status in `C.failedLegs` / the checks context, and +`gh pr checks ` lists the same per-leg rows with platform+category in the name — use +whichever is handy; the build timeline is the authoritative cross-check on the exact build.) + +Treat a target test as **VALIDATED-GREEN** only if, on `C.headSha`, the leg that runs its +category is green on **every platform it runs on** — a fix that repairs one platform must not +silently regress the same test's leg on another, so a green leg on the originally-red platform +alone is NOT enough. Across the target pipeline's platform legs (for a UI test: the Android, +iOS, Windows, and macOS/MacCatalyst legs running the target's category; for a device test: each +device-test platform), require ALL of: +- the target's category leg is `result == "succeeded"` **and** `state == "completed"` on + **every** platform that runs it, AND +- that leg is `failed` / `canceled` / aborted on **NO** platform, AND +- **no platform is left unverified.** For each platform family the target pipeline covers, that + platform's category leg must have CONCLUDED (`state == "completed"`) on `C.headSha`. If a + platform simply has no leg for the target's category, the test does not run there — that is + fine, not a gap. But if a platform's category leg has **not concluded** (`state` is + `inProgress` / pending, or an `/azp`-gated `maui-pr-uitests` / `maui-pr-devicetests` leg that + has not been kicked), the test's status on that platform is UNKNOWN → the fix is NOT yet + validated across platforms: record `skipped: target test green on but + not yet verified on (leg(s) pending / need /azp run) on PR #` + and stop this gate WITHOUT marking ready. + +A target test whose category leg never concluded on ANY platform (**not executed** anywhere — +e.g. its `/azp`-gated pipeline has not been kicked) is likewise NOT validated: record `skipped: +target test not yet executed on PR # ( not run — needs /azp run)` and stop +this gate WITHOUT marking ready. Do NOT overclaim — a green *sibling* leg (a different category +on the same platform) is not the target's leg, and a green leg on one platform is not a pass on +the others. + +**T3 — Mark ready + report.** If EVERY target test is VALIDATED-GREEN. The 🎯 comment, +the mark-ready, and the `p/0` label are THREE SEPARATE safe-outputs — the comment +existing does NOT prove the mark-ready took effect, so they are tracked independently: + +- **Comment idempotency (dup-suppress only — never gates the mark-ready).** We only reach + T3 while `C.isDraft == true` (the precondition stops an already-ready PR before here). So + if a prior bot comment for THIS head that carries the leading `🎯` anchor AND the + contiguous phrase `validated green on ` already exists (BOTH T3 variants — + `🎯 Target test validated green on ` and the build-only `🎯 Build validated green on + ` — contain that exact contiguous phrase, so this recognizes both; keying on + `target test` alone would instead miss the build-only comment and re-post it every sweep. + **Require the `🎯` anchor** so the match can NEVER be loosened to a gapped + `validated…green…on` form that would false-positive on the Step 3 `✅ … validated — the + fix's CI is green on ` surface comment and wrongly suppress the audit 🎯), the comment + landed on an earlier sweep but the **mark-ready did NOT take + effect** (the PR is still a draft) — set `SUPPRESS_COMMENT = true` (do not re-post the + duplicate 🎯 comment) but STILL complete the mark-ready + label below. Never treat the + comment's existence as "already marked ready" while the PR is still a draft. Record this + case as `re-marking PR # (🎯 present; mark-ready did not take on a prior sweep)`. +- **Atomicity budget pre-check (Hard-Rule 7).** Determine the outputs this gate will emit: + `mark_pull_request_as_ready_for_review` + `add_labels` always, plus `add_comment` unless + `SUPPRESS_COMMENT`. Verify a free per-run slot remains in EACH bucket you are about to + use. If ANY required bucket is exhausted, emit NONE of them — record `skipped: per-run + cap reached; deferring mark-ready PR # to next cycle` and stop this gate. Never mark a + PR ready (or label it) unless its 🎯 audit comment is GUARANTEED to exist for this exact + head SHA — either posted in this same sweep, or (in the `SUPPRESS_COMMENT` re-marking case) + already present from a prior sweep. The outputs you DO emit either all land or all defer + together; the audit trail must never be absent, but it is NOT re-posted when it already exists. +- **Dry-run gate (Step 0):** if `dry_run == "true"`, emit NOTHING — print the intended + readiness comment and "would mark ready + add p/0" to the run log, tally `dry-run: + would-mark-ready PR #`, and stop. +- Otherwise emit for THIS PR number ``: + 1. `add_comment` (ONLY if not `SUPPRESS_COMMENT`): `🎯 Target test validated green on + — passed on ALL platforms it runs on (, + buildId ). — not + caused by this fix."> Transitioning this PR from draft to ready for review and adding + `p/0`; a maintainer still reviews and merges.` (For a **build-only fix** — the T1 + whole-build fallback, no single target test — phrase the first clause as `🎯 Build + validated green on — the maui-pr build passed on ALL platforms (buildId + ).` instead of naming a test.) + 2. `mark_pull_request_as_ready_for_review` with `reason:` a one-line justification + naming the validated test(s) and ``. + 3. `add_labels` with `labels: ["p/0"]` for PR # — put the now-review-ready fix into + the team's p/0 priority queue so it is triaged, not lost in the draft backlog. (If + the PR somehow already carries `p/0`, this is a harmless no-op.) +- Record `marked-ready PR # (target green on ALL platforms on , + labeled p/0)` and stop. + #### Step 3.5.R — Maintainer change-request response (Track C) Reached from Step 3.5 gate 0. Goal: when an **eligible human reviewer** (Hard-Rule @@ -1383,7 +1669,19 @@ Filed by [`ci-status-fix`](https://github.com/dotnet/maui/blob/main/.github/work ### Step 8 — Per-issue tally + end-of-run summary -Per issue, append one outcome line to `/tmp/gh-aw/agent/coverage.txt`: +Per issue, append **one** outcome line to `/tmp/gh-aw/agent/coverage.txt` — the +terminal outcome for the cycle. When one cycle produces a chained pair — a PR gets a +same-cycle precursor line and **then** a Step 3.6 readiness line (`marked-ready` on the +draft→ready flip, or `already-ready` on a later steady-state sweep of an already-ready +PR) — record ONLY the terminal Step 3.6 readiness line: it supersedes ANY same-cycle +non-readiness precursor for that PR, so an aggregator keying on one-line-per-issue never +double-counts. The precursor may be a Step 3 green line (`surfaced-green` on the first ✅, +or `already-surfaced` when it already exists) OR — when an unrelated-flake red and a +target-green coincide — a Step 4 `annotated-flake` line; either way the readiness line is +terminal, and the superseded precursor's signal survives in the PR's 🎯/♻️ comment so no +information is lost. This covers the flip cycle (`surfaced-green` → `marked-ready`), the +post-flip steady state (`already-surfaced` → `already-ready`), and the flake-coincident +flip (`annotated-flake` → `marked-ready`): ``` # main attempt- @@ -1394,8 +1692,14 @@ Per issue, append one outcome line to `/tmp/gh-aw/agent/coverage.txt`: `advance-PR # attempt /10` (ADVANCE mode — new commit pushed to the existing PR), `surfaced-green PR #` (fix's own CI went green; commented for review, did not advance), `annotated-flake PR #` (red was unrelated flake; -commented, attempt NOT burned), `waiting PR #` (CI not settled yet), -`dry-run: would-`, `skipped: `. (The +commented, attempt NOT burned), `marked-ready PR #` (the specific fixed test +was confirmed green in the PR's own CI; draft flipped to ready for review), +`already-surfaced PR #` (the fix's ✅ green comment already existed this sweep; +re-post suppressed — superseded by the Step 3.6 readiness line when the PR also flips +ready that cycle), `already-ready PR #` (terminal steady state — the PR was flipped +ready on a prior cycle and remains green on an unchanged head; no action taken, +supersedes `already-surfaced`), `waiting PR #` (CI not settled yet), +`dry-run: would-`, `skipped: `. (The `needs-human-PR` outcome is reserved for the deferred hand-off — Step 6 currently records a skip instead, so it is not emitted.)
/merge` +or `sourceVersion == C.headSha`), read each build's **timeline** on `C.headSha` for the +pipeline(s) that RUN the target test — `maui-pr` (def 302) for unit/integration tests, +`maui-pr-uitests` (def 313) for Appium UI tests, `maui-pr-devicetests` (def 314) for device +tests: + +```bash +ORG=dnceng-public; PROJ=public; BUILD= +# Anonymous + Hard-Rule-8-compliant. NEVER call _apis/test/... (it 302-redirects to sign-in). +# Each type=="Job" record is one platform leg: result (succeeded/failed/canceled/null), +# state (completed/inProgress/pending), name (encodes " UITests "): +curl -s "https://dev.azure.com/$ORG/$PROJ/_apis/build/builds/$BUILD/timeline?api-version=7.1" \ + | tee /tmp/gh-aw/agent/timeline_${BUILD}.json \ + | jq -r '.records[] | select(.type=="Job") | "\(.result)\t\(.state)\t\(.name)"' +``` + +(The prefetch already exposes per-leg status in `C.failedLegs` / the checks context, and +`gh pr checks ` lists the same per-leg rows with platform+category in the name — use +whichever is handy; the build timeline is the authoritative cross-check on the exact build.) + +Treat a target test as **VALIDATED-GREEN** only if, on `C.headSha`, the leg that runs its +category is green on **every platform it runs on** — a fix that repairs one platform must not +silently regress the same test's leg on another, so a green leg on the originally-red platform +alone is NOT enough. Across the target pipeline's platform legs (for a UI test: the Android, +iOS, Windows, and macOS/MacCatalyst legs running the target's category; for a device test: each +device-test platform), require ALL of: +- the target's category leg is `result == "succeeded"` **and** `state == "completed"` on + **every** platform that runs it, AND +- that leg is `failed` / `canceled` / aborted on **NO** platform, AND +- **no platform is left unverified.** For each platform family the target pipeline covers, that + platform's category leg must have CONCLUDED (`state == "completed"`) on `C.headSha`. If a + platform simply has no leg for the target's category, the test does not run there — that is + fine, not a gap. But if a platform's category leg has **not concluded** (`state` is + `inProgress` / pending, or an `/azp`-gated `maui-pr-uitests` / `maui-pr-devicetests` leg that + has not been kicked), the test's status on that platform is UNKNOWN → the fix is NOT yet + validated across platforms: record `skipped: target test green on but + not yet verified on (leg(s) pending / need /azp run) on PR #` + and stop this gate WITHOUT marking ready. + +A target test whose category leg never concluded on ANY platform (**not executed** anywhere — +e.g. its `/azp`-gated pipeline has not been kicked) is likewise NOT validated: record `skipped: +target test not yet executed on PR # ( not run — needs /azp run)` and stop +this gate WITHOUT marking ready. Do NOT overclaim — a green *sibling* leg (a different category +on the same platform) is not the target's leg, and a green leg on one platform is not a pass on +the others. + +**T3 — Mark ready + report.** If EVERY target test is VALIDATED-GREEN. The 🎯 comment, +the mark-ready, and the `p/0` label are THREE SEPARATE safe-outputs — the comment +existing does NOT prove the mark-ready took effect, so they are tracked independently: + +- **Comment idempotency (dup-suppress only — never gates the mark-ready).** We only reach + T3 while `C.isDraft == true` (the precondition stops an already-ready PR before here). So + if a prior bot comment for THIS head that carries the leading `🎯` anchor AND the + contiguous phrase `validated green on ` already exists (BOTH T3 variants — + `🎯 Target test validated green on ` and the build-only `🎯 Build validated green on + ` — contain that exact contiguous phrase, so this recognizes both; keying on + `target test` alone would instead miss the build-only comment and re-post it every sweep. + **Require the `🎯` anchor** so the match can NEVER be loosened to a gapped + `validated…green…on` form that would false-positive on the Step 3 `✅ … validated — the + fix's CI is green on ` surface comment and wrongly suppress the audit 🎯), the comment + landed on an earlier sweep but the **mark-ready did NOT take + effect** (the PR is still a draft) — set `SUPPRESS_COMMENT = true` (do not re-post the + duplicate 🎯 comment) but STILL complete the mark-ready + label below. Never treat the + comment's existence as "already marked ready" while the PR is still a draft. Record this + case as `re-marking PR # (🎯 present; mark-ready did not take on a prior sweep)`. +- **Atomicity budget pre-check (Hard-Rule 7).** Determine the outputs this gate will emit: + `mark_pull_request_as_ready_for_review` + `add_labels` always, plus `add_comment` unless + `SUPPRESS_COMMENT`. Verify a free per-run slot remains in EACH bucket you are about to + use. If ANY required bucket is exhausted, emit NONE of them — record `skipped: per-run + cap reached; deferring mark-ready PR # to next cycle` and stop this gate. Never mark a + PR ready (or label it) unless its 🎯 audit comment is GUARANTEED to exist for this exact + head SHA — either posted in this same sweep, or (in the `SUPPRESS_COMMENT` re-marking case) + already present from a prior sweep. The outputs you DO emit either all land or all defer + together; the audit trail must never be absent, but it is NOT re-posted when it already exists. +- **Dry-run gate (Step 0):** if `dry_run == "true"`, emit NOTHING — print the intended + readiness comment and "would mark ready + add p/0" to the run log, tally `dry-run: + would-mark-ready PR #`, and stop. +- Otherwise emit for THIS PR number ``: + 1. `add_comment` (ONLY if not `SUPPRESS_COMMENT`): `🎯 Target test validated green on + — passed on ALL platforms it runs on (, + buildId ). — not + caused by this fix."> Transitioning this PR from draft to ready for review and adding + `p/0`; a maintainer still reviews and merges.` (For a **build-only fix** — the T1 + whole-build fallback, no single target test — phrase the first clause as `🎯 Build + validated green on — the maui-pr build passed on ALL platforms (buildId + ).` instead of naming a test.) + 2. `mark_pull_request_as_ready_for_review` with `reason:` a one-line justification + naming the validated test(s) and ``. + 3. `add_labels` with `labels: ["p/0"]` for PR # — put the now-review-ready fix into + the team's p/0 priority queue so it is triaged, not lost in the draft backlog. (If + the PR somehow already carries `p/0`, this is a harmless no-op.) +- Record `marked-ready PR # (target green on ALL platforms on , + labeled p/0)` and stop. + #### Step 3.5.R — Maintainer change-request response (Track C) Reached from Step 3.5 gate 0. Goal: when an **eligible human reviewer** (Hard-Rule @@ -1383,7 +1669,19 @@ Filed by [`ci-status-fix`](https://github.com/dotnet/maui/blob/main/.github/work ### Step 8 — Per-issue tally + end-of-run summary -Per issue, append one outcome line to `/tmp/gh-aw/agent/coverage.txt`: +Per issue, append **one** outcome line to `/tmp/gh-aw/agent/coverage.txt` — the +terminal outcome for the cycle. When one cycle produces a chained pair — a PR gets a +same-cycle precursor line and **then** a Step 3.6 readiness line (`marked-ready` on the +draft→ready flip, or `already-ready` on a later steady-state sweep of an already-ready +PR) — record ONLY the terminal Step 3.6 readiness line: it supersedes ANY same-cycle +non-readiness precursor for that PR, so an aggregator keying on one-line-per-issue never +double-counts. The precursor may be a Step 3 green line (`surfaced-green` on the first ✅, +or `already-surfaced` when it already exists) OR — when an unrelated-flake red and a +target-green coincide — a Step 4 `annotated-flake` line; either way the readiness line is +terminal, and the superseded precursor's signal survives in the PR's 🎯/♻️ comment so no +information is lost. This covers the flip cycle (`surfaced-green` → `marked-ready`), the +post-flip steady state (`already-surfaced` → `already-ready`), and the flake-coincident +flip (`annotated-flake` → `marked-ready`): ``` # main attempt- @@ -1394,8 +1692,14 @@ Per issue, append one outcome line to `/tmp/gh-aw/agent/coverage.txt`: `advance-PR # attempt /10` (ADVANCE mode — new commit pushed to the existing PR), `surfaced-green PR #` (fix's own CI went green; commented for review, did not advance), `annotated-flake PR #` (red was unrelated flake; -commented, attempt NOT burned), `waiting PR #` (CI not settled yet), -`dry-run: would-`, `skipped: `. (The +commented, attempt NOT burned), `marked-ready PR #` (the specific fixed test +was confirmed green in the PR's own CI; draft flipped to ready for review), +`already-surfaced PR #` (the fix's ✅ green comment already existed this sweep; +re-post suppressed — superseded by the Step 3.6 readiness line when the PR also flips +ready that cycle), `already-ready PR #` (terminal steady state — the PR was flipped +ready on a prior cycle and remains green on an unchanged head; no action taken, +supersedes `already-surfaced`), `waiting PR #` (CI not settled yet), +`dry-run: would-`, `skipped: `. (The `needs-human-PR` outcome is reserved for the deferred hand-off — Step 6 currently records a skip instead, so it is not emitted.)
` lists the same per-leg rows with platform+category in the name — use +whichever is handy; the build timeline is the authoritative cross-check on the exact build.) + +Treat a target test as **VALIDATED-GREEN** only if, on `C.headSha`, the leg that runs its +category is green on **every platform it runs on** — a fix that repairs one platform must not +silently regress the same test's leg on another, so a green leg on the originally-red platform +alone is NOT enough. Across the target pipeline's platform legs (for a UI test: the Android, +iOS, Windows, and macOS/MacCatalyst legs running the target's category; for a device test: each +device-test platform), require ALL of: +- the target's category leg is `result == "succeeded"` **and** `state == "completed"` on + **every** platform that runs it, AND +- that leg is `failed` / `canceled` / aborted on **NO** platform, AND +- **no platform is left unverified.** For each platform family the target pipeline covers, that + platform's category leg must have CONCLUDED (`state == "completed"`) on `C.headSha`. If a + platform simply has no leg for the target's category, the test does not run there — that is + fine, not a gap. But if a platform's category leg has **not concluded** (`state` is + `inProgress` / pending, or an `/azp`-gated `maui-pr-uitests` / `maui-pr-devicetests` leg that + has not been kicked), the test's status on that platform is UNKNOWN → the fix is NOT yet + validated across platforms: record `skipped: target test green on but + not yet verified on (leg(s) pending / need /azp run) on PR #` + and stop this gate WITHOUT marking ready. + +A target test whose category leg never concluded on ANY platform (**not executed** anywhere — +e.g. its `/azp`-gated pipeline has not been kicked) is likewise NOT validated: record `skipped: +target test not yet executed on PR # ( not run — needs /azp run)` and stop +this gate WITHOUT marking ready. Do NOT overclaim — a green *sibling* leg (a different category +on the same platform) is not the target's leg, and a green leg on one platform is not a pass on +the others. + +**T3 — Mark ready + report.** If EVERY target test is VALIDATED-GREEN. The 🎯 comment, +the mark-ready, and the `p/0` label are THREE SEPARATE safe-outputs — the comment +existing does NOT prove the mark-ready took effect, so they are tracked independently: + +- **Comment idempotency (dup-suppress only — never gates the mark-ready).** We only reach + T3 while `C.isDraft == true` (the precondition stops an already-ready PR before here). So + if a prior bot comment for THIS head that carries the leading `🎯` anchor AND the + contiguous phrase `validated green on ` already exists (BOTH T3 variants — + `🎯 Target test validated green on ` and the build-only `🎯 Build validated green on + ` — contain that exact contiguous phrase, so this recognizes both; keying on + `target test` alone would instead miss the build-only comment and re-post it every sweep. + **Require the `🎯` anchor** so the match can NEVER be loosened to a gapped + `validated…green…on` form that would false-positive on the Step 3 `✅ … validated — the + fix's CI is green on ` surface comment and wrongly suppress the audit 🎯), the comment + landed on an earlier sweep but the **mark-ready did NOT take + effect** (the PR is still a draft) — set `SUPPRESS_COMMENT = true` (do not re-post the + duplicate 🎯 comment) but STILL complete the mark-ready + label below. Never treat the + comment's existence as "already marked ready" while the PR is still a draft. Record this + case as `re-marking PR # (🎯 present; mark-ready did not take on a prior sweep)`. +- **Atomicity budget pre-check (Hard-Rule 7).** Determine the outputs this gate will emit: + `mark_pull_request_as_ready_for_review` + `add_labels` always, plus `add_comment` unless + `SUPPRESS_COMMENT`. Verify a free per-run slot remains in EACH bucket you are about to + use. If ANY required bucket is exhausted, emit NONE of them — record `skipped: per-run + cap reached; deferring mark-ready PR # to next cycle` and stop this gate. Never mark a + PR ready (or label it) unless its 🎯 audit comment is GUARANTEED to exist for this exact + head SHA — either posted in this same sweep, or (in the `SUPPRESS_COMMENT` re-marking case) + already present from a prior sweep. The outputs you DO emit either all land or all defer + together; the audit trail must never be absent, but it is NOT re-posted when it already exists. +- **Dry-run gate (Step 0):** if `dry_run == "true"`, emit NOTHING — print the intended + readiness comment and "would mark ready + add p/0" to the run log, tally `dry-run: + would-mark-ready PR #`, and stop. +- Otherwise emit for THIS PR number ``: + 1. `add_comment` (ONLY if not `SUPPRESS_COMMENT`): `🎯 Target test validated green on + — passed on ALL platforms it runs on (, + buildId ). — not + caused by this fix."> Transitioning this PR from draft to ready for review and adding + `p/0`; a maintainer still reviews and merges.` (For a **build-only fix** — the T1 + whole-build fallback, no single target test — phrase the first clause as `🎯 Build + validated green on — the maui-pr build passed on ALL platforms (buildId + ).` instead of naming a test.) + 2. `mark_pull_request_as_ready_for_review` with `reason:` a one-line justification + naming the validated test(s) and ``. + 3. `add_labels` with `labels: ["p/0"]` for PR # — put the now-review-ready fix into + the team's p/0 priority queue so it is triaged, not lost in the draft backlog. (If + the PR somehow already carries `p/0`, this is a harmless no-op.) +- Record `marked-ready PR # (target green on ALL platforms on , + labeled p/0)` and stop. + #### Step 3.5.R — Maintainer change-request response (Track C) Reached from Step 3.5 gate 0. Goal: when an **eligible human reviewer** (Hard-Rule @@ -1383,7 +1669,19 @@ Filed by [`ci-status-fix`](https://github.com/dotnet/maui/blob/main/.github/work ### Step 8 — Per-issue tally + end-of-run summary -Per issue, append one outcome line to `/tmp/gh-aw/agent/coverage.txt`: +Per issue, append **one** outcome line to `/tmp/gh-aw/agent/coverage.txt` — the +terminal outcome for the cycle. When one cycle produces a chained pair — a PR gets a +same-cycle precursor line and **then** a Step 3.6 readiness line (`marked-ready` on the +draft→ready flip, or `already-ready` on a later steady-state sweep of an already-ready +PR) — record ONLY the terminal Step 3.6 readiness line: it supersedes ANY same-cycle +non-readiness precursor for that PR, so an aggregator keying on one-line-per-issue never +double-counts. The precursor may be a Step 3 green line (`surfaced-green` on the first ✅, +or `already-surfaced` when it already exists) OR — when an unrelated-flake red and a +target-green coincide — a Step 4 `annotated-flake` line; either way the readiness line is +terminal, and the superseded precursor's signal survives in the PR's 🎯/♻️ comment so no +information is lost. This covers the flip cycle (`surfaced-green` → `marked-ready`), the +post-flip steady state (`already-surfaced` → `already-ready`), and the flake-coincident +flip (`annotated-flake` → `marked-ready`): ``` # main attempt- @@ -1394,8 +1692,14 @@ Per issue, append one outcome line to `/tmp/gh-aw/agent/coverage.txt`: `advance-PR # attempt /10` (ADVANCE mode — new commit pushed to the existing PR), `surfaced-green PR #` (fix's own CI went green; commented for review, did not advance), `annotated-flake PR #` (red was unrelated flake; -commented, attempt NOT burned), `waiting PR #` (CI not settled yet), -`dry-run: would-`, `skipped: `. (The +commented, attempt NOT burned), `marked-ready PR #` (the specific fixed test +was confirmed green in the PR's own CI; draft flipped to ready for review), +`already-surfaced PR #` (the fix's ✅ green comment already existed this sweep; +re-post suppressed — superseded by the Step 3.6 readiness line when the PR also flips +ready that cycle), `already-ready PR #` (terminal steady state — the PR was flipped +ready on a prior cycle and remains green on an unchanged head; no action taken, +supersedes `already-surfaced`), `waiting PR #` (CI not settled yet), +`dry-run: would-`, `skipped: `. (The `needs-human-PR` outcome is reserved for the deferred hand-off — Step 6 currently records a skip instead, so it is not emitted.)
` + and stop this gate WITHOUT marking ready. + +A target test whose category leg never concluded on ANY platform (**not executed** anywhere — +e.g. its `/azp`-gated pipeline has not been kicked) is likewise NOT validated: record `skipped: +target test not yet executed on PR # ( not run — needs /azp run)` and stop +this gate WITHOUT marking ready. Do NOT overclaim — a green *sibling* leg (a different category +on the same platform) is not the target's leg, and a green leg on one platform is not a pass on +the others. + +**T3 — Mark ready + report.** If EVERY target test is VALIDATED-GREEN. The 🎯 comment, +the mark-ready, and the `p/0` label are THREE SEPARATE safe-outputs — the comment +existing does NOT prove the mark-ready took effect, so they are tracked independently: + +- **Comment idempotency (dup-suppress only — never gates the mark-ready).** We only reach + T3 while `C.isDraft == true` (the precondition stops an already-ready PR before here). So + if a prior bot comment for THIS head that carries the leading `🎯` anchor AND the + contiguous phrase `validated green on ` already exists (BOTH T3 variants — + `🎯 Target test validated green on ` and the build-only `🎯 Build validated green on + ` — contain that exact contiguous phrase, so this recognizes both; keying on + `target test` alone would instead miss the build-only comment and re-post it every sweep. + **Require the `🎯` anchor** so the match can NEVER be loosened to a gapped + `validated…green…on` form that would false-positive on the Step 3 `✅ … validated — the + fix's CI is green on ` surface comment and wrongly suppress the audit 🎯), the comment + landed on an earlier sweep but the **mark-ready did NOT take + effect** (the PR is still a draft) — set `SUPPRESS_COMMENT = true` (do not re-post the + duplicate 🎯 comment) but STILL complete the mark-ready + label below. Never treat the + comment's existence as "already marked ready" while the PR is still a draft. Record this + case as `re-marking PR # (🎯 present; mark-ready did not take on a prior sweep)`. +- **Atomicity budget pre-check (Hard-Rule 7).** Determine the outputs this gate will emit: + `mark_pull_request_as_ready_for_review` + `add_labels` always, plus `add_comment` unless + `SUPPRESS_COMMENT`. Verify a free per-run slot remains in EACH bucket you are about to + use. If ANY required bucket is exhausted, emit NONE of them — record `skipped: per-run + cap reached; deferring mark-ready PR # to next cycle` and stop this gate. Never mark a + PR ready (or label it) unless its 🎯 audit comment is GUARANTEED to exist for this exact + head SHA — either posted in this same sweep, or (in the `SUPPRESS_COMMENT` re-marking case) + already present from a prior sweep. The outputs you DO emit either all land or all defer + together; the audit trail must never be absent, but it is NOT re-posted when it already exists. +- **Dry-run gate (Step 0):** if `dry_run == "true"`, emit NOTHING — print the intended + readiness comment and "would mark ready + add p/0" to the run log, tally `dry-run: + would-mark-ready PR #`, and stop. +- Otherwise emit for THIS PR number ``: + 1. `add_comment` (ONLY if not `SUPPRESS_COMMENT`): `🎯 Target test validated green on + — passed on ALL platforms it runs on (, + buildId ). — not + caused by this fix."> Transitioning this PR from draft to ready for review and adding + `p/0`; a maintainer still reviews and merges.` (For a **build-only fix** — the T1 + whole-build fallback, no single target test — phrase the first clause as `🎯 Build + validated green on — the maui-pr build passed on ALL platforms (buildId + ).` instead of naming a test.) + 2. `mark_pull_request_as_ready_for_review` with `reason:` a one-line justification + naming the validated test(s) and ``. + 3. `add_labels` with `labels: ["p/0"]` for PR # — put the now-review-ready fix into + the team's p/0 priority queue so it is triaged, not lost in the draft backlog. (If + the PR somehow already carries `p/0`, this is a harmless no-op.) +- Record `marked-ready PR # (target green on ALL platforms on , + labeled p/0)` and stop. + #### Step 3.5.R — Maintainer change-request response (Track C) Reached from Step 3.5 gate 0. Goal: when an **eligible human reviewer** (Hard-Rule @@ -1383,7 +1669,19 @@ Filed by [`ci-status-fix`](https://github.com/dotnet/maui/blob/main/.github/work ### Step 8 — Per-issue tally + end-of-run summary -Per issue, append one outcome line to `/tmp/gh-aw/agent/coverage.txt`: +Per issue, append **one** outcome line to `/tmp/gh-aw/agent/coverage.txt` — the +terminal outcome for the cycle. When one cycle produces a chained pair — a PR gets a +same-cycle precursor line and **then** a Step 3.6 readiness line (`marked-ready` on the +draft→ready flip, or `already-ready` on a later steady-state sweep of an already-ready +PR) — record ONLY the terminal Step 3.6 readiness line: it supersedes ANY same-cycle +non-readiness precursor for that PR, so an aggregator keying on one-line-per-issue never +double-counts. The precursor may be a Step 3 green line (`surfaced-green` on the first ✅, +or `already-surfaced` when it already exists) OR — when an unrelated-flake red and a +target-green coincide — a Step 4 `annotated-flake` line; either way the readiness line is +terminal, and the superseded precursor's signal survives in the PR's 🎯/♻️ comment so no +information is lost. This covers the flip cycle (`surfaced-green` → `marked-ready`), the +post-flip steady state (`already-surfaced` → `already-ready`), and the flake-coincident +flip (`annotated-flake` → `marked-ready`): ``` # main attempt- @@ -1394,8 +1692,14 @@ Per issue, append one outcome line to `/tmp/gh-aw/agent/coverage.txt`: `advance-PR # attempt /10` (ADVANCE mode — new commit pushed to the existing PR), `surfaced-green PR #` (fix's own CI went green; commented for review, did not advance), `annotated-flake PR #` (red was unrelated flake; -commented, attempt NOT burned), `waiting PR #` (CI not settled yet), -`dry-run: would-`, `skipped: `. (The +commented, attempt NOT burned), `marked-ready PR #` (the specific fixed test +was confirmed green in the PR's own CI; draft flipped to ready for review), +`already-surfaced PR #` (the fix's ✅ green comment already existed this sweep; +re-post suppressed — superseded by the Step 3.6 readiness line when the PR also flips +ready that cycle), `already-ready PR #` (terminal steady state — the PR was flipped +ready on a prior cycle and remains green on an unchanged head; no action taken, +supersedes `already-surfaced`), `waiting PR #` (CI not settled yet), +`dry-run: would-`, `skipped: `. (The `needs-human-PR` outcome is reserved for the deferred hand-off — Step 6 currently records a skip instead, so it is not emitted.)
( not run — needs /azp run)` and stop +this gate WITHOUT marking ready. Do NOT overclaim — a green *sibling* leg (a different category +on the same platform) is not the target's leg, and a green leg on one platform is not a pass on +the others. + +**T3 — Mark ready + report.** If EVERY target test is VALIDATED-GREEN. The 🎯 comment, +the mark-ready, and the `p/0` label are THREE SEPARATE safe-outputs — the comment +existing does NOT prove the mark-ready took effect, so they are tracked independently: + +- **Comment idempotency (dup-suppress only — never gates the mark-ready).** We only reach + T3 while `C.isDraft == true` (the precondition stops an already-ready PR before here). So + if a prior bot comment for THIS head that carries the leading `🎯` anchor AND the + contiguous phrase `validated green on ` already exists (BOTH T3 variants — + `🎯 Target test validated green on ` and the build-only `🎯 Build validated green on + ` — contain that exact contiguous phrase, so this recognizes both; keying on + `target test` alone would instead miss the build-only comment and re-post it every sweep. + **Require the `🎯` anchor** so the match can NEVER be loosened to a gapped + `validated…green…on` form that would false-positive on the Step 3 `✅ … validated — the + fix's CI is green on ` surface comment and wrongly suppress the audit 🎯), the comment + landed on an earlier sweep but the **mark-ready did NOT take + effect** (the PR is still a draft) — set `SUPPRESS_COMMENT = true` (do not re-post the + duplicate 🎯 comment) but STILL complete the mark-ready + label below. Never treat the + comment's existence as "already marked ready" while the PR is still a draft. Record this + case as `re-marking PR # (🎯 present; mark-ready did not take on a prior sweep)`. +- **Atomicity budget pre-check (Hard-Rule 7).** Determine the outputs this gate will emit: + `mark_pull_request_as_ready_for_review` + `add_labels` always, plus `add_comment` unless + `SUPPRESS_COMMENT`. Verify a free per-run slot remains in EACH bucket you are about to + use. If ANY required bucket is exhausted, emit NONE of them — record `skipped: per-run + cap reached; deferring mark-ready PR # to next cycle` and stop this gate. Never mark a + PR ready (or label it) unless its 🎯 audit comment is GUARANTEED to exist for this exact + head SHA — either posted in this same sweep, or (in the `SUPPRESS_COMMENT` re-marking case) + already present from a prior sweep. The outputs you DO emit either all land or all defer + together; the audit trail must never be absent, but it is NOT re-posted when it already exists. +- **Dry-run gate (Step 0):** if `dry_run == "true"`, emit NOTHING — print the intended + readiness comment and "would mark ready + add p/0" to the run log, tally `dry-run: + would-mark-ready PR #`, and stop. +- Otherwise emit for THIS PR number ``: + 1. `add_comment` (ONLY if not `SUPPRESS_COMMENT`): `🎯 Target test validated green on + — passed on ALL platforms it runs on (, + buildId ). — not + caused by this fix."> Transitioning this PR from draft to ready for review and adding + `p/0`; a maintainer still reviews and merges.` (For a **build-only fix** — the T1 + whole-build fallback, no single target test — phrase the first clause as `🎯 Build + validated green on — the maui-pr build passed on ALL platforms (buildId + ).` instead of naming a test.) + 2. `mark_pull_request_as_ready_for_review` with `reason:` a one-line justification + naming the validated test(s) and ``. + 3. `add_labels` with `labels: ["p/0"]` for PR # — put the now-review-ready fix into + the team's p/0 priority queue so it is triaged, not lost in the draft backlog. (If + the PR somehow already carries `p/0`, this is a harmless no-op.) +- Record `marked-ready PR # (target green on ALL platforms on , + labeled p/0)` and stop. + #### Step 3.5.R — Maintainer change-request response (Track C) Reached from Step 3.5 gate 0. Goal: when an **eligible human reviewer** (Hard-Rule @@ -1383,7 +1669,19 @@ Filed by [`ci-status-fix`](https://github.com/dotnet/maui/blob/main/.github/work ### Step 8 — Per-issue tally + end-of-run summary -Per issue, append one outcome line to `/tmp/gh-aw/agent/coverage.txt`: +Per issue, append **one** outcome line to `/tmp/gh-aw/agent/coverage.txt` — the +terminal outcome for the cycle. When one cycle produces a chained pair — a PR gets a +same-cycle precursor line and **then** a Step 3.6 readiness line (`marked-ready` on the +draft→ready flip, or `already-ready` on a later steady-state sweep of an already-ready +PR) — record ONLY the terminal Step 3.6 readiness line: it supersedes ANY same-cycle +non-readiness precursor for that PR, so an aggregator keying on one-line-per-issue never +double-counts. The precursor may be a Step 3 green line (`surfaced-green` on the first ✅, +or `already-surfaced` when it already exists) OR — when an unrelated-flake red and a +target-green coincide — a Step 4 `annotated-flake` line; either way the readiness line is +terminal, and the superseded precursor's signal survives in the PR's 🎯/♻️ comment so no +information is lost. This covers the flip cycle (`surfaced-green` → `marked-ready`), the +post-flip steady state (`already-surfaced` → `already-ready`), and the flake-coincident +flip (`annotated-flake` → `marked-ready`): ``` # main attempt- @@ -1394,8 +1692,14 @@ Per issue, append one outcome line to `/tmp/gh-aw/agent/coverage.txt`: `advance-PR # attempt /10` (ADVANCE mode — new commit pushed to the existing PR), `surfaced-green PR #` (fix's own CI went green; commented for review, did not advance), `annotated-flake PR #` (red was unrelated flake; -commented, attempt NOT burned), `waiting PR #` (CI not settled yet), -`dry-run: would-`, `skipped: `. (The +commented, attempt NOT burned), `marked-ready PR #` (the specific fixed test +was confirmed green in the PR's own CI; draft flipped to ready for review), +`already-surfaced PR #` (the fix's ✅ green comment already existed this sweep; +re-post suppressed — superseded by the Step 3.6 readiness line when the PR also flips +ready that cycle), `already-ready PR #` (terminal steady state — the PR was flipped +ready on a prior cycle and remains green on an unchanged head; no action taken, +supersedes `already-surfaced`), `waiting PR #` (CI not settled yet), +`dry-run: would-`, `skipped: `. (The `needs-human-PR` outcome is reserved for the deferred hand-off — Step 6 currently records a skip instead, so it is not emitted.)
`: + 1. `add_comment` (ONLY if not `SUPPRESS_COMMENT`): `🎯 Target test validated green on + — passed on ALL platforms it runs on (, + buildId ). — not + caused by this fix."> Transitioning this PR from draft to ready for review and adding + `p/0`; a maintainer still reviews and merges.` (For a **build-only fix** — the T1 + whole-build fallback, no single target test — phrase the first clause as `🎯 Build + validated green on — the maui-pr build passed on ALL platforms (buildId + ).` instead of naming a test.) + 2. `mark_pull_request_as_ready_for_review` with `reason:` a one-line justification + naming the validated test(s) and ``. + 3. `add_labels` with `labels: ["p/0"]` for PR # — put the now-review-ready fix into + the team's p/0 priority queue so it is triaged, not lost in the draft backlog. (If + the PR somehow already carries `p/0`, this is a harmless no-op.) +- Record `marked-ready PR # (target green on ALL platforms on , + labeled p/0)` and stop. + #### Step 3.5.R — Maintainer change-request response (Track C) Reached from Step 3.5 gate 0. Goal: when an **eligible human reviewer** (Hard-Rule @@ -1383,7 +1669,19 @@ Filed by [`ci-status-fix`](https://github.com/dotnet/maui/blob/main/.github/work ### Step 8 — Per-issue tally + end-of-run summary -Per issue, append one outcome line to `/tmp/gh-aw/agent/coverage.txt`: +Per issue, append **one** outcome line to `/tmp/gh-aw/agent/coverage.txt` — the +terminal outcome for the cycle. When one cycle produces a chained pair — a PR gets a +same-cycle precursor line and **then** a Step 3.6 readiness line (`marked-ready` on the +draft→ready flip, or `already-ready` on a later steady-state sweep of an already-ready +PR) — record ONLY the terminal Step 3.6 readiness line: it supersedes ANY same-cycle +non-readiness precursor for that PR, so an aggregator keying on one-line-per-issue never +double-counts. The precursor may be a Step 3 green line (`surfaced-green` on the first ✅, +or `already-surfaced` when it already exists) OR — when an unrelated-flake red and a +target-green coincide — a Step 4 `annotated-flake` line; either way the readiness line is +terminal, and the superseded precursor's signal survives in the PR's 🎯/♻️ comment so no +information is lost. This covers the flip cycle (`surfaced-green` → `marked-ready`), the +post-flip steady state (`already-surfaced` → `already-ready`), and the flake-coincident +flip (`annotated-flake` → `marked-ready`): ``` # main attempt- @@ -1394,8 +1692,14 @@ Per issue, append one outcome line to `/tmp/gh-aw/agent/coverage.txt`: `advance-PR # attempt /10` (ADVANCE mode — new commit pushed to the existing PR), `surfaced-green PR #` (fix's own CI went green; commented for review, did not advance), `annotated-flake PR #` (red was unrelated flake; -commented, attempt NOT burned), `waiting PR #` (CI not settled yet), -`dry-run: would-`, `skipped: `. (The +commented, attempt NOT burned), `marked-ready PR #` (the specific fixed test +was confirmed green in the PR's own CI; draft flipped to ready for review), +`already-surfaced PR #` (the fix's ✅ green comment already existed this sweep; +re-post suppressed — superseded by the Step 3.6 readiness line when the PR also flips +ready that cycle), `already-ready PR #` (terminal steady state — the PR was flipped +ready on a prior cycle and remains green on an unchanged head; no action taken, +supersedes `already-surfaced`), `waiting PR #` (CI not settled yet), +`dry-run: would-`, `skipped: `. (The `needs-human-PR` outcome is reserved for the deferred hand-off — Step 6 currently records a skip instead, so it is not emitted.)
(target green on ALL platforms on , + labeled p/0)` and stop. + #### Step 3.5.R — Maintainer change-request response (Track C) Reached from Step 3.5 gate 0. Goal: when an **eligible human reviewer** (Hard-Rule @@ -1383,7 +1669,19 @@ Filed by [`ci-status-fix`](https://github.com/dotnet/maui/blob/main/.github/work ### Step 8 — Per-issue tally + end-of-run summary -Per issue, append one outcome line to `/tmp/gh-aw/agent/coverage.txt`: +Per issue, append **one** outcome line to `/tmp/gh-aw/agent/coverage.txt` — the +terminal outcome for the cycle. When one cycle produces a chained pair — a PR gets a +same-cycle precursor line and **then** a Step 3.6 readiness line (`marked-ready` on the +draft→ready flip, or `already-ready` on a later steady-state sweep of an already-ready +PR) — record ONLY the terminal Step 3.6 readiness line: it supersedes ANY same-cycle +non-readiness precursor for that PR, so an aggregator keying on one-line-per-issue never +double-counts. The precursor may be a Step 3 green line (`surfaced-green` on the first ✅, +or `already-surfaced` when it already exists) OR — when an unrelated-flake red and a +target-green coincide — a Step 4 `annotated-flake` line; either way the readiness line is +terminal, and the superseded precursor's signal survives in the PR's 🎯/♻️ comment so no +information is lost. This covers the flip cycle (`surfaced-green` → `marked-ready`), the +post-flip steady state (`already-surfaced` → `already-ready`), and the flake-coincident +flip (`annotated-flake` → `marked-ready`): ``` # main attempt- @@ -1394,8 +1692,14 @@ Per issue, append one outcome line to `/tmp/gh-aw/agent/coverage.txt`: `advance-PR # attempt /10` (ADVANCE mode — new commit pushed to the existing PR), `surfaced-green PR #` (fix's own CI went green; commented for review, did not advance), `annotated-flake PR #` (red was unrelated flake; -commented, attempt NOT burned), `waiting PR #` (CI not settled yet), -`dry-run: would-`, `skipped: `. (The +commented, attempt NOT burned), `marked-ready PR #` (the specific fixed test +was confirmed green in the PR's own CI; draft flipped to ready for review), +`already-surfaced PR #` (the fix's ✅ green comment already existed this sweep; +re-post suppressed — superseded by the Step 3.6 readiness line when the PR also flips +ready that cycle), `already-ready PR #` (terminal steady state — the PR was flipped +ready on a prior cycle and remains green on an unchanged head; no action taken, +supersedes `already-surfaced`), `waiting PR #` (CI not settled yet), +`dry-run: would-`, `skipped: `. (The `needs-human-PR` outcome is reserved for the deferred hand-off — Step 6 currently records a skip instead, so it is not emitted.)
attempt /10` (ADVANCE mode — new commit pushed to the existing PR), `surfaced-green PR #` (fix's own CI went green; commented for review, did not advance), `annotated-flake PR #` (red was unrelated flake; -commented, attempt NOT burned), `waiting PR #` (CI not settled yet), -`dry-run: would-`, `skipped: `. (The +commented, attempt NOT burned), `marked-ready PR #` (the specific fixed test +was confirmed green in the PR's own CI; draft flipped to ready for review), +`already-surfaced PR #` (the fix's ✅ green comment already existed this sweep; +re-post suppressed — superseded by the Step 3.6 readiness line when the PR also flips +ready that cycle), `already-ready PR #` (terminal steady state — the PR was flipped +ready on a prior cycle and remains green on an unchanged head; no action taken, +supersedes `already-surfaced`), `waiting PR #` (CI not settled yet), +`dry-run: would-`, `skipped: `. (The `needs-human-PR` outcome is reserved for the deferred hand-off — Step 6 currently records a skip instead, so it is not emitted.)
` (CI not settled yet), -`dry-run: would-`, `skipped: `. (The +commented, attempt NOT burned), `marked-ready PR #` (the specific fixed test +was confirmed green in the PR's own CI; draft flipped to ready for review), +`already-surfaced PR #` (the fix's ✅ green comment already existed this sweep; +re-post suppressed — superseded by the Step 3.6 readiness line when the PR also flips +ready that cycle), `already-ready PR #` (terminal steady state — the PR was flipped +ready on a prior cycle and remains green on an unchanged head; no action taken, +supersedes `already-surfaced`), `waiting PR #` (CI not settled yet), +`dry-run: would-`, `skipped: `. (The `needs-human-PR` outcome is reserved for the deferred hand-off — Step 6 currently records a skip instead, so it is not emitted.)
` (CI not settled yet), +`dry-run: would-`, `skipped: `. (The `needs-human-PR` outcome is reserved for the deferred hand-off — Step 6 currently records a skip instead, so it is not emitted.)