diff --git a/.config/dotnet-tools.json b/.config/dotnet-tools.json index 7eae4d9a5834..10311f4926d5 100644 --- a/.config/dotnet-tools.json +++ b/.config/dotnet-tools.json @@ -24,7 +24,7 @@ "rollForward": false }, "microsoft.dotnet.xharness.cli": { - "version": "11.0.0-prerelease.26107.1", + "version": "11.0.0-prerelease.26229.1", "commands": [ "xharness" ], diff --git a/.github/ISSUE_TEMPLATE/bug-report.yml b/.github/ISSUE_TEMPLATE/bug-report.yml index 404f109e00e5..a096ab814aef 100644 --- a/.github/ISSUE_TEMPLATE/bug-report.yml +++ b/.github/ISSUE_TEMPLATE/bug-report.yml @@ -42,8 +42,10 @@ body: label: Version with bug description: In what version do you see this issue? Run `dotnet workload list` to find your version. options: - - 11.0.0-preview2 + - 11.0.0-preview.3 + - 11.0.0-preview.2 - 11.0.0-preview.1 + - 10.0.60 - 10.0.50 - 10.0.40 - 10.0.30 @@ -165,8 +167,10 @@ body: - 10.0.30 - 10.0.40 - 10.0.50 + - 10.0.60 - 11.0.0-preview.1 - - 11.0.0-preview.2 + - 11.0.0-preview.2 + - 11.0.0-preview.3 validations: required: true - type: dropdown diff --git a/.github/agent-pr-session/pr-31487.md b/.github/agent-pr-session/pr-31487.md deleted file mode 100644 index b15821b1c3a2..000000000000 --- a/.github/agent-pr-session/pr-31487.md +++ /dev/null @@ -1,261 +0,0 @@ -# PR Review: #31487 - [Android] Fixed duplicate title icon when setting TitleIconImageSource Multiple times - -**Date:** 2026-01-08 | **Issue:** [#31445](https://github.com/dotnet/maui/issues/31445) | **PR:** [#31487](https://github.com/dotnet/maui/pull/31487) - -## βœ… Final Recommendation: APPROVE - -| Phase | Status | -|-------|--------| -| Pre-Flight | βœ… COMPLETE | -| πŸ§ͺ Tests | βœ… COMPLETE | -| 🚦 Gate | βœ… PASSED | -| πŸ”§ Fix | βœ… COMPLETE | -| πŸ“‹ Report | βœ… COMPLETE | - ---- - -
-πŸ“‹ Issue Summary - -On Android, calling `NavigationPage.SetTitleIconImageSource(page, "image.png")` more than once for the same page results in the icon being rendered multiple times in the navigation bar. - -**Steps to Reproduce:** -1. Launch app on Android -2. Tap "Set TitleIconImageSource" once: icon appears -3. Tap it again: a second identical icon appears - -**Expected:** Single toolbar icon regardless of how many times SetTitleIconImageSource is called. - -**Actual:** Each repeated call adds an additional duplicate icon. - -**Platforms Affected:** -- [ ] iOS -- [x] Android -- [ ] Windows -- [ ] MacCatalyst - -**Version:** 9.0.100 SR10 - -
- -
-πŸ“ Files Changed - -| File | Type | Changes | -|------|------|---------| -| `src/Controls/src/Core/Platform/Android/Extensions/ToolbarExtensions.cs` | Fix | +17/-6 | -| `src/Controls/tests/TestCases.HostApp/Issues/Issue31445.cs` | Test | +38 | -| `src/Controls/tests/TestCases.Shared.Tests/Tests/Issues/Issue31445.cs` | Test | +23 | -| `snapshots/android/Issue31445DuplicateTitleIconDoesNotAppear.png` | Snapshot | binary | -| `snapshots/mac/Issue31445DuplicateTitleIconDoesNotAppear.png` | Snapshot | binary | -| `snapshots/windows/Issue31445DuplicateTitleIconDoesNotAppear.png` | Snapshot | binary | -| `snapshots/ios/Issue31445DuplicateTitleIconDoesNotAppear.png` | Snapshot | binary | - -
- -
-πŸ’¬ PR Discussion Summary - -**Key Comments:** -- Issue verified by LogishaSelvarajSF4525 on MAUI 9.0.0 & 9.0.100 -- PR triggered UI tests by jsuarezruiz -- PureWeen requested rebase - -**Reviewer Feedback:** -- Copilot review: Suggested testing with different image sources or rapid succession to validate fix better - -**Disagreements to Investigate:** -| File:Line | Reviewer Says | Author Says | Status | -|-----------|---------------|-------------|--------| -| Issue31445.cs:31 | Test with different images or rapid calls | N/A | ⚠️ INVESTIGATE | - -**Author Uncertainty:** -- None noted - -
- -
-πŸ§ͺ Tests - -**Status**: βœ… COMPLETE - -- [x] PR includes UI tests -- [x] Tests reproduce the issue -- [x] Tests follow naming convention (`Issue31445`) - -**Test Files:** -- HostApp: `src/Controls/tests/TestCases.HostApp/Issues/Issue31445.cs` -- NUnit: `src/Controls/tests/TestCases.Shared.Tests/Tests/Issues/Issue31445.cs` - -**Test Behavior:** -- Uses snapshot verification (`VerifyScreenshot()`) -- Navigates to test page, taps button to trigger duplicate icon scenario -- Verified to compile successfully - -
- -
-🚦 Gate - Test Verification - -**Status**: βœ… PASSED - -- [x] Tests FAIL without fix (bug reproduced - duplicate icons appeared) -- [x] Tests PASS with fix (single icon as expected) - -**Result:** PASSED βœ… - -**Verification Details:** -- Platform: Android (emulator-5554) -- Without fix: Test FAILED (screenshot mismatch - duplicate icons) -- With fix: Test PASSED (single icon verified) - -
- -
-πŸ”§ Fix Candidates - -**Status**: βœ… COMPLETE - -| # | Source | Approach | Test Result | Files Changed | Model | Notes | -|---|--------|----------|-------------|---------------|-------|-------| -| 1 | try-fix | Check for existing icon view at position 0, reuse if exists, only create new if needed | βœ… PASS | `ToolbarExtensions.cs` (+7) | Opus 4.5 | Works! Independently arrived at same solution logic as PR | -| 2 | try-fix | Dedupe defensively by scanning all toolbar children, keep first `ToolbarTitleIconImageView`, remove extras; then reuse/create | βœ… PASS | `ToolbarExtensions.cs` (+22/-5) | GPT 5.2 | More robust if child ordering changes or duplicates already exist | -| 3 | try-fix | Use `FindViewWithTag` to uniquely identify/retrieve the MAUI title icon | βœ… PASS | `ToolbarExtensions.cs` (+20/-6) | Gemini 2.0 Flash | Explicit identification; avoids index assumptions and iteration; most robust against external view insertions | -| PR | PR #31487 | Check for existing ToolbarTitleIconImageView before adding new one | βœ… PASS (Gate) | `ToolbarExtensions.cs` (+17/-6) | Author | Original PR - validated by Gate | - -**Exhausted:** Yes (3 passing alternatives found) - -**Selected Fix:** PR's fix - It’s simplest and sufficient. -- #3 (Tag) is the most "correct" for robustness but adds Tag management overhead. -- #2 (Dedupe) is good for cleanup. -- PR/#1 (Index 0) are standard for this codebase's patterns. - -**Comparison Notes:** -- PR/try-fix #1 rely on `GetChildAt(0)` being the title icon view when present -- try-fix #2 is more defensive: it collapses existing duplicates regardless of child index and then reuses/creates as needed -- try-fix #3 uses explicit tagging: precise but introduces new state (Tag) to manage - -
- ---- - -**Next Step:** Propose Alternative Fix #2 (Dedupe & Scan) to Author for Discussion - ---- - -## πŸ’¬ Draft Comment for Author - -Hi @PureWeen, - -Reviewing the fix in this PR, it works correctly for the reported issue and tests pass. - -I explored a couple of alternative approaches and found one that might offer slightly better robustness against edge cases, which I wanted to run by you: - -**Alternative: Dedupe & Scan** -Instead of just checking index 0, we could scan all children of the toolbar to find any `ToolbarTitleIconImageView` instances. - -```csharp -// Scan all children to find existing title icons -ToolbarTitleIconImageView? titleIcon = null; -for (int i = 0; i < nativeToolbar.ChildCount; i++) -{ - var child = nativeToolbar.GetChildAt(i); - if (child is ToolbarTitleIconImageView icon) - { - if (titleIcon == null) - titleIcon = icon; // Keep the first one found - else - nativeToolbar.RemoveView(icon); // Remove any extras (self-healing) - } -} -``` - -**Why consider this?** -1. **Robustness against Injection:** If another library inserts a view at index 0 (e.g., search bar), the current PR fix (checking only index 0) would fail to see the existing icon and create a duplicate. -2. **Self-Healing:** If the toolbar is already in a bad state (multiple icons from previous bugs), this approach cleans them up. - -**Trade-off:** -It involves a loop, so O(N) instead of O(1), but for a toolbar with very few items, this is negligible. - -Do you think the added robustness is worth the change, or should we stick to the simpler Index 0 check (current PR) which matches the existing removal logic? - ---- - -## πŸ“‹ Final Report - -### Summary - -PR #31487 correctly fixes the duplicate title icon issue on Android. The fix checks for an existing `ToolbarTitleIconImageView` at position 0 before creating a new one, preventing duplicate icons when `SetTitleIconImageSource` is called multiple times. - -### Root Cause - -The original `UpdateTitleIcon` method always created a new `ToolbarTitleIconImageView` and added it to position 0, without checking if one already existed. This caused duplicate icons when the method was called repeatedly. - -### Validation - -| Check | Result | -|-------|--------| -| Tests reproduce bug | βœ… Test fails without fix (duplicate icons) | -| Tests pass with fix | βœ… Test passes with fix (single icon) | -| Independent fix analysis | βœ… try-fix arrived at same solution | -| Code quality | βœ… Clean, minimal change | - -### Regression Analysis - -
-πŸ“œ Git History Analysis - -**Original Implementation:** `e2f3aaa222` (Oct 2021) by Shane Neuville -- Part of "[Android] ToolbarHandler and fixes for various page nesting scenarios (#2781)" -- The bug has existed since the original implementation - it was never designed to handle repeated calls - -**Key Finding:** The original code had a check for removing an existing icon when source is null/empty: -```csharp -if (nativeToolbar.GetChildAt(0) is ToolbarTitleIconImageView existingImageView) - nativeToolbar.RemoveView(existingImageView); -``` -But this check was **only in the removal path**, not in the creation path. The fix extends this pattern to also check before adding. - -**Related Toolbar Issues in This File:** -| Commit | Issue | Description | -|--------|-------|-------------| -| `a93e88c3de` | #7823 | Fix toolbar item icon not removed when navigating | -| `c04b7d79cc` | #19673 | Fixed android toolbar icon change | -| `158ed8b4f1` | #28767 | Removing outdated menu items after activity switch | - -**Pattern:** Multiple fixes in this file address issues where Android toolbar state isn't properly cleaned up or reused. This PR follows the same pattern. - -
- -
-πŸ”„ Platform Comparison - -| Platform | TitleIcon Implementation | Duplicate Prevention | -|----------|-------------------------|---------------------| -| **Android** | Creates `ToolbarTitleIconImageView`, adds to position 0 | ❌ Was missing (now fixed by PR) | -| **Windows** | Sets `TitleIconImageSource` property directly | βœ… Property-based, no duplicates possible | -| **iOS** | Uses `NavigationRenderer` with property binding | βœ… Property-based approach | - -**Why Android was vulnerable:** Android uses a view-based approach (adding/removing child views) while other platforms use property-based approaches. View management requires explicit duplicate checks. - -
- -
-⚠️ Risk Assessment - -**Regression Risk: LOW** - -1. **Minimal change** - Only modifies the creation logic, doesn't change removal -2. **Consistent pattern** - Uses same `GetChildAt(0)` check that already existed for removal -3. **Well-tested** - UI test verifies the specific scenario -4. **No side effects** - Reusing existing view is safe; `SetImageDrawable` handles updates - -**Potential Edge Cases (from Copilot review suggestion):** -- Setting different image sources rapidly β†’ Should work fine, image is updated on existing view -- Setting same source multiple times β†’ Explicitly tested, works correctly - -
- -### Recommendation - -**βœ… APPROVE** - The PR's approach is correct and validated by independent analysis. The fix is minimal, focused, and addresses the root cause. diff --git a/.github/aw/actions-lock.json b/.github/aw/actions-lock.json index 4a1a1a9ed381..f0f127ea9622 100644 --- a/.github/aw/actions-lock.json +++ b/.github/aw/actions-lock.json @@ -5,15 +5,15 @@ "version": "v8", "sha": "ed597411d8f924073f98dfc5c65a23a2325f34cd" }, - "github/gh-aw-actions/setup@v0.62.1": { - "repo": "github/gh-aw-actions/setup", - "version": "v0.62.1", - "sha": "95c4e2aa6adbdf63ff0b0fbf09945ad4f4716fea" + "actions/github-script@v9": { + "repo": "actions/github-script", + "version": "v9", + "sha": "3a2844b7e9c422d3c10d287c895573f7108da1b3" }, - "github/gh-aw-actions/setup@v0.62.2": { + "github/gh-aw-actions/setup@v0.68.3": { "repo": "github/gh-aw-actions/setup", - "version": "v0.62.2", - "sha": "20045bbd5ad2632b9809856c389708eab1bd16ef" + "version": "v0.68.3", + "sha": "ba90f2186d7ad780ec640f364005fa24e797b360" }, "github/gh-aw/actions/setup@v0.43.19": { "repo": "github/gh-aw/actions/setup", diff --git a/.github/copilot-instructions.md b/.github/copilot-instructions.md index 0e7463cd64e8..d95985e8a3f1 100644 --- a/.github/copilot-instructions.md +++ b/.github/copilot-instructions.md @@ -101,6 +101,30 @@ When referencing or triggering CI pipelines, use these current pipeline names: **⚠️ Old pipeline names** (e.g., `MAUI-UITests-public`, `MAUI-public`) are **outdated** and should NOT be used. Always use the names above. +### Investigating CI Failures + +**🚨 ALWAYS use the `azdo-build-investigator` skill when investigating CI failures or assessing merge readiness.** Its instructions direct you to invoke the `ci-analysis` skill first for the core investigation workflow, then apply MAUI-specific corrections (correct pipeline names, XHarness quirks, binlog guidance). + +Do NOT default to manually querying AzDO APIs or rely solely on `gh pr checks` pass/fail counts. + +**When to use it:** +- "How does CI look?" / "Is CI green?" / "Can we merge?" +- "What's failing?" / "Are these known failures?" +- "Is this PR safe to merge?" / "Any CI concerns?" +- After any PR push to verify the build + +**Verifying specific tests:** When asked "did test X pass?" or "did the new test run?", query the **actual AzDO test results** β€” do NOT infer whether a test ran by inspecting code attributes. Class-level traits, base class categories, and assembly-level attributes can all cause a test to run even when the method itself has no visible category. Check the evidence, not the code. + +**Anti-pattern:** Writing ad-hoc scripts to parse AzDO build timelines. The skills handle Helix work item details, known issue cross-referencing, and test result aggregation that manual approaches miss. + +### Gradle / Maven Dependency Failures (CFSClean) + +The official CI build uses CFSClean network isolation which blocks `repo.maven.apache.org`. All Gradle/Maven dependencies resolve through the `dotnet-public-maven` Azure Artifacts feed. + +**If CI fails with Gradle 401 errors** like `"No local versions of package"` or `"Please provide authentication to save package from upstream"`, it means a Maven package hasn't been ingested into the feed yet. **Fix:** run `./eng/ingest-maven-deps.sh` locally to pre-populate the feed. See `src/Core/AndroidNative/settings.gradle` for details. + +**Do NOT upgrade Gradle past 8.x** β€” the Android SDK's `net.android.init.gradle.kts` is incompatible with Gradle 9.x (`dotnet/android#10738`). + ### Code Formatting Always format code before committing: @@ -301,11 +325,6 @@ Skills are modular capabilities that can be invoked directly or used by agents. - **Two modes**: Verify failure only (test creation) or full verification (test + fix) - **Used by**: After creating tests, before considering PR complete -9. **pr-build-status** (`.github/skills/pr-build-status/SKILL.md`) - - **Purpose**: Retrieves Azure DevOps build information for PRs (build IDs, stage status, failed jobs) - - **Trigger phrases**: "check build for PR #XXXXX", "why did PR build fail", "get build status" - - **Used by**: When investigating CI failures - 10. **run-integration-tests** (`.github/skills/run-integration-tests/SKILL.md`) - **Purpose**: Build, pack, and run .NET MAUI integration tests locally - **Trigger phrases**: "run integration tests", "test templates locally", "run macOSTemplates tests", "run RunOniOS tests" diff --git a/.github/instructions/android.instructions.md b/.github/instructions/android.instructions.md index ba013f3c27a0..93d94cd3a19f 100644 --- a/.github/instructions/android.instructions.md +++ b/.github/instructions/android.instructions.md @@ -4,6 +4,9 @@ applyTo: - "**/Android/**/*.cs" - "**/Platforms/Android/**/*.cs" - "**/Platform/Android/**/*.cs" + - "**/AndroidNative/**" + - "eng/init.gradle" + - "eng/ingest-maven-deps.sh" --- # Android Platform Development Guidelines @@ -123,3 +126,8 @@ protected override void DisconnectHandler(RecyclerView platformView) | Listener not working | Check lifecycle (register/unregister) | | Memory leak | Ensure Dispose() called on Java.Lang.Object | | Threading error | Use `platformView.Post()` for UI thread | +| Gradle 401 / Maven dependency failure | Run `./eng/ingest-maven-deps.sh` β€” see `copilot-instructions.md` | + +## Gradle / Maven Dependency Failures + +CI uses CFSClean which blocks Maven Central. All deps go through the `dotnet-public-maven` Azure Artifacts feed. If a new package hasn't been ingested, CI fails with `XAGRDL0000` / 401. Run `./eng/ingest-maven-deps.sh` locally to fix. Do NOT upgrade Gradle past 8.x (`dotnet/android#10738`). diff --git a/.github/instructions/gh-aw-workflows.instructions.md b/.github/instructions/gh-aw-workflows.instructions.md deleted file mode 100644 index 2230e8c86492..000000000000 --- a/.github/instructions/gh-aw-workflows.instructions.md +++ /dev/null @@ -1,290 +0,0 @@ ---- -applyTo: - - ".github/workflows/*.md" - - ".github/workflows/*.lock.yml" ---- - -# gh-aw (GitHub Agentic Workflows) Guidelines - -## 🚨 Before You Build: Prefer Built-in gh-aw Features - -**CRITICAL RULE:** Before implementing any trigger, output, scheduling, or interaction mechanism in a gh-aw workflow, check whether gh-aw has a built-in feature that does it. gh-aw extends GitHub Actions with many convenience features β€” manually reimplementing them is always worse (more code, more bugs, missing platform integration like emoji reactions, sanitized inputs, and noise reduction). - -### Step 1: Check the anti-patterns table below -### Step 2: If not listed, check the [triggers reference](https://github.github.com/gh-aw/reference/triggers/), [frontmatter reference](https://github.github.com/gh-aw/reference/frontmatter/), and [safe-outputs reference](https://github.github.com/gh-aw/reference/safe-outputs/) -### Step 3: If a built-in exists, use it. If not, proceed with manual implementation. - -### Anti-Patterns: Manual Reimplementations to Avoid - -| If you're about to implement... | Use this built-in instead | Docs | -|---------------------------------|--------------------------|------| -| `issue_comment` + `startsWith(comment.body, '/cmd')` | `slash_command:` trigger | [Command Triggers](https://github.github.com/gh-aw/reference/command-triggers/) | -| Manual emoji reaction on triggering comment | `reaction:` field under `on:` | [Frontmatter](https://github.github.com/gh-aw/reference/frontmatter/) | -| Posting "workflow started/completed" status comments | `status-comment: true` under `on:` | [Frontmatter](https://github.github.com/gh-aw/reference/frontmatter/) | -| Fixed cron schedule (`0 9 * * 1`) for non-critical timing | `schedule: weekly on monday around 9:00` (fuzzy) | [Triggers](https://github.github.com/gh-aw/reference/triggers/) | -| Manual `if:` to skip bot-authored PRs | `skip-bots:` under `on:` | [Triggers](https://github.github.com/gh-aw/reference/triggers/) | -| Manual `if:` to skip by author role | `skip-roles:` under `on:` | [Triggers](https://github.github.com/gh-aw/reference/triggers/) | -| Manual label check + removal for one-shot commands | `label_command:` trigger | [Triggers](https://github.github.com/gh-aw/reference/triggers/) | -| Editing old comments to collapse them | `hide-older-comments: true` on `add-comment:` | [Safe Outputs](https://github.github.com/gh-aw/reference/safe-outputs/) | -| Creating no-op report issues | `noop: report-as-issue: false` | [Safe Outputs / Monitoring](https://github.github.com/gh-aw/patterns/monitoring/) | -| Auto-closing older issues from same workflow | `close-older-issues: true` on `create-issue:` | [Safe Outputs](https://github.github.com/gh-aw/reference/safe-outputs/) | -| Disabling workflow after a date | `stop-after:` under `on:` | [Triggers](https://github.github.com/gh-aw/reference/triggers/) | -| Manual approval gating | `manual-approval:` under `on:` | [Triggers](https://github.github.com/gh-aw/reference/triggers/) | -| Search-based skip logic in `steps:` | `skip-if-match:` / `skip-if-no-match:` under `on:` | [Triggers](https://github.github.com/gh-aw/reference/triggers/) | - -**Note:** gh-aw is actively developed. If a capability feels like something a framework would provide natively, check the reference docs β€” it probably exists even if it's not in this table yet. - -## Architecture - -gh-aw workflows are authored as `.md` files with YAML frontmatter, compiled to `.lock.yml` via `gh aw compile`. The lock file is auto-generated β€” **never edit it manually**. - -### Execution Model - -``` -activation job (renders prompt from base branch .md via runtime-import) - ↓ -agent job: - user steps: (pre-agent, OUTSIDE firewall, has GITHUB_TOKEN) - ↓ - platform steps: (configure git β†’ checkout_pr_branch.cjs β†’ install CLI) - ↓ - agent: (INSIDE sandboxed container, NO credentials) -``` - -| Context | Has GITHUB_TOKEN | Has gh CLI | Has git creds | Can execute scripts | -|---------|-----------------|-----------|---------------|-------------------| -| `steps:` (user) | βœ… Yes | βœ… Yes | βœ… Yes | βœ… Yes β€” **be careful** | -| Platform steps | βœ… Yes | βœ… Yes | βœ… Yes | Platform-controlled | -| Agent container | ❌ Scrubbed | ❌ Scrubbed | ❌ Scrubbed | βœ… But sandboxed | - -**⚠️ Agent container credential nuance:** `GITHUB_TOKEN` and `gh` CLI credentials are scrubbed inside the agent container. However, `COPILOT_TOKEN` (used for LLM inference) is present in the environment via `--env-all`. Any subprocess (e.g., `dotnet build`, `npm install`) inherits this variable. The AWF network firewall, `redact_secrets.cjs` (post-agent log scrubbing), and the threat detection agent limit the blast radius. See [Security Boundaries](#security-boundaries) below. - -### Step Ordering (Critical) - -User `steps:` **always run before** platform-generated steps. You cannot insert user steps after platform steps. - -The platform's `checkout_pr_branch.cjs` runs with `if: (github.event.pull_request) || (github.event.issue.pull_request)` β€” it is **skipped** for `workflow_dispatch` triggers. - -### Prompt Rendering - -The prompt is built in the **activation job** via `{{#runtime-import .github/workflows/.md}}`. This reads the `.md` file from the **base branch** workspace (before any PR checkout). The rendered prompt is uploaded as an artifact and downloaded by the agent job. - -- The agent prompt is always the base branch version β€” fork PRs cannot alter it -- The prompt references files on disk (e.g., `SKILL.md`) β€” those files must exist in the agent's workspace - -### Fork PR Activation Gate - -By default, `gh aw compile` automatically injects a fork guard into the activation job's `if:` condition: `head.repo.id == repository_id`. This blocks fork PRs on `pull_request` events. - -To **allow fork PRs**, add `forks: ["*"]` to the `pull_request` trigger in the `.md` frontmatter. The compiler removes the auto-injected guard from the compiled `if:` conditions. This is safe when the workflow uses the `Checkout-GhAwPr.ps1` pattern (checkout + trusted-infra restore) and the agent is sandboxed. - -## Security Boundaries - -### Key Principles (from [GitHub Security Lab](https://securitylab.github.com/resources/github-actions-preventing-pwn-requests/)) - -1. **Never execute untrusted PR code with elevated credentials.** The classic "pwn-request" attack is `pull_request_target` + checkout PR + run build scripts with `GITHUB_TOKEN`. The attack surface includes build scripts (`make`, `build.ps1`), package manager hooks (`npm postinstall`, MSBuild targets), and test runners. - -2. **Treating PR contents as passive data is safe.** Reading, analyzing, or diffing PR code is fine β€” the danger is *executing* it. Our gh-aw workflows read code for evaluation; they never build or run it. - -3. **`pull_request_target` grants write permissions and secrets access.** This is by design β€” the workflow YAML comes from the base branch (trusted). But any step that checks out and runs fork code in this context creates a vulnerability. - -4. **`pull_request` from forks has no secrets access.** GitHub withholds secrets because the workflow YAML comes from the fork (untrusted). This is the safe default for CI builds on fork PRs. - -5. **The `workflow_run` pattern separates privilege from code execution.** Build in an unprivileged `pull_request` job β†’ pass artifacts β†’ process in a privileged `workflow_run` job. This is architecturally what gh-aw does: agent runs read-only, `safe_outputs` job has write permissions. - -### gh-aw Defense Layers - -| Layer | What it does | What it doesn't do | -|-------|-------------|-------------------| -| **AWF network firewall** | Restricts outbound to allowlisted domains | Doesn't prevent reading env vars inside the container | -| **`redact_secrets.cjs`** | Scrubs known secret values from logs/artifacts post-agent | Doesn't catch encoded/obfuscated values | -| **Threat detection agent** | Reviews agent outputs before safe-outputs publishes them | Can miss novel exfiltration techniques | -| **Safe-outputs permission separation** | Write operations happen in separate job, not the agent | Agent can still request writes via safe-output tools | -| **`max: 1` on `add-comment`** | Limits agent to one comment | That one comment could contain sensitive data (mitigated by redaction) | -| **XPIA prompt** | Instructs LLM to resist prompt injection from untrusted content | LLM compliance is probabilistic, not guaranteed | -| **`pre_activation` role check** | Gates on write-access collaborators | Does not apply if `roles: all` is set | - -### Rules for gh-aw Workflow Authors - -- βœ… **DO** treat PR contents as passive data (read, analyze, diff) -- βœ… **DO** run data-gathering scripts in `steps:` (pre-agent, trusted context) not inside the agent -- βœ… **DO** use `Checkout-GhAwPr.ps1` for `workflow_dispatch` to restore trusted `.github/` from base -- ❌ **DO NOT** run `dotnet build`, `npm install`, or any build command on untrusted PR code inside the agent β€” build tool hooks (MSBuild targets, postinstall scripts) can read `COPILOT_TOKEN` from the environment -- ❌ **DO NOT** execute workspace scripts (`.ps1`, `.sh`, `.py`) after checking out a fork PR in `steps:` β€” those run with `GITHUB_TOKEN` -- ❌ **DO NOT** set `roles: all` on workflows that process PR content β€” this allows any user to trigger the workflow - -## Fork PR Handling - -### The "pwn-request" Threat Model - -The classic attack requires **checkout + execution** of fork code with elevated credentials. Checkout alone is not dangerous β€” the vulnerability is executing workspace scripts with `GITHUB_TOKEN`. - -Reference: https://securitylab.github.com/resources/github-actions-preventing-pwn-requests/ - -### Fork PR Behavior by Trigger - -| Trigger | `checkout_pr_branch.cjs` runs? | Fork handling | -|---------|-------------------------------|---------------| -| `pull_request` (default) | βœ… Yes | Blocked by auto-generated activation gate unless `forks: ["*"]` is set | -| `pull_request` + `forks: ["*"]` | βœ… Yes | βœ… Works β€” user steps restore trusted infra before agent runs | -| `workflow_dispatch` | ❌ Skipped | βœ… Works β€” user steps handle checkout and restore is final | -| `issue_comment` (same-repo) | βœ… Yes | βœ… Works β€” files already on PR branch | -| `issue_comment` (fork) | βœ… Yes | ⚠️ Works β€” `checkout_pr_branch.cjs` re-checks out fork branch after user steps, potentially overwriting restored infra. Acceptable because agent is sandboxed (no credentials, max 1 comment via safe-outputs). Pre-flight check catches missing `SKILL.md` if fork isn't rebased. | -| `slash_command` | βœ… Yes (compiles to `issue_comment` internally) | Same behavior as `issue_comment` above, but with platform-managed command matching, emoji reactions, and sanitized input. Prefer `slash_command:` over manual `issue_comment` + `startsWith()`. | - -### The `issue_comment` + Fork Problem - -For `/slash-command` triggers on fork PRs, `checkout_pr_branch.cjs` runs AFTER all user steps and re-checks out the fork branch. This overwrites any files restored by user steps (e.g., `.github/skills/`). A fork could include a crafted `SKILL.md` that alters the agent's evaluation behavior. - -**Accepted residual risk:** The agent runs in a sandboxed container with `GITHUB_TOKEN` and `gh` CLI credentials scrubbed. `COPILOT_TOKEN` (for LLM inference) remains in the environment but the AWF network firewall restricts outbound connections to an allowlist of domains, `redact_secrets.cjs` scrubs known secret values from logs/outputs post-agent, and the threat detection agent reviews outputs before they are published. The worst practical outcome is a manipulated evaluation comment (`safe-outputs: add-comment: max: 1`). The pre-flight check in the agent prompt catches the case where `SKILL.md` is missing entirely (fork not rebased on `main`). - -**Upstream issue:** [github/gh-aw#18481](https://github.com/github/gh-aw/issues/18481) β€” "Using gh-aw in forks of repositories" - -### Safe Pattern: Checkout + Restore - -Use the shared `.github/scripts/Checkout-GhAwPr.ps1` script, which implements checkout + restore in a single reusable step: - -```yaml -steps: - - name: Checkout PR and restore agent infrastructure - env: - GH_TOKEN: ${{ github.token }} - PR_NUMBER: ${{ github.event.pull_request.number || inputs.pr_number }} - run: pwsh .github/scripts/Checkout-GhAwPr.ps1 -``` - -The script: -1. Verifies the PR author has write access and rejects fork PRs -2. Captures the base branch SHA before checkout -3. Checks out the PR branch via `gh pr checkout` -4. Restores `.github/skills/`, `.github/instructions/`, and `.github/copilot-instructions.md` from the base branch SHA (fatal on failure) - -**Behavior by trigger:** -- **`workflow_dispatch`**: Platform checkout is skipped, so the restore IS the final workspace state (trusted files from base branch) -- **`slash_command`** (same-repo): Platform's `checkout_pr_branch.cjs` handles checkout. Skill files typically match main unless the PR modified them. -- **`slash_command`** (fork): Platform re-checks out fork branch after user steps, overwriting restored files. Agent is sandboxed; pre-flight in the prompt catches missing `SKILL.md` - -### Anti-Patterns - -**Do NOT skip checkout for fork PRs:** - -```bash -# ❌ ANTI-PATTERN: Makes fork PRs unevaluable -if [ "$HEAD_OWNER" != "$BASE_OWNER" ]; then - echo "Skipping checkout for fork PR" - exit 0 # Agent evaluates workflow branch instead of PR -fi -``` - -Skipping checkout means the agent evaluates the wrong files. The correct approach is: always check out the PR, then restore agent infrastructure from the base branch. - -**Do NOT execute workspace code after fork checkout:** - -```yaml -# ❌ DANGEROUS: runs fork code with GITHUB_TOKEN -- name: Checkout PR - run: gh pr checkout "$PR_NUMBER" ... -- name: Run analysis - run: pwsh .github/skills/some-script.ps1 -``` - -If you need to run scripts, either: -1. Run them **before** the checkout (from the base branch) -2. Run them **inside the agent container** (sandboxed, no tokens) - -## Compilation - -```bash -# Compile after every change to the .md source -gh aw compile .github/workflows/.md - -# This updates: -# - .github/workflows/.lock.yml (auto-generated) -# - .github/aw/actions-lock.json -``` - -**Always commit the compiled lock file alongside the source `.md`.** - -## Common Patterns - -### Pre-Agent Data Prep (the `steps:` pattern) - -Use `steps:` for any operation requiring GitHub API access that the agent needs: - -```yaml -steps: - - name: Fetch PR data - env: - GH_TOKEN: ${{ github.token }} - run: | - gh pr view "$PR_NUMBER" --json title,body > pr-metadata.json - gh pr diff "$PR_NUMBER" --name-only > changed-files.txt -``` - -### Safe Outputs (Posting Comments) - -```yaml -safe-outputs: - add-comment: - max: 1 - target: "*" # Required for workflow_dispatch (no triggering PR context) -``` - -### Concurrency - -Include all trigger-specific PR number sources: - -```yaml -concurrency: - group: "my-workflow-${{ github.event.issue.number || github.event.pull_request.number || inputs.pr_number || github.run_id }}" - cancel-in-progress: true -``` - -### Noise Reduction - -Filter `pull_request` triggers to relevant paths and add a gate step: - -```yaml -on: - pull_request: - paths: - - 'src/**/tests/**' - -steps: - - name: Gate β€” skip if no relevant files - if: github.event_name == 'pull_request' - run: | - FILES=$(gh pr diff "$PR_NUMBER" --name-only | grep -E '\.cs$' || true) - if [ -z "$FILES" ]; then exit 1; fi -``` - -Manual triggers (`workflow_dispatch`, `issue_comment`) should bypass the gate. Note: `exit 1` causes a red ❌ on non-matching PRs β€” this is intentional (no built-in "skip" mechanism in gh-aw steps). - -## Limitations - -| What | Behavior | Workaround | -|------|----------|------------| -| User steps always before platform steps | Cannot run user code after `checkout_pr_branch.cjs` | For `issue_comment` fork PRs, accept sandboxed residual risk; see [gh-aw#18481](https://github.com/github/gh-aw/issues/18481) | -| `--allow-all-tools` in lock.yml | Emitted by `gh aw compile` | Cannot override from `.md` source | -| MCP integrity filtering | Fork PRs blocked as "unapproved" | Use `steps:` checkout instead of MCP | -| `gh` CLI inside agent | Credentials scrubbed | Use `steps:` for API calls, or MCP tools | -| `issue_comment` trigger | Requires workflow on default branch | Must merge to `main` before `/slash-commands` work | -| Duplicate runs | gh-aw sometimes creates 2 runs per dispatch | Harmless, use concurrency groups | - -### Upstream References - -- [github/gh-aw#18481](https://github.com/github/gh-aw/issues/18481) β€” Fork support tracking issue -- [github/gh-aw#18518](https://github.com/github/gh-aw/issues/18518) β€” Fork detection in `gh aw init` -- [github/gh-aw#18521](https://github.com/github/gh-aw/issues/18521) β€” Fork support documentation - -## Troubleshooting - -| Symptom | Cause | Fix | -|---------|-------|-----| -| Agent evaluates wrong PR | `workflow_dispatch` checks out workflow branch | Add `gh pr checkout` in `steps:` | -| Agent can't find SKILL.md | Fork PR branch doesn't include `.github/skills/` | Rebase fork on `main`, or use `workflow_dispatch` with `pr_number` input | -| Fork PR skipped on `pull_request` | `forks: ["*"]` not in workflow frontmatter | Add `forks: ["*"]` under `pull_request:` in the `.md` source and recompile | -| `gh` commands fail in agent | Credentials scrubbed inside container | Move to `steps:` section | -| Lock file out of date | Forgot to recompile | Run `gh aw compile` | -| Integrity filtering warning | MCP reading fork PR data | Expected, non-blocking | -| `/slash-command` doesn't trigger | Workflow not on default branch | Merge to `main` first | diff --git a/.github/pr-review/pr-preflight.md b/.github/pr-review/pr-preflight.md index 0c5ae3f9b0b3..cef8034915c2 100644 --- a/.github/pr-review/pr-preflight.md +++ b/.github/pr-review/pr-preflight.md @@ -1,10 +1,10 @@ -# PR Pre-Flight β€” Context Gathering +# PR Pre-Flight β€” Context Gathering & Code Review -> **SCOPE:** Document only. No code analysis. No fix opinions. No running tests. +> **SCOPE:** Gather context, classify files, and perform deep code review. No code changes. No fix selection. No test execution. --- -## Steps +## Part A: Context Gathering (Steps 1–6) 1. **Read the issue** β€” full body + ALL comments via GitHub MCP tools 2. **Find the PR** β€” read description, diff summary, review comments, inline feedback @@ -12,6 +12,7 @@ 4. **Classify files** β€” separate fix files from test files, identify test type (UI / Device / Unit) 5. **Document edge cases** β€” from comments mentioning "what about...", "does this work with..." 6. **Record PR's fix** in Fix Candidates table (pending validation) +7. **Identify impacted UI test categories** β€” analyze which UI controls could be affected by this PR (see below) ```bash # Fetch PR metadata @@ -35,7 +36,86 @@ gh pr view XXXXX --json comments --jq '.comments[] | select(.body | contains("Fi --- -## Output File +## Step 7: Identify Impacted UI Test Categories + +After classifying files, determine which UI test categories could be affected by the PR changes. This enables targeted UI test runs instead of running the full matrix (~2h). + +**How to identify categories:** +1. Look at the **controls modified** in the PR (e.g., changes to `Button` handler β†’ `Button` category) +2. Consider **indirect impacts** (e.g., a layout change could affect `Layout`, `CollectionView`, `ListView`) +3. Check the **issue description** for mentions of specific controls +4. Consider **platform-specific impacts** (e.g., iOS SafeArea changes β†’ `SafeAreaEdges`) + +**Available categories:** +Read the canonical list from [`src/Controls/tests/TestCases.Shared.Tests/UITestCategories.cs`](../../src/Controls/tests/TestCases.Shared.Tests/UITestCategories.cs) β€” every `public const string` value in that file is a valid category. Only use category names defined there; AI-suggested names that aren't in the file will be filtered out by `detect-ui-test-categories.ps1` to avoid creating empty matrix jobs. + +**Output file:** +```bash +mkdir -p CustomAgentLogsTmp/PRState/{PRNumber}/PRAgent/uitests +``` + +Write `ai-categories.md`: +```markdown +Button β€” PR modifies ButtonHandler click event logic +Layout β€” Changes to StackLayout could affect child arrangement +``` + +One category per line, followed by ` β€” ` and a brief justification. Write `NONE` if the PR has no UI impact (e.g., docs-only, build scripts, backend-only changes). + +--- + +## Part B: Code Review (Step 8) + +> **Purpose:** Perform deep code analysis using the `code-review` skill to surface correctness issues, safety concerns, and MAUI convention violations BEFORE Try-Fix explores alternatives. These findings guide Try-Fix models toward higher-quality fixes. + +> **🚨 Independence-first requirement:** Step 8 MUST be invoked as a **separate sub-agent** (via the `task` tool with `agent_type: "general-purpose"`) so the code-review skill can form its assessment from the code BEFORE reading any PR narrative. The sub-agent receives ONLY the PR number β€” not the context gathered in Part A. This prevents anchoring bias. +> +> **Validation constraint:** The Step 8 prompt MUST NOT contain issue titles, root-cause descriptions, bug summaries, or any Part A content β€” only `PR #XXXXX`. If you find yourself adding context "to help" the sub-agent, you are violating independence-first. + +8. **Invoke the code-review skill as a sub-agent:** + + Use the `task` tool to launch a separate agent. The prompt MUST NOT contain issue titles, root-cause descriptions, or any Part A context β€” only the PR number. + + ```python + task( + name="code-review", + description="Code review for PR", + agent_type="general-purpose", + mode="sync", + prompt=""" + Run the code-review skill for PR #XXXXX. + Follow the full 6-step workflow in .github/skills/code-review/SKILL.md. + Output the review in the format specified by that skill. + """ + ) + ``` + + The sub-agent internally follows the code-review skill's 6-step workflow: + 1. Gather code context (independence-first β€” reads code BEFORE PR description) + 2. Load MAUI review rules from `.github/skills/code-review/references/review-rules.md` + 3. Form independent assessment + 4. Reconcile with PR narrative and prior reviews + 5. Check CI status + 6. Blast radius, failure-mode probing, and verdict + +**If Step 8 fails, times out, or returns malformed output:** +- Write `pre-flight/code-review.md` with: `## Code Review: SKIPPED\n\nReason: {failure description}` +- Set verdict to `SKIPPED` in the Code Review Summary section of `content.md` +- Omit `hints` from Try-Fix prompts (the `hints` field becomes optional when code review is unavailable) +- Do NOT apply the code-review hard gate in Phase 3 (Report) β€” treat as if code review was not run + +**Store the sub-agent's full output** in `pre-flight/code-review.md` β€” use the exact output format from the code-review skill (do NOT reformat or summarize into a different template). + +**Extract key items for Try-Fix consumption** and add to `content.md`: +- All ❌ Error findings (with file:line references) +- All ⚠️ Warning findings (with file:line references) +- Failure-mode probes and their answers +- Blast radius assessment summary +- The overall verdict and confidence level + +--- + +## Output Files ```bash mkdir -p CustomAgentLogsTmp/PRState/{PRNumber}/PRAgent/pre-flight @@ -52,16 +132,29 @@ Write `content.md`: - {Finding 1} - {Finding 2} +### Code Review Summary +**Verdict:** {LGTM / NEEDS_CHANGES / NEEDS_DISCUSSION / SKIPPED} +**Confidence:** {high / medium / low / N/A} +**Errors:** {count} | **Warnings:** {count} | **Suggestions:** {count} + +Key code review findings: +- {❌/⚠️/πŸ’‘} {Brief finding with file:line reference} +- ... +*(If SKIPPED: "Code review sub-agent failed or timed out. Reason: {details}")* + ### Fix Candidates | # | Source | Approach | Test Result | Files Changed | Notes | |---|--------|----------|-------------|---------------|-------| | PR | PR #XXXXX | {approach} | ⏳ PENDING (Gate) | `file.cs` | Original PR | ``` +Write `code-review.md` β€” the exact output from the code-review sub-agent, in the format specified by `.github/skills/code-review/SKILL.md` (Review Output Format section). Do NOT reformat or create a custom template β€” preserve the skill's native output verbatim. + --- ## Common Mistakes -- ❌ Researching root cause β€” save for Try-Fix phase -- ❌ Looking at implementation code β€” just gather context +- ❌ Skipping the code-review step β€” it provides critical findings for Try-Fix +- ❌ Reading the PR description before code in Step 7 β€” independence-first prevents anchoring bias - ❌ Running tests β€” that's the Gate phase +- ❌ Proposing fixes β€” save fix ideas for Try-Fix phase diff --git a/.github/pr-review/pr-report.md b/.github/pr-review/pr-report.md index 89651509fd3f..b715ba365327 100644 --- a/.github/pr-review/pr-report.md +++ b/.github/pr-review/pr-report.md @@ -12,18 +12,26 @@ - Phases 1-2 (Pre-Flight, Try-Fix) must be complete before starting - Gate result is available from the prompt (ran separately before this skill) +- **Read `pre-flight/content.md`** to get the code-review summary (verdict, confidence, error/warning counts) +- Optionally read `pre-flight/code-review.md` for full findings if needed for the recommendation --- ## Steps -1. **Determine recommendation:** +1. **Determine recommendation** (rows evaluated in order β€” first match wins): - | Condition | Recommendation | - |-----------|----------------| - | PR's fix selected and Gate passed | `βœ… APPROVE` | - | Alternative fix found via Try-Fix | `⚠️ REQUEST CHANGES` β€” suggest alternative | - | Gate failed | `⚠️ REQUEST CHANGES` β€” fix doesn't work | + | Priority | Condition | Recommendation | + |----------|-----------|----------------| + | 1 | Code review verdict is `NEEDS_CHANGES` (any ❌ errors) | `⚠️ REQUEST CHANGES` β€” code review found errors | + | 2 | Gate failed (tests fail with fix) | `⚠️ REQUEST CHANGES` β€” fix doesn't work | + | 3 | Alternative fix found via Try-Fix that is simpler/better | `⚠️ REQUEST CHANGES` β€” suggest alternative | + | 4 | Code review verdict is `NEEDS_DISCUSSION` | `⚠️ REQUEST CHANGES` β€” include code review concerns | + | 5 | PR's fix selected AND Gate passed AND code review LGTM or SKIPPED | `βœ… APPROVE` | + + **🚨 Hard gate:** If the code review (from Pre-Flight) has verdict `NEEDS_CHANGES`, the final recommendation MUST be `REQUEST CHANGES` regardless of Gate or Try-Fix results. Code-review ❌ Errors cannot be overridden by passing tests alone. + + **Code review SKIPPED:** If the code-review sub-agent failed or timed out (verdict = `SKIPPED`), the hard gate does NOT apply. Proceed as if code review was not available β€” base the recommendation on Gate and Try-Fix results only. Note in the report that code review was unavailable. 2. **Write output files** β€” Save recommendation to `content.md` @@ -47,10 +55,14 @@ Write `content.md`: | Phase | Status | Notes | |---|---|---| | Pre-Flight | βœ… COMPLETE | {notes} | +| Code Review | {verdict} ({confidence}) | {error_count} errors, {warning_count} warnings | | Gate | βœ… PASSED | {platform} | | Try-Fix | βœ… COMPLETE | {N} attempts, {M} passing | | Report | βœ… COMPLETE | | +### Code Review Impact on Try-Fix +{Brief description of how code-review findings influenced try-fix exploration. Did any model specifically address a code review ❌ Error? Did failure-mode probes reveal issues that guided fix approaches?} + ### Summary {Brief summary of the review} @@ -58,7 +70,7 @@ Write `content.md`: {Root cause analysis} ### Fix Quality -{Assessment of the fix} +{Assessment of the fix β€” informed by both gate results and code review findings} ``` --- diff --git a/.github/scripts/Checkout-GhAwPr.ps1 b/.github/scripts/Checkout-GhAwPr.ps1 deleted file mode 100644 index 1231e451be7d..000000000000 --- a/.github/scripts/Checkout-GhAwPr.ps1 +++ /dev/null @@ -1,116 +0,0 @@ -<# -.SYNOPSIS - Shared PR checkout and trusted-infra restore for gh-aw workflows. - -.DESCRIPTION - Checks out a PR branch and restores trusted agent infrastructure (skills, - instructions) from the base branch. This gives the agent the PR's code - changes with the latest skills and instructions from main. - - Currently used for workflow_dispatch triggers. For slash_command and - issue_comment triggers, the gh-aw platform's checkout_pr_branch.cjs - handles PR checkout automatically β€” but may overwrite trusted infra - with fork-supplied files. Call this script after platform checkout to - restore trusted .github/ from the base branch. - - SECURITY: Before checkout, the script verifies the PR author has - write access (write, maintain, or admin) and rejects fork PRs. - This prevents checkout of untrusted code in privileged contexts. - - DO NOT add steps after this that run scripts from the workspace - (e.g., ./build.sh, pwsh ./script.ps1). That would create a code - execution vulnerability. See: - https://securitylab.github.com/resources/github-actions-preventing-pwn-requests/ - -.NOTES - Required environment variables (set by the calling workflow step): - GH_TOKEN - GitHub token for API access - PR_NUMBER - PR number to check out - GITHUB_REPOSITORY - owner/repo (set by GitHub Actions) - GITHUB_ENV - path to env file (set by GitHub Actions) -#> - -$ErrorActionPreference = 'Stop' - -# ── Validate inputs ────────────────────────────────────────────────────────── - -if (-not $env:PR_NUMBER -or $env:PR_NUMBER -eq '0') { - Write-Host "No PR number available, using default checkout" - exit 0 -} - -$PrNumber = $env:PR_NUMBER - -# ── Verify PR is same-repo and author has write access ─────────────────────── - -$RawJson = gh pr view $PrNumber --repo $env:GITHUB_REPOSITORY --json author,isCrossRepository --jq '{author: .author.login, isFork: .isCrossRepository}' -if ($LASTEXITCODE -ne 0) { - Write-Host "❌ Failed to fetch PR #$PrNumber metadata" - exit 1 -} - -try { - $PrInfo = $RawJson | ConvertFrom-Json -} catch { - Write-Host "❌ PR #$PrNumber returned malformed JSON: $RawJson" - exit 1 -} - -if (-not $PrInfo -or -not $PrInfo.author) { - Write-Host "❌ PR #$PrNumber returned empty or malformed metadata" - exit 1 -} - -if ($PrInfo.isFork) { - Write-Host "⏭️ PR #$PrNumber is from a fork β€” skipping. Fork PRs are evaluated in the sandboxed agent container via the platform's checkout_pr_branch.cjs." - exit 1 -} - -$Permission = gh api "repos/$($env:GITHUB_REPOSITORY)/collaborators/$($PrInfo.author)/permission" --jq '.permission' -if ($LASTEXITCODE -ne 0) { - Write-Host "❌ Failed to check permissions for '$($PrInfo.author)'" - exit 1 -} - -$AllowedRoles = @('admin', 'write', 'maintain') -if ($Permission -notin $AllowedRoles) { - Write-Host "⏭️ PR author '$($PrInfo.author)' has '$Permission' access. workflow_dispatch only processes PRs from authors with write access." - exit 1 -} - -Write-Host "βœ… PR #$PrNumber by '$($PrInfo.author)' ($Permission access, same-repo)" - -# ── Save base branch SHA ───────────────────────────────────────────────────── - -$BaseSha = git rev-parse HEAD -if ($LASTEXITCODE -ne 0) { - Write-Host "❌ Failed to get current HEAD SHA" - exit 1 -} -Add-Content -Path $env:GITHUB_ENV -Value "BASE_SHA=$BaseSha" -Write-Host "Base branch SHA: $BaseSha" - -# ── Checkout PR branch ────────────────────────────────────────────────────── - -Write-Host "Checking out PR #$PrNumber..." -gh pr checkout $PrNumber --repo $env:GITHUB_REPOSITORY -if ($LASTEXITCODE -ne 0) { - Write-Host "❌ Failed to checkout PR #$PrNumber" - exit 1 -} -Write-Host "βœ… Checked out PR #$PrNumber" -git log --oneline -1 - -# ── Restore agent infrastructure from base branch ──────────────────────────── -# Replace skills and instructions with base branch versions to ensure the agent -# always uses trusted infrastructure from main. Uses git checkout to read files -# directly from the commit tree β€” works in shallow clones (no history traversal). -# Restore BEFORE deleting so a failure doesn't leave the workspace without infra. - -git checkout $BaseSha -- .github/skills/ .github/instructions/ .github/copilot-instructions.md 2>&1 -if ($LASTEXITCODE -eq 0) { - Write-Host "βœ… Restored agent infrastructure from base branch ($BaseSha)" -} else { - Write-Host "❌ Failed to restore agent infrastructure from base branch β€” aborting to prevent running with untrusted infra" - exit 1 -} diff --git a/.github/scripts/Review-PR.ps1 b/.github/scripts/Review-PR.ps1 index b4f43982bae3..acd7de121f0e 100644 --- a/.github/scripts/Review-PR.ps1 +++ b/.github/scripts/Review-PR.ps1 @@ -437,6 +437,69 @@ function Invoke-CopilotStep { return $exitCode } +# ═════════════════════════════════════════════════════════════════════════════ +# STEP 0.5: DETECT UI Test Categories (detection only β€” no pipeline trigger) +# ═════════════════════════════════════════════════════════════════════════════ + +Write-Host "" +Write-Host "╔═══════════════════════════════════════════════════════════╗" -ForegroundColor Cyan +Write-Host "β•‘ STEP 0.5: DETECT UI TEST CATEGORIES β•‘" -ForegroundColor Cyan +Write-Host "β•šβ•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•" -ForegroundColor Cyan + +$uitestCategories = "" + +$detectScript = Join-Path $RepoRoot "eng/scripts/detect-ui-test-categories.ps1" +if (Test-Path $detectScript) { + try { + $detectOutput = & pwsh -NoProfile -File $detectScript -PrNumber "$PRNumber" 2>&1 + $detectOutput | ForEach-Object { Write-Host " $_" } + + foreach ($line in $detectOutput) { + $lineStr = $line.ToString() + # Match even when the marker is followed by an empty value β€” `''` is + # the explicit "run all" sentinel emitted by the run-all returns in + # detect-ui-test-categories.ps1; treating it as "marker not seen" + # would lose that distinction. + if ($lineStr -match 'UITestCategoryList;isOutput=true\](.*)$') { + $uitestCategories = $Matches[1] + } + } + + if ($uitestCategories -eq 'NONE') { + Write-Host " ℹ️ No UI test categories needed (no UI-relevant changes)" -ForegroundColor DarkGray + } elseif ([string]::IsNullOrWhiteSpace($uitestCategories)) { + Write-Host " ℹ️ Full UI test matrix (no specific categories detected)" -ForegroundColor DarkGray + } else { + Write-Host " 🎯 Detected categories: $uitestCategories" -ForegroundColor Green + } + + # Write detection result for AI summary + $uitestOutputDir = Join-Path $RepoRoot "CustomAgentLogsTmp/PRState/$PRNumber/PRAgent/uitests" + New-Item -ItemType Directory -Force -Path $uitestOutputDir | Out-Null + if ($uitestCategories -eq 'NONE') { + "No UI test categories needed for this PR (no UI-relevant changes)." | Set-Content (Join-Path $uitestOutputDir "content.md") -Encoding UTF8 + } elseif ([string]::IsNullOrWhiteSpace($uitestCategories)) { + "Full UI test matrix will run (no specific categories detected from PR changes)." | Set-Content (Join-Path $uitestOutputDir "content.md") -Encoding UTF8 + } else { + "**Detected UI test categories:** ``$uitestCategories``" | Set-Content (Join-Path $uitestOutputDir "content.md") -Encoding UTF8 + } + } catch { + Write-Host " ⚠️ Category detection failed (non-fatal): $_" -ForegroundColor Yellow + } +} else { + Write-Host " ⚠️ detect-ui-test-categories.ps1 not found" -ForegroundColor Yellow +} + +# Belt-and-suspenders: the detect script's manual-PR mode does +# `git checkout $headSha`, leaving HEAD detached. Its own try/finally restores +# the previous ref, but if that finally is skipped (process killed, scripting +# error before the outer try opens) we'd run Step 1's gate against the wrong +# tree. Force HEAD back to the review branch and fail loudly if we can't. +git checkout $reviewBranch 2>$null | Out-Null +if ($LASTEXITCODE -ne 0) { + Write-Host " ⚠️ Failed to restore review branch '$reviewBranch' after Step 0.5 β€” Step 1 may run against the wrong tree" -ForegroundColor Red +} + # ═════════════════════════════════════════════════════════════════════════════ # STEP 1: Gate - Test Before and After Fix (script, no copilot agent) # ═════════════════════════════════════════════════════════════════════════════ @@ -451,17 +514,79 @@ New-Item -ItemType Directory -Force -Path $gateOutputDir | Out-Null # Detect tests in PR Write-Host " πŸ” Detecting tests in PR #$PRNumber..." -ForegroundColor Cyan -$detectScript = Join-Path $PSScriptRoot "shared/Detect-TestsInDiff.ps1" -& pwsh -NoProfile -Command "& '$detectScript' -PRNumber $PRNumber" 2>&1 | ForEach-Object { Write-Host " $_" } +$testDetectScript = Join-Path $PSScriptRoot "shared/Detect-TestsInDiff.ps1" +if (Test-Path $testDetectScript) { + $testDetectScript = (Resolve-Path $testDetectScript).Path + & pwsh -NoProfile -File $testDetectScript -PRNumber $PRNumber 2>&1 | ForEach-Object { Write-Host " $_" } +} else { + Write-Host " ⚠️ Detect-TestsInDiff.ps1 not found at $testDetectScript" -ForegroundColor Yellow +} # Determine platform for gate $gatePlatform = if ($Platform) { $Platform } else { "android" } Write-Host " πŸ§ͺ Running gate on platform: $gatePlatform" -ForegroundColor Cyan -$verifyScript = Join-Path $PSScriptRoot "../skills/verify-tests-fail-without-fix/scripts/verify-tests-fail.ps1" -$gateOutput = & pwsh -NoProfile -File "$verifyScript" -Platform $gatePlatform -PRNumber $PRNumber -RequireFullVerification 2>&1 -$gateExitCode = $LASTEXITCODE -$gateOutput | ForEach-Object { Write-Host " $_" } +$verifyScript = [System.IO.Path]::GetFullPath((Join-Path $PSScriptRoot "../skills/verify-tests-fail-without-fix/scripts/verify-tests-fail.ps1")) +if (-not (Test-Path $verifyScript)) { + Write-Host " ❌ verify-tests-fail.ps1 not found at: $verifyScript" -ForegroundColor Red + # $gateExitCode = 1 ensures the switch at line ~561 produces $gateResult = "FAILED" + $gateExitCode = 1 + $gateOutput = @("verify-tests-fail.ps1 not found at: $verifyScript") +} else { + +$maxGateAttempts = 3 +$gateExitCode = 1 +$gateOutput = @() +# Path is fixed across attempts β€” define once, then clear per-iteration so a stale +# report from attempt N-1 can't be misclassified as the current attempt's output. +$gateContentFile = Join-Path $gateOutputDir "verify-tests-fail/verification-report.md" + +for ($gateAttempt = 1; $gateAttempt -le $maxGateAttempts; $gateAttempt++) { + if ($gateAttempt -gt 1) { + Write-Host " πŸ”„ Retry $gateAttempt/$maxGateAttempts β€” previous attempt hit environment error" -ForegroundColor Yellow + } + # Clear previous attempt's report so a crash mid-run doesn't leak its classification into this one. + Remove-Item $gateContentFile -Force -ErrorAction SilentlyContinue + $gateOutput = & pwsh -NoProfile -File "$verifyScript" -Platform $gatePlatform -PRNumber $PRNumber -RequireFullVerification 2>&1 + $gateExitCode = $LASTEXITCODE + $gateOutput | ForEach-Object { Write-Host " $_" } + + # Check if this was an ENV ERROR (emulator timeout, ADB failure, etc.) + $isEnvError = $false + if ($gateExitCode -ne 0) { + if (Test-Path $gateContentFile) { + $gateContent = Get-Content $gateContentFile -Raw -ErrorAction SilentlyContinue + if ($gateContent -match 'ENV ERROR') { + $isEnvError = $true + Write-Host " ⚠️ Environment error detected (attempt $gateAttempt/$maxGateAttempts)" -ForegroundColor Yellow + } + } else { + # Verify script crashed BEFORE writing the report (e.g., emulator failed to + # start, ADB crash during setup, OOM kill). The most severe infra failures + # never reach the report-writing path, so a missing report alongside a + # non-zero exit is itself a strong signal we should retry rather than break. + $isEnvError = $true + Write-Host " ⚠️ Verification report missing after non-zero exit β€” treating as infra failure (attempt $gateAttempt/$maxGateAttempts)" -ForegroundColor Yellow + } + } + + if ($gateExitCode -eq 0 -or -not $isEnvError) { + break # Real pass or real failure β€” don't retry + } + if ($gateAttempt -lt $maxGateAttempts) { + Write-Host " ⏳ Waiting 30s before retry..." -ForegroundColor DarkGray + Start-Sleep -Seconds 30 + } +} +if ($isEnvError) { + # Reachable only if EVERY iteration was an env error: real pass/fail + # iterations `break` out of the loop (so $isEnvError would be reset to $false + # at the top of the next iteration but we'd never get here). $isEnvError + # here means "all $maxGateAttempts attempts hit env errors" β€” not "any". + Write-Host " ⚠️ All $maxGateAttempts gate attempts hit environment errors" -ForegroundColor Yellow +} + +} # end else (verify script exists) # Exit code: 0 = passed, 1 = verification failed, 2 = no tests detected $gateResult = switch ($gateExitCode) { @@ -472,35 +597,69 @@ $gateResult = switch ($gateExitCode) { $gateColor = switch ($gateResult) { "PASSED" { "Green" } "SKIPPED" { "Yellow" } default { "Red" } } Write-Host " πŸ“ Gate result: $gateResult" -ForegroundColor $gateColor -# Copy the verification report to gate/content.md if it exists +# Copy the verification report to gate/content.md (always overwrite β€” the report is the source of truth) $verificationReport = Join-Path $gateOutputDir "verify-tests-fail/verification-report.md" +# Capture last meaningful lines from gate output for fallback diagnostics +$gateLogTail = @($gateOutput | ForEach-Object { $_.ToString() } | Where-Object { $_ -match '\S' } | Select-Object -Last 20) -join "`n" + if (Test-Path $verificationReport) { - Copy-Item $verificationReport (Join-Path $gateOutputDir "content.md") -Force + $reportContent = Get-Content $verificationReport -Raw -ErrorAction SilentlyContinue + if ($reportContent) { + # Strip broken "Test Summary" blocks with empty values (from old verify script format) + $reportContent = $reportContent -replace '(?s)\*\*Test Summary:\*\*\s*\n- Total:\s*\n- Passed:\s*(True|False)\s*\n- Failed:\s*\n- Skipped:\s*\n?', '' + $reportContent | Set-Content (Join-Path $gateOutputDir "content.md") -Encoding UTF8 + } else { + # Report exists but has bad format β€” generate fallback with logs + Write-Host " ⚠️ Verification report has invalid format β€” using fallback" -ForegroundColor Yellow + $resultIcon = switch ($gateResult) { "PASSED" { "βœ…" } "SKIPPED" { "⚠️" } default { "❌" } } + @" +### Gate Result: $resultIcon $gateResult + +**Platform:** $($gatePlatform.ToUpper()) + +
+Gate output log + +`````` +$gateLogTail +`````` + +
+"@ | Set-Content (Join-Path $gateOutputDir "content.md") -Encoding UTF8 + } } elseif (-not (Test-Path (Join-Path $gateOutputDir "content.md"))) { - # Create gate content based on result if ($gateResult -eq "SKIPPED") { - $skipContent = @" + @" ### Gate Result: ⚠️ SKIPPED No tests were detected in this PR. -**Recommendation:** Add tests to verify the fix using the ``write-tests-agent``: +**Recommendation:** Add tests to verify the fix using the ``write-tests-agent``. +"@ | Set-Content (Join-Path $gateOutputDir "content.md") -Encoding UTF8 + } else { + $resultIcon = switch ($gateResult) { "PASSED" { "βœ…" } default { "❌" } } + @" +### Gate Result: $resultIcon $gateResult + +**Platform:** $($gatePlatform.ToUpper()) + +
+Gate output log `````` -@copilot write tests for this PR +$gateLogTail `````` -The agent will analyze the issue, determine the appropriate test type (UI test, device test, unit test, or XAML test), and create tests that verify the fix. -"@ - $skipContent | Set-Content (Join-Path $gateOutputDir "content.md") - } else { - "### Gate Result: $(if ($gateExitCode -eq 0) { 'βœ… PASSED' } else { '❌ FAILED' })`n`n**Platform:** $gatePlatform" | - Set-Content (Join-Path $gateOutputDir "content.md") +
+"@ | Set-Content (Join-Path $gateOutputDir "content.md") -Encoding UTF8 } } -# Post gate result as a separate PR comment -$postGateScript = Join-Path $PSScriptRoot "post-gate-comment.ps1" +# Post gate result by updating (or creating) the unified AI Summary comment. +# The same script is called again in STEP 3 once the review phases finish; here +# it runs early so the PR author sees the gate outcome without waiting for the +# full review. +$postGateScript = Join-Path $PSScriptRoot "post-ai-summary-comment.ps1" if (Test-Path $postGateScript) { try { if ($DryRun) { @@ -509,10 +668,10 @@ if (Test-Path $postGateScript) { & $postGateScript -PRNumber $PRNumber } } catch { - Write-Host " ⚠️ Failed to post gate comment (non-fatal): $_" -ForegroundColor Yellow + Write-Host " ⚠️ Failed to post gate section (non-fatal): $_" -ForegroundColor Yellow } } else { - Write-Host " ⚠️ post-gate-comment.ps1 not found" -ForegroundColor Yellow + Write-Host " ⚠️ post-ai-summary-comment.ps1 not found" -ForegroundColor Yellow } # Apply gate result label @@ -563,8 +722,10 @@ $autonomousRules **Gate result (already completed in a prior step):** $gateStatusForPrompt Do NOT re-run gate verification. The gate phase is handled separately. +⚠️ Do NOT create or overwrite ``gate/content.md`` β€” it is already generated by the gate script with detailed test output. πŸ“ Write phase output to ``CustomAgentLogsTmp/PRState/$PRNumber/PRAgent/{phase}/content.md`` +(phases: pre-flight, try-fix, report β€” NOT gate) "@ Invoke-CopilotStep -StepName "STEP 2: PR REVIEW" -Prompt $step2Prompt | Out-Null @@ -572,6 +733,42 @@ Invoke-CopilotStep -StepName "STEP 2: PR REVIEW" -Prompt $step2Prompt | Out-Null # Restore review branch β€” the Copilot agent may have switched branches (e.g. via gh pr checkout) git checkout $reviewBranch 2>$null | Out-Null +# ─── Tier 3 refresh: feed AI categories back into category detection ─── +# Step 0.5 ran detection without the AI tier (-AiCategories was empty). +# Pre-flight (Step 2) wrote `ai-categories.md`; re-run detection now so the +# unified comment reflects all three tiers before Step 3 posts. +$aiCategoriesFile = Join-Path $RepoRoot "CustomAgentLogsTmp/PRState/$PRNumber/PRAgent/uitests/ai-categories.md" +if ((Test-Path $detectScript) -and (Test-Path $aiCategoriesFile)) { + try { + # Pass as a single string (the script declares [string]$AiCategories); + # an array would not bind correctly across the pwsh -File boundary. + $aiCategoriesArg = (Get-Content $aiCategoriesFile -Raw).Trim() + if (-not [string]::IsNullOrWhiteSpace($aiCategoriesArg)) { + Write-Host " πŸ” Refreshing UI category detection with AI tier..." -ForegroundColor Cyan + $refreshOutput = & pwsh -NoProfile -File $detectScript -PrNumber "$PRNumber" -AiCategories $aiCategoriesArg 2>&1 + $refreshOutput | ForEach-Object { Write-Host " $_" } + + $refreshedCategories = $uitestCategories + foreach ($line in $refreshOutput) { + if ($line.ToString() -match 'UITestCategoryList;isOutput=true\](.*)$') { + $refreshedCategories = $Matches[1] + } + } + + $uitestOutputDir = Join-Path $RepoRoot "CustomAgentLogsTmp/PRState/$PRNumber/PRAgent/uitests" + if ($refreshedCategories -eq 'NONE') { + "No UI test categories needed for this PR (no UI-relevant changes)." | Set-Content (Join-Path $uitestOutputDir "content.md") -Encoding UTF8 + } elseif ([string]::IsNullOrWhiteSpace($refreshedCategories)) { + "Full UI test matrix will run (no specific categories detected from PR changes)." | Set-Content (Join-Path $uitestOutputDir "content.md") -Encoding UTF8 + } else { + "**Detected UI test categories:** ``$refreshedCategories``" | Set-Content (Join-Path $uitestOutputDir "content.md") -Encoding UTF8 + } + } + } catch { + Write-Host " ⚠️ AI-tier category refresh failed (non-fatal, keeping Step 0.5 result): $_" -ForegroundColor Yellow + } +} + # ═════════════════════════════════════════════════════════════════════════════ # STEP 3: Post AI Summary Comment (direct script invocation) # ═════════════════════════════════════════════════════════════════════════════ diff --git a/.github/scripts/post-ai-summary-comment.ps1 b/.github/scripts/post-ai-summary-comment.ps1 index b53807632259..6c5781e4bce9 100644 --- a/.github/scripts/post-ai-summary-comment.ps1 +++ b/.github/scripts/post-ai-summary-comment.ps1 @@ -13,9 +13,16 @@ After posting, the PR author is @-mentioned so they know to review. Content is auto-loaded from PRAgent phase files: + CustomAgentLogsTmp/PRState//PRAgent/gate/content.md (always shown first, open) CustomAgentLogsTmp/PRState//PRAgent/{pre-flight,try-fix,report}/content.md + CustomAgentLogsTmp/PRState//PRAgent/pre-flight/code-review.md - Gate is posted separately by post-gate-comment.ps1. + Gate is included as a section inside this unified comment β€” the script may + be called by Review-PR.ps1 twice per run: once after the gate completes + (gate-only update) and once after the review phases finish (full update). + + Any standalone legacy "" comment from older versions of + the script is deleted after a successful post to avoid duplicates. .PARAMETER PRNumber The pull request number (required) @@ -60,9 +67,35 @@ if (-not (Test-Path $PRAgentDir)) { } $phases = [ordered]@{ - "pre-flight" = @{ File = "pre-flight/content.md"; Icon = "πŸ”"; Title = "Pre-Flight β€” Context & Validation" } - "try-fix" = @{ File = "try-fix/content.md"; Icon = "πŸ”§"; Title = "Fix β€” Analysis & Comparison" } - "report" = @{ File = "report/content.md"; Icon = "πŸ“‹"; Title = "Report β€” Final Recommendation" } + "uitests" = @{ File = "uitests/content.md"; Icon = "πŸ§ͺ"; Title = "UI Tests β€” Category Detection" } + "pre-flight" = @{ File = "pre-flight/content.md"; Icon = "πŸ”"; Title = "Pre-Flight β€” Context & Validation" } + "code-review" = @{ File = "pre-flight/code-review.md"; Icon = "πŸ”¬"; Title = "Code Review β€” Deep Analysis" } + "try-fix" = @{ File = "try-fix/content.md"; Icon = "πŸ”§"; Title = "Fix β€” Analysis & Comparison" } + "report" = @{ File = "report/content.md"; Icon = "πŸ“‹"; Title = "Report β€” Final Recommendation" } +} + +# ─── Gate content (rendered first, always open) ─── +$gateSection = $null +$gateFilePath = Join-Path $PRAgentDir "gate/content.md" +if (Test-Path $gateFilePath) { + $gateContent = Get-Content $gateFilePath -Raw -Encoding UTF8 + if (-not [string]::IsNullOrWhiteSpace($gateContent)) { + Write-Host " βœ… gate ($((Get-Item $gateFilePath).Length) bytes)" -ForegroundColor Green + $gateSection = @" +
+🚦 Gate β€” Test Before & After Fix + +--- + +$gateContent + +
+"@ + } else { + Write-Host " ⏭️ gate (empty)" -ForegroundColor Gray + } +} else { + Write-Host " ⏭️ gate (not found)" -ForegroundColor Gray } $phaseSections = @() @@ -93,8 +126,8 @@ $content } } -if ($phaseSections.Count -eq 0) { - throw "No phase content found. Ensure at least one content.md exists in $PRAgentDir." +if (-not $gateSection -and $phaseSections.Count -eq 0) { + throw "No gate or phase content found. Ensure at least one of gate/content.md or {phase}/content.md exists in $PRAgentDir." } # ============================================================================ @@ -123,7 +156,13 @@ $timestamp = (Get-Date).ToUniversalTime().ToString("yyyy-MM-dd HH:mm UTC") # BUILD NEW SESSION BLOCK # ============================================================================ -$phaseContent = $phaseSections -join "`n`n---`n`n" +# Combine gate (always first, open) with phases (collapsed). When only one +# kind of content is available, the session still renders cleanly. +$sessionParts = @() +if ($gateSection) { $sessionParts += $gateSection } +if ($phaseSections.Count -gt 0) { $sessionParts += ($phaseSections -join "`n`n---`n`n") } +$phaseContent = $sessionParts -join "`n`n---`n`n" + $sessionMarkerStart = "" $sessionMarkerEnd = "" @@ -176,12 +215,17 @@ function Merge-Sessions { foreach ($sha in $orderedKeys) { $block = $sessions[$sha] if ($isFirst) { - # Ensure latest session has
- $block = $block -replace '', '
' + # Ensure ONLY the outer (session-wrapping) details tag is open. Inner + # phase tags must keep their original open/collapsed state β€” we used + # to re-open all of them via a global regex replace, which forced + # every phase to expand on each new session. + $rx = [regex]::new('') + $block = $rx.Replace($block, '
', 1) $isFirst = $false } else { - # Collapse older sessions - $block = $block -replace '', '
' + # Collapse the outer details of older sessions; leave inner phases alone. + $rx = [regex]::new('') + $block = $rx.Replace($block, '
', 1) } $allSessions += $block } @@ -300,3 +344,25 @@ try { } finally { Remove-Item $tempFile -ErrorAction SilentlyContinue } + +# ============================================================================ +# CLEAN UP LEGACY STANDALONE GATE COMMENTS +# ============================================================================ +# Earlier versions of this workflow posted gate results in a separate comment +# marked with . Now that the gate is included as a section in +# this unified comment, those legacy comments are duplicates and should go. + +try { + $legacyMarker = "" + $allRaw = gh api "repos/dotnet/maui/issues/$PRNumber/comments" --paginate 2>$null + if ($allRaw) { + $allComments = $allRaw | ConvertFrom-Json + $legacy = @($allComments | Where-Object { $_.body -and $_.body.Contains($legacyMarker) }) + foreach ($lc in $legacy) { + Write-Host "🧹 Deleting legacy gate comment (ID: $($lc.id))..." -ForegroundColor Gray + gh api --method DELETE "repos/dotnet/maui/issues/comments/$($lc.id)" 2>&1 | Out-Null + } + } +} catch { + Write-Host "⚠️ Legacy gate-comment cleanup failed (non-fatal): $_" -ForegroundColor Yellow +} diff --git a/.github/scripts/post-gate-comment.ps1 b/.github/scripts/post-gate-comment.ps1 deleted file mode 100644 index fcf365f2f286..000000000000 --- a/.github/scripts/post-gate-comment.ps1 +++ /dev/null @@ -1,271 +0,0 @@ -#!/usr/bin/env pwsh -<# -.SYNOPSIS - Posts or updates the gate verification comment on a GitHub Pull Request. - -.DESCRIPTION - Maintains ONE comment per PR, identified by marker. - Each gate run adds an expandable session keyed by HEAD commit SHA. - - Same commit SHA β†’ replaces that session in-place. - - New commit SHA β†’ prepends a new session (latest first). - Older sessions stay collapsed; the newest is expanded by default. - - After posting, the PR author is @-mentioned so they know to review. - - Reads content from CustomAgentLogsTmp/PRState//PRAgent/gate/content.md. - -.PARAMETER PRNumber - The pull request number (required) - -.PARAMETER DryRun - Print comment instead of posting - -.EXAMPLE - ./post-gate-comment.ps1 -PRNumber 12345 - -.EXAMPLE - ./post-gate-comment.ps1 -PRNumber 12345 -DryRun -#> - -param( - [Parameter(Mandatory = $true)] - [int]$PRNumber, - - [Parameter(Mandatory = $false)] - [switch]$DryRun -) - -$ErrorActionPreference = "Stop" -$MARKER = "" - -# ============================================================================ -# LOAD GATE CONTENT -# ============================================================================ - -$gateContentPath = "CustomAgentLogsTmp/PRState/$PRNumber/PRAgent/gate/content.md" -if (-not (Test-Path $gateContentPath)) { - $repoRoot = git rev-parse --show-toplevel 2>$null - if ($repoRoot) { - $gateContentPath = Join-Path $repoRoot "CustomAgentLogsTmp/PRState/$PRNumber/PRAgent/gate/content.md" - } -} - -if (-not (Test-Path $gateContentPath)) { - Write-Host "⚠️ No gate content found at: $gateContentPath" -ForegroundColor Yellow - exit 0 -} - -$gateContent = Get-Content $gateContentPath -Raw -Encoding UTF8 -if ([string]::IsNullOrWhiteSpace($gateContent)) { - Write-Host "⚠️ Gate content is empty" -ForegroundColor Yellow - exit 0 -} - -Write-Host "βœ… Loaded gate content ($($gateContent.Length) chars)" -ForegroundColor Green - -# ============================================================================ -# FETCH PR METADATA (commit + author) -# ============================================================================ - -try { - $commitJson = gh api "repos/dotnet/maui/pulls/$PRNumber/commits" --jq '.[-1] | {message: .commit.message, sha: .sha}' 2>$null | ConvertFrom-Json -} catch { - Write-Host "⚠️ Failed to fetch commit info: $_" -ForegroundColor Yellow - $commitJson = $null -} -$commitTitle = if ($commitJson) { ($commitJson.message -split "`n")[0] } else { "Unknown" } -$commitTitle = $commitTitle -replace '&','&' -replace '<','<' -replace '>','>' -$commitSha7 = if ($commitJson) { $commitJson.sha.Substring(0, 7) } else { "unknown" } -$commitFull = if ($commitJson) { $commitJson.sha } else { "" } -$commitUrl = if ($commitJson) { "https://github.com/dotnet/maui/commit/$commitFull" } else { "#" } - -try { - $prAuthor = gh api "repos/dotnet/maui/pulls/$PRNumber" --jq '.user.login' 2>$null -} catch { $prAuthor = $null } - -$timestamp = (Get-Date).ToUniversalTime().ToString("yyyy-MM-dd HH:mm UTC") - -# ============================================================================ -# BUILD NEW SESSION BLOCK -# ============================================================================ - -$sessionMarkerStart = "" -$sessionMarkerEnd = "" - -$newSessionBlock = @" -$sessionMarkerStart -
-🚦 Gate Session β€” $commitSha7 Β· $commitTitle Β· $timestamp - ---- - -$gateContent - ---- - -
-$sessionMarkerEnd -"@ - -# ============================================================================ -# MERGE WITH EXISTING SESSIONS -# ============================================================================ - -function Merge-Sessions { - param( - [string]$ExistingBody, - [string]$NewSession, - [string]$CommitSha7 - ) - - $sessionPattern = '(?s).*?' - $existingSessions = [regex]::Matches($ExistingBody, $sessionPattern) - - $sessions = [ordered]@{} - foreach ($match in $existingSessions) { - $sha = $match.Groups[1].Value - $sessions[$sha] = $match.Value - } - - $sessions[$CommitSha7] = $NewSession - - $orderedKeys = @($CommitSha7) + @($sessions.Keys | Where-Object { $_ -ne $CommitSha7 }) - - $allSessions = @() - $isFirst = $true - foreach ($sha in $orderedKeys) { - $block = $sessions[$sha] - if ($isFirst) { - $block = $block -replace '', '
' - $isFirst = $false - } else { - $block = $block -replace '', '
' - } - $allSessions += $block - } - - return ($allSessions -join "`n`n---`n`n") -} - -# ============================================================================ -# FIND EXISTING COMMENT & BUILD FINAL BODY -# ============================================================================ - -Write-Host "Checking for existing gate comment..." -ForegroundColor Yellow -$existingCommentId = $null -$existingBody = $null - -$existingRaw = gh api "repos/dotnet/maui/issues/$PRNumber/comments" --paginate 2>$null -$existingObj = $null -if ($existingRaw) { - try { - $allComments = $existingRaw | ConvertFrom-Json - $existingObj = @($allComments | Where-Object { $_.body -and $_.body.Contains($MARKER) }) | Select-Object -Last 1 - } catch { - Write-Host "⚠️ Could not parse comments: $_" -ForegroundColor Yellow - } -} - -if ($existingObj -and $existingObj.id) { - $existingCommentId = $existingObj.id - $existingBody = $existingObj.body - Write-Host "βœ“ Found existing comment (ID: $existingCommentId)" -ForegroundColor Green -} - -$authorPing = "" -if ($prAuthor) { - $authorPing = "> πŸ‘‹ @$prAuthor β€” new gate results are available. Please review the latest session below." -} - -if ($existingBody) { - $mergedSessions = Merge-Sessions -ExistingBody $existingBody -NewSession $newSessionBlock -CommitSha7 $commitSha7 - - $commentBody = @" -$MARKER - -## 🚦 Gate β€” Test Before and After Fix - -$authorPing - -$mergedSessions -"@ -} else { - $commentBody = @" -$MARKER - -## 🚦 Gate β€” Test Before and After Fix - -$authorPing - -$newSessionBlock -"@ -} - -$commentBody = $commentBody -replace "`n{4,}", "`n`n`n" - -# ============================================================================ -# DRY RUN -# ============================================================================ - -if ($DryRun) { - Write-Host "" - Write-Host "=== GATE COMMENT PREVIEW ===" -ForegroundColor Cyan - Write-Host $commentBody - Write-Host "=== END PREVIEW ===" -ForegroundColor Cyan - exit 0 -} - -# ============================================================================ -# POST OR UPDATE COMMENT -# ============================================================================ - -$tempFile = [System.IO.Path]::GetTempFileName() -try { - @{ body = $commentBody } | ConvertTo-Json -Depth 10 | Set-Content -Path $tempFile -Encoding UTF8 - - if ($existingCommentId) { - Write-Host "Updating gate comment (ID: $existingCommentId)..." -ForegroundColor Yellow - try { - gh api --method PATCH "repos/dotnet/maui/issues/comments/$existingCommentId" --input $tempFile 2>&1 | Out-Null - if ($LASTEXITCODE -ne 0) { throw "PATCH failed" } - Write-Host "βœ… Gate comment updated" -ForegroundColor Green - Write-Output "COMMENT_ID=$existingCommentId" - } catch { - Write-Host "⚠️ Could not update comment $existingCommentId : $_" -ForegroundColor Yellow - $botLogin = gh api user --jq .login 2>$null - if ($botLogin) { - $ownRaw = gh api "repos/dotnet/maui/issues/$PRNumber/comments" --paginate 2>$null - $ownCommentId = $null - if ($ownRaw) { - try { - $ownAll = $ownRaw | ConvertFrom-Json - $ownMatch = @($ownAll | Where-Object { $_.body -and $_.body.Contains($MARKER) -and $_.user.login -eq $botLogin }) | Select-Object -Last 1 - if ($ownMatch) { $ownCommentId = $ownMatch.id } - } catch { } - } - if ($ownCommentId -and $ownCommentId -ne "null") { - Write-Host " Retrying with own comment (ID: $ownCommentId)..." -ForegroundColor Yellow - gh api --method PATCH "repos/dotnet/maui/issues/comments/$ownCommentId" --input $tempFile 2>&1 | Out-Null - if ($LASTEXITCODE -eq 0) { - Write-Host "βœ… Gate comment updated (own comment)" -ForegroundColor Green - Write-Output "COMMENT_ID=$ownCommentId" - return - } - } - } - Write-Host " Creating new comment as fallback..." -ForegroundColor Yellow - $newJson = gh api --method POST "repos/dotnet/maui/issues/$PRNumber/comments" --input $tempFile - $newId = ($newJson | ConvertFrom-Json).id - Write-Host "βœ… Gate comment posted (ID: $newId)" -ForegroundColor Green - Write-Output "COMMENT_ID=$newId" - } - } else { - Write-Host "Creating new gate comment..." -ForegroundColor Yellow - $newJson = gh api --method POST "repos/dotnet/maui/issues/$PRNumber/comments" --input $tempFile - $newId = ($newJson | ConvertFrom-Json).id - Write-Host "βœ… Gate comment posted (ID: $newId)" -ForegroundColor Green - Write-Output "COMMENT_ID=$newId" - } -} finally { - Remove-Item $tempFile -ErrorAction SilentlyContinue -} diff --git a/.github/skills/azdo-build-investigator/SKILL.md b/.github/skills/azdo-build-investigator/SKILL.md index bd3ddc905332..f361fd45e11b 100644 --- a/.github/skills/azdo-build-investigator/SKILL.md +++ b/.github/skills/azdo-build-investigator/SKILL.md @@ -32,6 +32,10 @@ The `ci-analysis` skill and its `Get-CIStatus.ps1` script are loaded automatical Most failures are in `maui-pr`. Device test failures appear in `maui-pr-devicetests`. Focus on the first failing pipeline before checking others. +**When CI hasn't run:** Community PRs require a maintainer to trigger builds. Use `/azp run maui-pr` (or `maui-pr-devicetests`, `maui-pr-uitests`) in a PR comment, or trigger via Azure CLI. Not all pipelines run automatically β€” `maui-pr-devicetests` and `maui-pr-uitests` may need explicit triggers depending on the changed files. + +**Escalation:** For deep Helix log analysis (recurring failures, machine-specific issues, comparing passing vs. failing runs), escalate to the `helix-investigation` skill. + ## MAUI-Specific Quirks ### XHarness Exit-0 Blind Spot @@ -66,6 +70,37 @@ If available, use the `mcp-binlog-tool` MCP server to analyze downloaded `.binlo | `error CS####` | `maui-pr` | C# compiler error β€” check file/line | | `error XA####` | `maui-pr` | Android build error | | `XamlC` | `maui-pr` | XAML compiler β€” usually missing type or bad binding | +| `error XAGRDL0000` / `401` / `No local versions` | `maui-pr` or official build | Gradle/Maven feed issue β€” see below | | `XHarness timeout` | `maui-pr-devicetests` Helix logs | Test killed by infrastructure; may be transient | | `No test result files found` | `maui-pr-devicetests` Helix logs | Tests never ran or app crashed on launch | | UI test screenshot diff | `maui-pr-uitests` | Visual regression; check baseline images | + +## Test Count Deduplication + +When querying AzDO test results directly (e.g., via the `/test/runs/{id}/results` API), **always deduplicate before reporting counts**. MAUI UI tests produce multiple test runs per test because each test executes across: +- **Runtime variants**: CoreCLR and Mono +- **Platform versions**: e.g., iOS 18.5 and iOS latest, Android API 30 and API 36 +- **Retry attempts**: failed jobs are retried, each attempt publishes a new test run + +A single failing test can appear in 4–8+ test runs. Summing raw `totalTests - passedTests` across all runs inflates failure counts dramatically. + +**How to deduplicate**: Group by **test name + OS platform** (extract the OS token β€” `ios`, `android`, `mac`, `win` β€” from the run name as the grouping key). For example, "DatePicker_Format_D on iOS" vs "DatePicker_Format_D on Android" are distinct failures worth reporting separately. Collapse retries and runtime variants (coreclr/mono) of the same test on the same OS β€” if a test fails on both coreclr and mono for iOS, that's one issue, not two. + +### Gradle / Maven / CFSClean Failures + +**Error signatures:** +``` +error XAGRDL0000: Could not resolve com.android.tools.build:gradle:8.11.1 + > Received status code 401: Unauthorized - No local versions of package +``` +``` +error XAGRDL0000: Could not GET '...pkgs.dev.azure.com/.../maven/v1/...' + > Unauthorized - Please provide authentication to save package from upstream +``` + +**Fix:** Tell the user to run `./eng/ingest-maven-deps.sh` locally to pre-ingest packages into the feed. + +**Do NOT:** +- Remove CFSClean from `ci-official.yml` β€” security compliance requirement +- Upgrade Gradle past 8.x β€” `dotnet/android#10738` +- Add `mavenCentral()` or `google()` back β€” use the Azure Artifacts feed diff --git a/.github/skills/code-review/SKILL.md b/.github/skills/code-review/SKILL.md index ab07ead11c2f..3f0b145ffa04 100644 --- a/.github/skills/code-review/SKILL.md +++ b/.github/skills/code-review/SKILL.md @@ -17,7 +17,7 @@ Standalone skill that evaluates PR code changes for correctness, safety, perform **Do NOT use for:** "what does PR #XXXXX do?", "summarize PR", "describe the changes", or any informational query β€” just answer those directly without invoking this skill. > **How this differs from other skills:** -> - **`pr-review`** β€” End-to-end PR workflow (4 phases: pre-flight, gate, try-fix, report). Use when you want the full pipeline including test verification and fix attempts. +> - **`pr-review`** β€” End-to-end PR workflow (3 phases: pre-flight with code review, try-fix, report; gate runs separately). Use when you want the full pipeline including test verification and fix attempts. Pre-Flight invokes this skill as a sub-agent for independence-first code analysis. > - **`pr-finalize`** β€” Verifies PR title/description match implementation + light code review. Use before merging. > - **`code-review`** (this skill) β€” Deep code-only review with MAUI domain rules. Use when you want a thorough code analysis without running tests or modifying the PR. diff --git a/.github/skills/pr-review/SKILL.md b/.github/skills/pr-review/SKILL.md index fa17c664d410..eab358b1d112 100644 --- a/.github/skills/pr-review/SKILL.md +++ b/.github/skills/pr-review/SKILL.md @@ -18,7 +18,7 @@ End-to-end PR review workflow that orchestrates phases to explore independent fi ``` Gate (pre-run) β†’ Already completed by Review-PR.ps1 before this skill runs -Phase 1: Pre-Flight β†’ Gather context, classify files β†’ .github/pr-review/pr-preflight.md +Phase 1: Pre-Flight β†’ Gather context, classify files, code review β†’ .github/pr-review/pr-preflight.md Phase 2: Try-Fix β†’ ⚠️ MANDATORY multi-model exploration β†’ invoke try-fix skill (Γ—4 models) Phase 3: Report β†’ Write review recommendation β†’ .github/pr-review/pr-report.md ``` @@ -26,6 +26,7 @@ Phase 3: Report β†’ Write review recommendation β†’ .g > **Gate and Branch setup** are handled by `Review-PR.ps1` before this skill is invoked. The gate result is passed in the prompt. Do NOT re-run gate verification. **All phases write output to:** `CustomAgentLogsTmp/PRState/{PRNumber}/PRAgent/{phase}/content.md` +**Pre-Flight also writes:** `CustomAgentLogsTmp/PRState/{PRNumber}/PRAgent/pre-flight/code-review.md` --- @@ -44,7 +45,7 @@ Phase 3: Report β†’ Write review recommendation β†’ .g ### Multi-Model Configuration -Phase 3 uses these 4 AI models (run SEQUENTIALLY β€” they modify the same files): +Phase 2 uses these 4 AI models (run SEQUENTIALLY β€” they modify the same files): | Order | Model | |-------|-------| @@ -71,10 +72,20 @@ Phase 3 uses these 4 AI models (run SEQUENTIALLY β€” they modify the same files) > Read and follow `.github/pr-review/pr-preflight.md` -Gather context from the issue, PR, comments, and classify changed files. +Gather context from the issue, PR, comments, classify changed files, and **perform a deep code review** using the `code-review` skill. + +Pre-Flight now has two parts: +- **Part A (Steps 1–6):** Context gathering β€” read issue, PR, comments, classify files +- **Part B (Step 7):** Code review β€” independence-first code analysis using `.github/skills/code-review/SKILL.md` and `.github/skills/code-review/references/review-rules.md` + +**Outputs:** +- `pre-flight/content.md` β€” Context + code review summary +- `pre-flight/code-review.md` β€” Full code-review output (findings, blast radius, failure-mode probes, verdict) **Gate:** None β€” always runs. +**Why code review runs here:** The code-review findings (❌ Errors, ⚠️ Warnings, failure-mode probes, blast radius) become **structured hints for Phase 2 (Try-Fix)**. Instead of each model starting from scratch, they receive concrete code concerns to address, leading to higher-quality fix exploration. + --- ## Phase 2: Try-Fix β†’ Invoke `try-fix` Skill (Γ—4 Models) @@ -87,6 +98,8 @@ Even if the PR's fix looks correct and Gate passed, you MUST still run all 4 mod ### 🚨 CRITICAL: try-fix is Independent of PR's Fix +"Independent" means each model explores a **different fix approach** from the PR's fix β€” not that models are isolated from code-review context. Code-review findings are provided as advisory background to improve fix quality. + The purpose is NOT to re-test the PR's fix, but to: 1. **Generate independent fix ideas** β€” What would YOU do to fix this bug? 2. **Test those ideas empirically** β€” Actually implement and run tests @@ -119,10 +132,27 @@ prompt: | - target_files: - src/{area}/{file1}.cs - src/{area}/{file2}.cs + - hints: | + Code review found the following concerns (advisory β€” use to inform your approach, not as a checklist): + Errors: + - {❌ Error finding 1 with file:line reference} + # Include warnings ONLY if relevant to the root cause: + # Warnings: + # - {⚠️ Warning β€” omit if unrelated to root cause} + Failure modes: + - {Failure mode 1}: {What happens in this scenario} + Blast radius: {Summary β€” e.g., "Runs for ALL toolbar items at startup, not just badged ones"} + Code review verdict: {LGTM / NEEDS_CHANGES / NEEDS_DISCUSSION} (confidence: {high/medium/low}) Generate ONE independent fix idea. Review the PR's fix first to ensure your approach is DIFFERENT. + "Independent" means exploring a different fix approach β€” the code review context above is background + information to help you make better decisions, not a constraint on your exploration. ``` +**Include code review context in the `hints` field** (try-fix's documented optional input). If Pre-Flight code review found no issues, use `hints: "Code review found no issues (verdict: LGTM)"`. If code review was SKIPPED, omit the `hints` field entirely. + +**Selectivity:** Only include ❌ Error findings and failure-mode probes that are relevant to the bug being fixed. Omit πŸ’‘ Suggestions. Include ⚠️ Warnings only if directly related to the root cause. + **Wait for each to complete before starting the next.** **🧹 MANDATORY: Clean up between attempts:** @@ -198,7 +228,7 @@ Deliver the final review recommendation. > 🚨 **DO NOT post any comments.** All output goes to `CustomAgentLogsTmp/PRState/`. -**Gate:** Phases 1-3 must be complete. +**Gate:** Phases 1-2 must be complete. --- @@ -207,18 +237,19 @@ Deliver the final review recommendation. ``` CustomAgentLogsTmp/PRState/{PRNumber}/PRAgent/ β”œβ”€β”€ pre-flight/ -β”‚ └── content.md # Phase 1 output (pr-preflight) +β”‚ β”œβ”€β”€ content.md # Phase 1 output (context + code review summary) +β”‚ └── code-review.md # Full code-review skill output (findings, blast radius, verdict) β”œβ”€β”€ gate/ -β”‚ └── content.md # Phase 2 output (pr-gate) +β”‚ └── content.md # Gate output (pr-gate, run separately) β”œβ”€β”€ try-fix/ -β”‚ β”œβ”€β”€ content.md # Phase 3 summary +β”‚ β”œβ”€β”€ content.md # Phase 2 summary β”‚ └── attempt-{N}/ # Per-model attempt β”‚ β”œβ”€β”€ approach.md # What was tried β”‚ β”œβ”€β”€ result.txt # Pass / Fail / Blocked β”‚ β”œβ”€β”€ fix.diff # git diff of changes β”‚ └── analysis.md # Why it worked/failed └── report/ - └── content.md # Phase 4 output (pr-report) + └── content.md # Phase 3 output (pr-report) ``` --- @@ -227,10 +258,10 @@ CustomAgentLogsTmp/PRState/{PRNumber}/PRAgent/ | Phase | Instructions | Key Action | If Blocked | |-------|--------------|------------|------------| -| 1. Pre-Flight | `pr-preflight.md` | Read issue + PR context | Skip missing info, continue | -| 2. Gate | `pr-gate.md` | Verify tests via task agent | Document, continue to Try-Fix | -| 3. Try-Fix | `try-fix` skill (Γ—4) | **4-model exploration (MANDATORY)** | Skip failing models, continue | -| 4. Report | `pr-report.md` | Write review recommendation | Never skip | +| Gate (pre-run) | `pr-gate.md` | Verify tests (run by Review-PR.ps1) | Result passed in prompt β€” if missing, document and continue | +| 1. Pre-Flight | `pr-preflight.md` | Read issue + PR context + **code review** | Skip missing info; if code review fails, set verdict to SKIPPED | +| 2. Try-Fix | `try-fix` skill (Γ—4) | **4-model exploration with code-review hints (MANDATORY)** | Skip failing models, continue | +| 3. Report | `pr-report.md` | Write review recommendation | Never skip | --- diff --git a/.github/skills/run-device-tests/scripts/Run-DeviceTests.ps1 b/.github/skills/run-device-tests/scripts/Run-DeviceTests.ps1 index db77a90210be..5a94bf00fb2a 100644 --- a/.github/skills/run-device-tests/scripts/Run-DeviceTests.ps1 +++ b/.github/skills/run-device-tests/scripts/Run-DeviceTests.ps1 @@ -274,7 +274,11 @@ try { ) # Add RuntimeIdentifier if specified - if ($platformConfig.RuntimeIdentifier) { + # NOTE: For Windows we deliberately do NOT pass `-r` here; RuntimeIdentifierOverride + # is set in the windows-specific block below to ensure the RID propagates to ALL + # referenced projects (e.g. TestUtils.DeviceTests). Plain `-r` is suppressed on + # non-leaf project references and causes PRI/asset file resolution failures. + if ($platformConfig.RuntimeIdentifier -and $Platform -ne "windows") { $buildArgs += "-r", $platformConfig.RuntimeIdentifier } @@ -296,8 +300,27 @@ try { $buildArgs += "/p:AndroidPackageFormat=apk" } "windows" { + # NOTE: WindowsAppSDKSelfContained MUST NOT be passed via command line because it + # propagates to ALL referenced projects (including library dependencies like + # Graphics.csproj) and breaks them with: + # "WindowsAppSDKSelfContained requires a supported Windows architecture" + # Instead, pass _MauiDeviceTestUnpackaged=true. The + # Microsoft.Maui.TestUtils.DeviceTests.Runners.props file (imported from each + # device test csproj) converts that signal into WindowsAppSDKSelfContained=true + # ONLY on the device test project itself. + # + # Also: use RuntimeIdentifierOverride (NOT `-r`/RuntimeIdentifier) so the RID + # propagates to every ProjectReference (e.g. TestUtils.DeviceTests). Plain + # RuntimeIdentifier is auto-suppressed on non-leaf project references, which + # leaves dependency PRI/asset files in the non-RID output folder while the + # test app itself is built at the RID-specific path, producing PRI175 errors. + # + # See eng/devices/windows.cake (buildOnly task, lines 145-188) for the + # canonical CI pattern. + $buildArgs += "/p:RuntimeIdentifierOverride=$($platformConfig.RuntimeIdentifier)" $buildArgs += "/p:WindowsPackageType=None" - $buildArgs += "/p:WindowsAppSDKSelfContained=true" + $buildArgs += "/p:SelfContained=true" + $buildArgs += "/p:_MauiDeviceTestUnpackaged=true" $buildArgs += "/p:UseMonoRuntime=false" } } diff --git a/.github/skills/verify-tests-fail-without-fix/scripts/verify-tests-fail.ps1 b/.github/skills/verify-tests-fail-without-fix/scripts/verify-tests-fail.ps1 index 64b692527320..168f9211bf9e 100644 --- a/.github/skills/verify-tests-fail-without-fix/scripts/verify-tests-fail.ps1 +++ b/.github/skills/verify-tests-fail-without-fix/scripts/verify-tests-fail.ps1 @@ -1259,7 +1259,7 @@ function Write-MarkdownReport { $lines += "
" } - # ── Failure details (only if something went wrong) ── + # ── Failure details (shown directly β€” not collapsed) ── $failureLines = @() foreach ($r in $WithoutFixResultsList) { if ($r.Passed) { @@ -1272,7 +1272,7 @@ function Write-MarkdownReport { $failureLines += "- ❌ **$($r.TestName)** FAILED with fix (should pass)" if ($r.FailureReason) { $failureLines += " - ``$($r.FailureReason)``" } if ($r.FailureMessage) { - $msg = if ($r.FailureMessage.Length -gt 200) { $r.FailureMessage.Substring(0, 200) + "..." } else { $r.FailureMessage } + $msg = if ($r.FailureMessage.Length -gt 300) { $r.FailureMessage.Substring(0, 300) + "..." } else { $r.FailureMessage } $failureLines += " - ``$msg``" } } @@ -1280,13 +1280,29 @@ function Write-MarkdownReport { } if ($failureLines.Count -gt 0) { + # Count actual failed tests (lines beginning with "- ❌" or "- ⚠️") to decide + # whether to collapse. Sub-bullets (FailureReason / FailureMessage) start with + # two leading spaces so they don't match. + $failedTestCount = @($failureLines | Where-Object { $_ -match '^- (❌|⚠️)' }).Count + # Threshold: if more than 5 tests failed, collapse the section so the gate + # summary stays visible above the fold in PR comments. Below the threshold, + # show details inline so reviewers don't need an extra click. + $collapseFailures = $failedTestCount -gt 5 + $lines += "" - $lines += "
" - $lines += "⚠️ Issues found" - $lines += "" + if ($collapseFailures) { + $lines += "
" + $lines += "⚠️ Failure Details ($failedTestCount tests)" + $lines += "" + } else { + $lines += "#### ⚠️ Failure Details" + $lines += "" + } $lines += ($failureLines -join "`n") - $lines += "" - $lines += "
" + if ($collapseFailures) { + $lines += "" + $lines += "
" + } } # ── Fix files (collapsible) ── diff --git a/.github/workflows/copilot-evaluate-tests.lock.yml b/.github/workflows/copilot-evaluate-tests.lock.yml index 8f07ab6405a9..46dffc668d56 100644 --- a/.github/workflows/copilot-evaluate-tests.lock.yml +++ b/.github/workflows/copilot-evaluate-tests.lock.yml @@ -1,3 +1,5 @@ +# gh-aw-metadata: {"schema_version":"v3","frontmatter_hash":"3987c8ff8c6fc12c964100324d3531fc77e954c8823607239515783232e430c7","compiler_version":"v0.68.3","strict":true,"agent_id":"copilot","agent_model":"claude-sonnet-4.6"} +# gh-aw-manifest: {"version":1,"secrets":["COPILOT_GITHUB_TOKEN","GH_AW_GITHUB_MCP_SERVER_TOKEN","GH_AW_GITHUB_TOKEN","GITHUB_TOKEN"],"actions":[{"repo":"actions/checkout","sha":"de0fac2e4500dabe0009e67214ff5f5447ce83dd","version":"v6.0.2"},{"repo":"actions/download-artifact","sha":"3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c","version":"v8.0.1"},{"repo":"actions/github-script","sha":"373c709c69115d41ff229c7e5df9f8788daa9553","version":"v9"},{"repo":"actions/github-script","sha":"3a2844b7e9c422d3c10d287c895573f7108da1b3","version":"v9"},{"repo":"actions/upload-artifact","sha":"043fb46d1a93c77aae656e7c1c64a875d1fc6a0a","version":"v7.0.1"},{"repo":"github/gh-aw-actions/setup","sha":"ba90f2186d7ad780ec640f364005fa24e797b360","version":"v0.68.3"}],"containers":[{"image":"ghcr.io/github/gh-aw-firewall/agent:0.25.20"},{"image":"ghcr.io/github/gh-aw-firewall/api-proxy:0.25.20"},{"image":"ghcr.io/github/gh-aw-firewall/squid:0.25.20"},{"image":"ghcr.io/github/gh-aw-mcpg:v0.2.19"},{"image":"ghcr.io/github/github-mcp-server:v0.32.0"},{"image":"node:lts-alpine"}]} # ___ _ _ # / _ \ | | (_) # | |_| | __ _ ___ _ __ | |_ _ ___ @@ -12,7 +14,7 @@ # \ /\ / (_) | | | | ( | | | | (_) \ V V /\__ \ # \/ \/ \___/|_| |_|\_\|_| |_|\___/ \_/\_/ |___/ # -# This file was automatically generated by gh-aw (v0.62.2). DO NOT EDIT. +# This file was automatically generated by gh-aw (v0.68.3). DO NOT EDIT. # # To update this file, edit the corresponding .md file and run: # gh aw compile @@ -22,7 +24,27 @@ # # Evaluates test quality, coverage, and appropriateness on PRs that add or modify tests # -# gh-aw-metadata: {"schema_version":"v2","frontmatter_hash":"cbfd75d7a699f76155135d83866eda6476cef647c384243c8ee991065a2b44d7","compiler_version":"v0.62.2","strict":true} +# Secrets used: +# - COPILOT_GITHUB_TOKEN +# - GH_AW_GITHUB_MCP_SERVER_TOKEN +# - GH_AW_GITHUB_TOKEN +# - GITHUB_TOKEN +# +# Custom actions used: +# - actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2 +# - actions/download-artifact@3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c # v8.0.1 +# - actions/github-script@373c709c69115d41ff229c7e5df9f8788daa9553 # v9 +# - actions/github-script@3a2844b7e9c422d3c10d287c895573f7108da1b3 # v9 +# - actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1 +# - github/gh-aw-actions/setup@ba90f2186d7ad780ec640f364005fa24e797b360 # v0.68.3 +# +# Container images used: +# - ghcr.io/github/gh-aw-firewall/agent:0.25.20 +# - ghcr.io/github/gh-aw-firewall/api-proxy:0.25.20 +# - ghcr.io/github/gh-aw-firewall/squid:0.25.20 +# - ghcr.io/github/gh-aw-mcpg:v0.2.19 +# - ghcr.io/github/github-mcp-server:v0.32.0 +# - node:lts-alpine name: "Evaluate PR Tests" "on": @@ -32,8 +54,17 @@ name: "Evaluate PR Tests" types: - created - edited + # roles: # Roles processed as role check in pre-activation job + # - admin # Roles processed as role check in pre-activation job + # - maintain # Roles processed as role check in pre-activation job + # - write # Roles processed as role check in pre-activation job workflow_dispatch: inputs: + aw_context: + default: "" + description: Agent caller context (used internally by Agentic Workflows). + required: false + type: string pr_number: description: PR number to evaluate required: true @@ -47,7 +78,7 @@ name: "Evaluate PR Tests" permissions: {} concurrency: - cancel-in-progress: true + cancel-in-progress: false group: evaluate-pr-tests-${{ github.event.issue.number || inputs.pr_number || github.run_id }} run-name: "Evaluate PR Tests" @@ -56,9 +87,10 @@ jobs: activation: needs: pre_activation if: > - (needs.pre_activation.outputs.activated == 'true') && (github.event_name == 'issue_comment' || github.event_name == 'workflow_dispatch') + needs.pre_activation.outputs.activated == 'true' && (github.event_name == 'issue_comment' || github.event_name == 'workflow_dispatch') runs-on: ubuntu-slim permissions: + actions: read contents: read discussions: write issues: write @@ -71,56 +103,61 @@ jobs: lockdown_check_failed: ${{ steps.generate_aw_info.outputs.lockdown_check_failed == 'true' }} model: ${{ steps.generate_aw_info.outputs.model }} secret_verification_result: ${{ steps.validate-secret.outputs.verification_result }} + setup-trace-id: ${{ steps.setup.outputs.trace-id }} slash_command: ${{ needs.pre_activation.outputs.matched_command }} + stale_lock_file_failed: ${{ steps.check-lock-file.outputs.stale_lock_file_failed == 'true' }} text: ${{ steps.sanitized.outputs.text }} title: ${{ steps.sanitized.outputs.title }} steps: - name: Setup Scripts - uses: github/gh-aw-actions/setup@20045bbd5ad2632b9809856c389708eab1bd16ef # v0.62.2 + id: setup + uses: github/gh-aw-actions/setup@ba90f2186d7ad780ec640f364005fa24e797b360 # v0.68.3 with: destination: ${{ runner.temp }}/gh-aw/actions + job-name: ${{ github.job }} + trace-id: ${{ needs.pre_activation.outputs.setup-trace-id }} - name: Generate agentic run info id: generate_aw_info env: GH_AW_INFO_ENGINE_ID: "copilot" GH_AW_INFO_ENGINE_NAME: "GitHub Copilot CLI" GH_AW_INFO_MODEL: "claude-sonnet-4.6" - GH_AW_INFO_VERSION: "" - GH_AW_INFO_AGENT_VERSION: "latest" - GH_AW_INFO_CLI_VERSION: "v0.62.2" + GH_AW_INFO_VERSION: "1.0.21" + GH_AW_INFO_AGENT_VERSION: "1.0.21" + GH_AW_INFO_CLI_VERSION: "v0.68.3" GH_AW_INFO_WORKFLOW_NAME: "Evaluate PR Tests" GH_AW_INFO_EXPERIMENTAL: "false" GH_AW_INFO_SUPPORTS_TOOLS_ALLOWLIST: "true" GH_AW_INFO_STAGED: "false" GH_AW_INFO_ALLOWED_DOMAINS: '["defaults"]' GH_AW_INFO_FIREWALL_ENABLED: "true" - GH_AW_INFO_AWF_VERSION: "v0.24.3" + GH_AW_INFO_AWF_VERSION: "v0.25.20" GH_AW_INFO_AWMG_VERSION: "" GH_AW_INFO_FIREWALL_TYPE: "squid" GH_AW_COMPILED_STRICT: "true" - uses: actions/github-script@ed597411d8f924073f98dfc5c65a23a2325f34cd # v8 + uses: actions/github-script@373c709c69115d41ff229c7e5df9f8788daa9553 # v9 with: script: | const { setupGlobals } = require('${{ runner.temp }}/gh-aw/actions/setup_globals.cjs'); - setupGlobals(core, github, context, exec, io); + setupGlobals(core, github, context, exec, io, getOctokit); const { main } = require('${{ runner.temp }}/gh-aw/actions/generate_aw_info.cjs'); await main(core, context); - name: Add eyes reaction for immediate feedback id: react - if: github.event_name == 'issues' || github.event_name == 'issue_comment' || github.event_name == 'pull_request_review_comment' || github.event_name == 'discussion' || github.event_name == 'discussion_comment' || (github.event_name == 'pull_request') && (github.event.pull_request.head.repo.id == github.repository_id) - uses: actions/github-script@ed597411d8f924073f98dfc5c65a23a2325f34cd # v8 + if: github.event_name == 'issues' || github.event_name == 'issue_comment' || github.event_name == 'pull_request_review_comment' || github.event_name == 'discussion' || github.event_name == 'discussion_comment' || github.event_name == 'pull_request' && github.event.pull_request.head.repo.id == github.repository_id + uses: actions/github-script@373c709c69115d41ff229c7e5df9f8788daa9553 # v9 env: GH_AW_REACTION: "eyes" with: github-token: ${{ secrets.GITHUB_TOKEN }} script: | const { setupGlobals } = require('${{ runner.temp }}/gh-aw/actions/setup_globals.cjs'); - setupGlobals(core, github, context, exec, io); + setupGlobals(core, github, context, exec, io, getOctokit); const { main } = require('${{ runner.temp }}/gh-aw/actions/add_reaction.cjs'); await main(); - name: Validate COPILOT_GITHUB_TOKEN secret id: validate-secret - run: ${RUNNER_TEMP}/gh-aw/actions/validate_multi_secret.sh COPILOT_GITHUB_TOKEN 'GitHub Copilot CLI' https://github.github.com/gh-aw/reference/engines/#github-copilot-default + run: bash "${RUNNER_TEMP}/gh-aw/actions/validate_multi_secret.sh" COPILOT_GITHUB_TOKEN 'GitHub Copilot CLI' https://github.github.com/gh-aw/reference/engines/#github-copilot-default env: COPILOT_GITHUB_TOKEN: ${{ secrets.COPILOT_GITHUB_TOKEN }} - name: Checkout .github and .agents folders @@ -132,44 +169,56 @@ jobs: .agents sparse-checkout-cone-mode: true fetch-depth: 1 - - name: Check workflow file timestamps - uses: actions/github-script@ed597411d8f924073f98dfc5c65a23a2325f34cd # v8 + - name: Check workflow lock file + id: check-lock-file + uses: actions/github-script@373c709c69115d41ff229c7e5df9f8788daa9553 # v9 env: GH_AW_WORKFLOW_FILE: "copilot-evaluate-tests.lock.yml" + GH_AW_CONTEXT_WORKFLOW_REF: "${{ github.workflow_ref }}" with: script: | const { setupGlobals } = require('${{ runner.temp }}/gh-aw/actions/setup_globals.cjs'); - setupGlobals(core, github, context, exec, io); + setupGlobals(core, github, context, exec, io, getOctokit); const { main } = require('${{ runner.temp }}/gh-aw/actions/check_workflow_timestamp_api.cjs'); await main(); + - name: Check compile-agentic version + uses: actions/github-script@373c709c69115d41ff229c7e5df9f8788daa9553 # v9 + env: + GH_AW_COMPILED_VERSION: "v0.68.3" + with: + script: | + const { setupGlobals } = require('${{ runner.temp }}/gh-aw/actions/setup_globals.cjs'); + setupGlobals(core, github, context, exec, io, getOctokit); + const { main } = require('${{ runner.temp }}/gh-aw/actions/check_version_updates.cjs'); + await main(); - name: Compute current body text id: sanitized - uses: actions/github-script@ed597411d8f924073f98dfc5c65a23a2325f34cd # v8 + uses: actions/github-script@373c709c69115d41ff229c7e5df9f8788daa9553 # v9 env: - GH_AW_ALLOWED_BOTS: copilot-swe-agent[bot] + GH_AW_ALLOWED_BOTS: "copilot-swe-agent[bot]" with: script: | const { setupGlobals } = require('${{ runner.temp }}/gh-aw/actions/setup_globals.cjs'); - setupGlobals(core, github, context, exec, io); + setupGlobals(core, github, context, exec, io, getOctokit); const { main } = require('${{ runner.temp }}/gh-aw/actions/compute_text.cjs'); await main(); - name: Add comment with workflow run link id: add-comment - if: github.event_name == 'issues' || github.event_name == 'issue_comment' || github.event_name == 'pull_request_review_comment' || github.event_name == 'discussion' || github.event_name == 'discussion_comment' || (github.event_name == 'pull_request') && (github.event.pull_request.head.repo.id == github.repository_id) - uses: actions/github-script@ed597411d8f924073f98dfc5c65a23a2325f34cd # v8 + if: github.event_name == 'issues' || github.event_name == 'issue_comment' || github.event_name == 'pull_request_review_comment' || github.event_name == 'discussion' || github.event_name == 'discussion_comment' || github.event_name == 'pull_request' && github.event.pull_request.head.repo.id == github.repository_id + uses: actions/github-script@373c709c69115d41ff229c7e5df9f8788daa9553 # v9 env: GH_AW_WORKFLOW_NAME: "Evaluate PR Tests" GH_AW_SAFE_OUTPUT_MESSAGES: "{\"footer\":\"\\u003e πŸ§ͺ *Test evaluation by [{workflow_name}]({run_url})*\",\"runStarted\":\"πŸ”¬ Evaluating tests on this PR… [{workflow_name}]({run_url})\",\"runSuccess\":\"βœ… Test evaluation complete! [{workflow_name}]({run_url})\",\"runFailure\":\"❌ Test evaluation failed. [{workflow_name}]({run_url}) {status}\"}" with: script: | const { setupGlobals } = require('${{ runner.temp }}/gh-aw/actions/setup_globals.cjs'); - setupGlobals(core, github, context, exec, io); + setupGlobals(core, github, context, exec, io, getOctokit); const { main } = require('${{ runner.temp }}/gh-aw/actions/add_workflow_run_comment.cjs'); await main(); - name: Create prompt with built-in context env: GH_AW_PROMPT: /tmp/gh-aw/aw-prompts/prompt.txt - GH_AW_SAFE_OUTPUTS: ${{ env.GH_AW_SAFE_OUTPUTS }} + GH_AW_SAFE_OUTPUTS: ${{ runner.temp }}/gh-aw/safeoutputs/outputs.jsonl GH_AW_EXPR_A77326CF: ${{ github.event.issue.number || inputs.pr_number }} GH_AW_GITHUB_ACTOR: ${{ github.actor }} GH_AW_GITHUB_EVENT_COMMENT_ID: ${{ github.event.comment.id }} @@ -181,17 +230,18 @@ jobs: GH_AW_GITHUB_WORKSPACE: ${{ github.workspace }} GH_AW_INPUTS_SUPPRESS_OUTPUT: ${{ inputs.suppress_output }} GH_AW_IS_PR_COMMENT: ${{ github.event.issue.pull_request && 'true' || '' }} + # poutine:ignore untrusted_checkout_exec run: | - bash ${RUNNER_TEMP}/gh-aw/actions/create_prompt_first.sh + bash "${RUNNER_TEMP}/gh-aw/actions/create_prompt_first.sh" { - cat << 'GH_AW_PROMPT_EOF' + cat << 'GH_AW_PROMPT_76b04e0a6e258a6a_EOF' - GH_AW_PROMPT_EOF + GH_AW_PROMPT_76b04e0a6e258a6a_EOF cat "${RUNNER_TEMP}/gh-aw/prompts/xpia.md" cat "${RUNNER_TEMP}/gh-aw/prompts/temp_folder_prompt.md" cat "${RUNNER_TEMP}/gh-aw/prompts/markdown.md" cat "${RUNNER_TEMP}/gh-aw/prompts/safe_outputs_prompt.md" - cat << 'GH_AW_PROMPT_EOF' + cat << 'GH_AW_PROMPT_76b04e0a6e258a6a_EOF' Tools: add_comment, missing_tool, missing_data, noop @@ -223,20 +273,18 @@ jobs: {{/if}} - GH_AW_PROMPT_EOF + GH_AW_PROMPT_76b04e0a6e258a6a_EOF cat "${RUNNER_TEMP}/gh-aw/prompts/github_mcp_tools_with_safeoutputs_prompt.md" if [ "$GITHUB_EVENT_NAME" = "issue_comment" ] && [ -n "$GH_AW_IS_PR_COMMENT" ] || [ "$GITHUB_EVENT_NAME" = "pull_request_review_comment" ] || [ "$GITHUB_EVENT_NAME" = "pull_request_review" ]; then cat "${RUNNER_TEMP}/gh-aw/prompts/pr_context_prompt.md" fi - cat << 'GH_AW_PROMPT_EOF' + cat << 'GH_AW_PROMPT_76b04e0a6e258a6a_EOF' - GH_AW_PROMPT_EOF - cat << 'GH_AW_PROMPT_EOF' {{#runtime-import .github/workflows/copilot-evaluate-tests.md}} - GH_AW_PROMPT_EOF + GH_AW_PROMPT_76b04e0a6e258a6a_EOF } > "$GH_AW_PROMPT" - name: Interpolate variables and render templates - uses: actions/github-script@ed597411d8f924073f98dfc5c65a23a2325f34cd # v8 + uses: actions/github-script@373c709c69115d41ff229c7e5df9f8788daa9553 # v9 env: GH_AW_PROMPT: /tmp/gh-aw/aw-prompts/prompt.txt GH_AW_EXPR_A77326CF: ${{ github.event.issue.number || inputs.pr_number }} @@ -245,11 +293,11 @@ jobs: with: script: | const { setupGlobals } = require('${{ runner.temp }}/gh-aw/actions/setup_globals.cjs'); - setupGlobals(core, github, context, exec, io); + setupGlobals(core, github, context, exec, io, getOctokit); const { main } = require('${{ runner.temp }}/gh-aw/actions/interpolate_prompt.cjs'); await main(); - name: Substitute placeholders - uses: actions/github-script@ed597411d8f924073f98dfc5c65a23a2325f34cd # v8 + uses: actions/github-script@373c709c69115d41ff229c7e5df9f8788daa9553 # v9 env: GH_AW_PROMPT: /tmp/gh-aw/aw-prompts/prompt.txt GH_AW_EXPR_A77326CF: ${{ github.event.issue.number || inputs.pr_number }} @@ -268,7 +316,7 @@ jobs: with: script: | const { setupGlobals } = require('${{ runner.temp }}/gh-aw/actions/setup_globals.cjs'); - setupGlobals(core, github, context, exec, io); + setupGlobals(core, github, context, exec, io, getOctokit); const substitutePlaceholders = require('${{ runner.temp }}/gh-aw/actions/substitute_placeholders.cjs'); @@ -294,19 +342,23 @@ jobs: - name: Validate prompt placeholders env: GH_AW_PROMPT: /tmp/gh-aw/aw-prompts/prompt.txt - run: bash ${RUNNER_TEMP}/gh-aw/actions/validate_prompt_placeholders.sh + # poutine:ignore untrusted_checkout_exec + run: bash "${RUNNER_TEMP}/gh-aw/actions/validate_prompt_placeholders.sh" - name: Print prompt env: GH_AW_PROMPT: /tmp/gh-aw/aw-prompts/prompt.txt - run: bash ${RUNNER_TEMP}/gh-aw/actions/print_prompt_summary.sh + # poutine:ignore untrusted_checkout_exec + run: bash "${RUNNER_TEMP}/gh-aw/actions/print_prompt_summary.sh" - name: Upload activation artifact if: success() - uses: actions/upload-artifact@bbbca2ddaa5d8feaa63e36b76fdaad77386f024f # v7.0.0 + uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1 with: name: activation path: | /tmp/gh-aw/aw_info.json /tmp/gh-aw/aw-prompts/prompt.txt + /tmp/gh-aw/github_rate_limits.jsonl + if-no-files-found: ignore retention-days: 1 agent: @@ -324,81 +376,85 @@ jobs: GH_AW_MCP_LOG_DIR: /tmp/gh-aw/mcp-logs/safeoutputs GH_AW_WORKFLOW_ID_SANITIZED: copilotevaluatetests outputs: + agentic_engine_timeout: ${{ steps.detect-copilot-errors.outputs.agentic_engine_timeout || 'false' }} checkout_pr_success: ${{ steps.checkout-pr.outputs.checkout_pr_success || 'true' }} - detection_conclusion: ${{ steps.detection_conclusion.outputs.conclusion }} - detection_success: ${{ steps.detection_conclusion.outputs.success }} + effective_tokens: ${{ steps.parse-mcp-gateway.outputs.effective_tokens }} has_patch: ${{ steps.collect_output.outputs.has_patch }} - inference_access_error: ${{ steps.detect-inference-error.outputs.inference_access_error || 'false' }} + inference_access_error: ${{ steps.detect-copilot-errors.outputs.inference_access_error || 'false' }} + mcp_policy_error: ${{ steps.detect-copilot-errors.outputs.mcp_policy_error || 'false' }} model: ${{ needs.activation.outputs.model }} + model_not_supported_error: ${{ steps.detect-copilot-errors.outputs.model_not_supported_error || 'false' }} output: ${{ steps.collect_output.outputs.output }} output_types: ${{ steps.collect_output.outputs.output_types }} + setup-trace-id: ${{ steps.setup.outputs.trace-id }} steps: - name: Setup Scripts - uses: github/gh-aw-actions/setup@20045bbd5ad2632b9809856c389708eab1bd16ef # v0.62.2 + id: setup + uses: github/gh-aw-actions/setup@ba90f2186d7ad780ec640f364005fa24e797b360 # v0.68.3 with: destination: ${{ runner.temp }}/gh-aw/actions + job-name: ${{ github.job }} + trace-id: ${{ needs.activation.outputs.setup-trace-id }} - name: Set runtime paths + id: set-runtime-paths run: | - echo "GH_AW_SAFE_OUTPUTS=${RUNNER_TEMP}/gh-aw/safeoutputs/outputs.jsonl" >> "$GITHUB_ENV" - echo "GH_AW_SAFE_OUTPUTS_CONFIG_PATH=${RUNNER_TEMP}/gh-aw/safeoutputs/config.json" >> "$GITHUB_ENV" - echo "GH_AW_SAFE_OUTPUTS_TOOLS_PATH=${RUNNER_TEMP}/gh-aw/safeoutputs/tools.json" >> "$GITHUB_ENV" + { + echo "GH_AW_SAFE_OUTPUTS=${RUNNER_TEMP}/gh-aw/safeoutputs/outputs.jsonl" + echo "GH_AW_SAFE_OUTPUTS_CONFIG_PATH=${RUNNER_TEMP}/gh-aw/safeoutputs/config.json" + echo "GH_AW_SAFE_OUTPUTS_TOOLS_PATH=${RUNNER_TEMP}/gh-aw/safeoutputs/tools.json" + } >> "$GITHUB_OUTPUT" - name: Checkout repository uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2 with: persist-credentials: false - name: Create gh-aw temp directory - run: bash ${RUNNER_TEMP}/gh-aw/actions/create_gh_aw_tmp_dir.sh + run: bash "${RUNNER_TEMP}/gh-aw/actions/create_gh_aw_tmp_dir.sh" - name: Configure gh CLI for GitHub Enterprise - run: bash ${RUNNER_TEMP}/gh-aw/actions/configure_gh_for_ghe.sh + run: bash "${RUNNER_TEMP}/gh-aw/actions/configure_gh_for_ghe.sh" env: GH_TOKEN: ${{ github.token }} - env: GH_TOKEN: ${{ github.token }} PR_NUMBER: ${{ github.event.issue.number || inputs.pr_number }} name: Gate β€” skip if no test source files in diff - run: "# Verify this is an open PR\nif ! STATE=$(gh pr view \"$PR_NUMBER\" --repo \"$GITHUB_REPOSITORY\" --json state --jq .state 2>&1); then\n echo \"❌ Failed to fetch PR #$PR_NUMBER state: $STATE\"\n exit 1\nfi\nif [ \"$STATE\" != \"OPEN\" ]; then\n echo \"⏭️ PR #$PR_NUMBER is $STATE β€” skipping evaluation.\"\n exit 1\nfi\n# Try gh pr diff first; fall back to REST API only on command failure\nif DIFF_OUTPUT=$(gh pr diff \"$PR_NUMBER\" --repo \"$GITHUB_REPOSITORY\" --name-only 2>/dev/null); then\n TEST_FILES=$(echo \"$DIFF_OUTPUT\" \\\n | grep -E '\\.(cs|xaml)$' \\\n | grep -iE '(tests?/|TestCases|UnitTests|DeviceTests)' \\\n || true)\nelse\n # gh pr diff fails with HTTP 406 for PRs with 300+ files; use paginated files API\n if ! API_FILES=$(gh api \"repos/$GITHUB_REPOSITORY/pulls/$PR_NUMBER/files\" --paginate --jq '.[].filename' 2>&1); then\n echo \"❌ gh pr diff failed and REST API fallback also failed: $API_FILES\"\n exit 1\n fi\n TEST_FILES=$(echo \"$API_FILES\" \\\n | grep -E '\\.(cs|xaml)$' \\\n | grep -iE '(tests?/|TestCases|UnitTests|DeviceTests)' \\\n || true)\nfi\nif [ -z \"$TEST_FILES\" ]; then\n echo \"⏭️ No test source files (.cs/.xaml) found in PR diff. Nothing to evaluate.\"\n exit 1\nfi\necho \"βœ… Found test files to evaluate:\"\necho \"$TEST_FILES\" | head -20\n" - - env: - GH_TOKEN: ${{ github.token }} - PR_NUMBER: ${{ inputs.pr_number }} - if: github.event_name == 'workflow_dispatch' - name: Checkout PR and restore agent infrastructure - run: pwsh .github/scripts/Checkout-GhAwPr.ps1 + run: "# Verify this is an open PR\nif ! STATE=$(gh pr view \"$PR_NUMBER\" --repo \"$GITHUB_REPOSITORY\" --json state --jq .state 2>&1); then\n echo \"❌ Failed to fetch PR #$PR_NUMBER state: $STATE\"\n exit 1\nfi\nif [ \"$STATE\" != \"OPEN\" ]; then\n echo \"⏭️ PR #$PR_NUMBER is $STATE β€” skipping evaluation.\"\n exit 1\nfi\n# Try gh pr diff first; fall back to REST API only on command failure\nif DIFF_OUTPUT=$(gh pr diff \"$PR_NUMBER\" --repo \"$GITHUB_REPOSITORY\" --name-only 2>/dev/null); then\n TEST_FILES=$(echo \"$DIFF_OUTPUT\" \\\n | grep -E '\\.(cs|xaml)$' \\\n | grep -iE '(tests?/|TestCases|UnitTests|DeviceTests)' \\\n || true)\nelse\n # gh pr diff fails with HTTP 406 for PRs with 300+ files; use paginated files API\n if ! API_FILES=$(gh api \"repos/$GITHUB_REPOSITORY/pulls/$PR_NUMBER/files\" --paginate --jq '.[].filename' 2>&1); then\n echo \"❌ gh pr diff failed and REST API fallback also failed: $API_FILES\"\n exit 1\n fi\n TEST_FILES=$(echo \"$API_FILES\" \\\n | grep -E '\\.(cs|xaml)$' \\\n | grep -iE '(tests?/|TestCases|UnitTests|DeviceTests)' \\\n || true)\nfi\nif [ -z \"$TEST_FILES\" ]; then\n echo \"⏭️ No test source files (.cs/.xaml) found in PR diff. Nothing to evaluate.\"\n exit 1\nfi\necho \"βœ… Found test files to evaluate:\"\necho \"$TEST_FILES\" | head -20" - name: Configure Git credentials env: REPO_NAME: ${{ github.repository }} SERVER_URL: ${{ github.server_url }} + GITHUB_TOKEN: ${{ github.token }} run: | git config --global user.email "github-actions[bot]@users.noreply.github.com" git config --global user.name "github-actions[bot]" git config --global am.keepcr true # Re-authenticate git with GitHub token SERVER_URL_STRIPPED="${SERVER_URL#https://}" - git remote set-url origin "https://x-access-token:${{ github.token }}@${SERVER_URL_STRIPPED}/${REPO_NAME}.git" + git remote set-url origin "https://x-access-token:${GITHUB_TOKEN}@${SERVER_URL_STRIPPED}/${REPO_NAME}.git" echo "Git configured with standard GitHub Actions identity" - name: Checkout PR branch id: checkout-pr if: | - (github.event.pull_request) || (github.event.issue.pull_request) - uses: actions/github-script@ed597411d8f924073f98dfc5c65a23a2325f34cd # v8 + github.event.pull_request || github.event.issue.pull_request + uses: actions/github-script@373c709c69115d41ff229c7e5df9f8788daa9553 # v9 env: GH_TOKEN: ${{ secrets.GH_AW_GITHUB_MCP_SERVER_TOKEN || secrets.GH_AW_GITHUB_TOKEN || secrets.GITHUB_TOKEN }} with: github-token: ${{ secrets.GH_AW_GITHUB_MCP_SERVER_TOKEN || secrets.GH_AW_GITHUB_TOKEN || secrets.GITHUB_TOKEN }} script: | const { setupGlobals } = require('${{ runner.temp }}/gh-aw/actions/setup_globals.cjs'); - setupGlobals(core, github, context, exec, io); + setupGlobals(core, github, context, exec, io, getOctokit); const { main } = require('${{ runner.temp }}/gh-aw/actions/checkout_pr_branch.cjs'); await main(); - name: Install GitHub Copilot CLI - run: ${RUNNER_TEMP}/gh-aw/actions/install_copilot_cli.sh latest + run: bash "${RUNNER_TEMP}/gh-aw/actions/install_copilot_cli.sh" 1.0.21 env: GH_HOST: github.com - name: Install AWF binary - run: bash ${RUNNER_TEMP}/gh-aw/actions/install_awf_binary.sh v0.24.3 + run: bash "${RUNNER_TEMP}/gh-aw/actions/install_awf_binary.sh" v0.25.20 - name: Determine automatic lockdown mode for GitHub MCP Server id: determine-automatic-lockdown - uses: actions/github-script@ed597411d8f924073f98dfc5c65a23a2325f34cd # v8 + uses: actions/github-script@3a2844b7e9c422d3c10d287c895573f7108da1b3 # v9 env: GH_AW_GITHUB_TOKEN: ${{ secrets.GH_AW_GITHUB_TOKEN }} GH_AW_GITHUB_MCP_SERVER_TOKEN: ${{ secrets.GH_AW_GITHUB_MCP_SERVER_TOKEN }} @@ -407,106 +463,130 @@ jobs: const determineAutomaticLockdown = require('${{ runner.temp }}/gh-aw/actions/determine_automatic_lockdown.cjs'); await determineAutomaticLockdown(github, context, core); - name: Download container images - run: bash ${RUNNER_TEMP}/gh-aw/actions/download_docker_images.sh ghcr.io/github/gh-aw-firewall/agent:0.24.3 ghcr.io/github/gh-aw-firewall/api-proxy:0.24.3 ghcr.io/github/gh-aw-firewall/squid:0.24.3 ghcr.io/github/gh-aw-mcpg:v0.1.19 ghcr.io/github/github-mcp-server:v0.32.0 node:lts-alpine + run: bash "${RUNNER_TEMP}/gh-aw/actions/download_docker_images.sh" ghcr.io/github/gh-aw-firewall/agent:0.25.20 ghcr.io/github/gh-aw-firewall/api-proxy:0.25.20 ghcr.io/github/gh-aw-firewall/squid:0.25.20 ghcr.io/github/gh-aw-mcpg:v0.2.19 ghcr.io/github/github-mcp-server:v0.32.0 node:lts-alpine - name: Write Safe Outputs Config run: | - mkdir -p ${RUNNER_TEMP}/gh-aw/safeoutputs + mkdir -p "${RUNNER_TEMP}/gh-aw/safeoutputs" mkdir -p /tmp/gh-aw/safeoutputs mkdir -p /tmp/gh-aw/mcp-logs/safeoutputs - cat > ${RUNNER_TEMP}/gh-aw/safeoutputs/config.json << 'GH_AW_SAFE_OUTPUTS_CONFIG_EOF' - {"add_comment":{"max":1,"target":"*"},"missing_data":{},"missing_tool":{},"noop":{"max":1}} - GH_AW_SAFE_OUTPUTS_CONFIG_EOF + cat > "${RUNNER_TEMP}/gh-aw/safeoutputs/config.json" << 'GH_AW_SAFE_OUTPUTS_CONFIG_f147e6f2a8dceac4_EOF' + {"add_comment":{"hide_older_comments":true,"max":1,"target":"*"},"create_report_incomplete_issue":{},"missing_data":{},"missing_tool":{},"noop":{"max":1,"report-as-issue":"false"},"report_incomplete":{}} + GH_AW_SAFE_OUTPUTS_CONFIG_f147e6f2a8dceac4_EOF - name: Write Safe Outputs Tools - run: | - cat > ${RUNNER_TEMP}/gh-aw/safeoutputs/tools_meta.json << 'GH_AW_SAFE_OUTPUTS_TOOLS_META_EOF' - { - "description_suffixes": { - "add_comment": " CONSTRAINTS: Maximum 1 comment(s) can be added. Target: *." - }, - "repo_params": {}, - "dynamic_tools": [] - } - GH_AW_SAFE_OUTPUTS_TOOLS_META_EOF - cat > ${RUNNER_TEMP}/gh-aw/safeoutputs/validation.json << 'GH_AW_SAFE_OUTPUTS_VALIDATION_EOF' - { - "add_comment": { - "defaultMax": 1, - "fields": { - "body": { - "required": true, - "type": "string", - "sanitize": true, - "maxLength": 65000 - }, - "item_number": { - "issueOrPRNumber": true - }, - "repo": { - "type": "string", - "maxLength": 256 + env: + GH_AW_TOOLS_META_JSON: | + { + "description_suffixes": { + "add_comment": " CONSTRAINTS: Maximum 1 comment(s) can be added. Target: *. Supports reply_to_id for discussion threading." + }, + "repo_params": {}, + "dynamic_tools": [] + } + GH_AW_VALIDATION_JSON: | + { + "add_comment": { + "defaultMax": 1, + "fields": { + "body": { + "required": true, + "type": "string", + "sanitize": true, + "maxLength": 65000 + }, + "item_number": { + "issueOrPRNumber": true + }, + "reply_to_id": { + "type": "string", + "maxLength": 256 + }, + "repo": { + "type": "string", + "maxLength": 256 + } } - } - }, - "missing_data": { - "defaultMax": 20, - "fields": { - "alternatives": { - "type": "string", - "sanitize": true, - "maxLength": 256 - }, - "context": { - "type": "string", - "sanitize": true, - "maxLength": 256 - }, - "data_type": { - "type": "string", - "sanitize": true, - "maxLength": 128 - }, - "reason": { - "type": "string", - "sanitize": true, - "maxLength": 256 + }, + "missing_data": { + "defaultMax": 20, + "fields": { + "alternatives": { + "type": "string", + "sanitize": true, + "maxLength": 256 + }, + "context": { + "type": "string", + "sanitize": true, + "maxLength": 256 + }, + "data_type": { + "type": "string", + "sanitize": true, + "maxLength": 128 + }, + "reason": { + "type": "string", + "sanitize": true, + "maxLength": 256 + } } - } - }, - "missing_tool": { - "defaultMax": 20, - "fields": { - "alternatives": { - "type": "string", - "sanitize": true, - "maxLength": 512 - }, - "reason": { - "required": true, - "type": "string", - "sanitize": true, - "maxLength": 256 - }, - "tool": { - "type": "string", - "sanitize": true, - "maxLength": 128 + }, + "missing_tool": { + "defaultMax": 20, + "fields": { + "alternatives": { + "type": "string", + "sanitize": true, + "maxLength": 512 + }, + "reason": { + "required": true, + "type": "string", + "sanitize": true, + "maxLength": 256 + }, + "tool": { + "type": "string", + "sanitize": true, + "maxLength": 128 + } } - } - }, - "noop": { - "defaultMax": 1, - "fields": { - "message": { - "required": true, - "type": "string", - "sanitize": true, - "maxLength": 65000 + }, + "noop": { + "defaultMax": 1, + "fields": { + "message": { + "required": true, + "type": "string", + "sanitize": true, + "maxLength": 65000 + } + } + }, + "report_incomplete": { + "defaultMax": 5, + "fields": { + "details": { + "type": "string", + "sanitize": true, + "maxLength": 65000 + }, + "reason": { + "required": true, + "type": "string", + "sanitize": true, + "maxLength": 1024 + } } } } - } - GH_AW_SAFE_OUTPUTS_VALIDATION_EOF - node ${RUNNER_TEMP}/gh-aw/actions/generate_safe_outputs_tools.cjs + uses: actions/github-script@373c709c69115d41ff229c7e5df9f8788daa9553 # v9 + with: + script: | + const { setupGlobals } = require('${{ runner.temp }}/gh-aw/actions/setup_globals.cjs'); + setupGlobals(core, github, context, exec, io, getOctokit); + const { main } = require('${{ runner.temp }}/gh-aw/actions/generate_safe_outputs_tools.cjs'); + await main(); - name: Generate Safe Outputs MCP Server Config id: safe-outputs-config run: | @@ -529,6 +609,7 @@ jobs: id: safe-outputs-start env: DEBUG: '*' + GH_AW_SAFE_OUTPUTS: ${{ steps.set-runtime-paths.outputs.GH_AW_SAFE_OUTPUTS }} GH_AW_SAFE_OUTPUTS_PORT: ${{ steps.safe-outputs-config.outputs.safe_outputs_port }} GH_AW_SAFE_OUTPUTS_API_KEY: ${{ steps.safe-outputs-config.outputs.safe_outputs_api_key }} GH_AW_SAFE_OUTPUTS_TOOLS_PATH: ${{ runner.temp }}/gh-aw/safeoutputs/tools.json @@ -537,18 +618,19 @@ jobs: run: | # Environment variables are set above to prevent template injection export DEBUG + export GH_AW_SAFE_OUTPUTS export GH_AW_SAFE_OUTPUTS_PORT export GH_AW_SAFE_OUTPUTS_API_KEY export GH_AW_SAFE_OUTPUTS_TOOLS_PATH export GH_AW_SAFE_OUTPUTS_CONFIG_PATH export GH_AW_MCP_LOG_DIR - bash ${RUNNER_TEMP}/gh-aw/actions/start_safe_outputs_server.sh + bash "${RUNNER_TEMP}/gh-aw/actions/start_safe_outputs_server.sh" - name: Start MCP Gateway id: start-mcp-gateway env: - GH_AW_SAFE_OUTPUTS: ${{ env.GH_AW_SAFE_OUTPUTS }} + GH_AW_SAFE_OUTPUTS: ${{ steps.set-runtime-paths.outputs.GH_AW_SAFE_OUTPUTS }} GH_AW_SAFE_OUTPUTS_API_KEY: ${{ steps.safe-outputs-start.outputs.api_key }} GH_AW_SAFE_OUTPUTS_PORT: ${{ steps.safe-outputs-start.outputs.port }} GITHUB_MCP_GUARD_MIN_INTEGRITY: ${{ steps.determine-automatic-lockdown.outputs.min_integrity }} @@ -570,10 +652,10 @@ jobs: export DEBUG="*" export GH_AW_ENGINE="copilot" - export MCP_GATEWAY_DOCKER_COMMAND='docker run -i --rm --network host -v /var/run/docker.sock:/var/run/docker.sock -e MCP_GATEWAY_PORT -e MCP_GATEWAY_DOMAIN -e MCP_GATEWAY_API_KEY -e MCP_GATEWAY_PAYLOAD_DIR -e MCP_GATEWAY_PAYLOAD_SIZE_THRESHOLD -e DEBUG -e MCP_GATEWAY_LOG_DIR -e GH_AW_MCP_LOG_DIR -e GH_AW_SAFE_OUTPUTS -e GH_AW_SAFE_OUTPUTS_CONFIG_PATH -e GH_AW_SAFE_OUTPUTS_TOOLS_PATH -e GH_AW_ASSETS_BRANCH -e GH_AW_ASSETS_MAX_SIZE_KB -e GH_AW_ASSETS_ALLOWED_EXTS -e DEFAULT_BRANCH -e GITHUB_MCP_SERVER_TOKEN -e GITHUB_MCP_GUARD_MIN_INTEGRITY -e GITHUB_MCP_GUARD_REPOS -e GITHUB_REPOSITORY -e GITHUB_SERVER_URL -e GITHUB_SHA -e GITHUB_WORKSPACE -e GITHUB_TOKEN -e GITHUB_RUN_ID -e GITHUB_RUN_NUMBER -e GITHUB_RUN_ATTEMPT -e GITHUB_JOB -e GITHUB_ACTION -e GITHUB_EVENT_NAME -e GITHUB_EVENT_PATH -e GITHUB_ACTOR -e GITHUB_ACTOR_ID -e GITHUB_TRIGGERING_ACTOR -e GITHUB_WORKFLOW -e GITHUB_WORKFLOW_REF -e GITHUB_WORKFLOW_SHA -e GITHUB_REF -e GITHUB_REF_NAME -e GITHUB_REF_TYPE -e GITHUB_HEAD_REF -e GITHUB_BASE_REF -e GH_AW_SAFE_OUTPUTS_PORT -e GH_AW_SAFE_OUTPUTS_API_KEY -v /tmp/gh-aw/mcp-payloads:/tmp/gh-aw/mcp-payloads:rw -v /opt:/opt:ro -v /tmp:/tmp:rw -v '"${GITHUB_WORKSPACE}"':'"${GITHUB_WORKSPACE}"':rw ghcr.io/github/gh-aw-mcpg:v0.1.19' + export MCP_GATEWAY_DOCKER_COMMAND='docker run -i --rm --network host -v /var/run/docker.sock:/var/run/docker.sock -e MCP_GATEWAY_PORT -e MCP_GATEWAY_DOMAIN -e MCP_GATEWAY_API_KEY -e MCP_GATEWAY_PAYLOAD_DIR -e MCP_GATEWAY_PAYLOAD_SIZE_THRESHOLD -e DEBUG -e MCP_GATEWAY_LOG_DIR -e GH_AW_MCP_LOG_DIR -e GH_AW_SAFE_OUTPUTS -e GH_AW_SAFE_OUTPUTS_CONFIG_PATH -e GH_AW_SAFE_OUTPUTS_TOOLS_PATH -e GH_AW_ASSETS_BRANCH -e GH_AW_ASSETS_MAX_SIZE_KB -e GH_AW_ASSETS_ALLOWED_EXTS -e DEFAULT_BRANCH -e GITHUB_MCP_SERVER_TOKEN -e GITHUB_MCP_GUARD_MIN_INTEGRITY -e GITHUB_MCP_GUARD_REPOS -e GITHUB_REPOSITORY -e GITHUB_SERVER_URL -e GITHUB_SHA -e GITHUB_WORKSPACE -e GITHUB_TOKEN -e GITHUB_RUN_ID -e GITHUB_RUN_NUMBER -e GITHUB_RUN_ATTEMPT -e GITHUB_JOB -e GITHUB_ACTION -e GITHUB_EVENT_NAME -e GITHUB_EVENT_PATH -e GITHUB_ACTOR -e GITHUB_ACTOR_ID -e GITHUB_TRIGGERING_ACTOR -e GITHUB_WORKFLOW -e GITHUB_WORKFLOW_REF -e GITHUB_WORKFLOW_SHA -e GITHUB_REF -e GITHUB_REF_NAME -e GITHUB_REF_TYPE -e GITHUB_HEAD_REF -e GITHUB_BASE_REF -e GH_AW_SAFE_OUTPUTS_PORT -e GH_AW_SAFE_OUTPUTS_API_KEY -v /tmp/gh-aw/mcp-payloads:/tmp/gh-aw/mcp-payloads:rw -v /opt:/opt:ro -v /tmp:/tmp:rw -v '"${GITHUB_WORKSPACE}"':'"${GITHUB_WORKSPACE}"':rw ghcr.io/github/gh-aw-mcpg:v0.2.19' mkdir -p /home/runner/.copilot - cat << GH_AW_MCP_CONFIG_EOF | bash ${RUNNER_TEMP}/gh-aw/actions/start_mcp_gateway.sh + cat << GH_AW_MCP_CONFIG_a5bd4f57a227c878_EOF | bash "${RUNNER_TEMP}/gh-aw/actions/start_mcp_gateway.sh" { "mcpServers": { "github": { @@ -614,7 +696,7 @@ jobs: "payloadDir": "${MCP_GATEWAY_PAYLOAD_DIR}" } } - GH_AW_MCP_CONFIG_EOF + GH_AW_MCP_CONFIG_a5bd4f57a227c878_EOF - name: Download activation artifact uses: actions/download-artifact@3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c # v8.0.1 with: @@ -622,7 +704,7 @@ jobs: path: /tmp/gh-aw - name: Clean git credentials continue-on-error: true - run: bash ${RUNNER_TEMP}/gh-aw/actions/clean_git_credentials.sh + run: bash "${RUNNER_TEMP}/gh-aw/actions/clean_git_credentials.sh" - name: Execute GitHub Copilot CLI id: agentic_execution # Copilot CLI tool arguments (sorted): @@ -630,9 +712,10 @@ jobs: run: | set -o pipefail touch /tmp/gh-aw/agent-step-summary.md + (umask 177 && touch /tmp/gh-aw/agent-stdio.log) # shellcheck disable=SC1003 - sudo -E awf --env-all --container-workdir "${GITHUB_WORKSPACE}" --mount "${RUNNER_TEMP}/gh-aw:${RUNNER_TEMP}/gh-aw:ro" --mount "${RUNNER_TEMP}/gh-aw:/host${RUNNER_TEMP}/gh-aw:ro" --allow-domains "api.business.githubcopilot.com,api.enterprise.githubcopilot.com,api.github.com,api.githubcopilot.com,api.individual.githubcopilot.com,api.snapcraft.io,archive.ubuntu.com,azure.archive.ubuntu.com,crl.geotrust.com,crl.globalsign.com,crl.identrust.com,crl.sectigo.com,crl.thawte.com,crl.usertrust.com,crl.verisign.com,crl3.digicert.com,crl4.digicert.com,crls.ssl.com,github.com,host.docker.internal,json-schema.org,json.schemastore.org,keyserver.ubuntu.com,ocsp.digicert.com,ocsp.geotrust.com,ocsp.globalsign.com,ocsp.identrust.com,ocsp.sectigo.com,ocsp.ssl.com,ocsp.thawte.com,ocsp.usertrust.com,ocsp.verisign.com,packagecloud.io,packages.cloud.google.com,packages.microsoft.com,ppa.launchpad.net,raw.githubusercontent.com,registry.npmjs.org,s.symcb.com,s.symcd.com,security.ubuntu.com,telemetry.enterprise.githubcopilot.com,ts-crl.ws.symantec.com,ts-ocsp.ws.symantec.com,www.googleapis.com" --log-level info --proxy-logs-dir /tmp/gh-aw/sandbox/firewall/logs --enable-host-access --image-tag 0.24.3 --skip-pull --enable-api-proxy \ - -- /bin/bash -c '/usr/local/bin/copilot --add-dir /tmp/gh-aw/ --log-level all --log-dir /tmp/gh-aw/sandbox/agent/logs/ --add-dir "${GITHUB_WORKSPACE}" --disable-builtin-mcps --allow-all-tools --allow-all-paths --prompt "$(cat /tmp/gh-aw/aw-prompts/prompt.txt)"' 2>&1 | tee -a /tmp/gh-aw/agent-stdio.log + sudo -E awf --container-workdir "${GITHUB_WORKSPACE}" --mount "${RUNNER_TEMP}/gh-aw:${RUNNER_TEMP}/gh-aw:ro" --mount "${RUNNER_TEMP}/gh-aw:/host${RUNNER_TEMP}/gh-aw:ro" --env-all --exclude-env COPILOT_GITHUB_TOKEN --exclude-env GITHUB_MCP_SERVER_TOKEN --exclude-env MCP_GATEWAY_API_KEY --allow-domains api.business.githubcopilot.com,api.enterprise.githubcopilot.com,api.github.com,api.githubcopilot.com,api.individual.githubcopilot.com,api.snapcraft.io,archive.ubuntu.com,azure.archive.ubuntu.com,crl.geotrust.com,crl.globalsign.com,crl.identrust.com,crl.sectigo.com,crl.thawte.com,crl.usertrust.com,crl.verisign.com,crl3.digicert.com,crl4.digicert.com,crls.ssl.com,github.com,host.docker.internal,json-schema.org,json.schemastore.org,keyserver.ubuntu.com,ocsp.digicert.com,ocsp.geotrust.com,ocsp.globalsign.com,ocsp.identrust.com,ocsp.sectigo.com,ocsp.ssl.com,ocsp.thawte.com,ocsp.usertrust.com,ocsp.verisign.com,packagecloud.io,packages.cloud.google.com,packages.microsoft.com,ppa.launchpad.net,raw.githubusercontent.com,registry.npmjs.org,s.symcb.com,s.symcd.com,security.ubuntu.com,telemetry.enterprise.githubcopilot.com,ts-crl.ws.symantec.com,ts-ocsp.ws.symantec.com,www.googleapis.com --log-level info --proxy-logs-dir /tmp/gh-aw/sandbox/firewall/logs --audit-dir /tmp/gh-aw/sandbox/firewall/audit --enable-host-access --image-tag 0.25.20 --skip-pull --enable-api-proxy \ + -- /bin/bash -c 'node ${RUNNER_TEMP}/gh-aw/actions/copilot_driver.cjs /usr/local/bin/copilot --add-dir /tmp/gh-aw/ --log-level all --log-dir /tmp/gh-aw/sandbox/agent/logs/ --disable-builtin-mcps --no-ask-user --allow-all-tools --allow-all-paths --add-dir "${GITHUB_WORKSPACE}" --prompt "$(cat /tmp/gh-aw/aw-prompts/prompt.txt)"' 2>&1 | tee -a /tmp/gh-aw/agent-stdio.log env: COPILOT_AGENT_RUNNER_TYPE: STANDALONE COPILOT_GITHUB_TOKEN: ${{ secrets.COPILOT_GITHUB_TOKEN }} @@ -640,8 +723,8 @@ jobs: GH_AW_MCP_CONFIG: /home/runner/.copilot/mcp-config.json GH_AW_PHASE: agent GH_AW_PROMPT: /tmp/gh-aw/aw-prompts/prompt.txt - GH_AW_SAFE_OUTPUTS: ${{ env.GH_AW_SAFE_OUTPUTS }} - GH_AW_VERSION: v0.62.2 + GH_AW_SAFE_OUTPUTS: ${{ steps.set-runtime-paths.outputs.GH_AW_SAFE_OUTPUTS }} + GH_AW_VERSION: v0.68.3 GITHUB_API_URL: ${{ github.api_url }} GITHUB_AW: true GITHUB_HEAD_REF: ${{ github.head_ref }} @@ -655,40 +738,28 @@ jobs: GIT_COMMITTER_EMAIL: github-actions[bot]@users.noreply.github.com GIT_COMMITTER_NAME: github-actions[bot] XDG_CONFIG_HOME: /home/runner - - name: Detect inference access error - id: detect-inference-error + - name: Detect Copilot errors + id: detect-copilot-errors if: always() continue-on-error: true - run: bash ${RUNNER_TEMP}/gh-aw/actions/detect_inference_access_error.sh + run: node "${RUNNER_TEMP}/gh-aw/actions/detect_copilot_errors.cjs" - name: Configure Git credentials env: REPO_NAME: ${{ github.repository }} SERVER_URL: ${{ github.server_url }} + GITHUB_TOKEN: ${{ github.token }} run: | git config --global user.email "github-actions[bot]@users.noreply.github.com" git config --global user.name "github-actions[bot]" git config --global am.keepcr true # Re-authenticate git with GitHub token SERVER_URL_STRIPPED="${SERVER_URL#https://}" - git remote set-url origin "https://x-access-token:${{ github.token }}@${SERVER_URL_STRIPPED}/${REPO_NAME}.git" + git remote set-url origin "https://x-access-token:${GITHUB_TOKEN}@${SERVER_URL_STRIPPED}/${REPO_NAME}.git" echo "Git configured with standard GitHub Actions identity" - name: Copy Copilot session state files to logs if: always() continue-on-error: true - run: | - # Copy Copilot session state files to logs folder for artifact collection - # This ensures they are in /tmp/gh-aw/ where secret redaction can scan them - SESSION_STATE_DIR="$HOME/.copilot/session-state" - LOGS_DIR="/tmp/gh-aw/sandbox/agent/logs" - - if [ -d "$SESSION_STATE_DIR" ]; then - echo "Copying Copilot session state files from $SESSION_STATE_DIR to $LOGS_DIR" - mkdir -p "$LOGS_DIR" - cp -v "$SESSION_STATE_DIR"/*.jsonl "$LOGS_DIR/" 2>/dev/null || true - echo "Session state files copied successfully" - else - echo "No session-state directory found at $SESSION_STATE_DIR" - fi + run: bash "${RUNNER_TEMP}/gh-aw/actions/copy_copilot_session_state.sh" - name: Stop MCP Gateway if: always() continue-on-error: true @@ -697,14 +768,14 @@ jobs: MCP_GATEWAY_API_KEY: ${{ steps.start-mcp-gateway.outputs.gateway-api-key }} GATEWAY_PID: ${{ steps.start-mcp-gateway.outputs.gateway-pid }} run: | - bash ${RUNNER_TEMP}/gh-aw/actions/stop_mcp_gateway.sh "$GATEWAY_PID" + bash "${RUNNER_TEMP}/gh-aw/actions/stop_mcp_gateway.sh" "$GATEWAY_PID" - name: Redact secrets in logs if: always() - uses: actions/github-script@ed597411d8f924073f98dfc5c65a23a2325f34cd # v8 + uses: actions/github-script@373c709c69115d41ff229c7e5df9f8788daa9553 # v9 with: script: | const { setupGlobals } = require('${{ runner.temp }}/gh-aw/actions/setup_globals.cjs'); - setupGlobals(core, github, context, exec, io); + setupGlobals(core, github, context, exec, io, getOctokit); const { main } = require('${{ runner.temp }}/gh-aw/actions/redact_secrets.cjs'); await main(); env: @@ -715,18 +786,20 @@ jobs: SECRET_GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }} - name: Append agent step summary if: always() - run: bash ${RUNNER_TEMP}/gh-aw/actions/append_agent_step_summary.sh + run: bash "${RUNNER_TEMP}/gh-aw/actions/append_agent_step_summary.sh" - name: Copy Safe Outputs if: always() + env: + GH_AW_SAFE_OUTPUTS: ${{ steps.set-runtime-paths.outputs.GH_AW_SAFE_OUTPUTS }} run: | mkdir -p /tmp/gh-aw cp "$GH_AW_SAFE_OUTPUTS" /tmp/gh-aw/safeoutputs.jsonl 2>/dev/null || true - name: Ingest agent output id: collect_output if: always() - uses: actions/github-script@ed597411d8f924073f98dfc5c65a23a2325f34cd # v8 + uses: actions/github-script@373c709c69115d41ff229c7e5df9f8788daa9553 # v9 env: - GH_AW_SAFE_OUTPUTS: ${{ env.GH_AW_SAFE_OUTPUTS }} + GH_AW_SAFE_OUTPUTS: ${{ steps.set-runtime-paths.outputs.GH_AW_SAFE_OUTPUTS }} GH_AW_ALLOWED_DOMAINS: "api.business.githubcopilot.com,api.enterprise.githubcopilot.com,api.github.com,api.githubcopilot.com,api.individual.githubcopilot.com,api.snapcraft.io,archive.ubuntu.com,azure.archive.ubuntu.com,crl.geotrust.com,crl.globalsign.com,crl.identrust.com,crl.sectigo.com,crl.thawte.com,crl.usertrust.com,crl.verisign.com,crl3.digicert.com,crl4.digicert.com,crls.ssl.com,github.com,host.docker.internal,json-schema.org,json.schemastore.org,keyserver.ubuntu.com,ocsp.digicert.com,ocsp.geotrust.com,ocsp.globalsign.com,ocsp.identrust.com,ocsp.sectigo.com,ocsp.ssl.com,ocsp.thawte.com,ocsp.usertrust.com,ocsp.verisign.com,packagecloud.io,packages.cloud.google.com,packages.microsoft.com,ppa.launchpad.net,raw.githubusercontent.com,registry.npmjs.org,s.symcb.com,s.symcd.com,security.ubuntu.com,telemetry.enterprise.githubcopilot.com,ts-crl.ws.symantec.com,ts-ocsp.ws.symantec.com,www.googleapis.com" GITHUB_SERVER_URL: ${{ github.server_url }} GITHUB_API_URL: ${{ github.api_url }} @@ -734,27 +807,28 @@ jobs: with: script: | const { setupGlobals } = require('${{ runner.temp }}/gh-aw/actions/setup_globals.cjs'); - setupGlobals(core, github, context, exec, io); + setupGlobals(core, github, context, exec, io, getOctokit); const { main } = require('${{ runner.temp }}/gh-aw/actions/collect_ndjson_output.cjs'); await main(); - name: Parse agent logs for step summary if: always() - uses: actions/github-script@ed597411d8f924073f98dfc5c65a23a2325f34cd # v8 + uses: actions/github-script@373c709c69115d41ff229c7e5df9f8788daa9553 # v9 env: GH_AW_AGENT_OUTPUT: /tmp/gh-aw/sandbox/agent/logs/ with: script: | const { setupGlobals } = require('${{ runner.temp }}/gh-aw/actions/setup_globals.cjs'); - setupGlobals(core, github, context, exec, io); + setupGlobals(core, github, context, exec, io, getOctokit); const { main } = require('${{ runner.temp }}/gh-aw/actions/parse_copilot_log.cjs'); await main(); - name: Parse MCP Gateway logs for step summary if: always() - uses: actions/github-script@ed597411d8f924073f98dfc5c65a23a2325f34cd # v8 + id: parse-mcp-gateway + uses: actions/github-script@373c709c69115d41ff229c7e5df9f8788daa9553 # v9 with: script: | const { setupGlobals } = require('${{ runner.temp }}/gh-aw/actions/setup_globals.cjs'); - setupGlobals(core, github, context, exec, io); + setupGlobals(core, github, context, exec, io, getOctokit); const { main } = require('${{ runner.temp }}/gh-aw/actions/parse_mcp_gateway_log.cjs'); await main(); - name: Print firewall logs @@ -772,10 +846,26 @@ jobs: else echo 'AWF binary not installed, skipping firewall log summary' fi + - name: Parse token usage for step summary + if: always() + continue-on-error: true + uses: actions/github-script@373c709c69115d41ff229c7e5df9f8788daa9553 # v9 + with: + script: | + const { setupGlobals } = require('${{ runner.temp }}/gh-aw/actions/setup_globals.cjs'); + setupGlobals(core, github, context, exec, io, getOctokit); + const { main } = require('${{ runner.temp }}/gh-aw/actions/parse_token_usage.cjs'); + await main(); + - name: Write agent output placeholder if missing + if: always() + run: | + if [ ! -f /tmp/gh-aw/agent_output.json ]; then + echo '{"items":[]}' > /tmp/gh-aw/agent_output.json + fi - name: Upload agent artifacts if: always() continue-on-error: true - uses: actions/upload-artifact@bbbca2ddaa5d8feaa63e36b76fdaad77386f024f # v7.0.0 + uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1 with: name: agent path: | @@ -783,143 +873,27 @@ jobs: /tmp/gh-aw/sandbox/agent/logs/ /tmp/gh-aw/redacted-urls.log /tmp/gh-aw/mcp-logs/ - /tmp/gh-aw/sandbox/firewall/logs/ + /tmp/gh-aw/agent_usage.json /tmp/gh-aw/agent-stdio.log /tmp/gh-aw/agent/ + /tmp/gh-aw/github_rate_limits.jsonl /tmp/gh-aw/safeoutputs.jsonl /tmp/gh-aw/agent_output.json + /tmp/gh-aw/aw-*.patch + /tmp/gh-aw/aw-*.bundle + /tmp/gh-aw/sandbox/firewall/logs/ + /tmp/gh-aw/sandbox/firewall/audit/ if-no-files-found: ignore - # --- Threat Detection (inline) --- - - name: Check if detection needed - id: detection_guard - if: always() - env: - OUTPUT_TYPES: ${{ steps.collect_output.outputs.output_types }} - HAS_PATCH: ${{ steps.collect_output.outputs.has_patch }} - run: | - if [[ -n "$OUTPUT_TYPES" || "$HAS_PATCH" == "true" ]]; then - echo "run_detection=true" >> "$GITHUB_OUTPUT" - echo "Detection will run: output_types=$OUTPUT_TYPES, has_patch=$HAS_PATCH" - else - echo "run_detection=false" >> "$GITHUB_OUTPUT" - echo "Detection skipped: no agent outputs or patches to analyze" - fi - - name: Clear MCP configuration for detection - if: always() && steps.detection_guard.outputs.run_detection == 'true' - run: | - rm -f /tmp/gh-aw/mcp-config/mcp-servers.json - rm -f /home/runner/.copilot/mcp-config.json - rm -f "$GITHUB_WORKSPACE/.gemini/settings.json" - - name: Prepare threat detection files - if: always() && steps.detection_guard.outputs.run_detection == 'true' - run: | - mkdir -p /tmp/gh-aw/threat-detection/aw-prompts - cp /tmp/gh-aw/aw-prompts/prompt.txt /tmp/gh-aw/threat-detection/aw-prompts/prompt.txt 2>/dev/null || true - cp /tmp/gh-aw/agent_output.json /tmp/gh-aw/threat-detection/agent_output.json 2>/dev/null || true - for f in /tmp/gh-aw/aw-*.patch; do - [ -f "$f" ] && cp "$f" /tmp/gh-aw/threat-detection/ 2>/dev/null || true - done - echo "Prepared threat detection files:" - ls -la /tmp/gh-aw/threat-detection/ 2>/dev/null || true - - name: Setup threat detection - if: always() && steps.detection_guard.outputs.run_detection == 'true' - uses: actions/github-script@ed597411d8f924073f98dfc5c65a23a2325f34cd # v8 - env: - WORKFLOW_NAME: "Evaluate PR Tests" - WORKFLOW_DESCRIPTION: "Evaluates test quality, coverage, and appropriateness on PRs that add or modify tests" - HAS_PATCH: ${{ steps.collect_output.outputs.has_patch }} - with: - script: | - const { setupGlobals } = require('${{ runner.temp }}/gh-aw/actions/setup_globals.cjs'); - setupGlobals(core, github, context, exec, io); - const { main } = require('${{ runner.temp }}/gh-aw/actions/setup_threat_detection.cjs'); - await main(); - - name: Ensure threat-detection directory and log - if: always() && steps.detection_guard.outputs.run_detection == 'true' - run: | - mkdir -p /tmp/gh-aw/threat-detection - touch /tmp/gh-aw/threat-detection/detection.log - - name: Execute GitHub Copilot CLI - if: always() && steps.detection_guard.outputs.run_detection == 'true' - id: detection_agentic_execution - # Copilot CLI tool arguments (sorted): - # --allow-tool shell(cat) - # --allow-tool shell(grep) - # --allow-tool shell(head) - # --allow-tool shell(jq) - # --allow-tool shell(ls) - # --allow-tool shell(tail) - # --allow-tool shell(wc) - timeout-minutes: 20 - run: | - set -o pipefail - touch /tmp/gh-aw/agent-step-summary.md - # shellcheck disable=SC1003 - sudo -E awf --env-all --container-workdir "${GITHUB_WORKSPACE}" --mount "${RUNNER_TEMP}/gh-aw:${RUNNER_TEMP}/gh-aw:ro" --mount "${RUNNER_TEMP}/gh-aw:/host${RUNNER_TEMP}/gh-aw:ro" --allow-domains "api.business.githubcopilot.com,api.enterprise.githubcopilot.com,api.github.com,api.githubcopilot.com,api.individual.githubcopilot.com,github.com,host.docker.internal,raw.githubusercontent.com,registry.npmjs.org,telemetry.enterprise.githubcopilot.com" --log-level info --proxy-logs-dir /tmp/gh-aw/sandbox/firewall/logs --enable-host-access --image-tag 0.24.3 --skip-pull --enable-api-proxy \ - -- /bin/bash -c '/usr/local/bin/copilot --add-dir /tmp/gh-aw/ --log-level all --log-dir /tmp/gh-aw/sandbox/agent/logs/ --add-dir "${GITHUB_WORKSPACE}" --disable-builtin-mcps --allow-tool '\''shell(cat)'\'' --allow-tool '\''shell(grep)'\'' --allow-tool '\''shell(head)'\'' --allow-tool '\''shell(jq)'\'' --allow-tool '\''shell(ls)'\'' --allow-tool '\''shell(tail)'\'' --allow-tool '\''shell(wc)'\'' --prompt "$(cat /tmp/gh-aw/aw-prompts/prompt.txt)"' 2>&1 | tee -a /tmp/gh-aw/threat-detection/detection.log - env: - COPILOT_AGENT_RUNNER_TYPE: STANDALONE - COPILOT_GITHUB_TOKEN: ${{ secrets.COPILOT_GITHUB_TOKEN }} - COPILOT_MODEL: claude-sonnet-4.6 - GH_AW_PHASE: detection - GH_AW_PROMPT: /tmp/gh-aw/aw-prompts/prompt.txt - GH_AW_VERSION: v0.62.2 - GITHUB_API_URL: ${{ github.api_url }} - GITHUB_AW: true - GITHUB_HEAD_REF: ${{ github.head_ref }} - GITHUB_REF_NAME: ${{ github.ref_name }} - GITHUB_SERVER_URL: ${{ github.server_url }} - GITHUB_STEP_SUMMARY: /tmp/gh-aw/agent-step-summary.md - GITHUB_WORKSPACE: ${{ github.workspace }} - GIT_AUTHOR_EMAIL: github-actions[bot]@users.noreply.github.com - GIT_AUTHOR_NAME: github-actions[bot] - GIT_COMMITTER_EMAIL: github-actions[bot]@users.noreply.github.com - GIT_COMMITTER_NAME: github-actions[bot] - XDG_CONFIG_HOME: /home/runner - - name: Parse threat detection results - id: parse_detection_results - if: always() && steps.detection_guard.outputs.run_detection == 'true' - uses: actions/github-script@ed597411d8f924073f98dfc5c65a23a2325f34cd # v8 - with: - script: | - const { setupGlobals } = require('${{ runner.temp }}/gh-aw/actions/setup_globals.cjs'); - setupGlobals(core, github, context, exec, io); - const { main } = require('${{ runner.temp }}/gh-aw/actions/parse_threat_detection_results.cjs'); - await main(); - - name: Upload threat detection log - if: always() && steps.detection_guard.outputs.run_detection == 'true' - uses: actions/upload-artifact@bbbca2ddaa5d8feaa63e36b76fdaad77386f024f # v7.0.0 - with: - name: detection - path: /tmp/gh-aw/threat-detection/detection.log - if-no-files-found: ignore - - name: Set detection conclusion - id: detection_conclusion - if: always() - env: - RUN_DETECTION: ${{ steps.detection_guard.outputs.run_detection }} - DETECTION_SUCCESS: ${{ steps.parse_detection_results.outputs.success }} - run: | - if [[ "$RUN_DETECTION" != "true" ]]; then - echo "conclusion=skipped" >> "$GITHUB_OUTPUT" - echo "success=true" >> "$GITHUB_OUTPUT" - echo "Detection was not needed, marking as skipped" - elif [[ "$DETECTION_SUCCESS" == "true" ]]; then - echo "conclusion=success" >> "$GITHUB_OUTPUT" - echo "success=true" >> "$GITHUB_OUTPUT" - echo "Detection passed successfully" - else - echo "conclusion=failure" >> "$GITHUB_OUTPUT" - echo "success=false" >> "$GITHUB_OUTPUT" - echo "Detection found issues" - fi conclusion: needs: - activation - agent + - detection - safe_outputs - if: (always()) && ((needs.agent.result != 'skipped') || (needs.activation.outputs.lockdown_check_failed == 'true')) + if: > + always() && (needs.agent.result != 'skipped' || needs.activation.outputs.lockdown_check_failed == 'true' || + needs.activation.outputs.stale_lock_file_failed == 'true') runs-on: ubuntu-slim permissions: contents: read @@ -930,14 +904,18 @@ jobs: group: "gh-aw-conclusion-copilot-evaluate-tests" cancel-in-progress: false outputs: + incomplete_count: ${{ steps.report_incomplete.outputs.incomplete_count }} noop_message: ${{ steps.noop.outputs.noop_message }} tools_reported: ${{ steps.missing_tool.outputs.tools_reported }} total_count: ${{ steps.missing_tool.outputs.total_count }} steps: - name: Setup Scripts - uses: github/gh-aw-actions/setup@20045bbd5ad2632b9809856c389708eab1bd16ef # v0.62.2 + id: setup + uses: github/gh-aw-actions/setup@ba90f2186d7ad780ec640f364005fa24e797b360 # v0.68.3 with: destination: ${{ runner.temp }}/gh-aw/actions + job-name: ${{ github.job }} + trace-id: ${{ needs.activation.outputs.setup-trace-id }} - name: Download agent output artifact id: download-agent-output continue-on-error: true @@ -946,52 +924,92 @@ jobs: name: agent path: /tmp/gh-aw/ - name: Setup agent output environment variable + id: setup-agent-output-env if: steps.download-agent-output.outcome == 'success' run: | mkdir -p /tmp/gh-aw/ find "/tmp/gh-aw/" -type f -print - echo "GH_AW_AGENT_OUTPUT=/tmp/gh-aw/agent_output.json" >> "$GITHUB_ENV" - - name: Process No-Op Messages + echo "GH_AW_AGENT_OUTPUT=/tmp/gh-aw/agent_output.json" >> "$GITHUB_OUTPUT" + - name: Process no-op messages id: noop - uses: actions/github-script@ed597411d8f924073f98dfc5c65a23a2325f34cd # v8 + uses: actions/github-script@373c709c69115d41ff229c7e5df9f8788daa9553 # v9 env: - GH_AW_AGENT_OUTPUT: ${{ env.GH_AW_AGENT_OUTPUT }} + GH_AW_AGENT_OUTPUT: ${{ steps.setup-agent-output-env.outputs.GH_AW_AGENT_OUTPUT }} GH_AW_NOOP_MAX: "1" GH_AW_WORKFLOW_NAME: "Evaluate PR Tests" + GH_AW_RUN_URL: ${{ github.server_url }}/${{ github.repository }}/actions/runs/${{ github.run_id }} + GH_AW_AGENT_CONCLUSION: ${{ needs.agent.result }} + GH_AW_NOOP_REPORT_AS_ISSUE: "false" + with: + github-token: ${{ secrets.GH_AW_GITHUB_TOKEN || secrets.GITHUB_TOKEN }} + script: | + const { setupGlobals } = require('${{ runner.temp }}/gh-aw/actions/setup_globals.cjs'); + setupGlobals(core, github, context, exec, io, getOctokit); + const { main } = require('${{ runner.temp }}/gh-aw/actions/handle_noop_message.cjs'); + await main(); + - name: Log detection run + id: detection_runs + uses: actions/github-script@373c709c69115d41ff229c7e5df9f8788daa9553 # v9 + env: + GH_AW_AGENT_OUTPUT: ${{ steps.setup-agent-output-env.outputs.GH_AW_AGENT_OUTPUT }} + GH_AW_WORKFLOW_NAME: "Evaluate PR Tests" + GH_AW_RUN_URL: ${{ github.server_url }}/${{ github.repository }}/actions/runs/${{ github.run_id }} + GH_AW_DETECTION_CONCLUSION: ${{ needs.detection.outputs.detection_conclusion }} + GH_AW_DETECTION_REASON: ${{ needs.detection.outputs.detection_reason }} with: github-token: ${{ secrets.GH_AW_GITHUB_TOKEN || secrets.GITHUB_TOKEN }} script: | const { setupGlobals } = require('${{ runner.temp }}/gh-aw/actions/setup_globals.cjs'); - setupGlobals(core, github, context, exec, io); - const { main } = require('${{ runner.temp }}/gh-aw/actions/noop.cjs'); + setupGlobals(core, github, context, exec, io, getOctokit); + const { main } = require('${{ runner.temp }}/gh-aw/actions/handle_detection_runs.cjs'); await main(); - - name: Record Missing Tool + - name: Record missing tool id: missing_tool - uses: actions/github-script@ed597411d8f924073f98dfc5c65a23a2325f34cd # v8 + uses: actions/github-script@373c709c69115d41ff229c7e5df9f8788daa9553 # v9 env: - GH_AW_AGENT_OUTPUT: ${{ env.GH_AW_AGENT_OUTPUT }} + GH_AW_AGENT_OUTPUT: ${{ steps.setup-agent-output-env.outputs.GH_AW_AGENT_OUTPUT }} + GH_AW_MISSING_TOOL_CREATE_ISSUE: "true" GH_AW_WORKFLOW_NAME: "Evaluate PR Tests" with: github-token: ${{ secrets.GH_AW_GITHUB_TOKEN || secrets.GITHUB_TOKEN }} script: | const { setupGlobals } = require('${{ runner.temp }}/gh-aw/actions/setup_globals.cjs'); - setupGlobals(core, github, context, exec, io); + setupGlobals(core, github, context, exec, io, getOctokit); const { main } = require('${{ runner.temp }}/gh-aw/actions/missing_tool.cjs'); await main(); - - name: Handle Agent Failure + - name: Record incomplete + id: report_incomplete + uses: actions/github-script@373c709c69115d41ff229c7e5df9f8788daa9553 # v9 + env: + GH_AW_AGENT_OUTPUT: ${{ steps.setup-agent-output-env.outputs.GH_AW_AGENT_OUTPUT }} + GH_AW_REPORT_INCOMPLETE_CREATE_ISSUE: "true" + GH_AW_WORKFLOW_NAME: "Evaluate PR Tests" + with: + github-token: ${{ secrets.GH_AW_GITHUB_TOKEN || secrets.GITHUB_TOKEN }} + script: | + const { setupGlobals } = require('${{ runner.temp }}/gh-aw/actions/setup_globals.cjs'); + setupGlobals(core, github, context, exec, io, getOctokit); + const { main } = require('${{ runner.temp }}/gh-aw/actions/report_incomplete_handler.cjs'); + await main(); + - name: Handle agent failure id: handle_agent_failure if: always() - uses: actions/github-script@ed597411d8f924073f98dfc5c65a23a2325f34cd # v8 + uses: actions/github-script@373c709c69115d41ff229c7e5df9f8788daa9553 # v9 env: - GH_AW_AGENT_OUTPUT: ${{ env.GH_AW_AGENT_OUTPUT }} + GH_AW_AGENT_OUTPUT: ${{ steps.setup-agent-output-env.outputs.GH_AW_AGENT_OUTPUT }} GH_AW_WORKFLOW_NAME: "Evaluate PR Tests" GH_AW_RUN_URL: ${{ github.server_url }}/${{ github.repository }}/actions/runs/${{ github.run_id }} GH_AW_AGENT_CONCLUSION: ${{ needs.agent.result }} GH_AW_WORKFLOW_ID: "copilot-evaluate-tests" + GH_AW_ENGINE_ID: "copilot" GH_AW_SECRET_VERIFICATION_RESULT: ${{ needs.activation.outputs.secret_verification_result }} GH_AW_CHECKOUT_PR_SUCCESS: ${{ needs.agent.outputs.checkout_pr_success }} GH_AW_INFERENCE_ACCESS_ERROR: ${{ needs.agent.outputs.inference_access_error }} + GH_AW_MCP_POLICY_ERROR: ${{ needs.agent.outputs.mcp_policy_error }} + GH_AW_AGENTIC_ENGINE_TIMEOUT: ${{ needs.agent.outputs.agentic_engine_timeout }} + GH_AW_MODEL_NOT_SUPPORTED_ERROR: ${{ needs.agent.outputs.model_not_supported_error }} GH_AW_LOCKDOWN_CHECK_FAILED: ${{ needs.activation.outputs.lockdown_check_failed }} + GH_AW_STALE_LOCK_FILE_FAILED: ${{ needs.activation.outputs.stale_lock_file_failed }} GH_AW_SAFE_OUTPUT_MESSAGES: "{\"footer\":\"\\u003e πŸ§ͺ *Test evaluation by [{workflow_name}]({run_url})*\",\"runStarted\":\"πŸ”¬ Evaluating tests on this PR… [{workflow_name}]({run_url})\",\"runSuccess\":\"βœ… Test evaluation complete! [{workflow_name}]({run_url})\",\"runFailure\":\"❌ Test evaluation failed. [{workflow_name}]({run_url}) {status}\"}" GH_AW_GROUP_REPORTS: "false" GH_AW_FAILURE_REPORT_AS_ISSUE: "true" @@ -1000,85 +1018,232 @@ jobs: github-token: ${{ secrets.GH_AW_GITHUB_TOKEN || secrets.GITHUB_TOKEN }} script: | const { setupGlobals } = require('${{ runner.temp }}/gh-aw/actions/setup_globals.cjs'); - setupGlobals(core, github, context, exec, io); + setupGlobals(core, github, context, exec, io, getOctokit); const { main } = require('${{ runner.temp }}/gh-aw/actions/handle_agent_failure.cjs'); await main(); - - name: Handle No-Op Message - id: handle_noop_message - uses: actions/github-script@ed597411d8f924073f98dfc5c65a23a2325f34cd # v8 - env: - GH_AW_AGENT_OUTPUT: ${{ env.GH_AW_AGENT_OUTPUT }} - GH_AW_WORKFLOW_NAME: "Evaluate PR Tests" - GH_AW_RUN_URL: ${{ github.server_url }}/${{ github.repository }}/actions/runs/${{ github.run_id }} - GH_AW_AGENT_CONCLUSION: ${{ needs.agent.result }} - GH_AW_NOOP_MESSAGE: ${{ steps.noop.outputs.noop_message }} - GH_AW_NOOP_REPORT_AS_ISSUE: "false" - with: - github-token: ${{ secrets.GH_AW_GITHUB_TOKEN || secrets.GITHUB_TOKEN }} - script: | - const { setupGlobals } = require('${{ runner.temp }}/gh-aw/actions/setup_globals.cjs'); - setupGlobals(core, github, context, exec, io); - const { main } = require('${{ runner.temp }}/gh-aw/actions/handle_noop_message.cjs'); - await main(); - name: Update reaction comment with completion status id: conclusion - uses: actions/github-script@ed597411d8f924073f98dfc5c65a23a2325f34cd # v8 + uses: actions/github-script@373c709c69115d41ff229c7e5df9f8788daa9553 # v9 env: - GH_AW_AGENT_OUTPUT: ${{ env.GH_AW_AGENT_OUTPUT }} + GH_AW_AGENT_OUTPUT: ${{ steps.setup-agent-output-env.outputs.GH_AW_AGENT_OUTPUT }} GH_AW_COMMENT_ID: ${{ needs.activation.outputs.comment_id }} GH_AW_COMMENT_REPO: ${{ needs.activation.outputs.comment_repo }} GH_AW_RUN_URL: ${{ github.server_url }}/${{ github.repository }}/actions/runs/${{ github.run_id }} GH_AW_WORKFLOW_NAME: "Evaluate PR Tests" GH_AW_AGENT_CONCLUSION: ${{ needs.agent.result }} - GH_AW_DETECTION_CONCLUSION: ${{ needs.agent.outputs.detection_conclusion }} + GH_AW_DETECTION_CONCLUSION: ${{ needs.detection.outputs.detection_conclusion }} + GH_AW_DETECTION_REASON: ${{ needs.detection.outputs.detection_reason }} GH_AW_SAFE_OUTPUT_MESSAGES: "{\"footer\":\"\\u003e πŸ§ͺ *Test evaluation by [{workflow_name}]({run_url})*\",\"runStarted\":\"πŸ”¬ Evaluating tests on this PR… [{workflow_name}]({run_url})\",\"runSuccess\":\"βœ… Test evaluation complete! [{workflow_name}]({run_url})\",\"runFailure\":\"❌ Test evaluation failed. [{workflow_name}]({run_url}) {status}\"}" with: github-token: ${{ secrets.GH_AW_GITHUB_TOKEN || secrets.GITHUB_TOKEN }} script: | const { setupGlobals } = require('${{ runner.temp }}/gh-aw/actions/setup_globals.cjs'); - setupGlobals(core, github, context, exec, io); + setupGlobals(core, github, context, exec, io, getOctokit); const { main } = require('${{ runner.temp }}/gh-aw/actions/notify_comment_error.cjs'); await main(); + detection: + needs: + - activation + - agent + if: > + always() && needs.agent.result != 'skipped' && (needs.agent.outputs.output_types != '' || needs.agent.outputs.has_patch == 'true') + runs-on: ubuntu-latest + permissions: + contents: read + outputs: + detection_conclusion: ${{ steps.detection_conclusion.outputs.conclusion }} + detection_reason: ${{ steps.detection_conclusion.outputs.reason }} + detection_success: ${{ steps.detection_conclusion.outputs.success }} + steps: + - name: Setup Scripts + id: setup + uses: github/gh-aw-actions/setup@ba90f2186d7ad780ec640f364005fa24e797b360 # v0.68.3 + with: + destination: ${{ runner.temp }}/gh-aw/actions + job-name: ${{ github.job }} + trace-id: ${{ needs.activation.outputs.setup-trace-id }} + - name: Download agent output artifact + id: download-agent-output + continue-on-error: true + uses: actions/download-artifact@3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c # v8.0.1 + with: + name: agent + path: /tmp/gh-aw/ + - name: Setup agent output environment variable + id: setup-agent-output-env + if: steps.download-agent-output.outcome == 'success' + run: | + mkdir -p /tmp/gh-aw/ + find "/tmp/gh-aw/" -type f -print + echo "GH_AW_AGENT_OUTPUT=/tmp/gh-aw/agent_output.json" >> "$GITHUB_OUTPUT" + - name: Checkout repository for patch context + if: needs.agent.outputs.has_patch == 'true' + uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2 + with: + persist-credentials: false + # --- Threat Detection --- + - name: Clean stale firewall files from agent artifact + run: | + rm -rf /tmp/gh-aw/sandbox/firewall/logs + rm -rf /tmp/gh-aw/sandbox/firewall/audit + - name: Download container images + run: bash "${RUNNER_TEMP}/gh-aw/actions/download_docker_images.sh" ghcr.io/github/gh-aw-firewall/agent:0.25.20 ghcr.io/github/gh-aw-firewall/api-proxy:0.25.20 ghcr.io/github/gh-aw-firewall/squid:0.25.20 + - name: Check if detection needed + id: detection_guard + if: always() + env: + OUTPUT_TYPES: ${{ needs.agent.outputs.output_types }} + HAS_PATCH: ${{ needs.agent.outputs.has_patch }} + run: | + if [[ -n "$OUTPUT_TYPES" || "$HAS_PATCH" == "true" ]]; then + echo "run_detection=true" >> "$GITHUB_OUTPUT" + echo "Detection will run: output_types=$OUTPUT_TYPES, has_patch=$HAS_PATCH" + else + echo "run_detection=false" >> "$GITHUB_OUTPUT" + echo "Detection skipped: no agent outputs or patches to analyze" + fi + - name: Clear MCP configuration for detection + if: always() && steps.detection_guard.outputs.run_detection == 'true' + run: | + rm -f /tmp/gh-aw/mcp-config/mcp-servers.json + rm -f /home/runner/.copilot/mcp-config.json + rm -f "$GITHUB_WORKSPACE/.gemini/settings.json" + - name: Prepare threat detection files + if: always() && steps.detection_guard.outputs.run_detection == 'true' + run: | + mkdir -p /tmp/gh-aw/threat-detection/aw-prompts + cp /tmp/gh-aw/aw-prompts/prompt.txt /tmp/gh-aw/threat-detection/aw-prompts/prompt.txt 2>/dev/null || true + cp /tmp/gh-aw/agent_output.json /tmp/gh-aw/threat-detection/agent_output.json 2>/dev/null || true + for f in /tmp/gh-aw/aw-*.patch; do + [ -f "$f" ] && cp "$f" /tmp/gh-aw/threat-detection/ 2>/dev/null || true + done + for f in /tmp/gh-aw/aw-*.bundle; do + [ -f "$f" ] && cp "$f" /tmp/gh-aw/threat-detection/ 2>/dev/null || true + done + echo "Prepared threat detection files:" + ls -la /tmp/gh-aw/threat-detection/ 2>/dev/null || true + - name: Setup threat detection + if: always() && steps.detection_guard.outputs.run_detection == 'true' + uses: actions/github-script@373c709c69115d41ff229c7e5df9f8788daa9553 # v9 + env: + WORKFLOW_NAME: "Evaluate PR Tests" + WORKFLOW_DESCRIPTION: "Evaluates test quality, coverage, and appropriateness on PRs that add or modify tests" + HAS_PATCH: ${{ needs.agent.outputs.has_patch }} + with: + script: | + const { setupGlobals } = require('${{ runner.temp }}/gh-aw/actions/setup_globals.cjs'); + setupGlobals(core, github, context, exec, io, getOctokit); + const { main } = require('${{ runner.temp }}/gh-aw/actions/setup_threat_detection.cjs'); + await main(); + - name: Ensure threat-detection directory and log + if: always() && steps.detection_guard.outputs.run_detection == 'true' + run: | + mkdir -p /tmp/gh-aw/threat-detection + touch /tmp/gh-aw/threat-detection/detection.log + - name: Install GitHub Copilot CLI + run: bash "${RUNNER_TEMP}/gh-aw/actions/install_copilot_cli.sh" 1.0.21 + env: + GH_HOST: github.com + - name: Install AWF binary + run: bash "${RUNNER_TEMP}/gh-aw/actions/install_awf_binary.sh" v0.25.20 + - name: Execute GitHub Copilot CLI + if: always() && steps.detection_guard.outputs.run_detection == 'true' + id: detection_agentic_execution + # Copilot CLI tool arguments (sorted): + timeout-minutes: 20 + run: | + set -o pipefail + touch /tmp/gh-aw/agent-step-summary.md + (umask 177 && touch /tmp/gh-aw/threat-detection/detection.log) + # shellcheck disable=SC1003 + sudo -E awf --container-workdir "${GITHUB_WORKSPACE}" --mount "${RUNNER_TEMP}/gh-aw:${RUNNER_TEMP}/gh-aw:ro" --mount "${RUNNER_TEMP}/gh-aw:/host${RUNNER_TEMP}/gh-aw:ro" --env-all --exclude-env COPILOT_GITHUB_TOKEN --allow-domains api.business.githubcopilot.com,api.enterprise.githubcopilot.com,api.github.com,api.githubcopilot.com,api.individual.githubcopilot.com,github.com,host.docker.internal,telemetry.enterprise.githubcopilot.com --log-level info --proxy-logs-dir /tmp/gh-aw/sandbox/firewall/logs --audit-dir /tmp/gh-aw/sandbox/firewall/audit --enable-host-access --image-tag 0.25.20 --skip-pull --enable-api-proxy \ + -- /bin/bash -c 'node ${RUNNER_TEMP}/gh-aw/actions/copilot_driver.cjs /usr/local/bin/copilot --add-dir /tmp/gh-aw/ --log-level all --log-dir /tmp/gh-aw/sandbox/agent/logs/ --disable-builtin-mcps --no-ask-user --allow-all-tools --add-dir "${GITHUB_WORKSPACE}" --prompt "$(cat /tmp/gh-aw/aw-prompts/prompt.txt)"' 2>&1 | tee -a /tmp/gh-aw/threat-detection/detection.log + env: + COPILOT_AGENT_RUNNER_TYPE: STANDALONE + COPILOT_GITHUB_TOKEN: ${{ secrets.COPILOT_GITHUB_TOKEN }} + COPILOT_MODEL: claude-sonnet-4.6 + GH_AW_PHASE: detection + GH_AW_PROMPT: /tmp/gh-aw/aw-prompts/prompt.txt + GH_AW_VERSION: v0.68.3 + GITHUB_API_URL: ${{ github.api_url }} + GITHUB_AW: true + GITHUB_HEAD_REF: ${{ github.head_ref }} + GITHUB_REF_NAME: ${{ github.ref_name }} + GITHUB_SERVER_URL: ${{ github.server_url }} + GITHUB_STEP_SUMMARY: /tmp/gh-aw/agent-step-summary.md + GITHUB_WORKSPACE: ${{ github.workspace }} + GIT_AUTHOR_EMAIL: github-actions[bot]@users.noreply.github.com + GIT_AUTHOR_NAME: github-actions[bot] + GIT_COMMITTER_EMAIL: github-actions[bot]@users.noreply.github.com + GIT_COMMITTER_NAME: github-actions[bot] + XDG_CONFIG_HOME: /home/runner + - name: Upload threat detection log + if: always() && steps.detection_guard.outputs.run_detection == 'true' + uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1 + with: + name: detection + path: /tmp/gh-aw/threat-detection/detection.log + if-no-files-found: ignore + - name: Parse and conclude threat detection + id: detection_conclusion + if: always() + uses: actions/github-script@373c709c69115d41ff229c7e5df9f8788daa9553 # v9 + env: + RUN_DETECTION: ${{ steps.detection_guard.outputs.run_detection }} + GH_AW_DETECTION_CONTINUE_ON_ERROR: "true" + with: + script: | + const { setupGlobals } = require('${{ runner.temp }}/gh-aw/actions/setup_globals.cjs'); + setupGlobals(core, github, context, exec, io, getOctokit); + const { main } = require('${{ runner.temp }}/gh-aw/actions/parse_threat_detection_results.cjs'); + await main(); + pre_activation: if: github.event_name == 'issue_comment' || github.event_name == 'workflow_dispatch' runs-on: ubuntu-slim outputs: - activated: ${{ (steps.check_membership.outputs.is_team_member == 'true') && (steps.check_command_position.outputs.command_position_ok == 'true') }} + activated: ${{ steps.check_membership.outputs.is_team_member == 'true' && steps.check_command_position.outputs.command_position_ok == 'true' }} matched_command: ${{ steps.check_command_position.outputs.matched_command }} + setup-trace-id: ${{ steps.setup.outputs.trace-id }} steps: - name: Setup Scripts - uses: github/gh-aw-actions/setup@20045bbd5ad2632b9809856c389708eab1bd16ef # v0.62.2 + id: setup + uses: github/gh-aw-actions/setup@ba90f2186d7ad780ec640f364005fa24e797b360 # v0.68.3 with: destination: ${{ runner.temp }}/gh-aw/actions + job-name: ${{ github.job }} - name: Check team membership for command workflow id: check_membership - uses: actions/github-script@ed597411d8f924073f98dfc5c65a23a2325f34cd # v8 + uses: actions/github-script@373c709c69115d41ff229c7e5df9f8788daa9553 # v9 env: - GH_AW_REQUIRED_ROLES: admin,maintainer,write - GH_AW_ALLOWED_BOTS: copilot-swe-agent[bot] + GH_AW_REQUIRED_ROLES: "admin,maintain,write" + GH_AW_ALLOWED_BOTS: "copilot-swe-agent[bot]" with: github-token: ${{ secrets.GITHUB_TOKEN }} script: | const { setupGlobals } = require('${{ runner.temp }}/gh-aw/actions/setup_globals.cjs'); - setupGlobals(core, github, context, exec, io); + setupGlobals(core, github, context, exec, io, getOctokit); const { main } = require('${{ runner.temp }}/gh-aw/actions/check_membership.cjs'); await main(); - name: Check command position id: check_command_position - uses: actions/github-script@ed597411d8f924073f98dfc5c65a23a2325f34cd # v8 + uses: actions/github-script@373c709c69115d41ff229c7e5df9f8788daa9553 # v9 env: GH_AW_COMMANDS: "[\"evaluate-tests\"]" with: script: | const { setupGlobals } = require('${{ runner.temp }}/gh-aw/actions/setup_globals.cjs'); - setupGlobals(core, github, context, exec, io); + setupGlobals(core, github, context, exec, io, getOctokit); const { main } = require('${{ runner.temp }}/gh-aw/actions/check_command_position.cjs'); await main(); safe_outputs: - needs: agent - if: ((!cancelled()) && (needs.agent.result != 'skipped')) && (needs.agent.outputs.detection_success == 'true') + needs: + - activation + - agent + - detection + if: (!cancelled()) && needs.agent.result != 'skipped' && needs.detection.result == 'success' runs-on: ubuntu-slim permissions: contents: read @@ -1088,6 +1253,9 @@ jobs: timeout-minutes: 15 env: GH_AW_CALLER_WORKFLOW_ID: "${{ github.repository }}/copilot-evaluate-tests" + GH_AW_DETECTION_CONCLUSION: ${{ needs.detection.outputs.detection_conclusion }} + GH_AW_DETECTION_REASON: ${{ needs.detection.outputs.detection_reason }} + GH_AW_EFFECTIVE_TOKENS: ${{ needs.agent.outputs.effective_tokens }} GH_AW_ENGINE_ID: "copilot" GH_AW_ENGINE_MODEL: "claude-sonnet-4.6" GH_AW_SAFE_OUTPUT_MESSAGES: "{\"footer\":\"\\u003e πŸ§ͺ *Test evaluation by [{workflow_name}]({run_url})*\",\"runStarted\":\"πŸ”¬ Evaluating tests on this PR… [{workflow_name}]({run_url})\",\"runSuccess\":\"βœ… Test evaluation complete! [{workflow_name}]({run_url})\",\"runFailure\":\"❌ Test evaluation failed. [{workflow_name}]({run_url}) {status}\"}" @@ -1104,9 +1272,12 @@ jobs: process_safe_outputs_temporary_id_map: ${{ steps.process_safe_outputs.outputs.temporary_id_map }} steps: - name: Setup Scripts - uses: github/gh-aw-actions/setup@20045bbd5ad2632b9809856c389708eab1bd16ef # v0.62.2 + id: setup + uses: github/gh-aw-actions/setup@ba90f2186d7ad780ec640f364005fa24e797b360 # v0.68.3 with: destination: ${{ runner.temp }}/gh-aw/actions + job-name: ${{ github.job }} + trace-id: ${{ needs.activation.outputs.setup-trace-id }} - name: Download agent output artifact id: download-agent-output continue-on-error: true @@ -1115,12 +1286,14 @@ jobs: name: agent path: /tmp/gh-aw/ - name: Setup agent output environment variable + id: setup-agent-output-env if: steps.download-agent-output.outcome == 'success' run: | mkdir -p /tmp/gh-aw/ find "/tmp/gh-aw/" -type f -print - echo "GH_AW_AGENT_OUTPUT=/tmp/gh-aw/agent_output.json" >> "$GITHUB_ENV" + echo "GH_AW_AGENT_OUTPUT=/tmp/gh-aw/agent_output.json" >> "$GITHUB_OUTPUT" - name: Configure GH_HOST for enterprise compatibility + id: ghes-host-config shell: bash run: | # Derive GH_HOST from GITHUB_SERVER_URL so the gh CLI targets the correct @@ -1130,25 +1303,27 @@ jobs: echo "GH_HOST=${GH_HOST}" >> "$GITHUB_ENV" - name: Process Safe Outputs id: process_safe_outputs - uses: actions/github-script@ed597411d8f924073f98dfc5c65a23a2325f34cd # v8 + uses: actions/github-script@373c709c69115d41ff229c7e5df9f8788daa9553 # v9 env: - GH_AW_AGENT_OUTPUT: ${{ env.GH_AW_AGENT_OUTPUT }} + GH_AW_AGENT_OUTPUT: ${{ steps.setup-agent-output-env.outputs.GH_AW_AGENT_OUTPUT }} GH_AW_ALLOWED_DOMAINS: "api.business.githubcopilot.com,api.enterprise.githubcopilot.com,api.github.com,api.githubcopilot.com,api.individual.githubcopilot.com,api.snapcraft.io,archive.ubuntu.com,azure.archive.ubuntu.com,crl.geotrust.com,crl.globalsign.com,crl.identrust.com,crl.sectigo.com,crl.thawte.com,crl.usertrust.com,crl.verisign.com,crl3.digicert.com,crl4.digicert.com,crls.ssl.com,github.com,host.docker.internal,json-schema.org,json.schemastore.org,keyserver.ubuntu.com,ocsp.digicert.com,ocsp.geotrust.com,ocsp.globalsign.com,ocsp.identrust.com,ocsp.sectigo.com,ocsp.ssl.com,ocsp.thawte.com,ocsp.usertrust.com,ocsp.verisign.com,packagecloud.io,packages.cloud.google.com,packages.microsoft.com,ppa.launchpad.net,raw.githubusercontent.com,registry.npmjs.org,s.symcb.com,s.symcd.com,security.ubuntu.com,telemetry.enterprise.githubcopilot.com,ts-crl.ws.symantec.com,ts-ocsp.ws.symantec.com,www.googleapis.com" GITHUB_SERVER_URL: ${{ github.server_url }} GITHUB_API_URL: ${{ github.api_url }} - GH_AW_SAFE_OUTPUTS_HANDLER_CONFIG: "{\"add_comment\":{\"hide_older_comments\":true,\"max\":1,\"target\":\"*\"},\"missing_data\":{},\"missing_tool\":{},\"noop\":{\"max\":1,\"report-as-issue\":\"false\"}}" + GH_AW_SAFE_OUTPUTS_HANDLER_CONFIG: "{\"add_comment\":{\"hide_older_comments\":true,\"max\":1,\"target\":\"*\"},\"create_report_incomplete_issue\":{},\"missing_data\":{},\"missing_tool\":{},\"noop\":{\"max\":1,\"report-as-issue\":\"false\"},\"report_incomplete\":{}}" with: github-token: ${{ secrets.GH_AW_GITHUB_TOKEN || secrets.GITHUB_TOKEN }} script: | const { setupGlobals } = require('${{ runner.temp }}/gh-aw/actions/setup_globals.cjs'); - setupGlobals(core, github, context, exec, io); + setupGlobals(core, github, context, exec, io, getOctokit); const { main } = require('${{ runner.temp }}/gh-aw/actions/safe_output_handler_manager.cjs'); await main(); - - name: Upload safe output items + - name: Upload Safe Outputs Items if: always() - uses: actions/upload-artifact@bbbca2ddaa5d8feaa63e36b76fdaad77386f024f # v7.0.0 + uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1 with: - name: safe-output-items - path: /tmp/gh-aw/safe-output-items.jsonl + name: safe-outputs-items + path: | + /tmp/gh-aw/safe-output-items.jsonl + /tmp/gh-aw/temporary-id-map.json if-no-files-found: ignore diff --git a/.github/workflows/copilot-evaluate-tests.md b/.github/workflows/copilot-evaluate-tests.md index 7752f29a6fea..93235d7f9fc9 100644 --- a/.github/workflows/copilot-evaluate-tests.md +++ b/.github/workflows/copilot-evaluate-tests.md @@ -21,6 +21,7 @@ on: required: false type: boolean default: false + roles: [admin, maintain, write] bots: - "copilot-swe-agent[bot]" @@ -62,7 +63,7 @@ network: defaults concurrency: group: "evaluate-pr-tests-${{ github.event.issue.number || inputs.pr_number || github.run_id }}" - cancel-in-progress: true + cancel-in-progress: false timeout-minutes: 20 @@ -104,23 +105,6 @@ steps: fi echo "βœ… Found test files to evaluate:" echo "$TEST_FILES" | head -20 - - # For slash_command triggers, the gh-aw platform's checkout_pr_branch.cjs runs - # AFTER all user steps and overlays the PR branch onto the workspace. This means - # fork PRs can supply their own .github/skills/ and .github/instructions/. - # We cannot restore trusted infra here because the platform checkout runs later. - # Mitigation: agent is sandboxed (no credentials), max 1 comment via safe-outputs, - # and the agent prompt includes a pre-flight check that catches missing SKILL.md. - # See: .github/instructions/gh-aw-workflows.instructions.md "The issue_comment + Fork Problem" - - # For workflow_dispatch, the platform skips checkout entirely β€” this step is the - # only thing that gets the PR code onto disk and restores trusted infra from main. - - name: Checkout PR and restore agent infrastructure - if: github.event_name == 'workflow_dispatch' - env: - GH_TOKEN: ${{ github.token }} - PR_NUMBER: ${{ inputs.pr_number }} - run: pwsh .github/scripts/Checkout-GhAwPr.ps1 --- # Evaluate PR Tests @@ -132,7 +116,7 @@ Invoke the **evaluate-pr-tests** skill: read and follow `.github/skills/evaluate - **Repository**: ${{ github.repository }} - **PR Number**: ${{ github.event.issue.number || inputs.pr_number }} -The PR branch has been checked out for you. All files from the PR are available locally. +The PR is available via MCP tools. Use `gh pr view` and `gh pr diff` to access PR data. ## Pre-flight check diff --git a/docs/specs/XamlXCode.md b/docs/specs/XamlXCode.md deleted file mode 100644 index 4db3d7269c09..000000000000 --- a/docs/specs/XamlXCode.md +++ /dev/null @@ -1,221 +0,0 @@ -# XAML x:Code Directive - -## Overview - -The `x:Code` directive allows embedding inline C# member declarations directly in XAML files. The XAML source generator extracts these code blocks and emits them as part of a partial class, making them available alongside the code-behind. - -### Motivation - -Currently, any C# logic associated with a XAML page must live in a separate code-behind file. For simple cases β€” a single event handler, a helper method, a field β€” switching between XAML and code-behind adds friction. `x:Code` lets developers keep tightly-coupled logic next to the markup that uses it: - -- Small event handlers can live next to the control they serve -- Helper methods used by a single page don't need a separate file -- Prototyping is faster when everything is in one file - -### Example - -```xml - - - - -