Add gh-aw workflow for UI test category detection by kubaflo · Pull Request #34856 · dotnet/maui

kubaflo · 2026-04-07T18:29:25Z

Note

Are you waiting for the changes in this PR to be merged?
It would be very helpful if you could test the resulting artifacts from this PR and let us know in a comment if this change resolves your issue. Thank you!

Description

Adds a gh-aw (GitHub Agentic Workflows) workflow that automatically detects UI test categories from PR diffs and posts a comment listing which categories should run on CI pipelines.

What it does

When a PR adds or modifies UI test files, this workflow:

Detects [Category(UITestCategories.X)] attributes in added/modified lines of the PR diff
Posts a PR comment listing the detected categories and the pipeline filter to use
Supports UITestCategories.X, nameof(UITestCategories.X), and quoted string formats

Triggers

Trigger	When	Automatic?
`pull_request`	On PR open/sync/reopen/ready (NO paths filter)	✅ Yes
`workflow_dispatch`	Manual — enter PR number	Manual
`issue_comment` (`/detect-categories`)	Comment on PR	Manual

Important: No paths: filter on pull_request to avoid workflow indexing issues. A gate step checks if UI test files exist in the diff before proceeding.

Security model

Same as copilot-evaluate-tests (PR #34548):

gh-aw sandbox with scrubbed credentials
Safe outputs (max 1 PR comment per run)
Fork PR activation gate
Checkout-GhAwPr.ps1 for workflow_dispatch
Pinned actions via actions-lock.json

Files added/modified

.github/workflows/copilot-detect-categories.md — gh-aw workflow source
.github/workflows/copilot-detect-categories.lock.yml — Compiled workflow (auto-generated)
.github/aw/actions-lock.json — Updated with new action SHA

Relation to PR #33176

This workflow complements the Azure DevOps pipeline changes in PR #33176 (detect-ui-test-categories) by providing the same category detection as a GitHub-native agentic workflow that posts results as PR comments.

Testing

To trigger manually on any PR:

gh workflow run copilot-detect-categories.lock.yml --repo dotnet/maui --ref feature/copilot-detect-categories -f pr_number=<TEST_PR>

Adds a GitHub Agentic Workflow (copilot-detect-categories) that automatically detects UI test categories from PR diffs and posts a comment listing which categories should run on CI pipelines. Triggers: - pull_request (opened/sync/reopen/ready) — no paths filter, uses gate step - issue_comment (/detect-categories) - workflow_dispatch (manual with PR number) Security: same sandbox model as copilot-evaluate-tests — safe-outputs, fork guard, Checkout-GhAwPr.ps1 for workflow_dispatch. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

github-actions · 2026-04-07T18:29:36Z

🚀 Dogfood this PR with:

⚠️ WARNING: Do not do this without first carefully reviewing the code of this PR to satisfy yourself it is safe.

curl -fsSL https://raw.githubusercontent.com/dotnet/maui/main/eng/scripts/get-maui-pr.sh | bash -s -- 34856

Or

Run remotely in PowerShell:

iex "& { $(irm https://raw.githubusercontent.com/dotnet/maui/main/eng/scripts/get-maui-pr.ps1) } 34856"

Copilot

Pull request overview

Adds a new gh-aw (GitHub Agentic Workflows) workflow to detect UI test categories from PR diffs and post a PR comment describing which UI test categories were detected (and what to run).

Changes:

Added .github/workflows/copilot-detect-categories.md defining triggers, gating, diff parsing, and the agent prompt/comment format.
Added compiled workflow .github/workflows/copilot-detect-categories.lock.yml.
Updated .github/aw/actions-lock.json to include the new gh-aw setup action version/sha.

Reviewed changes

Copilot reviewed 3 out of 3 changed files in this pull request and generated 3 comments.

File	Description
.github/workflows/copilot-detect-categories.md	New gh-aw workflow source: gates on UI-test file changes, extracts `[Category(...)]` additions from PR diff, and instructs agent to post a summary comment.
.github/workflows/copilot-detect-categories.lock.yml	Auto-generated compiled workflow for the new gh-aw workflow.
.github/aw/actions-lock.json	Adds the pinned action entry needed by the compiled workflow.

Copilot · 2026-04-07T18:33:47Z

.github/workflows/copilot-detect-categories.md

+    run: |
+      TEST_FILES=$(gh pr diff "$PR_NUMBER" --repo "$GITHUB_REPOSITORY" --name-only \
+        | grep -E '\.(cs|xaml)$' \
+        | grep -iE '(TestCases\.Shared\.Tests|TestCases\.HostApp|UITest|Xaml\.UnitTests)' \


The gate’s path filter misses several existing MAUI UI test project paths (e.g., src/Controls/tests/TestCases.iOS.Tests, .WinUI.Tests, .Android.Tests, .Mac.Tests). As a result, PRs that only touch those tests will be incorrectly skipped. Consider broadening this to match TestCases\. or TestCases (similar to copilot-evaluate-tests.md) or explicitly include all TestCases.*.Tests variants.

Suggested change

| grep -iE '(TestCases\.Shared\.Tests|TestCases\.HostApp|UITest|Xaml\.UnitTests)' \

| grep -iE '(TestCases(\.[^/]+\.Tests)?|TestCases\.HostApp|UITest|Xaml\.UnitTests)' \

Copilot · 2026-04-07T18:33:47Z

.github/workflows/copilot-detect-categories.md

+      # Get list of changed test files
+      CHANGED_FILES=$(gh pr diff "$PR_NUMBER" --repo "$GITHUB_REPOSITORY" --name-only 2>/dev/null \
+        | grep -E '\.(cs|xaml)$' \
+        | grep -iE '(TestCases\.Shared\.Tests|TestCases\.HostApp|UITest|Xaml\.UnitTests)' \


Copilot · 2026-04-07T18:33:48Z

.github/workflows/copilot-detect-categories.md

+| Category | Pipeline Filter |
+|----------|----------------|
+| {CategoryName} | `Category={CategoryName}` |
+| ... | ... |
+
+**To run only these categories on CI**, use the following filter:
+
+```
+{comma-separated category list}
+```


The suggested pipeline filter syntax here appears incorrect for MAUI UI tests. eng/pipelines/common/ui-tests-steps.yml builds the --test-filter expression using TestCategory=<name> (not Category=<name>), after splitting a comma-separated category list. The comment format should reflect TestCategory={CategoryName} and/or explain that the pipeline input expects a comma-separated list (which becomes TestCategory=A|TestCategory=B).

- eng/pipelines/ci-regression-detection.yml: New pipeline that runs targeted UI tests (by category) and optional device tests (Helix). Accepts uiTestCategories parameter (comma-separated) and runDeviceTests boolean. Manual trigger only. - Updated copilot-detect-categories workflow to: - Detect BOTH UI test categories (UITestCategories.X) and device test categories (TestCategory.X) from PR diffs - Infer categories from source file paths when no test changes exist - Broader gate: any src/ .cs/.xaml file triggers analysis (not just tests) - Post comment with pipeline trigger instructions for maui-pr-regression-detection Note: Pipeline registration in AzDO requires admin permissions. The YAML is ready — needs to be registered as 'maui-pr-regression-detection' under \dotnet\maui. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Remove ci-regression-detection.yml and instead add category filtering parameters to the existing pipelines: - ci-uitests.yml: Add 'uiTestCategories' parameter. When set (comma- separated), overrides categoryGroupsToTest to run only specified categories. Empty (default) runs all categories as before. - ci-device-tests.yml: Add 'deviceTestCategories' parameter. When set (semicolon-separated), passes DeviceTestCategoryFilter to Helix. - stage-device-tests.yml: Wire extraHelixArguments through to all HelixProjectArguments (iOS, Catalyst, Android, Android CoreCLR, Windows). Previously declared but never used. - helix_xharness.proj: Add DeviceTestCategoryFilter support. When set, disables iOS category splitting and applies the filter to all Apple work items via CustomCommands and to Android via instrumentation args. - Updated gh-aw workflow to reference existing pipelines in the comment. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

When manually triggering maui-pr-uitests or maui-pr-devicetests, set prNumber to check out a specific PR branch before building. This enables testing any PR without merging to main. The checkout step fetches refs/pull/<N>/head and switches to it before the build starts. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot AI review requested due to automatic review settings April 7, 2026 18:29

Copilot started reviewing on behalf of kubaflo April 7, 2026 18:30 View session

github-actions bot mentioned this pull request Apr 7, 2026

[aw] Detect UI Test Categories failed #34857

Open

Copilot AI reviewed Apr 7, 2026

View reviewed changes

github-actions bot mentioned this pull request Apr 7, 2026

[aw] Detect Test Categories for Regression Detection failed #34858

Open

kubaflo and others added 2 commits April 7, 2026 22:17

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add gh-aw workflow for UI test category detection#34856

Add gh-aw workflow for UI test category detection#34856
kubaflo wants to merge 4 commits intomainfrom
feature/copilot-detect-categories

kubaflo commented Apr 7, 2026

Uh oh!

github-actions bot commented Apr 7, 2026 •

edited

Loading

Uh oh!

Copilot AI left a comment

Uh oh!

Copilot AI Apr 7, 2026

Uh oh!

Copilot AI Apr 7, 2026

Uh oh!

Copilot AI Apr 7, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

	\| grep -iE '(TestCases\.Shared\.Tests\|TestCases\.HostApp\|UITest\|Xaml\.UnitTests)' \
	\| grep -iE '(TestCases(\.[^/]+\.Tests)?\|TestCases\.HostApp\|UITest\|Xaml\.UnitTests)' \

	\| grep -iE '(TestCases\.Shared\.Tests\|TestCases\.HostApp\|UITest\|Xaml\.UnitTests)' \
	\| grep -iE '(TestCases\.(Android\|iOS\|Mac\|WinUI\|Shared)\.Tests\|TestCases\.HostApp\|UITest\|Xaml\.UnitTests)' \

Conversation

kubaflo commented Apr 7, 2026

Description

What it does

Triggers

Security model

Files added/modified

Relation to PR #33176

Testing

Uh oh!

github-actions bot commented Apr 7, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Copilot AI Apr 7, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Apr 7, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Apr 7, 2026

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

github-actions bot commented Apr 7, 2026 •

edited

Loading