Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions .github/README-AI.md
Original file line number Diff line number Diff line change
Expand Up @@ -250,7 +250,7 @@ Reusable skills in `.github/skills/` that agents can invoke:
- **`verify-tests-fail-without-fix/`** - Verifies UI tests catch bugs (auto-detects mode based on git diff)
- **`write-ui-tests/`** - Creates UI tests for issues following MAUI conventions
- **`write-xaml-tests/`** - Creates XAML unit tests for parsing, XamlC, and source generation issues
- **`pr-build-status/`** - Retrieves Azure DevOps build status for PRs
- **`azdo-build-investigator/`** - MAUI-specific CI investigation context (works with `ci-analysis` from arcade-skills plugin)

### Recent Improvements (January 2026)

Expand Down Expand Up @@ -365,7 +365,7 @@ For issues or questions about the AI agent instructions:

**Agent Files**:
- 4 agent files (pr.md, pr/post-gate.md, sandbox-agent.md, write-tests-agent.md)
- 5 skills (try-fix, verify-tests-fail-without-fix, write-ui-tests, write-xaml-tests, pr-build-status)
- 5 skills (try-fix, verify-tests-fail-without-fix, write-ui-tests, write-xaml-tests, azdo-build-investigator)
- All validated and consistent with consolidated structure

**Automation**:
Expand Down
5 changes: 0 additions & 5 deletions .github/copilot-instructions.md
Original file line number Diff line number Diff line change
Expand Up @@ -312,11 +312,6 @@ Skills are modular capabilities that can be invoked directly or used by agents.
- **Two modes**: Verify failure only (test creation) or full verification (test + fix)
- **Used by**: After creating tests, before considering PR complete

8. **pr-build-status** (`.github/skills/pr-build-status/SKILL.md`)
- **Purpose**: Retrieves Azure DevOps build information for PRs (build IDs, stage status, failed jobs)
- **Trigger phrases**: "check build for PR #XXXXX", "why did PR build fail", "get build status"
- **Used by**: When investigating CI failures

8. **run-integration-tests** (`.github/skills/run-integration-tests/SKILL.md`)
- **Purpose**: Build, pack, and run .NET MAUI integration tests locally
- **Trigger phrases**: "run integration tests", "test templates locally", "run macOSTemplates tests", "run RunOniOS tests"
Expand Down
13 changes: 13 additions & 0 deletions .github/copilot/settings.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
{
"extraKnownMarketplaces": {
"dotnet-arcade-skills": {
"source": {
"source": "github",
"repo": "dotnet/arcade-skills"
}
}
},
"enabledPlugins": {
"dotnet-dnceng@dotnet-arcade-skills": true
}
}
71 changes: 71 additions & 0 deletions .github/skills/azdo-build-investigator/SKILL.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,71 @@
---
name: azdo-build-investigator
description: "Investigate CI failures for dotnet/maui PRs — build errors, Helix test logs, and binlog analysis. Use when asked about failing checks, CI status, test failures, 'why is CI red', 'build failed', 'what's failing on PR', Helix failures, or device test failures."
metadata:
author: dotnet-maui
version: "2.0"
---

# dotnet/maui CI Investigation Context

This skill provides MAUI-specific context for CI investigation. Use it together with the `ci-analysis` skill (loaded from the `dotnet-dnceng@dotnet-arcade-skills` plugin via `.github/copilot/settings.json`).

> **First**: invoke the `ci-analysis` skill — it handles the core investigation workflow using `Get-CIStatus.ps1` and `gh` CLI (with MCP tools as optional enhancements if available). This skill provides MAUI-specific corrections and context on top of that.

## Script Location

The `ci-analysis` skill and its `Get-CIStatus.ps1` script are loaded automatically from the `dotnet/arcade-skills` plugin (configured in `.github/copilot/settings.json` via `enabledPlugins`). The CLI caches scripts to `~/.copilot/installed-plugins/dotnet-arcade-skills/`. No manual download is needed.

## MAUI CI Pipelines

> ⚠️ The `ci-analysis` skill's reference doc lists `maui-public` as the MAUI pipeline — **this is outdated**. The correct pipeline names are below.

| Pipeline Name | Definition ID | Purpose |
|---------------|---------------|---------|
| `maui-pr` | **302** | Main build — check this first |
| `maui-pr-devicetests` | **314** | Helix device tests (iOS, Android, Windows, MacCatalyst) |
| `maui-pr-uitests` | **313** | Appium-based UI tests |

**Organization**: `dnceng-public` / project `public`

**Investigation priority order**: `maui-pr` → `maui-pr-devicetests` → `maui-pr-uitests`

Most failures are in `maui-pr`. Device test failures appear in `maui-pr-devicetests`. Focus on the first failing pipeline before checking others.

## MAUI-Specific Quirks

### XHarness Exit-0 Blind Spot

XHarness (used for iOS/Android device tests in `maui-pr-devicetests`) **exits with code 0 even when tests fail**. This means:
- The ADO job shows ✅ "Succeeded"
- `ci-analysis` may report no failures
- But actual test failures are hidden inside the Helix work items

**How to detect hidden test failures**: Query the `ResultSummaryByBuild` Helix API endpoint:
```
GET https://helix.dot.net/api/2019-06-17/jobs/{correlationId}/aggregated
```
Look for `Failed` > 0 in the response even when the ADO build job shows green.

When `ci-analysis` reports a `maui-pr-devicetests` build as passing but the PR has a `s/agent-gate-failed` label or the user suspects device test failures, always cross-check Helix `ResultSummaryByBuild`.
Comment on lines +44 to +50

Copilot AI Mar 11, 2026

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The Helix summary endpoint is shown with a {correlationId} placeholder and referred to as ResultSummaryByBuild, but elsewhere in this repo Helix identifiers are treated as a Helix jobId (GUID) extracted from logs (e.g., jobs/{guid}/workitems). To avoid confusion for users following this doc, rename the placeholder to {jobId} (or {helixJobId}) and clarify where to obtain it (ADO timeline logs / Helix submission output).

Copilot uses AI. Check for mistakes.

### Container Artifact Binlogs

MAUI build artifacts are **Container type**, not `PipelineArtifact`. This means:
- `az pipelines runs artifact download` does **not** work for binlogs
- Artifact names are like `Windows_NT_Build Windows (Debug)_Attempt1` (not `binlog`)
- Download requires a Bearer token from `az account get-access-token --resource 499b84ac-1321-427f-aa17-267ca6975798`
- Use the ADO File Container API: `/_apis/resources/Containers/{id}?api-version=5.0-preview&$format=OctetStream`

If available, use the `mcp-binlog-tool` MCP server to analyze downloaded `.binlog` files. This is optional — the core investigation workflow works without it via `gh` CLI and REST APIs.

## Common MAUI Failure Patterns

| Pattern | Where | Notes |
|---------|-------|-------|
| `error CS####` | `maui-pr` | C# compiler error — check file/line |
| `error XA####` | `maui-pr` | Android build error |
| `XamlC` | `maui-pr` | XAML compiler — usually missing type or bad binding |
| `XHarness timeout` | `maui-pr-devicetests` Helix logs | Test killed by infrastructure; may be transient |
| `No test result files found` | `maui-pr-devicetests` Helix logs | Tests never ran or app crashed on launch |
| UI test screenshot diff | `maui-pr-uitests` | Visual regression; check baseline images |
114 changes: 0 additions & 114 deletions .github/skills/pr-build-status/SKILL.md

This file was deleted.

Loading
Loading