Skip to content
Closed
Show file tree
Hide file tree
Changes from 9 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions .github/README-AI.md
Original file line number Diff line number Diff line change
Expand Up @@ -250,7 +250,7 @@ Reusable skills in `.github/skills/` that agents can invoke:
- **`verify-tests-fail-without-fix/`** - Verifies UI tests catch bugs (auto-detects mode based on git diff)
- **`write-ui-tests/`** - Creates UI tests for issues following MAUI conventions
- **`write-xaml-tests/`** - Creates XAML unit tests for parsing, XamlC, and source generation issues
- **`pr-build-status/`** - Retrieves Azure DevOps build status for PRs
- **`azdo-build-investigator/`** - Retrieves Azure DevOps build status for PRs

### Recent Improvements (January 2026)

Expand Down Expand Up @@ -365,7 +365,7 @@ For issues or questions about the AI agent instructions:

**Agent Files**:
- 4 agent files (pr.md, pr/post-gate.md, sandbox-agent.md, write-tests-agent.md)
- 5 skills (try-fix, verify-tests-fail-without-fix, write-ui-tests, write-xaml-tests, pr-build-status)
- 5 skills (try-fix, verify-tests-fail-without-fix, write-ui-tests, write-xaml-tests, azdo-build-investigator)
- All validated and consistent with consolidated structure

**Automation**:
Expand Down
8 changes: 8 additions & 0 deletions .github/copilot-instructions.md
Original file line number Diff line number Diff line change
Expand Up @@ -40,6 +40,14 @@ This guidance assumes:
- **iOS/macOS**: Xcode (current stable version)
- **Windows**: Windows SDK

### Azure DevOps CI Access

- **Azure CLI (`az`)** — preferred over `curl`/`Invoke-RestMethod` for CI queries
- Install: `brew install azure-cli` (macOS) / `winget install Microsoft.AzureCLI` (Windows)
- Setup: `az extension add --name azure-devops` (`az login` optional — `dnceng-public` is publicly accessible)
- Defaults: `az devops configure --defaults organization=https://dev.azure.com/dnceng-public project=public`
- For structured CI queries, use the `azdo-build-investigator` skill scripts in `.github/skills/azdo-build-investigator/`

## Project Structure

### Important Directories
Expand Down
165 changes: 165 additions & 0 deletions .github/skills/azdo-build-investigator/SKILL.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,165 @@
---
name: azdo-build-investigator
description: "Investigate CI failures for dotnet/maui PRs — build errors, Helix test logs, and binlog analysis. Use when asked about failing checks, CI status, test failures, 'why is CI red', 'build failed', 'what's failing on PR', Helix failures, or device test failures."
metadata:
author: dotnet-maui
version: "1.2"
compatibility: Requires GitHub CLI (gh) authenticated with access to dotnet/maui repository.
---

# PR Build Status Skill

Investigate CI failures for dotnet/maui PRs — build errors, Helix test logs, and binlog analysis.

## Tools Required

This skill uses `bash` together with `pwsh` (PowerShell 7+) to run the PowerShell scripts. No file editing or other tools are required.

**If `gh` or `pwsh` is missing: stop immediately and tell the user to install the missing tool. Do NOT attempt to install it yourself.**

- `gh`: https://cli.github.com/
- `pwsh`: https://aka.ms/install-powershell

Optional for binlog analysis (MSBuild failures):
- `az` (Azure CLI): `brew install azure-cli` / `winget install Microsoft.AzureCLI`, then `az extension add --name azure-devops`
- `binlogtool`: `dotnet tool install -g binlogtool` (https://www.nuget.org/packages/binlogtool)

## Scripts

All scripts are in `.github/skills/azdo-build-investigator/scripts/`

### 1. Get Build IDs for a PR
```bash
pwsh .github/skills/azdo-build-investigator/scripts/Get-PrBuildIds.ps1 -PrNumber <PR_NUMBER>
```

### 2. Get Build Status
```bash
pwsh .github/skills/azdo-build-investigator/scripts/Get-BuildInfo.ps1 -BuildId <BUILD_ID>
# For failed jobs only:
pwsh .github/skills/azdo-build-investigator/scripts/Get-BuildInfo.ps1 -BuildId <BUILD_ID> -FailedOnly
```

### 3. Get Build Errors and Test Failures
```bash
# Get all errors (build errors + test failures)
pwsh .github/skills/azdo-build-investigator/scripts/Get-BuildErrors.ps1 -BuildId <BUILD_ID>

# Get only build/compilation errors
pwsh .github/skills/azdo-build-investigator/scripts/Get-BuildErrors.ps1 -BuildId <BUILD_ID> -ErrorsOnly

# Get only test failures
pwsh .github/skills/azdo-build-investigator/scripts/Get-BuildErrors.ps1 -BuildId <BUILD_ID> -TestsOnly
```

### 4. Get Helix Console Logs
```bash
# List all Helix work items and their status
pwsh .github/skills/azdo-build-investigator/scripts/Get-HelixLogs.ps1 -BuildId <BUILD_ID>

# Filter by platform
pwsh .github/skills/azdo-build-investigator/scripts/Get-HelixLogs.ps1 -BuildId <BUILD_ID> -Platform Windows

# Show console log content for failed work items
pwsh .github/skills/azdo-build-investigator/scripts/Get-HelixLogs.ps1 -BuildId <BUILD_ID> -ShowConsoleLog

# Filter by work item name and show more log lines
pwsh .github/skills/azdo-build-investigator/scripts/Get-HelixLogs.ps1 -BuildId <BUILD_ID> -WorkItem "*Lifecycle*" -ShowConsoleLog -TailLines 200
```

## Workflow

> **Focus on the first error chronologically — later errors usually cascade from the root cause.**

**Multiple pipelines**: PRs trigger multiple builds. Investigate in priority order:
1. **`maui-pr`** (main build) — check first, most failures here
2. **`maui-pr-devicetests`** — if device test failures
3. **`maui-pr-uitests`** — if UI test failures

### Standard Build Failures
1. Get build IDs: `Get-PrBuildIds.ps1 -PrNumber XXXXX`
- If output shows ⚠️ with no build IDs, CI was not triggered — read the diagnostic message
2. For each build, get status: `Get-BuildInfo.ps1 -BuildId YYYYY -FailedOnly`
3. For failed builds, get errors: `Get-BuildErrors.ps1 -BuildId YYYYY`
4. If errors say "Build FAILED" with no detail, check for binlog artifacts (see below)

### Helix Test Failures
1. Get build IDs: `Get-PrBuildIds.ps1 -PrNumber XXXXX`
2. Find the build with Helix jobs (e.g., `maui-pr-devicetests`, `maui-integration-tests`)
3. Get Helix logs: `Get-HelixLogs.ps1 -BuildId YYYYY -ShowConsoleLog`
4. For specific platform: `Get-HelixLogs.ps1 -BuildId YYYYY -Platform Windows -ShowConsoleLog`

### Binlog Analysis (MSBuild/XamlC/NuGet failures)

**When to use binlog analysis**:
- ✅ `Get-BuildErrors` returns generic "Build FAILED" with no error messages
- ✅ Errors mention MSBuild, XamlC, or NuGet restore issues
- ✅ Error says "See binlog for details"
- ❌ Helix test failures (use `Get-HelixLogs` instead)
- ❌ Clear error messages already visible in build logs

`.binlog` artifacts contain the full MSBuild structured log.

**Requires `binlogtool`** (`dotnet tool install -g binlogtool`). If not installed, tell the user and stop.

```bash
# Download the binlog artifact
az pipelines runs artifact download --run-id BUILD_ID --artifact-name "binlog" --path /tmp/maui-binlog \
--org https://dev.azure.com/dnceng-public --project public --detect false

# Search for errors (broad first, then narrow)
binlogtool search "/tmp/maui-binlog/*.binlog" "error"
binlogtool search "/tmp/maui-binlog/*.binlog" "error CS" # C# compiler
binlogtool search "/tmp/maui-binlog/*.binlog" "error NU" # NuGet
binlogtool search "/tmp/maui-binlog/*.binlog" "XamlC" # XAML compiler
binlogtool search "/tmp/maui-binlog/*.binlog" "XA" # Android build errors

# Reconstruct full text log (useful when you need context around an error)
binlogtool reconstruct "/tmp/maui-binlog/*.binlog" > /tmp/maui-build.log

# Detect double-write errors (multiple tasks writing to the same output file)
binlogtool doublewrites "/tmp/maui-binlog/*.binlog"

# Clean up
rm -rf /tmp/maui-binlog
```

## Understanding Helix Logs

Helix is the .NET engineering infrastructure that runs tests across multiple platforms and device types. Tests that run on Helix include:
- **Device tests** - Run on real devices/emulators (iOS, Android, Windows, MacCatalyst)
- **Integration tests** - Run on various OS configurations
- **Other distributed tests** - Any test scenario that requires Helix infrastructure

When Helix tests fail:

1. **Build stage** - Compiles and packages the test app/harness
2. **Helix submission** - Sends the work items to Helix queues
3. **Work item execution** - Helix runs the tests on target machines/devices
4. **Console log** - Contains stdout/stderr from the test execution

The `Get-HelixLogs.ps1` script retrieves the console logs which show:
- Test execution output
- Any crashes or errors
- Infrastructure issues (timeouts, installation failures, etc.)

## Common Build Error Patterns

| Pattern | Area | Notes |
|---------|------|-------|
| `error CS####` | C# compiler | Root cause; check file/line reference |
| `error NU1###` | NuGet restore | NU1301 = feed unreachable; NU11## = resolution failure |
| `XamlC` | XAML compiler | MAUI-specific; usually missing type or invalid binding |
| `##[error]` | ADO infrastructure | Pipeline-level error, not a build error |
| `System.TimeoutException` | Test infra | Infrastructure timeout; may be transient |
| `error MT####` | iOS/Mac linker | Linking failure; check build logs |
| `error BL####` | Build logic | MSBuild task failure |

## Common Helix Failure Patterns

| Pattern in Console Log | Meaning |
|------------------------|---------|
| "XHarness timeout" | Test took too long, killed by infrastructure |
| "No test result files found" | Tests never ran or process crashed |
| "error MT..." or "error BL..." | Build/linking error (check build logs instead) |
| Exit code non-zero | Test failures or infrastructure issues |
Original file line number Diff line number Diff line change
Expand Up @@ -62,4 +62,23 @@ $builds = $checks | Where-Object { $_.link -match "dev\.azure\.com" } | ForEach-
}
} | Sort-Object -Property Pipeline, BuildId -Unique

$builds
# Diagnose the no-build case — common when CI is skipped or not yet triggered
$noBuildRows = $builds | Where-Object { -not $_.BuildId }
if ($noBuildRows) {
Write-Host ""
Write-Host "⚠️ Some pipelines have no build ID — CI was not triggered for these:" -ForegroundColor Yellow
foreach ($row in $noBuildRows) {
Write-Host " Pipeline : $($row.Pipeline)" -ForegroundColor Yellow
Write-Host " State : $($row.State)" -ForegroundColor Yellow
Write-Host " Link : $($row.Link)" -ForegroundColor Yellow
Write-Host ""
Write-Host " Likely causes:" -ForegroundColor Cyan
Write-Host " • PR only modifies path-excluded files (e.g. .github/**, docs/**)" -ForegroundColor Cyan
Write-Host " • Build has not been queued yet (maintainer trigger required)" -ForegroundColor Cyan
Write-Host " • PR is in draft state" -ForegroundColor Cyan
Write-Host " To check path filters: look for 'paths.exclude' in eng/pipelines/ci.yml" -ForegroundColor Cyan
Write-Host ""
}
}

$builds | Where-Object { $_.BuildId }
114 changes: 0 additions & 114 deletions .github/skills/pr-build-status/SKILL.md

This file was deleted.

2 changes: 1 addition & 1 deletion .github/skills/pr-finalize/SKILL.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
---
name: pr-finalize
description: Finalizes any PR for merge by verifying title/description match implementation AND performing code review for best practices. Use when asked to "finalize PR", "check PR description", "review commit message", before merging any PR, or when PR implementation changed during review. Do NOT use for extracting lessons (use learn-from-pr), writing tests (use write-tests-agent), or investigating build failures (use pr-build-status).
description: Finalizes any PR for merge by verifying title/description match implementation AND performing code review for best practices. Use when asked to "finalize PR", "check PR description", "review commit message", before merging any PR, or when PR implementation changed during review. Do NOT use for extracting lessons (use learn-from-pr), writing tests (use write-tests-agent), or investigating build failures (use azdo-build-investigator).
---

# PR Finalize
Expand Down
Loading