diff --git a/.github/README-AI.md b/.github/README-AI.md index c703358dfade..843637a2988c 100644 --- a/.github/README-AI.md +++ b/.github/README-AI.md @@ -250,7 +250,7 @@ Reusable skills in `.github/skills/` that agents can invoke: - **`verify-tests-fail-without-fix/`** - Verifies UI tests catch bugs (auto-detects mode based on git diff) - **`write-ui-tests/`** - Creates UI tests for issues following MAUI conventions - **`write-xaml-tests/`** - Creates XAML unit tests for parsing, XamlC, and source generation issues -- **`pr-build-status/`** - Retrieves Azure DevOps build status for PRs +- **`azdo-build-investigator/`** - MAUI-specific CI investigation context (works with `ci-analysis` from arcade-skills plugin) ### Recent Improvements (January 2026) @@ -365,7 +365,7 @@ For issues or questions about the AI agent instructions: **Agent Files**: - 4 agent files (pr.md, pr/post-gate.md, sandbox-agent.md, write-tests-agent.md) -- 5 skills (try-fix, verify-tests-fail-without-fix, write-ui-tests, write-xaml-tests, pr-build-status) +- 5 skills (try-fix, verify-tests-fail-without-fix, write-ui-tests, write-xaml-tests, azdo-build-investigator) - All validated and consistent with consolidated structure **Automation**: diff --git a/.github/copilot-instructions.md b/.github/copilot-instructions.md index d0d99c2b8373..fb99bc2f87f5 100644 --- a/.github/copilot-instructions.md +++ b/.github/copilot-instructions.md @@ -312,11 +312,6 @@ Skills are modular capabilities that can be invoked directly or used by agents. - **Two modes**: Verify failure only (test creation) or full verification (test + fix) - **Used by**: After creating tests, before considering PR complete -8. **pr-build-status** (`.github/skills/pr-build-status/SKILL.md`) - - **Purpose**: Retrieves Azure DevOps build information for PRs (build IDs, stage status, failed jobs) - - **Trigger phrases**: "check build for PR #XXXXX", "why did PR build fail", "get build status" - - **Used by**: When investigating CI failures - 8. **run-integration-tests** (`.github/skills/run-integration-tests/SKILL.md`) - **Purpose**: Build, pack, and run .NET MAUI integration tests locally - **Trigger phrases**: "run integration tests", "test templates locally", "run macOSTemplates tests", "run RunOniOS tests" diff --git a/.github/copilot/settings.json b/.github/copilot/settings.json new file mode 100644 index 000000000000..35358294c16e --- /dev/null +++ b/.github/copilot/settings.json @@ -0,0 +1,13 @@ +{ + "extraKnownMarketplaces": { + "dotnet-arcade-skills": { + "source": { + "source": "github", + "repo": "dotnet/arcade-skills" + } + } + }, + "enabledPlugins": { + "dotnet-dnceng@dotnet-arcade-skills": true + } +} diff --git a/.github/skills/azdo-build-investigator/SKILL.md b/.github/skills/azdo-build-investigator/SKILL.md new file mode 100644 index 000000000000..bd3ddc905332 --- /dev/null +++ b/.github/skills/azdo-build-investigator/SKILL.md @@ -0,0 +1,71 @@ +--- +name: azdo-build-investigator +description: "Investigate CI failures for dotnet/maui PRs — build errors, Helix test logs, and binlog analysis. Use when asked about failing checks, CI status, test failures, 'why is CI red', 'build failed', 'what's failing on PR', Helix failures, or device test failures." +metadata: + author: dotnet-maui + version: "2.0" +--- + +# dotnet/maui CI Investigation Context + +This skill provides MAUI-specific context for CI investigation. Use it together with the `ci-analysis` skill (loaded from the `dotnet-dnceng@dotnet-arcade-skills` plugin via `.github/copilot/settings.json`). + +> **First**: invoke the `ci-analysis` skill — it handles the core investigation workflow using `Get-CIStatus.ps1` and `gh` CLI (with MCP tools as optional enhancements if available). This skill provides MAUI-specific corrections and context on top of that. + +## Script Location + +The `ci-analysis` skill and its `Get-CIStatus.ps1` script are loaded automatically from the `dotnet/arcade-skills` plugin (configured in `.github/copilot/settings.json` via `enabledPlugins`). The CLI caches scripts to `~/.copilot/installed-plugins/dotnet-arcade-skills/`. No manual download is needed. + +## MAUI CI Pipelines + +> ⚠️ The `ci-analysis` skill's reference doc lists `maui-public` as the MAUI pipeline — **this is outdated**. The correct pipeline names are below. + +| Pipeline Name | Definition ID | Purpose | +|---------------|---------------|---------| +| `maui-pr` | **302** | Main build — check this first | +| `maui-pr-devicetests` | **314** | Helix device tests (iOS, Android, Windows, MacCatalyst) | +| `maui-pr-uitests` | **313** | Appium-based UI tests | + +**Organization**: `dnceng-public` / project `public` + +**Investigation priority order**: `maui-pr` → `maui-pr-devicetests` → `maui-pr-uitests` + +Most failures are in `maui-pr`. Device test failures appear in `maui-pr-devicetests`. Focus on the first failing pipeline before checking others. + +## MAUI-Specific Quirks + +### XHarness Exit-0 Blind Spot + +XHarness (used for iOS/Android device tests in `maui-pr-devicetests`) **exits with code 0 even when tests fail**. This means: +- The ADO job shows ✅ "Succeeded" +- `ci-analysis` may report no failures +- But actual test failures are hidden inside the Helix work items + +**How to detect hidden test failures**: Query the `ResultSummaryByBuild` Helix API endpoint: +``` +GET https://helix.dot.net/api/2019-06-17/jobs/{correlationId}/aggregated +``` +Look for `Failed` > 0 in the response even when the ADO build job shows green. + +When `ci-analysis` reports a `maui-pr-devicetests` build as passing but the PR has a `s/agent-gate-failed` label or the user suspects device test failures, always cross-check Helix `ResultSummaryByBuild`. + +### Container Artifact Binlogs + +MAUI build artifacts are **Container type**, not `PipelineArtifact`. This means: +- `az pipelines runs artifact download` does **not** work for binlogs +- Artifact names are like `Windows_NT_Build Windows (Debug)_Attempt1` (not `binlog`) +- Download requires a Bearer token from `az account get-access-token --resource 499b84ac-1321-427f-aa17-267ca6975798` +- Use the ADO File Container API: `/_apis/resources/Containers/{id}?api-version=5.0-preview&$format=OctetStream` + +If available, use the `mcp-binlog-tool` MCP server to analyze downloaded `.binlog` files. This is optional — the core investigation workflow works without it via `gh` CLI and REST APIs. + +## Common MAUI Failure Patterns + +| Pattern | Where | Notes | +|---------|-------|-------| +| `error CS####` | `maui-pr` | C# compiler error — check file/line | +| `error XA####` | `maui-pr` | Android build error | +| `XamlC` | `maui-pr` | XAML compiler — usually missing type or bad binding | +| `XHarness timeout` | `maui-pr-devicetests` Helix logs | Test killed by infrastructure; may be transient | +| `No test result files found` | `maui-pr-devicetests` Helix logs | Tests never ran or app crashed on launch | +| UI test screenshot diff | `maui-pr-uitests` | Visual regression; check baseline images | diff --git a/.github/skills/pr-build-status/SKILL.md b/.github/skills/pr-build-status/SKILL.md deleted file mode 100644 index c4a542b8b9a3..000000000000 --- a/.github/skills/pr-build-status/SKILL.md +++ /dev/null @@ -1,114 +0,0 @@ ---- -name: pr-build-status -description: "Retrieve Azure DevOps build information for GitHub Pull Requests, including build IDs, stage status, failed jobs, and Helix console logs for any Helix-based test failures." -metadata: - author: dotnet-maui - version: "1.1" -compatibility: Requires GitHub CLI (gh) authenticated with access to dotnet/maui repository. ---- - -# PR Build Status Skill - -Retrieve Azure DevOps build information for GitHub Pull Requests, including Helix test logs. - -## Tools Required - -This skill uses `bash` together with `pwsh` (PowerShell 7+) to run the PowerShell scripts. No file editing or other tools are required. - -## When to Use - -- User asks about CI/CD status for a PR -- User asks about failed checks or builds -- User asks "what's failing on PR #XXXXX" -- User wants to see test results -- **User asks about Helix failures (device tests, integration tests, etc.)** -- **User needs to debug why tests are failing on Helix infrastructure** - -## Scripts - -All scripts are in `.github/skills/pr-build-status/scripts/` - -### 1. Get Build IDs for a PR -```bash -pwsh .github/skills/pr-build-status/scripts/Get-PrBuildIds.ps1 -PrNumber -``` - -### 2. Get Build Status -```bash -pwsh .github/skills/pr-build-status/scripts/Get-BuildInfo.ps1 -BuildId -# For failed jobs only: -pwsh .github/skills/pr-build-status/scripts/Get-BuildInfo.ps1 -BuildId -FailedOnly -``` - -### 3. Get Build Errors and Test Failures -```bash -# Get all errors (build errors + test failures) -pwsh .github/skills/pr-build-status/scripts/Get-BuildErrors.ps1 -BuildId - -# Get only build/compilation errors -pwsh .github/skills/pr-build-status/scripts/Get-BuildErrors.ps1 -BuildId -ErrorsOnly - -# Get only test failures -pwsh .github/skills/pr-build-status/scripts/Get-BuildErrors.ps1 -BuildId -TestsOnly -``` - -### 4. Get Helix Console Logs -```bash -# List all Helix work items and their status -pwsh .github/skills/pr-build-status/scripts/Get-HelixLogs.ps1 -BuildId - -# Filter by platform -pwsh .github/skills/pr-build-status/scripts/Get-HelixLogs.ps1 -BuildId -Platform Windows - -# Show console log content for failed work items -pwsh .github/skills/pr-build-status/scripts/Get-HelixLogs.ps1 -BuildId -ShowConsoleLog - -# Filter by work item name and show more log lines -pwsh .github/skills/pr-build-status/scripts/Get-HelixLogs.ps1 -BuildId -WorkItem "*Lifecycle*" -ShowConsoleLog -TailLines 200 -``` - -## Workflow - -### Standard Build Failures -1. Get build IDs: `Get-PrBuildIds.ps1 -PrNumber XXXXX` -2. For each build, get status: `Get-BuildInfo.ps1 -BuildId YYYYY -FailedOnly` -3. For failed builds, get errors: `Get-BuildErrors.ps1 -BuildId YYYYY` - -### Helix Test Failures -1. Get build IDs: `Get-PrBuildIds.ps1 -PrNumber XXXXX` -2. Find the build with Helix jobs (e.g., `maui-pr-devicetests`, `maui-integration-tests`) -3. Get Helix logs: `Get-HelixLogs.ps1 -BuildId YYYYY -ShowConsoleLog` -4. For specific platform: `Get-HelixLogs.ps1 -BuildId YYYYY -Platform Windows -ShowConsoleLog` - -## Understanding Helix Logs - -Helix is the .NET engineering infrastructure that runs tests across multiple platforms and device types. Tests that run on Helix include: -- **Device tests** - Run on real devices/emulators (iOS, Android, Windows, MacCatalyst) -- **Integration tests** - Run on various OS configurations -- **Other distributed tests** - Any test scenario that requires Helix infrastructure - -When Helix tests fail: - -1. **Build stage** - Compiles and packages the test app/harness -2. **Helix submission** - Sends the work items to Helix queues -3. **Work item execution** - Helix runs the tests on target machines/devices -4. **Console log** - Contains stdout/stderr from the test execution - -The `Get-HelixLogs.ps1` script retrieves the console logs which show: -- Test execution output -- Any crashes or errors -- Infrastructure issues (timeouts, installation failures, etc.) - -## Common Helix Failure Patterns - -| Pattern in Console Log | Meaning | -|------------------------|---------| -| "XHarness timeout" | Test took too long, killed by infrastructure | -| "No test result files found" | Tests never ran or process crashed | -| "error MT..." or "error BL..." | Build/linking error (check build logs instead) | -| Exit code non-zero | Test failures or infrastructure issues | - -## Prerequisites - -- `gh` (GitHub CLI) - authenticated -- `pwsh` (PowerShell 7+) diff --git a/.github/skills/pr-build-status/scripts/Get-BuildErrors.ps1 b/.github/skills/pr-build-status/scripts/Get-BuildErrors.ps1 deleted file mode 100644 index 2ad34c5e521e..000000000000 --- a/.github/skills/pr-build-status/scripts/Get-BuildErrors.ps1 +++ /dev/null @@ -1,208 +0,0 @@ -<# -.SYNOPSIS - Retrieves build errors and test failures from an Azure DevOps build. - -.DESCRIPTION - Queries the Azure DevOps build timeline to find failed jobs and tasks, - then extracts build errors (MSBuild errors, compilation failures) and - test failures with their details. - -.PARAMETER BuildId - The Azure DevOps build ID. - -.PARAMETER Org - The Azure DevOps organization. Defaults to 'dnceng-public'. - -.PARAMETER Project - The Azure DevOps project. Defaults to 'public'. - -.PARAMETER TestsOnly - If specified, only returns test results (no build errors). - -.PARAMETER ErrorsOnly - If specified, only returns build errors (no test results). - -.PARAMETER JobFilter - Optional filter to match job/task names (supports wildcards). - -.EXAMPLE - ./Get-BuildErrors.ps1 -BuildId 1240456 - -.EXAMPLE - ./Get-BuildErrors.ps1 -BuildId 1240456 -ErrorsOnly - -.EXAMPLE - ./Get-BuildErrors.ps1 -BuildId 1240456 -TestsOnly -JobFilter "*SafeArea*" - -.OUTPUTS - Objects with Type (BuildError/TestFailure), Source, Message, and Details properties. -#> - -[CmdletBinding()] -param( - [Parameter(Mandatory = $true, Position = 0)] - [string]$BuildId, - - [Parameter(Mandatory = $false)] - [string]$Org = "dnceng-public", - - [Parameter(Mandatory = $false)] - [string]$Project = "public", - - [Parameter(Mandatory = $false)] - [switch]$TestsOnly, - - [Parameter(Mandatory = $false)] - [switch]$ErrorsOnly, - - [Parameter(Mandatory = $false)] - [string]$JobFilter -) - -$ErrorActionPreference = "Stop" - -# Get build timeline -$timelineUrl = "https://dev.azure.com/$Org/$Project/_apis/build/builds/${BuildId}/timeline?api-version=7.0" - -try { - $timeline = Invoke-RestMethod -Uri $timelineUrl -Method Get -ContentType "application/json" -} -catch { - Write-Error "Failed to query Azure DevOps timeline API: $_" - exit 1 -} - -$allResults = @() - -# --- SECTION 1: Find Build Errors from Failed Tasks --- -if (-not $TestsOnly) { - $failedTasks = $timeline.records | Where-Object { - $_.type -eq "Task" -and - $_.result -eq "failed" -and - $_.log.url -and - (-not $JobFilter -or $_.name -like $JobFilter) - } - - foreach ($task in $failedTasks) { - Write-Host "Analyzing failed task: $($task.name)" -ForegroundColor Red - - try { - $log = Invoke-RestMethod -Uri $task.log.url -Method Get - $lines = $log -split "`n" - - # Find MSBuild errors and ##[error] markers - $errorLines = $lines | Where-Object { - $_ -match ": error [A-Z]+\d*:" -or # MSBuild errors (CS1234, MT1234, etc.) - $_ -match ": Error :" -or # Xamarin.Shared.Sdk errors - $_ -match "##\[error\]" # Azure DevOps error markers - } - - foreach ($errorLine in $errorLines) { - # Clean up the line - $cleanLine = $errorLine -replace "^\d{4}-\d{2}-\d{2}T[\d:.]+Z\s*", "" - $cleanLine = $cleanLine -replace "##\[error\]", "" - - # Skip generic "exited with code" errors - we want the actual error - if ($cleanLine -match "exited with code") { - continue - } - - $allResults += [PSCustomObject]@{ - Type = "BuildError" - Source = $task.name - Message = $cleanLine.Trim() - Details = "" - } - } - } - catch { - Write-Warning "Failed to fetch log for task $($task.name): $_" - } - } -} - -# --- SECTION 2: Find Test Failures from Jobs --- -if (-not $ErrorsOnly) { - $jobs = $timeline.records | Where-Object { - $_.type -eq "Job" -and - $_.log.url -and - $_.state -eq "completed" -and - $_.result -eq "failed" -and - (-not $JobFilter -or $_.name -like $JobFilter) - } - - foreach ($job in $jobs) { - Write-Host "Analyzing job for test failures: $($job.name)" -ForegroundColor Yellow - - try { - $logContent = Invoke-RestMethod -Uri $job.log.url -Method Get - $lines = $logContent -split "`n" - - # Find test result lines: "Failed [duration]" - for ($i = 0; $i -lt $lines.Count; $i++) { - if ($lines[$i] -match "^\d{4}-\d{2}-\d{2}.*\s+Failed\s+(\S+)\s+\[([^\]]+)\]") { - $testName = $matches[1] - $duration = $matches[2] - - $errorMessage = "" - $stackTrace = "" - - # Look ahead for error message and stack trace - for ($j = $i + 1; $j -lt $lines.Count; $j++) { - $line = $lines[$j] - $cleanLine = $line -replace "^\d{4}-\d{2}-\d{2}T[\d:.]+Z\s*", "" - - if ($cleanLine -match "^\s*Error Message:") { - for ($k = $j + 1; $k -lt [Math]::Min($j + 10, $lines.Count); $k++) { - $msgLine = $lines[$k] -replace "^\d{4}-\d{2}-\d{2}T[\d:.]+Z\s*", "" - if ($msgLine -match "^\s*Stack Trace:" -or [string]::IsNullOrWhiteSpace($msgLine)) { - break - } - $errorMessage += $msgLine.Trim() + " " - } - } - - if ($cleanLine -match "^\s*Stack Trace:") { - for ($k = $j + 1; $k -lt [Math]::Min($j + 5, $lines.Count); $k++) { - $stLine = $lines[$k] -replace "^\d{4}-\d{2}-\d{2}T[\d:.]+Z\s*", "" - if ($stLine -match "at .+ in .+:line \d+") { - $stackTrace = $stLine.Trim() - break - } - } - break - } - - # Stop if we hit the next test - if ($cleanLine -match "^\s*(Passed|Failed|Skipped)\s+\S+\s+\[") { - break - } - } - - $allResults += [PSCustomObject]@{ - Type = "TestFailure" - Source = $job.name - Message = $testName - Details = if ($errorMessage) { "$errorMessage`n$stackTrace".Trim() } else { $stackTrace } - } - } - } - } - catch { - Write-Warning "Failed to fetch log for job $($job.name): $_" - } - } -} - -# Remove duplicate errors (same message from same source) -$uniqueResults = $allResults | Group-Object -Property Type, Source, Message | ForEach-Object { - $_.Group | Select-Object -First 1 -} - -# Summary -$buildErrors = ($uniqueResults | Where-Object { $_.Type -eq "BuildError" }).Count -$testFailures = ($uniqueResults | Where-Object { $_.Type -eq "TestFailure" }).Count - -Write-Host "`nSummary: $buildErrors build error(s), $testFailures test failure(s)" -ForegroundColor Cyan - -$uniqueResults diff --git a/.github/skills/pr-build-status/scripts/Get-BuildInfo.ps1 b/.github/skills/pr-build-status/scripts/Get-BuildInfo.ps1 deleted file mode 100644 index 12fbc89a5541..000000000000 --- a/.github/skills/pr-build-status/scripts/Get-BuildInfo.ps1 +++ /dev/null @@ -1,104 +0,0 @@ -<# -.SYNOPSIS - Retrieves detailed status information for an Azure DevOps build. - -.DESCRIPTION - Queries the Azure DevOps build timeline API and returns comprehensive - information about the build including all stages, their status, and - any failed or canceled jobs. - -.PARAMETER BuildId - The Azure DevOps build ID. - -.PARAMETER Org - The Azure DevOps organization. Defaults to 'dnceng-public'. - -.PARAMETER Project - The Azure DevOps project. Defaults to 'public'. - -.PARAMETER FailedOnly - If specified, only returns failed or canceled stages and jobs. - -.EXAMPLE - ./Get-BuildInfo.ps1 -BuildId 1240455 - -.EXAMPLE - ./Get-BuildInfo.ps1 -BuildId 1240455 -FailedOnly - -.EXAMPLE - ./Get-BuildInfo.ps1 -BuildId 1240455 -Org "dnceng-public" -Project "public" - -.OUTPUTS - Object with BuildId, Status, Result, Stages, and FailedJobs properties. -#> - -[CmdletBinding()] -param( - [Parameter(Mandatory = $true, Position = 0)] - [string]$BuildId, - - [Parameter(Mandatory = $false)] - [string]$Org = "dnceng-public", - - [Parameter(Mandatory = $false)] - [string]$Project = "public", - - [Parameter(Mandatory = $false)] - [switch]$FailedOnly -) - -$ErrorActionPreference = "Stop" - -# Get build info -$buildUrl = "https://dev.azure.com/$Org/$Project/_apis/build/builds/${BuildId}?api-version=7.0" -$timelineUrl = "https://dev.azure.com/$Org/$Project/_apis/build/builds/$BuildId/timeline?api-version=7.0" - -try { - $build = Invoke-RestMethod -Uri $buildUrl -Method Get -ContentType "application/json" - $timeline = Invoke-RestMethod -Uri $timelineUrl -Method Get -ContentType "application/json" -} -catch { - Write-Error "Failed to query Azure DevOps API: $_" - exit 1 -} - -# Extract stages -$stages = $timeline.records | Where-Object { $_.type -eq "Stage" } | ForEach-Object { - [PSCustomObject]@{ - Name = $_.name - State = $_.state - Result = $_.result - } -} | Sort-Object -Property { $_.State -eq "completed" }, { $_.State -eq "inProgress" } - -# Extract failed/canceled jobs -$failedJobs = $timeline.records | - Where-Object { - ($_.type -eq "Stage" -or $_.type -eq "Job") -and - ($_.result -eq "failed" -or $_.result -eq "canceled") - } | - ForEach-Object { - [PSCustomObject]@{ - Name = $_.name - Type = $_.type - Result = $_.result - } - } | Sort-Object -Property Type, Name - -if ($FailedOnly) { - $failedJobs -} -else { - [PSCustomObject]@{ - BuildId = $BuildId - BuildNumber = $build.buildNumber - Status = $build.status - Result = $build.result - Pipeline = $build.definition.name - StartTime = $build.startTime - FinishTime = $build.finishTime - Stages = $stages - FailedJobs = $failedJobs - Link = "https://dev.azure.com/$Org/$Project/_build/results?buildId=$BuildId" - } -} diff --git a/.github/skills/pr-build-status/scripts/Get-HelixLogs.ps1 b/.github/skills/pr-build-status/scripts/Get-HelixLogs.ps1 deleted file mode 100644 index 64a2801aabeb..000000000000 --- a/.github/skills/pr-build-status/scripts/Get-HelixLogs.ps1 +++ /dev/null @@ -1,247 +0,0 @@ -<# -.SYNOPSIS - Retrieves Helix console logs for failed work items from an Azure DevOps build. - -.DESCRIPTION - Parses Azure DevOps build logs to extract Helix job IDs, then queries the Helix API - to get console logs for failed work items. This is useful for debugging any test - failures that run on Helix infrastructure (device tests, integration tests, etc.). - -.PARAMETER BuildId - The Azure DevOps build ID. - -.PARAMETER Org - The Azure DevOps organization. Defaults to 'dnceng-public'. - -.PARAMETER Project - The Azure DevOps project. Defaults to 'public'. - -.PARAMETER Platform - Optional filter for platform (e.g., 'Windows', 'iOS', 'Android', 'MacCatalyst'). - -.PARAMETER WorkItem - Optional filter for specific work item name (supports wildcards). - -.PARAMETER ShowConsoleLog - If specified, displays the full console log content for each failed work item. - -.PARAMETER TailLines - Number of lines to show from the end of console logs. Default is 100. - -.EXAMPLE - ./Get-HelixLogs.ps1 -BuildId 1255952 - -.EXAMPLE - ./Get-HelixLogs.ps1 -BuildId 1255952 -Platform Windows -ShowConsoleLog - -.EXAMPLE - ./Get-HelixLogs.ps1 -BuildId 1255952 -WorkItem "*Lifecycle*" -ShowConsoleLog -TailLines 200 - -.OUTPUTS - Objects with JobId, WorkItem, Queue, ConsoleUrl, and optionally ConsoleLog properties. -#> - -[CmdletBinding()] -param( - [Parameter(Mandatory = $true, Position = 0)] - [string]$BuildId, - - [Parameter(Mandatory = $false)] - [string]$Org = "dnceng-public", - - [Parameter(Mandatory = $false)] - [string]$Project = "public", - - [Parameter(Mandatory = $false)] - [string]$Platform, - - [Parameter(Mandatory = $false)] - [string]$WorkItem, - - [Parameter(Mandatory = $false)] - [switch]$ShowConsoleLog, - - [Parameter(Mandatory = $false)] - [int]$TailLines = 100 -) - -$ErrorActionPreference = "Stop" - -# Get build timeline -$timelineUrl = "https://dev.azure.com/$Org/$Project/_apis/build/builds/${BuildId}/timeline?api-version=7.0" - -try { - $timeline = Invoke-RestMethod -Uri $timelineUrl -Method Get -ContentType "application/json" -} -catch { - Write-Error "Failed to query Azure DevOps timeline API: $_" - exit 1 -} - -# Build platform filter pattern -$platformPattern = if ($Platform) { - switch ($Platform.ToLower()) { - "windows" { "*Windows*" } - "ios" { "*iOS*" } - "android" { "*Android*" } - "maccatalyst" { "*MacCatalyst*|*Catalyst*" } - default { "*$Platform*" } - } -} else { - "*" -} - -# Find Helix-related tasks with logs (looking for DeviceTests tasks that submit to Helix) -$helixTasks = $timeline.records | Where-Object { - $_.name -like "*DeviceTests*" -and - $_.log.url -and - ($_.result -eq "failed" -or $_.result -eq "succeeded") -and - (-not $Platform -or $_.name -like $platformPattern) -} - -if (-not $helixTasks) { - Write-Host "No Helix-related tasks found in build $BuildId" -ForegroundColor Yellow - exit 0 -} - -$allHelixJobs = @{} -$allResults = @() - -foreach ($task in $helixTasks) { - Write-Host "Scanning task: $($task.name) [$($task.result)]" -ForegroundColor $(if ($task.result -eq "failed") { "Red" } else { "Gray" }) - - try { - $logContent = Invoke-RestMethod -Uri $task.log.url -Method Get - - # Extract Helix job IDs from log (pattern: "jobs/{guid}/workitems") - $jobMatches = [regex]::Matches($logContent, "jobs/([a-f0-9-]{36})/workitems") - - foreach ($match in $jobMatches) { - $jobId = $match.Groups[1].Value - if (-not $allHelixJobs.ContainsKey($jobId)) { - $allHelixJobs[$jobId] = @{ - Task = $task.name - Result = $task.result - } - } - } - } - catch { - Write-Warning "Failed to fetch log for task $($task.name): $_" - } -} - -if ($allHelixJobs.Count -eq 0) { - Write-Host "No Helix job IDs found in build logs" -ForegroundColor Yellow - exit 0 -} - -Write-Host "`nFound $($allHelixJobs.Count) Helix job(s)" -ForegroundColor Cyan - -# Query each Helix job for work items -foreach ($jobId in $allHelixJobs.Keys) { - $jobInfo = $allHelixJobs[$jobId] - Write-Host "`n--- Helix Job: $jobId ---" -ForegroundColor Yellow - Write-Host "From task: $($jobInfo.Task)" -ForegroundColor Gray - - try { - # Get job details for queue info - $jobDetailsUrl = "https://helix.dot.net/api/jobs/${jobId}/details?api-version=2019-06-17" - $jobDetails = Invoke-RestMethod -Uri $jobDetailsUrl -Method Get - $queue = $jobDetails.QueueId - - Write-Host "Queue: $queue" -ForegroundColor Gray - - # Get work items - $workItemsUrl = "https://helix.dot.net/api/jobs/${jobId}/workitems?api-version=2019-06-17" - $workItems = Invoke-RestMethod -Uri $workItemsUrl -Method Get - - foreach ($wi in $workItems) { - # Skip the controller work item - if ($wi.Name -eq "HelixController Work Queueing") { - continue - } - - # Apply work item filter if specified - if ($WorkItem -and $wi.Name -notlike $WorkItem) { - continue - } - - $consoleUrl = "https://helix.dot.net/api/2019-06-17/jobs/${jobId}/workitems/$($wi.Name)/console" - - # Determine if this is a failure - # Check: 1) ExitCode non-zero, 2) Parent task failed - $isFailed = $false - if ($wi.ExitCode -and $wi.ExitCode -ne 0) { - $isFailed = $true - } - elseif ($jobInfo.Result -eq "failed") { - $isFailed = $true - } - - # For work items from failed tasks, always try to get console log to check for errors - $shouldFetchLog = $ShowConsoleLog -and ($isFailed -or $jobInfo.Result -eq "failed") - - $consoleLogContent = $null - if ($shouldFetchLog) { - Write-Host "`nFetching console log for: $($wi.Name)" -ForegroundColor Cyan - - try { - $consoleLog = Invoke-RestMethod -Uri $consoleUrl -Method Get - - # Check if the log indicates a failure (even if ExitCode wasn't set) - if ($consoleLog -match "exited with (code )?1\]|ERROR:|FAILED|exception|crash") { - $isFailed = $true - } - - # Get tail lines - $lines = $consoleLog -split "`n" - if ($TailLines -gt 0 -and $lines.Count -gt $TailLines) { - $consoleLogContent = ($lines | Select-Object -Last $TailLines) -join "`n" - Write-Host "... (showing last $TailLines of $($lines.Count) lines)" -ForegroundColor Gray - } - else { - $consoleLogContent = $consoleLog - } - - Write-Host $consoleLogContent - } - catch { - Write-Warning "Failed to fetch console log: $_" - } - } - - $result = [PSCustomObject]@{ - JobId = $jobId - WorkItem = $wi.Name - State = $wi.State - ExitCode = $wi.ExitCode - Queue = $queue - IsFailed = $isFailed - ConsoleUrl = $consoleUrl - ConsoleLog = $consoleLogContent - } - - $allResults += $result - - # Print summary line - $statusColor = if ($isFailed) { "Red" } else { "Green" } - $statusSymbol = if ($isFailed) { "X" } else { "√" } - Write-Host " [$statusSymbol] $($wi.Name) (Exit: $($wi.ExitCode))" -ForegroundColor $statusColor - } - } - catch { - Write-Warning "Failed to query Helix job $jobId`: $_" - } -} - -# Summary -$failedCount = ($allResults | Where-Object { $_.IsFailed }).Count -$totalCount = $allResults.Count - -Write-Host "`n========================================" -ForegroundColor Cyan -Write-Host "Summary: $failedCount failed / $totalCount total work items" -ForegroundColor $(if ($failedCount -gt 0) { "Red" } else { "Green" }) -Write-Host "========================================" -ForegroundColor Cyan - -# Output results -$allResults diff --git a/.github/skills/pr-build-status/scripts/Get-PrBuildIds.ps1 b/.github/skills/pr-build-status/scripts/Get-PrBuildIds.ps1 deleted file mode 100644 index 8df3966a3577..000000000000 --- a/.github/skills/pr-build-status/scripts/Get-PrBuildIds.ps1 +++ /dev/null @@ -1,65 +0,0 @@ -<# -.SYNOPSIS - Retrieves Azure DevOps build IDs associated with a GitHub PR. - -.DESCRIPTION - Queries GitHub PR checks and extracts the Azure DevOps build IDs, - pipeline names, states, and links for each unique build. - -.PARAMETER PrNumber - The GitHub Pull Request number. - -.PARAMETER Repo - The GitHub repository in 'owner/repo' format. Defaults to 'dotnet/maui'. - -.EXAMPLE - ./Get-PrBuildIds.ps1 -PrNumber 33251 - -.EXAMPLE - ./Get-PrBuildIds.ps1 -PrNumber 33251 -Repo "dotnet/maui" - -.OUTPUTS - Array of objects with Pipeline, BuildId, State, and Link properties. -#> - -[CmdletBinding()] -param( - [Parameter(Mandatory = $true, Position = 0)] - [int]$PrNumber, - - [Parameter(Mandatory = $false)] - [string]$Repo = "dotnet/maui" -) - -$ErrorActionPreference = "Stop" - -# Validate prerequisites -if (-not (Get-Command "gh" -ErrorAction SilentlyContinue)) { - Write-Error "GitHub CLI (gh) is not installed. Install from https://cli.github.com/" - exit 1 -} - -# Get PR checks from GitHub -$checksJson = gh pr checks $PrNumber --repo $Repo --json name,link,state 2>&1 - -if ($LASTEXITCODE -ne 0) { - Write-Error "Failed to get PR checks: $checksJson" - exit 1 -} - -$checks = $checksJson | ConvertFrom-Json - -# Filter to Azure DevOps checks and extract build IDs -$builds = $checks | Where-Object { $_.link -match "dev\.azure\.com" } | ForEach-Object { - $buildId = if ($_.link -match "buildId=(\d+)") { $matches[1] } else { $null } - $pipeline = ($_.name -split " ")[0] - - [PSCustomObject]@{ - Pipeline = $pipeline - BuildId = $buildId - State = $_.state - Link = $_.link - } -} | Sort-Object -Property Pipeline, BuildId -Unique - -$builds diff --git a/.github/skills/pr-finalize/SKILL.md b/.github/skills/pr-finalize/SKILL.md index c1c8af7144de..d2efd2d25ebf 100644 --- a/.github/skills/pr-finalize/SKILL.md +++ b/.github/skills/pr-finalize/SKILL.md @@ -1,6 +1,6 @@ --- name: pr-finalize -description: Finalizes any PR for merge by verifying title/description match implementation AND performing code review for best practices. Use when asked to "finalize PR", "check PR description", "review commit message", before merging any PR, or when PR implementation changed during review. Do NOT use for extracting lessons (use learn-from-pr), writing tests (use write-tests-agent), or investigating build failures (use pr-build-status). +description: Finalizes any PR for merge by verifying title/description match implementation AND performing code review for best practices. Use when asked to "finalize PR", "check PR description", "review commit message", before merging any PR, or when PR implementation changed during review. Do NOT use for extracting lessons (use learn-from-pr), writing tests (use write-tests-agent), or investigating build failures (use azdo-build-investigator and ci-analysis). --- # PR Finalize