Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
47 commits
Select commit Hold shift + click to select a range
f15d0d1
try build tests with coreclr
rmarinho Dec 18, 2025
10ed1a2
Remove iOS
rmarinho Dec 18, 2025
ad1bdce
Dont include iOS tf
rmarinho Dec 18, 2025
3307c71
Small fixes
rmarinho Dec 18, 2025
e1405bc
[ci] try build tests with coreclr (#33212)
rmarinho Dec 18, 2025
f2982aa
Continue on error
rmarinho Dec 19, 2025
2644414
Add gitignore for TypeScript source map artifacts (#33222)
Copilot Dec 19, 2025
27192d4
[Testing] Fix for flaky UITests in CI - 4 (#33215)
TamilarasanSF4853 Dec 19, 2025
f526b77
[ci] Try use netcore public (#33236)
rmarinho Dec 19, 2025
a1abaf8
[ci] Use windows pool to queue device tests (#33250)
rmarinho Dec 19, 2025
a14b855
[Testing] Rework Catalyst Test Runner to use startup arguments instea…
NafeelaNazhir Dec 22, 2025
b763385
Disable AppThemeBinding source generation for .NET 10 (#33219)
Copilot Dec 22, 2025
69613dd
[Android] Implemented `UseMaterial3` build property and provided basi…
NirmalKumarYuvaraj Dec 23, 2025
e7f8f08
Split iOS device tests by category in Helix (#33263)
PureWeen Dec 23, 2025
1f06a2b
[Testing] Fix for flaky UITests in CI - 5 (#33259)
TamilarasanSF4853 Dec 26, 2025
dec14ce
Add `UseMaterial3` build property to Sandbox, HostApp and Controls.Sa…
NirmalKumarYuvaraj Dec 26, 2025
eacd06d
Fixed the NRE in CarouselViewController on iOS 15.5 & 16.4 (#30838)
Ahamed-Ali Dec 4, 2025
2599c5a
Fix for TabBar Navigation does not invoke its IQueryAttributable.Appl…
SuthiYuvaraj Dec 4, 2025
7fdb7d9
Add unit tests for TabBar and FlyoutItem navigation ApplyQueryAttribu…
StephaneDelcroix Dec 4, 2025
4535e80
[C] Fix binding to interface-inherited properties like IReadOnlyList<…
StephaneDelcroix Dec 5, 2025
093ed1f
Fix #31939: CommandParameter TemplateBinding lost during reparenting …
StephaneDelcroix Dec 5, 2025
cf47f49
[XSG][BindingSourceGen] Add support for RelayCommand to compiled bind…
Copilot Dec 5, 2025
54b67d1
Update logic for large title display mode on iOS - shell (#33039)
kubaflo Dec 8, 2025
cb291d7
Fix for mediapicker (#32952)
HarishwaranVijayakumar Dec 9, 2025
4be1a3d
[iOS] Fixed memory leak with PopToRootAsync when using TitleView (#28…
Vignesh-SF3580 Dec 16, 2025
0e7c21a
[iOS, Mac, Windows] Fixed CharacterSpacing for SearchBar text and pla…
Dhivya-SF4094 Dec 16, 2025
3b39bb2
Fix ScrollToPosition.Center behavior in ScrollView on iOS and MacCata…
devanathan-vaithiyanathan Dec 16, 2025
e520a93
Fixed the FlyoutPage.Flyout Disappearing When Maximizing the Window o…
NanthiniMahalingam Dec 16, 2025
cff7f35
Revert "Update logic for large title display mode on iOS - shell (#33…
jfversluis Dec 19, 2025
3ce1420
[iOS, macOS] Fixed CollectionView group header size changes with Item…
NanthiniMahalingam Dec 19, 2025
4cba013
[Testing] Fixed Test case failure in PR 33185 - [12/22/2025] Candidat…
TamilarasanSF4853 Dec 22, 2025
7fbe83d
[Testing] Re-saved ShouldFlyoutBeVisibleAfterMaximizingWindow test ca…
TamilarasanSF4853 Dec 23, 2025
5b08ad1
[Testing] Fixed Test case failure in PR 33185 - [12/22/2025] Candidat…
TamilarasanSF4853 Dec 26, 2025
5f19e7e
[Testing] Fixed Test case failure in PR 33185 - [12/22/2025] Candidat…
TamilarasanSF4853 Dec 29, 2025
7b45118
December 22nd, Candidate (#33185)
PureWeen Dec 29, 2025
cb9c7cd
[Testing] Fix for flaky UITests in CI - 7 (#33313)
TamilarasanSF4853 Dec 30, 2025
a213790
[XSG] Skip compiled binding when RelativeSource is used (#33248)
StephaneDelcroix Dec 30, 2025
b494233
Enforce exactly one Category attribute per UI test (#33242)
jfversluis Dec 30, 2025
80b6844
Improve Controls Core API docs 2 (#33243)
jfversluis Dec 30, 2025
8fe90ca
Fix OnPlatform SourceGen for abstract types and protected constructor…
StephaneDelcroix Dec 31, 2025
235cb6f
Update investigations docs (#32949)
Copilot Dec 31, 2025
a6fbcb7
Merge branch 'main' into merge_net10.0
PureWeen Jan 1, 2026
3f5d3c1
Try xcode 26.2.0
rmarinho Jan 5, 2026
65ef482
Try build on maui pool
rmarinho Jan 5, 2026
7cca9f8
Update iOS versions
rmarinho Jan 5, 2026
40cc655
Update nuget.config
rmarinho Jan 5, 2026
95c545f
Fix feed
rmarinho Jan 5, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
The table of contents is too big for display.
Diff view
Diff view
  •  
  •  
  •  
162 changes: 162 additions & 0 deletions .github/instructions/helix-device-tests.instructions.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,162 @@
---
applyTo: "eng/helix_xharness.proj,eng/pipelines/**/device-tests*.yml,eng/pipelines/**/stage-device-tests.yml,src/**/DeviceTests/**"
description: "Guidelines for running and configuring .NET MAUI device tests on Helix infrastructure"
---

# Helix Device Tests Guidelines

This document provides guidance for working with .NET MAUI device tests that run on Helix infrastructure using XHarness.

## Overview

.NET MAUI uses [.NET Engineering Services Helix](https://helix.dot.net) with XHarness to run device tests across multiple platforms in parallel. This provides cloud-based device testing infrastructure.

### Device Test Projects

- `Controls.DeviceTests` - UI control tests
- `Core.DeviceTests` - Core framework tests
- `Graphics.DeviceTests` - Graphics and drawing tests
- `Essentials.DeviceTests` - Platform API tests
- `MauiBlazorWebView.DeviceTests` - Blazor WebView tests

### Available Helix Queues

Current configuration uses:
- **iOS**: `osx.15.arm64.maui.open`
- **Mac Catalyst**: `osx.15.arm64.maui.open`
- **Android**: `ubuntu.2204.amd64.android.33.open`

Check available queues at [helix.dot.net](https://helix.dot.net).

## Key Configuration Files

| File | Purpose |
|------|---------|
| `eng/helix_xharness.proj` | Main Helix configuration - defines scenarios, queues, and work items |
| `eng/pipelines/common/stage-device-tests.yml` | Pipeline template for device tests |
| `eng/test-configuration.json` | Test retry configuration |

## iOS Category Splitting

For iOS, Controls.DeviceTests heavy categories are split into separate Helix work items. This mirrors the old cake-based approach and enables parallel execution for the slowest tests.

**How it works:**
1. Heavy categories are defined in `ControlsTestCategoriesToSkipForRestOfTests` property in `helix_xharness.proj`
2. The `ControlsTestCategoriesToRunIndividually` ItemGroup is populated from that property
3. Each heavy category becomes a separate Helix work item
4. All other Controls tests run together in a single "General" work item
5. XHarness passes `--set-env="TestFilter=Category=X"` for individual categories
6. XHarness passes `--set-env="TestFilter=SkipCategories=X;Y;Z"` for the "General" work item
7. Core.DeviceTests runs as a single work item (no splitting)

**Heavy categories that run separately:**
- CollectionView, Shell, HybridWebView

**Keep in sync:** If adding new heavy categories, update the `ControlsTestCategoriesToSkipForRestOfTests` property in `eng/helix_xharness.proj`.

## Running Device Tests Locally

### Prerequisites

From the repository root:

```bash
# 1. Restore dotnet tools
dotnet tool restore

# 2. Build MSBuild tasks (required)
./build.sh -restore -build -configuration Release -projects $(PWD)/Microsoft.Maui.BuildTasks.slnf /bl:BuildBuildTasks.binlog -warnAsError false

# 3. Build device tests
./build.sh -restore -build -configuration Release /p:BuildDeviceTests=true /bl:BuildDeviceTests.binlog -warnAsError false
```

### Submit to Helix

Set required environment variables:

```bash
export BUILD_REASON=pr
export BUILD_REPOSITORY_NAME=maui
export BUILD_SOURCEBRANCH=main
export SYSTEM_TEAMPROJECT=dnceng
export SYSTEM_ACCESSTOKEN=''
```

Submit tests:

```bash
# Android
./eng/common/msbuild.sh ./eng/helix_xharness.proj /restore /p:TreatWarningsAsErrors=false /t:Test /p:TargetOS=android /bl:sendhelix_android.binlog

# iOS
./eng/common/msbuild.sh ./eng/helix_xharness.proj /restore /p:TreatWarningsAsErrors=false /t:Test /p:TargetOS=ios /bl:sendhelix_ios.binlog

# Mac Catalyst
./eng/common/msbuild.sh ./eng/helix_xharness.proj /restore /p:TreatWarningsAsErrors=false /t:Test /p:TargetOS=maccatalyst /bl:sendhelix_catalyst.binlog
```

### Windows Commands

```cmd
set BUILD_REASON=pr
set BUILD_REPOSITORY_NAME=maui
set BUILD_SOURCEBRANCH=main
set SYSTEM_TEAMPROJECT=dnceng
set SYSTEM_ACCESSTOKEN=

.\build.cmd -restore -build -configuration Release -projects ".\Microsoft.Maui.BuildTasks.slnf" /bl:BuildBuildTasks.binlog -warnAsError false
.\build.cmd -restore -build -configuration Release /p:BuildDeviceTests=true /bl:BuildDeviceTests.binlog -warnAsError false
.\eng\common\msbuild.cmd .\eng\helix_xharness.proj /restore /p:TreatWarningsAsErrors=false /t:Test /p:TargetOS=android /bl:sendhelix.binlog
```

### Validate MSBuild Logic Only

To validate the helix proj without submitting (requires built artifacts):

```bash
dotnet msbuild eng/helix_xharness.proj /t:DiscoverTestBundles /p:TargetOS=ios /p:_MauiDotNetTfm=net10.0 /p:RepoRoot=$(pwd)/ -v:n
```

## Configuration Details

The `eng/helix_xharness.proj` configuration includes:

- **Timeouts**: 2-hour work item timeout, 1-hour 15-min test timeout for category splits
- **Test Discovery**: Automatically discovers test bundles for each scenario
- **Platform Targeting**: Uses `TargetOS` property (ios, maccatalyst, android)
- **Queue Selection**: Platform-appropriate Helix queues
- **XHarness Integration**: Uses XHarness CLI for device orchestration

### CustomCommands for Category Filtering

When using category splitting, `CustomCommands` metadata overrides the default xharness invocation:

```xml
<CustomCommands>xharness apple test --target "$target" --app "$app" --output-directory "$output_directory" --timeout "$timeout" --launch-timeout "$launch_timeout" --set-env="TestFilter=Category=CategoryName"</CustomCommands>
```

**Important**: Keep CustomCommands as a single line. Multi-line commands with `set -ex` can cause parse errors.

## Troubleshooting

### Common Issues

1. **Build failures**: Ensure MSBuild tasks are built first
2. **Missing devices**: Check queue availability at [helix.dot.net](https://helix.dot.net)
3. **Authentication**: For CI, ensure proper Azure DevOps access tokens
4. **Timeouts**: Adjust `TestTimeout` and `WorkItemTimeout` for complex scenarios
5. **CustomCommands parse errors**: Keep commands on single line, avoid shell constructs like `set -ex`

### Logging and Diagnostics

- Use `/bl:filename.binlog` for detailed MSBuild logs
- Add `-verbosity:diag` for maximum diagnostic output
- Check Helix job results at the URL provided after submission
- Individual work item logs available at `https://helix.dot.net/api/2019-06-17/jobs/{jobId}/workitems/{workItemName}/console`

## Additional Resources

- [XHarness on Helix Documentation](https://github.com/dotnet/arcade/blob/main/src/Microsoft.DotNet.Helix/Sdk/tools/xharness-runner/Readme.md)
- [Helix SDK Documentation](https://github.com/dotnet/arcade/blob/main/src/Microsoft.DotNet.Helix/Sdk/Readme.md)
- [Example Helix Run](https://dev.azure.com/dnceng-public/public/_build/results?buildId=1115383&view=results)
4 changes: 4 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -383,3 +383,7 @@ snapshots-diff/
temp
.packages
/src/Templates/.tempTemplateOutput

# TypeScript source map files (generated artifacts)
# Note: CSS map files in templates (e.g., bootstrap) are intentionally tracked
*.js.map
Loading
Loading