[Testing] Resaved the iOS 26.4 images#35061
Merged
Merged
Conversation
Contributor
|
🚀 Dogfood this PR with:
curl -fsSL https://raw.githubusercontent.com/dotnet/maui/main/eng/scripts/get-maui-pr.sh | bash -s -- 35061Or
iex "& { $(irm https://raw.githubusercontent.com/dotnet/maui/main/eng/scripts/get-maui-pr.ps1) } 35061" |
Contributor
|
Hey there @@TamilarasanSF4853! Thank you so much for your PR! Someone from the team will get assigned to your PR shortly and we'll get it reviewed. |
dotnet-policy-service
Bot
added
the
partner/syncfusion
Member
|
/azp run maui-pr-uitests, maui-pr-devicetests |
|
Azure Pipelines successfully started running 2 pipeline(s). |
Adds updated baselines for Slider (43), ScrollView (26), Editor (2), Shape/Polygon (4), and FlexLayout (1) that were missed in the initial update. Images sourced from CI build 1389538 artifacts. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Member
|
/azp run maui-pr-uitests, maui-pr-devicetests |
|
Azure Pipelines successfully started running 2 pipeline(s). |
This was referenced Apr 21, 2026
Open
PureWeen
pushed a commit
that referenced
this pull request
Apr 22, 2026
Resaved the iOS 26 images due to the iOS 26.4 version used on the current CI. --------- Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
3 tasks
This was referenced Apr 22, 2026
kubaflo
pushed a commit
that referenced
this pull request
May 9, 2026
AcesShared agents have iOS 26.0, 26.3.1, and 26.4 pre-installed. PR #35061 resaved all ios-26 snapshot baselines on iOS 26.4, so using iOS 26.0 (our previous ascending sort) caused pixel diffs on every single visual test. Switch to descending sort for all iOS versions so iOS-26-4 wins, matching what the main CI pipeline (Cake runner) uses. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
kubaflo
pushed a commit
that referenced
this pull request
May 10, 2026
PR #35061 resaved all ios-26 baselines on iOS 26.4 (build 23E244). provision.yml only downloads iOS 26.0. AcesShared agents inconsistently have 26.4 pre-installed — some do, most don't. Add an explicit xcodebuild -downloadPlatform iOS -buildVersion 26.4 step after provision so every Deep stage agent has it. The main CI pipeline (maui-pr-uitests) has the same inconsistency: ViewBaseTests also fails on agents without 26.4. Our pipeline fixes this by ensuring 26.4 is always available. Gate+TryFix stays disabled for faster iteration. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
kubaflo
pushed a commit
that referenced
this pull request
May 10, 2026
AcesShared pool has two VM images: - ACES_VM_SharedPool_Sequoia (macOS 15.x) — only iOS 26.0/26.1 - ACES_VM_SharedPool_Tahoe (macOS 26.x) — has iOS 26.4 pre-installed PR #35061 resaved ios-26 baselines on iOS 26.4. Without this demand, 50% of agents get Sequoia VMs where iOS 26.4 is unavailable and all visual tests fail with pixel diffs. The main maui-pr-uitests pipeline also uses AcesShared without this demand and has the same inconsistency. Our pipeline fixes it. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
kubaflo
pushed a commit
that referenced
this pull request
May 10, 2026
…untime provision.yml downloads iOS 26.0, but ios-26 baselines were captured on iOS 26.4 (PR #35061). Tahoe agents ship iOS 26.3.1 or 26.4 pre- installed. By setting skipSimulatorSetup: true for the Deep stage, we avoid installing 26.0 and let the agent's pre-installed runtime (26.3.1+) be used, which is closer to the baselines. Combined with the Tahoe pool demand, this ensures: - Agent has macOS 26.x (Tahoe image) - iOS 26.3.1+ pre-installed (no 26.0 download polluting the list) - Descending sort picks highest available runtime Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
kubaflo
pushed a commit
that referenced
this pull request
May 10, 2026
Tahoe agents have macOS 26.4 with Xcode 26.0.1-26.3 installed, but iOS 26.4 simulator is NOT always pre-installed (some VMs only have 26.3.1). Baselines were captured on 26.4 (PR #35061). Switch to the highest available Xcode (26.3) which can download iOS 26.4, then restore the default Xcode (26.0.1) for the build. This ensures Start-Emulator.ps1 descending sort picks 26.4. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
kubaflo
pushed a commit
that referenced
this pull request
May 19, 2026
AcesShared agents have iOS 26.0, 26.3.1, and 26.4 pre-installed. PR #35061 resaved all ios-26 snapshot baselines on iOS 26.4, so using iOS 26.0 (our previous ascending sort) caused pixel diffs on every single visual test. Switch to descending sort for all iOS versions so iOS-26-4 wins, matching what the main CI pipeline (Cake runner) uses. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
kubaflo
pushed a commit
that referenced
this pull request
May 19, 2026
PR #35061 resaved all ios-26 baselines on iOS 26.4 (build 23E244). provision.yml only downloads iOS 26.0. AcesShared agents inconsistently have 26.4 pre-installed — some do, most don't. Add an explicit xcodebuild -downloadPlatform iOS -buildVersion 26.4 step after provision so every Deep stage agent has it. The main CI pipeline (maui-pr-uitests) has the same inconsistency: ViewBaseTests also fails on agents without 26.4. Our pipeline fixes this by ensuring 26.4 is always available. Gate+TryFix stays disabled for faster iteration. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
kubaflo
pushed a commit
that referenced
this pull request
May 19, 2026
AcesShared pool has two VM images: - ACES_VM_SharedPool_Sequoia (macOS 15.x) — only iOS 26.0/26.1 - ACES_VM_SharedPool_Tahoe (macOS 26.x) — has iOS 26.4 pre-installed PR #35061 resaved ios-26 baselines on iOS 26.4. Without this demand, 50% of agents get Sequoia VMs where iOS 26.4 is unavailable and all visual tests fail with pixel diffs. The main maui-pr-uitests pipeline also uses AcesShared without this demand and has the same inconsistency. Our pipeline fixes it. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
kubaflo
pushed a commit
that referenced
this pull request
May 19, 2026
…untime provision.yml downloads iOS 26.0, but ios-26 baselines were captured on iOS 26.4 (PR #35061). Tahoe agents ship iOS 26.3.1 or 26.4 pre- installed. By setting skipSimulatorSetup: true for the Deep stage, we avoid installing 26.0 and let the agent's pre-installed runtime (26.3.1+) be used, which is closer to the baselines. Combined with the Tahoe pool demand, this ensures: - Agent has macOS 26.x (Tahoe image) - iOS 26.3.1+ pre-installed (no 26.0 download polluting the list) - Descending sort picks highest available runtime Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
kubaflo
pushed a commit
that referenced
this pull request
May 19, 2026
Tahoe agents have macOS 26.4 with Xcode 26.0.1-26.3 installed, but iOS 26.4 simulator is NOT always pre-installed (some VMs only have 26.3.1). Baselines were captured on 26.4 (PR #35061). Switch to the highest available Xcode (26.3) which can download iOS 26.4, then restore the default Xcode (26.0.1) for the build. This ensures Start-Emulator.ps1 descending sort picks 26.4. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
kubaflo
pushed a commit
that referenced
this pull request
May 22, 2026
AcesShared agents have iOS 26.0, 26.3.1, and 26.4 pre-installed. PR #35061 resaved all ios-26 snapshot baselines on iOS 26.4, so using iOS 26.0 (our previous ascending sort) caused pixel diffs on every single visual test. Switch to descending sort for all iOS versions so iOS-26-4 wins, matching what the main CI pipeline (Cake runner) uses. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
kubaflo
pushed a commit
that referenced
this pull request
May 22, 2026
PR #35061 resaved all ios-26 baselines on iOS 26.4 (build 23E244). provision.yml only downloads iOS 26.0. AcesShared agents inconsistently have 26.4 pre-installed — some do, most don't. Add an explicit xcodebuild -downloadPlatform iOS -buildVersion 26.4 step after provision so every Deep stage agent has it. The main CI pipeline (maui-pr-uitests) has the same inconsistency: ViewBaseTests also fails on agents without 26.4. Our pipeline fixes this by ensuring 26.4 is always available. Gate+TryFix stays disabled for faster iteration. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
kubaflo
pushed a commit
that referenced
this pull request
May 22, 2026
AcesShared pool has two VM images: - ACES_VM_SharedPool_Sequoia (macOS 15.x) — only iOS 26.0/26.1 - ACES_VM_SharedPool_Tahoe (macOS 26.x) — has iOS 26.4 pre-installed PR #35061 resaved ios-26 baselines on iOS 26.4. Without this demand, 50% of agents get Sequoia VMs where iOS 26.4 is unavailable and all visual tests fail with pixel diffs. The main maui-pr-uitests pipeline also uses AcesShared without this demand and has the same inconsistency. Our pipeline fixes it. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
kubaflo
pushed a commit
that referenced
this pull request
May 22, 2026
…untime provision.yml downloads iOS 26.0, but ios-26 baselines were captured on iOS 26.4 (PR #35061). Tahoe agents ship iOS 26.3.1 or 26.4 pre- installed. By setting skipSimulatorSetup: true for the Deep stage, we avoid installing 26.0 and let the agent's pre-installed runtime (26.3.1+) be used, which is closer to the baselines. Combined with the Tahoe pool demand, this ensures: - Agent has macOS 26.x (Tahoe image) - iOS 26.3.1+ pre-installed (no 26.0 download polluting the list) - Descending sort picks highest available runtime Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
kubaflo
pushed a commit
that referenced
this pull request
May 22, 2026
Tahoe agents have macOS 26.4 with Xcode 26.0.1-26.3 installed, but iOS 26.4 simulator is NOT always pre-installed (some VMs only have 26.3.1). Baselines were captured on 26.4 (PR #35061). Switch to the highest available Xcode (26.3) which can download iOS 26.4, then restore the default Xcode (26.0.1) for the build. This ensures Start-Emulator.ps1 descending sort picks 26.4. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
rmarinho
pushed a commit
that referenced
this pull request
May 22, 2026
## Description
Extends the `maui-copilot` DevDiv pipeline (pipeline 27723) with a
3-stage architecture that runs real UI tests on platform-pool agents and
reports results directly in the AI summary PR comment.
### Pipeline Workflow
```
┌─────────────────────────────────────────────────────────┐
│ Stage 1: ReviewPR │
│ │
│ STEP 1: Branch Setup (checkout + cherry-pick PR) │
│ STEP 2: Detect UI Test Categories │
│ STEP 3: Run Detected UI Tests (in-process, fast) │
│ STEP 4: Regression Cross-Reference │
│ STEP 5: Gate — verify tests fail/pass before/after fix │
│ STEP 6: Code Review — deep analysis via Copilot agent │
│ │
│ Outputs → CopilotLogs artifact + detectedCategories │
└──────────────────────┬──────────────────────────────────┘
│
┌──────────────────────▼──────────────────────────────────┐
│ Stage 2: RunDeepUITests (platform-pool agent) │
│ │
│ iOS: AcesShared Tahoe + iOS 26.4 │
│ Android: ubuntu-22.04 + KVM + AVD │
│ │
│ Runs BuildAndRunHostApp.ps1 per detected category │
│ Outputs → drop-deep-uitests artifact (TRX + diffs) │
└──────────────────────┬──────────────────────────────────┘
│
┌──────────────────────▼──────────────────────────────────┐
│ Stage 3: PostResults │
│ │
│ 1. Download CopilotLogs (review content files) │
│ 2. Download drop-deep-uitests (TRX results) │
│ 3. Merge deep results into uitests/content.md │
│ 4. Post full AI Summary comment on PR │
│ 5. Apply labels (s/agent-reviewed, etc.) │
│ │
│ One comment with everything — no patching needed │
└─────────────────────────────────────────────────────────┘
```
### What's New
**Deep UI Test Execution (Stage 2)**
- Runs detected UI test categories on proper platform-pool agents (not
in-process on Linux)
- **iOS**: AcesShared Tahoe agents with iOS 26.4 simulator, iPhone 11
Pro (matching `ios-26` baselines from PR #35061)
- **Android**: ubuntu-22.04 with KVM, AVD boot with `-partition-size
2048`, `ignoreHiddenApiPolicyError` capability
- TRX results + snapshot-diff PNGs published as `drop-deep-uitests`
artifact
**Unified Comment Posting (Stage 3)**
- Comment posting and label application deferred to Stage 3 (after deep
tests complete)
- Single AI summary comment includes ALL results: code review + deep
test results
- Nested collapsible `<details>` for failed tests with full error +
stack trace
- Dynamic section title: `🧪 UI Tests — CollectionView, TabbedPage`
- Artifact download link for snapshot-diff PNGs
**Android Emulator Improvements**
- AVD boot step with proper partition size, ADB key pre-authorization,
boot wait
- `DEVICE_UDID` pass-through prevents double emulator boot
- Disk cleanup on hosted ubuntu agents (frees ~22GB)
- KVM enablement + `appium:ignoreHiddenApiPolicyError` for API 30
**iOS Simulator Improvements**
- Tahoe pool demand ensures macOS 26.x agents
- Explicit iOS 26.4 download via latest Xcode
- Auto-creates iPhone 11 Pro for baseline resolution match
### Validation
Tested across 30+ pipeline iterations on 6 PRs:
| PR | iOS | Android |
|---|---|---|
| 35358 (ViewBaseTests) | **112/112 ALL PASS** ✅ | **118/119 PASS** ✅ |
| 35359 (TabbedPage) | 44/50 (1 real failure) | 74/75 (1 real failure) |
| 35356 (CollectionView) | **415/417 PASS** ✅ | 593/619 (26 real
failures) |
---------
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Resaved the iOS 26 images due to the iOS 26.4 version used on the current CI.