Skip to content

Separate snapshot diffs artifacts in CI #31724

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
jfversluis merged 13 commits into from
Sep 24, 2025
Merged

Separate snapshot diffs artifacts in CI #31724

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
13 commits
Select commit Hold shift + click to select a range
26e14f8
Update ui-tests-steps.yml
jfversluis Sep 23, 2025
99c982a
Fake test snapshot diff
jfversluis Sep 23, 2025
03fd35f
Replace hardcoded tfm
jfversluis Sep 23, 2025
e0f16f5
Update ui-tests-steps.yml
jfversluis Sep 23, 2025
3c37735
Update ui-tests-steps.yml
jfversluis Sep 23, 2025
288f86d
Different approach
jfversluis Sep 23, 2025
38526bf
Update ui-tests.yml
jfversluis Sep 23, 2025
f4cfa6e
Update ui-tests.yml
jfversluis Sep 23, 2025
05dae9e
Update ui-tests.yml
jfversluis Sep 23, 2025
713ac9a
Reusable template
jfversluis Sep 23, 2025
80cd9a9
Revert "Fake test snapshot diff"
jfversluis Sep 23, 2025
787fe09
Update eng/pipelines/common/ui-tests-collect-snapshot-diffs.yml
jfversluis Sep 23, 2025
9fefdfa
Also check files in subfolders
jfversluis Sep 24, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
36 changes: 36 additions & 0 deletions eng/pipelines/common/ui-tests-collect-snapshot-diffs.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,36 @@
parameters:
platform: '' # The platform name for logging (e.g., 'Android', 'iOS', 'Windows', 'Mac')
artifactName: '' # The artifact name to publish (e.g., 'uitest-snapshot-results-android')

steps:
- task: PowerShell@2
condition: always()
displayName: Check for ${{ parameters.platform }} snapshot diffs
name: Check${{ replace(parameters.platform, ' ', '') }}Snapshots # NOTE: This step name must match the variable reference in the condition below
inputs:
targetType: 'inline'
script: |
$snapshotDiffPath = "$(Build.ArtifactStagingDirectory)/Controls.TestCases.Shared.Tests/snapshots-diff"
Write-Host "Checking for ${{ parameters.platform }} snapshot diffs at: $snapshotDiffPath"

if (Test-Path $snapshotDiffPath) {
$diffFiles = Get-ChildItem $snapshotDiffPath -File -Recurse

if ($diffFiles.Count -gt 0) {
Write-Host "✅ Found $($diffFiles.Count) ${{ parameters.platform }} snapshot diff files in snapshots-diff directory"
Write-Host "##vso[task.setvariable variable=snapshotsExist;isOutput=true]true"
} else {
Write-Host "ℹ️ snapshots-diff directory exists but no ${{ parameters.platform }} diff files found"
Write-Host "##vso[task.setvariable variable=snapshotsExist;isOutput=true]false"
}
} else {
Write-Host "❌ No ${{ parameters.platform }} snapshots-diff directory found at: $snapshotDiffPath"
Write-Host "##vso[task.setvariable variable=snapshotsExist;isOutput=true]false"
}

- task: PublishPipelineArtifact@1
condition: eq(variables['Check${{ replace(parameters.platform, ' ', '') }}Snapshots.snapshotsExist'], 'true') # NOTE: This variable reference must match the step name above
displayName: Publish ${{ parameters.platform }} snapshot diffs
inputs:
targetPath: $(Build.ArtifactStagingDirectory)/Controls.TestCases.Shared.Tests
artifact: ${{ parameters.artifactName }}
81 changes: 1 addition & 80 deletions eng/pipelines/common/ui-tests-steps.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -170,90 +170,11 @@ steps:
condition: always()
displayName: publish artifacts

# Enable Notification Center re-enabled only for catalyst
- ${{ if eq(parameters.platform, 'catalyst')}}:
- bash: |
chmod +x $(System.DefaultWorkingDirectory)/eng/scripts/enable-notification-center.sh
$(System.DefaultWorkingDirectory)/eng/scripts/enable-notification-center.sh
displayName: 'Enable Notification Center'
continueOnError: true
timeoutInMinutes: 60

- ${{ if eq(parameters.platform, 'android')}}:
- task: PowerShell@2
inputs:
targetType: 'inline'
script: |
$folderPath = "${{ parameters.checkoutDirectory }}/artifacts/bin/Controls.TestCases.Android.Tests/${{ parameters.configuration }}/net9.0/snapshots-diff"
if (Test-Path $folderPath) {
Write-Host "##vso[task.setvariable variable=androidFolderExists;isOutput=true]true"
} else {
Write-Host "##vso[task.setvariable variable=androidFolderExists;isOutput=true]false"
}
name: CheckAndroidFolderExists

- task: PublishBuildArtifacts@1
condition: eq(variables['CheckAndroidFolderExists.androidFolderExists'], 'true')
displayName: publish Android screenshots artifacts
inputs:
PathToPublish: ${{ parameters.checkoutDirectory }}/artifacts/bin/Controls.TestCases.Android.Tests/${{ parameters.configuration }}/net9.0/snapshots-diff
ArtifactName: uitest-snapshot-results-android

- ${{ if eq(parameters.platform, 'ios')}}:
- task: PowerShell@2
inputs:
targetType: 'inline'
script: |
$folderPath = "${{ parameters.checkoutDirectory }}/artifacts/bin/Controls.TestCases.iOS.Tests/${{ parameters.configuration }}/net9.0/snapshots-diff"
if (Test-Path $folderPath) {
Write-Host "##vso[task.setvariable variable=iosFolderExists;isOutput=true]true"
} else {
Write-Host "##vso[task.setvariable variable=iosFolderExists;isOutput=true]false"
}
name: CheckIosFolderExists

- task: PublishBuildArtifacts@1
condition: eq(variables['CheckIosFolderExists.iosFolderExists'], 'true')
displayName: publish iOS screenshots artifacts
inputs:
PathToPublish: ${{ parameters.checkoutDirectory }}/artifacts/bin/Controls.TestCases.iOS.Tests/${{ parameters.configuration }}/net9.0/snapshots-diff
ArtifactName: uitest-snapshot-results-ios

- ${{ if eq(parameters.platform, 'windows')}}:
- task: PowerShell@2
inputs:
targetType: 'inline'
script: |
$folderPath = "${{ parameters.checkoutDirectory }}/artifacts/bin/Controls.TestCases.WinUI.Tests/${{ parameters.configuration }}/net9.0/snapshots-diff"
if (Test-Path $folderPath) {
Write-Host "##vso[task.setvariable variable=windowsFolderExists;isOutput=true]true"
} else {
Write-Host "##vso[task.setvariable variable=windowsFolderExists;isOutput=true]false"
}
name: CheckWindowsFolderExists

- task: PublishBuildArtifacts@1
condition: eq(variables['CheckWindowsFolderExists.windowsFolderExists'], 'true')
displayName: publish Windows screenshots artifacts
inputs:
PathToPublish: ${{ parameters.checkoutDirectory }}/artifacts/bin/Controls.TestCases.WinUI.Tests/${{ parameters.configuration }}/net9.0/snapshots-diff
ArtifactName: uitest-snapshot-results-windows

- ${{ if eq(parameters.platform, 'catalyst')}}:
- task: PowerShell@2
inputs:
targetType: 'inline'
script: |
$folderPath = "${{ parameters.checkoutDirectory }}/artifacts/bin/Controls.TestCases.Mac.Tests/${{ parameters.configuration }}/net9.0/snapshots-diff"
if (Test-Path $folderPath) {
Write-Host "##vso[task.setvariable variable=macFolderExists;isOutput=true]true"
} else {
Write-Host "##vso[task.setvariable variable=macFolderExists;isOutput=true]false"
}
name: CheckMacFolderExists

- task: PublishBuildArtifacts@1
condition: eq(variables['CheckMacFolderExists.macFolderExists'], 'true')
displayName: publish Mac screenshots artifacts
inputs:
PathToPublish: ${{ parameters.checkoutDirectory }}/artifacts/bin/Controls.TestCases.Mac.Tests/${{ parameters.configuration }}/net9.0/snapshots-diff
ArtifactName: uitest-snapshot-results-mac
36 changes: 36 additions & 0 deletions eng/pipelines/common/ui-tests.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -130,6 +130,12 @@ stages:
provisionatorChannel: ${{ parameters.provisionatorChannel }}
testFilter: $(CATEGORYGROUP)
skipProvisioning: ${{ parameters.skipProvisioning }}

# Collect and publish Android snapshot diffs
- template: ui-tests-collect-snapshot-diffs.yml
parameters:
platform: 'Android'
artifactName: 'uitest-snapshot-results-android-$(System.JobName)-$(System.JobAttempt)'

- stage: ios_ui_tests_mono
displayName: iOS UITests Mono
Expand Down Expand Up @@ -171,6 +177,12 @@ stages:
runtimeVariant : "Mono"
testFilter: $(CATEGORYGROUP)
skipProvisioning: ${{ parameters.skipProvisioning }}

# Collect and publish iOS snapshot diffs
- template: ui-tests-collect-snapshot-diffs.yml
parameters:
platform: 'iOS'
artifactName: 'uitest-snapshot-results-ios-$(System.JobName)-$(System.JobAttempt)'

- stage: ios_ui_tests_mono_cv2
displayName: iOS UITests Mono CollectionView2
Expand Down Expand Up @@ -208,6 +220,12 @@ stages:
testFilter: "CollectionView"
testConfigurationArgs: "CollectionView2"
skipProvisioning: ${{ parameters.skipProvisioning }}

# Collect and publish iOS CV2 snapshot diffs
- template: ui-tests-collect-snapshot-diffs.yml
parameters:
platform: 'iOS CV2'
artifactName: 'uitest-snapshot-results-ios-cv2-$(System.JobName)-$(System.JobAttempt)'

- stage: ios_ui_tests_mono_carv2
displayName: iOS UITests Mono CarouselView2
Expand Down Expand Up @@ -245,6 +263,12 @@ stages:
testFilter: "CarouselView"
testConfigurationArgs: "CollectionView2"
skipProvisioning: ${{ parameters.skipProvisioning }}

# Collect and publish iOS CARV2 snapshot diffs
- template: ui-tests-collect-snapshot-diffs.yml
parameters:
platform: 'iOS CARV2'
artifactName: 'uitest-snapshot-results-ios-carv2-$(System.JobName)-$(System.JobAttempt)'

# NativeAOT iOS UI tests stage - only run when both BuildNativeAOT and RunNativeAOT parameters are true
- ${{ if and(eq(parameters.BuildNativeAOT, true), eq(parameters.RunNativeAOT, true)) }}:
Expand Down Expand Up @@ -319,6 +343,12 @@ stages:
provisionatorChannel: ${{ parameters.provisionatorChannel }}
testFilter: $(CATEGORYGROUP)
skipProvisioning: ${{ parameters.skipProvisioning }}

# Collect and publish Windows snapshot diffs
- template: ui-tests-collect-snapshot-diffs.yml
parameters:
platform: 'Windows'
artifactName: 'uitest-snapshot-results-windows-$(System.JobName)-$(System.JobAttempt)'

- stage: mac_ui_tests
displayName: macOS UITests
Expand Down Expand Up @@ -351,3 +381,9 @@ stages:
provisionatorChannel: ${{ parameters.provisionatorChannel }}
testFilter: $(CATEGORYGROUP)
skipProvisioning: ${{ parameters.skipProvisioning }}

# Collect and publish Mac snapshot diffs
- template: ui-tests-collect-snapshot-diffs.yml
parameters:
platform: 'Mac'
artifactName: 'uitest-snapshot-results-mac-$(System.JobName)-$(System.JobAttempt)'
Loading