[Testing] Rework Catalyst Test Runner to use startup arguments instead of UI navigation by NafeelaNazhir · Pull Request #31673 · dotnet/maui

NafeelaNazhir · 2025-09-18T10:30:52Z

Description of Change

This PR reopens the work from the closed #30651

This pull request introduces functionality to streamline test execution by allowing tests to be launched directly via startup arguments or programmatically. Key changes include adding support for passing test names as environment variables, introducing a PageFactory mechanism for dynamic page creation, and enhancing test lifecycle management for specific platforms like Mac Catalyst.

Note: This PR is an extension of the PR(#30286) with a proper implementation.

Test Execution Enhancements:

Startup Argument Support: Added logic to retrieve test names from environment variables and dynamically load the corresponding test page in CreateDefaultMainPage() (MauiProgram.cs).
Dynamic Page Creation: Introduced a PageFactory property in the IssueModel class to enable dynamic test page instantiation (TestCases.cs).
Direct Test Page Retrieval: Implemented TryToGetTestPage() in TestCaseScreen to retrieve test pages by description using PageFactory (TestCases.cs).

Platform-Specific Test Lifecycle Improvements:

Mac Catalyst Test Configuration: Enhanced _IssuesUITest to pass test names as startup arguments and manage app launch/close lifecycle specifically for Mac Catalyst (_IssuesUITest.cs). [1] [2]
** Appium Options for Mac Catalyst**: Updated AppiumCatalystApp to include environment variables in Appium options for test execution (AppiumCatalystApp.cs).

Helper Methods for Test Execution:

Parameterized App Launch: Added a method to launch apps with additional parameters, supporting test-specific configurations (HelperExtensions.cs).
Mac-Specific App Closure: Modified FixtureOneTimeTearDown() to handle app closure for Mac Catalyst during test teardown (UITestBase.cs).

Output

467025436-e594828d-93bb-4e41-b583-7d5b4ab2a0b9.mov

Issues Fixed

Fixes #30285

Copilot

Pull Request Overview

This PR reworks the Mac Catalyst test runner to enable direct test execution via startup arguments instead of requiring UI navigation. The change streamlines test execution by allowing tests to launch directly through environment variables and programmatic page creation.

Key Changes:

Enhanced test lifecycle management for Mac Catalyst with direct page loading via startup arguments
Introduced a new _GalleryUITest base class to simplify gallery-based test implementations
Added PageFactory mechanism for dynamic test page instantiation

Reviewed Changes

Copilot reviewed 76 out of 78 changed files in this pull request and generated 3 comments.

Show a summary per file

File	Description
`src/TestUtils/src/UITest.NUnit/UITestContextBase.cs`	Added Mac-specific app launch handling in InitialSetup
`src/TestUtils/src/UITest.NUnit/UITestBase.cs`	Enhanced teardown logic with Mac-specific app closure
`src/TestUtils/src/UITest.Appium/HelperExtensions.cs`	Added LaunchApp overload to support test name parameters
`src/TestUtils/src/UITest.Appium/AppiumCatalystApp.cs`	Enhanced Appium options to include environment variables
`src/TestUtils/src/UITest.Appium/Actions/AppiumLifecycleActions.cs`	Modified Mac app launch to use environment variables
`src/Controls/tests/TestCases.Shared.Tests/UtilExtensions.cs`	Updated NavigateToGallery to skip navigation for Catalyst
`src/Controls/tests/TestCases.Shared.Tests/UITest.cs`	Implemented Close and LaunchAppWithTest methods
`src/Controls/tests/TestCases.Shared.Tests/Tests/Issues/_IssuesUITest.cs`	Added Mac Catalyst startup argument support
`src/Controls/tests/TestCases.Shared.Tests/Tests/Issues/_GalleryUITest.cs`	New base class for gallery-based tests
`src/Controls/tests/TestCases.HostApp/TestCases.cs`	Added PageFactory property and TryToGetTestPage method
`src/Controls/tests/TestCases.HostApp/MauiProgram.cs`	Enhanced CreateDefaultMainPage with startup argument handling
Various test classes	Updated to inherit from _GalleryUITest and added GalleryPageName properties

Copilot · 2025-09-18T10:38:11Z

src/Controls/tests/TestCases.Shared.Tests/Tests/Issues/Bugzilla/Bugzilla40092.cs

 	public void AllScreenIsBlack()
 	{
-#if !ANDROID // Display alert not shown in android also this is not a needed one for ensuring this case, so ignored the below steps on Android.
+#if !ANDROID && !MACCATALYST // Display alert is not shown on Android, and it is not required to validate this case, so the following steps are skipped for Android. For Catalyst, the test page loading flow has been modified, making the popup validation unnecessary here as well.


[nitpick] The comment should be split into separate lines for better readability and clarity. Consider formatting as a multi-line comment or breaking it into shorter, more focused comments.

Suggested change

#if !ANDROID && !MACCATALYST // Display alert is not shown on Android, and it is not required to validate this case, so the following steps are skipped for Android. For Catalyst, the test page loading flow has been modified, making the popup validation unnecessary here as well.

// Display alert is not shown on Android, and it is not required to validate this case,

// so the following steps are skipped for Android.

// For Catalyst, the test page loading flow has been modified,

// making the popup validation unnecessary here as well.

#if !ANDROID && !MACCATALYST

Copilot · 2025-09-18T10:38:11Z

src/Controls/tests/TestCases.Shared.Tests/Tests/FeatureMatrix/SearchBarFeatureTests.cs

-	public SearchBarFeatureTests(TestDevice testDevice) : base(testDevice)
-	{
-	}
+	override public string GalleryPageName => SearchBarFeatureMatrix;


The access modifier 'override' should come after 'public' according to C# coding conventions. It should be 'public override string GalleryPageName'.

Suggested change

override public string GalleryPageName => SearchBarFeatureMatrix;

public override string GalleryPageName => SearchBarFeatureMatrix;

Copilot · 2025-09-18T10:38:12Z

src/Controls/tests/TestCases.Shared.Tests/Tests/FeatureMatrix/ScrollViewFeatureTests.cs

+public class ScrollViewFeatureTests : _GalleryUITest
 {
 	public const string ScrollViewFeatureMatrix = "ScrollView Feature Matrix";
+	override public string GalleryPageName => ScrollViewFeatureMatrix;


The access modifier 'override' should come after 'public' according to C# coding conventions. It should be 'public override string GalleryPageName'.

Suggested change

override public string GalleryPageName => ScrollViewFeatureMatrix;

public override string GalleryPageName => ScrollViewFeatureMatrix;

jsuarezruiz · 2025-09-18T11:31:20Z

/azp run

azure-pipelines · 2025-09-18T11:31:42Z

Azure Pipelines successfully started running 3 pipeline(s).

jsuarezruiz · 2025-09-26T07:10:42Z

/azp run

azure-pipelines · 2025-09-26T07:11:03Z

Azure Pipelines successfully started running 3 pipeline(s).

jsuarezruiz · 2025-09-26T07:40:44Z

src/Controls/tests/TestCases.HostApp/TestCases.cs

+			/// then by description if no match is found by name. If a match is found,
+			/// it invokes the associated PageFactory to create the page.
+			/// </remarks>
+			public Page TryToGetTestPage(string name)


Can try the exact match first:

var exactMatch = TestCases.FirstOrDefault(x => string.Equals(x.Description, description, StringComparison.OrdinalIgnoreCase));

if not find it, the current partial match as fallback.

@jsuarezruiz I have addressed the feedback in the latest commit.

src/Controls/tests/TestCases.Shared.Tests/Tests/Issues/_GalleryUITest.cs

jsuarezruiz · 2025-10-27T15:13:16Z

/azp run

azure-pipelines · 2025-10-27T15:13:40Z

Azure Pipelines successfully started running 3 pipeline(s).

PureWeen · 2025-10-31T23:38:14Z

/rebase

PureWeen · 2025-12-04T00:46:05Z

/rebase

sheiksyedm · 2025-12-07T13:38:33Z

/azp run MAUI-UITests-public

azure-pipelines · 2025-12-07T13:38:45Z

Azure Pipelines successfully started running 1 pipeline(s).

github-actions · 2025-12-09T06:56:05Z

🚀 Dogfood this PR with:

⚠️ WARNING: Do not do this without first carefully reviewing the code of this PR to satisfy yourself it is safe.

curl -fsSL https://raw.githubusercontent.com/dotnet/maui/main/eng/scripts/get-maui-pr.sh | bash -s -- 31673

Or

Run remotely in PowerShell:

iex "& { $(irm https://raw.githubusercontent.com/dotnet/maui/main/eng/scripts/get-maui-pr.ps1) } 31673"

sheiksyedm · 2025-12-11T15:01:55Z

/azp run

azure-pipelines · 2025-12-11T15:02:22Z

Azure Pipelines successfully started running 3 pipeline(s).

sheiksyedm · 2025-12-15T18:45:08Z

/rebase

sheiksyedm · 2025-12-18T08:05:33Z

/rebase

PureWeen

Amazing!!!

Going to merge as is but I was curious on the tests where you have it switched to CarouselView2 and CollectionView2. Do we need to log those as bugs?

…d of UI navigation (#31673) ### Description of Change #### This PR reopens the work from the closed [#30651](#30651) This pull request introduces functionality to streamline test execution by allowing tests to be launched directly via startup arguments or programmatically. Key changes include adding support for passing test names as environment variables, introducing a `PageFactory` mechanism for dynamic page creation, and enhancing test lifecycle management for specific platforms like Mac Catalyst. Note: This PR is an extension of the PR(#30286) with a proper implementation. #### Test Execution Enhancements: - **Startup Argument Support**: Added logic to retrieve test names from environment variables and dynamically load the corresponding test page in `CreateDefaultMainPage()` (`MauiProgram.cs`). - **Dynamic Page Creation**: Introduced a `PageFactory` property in the `IssueModel` class to enable dynamic test page instantiation (`TestCases.cs`). - **Direct Test Page Retrieval**: Implemented `TryToGetTestPage()` in `TestCaseScreen` to retrieve test pages by description using `PageFactory` (`TestCases.cs`). #### Platform-Specific Test Lifecycle Improvements: - **Mac Catalyst Test Configuration**: Enhanced `_IssuesUITest` to pass test names as startup arguments and manage app launch/close lifecycle specifically for Mac Catalyst (`_IssuesUITest.cs`). [[1]](https://github.com/dotnet/maui/pull/30651/files#diff-6e8c0ea1f2979b5484d14facb1aa5b2ccef3ee748f1606eaf46220ac0936e3edR22-R46) [[2]](https://github.com/dotnet/maui/pull/30651/files#diff-6e8c0ea1f2979b5484d14facb1aa5b2ccef3ee748f1606eaf46220ac0936e3edR58-R61) - ** Appium Options for Mac Catalyst**: Updated `AppiumCatalystApp` to include environment variables in Appium options for test execution (`AppiumCatalystApp.cs`). #### Helper Methods for Test Execution: - **Parameterized App Launch**: Added a method to launch apps with additional parameters, supporting test-specific configurations (`HelperExtensions.cs`). - **Mac-Specific App Closure**: Modified `FixtureOneTimeTearDown()` to handle app closure for Mac Catalyst during test teardown (`UITestBase.cs`). ### Output https://github.com/user-attachments/assets/82fc12ad-b0d3-4bad-901f-0ab6b2bddda1 ### Issues Fixed Fixes #30285

NafeelaNazhir added the area-testing Unit tests, device tests label Sep 18, 2025

dotnet-policy-service bot added community ✨ Community Contribution partner/syncfusion Issues / PR's with Syncfusion collaboration labels Sep 18, 2025

NafeelaNazhir marked this pull request as ready for review September 18, 2025 10:36

Copilot AI review requested due to automatic review settings September 18, 2025 10:37

Copilot AI reviewed Sep 18, 2025

View reviewed changes

jsuarezruiz self-assigned this Sep 18, 2025

PureWeen added this to the .NET 10.0 GA milestone Sep 20, 2025

PureWeen added this to MAUI SDK Ongoing Sep 20, 2025

github-project-automation bot moved this to Todo in MAUI SDK Ongoing Sep 20, 2025

NafeelaNazhir force-pushed the enhance_mac_uiTest branch from 59f0480 to 0bb18a0 Compare September 26, 2025 05:59

jsuarezruiz reviewed Sep 26, 2025

View reviewed changes

PureWeen moved this from Todo to Ready To Review in MAUI SDK Ongoing Oct 10, 2025

rmarinho modified the milestones: .NET 10.0 GA, .NET 10 SR1 Oct 22, 2025

NafeelaNazhir force-pushed the enhance_mac_uiTest branch from 7a5a026 to 3e7258a Compare October 30, 2025 07:51

PureWeen added the p/0 Current heighest priority issues that we are targeting for a release. label Oct 31, 2025

github-actions bot force-pushed the enhance_mac_uiTest branch from 3e7258a to 29d8e98 Compare October 31, 2025 23:39

NafeelaNazhir force-pushed the enhance_mac_uiTest branch from 16e932a to f34cc78 Compare December 4, 2025 08:33

PureWeen unassigned jsuarezruiz Dec 8, 2025

NafeelaNazhir force-pushed the enhance_mac_uiTest branch from a80e751 to 8483cb8 Compare December 10, 2025 11:04

github-actions bot force-pushed the enhance_mac_uiTest branch from 71d6efb to ef68d19 Compare December 15, 2025 18:45

build-analysis bot mentioned this pull request Dec 16, 2025

"We stopped hearing from agent Azure Pipelines 32. Verify the agent machine is running and has a healthy network connection" dotnet/dnceng#1886

Open

3 tasks

NafeelaNazhir added 10 commits December 18, 2025 08:06

modified files to enhance mac ui test flow

a5712ff

Modified newly added feature matrix test classes

aec485b

Changes updated

3e81f92

Modified newly added feature matrix

b4024f0

modified newly added feature matrix

6ad1501

changes updated

35dceab

changes updated

de077ee

modified new feature matrix tests

aa86b86

resave the image

6f9dbc3

Added delay

c465b73

github-actions bot force-pushed the enhance_mac_uiTest branch from 6e6efcd to c465b73 Compare December 18, 2025 08:06

PureWeen approved these changes Dec 22, 2025

View reviewed changes

github-project-automation bot moved this from Ready To Review to Approved in MAUI SDK Ongoing Dec 22, 2025

PureWeen merged commit a14b855 into dotnet:main Dec 22, 2025
159 checks passed

github-project-automation bot moved this from Approved to Done in MAUI SDK Ongoing Dec 22, 2025

github-actions bot locked and limited conversation to collaborators Jan 22, 2026

	override public string GalleryPageName => SearchBarFeatureMatrix;
	public override string GalleryPageName => SearchBarFeatureMatrix;

Conversation

NafeelaNazhir commented Sep 18, 2025

Description of Change

This PR reopens the work from the closed #30651

Test Execution Enhancements:

Platform-Specific Test Lifecycle Improvements:

Helper Methods for Test Execution:

Output

Issues Fixed

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull Request Overview

Key Changes:

Reviewed Changes

Uh oh!

Copilot AI Sep 18, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI Sep 18, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI Sep 18, 2025

Choose a reason for hiding this comment

Uh oh!

jsuarezruiz commented Sep 18, 2025

Uh oh!

azure-pipelines bot commented Sep 18, 2025

Uh oh!

jsuarezruiz commented Sep 26, 2025

Uh oh!

azure-pipelines bot commented Sep 26, 2025

Uh oh!

jsuarezruiz Sep 26, 2025

Choose a reason for hiding this comment

Uh oh!

NafeelaNazhir Oct 6, 2025

Choose a reason for hiding this comment

Uh oh!

Uh oh!

jsuarezruiz commented Oct 27, 2025

Uh oh!

azure-pipelines bot commented Oct 27, 2025

Uh oh!

PureWeen commented Oct 31, 2025

Uh oh!

PureWeen commented Dec 4, 2025

Uh oh!

sheiksyedm commented Dec 7, 2025

Uh oh!

azure-pipelines bot commented Dec 7, 2025

Uh oh!

github-actions bot commented Dec 9, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

sheiksyedm commented Dec 11, 2025

Uh oh!

azure-pipelines bot commented Dec 11, 2025

Uh oh!

sheiksyedm commented Dec 15, 2025

Uh oh!

sheiksyedm commented Dec 18, 2025

Uh oh!

PureWeen left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

github-actions bot commented Dec 9, 2025 •

edited

Loading