[Testing] Fix for flaky UITests in CI - 3 by TamilarasanSF4853 · Pull Request #33689 · dotnet/maui

TamilarasanSF4853 · 2026-01-23T13:54:53Z

This pull request includes targeted improvements to test reliability and platform-specific code handling in the test suite. The main changes address platform differences for cookie retrieval and enhance test stability by adding retries and synchronization points.

Platform-specific handling:

In CookiesCorrectlyLoadWithMultipleWebViews test, added a failing condition to restrict it on the Android platform.

Test stability improvements:

In SafeAreaBorderMultipleOrientationChanges test, the [Retry(2)] attribute was added to reduce flakiness by retrying failed tests up to two times.
In ApplyingRotationWithZeroDurationShouldNotCrash test, an additional App.WaitForElement("RotateButton") was inserted to ensure UI stability before proceeding with the next action.

dotnet-policy-service · 2026-01-23T13:55:17Z

Hey there @@TamilarasanSF4853! Thank you so much for your PR! Someone from the team will get assigned to your PR shortly and we'll get it reviewed.

Copilot

Pull request overview

Improves UI test reliability by adding retries/synchronization and adjusting cookie retrieval logic for Android.

Changes:

Add an NUnit retry to reduce flakiness in the SafeArea border orientation test.
Add extra synchronization between taps in a rotation regression test.
Add an Android-specific cookie retrieval path in the HostApp issue page.

Reviewed changes

Copilot reviewed 3 out of 4 changed files in this pull request and generated 2 comments.

File	Description
src/Controls/tests/TestCases.Shared.Tests/Tests/Issues/Issue28986_SafeAreaBorderOrientation.cs	Adds `[Retry(2)]` to mitigate intermittent failures during repeated orientation changes.
src/Controls/tests/TestCases.Shared.Tests/Tests/Issues/Issue18420.cs	Adds additional `WaitForElement` calls to stabilize repeated tap interactions.
src/Controls/tests/TestCases.HostApp/Issues/Issue18452.cs	Minor formatting/newline change only.
src/Controls/tests/TestCases.HostApp/Issues/Issue12134.cs	Adds Android cookie retrieval via `Android.Webkit.CookieManager` instead of JS evaluation.

Copilot · 2026-01-23T14:07:14Z

src/Controls/tests/TestCases.HostApp/Issues/Issue12134.cs

 		private async void WebViewOnNavigated(object sender, WebNavigatedEventArgs e)
 		{
+#if ANDROID
+			var result = Android.Webkit.CookieManager.Instance?.GetCookie("https://dotnet.microsoft.com");
+#else


WebViewOnNavigated is marked async, but under #if ANDROID there is no await, which produces CS1998. This repo sets TreatWarningsAsErrors=true (see Directory.Build.props:16), so Android builds will fail. Consider restructuring so the method always awaits (e.g., wrap the Android cookie lookup in a Task and await it), or conditional-compile a non-async Android handler.

Copilot · 2026-01-23T14:07:15Z

src/Controls/tests/TestCases.HostApp/Issues/Issue12134.cs

+#else
 			var result = await ((WebView)sender).EvaluateJavaScriptAsync("document.cookie");
+#endif
 			_label.Text = result.Contains(_guid.ToString(), StringComparison.OrdinalIgnoreCase) ? "Success" : "Failed";


Android.Webkit.CookieManager.Instance?.GetCookie(...) can return null (or the Instance can be null), and the next line calls result.Contains(...), which will throw on Android. Please make the check null-safe and treat null/empty as "Failed".

Suggested change

_label.Text = result.Contains(_guid.ToString(), StringComparison.OrdinalIgnoreCase) ? "Success" : "Failed";

var success = !string.IsNullOrEmpty(result) && result.Contains(_guid.ToString(), StringComparison.OrdinalIgnoreCase);

_label.Text = success ? "Success" : "Failed";

rmarinho · 2026-01-26T12:00:34Z

/azp run maui-pr-uitests

azure-pipelines · 2026-01-26T12:00:46Z

Azure Pipelines successfully started running 1 pipeline(s).

sheiksyedm · 2026-01-28T11:49:08Z

/azp run maui-pr-uitests

azure-pipelines · 2026-01-28T11:49:20Z

Azure Pipelines successfully started running 1 pipeline(s).

- Issue18751/18896 local images: covered by #33737 - Issue12134 timeout: covered by #33689 (uses native CookieManager)

PureWeen

looks like CookiesCorrectlyLoadWithMultipleWebViews failed

- Issue18751/18896 local images: covered by #33737 - Issue12134 timeout: covered by #33689 (uses native CookieManager)

TamilarasanSF4853 · 2026-01-29T10:20:07Z

looks like CookiesCorrectlyLoadWithMultipleWebViews failed

@PureWeen Restricted the CookiesCorrectlyLoadWithMultipleWebViews test for Android.

sheiksyedm · 2026-01-29T14:09:40Z

/azp run maui-pr-uitests

azure-pipelines · 2026-01-29T14:09:52Z

Azure Pipelines successfully started running 1 pipeline(s).

- Issue18751/18896 local images: covered by #33737 - Issue12134 timeout: covered by #33689 (uses native CookieManager)

…33749) > [!NOTE] > Are you waiting for the changes in this PR to be merged? > It would be very helpful if you could [test the resulting artifacts](https://github.com/dotnet/maui/wiki/Testing-PR-Builds) from this PR and let us know in a comment if this change resolves your issue. Thank you! ## Description This PR improves the reliability of several flaky UI tests by: 1. **Replacing `Thread.Sleep` with `retryTimeout`** - The `retryTimeout` parameter keeps retrying screenshot comparisons until they match or timeout, adapting to actual UI timing across different machines/load conditions. 2. **Fixing Issue10563 SwipeView button animations** - Added flat `Background = SolidColorBrush(Colors.LightGray)` to buttons to prevent ripple animation artifacts that caused visual differences on CI. ## Changes | Test | Issue | Fix | |------|-------|-----| | Issue10563 | Button ripple animation artifacts in SwipeView test | Flat button background + updated baselines | | Issue18857 | Thread.Sleep causes flakiness | `retryTimeout: TimeSpan.FromSeconds(2)` | | Issue24856 | Thread.Sleep causes flakiness | `retryTimeout: TimeSpan.FromSeconds(2)` | | Issue25192 | Thread.Sleep causes flakiness | `retryTimeout: TimeSpan.FromSeconds(2)` | | Issue27418 | Thread.Sleep causes flakiness | `retryTimeout: TimeSpan.FromSeconds(2)` | | Issue28657 | Thread.Sleep causes flakiness | `retryTimeout: TimeSpan.FromSeconds(2)` | | Issue30575 | Thread.Sleep causes flakiness | `retryTimeout: TimeSpan.FromSeconds(3)` | ## Related PRs (not overlapping) - #33737 - Fixes Issue18751/18896 (local monkey images) - #33689 - Fixes Issue12134 (WebView cookies) --------- Co-authored-by: Ahamed-Ali <102580874+Ahamed-Ali@users.noreply.github.com>

…otnet#33749) > [!NOTE] > Are you waiting for the changes in this PR to be merged? > It would be very helpful if you could [test the resulting artifacts](https://github.com/dotnet/maui/wiki/Testing-PR-Builds) from this PR and let us know in a comment if this change resolves your issue. Thank you! ## Description This PR improves the reliability of several flaky UI tests by: 1. **Replacing `Thread.Sleep` with `retryTimeout`** - The `retryTimeout` parameter keeps retrying screenshot comparisons until they match or timeout, adapting to actual UI timing across different machines/load conditions. 2. **Fixing Issue10563 SwipeView button animations** - Added flat `Background = SolidColorBrush(Colors.LightGray)` to buttons to prevent ripple animation artifacts that caused visual differences on CI. ## Changes | Test | Issue | Fix | |------|-------|-----| | Issue10563 | Button ripple animation artifacts in SwipeView test | Flat button background + updated baselines | | Issue18857 | Thread.Sleep causes flakiness | `retryTimeout: TimeSpan.FromSeconds(2)` | | Issue24856 | Thread.Sleep causes flakiness | `retryTimeout: TimeSpan.FromSeconds(2)` | | Issue25192 | Thread.Sleep causes flakiness | `retryTimeout: TimeSpan.FromSeconds(2)` | | Issue27418 | Thread.Sleep causes flakiness | `retryTimeout: TimeSpan.FromSeconds(2)` | | Issue28657 | Thread.Sleep causes flakiness | `retryTimeout: TimeSpan.FromSeconds(2)` | | Issue30575 | Thread.Sleep causes flakiness | `retryTimeout: TimeSpan.FromSeconds(3)` | ## Related PRs (not overlapping) - dotnet#33737 - Fixes Issue18751/18896 (local monkey images) - dotnet#33689 - Fixes Issue12134 (WebView cookies) --------- Co-authored-by: Ahamed-Ali <102580874+Ahamed-Ali@users.noreply.github.com>

jfversluis · 2026-02-17T14:28:02Z

Wrong button...

TamilarasanSF4853 added 3 commits January 23, 2026 11:53

updated test sample

b20033c

added retry

420d6b6

added changes

b15cd42

dotnet-policy-service bot added the community ✨ Community Contribution label Jan 23, 2026

dotnet-policy-service bot added the partner/syncfusion Issues / PR's with Syncfusion collaboration label Jan 23, 2026

sheiksyedm marked this pull request as ready for review January 23, 2026 13:57

Copilot AI review requested due to automatic review settings January 23, 2026 13:57

sheiksyedm added the area-testing Unit tests, device tests label Jan 23, 2026

Copilot started reviewing on behalf of sheiksyedm January 23, 2026 13:57 View session

Copilot AI reviewed Jan 23, 2026

View reviewed changes

build-analysis bot mentioned this pull request Jan 23, 2026

"We stopped hearing from agent Azure Pipelines 32. Verify the agent machine is running and has a healthy network connection" dotnet/dnceng#1886

Open

3 tasks

This was referenced Jan 26, 2026

CI UItests failure System.TimeoutException : Timed out waiting for element... #33220

Open

System.TimeoutException #33245

Open

added changes

055d7e3

PureWeen added a commit that referenced this pull request Jan 29, 2026

Remove changes covered by other PRs (#33737, #33689)

c5d25d5

- Issue18751/18896 local images: covered by #33737 - Issue12134 timeout: covered by #33689 (uses native CookieManager)

PureWeen mentioned this pull request Jan 29, 2026

[Testing] Fix flaky UI tests: retryTimeout and SwipeView button fix #33749

Merged

PureWeen requested changes Jan 29, 2026

View reviewed changes

PureWeen added a commit that referenced this pull request Jan 29, 2026

Remove changes covered by other PRs (#33737, #33689)

8de0481

- Issue18751/18896 local images: covered by #33737 - Issue12134 timeout: covered by #33689 (uses native CookieManager)

added android failing condition

bdfa677

build-analysis bot mentioned this pull request Jan 29, 2026

Project build failure causing test run abort in RunOniOS_BlazorRelease on mac_runios_blazor_release job #33744

Closed

github-actions bot pushed a commit that referenced this pull request Jan 29, 2026

Remove changes covered by other PRs (#33737, #33689)

b6349bb

- Issue18751/18896 local images: covered by #33737 - Issue12134 timeout: covered by #33689 (uses native CookieManager)

Ahamed-Ali added the testing-flakiness label Feb 17, 2026

jfversluis approved these changes Feb 17, 2026

View reviewed changes

jfversluis closed this Feb 17, 2026

jfversluis reopened this Feb 17, 2026

jfversluis merged commit fcb0d97 into dotnet:main Feb 17, 2026
139 of 149 checks passed

This was referenced Feb 18, 2026

[repo-status] Daily Repo Status - February 18, 2026 #34106

Closed

[repo-status] Daily Repo Status - Thursday, February 19, 2026 🌟 #34123

Closed

[repo-status] Daily Repo Status - Thursday, February 20, 2026 🌟 #34147

Open

	_label.Text = result.Contains(_guid.ToString(), StringComparison.OrdinalIgnoreCase) ? "Success" : "Failed";
	var success = !string.IsNullOrEmpty(result) && result.Contains(_guid.ToString(), StringComparison.OrdinalIgnoreCase);
	_label.Text = success ? "Success" : "Failed";

Comments

Conversation

TamilarasanSF4853 commented Jan 23, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

dotnet-policy-service bot commented Jan 23, 2026

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Copilot AI Jan 23, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Jan 23, 2026

Choose a reason for hiding this comment

Uh oh!

rmarinho commented Jan 26, 2026

Uh oh!

azure-pipelines bot commented Jan 26, 2026

Uh oh!

sheiksyedm commented Jan 28, 2026

Uh oh!

azure-pipelines bot commented Jan 28, 2026

Uh oh!

PureWeen left a comment

Choose a reason for hiding this comment

Uh oh!

TamilarasanSF4853 commented Jan 29, 2026

Uh oh!

sheiksyedm commented Jan 29, 2026

Uh oh!

azure-pipelines bot commented Jan 29, 2026

Uh oh!

jfversluis commented Feb 17, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

TamilarasanSF4853 commented Jan 23, 2026 •

edited

Loading