[inflight/Candidate] Fix Android Shell Flyout UI tests failure in Candidate PR #36411 by devanathan-vaithiyanathan · Pull Request #36672 · dotnet/maui

devanathan-vaithiyanathan · 2026-07-20T09:49:34Z

Issue Details

Android Shell Flyout UI Test Failures in Candidate PR #36411 due to PR #34510

Description of Change

On Android, the Shell flyout was incorrectly applying bottom padding even when the flyout didn't reach the bottom safe area. The fix checks if the view actually overlaps the bottom safe area before applying the bottom inset padding.

Issues Fixed

Fixes #36625

Test failure details

VerifyShellFlyout_HeightAndWidthWithBackgroundColor
VerifyShellFlyout_HeightAndWidthWithBackgroundImage
VerifyShellFlyout_BackgroundImageWithHeaderAndFooter
VerifyShellFlyout_Height, ShouldFlyoutTextWrapsInLandscape

azure-pipelines · 2026-07-20T09:49:51Z

Azure Pipelines: Successfully started running 1 pipeline(s). There may be pipelines that require an authorized user to comment /azp run to run.

sheiksyedm · 2026-07-20T13:09:47Z

/azp run maui-pr-uitests , maui-pr-devicetests

azure-pipelines · 2026-07-20T13:09:54Z

Azure Pipelines: Successfully started running 1 pipeline(s). There may be pipelines that require an authorized user to comment /azp run to run.

azure-pipelines · 2026-07-20T13:10:02Z

Azure Pipelines: Successfully started running 2 pipeline(s).

kubaflo · 2026-07-20T20:33:56Z

Tests Failure Analysis

@devanathan-vaithiyanathan — test-failure review results are available based on commit d943d82.

Test Failure Review: Needs human investigation - click to expand

Overall verdict: Needs human investigation

The deterministic gate caps this PR at Needs human investigation: 252 of 253 distinct failures are unattributed, 133 failed build legs produced no extractable failure, 2 red checks have no mapped AzDO evidence, 1 check was aborted, and 3 green device-test checks could not be confirmed as all-clean. No leg was proven red-on-PR/green-on-base, and 36 distinct failures were also observed on the base branch, so the current evidence does not support a PR-caused or unrelated-only conclusion.

Coverage: 175 checks · 96 passing · 79 failing · 0 pending · 0 inaccessible · 2 unmapped · 133 unexplained build legs · 0 unaccounted failing checks · 1 aborted failing checks · 0 canceled-build checks · 3 device-test unverified · 252 unattributed · 0 regressed-vs-base. Deterministic ceiling: Needs human investigation — 2 failing checks have no inspectable AzDO build evidence (pre_activation, Build Analysis); 133 failed build legs produced no extractable failure; 252 failures could not be attributed deterministically; maui-pr-uitests (macOS UITests Controls CollectionView) did not finish cleanly; and 3 green device-test checks could not be confirmed Failed == 0.

Failure	Verdict	On base?	Evidence
`pre_activation` and `Build Analysis`	Insufficient data	no AzDO evidence	Gate reports 2 unmapped failing checks with no inspectable AzDO build evidence, so these red checks cannot be responsibly classified from the gathered data.
Android SDK provisioning: `to find package 'platform-tools;35.0.2'` on macOS/android/unknown	Needs human investigation	also-red	`deterministicAttribution=indeterminate`; exact failure also appears on base, and the same legs were red on base across up to 5 sampled base builds, but this is an infrastructure/provisioning signature and not an exact dismissible known issue.
`maui-pr` integration/build/publish failures, including `PublishTestResults - build error`, Windows device-test build errors, and many RunOniOS/AOT/Blazor/WindowsTemplates legs	Needs human investigation	also-red or unknown	133 failed build legs produced no extractable failure, and baseline rows for `maui-pr` inspected only the first 8 of 84 failed build logs. The gate therefore forbids treating these as green or unrelated without manual log inspection.
Device-test failures: `ItemsUpdateWithCollectionChanges`, incomplete `com.microsoft.maui.controls.devicetests-Signed`, `Controls Tests`, and related Helix work-item failures	Needs human investigation	mostly no exact base match	`deterministicAttribution=indeterminate`; Helix work items crashed or were incomplete, and 3 green device-test checks were unverified despite XHarness exit-0 behavior. Of the completed device-test evidence, examples include 962 passed / 1 failed / 12 skipped and 785 passed / 1 failed / 20 skipped before incomplete runs.
Android Shell/Flyout UI failures, including `VerifyShellFlyout_`, `VerifyCustomFlyoutContent`, `VerifyDefaultFlyoutItemsRendering`, `ToolbarExtendsAllTheWayLeftAndRight_*`, and `LayoutShouldBeCorrectOnFirstNavigation`	Needs human investigation	mixed, several also-red	These overlap the PR's Android Shell handler/snapshot scope, but the deterministic attribution remains `indeterminate`; several same-leg samples were already red on base and some exact tests are also on base, while failures include timeouts and missing/sized visual baselines.
iOS Shell flyout layout matrix: `FlyoutHeaderContentAndFooterAllMeasureCorrectly(...)` and `FlyoutContentSetsCorrectBottomPaddingWhenMinHeightIsSetForFlyoutHeader`	Needs human investigation	no exact base match	`deterministicAttribution=indeterminate`; failures report consistent 34px footer/layout differences, but the PR changed Android Shell files and snapshots, not iOS code, and the base comparison is inconclusive rather than clean.
`CollectionViewWithRefreshViewShouldNotReset`	Likely unrelated	yes	`deterministicAttribution=pre-existing-on-base`; the exact failure is also present on the base branch with the same leg also red across sampled base builds, so this one is pre-existing.
Other UI visual, timeout, and app-crash failures across Android, iOS, macOS, and Windows controls	Needs human investigation	mixed	Most rows are `deterministicAttribution=indeterminate`; examples include SwipeView visual diffs/timeouts, Border snapshot diffs, shadow null references/app crashes, CarouselView timeouts, SafeArea timeouts, and Windows snapshot baselines. They are not proven PR-caused, but also are not safely dismissible from the gathered evidence.

Recommended action

Do not merge on this CI run. Manually inspect the unmapped red checks and the 133 unexplained failed build legs first, then rerun or re-gather after confirming the Android Shell UI-test failures and device-test Helix results; only the CollectionViewWithRefreshViewShouldNotReset row is confidently pre-existing.

Evidence details

PR scope: 3 changed files, inferred platform android, area hints Shell and Handler; changed files are src/Controls/src/Core/Compatibility/Handlers/Shell/Android/ShellFlyoutTemplatedContentRenderer.cs plus two Android Shell snapshot baselines.
PR build evidence: maui-pr build 1517761, maui-pr-uitests build 1517762, and maui-pr-devicetests build 1517763 all have failing checks.
Baseline evidence: maui-pr base build 1517909 failed with 9 extracted failures, maui-pr-devicetests base build 1517911 failed with 3 extracted failures, and maui-pr-uitests base build 1517910 failed with 70 extracted failures. All three baseline rows are incomplete because only the first 8 failed build logs were inspected.
Gate evidence: 253 distinct PR failures; 36 also on base; 0 known-issue matches; 0 retried-still-failing; 0 deterministic regressed-vs-base; 252 unattributed; 133 unexplained failed build legs; 2 unmapped failing checks; 1 aborted failing check; 3 unverified green device-test checks.
Known issue and ci-scan evidence: known-issue query succeeded with 1 matcher and 0 matches; ci-scan query succeeded for branch family inflight/candidate with 47 matchers, 0 matches, and 0 demotions.
Visual evidence limitation: visual comparison discovery inspected only the first 100 of 201 failed test results. Visual panels and image links are intentionally omitted here for the local runner to merge separately.
MAUI CI limitation: green device-test checks are not trusted unless Failed == 0 is positively confirmed because XHarness can exit 0 even when device tests fail; the gate reports 3 such checks as unverified.

Visual failure comparisons

Full-resolution CI baseline, actual, and diff images are embedded below. They supplement the failure classification and do not change the deterministic verdict ceiling.
Relationship labels use deterministic exact test-and-platform baseline evidence plus exact changed snapshot/test scope; missing or mixed evidence remains Needs human investigation.

VerifyBorderWithStrokeDashArrayValue - android - Needs human investigation - visual comparison

CI reported 9.55% difference in build 1517762.

Relationship to PR: Needs human investigation - No decisive exact test-and-platform baseline attribution was available.

CI baseline	Fresh PR actual	CI diff

VerifyBorderWithNullStrokeDashArray - android - Needs human investigation - visual comparison

CI reported 11.03% difference in build 1517762.

Relationship to PR: Needs human investigation - No decisive exact test-and-platform baseline attribution was available.

CI baseline	Fresh PR actual	CI diff

VerifyBorderWithStrokeDashArrayValue - windows - Needs human investigation - visual comparison

CI reported 2.99% difference in build 1517762.

Relationship to PR: Needs human investigation - No decisive exact test-and-platform baseline attribution was available.

CI baseline	Fresh PR actual	CI diff

VerifyBorderWithNullStrokeDashArray - windows - Needs human investigation - visual comparison

CI reported 3.17% difference in build 1517762.

Relationship to PR: Needs human investigation - No decisive exact test-and-platform baseline attribution was available.

CI baseline	Fresh PR actual	CI diff

ShouldFlyoutTextWrapsInLandscape - android - Needs human investigation - visual comparison

CI reported 7.96% difference in build 1517762.

Relationship to PR: Needs human investigation - No decisive exact test-and-platform baseline attribution was available.

CI baseline	Fresh PR actual	CI diff
Baseline unavailable: ambiguous across multiple snapshot environments

VerifySwipeViewWithLabelContentAndThreshold - android - Needs human investigation - visual comparison

CI reported 0.68% difference in build 1517762.

Relationship to PR: Needs human investigation - No decisive exact test-and-platform baseline attribution was available.

CI baseline	Fresh PR actual	CI diff

VerifySwipeViewWithImageContentAndThreshold - android - Needs human investigation - visual comparison

CI reported 0.68% difference in build 1517762.

Relationship to PR: Needs human investigation - No decisive exact test-and-platform baseline attribution was available.

CI baseline	Fresh PR actual	CI diff

Material3Editor_VerifyzEditorTextWhenAutoSizeTextChangesSet - android - Needs human investigation - visual comparison

CI reported 0.91% difference in build 1517762.

Relationship to PR: Needs human investigation - No decisive exact test-and-platform baseline attribution was available.

CI baseline	Fresh PR actual	CI diff

VerifySwipeViewWithCollectionViewContentAndThreshold - android - Needs human investigation - visual comparison

CI reported 0.68% difference in build 1517762.

Relationship to PR: Needs human investigation - No decisive exact test-and-platform baseline attribution was available.

CI baseline	Fresh PR actual	CI diff

Material3Editor_VerifyzEditorPlaceholderWithAutoSizeTextChanges - android - Needs human investigation - visual comparison

CI reported 1.38% difference in build 1517762.

Relationship to PR: Needs human investigation - No decisive exact test-and-platform baseline attribution was available.

CI baseline	Fresh PR actual	CI diff

Material3Editor_VerifyzEditorPlaceholderWithAutoSizeDiabled - android - Needs human investigation - visual comparison

CI reported 0.75% difference in build 1517762.

Relationship to PR: Needs human investigation - No decisive exact test-and-platform baseline attribution was available.

CI baseline	Fresh PR actual	CI diff

ItemImageSourceShouldBeVisible - ios - Needs human investigation - visual comparison

CI reported 1.71% difference in build 1517762.

Relationship to PR: Needs human investigation - No decisive exact test-and-platform baseline attribution was available.

CI baseline	Fresh PR actual	CI diff
Baseline unavailable: ambiguous across multiple snapshot environments

VerifyEditorTextWhenFontAttributesSet - windows - Needs human investigation - visual comparison

CI reported 0.60% difference in build 1517762.

Relationship to PR: Needs human investigation - No decisive exact test-and-platform baseline attribution was available.

CI baseline	Fresh PR actual	CI diff

Material3Editor_VerifyEditorPlaceholderWithShadow - android - Needs human investigation - visual comparison

CI reported 0.58% difference in build 1517762.

Relationship to PR: Needs human investigation - No decisive exact test-and-platform baseline attribution was available.

CI baseline	Fresh PR actual	CI diff

Visual output was bounded for comment safety; 9 additional comparison(s) were omitted.

kubaflo

Could you please check if test failures are related?

devanathan-vaithiyanathan · 2026-07-21T08:07:56Z

Could you please check if test failures are related?

@kubaflo , I have ensured the test locally, failures are not related to my fix.

> [!NOTE] > Are you waiting for the changes in this PR to be merged? > It would be very helpful if you could [test the resulting artifacts](https://github.com/dotnet/maui/wiki/Testing-PR-Builds) from this PR and let us know in a comment if this change resolves your issue. Thank you! ### Description of Change Automates visual snapshot evidence in `/review tests`. When public AzDO results contain failed screenshot comparisons, the command now emits exactly one test-failure analysis comment containing bounded, expandable baseline/actual/diff panels. Visual evidence remains supplementary: it does not change `gate.verdictCeiling`, deterministic attribution, or the merge-readiness verdict. #### One-comment flow 1. Trusted pre-activation code discovers failed visual results through the public AzDO `resultsbybuild` API, including retry-suffixed attachments such as `Snapshot[1].png` and `Snapshot-diff[1].png`. 2. It resolves baselines from the exact source version tested by AzDO and maps runtime evidence to the correct snapshot directory (`ios-26`, `android-notch-36`, `mac`, or `windows`). 3. It streams and validates bounded PNG assets, then stores them on `review-tests-assets` using immutable commit-pinned `raw.githubusercontent.com` URLs. 4. The Copilot agent emits the normal single `add_comment` analysis payload with a trusted insertion marker. 5. A sealed post-step validates the published asset manifest and injects as many expandable comparison panels as fit into that same comment. Each collapsed panel shows a conservative relationship label: - `Likely PR-caused` for an exact test/platform base regression or directly changed snapshot/test; - `Likely unrelated` for an exact base/known-issue match without direct visual scope; - `Needs human investigation` for unmatched or mixed evidence. 6. Excess comparisons are summarized as omitted instead of creating a companion comment. The local `.github/scripts/Review-Tests.ps1 -PostComment` path uses the same merger. It also recognizes complete reports returned in Copilot's final response, preserving nested evidence code fences without wrapping a second title or badge section. ### Security and Failure Safety - PR text, logs, test names, attachment metadata, changed files, and visual labels remain untrusted input. - The merger script and visual context are copied to a root-owned location before the workflow checks out the untrusted PR branch. - The post-step runs without `COPILOT_GITHUB_TOKEN`, `GH_TOKEN`, or `GITHUB_TOKEN`. - AzDO attachment URLs must match the expected public project and attachment route. - Published assets are size-bounded, signature-checked PNGs with validated dimensions and repository paths. - Raw image URLs must match the exact repository, asset commit, PR directory, and safe filename. - Labels are HTML-escaped and `@` is neutralized before insertion. - Relationship labels use fixed trusted text. Untrusted attribution values are never rendered, and same-named snapshots changed on another platform do not count as PR scope. - The final body is checked panel-by-panel against conservative limits of 45 URLs, 10 mentions, and 60,000 UTF-16 characters, below gh-aw's throwing limits. - The analysis JSON update is atomic (written to a temp file, then renamed over the original). Invalid context, malformed output, missing analysis payloads, limit failures, and dry-run/noop output leave the original analysis unchanged. - The publisher never creates or patches PR comments; only the existing gh-aw `add_comment` payload is mutated. ### What NOT to Do - Do not use the ordinary anonymous AzDO test-runs listing for discovery; it redirects to sign-in. Use the public failed-results endpoint. - Do not resolve baselines from the current PR head; use the source version actually tested by the selected AzDO build. - Do not let the agent construct or trust visual asset URLs. - Do not publish a second companion comment; merge bounded panels into the single analysis payload. ### Validation - 98 focused Pester tests pass. - Changed PowerShell scripts parse successfully. - `gh aw compile copilot-review-tests --approve` completes without errors or warnings. - A real `agent_output.json` from gh-aw run [29674953402](https://github.com/dotnet/maui/actions/runs/29674953402) was replayed through the post-step: - one `add_comment` item remained one item; - five visual panels were inserted; - the final body contained 26 URLs, one mention, and 9,585 characters. ### Live Single-Comment Examples The exact local `/review tests` path from this branch posted or repaired these merged comments after the PRs' `/azp run` pipelines completed: | PR | Single merged result | Included evidence | Relationship labels | Final limits | | --- | --- | --- | --- | --- | | #36413 | [Test-failure analysis with visual panels](#36413 (comment)) | 5 panels / 15 images | 1 PR-caused, 4 investigate | 23 URLs, 13,156 chars | | #36631 | [Test-failure analysis with visual panels](#36631 (comment)) | 6 panels / 18 images | 2 PR-caused, 4 investigate | 31 URLs, 21,367 chars | | #36395 | [Test-failure analysis with visual panels](#36395 (comment)) | 11 panels / 33 images; 19 omitted | 11 investigate | 43 URLs, 19,858 chars | | #36404 | [Test-failure analysis with visual panels](#36404 (comment)) | 14 panels / 40 images; 81 omitted | 14 investigate | 45 URLs, 23,779 chars | | #35846 | [Test-failure analysis with visual panels](#35846 (comment)) | 10 panels / 30 images; 9 omitted | 10 investigate | 43 URLs, 22,496 chars | | #36277 | [Test-failure analysis with visual panels](#36277 (comment)) | 7 panels / 19 images | 3 PR-caused, 4 investigate | 31 URLs, 18,355 chars | | #36170 | [Test-failure analysis with visual panels](#36170 (comment)) | 11 panels / 33 images; 8 omitted | 11 investigate | 44 URLs, 23,180 chars | | #35578 | [Test-failure analysis with visual panels](#35578 (comment)) | 12 panels / 36 images; 50 omitted | 12 investigate | 44 URLs, 25,443 chars | | #36672 | [Test-failure analysis with visual panels](#36672 (comment)) | 14 panels / 40 images; 9 omitted | 14 investigate | 45 URLs, 25,915 chars | | #31755 | [Test-failure analysis with visual panels](#31755 (comment)) | 12 panels / 36 images; 3 omitted | 12 investigate | 44 URLs, 22,728 chars | | #34637 | [Test-failure analysis with visual panels](#34637 (comment)) | 9 panels / 27 images; 81 omitted | 9 investigate | 43 URLs, 22,325 chars | | #35156 | [Test-failure analysis with visual panels](#35156 (comment)) | 2 panels / 6 images | 1 PR-caused, 1 investigate | 19 URLs, 11,911 chars | | #35885 | [Test-failure analysis with no visual failures](#35885 (comment)) | 0 panels / 0 images | No visual failures detected | 8 URLs, 4,239 chars | | #36577 | [Test-failure analysis with visual panels](#36577 (comment)) | 1 panel / 3 images | 1 investigate | 44 URLs, 21,768 chars | | #36212 | [Test-failure analysis with no visual failures](#36212 (comment)) | 0 panels / 0 images | No visual failures detected | 5 URLs, 5,982 chars | Each result contains one `Tests Failure Analysis` title and one merged review marker. Across 114 rendered panels, all 336 embedded image URLs returned HTTP 200. Seven panels were safely classified as likely PR-caused; no panel in this sample had enough exact evidence to be safely classified as likely unrelated, so the remaining 107 stayed at `Needs human investigation`. Another 260 comparisons were omitted safely by the comment limits. The latest eight-example batch was regenerated concurrently, and #36672, #31755, #34637, #35156, #35885, #36577, and #36212 were added afterward. The current [`review-tests-assets` head](f937993) retains the full asset history. The protected `copilot-pat-pool` environment rejects feature-branch `workflow_dispatch` runs before job execution. The live local-runner examples validate comment generation and asset publication, while the real gh-aw output replay validates the workflow post-step mutation without weakening that branch protection. ### Issues Fixed N/A - reviewer workflow enhancement. --------- Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Co-authored-by: kubaflo <kubaflo@users.noreply.github.com> Copilot-Session: a280b482-e102-4ca0-9ff9-1cfe1946e21f Copilot-Session: d00747b7-96f3-4e7a-8dfb-e3a48db04b2d Copilot-Session: 478d195b-20f3-4bc6-aeed-f6aa88b55fda

devanathan-vaithiyanathan added 2 commits July 16, 2026 19:48

Update ShellFlyoutTemplatedContentRenderer.cs

1aae26a

fix updated

d943d82

devanathan-vaithiyanathan had a problem deploying to copilot-pat-pool July 20, 2026 09:49 — with GitHub Actions Failure

devanathan-vaithiyanathan changed the title ~~Fix Android Shell Flyout UI tests failure in Candidate PR #36411~~ [inflight/Candidate] Fix Android Shell Flyout UI tests failure in Candidate PR #36411 Jul 20, 2026

dotnet-policy-service Bot added the partner/syncfusion Issues / PR's with Syncfusion collaboration label Jul 20, 2026

devanathan-vaithiyanathan added the community ✨ Community Contribution label Jul 20, 2026

sheiksyedm marked this pull request as ready for review July 20, 2026 13:09

This was referenced Jul 20, 2026

slow macOS - "##[error]The job running on agent Azure Pipelines 9 ran longer than the maximum time of 60 minutes." dotnet/dnceng#1883

Open

The Operation will be canceled. The next steps may not contain expected logs. dotnet/dnceng#3008

Open

kubaflo added a commit that referenced this pull request Jul 20, 2026

[skip ci] Store /review tests visuals for PR #36672 at d943d82

cb2ee30

kubaflo mentioned this pull request Jul 20, 2026

Automate visual failure comparisons in /review tests #36666

Merged

kubaflo requested changes Jul 20, 2026

View reviewed changes

github-actions Bot mentioned this pull request Jul 21, 2026

[PR Review Queue] 2026-07-21 #36696

Closed

kubaflo merged commit b58b73a into dotnet:inflight/candidate Jul 21, 2026
96 of 175 checks passed

github-actions Bot added this to the .NET 10 SR10 milestone Jul 21, 2026

Navigation Menu

Uh oh!

[inflight/Candidate] Fix Android Shell Flyout UI tests failure in Candidate PR #36411 - #36672

[inflight/Candidate] Fix Android Shell Flyout UI tests failure in Candidate PR #36411#36672
kubaflo merged 2 commits into
dotnet:inflight/candidatefrom
devanathan-vaithiyanathan:fix-34510-candidate-failure

devanathan-vaithiyanathan commented Jul 20, 2026 •

edited

Loading

Uh oh!

azure-pipelines Bot commented Jul 20, 2026

Uh oh!

sheiksyedm commented Jul 20, 2026

Uh oh!

azure-pipelines Bot commented Jul 20, 2026

Uh oh!

azure-pipelines Bot commented Jul 20, 2026

Uh oh!

kubaflo commented Jul 20, 2026

Recommended action

Visual failure comparisons

Uh oh!

kubaflo left a comment

Uh oh!

devanathan-vaithiyanathan commented Jul 21, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Uh oh!

Conversation

devanathan-vaithiyanathan commented Jul 20, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Issue Details

Description of Change

Issues Fixed

Test failure details

Uh oh!

azure-pipelines Bot commented Jul 20, 2026

Uh oh!

sheiksyedm commented Jul 20, 2026

Uh oh!

azure-pipelines Bot commented Jul 20, 2026

Uh oh!

azure-pipelines Bot commented Jul 20, 2026

Uh oh!

kubaflo commented Jul 20, 2026

Tests Failure Analysis

Recommended action

Visual failure comparisons

Uh oh!

kubaflo left a comment

Choose a reason for hiding this comment

Uh oh!

devanathan-vaithiyanathan commented Jul 21, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

devanathan-vaithiyanathan commented Jul 20, 2026 •

edited

Loading