[Testing] Feature Matrix UITest Cases for Shell SearchHandler - #36631
Merged
kubaflo merged 20 commits intoJul 20, 2026
Conversation
…dotnet#36461) > [!NOTE] > Are you waiting for the changes in this PR to be merged? > It would be very helpful if you could [test the resulting artifacts](https://github.com/dotnet/maui/wiki/Testing-PR-Builds) from this PR and let us know in a comment if this change resolves your issue. Thank you! ## What this changes Follow-up to dotnet#36317 (the self-watching `ci-status-fix` loop). Today, when a `[ci-fix]` draft PR's CI comes back green — or red only on **unrelated** flakes — the loop posts a comment but **leaves the PR as a draft forever**. A maintainer has to notice it, confirm the specific fixed test actually passed, and flip it to ready. In practice these validated-green draft PRs sit unreviewed for weeks. This PR adds a **target-test verification + mark-ready gate (Step 3.6)** to both twins (`main` + `net11.0`). When a draft `[ci-fix]` / `[ci-fix-net11]` PR reaches the green-surface or unrelated-flake branch, the loop now drills into the PR's **own** AzDO test-results for the **specific test(s)** the fix targeted. If every target test is `Passed` on ≥1 leg and `Failed` on none (VALIDATED-GREEN), it: - posts a `🎯 Target test validated green on <headSha>` comment (naming the test + legs + buildId), and - transitions the draft PR to **ready for review**. This is a **state transition only** — it never approves and never merges; a human still reviews and merges. Overall red on **unrelated** legs no longer keeps a validated fix parked as a draft. ## How it works - **Preconditions** (all required): PR `isDraft == true`; it is unmistakably this workflow's own PR (`[ci-fix]`/`[ci-fix-net11]` title prefix **and** `agentic-workflows` label); and it was reached from the green or **unrelated-flake** path (never from caused-by-fix — that path advances an attempt instead). - **T1 — identify target test(s)** from the `[ci-scan]` issue signature + the PR diff. If no specific test can be identified (e.g. a product build-break), it records a skip — build-only fixes are validated by overall-green, which the existing green branch already handles. - **T2 — drill AzDO test-results** for the build(s) on the PR's current head SHA, filtered to the target test's `testCaseTitle`. A test that never ran (e.g. an `/azp`-gated `maui-pr-uitests`/`maui-pr-devicetests` leg that wasn't kicked) is **not** validated — the loop records an honest "not yet executed" skip and does **not** mark ready. No overclaiming: a green *sibling* leg is not the target test. - **T3 — mark ready + report**, guarded by a per-head-SHA idempotency marker and the existing `dry_run` gate (dry-run emits nothing and tallies `would-mark-ready`). ## Safe-output Adds the `mark-pull-request-as-ready-for-review` safe-output to both twins (`max: 3`, `target: "*"`, `required-title-prefix` + `required-labels`). Unlike `update-pull-request`, this output's `required-*` guards **do** survive the gh-aw v0.80.9 compile (verified against the generated locks), so which-PR scoping is enforced at the handler level in addition to the Step 3.6 preconditions and `min-integrity: approved`. No gh-aw version bump is required — the capability already exists at our pinned v0.80.9. ## Enabling fix — the loop's own create-PR commit no longer counts as "human engaged" While validating this feature against dotnet#36429 I found the mark-ready path was **unreachable for every loop-owned draft PR**, and traced it to a regression from dotnet#36317's own review-hardening. gh-aw's `create_pull_request` builds a PR's initial commit through the GitHub API, which stamps `author=github-actions[bot]` but **`committer=web-flow`**. Commit `2f6b77b330` (in dotnet#36317) removed `web-flow` from the prefetch's bot-login denylist so a maintainer "Update branch" would correctly hand the PR off — but that also made `Test-AnyHumanCommitActor`'s committer check read the loop's **own** first commit as human engagement. Result: every freshly opened `[ci-fix]` draft PR computed `humanEngaged=true` from commit #1, so the watch loop skipped it forever — never surfacing green, never marking ready. This regression is live on `main` today (all four open loop-owned draft PRs have `committer=web-flow`). Fix (`Query-CiFixPRs.ps1`): a human **committer** only trips the hand-off when the commit **author** is not one of this workflow's own bot identities (`$LoopBotCommitAuthors`). A human **author** still counts unconditionally, so maintainer direct commits and web-flow-authored "Update branch" merges continue to hand off correctly. Unit-tested across all six author/committer shapes. ## Validation - Both twins recompiled with `gh aw compile` (0 errors / 0 warnings); locks show no action-SHA or `compiler_version` drift (only frontmatter/body hash + the new handler config). - `mark_pull_request_as_ready_for_review` config confirmed present in **both** locks with the correct `[ci-fix] ` / `[ci-fix-net11] ` prefixes and `agentic-workflows` label; the `safe_outputs` job carries `pull-requests: write`. - Twin symmetry preserved (only `[ci-fix]` ↔ `[ci-fix-net11]` / `ci-scan` ↔ `ci-scan-net11` token differences). - **Live dry-run against dotnet#36429** (scoped `workflow_dispatch`, `dry_run=true`, this branch): the workflow prefetch now computes `humanEngaged=false` for dotnet#36429 (was `true` pre-fix), and the agent's gate walk correctly advances **past** the human-engaged gate to the CI-pending WAIT gate — `checksSettled=false` because dotnet#36429's macOS `SafeAreaEdges` leg is still queued. Once that leg settles green, the same run path reaches Step 3.6 and marks the PR ready. Emitted zero writes (dry-run). Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> --------- Co-authored-by: PureWeen <223556219+Copilot@users.noreply.github.com>
…le grouped comments (dotnet#36533) <!-- Please let the below note in for people that find this PR --> > [!NOTE] > Are you waiting for the changes in this PR to be merged? > It would be very helpful if you could [test the resulting artifacts](https://github.com/dotnet/maui/wiki/Testing-PR-Builds) from this PR and let us know in a comment if this change resolves your issue. Thank you! ### What this changes The `/review tests` **Test Failure Analysis** workflow classifies whether a PR's CI failures are PR-caused or unrelated. Its deterministic gate computed `legsRegressedVsBase` — the "Likely PR-caused" headline — by comparing each red PR leg against **a single most-recent base build**. Flaky UI-test legs that happened to be green on that one base build produced false **"regressed-vs-base / Likely PR-caused"** positives, so the generated comments carried little signal. This PR makes the regression diff sample **several recent base builds of the PR's own base branch** (`main` or `net11.0`) and only calls a leg a regression when it is green across enough of them and red on none — and switches the LLM engine to Opus. It also **replaces the per-test result table with a skimmable, root-cause-grouped bullet list** (the deep-UI-test-analysis style) so a run with hundreds of failures collapses to a handful of readable groups inside the existing compact badge + collapsible header instead of a giant table. ### Changes - **`Gather-TestFailureContext.ps1`** — new `Get-AggregatedBaseLegMap` aggregator + params `RegressionBaseBuilds=5` and `MinBaseGreenSamples=2`. A leg is a regression only if it is green on **≥ `MinBaseGreenSamples`** base builds **and red on NONE**. Per base build, a leg that failed even one attempt counts RED (a retry that later passed does not clear a base flake). New leg states `flaky-on-base` / `succeeded-on-base-unconfirmed` and new per-failure fields `baseSampleCount` / `baseGreenCount` / `baseFailedCount`. - **Asymmetric conservatism (preserved)** — multi-build sampling widens **only** the false-RED (assert-a-regression) side. The test-level dismissal / false-GREEN side stays single-build-strict on purpose, so this change never turns a real failure green. - **`copilot-review-tests.md` / `.lock.yml`** — engine model `claude-sonnet-4.6` → `claude-opus-4.8`; **replaced only the per-test Markdown table** (an unreadable wall of text on runs with hundreds of failures) with a deep-UI-test-analysis style **root-cause-grouped** bullet list using subtle tokens (`✗ PR-related` / `ℹ Uncertain` / `● Unrelated`). The compact **badge row and the `Test Failure Review: [verdict] - click to expand` collapsible are kept** so the collapsed comment stays a one-glance summary; the badge set now surfaces the key multi-build signal — `Overall` / `Failures` / **`Regressed vs base`** (replacing the old `Platform` badge) / `Baseline on base`. Everything detailed (verdict sentence, grouped bullets, coverage counts, this-PR + base-sampling build links, recommended action) lives inside the collapsible. `img.shields.io` stays in the network allowlist. Lock recompiled (gh-aw v0.80.9). - **`SKILL.md` / `maui-ci-facts.md`** — document the multi-build sampling, the new states/fields, and soften the "green on base is proof" wording to require green across several base builds and red on none. ### Validation Ran the gatherer locally against **PR dotnet#36478** (base `main`), before vs after: | | `legsRegressedVsBase` | Verdict | |---|---|---| | **Before** (single base build) | **14** | Not ready | | **After** (5 base builds) | **0** | Needs human investigation | The `ValidateDynamic*` (×9), `CollectionViewInfiniteScroll`, and `Issue17400` legs — each green on 4/5 base builds and red on 1/5 — are now correctly classified as `flaky-on-base` instead of PR regressions. ### Notes - Extracted as a focused, standalone change from the broader `improved-reviewer` work (dotnet#36473). - CI-tooling-only change (`.github/`); no framework/runtime code is touched. --------- Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
… twin) (dotnet#36601) > [!NOTE] > Are you waiting for the changes in this PR to be merged? > It would be very helpful if you could [test the resulting artifacts](https://github.com/dotnet/maui/wiki/Testing-PR-Builds) from this PR and let us know in a comment if this change resolves your issue. Thank you! ## Problem The **net11 twin** of the CI Failure Fixer (`ci-status-fix-net11`) has been failing in its `safe_outputs` job on most scheduled runs since ~2026-07-10 (red on 07-10, 07-12, 07-13, 07-14, and 07-15 ×2). Every failure is the same error: ``` Cannot create pull request: patch modifies files outside the allowed-files list (src/AI/tests/Essentials.AI.UnitTests/...) ``` `net11.0` gained a new **`src/AI/`** source tree (Essentials.AI / streaming-JSON) that **does not exist on `main`** (it 404s there). The agent repeatedly finds a flaky `Essentials.AI.UnitTests` test, builds a valid fix, and emits `create_pull_request` — but `src/AI/**` was never added to the enforced `allowed-files` allowlist. The handler rejects the patch, so: - the whole scheduled run goes **red** every time, - that AI-test flake can **never** be filed, and - one create-PR slot is **burned** each sweep. The `main` twin is unaffected because there is no `src/AI/` tree on `main`. ## Fix Add `src/AI/**` to **both** `allowed-files` blocks (`create-pull-request` and `push-to-pull-request-branch`) in **both** twins: - On **net11.0** it unblocks the failing PR creation. - On **main** it is a harmless no-op (no `src/AI/` tree exists there to match) and keeps the two workflow files byte-identical per the established mirror invariant — future-proofing a forward-port of the AI tree. Both `.lock.yml` files were recompiled with `gh aw compile` (0 errors / 0 warnings). The only substantive lock delta is the prepended `"src/AI/**"` entry in the two `allowed_files` arrays (plus the metadata hash line). No other handler config changed — `max` values, `add_labels allowed:[p/0]`, `mark_pull_request_as_ready_for_review`, `protected_files`, `base_branch`, and `title_prefix` are all unchanged. ## Verification - `gh aw compile ci-status-fix` and `gh aw compile ci-status-fix-net11` → **0 errors / 0 warnings** each. - Twin-symmetry invariant preserved (transformed-net11 vs main diff = **124 lines**, the pre-existing cosmetic delta — unchanged by this PR). - `gh aw lint` → no lint issues. - Confirmed via lock diff that the sole config change is `"src/AI/**"` prepended to the two `allowed_files` arrays in each twin. Co-authored-by: PureWeen <223556219+Copilot@users.noreply.github.com>
…Copilot follow-ups from dotnet#36213) (dotnet#36483) <!-- Please let the below note in for people that find this PR --> > [!NOTE] > Are you waiting for the changes in this PR to be merged? > It would be very helpful if you could [test the resulting artifacts](https://github.com/dotnet/maui/wiki/Testing-PR-Builds) from this PR and let us know in a comment if this change resolves your issue. Thank you! ### What Two small follow-up fixes to the release-readiness reporting skill, closing two **low-severity** edge cases that the GitHub Copilot reviewer flagged on dotnet#36213 and that shipped into `main`. Both are docs/skill-only (PowerShell + tests) — no product code, no public API. **1. `Test-PluginEnabled` — minified `settings.json` false negative** `.github/skills/dependency-flow/scripts/Get-PreviewReleaseReadiness.ps1` The enabled-plugin matcher was anchored to the start of a physical line (`(?m)^\s*`). A **minified / single-line** `settings.json` (e.g. `{"enabledPlugins":{"dotnet-release-tracker@dotnet-release":true}}`) therefore failed to match, so an *enabled* plugin was reported as **not** enabled (a false-negative that wrongly degrades to `AVAILABLE_NOT_ENABLED`). It fails safe — it never produces a false *enabled* — but it's still wrong for anyone whose settings file isn't pretty-printed. Fix: anchor the key to a JSON boundary (`{`, `,`, or whitespace) via a look-behind `(?<=[{,\s])` instead of a line start. Comment-avoidance is already handled by the string-aware `Remove-JsoncComments` scrub applied just below, so the line anchor was redundant. **2. `Test-IsSdkBumpPr` — `dotnet-optimization` collision** `.github/skills/release-readiness/scripts/Get-PreviewReadiness.ps1` `'(?i)\bBump\b.*dotnet/(dotnet|sdk)\b'` — the trailing `\b` sits between `t` and `-`, so `Bump dotnet/dotnet-optimization …` was misclassified as an SDK/VMR bump (which would attach a spurious "verify blessed build locally" emphasis). Fix: use the `(?![\w-])` boundary that its sibling matchers already use (`selectPin`, `Get-ComponentFlowSignal`). Practically dormant on maui today (real dep-flow PRs are titled `[netN.0] Update dependencies from…`), but now correct. ### Tests Added hermetic regression guards in `Test-ReleaseReadiness.ps1`: - `Test-PluginEnabled`: minified, pretty, suffix-only-key (no false positive), and absent-entry cases (writes fixtures into a throwaway `HOME`/`USERPROFILE`, restored in `finally`; no `gh`/network). - `Test-IsSdkBumpPr`: `dotnet/dotnet-optimization` does **not** collide → `false`; a real `dotnet/sdk` later in the same title still → `true`. This mirrors the `Get-ComponentFlowSignal` collision guard that already existed — the sibling matcher just never got the parallel assertion (the exact gap this closes). Suite: **853 passed / 0 failed** (`-SkipE2E`). ### Why low-risk Skill/tooling only. Fix 1 only ever *widens* a previously-too-narrow match and still can't produce a false enable; Fix 2 only *narrows* an over-broad match to exclude a hyphenated sibling. Both are covered by new tests that fail against the old patterns. Co-authored-by: PureWeen <223556219+Copilot@users.noreply.github.com>
<!-- Please let the below note in for people that find this PR --> > [!NOTE] > Are you waiting for the changes in this PR to be merged? > It would be very helpful if you could [test the resulting artifacts](https://github.com/dotnet/maui/wiki/Testing-PR-Builds) from this PR and let us know in a comment if this change resolves your issue. Thank you! ## Summary The live manual validation of Action Pin Refresh ([run 29052735587](https://github.com/dotnet/maui/actions/runs/29052735587)) correctly produced a no-op: the runner had gh-aw v0.81.6 preinstalled, but the v0.80.9-compiled workflow needed its matching CLI. `gh extension remove gh-aw` could not authenticate because the agent had only the Copilot inference PAT, and direct binary download was blocked. This follow-up supplies the agent's `GH_TOKEN` from the workflow's existing read-only `${{ github.token }}` permissions, leaving the Copilot PAT limited to inference. It also reads and validates the required gh-aw version from `aw-actions-update.lock.yml` metadata, avoiding manual pin drift when the dedicated version-updater changes the compiler version. ## Scope - `.github/workflows/aw-actions-update.md` - Regenerated `.github/workflows/aw-actions-update.lock.yml` (compiled with gh-aw v0.80.9) The safe-output remains constrained to `.github/aw/actions-lock.json`; permissions and write scope are unchanged. ## Validation - `gh aw compile aw-actions-update` - gh-aw workflow security scanner: `aw-actions-update.md` passed - `gh aw lint .github/workflows/aw-actions-update.lock.yml` reported no lint issues; its local actionlint integration exited 125 due to a tooling error. Co-authored-by: PureWeen <223556219+Copilot@users.noreply.github.com>
…ry inline-findings.json write (dotnet#36002) > [!NOTE] > Are you waiting for the changes in this PR to be merged? > It would be very helpful if you could [test the resulting artifacts](https://github.com/dotnet/maui/wiki/Testing-PR-Builds) from this PR and let us know in a comment if this change resolves your issue. Thank you! ## What & why Fixes to the Copilot PR-reviewer pipeline, driven by analyzing **64 recent `maui-copilot` PR-reviewer CI sessions** (most-recent run per PR, last 7 days). Three independent analysis agents mined disjoint batches and **converged on the same top two systemic issues** — strong signal these are structural, not anecdotal. Fixes #1 and dotnet#2 are **prompt/instruction-only**; the PR additionally ships **regression-guard evals** for both, plus a **token-auth migration** of the eval harness workflow (see Scope & safety). ### 1. `gh` CLI is unauthenticated by design, but the prompt doesn't say so — ~61/64 sessions The `CopilotReview` task intentionally strips all GitHub tokens (`copilot ... --secret-env-vars=GH_TOKEN,COPILOT_GITHUB_TOKEN,GITHUB_TOKEN`), so `gh pr view` / `gh issue view` / `gh api` always fail inside the agent run. But `pr-preflight.md` still *leads* with those commands, so nearly every session burns 1-3 turns watching them fail and pivoting — and several then **wrongly log an "environment blocker" or lower review confidence**. **Fix:** `pr-preflight.md` now opens with an explicit "Environment & Authentication" note stating that unauthenticated `gh` is **expected, not a blocker** (don't stop, don't reduce confidence), and provides **local-`git` + unauthenticated public-REST (`curl`)** recipes that work in CI. The original `gh` recipes are retained for local `pr-review` runs where a token is present. ### 2. Agent refuses to write the REQUIRED `inline-findings.json` — ~34/64 sessions The expert-review step over-generalizes a host "do not write output files" guardrail and refuses to write `inline-findings.json`, dumping the JSON into chat instead. There is **no fallback** — `post-inline-review.ps1` finds no file (`Test-Path` fails) and **inline review comments are silently dropped** in roughly half of sessions. This is the most damaging correctness issue found. **Fix:** the STEP 5b prompt in `Review-PR.ps1` and the `maui-expert-reviewer` agent now **explicitly authorize and mandate** writing the file, state that the general "don't write review output" guidance does **not** apply to this required artifact, and forbid substituting a chat-text dump (with an orchestrator-writes-it-itself fallback instruction). ## Files - `.github/pr-review/pr-preflight.md` — environment note + local-first / `curl` recipes (#1) - `.github/scripts/Review-PR.ps1` — strengthened STEP 5b inline-findings mandate (dotnet#2, prompt string only) - `.github/agents/maui-expert-reviewer.md` — reinforce required file write, no chat-dump (dotnet#2) - `.github/skills/pr-review/tests/eval.gh-auth.vally.yaml` — **new** regression guard for #1 - `.github/skills/code-review/tests/eval.inline-findings.vally.yaml` — **new** regression guard for dotnet#2 - `.github/workflows/skill-validation.yml` — **token-auth migration** for the eval harness (see Scope & safety) ## Scope & safety - **Fixes #1 and dotnet#2 are prompt/instruction text only.** No pipeline control-flow or security-boundary changes — the #1 note merely *documents* the existing token-stripping (reinforces it, never weakens it). `Review-PR.ps1`'s change is confined to a here-string prompt; `pwsh` parse check passes and `$PRNumber` interpolation is preserved. - **`skill-validation.yml` is a token-provisioning migration (not prompt text).** The prior `COPILOT_GITHUB_TOKEN*` secrets were rotated out with the removal of `gh-aw-agents`, so the eval harness now sources Copilot auth from the **`copilot-pat-pool`** environment (`COPILOT_PAT_0..9`, index/run-id modulo selection, masked before `GITHUB_OUTPUT`). This is a **like-for-like** auth-source swap — same `pull_request_target` exposure model as before, no new secret exposure and no security loosening.⚠️ **Merge prerequisite:** the `copilot-pat-pool` environment must be populated with `COPILOT_PAT_0..9` (and must NOT carry required-reviewer/wait-timer protection rules, which would hang the `pull_request_target` job) so eval goes green immediately after merge. ## Evals (regression guards for these fixes) Now that we run [`vally`](https://www.npmjs.com/package/@microsoft/vally-cli) eval suites, each fix ships with a guard that reproduces the exact failure mode it fixes. Both pass `vally lint --strict` and run on this PR (each lives under its skill's `tests/` dir, which flags that skill as changed). - `.github/skills/pr-review/tests/eval.gh-auth.vally.yaml` (#1) — the vally eval step is already tokenless, so it **natively reproduces** the CI condition where `gh` is unauthenticated. The agent must classify that as **expected, not a blocker**, pivot to the local-first `git`/anonymous-REST path, and **not lower its review confidence**. Structural floor: the agent must end with `GH_AUTH_BLOCKER: no` — a *necessary, not sufficient* signal — with an LLM judge scoring the reasoning. - `.github/skills/code-review/tests/eval.inline-findings.vally.yaml` (dotnet#2) — pins a worktree to a real regression commit so the agent has a genuine diff, then must **write** `inline-findings.json` (canonical `path`/`line`/`body` schema) to the path the pipeline reads from disk and prove it landed (read back, echo `FILE_OK:[…`). An agent that refuses (`"prohibited"`) and chat-dumps the JSON has no file to echo and fails. An LLM judge scores finding quality, non-refusal, and requires transcript evidence of an actual write tool call (not just a fabricated `FILE_OK:` line). Each suite keeps two graders — one structural floor + one LLM judge. Both floors are *satisfiable by a partial regression* (the prompt hands the agent the giveaway token), so `scoring.threshold` is set to **0.7** (above the house 0.6): with the unweighted `mean(floor, judge_norm)` aggregate, a floor-1.0-but-judge-failing run scores `(1.0 + 0.25)/2 = 0.625`, which 0.6 would **pass** but 0.7 **fails**. This makes the **LLM judge load-bearing** (must reach ≥3/5) rather than letting the spoofable floor decide. Live-validated: good path 1.00, partial-regression 0.625. ## Follow-ups (identified, not in this PR) Same analysis surfaced: PowerShell-vs-bash redirection footguns in `try-fix` recipes; test-result classification keyed off exit code instead of TRX `failed=` count; winners crowned with no regression evidence; gate=FAILED conflating real regressions with wrong-platform/ineffective tests; and the agent re-deriving the true PR diff base each run. These can be addressed in separate PRs. --------- Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…et#36007) > [!NOTE] > Are you waiting for the changes in this PR to be merged? > It would be very helpful if you could [test the resulting artifacts](https://github.com/dotnet/maui/wiki/Testing-PR-Builds) from this PR and let us know in a comment if this change resolves your issue. Thank you! ## What this adds A reusable, **local-only** dotnet/maui skill — `.github/skills/analyze-sessions` — that analyzes a contributor's Copilot CLI sessions to drive iterative improvements to the **PR-review agent** (and other agents/skills/instruction files). It automates the loop the team has been running by hand: **select → extract → score → judge → cluster → propose → emit-eval**. The emit-eval step is what makes it iterative — every recurring failure mode becomes a `vally` guard-eval, the exact mechanism PR dotnet#36002 used by hand, now run over the whole fleet of local sessions. ## Architecture — one engine, two front doors ``` local front door ─┐ -Repository/-Last │ scripts/Get-SessionAnalysis.ps1 (deterministic, NO LLM) -SessionId ─┤ select → extract → score → digest → redact │ • dotnet-replay --summary --json (normalization) CI front door ─┤ • thin raw events.jsonl scan (success/tokens/…) -EventsPath/-Dir ─┘ emits: session-analysis.md + JSON contract │ ▼ redacted digests + ranking agent, in the contributor's OWN session: judge → cluster → propose → emit guard-eval ``` - **Deterministic shared core** (`Get-SessionAnalysis.ps1`, no LLM) does select/extract/score/digest + redaction. It **wraps `dotnet-replay` v0.9.1** for normalization and adds a thin raw `events.jsonl` scan for the signals replay's `--json` omits (per-tool `success`, `outputTokens`, truncations/compactions, aborts, retries, subagent failures). - **Local front door:** `-Repository` / `-Last` / `-SessionId` select from `session-store.db`. - **CI front door:** `-EventsPath` / `-EventsDir` point the *same* engine at already-downloaded AzDO `events.jsonl` artifacts — one engine reused by the existing CI-session pipeline, inside its artifact boundary. - The **judge / cluster / propose / emit-eval** steps run in the contributor's **own Copilot session** (no third-party endpoint), driven by `SKILL.md`. ## Privacy / safety - **Local-only by default** — reads `~/.copilot/…`, writes a report into the session workspace. It **never** opens a gist and **never** POSTs a transcript. - **Redaction on by default** — home paths → `~`, tokens (`ghp_`/`gho_`/`Bearer`/`password=`/`key=`), and emails are stripped from the report **and** any emitted eval. - The LLM-judge runs through the contributor's own auth/quota; cross-machine sharing is explicit, manual, opt-in. ## Deliverables | File | Purpose | |------|---------| | `SKILL.md` | Triggers, 6-phase workflow, judge rubric, learn-from-pr proposal taxonomy, dotnet#36002 emit-eval template, privacy model, when-NOT-to-use | | `scripts/Get-SessionAnalysis.ps1` | The deterministic shared core (PowerShell — matches every other repo skill script) | | `references/design-rationale.md` | Cites `dotnet-replay` (+ the `--json` gap), the `events.jsonl` 35-event schema, the privacy model, the two-front-doors architecture, and the hand-run proof-of-concept | | `tests/eval.vally.yaml` | Capability + privacy suite: privacy floor (`SHARE_ACTION: none`), capability (`PROPOSED_EVAL: yes`), and negative-trigger — each a refutation-proof structural floor + LLM judge | ## Verification - Core validated end-to-end against real local maui sessions; ranking is driven by genuine inefficiency (failures, retries, truncations, tokens) rather than calendar span (resumed-session wall-clock is capped for scoring). - Metrics spot-checked against raw `events.jsonl` (tool-failure count, `outputTokens`, compactions all matched exactly). - Both front doors exercised (local DB select **and** `-EventsPath`); `-Json` contract is valid JSON; redaction confirmed against synthetic secrets. - `npx -y @microsoft/vally-cli@0.6.0 lint --eval-spec .github/skills/analyze-sessions/tests/eval.vally.yaml --strict` **passes**. No production code changes — this is additive tooling under `.github/skills/`. --------- Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
NafeelaNazhir
had a problem deploying
to
copilot-pat-pool
July 17, 2026 13:16 — with 
GitHub Actions
Failure
GitHub Actions
Failure
Contributor
|
🚀 Dogfood this PR with:
curl -fsSL https://raw.githubusercontent.com/dotnet/maui/main/eng/scripts/get-maui-pr.sh | bash -s -- 36631Or
iex "& { $(irm https://raw.githubusercontent.com/dotnet/maui/main/eng/scripts/get-maui-pr.ps1) } 36631" |
|
Azure Pipelines: Successfully started running 1 pipeline(s). There may be pipelines that require an authorized user to comment /azp run to run. |
Contributor
|
Hey there @@NafeelaNazhir! Thank you so much for your PR! Someone from the team will get assigned to your PR shortly and we'll get it reviewed. |
dotnet-policy-service
Bot
added
the
partner/syncfusion
Contributor
|
/azp run maui-pr-uitests |
|
Azure Pipelines: Successfully started running 1 pipeline(s). |
kubaflo
added a commit
that referenced
this pull request
Jul 19, 2026
Contributor
Tests Failure Analysis
Test Failure Review: Not ready - click to expandOverall verdict: Not ready The deterministic gate found 7 failures red on this PR while the same legs were green across sampled base builds, so CI is not ready. None of the 39 distinct failures exact-match the base branch; 32 remain unattributed because the relevant UI legs are flaky/incomplete on base, logs were incomplete, or the checks were aborted/unmapped. Coverage: 139 checks · 127 passing · 12 failing · 0 pending · 0 inaccessible · 1 unmapped · 25 unexplained build legs · 0 unaccounted failing checks · 2 aborted failing checks · 0 canceled-build checks · 0 device-test unverified · 32 unattributed · 7 regressed-vs-base · 1 demoted by ci-scan. Deterministic ceiling: Not ready — 1 failing check has no inspectable AzDO build evidence (Build Analysis); 25 failed build legs produced no extractable failure; 32 failures could not be attributed deterministically; 2 macOS UI-test checks were cancelled; 7 failures are red on the PR but green on sampled base builds.
Recommended actionInvestigate the Shell SearchHandler UI-test changes and snapshots first, especially Evidence detailsPR #36631 changes 84 test-related files, including Shell SearchHandler host pages, Failing checks include Build Analysis; Base comparison inspected The strongest PR-caused evidence is the computed job-level baseline diff: Limitations: Build Analysis had no inspectable AzDO build evidence in the gathered mapping; 25 failed build legs produced no extractable failure; two macOS UI-test checks were cancelled; and several UI failures are indeterminate because the sampled base UI legs were already flaky or the baseline extraction was incomplete. Visual comparison metadata was considered as evidence only; individual visual image links and panels are intentionally omitted here. Visual failure comparisonsFull-resolution CI baseline, actual, and diff images are embedded below. They supplement the failure classification and do not change the deterministic verdict ceiling.
|
| CI baseline | Fresh PR actual | CI diff |
|---|---|---|
 |
 |
 |
CollectionViewSelectionChangesVisualState - macos - Needs human investigation - visual comparison
CI reported 0.54% difference in build 1517676.
Relationship to PR: Needs human investigation - No decisive exact test-and-platform baseline attribution was available.
| CI baseline | Fresh PR actual | CI diff |
|---|---|---|
 |
 |
 |
VerifyShellSearch_SearchBoxVisibilityExpanded - windows - Likely PR-caused - visual comparison
CI reported 0.87% difference in build 1517676.
Relationship to PR: Likely PR-caused - This PR changes the exact snapshot or visual test.
| CI baseline | Fresh PR actual | CI diff |
|---|---|---|
 |
 |
 |
VerifyShellSearch_SearchBoxVisibilityCollapsible - windows - Likely PR-caused - visual comparison
CI reported 0.90% difference in build 1517676.
Relationship to PR: Likely PR-caused - This PR changes the exact snapshot or visual test.
| CI baseline | Fresh PR actual | CI diff |
|---|---|---|
 |
 |
 |
ShellPages_FlowDirectionRTL - windows - Needs human investigation - visual comparison
CI reported 0.71% difference in build 1517676.
Relationship to PR: Needs human investigation - No decisive exact test-and-platform baseline attribution was available.
| CI baseline | Fresh PR actual | CI diff |
|---|---|---|
 |
 |
 |
VerticalStackLayout_Spacing_With_Width - windows - Needs human investigation - visual comparison
CI reported 0.94% difference in build 1517676.
Relationship to PR: Needs human investigation - No decisive exact test-and-platform baseline attribution was available.
| CI baseline | Fresh PR actual | CI diff |
|---|---|---|
 |
 |
 |
sheiksyedm
marked this pull request as ready for review
July 20, 2026 06:02
|
Azure Pipelines: Successfully started running 1 pipeline(s). There may be pipelines that require an authorized user to comment /azp run to run. |
Contributor
There was a problem hiding this comment.
Pull request overview
This PR extends the Controls test HostApp Shell Feature Matrix with a dedicated SearchHandler feature page + options page, and adds Appium UI tests to exercise SearchHandler properties/behaviors across platforms (with several cases gated/ignored due to known issues).
Changes:
- Added ShellSearchControlPage (Shell + SampleSearchHandler) and ShellSearchOptionsPage to manipulate SearchHandler settings at runtime.
- Expanded ShellViewModel with SearchHandler-related bindable properties, commands, and a
Reset()routine to restore defaults between option runs. - Added ShellSearchHandlerFeatureTests to validate SearchHandler visuals/behavior via Appium (with known-failing scenarios marked ignored or behind
TEST_FAILS_ON_*).
Reviewed changes
Copilot reviewed 8 out of 84 changed files in this pull request and generated 4 comments.
Show a summary per file
| File | Description |
|---|---|
| src/Controls/tests/TestCases.Shared.Tests/Tests/FeatureMatrix/ShellSearchHandlerFeatureTests.cs | New Appium feature-matrix tests for Shell SearchHandler properties and behaviors. |
| src/Controls/tests/TestCases.HostApp/FeatureMatrix/Shell/ShellViewModel.cs | Adds SearchHandler-related state + commands and a Reset() method to support the feature matrix UI. |
| src/Controls/tests/TestCases.HostApp/FeatureMatrix/Shell/ShellSearch/ShellSearchOptionsPage.xaml.cs | Code-behind wiring for options UI to update the SearchHandler-related properties on the view model. |
| src/Controls/tests/TestCases.HostApp/FeatureMatrix/Shell/ShellSearch/ShellSearchOptionsPage.xaml | Options UI (entries/radio buttons) for exercising SearchHandler properties. |
| src/Controls/tests/TestCases.HostApp/FeatureMatrix/Shell/ShellSearch/ShellSearchControlPage.xaml.cs | Implements SampleSearchHandler + Shell page logic (reset, navigation to options, icon toggles, etc.). |
| src/Controls/tests/TestCases.HostApp/FeatureMatrix/Shell/ShellSearch/ShellSearchControlPage.xaml | Declares the Shell + SearchHandler bindings and the in-app controls/logging used by the UI tests. |
| src/Controls/tests/TestCases.HostApp/FeatureMatrix/Shell/ShellFeaturePage.xaml.cs | Adds a button handler to launch the Shell Search feature page. |
| src/Controls/tests/TestCases.HostApp/FeatureMatrix/Shell/ShellFeaturePage.xaml | Adds a “Shell Search” entry point button to the Shell Feature Matrix main page. |
Contributor
|
/azp run |
|
Azure Pipelines: Successfully started running 3 pipeline(s). |
kubaflo
added a commit
that referenced
this pull request
Jul 20, 2026
kubaflo
added a commit
that referenced
this pull request
Jul 20, 2026
kubaflo
added a commit
that referenced
this pull request
Jul 20, 2026
kubaflo
added a commit
that referenced
this pull request
Jul 22, 2026
This pull request extends the Shell feature matrix in the test host app by adding a dedicated **SearchHandler** feature page and comprehensive Appium UI test coverage for **SearchHandler** properties and behaviors across **Android**, **iOS/MacCatalyst**, and **Windows**. * Added `ShellSearchControlPage.xaml` / `ShellSearchControlPage.xaml.cs` with a `SampleSearchHandler` implementation backed by sample **Fruits** and **Birds** data sources. The page supports query changes, item selection, and command execution through `ShellViewModel`. * Added `ShellSearchOptionsPage.xaml` / `ShellSearchOptionsPage.xaml.cs`, providing runtime controls for validating the following `SearchHandler` properties: BackgroundColor, CancelButtonColor, TextColor, PlaceholderColor, CharacterSpacing, FontAttributes, FontAutoScalingEnabled, FontFamily, FontSize, HorizontalTextAlignment, VerticalTextAlignment, TextTransform, Keyboard, Placeholder, IsSearchEnabled, ShowsResults, SearchBoxVisibility, Query, SelectedItem, ItemsSourceMode, ClearIcon, QueryIcon, ClearPlaceholderEnabled, ClearPlaceholderIcon, and ClearPlaceholderCommandParameter. * Added `ShellSearchHandlerFeatureTests.cs` with Appium UI tests covering SearchHandler properties and behaviors, including Android and iOS baseline screenshots. * #35085 * #35624 * #35667 * #35736 * #35764 * #35767 * #36629 * #35516 * #28619 --------- Co-authored-by: Shane Neuville <5375137+PureWeen@users.noreply.github.com> Co-authored-by: PureWeen <223556219+Copilot@users.noreply.github.com> Co-authored-by: Jakub Florkowski <42434498+kubaflo@users.noreply.github.com>
Dhivya-SF4094
pushed a commit
to Dhivya-SF4094/maui
that referenced
this pull request
Jul 24, 2026
…#36631) This pull request extends the Shell feature matrix in the test host app by adding a dedicated **SearchHandler** feature page and comprehensive Appium UI test coverage for **SearchHandler** properties and behaviors across **Android**, **iOS/MacCatalyst**, and **Windows**. * Added `ShellSearchControlPage.xaml` / `ShellSearchControlPage.xaml.cs` with a `SampleSearchHandler` implementation backed by sample **Fruits** and **Birds** data sources. The page supports query changes, item selection, and command execution through `ShellViewModel`. * Added `ShellSearchOptionsPage.xaml` / `ShellSearchOptionsPage.xaml.cs`, providing runtime controls for validating the following `SearchHandler` properties: BackgroundColor, CancelButtonColor, TextColor, PlaceholderColor, CharacterSpacing, FontAttributes, FontAutoScalingEnabled, FontFamily, FontSize, HorizontalTextAlignment, VerticalTextAlignment, TextTransform, Keyboard, Placeholder, IsSearchEnabled, ShowsResults, SearchBoxVisibility, Query, SelectedItem, ItemsSourceMode, ClearIcon, QueryIcon, ClearPlaceholderEnabled, ClearPlaceholderIcon, and ClearPlaceholderCommandParameter. * Added `ShellSearchHandlerFeatureTests.cs` with Appium UI tests covering SearchHandler properties and behaviors, including Android and iOS baseline screenshots. * dotnet#35085 * dotnet#35624 * dotnet#35667 * dotnet#35736 * dotnet#35764 * dotnet#35767 * dotnet#36629 * dotnet#35516 * dotnet#28619 --------- Co-authored-by: Shane Neuville <5375137+PureWeen@users.noreply.github.com> Co-authored-by: PureWeen <223556219+Copilot@users.noreply.github.com> Co-authored-by: Jakub Florkowski <42434498+kubaflo@users.noreply.github.com>
PureWeen
pushed a commit
that referenced
this pull request
Jul 24, 2026
<!-- Please let the below note in for people that find this PR --> > [!NOTE] > Are you waiting for the changes in this PR to be merged? > It would be very helpful if you could [test the resulting artifacts](https://github.com/dotnet/maui/wiki/Testing-PR-Builds) from this PR and let us know in a comment if this change resolves your issue. Thank you! ### Description of Change Automates visual snapshot evidence in `/review tests`. When public AzDO results contain failed screenshot comparisons, the command now emits exactly one test-failure analysis comment containing bounded, expandable baseline/actual/diff panels. Visual evidence remains supplementary: it does not change `gate.verdictCeiling`, deterministic attribution, or the merge-readiness verdict. #### One-comment flow 1. Trusted pre-activation code discovers failed visual results through the public AzDO `resultsbybuild` API, including retry-suffixed attachments such as `Snapshot[1].png` and `Snapshot-diff[1].png`. 2. It resolves baselines from the exact source version tested by AzDO and maps runtime evidence to the correct snapshot directory (`ios-26`, `android-notch-36`, `mac`, or `windows`). 3. It streams and validates bounded PNG assets, then stores them on `review-tests-assets` using immutable commit-pinned `raw.githubusercontent.com` URLs. 4. The Copilot agent emits the normal single `add_comment` analysis payload with a trusted insertion marker. 5. A sealed post-step validates the published asset manifest and injects as many expandable comparison panels as fit into that same comment. Each collapsed panel shows a conservative relationship label: - `Likely PR-caused` for an exact test/platform base regression or directly changed snapshot/test; - `Likely unrelated` for an exact base/known-issue match without direct visual scope; - `Needs human investigation` for unmatched or mixed evidence. 6. Excess comparisons are summarized as omitted instead of creating a companion comment. The local `.github/scripts/Review-Tests.ps1 -PostComment` path uses the same merger. It also recognizes complete reports returned in Copilot's final response, preserving nested evidence code fences without wrapping a second title or badge section. ### Security and Failure Safety - PR text, logs, test names, attachment metadata, changed files, and visual labels remain untrusted input. - The merger script and visual context are copied to a root-owned location before the workflow checks out the untrusted PR branch. - The post-step runs without `COPILOT_GITHUB_TOKEN`, `GH_TOKEN`, or `GITHUB_TOKEN`. - AzDO attachment URLs must match the expected public project and attachment route. - Published assets are size-bounded, signature-checked PNGs with validated dimensions and repository paths. - Raw image URLs must match the exact repository, asset commit, PR directory, and safe filename. - Labels are HTML-escaped and `@` is neutralized before insertion. - Relationship labels use fixed trusted text. Untrusted attribution values are never rendered, and same-named snapshots changed on another platform do not count as PR scope. - The final body is checked panel-by-panel against conservative limits of 45 URLs, 10 mentions, and 60,000 UTF-16 characters, below gh-aw's throwing limits. - The analysis JSON update is atomic (written to a temp file, then renamed over the original). Invalid context, malformed output, missing analysis payloads, limit failures, and dry-run/noop output leave the original analysis unchanged. - The publisher never creates or patches PR comments; only the existing gh-aw `add_comment` payload is mutated. ### What NOT to Do - Do not use the ordinary anonymous AzDO test-runs listing for discovery; it redirects to sign-in. Use the public failed-results endpoint. - Do not resolve baselines from the current PR head; use the source version actually tested by the selected AzDO build. - Do not let the agent construct or trust visual asset URLs. - Do not publish a second companion comment; merge bounded panels into the single analysis payload. ### Validation - 98 focused Pester tests pass. - Changed PowerShell scripts parse successfully. - `gh aw compile copilot-review-tests --approve` completes without errors or warnings. - A real `agent_output.json` from gh-aw run [29674953402](https://github.com/dotnet/maui/actions/runs/29674953402) was replayed through the post-step: - one `add_comment` item remained one item; - five visual panels were inserted; - the final body contained 26 URLs, one mention, and 9,585 characters. ### Live Single-Comment Examples The exact local `/review tests` path from this branch posted or repaired these merged comments after the PRs' `/azp run` pipelines completed: | PR | Single merged result | Included evidence | Relationship labels | Final limits | | --- | --- | --- | --- | --- | | #36413 | [Test-failure analysis with visual panels](#36413 (comment)) | 5 panels / 15 images | 1 PR-caused, 4 investigate | 23 URLs, 13,156 chars | | #36631 | [Test-failure analysis with visual panels](#36631 (comment)) | 6 panels / 18 images | 2 PR-caused, 4 investigate | 31 URLs, 21,367 chars | | #36395 | [Test-failure analysis with visual panels](#36395 (comment)) | 11 panels / 33 images; 19 omitted | 11 investigate | 43 URLs, 19,858 chars | | #36404 | [Test-failure analysis with visual panels](#36404 (comment)) | 14 panels / 40 images; 81 omitted | 14 investigate | 45 URLs, 23,779 chars | | #35846 | [Test-failure analysis with visual panels](#35846 (comment)) | 10 panels / 30 images; 9 omitted | 10 investigate | 43 URLs, 22,496 chars | | #36277 | [Test-failure analysis with visual panels](#36277 (comment)) | 7 panels / 19 images | 3 PR-caused, 4 investigate | 31 URLs, 18,355 chars | | #36170 | [Test-failure analysis with visual panels](#36170 (comment)) | 11 panels / 33 images; 8 omitted | 11 investigate | 44 URLs, 23,180 chars | | #35578 | [Test-failure analysis with visual panels](#35578 (comment)) | 12 panels / 36 images; 50 omitted | 12 investigate | 44 URLs, 25,443 chars | | #36672 | [Test-failure analysis with visual panels](#36672 (comment)) | 14 panels / 40 images; 9 omitted | 14 investigate | 45 URLs, 25,915 chars | | #31755 | [Test-failure analysis with visual panels](#31755 (comment)) | 12 panels / 36 images; 3 omitted | 12 investigate | 44 URLs, 22,728 chars | | #34637 | [Test-failure analysis with visual panels](#34637 (comment)) | 9 panels / 27 images; 81 omitted | 9 investigate | 43 URLs, 22,325 chars | | #35156 | [Test-failure analysis with visual panels](#35156 (comment)) | 2 panels / 6 images | 1 PR-caused, 1 investigate | 19 URLs, 11,911 chars | | #35885 | [Test-failure analysis with no visual failures](#35885 (comment)) | 0 panels / 0 images | No visual failures detected | 8 URLs, 4,239 chars | | #36577 | [Test-failure analysis with visual panels](#36577 (comment)) | 1 panel / 3 images | 1 investigate | 44 URLs, 21,768 chars | | #36212 | [Test-failure analysis with no visual failures](#36212 (comment)) | 0 panels / 0 images | No visual failures detected | 5 URLs, 5,982 chars | Each result contains one `Tests Failure Analysis` title and one merged review marker. Across 114 rendered panels, all 336 embedded image URLs returned HTTP 200. Seven panels were safely classified as likely PR-caused; no panel in this sample had enough exact evidence to be safely classified as likely unrelated, so the remaining 107 stayed at `Needs human investigation`. Another 260 comparisons were omitted safely by the comment limits. The latest eight-example batch was regenerated concurrently, and #36672, #31755, #34637, #35156, #35885, #36577, and #36212 were added afterward. The current [`review-tests-assets` head](f937993) retains the full asset history. The protected `copilot-pat-pool` environment rejects feature-branch `workflow_dispatch` runs before job execution. The live local-runner examples validate comment generation and asset publication, while the real gh-aw output replay validates the workflow post-step mutation without weakening that branch protection. ### Issues Fixed N/A - reviewer workflow enhancement. --------- Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Co-authored-by: kubaflo <kubaflo@users.noreply.github.com> Copilot-Session: a280b482-e102-4ca0-9ff9-1cfe1946e21f Copilot-Session: d00747b7-96f3-4e7a-8dfb-e3a48db04b2d Copilot-Session: 478d195b-20f3-4bc6-aeed-f6aa88b55fda
kubaflo
added a commit
that referenced
this pull request
Jul 28, 2026
This pull request extends the Shell feature matrix in the test host app by adding a dedicated **SearchHandler** feature page and comprehensive Appium UI test coverage for **SearchHandler** properties and behaviors across **Android**, **iOS/MacCatalyst**, and **Windows**. * Added `ShellSearchControlPage.xaml` / `ShellSearchControlPage.xaml.cs` with a `SampleSearchHandler` implementation backed by sample **Fruits** and **Birds** data sources. The page supports query changes, item selection, and command execution through `ShellViewModel`. * Added `ShellSearchOptionsPage.xaml` / `ShellSearchOptionsPage.xaml.cs`, providing runtime controls for validating the following `SearchHandler` properties: BackgroundColor, CancelButtonColor, TextColor, PlaceholderColor, CharacterSpacing, FontAttributes, FontAutoScalingEnabled, FontFamily, FontSize, HorizontalTextAlignment, VerticalTextAlignment, TextTransform, Keyboard, Placeholder, IsSearchEnabled, ShowsResults, SearchBoxVisibility, Query, SelectedItem, ItemsSourceMode, ClearIcon, QueryIcon, ClearPlaceholderEnabled, ClearPlaceholderIcon, and ClearPlaceholderCommandParameter. * Added `ShellSearchHandlerFeatureTests.cs` with Appium UI tests covering SearchHandler properties and behaviors, including Android and iOS baseline screenshots. * #35085 * #35624 * #35667 * #35736 * #35764 * #35767 * #36629 * #35516 * #28619 --------- Co-authored-by: Shane Neuville <5375137+PureWeen@users.noreply.github.com> Co-authored-by: PureWeen <223556219+Copilot@users.noreply.github.com> Co-authored-by: Jakub Florkowski <42434498+kubaflo@users.noreply.github.com>
kubaflo
added a commit
that referenced
this pull request
Jul 29, 2026
This pull request extends the Shell feature matrix in the test host app by adding a dedicated **SearchHandler** feature page and comprehensive Appium UI test coverage for **SearchHandler** properties and behaviors across **Android**, **iOS/MacCatalyst**, and **Windows**. * Added `ShellSearchControlPage.xaml` / `ShellSearchControlPage.xaml.cs` with a `SampleSearchHandler` implementation backed by sample **Fruits** and **Birds** data sources. The page supports query changes, item selection, and command execution through `ShellViewModel`. * Added `ShellSearchOptionsPage.xaml` / `ShellSearchOptionsPage.xaml.cs`, providing runtime controls for validating the following `SearchHandler` properties: BackgroundColor, CancelButtonColor, TextColor, PlaceholderColor, CharacterSpacing, FontAttributes, FontAutoScalingEnabled, FontFamily, FontSize, HorizontalTextAlignment, VerticalTextAlignment, TextTransform, Keyboard, Placeholder, IsSearchEnabled, ShowsResults, SearchBoxVisibility, Query, SelectedItem, ItemsSourceMode, ClearIcon, QueryIcon, ClearPlaceholderEnabled, ClearPlaceholderIcon, and ClearPlaceholderCommandParameter. * Added `ShellSearchHandlerFeatureTests.cs` with Appium UI tests covering SearchHandler properties and behaviors, including Android and iOS baseline screenshots. * #35085 * #35624 * #35667 * #35736 * #35764 * #35767 * #36629 * #35516 * #28619 --------- Co-authored-by: Shane Neuville <5375137+PureWeen@users.noreply.github.com> Co-authored-by: PureWeen <223556219+Copilot@users.noreply.github.com> Co-authored-by: Jakub Florkowski <42434498+kubaflo@users.noreply.github.com>
kubaflo
added a commit
that referenced
this pull request
Aug 7, 2026
This pull request extends the Shell feature matrix in the test host app by adding a dedicated **SearchHandler** feature page and comprehensive Appium UI test coverage for **SearchHandler** properties and behaviors across **Android**, **iOS/MacCatalyst**, and **Windows**. * Added `ShellSearchControlPage.xaml` / `ShellSearchControlPage.xaml.cs` with a `SampleSearchHandler` implementation backed by sample **Fruits** and **Birds** data sources. The page supports query changes, item selection, and command execution through `ShellViewModel`. * Added `ShellSearchOptionsPage.xaml` / `ShellSearchOptionsPage.xaml.cs`, providing runtime controls for validating the following `SearchHandler` properties: BackgroundColor, CancelButtonColor, TextColor, PlaceholderColor, CharacterSpacing, FontAttributes, FontAutoScalingEnabled, FontFamily, FontSize, HorizontalTextAlignment, VerticalTextAlignment, TextTransform, Keyboard, Placeholder, IsSearchEnabled, ShowsResults, SearchBoxVisibility, Query, SelectedItem, ItemsSourceMode, ClearIcon, QueryIcon, ClearPlaceholderEnabled, ClearPlaceholderIcon, and ClearPlaceholderCommandParameter. * Added `ShellSearchHandlerFeatureTests.cs` with Appium UI tests covering SearchHandler properties and behaviors, including Android and iOS baseline screenshots. * #35085 * #35624 * #35667 * #35736 * #35764 * #35767 * #36629 * #35516 * #28619 --------- Co-authored-by: Shane Neuville <5375137+PureWeen@users.noreply.github.com> Co-authored-by: PureWeen <223556219+Copilot@users.noreply.github.com> Co-authored-by: Jakub Florkowski <42434498+kubaflo@users.noreply.github.com>
kubaflo
added a commit
that referenced
this pull request
Aug 12, 2026
This pull request extends the Shell feature matrix in the test host app by adding a dedicated **SearchHandler** feature page and comprehensive Appium UI test coverage for **SearchHandler** properties and behaviors across **Android**, **iOS/MacCatalyst**, and **Windows**. * Added `ShellSearchControlPage.xaml` / `ShellSearchControlPage.xaml.cs` with a `SampleSearchHandler` implementation backed by sample **Fruits** and **Birds** data sources. The page supports query changes, item selection, and command execution through `ShellViewModel`. * Added `ShellSearchOptionsPage.xaml` / `ShellSearchOptionsPage.xaml.cs`, providing runtime controls for validating the following `SearchHandler` properties: BackgroundColor, CancelButtonColor, TextColor, PlaceholderColor, CharacterSpacing, FontAttributes, FontAutoScalingEnabled, FontFamily, FontSize, HorizontalTextAlignment, VerticalTextAlignment, TextTransform, Keyboard, Placeholder, IsSearchEnabled, ShowsResults, SearchBoxVisibility, Query, SelectedItem, ItemsSourceMode, ClearIcon, QueryIcon, ClearPlaceholderEnabled, ClearPlaceholderIcon, and ClearPlaceholderCommandParameter. * Added `ShellSearchHandlerFeatureTests.cs` with Appium UI tests covering SearchHandler properties and behaviors, including Android and iOS baseline screenshots. * #35085 * #35624 * #35667 * #35736 * #35764 * #35767 * #36629 * #35516 * #28619 --------- Co-authored-by: Shane Neuville <5375137+PureWeen@users.noreply.github.com> Co-authored-by: PureWeen <223556219+Copilot@users.noreply.github.com> Co-authored-by: Jakub Florkowski <42434498+kubaflo@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This pull request extends the Shell feature matrix in the test host app by adding a dedicated SearchHandler feature page and comprehensive Appium UI test coverage for SearchHandler properties and behaviors across Android, iOS/MacCatalyst, and Windows.
Changes
Shell SearchHandler Feature Matrix
ShellSearchControlPage.xaml/ShellSearchControlPage.xaml.cswith aSampleSearchHandlerimplementation backed by sample Fruits and Birds data sources. The page supports query changes, item selection, and command execution throughShellViewModel.ShellSearchOptionsPage.xaml/ShellSearchOptionsPage.xaml.cs, providing runtime controls for validating the followingSearchHandlerproperties:BackgroundColor, CancelButtonColor, TextColor, PlaceholderColor, CharacterSpacing, FontAttributes, FontAutoScalingEnabled, FontFamily, FontSize, HorizontalTextAlignment, VerticalTextAlignment, TextTransform, Keyboard, Placeholder, IsSearchEnabled, ShowsResults, SearchBoxVisibility, Query, SelectedItem, ItemsSourceMode, ClearIcon, QueryIcon, ClearPlaceholderEnabled, ClearPlaceholderIcon, and ClearPlaceholderCommandParameter.
ShellSearchHandlerFeatureTests.cswith Appium UI tests covering SearchHandler properties and behaviors, including Android and iOS baseline screenshots.New Issues Identified
Existing Issues Identified