Fix dotnet-maui skill regressions and sharpen decisiveness

[kubaflo]

Summary

Addresses #895 — the cross-family evaluation for the dotnet-maui plugin (8 skills, 48% pass, 1 P0).

Every claim in this PR was validated empirically by building a real .NET MAUI app against the current SDK — not by source reading alone. That mattered: one of my initial "fixes" turned out to be wrong and was reverted (see below).

🔴 P0 — maui-collectionview (FIX-REGRESSION)

Root cause found: the skill documented markup that does not compile.

ItemSizingStrategy was shown on LinearItemsLayout in both SKILL.md and references/collectionview-api.md. It is declared on StructuredItemsView, so the documented form is a hard build break:

error MAUIX2002: No accessible property, BindableProperty, or event found for
"ItemSizingStrategy", or mismatching type between value and property.

This is a plausible driver of the "skilled is worse than baseline" signal — the skill actively taught broken XAML. Corrected to set it on <CollectionView>, keeping the invalid form as an explicit ❌ counter-example.

Per the CTA's FIX-REGRESSION guidance, also added:

• A Scope Control section with explicit stop-conditions (do NOT act when…) targeting over-application: don't answer beyond the question, don't rewrite working code, don't migrate a working ListView, don't churn style.
• A narrowed frontmatter trigger (excludes CarouselView/BindableLayout, handler customization, and binding questions that merely mention a list).
• Demoted prescriptive edits to suggestions the agent may decline, reducing the non-negotiable rules to the three that actually break code.
• A validation checklist.

🔴 HIGH — maui-app-lifecycle (non-compiling code, newly found)

The iOS ConfigureLifecycleEvents examples called .DidBecomeActive() / .WillResignActive(), which don't exist:

error CS1061: 'IiOSLifecycleBuilder' does not contain a definition for 'DidBecomeActive'

Corrected to .OnActivated() / .OnResignActivation(). The platform-mapping tables now show both the UIKit selector and the builder method, because the names deliberately differ — that mismatch is exactly what made this easy to get wrong.

Correctness fixes

Skill	Fix
maui-data-binding	The XC0022–XC0025 meanings table was wrong on all four rows. Corrected against Build.Tasks/BuildException.cs + ErrorMessages.resx.
maui-safe-area	Claimed WindowSoftInputModeAdjust.Resize was "removed" — it still compiles and is not obsolete. Reframed as Android-only/superseded. Brevity deliberately preserved (EFFICIENT-WIN).

🟡 STRENGTHEN — decisiveness

Ties dominated maui-shell-navigation, maui-theming, maui-data-binding and maui-dependency-injection: the content was correct but read as reference prose the model already knows, so the judge saw no behaviour change.

Each now has a compact "Rules That Change the Answer" table converting prose into when-A-do-B guidance (e.g. IQueryAttributable on the ViewModel not the Page; ContentTemplate for lazy page creation; AddScoped silently behaving as Singleton without an IServiceScope), plus an explicit "don't churn working code" stop-condition to avoid trading ties for losses.

⚠️ Notably NOT changed

• maui-theming's {AppThemeBinding Light={DynamicResource …}} guidance. Reading AppThemeBindingExtension suggested the nested DynamicResource could never resolve, and I initially "fixed" it. A runtime probe disproved that: after swapping the dictionary the value tracked it (#FF0000 → #0000FF) while a StaticResource control stayed frozen at #FF0000. The original guidance is correct — reverted. (This is why the empirical step was worth it.)
• maui-shell-navigation fixture/SDK pinning. Per the [skills-eval] Judge-side transient failures (disabled PAT + session.idle timeout) corrupt reliability verdicts #909 correction, the errored trial was a judge-side disabled-PAT failure, not fixture nondeterminism — so it was treated as STRENGTHEN, not FIX-RELIABILITY.
• dotnet-maui-doctor (EXEMPLAR) — unchanged.

Validation

• skill-validator check --plugin ./plugins/dotnet-maui → ✅ all checks passed (8 skills, 0 errors, 26 reference files scanned).
• A net11.0-maccatalyst MAUI app builds clean with every corrected snippet (CollectionView grid/selection/refresh/swipe/empty-view, Shell ContentTemplate, GetDeferral/Complete, ShellNavigationQueryParameters, IQueryAttributable, DI registrations, iOS lifecycle).
• Both original buggy forms reproduce the exact errors quoted above.

Full cross-family re-evaluation still to be triggered per the instructions in #895.

[Copilot]

Pull request overview

This PR updates the dotnet-maui plugin’s skill documentation to fix known non-compiling snippets (regressions) and to make several skills more behaviorally decisive by adding concise “Rules That Change the Answer” / scope-control guidance.

Changes:

• Fixes MAUI documentation examples that previously produced build errors (CollectionView ItemSizingStrategy placement; iOS lifecycle builder method names).
• Adds “Rules That Change the Answer” (and related stop-conditions / validation checklists) to reduce ties and prevent over-application.
• Clarifies/adjusts guidance around safe-area + keyboard avoidance and compiled-binding warning meanings.

Reviewed changes

Copilot reviewed 9 out of 9 changed files in this pull request and generated 1 comment.

Show a summary per file

File	Description
plugins/dotnet-maui/skills/maui-theming/SKILL.md	Adds a “Rules That Change the Answer” section and clarifies combined theming guidance.
plugins/dotnet-maui/skills/maui-shell-navigation/SKILL.md	Adds a “Rules That Change the Answer” table and stop-conditions for Shell navigation.
plugins/dotnet-maui/skills/maui-safe-area/SKILL.md	Reframes WindowSoftInputModeAdjust.Resize as supported but Android-only; clarifies obsoletion status.
plugins/dotnet-maui/skills/maui-dependency-injection/SKILL.md	Adds a compact rules table emphasizing lifetimes and MAUI scoped behavior.
plugins/dotnet-maui/skills/maui-data-binding/SKILL.md	Adds rules table and corrects the XC0022–XC0025 meanings section.
plugins/dotnet-maui/skills/maui-collectionview/SKILL.md	Adds scope control/stop-conditions, fixes ItemSizingStrategy guidance, and adds validation checklist.
plugins/dotnet-maui/skills/maui-collectionview/references/collectionview-api.md	Corrects ItemSizingStrategy placement and adds explicit ✅/❌ examples.
plugins/dotnet-maui/skills/maui-app-lifecycle/SKILL.md	Fixes iOS lifecycle builder method names and adds a mapping table note.
plugins/dotnet-maui/skills/maui-app-lifecycle/references/lifecycle-api.md	Mirrors the lifecycle mapping fix + note in the reference doc.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[kubaflo]

Evaluation results (skilled vs. baseline)

Per @AbhitejJohn's verification guidance in #895 (comment), I ran the skilled-vs-baseline comparison for the P0 skill (maui-collectionview). The /evaluate gate requires write access (Actor kubaflo has permission: read → Actor does not have write access), so I ran the same harness locally instead:

• vally 0.10.0 (@microsoft/vally-cli), driving dotnet-skills.experiment.yaml unmodified except runs: 3
• Executor claude-opus-4.6, judge gpt-5.5 — cross-family, judge ≠ executor family, matching the CI grid's rule
• --eval-filter tests/dotnet-maui/maui-collectionview/eval.yaml, n=3 per stimulus → 12 trials per arm
• Both arms run identically; the only difference is the skill content

Headline

Arm	Win rate	Wins	Ties	Losses	Mean impact	Errors
main (original skill)	17%	2	4	6	−0.13	0
This PR (first commit)	25%	3	4	5	−0.07	0
This PR (head)	42%	5	6	1	+0.13	0

Losses 6 → 1, and mean impact flips from negative to positive.

The main arm independently reproduced the exact regression signature reported in #895 — 6 losses over 12 trials (the issue's table records Loss▵ 6 for this skill). That reproduction is what gives me confidence the harness is measuring the right thing.

⚠️ Caveat, stated plainly: n=12 on a single executor family. The head arm's 95% CI is [−0.03, +0.30], so this is directional, not statistically significant. The full 5-family grid should still be run before trusting the magnitude. The loss collapse (6 → 1) is the effect the FIX-REGRESSION change was aiming for.

The second commit was driven by these results

The first commit only moved losses 6 → 5. Reading the judge's per-trial rationales identified two concrete causes, which b07628c fixes:

1. "Response B introduces an unresolved x:DataType prefix unless additional namespace declarations are added" — penalised on 2 trials.

   The skill used x:DataType="models:…" in six snippets but never declared xmlns:models, so its examples weren't copy-pasteable. Basic Setup is now a complete ContentPage including the declaration.

2. On the ListView/ViewCell stimulus, baseline won by supplying nuance the skill lacked — that ListView still exists, plus event/API replacements. Added a Migrating from ListView table (every member verified against PublicAPI.Shipped.txt), including the commonly-missed fact that CollectionView has no separator API.

Coverage gap worth flagging

None of the four maui-collectionview stimuli exercise ItemSizingStrategy or performance guidance — so the eval cannot detect the build-break this PR fixes:

error MAUIX2002: No accessible property, BindableProperty, or event found for "ItemSizingStrategy"

That defect was caught by compiling the documented XAML, not by the eval. A stimulus covering item sizing / performance would close the gap.

Compile validation

Separately, every corrected snippet was verified by building a real net11.0-maccatalyst MAUI app, and both original buggy forms reproduce their exact errors:

Claim	Result
ItemSizingStrategy on LinearItemsLayout (original)	❌ error MAUIX2002
ItemSizingStrategy on CollectionView (fixed)	✅ builds
.DidBecomeActive() / .WillResignActive() (original)	❌ error CS1061: 'IiOSLifecycleBuilder' does not contain a definition for 'DidBecomeActive'
.OnActivated() / .OnResignActivation() (fixed)	✅ builds
WindowSoftInputModeAdjust.Resize ("removed")	✅ builds, not obsolete → claim corrected
{AppThemeBinding Light={DynamicResource …}}	✅ builds and is dynamic at runtime (#FF0000 → #0000FF after a dictionary swap, vs. StaticResource frozen at #FF0000) → my proposed "fix" was wrong and was reverted

skill-validator check --plugin ./plugins/dotnet-maui → ✅ all checks passed (8 skills, 0 errors, 26 reference files).

A maintainer will still need to post /evaluate to run the full 5-family grid.

[Copilot]

Pull request overview

Copilot reviewed 9 out of 9 changed files in this pull request and generated no new comments.

Comments suppressed due to low confidence (1)

plugins/dotnet-maui/skills/maui-theming/SKILL.md:62

• In the theming rules table, the inline code uses \| to escape the pipe character. Because this is inside a code span, the escape isn't needed for Markdown tables and it produces an invalid C# expression if someone copies it (bitwise OR is |, not \|).

| **Runtime-swapped values must be dynamic** | `{DynamicResource Key}` | `{StaticResource Key}` | `StaticResource` resolves once at load and never updates when dictionaries are swapped. |
| **Android must declare `UiMode`** | `ConfigurationChanges = … \| ConfigChanges.UiMode` on `MainActivity` | Omitting it | Without it Android restarts the activity on theme change — navigation state is lost and it looks like a crash. |
| **Force a theme via `UserAppTheme`** | `Application.Current.UserAppTheme = AppTheme.Dark` | Manually re-assigning colors | `UserAppTheme` overrides the OS; `AppTheme.Unspecified` returns to following the system. |

[Copilot]

Pull request overview

Copilot reviewed 9 out of 9 changed files in this pull request and generated no new comments.

Comments suppressed due to low confidence (1)

plugins/dotnet-maui/skills/maui-theming/SKILL.md:61

• In this table row, the escaped pipe (\|) is inside an inline code span, so it will render with the backslash visible (and users may copy it). Consider using an HTML entity for the pipe so the rendered code shows the correct | without breaking the markdown table.

| **Android must declare `UiMode`** | `ConfigurationChanges = … \| ConfigChanges.UiMode` on `MainActivity` | Omitting it | Without it Android restarts the activity on theme change — navigation state is lost and it looks like a crash. |

[Copilot]

Pull request overview

Copilot reviewed 9 out of 9 changed files in this pull request and generated 1 comment.

[Copilot]

Pull request overview

Copilot reviewed 9 out of 9 changed files in this pull request and generated 2 comments.

[Copilot]

Pull request overview

Copilot reviewed 9 out of 9 changed files in this pull request and generated 1 comment.

[Copilot]

Pull request overview

Copilot reviewed 15 out of 15 changed files in this pull request and generated no new comments.

Comments suppressed due to low confidence (4)

tests/dotnet-maui/maui-theming/eval.yaml:128

• In this new stimulus, the rubric items are unquoted YAML scalars. The skill-coverage workflow only extracts rubric evidence from single-line quoted items, so these won't contribute to coverage reports. Quote each rubric entry (like the earlier rubric items in this file).

    rubric:
      - Removes only the previously applied theme dictionary, then adds the new one
      - Does NOT call MergedDictionaries.Clear(), which would also remove the template's Colors.xaml and Styles.xaml and unstyle the whole app
      - Keeps a reference to the current theme so it can be removed on the next switch
      - Produces code that leaves the app's existing styles intact after repeated switches

tests/dotnet-maui/maui-shell-navigation/eval.yaml:133

• These newly added rubric items are unquoted. Skill coverage extraction only counts rubric entries that are quoted on a single line, so this scenario will show 0 rubric evidence in coverage reports. Wrap each rubric string in quotes.

    rubric:
      - Sets an explicit Route on the ShellContent elements in AppShell.xaml
      - Explains that without Route, MAUI auto-generates an unstable name (D_FAULT_ prefixed) that shifts when pages are reordered or added
      - Shows an absolute GoToAsync route (leading //) built from the explicit route names
      - Does NOT rely on auto-generated route names for deep linking

tests/dotnet-maui/maui-dependency-injection/eval.yaml:125

• The rubric strings for this new scenario are unquoted YAML scalars. The skill-coverage workflow only picks up quoted rubric entries, so these won't count toward coverage evidence. Quote each rubric item.

    rubric:
      - Explains that ContentTemplate pages are created with Activator.CreateInstance and bypass DI entirely
      - Explains that routes resolved via GoToAsync go through the service provider and inject constructor dependencies
      - Does NOT claim that dependencies are silently set to null without an exception
      - Recommends navigating by route, or giving the page a parameterless constructor, for the ContentTemplate case

tests/dotnet-maui/maui-collectionview/eval.yaml:129

• The new rubric entries are unquoted, but skill-coverage evidence extraction only counts rubric items written as single-line quoted strings. Quote these rubric lines so the coverage report reflects this new scenario.

    rubric:
      - Sets ItemSizingStrategy="MeasureFirstItem" on the CollectionView element itself
      - Does NOT set ItemSizingStrategy on LinearItemsLayout or GridItemsLayout, which does not compile
      - Explains that MeasureFirstItem is only safe when every item is the same size
      - Warns that variable-height items will be clipped or stretched, and that MeasureAllItems is the default

[github-actions]

👋 /evaluate needs the exact commit to evaluate.

Two ways to run it:

1. Review flow (recommended — no SHA to copy): open Files changed → Review changes, type /evaluate in the review box, and Submit review. GitHub binds the run to the exact commit you reviewed.
2. Comment flow: copy-paste this command for the current head:

/evaluate a1eec661a128d684bdf2a012d3f48b2a8b50aebc

[Copilot]

Pull request overview

Copilot reviewed 15 out of 15 changed files in this pull request and generated no new comments.

Comments suppressed due to low confidence (4)

tests/dotnet-maui/maui-shell-navigation/eval.yaml:133

• These newly added rubric items are unquoted. The skill-coverage workflow only counts single-line quoted rubric entries, so these won’t be picked up in coverage reports unless they’re wrapped in quotes.

    rubric:
      - Sets an explicit Route on the ShellContent elements in AppShell.xaml
      - Explains that without Route, MAUI auto-generates an unstable name (D_FAULT_ prefixed) that shifts when pages are reordered or added
      - Shows an absolute GoToAsync route (leading //) built from the explicit route names
      - Does NOT rely on auto-generated route names for deep linking

tests/dotnet-maui/maui-theming/eval.yaml:128

• The rubric items in this new stimulus are unquoted, but the repo’s skill-coverage tooling only extracts rubric evidence from single-line quoted items. Leaving these unquoted means they won’t count toward coverage reporting for this skill.

    rubric:
      - Removes only the previously applied theme dictionary, then adds the new one
      - Does NOT call MergedDictionaries.Clear(), which would also remove the template's Colors.xaml and Styles.xaml and unstyle the whole app
      - Keeps a reference to the current theme so it can be removed on the next switch
      - Produces code that leaves the app's existing styles intact after repeated switches

tests/dotnet-maui/maui-dependency-injection/eval.yaml:125

• These rubric items were added unquoted, which means the skill-coverage script won’t extract them as evidence (it only matches quoted rubric list items). Quoting them keeps coverage reporting accurate.

    rubric:
      - Explains that ContentTemplate pages are created with Activator.CreateInstance and bypass DI entirely
      - Explains that routes resolved via GoToAsync go through the service provider and inject constructor dependencies
      - Does NOT claim that dependencies are silently set to null without an exception
      - Recommends navigating by route, or giving the page a parameterless constructor, for the ContentTemplate case

tests/dotnet-maui/maui-collectionview/eval.yaml:129

• This new stimulus adds unquoted rubric items. The repo’s skill-coverage report only counts single-line quoted rubric entries, so these won’t contribute to coverage unless they’re quoted.

    rubric:
      - Sets ItemSizingStrategy="MeasureFirstItem" on the CollectionView element itself
      - Does NOT set ItemSizingStrategy on LinearItemsLayout or GridItemsLayout, which does not compile
      - Explains that MeasureFirstItem is only safe when every item is the same size
      - Warns that variable-height items will be clipped or stretched, and that MeasureAllItems is the default

[github-actions]

📊 Skill Evaluation Results

7 skill(s) evaluated — 7 improved, 0 no credible improvement. A skill passes only on a credible improvement over baseline (mean preference > 0 with its 95% CI above 0); ⚠️ marks a comparison that couldn't complete (errored/unmatched trials).

Skill	Result	Δ Preference [95% CI]	W/T/L	Quality	Baseline	Overfit	Skills Loaded
maui-app-lifecycle	✅	+51.7% [+31.8%, +71.5%]	11/1/0	4.4/5	3.4/5	✅ 0.19	4/4 · 4/4 (plugin)
maui-collectionview	✅	+32.0% [+22.8%, +41.2%]	12/3/0	5.0/5	5.0/5	✅ 0.07	5/5 · 5/5 (plugin)
maui-data-binding	✅	+40.0% [+40.0%, +40.0%]	12/0/0	4.9/5	4.2/5	✅ 0.11	4/4 · 4/4 (plugin)
maui-dependency-injection	✅	+41.3% [+30.7%, +52.0%]	14/1/0	4.8/5	4.6/5	✅ 0.07	5/5 · 5/5 (plugin)
maui-safe-area	✅	+100.0% [+100.0%, +100.0%]	12/0/0	5.0/5	2.2/5	✅ 0.08	4/4 · 4/4 (plugin)
maui-shell-navigation	✅	+46.7% [+29.6%, +63.8%]	13/2/0	5.0/5	3.8/5	✅ 0.12	5/5 · 5/5 (plugin)
maui-theming	✅	+34.7% [+26.9%, +42.5%]	13/2/0	5.0/5	5.0/5	✅ 0.10	5/5 · 5/5 (plugin)

ℹ️ Column legend

• Δ Preference — mean head-to-head preference of skilled vs baseline (−100%…+100%), judged by vally compare.
• [95% CI] — 95% confidence interval on that mean; a skill passes only when the whole interval is above 0.
• W/T/L — wins / ties / losses across trials.
• Quality / Baseline — mean absolute judge score 0–5 (skilled isolated vs skill-free control).
• Overfit — overfitting-judge severity (✅ Low, 🟡 Moderate, 🔴 High, — none) with its score.
• Skills Loaded — of the scenarios that expect activation, how many actually activated / that total (plugin run shown when present); ⚠️ marks a scenario that expected activation but didn't activate.

✅ maui-app-lifecycle — details

Reason: Mean preference +51.7% [95% CI 31.8%, 71.5%], win rate 91.7% (11W/1T/0L over 12 trial(s)) — credibly better

Scenario	Mean preference	Trials (W/T/L)
▲ Avoid legacy Xamarin.Forms lifecycle methods	+40.0%	3/0/0
▲ Platform-specific lifecycle mapping	+40.0%	3/0/0
▲ Save and restore state on background	+100.0%	3/0/0
▲ Window lifecycle event subscription	+26.7%	2/1/0

✅ maui-collectionview — details

Reason: Mean preference +32.0% [95% CI 22.8%, 41.2%], win rate 80.0% (12W/3T/0L over 15 trial(s)) — credibly better

Scenario	Mean preference	Trials (W/T/L)
▲ Avoid ListView and ViewCell mistakes	+40.0%	3/0/0
▲ Basic CollectionView with data binding and DataTemplate	+26.7%	2/1/0
▲ Grid layout with CollectionView	+13.3%	1/2/0
▲ ItemSizingStrategy placement for uniform items	+40.0%	3/0/0
▲ Selection and pull-to-refresh with CollectionView	+40.0%	3/0/0

✅ maui-data-binding — details

Reason: Mean preference +40.0% [95% CI 40.0%, 40.0%], win rate 100.0% (12W/0T/0L over 12 trial(s)) — credibly better

Scenario	Mean preference	Trials (W/T/L)
▲ Create and use an IValueConverter	+40.0%	3/0/0
▲ Diagnose missing BindingContext and non-compiled bindings	+40.0%	3/0/0
▲ Implement MVVM ViewModel with ObservableObject	+40.0%	3/0/0
▲ Set up compiled bindings with x:DataType on a page	+40.0%	3/0/0

✅ maui-dependency-injection — details

Reason: Mean preference +41.3% [95% CI 30.7%, 52.0%], win rate 93.3% (14W/1T/0L over 15 trial(s)) — credibly better

Scenario	Mean preference	Trials (W/T/L)
▲ Avoid AddScoped pitfall in MAUI	+40.0%	3/0/0
▲ Diagnose a page whose injected dependencies are missing	+46.7%	2/1/0
▲ Platform-specific service registration with fallback	+40.0%	3/0/0
▲ Register services with correct lifetimes in MauiProgram.cs	+40.0%	3/0/0
▲ Shell navigation auto-resolves DI-registered pages	+40.0%	3/0/0

✅ maui-safe-area — details

Reason: Mean preference +100.0% [95% CI 100.0%, 100.0%], win rate 100.0% (12W/0T/0L over 12 trial(s)) — credibly better

Scenario	Mean preference	Trials (W/T/L)
▲ Avoid deprecated UseSafeArea in new .NET 10 project	+100.0%	3/0/0
▲ Edge-to-edge layout with SafeAreaEdges	+100.0%	3/0/0
▲ Handle notch and status bar safe areas on iOS	+100.0%	3/0/0
▲ Keyboard avoidance with safe area for chat UI	+100.0%	3/0/0

✅ maui-shell-navigation — details

Reason: Mean preference +46.7% [95% CI 29.6%, 63.8%], win rate 86.7% (13W/2T/0L over 15 trial(s)) — credibly better

Scenario	Mean preference	Trials (W/T/L)
▲ Diagnose common Shell navigation mistakes	+40.0%	3/0/0
▲ Handle back navigation and unsaved changes guard	+100.0%	3/0/0
▲ Navigate with parameters using GoToAsync	+26.7%	2/1/0
▲ Set up Shell navigation with tabs and flyout	+26.7%	2/1/0
▲ Stable routes for deep linking into tabs	+40.0%	3/0/0

✅ maui-theming — details

Reason: Mean preference +34.7% [95% CI 26.9%, 42.5%], win rate 86.7% (13W/2T/0L over 15 trial(s)) — credibly better

Scenario	Mean preference	Trials (W/T/L)
▲ Add light/dark mode support using AppThemeBinding	+40.0%	3/0/0
▲ Avoid common theming mistakes	+40.0%	3/0/0
▲ Create custom themes with ResourceDictionary switching	+40.0%	3/0/0
▲ Detect and respond to system theme changes	+26.7%	2/1/0
▲ Swap theme dictionaries without destroying app styles	+26.7%	2/1/0

🔍 Full Results - additional metrics and failure investigation steps

▶ Sessions Visualisation -- interactive replay of all evaluation sessions
📊 Session Analytics (preview) -- aggregated metrics across evaluation sessions

[kubaflo]

Ready for re-review — 7/7 confirmed on the current head

@Redth @jfversluis @davidortinau — this is ready whenever you have time. @AbhitejJohn / @JanKrivanek, flagging you too since you've been through the earlier rounds.

The latest evaluation ran against a1eec661, the exact current head — not an older commit:

Skill		Δ Preference [95% CI]	W/T/L	Overfit
maui-safe-area	✅	+100.0% [+100.0%, +100.0%]	12/0/0	✅ 0.08
maui-app-lifecycle	✅	+51.7% [+31.8%, +71.5%]	11/1/0	✅ 0.19
maui-shell-navigation	✅	+46.7% [+29.6%, +63.8%]	13/2/0	✅ 0.12
maui-dependency-injection	✅	+41.3% [+30.7%, +52.0%]	14/1/0	✅ 0.07
maui-data-binding	✅	+40.0% [+40.0%, +40.0%]	12/0/0	✅ 0.11
maui-theming	✅	+34.7% [+26.9%, +42.5%]	13/2/0	✅ 0.10
maui-collectionview	✅	+32.0% [+22.8%, +41.2%]	12/3/0	✅ 0.07

87 wins, 9 ties, 0 losses. Every CI clear of zero, every overfit score ✅ Low, and every skill activating 100% of the time it should.

What this fixes

Issue #895 had maui-collectionview as FIX-REGRESSION 🔴 P0 — "regresses ~30% of trials", 6 losses, plugin at 48% pass. It's now +32% with zero losses, plugin at 100%.

What's worth your attention as a reviewer

The substantive changes are four bugs that were teaching code that doesn't work, each verified by building or running a real MAUI app rather than by reading docs:

Bug	Evidence
ItemSizingStrategy documented on LinearItemsLayout	error MAUIX2002 — it's declared on StructuredItemsView
iOS .DidBecomeActive() / .WillResignActive()	error CS1061 — real names are .OnActivated() / .OnResignActivation()
MergedDictionaries.Clear() to swap themes	Runtime: merged count 2→1, the template's Primary resource stops resolving — silently unstyles the whole app
ShellContent without explicit Route=	Runtime: D_FAULT_ShellContent2, D_FAULT_ShellContent5 — non-sequential, order-dependent, can't be deep-linked

Plus two doc corrections: the XC0022–XC0025 table (wrong on all four rows) and the "unregistered pages get silently null dependencies" claim (false — ActivatorUtilities throws). All six are now guarded by regression scenarios in tests/, so they can't creep back.

Note the PR touches tests/dotnet-maui/ as well as plugins/, so it's in your CODEOWNERS scope both ways.

Two open questions for you

1. Two rubrics currently penalise accuracy. maui-safe-area asserts WindowSoftInputModeAdjust.Resize was "replaced by" the new API (it's neither removed nor obsolete), and maui-dependency-injection asserts Scoped "behaves like Singleton" (MAUI creates one IServiceScope per window via MauiContextExtensions.MakeWindowScope). I deliberately left those rubrics untouched so the before/after comparison stayed honest — happy to fix here or in a follow-up, your call.
2. Gate sensitivity. At runs: 1 with 4 scenarios, a single tie is mathematically disqualifying (3W/1T → CI [−1.8%, +61.8%]). I have three cases in this PR where byte-identical content flipped ✅→❌ between runs. runs: 3 or ≥5 scenarios per skill removes it — worth considering for the gate generally.

Happy to make any changes you'd like.

[github-actions]

📊 Skill Evaluation Results

7 skill(s) evaluated — 7 improved, 0 no credible improvement. A skill passes only on a credible improvement over baseline (mean preference > 0 with its 95% CI above 0); ⚠️ marks a comparison that couldn't complete (errored/unmatched trials).

Skill	Result	Δ Preference [95% CI]	W/T/L	Quality	Baseline	Overfit	Skills Loaded
maui-app-lifecycle	✅	+51.7% [+31.8%, +71.5%]	11/1/0	4.5/5	3.4/5	✅ 0.20	4/4 · 4/4 (plugin)
maui-collectionview	✅	+29.3% [+19.2%, +39.5%]	11/4/0	5.0/5	5.0/5	✅ 0.07	5/5 · 5/5 (plugin)
maui-data-binding	✅	+40.0% [+40.0%, +40.0%]	12/0/0	5.0/5	4.1/5	✅ 0.11	4/4 · 4/4 (plugin)
maui-dependency-injection	✅	+44.0% [+35.4%, +52.6%]	15/0/0	5.0/5	4.5/5	✅ 0.08	5/5 · 5/5 (plugin)
maui-safe-area	✅	+100.0% [+100.0%, +100.0%]	12/0/0	5.0/5	1.8/5	✅ 0.08	4/4 · 4/4 (plugin)
maui-shell-navigation	✅	+40.0% [+23.8%, +56.2%]	12/3/0	5.0/5	3.9/5	✅ 0.12	5/5 · 5/5 (plugin)
maui-theming	✅	+37.3% [+31.6%, +43.1%]	14/1/0	5.0/5	4.9/5	✅ 0.10	5/5 · 5/5 (plugin)

ℹ️ Column legend

• Δ Preference — mean head-to-head preference of skilled vs baseline (−100%…+100%), judged by vally compare.
• [95% CI] — 95% confidence interval on that mean; a skill passes only when the whole interval is above 0.
• W/T/L — wins / ties / losses across trials.
• Quality / Baseline — mean absolute judge score 0–5 (skilled isolated vs skill-free control).
• Overfit — overfitting-judge severity (✅ Low, 🟡 Moderate, 🔴 High, — none) with its score.
• Skills Loaded — of the scenarios that expect activation, how many actually activated / that total (plugin run shown when present); ⚠️ marks a scenario that expected activation but didn't activate.

✅ maui-app-lifecycle — details

Reason: Mean preference +51.7% [95% CI 31.8%, 71.5%], win rate 91.7% (11W/1T/0L over 12 trial(s)) — credibly better

Scenario	Mean preference	Trials (W/T/L)
▲ Avoid legacy Xamarin.Forms lifecycle methods	+40.0%	3/0/0
▲ Platform-specific lifecycle mapping	+40.0%	3/0/0
▲ Save and restore state on background	+100.0%	3/0/0
▲ Window lifecycle event subscription	+26.7%	2/1/0

✅ maui-collectionview — details

Reason: Mean preference +29.3% [95% CI 19.2%, 39.5%], win rate 73.3% (11W/4T/0L over 15 trial(s)) — credibly better

Scenario	Mean preference	Trials (W/T/L)
▲ Avoid ListView and ViewCell mistakes	+13.3%	1/2/0
▲ Basic CollectionView with data binding and DataTemplate	+26.7%	2/1/0
▲ Grid layout with CollectionView	+40.0%	3/0/0
▲ ItemSizingStrategy placement for uniform items	+40.0%	3/0/0
▲ Selection and pull-to-refresh with CollectionView	+26.7%	2/1/0

✅ maui-data-binding — details

Reason: Mean preference +40.0% [95% CI 40.0%, 40.0%], win rate 100.0% (12W/0T/0L over 12 trial(s)) — credibly better

Scenario	Mean preference	Trials (W/T/L)
▲ Create and use an IValueConverter	+40.0%	3/0/0
▲ Diagnose missing BindingContext and non-compiled bindings	+40.0%	3/0/0
▲ Implement MVVM ViewModel with ObservableObject	+40.0%	3/0/0
▲ Set up compiled bindings with x:DataType on a page	+40.0%	3/0/0

✅ maui-dependency-injection — details

Reason: Mean preference +44.0% [95% CI 35.4%, 52.6%], win rate 100.0% (15W/0T/0L over 15 trial(s)) — credibly better

Scenario	Mean preference	Trials (W/T/L)
▲ Avoid AddScoped pitfall in MAUI	+40.0%	3/0/0
▲ Diagnose a page whose injected dependencies are missing	+60.0%	3/0/0
▲ Platform-specific service registration with fallback	+40.0%	3/0/0
▲ Register services with correct lifetimes in MauiProgram.cs	+40.0%	3/0/0
▲ Shell navigation auto-resolves DI-registered pages	+40.0%	3/0/0

✅ maui-safe-area — details

Reason: Mean preference +100.0% [95% CI 100.0%, 100.0%], win rate 100.0% (12W/0T/0L over 12 trial(s)) — credibly better

Scenario	Mean preference	Trials (W/T/L)
▲ Avoid deprecated UseSafeArea in new .NET 10 project	+100.0%	3/0/0
▲ Edge-to-edge layout with SafeAreaEdges	+100.0%	3/0/0
▲ Handle notch and status bar safe areas on iOS	+100.0%	3/0/0
▲ Keyboard avoidance with safe area for chat UI	+100.0%	3/0/0

✅ maui-shell-navigation — details

Reason: Mean preference +40.0% [95% CI 23.8%, 56.2%], win rate 80.0% (12W/3T/0L over 15 trial(s)) — credibly better

Scenario	Mean preference	Trials (W/T/L)
▲ Diagnose common Shell navigation mistakes	+26.7%	2/1/0
▲ Handle back navigation and unsaved changes guard	+80.0%	3/0/0
▲ Navigate with parameters using GoToAsync	+13.3%	1/2/0
▲ Set up Shell navigation with tabs and flyout	+40.0%	3/0/0
▲ Stable routes for deep linking into tabs	+40.0%	3/0/0

✅ maui-theming — details

Reason: Mean preference +37.3% [95% CI 31.6%, 43.1%], win rate 93.3% (14W/1T/0L over 15 trial(s)) — credibly better

Scenario	Mean preference	Trials (W/T/L)
▲ Add light/dark mode support using AppThemeBinding	+40.0%	3/0/0
▲ Avoid common theming mistakes	+40.0%	3/0/0
▲ Create custom themes with ResourceDictionary switching	+40.0%	3/0/0
▲ Detect and respond to system theme changes	+26.7%	2/1/0
▲ Swap theme dictionaries without destroying app styles	+40.0%	3/0/0

🔍 Full Results - additional metrics and failure investigation steps

[kubaflo]

Thanks @AbhitejJohn — and good catch. Straight answer: no, not on the final changes.

Your cross-family run (30313685452, 23/35, no regressions) was against 9c02b413. Six commits have landed since, and they're the substantive ones — not cosmetics:

Commit	What changed
5812d5af	First regression-guard scenario + version-pinned the SDK-sensitive claims
1db8796d	theming: single-source palette instead of scattered literals
714b5975	theming: never MergedDictionaries.Clear() (runtime-verified: merged 2→1, template's Primary stops resolving)
6d1201c3	shell: explicit Route= (runtime-verified: D_FAULT_ShellContent2/5) · DI: IQueryAttributable for nav params
cf626334	3 more guard scenarios — tests/ changed, so the eval surface itself is different
fcb7e560	SetAppTheme wrong-property-owner fix (your reviewer bot caught it)

Those are precisely the commits that took the gate from 4/7 to 7/7, so the old cross-family numbers understate the current state — but they equally don't prove it. A fresh matrix is the right ask.

What I can vouch for on the current head a1eec661: two independent gate runs, both 7/7 with identical totals — 87W / 9T / 0L, every CI clear of zero, every overfit ✅ Low, 100% activation:

Run on a1eec661	Result
13:06	7/7 · 87W/9T/0L
15:11	7/7 · 87W/9T/0L

Two runs agreeing exactly on byte-identical content is itself a result — earlier in this PR, identical content flipped ✅→❌ three times at 4 scenarios. The 5th scenario per skill removed that.

But that's still one executor family. It doesn't tell you whether the theming/shell/DI rewrites help or hurt sonnet/haiku/mai, which is exactly what the matrix is for.

Could you re-dispatch the cross-family workflow against a1eec661? I can't — fork PR, Actor kubaflo has permission: read.

In the meantime I've kicked off a local cross-family approximation on the three most-changed skills (theming, shell-navigation, DI) with a GPT executor judged by Claude — judge ≠ executor family, same as your matrix rule. It's directional only (one extra family, my machine), but it should flag any obvious non-Claude regression before you spend CI on it. I'll post the numbers when it finishes.

I would also hope that Jonathan/Gerald would be the final approvers cause they know maui so much better than me.

Completely agree — @jfversluis and @Redth are the right final approvers; they're already CODEOWNERS on both plugins/dotnet-maui/ and tests/dotnet-maui/. Your eval-side review has been the more valuable half here anyway: the "no eval guarding the regression" point is what produced the four guard scenarios, and your reviewer bot caught a real bug I'd introduced.

Worth their attention specifically: the four fixes are things that were teaching code that doesn't work — MAUIX2002 on ItemSizingStrategy, CS1061 on the iOS lifecycle builder, Clear() silently unstyling the app, and unstable D_FAULT_ routes. Each verified by building or running a real MAUI app, not by reading docs.

[AbhitejJohn]

🔄 Re-running cross-family evaluation on the latest commit (a1eec661), triggered on behalf of a maintainer.

• Scope: plugins=dotnet-maui, all 7 changed skills · executors opus, gpt, sonnet46, haiku, mai · runs=3, workers=3 (35 matrix cells)
• Judge: cross-family (judge ≠ executor) — opus → gpt-5.5; all others → claude-opus-4.8
• ETA: ~60–90 minutes (the previous 35-cell run took ~56 min)

Updated results will be posted here when the run completes.

[kubaflo]

Thanks for kicking that off @AbhitejJohn — that's exactly the right call, and the official matrix is the number that counts.

My local approximation finished in the meantime. Directional only, but it points the same way:

Executor gpt-5.5 → judge claude-opus-4.8 (cross-family, judge ≠ executor), 3 most-changed skills, runs: 3:

Skill	W/T/L	Mean	Scored / errored
maui-theming	7/0/0	+0.40	7 / 8
maui-shell-navigation	5/0/0	+0.40	5 / 10
maui-dependency-injection	7/0/1	+0.30	8 / 7
pooled	19/0/1	+0.36 [+0.28, +0.44]	20 / 25

⚠️ Heavy caveat: 25 of 45 trials errored — local token-pool/rate-limit noise on my machine, not a content signal. Vally excludes errored trials from the verdict (same convention as the #909 correction), but a 56% error rate means treat this as a smoke test, not a measurement. Yours is the real one.

What it's useful for: the three skills that changed most since your 9c02b413 matrix — the theming palette/Clear() rewrites, the shell Route= rule, and the DI IQueryAttributable section — show no sign of a non-Claude regression. That was the open risk in asking you to re-run.

The single loss is on Shell navigation auto-resolves DI-registered pages, and the judge called it modest — baseline was more literally responsive to "full wiring from MauiProgram.cs through to GoToAsync" by including App.xaml.cs and the AppShell.xaml ShellContent. Worth watching in your run; if it reproduces across families I'll widen that section rather than leave it.

[kubaflo]

⚠️ Correction — I need to flag something before your run finishes

@AbhitejJohn @JanKrivanek — while cleaning up after the local run I found that cf626334 accidentally included a change to dotnet-skills.experiment.yaml:

 overrides:
   model: claude-opus-4.6
   judge_model: claude-opus-4.6
-  runs: 1
+  runs: 3

That file is not dotnet-maui-scoped — it drives the experiment for every plugin's evals, so it tripled trial counts repo-wide. It leaked in from my local measurement loop (I set runs: 3 for a usable sample and restore it from a copy afterwards; on that one commit the restore was missed and git add -A picked it up).

Reverted in b02c5a20. No skill or eval content is affected — the diff is that one line.

How this distorted the numbers I reported

Gate runs from cf626334 onward executed at 3 trials per scenario instead of 1 — which is why they show 12–15 trials per skill rather than 4. I previously attributed that to a maintainer-side change; it was mine.

The measurements are still valid — more trials of the same content, not different content — but they are not what the gate produces at upstream's runs: 1, and the two 7/7 results on a1eec661 should be read with that caveat.

It also makes my earlier "consider keeping runs: 3" suggestion look self-serving in hindsight, which I regret. The underlying evidence stands on its own (byte-identical content flipped ✅→❌ three times at 4 scenarios × 1 run), but that's a deliberate call for you to make — not something that should ride in on an unnoticed diff.

What this means for the run you just started

Your cross-family dispatch passes runs=3 explicitly as a workflow input, so it is unaffected — it was always going to run 3 trials per cell regardless of this file. Its results stand.

The thing to re-check is the PR gate on the new head b02c5a20: it'll now run at runs: 1 as upstream intends, which is the stricter bar, and is where the single-tie sensitivity I described will bite again. If a skill drops to ❌ there on an unchanged-content commit, that's the sampling artifact — not a regression — and the cross-family matrix is the better evidence.

Sorry for the noise. Better to catch it now than to have it merge silently.

[Copilot]

Pull request overview

Copilot reviewed 14 out of 14 changed files in this pull request and generated no new comments.

Comments suppressed due to low confidence (4)

tests/dotnet-maui/maui-theming/eval.yaml:128

• These newly added rubric items are unquoted YAML scalars. Our skill-coverage tooling only extracts rubric evidence from single-line quoted items, so these won’t contribute to coverage metrics and can under-report evidence for this scenario.

    rubric:
      - Removes only the previously applied theme dictionary, then adds the new one
      - Does NOT call MergedDictionaries.Clear(), which would also remove the template's Colors.xaml and Styles.xaml and unstyle the whole app
      - Keeps a reference to the current theme so it can be removed on the next switch
      - Produces code that leaves the app's existing styles intact after repeated switches

tests/dotnet-maui/maui-shell-navigation/eval.yaml:133

• These newly added rubric items are unquoted YAML scalars. Our skill-coverage tooling only extracts rubric evidence from single-line quoted items, so these won’t contribute to coverage metrics and can under-report evidence for this scenario.

    rubric:
      - Sets an explicit Route on the ShellContent elements in AppShell.xaml
      - Explains that without Route, MAUI auto-generates an unstable name (D_FAULT_ prefixed) that shifts when pages are reordered or added
      - Shows an absolute GoToAsync route (leading //) built from the explicit route names
      - Does NOT rely on auto-generated route names for deep linking

tests/dotnet-maui/maui-dependency-injection/eval.yaml:125

• These newly added rubric items are unquoted YAML scalars. Our skill-coverage tooling only extracts rubric evidence from single-line quoted items, so these won’t contribute to coverage metrics and can under-report evidence for this scenario.

    rubric:
      - Explains that ContentTemplate pages are created with Activator.CreateInstance and bypass DI entirely
      - Explains that routes resolved via GoToAsync go through the service provider and inject constructor dependencies
      - Does NOT claim that dependencies are silently set to null without an exception
      - Recommends navigating by route, or giving the page a parameterless constructor, for the ContentTemplate case

tests/dotnet-maui/maui-collectionview/eval.yaml:129

• These newly added rubric items are unquoted YAML scalars. Our skill-coverage tooling only extracts rubric evidence from single-line quoted items, so these won’t contribute to coverage metrics and can under-report evidence for this scenario.

    rubric:
      - Sets ItemSizingStrategy="MeasureFirstItem" on the CollectionView element itself
      - Does NOT set ItemSizingStrategy on LinearItemsLayout or GridItemsLayout, which does not compile
      - Explains that MeasureFirstItem is only safe when every item is the same size
      - Warns that variable-height items will be clipped or stretched, and that MeasureAllItems is the default

[kubaflo]

/evaluate b02c5a2

[github-actions]

📊 Skill Evaluation Results

7 skill(s) evaluated — 7 improved, 0 no credible improvement. A skill passes only on a credible improvement over baseline (mean preference > 0 with its 95% CI above 0); ⚠️ marks a comparison that couldn't complete (errored/unmatched trials).

Skill	Result	Δ Preference [95% CI]	W/T/L	Quality	Baseline	Overfit	Skills Loaded
maui-app-lifecycle	✅	+55.0% [+7.3%, +102.7%]	4/0/0	4.5/5	3.4/5	✅ 0.16	4/4 · 4/4 (plugin)
maui-collectionview	✅	+40.0% [+40.0%, +40.0%]	5/0/0	5.0/5	5.0/5	✅ 0.07	5/5 · 5/5 (plugin)
maui-data-binding	✅	+40.0% [+40.0%, +40.0%]	4/0/0	5.0/5	3.9/5	✅ 0.11	4/4 · 4/4 (plugin)
maui-dependency-injection	✅	+40.0% [+40.0%, +40.0%]	5/0/0	5.0/5	4.6/5	✅ 0.08	5/5 · 5/5 (plugin)
maui-safe-area	✅	+85.0% [+37.3%, +132.7%]	4/0/0	5.0/5	2.1/5	✅ 0.08	4/4 · 4/4 (plugin)
maui-shell-navigation	✅	+56.0% [+2.2%, +109.8%]	4/1/0	5.0/5	3.2/5	✅ 0.14	5/5 · 5/5 (plugin)
maui-theming	✅	+32.0% [+9.8%, +54.2%]	4/1/0	5.0/5	4.9/5	✅ 0.10	5/5 · 5/5 (plugin)

ℹ️ Column legend

• Δ Preference — mean head-to-head preference of skilled vs baseline (−100%…+100%), judged by vally compare.
• [95% CI] — 95% confidence interval on that mean; a skill passes only when the whole interval is above 0.
• W/T/L — wins / ties / losses across trials.
• Quality / Baseline — mean absolute judge score 0–5 (skilled isolated vs skill-free control).
• Overfit — overfitting-judge severity (✅ Low, 🟡 Moderate, 🔴 High, — none) with its score.
• Skills Loaded — of the scenarios that expect activation, how many actually activated / that total (plugin run shown when present); ⚠️ marks a scenario that expected activation but didn't activate.

✅ maui-app-lifecycle — details

Reason: Mean preference +55.0% [95% CI 7.3%, 102.7%], win rate 100.0% (4W/0T/0L over 4 trial(s)) — credibly better

Scenario	Mean preference	Trials (W/T/L)
▲ Avoid legacy Xamarin.Forms lifecycle methods	+40.0%	1/0/0
▲ Platform-specific lifecycle mapping	+40.0%	1/0/0
▲ Save and restore state on background	+100.0%	1/0/0
▲ Window lifecycle event subscription	+40.0%	1/0/0

✅ maui-collectionview — details

Reason: Mean preference +40.0% [95% CI 40.0%, 40.0%], win rate 100.0% (5W/0T/0L over 5 trial(s)) — credibly better

Scenario	Mean preference	Trials (W/T/L)
▲ Avoid ListView and ViewCell mistakes	+40.0%	1/0/0
▲ Basic CollectionView with data binding and DataTemplate	+40.0%	1/0/0
▲ Grid layout with CollectionView	+40.0%	1/0/0
▲ ItemSizingStrategy placement for uniform items	+40.0%	1/0/0
▲ Selection and pull-to-refresh with CollectionView	+40.0%	1/0/0

✅ maui-data-binding — details

Reason: Mean preference +40.0% [95% CI 40.0%, 40.0%], win rate 100.0% (4W/0T/0L over 4 trial(s)) — credibly better

Scenario	Mean preference	Trials (W/T/L)
▲ Create and use an IValueConverter	+40.0%	1/0/0
▲ Diagnose missing BindingContext and non-compiled bindings	+40.0%	1/0/0
▲ Implement MVVM ViewModel with ObservableObject	+40.0%	1/0/0
▲ Set up compiled bindings with x:DataType on a page	+40.0%	1/0/0

✅ maui-dependency-injection — details

Reason: Mean preference +40.0% [95% CI 40.0%, 40.0%], win rate 100.0% (5W/0T/0L over 5 trial(s)) — credibly better

Scenario	Mean preference	Trials (W/T/L)
▲ Avoid AddScoped pitfall in MAUI	+40.0%	1/0/0
▲ Diagnose a page whose injected dependencies are missing	+40.0%	1/0/0
▲ Platform-specific service registration with fallback	+40.0%	1/0/0
▲ Register services with correct lifetimes in MauiProgram.cs	+40.0%	1/0/0
▲ Shell navigation auto-resolves DI-registered pages	+40.0%	1/0/0

✅ maui-safe-area — details

Reason: Mean preference +85.0% [95% CI 37.3%, 132.7%], win rate 100.0% (4W/0T/0L over 4 trial(s)) — credibly better

Scenario	Mean preference	Trials (W/T/L)
▲ Avoid deprecated UseSafeArea in new .NET 10 project	+100.0%	1/0/0
▲ Edge-to-edge layout with SafeAreaEdges	+100.0%	1/0/0
▲ Handle notch and status bar safe areas on iOS	+40.0%	1/0/0
▲ Keyboard avoidance with safe area for chat UI	+100.0%	1/0/0

✅ maui-shell-navigation — details

Reason: Mean preference +56.0% [95% CI 2.2%, 109.8%], win rate 80.0% (4W/1T/0L over 5 trial(s)) — credibly better

Scenario	Mean preference	Trials (W/T/L)
▲ Diagnose common Shell navigation mistakes	+40.0%	1/0/0
▲ Handle back navigation and unsaved changes guard	+100.0%	1/0/0
= Navigate with parameters using GoToAsync	+0.0%	0/1/0
▲ Set up Shell navigation with tabs and flyout	+100.0%	1/0/0
▲ Stable routes for deep linking into tabs	+40.0%	1/0/0

✅ maui-theming — details

Reason: Mean preference +32.0% [95% CI 9.8%, 54.2%], win rate 80.0% (4W/1T/0L over 5 trial(s)) — credibly better

Scenario	Mean preference	Trials (W/T/L)
▲ Add light/dark mode support using AppThemeBinding	+40.0%	1/0/0
▲ Avoid common theming mistakes	+40.0%	1/0/0
▲ Create custom themes with ResourceDictionary switching	+40.0%	1/0/0
▲ Detect and respond to system theme changes	+40.0%	1/0/0
= Swap theme dictionaries without destroying app styles	+0.0%	0/1/0

🔍 Full Results - additional metrics and failure investigation steps

▶ Sessions Visualisation -- interactive replay of all evaluation sessions
📊 Session Analytics (preview) -- aggregated metrics across evaluation sessions

[kubaflo]

✅ 7/7 at the restored runs: 1 — the strict bar

The gate has now run on b02c5a20 (the revert commit), i.e. at upstream's runs: 1 rather than the runs: 3 my leaked config had been forcing. All 7 still pass:

Skill		Δ Preference [95% CI]	W/T/L
maui-safe-area	✅	+85.0% [+37.3%, +132.7%]	4/0/0
maui-shell-navigation	✅	+56.0% [+2.2%, +109.8%]	4/1/0
maui-app-lifecycle	✅	+55.0% [+7.3%, +102.7%]	4/0/0
maui-collectionview	✅	+40.0% [+40.0%, +40.0%]	5/0/0
maui-data-binding	✅	+40.0% [+40.0%, +40.0%]	4/0/0
maui-dependency-injection	✅	+40.0% [+40.0%, +40.0%]	5/0/0
maui-theming	✅	+32.0% [+9.8%, +54.2%]	4/1/0

30 wins, 2 ties, 0 losses. All overfit ✅ Low, all activation 100%.

The 5-scenario design is what carried it

This run is a clean natural experiment for the tie-sensitivity point:

• maui-shell-navigation and maui-theming each drew a tie — and both still passed, at 4W/1T over 5 scenarios.
• At 4 scenarios that same 3W/1T shape yields [−1.8%, +61.8%] and fails.

So the three skills that previously flapped are now insulated by the regression-guard scenarios, and they earned it on the first strict run. The four skills still on 4 scenarios (app-lifecycle, data-binding, safe-area) happened to draw no ties this time — they remain the ones exposed to the artifact, which is a repo-wide observation rather than anything this PR needs to fix.

Where the PR stands

• 7/7 at runs: 1 (this run) and 7/7 twice at runs: 3 on a1eec661 — consistent across both settings.
• Cross-family on 9c02b413: 23/35 cells, no regressions, every cell net-positive or neutral.
• Local cross-family smoke test on the changed skills: 19W/0T/1L (directional; high error rate disclosed).
• Scope verified: exactly 14 files, all under plugins/dotnet-maui/ and tests/dotnet-maui/.

@AbhitejJohn — the cross-family run you announced at 19:01Z hasn't posted results and I can't see an evaluation-run dispatch for it in the workflow list; it may not have taken. No rush from my side given the above, but flagging in case it failed silently.

@jfversluis @Redth — ready for your review whenever suits.

[AbhitejJohn]

@kubaflo : Sorry, yes. It's underway - needed to be unstuck.

[AbhitejJohn]

🧪 Cross-family evaluation — re-run on latest commit (a1eec661)

Run: https://github.com/dotnet/skills/actions/runs/30496418697
Scope: plugins=dotnet-maui, all 7 changed skills · runs=3, workers=3 · 35 matrix cells
Overall: 29/35 pass — up from 23/35 on the previous commit, still no regressions (every cell's mean preference ≥ 0).

Judge rule (verified): opus → gpt-5.5; gpt/sonnet46/haiku/mai → claude-opus-4.8.

Pass matrix (✅ pass · ⚠️ inconclusive-unmatched · ➖ not-credible)

Skill	opus	gpt	sonnet46	haiku	mai	Pass	Δ vs prev
maui-app-lifecycle	✅	✅	✅	✅	✅	5/5	=
maui-theming	✅	✅	✅	✅	✅	5/5	🔼 +3
maui-safe-area	✅	✅	⚠️	✅	✅	4/5	=
maui-dependency-injection	✅	✅	✅	⚠️	✅	4/5	🔼 +2
maui-data-binding	➖	✅	✅	✅	✅	4/5	🔼 +1
maui-shell-navigation	✅	✅	✅	⚠️	✅	4/5	🔽 −1
maui-collectionview	➖	✅	✅	✅	➖	3/5	🔼 +1
Per-executor	5/7	7/7	6/7	5/7	6/7	29/35

Per-cell detail (win rate W/T/L · mean preference)

Skill	opus	gpt	sonnet46	haiku	mai
app-lifecycle	✅ 83% 10/1/1 +0.45	✅ 100% 12/0/0 +0.45	✅ 92% 11/1/0 +0.52	✅ 100% 12/0/0 +1.00	✅ 100% 12/0/0 +0.75
collectionview	➖ 33% 5/6/4 +0.03	✅ 73% 11/4/0 +0.29	✅ 53% 8/6/1 +0.19	✅ 100% 15/0/0 +0.48	➖ 53% 8/5/2 +0.16
data-binding	➖ 58% 7/1/4 +0.10	✅ 100% 12/0/0 +0.40	✅ 75% 9/3/0 +0.30	✅ 100% 12/0/0 +0.45	✅ 67% 8/3/1 +0.23
dependency-injection	✅ 93% 14/0/1 +0.43	✅ 73% 11/2/2 +0.24	✅ 87% 13/0/2 +0.45	⚠️ 100% 14/0/0 +0.70	✅ 73% 11/4/0 +0.29
safe-area	✅ 83% 10/2/0 +0.58	✅ 83% 10/1/1 +0.55	⚠️ 89% 8/0/1 +0.84	✅ 100% 12/0/0 +1.00	✅ 92% 11/1/0 +0.87
shell-navigation	✅ 93% 14/0/1 +0.35	✅ 87% 13/2/0 +0.43	✅ 93% 14/1/0 +0.57	⚠️ 93% 13/1/0 +0.76	✅ 80% 12/0/3 +0.32
theming	✅ 80% 12/3/0 +0.32	✅ 100% 15/0/0 +0.40	✅ 93% 14/1/0 +0.41	✅ 100% 15/0/0 +0.76	✅ 73% 11/2/2 +0.48

What changed vs the previous commit (23/35 → 29/35)

• maui-theming fixed across the board — was 2/5, now 5/5 (opus, sonnet46, mai all flipped to conclusive passes).
• maui-dependency-injection 2/5 → 4/5; maui-data-binding 3/5 → 4/5; maui-collectionview 2/5 → 3/5.
• mai executor jumped 2/7 → 6/7; sonnet46 4/7 → 6/7; opus 4/7 → 5/7.
• Minor slips: maui-shell-navigation 5/5 → 4/5 and maui-safe-area sonnet46 now ⚠️ — both are unmatched-trajectory artifacts (still strongly positive: haiku shell-nav +0.76; sonnet46 safe-area +0.84), not quality losses.

Interpretation

• No regressions — all 35 cells net-positive. Remaining non-passes are the two familiar soft spots (maui-collectionview for opus/mai, opus maui-data-binding) where gains are real but small (CI includes 0), plus a few unmatched-trajectory inconclusives with strong positive means.
• gpt 7/7 again; the latest commit meaningfully lifts the Claude-family and MAI executors. Net a clear improvement over the prior revision.

Legend: ⚠️ inconclusive-unmatched = positive signal but ≥1 trajectory unmatched; ➖ not-credible = 95% CI includes 0 (small margin). Neither indicates a regression.