Skip to content

History-trained agentic files + expert reviewer (proposal)#35062

Closed
T-Gro wants to merge 2 commits into
dotnet:mainfrom
T-Gro:feature/expert-reviewer-extraction
Closed

History-trained agentic files + expert reviewer (proposal)#35062
T-Gro wants to merge 2 commits into
dotnet:mainfrom
T-Gro:feature/expert-reviewer-extraction

Conversation

@T-Gro
Copy link
Copy Markdown
Member

@T-Gro T-Gro commented Apr 21, 2026

Note

Are you waiting for the changes in this PR to be merged?
It would be very helpful if you could test the resulting artifacts from this PR and let us know in a comment if this change resolves your issue. Thank you!

Description

Replaces review-rules.md (flat 345-line checklist) with a dimensional expert review agent. Single source of truth for all review rules, organized into 30 dimensions for per-dimension sub-agent evaluation. Adds inline file:line PR comments alongside the existing wall-of-text summary.

Extracted from 28k review comments across 5 maintainers via extraction-pipeline. No functional code changes.

What changed

Before: review-rules.md had 345 lines of flat rules. code-review skill loaded them all into one context. Output was a single wall-of-text PR comment.

After: Rules absorbed into maui-expert-reviewer.md as 30 dimensions with 200+ CHECK items. Each dimension runs as an independent sub-agent with focused context. Output is inline file:line PR comments via inline-findings.json.

CI Flow

Review-PR.ps1 prompt:
  1. code-review → maui-expert-reviewer agent → inline-findings.json
  2. pr-review → Pre-Flight → Try-Fix → Report (sees findings, no duplication)

Posting:
  post-inline-review.ps1    → .json → GitHub file:line comments (NEW)
  post-ai-summary-comment.ps1 → {phase}/content.md → wall-of-text (existing)

CI: COMMENTS_VIA_FILE=true → agent writes .json, script posts
Local: agent writes .json, code-review posts directly via gh api

Files

Action File What
Add agents/maui-expert-reviewer.md 30 dimensions, 200+ CHECKs, routing table
Add instructions/collectionview-{android,ios,windows} Platform-isolated CV rules
Add instructions/{handler-patterns,layout-system,performance-hotpaths,public-api,threading-async} Domain-specific ambient guidance
Add scripts/post-inline-review.ps1 Posts .json as GitHub PR review
Del skills/code-review/references/review-rules.md Absorbed into agent
Mod skills/code-review/SKILL.md Delegates to agent
Mod scripts/Review-PR.ps1 Prompt + inline posting wiring
Mod eng/pipelines/ci-copilot.yml COMMENTS_VIA_FILE env var

Copilot AI review requested due to automatic review settings April 21, 2026 14:19
@github-actions
Copy link
Copy Markdown
Contributor

github-actions Bot commented Apr 21, 2026

🚀 Dogfood this PR with:

⚠️ WARNING: Do not do this without first carefully reviewing the code of this PR to satisfy yourself it is safe.

curl -fsSL https://raw.githubusercontent.com/dotnet/maui/main/eng/scripts/get-maui-pr.sh | bash -s -- 35062

Or

  • Run remotely in PowerShell:
iex "& { $(irm https://raw.githubusercontent.com/dotnet/maui/main/eng/scripts/get-maui-pr.ps1) } 35062"

Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds a proposal set of GitHub Copilot “agentic artifacts” for .NET MAUI: a MAUI expert reviewer agent, a routing skill to activate review dimensions based on changed paths/platforms, and scoped instruction files for common MAUI hotspots (handlers/layout/perf/public API/threading + CollectionView per platform).

Changes:

  • Added maui-expert-reviewer agent with a dimension-based review workflow and CHECK items.
  • Added expert-review routing skill with a folder→dimension table and platform detection rules.
  • Added 8 folder-scoped instruction files for CollectionView (Android/iOS/Windows) and general MAUI review areas.

Reviewed changes

Copilot reviewed 10 out of 10 changed files in this pull request and generated 9 comments.

Show a summary per file
File Description
.github/agents/maui-expert-reviewer.md New “expert reviewer” agent definition and workflow across 21 MAUI review dimensions.
.github/skills/expert-review/SKILL.md New dimension router skill (path routing table + platform detection + selection algorithm).
.github/instructions/collectionview-android.instructions.md Android CollectionView (RecyclerView) review guidance.
.github/instructions/collectionview-ios.instructions.md iOS/MacCatalyst CollectionView (UICollectionView/Items2) review guidance + deprecation notes.
.github/instructions/collectionview-windows.instructions.md Windows CollectionView (WinUI) review guidance.
.github/instructions/handler-patterns.instructions.md Handler mapper/lifecycle/null-safety/native-defaults guidance.
.github/instructions/layout-system.instructions.md Layout measure/arrange contract + constraint propagation + perf notes.
.github/instructions/performance-hotpaths.instructions.md Hot-path allocation/perf guidance for layout/scroll/binding paths.
.github/instructions/public-api.instructions.md Public API guidance + PublicAPI.Unshipped workflow pointer.
.github/instructions/threading-async.instructions.md Threading/async/UI-thread dispatch and race-condition guidance.

Comment thread .github/skills/expert-review/SKILL.md Outdated
Comment thread .github/skills/expert-review/SKILL.md Outdated
Comment thread .github/skills/expert-review/SKILL.md Outdated
Comment thread .github/instructions/threading-async.instructions.md Outdated
Comment thread .github/skills/expert-review/SKILL.md Outdated
Comment thread .github/skills/expert-review/SKILL.md Outdated
Comment thread .github/agents/maui-expert-reviewer.md Outdated
Comment thread .github/agents/maui-expert-reviewer.md Outdated
Comment thread .github/skills/expert-review/SKILL.md Outdated
@PureWeen
Copy link
Copy Markdown
Member

PureWeen commented Apr 21, 2026

Code Review — PR #35062 (Updated)

Scope: Docs/agent/skill/instruction changes only — no src/ changes in this PR


Overall Assessment

The data in this PR is genuinely valuable — ~57 of the 97 CHECK items are novel content not found anywhere in the existing review infrastructure. The extraction surfaced institutional knowledge that was previously only in PR comment threads (native defaults preservation, CollectionView per-platform specifics, trimming/AOT patterns, type choice guidance). The routing table concept — mapping file paths to review focus areas — is the single most innovative idea in the PR.

However, the architecture creates a parallel review system alongside the existing code-review skill, which already covers a substantial portion of the same CHECK items in different phrasing. Two parallel rule sets will diverge over time.


Recommendation: Restructure, Don't Discard

✅ KEEP all 8 instruction files (with fixes)

These follow the repo's existing pattern exactly (android.instructions.md, safe-area-ios.instructions.md, etc.) and use applyTo scoping correctly. They provide ambient context automatically when relevant files are touched.

The 3 CollectionView per-platform files are especially valuable and should be kept as separate applyTo-scoped files:

  • collectionview-android.instructions.md — RecyclerView adapter/ViewHolder/LayoutManager patterns (Items/ handler)
  • collectionview-ios.instructions.md — UICollectionView cell measurement/memory/static callbacks (Items2/ handler)
  • collectionview-windows.instructions.md — WinUI ListView/ItemsRepeater and NaN conventions (Items/ handler)

These are scoped to their respective handler directories and will grow as native platform guidance is added over time.

The remaining 5 general instruction files:

  • handler-patterns.instructions.md
  • layout-system.instructions.md
  • performance-hotpaths.instructions.md
  • public-api.instructions.md
  • threading-async.instructions.md

Fix these issues before merging:

  • Convert applyTo from comma-separated strings to YAML array format (matching the existing repo convention in android.instructions.md, safe-area-ios.instructions.md, etc.)
  • layout-system.instructions.md: change Layout*Layout/** (currently overfires on data-type files like LayoutOptions.cs)
  • performance-hotpaths.instructions.md: narrow applyTo from src/Core/src/** to specific hot-path dirs (Layouts/**, Layout/**, scroll/binding paths)
  • public-api.instructions.md: narrow scope to **/PublicAPI.Unshipped.txt (the Core/Controls scope adds context noise for cold-path edits)
  • threading-async.instructions.md: add **/Platforms/** (plural) to catch test/template platform dirs

✅ KEEP the routing table — merge into code-review

The path→dimension routing belongs as a pre-processing step in the existing code-review skill, not as a standalone skill. Concretely:

  1. Extract the routing table from expert-review/SKILL.md into .github/skills/code-review/references/routing-table.md
  2. Add a "Step 0: Route changed files" to code-review/SKILL.md
  3. Fix the brace expansion patterns — {Application,Border,Button,...} is shell syntax, not valid glob. Expand to explicit rows or use **/Handlers/** with override patterns
  4. Fix the wrong base paths — src/Controls/src/Core/Handlers/{Button,...} should be src/Core/src/Handlers/ (Button, Entry, Label, etc. live in Core, not Controls)
  5. Resolve the algorithm contradiction — the intro says "union all matched dimensions" (additive) while Step 1 says "longest prefix wins" (exclusive). Pick one strategy and be consistent

✅ MERGE ~57 novel CHECK items into review-rules.md

These should enrich the existing .github/skills/code-review/references/review-rules.md, not live in a parallel file:

Novel content Where it goes in review-rules.md
Native Platform Defaults (Dim 14) — capture before override, restore on clear New §: "Native Defaults Preservation"
CollectionView Android specifics — range notifications, ViewHolder recycling Expand existing §10 or add §10a
CollectionView iOS specifics — cell measurement scoping, static callbacks Expand existing §10 or add §10b
Trimming/AOT (Dim 18) — Type.GetType() restrictions, DynamicallyAccessedMembers New §: "Trimming & AOT"
Type Choice (Dim 17) — struct boxing, record vs struct New §: "Type Choice"
Complexity Reduction (Dim 16) — don't reimplement existing infra New §: "Complexity"
Logic Verification — "verify against original repro, not just new test" Add to existing §21
Architectural Layer — compatibility shim isolation, cross-cutting via IView Add to existing §5 or new §
"Concrete scenario" requirement — discard findings without one Add to code-review workflow

⚠️ Recommend not adding the agent (maui-expert-reviewer)

The 334-line agent definition substantially overlaps with review-rules.md. Its 4-wave workflow restructures code-review's existing workflow without being fundamentally different. Having it in .github/agents/ creates a parallel invocation path that bypasses the pr-review → code-review pipeline. The agent also contradicts itself internally — "Wave 3" instructs posting ≤15 inline PR comments, while a later "Operational Notes" line says "terminal only, do NOT post PR comments."

The novel content survives in review-rules.md. The routing survives in code-review/references/routing-table.md. The instruction files provide platform-scoped context automatically.

⚠️ Recommend not adding the routing skill (expert-review/)

Its routing table moves into code-review/references/routing-table.md. The skill name mismatch (expert-reviewing in frontmatter vs expert-review/ folder) is one symptom — the deeper issue is that this functionality belongs inside code-review, not alongside it.


Summary

Component Action Rationale
3 CollectionView per-platform files ✅ Keep (with applyTo fixes) Per-platform handler guidance, will grow over time
5 general instruction files ✅ Keep (with applyTo fixes) Follows existing repo pattern, auto-scoped
Routing table ✅ Keep → move to code-review/references/ Best idea in the PR, wrong container
~57 novel CHECK items ✅ Keep → merge into review-rules.md Valuable institutional knowledge
"Concrete scenario" rule ✅ Keep → add to code-review workflow High-signal improvement
maui-expert-reviewer agent ⚠️ Don't add Overlaps existing system, internal contradictions, bypasses pipeline
expert-review/ skill ⚠️ Don't add Routing table moves to code-review

The PR's data is excellent work — the novel content is a strong hit rate from the extraction pipeline. The restructuring above preserves 100% of the new knowledge while avoiding a parallel review system.

@dotnet dotnet deleted a comment from kubaflo Apr 22, 2026
@github-actions
Copy link
Copy Markdown
Contributor

github-actions Bot commented Apr 22, 2026

🔍 Skill Validation Results

❌ Static Checks Failed

Skills checked: 15 | Agents checked: 0

Full validator output
Found 3 skill(s)
[code-review] 📊 code-review: 2,327 BPE tokens [chars/4: 2,361] (detailed ✓), 27 sections, 8 code blocks
[pr-review] 📊 pr-review: 3,269 BPE tokens [chars/4: 3,161] (standard ~), 22 sections, 7 code blocks
[pr-review]    ⚠  Skill is 3,269 BPE tokens (chars/4 estimate: 3,161) — approaching "comprehensive" range where gains diminish.
[try-fix] 📊 try-fix: 3,906 BPE tokens [chars/4: 4,077] (standard ~), 37 sections, 12 code blocks
[try-fix]    ⚠  Skill is 3,906 BPE tokens (chars/4 estimate: 4,077) — approaching "comprehensive" range where gains diminish.
✅ All checks passed (3 skill(s))
No agents found in the specified paths: "/home/runner/work/maui/maui/.github/agents"

⏭️ LLM Evaluation: Skipped

💡 LLM evaluation was not run for this external PR.
A repository contributor can post /evaluate-skills on this PR to trigger full evaluation.

🔍 Full results and investigation steps

@T-Gro T-Gro force-pushed the feature/expert-reviewer-extraction branch 7 times, most recently from 484b9bc to 4349b52 Compare April 22, 2026 18:11
Comment thread .github/agents/maui-expert-reviewer.md Outdated
Comment thread .github/agents/maui-expert-reviewer.md Outdated
Comment thread .github/agents/maui-expert-reviewer.md Outdated
Comment thread .github/agents/maui-expert-reviewer.md
…indings

Adds a MAUI expert review agent trained on 28,180 review comments from 5 core
maintainers, plus 8 folder-scoped instruction files and inline review posting.

## What's added
- .github/agents/maui-expert-reviewer.md (665 lines)
  30 review dimensions, 200+ CHECK items, per-dimension sub-agent workflow.
  Writes inline-findings.json for file:line PR review comments.
  Dual mode: posts directly when local, writes JSON when in CI.
- .github/instructions/ (8 new files, 261 lines total)
  CollectionView Android/iOS/Windows (platform-isolated), handler patterns,
  layout system, performance hot paths, public API, threading/async.
- .github/scripts/post-inline-review.ps1 (185 lines)
  Posts inline-findings.json as GitHub PR review with file:line comments.

## What's changed
- .github/skills/code-review/SKILL.md — delegates to expert reviewer agent
- .github/skills/code-review/references/review-rules.md — deleted (absorbed into agent)
- .github/skills/code-review/tests/eval.yaml — updated assertions
- .github/scripts/Review-PR.ps1 — wired inline findings posting in Step 3
- eng/pipelines/ci-copilot.yml — added COMMENTS_VIA_FILE env var

## Architecture
Dual posting mode controlled by COMMENTS_VIA_FILE env var:
- CI (set by ci-copilot.yml): agent writes JSON, post-inline-review.ps1 posts
- Local (@maui-expert-reviewer): agent posts directly via gh api
Two PATs in CI: COPILOT_TOKEN for LLM, GH_COMMENT_TOKEN for GitHub API.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@T-Gro T-Gro force-pushed the feature/expert-reviewer-extraction branch 2 times, most recently from 53fb77f to f1e1bf7 Compare April 29, 2026 15:02
try-fix skill now self-reviews its changes against expert reviewer
dimensions and fixes violations before reporting back. Applied in
try-fix/SKILL.md so it works regardless of who invokes try-fix.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@T-Gro T-Gro force-pushed the feature/expert-reviewer-extraction branch from f1e1bf7 to baa4792 Compare April 29, 2026 15:03
@kubaflo
Copy link
Copy Markdown
Contributor

kubaflo commented May 2, 2026

Closed in favour of #35198

@kubaflo kubaflo closed this May 2, 2026
@T-Gro T-Gro added s/agent-review-incomplete s/agent-reviewed PR was reviewed by AI agent workflow (full 4-phase review) labels May 5, 2026
PureWeen pushed a commit that referenced this pull request May 7, 2026
> [!NOTE]
> Are you waiting for the changes in this PR to be merged?
> It would be very helpful if you could [test the resulting
artifacts](https://github.com/dotnet/maui/wiki/Testing-PR-Builds) from
this PR and let us know in a comment if this change resolves your issue.
Thank you!

## Description

Replaces `review-rules.md` (flat 345-line checklist) with a dimensional
expert review agent. Single source of truth for all review rules,
organized into 30 dimensions for per-dimension sub-agent evaluation.
Adds inline file:line PR comments alongside the existing wall-of-text
summary.

Extracted from 28k review comments across 5 maintainers via
[extraction-pipeline](https://github.com/dotnet/fsharp/blob/main/.github/agents/extraction-pipeline.md).
No functional code changes.

Recreated from #35062 on a dotnet/maui branch (originally opened from a
fork).

## What changed

**Before:** `review-rules.md` had 345 lines of flat rules. `code-review`
skill loaded them all into one context. Output was a single wall-of-text
PR comment.

**After:** Rules absorbed into `maui-expert-reviewer.md` as 30
dimensions with 200+ CHECK items. Each dimension runs as an independent
sub-agent with focused context. Output is inline file:line PR comments
via `inline-findings.json`.

## CI Flow

```
Review-PR.ps1 prompt:
  1. code-review → maui-expert-reviewer agent → inline-findings.json
  2. pr-review → Pre-Flight → Try-Fix → Report (sees findings, no duplication)

Posting:
  post-inline-review.ps1    → .json → GitHub file:line comments (NEW)
  post-ai-summary-comment.ps1 → {phase}/content.md → wall-of-text (existing)

CI: COMMENTS_VIA_FILE=true → agent writes .json, script posts
Local: agent writes .json, code-review posts directly via gh api
```

## Files

| Action | File | What |
|--------|------|------|
| **Add** | `agents/maui-expert-reviewer.md` | 30 dimensions, 200+
CHECKs, routing table |
| **Add** | `instructions/collectionview-{android,ios,windows}` |
Platform-isolated CV rules |
| **Add** |
`instructions/{handler-patterns,layout-system,performance-hotpaths,public-api,threading-async}`
| Domain-specific ambient guidance |
| **Add** | `scripts/post-inline-review.ps1` | Posts .json as GitHub PR
review |
| **Del** | `skills/code-review/references/review-rules.md` | Absorbed
into agent |
| **Mod** | `skills/code-review/SKILL.md` | Delegates to agent |
| **Mod** | `scripts/Review-PR.ps1` | Prompt + inline posting wiring |
| **Mod** | `eng/pipelines/ci-copilot.yml` | `COMMENTS_VIA_FILE` env var
|

---------

Co-authored-by: kubaflo <kubaflo@users.noreply.github.com>
Co-authored-by: T-Gro <46543583+T-Gro@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Tomas Grosup <tomasgrosup@microsoft.com>
devanathan-vaithiyanathan pushed a commit to devanathan-vaithiyanathan/maui that referenced this pull request Jun 1, 2026
> [!NOTE]
> Are you waiting for the changes in this PR to be merged?
> It would be very helpful if you could [test the resulting
artifacts](https://github.com/dotnet/maui/wiki/Testing-PR-Builds) from
this PR and let us know in a comment if this change resolves your issue.
Thank you!

## Description

Replaces `review-rules.md` (flat 345-line checklist) with a dimensional
expert review agent. Single source of truth for all review rules,
organized into 30 dimensions for per-dimension sub-agent evaluation.
Adds inline file:line PR comments alongside the existing wall-of-text
summary.

Extracted from 28k review comments across 5 maintainers via
[extraction-pipeline](https://github.com/dotnet/fsharp/blob/main/.github/agents/extraction-pipeline.md).
No functional code changes.

Recreated from dotnet#35062 on a dotnet/maui branch (originally opened from a
fork).

## What changed

**Before:** `review-rules.md` had 345 lines of flat rules. `code-review`
skill loaded them all into one context. Output was a single wall-of-text
PR comment.

**After:** Rules absorbed into `maui-expert-reviewer.md` as 30
dimensions with 200+ CHECK items. Each dimension runs as an independent
sub-agent with focused context. Output is inline file:line PR comments
via `inline-findings.json`.

## CI Flow

```
Review-PR.ps1 prompt:
  1. code-review → maui-expert-reviewer agent → inline-findings.json
  2. pr-review → Pre-Flight → Try-Fix → Report (sees findings, no duplication)

Posting:
  post-inline-review.ps1    → .json → GitHub file:line comments (NEW)
  post-ai-summary-comment.ps1 → {phase}/content.md → wall-of-text (existing)

CI: COMMENTS_VIA_FILE=true → agent writes .json, script posts
Local: agent writes .json, code-review posts directly via gh api
```

## Files

| Action | File | What |
|--------|------|------|
| **Add** | `agents/maui-expert-reviewer.md` | 30 dimensions, 200+
CHECKs, routing table |
| **Add** | `instructions/collectionview-{android,ios,windows}` |
Platform-isolated CV rules |
| **Add** |
`instructions/{handler-patterns,layout-system,performance-hotpaths,public-api,threading-async}`
| Domain-specific ambient guidance |
| **Add** | `scripts/post-inline-review.ps1` | Posts .json as GitHub PR
review |
| **Del** | `skills/code-review/references/review-rules.md` | Absorbed
into agent |
| **Mod** | `skills/code-review/SKILL.md` | Delegates to agent |
| **Mod** | `scripts/Review-PR.ps1` | Prompt + inline posting wiring |
| **Mod** | `eng/pipelines/ci-copilot.yml` | `COMMENTS_VIA_FILE` env var
|

---------

Co-authored-by: kubaflo <kubaflo@users.noreply.github.com>
Co-authored-by: T-Gro <46543583+T-Gro@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Tomas Grosup <tomasgrosup@microsoft.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

community ✨ Community Contribution s/agent-reviewed PR was reviewed by AI agent workflow (full 4-phase review)

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants