Conversation
909830b to
5f8e197
Compare
d61d2af to
5e92ccf
Compare
0e0aea1 to
fac75f8
Compare
.github/agents/pr-reviewer-gate.md
Outdated
There was a problem hiding this comment.
We should also handle the case where a PR has no tests. Maybe, in that situation, the agent should attempt to design a reasonable test that would fail without the fix.
If the agent succeeds, it should post a PR comment proposing the test it came up with. If it cannot come up with a meaningful test, it should proceed to the next phase, but leave a comment explaining that no test was added because it was difficult to design one.
We can also think about a new tag like: needs-test
Ideally, the agent should also do a quick check whether any of existing test should already be covering this scenario. This could help us identify gaps or faulty tests.
| This skill guides you through analyzing an issue and developing your own fix approach BEFORE looking at how the PR solved it. This ensures you form an independent opinion and can provide meaningful review feedback. | ||
|
|
||
| ## When to Use | ||
|
|
There was a problem hiding this comment.
Should we also add something like:
when doing pr-reviewer-analyze session
|
I think that for each sub-agent, we should explicitly document which skills it is expected to use. At the same time, each skill definition should clearly state which agents are most likely to use it, even if that information is partially redundant - just to be sure. |
.github/agents/pr-reviewer.md
Outdated
| @@ -1,392 +1,306 @@ | |||
| --- | |||
| name: pr-reviewer | |||
There was a problem hiding this comment.
The main pr-reviewer repeats a lot of the same steps that already are in the sub-agents.
| @@ -0,0 +1,84 @@ | |||
| --- | |||
There was a problem hiding this comment.
Here we could also check if there's a possibility of writing better test. For example unit test vs UiTest, UiTest without snapshots or UiTest without Task.Delay
| @@ -0,0 +1,82 @@ | |||
| --- | |||
There was a problem hiding this comment.
I think it would make sense to rename this to pr-reviewer-independent-analyze.
The current name is a bit ambiguous, while the new one makes it explicit that this step is meant to perform an independent analysis before looking at the instruction.
.github/agents/pr-reviewer-report.md
Outdated
| @@ -0,0 +1,94 @@ | |||
| --- | |||
There was a problem hiding this comment.
What about adding a tag pr-reviewer-agent-approved?
|
I've tested it with this pr: dotnet#33359 (with the pr-reviewer-init included) Initially, before I could use the staged approach, I had to explicitly tell it not to switch branches and to cherry-pick commits into the current one, so that the required agents and skills were available. Thanks to it, it completed a full review cycle, but it only used the pr-reviewer agent.
In the second round, I adjusted the prompt to: “please review XXX, don’t switch branches, cherry-pick commits instead, and make sure to switch between agents as defined in the instructions.” WIth such prompt the agents switching feature worked properly :) |
- Document CLI vs VS Code feature support - Define constraints (30K chars, 64 char name, 1024 char description) - List anti-patterns to avoid - Include best practices for discovery and conciseness - Add minimal structure template and checklist
- pr-reviewer-gate: Verify tests catch the bug - pr-reviewer-analyze: Independent root cause analysis - pr-reviewer-compare: Compare PR fix vs alternatives - pr-reviewer-regression: Check for regression risks - pr-reviewer-report: Generate final review report - pr-reviewer-loop.ps1: Orchestrator script that invokes agents sequentially Each agent reads/writes to a shared state file (pr-XXXXX-review.md) to track progress across phases.
- Add Tools array to each phase definition - Use --allow-tool flags instead of --allow-all-tools - Add detailed logging for agent invocation - Show state file section after each phase - Display command being executed with tools
- New pr-reviewer-init agent fetches PR data, comments, and previous runs - Extracts edge cases and regression concerns from discussion - Looks for previous pr-reviewer runs in PR comments to resume context - Replaces inline Initialize-StateFile function in loop script - Init is now the first phase (default), creates state file with rich context
7d9496c to
f65b55f
Compare
- Update pr-reviewer.md with collapsible <details> sections for readable state files - Keep verify-tests-fail-without-fix skill (used by Phase 0) - Add agents.instructions.md and skills.instructions.md for development guidelines - Remove unused phase-specific agents and skills that are now inline
- Add Step 4d to detect and import prior agent reviews from PR comments - Prior reviews with phase tables and Final Recommendation should be used as state file content - Add to Common Mistakes: ignoring prior agent reviews
- Delete issue-resolver.md (consolidated into pr agent) - Update pr.md to focus on PR review workflow - Add guidance to use /delegate for issues without PRs - Keep all trigger phrases for PR-related requests
- Remove restriction requiring PR to exist - Add conditional steps for issue-only vs PR starting points - Clarify state file naming (use issue # if no PR yet) - Add 'look at implementation code' to Pre-Flight restrictions - Update Tests phase to check for existing tests in both modes
- Split pr.md into phases 1-3 (pre-gate) and pr/post-gate.md (phases 4-7) - Add scope constraints at top of each phase to prevent premature analysis - Add explicit STOP marker after Gate phase - Remove phases 4-7 from state file template (added after Gate passes) - Move post-gate.md to subfolder so it doesn't appear as separate agent - Fix dangling references to removed 'Phase Gate Check' section
Co-authored-by: PureWeen <5375137+PureWeen@users.noreply.github.com>
Co-authored-by: PureWeen <5375137+PureWeen@users.noreply.github.com>
…33398) Introduces a consolidated 5-phase PR agent workflow that provides structured, end-to-end assistance for investigating issues, developing fixes, and creating pull requests. ### PR Agent Workflow The new `pr` agent replaces the separate `issue-resolver` and `pr-reviewer` agents with a unified 5-phase workflow: 1. **Pre-Flight** - Context gathering and state file creation 2. **Tests** - Create or verify reproduction tests using the `write-tests` skill 3. **Gate** - Verify tests catch the issue (mandatory checkpoint before proceeding) 4. **Fix** - Explore and test fix approaches using the `try-fix` skill 5. **Report** - Present findings and create PR (with explicit user confirmation) ### Agent Changes - **Removed**: `issue-resolver.md`, `pr-reviewer.md` - **Added**: `pr.md` - Main agent with phases 1-3 (Pre-Flight, Tests, Gate) - **Added**: `pr/post-gate.md` - Phases 4-5 (Fix, Report) ### New Skills - `try-fix/` - Proposes ONE independent fix approach per invocation, applies it, runs tests, records result with failure analysis, then reverts. Learns from prior attempts. Max 5 attempts per session. - `verify-tests-fail-without-fix/` - Auto-detects fix files from git diff, verifies tests FAIL without fix and PASS with fix - `write-tests/` - Creates UI test pages and NUnit tests following MAUI conventions, with verification that tests reproduce the bug ### New Instructions - `agents.instructions.md` - Copilot CLI agent authoring guidelines (constraints, best practices, anti-patterns) - `skills.instructions.md` - Skill development standards (structure, naming, script organization) ### Script Updates - Added `-Rebuild` parameter to `BuildAndRunHostApp.ps1` and `Build-AndDeploy.ps1` for clean builds during test verification ### Documentation Updates - Updated references from `issue-resolver`/`pr-reviewer` to `pr` agent across README-AI.md, copilot-instructions.md, sandbox.instructions.md, uitest-coding-agent.md - Expanded `collectionview-handler-detection.instructions.md` with platform-specific guidance (Items2/ is iOS/MacCatalyst only, Items/ is the only Android/Windows implementation) <!-- START COPILOT CODING AGENT SUFFIX --> <!-- START COPILOT ORIGINAL PROMPT --> <details> <summary>Original prompt</summary> > Create a PR with all the changes from this PR on the PureWeen fork PureWeen#9 </details> <!-- START COPILOT CODING AGENT TIPS --> --- 💬 We'd love your input! Share your thoughts on Copilot coding agent in our [2 minute survey](https://gh.io/copilot-coding-agent-survey). --------- Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com> Co-authored-by: PureWeen <5375137+PureWeen@users.noreply.github.com> Co-authored-by: Shane Neuville <shneuvil@microsoft.com>
…otnet#33398) Introduces a consolidated 5-phase PR agent workflow that provides structured, end-to-end assistance for investigating issues, developing fixes, and creating pull requests. ### PR Agent Workflow The new `pr` agent replaces the separate `issue-resolver` and `pr-reviewer` agents with a unified 5-phase workflow: 1. **Pre-Flight** - Context gathering and state file creation 2. **Tests** - Create or verify reproduction tests using the `write-tests` skill 3. **Gate** - Verify tests catch the issue (mandatory checkpoint before proceeding) 4. **Fix** - Explore and test fix approaches using the `try-fix` skill 5. **Report** - Present findings and create PR (with explicit user confirmation) ### Agent Changes - **Removed**: `issue-resolver.md`, `pr-reviewer.md` - **Added**: `pr.md` - Main agent with phases 1-3 (Pre-Flight, Tests, Gate) - **Added**: `pr/post-gate.md` - Phases 4-5 (Fix, Report) ### New Skills - `try-fix/` - Proposes ONE independent fix approach per invocation, applies it, runs tests, records result with failure analysis, then reverts. Learns from prior attempts. Max 5 attempts per session. - `verify-tests-fail-without-fix/` - Auto-detects fix files from git diff, verifies tests FAIL without fix and PASS with fix - `write-tests/` - Creates UI test pages and NUnit tests following MAUI conventions, with verification that tests reproduce the bug ### New Instructions - `agents.instructions.md` - Copilot CLI agent authoring guidelines (constraints, best practices, anti-patterns) - `skills.instructions.md` - Skill development standards (structure, naming, script organization) ### Script Updates - Added `-Rebuild` parameter to `BuildAndRunHostApp.ps1` and `Build-AndDeploy.ps1` for clean builds during test verification ### Documentation Updates - Updated references from `issue-resolver`/`pr-reviewer` to `pr` agent across README-AI.md, copilot-instructions.md, sandbox.instructions.md, uitest-coding-agent.md - Expanded `collectionview-handler-detection.instructions.md` with platform-specific guidance (Items2/ is iOS/MacCatalyst only, Items/ is the only Android/Windows implementation) <!-- START COPILOT CODING AGENT SUFFIX --> <!-- START COPILOT ORIGINAL PROMPT --> <details> <summary>Original prompt</summary> > Create a PR with all the changes from this PR on the PureWeen fork PureWeen#9 </details> <!-- START COPILOT CODING AGENT TIPS --> --- 💬 We'd love your input! Share your thoughts on Copilot coding agent in our [2 minute survey](https://gh.io/copilot-coding-agent-survey). --------- Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com> Co-authored-by: PureWeen <5375137+PureWeen@users.noreply.github.com> Co-authored-by: Shane Neuville <shneuvil@microsoft.com>
Adds the issue-triage skill from PureWeen#9 to help triage open GitHub issues that need milestones, labels, or investigation. ## Changes - **`.github/skills/issue-triage/SKILL.md`** - Skill definition with workflow, trigger phrases, and milestone suggestion logic - **`scripts/init-triage-session.ps1`** - Initializes session with current milestones and labels from dotnet/maui - **`scripts/query-issues.ps1`** - Queries open issues with platform/area/age filters, fetches linked PRs, suggests milestones dynamically - **`scripts/record-triage.ps1`** - Tracks triaged issues in session file - **`.github/copilot-instructions.md`** - Added "Reusable Skills" section documenting the new skill ## Dynamic Milestone Detection Milestones are queried dynamically from dotnet/maui rather than being hardcoded (since SR milestones change monthly): - **SR milestones**: Identified by pattern `\.NET.*SR\d+`, sorted by due date to determine current/next SR - **Servicing milestone**: Identified by pattern `\.NET.*Servicing` - Suggestions use the dynamically discovered milestones based on issue characteristics (regressions → current SR, PRs → Servicing, etc.) ## Usage ```bash # Initialize triage session pwsh .github/skills/issue-triage/scripts/init-triage-session.ps1 # Query Android issues needing triage pwsh .github/skills/issue-triage/scripts/query-issues.ps1 -Platform android -Limit 20 # Record a triage decision pwsh .github/skills/issue-triage/scripts/record-triage.ps1 -IssueNumber 33272 -Milestone "Backlog" ``` Trigger phrases: "find issues to triage", "triage Android issues", "what issues need attention" <!-- START COPILOT CODING AGENT SUFFIX --> <!-- START COPILOT ORIGINAL PROMPT --> <details> <summary>Original prompt</summary> > Create a PR for an issue-triage skill that uses the skill from here PureWeen#9. Just grab the issue-triage skill from this PR please </details> <!-- START COPILOT CODING AGENT TIPS --> --- ✨ Let Copilot coding agent [set things up for you](https://github.com/dotnet/maui/issues/new?title=✨+Set+up+Copilot+instructions&body=Configure%20instructions%20for%20this%20repository%20as%20documented%20in%20%5BBest%20practices%20for%20Copilot%20coding%20agent%20in%20your%20repository%5D%28https://gh.io/copilot-coding-agent-tips%29%2E%0A%0A%3COnboard%20this%20repo%3E&assignees=copilot) — coding agent works faster and does higher quality work when set up for your repo. --------- Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com> Co-authored-by: PureWeen <5375137+PureWeen@users.noreply.github.com> Co-authored-by: Shane Neuville <shneuvil@microsoft.com>
Adds the issue-triage skill from PureWeen#9 to help triage open GitHub issues that need milestones, labels, or investigation. ## Changes - **`.github/skills/issue-triage/SKILL.md`** - Skill definition with workflow, trigger phrases, and milestone suggestion logic - **`scripts/init-triage-session.ps1`** - Initializes session with current milestones and labels from dotnet/maui - **`scripts/query-issues.ps1`** - Queries open issues with platform/area/age filters, fetches linked PRs, suggests milestones dynamically - **`scripts/record-triage.ps1`** - Tracks triaged issues in session file - **`.github/copilot-instructions.md`** - Added "Reusable Skills" section documenting the new skill ## Dynamic Milestone Detection Milestones are queried dynamically from dotnet/maui rather than being hardcoded (since SR milestones change monthly): - **SR milestones**: Identified by pattern `\.NET.*SR\d+`, sorted by due date to determine current/next SR - **Servicing milestone**: Identified by pattern `\.NET.*Servicing` - Suggestions use the dynamically discovered milestones based on issue characteristics (regressions → current SR, PRs → Servicing, etc.) ## Usage ```bash # Initialize triage session pwsh .github/skills/issue-triage/scripts/init-triage-session.ps1 # Query Android issues needing triage pwsh .github/skills/issue-triage/scripts/query-issues.ps1 -Platform android -Limit 20 # Record a triage decision pwsh .github/skills/issue-triage/scripts/record-triage.ps1 -IssueNumber 33272 -Milestone "Backlog" ``` Trigger phrases: "find issues to triage", "triage Android issues", "what issues need attention" <!-- START COPILOT CODING AGENT SUFFIX --> <!-- START COPILOT ORIGINAL PROMPT --> <details> <summary>Original prompt</summary> > Create a PR for an issue-triage skill that uses the skill from here PureWeen#9. Just grab the issue-triage skill from this PR please </details> <!-- START COPILOT CODING AGENT TIPS --> --- ✨ Let Copilot coding agent [set things up for you](https://github.com/dotnet/maui/issues/new?title=✨+Set+up+Copilot+instructions&body=Configure%20instructions%20for%20this%20repository%20as%20documented%20in%20%5BBest%20practices%20for%20Copilot%20coding%20agent%20in%20your%20repository%5D%28https://gh.io/copilot-coding-agent-tips%29%2E%0A%0A%3COnboard%20this%20repo%3E&assignees=copilot) — coding agent works faster and does higher quality work when set up for your repo. --------- Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com> Co-authored-by: PureWeen <5375137+PureWeen@users.noreply.github.com> Co-authored-by: Shane Neuville <shneuvil@microsoft.com>
Adds the `find-reviewable-pr` skill from <a href="https://github.com/PureWeen/maui/pull/9">PureWeen/maui PR #9</a> to help find open PRs in the dotnet/maui and dotnet/docs-maui repositories that are good candidates for review. ## Changes Made - **`.github/skills/find-reviewable-pr/SKILL.md`** - Skill documentation including usage guide, priority categories, script parameters, and workflow for reviewing PRs - **`.github/skills/find-reviewable-pr/scripts/query-reviewable-prs.ps1`** - PowerShell script that queries GitHub for open PRs and prioritizes them dynamically by milestone (lower SR numbers first), P/0 priority, partner status, community contributions, and recency ## Features The skill provides: - Priority-based PR categorization (P/0, Milestoned, Partner, Community, Recent) - **Dynamic milestone sorting**: Milestones are sorted by SR number (e.g., SR5 before SR6) so prioritization remains correct as milestones advance monthly - Platform filtering (Android, iOS, Windows, MacCatalyst) - Multiple output formats (review, table, JSON) - Complexity assessment for PRs - Review status tracking - **docs-maui support**: Returns 5 priority PRs and 5 recent PRs from dotnet/docs-maui by default - New `-DocsLimit` parameter to control docs-maui PR count - New `-Category docs-maui` filter option to show only docs-maui PRs - **Recent PRs**: Shows 5 recent PRs from both maui and docs-maui by default - New `-RecentLimit` parameter to control recent PRs from maui (default: 5) <!-- START COPILOT ORIGINAL PROMPT --> <details> <summary>Original prompt</summary> > Create a PR that just grabs the find-reviewable-pr's skill from this PR PureWeen#9 </details> <!-- START COPILOT CODING AGENT TIPS --> --- 💡 You can make Copilot smarter by setting up custom instructions, customizing its development environment and configuring Model Context Protocol (MCP) servers. Learn more [Copilot coding agent tips](https://gh.io/copilot-coding-agent-tips) in the docs. --------- Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com> Co-authored-by: PureWeen <5375137+PureWeen@users.noreply.github.com> Co-authored-by: Shane Neuville <shneuvil@microsoft.com> Co-authored-by: Jakub Florkowski <kubaflo123@gmail.com>
Adds the `find-reviewable-pr` skill from <a href="https://github.com/PureWeen/maui/pull/9">PureWeen/maui PR #9</a> to help find open PRs in the dotnet/maui and dotnet/docs-maui repositories that are good candidates for review. ## Changes Made - **`.github/skills/find-reviewable-pr/SKILL.md`** - Skill documentation including usage guide, priority categories, script parameters, and workflow for reviewing PRs - **`.github/skills/find-reviewable-pr/scripts/query-reviewable-prs.ps1`** - PowerShell script that queries GitHub for open PRs and prioritizes them dynamically by milestone (lower SR numbers first), P/0 priority, partner status, community contributions, and recency ## Features The skill provides: - Priority-based PR categorization (P/0, Milestoned, Partner, Community, Recent) - **Dynamic milestone sorting**: Milestones are sorted by SR number (e.g., SR5 before SR6) so prioritization remains correct as milestones advance monthly - Platform filtering (Android, iOS, Windows, MacCatalyst) - Multiple output formats (review, table, JSON) - Complexity assessment for PRs - Review status tracking - **docs-maui support**: Returns 5 priority PRs and 5 recent PRs from dotnet/docs-maui by default - New `-DocsLimit` parameter to control docs-maui PR count - New `-Category docs-maui` filter option to show only docs-maui PRs - **Recent PRs**: Shows 5 recent PRs from both maui and docs-maui by default - New `-RecentLimit` parameter to control recent PRs from maui (default: 5) <!-- START COPILOT ORIGINAL PROMPT --> <details> <summary>Original prompt</summary> > Create a PR that just grabs the find-reviewable-pr's skill from this PR PureWeen#9 </details> <!-- START COPILOT CODING AGENT TIPS --> --- 💡 You can make Copilot smarter by setting up custom instructions, customizing its development environment and configuring Model Context Protocol (MCP) servers. Learn more [Copilot coding agent tips](https://gh.io/copilot-coding-agent-tips) in the docs. --------- Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com> Co-authored-by: PureWeen <5375137+PureWeen@users.noreply.github.com> Co-authored-by: Shane Neuville <shneuvil@microsoft.com> Co-authored-by: Jakub Florkowski <kubaflo123@gmail.com>

Example
dotnet#32456 (comment)
Summary
This PR adds a PR Reviewer “Detailed Agent” that automatically reviews pull requests and produces a structured, high-signal review focused on correctness, safety, maintainability, and developer experience.
Rather than leaving ad‑hoc comments, the agent builds a holistic understanding of the change by reading the PR context (title/description), the full diff, and relevant surrounding code, then returns a consistent review output that is easy to act on.
How the PR Reviewer Detailed Agent works
1) Inputs it uses
The agent gathers and normalizes the following signals from the pull request:
2) Review pipeline (high level)
The agent follows a predictable set of steps:
Understand intent
Extracts what the PR is trying to accomplish (feature, fix, refactor) and what “done” should look like.
Diff analysis
Scans each changed file and hunk to identify:
Risk and impact assessment
Flags areas that are likely to cause regressions (core flows, shared utilities, public APIs) and calls out backwards-compatibility concerns.
Actionable recommendations
Produces concrete suggestions, including what to change and why, preferring the smallest safe fix over large rewrites.
3) Output format
The agent produces a detailed, structured review (vs. a generic “LGTM”), typically including:
4) What it optimizes for
Why this is useful