Skip to content

Simplify agent instructions and consolidate sandbox testing workflow#32950

Merged
PureWeen merged 15 commits intomainfrom
copilot/change-pr-reviewer-agent
Dec 4, 2025
Merged

Simplify agent instructions and consolidate sandbox testing workflow#32950
PureWeen merged 15 commits intomainfrom
copilot/change-pr-reviewer-agent

Conversation

Copy link
Contributor

Copilot AI commented Dec 2, 2025

Note

Are you waiting for the changes in this PR to be merged?
It would be very helpful if you could test the resulting artifacts from this PR and let us know in a comment if this change resolves your issue. Thank you!

Description of Change

Dramatically simplifies the agent instruction system by consolidating scattered instruction files into agent files directly and creating a single comprehensive sandbox testing workflow.

Major Changes:

  • Removed ~8,500 lines of nested, complex instruction files across pr-reviewer-agent/, issue-resolver-agent/, and shared/ directories
  • Consolidated pr-reviewer instructions directly into .github/agents/pr-reviewer.md (392 lines, self-contained with complete workflow, checkpoints, and examples)
  • Consolidated issue-resolver instructions directly into .github/agents/issue-resolver.md (619 lines, complete lifecycle from reproduction to PR submission)
  • Created single sandbox.instructions.md (882 lines) replacing scattered sandbox testing patterns across multiple files
  • Re-added sandbox-agent.md as explicit agent (workaround for GitHub Copilot CLI limitations with frontmatter pattern matching)
  • Moved collectionview-handler-detection.md to main instructions directory for broader access
  • Streamlined uitests.instructions.md to focus on core testing patterns (reduced from 542 to 343 lines)
  • Removed outdated documentation including PR-Test-Validation-Guide.md (355 lines)

Files Deleted:

  • 8 pr-reviewer-agent files (3,471 lines): README, quick-start, core-guidelines, testing-guidelines, error-handling, checkpoint-resume, sandbox-setup, output-format
  • 8 issue-resolver-agent files (3,288 lines): README, quick-start, core-workflow, reproduction, solution-development, error-handling, pr-submission, quick-ref
  • 4 shared instruction files (1,704 lines): checkpoints, error-handling-common, fix-patterns, platform-workflows

Result:

  • Much simpler, more maintainable agent configuration
  • All guidance consolidated in fewer, more focused files (from 24 nested files to 4 core files)
  • Net reduction of ~10,000 lines while maintaining essential functionality
  • Clearer separation of concerns between agents
  • Agents are now self-contained single files instead of multi-file packages requiring navigation

Trade-offs:

  • Loss of detailed error recovery patterns (agents may need additional guidance when encountering edge cases)
  • Checkpoint system simplified from formal structure to pause points
  • Platform-specific guidance consolidated but less detailed
  • Error handling moved from standalone files into inline sections

Issues Fixed

Fixes #

Original prompt

Create a pr that changes the pr-reviewer agent


💡 You can make Copilot smarter by setting up custom instructions, customizing its development environment and configuring Model Context Protocol (MCP) servers. Learn more Copilot coding agent tips in the docs.

Copilot AI changed the title [WIP] Change the PR reviewer agent Change pr-reviewer agent to use UI tests only, disconnecting from sandbox Dec 2, 2025
Copilot AI requested a review from PureWeen December 2, 2025 01:22
@PureWeen
Copy link
Member

PureWeen commented Dec 3, 2025

@copilot

Update description and title to summarize latest changes

Copy link
Contributor Author

Copilot AI commented Dec 3, 2025

@copilot

Update description and title to summarize latest changes

Updated PR title and description to reflect the actual changes:

New Title: Simplify agent instructions and consolidate sandbox testing workflow

Summary of changes:

  • Removed ~8,500 lines of nested, complex instruction files
  • Consolidated pr-reviewer-agent and issue-resolver-agent instructions into agent files directly
  • Created single consolidated sandbox.instructions.md (replacing scattered files)
  • Re-added sandbox-agent as explicit agent (workaround for CLI frontmatter limitations)
  • Moved collectionview-handler-detection.md to main instructions directory
  • Streamlined uitests.instructions.md to focus on core patterns
  • Deleted outdated PR-Test-Validation-Guide.md documentation

Net result: Much simpler, more maintainable agent configuration with ~10k fewer lines.

Copilot AI changed the title Change pr-reviewer agent to use UI tests only, disconnecting from sandbox Simplify agent instructions and consolidate sandbox testing workflow Dec 3, 2025
@PureWeen
Copy link
Member

PureWeen commented Dec 3, 2025

@copilot can you assess all the changes and make any recommendations for changes?

Don't change anything yet just analyze the changes and tell me your opinion.

Don't be agreeable be as honest and deeply analyze as much as you can

Make sure to understand the purpose of each change and file and then deep think about the changes

Copy link
Contributor Author

Copilot AI commented Dec 3, 2025

@copilot can you assess all the changes and make any recommendations for changes?

Don't change anything yet just analyze the changes and tell me your opinion....

Deep Analysis of Changes

I've analyzed all 38 changed files (~10k lines removed, ~2k added). Here's my honest assessment:

✅ What Works Well

1. Massive reduction in cognitive load

  • Deleted 8 nested issue-resolver-agent files (3,288 lines)
  • Deleted 8 nested pr-reviewer-agent files (3,471 lines)
  • Deleted 4 shared files (1,704 lines)
  • Result: Agents went from multi-file packages requiring navigation to single, focused files

2. Consolidation is effective

  • pr-reviewer.md (392 lines) now contains everything essential - workflow, checkpoints, examples
  • issue-resolver.md (619 lines) is self-contained with full lifecycle
  • Much easier to maintain - one file vs 10+ files per agent

3. Smart preservation

  • sandbox.instructions.md (882 lines) consolidates scattered sandbox patterns with proper applyTo frontmatter
  • Critical validation rules preserved and emphasized upfront
  • Platform-specific patterns retained

⚠️ Concerns & Questions

1. Loss of detailed error patterns

  • Old pr-reviewer-agent/error-handling.md had 9 specific mistake patterns with recovery steps (457 lines)
  • Old issue-resolver-agent/error-handling.md had comprehensive troubleshooting (657 lines)
  • Question: Are common errors now harder to recover from? The new files mention errors but lack the detailed recovery patterns.

2. Checkpoint system simplified away

  • Old shared/checkpoints.md (392 lines) had structured checkpoint/resume system
  • New approach uses pause points but lacks the formal checkpoint structure
  • Risk: Without structured checkpoints, agents might skip validation steps or not document pause points consistently

3. Sandbox-agent as workaround

  • Comment says it exists because "GitHub Copilot CLI is not reliable with applyTo frontmatter"
  • This is a band-aid for a tool limitation, not a design improvement
  • Question: Is this temporary? Will it be removed when CLI improves?

4. Platform-specific guidance removed

  • Old structure had platform directories with Android/iOS/Windows specific patterns
  • Now consolidated but less detailed for platform-specific edge cases
  • Trade-off: Simpler but potentially less comprehensive for platform quirks

5. Testing workflow clarity

  • Old had separate testing-guidelines.md, quick-ref.md, quick-start.md progression
  • New is flatter but might be harder to find specific patterns quickly
  • Question: Will agents spend more time searching through longer single files?

🔴 Potential Issues

1. uitests.instructions.md over-simplified

  • Went from 542→343 lines (36% reduction)
  • Risk: Critical patterns might be missing. Need to verify all essential guidance preserved.

2. Deleted PR-Test-Validation-Guide.md

  • 355 lines of validation procedures gone
  • Question: Was this redundant? Or is valuable testing guidance now missing?

3. Loss of fix-patterns.md

  • 699 lines of common fix patterns deleted
  • Risk: Issue-resolver agent might struggle with common scenarios that had established patterns

💡 Recommendations

1. Add error recovery index
Each agent file could benefit from a "Common Errors" section linking to recovery steps:

## Common Errors
- Build fails → [Step 2.1]
- App crashes → [Step 6.2]  
- Test timeout → [Step 6.3]

2. Consider micro-files for reusable patterns
Instead of deleting fix-patterns.md entirely, could distill to essential patterns as:

  • .github/instructions/common-fixes.md (100-150 lines)
  • Referenced from both agents

3. Document the sandbox-agent workaround
Add issue tracking or comment explaining:

  • Why this workaround exists
  • When it can be removed
  • What the proper solution is (CLI supporting applyTo)

4. Validate critical content preserved
Audit these specific areas:

  • Android Fast Deployment noReset requirement (critical for tests)
  • iOS device selection patterns
  • PublicAPI.Unshipped.txt validation
  • SafeArea measurement patterns

Bottom Line

This is a net improvement - the simplification makes agents more approachable and maintainable. However, there's risk of losing valuable troubleshooting knowledge that took time to develop.

Key concern: When things go wrong (build fails, test crashes, environment issues), will agents have enough guidance to recover? The old error-handling files were verbose but battle-tested.

Suggestion: Consider a follow-up PR adding back essential error patterns as lightweight "Common Issues" sections within each agent file.

Copilot AI and others added 9 commits December 3, 2025 12:21
…dency

Co-authored-by: PureWeen <5375137+PureWeen@users.noreply.github.com>
Co-authored-by: PureWeen <5375137+PureWeen@users.noreply.github.com>
Co-authored-by: PureWeen <5375137+PureWeen@users.noreply.github.com>
@PureWeen PureWeen force-pushed the copilot/change-pr-reviewer-agent branch from 6dae6df to fca377a Compare December 3, 2025 18:22
@PureWeen PureWeen marked this pull request as ready for review December 3, 2025 18:50
Copilot AI review requested due to automatic review settings December 3, 2025 18:50
Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR dramatically simplifies the .NET MAUI agent instruction system by consolidating scattered documentation into focused, self-contained files. The change removes approximately 10,000 lines of nested instructions across 20+ files and consolidates them into 4 core instruction files.

Key Changes:

  • Consolidated all sandbox testing patterns into single sandbox.instructions.md (882 lines)
  • Moved CollectionView handler detection to main instructions directory as collectionview-handler-detection.instructions.md
  • Streamlined uitests.instructions.md by removing redundant manual command patterns (now references BuildAndRunHostApp.ps1 script)
  • Updated all cross-references in prompt files and common testing patterns
  • Deleted 20 nested instruction files from pr-reviewer-agent/, issue-resolver-agent/, and shared/ directories
  • Added applyTo frontmatter to specify when instructions apply

Reviewed changes

Copilot reviewed 38 out of 38 changed files in this pull request and generated no comments.

Show a summary per file
File Description
sandbox.instructions.md New comprehensive 882-line guide consolidating all sandbox testing workflows, validation rules, troubleshooting, and platform patterns
collectionview-handler-detection.instructions.md Moved from pr-reviewer-agent directory to main instructions; simplified to focus on Items vs Items2 handler status
uitests.instructions.md Streamlined by removing manual command patterns, now focuses on BuildAndRunHostApp.ps1 script usage
xaml-unittests.instructions.md Added applyTo frontmatter for better scope definition
RunWithAppiumTest.template.cs Updated screenshot paths to be relative (script runs from Sandbox directory)
*.prompt.md files Updated references to point to new consolidated instruction files
common-testing-patterns.md Updated documentation references to new agent file locations
Deleted files Removed 20 nested instruction files totaling ~10,000 lines

@github-actions github-actions bot locked and limited conversation to collaborators Jan 16, 2026
@kubaflo kubaflo added area-ai-agents Copilot CLI agents, agent skills, AI-assisted development and removed area-ai labels Jan 28, 2026
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

area-ai-agents Copilot CLI agents, agent skills, AI-assisted development

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants