Skip to content

Improve issue-triage skill: Add gh CLI checks and fix workflow#33750

Merged
PureWeen merged 2 commits intomainfrom
improve-issue-triage-skill
Jan 29, 2026
Merged

Improve issue-triage skill: Add gh CLI checks and fix workflow#33750
PureWeen merged 2 commits intomainfrom
improve-issue-triage-skill

Conversation

@PureWeen
Copy link
Member

Description

Improves the issue-triage skill to prevent common workflow mistakes and ensure proper usage of GitHub CLI.

Changes

1. Prerequisites and Validation

  • Added GitHub CLI (gh) prerequisite checks to both scripts
  • Scripts now exit gracefully with installation instructions if gh is not found
  • Updated SKILL.md with Prerequisites section and installation guide for all platforms

2. Milestone Logic Fixes

  • Fixed init-triage-session.ps1 to use Invoke-RestMethod for fetching milestones (doesn't require gh)
  • Added documentation: milestone names must come from session output, not assumptions
  • Included examples of correct vs incorrect milestone suggestions

3. Workflow Documentation

  • Added critical rule: ALWAYS use skill scripts, never ad-hoc GitHub API queries
  • Documented Step 6: Automatically reload issue batches when empty (don't prompt user)
  • Added Common Mistakes section with anti-patterns table
  • Explained what each script does and why to use them

4. Testing

  • ✅ Verified init-triage-session.ps1 works with gh installed
  • ✅ Verified query-issues.ps1 applies proper exclusion filters
  • ✅ Confirmed prerequisite check catches missing gh CLI

Why These Changes

During a triage session, the agent bypassed the skill's scripts and used ad-hoc GitHub queries, which missed important exclusion filters (s/needs-info, s/needs-repro, area-blazor, etc.). This resulted in incorrect issue counts and suggestions.

Additionally, the agent assumed milestone names like "SR2" and "SR3" without checking what milestones actually exist, leading to invalid suggestions.

These changes ensure future agents (and users) follow the proper workflow and use accurate data.

Version Bump

  • Version: 2.2 → 2.3

- Add prerequisite checks for GitHub CLI (gh) in both scripts
- Exit with clear installation instructions if gh is not found
- Update SKILL.md with Prerequisites section and installation guide
- Fix init-triage-session.ps1 to use Invoke-RestMethod for milestones
- Add critical workflow rules: always use skill scripts, auto-reload batches
- Add Common Mistakes section to document anti-patterns
- Version bump to 2.3
- Document that milestone names must come from session init, not assumptions

Fixes issue where agents would bypass skill scripts and use ad-hoc queries,
missing important exclusion filters (s/needs-info, s/needs-repro, etc.)
Copilot AI review requested due to automatic review settings January 28, 2026 19:30
Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR improves the issue-triage skill by adding prerequisite validation for GitHub CLI and enhancing workflow documentation to prevent common mistakes during issue triage sessions.

Changes:

  • Added GitHub CLI (gh) prerequisite checks with installation instructions to both init-triage-session.ps1 and query-issues.ps1
  • Updated init-triage-session.ps1 to use Invoke-RestMethod instead of gh api for fetching milestones
  • Enhanced SKILL.md with comprehensive documentation including prerequisites section, critical workflow rules, milestone naming guidance, and common mistakes table
  • Bumped skill version from 2.2 to 2.3

Reviewed changes

Copilot reviewed 3 out of 3 changed files in this pull request and generated 2 comments.

File Description
.github/skills/issue-triage/scripts/query-issues.ps1 Added prerequisite check for GitHub CLI with graceful exit and installation instructions
.github/skills/issue-triage/scripts/init-triage-session.ps1 Added prerequisite check and migrated milestone fetching from gh api to Invoke-RestMethod
.github/skills/issue-triage/SKILL.md Added prerequisites section, critical workflow rules, enhanced milestone documentation, and common mistakes table
Comments suppressed due to low confidence (1)

.github/skills/issue-triage/scripts/init-triage-session.ps1:96

  • The milestone fetching now uses Invoke-RestMethod without authentication, which will hit GitHub's rate limit for unauthenticated requests (60 requests per hour per IP). For a collaborative environment or CI scenarios, this could cause failures. Consider adding authentication headers using a GitHub token from environment variables (e.g., $env:GITHUB_TOKEN) or falling back to authenticated gh api calls when available to get the higher rate limit of 5000 requests per hour.
try {
    $msData = Invoke-RestMethod -Uri "https://api.github.com/repos/dotnet/maui/milestones?state=open&per_page=100"
    
    foreach ($ms in $msData) {
        $milestones += [PSCustomObject]@{
            Number = $ms.number
            Title = $ms.title
            DueOn = $ms.due_on
            OpenIssues = $ms.open_issues
        }
    }

Write-Host ""
exit 1
}

Copy link

Copilot AI Jan 28, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The prerequisite check validates that gh CLI is installed, but it only reports success without verifying authentication status. After the check passes, the script still uses gh api at line 153 to fetch labels, which will fail if the user hasn't authenticated with gh auth login. Consider adding an authentication check (e.g., gh auth status) or catching the authentication error at line 153 with a more specific error message directing users to run gh auth login.

Suggested change
# Verify GitHub CLI authentication
try {
# gh auth status will throw if not authenticated or if there is a problem with the auth setup
gh auth status 1>$null 2>$null
Write-Host " ✅ GitHub CLI authentication verified" -ForegroundColor Green
} catch {
Write-Host ""
Write-Host " ❌ GitHub CLI (gh) is not authenticated" -ForegroundColor Red
Write-Host ""
Write-Host " The issue-triage skill requires an authenticated GitHub CLI session" -ForegroundColor Yellow
Write-Host " to query milestones, labels, and issues via the GitHub API." -ForegroundColor Yellow
Write-Host ""
Write-Host " Please authenticate with:" -ForegroundColor Cyan
Write-Host " gh auth login" -ForegroundColor White
Write-Host ""
exit 1
}

Copilot uses AI. Check for mistakes.
Write-Host ""
exit 1
}

Copy link

Copilot AI Jan 28, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Similar to init-triage-session.ps1, this prerequisite check validates that gh CLI is installed but doesn't verify authentication status. The script uses gh commands later (lines 150, 229, 348) which will fail if the user hasn't authenticated. Consider adding an authentication check (e.g., gh auth status) or providing more specific error messages when gh commands fail due to missing authentication.

Suggested change
# Verify GitHub CLI authentication status
& gh auth status 1>$null 2>$null
if ($LASTEXITCODE -ne 0) {
Write-Host ""
Write-Host "❌ GitHub CLI (gh) is not authenticated" -ForegroundColor Red
Write-Host ""
Write-Host "The issue-triage skill requires an authenticated GitHub CLI session to query issues." -ForegroundColor Yellow
Write-Host ""
Write-Host "Please authenticate by running:" -ForegroundColor Cyan
Write-Host " gh auth login" -ForegroundColor White
Write-Host ""
exit 1
}

Copilot uses AI. Check for mistakes.
@kubaflo kubaflo added area-ai area-ai-agents Copilot CLI agents, agent skills, AI-assisted development and removed area-ai labels Jan 28, 2026
Address Copilot review feedback: verify GitHub CLI authentication
before running API queries, not just that gh is installed.
@PureWeen PureWeen merged commit 44a8f30 into main Jan 29, 2026
2 of 3 checks passed
@PureWeen PureWeen deleted the improve-issue-triage-skill branch January 29, 2026 21:46
kubaflo pushed a commit to kubaflo/maui that referenced this pull request Feb 2, 2026
…t#33750)

## Description

Improves the issue-triage skill to prevent common workflow mistakes and
ensure proper usage of GitHub CLI.

## Changes

### 1. Prerequisites and Validation
- Added GitHub CLI (gh) prerequisite checks to both scripts
- Scripts now exit gracefully with installation instructions if gh is
not found
- Updated SKILL.md with Prerequisites section and installation guide for
all platforms

### 2. Milestone Logic Fixes
- Fixed init-triage-session.ps1 to use Invoke-RestMethod for fetching
milestones (doesn't require gh)
- Added documentation: milestone names must come from session output,
not assumptions
- Included examples of correct vs incorrect milestone suggestions

### 3. Workflow Documentation
- Added critical rule: ALWAYS use skill scripts, never ad-hoc GitHub API
queries
- Documented Step 6: Automatically reload issue batches when empty
(don't prompt user)
- Added Common Mistakes section with anti-patterns table
- Explained what each script does and why to use them

### 4. Testing
- ✅ Verified init-triage-session.ps1 works with gh installed
- ✅ Verified query-issues.ps1 applies proper exclusion filters
- ✅ Confirmed prerequisite check catches missing gh CLI

## Why These Changes

During a triage session, the agent bypassed the skill's scripts and used
ad-hoc GitHub queries, which missed important exclusion filters
(s/needs-info, s/needs-repro, area-blazor, etc.). This resulted in
incorrect issue counts and suggestions.

Additionally, the agent assumed milestone names like "SR2" and "SR3"
without checking what milestones actually exist, leading to invalid
suggestions.

These changes ensure future agents (and users) follow the proper
workflow and use accurate data.

## Version Bump

- Version: 2.2 → 2.3
@kubaflo kubaflo added the copilot label Feb 6, 2026
This was referenced Feb 12, 2026
This was referenced Feb 16, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area-ai-agents Copilot CLI agents, agent skills, AI-assisted development copilot

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants

Comments