Fix test-anti-patterns plugin activation by disambiguating competing descriptions - #552
Conversation
…descriptions The test-anti-patterns skill was not reliably activated in plugin mode because sibling descriptions overlapped: - test-quality-auditor agent: narrowed to multi-skill audit pipelines only, explicitly excluding single-file reviews - writing-mstest-tests skill: removed 'review' and 'anti-patterns' from description, added explicit exclusion for test quality auditing - test-anti-patterns skill: expanded trigger phrases to better match review prompts (flaky tests, duplication, tests that pass without verifying)
|
/evaluate |
There was a problem hiding this comment.
Pull request overview
This PR updates skill/agent metadata in the dotnet-test plugin to reduce intermittent plugin activation flakiness by disambiguating overlapping descriptions, ensuring prompts route to test-anti-patterns instead of competing targets.
Changes:
- Narrowed
test-quality-auditoragent description to emphasize multi-skill, workspace-level audit pipelines (and explicitly exclude single-file/snippet reviews). - Refocused
writing-mstest-testsskill description on authoring/implementing MSTest tests and explicitly deferred anti-pattern auditing totest-anti-patterns. - Expanded
test-anti-patternsskill description with additional trigger phrases and anti-pattern types to better match eval prompts.
Show a summary per file
| File | Description |
|---|---|
| plugins/dotnet-test/skills/writing-mstest-tests/SKILL.md | Updates skill frontmatter description to avoid competing with anti-pattern review routing. |
| plugins/dotnet-test/skills/test-anti-patterns/SKILL.md | Broadens frontmatter description to better capture anti-pattern/flakiness/duplication prompts. |
| plugins/dotnet-test/agents/test-quality-auditor.agent.md | Tightens agent description to broad, multi-skill suite audits (not single-file reviews). |
Copilot's findings
Tip
Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
- Files reviewed: 3/3 changed files
- Comments generated: 1
Skill Validation Results
[1]
Model: claude-opus-4.6 | Judge: claude-opus-4.6 🔍 Full Results - additional metrics and failure investigation steps ▶ Sessions Visualisation -- interactive replay of all evaluation sessions |
|
/evaluate |
Skill Validation Results
[1]
Model: claude-opus-4.6 | Judge: claude-opus-4.6 🔍 Full Results - additional metrics and failure investigation steps ▶ Sessions Visualisation -- interactive replay of all evaluation sessions |
|
/evaluate |
There was a problem hiding this comment.
Copilot's findings
Tip
Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
- Files reviewed: 3/3 changed files
- Comments generated: 2
Skill Validation Results
[1]
Model: claude-opus-4.6 | Judge: claude-opus-4.6 🔍 Full Results - additional metrics and failure investigation steps ▶ Sessions Visualisation -- interactive replay of all evaluation sessions |
# Conflicts: # plugins/dotnet-test/skills/writing-mstest-tests/SKILL.md
There was a problem hiding this comment.
Copilot's findings
Tip
Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
- Files reviewed: 3/3 changed files
- Comments generated: 2
|
/evaluate |
Skill Validation Results
[1]
Model: claude-opus-4.6 | Judge: claude-opus-4.6 🔍 Full Results - additional metrics and failure investigation steps ▶ Sessions Visualisation -- interactive replay of all evaluation sessions |
Problem
The
test-anti-patternsskill was intermittently failing plugin activation checks on CI for these scenarios:In the plugin run (all skills + all agents loaded), the model sometimes chose competing targets instead of
test-anti-patterns:test-quality-auditoragent description ("audit a test suite, find weak tests, check test health") overlapped with test review promptswriting-mstest-testsskill description included "review MSTest tests" and "common anti-patterns"Fix
Three description changes to disambiguate:
test-quality-auditor.agent.md— Narrowed to "multi-skill audit pipelines for comprehensive workspace-level assessment". Added explicit "Do NOT use for reviewing a single test file, class, or inline code snippet."writing-mstest-tests/SKILL.md— Removed "review" and "anti-patterns" from description. Added explicit "Do NOT use for reviewing or auditing existing test quality — use test-anti-patterns instead."test-anti-patterns/SKILL.md— Expanded trigger phrases to better match eval prompts: "investigate flaky tests, find test repetition or duplication, tests pass but don't actually verify anything." Listed more anti-pattern types in the description.Validation
check --plugin ./plugins/dotnet-testpasses