Fix dotnet-test-frameworks skill activation for evaluations

[Evangelink]

Problem

The dotnet-test-frameworks skill evaluations fail on CI with SkillNotActivated for all 6 scenarios:

• Cross-framework assertion equivalence mapping
• Identify TUnit framework and its unique attributes
• Replace try-catch with framework-native exception assertions
• Skip annotations across all four frameworks
• Convert NUnit lifecycle methods to xUnit equivalents
• Identify integration tests by markers and code patterns

Root Cause

The skill had two properties preventing activation:

1. user-invocable: false in frontmatter — signals to the agent system not to present the skill for direct user prompts
2. Description said "DO NOT USE directly" — explicitly instructed the model to never activate the skill

All eval scenarios default to expect_activation: true, but the validator has no automatic linkage between user-invocable: false and expect_activation. The model sees "DO NOT USE directly", never loads the SKILL.md, and the evaluator reports failure.

Fix

• Remove user-invocable: false from both dotnet-test and dotnet-experimental copies
• Rewrite description with activation keywords matching eval prompts (assertion mapping, framework identification, skip annotations, exception assertions, lifecycle conversion, integration test markers)
• Preserve note that other analysis skills also load this as reference data

Validation

• skill-validator check passes for both plugins
• Description is 704 chars (under 1024 limit)
• All 513 skill-validator unit tests pass

[Copilot]

Pull request overview

Updates the dotnet-test-frameworks skill metadata so it can activate during CI evaluations, preventing SkillNotActivated failures for the dotnet-test evaluation scenarios.

Changes:

• Removed user-invocable: false from the dotnet-test-frameworks and exp-dotnet-test-frameworks skill frontmatter.
• Rewrote the skill descriptions to remove “DO NOT USE directly” language and add activation-oriented “Use when …” keywords aligned to eval prompts.

Show a summary per file

File	Description
plugins/dotnet-test/skills/dotnet-test-frameworks/SKILL.md	Makes the skill eligible to activate in evals by updating frontmatter and description keywords.
plugins/dotnet-experimental/skills/exp-dotnet-test-frameworks/SKILL.md	Mirrors the frontmatter/description activation changes for the experimental plugin copy.

Copilot's findings

Tip

Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

• Files reviewed: 2/2 changed files
• Comments generated: 1

[Evangelink]

/evaluate

[github-actions]

Skill Validation Results

Skill	Scenario	Quality	Skills Loaded	Overfit	Verdict
dotnet-test-frameworks	Cross-framework assertion equivalence mapping	5.0/5 → 5.0/5	✅ dotnet-test-frameworks; tools: report_intent, skill	✅ 0.09	❌ [1]
dotnet-test-frameworks	Identify TUnit framework and its unique attributes	5.0/5 → 4.7/5 🔴	✅ dotnet-test-frameworks; tools: report_intent, skill	✅ 0.09	❌ [2]
dotnet-test-frameworks	Replace try-catch with framework-native exception assertions	4.3/5 → 4.7/5 🟢	✅ dotnet-test-frameworks; tools: report_intent, skill / ⚠️ NOT ACTIVATED	✅ 0.09	✅ [3]
dotnet-test-frameworks	Skip annotations across all four frameworks	5.0/5 → 5.0/5	✅ dotnet-test-frameworks; tools: report_intent, skill	✅ 0.09	❌ [4]
dotnet-test-frameworks	Convert NUnit lifecycle methods to xUnit equivalents	5.0/5 → 5.0/5	✅ dotnet-test-frameworks; tools: report_intent, skill / ⚠️ NOT ACTIVATED	✅ 0.09	❌ [5]
dotnet-test-frameworks	Identify integration tests by markers and code patterns	5.0/5 → 5.0/5	✅ dotnet-test-frameworks; tools: report_intent, skill / ⚠️ NOT ACTIVATED	✅ 0.09	❌ [6]

[1] (Plugin) Quality unchanged but weighted score is -9.5% due to: tokens (12352 → 36325), tool calls (0 → 2), time (11.9s → 21.7s)
[2] ⚠️ High run-to-run variance (CV=0.63) — consider re-running with --runs 5
[3] ⚠️ High run-to-run variance (CV=0.99) — consider re-running with --runs 5
[4] (Plugin) Quality unchanged but weighted score is -10.8% due to: tokens (12229 → 35912), tool calls (0 → 2), time (10.6s → 19.3s), quality
[5] (Isolated) Quality unchanged but weighted score is -7.9% due to: tokens (12719 → 27575), tool calls (0 → 2)
[6] ⚠️ High run-to-run variance (CV=35.16) — consider re-running with --runs 5. (Plugin) Quality unchanged but weighted score is -1.2% due to: tokens (12744 → 17768)

Model: claude-opus-4.6 | Judge: claude-opus-4.6

🔍 Full Results - additional metrics and failure investigation steps

[Evangelink]

/evaluate

[github-actions]

Skill Validation Results

Skill	Scenario	Quality	Skills Loaded	Overfit	Verdict
dotnet-test-frameworks	Cross-framework assertion equivalence mapping	5.0/5 → 5.0/5	✅ dotnet-test-frameworks; tools: report_intent, skill / ℹ️ not activated (expected)	✅ 0.16	❌ [1]
dotnet-test-frameworks	Identify TUnit framework and its unique attributes	5.0/5 → 5.0/5	✅ dotnet-test-frameworks; tools: report_intent, skill / ℹ️ not activated (expected)	✅ 0.16	❌ [2]
dotnet-test-frameworks	Replace try-catch with framework-native exception assertions	4.0/5 → 4.3/5 🟢	✅ dotnet-test-frameworks; tools: report_intent, skill / ℹ️ not activated (expected)	✅ 0.16	✅ [3]
dotnet-test-frameworks	Skip annotations across all four frameworks	5.0/5 → 5.0/5	✅ dotnet-test-frameworks; tools: skill, report_intent / ✅ dotnet-test-frameworks; tools: report_intent, skill	✅ 0.16	❌ [4]
dotnet-test-frameworks	Convert NUnit lifecycle methods to xUnit equivalents	5.0/5 → 5.0/5	✅ dotnet-test-frameworks; tools: report_intent, skill / ℹ️ not activated (expected)	✅ 0.16	❌ [5]
dotnet-test-frameworks	Identify integration tests by markers and code patterns	5.0/5 → 5.0/5	✅ dotnet-test-frameworks; tools: report_intent, skill / ℹ️ not activated (expected)	✅ 0.16	❌ [6]

[1] (Isolated) Quality unchanged but weighted score is -4.4% due to: tokens (12368 → 26716), tool calls (0 → 2), time (16.6s → 22.9s)
[2] ⚠️ High run-to-run variance (CV=1.01) — consider re-running with --runs 5. (Isolated) Quality unchanged but weighted score is -2.8% due to: tokens (12417 → 26827), tool calls (0 → 2), time (15.3s → 19.4s)
[3] ⚠️ High run-to-run variance (CV=1.66) — consider re-running with --runs 5
[4] (Plugin) Quality unchanged but weighted score is -8.3% due to: tokens (12224 → 29708), tool calls (0 → 1), time (14.6s → 19.5s)
[5] ⚠️ High run-to-run variance (CV=0.81) — consider re-running with --runs 5. (Isolated) Quality unchanged but weighted score is -6.7% due to: tokens (12686 → 22533), tool calls (0 → 1)
[6] ⚠️ High run-to-run variance (CV=1.91) — consider re-running with --runs 5. (Isolated) Quality unchanged but weighted score is -22.6% due to: judgment, quality, tokens (12798 → 22540), tool calls (0 → 1)

Model: claude-opus-4.6 | Judge: claude-opus-4.6

🔍 Full Results - additional metrics and failure investigation steps