dotnet-test: reduce assertion-quality over-structuring on small suites

[Evangelink]

Problem

In the skill-validator dashboard, the assertion-quality scenario "Flag assertion-free tests and trivial-only assertions" regressed: baseline 4.3/5 vs SKILLED 4.0/5 (−0.3 quality decline). The fixture is tiny (8 test methods in one file), and the skill pushes a heavy multi-section structured report (Summary Dashboard + 12 categories + Category Breakdown + Gap Analysis + Recommendations). On a trivial input this over-structuring/verbosity scores below a focused freeform baseline.

Change

A single, principle-based addition to Step 6 ("Report findings") in plugins/dotnet-test/skills/assertion-quality/SKILL.md: scale report depth to suite size/complexity. For small/simple inputs, answer directly and concisely (which tests are assertion-free/trivial, the overall verdict, concrete recommendations) rather than emitting every dashboard section. The full template remains the default for substantial suites, and no rubric-relevant substance is dropped.

Safety

• Generic and principle-based — not overfit to this scenario.
• Full capability preserved for large suites.
• Does not weaken rubric behaviors: still identifies assertion-free tests, distinguishes intentional smoke tests from real verification, and gives concrete recommendations.
• Independent of PR dotnet-test: raise timeout for assertion-quality decline-write-tests scenario #862 (branched from main).

Co-authored-by: Copilot App 223556219+Copilot@users.noreply.github.com

[Evangelink]

/evaluate

[Copilot]

Pull request overview

This PR updates the dotnet-test plugin’s assertion-quality skill instructions to reduce over-structured, dashboard-heavy reporting when evaluating very small/simple test suites, aiming to improve quality scores by keeping outputs focused and signal-dense.

Changes:

• Add guidance in Step 6 (“Report findings”) to scale report depth based on suite size/complexity.
• Encourage concise, direct answers for small suites while preserving rubric-critical substance.

Show a summary per file

File	Description
plugins/dotnet-test/skills/assertion-quality/SKILL.md	Adds report-scaling guidance to avoid verbose templated dashboards on tiny suites.

Copilot's findings

Tip

Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

• Files reviewed: 1/1 changed files
• Comments generated: 1

[github-actions]

Skill Validation Results

Skill	Scenario	Quality	Skills Loaded	Overfit	Verdict
assertion-quality	Identify low assertion diversity in equality-dominated test suite	3.3/5 → 5.0/5 🟢	✅ assertion-quality; tools: skill	🟡 0.23	✅
assertion-quality	Flag assertion-free tests and trivial-only assertions	4.3/5 → 4.7/5 🟢	✅ assertion-quality; tools: skill, glob / ✅ assertion-quality; tools: skill	🟡 0.23	✅ [1]
assertion-quality	Recognize well-diversified assertion usage	3.7/5 → 5.0/5 🟢	✅ assertion-quality; tools: skill, glob / ✅ assertion-quality; tools: skill	🟡 0.23	✅
assertion-quality	Identify self-referential assertions in identity and round-trip tests	3.0/5 → 4.3/5 🟢	✅ assertion-quality; tools: skill, glob	🟡 0.23	✅
assertion-quality	Decline request to write new tests from scratch	4.0/5 → 4.3/5 🟢	ℹ️ not activated (expected)	🟡 0.23	✅ [2]
assertion-quality	Polyglot: evaluate shallow assertions in a Jest/TypeScript OrderService suite	4.3/5 → 5.0/5 🟢	✅ assertion-quality; tools: skill, glob	🟡 0.23	✅ [3]

[1] ⚠️ High run-to-run variance (CV=138%) — consider re-running with --runs 5
[2] ⚠️ High run-to-run variance (CV=69%) — consider re-running with --runs 5
[3] ⚠️ High run-to-run variance (CV=229%) — consider re-running with --runs 5

Model: claude-opus-4.6 | Judge: claude-opus-4.6

🔍 Full Results - additional metrics and failure investigation steps

▶ Sessions Visualisation -- interactive replay of all evaluation sessions
📊 Session Analytics (preview) -- aggregated metrics across evaluation sessions

[github-actions]

👋 @Evangelink — this PR has 1 unresolved review thread(s). When you're ready, please address the feedback and push an update; the triage bot will pick up the next state automatically. (Add the no-stale label to silence further pings.)