Add exp-mock-usage-analysis skill and evaluation

[Evangelink]

New experimental skill that detects unused, redundant, or unnecessary mocks in .NET test suites. Covers Moq, NSubstitute, FakeItEasy, and manual test doubles.

• SKILL.md with 6-step workflow: gather code, classify mocks, detect anti-patterns, incorporate runtime data, generate recommendations, report findings
• eval.yaml with 10 scenarios: trivial type mocking, unused setups, redundant configs, stable utility mocks, well-placed mocks (no false positives), NSubstitute support, FakeItEasy support, setup sprawl, and 2 non-activation boundary tests
• Fixture files for each scenario with production + test code
• CODEOWNERS entry for @dotnet/dotnet-testing

[Evangelink]

/evaluate

[github-actions]

Skill Validation Results

Skill	Scenario	Quality	Skills Loaded	Overfit	Verdict
exp-mock-usage-analysis	Detect mocking of DTOs, records, and enums	5.0/5 → 5.0/5	✅ exp-mock-usage-analysis; tools: skill	✅ 0.06	❌ [1]
exp-mock-usage-analysis	Detect unused and unreachable mock setups	3.3/5 → 5.0/5 🟢	✅ exp-mock-usage-analysis; tools: skill	✅ 0.06	✅
exp-mock-usage-analysis	Detect redundant mock configurations duplicated across tests	3.0/5 → 4.0/5 🟢	✅ exp-mock-usage-analysis; tools: skill, edit	✅ 0.06	✅
exp-mock-usage-analysis	Detect mocking of stable framework types	3.0/5 → 5.0/5 🟢	✅ exp-mock-usage-analysis; tools: skill / ⚠️ NOT ACTIVATED	✅ 0.06	✅
exp-mock-usage-analysis	Recognize well-placed mocks without inventing false positives	5.0/5 → 5.0/5	✅ exp-mock-usage-analysis; tools: skill	✅ 0.06	❌ [2]
exp-mock-usage-analysis	Analyze mock usage in NSubstitute tests	3.0/5 ⏰ → 5.0/5 🟢	✅ exp-mock-usage-analysis; tools: skill	✅ 0.06	✅
exp-mock-usage-analysis	Analyze mock usage in FakeItEasy tests	4.3/5 → 4.7/5 🟢	✅ exp-mock-usage-analysis; tools: skill	✅ 0.06	❌
exp-mock-usage-analysis	Detect excessive mock configuration sprawl	3.0/5 → 4.0/5 🟢	✅ exp-mock-usage-analysis; tools: skill	✅ 0.06	✅
exp-mock-usage-analysis	Decline when asked about non-mock test anti-patterns	5.0/5 → 3.7/5 ⏰ 🔴	ℹ️ not activated (expected)	✅ 0.06	❌
exp-mock-usage-analysis	Decline mock framework migration request	5.0/5 → 5.0/5	ℹ️ not activated (expected)	✅ 0.06	❌ [3]

[1] (Plugin) Quality unchanged but weighted score is -11.5% due to: tokens (27055 → 58677), quality, time (25.6s → 61.0s), tool calls (4 → 6)
[2] (Plugin) Quality unchanged but weighted score is -3.4% due to: tokens (27136 → 50767), tool calls (3 → 4), time (44.8s → 56.7s)
[3] (Isolated) Quality unchanged but weighted score is -1.2% due to: time (6.7s → 9.3s)

⏰ timeout — run(s) hit the (60s, 120s) scenario timeout limit; scoring may be impacted by aborting model execution before it could produce its full output (increase via timeout in eval.yaml)

Model: claude-opus-4.6 | Judge: claude-opus-4.6

📖 See docs/InvestigatingResults.md for how to diagnose failures. Additional debugging guidance may be provided by your workflow.

🔍 Full Results - additional metrics and failure investigation steps

[Evangelink]

/evaluate

[github-actions]

❌ Evaluation failed. View workflow run

[Evangelink]

/evaluate

[github-actions]

Skill Validation Results

Skill	Scenario	Quality	Skills Loaded	Overfit	Verdict
exp-mock-usage-analysis	Detect unused and unreachable mock setups	3.0/5 → 2.3/5 🔴	✅ exp-mock-usage-analysis; tools: skill	✅ 0.08	❌
exp-mock-usage-analysis	Detect redundant mock configurations duplicated across tests	3.3/5 → 1.0/5 🔴	✅ exp-mock-usage-analysis; tools: skill / ⚠️ NOT ACTIVATED	✅ 0.08	❌
exp-mock-usage-analysis	Detect mocking of stable framework types	3.0/5 → 2.3/5 🔴	✅ exp-mock-usage-analysis; tools: skill / ⚠️ NOT ACTIVATED	✅ 0.08	❌
exp-mock-usage-analysis	Analyze mock usage in NSubstitute tests	2.3/5 → 1.0/5 🔴	✅ exp-mock-usage-analysis; tools: skill	✅ 0.08	❌
exp-mock-usage-analysis	Analyze mock usage in FakeItEasy tests	3.3/5 → 1.0/5 🔴	✅ exp-mock-usage-analysis; tools: skill	✅ 0.08	❌
exp-mock-usage-analysis	Detect excessive mock configuration sprawl	2.7/5 → 3.7/5 🟢	✅ exp-mock-usage-analysis; tools: skill	✅ 0.08	✅

Model: claude-opus-4.6 | Judge: claude-opus-4.6

📖 See docs/InvestigatingResults.md for how to diagnose failures. Additional debugging guidance may be provided by your workflow.

🔍 Full Results - additional metrics and failure investigation steps

[Evangelink]

/evaluate

[github-actions]

Skill Validation Results

Skill	Scenario	Quality	Skills Loaded	Overfit	Verdict
exp-mock-usage-analysis	Detect unused and unreachable mock setups	4.3/5 → 5.0/5 🟢	✅ exp-mock-usage-analysis; tools: skill	✅ 0.09	✅ [1]
exp-mock-usage-analysis	Detect redundant mock configurations duplicated across tests	3.3/5 → 3.7/5 🟢	✅ exp-mock-usage-analysis; tools: skill, edit	✅ 0.09	❌ [2]
exp-mock-usage-analysis	Detect mocking of stable framework types	3.0/5 → 5.0/5 🟢	✅ exp-mock-usage-analysis; tools: glob, skill / ✅ exp-mock-usage-analysis; tools: skill	✅ 0.09	✅ [3]
exp-mock-usage-analysis	Analyze mock usage in NSubstitute tests	3.0/5 → 5.0/5 🟢	✅ exp-mock-usage-analysis; tools: skill, glob / ✅ exp-mock-usage-analysis; tools: skill	✅ 0.09	✅ [4]
exp-mock-usage-analysis	Analyze mock usage in FakeItEasy tests	4.7/5 → 4.7/5	✅ exp-mock-usage-analysis; tools: skill, glob / ✅ exp-mock-usage-analysis; tools: skill	✅ 0.09	❌ [5]
exp-mock-usage-analysis	Detect excessive mock configuration sprawl	2.7/5 → 4.0/5 🟢	✅ exp-mock-usage-analysis; tools: skill	✅ 0.09	✅ [6]

[1] ⚠️ High run-to-run variance (CV=1.33) — consider re-running with --runs 5
[2] ⚠️ High run-to-run variance (CV=4.68) — consider re-running with --runs 5. (Isolated) Quality improved but weighted score is -2.4% due to: tokens (33793 → 69141), time (32.5s → 67.4s), tool calls (4 → 5)
[3] ⚠️ High run-to-run variance (CV=0.66) — consider re-running with --runs 5
[4] ⚠️ High run-to-run variance (CV=0.51) — consider re-running with --runs 5
[5] ⚠️ High run-to-run variance (CV=0.51) — consider re-running with --runs 5. (Isolated) Quality unchanged but weighted score is -8.7% due to: quality, tokens (35934 → 59078), tool calls (4 → 6)
[6] ⚠️ High run-to-run variance (CV=0.79) — consider re-running with --runs 5

Model: claude-opus-4.6 | Judge: claude-opus-4.6

🔍 Full Results - additional metrics and failure investigation steps

[Copilot]

Pull request overview

Adds a new experimental .NET test mock usage analysis skill to the dotnet-experimental plugin, along with an evaluation suite and scenario fixtures to measure detection of unused/unreachable/redundant mock configuration across common mocking frameworks.

Changes:

• Added exp-mock-usage-analysis skill definition (workflow + framework terminology guidance).
• Added evaluation scenarios (eval.yaml) and fixture source files for Moq/NSubstitute/FakeItEasy mock-usage patterns.
• Added design notes and CODEOWNERS entries for the new skill/tests.

Reviewed changes

Copilot reviewed 21 out of 21 changed files in this pull request and generated 3 comments.

Show a summary per file

File	Description
tests/dotnet-experimental/exp-mock-usage-analysis/well-placed-mocks/AppointmentSchedulerTests.cs	Fixture tests demonstrating “well-placed mocks” patterns.
tests/dotnet-experimental/exp-mock-usage-analysis/well-placed-mocks/AppointmentScheduler.cs	Fixture production code for “well-placed mocks” tests.
tests/dotnet-experimental/exp-mock-usage-analysis/unused-setups/InventoryServiceTests.cs	Moq fixture tests with intentionally unused/unreachable setups.
tests/dotnet-experimental/exp-mock-usage-analysis/unused-setups/InventoryService.cs	Fixture production code to support unused/unreachable setup tracing.
tests/dotnet-experimental/exp-mock-usage-analysis/stable-utility-mocks/AuthenticationServiceTests.cs	Fixture tests demonstrating mocking ILogger/IOptions patterns.
tests/dotnet-experimental/exp-mock-usage-analysis/stable-utility-mocks/AuthenticationService.cs	Fixture production code for stable-utility mocking scenarios.
tests/dotnet-experimental/exp-mock-usage-analysis/setup-sprawl/ReportGeneratorTests.cs	Fixture tests with heavy mock setup sprawl/duplication.
tests/dotnet-experimental/exp-mock-usage-analysis/setup-sprawl/ReportGenerator.cs	Fixture production code to reveal unused setups and dependency sprawl.
tests/dotnet-experimental/exp-mock-usage-analysis/redundant-mocks/BillingServiceTests.cs	Fixture tests with redundant mock boilerplate and interaction-only assertions.
tests/dotnet-experimental/exp-mock-usage-analysis/redundant-mocks/BillingService.cs	Fixture production code to support redundant/unreachable setup analysis.
tests/dotnet-experimental/exp-mock-usage-analysis/nsubstitute-mocks/ShippingServiceTests.cs	NSubstitute fixture tests including unused/unreachable setups and interaction-only checks.
tests/dotnet-experimental/exp-mock-usage-analysis/nsubstitute-mocks/ShippingService.cs	Fixture production code for NSubstitute scenarios.
tests/dotnet-experimental/exp-mock-usage-analysis/mocking-trivial-types/OrderServiceTests.cs	Fixture tests demonstrating over-mocking trivial/value types.
tests/dotnet-experimental/exp-mock-usage-analysis/mocking-trivial-types/OrderService.cs	Fixture production code for trivial-type mocking examples.
tests/dotnet-experimental/exp-mock-usage-analysis/mocking-trivial-types/Models.cs	Fixture DTO/value-object models used by trivial-type mocking examples.
tests/dotnet-experimental/exp-mock-usage-analysis/fakeiteasy-mocks/NotificationDispatcherTests.cs	FakeItEasy fixture tests with intentionally unnecessary A.CallTo setups.
tests/dotnet-experimental/exp-mock-usage-analysis/fakeiteasy-mocks/NotificationDispatcher.cs	Fixture production code for FakeItEasy scenarios.
tests/dotnet-experimental/exp-mock-usage-analysis/eval.yaml	Evaluation scenarios driving the skill-validator runs for this skill.
plugins/dotnet-experimental/skills/exp-mock-usage-analysis/SKILL.md	Skill instructions emphasizing code-path tracing and framework-accurate terminology.
docs/dotnet-experimental/exp-mock-usage-analysis-design-notes.md	Design notes documenting iteration history and eval outcomes.
.github/CODEOWNERS	Adds ownership for the new skill + tests under @dotnet/dotnet-testing.

Comments suppressed due to low confidence (1)

tests/dotnet-experimental/exp-mock-usage-analysis/eval.yaml:15

• The scenario header comments are inconsistently numbered (e.g., Scenario 1, then Scenario 3, then Scenario 4 twice, then Scenario 7/8). This is confusing when maintaining/triaging eval results; renumber the headers to match the actual ordering or remove the numeric labels and rely on the name: fields.

  # ============================================================
  # Scenario 1: Unused mock setups
  # ============================================================
  - name: "Detect unused and unreachable mock setups"
    prompt: |
      I've been told our tests have a lot of dead mock setup code — setups for

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[Evangelink]

/evaluate

[github-actions]

Skill Validation Results

Skill	Scenario	Quality	Skills Loaded	Overfit	Verdict
exp-mock-usage-analysis	Detect unused and unreachable mock setups	4.0/5 → 4.7/5 🟢	✅ exp-mock-usage-analysis; tools: skill	✅ 0.12	✅ [1]
exp-mock-usage-analysis	Detect redundant mock configurations duplicated across tests	3.0/5 → 3.7/5 🟢	✅ exp-mock-usage-analysis; tools: skill, edit	✅ 0.12	✅ [2]
exp-mock-usage-analysis	Detect mocking of stable framework types	3.0/5 → 5.0/5 🟢	✅ exp-mock-usage-analysis; tools: skill	✅ 0.12	✅
exp-mock-usage-analysis	Analyze mock usage in NSubstitute tests	4.0/5 → 4.7/5 🟢	✅ exp-mock-usage-analysis; tools: skill	✅ 0.12	✅ [3]
exp-mock-usage-analysis	Analyze mock usage in FakeItEasy tests	5.0/5 → 5.0/5	✅ exp-mock-usage-analysis; tools: skill	✅ 0.12	❌ [4]
exp-mock-usage-analysis	Detect excessive mock configuration sprawl	3.0/5 → 4.0/5 🟢	✅ exp-mock-usage-analysis; tools: skill	✅ 0.12	✅

[1] ⚠️ High run-to-run variance (CV=2.83) — consider re-running with --runs 5
[2] ⚠️ High run-to-run variance (CV=1.01) — consider re-running with --runs 5
[3] ⚠️ High run-to-run variance (CV=4.38) — consider re-running with --runs 5
[4] ⚠️ High run-to-run variance (CV=0.66) — consider re-running with --runs 5. (Plugin) Quality unchanged but weighted score is -2.6% due to: tokens (27326 → 48006), tool calls (3 → 4), time (41.7s → 55.4s)

Model: claude-opus-4.6 | Judge: claude-opus-4.6

🔍 Full Results - additional metrics and failure investigation steps