The csharp-scripts skill itself is fine — dotnet file.cs is a valid and useful tool for .NET-specific tasks. The problem is the skill's description is over-broad: words like "script", "experimentation", "prototyping", "quick", "without creating a full project" match virtually any throwaway coding task, causing the skill to activate even when the prompt has no .NET dependency. Once activated, the agent switches from Python (the natural choice for throwaway computations) to dotnet file.cs, which involves SDK resolution, compilation, and NuGet restore overhead — burning significantly more tokens for equal or worse results.
Eval results (claude-opus-4.6, 3 runs)
|
Baseline (no skill) |
With csharp-scripts |
| Language chosen |
Python |
C# |
| Tokens |
39,973 |
189,020 (+375%) |
| Quality (rubric) |
5.0/5 |
4.9/5 |
| Quality (overall) |
5.0/5 |
4.3/5 |
| Score |
— |
-15.3% (significant, CI [-37.0%, -8.0%]) |
Suggestion
Narrow the description to trigger only when the user explicitly wants C# — e.g. replace "quick experimentation, prototyping, and concept testing" with "quick C# experimentation" or "testing a C# language feature". The skill should activate when someone asks to run C# code, not when they just want "a quick script".
Context
Discovered while evaluating a dotnet-fsi-interactive skill for interactive .NET REPL usage. During A/B testing, csharp-scripts consistently over-activated on prompts that had no C#/.NET requirement, outcompeting both the baseline (Python) and the REPL skill with worse token efficiency.
eval.yaml used for reproduction
scenarios:
- name: "Quick script to prototype a calendar computation"
prompt: >
Write a quick script to experiment with finding the earliest Friday the
13th after year 0 in the Gregorian calendar. Also count how many Friday
the 13ths exist in a full 400-year cycle. Execute it and show results —
no need for a full project, just a small script for quick prototyping.
assertions:
- type: "exit_success"
- type: "output_contains"
value: "Friday"
- type: "output_matches"
pattern: "(April|4/13|0001-04-13)"
- type: "output_contains"
value: "688"
expect_tools: ["bash"]
max_turns: 10
rubric:
- "The agent correctly finds April 13, 0001 as the earliest Friday the 13th"
- "The agent correctly counts 688 Friday the 13ths in 400 years"
- "The agent verified by running code"
timeout: 120
Run with: dotnet run --project eng/skill-validator/src/SkillValidator.csproj -- --runs 3 --tests-dir tests/dotnet plugins/dotnet/skills/csharp-scripts
The
csharp-scriptsskill itself is fine —dotnet file.csis a valid and useful tool for .NET-specific tasks. The problem is the skill's description is over-broad: words like "script", "experimentation", "prototyping", "quick", "without creating a full project" match virtually any throwaway coding task, causing the skill to activate even when the prompt has no .NET dependency. Once activated, the agent switches from Python (the natural choice for throwaway computations) todotnet file.cs, which involves SDK resolution, compilation, and NuGet restore overhead — burning significantly more tokens for equal or worse results.Eval results (claude-opus-4.6, 3 runs)
Suggestion
Narrow the description to trigger only when the user explicitly wants C# — e.g. replace "quick experimentation, prototyping, and concept testing" with "quick C# experimentation" or "testing a C# language feature". The skill should activate when someone asks to run C# code, not when they just want "a quick script".
Context
Discovered while evaluating a
dotnet-fsi-interactiveskill for interactive .NET REPL usage. During A/B testing,csharp-scriptsconsistently over-activated on prompts that had no C#/.NET requirement, outcompeting both the baseline (Python) and the REPL skill with worse token efficiency.eval.yaml used for reproduction
Run with:
dotnet run --project eng/skill-validator/src/SkillValidator.csproj -- --runs 3 --tests-dir tests/dotnet plugins/dotnet/skills/csharp-scripts