Skip to content

Update file-based app skill to mention features from 10.0.300 - #602

Merged
jjonescz merged 5 commits into
dotnet:mainfrom
jjonescz:fba-300
May 11, 2026
Merged

Update file-based app skill to mention features from 10.0.300#602
jjonescz merged 5 commits into
dotnet:mainfrom
jjonescz:fba-300

Conversation

@jjonescz

@jjonescz jjonescz commented Apr 30, 2026

Copy link
Copy Markdown
Member

Resolves #150.
Resolves #531.
Resolves #255.

Co-authored-by: Copilot <copilot@github.com>
Copilot AI review requested due to automatic review settings April 30, 2026 11:25
@jjonescz
jjonescz requested review from a team, dbreshears and timheuer as code owners April 30, 2026 11:25
@github-actions

Copy link
Copy Markdown
Contributor

Note

This PR is from a fork and modifies infrastructure files (eng/ or .github/).

Changes to infrastructure typically need to be submitted from a branch in dotnet/skills (not a fork) so that CI workflows run with the correct permissions and secrets.

Please consider recreating this PR from an upstream branch. If you don't have push access to dotnet/skills, ask a maintainer to push your branch for you.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Updates the csharp-scripts skill documentation and evaluation scenarios to use “file-based app” terminology and document new multi-file composition capabilities introduced in .NET SDK 10.0.300.

Changes:

  • Renames user-facing wording from “C# scripts” to “file-based C# apps” and clarifies when to use vs. avoid this workflow.
  • Documents new directives/features (#:include, #:exclude, and #:ref, including transitive directives guidance).
  • Adds a new eval scenario intended to exercise multi-file composition using include/exclude directives.

Reviewed changes

Copilot reviewed 3 out of 3 changed files in this pull request and generated 1 comment.

File Description
tests/dotnet/csharp-scripts/eval.yaml Renames an existing scenario and adds a new scenario for multi-file file-based apps.
plugins/dotnet/skills/csharp-scripts/SKILL.md Updates terminology and expands guidance to cover .NET 10.0.300 multi-file composition and #:ref.
.github/CODEOWNERS Changes ownership for the skill/tests from @dotnet/roslyn to @dotnet/run-file.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread tests/dotnet/csharp-scripts/eval.yaml
Co-authored-by: Copilot <copilot@github.com>
Comment thread tests/dotnet/csharp-scripts/eval.yaml
Co-authored-by: Copilot <copilot@github.com>
Copilot AI review requested due to automatic review settings May 4, 2026 10:57
@jjonescz

jjonescz commented May 4, 2026

Copy link
Copy Markdown
Member Author

/evaluate

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 3 out of 3 changed files in this pull request and generated 4 comments.


💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread tests/dotnet/csharp-scripts/eval.yaml Outdated
Comment thread tests/dotnet/csharp-scripts/eval.yaml
Comment thread tests/dotnet/csharp-scripts/eval.yaml
Comment thread plugins/dotnet/skills/csharp-scripts/SKILL.md Outdated
@jjonescz

jjonescz commented May 4, 2026

Copy link
Copy Markdown
Member Author

/evaluate

github-actions Bot added a commit that referenced this pull request May 4, 2026
@github-actions

github-actions Bot commented May 4, 2026

Copy link
Copy Markdown
Contributor

Skill Validation Results

Skill Scenario Quality Skills Loaded Overfit Verdict
csharp-scripts Avoid activating for language-agnostic calendar script 5.0/5 → 4.3/5 🔴 ✅ csharp-scripts; tools: skill, create 🟡 0.29
csharp-scripts Test a C# language feature with a file-based app 2.7/5 ⏰ → 3.7/5 🟢 ✅ csharp-scripts; tools: skill, create, edit 🟡 0.29
csharp-scripts Compose a file-based app from helper files 1.7/5 ⏰ → 2.3/5 ⏰ 🟢 ✅ csharp-scripts; tools: skill, edit / ✅ csharp-scripts; tools: stop_bash, skill, edit 🟡 0.29 [1]

[1] ⚠️ High run-to-run variance (CV=1.30) — consider re-running with --runs 5

timeout — run(s) hit the (120s) scenario timeout limit; scoring may be impacted by aborting model execution before it could produce its full output (increase via timeout in eval.yaml)

Model: claude-opus-4.6 | Judge: claude-opus-4.6

🔍 Full Results - additional metrics and failure investigation steps

▶ Sessions Visualisation -- interactive replay of all evaluation sessions

@jjonescz

jjonescz commented May 5, 2026

Copy link
Copy Markdown
Member Author

@dotnet/run-file @dotnet/skills-csharp-language-reviewers for review, thanks

Comment thread plugins/dotnet/skills/csharp-scripts/SKILL.md
Copilot AI review requested due to automatic review settings May 11, 2026 07:21

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 3 out of 3 changed files in this pull request and generated 3 comments.

Comment thread tests/dotnet/csharp-scripts/eval.yaml
Comment thread tests/dotnet/csharp-scripts/eval.yaml
Comment thread tests/dotnet/csharp-scripts/eval.yaml
@jjonescz

Copy link
Copy Markdown
Member Author

/evaluate

github-actions Bot added a commit that referenced this pull request May 11, 2026
@github-actions

Copy link
Copy Markdown
Contributor

Skill Validation Results

Skill Scenario Quality Skills Loaded Overfit Verdict
csharp-scripts Avoid activating for language-agnostic calendar script 5.0/5 → 5.0/5 ℹ️ not activated (expected) / ✅ csharp-scripts; tools: skill, create, edit 🟡 0.36
csharp-scripts Test a C# language feature with a file-based app 3.0/5 → 4.0/5 🟢 ✅ csharp-scripts; tools: skill, create, edit / ✅ csharp-scripts; tools: skill, create, edit, view 🟡 0.36
csharp-scripts Compose a file-based app from helper files 1.7/5 ⏰ → 5.0/5 🟢 ✅ csharp-scripts; tools: skill / ✅ csharp-scripts; tools: skill, edit 🟡 0.36

timeout — run(s) hit the (120s) scenario timeout limit; scoring may be impacted by aborting model execution before it could produce its full output (increase via timeout in eval.yaml)

Model: claude-opus-4.6 | Judge: claude-opus-4.6

🔍 Full Results - additional metrics and failure investigation steps

▶ Sessions Visualisation -- interactive replay of all evaluation sessions

@jjonescz
jjonescz merged commit 3aefa0f into dotnet:main May 11, 2026
41 checks passed
@jjonescz
jjonescz deleted the fba-300 branch May 11, 2026 07:53
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

4 participants