run-tests: fix evals (query-filter regex, sibling skills, command observability)

plugins/dotnet-test/skills/code-testing-extensions/SKILL.md

@@ -6,6 +6,7 @@ description: >-
   (e.g., dotnet.md for .NET, cpp.md for C++). Do not use directly — invoked
   by code-testing agents and skills that need language-specific references.
 user-invocable: false
+disable-model-invocation: true
 license: MIT
 ---
 

plugins/dotnet-test/skills/dotnet-test-frameworks/SKILL.md

@@ -2,6 +2,7 @@
 name: dotnet-test-frameworks
 description: "Reference data for .NET test framework detection patterns, assertion APIs, skip annotations, setup/teardown methods, and common test smell indicators across MSTest, xUnit, NUnit, and TUnit. Loaded by test analysis skills (test-anti-patterns) as framework-specific lookup tables."
 user-invocable: false
+disable-model-invocation: true
 license: MIT
 ---
 

plugins/dotnet-test/skills/filter-syntax/SKILL.md

@@ -2,6 +2,7 @@
 name: filter-syntax
 description: "Reference data for test filter syntax across all platform and framework combinations: VSTest --filter expressions, MTP filters for MSTest/NUnit/xUnit v3/TUnit, and VSTest-to-MTP filter translation. DO NOT USE directly — loaded by run-tests, mtp-hot-reload, and migrate-vstest-to-mtp when they need filter syntax."
 user-invocable: false
+disable-model-invocation: true
 license: MIT
 ---
 

plugins/dotnet-test/skills/find-untested-sources-polyglot/SKILL.md

@@ -8,6 +8,7 @@ description: >
   without tests, polyglot test-pairing map.
   DO NOT USE FOR: coverage, CRAP risk. For .NET-only repos prefer
   `find-untested-sources`.
+disable-model-invocation: true
 license: MIT
 ---
 

plugins/dotnet-test/skills/find-untested-sources/SKILL.md

@@ -8,6 +8,7 @@ description: >
   without tests, build a test-pairing map.
   DO NOT USE FOR: coverage (use `coverage-analysis`), CRAP risk ranking,
   assertion gaps.
+disable-model-invocation: true
 license: MIT
 ---
 

plugins/dotnet-test/skills/platform-detection/SKILL.md

@@ -2,6 +2,7 @@
 name: platform-detection
 description: "Reference data for detecting the test platform (VSTest vs Microsoft.Testing.Platform) and test framework (MSTest, xUnit, NUnit, TUnit) from project files. DO NOT USE directly — loaded by run-tests, mtp-hot-reload, and migrate-vstest-to-mtp when they need detection logic."
 user-invocable: false
+disable-model-invocation: true
 license: MIT
 ---
 

plugins/dotnet-test/skills/run-tests/SKILL.md

@@ -1,21 +1,19 @@
 ---
 name: run-tests
 description: >
-  For `dotnet test`: figures out which test platform (VSTest vs
-  Microsoft.Testing.Platform) a project uses from `Directory.Build.props`,
-  `global.json`, and `.csproj`, then picks the matching command syntax. USE
-  FOR: running, filtering, or troubleshooting `dotnet test`; identifying the
-  test runner/platform from project files; `--` separator rules on .NET SDK
-  8/9 vs 10+; choosing the right filter syntax for MSTest / xUnit / NUnit /
-  TUnit (--filter, --filter-class, --filter-trait, --filter-query,
-  --treenode-filter); TRX/reporting (--report-trx vs --logger trx);
-  blame/hang/crash diagnostics (--blame-hang-timeout, --blame-crash); running
-  tests against a single target framework when a project targets multiple
-  TFMs (e.g., `<TargetFrameworks>net8.0;net9.0</TargetFrameworks>`,
-  `--framework <TFM>`); and avoiding MTP/VSTest argument mixups (--logger
-  trx on MTP, --report-trx on VSTest, --blame on MTP).
-  DO NOT USE FOR: writing/generating test code, CI/CD config, or debugging
-  failing test logic.
+  Run, filter, or troubleshoot .NET tests with `dotnet test`. USE FOR:
+  running all tests in a project or solution; running only a subset (a
+  specific test class, category, or trait) via filters; running a single
+  target framework in a multi-TFM project (`--framework`); producing TRX
+  reports; collecting crash or hang dumps; diagnosing why `dotnet test`
+  fails or uses the wrong argument syntax. Detects the test platform
+  (VSTest vs Microsoft.Testing.Platform) and framework
+  (MSTest/xUnit/NUnit/TUnit), then picks the matching command: the `--`
+  separator on .NET SDK 8/9 vs 10+, the right filter flag (--filter,
+  --filter-class, --filter-trait, --filter-query, --treenode-filter), and
+  TRX/blame flags. DO NOT USE FOR: writing test code (use
+  code-testing-agent), iterating on failing tests without rebuilding (use
+  mtp-hot-reload), CI/CD config, or debugging test logic.
 license: MIT
 ---
 
@@ -216,7 +214,7 @@ See the `filter-syntax` skill for the complete filter syntax for each platform a
 
 - **VSTest** (MSTest, xUnit v2, NUnit): `dotnet test --filter <EXPRESSION>` with `=`, `!=`, `~`, `!~` operators
 - **MTP -- MSTest and NUnit**: Same `--filter` syntax as VSTest; pass after `--` on SDK 8/9, directly on SDK 10+
-- **MTP -- xUnit v3**: Uses `--filter-class`, `--filter-method`, `--filter-trait` (not VSTest expression syntax)
+- **MTP -- xUnit v3**: Uses `--filter-class`, `--filter-method`, `--filter-trait` (not VSTest expression syntax). For a **single combined expression** (e.g., a class-name pattern AND a trait), use `--filter-query` with the xUnit v3 query filter language: path segments `/<assembly>/<namespace>/<class>/<method>` with `*` wildcards and a `[Trait=Value]` qualifier — for example `dotnet test -- --filter-query "/*/*/*IntegrationTests*/*[Category=Smoke]"`. See the `filter-syntax` skill for the full query language.
 - **MTP -- TUnit**: Uses `--treenode-filter` with path-based syntax
 
 #### When the user names a test category, trait, or group

plugins/dotnet-test/skills/test-analysis-extensions/SKILL.md

@@ -12,6 +12,7 @@ description: >-
   annotations, sleep patterns, mystery guest indicators, integration
   markers, setup/teardown, tag-support capability).
 user-invocable: false
+disable-model-invocation: true
 license: MIT
 ---
 

tests/dotnet-test/run-tests/eval.yaml

@@ -4,7 +4,7 @@ scenarios:
   # ============================================================================
 
   - name: "Run tests in a VSTest MSTest project"
-    prompt: "I need to run all the tests in this project. Can you help me?"
+    prompt: "I need to run all the tests in this project. Can you help me? Show me the exact command to use."
     setup:
       files:
         - path: "TestProject.csproj"
@@ -23,7 +23,7 @@ scenarios:
     timeout: 180
 
   - name: "Run tests with trx reporting on MTP project (SDK 9)"
-    prompt: "I have an MSTest project using Microsoft.Testing.Platform. I need to run the tests and produce a TRX report. I'm on .NET SDK 9."
+    prompt: "I have an MSTest project using Microsoft.Testing.Platform. I need to run the tests and produce a TRX report. I'm on .NET SDK 9. Show me the exact command."
     setup:
       files:
         - path: "TestProject.csproj"
@@ -47,7 +47,7 @@ scenarios:
     timeout: 480
 
   - name: "Run tests with blame-hang on MTP project (SDK 10)"
-    prompt: "My tests are occasionally hanging. I want to run them with dotnet test using a hang timeout of 5 minutes to detect the culprit. Here's my project."
+    prompt: "My tests are occasionally hanging. I want to run them with dotnet test using a hang timeout of 5 minutes to detect the culprit. Here's my project. Show me the exact command."
     setup:
       files:
         - path: "TestProject.csproj"
@@ -73,7 +73,7 @@ scenarios:
     timeout: 420
 
   - name: "Run tests on a specific TFM with TRX in a multi-TFM MTP project (SDK 9)"
-    prompt: "This MSTest project uses Microsoft.Testing.Platform and targets both net8.0 and net9.0. I'm on .NET SDK 9. I want to run only the net9.0 tests and produce a TRX report."
+    prompt: "This MSTest project uses Microsoft.Testing.Platform and targets both net8.0 and net9.0. I'm on .NET SDK 9. I want to run only the net9.0 tests and produce a TRX report. Show me the exact command."
     setup:
       files:
         - path: "TestProject.csproj"
@@ -102,6 +102,7 @@ scenarios:
   - name: "Filter MSTest tests by category on VSTest"
     prompt: "I only want to run the integration tests using dotnet test. How do I filter by the Integration category?"
     setup:
+      additional_required_skills: ["filter-syntax"]
       files:
         - path: "TestProject.csproj"
           source: "fixtures/vstest-mstest/TestProject.csproj"
@@ -124,6 +125,7 @@ scenarios:
   - name: "Filter NUnit tests by class name on VSTest"
     prompt: "I want to run only the ReportExporterTests class using dotnet test, not the ReportGeneratorTests. How do I do that?"
     setup:
+      additional_required_skills: ["filter-syntax"]
       files:
         - path: "TestProject.csproj"
           source: "fixtures/vstest-nunit/TestProject.csproj"
@@ -146,6 +148,7 @@ scenarios:
   - name: "Filter xUnit v3 tests by class on MTP"
     prompt: "I need to run only the ShoppingCartTests class using dotnet test, not the InventoryServiceTests. Can you show me the command?"
     setup:
+      additional_required_skills: ["filter-syntax"]
       files:
         - path: "TestProject.csproj"
           source: "fixtures/mtp-xunit-v3/TestProject.csproj"
@@ -170,6 +173,7 @@ scenarios:
   - name: "Filter xUnit v3 tests by trait on MTP"
     prompt: "I want to run only the tests with the Category=Integration trait in this xUnit project using dotnet test."
     setup:
+      additional_required_skills: ["filter-syntax"]
       files:
         - path: "TestProject.csproj"
           source: "fixtures/mtp-xunit-v3/TestProject.csproj"
@@ -194,6 +198,7 @@ scenarios:
   - name: "Filter xUnit v3 tests by class pattern and trait using query filter language"
     prompt: "I have an xUnit v3 project with several test classes. I need to run only the Smoke tests from classes whose name contains 'Integration' — I want a single combined filter expression for dotnet test rather than separate flags."
     setup:
+      additional_required_skills: ["filter-syntax"]
       files:
         - path: "TestProject.csproj"
           source: "fixtures/mtp-xunit-v3/TestProject.csproj"
@@ -205,7 +210,7 @@ scenarios:
       - type: "output_matches"
         pattern: "--\\s+--filter-query"
       - type: "output_matches"
-        pattern: '--filter-query\\s+"(?=[^"]*Integration)(?=[^"]*\\[Category=Smoke\\])[^"]+"'
+        pattern: '--filter-query\s+"(?=[^"]*Integration)(?=[^"]*\[Category=Smoke\])[^"]+"'
       - type: "exit_success"
     rubric:
       - "Recognized that xUnit v3 on MTP supports a query filter language for complex combined expressions"
@@ -219,6 +224,7 @@ scenarios:
   - name: "Filter TUnit tests by class using treenode-filter"
     prompt: "I want to run only the SmsNotificationTests from this TUnit project using dotnet test. What's the command?"
     setup:
+      additional_required_skills: ["filter-syntax"]
       files:
         - path: "TestProject.csproj"
           source: "fixtures/mtp-tunit/TestProject.csproj"
@@ -242,6 +248,7 @@ scenarios:
   - name: "Combine multiple filter criteria on VSTest MSTest"
     prompt: "I want to run only the unit tests in the OrderServiceTests class using dotnet test. How do I combine these two filter conditions?"
     setup:
+      additional_required_skills: ["filter-syntax"]
       files:
         - path: "TestProject.csproj"
           source: "fixtures/vstest-mstest/TestProject.csproj"
@@ -270,7 +277,7 @@ scenarios:
   # ============================================================================
 
   - name: "MTP project on SDK 9 must use -- separator for args"
-    prompt: "I want to run these tests with dotnet test and collect a crash dump if any test crashes the host."
+    prompt: "I want to run these tests with dotnet test and collect a crash dump if any test crashes the host. Show me the exact command."
     setup:
       files:
         - path: "TestProject.csproj"
@@ -294,7 +301,7 @@ scenarios:
     timeout: 240
 
   - name: "MTP project on SDK 10 passes args directly"
-    prompt: "I want to run these tests with dotnet test and produce a TRX report."
+    prompt: "I want to run these tests with dotnet test and produce a TRX report. Show me the exact command."
     setup:
       files:
         - path: "TestProject.csproj"
@@ -320,8 +327,9 @@ scenarios:
     timeout: 300
 
   - name: "Detect test platform from Directory.Build.props"
-    prompt: "I want to run the integration tests in this project with dotnet test. Can you figure out which test platform it uses and run them?"
+    prompt: "I want to run the integration tests in this project with dotnet test. Can you figure out which test platform it uses and run them? Show me the exact command you run."
     setup:
+      additional_required_skills: ["platform-detection", "filter-syntax"]
       files:
         - path: "TestProject.csproj"
           content: |
@@ -383,7 +391,7 @@ scenarios:
     timeout: 240
 
   - name: "Negative test: do not use MTP syntax for a VSTest project"
-    prompt: "Run my tests with dotnet test and generate a TRX report."
+    prompt: "Run my tests with dotnet test and generate a TRX report. Show me the exact command."
     setup:
       files:
         - path: "TestProject.csproj"