Skip to content
Merged
Show file tree
Hide file tree
Changes from 5 commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -36,7 +36,14 @@ Analyze .NET test code for maintainability issues: duplicated boilerplate, copy-

### Step 1: Gather the test code

Read all test files the user provides or references. If the user points to a directory or project, scan for all test files — see the `dotnet-test-frameworks` skill for framework-specific markers.
Read all test files the user provides or references. If the user points to a directory or project, scan for all test files using these framework markers:

| Framework | Test class markers | Test method markers |
|-----------|--------------------|---------------------|
| MSTest | `[TestClass]` | `[TestMethod]`, `[DataTestMethod]` |
| xUnit | *(none — convention-based)* | `[Fact]`, `[Theory]` |
| NUnit | `[TestFixture]` | `[Test]`, `[TestCase]`, `[TestCaseSource]` |
| TUnit | *(none — convention-based)* | `[Test]` |

### Step 2: Identify maintainability issues

Expand Down
1 change: 0 additions & 1 deletion plugins/dotnet-test/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -71,7 +71,6 @@ For non-.NET languages, use the native coverage tool: `coverage.py`/`pytest-cov`
| **test-analysis-extensions** | Language-specific guidance loaded by the polyglot analysis skills (test markers, assertion APIs, sleeps, skips, mystery-guest indicators, integration markers, tag-support capability) |
| **platform-detection** *(.NET)* | Detect VSTest vs MTP and identify the test framework from project files |
| **filter-syntax** *(.NET)* | Test filter syntax reference for VSTest and MTP across all frameworks |
| **dotnet-test-frameworks** *(.NET)* | Framework detection patterns, assertion APIs, skip annotations, and lifecycle methods (kept for backward compatibility with .NET-only skills like `writing-mstest-tests`) |

## Agents

Expand Down
139 changes: 0 additions & 139 deletions plugins/dotnet-test/skills/dotnet-test-frameworks/SKILL.md

This file was deleted.

Original file line number Diff line number Diff line change
Expand Up @@ -2,8 +2,6 @@

Reference data for analyzing .NET test code. Used by the polyglot test analysis skills (`assertion-quality`, `test-anti-patterns`, `test-gap-analysis`, `test-smell-detection`, `test-tagging`).

> See also: the standalone `dotnet-test-frameworks` skill, which carries the same data and is loaded by .NET-only skills.

## Capability tags

| Capability | Support |
Expand Down
176 changes: 16 additions & 160 deletions tests/dotnet-experimental/exp-test-maintainability/eval.vally.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -6,99 +6,14 @@ config:
stimuli:
- name: Recommend data-driven patterns with display names for unclear parameters
prompt: |
I need to add more test cases for our validation logic but each new case needs a whole new method.
Can you suggest a better structure?

```csharp
using Microsoft.VisualStudio.TestTools.UnitTesting;

namespace Validation.Tests;

[TestClass]
public sealed class InputValidatorTests
{
[TestMethod]
public void Validate_EmptyString_ReturnsFalse()
{
var validator = new InputValidator();
var result = validator.Validate("");
Assert.IsFalse(result.IsValid);
Assert.AreEqual("Input cannot be empty", result.ErrorMessage);
}

[TestMethod]
public void Validate_TooShort_ReturnsFalse()
{
var validator = new InputValidator();
var result = validator.Validate("ab");
Assert.IsFalse(result.IsValid);
Assert.AreEqual("Input must be at least 3 characters", result.ErrorMessage);
}

[TestMethod]
public void Validate_TooLong_ReturnsFalse()
{
var validator = new InputValidator();
var result = validator.Validate(new string('x', 256));
Assert.IsFalse(result.IsValid);
Assert.AreEqual("Input must not exceed 255 characters", result.ErrorMessage);
}

[TestMethod]
public void Validate_ContainsSpecialChars_ReturnsFalse()
{
var validator = new InputValidator();
var result = validator.Validate("hello<script>");
Assert.IsFalse(result.IsValid);
Assert.AreEqual("Input contains invalid characters", result.ErrorMessage);
}

[TestMethod]
public void Validate_ValidInput_ReturnsTrue()
{
var validator = new InputValidator();
var result = validator.Validate("hello-world");
Assert.IsTrue(result.IsValid);
Assert.IsNull(result.ErrorMessage);
}

[TestMethod]
public void Validate_MinLength_ReturnsTrue()
{
var validator = new InputValidator();
var result = validator.Validate("abc");
Assert.IsTrue(result.IsValid);
Assert.IsNull(result.ErrorMessage);
}

[TestMethod]
public void Validate_MaxLength_ReturnsTrue()
{
var validator = new InputValidator();
var result = validator.Validate(new string('a', 255));
Assert.IsTrue(result.IsValid);
Assert.IsNull(result.ErrorMessage);
}

[TestMethod]
public void Validate_Null_ReturnsFalse()
{
var validator = new InputValidator();
var result = validator.Validate(null);
Assert.IsFalse(result.IsValid);
Assert.AreEqual("Input cannot be empty", result.ErrorMessage);
}

[TestMethod]
public void Validate_WhitespaceOnly_ReturnsFalse()
{
var validator = new InputValidator();
var result = validator.Validate(" ");
Assert.IsFalse(result.IsValid);
Assert.AreEqual("Input cannot be empty", result.ErrorMessage);
}
}
```
I keep adding test cases to my Validation.Tests project and each new case
needs a whole new method. Can you analyze the project and suggest a better structure?
environment:
files:
- src: fixtures/data-driven-candidate/Validation.Tests/Validation.Tests.csproj
dest: Validation.Tests/Validation.Tests.csproj
- src: fixtures/data-driven-candidate/Validation.Tests/InputValidatorTests.cs
dest: Validation.Tests/InputValidatorTests.cs
graders:
- type: output-matches
config:
Expand All @@ -117,73 +32,14 @@ stimuli:
- Acknowledged that the tests are otherwise well-structured (focused, clear assertions, proper naming)
- name: Recognize well-maintained tests that need minimal changes
prompt: |
Can you review these tests for maintainability? I want to make sure they'll be easy
to update as the codebase evolves.

```csharp
using Microsoft.VisualStudio.TestTools.UnitTesting;

namespace Auth.Tests;

[TestClass]
public sealed class TokenServiceTests
{
[TestMethod]
[DataRow("user@example.com", "admin", DisplayName = "Admin user gets full-access token")]
[DataRow("viewer@example.com", "viewer", DisplayName = "Viewer gets read-only token")]
[DataRow("api@example.com", "service", DisplayName = "Service account gets API token")]
public void GenerateToken_ValidUser_ReturnsTokenWithCorrectRole(string email, string role)
{
var service = CreateTokenService();

var token = service.GenerateToken(email, role);

Assert.IsNotNull(token);
Assert.AreEqual(role, token.Role);
Assert.IsTrue(token.ExpiresAt > DateTime.UtcNow);
}

[TestMethod]
public void GenerateToken_ExpiredCredentials_ThrowsAuthException()
{
var service = CreateTokenService(clockOffset: TimeSpan.FromDays(-1));

Assert.ThrowsException<AuthenticationException>(
() => service.GenerateToken("user@example.com", "admin"));
}

[TestMethod]
public void RevokeToken_ValidToken_MarksRevoked()
{
var service = CreateTokenService();
var token = service.GenerateToken("user@example.com", "admin");

service.RevokeToken(token.Id);

Assert.IsTrue(service.IsRevoked(token.Id));
}

[TestMethod]
public void RevokeToken_AlreadyRevoked_IsIdempotent()
{
var service = CreateTokenService();
var token = service.GenerateToken("user@example.com", "admin");
service.RevokeToken(token.Id);

service.RevokeToken(token.Id); // second call

Assert.IsTrue(service.IsRevoked(token.Id));
}

private static TokenService CreateTokenService(TimeSpan? clockOffset = null)
{
var clock = clockOffset.HasValue
? new FakeClock(DateTimeOffset.UtcNow + clockOffset.Value)
: new FakeClock(DateTimeOffset.UtcNow);
return new TokenService(clock, new InMemoryTokenStore());
}
}
```
Can you review my Auth.Tests project for maintainability? I want to make sure
the tests will be easy to update as the codebase evolves.
environment:
files:
- src: fixtures/well-maintained/Auth.Tests/Auth.Tests.csproj
dest: Auth.Tests/Auth.Tests.csproj
- src: fixtures/well-maintained/Auth.Tests/TokenServiceTests.cs
dest: Auth.Tests/TokenServiceTests.cs
graders:
- type: output-matches
config:
Expand Down
Loading
Loading