feat: add DisplayName, Skip, and Categories support for parameterized tests

[thomhurst]

Summary

• Adds TestDataRow<T> wrapper type for method/class data sources to specify per-row metadata
• Adds DisplayName, Skip, and Categories properties to [Arguments] attribute
• Supports $paramName and $arg1 substitution in display names
• Works with any IDataSourceAttribute implementation

Closes #4212

Features

TestDataRow Wrapper

public static IEnumerable<TestDataRow<(string Username, string Password)>> GetCredentials()
{
    yield return new(("admin", "secret123"), DisplayName: "Admin login");
    yield return new(("guest", "guest"), DisplayName: "Guest login");
    yield return new(("", ""), DisplayName: "Empty credentials", Skip: "Not implemented yet");
}

Arguments Attribute Properties

[Test]
[Arguments(1, 2, 3, DisplayName = "Adding $arg1 + $arg2 = $arg3")]
[Arguments(5, 5, 10, DisplayName = "Five plus five", Categories = new[] { "Math" })]
public async Task Addition(int a, int b, int expected) { }

New Files

• TUnit.Core/TestDataRow.cs - Public wrapper type
• TUnit.Core/TestDataRowMetadata.cs - Internal metadata record
• TUnit.Core/Helpers/TestDataRowUnwrapper.cs - AOT-compatible unwrapping
• TUnit.Core/Helpers/DisplayNameSubstitutor.cs - Parameter substitution
• TUnit.Engine/Helpers/DataSourceMetadataExtractor.cs - Reflection mode support
• docs/docs/test-authoring/test-data-row.md - Documentation

Test plan

• Verify [Arguments(DisplayName = "...")] shows custom name in test output
• Verify TestDataRow<T> with DisplayName works with MethodDataSource
• Verify $paramName substitution works correctly
• Verify Skip property skips individual test cases
• Verify Categories property applies categories for filtering
• Verify AOT compatibility (no reflection in hot paths)

🤖 Generated with Claude Code

[thomhurst]

Summary

Adds TestDataRow wrapper and metadata properties (DisplayName, Skip, Categories) to [Arguments] attribute for customizing individual parameterized test cases.

Critical Issues

BLOCKING: Dual-Mode Violation (CRITICAL RULE #1)

Problem: This feature only works in reflection mode (TUnit.Engine) but NOT in source-gen mode (TUnit.Core.SourceGenerator).

Evidence:

• No source generator files modified in this PR
• DataSourceMetadataExtractor.cs uses reflection to extract metadata
• This code path only executes in TUnit.Engine/Building/TestBuilder.cs (reflection mode)

Impact: Users running with source generators will not get DisplayName, Skip, or Categories metadata from [Arguments] attributes or TestDataRow wrappers.

Required Fix: Add source generator support in TUnit.Core.SourceGenerator to:

1. Detect DisplayName, Skip, Categories properties on ArgumentsAttribute
2. Detect TestDataRow wrappers in data source methods (compile-time analysis)
3. Generate equivalent metadata collection code

See .claude/docs/mandatory-rules.md - Rule #1 requires both source-gen AND reflection modes to work.

AOT Compatibility Concerns

Issue: DataSourceMetadataExtractor.cs uses UnconditionalSuppressMessage with weak justification.

Suggestion: Add [DynamicallyAccessedMembers(DynamicallyAccessedMemberTypes.PublicProperties)] to ensure the trimmer preserves properties needed for reflection.

Suggestions

Performance: DisplayNameSubstitutor allocations

Location: TUnit.Core/Helpers/DisplayNameSubstitutor.cs:37-53

Multiple string allocations in loop. Consider StringBuilder if this becomes a bottleneck, though current implementation is likely acceptable since it runs once per test during discovery.

Verdict

REQUEST CHANGES - Critical dual-mode violation must be fixed before merge.

The feature is well-designed and code quality is good, but it fundamentally violates TUnit critical rule #1 by only supporting reflection mode.

[Copilot]

Pull request overview

This pull request adds support for per-row metadata (DisplayName, Skip, Categories) to parameterized tests. It introduces a new TestDataRow<T> wrapper type for method/class data sources and adds corresponding properties to the [Arguments] attribute. Display names support parameter substitution using $paramName or $arg1 syntax.

Key Changes:

• New public API: TestDataRow<T> record type for wrapping test data with metadata
• ArgumentsAttribute extended with DisplayName, Skip, and Categories properties
• Display name substitution engine with parameter and positional placeholder support

Reviewed changes

Copilot reviewed 14 out of 14 changed files in this pull request and generated 4 comments.

Show a summary per file

File	Description
TUnit.Core/TestDataRow.cs	New public wrapper type with ITestDataRow interface for AOT compatibility
TUnit.Core/TestDataRowMetadata.cs	Internal metadata record with merge logic for precedence handling
TUnit.Core/Helpers/TestDataRowUnwrapper.cs	AOT-compatible unwrapping logic using interface-based access
TUnit.Core/Helpers/DisplayNameSubstitutor.cs	Parameter substitution engine for $paramName and $arg1 placeholders
TUnit.Core/Attributes/TestData/ArgumentsAttribute.cs	Added DisplayName and Categories properties to both generic and non-generic versions
TUnit.Core/TestContext.cs	Added DataSourceDisplayName property and SetDataSourceDisplayName method
TUnit.Core/TestContext.Metadata.cs	Updated GetDisplayName to check DataSourceDisplayName and apply substitution
TUnit.Core/TestDataCombination.cs	Added Skip and Categories properties for data source metadata
TUnit.Engine/Helpers/DataSourceMetadataExtractor.cs	Reflection-based metadata extraction from IDataSourceAttribute implementations
TUnit.Engine/Helpers/DataUnwrapper.cs	Enhanced with metadata-aware unwrapping methods
TUnit.Engine/Building/TestBuilder.cs	Metadata application logic in BuildTestAsync for reflection mode
docs/docs/test-authoring/test-data-row.md	Comprehensive documentation with examples for all features
docs/docs/test-authoring/arguments.md	Updated with metadata property examples and usage guidance
docs/sidebars.ts	Added new documentation page to sidebar navigation

[Copilot]

The TestDataCombination class now has Skip and Categories properties, but there's no corresponding logic in the source generator to populate these properties when creating TestDataCombination instances from ArgumentsAttribute metadata.

For dual-mode parity, the source generator needs to extract these properties from ArgumentsAttribute when generating test combinations. The reflection mode handles this via DataSourceMetadataExtractor.ExtractFromAttribute and TestBuilder.BuildTestAsync (lines 874-897), but the source-generated code path needs equivalent logic.

[Copilot]

The new DisplayName and Categories properties are defined but not applied to the test context. The ArgumentsAttribute implements ITestRegisteredEventReceiver with an OnTestRegistered method (lines 81-90) that only handles the Skip property.

For dual-mode parity, the OnTestRegistered method needs to be updated to also apply DisplayName and Categories properties, similar to how the reflection mode handles this in TestBuilder.cs lines 874-897. Without this, the source-generated mode won't respect these properties.

The OnTestRegistered method should:

1. Call context.SetDataSourceDisplayName(DisplayName) when DisplayName is not null/empty
2. Add Categories to context.TestContext.Metadata.TestDetails.Categories when Categories is not null/empty

[Copilot]

The new DisplayName and Categories properties are defined but not applied to the test context. The generic ArgumentsAttribute also implements ITestRegisteredEventReceiver with an OnTestRegistered method (lines 120-129) that only handles the Skip property.

For dual-mode parity, the OnTestRegistered method needs to be updated to also apply DisplayName and Categories properties, similar to how the reflection mode handles this in TestBuilder.cs lines 874-897. Without this, the source-generated mode won't respect these properties.

The OnTestRegistered method should:

1. Call context.SetDataSourceDisplayName(DisplayName) when DisplayName is not null/empty
2. Add Categories to context.TestContext.Metadata.TestDetails.Categories when Categories is not null/empty

[Copilot]

The substitution logic checks for a dollar sign in the display name and performs replacements, but consider edge cases where parameter names or values themselves might contain dollar signs. While unlikely in typical scenarios, this could lead to unintended substitutions.

Consider using a more explicit placeholder syntax (e.g., {paramName} or {{paramName}}) or documenting this limitation if the current approach is intentional.

[thomhurst]

Summary

This PR adds DisplayName, Skip, and Categories support for parameterized tests via ArgumentsAttribute properties and a new TestDataRow wrapper for method/class data sources.

Critical Issues

None found ✅

The second commit successfully addresses the dual-mode requirement by implementing DisplayName and Categories handling in ArgumentsAttribute.OnTestRegistered (source-gen mode), and DataSourceMetadataExtractor properly handles reflection mode with appropriate AOT annotations.

[thomhurst]

Suggestions

1. Consider caching in DisplayNameSubstitutor

Location: TUnit.Core/Helpers/DisplayNameSubstitutor.cs:22-58

The Substitute method uses multiple string.Contains and string.Replace calls in a loop. For frequently-used display names, this could be optimized by using a compiled regex or Span based approach if this becomes a hot path.

Impact: Minor - only affects display name generation, not test execution. Worth monitoring if users report slowdowns with many parameterized tests.

[thomhurst]

2. Categories list duplication check could be more efficient

Location: TUnit.Core/Attributes/TestData/ArgumentsAttribute.cs:94-99 (and similar in ArgumentsAttribute)

If Categories is a List, the Contains check is O(n). If many categories are added across multiple data sources, consider using a HashSet for Categories if not already, or checking the implementation of context.TestDetails.Categories.

Impact: Low - unlikely to have many categories, but worth noting for consistency.

[thomhurst]

Previous Review Status

The second commit (436f382) addresses previous review feedback by:

• ✅ Adding DisplayName/Categories handling to ArgumentsAttribute.OnTestRegistered (fixes dual-mode violation)
• ✅ Adding proper [DynamicDependency] annotations to DataSourceMetadataExtractor for AOT compatibility

Verdict

✅ APPROVE - No critical issues. The implementation correctly follows TUnit's critical rules:

• Dual-Mode: ✅ Works in both source-gen (ArgumentsAttribute.OnTestRegistered) and reflection (DataSourceMetadataExtractor) modes
• AOT Compatible: ✅ Uses interface-based access (ITestDataRow) + DynamicDependency annotations
• No VSTest: ✅ N/A - no test platform changes
• Performance: ✅ Reasonable - minimal allocations, no obvious hot path concerns
• Snapshot Testing: ✅ N/A - no source generator output changes requiring snapshots

The suggestions above are optional optimizations, not blocking issues.

[thomhurst]

Summary

Adds TestDataRow<T> wrapper and DisplayName, Skip, Categories properties to [Arguments] attribute for per-row test metadata.

Critical Issues

❌ BLOCKING: Dual-Mode Rule Violation (#1)

This PR implements the new metadata features (DisplayName, Skip, Categories) only in reflection mode (TUnit.Engine). The source generator mode (TUnit.Core.SourceGenerator) is missing the implementation.

Evidence:

The reflection mode correctly extracts these properties in:

• TUnit.Engine/Helpers/DataSourceMetadataExtractor.cs:26 - Extracts DisplayName, Skip, Categories from ArgumentsAttribute
• TUnit.Engine/Building/TestBuilder.cs:324-330 - Applies metadata to test context
• TUnit.Core/Attributes/TestData/ArgumentsAttribute.cs:86-102 - OnTestRegistered applies DisplayName and Categories

However, the source generator does NOT preserve these properties:

• When generating ArgumentsAttribute instances, the source generator only copies constructor arguments (the data values)
• Named properties like DisplayName, Skip, and Categories are not extracted from NamedArguments
• This means source-gen tests will NOT respect these new properties

Required Fix:

Update TUnit.Core.SourceGenerator/Generators/TestMetadataGenerator.cs in the GenerateArgumentsAttributeWithParameterTypes method (around line 764) to:

1. Extract NamedArguments from the AttributeData:

   var skipValue = attr.NamedArguments.FirstOrDefault(x => x.Key == "Skip").Value;
   var displayNameValue = attr.NamedArguments.FirstOrDefault(x => x.Key == "DisplayName").Value;
   var categoriesValue = attr.NamedArguments.FirstOrDefault(x => x.Key == "Categories").Value;

2. Include them in the generated code with object initializer syntax (similar to how AttributeWriter.GetAttributeObjectInitializer does it)

3. Update snapshot tests to reflect the new property initialization

Why This Matters:

Without this fix, tests written like:

[Arguments(1, 2, DisplayName = "Custom name", Skip = "Not ready")]

Will work correctly in reflection mode but will silently ignore DisplayName/Skip/Categories in source generator mode, creating inconsistent behavior.

---

✅ Snapshot Tests: Correctly updated .verified.txt files (no .received.txt committed)

✅ AOT Compatibility: Proper use of DynamicallyAccessedMembers and ITestDataRow interface to avoid reflection in hot paths

✅ Performance: TestDataRowUnwrapper uses interface-based access instead of reflection

Verdict

⚠️ REQUEST CHANGES - Must implement dual-mode support in source generator before merge