Skip to content

perf: avoid formatting interface names for data-source checks - #6782

Merged
thomhurst merged 1 commit into
mainfrom
perf/generator-interface-checks
Sep 11, 2026
Merged

thomhurst merged 1 commit into
mainfrom
perf/generator-interface-checks

Conversation

@thomhurst

@thomhurst thomhurst commented Sep 11, 2026

Copy link
Copy Markdown
Owner

Data-source detection repeatedly formats fully qualified names for implemented interfaces, including unrelated generic interfaces. This work runs while filtering and emitting test attributes.

Reject interfaces with a different simple name, then recognize the usual non-generic TUnit.Core.IDataSourceAttribute through symbol properties. Keep the previous display-name comparison as a fallback for unusual shapes. Route both existing detection entry points through the same implementation. Generated source remains unchanged.

Focused BenchmarkDotNet results using actual baseline (e322d68799) and modified generator assemblies:

Attribute symbol Before mean After mean Before allocated After allocated
ArgumentsAttribute 187.82 ns 31.86 ns 152 B 48 B
ArgumentsAttribute<int> 528.26 ns 45.71 ns 408 B 56 B

The checks are approximately 6–12× faster and allocate 104–352 fewer bytes per call.

Complete TestMetadataGenerator passes over 10,000 tests in 100 classes:

Workload Before mean After mean Before allocated After allocated
Bare tests (control) 218.8 ms 162.9 ms 178.24 MB 178.55 MB
Inline data 349.5 ms 341.4 ms 383.81 MB 381.91 MB
Typed inline data 454.3 ms 307.8 ms 392.98 MB 382.90 MB

The allocation reductions are approximately 1.9 MB and 10.1 MB for the data-driven suites. Full-generator timing improvements are inconclusive: confidence intervals overlap and the bare-test control also changed substantially. These measurements exclude parsing and compilation of generated code; no whole-build speedup is claimed.

Validation: 133 generator tests pass with one existing skip and no snapshot changes. All 11 new classification cases pass on .NET 10 and .NET Framework 4.7.2. Roslyn 4.4 and 4.14 builds pass without warnings/errors. Benchmark setup verifies identical generated filenames and source text for all three workloads. Tests explicitly preserve legacy matching of nested interfaces with the same display name.

Environment: BenchmarkDotNet 0.15.8, Roslyn 4.14.0, Windows 11, i7-12700K, .NET 10.0.12, SDK 11.0.100-preview.7.26381.103; 15 measured iterations, 6 warmups, 1 launch. Full reports, confidence intervals, source, and reproduction commands are in the PR comment.

Summary by CodeRabbit

  • Bug Fixes

    • Improved recognition of data source attributes across direct, inherited, indirect, generic, nested-interface, and namespace variations.
    • Preserved existing behavior for attributes identified by their display name.
    • Added safe handling for missing or null attributes, preventing incorrect data source classification.
  • Tests

    • Added comprehensive coverage for data source attribute detection, including generated source validation and diagnostic checks.

@thomhurst
thomhurst deployed to Pull Requests September 11, 2026 15:38 — with GitHub Actions Active
@thomhurst
thomhurst deployed to Pull Requests September 11, 2026 15:38 — with GitHub Actions Active
@thomhurst
thomhurst deployed to Pull Requests September 11, 2026 15:38 — with GitHub Actions Active
@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Sep 11, 2026

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review Completed 2026-09-11T15:41:24.946133Z e2a4cc3 PR opened
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@coderabbitai

coderabbitai Bot commented Sep 11, 2026

Copy link
Copy Markdown

Review Change StackReview Change Stack

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Advanced

Run ID: 55248e83-9a17-4490-9602-4678beb32b37

📥 Commits

Reviewing files that changed from the base of the PR and between e322d68 and e2a4cc3.

📒 Files selected for processing (3)
  • src/TUnit.Core.SourceGenerator/CodeGenerators/Helpers/DataSourceAttributeHelper.cs
  • src/TUnit.Core.SourceGenerator/Extensions/AttributeDataExtensions.cs
  • tests/TUnit.Core.SourceGenerator.Tests/DataSourceAttributeDetectionTests.cs

Included review availability: Your plan provides up to 8 included reviews per hour; 5 remain after this review.


📝 Walkthrough

Walkthrough

The change centralizes IsDataSourceAttribute detection in DataSourceAttributeHelper and adds source-generator tests for valid, invalid, nested, generic, display-name, and null cases.

Changes

Data source attribute detection

Layer / File(s) Summary
Centralized detection delegation
src/TUnit.Core.SourceGenerator/CodeGenerators/Helpers/DataSourceAttributeHelper.cs, src/TUnit.Core.SourceGenerator/Extensions/AttributeDataExtensions.cs
IsDataSourceAttribute delegates detection to DataSourceAttributeHelper.IsDataSourceAttribute. Required namespace imports support the helper and GloballyQualified() extension.
Detection behavior tests
tests/TUnit.Core.SourceGenerator.Tests/DataSourceAttributeDetectionTests.cs
Tests cover direct, inherited, indirect, generic, namespace, nested-interface, display-name, and null attribute cases.

Priority: ⬇️ Low

Estimated code review effort: 2 (Simple) | ~10 minutes

Change: Refactor

Merge Risk: ⚪ Minimal · up to e2a4c

The data-source detection optimization preserves supported detection behavior and has focused test coverage; no actionable merge risk remains.

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 9 functions across 3 files. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title clearly and concisely describes the main change: improving data-source checks by avoiding interface-name formatting.
  • Fix all pre-merge checks with AI
✨ Finishing Touches 💡 1
📝 Generate docstrings 💡
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch perf/generator-interface-checks

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

A rabbit reads each line,
The patch grows clear beneath the moon,
Small changes hop in place,
Tests guard the garden path,
Reviews bloom before the dawn.

Comment @coderabbitai help to get the list of available commands.

@thomhurst

Copy link
Copy Markdown
Owner Author

Reproduction and validation evidence

This compares the actual generator assembly built from baseline e322d68799 with the modified assembly. Both run in the same Roslyn 4.14.0 host, loaded through separate AssemblyLoadContext instances. Module IDs:

  • Before: 6d0870b6-31d9-4530-b829-2e11fb8bbeb5
  • After: ac94805c-b54c-4c1a-a167-178695d3ac07

Build src/TUnit.Core.SourceGenerator/TUnit.Core.SourceGenerator.csproj -c Release at each revision. In a standalone benchmark directory, copy the two resulting netstandard2.0/TUnit.Core.SourceGenerator.dll files into before/ and after/. Copy a locally built TUnit.Core.dll and the repository's global.json into the directory root. This run used the 99.99.99.0 Core assembly from the preceding generator benchmarks. Create the project and program below, then run these commands sequentially:

dotnet run -c Release -- --filter '*' --job Dry
dotnet run -c Release --no-build -- --filter '*InterfaceCheckBenchmarks*' --iterationCount 15 --warmupCount 6 --launchCount 1 --exporters json
dotnet run -c Release --no-build -- --filter '*MetadataGeneratorBenchmarks*' --iterationCount 15 --warmupCount 6 --launchCount 1 --invocationCount 1 --unrollFactor 1 --exporters json

The focused benchmark classifies real ArgumentsAttribute and constructed ArgumentsAttribute<int> symbols. Compilation creation, symbol lookup, reflection, and delegate creation happen only in setup. Measured calls use delegates to the actual before/after helper methods and return their boolean results.

The complete TestMetadataGenerator workload contains 100 classes with 100 test methods each. Setup checks compilation errors, generator diagnostics, and exact equality of every generated filename and source file. Each measured operation starts from the initial driver; returned drivers are not retained, so generation runs again rather than using cached outputs. Parsing and initial compilation validation are excluded. Bare tests provide a control with no data-source attributes.

Validation commands:

dotnet build src/TUnit.Core/TUnit.Core.csproj -c Release -f netstandard2.0
dotnet run --project tests/TUnit.Core.SourceGenerator.Tests -c Release -f net10.0
dotnet run --project tests/TUnit.Core.SourceGenerator.Tests -c Release -f net472 -- --treenode-filter '/*/*/DataSourceAttributeDetectionTests/*'
dotnet build src/TUnit.Core.SourceGenerator.Roslyn414/TUnit.Core.SourceGenerator.Roslyn414.csproj -c Release
dotnet build src/TUnit.Core.SourceGenerator.Roslyn44/TUnit.Core.SourceGenerator.Roslyn44.csproj -c Release

Results: 133 passed and one existing skip in the full generator suite, with no snapshot changes. All 11 new cases pass on .NET 10 and .NET Framework 4.7.2. Both Roslyn variant builds pass with zero warnings/errors. Tests cover direct/inherited/indirect interfaces, generic attributes, similarly named interfaces in other namespaces, generic interface lookalikes, nested types, and null attributes. The nested-type fallback deliberately preserves the previous display-name matching behavior.

Focused classification results:


BenchmarkDotNet v0.15.8, Windows 11 (10.0.26200.9168/25H2/2025Update/HudsonValley2)
12th Gen Intel Core i7-12700K 3.60GHz, 1 CPU, 20 logical and 12 physical cores
.NET SDK 11.0.100-preview.7.26381.103
  [Host]     : .NET 10.0.12 (10.0.12, 10.0.1226.42308), X64 RyuJIT x86-64-v3
  Job-OWOXYI : .NET 10.0.12 (10.0.12, 10.0.1226.42308), X64 RyuJIT x86-64-v3

IterationCount=15  LaunchCount=1  WarmupCount=6  

Method Typed Mean Error StdDev Ratio RatioSD Gen0 Allocated Alloc Ratio
Before False 187.82 ns 3.434 ns 3.212 ns 1.00 0.02 0.0114 152 B 1.00
After False 31.86 ns 0.279 ns 0.233 ns 0.17 0.00 0.0036 48 B 0.32
Before True 528.26 ns 4.386 ns 3.663 ns 1.00 0.01 0.0305 408 B 1.00
After True 45.71 ns 5.615 ns 5.252 ns 0.09 0.01 0.0042 56 B 0.14

Complete metadata-generator results:


BenchmarkDotNet v0.15.8, Windows 11 (10.0.26200.9168/25H2/2025Update/HudsonValley2)
12th Gen Intel Core i7-12700K 3.60GHz, 1 CPU, 20 logical and 12 physical cores
.NET SDK 11.0.100-preview.7.26381.103
  [Host]     : .NET 10.0.12 (10.0.12, 10.0.1226.42308), X64 RyuJIT x86-64-v3
  Job-NOAUNV : .NET 10.0.12 (10.0.12, 10.0.1226.42308), X64 RyuJIT x86-64-v3

InvocationCount=1  IterationCount=15  LaunchCount=1  
UnrollFactor=1  WarmupCount=6  

Method Scenario Mean Error StdDev Ratio RatioSD Gen0 Gen1 Gen2 Allocated Alloc Ratio
Before Bare 218.8 ms 39.75 ms 37.18 ms 1.03 0.26 12000.0000 5000.0000 1000.0000 178.24 MB 1.00
After Bare 162.9 ms 17.97 ms 15.00 ms 0.77 0.16 12000.0000 5000.0000 1000.0000 178.55 MB 1.00
Before InlineData 349.5 ms 66.89 ms 62.57 ms 1.03 0.26 29000.0000 9000.0000 3000.0000 383.81 MB 1.00
After InlineData 341.4 ms 44.01 ms 41.16 ms 1.01 0.22 29000.0000 9000.0000 3000.0000 381.91 MB 1.00
Before TypedInlineData 454.3 ms 78.43 ms 73.36 ms 1.03 0.24 29000.0000 8000.0000 2000.0000 392.98 MB 1.00
After TypedInlineData 307.8 ms 96.31 ms 80.42 ms 0.70 0.22 27000.0000 7000.0000 1000.0000 382.9 MB 0.97
Benchmark project and source
<Project Sdk="Microsoft.NET.Sdk">
  <PropertyGroup>
    <OutputType>Exe</OutputType>
    <TargetFramework>net10.0</TargetFramework>
    <ImplicitUsings>enable</ImplicitUsings>
    <Nullable>enable</Nullable>
  </PropertyGroup>
  <ItemGroup>
    <PackageReference Include="BenchmarkDotNet" Version="0.15.8" />
    <PackageReference Include="Microsoft.CodeAnalysis.CSharp" Version="4.14.0" />
    <Reference Include="TUnit.Core"><HintPath>TUnit.Core.dll</HintPath></Reference>
    <None Update="before/*.dll;after/*.dll" CopyToOutputDirectory="PreserveNewest" />
  </ItemGroup>
</Project>
using BenchmarkDotNet.Attributes;
using BenchmarkDotNet.Running;
using Microsoft.CodeAnalysis;
using Microsoft.CodeAnalysis.CSharp;

using System.Reflection;
using System.Runtime.Loader;
using System.Text;

BenchmarkSwitcher.FromAssembly(typeof(Program).Assembly).Run(args);

public static class Versions
{
    public static Assembly Load(string name)
    {
        var assembly = new AssemblyLoadContext(name).LoadFromAssemblyPath(
            Path.Combine(AppContext.BaseDirectory, name, "TUnit.Core.SourceGenerator.dll"));
        Console.WriteLine($"{name} MVID: {assembly.ManifestModule.ModuleVersionId}");
        return assembly;
    }
}

[MemoryDiagnoser]
public class MetadataGeneratorBenchmarks
{
    [Params("Bare", "InlineData", "TypedInlineData")]
    public string Scenario { get; set; } = "Bare";
    private CSharpCompilation _compilation = null!;
    private GeneratorDriver _before = null!;
    private GeneratorDriver _after = null!;

    [GlobalSetup]
    public void Setup()
    {
        var trees = new List<SyntaxTree>();
        for (var c = 0; c < 100; c++)
        {
            var source = new StringBuilder("using System; using TUnit.Core; public class Tests" + c + " {");
            for (var m = 0; m < 100; m++)
            {
                source.Append("[Test]");
                source.Append(Scenario switch { "InlineData" => "[Arguments(42)] public void Test", "TypedInlineData" => "[Arguments<int>(42)] public void Test", _ => "public void Test" });
                source.Append(m).Append(Scenario != "Bare" ? "(int value) {" : "() {");
                source.Append("int x = 42; if (x * x + 1 != 1765) throw new Exception(); }");
            }
            source.Append('}');
            trees.Add(CSharpSyntaxTree.ParseText(source.ToString()));
        }
        var references = ((string)AppContext.GetData("TRUSTED_PLATFORM_ASSEMBLIES")!).Split(Path.PathSeparator)
            .Select(p => MetadataReference.CreateFromFile(p)).ToList();
        references.Add(MetadataReference.CreateFromFile(typeof(TUnit.Core.TestAttribute).Assembly.Location));
        _compilation = CSharpCompilation.Create("SyntheticSuite", trees, references,
            new CSharpCompilationOptions(OutputKind.DynamicallyLinkedLibrary));
        var errors = _compilation.GetDiagnostics().Where(d => d.Severity == DiagnosticSeverity.Error).ToArray();
        if (errors.Length != 0) throw new InvalidOperationException(string.Join("\n", errors.Select(e => e.ToString())));
        _before = LoadDriver("before");
        _after = LoadDriver("after");
        var beforeResult = _before.RunGenerators(_compilation).GetRunResult();
        var afterResult = _after.RunGenerators(_compilation).GetRunResult();
        if (beforeResult.Diagnostics.Concat(afterResult.Diagnostics).Any(d => d.Severity == DiagnosticSeverity.Error))
            throw new InvalidOperationException("Generator diagnostics");
        var beforeSources = beforeResult.Results.Single().GeneratedSources;
        var afterSources = afterResult.Results.Single().GeneratedSources;
        if (beforeSources.Length != 100 || afterSources.Length != 100 ||
            !beforeSources.Select(s => (s.HintName, s.SourceText.ToString()))
                .SequenceEqual(afterSources.Select(s => (s.HintName, s.SourceText.ToString()))))
            throw new InvalidOperationException("Generated sources differ or are missing.");
    }

    private static GeneratorDriver LoadDriver(string name) => CSharpGeneratorDriver.Create(
        (IIncrementalGenerator)Activator.CreateInstance(Versions.Load(name).GetType(
            "TUnit.Core.SourceGenerator.Generators.TestMetadataGenerator")!)!);

    [Benchmark(Baseline = true)]
    public GeneratorDriver Before() => _before.RunGenerators(_compilation);

    [Benchmark]
    public GeneratorDriver After() => _after.RunGenerators(_compilation);
}

[MemoryDiagnoser]
public class InterfaceCheckBenchmarks
{
    [Params(false, true)]
    public bool Typed { get; set; }
    private INamedTypeSymbol _attributeClass = null!;
    private Func<INamedTypeSymbol, bool> _before = null!;
    private Func<INamedTypeSymbol, bool> _after = null!;

    [GlobalSetup]
    public void Setup()
    {
        var references = ((string)AppContext.GetData("TRUSTED_PLATFORM_ASSEMBLIES")!).Split(Path.PathSeparator)
            .Select(p => MetadataReference.CreateFromFile(p)).ToList();
        references.Add(MetadataReference.CreateFromFile(typeof(TUnit.Core.TestAttribute).Assembly.Location));
        var compilation = CSharpCompilation.Create("AttributeSymbols", references: references);
        _attributeClass = Typed
            ? compilation.GetTypeByMetadataName("TUnit.Core.ArgumentsAttribute`1")!.Construct(compilation.GetSpecialType(SpecialType.System_Int32))
            : compilation.GetTypeByMetadataName("TUnit.Core.ArgumentsAttribute")!;
        _before = Load("before");
        _after = Load("after");
        if (!_before(_attributeClass) || !_after(_attributeClass))
            throw new InvalidOperationException("Expected both argument attributes to be recognized.");
    }

    private static Func<INamedTypeSymbol, bool> Load(string name) =>
        Versions.Load(name).GetType("TUnit.Core.SourceGenerator.CodeGenerators.Helpers.DataSourceAttributeHelper")!
            .GetMethod("IsDataSourceAttribute")!.CreateDelegate<Func<INamedTypeSymbol, bool>>();

    [Benchmark(Baseline = true)]
    public bool Before() => _before(_attributeClass);

    [Benchmark]
    public bool After() => _after(_attributeClass);
}

@greptile-apps

greptile-apps Bot commented Sep 11, 2026

Copy link
Copy Markdown

Greptile Summary

This PR optimizes data-source attribute detection by avoiding fully qualified interface-name formatting in the common case while retaining the previous comparison as a compatibility fallback.

  • Routes AttributeDataExtensions.IsDataSourceAttribute through the shared helper.
  • Uses symbol properties to recognize the ordinary non-generic TUnit.Core.IDataSourceAttribute.
  • Adds classification tests for direct, inherited, indirect, generic, unrelated, nested, and null cases.

Confidence Score: 5/5

The PR appears safe to merge, with no concrete correctness, compatibility, security, or repository-rule issues identified.

The optimized path preserves the ordinary interface classification and retains the previous display-name comparison as a fallback, while the added tests cover the relevant behavioral boundaries.

Important Files Changed

Filename Overview
src/TUnit.Core.SourceGenerator/CodeGenerators/Helpers/DataSourceAttributeHelper.cs Adds an allocation-conscious symbol-property fast path and retains legacy display-name matching for unusual symbols.
src/TUnit.Core.SourceGenerator/Extensions/AttributeDataExtensions.cs Delegates data-source classification to the shared helper without changing the extension contract.
tests/TUnit.Core.SourceGenerator.Tests/DataSourceAttributeDetectionTests.cs Adds focused coverage for accepted and rejected interface shapes, legacy nested-name behavior, and null input.

Flowchart

%%{init: {'theme': 'neutral'}}%%
flowchart TD
    A[AttributeData or named type] --> B[DataSourceAttributeHelper]
    B --> C{Interface name matches?}
    C -- No --> D[Continue scanning]
    C -- Yes --> E{Ordinary non-generic interface in TUnit.Core?}
    E -- Yes --> F[Data source]
    E -- No --> G{Legacy fully qualified name matches?}
    G -- Yes --> F
    G -- No --> D
    D --> H{More interfaces?}
    H -- Yes --> C
    H -- No --> I[Not a data source]
Loading

Reviews (1): Last reviewed commit: "perf: avoid formatting interface names f..." | Re-trigger Greptile

@thomhurst
thomhurst enabled auto-merge (squash) September 11, 2026 15:54
@thomhurst
thomhurst merged commit e865f18 into main Sep 11, 2026
14 checks passed
@thomhurst
thomhurst deleted the perf/generator-interface-checks branch September 11, 2026 16:13
This was referenced Sep 14, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant