-
-
Notifications
You must be signed in to change notification settings - Fork 108
perf: remove telemetry overhead from benchmark projects #4314
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
Benchmark projects now reference TUnit.Engine and TUnit.Assertions directly instead of the TUnit metapackage, which includes: - Microsoft.Testing.Extensions.Telemetry (AppInsights) - Microsoft.Testing.Extensions.CodeCoverage - Microsoft.Testing.Extensions.TrxReport This eliminates ~35-40% overhead in single test execution benchmarks caused by telemetry initialization (AppInsights background threads, I/O completion port waiting). Measured improvement: - Before: ~565ms for single test execution - After: ~370ms for single test execution Co-Authored-By: Claude Opus 4.5 <[email protected]>
SummaryRemoves telemetry overhead from benchmark projects by switching from TUnit metapackage to direct TUnit.Engine + TUnit.Assertions references. Critical IssuesMissing Source Generator in UnifiedTests The tools/speed-comparison/UnifiedTests/UnifiedTests.csproj change removes the TUnit metapackage but fails to add the required source generator reference. The test files use TUnit attributes ([TestClass], [DataDrivenTest], etc.) which require source generation. Required fix: Add the source generator reference like TUnit.PerformanceBenchmarks does on line 18. Note: TUnit.PerformanceBenchmarks already has this correctly. SuggestionsNone - the approach is sound once the source generator is added. VerdictREQUEST CHANGES - Critical source generator missing in UnifiedTests project |
Summary
TUnit.EngineandTUnit.Assertionsdirectly instead of theTUnitmetapackageProblem
The
TUnitmetapackage includesMicrosoft.Testing.Extensions.Telemetrywhich adds AppInsights telemetry. During profiling, we found that 91% of single test execution time was spent waiting, primarily due to:GetQueuedCompletionStatus)Changes
TUnit.PerformanceBenchmarksTUnit.csprojTUnit.Engine+TUnit.AssertionsUnifiedTests(speed-comparison)TUnitpackageTUnit.Engine+TUnit.AssertionsMeasured Improvement
Test plan
TUnit.PerformanceBenchmarksproject and verify tests runTestFramework=TUNIT🤖 Generated with Claude Code