From 7fd4a2e4594c528e1e42605fccb7895fb1b7c29d Mon Sep 17 00:00:00 2001 From: Tom Longhurst <30480171+thomhurst@users.noreply.github.com> Date: Sun, 18 Jan 2026 00:05:10 +0000 Subject: [PATCH 01/13] docs: add IDE streaming sink design for issue #4495 Design for real-time IDE output streaming via ILogSink: - Cumulative replacement approach (no duplication handling needed) - 1-second throttling per test - Streams both stdout and stderr - On by default for IDE clients --- .../2026-01-18-ide-streaming-sink-design.md | 130 ++++++++++++++++++ 1 file changed, 130 insertions(+) create mode 100644 docs/plans/2026-01-18-ide-streaming-sink-design.md diff --git a/docs/plans/2026-01-18-ide-streaming-sink-design.md b/docs/plans/2026-01-18-ide-streaming-sink-design.md new file mode 100644 index 0000000000..d8789ba100 --- /dev/null +++ b/docs/plans/2026-01-18-ide-streaming-sink-design.md @@ -0,0 +1,130 @@ +# IDE Streaming Sink Design + +**Date:** 2026-01-18 +**Issue:** [#4495](https://github.com/thomhurst/TUnit/issues/4495) +**Status:** Approved + +## Overview + +Implement real-time IDE output streaming via a new `IdeStreamingSink` that sends test output to IDEs during test execution, not just at completion. + +## Background + +PR #4493 introduced an extensible log sink framework (`ILogSink`, `TUnitLoggerFactory`). A previous attempt at IDE streaming was removed due to output duplication issues. This design takes a simpler approach. + +## Design Decisions + +| Decision | Choice | Rationale | +|----------|--------|-----------| +| **Duplication handling** | Cumulative replacement | Each update sends full output; IDE displays latest. Simple, no coordination needed. | +| **Throttling** | 1 second intervals | Prevents flooding IDE with rapid writes. Uses latest snapshot per interval. | +| **Output types** | Both stdout and stderr | Stream `StandardOutputProperty` and `StandardErrorProperty` | +| **Activation** | On by default for IDE | Uses existing `VerbosityService.IsIdeClient` detection | + +## Architecture + +### Data Flow + +``` +Console.WriteLine() + -> StandardOutConsoleInterceptor + -> LogSinkRouter routes to all sinks: + -> TestOutputSink: accumulates to Context.OutputWriter (always) + -> IdeStreamingSink: marks test as "dirty" (IDE only) + -> Timer fires every 1s per test + -> GetStandardOutput() + GetErrorOutput() + -> Send TestNodeUpdateMessage with InProgressTestNodeStateProperty +``` + +### Class Structure + +```csharp +internal sealed class IdeStreamingSink : ILogSink, IAsyncDisposable +{ + private readonly TUnitMessageBus _messageBus; + private readonly SessionUid _sessionUid; + private readonly ConcurrentDictionary _activeTests = new(); + private readonly TimeSpan _throttleInterval = TimeSpan.FromSeconds(1); +} + +private sealed class TestStreamingState : IDisposable +{ + public TestContext TestContext { get; } + public Timer Timer { get; } + public bool IsDirty { get; set; } // Has new output since last send +} +``` + +### Core Logic + +1. **On `Log()` call with `TestContext`:** + - Get or create `TestStreamingState` for this test + - Mark as dirty (`IsDirty = true`) + - Timer is already running (started on first log) + +2. **On timer tick (every 1 second):** + - Check if test completed (passive cleanup) - if so, dispose and remove + - If `IsDirty` is false, skip (no new output) + - Set `IsDirty = false` + - Call `testContext.GetStandardOutput()` and `GetErrorOutput()` + - Send `TestNodeUpdateMessage` with `InProgressTestNodeStateProperty` + output properties + +3. **On dispose:** Cancel all timers, clear dictionary + +### TestNode Creation + +```csharp +private TestNode CreateOutputUpdateNode(TestContext testContext, string? output, string? error) +{ + var properties = new List + { + InProgressTestNodeStateProperty.CachedInstance + }; + + if (!string.IsNullOrEmpty(output)) + properties.Add(new StandardOutputProperty(output)); + + if (!string.IsNullOrEmpty(error)) + properties.Add(new StandardErrorProperty(error)); + + return new TestNode + { + Uid = new TestNodeUid(testContext.TestDetails.TestId), + DisplayName = testContext.GetDisplayName(), + Properties = new PropertyBag(properties) + }; +} +``` + +### Registration + +In `TUnitServiceProvider.cs`: + +```csharp +// After existing sink registrations +if (VerbosityService.IsIdeClient) +{ + TUnitLoggerFactory.AddSink(new IdeStreamingSink( + MessageBus, + context.Request.Session.SessionUid)); +} +``` + +## Files to Modify + +| File | Change | +|------|--------| +| `TUnit.Engine/Logging/IdeStreamingSink.cs` | **Create** - new sink implementation | +| `TUnit.Engine/Framework/TUnitServiceProvider.cs` | **Modify** - register sink for IDE clients | + +## Testing Strategy + +1. Manual testing in Visual Studio and Rider +2. Verify output appears during long-running tests +3. Verify no duplication at test completion +4. Verify console mode is unaffected + +## Related + +- PR #4493 - Extensible log sink architecture +- Issue #4478 - Original user feature request From cfa98897e653b376f4e690a13027499f43ad12d1 Mon Sep 17 00:00:00 2001 From: Tom Longhurst <30480171+thomhurst@users.noreply.github.com> Date: Sun, 18 Jan 2026 00:09:05 +0000 Subject: [PATCH 02/13] feat: implement IdeStreamingSink for real-time IDE output (#4495) Adds real-time output streaming to IDEs during test execution: - IdeStreamingSink: New log sink that streams cumulative output every 1 second - Uses timer-based throttling to prevent flooding IDE with messages - Sends both StandardOutputProperty and StandardErrorProperty - Only activated for IDE clients (not console) - Passive cleanup when tests complete Closes #4495 --- .../Framework/TUnitServiceProvider.cs | 6 + TUnit.Engine/Logging/IdeStreamingSink.cs | 176 ++++++++++++++++++ TUnit.Engine/TUnitMessageBus.cs | 8 + 3 files changed, 190 insertions(+) create mode 100644 TUnit.Engine/Logging/IdeStreamingSink.cs diff --git a/TUnit.Engine/Framework/TUnitServiceProvider.cs b/TUnit.Engine/Framework/TUnitServiceProvider.cs index 66e3822a51..2974646dec 100644 --- a/TUnit.Engine/Framework/TUnitServiceProvider.cs +++ b/TUnit.Engine/Framework/TUnitServiceProvider.cs @@ -152,6 +152,12 @@ public TUnitServiceProvider(IExtension extension, StandardErrorConsoleInterceptor.DefaultError)); } + // IdeStreamingSink: For IDE clients - real-time output streaming + if (VerbosityService.IsIdeClient) + { + TUnitLoggerFactory.AddSink(new IdeStreamingSink(MessageBus)); + } + CancellationToken = Register(new EngineCancellationToken()); EventReceiverOrchestrator = Register(new EventReceiverOrchestrator(Logger)); diff --git a/TUnit.Engine/Logging/IdeStreamingSink.cs b/TUnit.Engine/Logging/IdeStreamingSink.cs new file mode 100644 index 0000000000..cfff4a54c0 --- /dev/null +++ b/TUnit.Engine/Logging/IdeStreamingSink.cs @@ -0,0 +1,176 @@ +using System.Collections.Concurrent; +using Microsoft.Testing.Platform.Extensions.Messages; +using TUnit.Core; +using TUnit.Core.Logging; + +#pragma warning disable TPEXP + +namespace TUnit.Engine.Logging; + +/// +/// A log sink that streams test output in real-time to IDE test explorers. +/// Sends cumulative output snapshots every 1 second during test execution. +/// Only activated when running in an IDE environment (not console). +/// +internal sealed class IdeStreamingSink : ILogSink, IAsyncDisposable +{ + private readonly TUnitMessageBus _messageBus; + private readonly ConcurrentDictionary _activeTests = new(); + private readonly TimeSpan _throttleInterval = TimeSpan.FromSeconds(1); + + public IdeStreamingSink(TUnitMessageBus messageBus) + { + _messageBus = messageBus; + } + + public bool IsEnabled(LogLevel level) => true; + + public void Log(LogLevel level, string message, Exception? exception, Context? context) + { + if (context is not TestContext testContext) + { + return; + } + + var testId = testContext.TestDetails.TestId; + + var state = _activeTests.GetOrAdd(testId, _ => CreateStreamingState(testContext)); + + state.MarkDirty(); + } + + public ValueTask LogAsync(LogLevel level, string message, Exception? exception, Context? context) + { + Log(level, message, exception, context); + return ValueTask.CompletedTask; + } + + private TestStreamingState CreateStreamingState(TestContext testContext) + { + var state = new TestStreamingState(testContext); + + state.Timer = new Timer( + callback: _ => OnTimerTick(testContext.TestDetails.TestId), + state: null, + dueTime: _throttleInterval, + period: _throttleInterval); + + return state; + } + + private void OnTimerTick(string testId) + { + if (!_activeTests.TryGetValue(testId, out var state)) + { + return; + } + + // Passive cleanup: if test completed, dispose and remove + if (state.TestContext.Result is not null) + { + CleanupTest(testId, state); + return; + } + + // Skip if no new output since last send + if (!state.TryConsumeAndReset()) + { + return; + } + + // Send cumulative output snapshot + _ = SendOutputUpdateAsync(state.TestContext); + } + + private async Task SendOutputUpdateAsync(TestContext testContext) + { + try + { + var output = testContext.GetStandardOutput(); + var error = testContext.GetErrorOutput(); + + if (string.IsNullOrEmpty(output) && string.IsNullOrEmpty(error)) + { + return; + } + + var testNode = CreateOutputUpdateNode(testContext, output, error); + await _messageBus.PublishOutputUpdate(testNode).ConfigureAwait(false); + } + catch + { + // Swallow exceptions to prevent disrupting test execution + } + } + + private static TestNode CreateOutputUpdateNode(TestContext testContext, string? output, string? error) + { + var properties = new List + { + InProgressTestNodeStateProperty.CachedInstance + }; + + if (!string.IsNullOrEmpty(output)) + { + properties.Add(new StandardOutputProperty(output!)); + } + + if (!string.IsNullOrEmpty(error)) + { + properties.Add(new StandardErrorProperty(error!)); + } + + return new TestNode + { + Uid = new TestNodeUid(testContext.TestDetails.TestId), + DisplayName = testContext.GetDisplayName(), + Properties = new PropertyBag(properties) + }; + } + + private void CleanupTest(string testId, TestStreamingState state) + { + state.Dispose(); + _activeTests.TryRemove(testId, out _); + } + + public async ValueTask DisposeAsync() + { + foreach (var kvp in _activeTests) + { + kvp.Value.Dispose(); + } + + _activeTests.Clear(); + + await ValueTask.CompletedTask; + } + + private sealed class TestStreamingState : IDisposable + { + private int _isDirty; + + public TestContext TestContext { get; } + public Timer? Timer { get; set; } + + public TestStreamingState(TestContext testContext) + { + TestContext = testContext; + } + + public void MarkDirty() + { + Interlocked.Exchange(ref _isDirty, 1); + } + + public bool TryConsumeAndReset() + { + return Interlocked.Exchange(ref _isDirty, 0) == 1; + } + + public void Dispose() + { + Timer?.Dispose(); + } + } +} diff --git a/TUnit.Engine/TUnitMessageBus.cs b/TUnit.Engine/TUnitMessageBus.cs index 2195b7ae06..70921436a6 100644 --- a/TUnit.Engine/TUnitMessageBus.cs +++ b/TUnit.Engine/TUnitMessageBus.cs @@ -131,6 +131,14 @@ public ValueTask SessionArtifact(Artifact artifact) )); } + public ValueTask PublishOutputUpdate(TestNode testNode) + { + return new ValueTask(context.MessageBus.PublishAsync(this, new TestNodeUpdateMessage( + sessionUid: _sessionSessionUid, + testNode: testNode + ))); + } + private static TestNodeStateProperty GetFailureStateProperty(TestContext testContext, Exception e, TimeSpan duration) { if (testContext.Metadata.TestDetails.Timeout != null From 3b322e9ccdb7723ff5ad2d84fa8cd01ad3e4372c Mon Sep 17 00:00:00 2001 From: Tom Longhurst <30480171+thomhurst@users.noreply.github.com> Date: Sun, 18 Jan 2026 00:14:56 +0000 Subject: [PATCH 03/13] chore: remove redundant platform attributes from test NuGet package modules --- TUnit.Pipeline/Modules/TestNugetPackageModule.cs | 2 -- 1 file changed, 2 deletions(-) diff --git a/TUnit.Pipeline/Modules/TestNugetPackageModule.cs b/TUnit.Pipeline/Modules/TestNugetPackageModule.cs index 537ce328fc..f748e04914 100644 --- a/TUnit.Pipeline/Modules/TestNugetPackageModule.cs +++ b/TUnit.Pipeline/Modules/TestNugetPackageModule.cs @@ -16,13 +16,11 @@ public class TestNugetPackageModule : AbstractTestNugetPackageModule public override string ProjectName => "TUnit.NugetTester.csproj"; } -[RunOnWindowsOnly, RunOnLinuxOnly] public class TestFSharpNugetPackageModule : AbstractTestNugetPackageModule { public override string ProjectName => "TUnit.NugetTester.FSharp.fsproj"; } -[RunOnWindowsOnly, RunOnLinuxOnly] public class TestVBNugetPackageModule : AbstractTestNugetPackageModule { public override string ProjectName => "TUnit.NugetTester.VB.vbproj"; From 49e3072541e3eb7866fb1bf9cd59d119bd1b8751 Mon Sep 17 00:00:00 2001 From: Tom Longhurst <30480171+thomhurst@users.noreply.github.com> Date: Sun, 18 Jan 2026 00:43:53 +0000 Subject: [PATCH 04/13] fix: add defensive error handling to IdeStreamingSink - Wrap Log() and OnTimerTick() in try-catch to prevent crashes - Add null checks for TestDetails before accessing TestId - Handle nullable TestNode from CreateOutputUpdateNode --- TUnit.Engine/Logging/IdeStreamingSink.cs | 75 ++++++++++++++++-------- 1 file changed, 52 insertions(+), 23 deletions(-) diff --git a/TUnit.Engine/Logging/IdeStreamingSink.cs b/TUnit.Engine/Logging/IdeStreamingSink.cs index cfff4a54c0..b961559f1c 100644 --- a/TUnit.Engine/Logging/IdeStreamingSink.cs +++ b/TUnit.Engine/Logging/IdeStreamingSink.cs @@ -27,16 +27,27 @@ public IdeStreamingSink(TUnitMessageBus messageBus) public void Log(LogLevel level, string message, Exception? exception, Context? context) { - if (context is not TestContext testContext) + try { - return; - } + if (context is not TestContext testContext) + { + return; + } - var testId = testContext.TestDetails.TestId; + // Only stream for tests that have started execution + if (testContext.TestDetails?.TestId is not { } testId) + { + return; + } - var state = _activeTests.GetOrAdd(testId, _ => CreateStreamingState(testContext)); + var state = _activeTests.GetOrAdd(testId, _ => CreateStreamingState(testContext)); - state.MarkDirty(); + state.MarkDirty(); + } + catch + { + // Swallow exceptions to prevent disrupting test execution + } } public ValueTask LogAsync(LogLevel level, string message, Exception? exception, Context? context) @@ -60,26 +71,33 @@ private TestStreamingState CreateStreamingState(TestContext testContext) private void OnTimerTick(string testId) { - if (!_activeTests.TryGetValue(testId, out var state)) + try { - return; - } + if (!_activeTests.TryGetValue(testId, out var state)) + { + return; + } - // Passive cleanup: if test completed, dispose and remove - if (state.TestContext.Result is not null) - { - CleanupTest(testId, state); - return; - } + // Passive cleanup: if test completed, dispose and remove + if (state.TestContext.Result is not null) + { + CleanupTest(testId, state); + return; + } - // Skip if no new output since last send - if (!state.TryConsumeAndReset()) + // Skip if no new output since last send + if (!state.TryConsumeAndReset()) + { + return; + } + + // Send cumulative output snapshot + _ = SendOutputUpdateAsync(state.TestContext); + } + catch { - return; + // Swallow exceptions to prevent crashing thread pool } - - // Send cumulative output snapshot - _ = SendOutputUpdateAsync(state.TestContext); } private async Task SendOutputUpdateAsync(TestContext testContext) @@ -95,6 +113,11 @@ private async Task SendOutputUpdateAsync(TestContext testContext) } var testNode = CreateOutputUpdateNode(testContext, output, error); + if (testNode is null) + { + return; + } + await _messageBus.PublishOutputUpdate(testNode).ConfigureAwait(false); } catch @@ -103,8 +126,14 @@ private async Task SendOutputUpdateAsync(TestContext testContext) } } - private static TestNode CreateOutputUpdateNode(TestContext testContext, string? output, string? error) + private static TestNode? CreateOutputUpdateNode(TestContext testContext, string? output, string? error) { + // Defensive: ensure TestDetails is available + if (testContext.TestDetails?.TestId is not { } testId) + { + return null; + } + var properties = new List { InProgressTestNodeStateProperty.CachedInstance @@ -122,7 +151,7 @@ private static TestNode CreateOutputUpdateNode(TestContext testContext, string? return new TestNode { - Uid = new TestNodeUid(testContext.TestDetails.TestId), + Uid = new TestNodeUid(testId), DisplayName = testContext.GetDisplayName(), Properties = new PropertyBag(properties) }; From d12e293650e9d3357dc16e6a8e8037be1564768a Mon Sep 17 00:00:00 2001 From: Tom Longhurst <30480171+thomhurst@users.noreply.github.com> Date: Sun, 18 Jan 2026 00:47:02 +0000 Subject: [PATCH 05/13] fix: only stream output for tests that have actually started Check that TestStart is set before streaming output updates. This ensures we don't interfere with test discovery. --- TUnit.Engine/Logging/IdeStreamingSink.cs | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/TUnit.Engine/Logging/IdeStreamingSink.cs b/TUnit.Engine/Logging/IdeStreamingSink.cs index b961559f1c..f392f91fb4 100644 --- a/TUnit.Engine/Logging/IdeStreamingSink.cs +++ b/TUnit.Engine/Logging/IdeStreamingSink.cs @@ -34,8 +34,9 @@ public void Log(LogLevel level, string message, Exception? exception, Context? c return; } - // Only stream for tests that have started execution - if (testContext.TestDetails?.TestId is not { } testId) + // Only stream for tests that have started execution (TestStart is set) + if (testContext.TestDetails?.TestId is not { } testId || + testContext.Execution.TestStart is null) { return; } From d4c1f3b75cc4bd9368794921e71caee3ec1e2544 Mon Sep 17 00:00:00 2001 From: Tom Longhurst <30480171+thomhurst@users.noreply.github.com> Date: Sun, 18 Jan 2026 00:49:44 +0000 Subject: [PATCH 06/13] fix: address PR review comments - Revert unrelated TestNugetPackageModule.cs changes - Fix timer disposal race by stopping timer before disposing - Add documentation about passive cleanup strategy and thread safety --- TUnit.Engine/Logging/IdeStreamingSink.cs | 14 ++++++++++++++ TUnit.Pipeline/Modules/TestNugetPackageModule.cs | 2 ++ 2 files changed, 16 insertions(+) diff --git a/TUnit.Engine/Logging/IdeStreamingSink.cs b/TUnit.Engine/Logging/IdeStreamingSink.cs index f392f91fb4..eb724d89a3 100644 --- a/TUnit.Engine/Logging/IdeStreamingSink.cs +++ b/TUnit.Engine/Logging/IdeStreamingSink.cs @@ -12,6 +12,18 @@ namespace TUnit.Engine.Logging; /// Sends cumulative output snapshots every 1 second during test execution. /// Only activated when running in an IDE environment (not console). /// +/// +/// +/// Cleanup Strategy: Uses passive cleanup - each timer tick checks if the test +/// has completed (Result is not null) and cleans up if so. This avoids the need to +/// register for test completion events while ensuring timely resource release. +/// +/// +/// Thread Safety: Uses Interlocked operations for the dirty flag and +/// ConcurrentDictionary for test state tracking. Timer callbacks are wrapped +/// in try-catch to prevent thread pool crashes. +/// +/// internal sealed class IdeStreamingSink : ILogSink, IAsyncDisposable { private readonly TUnitMessageBus _messageBus; @@ -200,6 +212,8 @@ public bool TryConsumeAndReset() public void Dispose() { + // Stop timer before disposing to prevent callback race + Timer?.Change(Timeout.Infinite, Timeout.Infinite); Timer?.Dispose(); } } diff --git a/TUnit.Pipeline/Modules/TestNugetPackageModule.cs b/TUnit.Pipeline/Modules/TestNugetPackageModule.cs index f748e04914..537ce328fc 100644 --- a/TUnit.Pipeline/Modules/TestNugetPackageModule.cs +++ b/TUnit.Pipeline/Modules/TestNugetPackageModule.cs @@ -16,11 +16,13 @@ public class TestNugetPackageModule : AbstractTestNugetPackageModule public override string ProjectName => "TUnit.NugetTester.csproj"; } +[RunOnWindowsOnly, RunOnLinuxOnly] public class TestFSharpNugetPackageModule : AbstractTestNugetPackageModule { public override string ProjectName => "TUnit.NugetTester.FSharp.fsproj"; } +[RunOnWindowsOnly, RunOnLinuxOnly] public class TestVBNugetPackageModule : AbstractTestNugetPackageModule { public override string ProjectName => "TUnit.NugetTester.VB.vbproj"; From ce68e091c25c36351102bb3ac48fbb8f227efa57 Mon Sep 17 00:00:00 2001 From: Tom Longhurst <30480171+thomhurst@users.noreply.github.com> Date: Sun, 18 Jan 2026 01:00:22 +0000 Subject: [PATCH 07/13] fix: remove output properties from streaming updates to prevent duplication IDEs like Rider concatenate StandardOutputProperty from each update rather than replacing it, causing output to appear multiple times during streaming. Changed to send only InProgressTestNodeStateProperty as a "heartbeat" to indicate the test is still running. The final test result includes the complete output without duplication. --- TUnit.Engine/Logging/IdeStreamingSink.cs | 33 +++++------------------- 1 file changed, 7 insertions(+), 26 deletions(-) diff --git a/TUnit.Engine/Logging/IdeStreamingSink.cs b/TUnit.Engine/Logging/IdeStreamingSink.cs index eb724d89a3..8504bce250 100644 --- a/TUnit.Engine/Logging/IdeStreamingSink.cs +++ b/TUnit.Engine/Logging/IdeStreamingSink.cs @@ -117,15 +117,7 @@ private async Task SendOutputUpdateAsync(TestContext testContext) { try { - var output = testContext.GetStandardOutput(); - var error = testContext.GetErrorOutput(); - - if (string.IsNullOrEmpty(output) && string.IsNullOrEmpty(error)) - { - return; - } - - var testNode = CreateOutputUpdateNode(testContext, output, error); + var testNode = CreateOutputUpdateNode(testContext); if (testNode is null) { return; @@ -139,7 +131,7 @@ private async Task SendOutputUpdateAsync(TestContext testContext) } } - private static TestNode? CreateOutputUpdateNode(TestContext testContext, string? output, string? error) + private static TestNode? CreateOutputUpdateNode(TestContext testContext) { // Defensive: ensure TestDetails is available if (testContext.TestDetails?.TestId is not { } testId) @@ -147,26 +139,15 @@ private async Task SendOutputUpdateAsync(TestContext testContext) return null; } - var properties = new List - { - InProgressTestNodeStateProperty.CachedInstance - }; - - if (!string.IsNullOrEmpty(output)) - { - properties.Add(new StandardOutputProperty(output!)); - } - - if (!string.IsNullOrEmpty(error)) - { - properties.Add(new StandardErrorProperty(error!)); - } - + // Note: We intentionally do NOT include StandardOutputProperty/StandardErrorProperty here. + // IDEs like Rider concatenate output from each update rather than replacing it. + // The final test result will include the complete output. + // This update serves as a "heartbeat" to show the test is still running. return new TestNode { Uid = new TestNodeUid(testId), DisplayName = testContext.GetDisplayName(), - Properties = new PropertyBag(properties) + Properties = new PropertyBag(InProgressTestNodeStateProperty.CachedInstance) }; } From ef2e11ce4b4ce9642ff9f8067b9b8d6ff92099fe Mon Sep 17 00:00:00 2001 From: Tom Longhurst <30480171+thomhurst@users.noreply.github.com> Date: Sun, 18 Jan 2026 01:08:16 +0000 Subject: [PATCH 08/13] fix: use delta output to prevent duplication in Rider Instead of sending cumulative output which Rider concatenates, track the last sent position and only send new content since the last update. This prevents output duplication while still showing real-time streaming. --- TUnit.Engine/Logging/IdeStreamingSink.cs | 76 ++++++++++++++++++++---- 1 file changed, 65 insertions(+), 11 deletions(-) diff --git a/TUnit.Engine/Logging/IdeStreamingSink.cs b/TUnit.Engine/Logging/IdeStreamingSink.cs index 8504bce250..01b5c82ee8 100644 --- a/TUnit.Engine/Logging/IdeStreamingSink.cs +++ b/TUnit.Engine/Logging/IdeStreamingSink.cs @@ -9,11 +9,16 @@ namespace TUnit.Engine.Logging; /// /// A log sink that streams test output in real-time to IDE test explorers. -/// Sends cumulative output snapshots every 1 second during test execution. +/// Sends delta output (new content since last update) every 1 second during test execution. /// Only activated when running in an IDE environment (not console). /// /// /// +/// Delta Streaming: Sends only new output since the last update, not cumulative. +/// IDEs like Rider concatenate output from each TestNodeUpdateMessage, so sending deltas +/// builds up the correct output. The final test result contains the complete output. +/// +/// /// Cleanup Strategy: Uses passive cleanup - each timer tick checks if the test /// has completed (Result is not null) and cleans up if so. This avoids the need to /// register for test completion events while ensuring timely resource release. @@ -104,8 +109,14 @@ private void OnTimerTick(string testId) return; } - // Send cumulative output snapshot - _ = SendOutputUpdateAsync(state.TestContext); + // Send delta output (only new content since last update) + var (outputDelta, errorDelta) = state.GetOutputDelta(); + if (outputDelta is null && errorDelta is null) + { + return; + } + + _ = SendOutputUpdateAsync(state.TestContext, outputDelta, errorDelta); } catch { @@ -113,11 +124,11 @@ private void OnTimerTick(string testId) } } - private async Task SendOutputUpdateAsync(TestContext testContext) + private async Task SendOutputUpdateAsync(TestContext testContext, string? outputDelta, string? errorDelta) { try { - var testNode = CreateOutputUpdateNode(testContext); + var testNode = CreateOutputUpdateNode(testContext, outputDelta, errorDelta); if (testNode is null) { return; @@ -131,7 +142,7 @@ private async Task SendOutputUpdateAsync(TestContext testContext) } } - private static TestNode? CreateOutputUpdateNode(TestContext testContext) + private static TestNode? CreateOutputUpdateNode(TestContext testContext, string? outputDelta, string? errorDelta) { // Defensive: ensure TestDetails is available if (testContext.TestDetails?.TestId is not { } testId) @@ -139,15 +150,29 @@ private async Task SendOutputUpdateAsync(TestContext testContext) return null; } - // Note: We intentionally do NOT include StandardOutputProperty/StandardErrorProperty here. - // IDEs like Rider concatenate output from each update rather than replacing it. - // The final test result will include the complete output. - // This update serves as a "heartbeat" to show the test is still running. + // Build properties list with delta output + // We send only new content since the last update (delta) because IDEs like Rider + // concatenate output from each update. This way the final result is correct. + var properties = new List(3) + { + InProgressTestNodeStateProperty.CachedInstance + }; + + if (!string.IsNullOrEmpty(outputDelta)) + { + properties.Add(new StandardOutputProperty(outputDelta!)); + } + + if (!string.IsNullOrEmpty(errorDelta)) + { + properties.Add(new StandardErrorProperty(errorDelta!)); + } + return new TestNode { Uid = new TestNodeUid(testId), DisplayName = testContext.GetDisplayName(), - Properties = new PropertyBag(InProgressTestNodeStateProperty.CachedInstance) + Properties = new PropertyBag(properties) }; } @@ -172,6 +197,8 @@ public async ValueTask DisposeAsync() private sealed class TestStreamingState : IDisposable { private int _isDirty; + private int _lastOutputPosition; + private int _lastErrorPosition; public TestContext TestContext { get; } public Timer? Timer { get; set; } @@ -191,6 +218,33 @@ public bool TryConsumeAndReset() return Interlocked.Exchange(ref _isDirty, 0) == 1; } + /// + /// Gets only the new output since the last call (delta). + /// IDEs like Rider append each update, so sending deltas builds up the correct output. + /// + public (string? Output, string? Error) GetOutputDelta() + { + var fullOutput = TestContext.GetStandardOutput(); + var fullError = TestContext.GetErrorOutput(); + + string? outputDelta = null; + string? errorDelta = null; + + if (!string.IsNullOrEmpty(fullOutput) && fullOutput.Length > _lastOutputPosition) + { + outputDelta = fullOutput.Substring(_lastOutputPosition); + _lastOutputPosition = fullOutput.Length; + } + + if (!string.IsNullOrEmpty(fullError) && fullError.Length > _lastErrorPosition) + { + errorDelta = fullError.Substring(_lastErrorPosition); + _lastErrorPosition = fullError.Length; + } + + return (outputDelta, errorDelta); + } + public void Dispose() { // Stop timer before disposing to prevent callback race From ead93c70747bfd4944979bd0a6a2828ed0e2e926 Mon Sep 17 00:00:00 2001 From: Tom Longhurst <30480171+thomhurst@users.noreply.github.com> Date: Sun, 18 Jan 2026 01:11:43 +0000 Subject: [PATCH 09/13] fix: use cumulative output since Rider replaces (not concatenates) Rider replaces the displayed output with each update rather than concatenating. Changed from delta to cumulative snapshots so the full output is always visible. --- TUnit.Engine/Logging/IdeStreamingSink.cs | 23 ++++++++++++----------- 1 file changed, 12 insertions(+), 11 deletions(-) diff --git a/TUnit.Engine/Logging/IdeStreamingSink.cs b/TUnit.Engine/Logging/IdeStreamingSink.cs index 01b5c82ee8..5ac4886fb3 100644 --- a/TUnit.Engine/Logging/IdeStreamingSink.cs +++ b/TUnit.Engine/Logging/IdeStreamingSink.cs @@ -9,14 +9,13 @@ namespace TUnit.Engine.Logging; /// /// A log sink that streams test output in real-time to IDE test explorers. -/// Sends delta output (new content since last update) every 1 second during test execution. +/// Sends cumulative output snapshots every 1 second during test execution. /// Only activated when running in an IDE environment (not console). /// /// /// -/// Delta Streaming: Sends only new output since the last update, not cumulative. -/// IDEs like Rider concatenate output from each TestNodeUpdateMessage, so sending deltas -/// builds up the correct output. The final test result contains the complete output. +/// Cumulative Streaming: Sends full output each update. Rider replaces the displayed +/// output with each TestNodeUpdateMessage (not concatenates), so cumulative snapshots are needed. /// /// /// Cleanup Strategy: Uses passive cleanup - each timer tick checks if the test @@ -109,14 +108,17 @@ private void OnTimerTick(string testId) return; } - // Send delta output (only new content since last update) - var (outputDelta, errorDelta) = state.GetOutputDelta(); - if (outputDelta is null && errorDelta is null) + // Send cumulative output snapshot + // Rider replaces the displayed output with each update (not concatenates) + var output = state.TestContext.GetStandardOutput(); + var error = state.TestContext.GetErrorOutput(); + + if (string.IsNullOrEmpty(output) && string.IsNullOrEmpty(error)) { return; } - _ = SendOutputUpdateAsync(state.TestContext, outputDelta, errorDelta); + _ = SendOutputUpdateAsync(state.TestContext, output, error); } catch { @@ -150,9 +152,8 @@ private async Task SendOutputUpdateAsync(TestContext testContext, string? output return null; } - // Build properties list with delta output - // We send only new content since the last update (delta) because IDEs like Rider - // concatenate output from each update. This way the final result is correct. + // Build properties list with cumulative output + // Rider replaces the displayed output with each update, so we send full snapshots. var properties = new List(3) { InProgressTestNodeStateProperty.CachedInstance From 27961681622942a06fb560abc46288ad008443aa Mon Sep 17 00:00:00 2001 From: Tom Longhurst <30480171+thomhurst@users.noreply.github.com> Date: Sun, 18 Jan 2026 01:14:07 +0000 Subject: [PATCH 10/13] fix: send heartbeat after content to prevent Rider concatenation Rider concatenates the previous update with the current update. By sending a heartbeat (no output) after each content update, the next content update concatenates with empty instead of the previous content, preventing duplication. --- TUnit.Engine/Logging/IdeStreamingSink.cs | 57 +++++++++++++++++++++--- 1 file changed, 51 insertions(+), 6 deletions(-) diff --git a/TUnit.Engine/Logging/IdeStreamingSink.cs b/TUnit.Engine/Logging/IdeStreamingSink.cs index 5ac4886fb3..d8b65085d4 100644 --- a/TUnit.Engine/Logging/IdeStreamingSink.cs +++ b/TUnit.Engine/Logging/IdeStreamingSink.cs @@ -14,8 +14,9 @@ namespace TUnit.Engine.Logging; /// /// /// -/// Cumulative Streaming: Sends full output each update. Rider replaces the displayed -/// output with each TestNodeUpdateMessage (not concatenates), so cumulative snapshots are needed. +/// Cumulative Streaming with Heartbeat: Sends full output each update, followed by a +/// heartbeat (no output). Rider concatenates the previous update with the current update, so +/// the heartbeat clears the "previous" to prevent duplication on the next content update. /// /// /// Cleanup Strategy: Uses passive cleanup - each timer tick checks if the test @@ -109,7 +110,9 @@ private void OnTimerTick(string testId) } // Send cumulative output snapshot - // Rider replaces the displayed output with each update (not concatenates) + // Rider concatenates the previous update with the current update. + // To prevent duplication, we send a heartbeat (no output) after each content update, + // so the next content update concatenates with empty = just the current content. var output = state.TestContext.GetStandardOutput(); var error = state.TestContext.GetErrorOutput(); @@ -118,7 +121,7 @@ private void OnTimerTick(string testId) return; } - _ = SendOutputUpdateAsync(state.TestContext, output, error); + _ = SendOutputUpdateWithFollowUpHeartbeatAsync(state.TestContext, output, error); } catch { @@ -126,11 +129,53 @@ private void OnTimerTick(string testId) } } - private async Task SendOutputUpdateAsync(TestContext testContext, string? outputDelta, string? errorDelta) + private async Task SendOutputUpdateWithFollowUpHeartbeatAsync(TestContext testContext, string? output, string? error) { try { - var testNode = CreateOutputUpdateNode(testContext, outputDelta, errorDelta); + var testNode = CreateOutputUpdateNode(testContext, output, error); + if (testNode is null) + { + return; + } + + // Send the content update + await _messageBus.PublishOutputUpdate(testNode).ConfigureAwait(false); + + // Send a follow-up heartbeat (no output) to clear the "previous update" + // This prevents Rider from concatenating this content with the next content update + var heartbeat = CreateHeartbeatNode(testContext); + if (heartbeat is not null) + { + await _messageBus.PublishOutputUpdate(heartbeat).ConfigureAwait(false); + } + } + catch + { + // Swallow exceptions to prevent disrupting test execution + } + } + + private static TestNode? CreateHeartbeatNode(TestContext testContext) + { + if (testContext.TestDetails?.TestId is not { } testId) + { + return null; + } + + return new TestNode + { + Uid = new TestNodeUid(testId), + DisplayName = testContext.GetDisplayName(), + Properties = new PropertyBag(InProgressTestNodeStateProperty.CachedInstance) + }; + } + + private async Task SendOutputUpdateAsync(TestContext testContext, string? output, string? error) + { + try + { + var testNode = CreateOutputUpdateNode(testContext, output, error); if (testNode is null) { return; From 83d231f0be6c643551075a0d75ed7f6416fe83cc Mon Sep 17 00:00:00 2001 From: Tom Longhurst <30480171+thomhurst@users.noreply.github.com> Date: Sun, 18 Jan 2026 01:25:39 +0000 Subject: [PATCH 11/13] fix: add atomic completion flag to prevent updates after final state Use an atomic _isCompleted flag to track when we've detected test completion. Check this flag before every send operation to ensure we never send InProgressTestNodeStateProperty after the final test state has been sent. This prevents IDEs from incorrectly showing tests as still in progress after they've completed. --- TUnit.Engine/Logging/IdeStreamingSink.cs | 47 ++++++++++++++++++++++-- 1 file changed, 44 insertions(+), 3 deletions(-) diff --git a/TUnit.Engine/Logging/IdeStreamingSink.cs b/TUnit.Engine/Logging/IdeStreamingSink.cs index d8b65085d4..ed88ea8d01 100644 --- a/TUnit.Engine/Logging/IdeStreamingSink.cs +++ b/TUnit.Engine/Logging/IdeStreamingSink.cs @@ -96,8 +96,17 @@ private void OnTimerTick(string testId) return; } - // Passive cleanup: if test completed, dispose and remove + // Passive cleanup: if test completed, mark as completed and cleanup + // The atomic flag ensures we never send updates after detecting completion if (state.TestContext.Result is not null) + { + state.TryMarkCompleted(); + CleanupTest(testId, state); + return; + } + + // Double-check: if already marked completed by another path, don't proceed + if (state.IsCompleted) { CleanupTest(testId, state); return; @@ -121,7 +130,7 @@ private void OnTimerTick(string testId) return; } - _ = SendOutputUpdateWithFollowUpHeartbeatAsync(state.TestContext, output, error); + _ = SendOutputUpdateWithFollowUpHeartbeatAsync(state, output, error); } catch { @@ -129,10 +138,19 @@ private void OnTimerTick(string testId) } } - private async Task SendOutputUpdateWithFollowUpHeartbeatAsync(TestContext testContext, string? output, string? error) + private async Task SendOutputUpdateWithFollowUpHeartbeatAsync(TestStreamingState state, string? output, string? error) { try { + var testContext = state.TestContext; + + // Don't send if test already completed - final state has been sent + if (state.IsCompleted || testContext.Result is not null) + { + state.TryMarkCompleted(); + return; + } + var testNode = CreateOutputUpdateNode(testContext, output, error); if (testNode is null) { @@ -144,6 +162,14 @@ private async Task SendOutputUpdateWithFollowUpHeartbeatAsync(TestContext testCo // Send a follow-up heartbeat (no output) to clear the "previous update" // This prevents Rider from concatenating this content with the next content update + // CRITICAL: Check again that test hasn't completed - we must never send + // InProgressTestNodeStateProperty after the final state has been sent + if (state.IsCompleted || testContext.Result is not null) + { + state.TryMarkCompleted(); + return; + } + var heartbeat = CreateHeartbeatNode(testContext); if (heartbeat is not null) { @@ -243,6 +269,7 @@ public async ValueTask DisposeAsync() private sealed class TestStreamingState : IDisposable { private int _isDirty; + private int _isCompleted; // Set to 1 once we detect test completion - never send after this private int _lastOutputPosition; private int _lastErrorPosition; @@ -264,6 +291,20 @@ public bool TryConsumeAndReset() return Interlocked.Exchange(ref _isDirty, 0) == 1; } + /// + /// Atomically marks this test as completed. Once marked, no more updates will be sent. + /// + /// True if this call marked completion (first caller), false if already marked. + public bool TryMarkCompleted() + { + return Interlocked.Exchange(ref _isCompleted, 1) == 0; + } + + /// + /// Returns true if this test has been marked as completed. + /// + public bool IsCompleted => Interlocked.CompareExchange(ref _isCompleted, 0, 0) == 1; + /// /// Gets only the new output since the last call (delta). /// IDEs like Rider append each update, so sending deltas builds up the correct output. From c2f204a2b196a838e803aa57a6b717389b3c4126 Mon Sep 17 00:00:00 2001 From: Tom Longhurst <30480171+thomhurst@users.noreply.github.com> Date: Sun, 18 Jan 2026 01:27:24 +0000 Subject: [PATCH 12/13] chore: remove unused delta streaming code Remove GetOutputDelta(), SendOutputUpdateAsync(), and related fields that were left over from the delta streaming approach. The final implementation uses cumulative snapshots + heartbeat instead. --- TUnit.Engine/Logging/IdeStreamingSink.cs | 57 +++--------------------- 1 file changed, 5 insertions(+), 52 deletions(-) diff --git a/TUnit.Engine/Logging/IdeStreamingSink.cs b/TUnit.Engine/Logging/IdeStreamingSink.cs index ed88ea8d01..7309ec9db3 100644 --- a/TUnit.Engine/Logging/IdeStreamingSink.cs +++ b/TUnit.Engine/Logging/IdeStreamingSink.cs @@ -197,25 +197,7 @@ private async Task SendOutputUpdateWithFollowUpHeartbeatAsync(TestStreamingState }; } - private async Task SendOutputUpdateAsync(TestContext testContext, string? output, string? error) - { - try - { - var testNode = CreateOutputUpdateNode(testContext, output, error); - if (testNode is null) - { - return; - } - - await _messageBus.PublishOutputUpdate(testNode).ConfigureAwait(false); - } - catch - { - // Swallow exceptions to prevent disrupting test execution - } - } - - private static TestNode? CreateOutputUpdateNode(TestContext testContext, string? outputDelta, string? errorDelta) + private static TestNode? CreateOutputUpdateNode(TestContext testContext, string? output, string? error) { // Defensive: ensure TestDetails is available if (testContext.TestDetails?.TestId is not { } testId) @@ -230,14 +212,14 @@ private async Task SendOutputUpdateAsync(TestContext testContext, string? output InProgressTestNodeStateProperty.CachedInstance }; - if (!string.IsNullOrEmpty(outputDelta)) + if (!string.IsNullOrEmpty(output)) { - properties.Add(new StandardOutputProperty(outputDelta!)); + properties.Add(new StandardOutputProperty(output!)); } - if (!string.IsNullOrEmpty(errorDelta)) + if (!string.IsNullOrEmpty(error)) { - properties.Add(new StandardErrorProperty(errorDelta!)); + properties.Add(new StandardErrorProperty(error!)); } return new TestNode @@ -270,8 +252,6 @@ private sealed class TestStreamingState : IDisposable { private int _isDirty; private int _isCompleted; // Set to 1 once we detect test completion - never send after this - private int _lastOutputPosition; - private int _lastErrorPosition; public TestContext TestContext { get; } public Timer? Timer { get; set; } @@ -305,33 +285,6 @@ public bool TryMarkCompleted() /// public bool IsCompleted => Interlocked.CompareExchange(ref _isCompleted, 0, 0) == 1; - /// - /// Gets only the new output since the last call (delta). - /// IDEs like Rider append each update, so sending deltas builds up the correct output. - /// - public (string? Output, string? Error) GetOutputDelta() - { - var fullOutput = TestContext.GetStandardOutput(); - var fullError = TestContext.GetErrorOutput(); - - string? outputDelta = null; - string? errorDelta = null; - - if (!string.IsNullOrEmpty(fullOutput) && fullOutput.Length > _lastOutputPosition) - { - outputDelta = fullOutput.Substring(_lastOutputPosition); - _lastOutputPosition = fullOutput.Length; - } - - if (!string.IsNullOrEmpty(fullError) && fullError.Length > _lastErrorPosition) - { - errorDelta = fullError.Substring(_lastErrorPosition); - _lastErrorPosition = fullError.Length; - } - - return (outputDelta, errorDelta); - } - public void Dispose() { // Stop timer before disposing to prevent callback race From 974630c07909e14223de64afa6d32901dbfe99a7 Mon Sep 17 00:00:00 2001 From: Tom Longhurst <30480171+thomhurst@users.noreply.github.com> Date: Sun, 18 Jan 2026 01:36:39 +0000 Subject: [PATCH 13/13] chore: ignore docs/plans folder --- .gitignore | 2 +- ...-01-11-lazy-test-materialization-design.md | 362 ------- ...-01-14-source-generator-overhaul-design.md | 324 ------ ...5-generic-type-source-generation-design.md | 312 ------ ...6-01-16-generic-method-discovery-design.md | 237 ----- docs/plans/2026-01-17-log-streaming-design.md | 407 ------- ...2026-01-17-log-streaming-implementation.md | 999 ------------------ .../2026-01-18-ide-streaming-sink-design.md | 130 --- 8 files changed, 1 insertion(+), 2772 deletions(-) delete mode 100644 docs/plans/2026-01-11-lazy-test-materialization-design.md delete mode 100644 docs/plans/2026-01-14-source-generator-overhaul-design.md delete mode 100644 docs/plans/2026-01-15-generic-type-source-generation-design.md delete mode 100644 docs/plans/2026-01-16-generic-method-discovery-design.md delete mode 100644 docs/plans/2026-01-17-log-streaming-design.md delete mode 100644 docs/plans/2026-01-17-log-streaming-implementation.md delete mode 100644 docs/plans/2026-01-18-ide-streaming-sink-design.md diff --git a/.gitignore b/.gitignore index 01a7753946..efba1d4528 100644 --- a/.gitignore +++ b/.gitignore @@ -429,7 +429,7 @@ nul .claude/agents # Documentation plans -doc/plans/ +docs/plans/ # Speedscope profiling files *speedscope*.json diff --git a/docs/plans/2026-01-11-lazy-test-materialization-design.md b/docs/plans/2026-01-11-lazy-test-materialization-design.md deleted file mode 100644 index 715c972b94..0000000000 --- a/docs/plans/2026-01-11-lazy-test-materialization-design.md +++ /dev/null @@ -1,362 +0,0 @@ -# Lazy Test Materialization Design - -## Problem Statement - -TUnit's test discovery is ~9% slower than MSTest for single/few test scenarios. Profiling reveals the bottleneck is **eager materialization**: every test creates a full `TestMetadata` object during discovery, even tests that won't run due to filtering. - -Current pipeline: -``` -Source Gen → Full TestMetadata (20+ properties, delegates) → Filter → Build → Execute - ↑ EXPENSIVE ↑ Most tests discarded -``` - -Proposed pipeline: -``` -Source Gen → Lightweight Descriptor → Filter → Lazy Materialize → Build → Execute - ↑ CHEAP ↑ Only matching tests -``` - -## Current Architecture - -### TestMetadata (Heavyweight) - -```csharp -public abstract class TestMetadata -{ - // Identity (needed for filtering) - public required string TestName { get; init; } - public required Type TestClassType { get; init; } - public required string TestMethodName { get; init; } - - // Location (needed for display) - public required string FilePath { get; init; } - public required int LineNumber { get; init; } - - // Expensive (delegates, reflection, allocations) - public Func InstanceFactory { get; init; } - public Func? TestInvoker { get; init; } - public required Func AttributeFactory { get; init; } - public required IDataSourceAttribute[] DataSources { get; init; } - public required IDataSourceAttribute[] ClassDataSources { get; init; } - public required PropertyDataSource[] PropertyDataSources { get; init; } - public required MethodMetadata MethodMetadata { get; init; } - // ... 15+ more properties -} -``` - -**Problem**: All 20+ properties are populated during discovery, including: -- Delegates that capture closures (allocations) -- Arrays that are never used if test doesn't match filter -- Attribute factories that instantiate attributes - -### ITestSource Interface - -```csharp -public interface ITestSource -{ - IAsyncEnumerable GetTestsAsync(string testSessionId, CancellationToken ct); -} -``` - -**Problem**: Returns full `TestMetadata`, forcing eager materialization. - -## Proposed Architecture - -### Phase 1: Lightweight TestDescriptor - -```csharp -/// -/// Minimal test identity for fast enumeration and filtering. -/// No allocations beyond the struct itself. -/// -public readonly struct TestDescriptor -{ - // Identity (for filtering) - all value types or interned strings - public required string TestId { get; init; } - public required string ClassName { get; init; } - public required string MethodName { get; init; } - public required string FullyQualifiedName { get; init; } - - // Location (for display) - public required string FilePath { get; init; } - public required int LineNumber { get; init; } - - // Filter hints (pre-computed during source gen) - public required string[] Categories { get; init; } // [Category("X")] values - public required string[] Traits { get; init; } // Trait key=value pairs - public required bool HasDataSource { get; init; } // Quick check for parameterized - public required int RepeatCount { get; init; } // Pre-extracted - - // Lazy materialization - public required Func Materializer { get; init; } -} -``` - -**Key properties**: -- Struct (stack allocated) -- Only filter-relevant data -- Pre-computed filter hints (source gen does the work) -- Single `Materializer` delegate defers expensive work - -### Phase 2: Two-Phase Interface - -```csharp -/// -/// Fast test enumeration for filtering. -/// -public interface ITestDescriptorSource -{ - IEnumerable EnumerateTests(); -} - -/// -/// Full test source (backward compatible). -/// -public interface ITestSource : ITestDescriptorSource -{ - IAsyncEnumerable GetTestsAsync(string testSessionId, CancellationToken ct); -} -``` - -### Phase 3: Source Generator Changes - -Current generated code (simplified): -```csharp -public class MyTestClass_Tests : ITestSource -{ - public async IAsyncEnumerable GetTestsAsync(string sessionId, ...) - { - yield return new SourceGeneratedTestMetadata - { - TestName = "MyTest", - TestClassType = typeof(MyTestClass), - InstanceFactory = (types, args) => new MyTestClass(), - TestInvoker = (instance, args) => ((MyTestClass)instance).MyTest(), - AttributeFactory = () => new[] { new TestAttribute() }, - DataSources = new[] { ... }, - // ... 15+ more expensive properties - }; - } -} -``` - -Proposed generated code: -```csharp -public class MyTestClass_Tests : ITestSource -{ - // Pre-computed at compile time (static readonly) - private static readonly TestDescriptor _descriptor = new() - { - TestId = "MyTestClass.MyTest", - ClassName = "MyTestClass", - MethodName = "MyTest", - FullyQualifiedName = "MyNamespace.MyTestClass.MyTest", - FilePath = "MyTestClass.cs", - LineNumber = 42, - Categories = new[] { "Unit" }, // Pre-extracted - Traits = Array.Empty(), - HasDataSource = false, - RepeatCount = 0, - Materializer = MaterializeTest - }; - - // Fast path: Just return pre-computed descriptor - public IEnumerable EnumerateTests() - { - yield return _descriptor; - } - - // Slow path: Full materialization (only called for matching tests) - private static TestMetadata MaterializeTest(string sessionId) - { - return new SourceGeneratedTestMetadata - { - // ... full properties - }; - } - - // Backward compatible - public async IAsyncEnumerable GetTestsAsync(string sessionId, ...) - { - yield return MaterializeTest(sessionId); - } -} -``` - -### Phase 4: Pipeline Changes - -```csharp -internal sealed class TestBuilderPipeline -{ - public async Task> BuildTestsAsync( - string testSessionId, - ITestExecutionFilter? filter = null) - { - // Phase 1: Fast enumeration - var descriptors = _dataCollector.EnumerateDescriptors(); - - // Phase 2: Filter (no materialization yet) - var matchingDescriptors = filter != null - ? descriptors.Where(d => FilterMatches(d, filter)) - : descriptors; - - // Phase 3: Lazy materialization (only matching tests) - var metadata = matchingDescriptors - .Select(d => d.Materializer(testSessionId)); - - // Phase 4: Build executable tests - return await BuildTestsFromMetadataAsync(metadata); - } - - private static bool FilterMatches(TestDescriptor d, ITestExecutionFilter filter) - { - // Fast filter check using pre-computed hints - // No attribute instantiation, no reflection - return filter.Matches(d.FullyQualifiedName, d.Categories, d.Traits); - } -} -``` - -## Data Source Deferral (Advanced) - -For parameterized tests, data sources can be deferred even further: - -```csharp -public readonly struct TestDescriptor -{ - // For parameterized tests, descriptor represents the "template" - // Each data row becomes a separate test during materialization - public required bool HasDataSource { get; init; } - public required int EstimatedDataRowCount { get; init; } // Hint for capacity -} -``` - -During materialization: -```csharp -private static IEnumerable MaterializeTest(string sessionId) -{ - // Data source evaluation happens here, after filtering - foreach (var dataRow in GetDataSource()) - { - yield return new SourceGeneratedTestMetadata - { - // ... properties with dataRow values - }; - } -} -``` - -## Implementation Plan - -### Step 1: Add TestDescriptor (Non-Breaking) - -1. Create `TUnit.Core/TestDescriptor.cs` -2. Create `TUnit.Core/Interfaces/SourceGenerator/ITestDescriptorSource.cs` -3. Make `ITestSource` extend `ITestDescriptorSource` with default implementation -4. Add unit tests - -**Estimated scope**: 2 new files, 0 breaking changes - -### Step 2: Update Source Generator - -1. Modify `TestMetadataGenerator.cs` to generate: - - Static `TestDescriptor` field with pre-computed values - - `EnumerateTests()` method returning descriptors - - `MaterializeTest()` factory method -2. Extract filter hints at compile time (categories, traits) -3. Update snapshot tests - -**Estimated scope**: ~300 lines changed in source generator - -### Step 3: Update Pipeline - -1. Add `ITestDataCollector.EnumerateDescriptors()` method -2. Update `TestBuilderPipeline` to use two-phase approach -3. Implement fast filter matching against descriptors -4. Add fallback to full materialization for complex filters - -**Estimated scope**: ~150 lines in pipeline - -### Step 4: Optimize Reflection Mode - -1. Update `ReflectionTestDataCollector` to support descriptors -2. Cache descriptor data per-type (not per-test) -3. Implement lazy materialization for reflection mode - -**Estimated scope**: ~200 lines - -### Step 5: Benchmarks and Validation - -1. Run speed-comparison benchmarks -2. Target: Match or beat MSTest for single test execution -3. Validate AOT compatibility -4. Run full test suite - -## Performance Expectations - -| Scenario | Current | Expected | Improvement | -|----------|---------|----------|-------------| -| Single test (no filter) | 596ms | ~530ms | ~11% | -| Single test (with filter) | 596ms | ~480ms | ~20% | -| 1000 tests, run 10 | N/A | -30% time | Significant | -| Full suite | baseline | ~same | No regression | - -Key wins: -1. **No delegate allocation** during enumeration (major GC improvement) -2. **No attribute instantiation** until materialization -3. **Pre-computed filter hints** avoid runtime reflection -4. **Only materialize tests that will run** - -## Risks and Mitigations - -### Risk: Breaking change for custom test sources - -**Mitigation**: `ITestDescriptorSource` has default implementation that delegates to `GetTestsAsync()`. Existing sources continue to work, just without optimization. - -### Risk: Source generator complexity - -**Mitigation**: Implement incrementally. Phase 1 just adds descriptor alongside existing code. Only remove old code after validation. - -### Risk: Filter hint extraction misses edge cases - -**Mitigation**: Complex filters fall back to full materialization. Fast path is optimization, not requirement. - -### Risk: Memory overhead of descriptor + metadata - -**Mitigation**: Descriptor is struct (stack allocated). Materializer delegate is shared (static method). Net memory should decrease. - -## Alternatives Considered - -### Alternative 1: Lazy property initialization - -Instead of separate descriptor, make `TestMetadata` properties lazy. - -**Rejected**: Still allocates the object, still creates delegate captures. Doesn't solve GC pressure. - -### Alternative 2: Compiled filter expressions - -Generate filter-specific code at compile time. - -**Rejected**: Too complex, doesn't handle runtime filters (VS Test Explorer). - -### Alternative 3: Just optimize hot paths - -Continue with micro-optimizations in existing architecture. - -**Rejected**: Diminishing returns. Already applied sequential processing optimization. Fundamental architecture limits further gains. - -## Success Criteria - -1. Single test execution time <= MSTest (currently MSTest ~553ms, TUnit ~540ms after PR #4299) -2. No performance regression for full test suite -3. All existing tests pass -4. AOT/trimming compatibility maintained -5. Backward compatible with custom `ITestSource` implementations - -## Next Steps - -1. Review and approve this design -2. Create feature branch: `feature/lazy-test-materialization` -3. Implement Step 1 (TestDescriptor) -4. Iterate through remaining steps -5. Performance validation at each step diff --git a/docs/plans/2026-01-14-source-generator-overhaul-design.md b/docs/plans/2026-01-14-source-generator-overhaul-design.md deleted file mode 100644 index f6a7ad4de5..0000000000 --- a/docs/plans/2026-01-14-source-generator-overhaul-design.md +++ /dev/null @@ -1,324 +0,0 @@ -# Source Generator Overhaul Design - -**Date:** 2026-01-14 -**Status:** Approved -**Problem:** Build times increased from ~2s to ~20s (11x regression) - -## Root Cause Analysis - -### Primary Issue: Storing Roslyn Symbols in Models - -Multiple generators store `ISymbol`, `SyntaxNode`, `SemanticModel`, and `GeneratorAttributeSyntaxContext` in models that flow through the incremental pipeline. These types cannot be properly cached by Roslyn because they reference the `Compilation` object which changes on every keystroke. - -**Affected Models:** - -| Model | Problematic Fields | -|-------|-------------------| -| `TestMethodMetadata` | `IMethodSymbol`, `INamedTypeSymbol`, `MethodDeclarationSyntax`, `GeneratorAttributeSyntaxContext`, `AttributeData[]` | -| `HooksDataModel` | `GeneratorAttributeSyntaxContext`, `IMethodSymbol`, `INamedTypeSymbol` | -| `InheritsTestsClassMetadata` | `INamedTypeSymbol`, `ClassDeclarationSyntax`, `GeneratorAttributeSyntaxContext` | -| `PropertyInjectionContext` | `INamedTypeSymbol` | -| `PropertyWithDataSource` | `IPropertySymbol`, `AttributeData` | - -### Secondary Issues - -1. **Broad Syntax Predicates:** `PropertyInjectionSourceGenerator` and `StaticPropertyInitializationGenerator` use `CreateSyntaxProvider` matching ALL classes instead of targeting specific attributes. - -2. **Full Compilation Scanning:** `AotConverterGenerator` iterates through ALL syntax trees on every compilation change. - -3. **Non-Deterministic Output:** Three generators use `Guid.NewGuid()` in filenames or class names, preventing caching. - -## Solution: The "Extracted Data" Pattern - -### Core Principle - -All symbol analysis happens in the `transform` function. Models contain ONLY: -- Primitive types (`string`, `int`, `bool`, `enum`) -- Arrays/collections of primitives -- Other "extracted data" models -- **NEVER** `ISymbol`, `SyntaxNode`, `SemanticModel`, `Compilation`, or `GeneratorAttributeSyntaxContext` - -### Pipeline Pattern - -``` -ForAttributeWithMetadataName("Attribute.Name") - ↓ -Transform: Extract ALL data as primitives - ↓ -Combine with enabledProvider - ↓ -RegisterSourceOutput: Generate code using only primitives -``` - -## Proposed Architecture - -### Generator Count: 9 → 5 - -| Generator | Responsibility | -|-----------|---------------| -| `TestMetadataGenerator` | Test discovery, registration, AND AOT converters | -| `HookMetadataGenerator` | All hook types (Before/After × Each/Every × Assembly/Class/Test) | -| `PropertyDataSourceGenerator` | Instance + static property injection (unified) | -| `DynamicTestsGenerator` | Runtime-generated tests via [DynamicTestSource] | -| `InfrastructureGenerator` | Module initializer setup + assembly loading | - -### Moved to TUnit.Analyzers - -| Analyzer | Responsibility | -|----------|---------------| -| `LanguageVersionAnalyzer` | Reports error if C# < 12 | - -## New Model Definitions - -### TestMethodModel - -```csharp -public sealed class TestMethodModel : IEquatable -{ - // Type identity - public required string FullyQualifiedTypeName { get; init; } - public required string MinimalTypeName { get; init; } - public required string Namespace { get; init; } - public required string AssemblyName { get; init; } - - // Method identity - public required string MethodName { get; init; } - public required string FilePath { get; init; } - public required int LineNumber { get; init; } - - // Generics (extracted as strings) - public required bool IsGenericType { get; init; } - public required bool IsGenericMethod { get; init; } - public required EquatableArray TypeParameters { get; init; } - public required EquatableArray MethodTypeParameters { get; init; } - public required EquatableArray TypeConstraints { get; init; } - - // Method signature - public required string ReturnType { get; init; } - public required EquatableArray Parameters { get; init; } - - // Attributes (fully extracted) - public required EquatableArray Attributes { get; init; } - - // Data sources - public required EquatableArray DataSources { get; init; } - - // AOT converters (integrated) - public required EquatableArray TypesNeedingConverters { get; init; } - - // Inheritance - public required int InheritanceDepth { get; init; } -} -``` - -### HookModel - -```csharp -public sealed class HookModel : IEquatable -{ - // Identity - public required string FullyQualifiedTypeName { get; init; } - public required string MinimalTypeName { get; init; } - public required string MethodName { get; init; } - public required string FilePath { get; init; } - public required int LineNumber { get; init; } - - // Hook configuration - public required HookLevel Level { get; init; } - public required HookType Type { get; init; } - public required HookTiming Timing { get; init; } - public required int Order { get; init; } - public required string? HookExecutorTypeName { get; init; } - - // Method info - public required bool IsStatic { get; init; } - public required bool IsAsync { get; init; } - public required bool ReturnsVoid { get; init; } - public required EquatableArray ParameterTypes { get; init; } - - // Class info - public required bool ClassIsStatic { get; init; } - public required EquatableArray ClassTypeParameters { get; init; } -} - -public enum HookLevel { Assembly, Class, Test } -public enum HookType { Before, After } -public enum HookTiming { Each, Every } -``` - -### PropertyDataModel - -```csharp -public sealed class PropertyDataModel : IEquatable -{ - // Property identity - public required string PropertyName { get; init; } - public required string PropertyTypeName { get; init; } - public required string ContainingTypeName { get; init; } - public required string MinimalContainingTypeName { get; init; } - public required string Namespace { get; init; } - - // Property characteristics - public required bool IsStatic { get; init; } - public required bool HasPublicSetter { get; init; } - - // Data source (extracted) - public required DataSourceModel DataSource { get; init; } -} -``` - -### Supporting Models - -```csharp -public sealed class ExtractedAttribute : IEquatable -{ - public required string FullyQualifiedName { get; init; } - public required EquatableArray ConstructorArguments { get; init; } - public required EquatableArray NamedArguments { get; init; } -} - -public sealed class TypedConstantModel : IEquatable -{ - public required string TypeName { get; init; } - public required string? Value { get; init; } - public required TypedConstantKind Kind { get; init; } - public required EquatableArray? ArrayValues { get; init; } -} - -public sealed class NamedArgumentModel : IEquatable -{ - public required string Name { get; init; } - public required TypedConstantModel Value { get; init; } -} - -public sealed class ParameterModel : IEquatable -{ - public required string Name { get; init; } - public required string TypeName { get; init; } - public required bool HasDefaultValue { get; init; } - public required string? DefaultValue { get; init; } -} - -public sealed class DataSourceModel : IEquatable -{ - public required DataSourceKind Kind { get; init; } - public required string? MethodName { get; init; } - public required string? ContainingTypeName { get; init; } - public required EquatableArray Arguments { get; init; } - public required ExtractedAttribute SourceAttribute { get; init; } -} -``` - -### EquatableArray Utility - -```csharp -public readonly struct EquatableArray : IEquatable>, IEnumerable - where T : IEquatable -{ - private readonly T[] _array; - - public EquatableArray(T[] array) => _array = array ?? Array.Empty(); - - public bool Equals(EquatableArray other) - { - if (_array.Length != other._array.Length) - return false; - - for (int i = 0; i < _array.Length; i++) - { - if (!_array[i].Equals(other._array[i])) - return false; - } - return true; - } - - public override int GetHashCode() - { - unchecked - { - int hash = 17; - foreach (var item in _array) - hash = hash * 31 + item.GetHashCode(); - return hash; - } - } - - // IEnumerable implementation... -} -``` - -## Property Data Source Handling - -Since users can create custom data source attributes, `ForAttributeWithMetadataName` cannot be used. Instead: - -```csharp -// Target properties with any attribute -predicate: static (s, _) => s is PropertyDeclarationSyntax { AttributeLists.Count: > 0 } - -// Then check inheritance in transform -static bool IsDataSourceAttribute(INamedTypeSymbol? attrType) -{ - while (attrType != null) - { - var name = attrType.ToDisplayString(); - if (name is "TUnit.Core.DataSourceGeneratorAttribute" - or "TUnit.Core.ArgumentsAttribute" - or "TUnit.Core.MethodDataSourceAttribute" - /* etc */) - return true; - attrType = attrType.BaseType; - } - return false; -} -``` - -## InfrastructureGenerator (Consolidated) - -Combines `DisableReflectionScannerGenerator` and `AssemblyLoaderGenerator`: - -```csharp -file static class TUnitInfrastructure -{ - [ModuleInitializer] - public static void Initialize() - { - global::TUnit.Core.SourceRegistrar.IsEnabled = true; - - // Assembly loading - global::TUnit.Core.SourceRegistrar.RegisterAssembly(() => - global::System.Reflection.Assembly.Load("AssemblyName, Version=...")); - // ... - } -} -``` - -Key improvements: -- Single deterministic output file -- No GUIDs (uses `file` keyword for collision avoidance) - -## Implementation Plan - -| Phase | Task | Risk | Impact | -|-------|------|------|--------| -| 1 | Create `EquatableArray` and primitive model infrastructure | Low | Foundation | -| 2 | Fix `TestMetadataGenerator` - largest impact on build times | Medium | High | -| 3 | Fix `HookMetadataGenerator` | Medium | Medium | -| 4 | Unify and fix `PropertyDataSourceGenerator` | Medium | Medium | -| 5 | Fix `DynamicTestsGenerator` (remove GUID) | Low | Low | -| 6 | Create `InfrastructureGenerator` (consolidate utilities) | Low | Low | -| 7 | Move `LanguageVersionCheck` to Analyzers | Low | Low | -| 8 | Delete old generators and models | Low | Cleanup | - -## Expected Outcomes - -- Build times return to ~2-3 seconds range -- Incremental compilation works correctly (typing doesn't trigger full regeneration) -- Cleaner, more maintainable generator codebase -- Reduced generator count (9 → 5) - -## Testing Strategy - -1. Run existing snapshot tests after each phase -2. Benchmark build times after Phase 2 (TestMetadataGenerator) -3. Verify incremental compilation with IDE typing tests -4. Full test suite must pass before merging diff --git a/docs/plans/2026-01-15-generic-type-source-generation-design.md b/docs/plans/2026-01-15-generic-type-source-generation-design.md deleted file mode 100644 index d7e1777a29..0000000000 --- a/docs/plans/2026-01-15-generic-type-source-generation-design.md +++ /dev/null @@ -1,312 +0,0 @@ -# Generic Type Source Generation Design - -**Date:** 2026-01-15 -**Status:** Proposed -**Issue:** #4431 -**PR:** #4434 - -## Problem Statement - -The `PropertyInjectionSourceGenerator` currently skips generic types entirely: - -```csharp -// PropertyInjectionSourceGenerator.cs lines 103-105 -if (containingType.IsUnboundGenericType || containingType.TypeParameters.Length > 0) - return null; -``` - -This means generic types like `CustomWebApplicationFactory` never get source-generated metadata for: -- Properties with `IDataSourceAttribute` (e.g., `[ClassDataSource]`) -- Nested `IAsyncInitializer` properties - -**Impact:** -- In non-AOT scenarios, the reflection fallback works but is suboptimal -- In AOT scenarios, this is completely broken - no metadata means no initialization - -## Solution Overview - -Generate source metadata for **concrete instantiations** of generic types discovered at compile time, while keeping the reflection fallback for runtime-only types. - -### Discovery Sources - -1. **Inheritance chains** - `class MyTests : GenericBase` -2. **`IDataSourceAttribute` type arguments** - `[SomeDataSource>]` -3. **Base type arguments** - Walking up inheritance hierarchies - -### Key Principle - -Once we discover a concrete type like `CustomWebApplicationFactory`, we treat it identically to a non-generic type for code generation. - -## Architecture - -### Current State - -``` -PropertyInjectionSourceGenerator -├── Pipeline 1: Property Data Sources (non-generic types only) -└── Pipeline 2: IAsyncInitializer Properties (non-generic types only) -``` - -### Proposed State - -``` -PropertyInjectionSourceGenerator -├── Pipeline 1: Property Data Sources (non-generic types) -├── Pipeline 2: IAsyncInitializer Properties (non-generic types) -├── Pipeline 3: Concrete Generic Type Discovery -│ └── Scans compilation for all concrete instantiations -├── Pipeline 4: Generic Property Data Sources -│ └── Generates PropertySource for concrete generic types -└── Pipeline 5: Generic IAsyncInitializer Properties - └── Generates InitializerPropertyRegistry for concrete generic types -``` - -## Detailed Design - -### Pipeline 3: Concrete Generic Type Discovery - -**Model:** - -```csharp -record ConcreteGenericTypeModel -{ - INamedTypeSymbol ConcreteType { get; } // e.g., CustomWebApplicationFactory - INamedTypeSymbol GenericDefinition { get; } // e.g., CustomWebApplicationFactory<> - string SafeTypeName { get; } // For file naming -} -``` - -**Discovery Implementation:** - -```csharp -var concreteGenericTypes = context.SyntaxProvider - .CreateSyntaxProvider( - predicate: static (node, _) => node is TypeDeclarationSyntax or PropertyDeclarationSyntax, - transform: static (ctx, _) => ExtractConcreteGenericTypes(ctx)) - .Where(static x => x.Length > 0) - .SelectMany(static (x, _) => x) - .Collect() - .Select(static (types, _) => types.Distinct(SymbolEqualityComparer.Default)); -``` - -**Discovery from Inheritance:** - -```csharp -private static IEnumerable DiscoverFromInheritance(INamedTypeSymbol type) -{ - var baseType = type.BaseType; - while (baseType != null && baseType.SpecialType != SpecialType.System_Object) - { - if (baseType.IsGenericType && !baseType.IsUnboundGenericType) - { - yield return baseType; // Concrete generic like GenericBase - } - baseType = baseType.BaseType; - } -} -``` - -**Discovery from IDataSourceAttribute:** - -```csharp -private static IEnumerable DiscoverFromAttributes( - IPropertySymbol property, - INamedTypeSymbol dataSourceInterface) -{ - foreach (var attr in property.GetAttributes()) - { - if (attr.AttributeClass?.AllInterfaces.Contains(dataSourceInterface) != true) - continue; - - // Check attribute type arguments - if (attr.AttributeClass is { IsGenericType: true, TypeArguments.Length: > 0 }) - { - foreach (var typeArg in attr.AttributeClass.TypeArguments) - { - if (typeArg is INamedTypeSymbol { IsGenericType: true, IsUnboundGenericType: false } concreteGeneric) - { - yield return concreteGeneric; - } - } - } - } -} -``` - -### Pipeline 4 & 5: Generation for Concrete Generic Types - -Reuses the same generation logic as Pipelines 1 & 2, just with concrete generic types. - -**Example Generated Output:** - -```csharp -// For CustomWebApplicationFactory -internal static class CustomWebApplicationFactory_Program_PropertyInjectionInitializer -{ - [ModuleInitializer] - public static void Initialize() - { - PropertySourceRegistry.Register( - typeof(CustomWebApplicationFactory), - new CustomWebApplicationFactory_Program_PropertySource()); - } -} - -internal sealed class CustomWebApplicationFactory_Program_PropertySource : IPropertySource -{ - public Type Type => typeof(CustomWebApplicationFactory); - public bool ShouldInitialize => true; - - public IEnumerable GetPropertyMetadata() - { - yield return new PropertyInjectionMetadata - { - PropertyName = "Postgres", - PropertyType = typeof(InMemoryPostgres), - ContainingType = typeof(CustomWebApplicationFactory), - CreateDataSource = () => new ClassDataSourceAttribute - { - Shared = SharedType.PerTestSession - }, - SetProperty = (instance, value) => - ((CustomWebApplicationFactory)instance).Postgres = (InMemoryPostgres)value - }; - } -} -``` - -### Deduplication - -The same concrete type might be discovered from multiple sources. Deduplication uses `SymbolEqualityComparer.Default` on the collected types before generation. - -**Safe File Naming:** - -```csharp -private static string GetSafeTypeName(INamedTypeSymbol concreteType) -{ - var fullName = concreteType.ToDisplayString(SymbolDisplayFormat.FullyQualifiedFormat); - return fullName - .Replace("global::", "") - .Replace("<", "_") - .Replace(">", "") - .Replace(",", "_") - .Replace(".", "_") - .Replace(" ", ""); -} -``` - -### Inheritance Chain Handling - -When discovering `CustomWebApplicationFactory`, also generate for generic base types: - -``` -CustomWebApplicationFactory - └── TestWebApplicationFactory (generate metadata) - └── WebApplicationFactory (generate metadata if relevant) -``` - -## Implementation Plan - -### Phase 1: Core Discovery Infrastructure -1. Create `ConcreteGenericTypeDiscoverer` helper class -2. Implement discovery from inheritance chains -3. Implement discovery from `IDataSourceAttribute` type arguments -4. Add deduplication logic - -### Phase 2: Extend PropertyInjectionSourceGenerator -1. Add Pipeline 3: Concrete generic type collection -2. Add Pipeline 4: Generic property data source generation -3. Add Pipeline 5: Generic IAsyncInitializer property generation -4. Update safe file naming for generic type arguments - -### Phase 3: Handle Inheritance Chains -1. Walk up base types when discovering concrete generic type -2. Construct concrete version of each generic base type -3. Generate metadata for each hierarchy level - -### Phase 4: Testing -1. Source generator unit tests for generic type scenarios -2. Integration tests for end-to-end behavior -3. Specific test for issue #4431 scenario -4. AOT compatibility verification - -### Phase 5: Cleanup -1. Update PR #4434 with complete fix -2. Update documentation if needed - -## Testing Strategy - -### Unit Tests (Source Generator) - -```csharp -// Generic class with IDataSourceAttribute property -[Fact] -public async Task GenericClass_WithDataSourceProperty_GeneratesMetadata(); - -// Generic class implementing IAsyncInitializer -[Fact] -public async Task GenericClass_ImplementingIAsyncInitializer_GeneratesMetadata(); - -// Discovery via inheritance -[Fact] -public async Task Discovery_ViaInheritance_FindsConcreteType(); - -// Discovery via IDataSourceAttribute type argument -[Fact] -public async Task Discovery_ViaDataSourceAttribute_FindsConcreteType(); - -// Nested generics -[Fact] -public async Task Discovery_NestedGenerics_FindsAllConcreteTypes(); - -// Inheritance chain walking -[Fact] -public async Task Discovery_WalksInheritanceChain_FindsBaseTypes(); - -// Deduplication -[Fact] -public async Task Discovery_DuplicateUsages_GeneratesOnce(); -``` - -### Integration Tests (Engine) - -```csharp -// Issue #4431 scenario -[Fact] -public async Task GenericWebApplicationFactory_InitializesNestedInitializers(); - -// Shared data source with generic fixture -[Fact] -public async Task GenericFixture_SharedDataSource_InitializedBeforeTest(); - -// Multiple instantiations -[Fact] -public async Task SameGeneric_DifferentTypeArgs_BothWork(); -``` - -### AOT Verification - -```csharp -// Verify source-gen metadata exists -[Fact] -public async Task GenericTypes_HaveSourceGenMetadata_NoReflectionFallback(); -``` - -## File Changes - -- `PropertyInjectionSourceGenerator.cs` - Major changes (new pipelines) -- New: `ConcreteGenericTypeDiscoverer.cs` - Discovery helper -- New: `ConcreteGenericTypeModel.cs` - Model for discovered types -- New tests in `TUnit.Core.SourceGenerator.Tests` -- New tests in `TUnit.Engine.Tests` - -## Backward Compatibility - -- Fully backward compatible -- Non-generic types continue to work unchanged -- Generic types that previously fell back to reflection now get source-gen metadata -- Reflection fallback remains for runtime-only types (non-AOT scenarios) - -## Open Questions - -None - design is complete and ready for implementation. diff --git a/docs/plans/2026-01-16-generic-method-discovery-design.md b/docs/plans/2026-01-16-generic-method-discovery-design.md deleted file mode 100644 index 41a1c2bb92..0000000000 --- a/docs/plans/2026-01-16-generic-method-discovery-design.md +++ /dev/null @@ -1,237 +0,0 @@ -# Generic Method Discovery with [GenerateGenericTest] + [MethodDataSource] - -**Issue:** [#4440](https://github.com/thomhurst/TUnit/issues/4440) -**Date:** 2026-01-16 -**Status:** Design Complete - -## Problem Statement - -When a generic **method** (not class) has both `[GenerateGenericTest]` and `[MethodDataSource]` attributes, tests fail to be discovered at runtime in reflection mode, though the source generator produces correct metadata. - -```csharp -public class NonGenericClassWithGenericMethodAndDataSource -{ - [Test] - [GenerateGenericTest(typeof(int))] - [GenerateGenericTest(typeof(double))] - [MethodDataSource(nameof(GetStrings))] - public async Task GenericMethod_With_DataSource(string input) { } -} -``` - -**Expected:** 4 tests (int×"hello", int×"world", double×"hello", double×"world") -**Actual:** 0 tests discovered in reflection mode - -## Root Cause Analysis - -| Mode | Class-level `[GenerateGenericTest]` | Method-level `[GenerateGenericTest]` | -|------|-------------------------------------|--------------------------------------| -| Source Generator | Handled (lines 3680-3733) | Handled (lines 3735-3751) | -| Reflection | Handled (lines 588-625) | **NOT handled** | - -In `ReflectionTestDataCollector.cs`, lines 588 and 716 only check for `[GenerateGenericTest]` on classes: -```csharp -var generateGenericTestAttributes = genericTypeDefinition.GetCustomAttributes(inherit: false).ToArray(); -``` - -No equivalent code exists for method-level `[GenerateGenericTest]`. - -## Solution Design - -### Architecture Overview - -Refactor discovery to separate concerns into three distinct responsibilities: - -``` -DiscoverTestsAsync(assembly) - └─> for each type: - └─> DiscoverTestsFromTypeAsync(type) - └─> for each (concreteClass, classData) in ResolveClassInstantiations(type): - └─> for each method in GetTestMethods(concreteClass): - └─> for each concreteMethod in ResolveMethodInstantiations(method): - └─> BuildTestMetadata(concreteClass, concreteMethod, classData) -``` - -| Method | Responsibility | -|--------|----------------| -| `ResolveClassInstantiationsAsync` | Yields `(Type, object[]?)` for each concrete class variant | -| `ResolveMethodInstantiations` | Yields `MethodInfo` for each concrete method variant | -| `BuildTestMetadata` | Creates metadata from concrete class + concrete method (unchanged) | - -### Method 1: `ResolveClassInstantiationsAsync` - -**Signature:** -```csharp -private static async IAsyncEnumerable<(Type ConcreteType, object?[]? ClassData)> ResolveClassInstantiationsAsync( - [DynamicallyAccessedMembers(...)] Type type, - [EnumeratorCancellation] CancellationToken cancellationToken) -``` - -**Logic:** -1. If type is NOT a generic type definition: - - Yield `(type, null)` once - -2. If type IS a generic type definition: - - Check for `[GenerateGenericTest]` attributes on class - - For each attribute: extract type args, validate constraints, yield `(concreteType, null)` - - Check for class-level data sources - - For each data row: infer type args, yield `(concreteType, dataRow)` - - If neither found: yield nothing (can't resolve open generic) - -### Method 2: `ResolveMethodInstantiations` - -**Signature:** -```csharp -private static IEnumerable ResolveMethodInstantiations( - Type concreteClassType, - MethodInfo method) -``` - -**Note:** Synchronous because `[GenerateGenericTest]` attributes are available immediately. - -**Logic:** -1. If method is NOT a generic method definition: - - Yield `method` once - -2. If method IS a generic method definition: - - Get `[GenerateGenericTest]` attributes from method - - If attributes found: - - For each: extract type args, validate constraints, yield `method.MakeGenericMethod(typeArgs)` - - If no attributes: - - Yield method as-is (TestBuilder will attempt inference from data sources) - -### Error Handling - -All errors become **visible failed tests** in the test explorer: - -```csharp -private static TestMetadata CreateFailedDiscoveryTest( - Type? testClass, - MethodInfo? testMethod, - string errorMessage, - Exception? exception = null) -{ - var fullMessage = exception != null - ? $"{errorMessage}\n\nException: {exception.GetType().Name}: {exception.Message}\n{exception.StackTrace}" - : errorMessage; - - return new FailedTestMetadata - { - TestName = testMethod?.Name ?? testClass?.Name ?? "Unknown", - TestClassType = testClass ?? typeof(object), - TestMethodName = testMethod?.Name ?? "DiscoveryError", - FailureReason = fullMessage, - DiscoveryException = exception ?? new TestDiscoveryException(errorMessage) - }; -} -``` - -**Error scenarios that create failed tests:** -- Constraint violations when calling `MakeGenericType`/`MakeGenericMethod` -- Type argument count mismatch -- Data source retrieval failures -- Reflection failures - -**What users see:** -``` -❌ GenericMethod_With_DataSource (Discovery Failed) - - [GenerateGenericTest] provides 1 type argument(s) but method - 'GenericMethod_With_DataSource' requires 2. - Provided: [Int32] -``` - -### TestBuilder Integration - -No changes needed to TestBuilder. By the time metadata reaches TestBuilder: -- Class type is concrete -- Method is concrete (with `GenericMethodTypeArguments` populated) -- `[MethodDataSource]` is preserved on the concrete method - -TestBuilder's existing data source handling works unchanged. - -## Implementation Plan - -### Step 1: Add Resolution Methods - -**File:** `TUnit.Engine/Discovery/ReflectionTestDataCollector.cs` - -Add two new methods: -- `ResolveClassInstantiationsAsync` - extracts and refactors existing logic from `DiscoverTestsFromGenericTypeAsync` -- `ResolveMethodInstantiations` - new logic for method-level `[GenerateGenericTest]` - -### Step 2: Add Error Handling Infrastructure - -**File:** `TUnit.Engine/Discovery/ReflectionTestDataCollector.cs` - -Add helper method: -- `CreateFailedDiscoveryTest` - creates visible failed test metadata for errors - -### Step 3: Refactor Main Discovery Loop - -**File:** `TUnit.Engine/Discovery/ReflectionTestDataCollector.cs` - -Refactor `DiscoverTestsFromTypeAsync` to use the new resolution methods: -```csharp -await foreach (var (concreteClass, classData) in ResolveClassInstantiationsAsync(type, cancellationToken)) -{ - foreach (var method in GetTestMethods(concreteClass)) - { - foreach (var concreteMethod in ResolveMethodInstantiations(concreteClass, method)) - { - yield return await BuildTestMetadata(concreteClass, concreteMethod, classData); - } - } -} -``` - -### Step 4: Add Test Fixtures - -**File:** `TUnit.TestProject/Bugs/4440/GenericMethodDiscoveryTests.cs` - -Create test fixtures covering: -- Non-generic class + generic method + data source (original bug) -- Generic class + generic method + data source (cartesian product) -- Error cases (constraint violations, type arg mismatches) - -### Step 5: Add Unit Tests - -**File:** `TUnit.Engine.Tests/ResolveClassInstantiationsTests.cs` -**File:** `TUnit.Engine.Tests/ResolveMethodInstantiationsTests.cs` - -Unit tests for the new resolution methods in isolation. - -### Step 6: Expand Integration Tests - -**File:** `TUnit.Engine.Tests/GenericMethodWithDataSourceTests.cs` - -Expand existing tests to run in both modes and verify parity. - -## Test Matrix - -| Scenario | Expected Tests | Source Gen | Reflection | -|----------|---------------|------------|------------| -| Non-generic class, generic method, 2 types, 2 data | 4 | Must pass | Must pass | -| Generic class (2 types), non-generic method, 3 data | 6 | Must pass | Must pass | -| Generic class (2 types), generic method (2 types), 2 data | 8 | Must pass | Must pass | -| Constraint violation | 1 failed | Must show error | Must show error | -| Type arg count mismatch | 1 failed | Must show error | Must show error | - -## Files Changed - -| File | Change | -|------|--------| -| `TUnit.Engine/Discovery/ReflectionTestDataCollector.cs` | Add resolution methods, refactor discovery loop | -| `TUnit.TestProject/Bugs/4440/GenericMethodDiscoveryTests.cs` | New test fixtures | -| `TUnit.Engine.Tests/GenericMethodWithDataSourceTests.cs` | Expand integration tests | -| `TUnit.Engine.Tests/ResolveClassInstantiationsTests.cs` | New unit tests | -| `TUnit.Engine.Tests/ResolveMethodInstantiationsTests.cs` | New unit tests | - -## Success Criteria - -1. Issue #4440 scenario discovers 4 tests in both modes -2. Cartesian product (class + method generics) works correctly -3. All errors visible as failed tests in test explorer -4. Source generator and reflection modes produce identical test counts -5. No regression in existing generic class tests -6. All existing tests continue to pass diff --git a/docs/plans/2026-01-17-log-streaming-design.md b/docs/plans/2026-01-17-log-streaming-design.md deleted file mode 100644 index 7e9fd53c30..0000000000 --- a/docs/plans/2026-01-17-log-streaming-design.md +++ /dev/null @@ -1,407 +0,0 @@ -# Log Streaming Plugin System Design - -**Date:** 2026-01-17 -**Issue:** [#4478 - Stream logs](https://github.com/thomhurst/TUnit/issues/4478) -**Status:** Draft - -## Problem Statement - -Currently, when using TUnit's logging with `TestContext.GetDefaultLogger()`, log output only appears in the IDE (e.g., Rider) after test completion. Users expect real-time log streaming during test execution, similar to NUnit's behavior. - -```csharp -[Test] -public async Task X() -{ - for (int i = 0; i < 3; i += 1) - { - TestContext.Current!.GetDefaultLogger().LogInformation(i.ToString()); - await Task.Delay(1000); - } -} -``` - -**Current behavior:** All 3 log messages appear after the test completes. -**Expected behavior:** Each log message appears as it's written. - -## Root Cause - -Microsoft Testing Platform has two output channels: -1. **Real-time:** `IOutputDevice.DisplayAsync()` - streams directly to IDEs during execution -2. **Historical:** `StandardOutputProperty` on `TestNodeUpdateMessage` - bundled at test completion - -`DefaultLogger` writes to `context.OutputWriter` (historical) and `OriginalConsoleOut` (console), but never uses `IOutputDevice.DisplayAsync()` for real-time IDE streaming. - -## Solution: Plugin-Based Log Sink System - -Inspired by ASP.NET Core's logging architecture, we'll introduce a plugin system that: -1. Allows multiple log destinations (sinks) -2. Enables real-time streaming via `IOutputDevice` -3. Maintains backward compatibility with historical capture -4. Opens extensibility for custom sinks (Seq, file, etc.) - -## Design - -### Core Interfaces (TUnit.Core) - -#### ILogSink - -```csharp -namespace TUnit.Core.Logging; - -/// -/// Represents a destination for log messages. Implement this interface -/// to create custom log sinks (e.g., file, Seq, Application Insights). -/// -public interface ILogSink -{ - /// - /// Asynchronously logs a message. - /// - /// The log level. - /// The formatted message. - /// Optional exception. - /// The current context (TestContext, ClassHookContext, etc.), or null if outside test execution. - ValueTask LogAsync(LogLevel level, string message, Exception? exception, Context? context); - - /// - /// Synchronously logs a message. - /// - void Log(LogLevel level, string message, Exception? exception, Context? context); - - /// - /// Determines if this sink should receive messages at the specified level. - /// - bool IsEnabled(LogLevel level); -} -``` - -**Design notes:** -- Both sync and async methods match existing `ILogger` pattern -- `Context?` is nullable for console output outside test execution -- Sinks can cast to `TestContext` when they need test-specific info (test name, class, etc.) -- `IsEnabled` allows sinks to filter by level for performance -- If sink implements `IAsyncDisposable`, TUnit calls it at session end - -#### TUnitLoggerFactory - -```csharp -namespace TUnit.Core.Logging; - -/// -/// Factory for configuring and managing log sinks. -/// -public static class TUnitLoggerFactory -{ - private static readonly List Sinks = new(); - private static readonly object Lock = new(); - - /// - /// Registers a log sink to receive log messages. - /// Call this in [Before(Assembly)] or before tests run. - /// - public static void AddSink(ILogSink sink) - { - lock (Lock) - { - Sinks.Add(sink); - } - } - - /// - /// Registers a log sink by type. TUnit will instantiate it. - /// - public static void AddSink() where TSink : ILogSink, new() - { - AddSink(new TSink()); - } - - internal static IReadOnlyList GetSinks() => Sinks; - - internal static async ValueTask DisposeAllAsync() - { - foreach (var sink in Sinks) - { - if (sink is IAsyncDisposable disposable) - { - try - { - await disposable.DisposeAsync(); - } - catch - { - // Swallow disposal errors - } - } - } - Sinks.Clear(); - } -} -``` - -### Routing Changes - -#### DefaultLogger Modifications - -```csharp -// In DefaultLogger.WriteToOutput / WriteToOutputAsync: - -protected virtual void WriteToOutput(string message, bool isError) -{ - var level = isError ? LogLevel.Error : LogLevel.Information; - - // Historical capture (unchanged) - if (isError) - context.ErrorOutputWriter.WriteLine(message); - else - context.OutputWriter.WriteLine(message); - - // Real-time streaming to sinks (new) - foreach (var sink in TUnitLoggerFactory.GetSinks()) - { - if (!sink.IsEnabled(level)) - continue; - - try - { - sink.Log(level, message, exception: null, context); - } - catch (Exception ex) - { - GlobalContext.Current.OriginalConsoleError.WriteLine( - $"[TUnit] Log sink {sink.GetType().Name} failed: {ex.Message}"); - } - } -} - -protected virtual async ValueTask WriteToOutputAsync(string message, bool isError) -{ - var level = isError ? LogLevel.Error : LogLevel.Information; - - // Historical capture (unchanged) - if (isError) - await context.ErrorOutputWriter.WriteLineAsync(message); - else - await context.OutputWriter.WriteLineAsync(message); - - // Real-time streaming to sinks (new) - foreach (var sink in TUnitLoggerFactory.GetSinks()) - { - if (!sink.IsEnabled(level)) - continue; - - try - { - await sink.LogAsync(level, message, exception: null, context); - } - catch (Exception ex) - { - await GlobalContext.Current.OriginalConsoleError.WriteLineAsync( - $"[TUnit] Log sink {sink.GetType().Name} failed: {ex.Message}"); - } - } -} -``` - -#### Console Interceptor Modifications - -Route `Console.WriteLine` through sinks for real-time streaming: - -```csharp -// In StandardOutConsoleInterceptor, after writing to context: - -private void RouteToSinks(string? value) -{ - if (string.IsNullOrEmpty(value)) - return; - - var sinks = TUnitLoggerFactory.GetSinks(); - if (sinks.Count == 0) - return; - - var context = Context.Current; // may be null outside test execution - - foreach (var sink in sinks) - { - if (!sink.IsEnabled(LogLevel.Information)) - continue; - - try - { - sink.Log(LogLevel.Information, value, exception: null, context); - } - catch (Exception ex) - { - // Write to original console to avoid recursion - GlobalContext.Current.OriginalConsoleError.WriteLine( - $"[TUnit] Log sink {sink.GetType().Name} failed: {ex.Message}"); - } - } -} -``` - -### Engine's Built-in Sink (TUnit.Engine) - -```csharp -namespace TUnit.Engine.Logging; - -/// -/// Built-in sink that streams logs to IDEs via Microsoft Testing Platform's IOutputDevice. -/// Automatically registered by TUnit.Engine at startup. -/// -internal class OutputDeviceLogSink : ILogSink -{ - private readonly IOutputDevice _outputDevice; - private readonly LogLevel _minLevel; - - public OutputDeviceLogSink(IOutputDevice outputDevice, LogLevel minLevel = LogLevel.Information) - { - _outputDevice = outputDevice; - _minLevel = minLevel; - } - - public bool IsEnabled(LogLevel level) => level >= _minLevel; - - public void Log(LogLevel level, string message, Exception? exception, Context? context) - { - // Fire and forget for sync path - IOutputDevice is async-only - _ = LogAsync(level, message, exception, context); - } - - public async ValueTask LogAsync(LogLevel level, string message, Exception? exception, Context? context) - { - if (!IsEnabled(level)) - return; - - var color = GetConsoleColor(level); - - await _outputDevice.DisplayAsync( - this, - new FormattedTextOutputDeviceData(message) - { - ForegroundColor = new SystemConsoleColor { ConsoleColor = color } - }, - CancellationToken.None); - } - - private static ConsoleColor GetConsoleColor(LogLevel level) => level switch - { - LogLevel.Error => ConsoleColor.Red, - LogLevel.Warning => ConsoleColor.Yellow, - LogLevel.Debug => ConsoleColor.Gray, - _ => ConsoleColor.White - }; -} -``` - -**Registration during test session initialization:** - -```csharp -// In TUnitTestFramework or test session initialization: -var outputDevice = serviceProvider.GetRequiredService(); -TUnitLoggerFactory.AddSink(new OutputDeviceLogSink(outputDevice)); -``` - -## Data Flow - -``` -┌─────────────────────────────────────────────────────────────────┐ -│ Test Code │ -│ - TestContext.GetDefaultLogger().LogInformation("...") │ -│ - Console.WriteLine("...") │ -└──────────────────────────┬──────────────────────────────────────┘ - │ - ▼ -┌─────────────────────────────────────────────────────────────────┐ -│ DefaultLogger / Console Interceptor │ -│ 1. Write to context.OutputWriter (historical capture) │ -│ 2. Route to all registered ILogSink instances │ -└──────────────────────────┬──────────────────────────────────────┘ - │ - ┌───────────────┼───────────────┐ - ▼ ▼ ▼ -┌─────────────────┐ ┌─────────────┐ ┌─────────────────┐ -│ OutputDevice │ │ User's Seq │ │ User's File │ -│ LogSink │ │ LogSink │ │ LogSink │ -│ (built-in) │ │ (custom) │ │ (custom) │ -└────────┬────────┘ └──────┬──────┘ └────────┬────────┘ - │ │ │ - ▼ ▼ ▼ - IDE Real-time Seq Server Log File -``` - -## User Registration Example - -```csharp -[assembly: Before(Assembly)] -public static class LoggingSetup -{ - public static Task BeforeAssembly() - { - // Add custom sinks - TUnitLoggerFactory.AddSink(new SeqLogSink("http://localhost:5341")); - TUnitLoggerFactory.AddSink(); - return Task.CompletedTask; - } -} - -// Example custom sink -public class FileLogSink : ILogSink, IAsyncDisposable -{ - private readonly StreamWriter _writer; - - public FileLogSink() - { - _writer = new StreamWriter("test-log.txt", append: true); - } - - public bool IsEnabled(LogLevel level) => true; - - public void Log(LogLevel level, string message, Exception? exception, Context? context) - { - var testName = (context as TestContext)?.TestDetails.TestName ?? "N/A"; - _writer.WriteLine($"[{DateTime.Now:HH:mm:ss}] [{level}] [{testName}] {message}"); - } - - public async ValueTask LogAsync(LogLevel level, string message, Exception? exception, Context? context) - { - var testName = (context as TestContext)?.TestDetails.TestName ?? "N/A"; - await _writer.WriteLineAsync($"[{DateTime.Now:HH:mm:ss}] [{level}] [{testName}] {message}"); - } - - public async ValueTask DisposeAsync() - { - await _writer.DisposeAsync(); - } -} -``` - -## Files to Create/Modify - -| File | Action | Description | -|------|--------|-------------| -| `TUnit.Core/Logging/ILogSink.cs` | Create | New sink interface | -| `TUnit.Core/Logging/TUnitLoggerFactory.cs` | Create | Sink registration | -| `TUnit.Core/Logging/DefaultLogger.cs` | Modify | Route to sinks | -| `TUnit.Core/Logging/StandardOutConsoleInterceptor.cs` | Modify | Route console to sinks | -| `TUnit.Engine/Logging/OutputDeviceLogSink.cs` | Create | Built-in IDE streaming sink | -| `TUnit.Engine/Services/TUnitTestFramework.cs` | Modify | Register OutputDeviceLogSink | - -## Error Handling - -- Sink failures are caught and logged to `OriginalConsoleError` -- Failures do not break tests or stop other sinks from receiving messages -- Disposal errors are swallowed during cleanup - -## Backward Compatibility - -- No breaking changes to existing APIs -- Historical capture via `context.OutputWriter` unchanged -- Existing behavior preserved if no custom sinks registered -- `OutputDeviceLogSink` registered automatically by Engine - -## Future Considerations - -- Built-in sinks package (file, JSON, etc.) -- Structured logging support with semantic properties -- Log level configuration per sink -- Async batching for high-throughput scenarios diff --git a/docs/plans/2026-01-17-log-streaming-implementation.md b/docs/plans/2026-01-17-log-streaming-implementation.md deleted file mode 100644 index ff08669d03..0000000000 --- a/docs/plans/2026-01-17-log-streaming-implementation.md +++ /dev/null @@ -1,999 +0,0 @@ -# Log Streaming Plugin System Implementation Plan - -> **For Claude:** REQUIRED SUB-SKILL: Use superpowers:executing-plans to implement this plan task-by-task. - -**Goal:** Enable real-time log streaming to IDEs during test execution via a plugin-based ILogSink system. - -**Architecture:** Introduce `ILogSink` interface in TUnit.Core that receives log messages. `TUnitLoggerFactory` manages sink registration. `DefaultLogger` and console interceptors route to all registered sinks. TUnit.Engine registers `OutputDeviceLogSink` at startup which uses `IOutputDevice.DisplayAsync()` for real-time IDE streaming. - -**Tech Stack:** .NET, Microsoft Testing Platform, IOutputDevice - -**Design Document:** `docs/plans/2026-01-17-log-streaming-design.md` - ---- - -## Task 1: Create ILogSink Interface - -**Files:** -- Create: `TUnit.Core/Logging/ILogSink.cs` - -**Step 1: Create the interface file** - -```csharp -namespace TUnit.Core.Logging; - -/// -/// Represents a destination for log messages. Implement this interface -/// to create custom log sinks (e.g., file, Seq, Application Insights). -/// -public interface ILogSink -{ - /// - /// Asynchronously logs a message. - /// - /// The log level. - /// The formatted message. - /// Optional exception. - /// The current context (TestContext, ClassHookContext, etc.), or null if outside test execution. - ValueTask LogAsync(LogLevel level, string message, Exception? exception, Context? context); - - /// - /// Synchronously logs a message. - /// - void Log(LogLevel level, string message, Exception? exception, Context? context); - - /// - /// Determines if this sink should receive messages at the specified level. - /// - bool IsEnabled(LogLevel level); -} -``` - -**Step 2: Verify it compiles** - -Run: `dotnet build TUnit.Core/TUnit.Core.csproj -c Release` -Expected: Build succeeded - -**Step 3: Commit** - -```bash -git add TUnit.Core/Logging/ILogSink.cs -git commit -m "feat(logging): add ILogSink interface for log destinations" -``` - ---- - -## Task 2: Create TUnitLoggerFactory - -**Files:** -- Create: `TUnit.Core/Logging/TUnitLoggerFactory.cs` - -**Step 1: Create the factory class** - -```csharp -namespace TUnit.Core.Logging; - -/// -/// Factory for configuring and managing log sinks. -/// -public static class TUnitLoggerFactory -{ - private static readonly List _sinks = []; - private static readonly Lock _lock = new(); - - /// - /// Registers a log sink to receive log messages. - /// Call this in [Before(Assembly)] or before tests run. - /// - public static void AddSink(ILogSink sink) - { - lock (_lock) - { - _sinks.Add(sink); - } - } - - /// - /// Registers a log sink by type. TUnit will instantiate it. - /// - public static void AddSink() where TSink : ILogSink, new() - { - AddSink(new TSink()); - } - - /// - /// Gets all registered sinks. For internal use. - /// - internal static IReadOnlyList GetSinks() - { - lock (_lock) - { - return _sinks.ToArray(); - } - } - - /// - /// Disposes all sinks that implement IAsyncDisposable. - /// Called at end of test session. - /// - internal static async ValueTask DisposeAllAsync() - { - ILogSink[] sinksToDispose; - lock (_lock) - { - sinksToDispose = _sinks.ToArray(); - _sinks.Clear(); - } - - foreach (var sink in sinksToDispose) - { - if (sink is IAsyncDisposable disposable) - { - try - { - await disposable.DisposeAsync(); - } - catch - { - // Swallow disposal errors - } - } - } - } - - /// - /// Clears all registered sinks. For testing purposes. - /// - internal static void Clear() - { - lock (_lock) - { - _sinks.Clear(); - } - } -} -``` - -**Step 2: Verify it compiles** - -Run: `dotnet build TUnit.Core/TUnit.Core.csproj -c Release` -Expected: Build succeeded - -**Step 3: Commit** - -```bash -git add TUnit.Core/Logging/TUnitLoggerFactory.cs -git commit -m "feat(logging): add TUnitLoggerFactory for sink registration" -``` - ---- - -## Task 3: Add Internal Sink Routing Helper - -**Files:** -- Create: `TUnit.Core/Logging/LogSinkRouter.cs` - -**Step 1: Create router helper to avoid code duplication** - -```csharp -namespace TUnit.Core.Logging; - -/// -/// Internal helper for routing log messages to all registered sinks. -/// -internal static class LogSinkRouter -{ - public static void RouteToSinks(LogLevel level, string message, Exception? exception, Context? context) - { - var sinks = TUnitLoggerFactory.GetSinks(); - if (sinks.Count == 0) - { - return; - } - - foreach (var sink in sinks) - { - if (!sink.IsEnabled(level)) - { - continue; - } - - try - { - sink.Log(level, message, exception, context); - } - catch (Exception ex) - { - // Write to original console to avoid recursion - GlobalContext.Current.OriginalConsoleError.WriteLine( - $"[TUnit] Log sink {sink.GetType().Name} failed: {ex.Message}"); - } - } - } - - public static async ValueTask RouteToSinksAsync(LogLevel level, string message, Exception? exception, Context? context) - { - var sinks = TUnitLoggerFactory.GetSinks(); - if (sinks.Count == 0) - { - return; - } - - foreach (var sink in sinks) - { - if (!sink.IsEnabled(level)) - { - continue; - } - - try - { - await sink.LogAsync(level, message, exception, context); - } - catch (Exception ex) - { - // Write to original console to avoid recursion - await GlobalContext.Current.OriginalConsoleError.WriteLineAsync( - $"[TUnit] Log sink {sink.GetType().Name} failed: {ex.Message}"); - } - } - } -} -``` - -**Step 2: Verify it compiles** - -Run: `dotnet build TUnit.Core/TUnit.Core.csproj -c Release` -Expected: Build succeeded - -**Step 3: Commit** - -```bash -git add TUnit.Core/Logging/LogSinkRouter.cs -git commit -m "feat(logging): add LogSinkRouter helper for sink dispatch" -``` - ---- - -## Task 4: Modify DefaultLogger to Route to Sinks - -**Files:** -- Modify: `TUnit.Core/Logging/DefaultLogger.cs` - -**Step 1: Update WriteToOutput to route to sinks** - -Find the `WriteToOutput` method (around line 125) and replace with: - -```csharp -/// -/// Writes the message to the output. -/// Override this method to customize how messages are written. -/// -/// The formatted message to write. -/// True if this is an error-level message. -protected virtual void WriteToOutput(string message, bool isError) -{ - var level = isError ? LogLevel.Error : LogLevel.Information; - - // Historical capture - if (isError) - { - context.ErrorOutputWriter.WriteLine(message); - } - else - { - context.OutputWriter.WriteLine(message); - } - - // Real-time streaming to sinks - LogSinkRouter.RouteToSinks(level, message, null, context); -} -``` - -**Step 2: Update WriteToOutputAsync to route to sinks** - -Find the `WriteToOutputAsync` method (around line 146) and replace with: - -```csharp -/// -/// Asynchronously writes the message to the output. -/// Override this method to customize how messages are written. -/// -/// The formatted message to write. -/// True if this is an error-level message. -/// A task representing the async operation. -protected virtual async ValueTask WriteToOutputAsync(string message, bool isError) -{ - var level = isError ? LogLevel.Error : LogLevel.Information; - - // Historical capture - if (isError) - { - await context.ErrorOutputWriter.WriteLineAsync(message); - } - else - { - await context.OutputWriter.WriteLineAsync(message); - } - - // Real-time streaming to sinks - await LogSinkRouter.RouteToSinksAsync(level, message, null, context); -} -``` - -**Step 3: Verify it compiles** - -Run: `dotnet build TUnit.Core/TUnit.Core.csproj -c Release` -Expected: Build succeeded - -**Step 4: Commit** - -```bash -git add TUnit.Core/Logging/DefaultLogger.cs -git commit -m "feat(logging): route DefaultLogger output to registered sinks" -``` - ---- - -## Task 5: Modify Console Interceptor to Route to Sinks - -**Files:** -- Modify: `TUnit.Engine/Logging/StandardOutConsoleInterceptor.cs` - -**Step 1: Find the interceptor and understand its structure** - -Read the file first to understand how it intercepts console output. - -Run: Read `TUnit.Engine/Logging/StandardOutConsoleInterceptor.cs` - -**Step 2: Add sink routing after console capture** - -The interceptor likely has Write/WriteLine methods that capture output. Add routing to sinks after capturing. The exact modification depends on the file structure, but the pattern is: - -After any line that writes to the context's output (like `Context.Current?.OutputWriter?.WriteLine(value)`), add: - -```csharp -// Route to sinks for real-time streaming -LogSinkRouter.RouteToSinks(LogLevel.Information, value?.ToString() ?? string.Empty, null, Context.Current); -``` - -**Step 3: Add using statement if needed** - -Add at top of file: -```csharp -using TUnit.Core.Logging; -``` - -**Step 4: Verify it compiles** - -Run: `dotnet build TUnit.Engine/TUnit.Engine.csproj -c Release` -Expected: Build succeeded - -**Step 5: Commit** - -```bash -git add TUnit.Engine/Logging/StandardOutConsoleInterceptor.cs -git commit -m "feat(logging): route Console.WriteLine to registered sinks" -``` - ---- - -## Task 6: Modify Console Error Interceptor (if separate) - -**Files:** -- Modify: `TUnit.Engine/Logging/StandardErrorConsoleInterceptor.cs` (if exists) - -**Step 1: Check if file exists and apply same pattern** - -If there's a separate error interceptor, apply the same changes as Task 5 but use `LogLevel.Error`: - -```csharp -LogSinkRouter.RouteToSinks(LogLevel.Error, value?.ToString() ?? string.Empty, null, Context.Current); -``` - -**Step 2: Verify it compiles** - -Run: `dotnet build TUnit.Engine/TUnit.Engine.csproj -c Release` -Expected: Build succeeded - -**Step 3: Commit (if changes made)** - -```bash -git add TUnit.Engine/Logging/StandardErrorConsoleInterceptor.cs -git commit -m "feat(logging): route Console.Error to registered sinks" -``` - ---- - -## Task 7: Create OutputDeviceLogSink in TUnit.Engine - -**Files:** -- Create: `TUnit.Engine/Logging/OutputDeviceLogSink.cs` - -**Step 1: Create the sink that streams to IDEs** - -```csharp -using Microsoft.Testing.Platform.OutputDevice; -using TUnit.Core; -using TUnit.Core.Logging; - -namespace TUnit.Engine.Logging; - -/// -/// Built-in sink that streams logs to IDEs via Microsoft Testing Platform's IOutputDevice. -/// Automatically registered by TUnit.Engine at startup. -/// -internal class OutputDeviceLogSink : ILogSink, IOutputDeviceDataProducer -{ - private readonly IOutputDevice _outputDevice; - private readonly LogLevel _minLevel; - - public OutputDeviceLogSink(IOutputDevice outputDevice, LogLevel minLevel = LogLevel.Information) - { - _outputDevice = outputDevice; - _minLevel = minLevel; - } - - public string Uid => "TUnit.OutputDeviceLogSink"; - public string Version => typeof(OutputDeviceLogSink).Assembly.GetName().Version?.ToString() ?? "1.0.0"; - public string DisplayName => "TUnit Log Sink"; - public string Description => "Streams test logs to IDE in real-time"; - - public Task IsEnabledAsync() => Task.FromResult(true); - - public bool IsEnabled(LogLevel level) => level >= _minLevel; - - public void Log(LogLevel level, string message, Exception? exception, Context? context) - { - if (!IsEnabled(level)) - { - return; - } - - // Fire and forget for sync path - IOutputDevice is async-only - _ = LogAsync(level, message, exception, context); - } - - public async ValueTask LogAsync(LogLevel level, string message, Exception? exception, Context? context) - { - if (!IsEnabled(level)) - { - return; - } - - try - { - var color = GetConsoleColor(level); - - await _outputDevice.DisplayAsync( - this, - new FormattedTextOutputDeviceData(message) - { - ForegroundColor = new SystemConsoleColor { ConsoleColor = color } - }, - CancellationToken.None); - } - catch - { - // Swallow errors - logging should not break tests - } - } - - private static ConsoleColor GetConsoleColor(LogLevel level) => level switch - { - LogLevel.Error => ConsoleColor.Red, - LogLevel.Warning => ConsoleColor.Yellow, - LogLevel.Debug => ConsoleColor.Gray, - LogLevel.Trace => ConsoleColor.DarkGray, - _ => ConsoleColor.White - }; -} -``` - -**Step 2: Verify it compiles** - -Run: `dotnet build TUnit.Engine/TUnit.Engine.csproj -c Release` -Expected: Build succeeded - -**Step 3: Commit** - -```bash -git add TUnit.Engine/Logging/OutputDeviceLogSink.cs -git commit -m "feat(logging): add OutputDeviceLogSink for real-time IDE streaming" -``` - ---- - -## Task 8: Register OutputDeviceLogSink at Startup - -**Files:** -- Modify: Find the test framework initialization (likely `TUnit.Engine/Services/TUnitTestFramework.cs` or similar) - -**Step 1: Find where IOutputDevice is available** - -Search for where `IOutputDevice` is injected or retrieved from the service provider. - -Run: `grep -r "IOutputDevice" TUnit.Engine/ --include="*.cs"` - -**Step 2: Register the sink during initialization** - -At the point where `IOutputDevice` is available (likely in a constructor or initialization method), add: - -```csharp -// Register the built-in sink for real-time IDE streaming -var outputDeviceSink = new OutputDeviceLogSink(outputDevice); -TUnitLoggerFactory.AddSink(outputDeviceSink); -``` - -Add using statement: -```csharp -using TUnit.Core.Logging; -``` - -**Step 3: Verify it compiles** - -Run: `dotnet build TUnit.Engine/TUnit.Engine.csproj -c Release` -Expected: Build succeeded - -**Step 4: Commit** - -```bash -git add TUnit.Engine/Services/*.cs -git commit -m "feat(logging): register OutputDeviceLogSink at test session startup" -``` - ---- - -## Task 9: Dispose Sinks at Session End - -**Files:** -- Modify: Find session cleanup code (likely `TUnit.Engine/Services/TUnitTestFramework.cs` or `OnTestSessionFinishing` handler) - -**Step 1: Find session end hook** - -Search for cleanup or disposal code: - -Run: `grep -r "OnTestSessionFinishing\|Dispose\|Cleanup" TUnit.Engine/Services/ --include="*.cs"` - -**Step 2: Add sink disposal** - -At session end, add: - -```csharp -await TUnitLoggerFactory.DisposeAllAsync(); -``` - -**Step 3: Verify it compiles** - -Run: `dotnet build TUnit.Engine/TUnit.Engine.csproj -c Release` -Expected: Build succeeded - -**Step 4: Commit** - -```bash -git add TUnit.Engine/Services/*.cs -git commit -m "feat(logging): dispose sinks at test session end" -``` - ---- - -## Task 10: Write Unit Tests for TUnitLoggerFactory - -**Files:** -- Create: `TUnit.UnitTests/LogSinkTests.cs` - -**Step 1: Create test file with basic tests** - -```csharp -using TUnit.Core.Logging; - -namespace TUnit.UnitTests; - -public class LogSinkTests -{ - [Before(Test)] - public void Setup() - { - TUnitLoggerFactory.Clear(); - } - - [After(Test)] - public void Cleanup() - { - TUnitLoggerFactory.Clear(); - } - - [Test] - public void AddSink_RegistersSink() - { - // Arrange - var sink = new TestLogSink(); - - // Act - TUnitLoggerFactory.AddSink(sink); - - // Assert - var sinks = TUnitLoggerFactory.GetSinks(); - await Assert.That(sinks).Contains(sink); - } - - [Test] - public void AddSink_Generic_CreatesSinkInstance() - { - // Act - TUnitLoggerFactory.AddSink(); - - // Assert - var sinks = TUnitLoggerFactory.GetSinks(); - await Assert.That(sinks).HasCount().EqualTo(1); - await Assert.That(sinks[0]).IsTypeOf(); - } - - [Test] - public async Task DisposeAllAsync_DisposesAsyncDisposableSinks() - { - // Arrange - var sink = new DisposableTestLogSink(); - TUnitLoggerFactory.AddSink(sink); - - // Act - await TUnitLoggerFactory.DisposeAllAsync(); - - // Assert - await Assert.That(sink.WasDisposed).IsTrue(); - await Assert.That(TUnitLoggerFactory.GetSinks()).IsEmpty(); - } - - [Test] - public void Clear_RemovesAllSinks() - { - // Arrange - TUnitLoggerFactory.AddSink(new TestLogSink()); - TUnitLoggerFactory.AddSink(new TestLogSink()); - - // Act - TUnitLoggerFactory.Clear(); - - // Assert - await Assert.That(TUnitLoggerFactory.GetSinks()).IsEmpty(); - } - - private class TestLogSink : ILogSink - { - public List Messages { get; } = []; - - public bool IsEnabled(LogLevel level) => true; - - public void Log(LogLevel level, string message, Exception? exception, Context? context) - { - Messages.Add(message); - } - - public ValueTask LogAsync(LogLevel level, string message, Exception? exception, Context? context) - { - Messages.Add(message); - return ValueTask.CompletedTask; - } - } - - private class DisposableTestLogSink : TestLogSink, IAsyncDisposable - { - public bool WasDisposed { get; private set; } - - public ValueTask DisposeAsync() - { - WasDisposed = true; - return ValueTask.CompletedTask; - } - } -} -``` - -**Step 2: Run tests** - -Run: `dotnet run --project TUnit.UnitTests/TUnit.UnitTests.csproj -c Release --no-build -f net8.0 -- --treenode-filter "/*/*/LogSinkTests/*"` -Expected: All tests pass - -**Step 3: Commit** - -```bash -git add TUnit.UnitTests/LogSinkTests.cs -git commit -m "test(logging): add unit tests for TUnitLoggerFactory" -``` - ---- - -## Task 11: Write Unit Tests for LogSinkRouter - -**Files:** -- Modify: `TUnit.UnitTests/LogSinkTests.cs` - -**Step 1: Add router tests to the test file** - -```csharp -public class LogSinkRouterTests -{ - [Before(Test)] - public void Setup() - { - TUnitLoggerFactory.Clear(); - } - - [After(Test)] - public void Cleanup() - { - TUnitLoggerFactory.Clear(); - } - - [Test] - public void RouteToSinks_SendsMessageToAllEnabledSinks() - { - // Arrange - var sink1 = new TestLogSink(); - var sink2 = new TestLogSink(); - TUnitLoggerFactory.AddSink(sink1); - TUnitLoggerFactory.AddSink(sink2); - - // Act - LogSinkRouter.RouteToSinks(LogLevel.Information, "test message", null, null); - - // Assert - await Assert.That(sink1.Messages).Contains("test message"); - await Assert.That(sink2.Messages).Contains("test message"); - } - - [Test] - public void RouteToSinks_SkipsDisabledSinks() - { - // Arrange - var enabledSink = new TestLogSink(); - var disabledSink = new TestLogSink { MinLevel = LogLevel.Error }; - TUnitLoggerFactory.AddSink(enabledSink); - TUnitLoggerFactory.AddSink(disabledSink); - - // Act - LogSinkRouter.RouteToSinks(LogLevel.Information, "test message", null, null); - - // Assert - await Assert.That(enabledSink.Messages).Contains("test message"); - await Assert.That(disabledSink.Messages).IsEmpty(); - } - - [Test] - public void RouteToSinks_ContinuesAfterSinkFailure() - { - // Arrange - var failingSink = new FailingLogSink(); - var workingSink = new TestLogSink(); - TUnitLoggerFactory.AddSink(failingSink); - TUnitLoggerFactory.AddSink(workingSink); - - // Act - should not throw - LogSinkRouter.RouteToSinks(LogLevel.Information, "test message", null, null); - - // Assert - working sink still received message - await Assert.That(workingSink.Messages).Contains("test message"); - } - - private class TestLogSink : ILogSink - { - public List Messages { get; } = []; - public LogLevel MinLevel { get; set; } = LogLevel.Trace; - - public bool IsEnabled(LogLevel level) => level >= MinLevel; - - public void Log(LogLevel level, string message, Exception? exception, Context? context) - { - Messages.Add(message); - } - - public ValueTask LogAsync(LogLevel level, string message, Exception? exception, Context? context) - { - Messages.Add(message); - return ValueTask.CompletedTask; - } - } - - private class FailingLogSink : ILogSink - { - public bool IsEnabled(LogLevel level) => true; - - public void Log(LogLevel level, string message, Exception? exception, Context? context) - { - throw new InvalidOperationException("Sink failed"); - } - - public ValueTask LogAsync(LogLevel level, string message, Exception? exception, Context? context) - { - throw new InvalidOperationException("Sink failed"); - } - } -} -``` - -**Step 2: Run tests** - -Run: `dotnet run --project TUnit.UnitTests/TUnit.UnitTests.csproj -c Release -f net8.0 -- --treenode-filter "/*/*/LogSinkRouterTests/*"` -Expected: All tests pass - -**Step 3: Commit** - -```bash -git add TUnit.UnitTests/LogSinkTests.cs -git commit -m "test(logging): add unit tests for LogSinkRouter" -``` - ---- - -## Task 12: Integration Test - End to End - -**Files:** -- Create: `TUnit.UnitTests/LogStreamingIntegrationTests.cs` - -**Step 1: Create integration test** - -```csharp -using TUnit.Core.Logging; - -namespace TUnit.UnitTests; - -public class LogStreamingIntegrationTests -{ - [Before(Test)] - public void Setup() - { - TUnitLoggerFactory.Clear(); - } - - [After(Test)] - public void Cleanup() - { - TUnitLoggerFactory.Clear(); - } - - [Test] - public async Task DefaultLogger_RoutesToRegisteredSinks() - { - // Arrange - var captureSink = new CapturingLogSink(); - TUnitLoggerFactory.AddSink(captureSink); - - var testContext = TestContext.Current; - var logger = testContext!.GetDefaultLogger(); - - // Act - await logger.LogInformationAsync("Hello from test"); - - // Assert - await Assert.That(captureSink.Messages).Contains(m => m.Contains("Hello from test")); - } - - private class CapturingLogSink : ILogSink - { - public List Messages { get; } = []; - - public bool IsEnabled(LogLevel level) => true; - - public void Log(LogLevel level, string message, Exception? exception, Context? context) - { - Messages.Add(message); - } - - public ValueTask LogAsync(LogLevel level, string message, Exception? exception, Context? context) - { - Messages.Add(message); - return ValueTask.CompletedTask; - } - } -} -``` - -**Step 2: Run test** - -Run: `dotnet run --project TUnit.UnitTests/TUnit.UnitTests.csproj -c Release -f net8.0 -- --treenode-filter "/*/*/LogStreamingIntegrationTests/*"` -Expected: Test passes - -**Step 3: Commit** - -```bash -git add TUnit.UnitTests/LogStreamingIntegrationTests.cs -git commit -m "test(logging): add integration test for log streaming" -``` - ---- - -## Task 13: Run Full Test Suite - -**Files:** None (verification only) - -**Step 1: Build entire solution** - -Run: `dotnet build TUnit.sln -c Release` -Expected: Build succeeded - -**Step 2: Run unit tests** - -Run: `dotnet run --project TUnit.UnitTests/TUnit.UnitTests.csproj -c Release --no-build -f net8.0` -Expected: All tests pass - -**Step 3: Run analyzer tests** - -Run: `dotnet run --project TUnit.Analyzers.Tests/TUnit.Analyzers.Tests.csproj -c Release -f net8.0` -Expected: All tests pass - ---- - -## Task 14: Update Public API Surface (if using PublicAPI analyzers) - -**Files:** -- Modify: `TUnit.Core/PublicAPI.Shipped.txt` or `PublicAPI.Unshipped.txt` - -**Step 1: Check if public API tracking is used** - -Run: `ls TUnit.Core/PublicAPI*.txt 2>/dev/null || echo "No PublicAPI files"` - -**Step 2: If files exist, add new public types** - -Add to `PublicAPI.Unshipped.txt`: -``` -TUnit.Core.Logging.ILogSink -TUnit.Core.Logging.ILogSink.IsEnabled(TUnit.Core.Logging.LogLevel) -> bool -TUnit.Core.Logging.ILogSink.Log(TUnit.Core.Logging.LogLevel, string!, System.Exception?, TUnit.Core.Context?) -> void -TUnit.Core.Logging.ILogSink.LogAsync(TUnit.Core.Logging.LogLevel, string!, System.Exception?, TUnit.Core.Context?) -> System.Threading.Tasks.ValueTask -TUnit.Core.Logging.TUnitLoggerFactory -TUnit.Core.Logging.TUnitLoggerFactory.AddSink(TUnit.Core.Logging.ILogSink!) -> void -TUnit.Core.Logging.TUnitLoggerFactory.AddSink() -> void -``` - -**Step 3: Commit** - -```bash -git add TUnit.Core/PublicAPI*.txt -git commit -m "docs: update public API surface for log sink types" -``` - ---- - -## Task 15: Final Verification and Squash (Optional) - -**Step 1: Verify all tests pass** - -Run: `dotnet run --project TUnit.UnitTests/TUnit.UnitTests.csproj -c Release -f net8.0` -Expected: All tests pass including new log sink tests - -**Step 2: Review git log** - -Run: `git log --oneline -15` - -**Step 3: Create final summary commit or squash if desired** - -If keeping granular commits: -```bash -git push -u origin feature/log-streaming -``` - -If squashing: -```bash -git rebase -i main -# Squash commits as desired -git push -u origin feature/log-streaming -``` - ---- - -## Summary - -| Task | Description | Files | -|------|-------------|-------| -| 1 | Create ILogSink interface | `TUnit.Core/Logging/ILogSink.cs` | -| 2 | Create TUnitLoggerFactory | `TUnit.Core/Logging/TUnitLoggerFactory.cs` | -| 3 | Create LogSinkRouter helper | `TUnit.Core/Logging/LogSinkRouter.cs` | -| 4 | Modify DefaultLogger | `TUnit.Core/Logging/DefaultLogger.cs` | -| 5-6 | Modify Console Interceptors | `TUnit.Engine/Logging/Standard*ConsoleInterceptor.cs` | -| 7 | Create OutputDeviceLogSink | `TUnit.Engine/Logging/OutputDeviceLogSink.cs` | -| 8 | Register sink at startup | `TUnit.Engine/Services/*.cs` | -| 9 | Dispose sinks at session end | `TUnit.Engine/Services/*.cs` | -| 10-12 | Write tests | `TUnit.UnitTests/LogSink*.cs` | -| 13 | Full test suite verification | - | -| 14 | Update public API | `TUnit.Core/PublicAPI*.txt` | -| 15 | Final verification | - | diff --git a/docs/plans/2026-01-18-ide-streaming-sink-design.md b/docs/plans/2026-01-18-ide-streaming-sink-design.md deleted file mode 100644 index d8789ba100..0000000000 --- a/docs/plans/2026-01-18-ide-streaming-sink-design.md +++ /dev/null @@ -1,130 +0,0 @@ -# IDE Streaming Sink Design - -**Date:** 2026-01-18 -**Issue:** [#4495](https://github.com/thomhurst/TUnit/issues/4495) -**Status:** Approved - -## Overview - -Implement real-time IDE output streaming via a new `IdeStreamingSink` that sends test output to IDEs during test execution, not just at completion. - -## Background - -PR #4493 introduced an extensible log sink framework (`ILogSink`, `TUnitLoggerFactory`). A previous attempt at IDE streaming was removed due to output duplication issues. This design takes a simpler approach. - -## Design Decisions - -| Decision | Choice | Rationale | -|----------|--------|-----------| -| **Duplication handling** | Cumulative replacement | Each update sends full output; IDE displays latest. Simple, no coordination needed. | -| **Throttling** | 1 second intervals | Prevents flooding IDE with rapid writes. Uses latest snapshot per interval. | -| **Output types** | Both stdout and stderr | Stream `StandardOutputProperty` and `StandardErrorProperty` | -| **Activation** | On by default for IDE | Uses existing `VerbosityService.IsIdeClient` detection | - -## Architecture - -### Data Flow - -``` -Console.WriteLine() - -> StandardOutConsoleInterceptor - -> LogSinkRouter routes to all sinks: - -> TestOutputSink: accumulates to Context.OutputWriter (always) - -> IdeStreamingSink: marks test as "dirty" (IDE only) - -> Timer fires every 1s per test - -> GetStandardOutput() + GetErrorOutput() - -> Send TestNodeUpdateMessage with InProgressTestNodeStateProperty -``` - -### Class Structure - -```csharp -internal sealed class IdeStreamingSink : ILogSink, IAsyncDisposable -{ - private readonly TUnitMessageBus _messageBus; - private readonly SessionUid _sessionUid; - private readonly ConcurrentDictionary _activeTests = new(); - private readonly TimeSpan _throttleInterval = TimeSpan.FromSeconds(1); -} - -private sealed class TestStreamingState : IDisposable -{ - public TestContext TestContext { get; } - public Timer Timer { get; } - public bool IsDirty { get; set; } // Has new output since last send -} -``` - -### Core Logic - -1. **On `Log()` call with `TestContext`:** - - Get or create `TestStreamingState` for this test - - Mark as dirty (`IsDirty = true`) - - Timer is already running (started on first log) - -2. **On timer tick (every 1 second):** - - Check if test completed (passive cleanup) - if so, dispose and remove - - If `IsDirty` is false, skip (no new output) - - Set `IsDirty = false` - - Call `testContext.GetStandardOutput()` and `GetErrorOutput()` - - Send `TestNodeUpdateMessage` with `InProgressTestNodeStateProperty` + output properties - -3. **On dispose:** Cancel all timers, clear dictionary - -### TestNode Creation - -```csharp -private TestNode CreateOutputUpdateNode(TestContext testContext, string? output, string? error) -{ - var properties = new List - { - InProgressTestNodeStateProperty.CachedInstance - }; - - if (!string.IsNullOrEmpty(output)) - properties.Add(new StandardOutputProperty(output)); - - if (!string.IsNullOrEmpty(error)) - properties.Add(new StandardErrorProperty(error)); - - return new TestNode - { - Uid = new TestNodeUid(testContext.TestDetails.TestId), - DisplayName = testContext.GetDisplayName(), - Properties = new PropertyBag(properties) - }; -} -``` - -### Registration - -In `TUnitServiceProvider.cs`: - -```csharp -// After existing sink registrations -if (VerbosityService.IsIdeClient) -{ - TUnitLoggerFactory.AddSink(new IdeStreamingSink( - MessageBus, - context.Request.Session.SessionUid)); -} -``` - -## Files to Modify - -| File | Change | -|------|--------| -| `TUnit.Engine/Logging/IdeStreamingSink.cs` | **Create** - new sink implementation | -| `TUnit.Engine/Framework/TUnitServiceProvider.cs` | **Modify** - register sink for IDE clients | - -## Testing Strategy - -1. Manual testing in Visual Studio and Rider -2. Verify output appears during long-running tests -3. Verify no duplication at test completion -4. Verify console mode is unaffected - -## Related - -- PR #4493 - Extensible log sink architecture -- Issue #4478 - Original user feature request