diff --git a/docs/docs/guides/html-report.md b/docs/docs/guides/html-report.md
index 325e24cd760..16c02002948 100644
--- a/docs/docs/guides/html-report.md
+++ b/docs/docs/guides/html-report.md
@@ -21,7 +21,7 @@ The OS and runtime version are included automatically so that matrix builds (mul
Open it in any modern browser. The report is fully self-contained (single HTML file) and works offline.
-A machine-readable JSON sidecar (`{AssemblyName}-{os}-{tfm}.tunit-report.json`) is written alongside the HTML report. It powers [report aggregation](/docs/guides/report-aggregation) — merging reports from multiple test projects into one — and can be disabled with `TUNIT_DISABLE_JSON_REPORT=true`.
+A machine-readable JSON sidecar (`{AssemblyName}-{os}-{tfm}.tunit-report.json`) is written alongside the HTML report. It powers [report aggregation](/docs/guides/report-aggregation) — merging reports from multiple test projects into one — and can be disabled with `TUNIT_DISABLE_JSON_REPORT=true` or `context.Settings.Reporting.JsonReportEnabled = false`.
Running many test projects and want **one combined report instead of one per project**? See [Aggregated Reports](/docs/guides/report-aggregation).
@@ -53,6 +53,8 @@ export TUNIT_DISABLE_HTML_REPORTER=true
Accepts: `true`, `1`, `yes` (case-insensitive).
+For version-controlled project configuration, set `context.Settings.Reporting.HtmlReportEnabled = false` in a `[Before(HookType.TestDiscovery)]` hook instead.
+
### Deprecated: `--report-html` Flag
The `--report-html` flag is deprecated since the report is now generated by default. Using it will show a deprecation warning but will not cause an error.
@@ -135,6 +137,8 @@ This is useful if you:
The report file and the `GITHUB_STEP_SUMMARY` are still generated.
+This can also be configured in code with `context.Settings.Reporting.ArtifactUploadEnabled = false`.
+
### Viewing the Report
After the workflow run completes:
@@ -234,7 +238,7 @@ The collector uses **smart sampling**: spans from known test traces are fully re
### Report Not Generated
-- Check that `TUNIT_DISABLE_HTML_REPORTER` is not set in your environment
+- Check that `TUNIT_DISABLE_HTML_REPORTER` is not set and `context.Settings.Reporting.HtmlReportEnabled` is not `false`
- Verify that the `TestResults/` directory is writable
- Check the console output for any warning messages about report generation failures
diff --git a/docs/docs/guides/report-aggregation.md b/docs/docs/guides/report-aggregation.md
index 6cc1e5881b6..f399d78b215 100644
--- a/docs/docs/guides/report-aggregation.md
+++ b/docs/docs/guides/report-aggregation.md
@@ -12,7 +12,7 @@ Report aggregation merges all of that into **one combined HTML report and one Gi
## How It Works
-1. Alongside every HTML report, TUnit writes a machine-readable sidecar: `{AssemblyName}-{os}-{tfm}.tunit-report.json`. This is on by default (disable with `TUNIT_DISABLE_JSON_REPORT=true`).
+1. Alongside every HTML report, TUnit writes a machine-readable sidecar: `{AssemblyName}-{os}-{tfm}.tunit-report.json`. This is on by default (disable with `TUNIT_DISABLE_JSON_REPORT=true` or `context.Settings.Reporting.JsonReportEnabled = false`).
2. With aggregation enabled, each test process also copies its sidecar into a directory shared by all sibling processes.
3. As each process finishes, it takes a cross-process lock, reads *all* sidecars present so far, and regenerates the merged HTML report and the summary block. The last process to finish naturally leaves the complete aggregate — no process ever needs to know whether it is the last one.
@@ -166,11 +166,11 @@ tunit-report merge --directory [options]
| --- | --- |
| `TUNIT_AGGREGATE_REPORTS` | Unset (default) — cooperative merge wherever a shared directory is resolvable (GitHub Actions, or explicit `TUNIT_AGGREGATE_DIR`); silently off otherwise. `defer` — persist sidecars + merged HTML only; no summary blocks (multi-step scenarios). `off` (also `false`/`0`/`no`/`disabled`/`none`) — no aggregation. |
| `TUNIT_AGGREGATE_DIR` | Shared directory for sidecars and the merged report. Required outside GitHub Actions; optional override on GitHub Actions. |
-| `TUNIT_DISABLE_JSON_REPORT` | Disables the JSON sidecar written next to the HTML report. Note: sidecars are what aggregation and `tunit-report` consume. |
+| `TUNIT_DISABLE_JSON_REPORT` | Disables the JSON sidecar written next to the HTML report. Programmatic equivalent: `context.Settings.Reporting.JsonReportEnabled = false`. Note: sidecars are what aggregation and `tunit-report` consume. |
## Notes & Limitations
-- Aggregation is driven by the HTML reporter's data pipeline — if you set `TUNIT_DISABLE_HTML_REPORTER`, no sidecars are produced and there is nothing to merge.
+- Aggregation is driven by the HTML reporter's data pipeline — if you set `TUNIT_DISABLE_HTML_REPORTER` or `context.Settings.Reporting.HtmlReportEnabled = false`, no sidecars are produced and there is nothing to merge.
- With cooperative mode (the default) across *multiple steps in the same job*, each step appends its own progressively-larger block (earlier steps' blocks can't be rewritten). Use `defer` + the tool for that layout, or `off` to restore per-suite blocks.
- Suites are identified per assembly + OS + TFM, so multi-targeted projects appear as separate rows (e.g. `MyTests (.NET 8.0.x)` / `MyTests (.NET 9.0.x)`).
- The GitHub step summary is capped at 1 MB by GitHub; the aggregated block replaces N per-suite blocks, so it usually *reduces* summary size.
diff --git a/docs/docs/reference/command-line-flags.md b/docs/docs/reference/command-line-flags.md
index 2e86de5f3e6..ef2d60f2ca9 100644
--- a/docs/docs/reference/command-line-flags.md
+++ b/docs/docs/reference/command-line-flags.md
@@ -137,7 +137,8 @@ Please note that for the coverage and trx report, you need to install [additiona
--report-html
(Deprecated) The HTML report is now generated by default.
- Disable it with the TUNIT_DISABLE_HTML_REPORTER environment variable.
+ Disable it with TUNIT_DISABLE_HTML_REPORTER or set
+ context.Settings.Reporting.HtmlReportEnabled = false.
--report-html-filename
Path for the HTML test report file
diff --git a/docs/docs/reference/environment-variables.md b/docs/docs/reference/environment-variables.md
index 0f32f77ac35..f78d7e2b142 100644
--- a/docs/docs/reference/environment-variables.md
+++ b/docs/docs/reference/environment-variables.md
@@ -60,9 +60,11 @@ Accepts truthy values: `true`, `1`, `yes` (case-insensitive).
**Use case:** When you don't need the HTML report or want to reduce disk I/O. The report is written to `TestResults/{AssemblyName}-report.html` by default.
+**Programmatic equivalent:** `context.Settings.Reporting.HtmlReportEnabled = false`
+
### TUNIT_DISABLE_ARTIFACT_UPLOAD
-Skips the autmoatic upload of the html report but still generates the files.
+Skips automatic upload of the HTML report but still generates the files.
```bash
export TUNIT_DISABLE_ARTIFACT_UPLOAD=true
@@ -77,6 +79,20 @@ GitHub changed the server-side behaviour of `upload-artifacts` since v4 and hasn
Forgejo and gitea don't implement this new artifact endpoint but the runners set `GITHUB_ACTIONS=true`.
In this case it attempts to upload the report a few times until it eventually backs off after 30s.
+**Programmatic equivalent:** `context.Settings.Reporting.ArtifactUploadEnabled = false`
+
+### TUNIT_DISABLE_JSON_REPORT
+
+Disables the machine-readable JSON sidecar written alongside the HTML report.
+
+```bash
+export TUNIT_DISABLE_JSON_REPORT=true
+```
+
+Accepts truthy values: `true`, `1`, `yes` (case-insensitive).
+
+**Programmatic equivalent:** `context.Settings.Reporting.JsonReportEnabled = false`
+
### TUNIT_DISABLE_JUNIT_REPORTER
Disables the JUnit XML reporter.
@@ -284,8 +300,9 @@ When the same setting is configured in multiple places, TUnit follows this prior
| `TUNIT_DISABLE_GITHUB_REPORTER` | - | Disables GitHub reporter |
| `TUNIT_DISABLE_JUNIT_REPORTER` | - | Disables JUnit reporter |
| `TUNIT_ENABLE_JUNIT_REPORTER` | - | Enables JUnit reporter |
-| `TUNIT_DISABLE_HTML_REPORTER` | - | Disables HTML report generation |
-| `TUNIT_DISABLE_ARTIFACT_UPLOAD` | - | Keeps the HTML report file but skips the GitHub Actions artifact upload |
+| `TUNIT_DISABLE_HTML_REPORTER` | - | Disables HTML report generation (`context.Settings.Reporting.HtmlReportEnabled = false`) |
+| `TUNIT_DISABLE_JSON_REPORT` | - | Disables the machine-readable JSON sidecar (`context.Settings.Reporting.JsonReportEnabled = false`) |
+| `TUNIT_DISABLE_ARTIFACT_UPLOAD` | - | Keeps the HTML report file but skips the GitHub Actions artifact upload (`context.Settings.Reporting.ArtifactUploadEnabled = false`) |
| `JUNIT_XML_OUTPUT_PATH` | - | JUnit output path |
| `TUNIT_MAX_PARALLEL_TESTS` | `--maximum-parallel-tests` | Max parallel tests |
| `TUNIT_EXECUTION_MODE` | `--reflection` | Selects source-generation (`sourcegeneration`/`aot`) or `reflection` execution mode |
diff --git a/docs/docs/reference/programmatic-configuration.md b/docs/docs/reference/programmatic-configuration.md
index 9c08e4973e6..ad9ee85c755 100644
--- a/docs/docs/reference/programmatic-configuration.md
+++ b/docs/docs/reference/programmatic-configuration.md
@@ -15,6 +15,7 @@ Settings are organized into logical groups:
- `Parallelism` — concurrent test execution limits
- `Execution` — runtime behavior such as fail-fast
- `Display` — output and display options
+- `Reporting` — HTML report generation and publishing
- `Mocks` — defaults for TUnit.Mocks when the package is referenced
## Usage
@@ -33,6 +34,7 @@ public class TestSetup
context.Settings.Timeouts.DefaultTestTimeout = TimeSpan.FromMinutes(5);
context.Settings.Timeouts.DefaultHookTimeout = TimeSpan.FromMinutes(2);
context.Settings.Execution.FailFast = true;
+ context.Settings.Reporting.HtmlReportEnabled = false;
context.Settings.Mocks.DefaultMode = MockBehavior.Strict;
return Task.CompletedTask;
@@ -73,6 +75,16 @@ Settings are accessed exclusively through `context.Settings` in the discovery ho
|---|---|---|---|
| `FailFast` | `bool` | `false` | Cancels the remaining test run after the first test failure. |
+### `context.Settings.Reporting`
+
+| Property | Type | Default | Description |
+|---|---|---|---|
+| `HtmlReportEnabled` | `bool` | `true` | Generates the HTML test report. |
+| `JsonReportEnabled` | `bool` | `true` | Generates the machine-readable JSON sidecar used by report aggregation. |
+| `ArtifactUploadEnabled` | `bool` | `true` | Uploads the HTML report as an artifact when supported by the CI environment. |
+
+The corresponding `TUNIT_DISABLE_HTML_REPORTER`, `TUNIT_DISABLE_JSON_REPORT`, and `TUNIT_DISABLE_ARTIFACT_UPLOAD` environment variables take precedence over these values.
+
### `context.Settings.Mocks`
Available when `TUnit.Mocks` is referenced.
diff --git a/src/TUnit.Core/Settings/ReportingSettings.cs b/src/TUnit.Core/Settings/ReportingSettings.cs
new file mode 100644
index 00000000000..a198a358b27
--- /dev/null
+++ b/src/TUnit.Core/Settings/ReportingSettings.cs
@@ -0,0 +1,28 @@
+namespace TUnit.Core.Settings;
+
+///
+/// Controls built-in report generation and publishing.
+///
+public sealed class ReportingSettings
+{
+ internal ReportingSettings() { }
+
+ ///
+ /// Whether to generate the HTML test report. Default: true.
+ /// Precedence: TUNIT_DISABLE_HTML_REPORTER → TUnitSettings → built-in default.
+ ///
+ public bool HtmlReportEnabled { get; set; } = true;
+
+ ///
+ /// Whether to generate the machine-readable JSON report sidecar. Default: true.
+ /// Precedence: TUNIT_DISABLE_JSON_REPORT → TUnitSettings → built-in default.
+ ///
+ public bool JsonReportEnabled { get; set; } = true;
+
+ ///
+ /// Whether to upload the HTML report as an artifact when supported by the CI environment.
+ /// Default: true.
+ /// Precedence: TUNIT_DISABLE_ARTIFACT_UPLOAD → TUnitSettings → built-in default.
+ ///
+ public bool ArtifactUploadEnabled { get; set; } = true;
+}
diff --git a/src/TUnit.Core/Settings/TUnitSettings.cs b/src/TUnit.Core/Settings/TUnitSettings.cs
index cc15574a89b..7ca50b384a0 100644
--- a/src/TUnit.Core/Settings/TUnitSettings.cs
+++ b/src/TUnit.Core/Settings/TUnitSettings.cs
@@ -38,4 +38,9 @@ internal TUnitSettings() { }
/// Controls test run behavior.
///
public ExecutionSettings Execution { get; } = new();
+
+ ///
+ /// Controls report generation and publishing.
+ ///
+ public ReportingSettings Reporting { get; } = new();
}
diff --git a/src/TUnit.Engine/Reporters/Aggregation/ReportAggregator.cs b/src/TUnit.Engine/Reporters/Aggregation/ReportAggregator.cs
index acba21e926d..72aac1d2ade 100644
--- a/src/TUnit.Engine/Reporters/Aggregation/ReportAggregator.cs
+++ b/src/TUnit.Engine/Reporters/Aggregation/ReportAggregator.cs
@@ -39,9 +39,9 @@ internal sealed class ReportAggregator
private const string SidecarSearchPattern = "*" + ReportDataJson.SidecarExtension;
private const string LockFileName = ".tunit-aggregate.lock";
- // Lock contention is expected (N processes finishing together, each holding the lock
- // for a full merge), so wait far longer than the file-write retry defaults.
- private const int LockMaxAttempts = 60;
+ // Bound lock contention to roughly ten seconds. Reporting must never hang test-suite
+ // completion; timed-out writers leave their sidecar for a later aggregate refresh.
+ private const int LockMaxAttempts = 30;
private const int LockRetryDelayMs = 250;
internal AggregationMode Mode { get; }
@@ -130,12 +130,107 @@ internal string WriteSidecar(byte[] sidecarUtf8Json, string assemblyName, string
{
System.IO.Directory.CreateDirectory(Directory);
- var fileName = $"{PathValidator.SanitizeFileName(assemblyName)}-{ShortHash(suiteSalt)}{ReportDataJson.SidecarExtension}";
- var path = Path.Combine(Directory, fileName);
+ var path = GetSidecarPath(assemblyName, suiteSalt);
AtomicFile.WriteAllBytes(path, sidecarUtf8Json);
return path;
}
+ internal void WritePendingSidecar(byte[] sidecarUtf8Json, string assemblyName, string suiteSalt)
+ {
+ System.IO.Directory.CreateDirectory(Directory);
+ var path = GetPendingSidecarPath(assemblyName, suiteSalt);
+ AtomicFile.WriteAllBytes(path, sidecarUtf8Json);
+ }
+
+ internal void DeletePendingSidecar(string assemblyName, string suiteSalt)
+ {
+ var path = GetPendingSidecarPath(assemblyName, suiteSalt);
+ File.Delete(path);
+ }
+
+ internal string? ReadEffectiveSidecarGeneration(string assemblyName, string suiteSalt)
+ {
+ var pendingPath = GetPendingSidecarPath(assemblyName, suiteSalt);
+ var sidecarPath = File.Exists(pendingPath)
+ ? pendingPath
+ : GetSidecarPath(assemblyName, suiteSalt);
+ return File.Exists(sidecarPath)
+ ? ReportDataJson.GetPublicationGeneration(File.ReadAllBytes(sidecarPath))
+ : null;
+ }
+
+ internal void ExcludeSidecar(string assemblyName, string suiteSalt)
+ {
+ System.IO.Directory.CreateDirectory(Directory);
+ var currentGeneration = ReadEffectiveSidecarGeneration(assemblyName, suiteSalt);
+ AtomicFile.WriteAllText(GetExclusionMarkerPath(assemblyName, suiteSalt), currentGeneration ?? "");
+ }
+
+ internal void ExcludeSidecarIfGenerationMatches(string assemblyName, string suiteSalt, string? expectedGeneration)
+ {
+ System.IO.Directory.CreateDirectory(Directory);
+ var currentGeneration = ReadEffectiveSidecarGeneration(assemblyName, suiteSalt);
+ if (expectedGeneration is null
+ ? currentGeneration is not null
+ : !expectedGeneration.Equals(currentGeneration, StringComparison.Ordinal))
+ {
+ return;
+ }
+
+ AtomicFile.WriteAllText(GetExclusionMarkerPath(assemblyName, suiteSalt), currentGeneration ?? "");
+ }
+
+ internal void IncludeSidecar(string assemblyName, string suiteSalt)
+ {
+ File.Delete(GetExclusionMarkerPath(assemblyName, suiteSalt));
+ }
+
+ internal IDisposable BeginSidecarPublication(string assemblyName, string suiteSalt)
+ {
+ System.IO.Directory.CreateDirectory(Directory);
+ var lockPath = GetPublishingMarkerPath(assemblyName, suiteSalt);
+ return new FileStream(lockPath, FileMode.OpenOrCreate, FileAccess.ReadWrite, FileShare.None);
+ }
+
+ internal IDisposable? TryAcquireSidecarPublication(string assemblyName, string suiteSalt)
+ {
+ try
+ {
+ return BeginSidecarPublication(assemblyName, suiteSalt);
+ }
+ catch (Exception ex) when (ex is IOException or UnauthorizedAccessException)
+ {
+ return null;
+ }
+ }
+
+ internal async Task AcquireSidecarPublicationAsync(
+ string assemblyName,
+ string suiteSalt,
+ CancellationToken cancellationToken)
+ {
+ System.IO.Directory.CreateDirectory(Directory);
+ return await AcquireFileLockAsync(
+ GetPublishingMarkerPath(assemblyName, suiteSalt),
+ cancellationToken,
+ "Warning: Report sidecar publication lock timed out; keeping the local per-suite report.");
+ }
+
+ internal bool HasSidecarState(string assemblyName, string suiteSalt)
+ {
+ if (!System.IO.Directory.Exists(Directory))
+ {
+ return false;
+ }
+
+ var sidecarPath = GetSidecarPath(assemblyName, suiteSalt);
+ var pendingPath = GetPendingSidecarPath(assemblyName, suiteSalt);
+ return File.Exists(sidecarPath)
+ || File.Exists(pendingPath)
+ || File.Exists(sidecarPath + ReportDataJson.SidecarExclusionExtension)
+ || File.Exists(sidecarPath + ReportDataJson.SidecarPublishingExtension);
+ }
+
///
/// Reads every sidecar currently present in the shared directory. Unreadable or
/// foreign files are skipped — a crashed sibling must not break the merge.
@@ -154,12 +249,20 @@ internal List ReadAllSidecars()
// as the tunit-report tool does.
var seenDigests = new HashSet(StringComparer.Ordinal);
using var sha = SHA256.Create();
- foreach (var file in System.IO.Directory.GetFiles(Directory, SidecarSearchPattern))
+ var effectiveSidecars = ReportDataJson.SelectEffectiveSidecars(
+ System.IO.Directory.GetFiles(Directory, SidecarSearchPattern));
+ foreach (var file in effectiveSidecars)
{
try
{
+ if (ReportDataJson.IsSidecarPublicationInProgress(file))
+ {
+ continue;
+ }
+
var bytes = File.ReadAllBytes(file);
- if (seenDigests.Add(Convert.ToBase64String(sha.ComputeHash(bytes)))
+ if (!ReportDataJson.IsSidecarExcluded(file, bytes)
+ && seenDigests.Add(Convert.ToBase64String(sha.ComputeHash(bytes)))
&& ReportDataJson.TryDeserialize((ReadOnlyMemory)bytes) is { } data)
{
results.Add(data);
@@ -178,14 +281,22 @@ internal List ReadAllSidecars()
///
/// Acquires the cross-process aggregation lock. Every writer performs its whole
/// read-merge-write cycle under this lock, so merges never interleave. Returns
- /// when the lock cannot be acquired within the timeout;
- /// callers should then skip merging (a later sibling will produce a fresher merge).
+ /// after a bounded wait so reporting cannot hang the run.
///
internal async Task AcquireLockAsync(CancellationToken cancellationToken)
{
System.IO.Directory.CreateDirectory(Directory);
- var lockPath = Path.Combine(Directory, LockFileName);
+ return await AcquireFileLockAsync(
+ Path.Combine(Directory, LockFileName),
+ cancellationToken,
+ "Warning: Report aggregation lock timed out; keeping per-suite reports and deferring aggregate refresh.");
+ }
+ private static async Task AcquireFileLockAsync(
+ string lockPath,
+ CancellationToken cancellationToken,
+ string timeoutWarning)
+ {
for (var attempt = 1; attempt <= LockMaxAttempts; attempt++)
{
cancellationToken.ThrowIfCancellationRequested();
@@ -195,8 +306,6 @@ internal List ReadAllSidecars()
}
catch (Exception ex) when (ex is IOException or UnauthorizedAccessException)
{
- // Contention (or a permissions hiccup) on the final attempt must still fall
- // through to the graceful "skip this merge" path, never escape as a throw.
if (attempt == LockMaxAttempts)
{
break;
@@ -206,7 +315,7 @@ internal List ReadAllSidecars()
}
}
- Console.WriteLine("Warning: Could not acquire the report aggregation lock; skipping merge for this process.");
+ Console.WriteLine(timeoutWarning);
return null;
}
@@ -231,4 +340,23 @@ private static string ShortHash(string value)
var hash = sha.ComputeHash(Encoding.UTF8.GetBytes(value));
return BitConverter.ToString(hash, 0, 4).Replace("-", "").ToLowerInvariant();
}
+
+ private string GetExclusionMarkerPath(string assemblyName, string suiteSalt)
+ {
+ return GetSidecarPath(assemblyName, suiteSalt) + ReportDataJson.SidecarExclusionExtension;
+ }
+
+ private string GetPublishingMarkerPath(string assemblyName, string suiteSalt)
+ {
+ return GetSidecarPath(assemblyName, suiteSalt) + ReportDataJson.SidecarPublishingExtension;
+ }
+
+ private string GetPendingSidecarPath(string assemblyName, string suiteSalt)
+ => ReportDataJson.GetPendingSidecarPath(GetSidecarPath(assemblyName, suiteSalt));
+
+ private string GetSidecarPath(string assemblyName, string suiteSalt)
+ {
+ var fileName = $"{PathValidator.SanitizeFileName(assemblyName)}-{ShortHash(suiteSalt)}{ReportDataJson.SidecarExtension}";
+ return Path.Combine(Directory, fileName);
+ }
}
diff --git a/src/TUnit.Engine/Reporters/Aggregation/ReportDataJson.cs b/src/TUnit.Engine/Reporters/Aggregation/ReportDataJson.cs
index ee2ebecf3dd..2088bb690b2 100644
--- a/src/TUnit.Engine/Reporters/Aggregation/ReportDataJson.cs
+++ b/src/TUnit.Engine/Reporters/Aggregation/ReportDataJson.cs
@@ -25,10 +25,108 @@ internal static class ReportDataJson
/// File extension shared by every sidecar so aggregators can discover them.
internal const string SidecarExtension = ".tunit-report.json";
+ internal const string SidecarPendingSegment = ".pending";
+ internal const string SidecarExclusionExtension = ".excluded";
+ internal const string SidecarPublishingExtension = ".publishing";
/// Merged HTML report filename, shared by the engine and the tool's default output.
internal const string MergedReportFileName = "merged-report.html";
+ internal static bool IsSidecarExcluded(string sidecarPath, ReadOnlySpan sidecarUtf8Json)
+ {
+ var canonicalPath = GetCanonicalSidecarPath(sidecarPath);
+ var exclusionPath = canonicalPath + SidecarExclusionExtension;
+ if (!File.Exists(exclusionPath))
+ {
+ return false;
+ }
+
+ var generation = GetPublicationGeneration(sidecarUtf8Json);
+ if (generation is null)
+ {
+ // Exclusions created for sidecars from before generation tracking remain valid.
+ return true;
+ }
+
+ try
+ {
+ return File.ReadAllText(exclusionPath).Equals(generation, StringComparison.Ordinal);
+ }
+ catch (Exception ex) when (ex is IOException or UnauthorizedAccessException)
+ {
+ return true;
+ }
+ }
+
+ internal static bool IsSidecarPublicationInProgress(string sidecarPath)
+ {
+ // Pending sidecars are complete atomic files written only after a publisher's
+ // per-suite lock wait expires. The canonical publisher cannot mutate them.
+ if (IsPendingSidecar(sidecarPath))
+ {
+ return false;
+ }
+
+ var publicationLockPath = sidecarPath + SidecarPublishingExtension;
+ if (!File.Exists(publicationLockPath))
+ {
+ return false;
+ }
+
+ try
+ {
+ // Lock file is stable: deleting it after releasing the handle lets another
+ // process lock the old inode while a third process creates and locks a new one.
+ using (new FileStream(publicationLockPath, FileMode.Open, FileAccess.ReadWrite, FileShare.None))
+ {
+ }
+ return false;
+ }
+ catch (Exception ex) when (ex is IOException or UnauthorizedAccessException)
+ {
+ return true;
+ }
+ }
+
+ internal static string? GetPublicationGeneration(ReadOnlySpan sidecarUtf8Json)
+ {
+ try
+ {
+ var reader = new Utf8JsonReader(sidecarUtf8Json);
+ while (reader.Read())
+ {
+ if (reader.TokenType == JsonTokenType.PropertyName
+ && reader.ValueTextEquals("publicationGeneration")
+ && reader.Read()
+ && reader.TokenType == JsonTokenType.String)
+ {
+ return reader.GetString();
+ }
+ }
+ }
+ catch (JsonException)
+ {
+ }
+
+ return null;
+ }
+
+ internal static bool IsPendingSidecar(string sidecarPath)
+ => sidecarPath.EndsWith(SidecarPendingSegment + SidecarExtension, StringComparison.Ordinal);
+
+ internal static string GetCanonicalSidecarPath(string sidecarPath)
+ => IsPendingSidecar(sidecarPath)
+ ? sidecarPath[..^(SidecarPendingSegment.Length + SidecarExtension.Length)] + SidecarExtension
+ : sidecarPath;
+
+ internal static string GetPendingSidecarPath(string canonicalSidecarPath)
+ => canonicalSidecarPath[..^SidecarExtension.Length] + SidecarPendingSegment + SidecarExtension;
+
+ internal static IEnumerable SelectEffectiveSidecars(IEnumerable sidecarPaths)
+ => sidecarPaths
+ .GroupBy(GetCanonicalSidecarPath, StringComparer.Ordinal)
+ .Select(group => group.FirstOrDefault(IsPendingSidecar) ?? group.First());
+
///
/// Serializes straight to UTF-8 bytes — callers write the same payload to more than one
/// file, so producing bytes once avoids a UTF-8 → string → UTF-8 round trip per copy.
@@ -50,6 +148,7 @@ private static void Write(Utf8JsonWriter w, ReportData data)
{
w.WriteStartObject();
w.WriteNumber("schemaVersion", SchemaVersion);
+ w.WriteString("publicationGeneration", data.PublicationGeneration ?? Guid.NewGuid().ToString("N"));
w.WriteString("assemblyName", data.AssemblyName);
w.WriteString("machineName", data.MachineName);
w.WriteString("timestamp", data.Timestamp);
@@ -322,6 +421,7 @@ private static bool IsMalformedSidecar(Exception ex)
return new ReportData
{
AssemblyName = GetString(root, "assemblyName") ?? "Unknown",
+ PublicationGeneration = GetString(root, "publicationGeneration"),
MachineName = GetString(root, "machineName") ?? "",
Timestamp = GetString(root, "timestamp") ?? "",
TUnitVersion = GetString(root, "tunitVersion") ?? "",
diff --git a/src/TUnit.Engine/Reporters/GitHubReporter.cs b/src/TUnit.Engine/Reporters/GitHubReporter.cs
index cf3c21d5b0b..9348d130211 100644
--- a/src/TUnit.Engine/Reporters/GitHubReporter.cs
+++ b/src/TUnit.Engine/Reporters/GitHubReporter.cs
@@ -470,6 +470,16 @@ public async Task AfterRunAsync(int exitCode, CancellationToken cancellation)
///
internal bool SuppressPerSuiteSummary { get; set; }
+ internal void ResetSessionState()
+ {
+ _latestUpdates.Clear();
+ _terminalStateCounts.Clear();
+ ArtifactUrl = null;
+ ShowArtifactUploadTip = false;
+ SuppressPerSuiteSummary = false;
+ _runStopwatch = Stopwatch.StartNew();
+ }
+
///
/// Renders the aggregated block and rewrites the marked summary region. Called by
/// HtmlReporter inside the aggregation lock, so the whole merge is one lock cycle.
@@ -494,6 +504,14 @@ internal void WriteAggregatedSummary(IReadOnlyList suites, stri
GitHubSummaryRegion.ReplaceOrAppend(_outputSummaryFilePath, content, MaxFileSizeInBytes);
}
+ internal void ClearAggregatedSummary()
+ {
+ if (_outputSummaryFilePath is not null)
+ {
+ GitHubSummaryRegion.ReplaceOrAppend(_outputSummaryFilePath, string.Empty, MaxFileSizeInBytes);
+ }
+ }
+
private async Task WriteFile(string contents)
{
var fileInfo = new FileInfo(_outputSummaryFilePath);
diff --git a/src/TUnit.Engine/Reporters/Html/HtmlReportDataModel.cs b/src/TUnit.Engine/Reporters/Html/HtmlReportDataModel.cs
index eac0d3b23a7..95ec9473143 100644
--- a/src/TUnit.Engine/Reporters/Html/HtmlReportDataModel.cs
+++ b/src/TUnit.Engine/Reporters/Html/HtmlReportDataModel.cs
@@ -12,6 +12,10 @@ internal sealed class ReportData
[JsonPropertyName("assemblyName")]
public required string AssemblyName { get; init; }
+ /// Atomic sidecar generation used only by aggregation cleanup.
+ [JsonIgnore]
+ public string? PublicationGeneration { get; init; }
+
[JsonPropertyName("machineName")]
public required string MachineName { get; init; }
diff --git a/src/TUnit.Engine/Reporters/Html/HtmlReporter.cs b/src/TUnit.Engine/Reporters/Html/HtmlReporter.cs
index 4f3f4a51846..7ff1cf76618 100644
--- a/src/TUnit.Engine/Reporters/Html/HtmlReporter.cs
+++ b/src/TUnit.Engine/Reporters/Html/HtmlReporter.cs
@@ -12,6 +12,7 @@
using Microsoft.Testing.Platform.Services;
using Microsoft.Testing.Platform.TestHost;
using TUnit.Core;
+using TUnit.Core.Settings;
using TUnit.Engine.Configuration;
using TUnit.Engine.Constants;
using TUnit.Engine.Exceptions;
@@ -27,6 +28,8 @@ namespace TUnit.Engine.Reporters.Html;
internal sealed class HtmlReporter(IExtension extension) : IDataConsumer, IDataProducer, ITestHostApplicationLifetime, ITestSessionLifetimeHandler, IFilterReceiver, IDisposable
{
+ private const int HtmlReportEnabledUnresolved = -1;
+
// System.Text.Json's Utf8JsonWriter limits a single string token to int.MaxValue / 6 characters.
// Truncate large outputs early so report generation never fails for test suites with excessive logging.
internal const int MaxOutputLength = 1 * 1024 * 1024; // 1 MB
@@ -35,7 +38,9 @@ internal sealed class HtmlReporter(IExtension extension) : IDataConsumer, IDataP
private IMessageBus? _messageBus;
private string _resultsDirectory = "TestResults";
private readonly ConcurrentDictionary _updates = [];
+ private readonly object _htmlReportStateLock = new();
private GitHubReporter? _githubReporter;
+ private int _htmlReportEnabledAfterDiscovery = HtmlReportEnabledUnresolved;
#if NET
private ActivityCollector? _activityCollector;
@@ -43,7 +48,7 @@ internal sealed class HtmlReporter(IExtension extension) : IDataConsumer, IDataP
public async Task IsEnabledAsync()
{
- if (IsTruthyEnv(Environment.GetEnvironmentVariable(EnvironmentConstants.DisableHtmlReporter)))
+ if (!IsHtmlReportEnabled())
{
return false;
}
@@ -61,6 +66,11 @@ public async Task IsEnabledAsync()
public Task ConsumeAsync(IDataProducer dataProducer, IData value, CancellationToken cancellationToken)
{
+ if (!IsHtmlReportEnabledForRun())
+ {
+ return Task.CompletedTask;
+ }
+
var testNodeUpdateMessage = (TestNodeUpdateMessage)value;
// Keep only the update we'll report per test: a final-state update always wins over a
// non-final one, otherwise the latest wins. The engine emits a single final update per
@@ -89,10 +99,6 @@ private static bool HasFinalState(TestNodeUpdateMessage update)
public Task BeforeRunAsync(CancellationToken cancellationToken)
{
-#if NET
- _activityCollector = new ActivityCollector();
- _activityCollector.Start();
-#endif
return Task.CompletedTask;
}
@@ -100,21 +106,62 @@ public Task AfterRunAsync(int exitCode, CancellationToken cancellation)
=> Task.CompletedTask; // All work happens in OnTestSessionFinishingAsync.
public Task OnTestSessionStartingAsync(ITestSessionContext testSessionContext)
- => Task.CompletedTask;
+ {
+ lock (_htmlReportStateLock)
+ {
+ _updates.Clear();
+ _githubReporter?.ResetSessionState();
+ Volatile.Write(ref _htmlReportEnabledAfterDiscovery, HtmlReportEnabledUnresolved);
+#if NET
+ DisposeActivityCollection();
+ // Discovery hooks may re-enable reporting for this session. Start before
+ // discovery so those early spans are retained, then resolve the setting
+ // on the first post-discovery update.
+ StartActivityCollection();
+#endif
+ }
+
+ return Task.CompletedTask;
+ }
public async Task OnTestSessionFinishingAsync(ITestSessionContext testSessionContext)
{
try
{
#if NET
- _activityCollector?.Stop();
+ StopActivityCollection();
+#endif
+
+ if (!IsHtmlReportEnabledForRun())
+ {
+#if NET
+ TraceRegistry.Clear();
#endif
+ _updates.Clear();
+ var disabledOutputPath = _outputPath ?? GetDefaultOutputPath();
+ var aggregator = ReportAggregator.TryCreateFromEnvironment(Environment.GetEnvironmentVariable);
+ await DeleteSidecarsAndRefreshAggregateAsync(
+ GetAssemblyName(),
+ disabledOutputPath,
+ aggregator);
+ return;
+ }
if (_updates.Count == 0)
{
#if NET
TraceRegistry.Clear();
#endif
+ if (!IsJsonReportEnabled())
+ {
+ var emptyOutputPath = _outputPath ?? GetDefaultOutputPath();
+ var aggregator = ReportAggregator.TryCreateFromEnvironment(Environment.GetEnvironmentVariable);
+ await DeleteSidecarsAndRefreshAggregateAsync(
+ GetAssemblyName(),
+ emptyOutputPath,
+ aggregator);
+ }
+
return;
}
@@ -163,26 +210,36 @@ public async Task OnTestSessionFinishingAsync(ITestSessionContext testSessionCon
{
Console.WriteLine($"Warning: HTML report generation failed: {ex.Message}");
}
+ finally
+ {
+#if NET
+ DisposeActivityCollection();
+#endif
+ }
}
- private async Task TryWriteSidecarAndAggregateAsync(ReportData reportData, string htmlOutputPath, CancellationToken cancellationToken)
+ internal async Task TryWriteSidecarAndAggregateAsync(ReportData reportData, string htmlOutputPath, CancellationToken cancellationToken)
{
+ var aggregator = ReportAggregator.TryCreateFromEnvironment(Environment.GetEnvironmentVariable);
+
+ if (!IsJsonReportEnabled())
+ {
+ await DeleteSidecarsAndRefreshAggregateAsync(reportData.AssemblyName, htmlOutputPath, aggregator);
+ return;
+ }
+
// Serialized once; the same bytes back both the local sidecar and the shared copy.
var sidecarBytes = ReportDataJson.SerializeToBytes(reportData);
- if (!IsTruthyEnv(Environment.GetEnvironmentVariable(EnvironmentConstants.DisableJsonReport)))
+ try
{
- try
- {
- AtomicFile.WriteAllBytes(GetSidecarPath(htmlOutputPath), sidecarBytes);
- }
- catch (Exception ex)
- {
- Console.WriteLine($"Warning: Failed to write JSON report sidecar: {ex.Message}");
- }
+ AtomicFile.WriteAllBytes(GetSidecarPath(htmlOutputPath), sidecarBytes);
+ }
+ catch (Exception ex)
+ {
+ Console.WriteLine($"Warning: Failed to write JSON report sidecar: {ex.Message}");
}
- var aggregator = ReportAggregator.TryCreateFromEnvironment(Environment.GetEnvironmentVariable);
if (aggregator is null)
{
return;
@@ -190,44 +247,152 @@ private async Task TryWriteSidecarAndAggregateAsync(ReportData reportData, strin
try
{
- aggregator.WriteSidecar(sidecarBytes, reportData.AssemblyName, suiteSalt: htmlOutputPath);
+ // Serialize the shared write with cleanup so cleanup cannot mistake an in-flight
+ // enabled generation for the stale generation it intended to remove.
+ using var publicationMarker = await aggregator.AcquireSidecarPublicationAsync(
+ reportData.AssemblyName,
+ htmlOutputPath,
+ CancellationToken.None);
+ if (publicationMarker is null)
+ {
+ // Publication-lock contention must not discard completed suite results.
+ // A separate pending slot cannot overwrite the active publisher's canonical
+ // generation and remains available to this or any later aggregate refresh.
+ aggregator.WritePendingSidecar(sidecarBytes, reportData.AssemblyName, suiteSalt: htmlOutputPath);
+ }
+ else
+ {
+ // Owning the lock makes it safe for a newer normal publication to replace
+ // any pending timeout result left by an earlier publisher.
+ aggregator.DeletePendingSidecar(reportData.AssemblyName, htmlOutputPath);
+ aggregator.WriteSidecar(sidecarBytes, reportData.AssemblyName, suiteSalt: htmlOutputPath);
+ aggregator.IncludeSidecar(reportData.AssemblyName, htmlOutputPath);
+ }
- // Aggregation is committed for this suite: whatever happens below, the classic
- // per-suite block must not be appended on top of the aggregated one. (If we
- // fail past this point, a sibling that merges after us still renders this
- // suite's results from the sidecar just written.)
- if (_githubReporter is not null)
+ if (aggregator.Mode == AggregationMode.Defer && _githubReporter is not null)
{
_githubReporter.SuppressPerSuiteSummary = true;
}
- // Every finishing process regenerates the merged outputs from all sidecars
- // present so far; the last one to finish leaves the complete aggregate.
- using var aggregationLock = await aggregator.AcquireLockAsync(cancellationToken);
+ IDisposable? aggregationLock;
+ try
+ {
+ aggregationLock = await aggregator.AcquireLockAsync(cancellationToken);
+ }
+ catch (OperationCanceledException)
+ {
+ aggregationLock = null;
+ }
+
if (aggregationLock is null)
{
return;
}
- var suites = aggregator.ReadAllSidecars();
- if (suites.Count == 0)
+ using (aggregationLock)
+ {
+ publicationMarker?.Dispose();
+
+ RefreshAggregatedOutputs(aggregator);
+ if (_githubReporter is not null)
+ {
+ _githubReporter.SuppressPerSuiteSummary = true;
+ }
+ }
+ }
+ catch (Exception ex)
+ {
+ Console.WriteLine($"Warning: Report aggregation failed: {ex.Message}");
+ }
+ }
+
+ private async Task DeleteSidecarsAndRefreshAggregateAsync(
+ string assemblyName,
+ string htmlOutputPath,
+ ReportAggregator? aggregator)
+ {
+ TryDeleteReportFile(() => File.Delete(GetSidecarPath(htmlOutputPath)));
+
+ if (aggregator is null)
+ {
+ return;
+ }
+
+ if (!aggregator.HasSidecarState(assemblyName, htmlOutputPath))
+ {
+ return;
+ }
+
+ try
+ {
+ var expectedGeneration = aggregator.ReadEffectiveSidecarGeneration(assemblyName, htmlOutputPath);
+ using var publicationMarker = aggregator.TryAcquireSidecarPublication(assemblyName, htmlOutputPath);
+ if (publicationMarker is null)
+ {
+ // An enabled publication may already have installed its generation while
+ // holding this lock. Cleanup must not wait, acquire next, and hide it.
+ return;
+ }
+
+ // Exclude only the generation observed before the lock attempt. A publisher that
+ // won the per-suite lock in the meantime installed a newer generation and survives.
+ aggregator.ExcludeSidecarIfGenerationMatches(assemblyName, htmlOutputPath, expectedGeneration);
+
+ // The exclusion is durable and generation-scoped, so do not hold the per-suite
+ // lock during the bounded aggregate-lock wait. A staged enabled publication
+ // must be able to acquire it and clear the exclusion before either wait expires.
+ publicationMarker.Dispose();
+
+ // Cleanup must survive session cancellation or stale enabled-run sidecars can
+ // re-enter a sibling's aggregate. Lock acquisition remains time-bounded.
+ using var aggregationLock = await aggregator.AcquireLockAsync(CancellationToken.None);
+ if (aggregationLock is null)
{
return;
}
- aggregator.WriteMergedHtml(suites);
- Console.WriteLine($"Merged HTML test report ({suites.Count} {(suites.Count == 1 ? "suite" : "suites")} so far) written to: {aggregator.MergedReportPath}");
+ RefreshAggregatedOutputs(aggregator);
+ }
+ catch (Exception ex) when (ex is not OperationCanceledException)
+ {
+ Console.WriteLine($"Warning: Report aggregation cleanup failed: {ex.Message}");
+ }
+ }
- // The step summary is rewritten in the same lock cycle: one env parse, one
- // lock acquisition and one sidecar scan per process for both merged outputs.
+ private void RefreshAggregatedOutputs(ReportAggregator aggregator)
+ {
+ var suites = aggregator.ReadAllSidecars();
+ if (suites.Count == 0)
+ {
+ TryDeleteReportFile(() => File.Delete(aggregator.MergedReportPath));
if (aggregator.Mode == AggregationMode.Cooperative)
{
- _githubReporter?.WriteAggregatedSummary(suites, aggregator.MergedReportPath);
+ _githubReporter?.ClearAggregatedSummary();
}
+
+ return;
+ }
+
+ aggregator.WriteMergedHtml(suites);
+ Console.WriteLine($"Merged HTML test report ({suites.Count} {(suites.Count == 1 ? "suite" : "suites")} so far) written to: {aggregator.MergedReportPath}");
+
+ // The step summary is rewritten in the same lock cycle: one env parse, one
+ // lock acquisition and one sidecar scan per process for both merged outputs.
+ if (aggregator.Mode == AggregationMode.Cooperative)
+ {
+ _githubReporter?.WriteAggregatedSummary(suites, aggregator.MergedReportPath);
+ }
+ }
+
+ private static void TryDeleteReportFile(Action deleteFile)
+ {
+ try
+ {
+ deleteFile();
}
catch (Exception ex)
{
- Console.WriteLine($"Warning: Report aggregation failed: {ex.Message}");
+ Console.WriteLine($"Warning: Failed to remove disabled report file: {ex.Message}");
}
}
@@ -238,6 +403,53 @@ private static bool IsTruthyEnv(string? value)
value.Equals("1", StringComparison.Ordinal) ||
value.Equals("yes", StringComparison.OrdinalIgnoreCase));
+ internal static bool IsHtmlReportEnabled()
+ => !IsTruthyEnv(Environment.GetEnvironmentVariable(EnvironmentConstants.DisableHtmlReporter))
+ && TUnitSettings.Default.Reporting.HtmlReportEnabled;
+
+ internal static bool IsJsonReportEnabled()
+ => !IsTruthyEnv(Environment.GetEnvironmentVariable(EnvironmentConstants.DisableJsonReport))
+ && TUnitSettings.Default.Reporting.JsonReportEnabled;
+
+ internal static bool IsArtifactUploadEnabled()
+ => !IsTruthyEnv(Environment.GetEnvironmentVariable(EnvironmentConstants.DisableArtifactUpload))
+ && TUnitSettings.Default.Reporting.ArtifactUploadEnabled;
+
+ internal bool IsHtmlReportEnabledForRun()
+ {
+ var resolved = Volatile.Read(ref _htmlReportEnabledAfterDiscovery);
+ if (resolved != HtmlReportEnabledUnresolved)
+ {
+ return resolved == 1;
+ }
+
+ lock (_htmlReportStateLock)
+ {
+ resolved = Volatile.Read(ref _htmlReportEnabledAfterDiscovery);
+ if (resolved != HtmlReportEnabledUnresolved)
+ {
+ return resolved == 1;
+ }
+
+ var enabled = IsHtmlReportEnabled();
+
+#if NET
+ if (enabled)
+ {
+ StartActivityCollection();
+ }
+ else
+ {
+ DisposeActivityCollection();
+ TraceRegistry.Clear();
+ }
+#endif
+
+ Volatile.Write(ref _htmlReportEnabledAfterDiscovery, enabled ? 1 : 0);
+ return enabled;
+ }
+ }
+
// Default HTML report is "{name}-{os}-{tfm}-report.html"; the sidecar drops the
// "-report" stem so the default pair reads "{name}-{os}-{tfm}.tunit-report.json".
internal static string GetSidecarPath(string htmlOutputPath)
@@ -253,7 +465,7 @@ internal static string GetSidecarPath(string htmlOutputPath)
internal async Task PublishArtifactAsync(string outputPath, SessionUid sessionUid, CancellationToken cancellationToken)
{
- if (_messageBus is null)
+ if (_messageBus is null || !IsArtifactUploadEnabled())
{
return;
}
@@ -270,10 +482,34 @@ internal async Task PublishArtifactAsync(string outputPath, SessionUid sessionUi
public void Dispose()
{
#if NET
- _activityCollector?.Dispose();
+ DisposeActivityCollection();
#endif
}
+#if NET
+ internal bool HasActivityCollector => _activityCollector is not null;
+
+ private void StartActivityCollection()
+ {
+ if (_activityCollector is not null)
+ {
+ return;
+ }
+
+ _activityCollector = new ActivityCollector();
+ _activityCollector.Start();
+ }
+
+ internal void StopActivityCollection()
+ => _activityCollector?.Stop();
+
+ private void DisposeActivityCollection()
+ {
+ _activityCollector?.Dispose();
+ _activityCollector = null;
+ }
+#endif
+
public string? Filter { get; set; }
internal void SetOutputPath(string path)
@@ -307,7 +543,7 @@ internal void SetResultsDirectory(string path)
internal ReportData BuildReportData()
{
- var assemblyName = Assembly.GetEntryAssembly()?.GetName().Name ?? "TestResults";
+ var assemblyName = GetAssemblyName();
var tunitVersion = typeof(HtmlReporter).Assembly.GetName().Version?.ToString() ?? "unknown";
// Get the last update with a final state for each test
@@ -747,13 +983,16 @@ private static (string Status, ReportExceptionData? Exception, string? SkipReaso
private string GetDefaultOutputPath()
{
- var assemblyName = Assembly.GetEntryAssembly()?.GetName().Name ?? "TestResults";
+ var assemblyName = GetAssemblyName();
var sanitizedName = PathValidator.SanitizeFileName(assemblyName);
var os = GetShortOsName();
var tfm = GetShortFrameworkName();
return Path.GetFullPath(Path.Combine(_resultsDirectory, $"{sanitizedName}-{os}-{tfm}-report.html"));
}
+ private static string GetAssemblyName()
+ => Assembly.GetEntryAssembly()?.GetName().Name ?? "TestResults";
+
private static string GetShortOsName()
{
#if NET
@@ -857,7 +1096,7 @@ private static bool IsFileLocked(IOException exception)
return null;
}
- if (IsTruthyEnv(Environment.GetEnvironmentVariable(EnvironmentConstants.DisableArtifactUpload)))
+ if (!IsArtifactUploadEnabled())
{
return null;
}
diff --git a/src/TUnit.Reporting.Tool/Program.cs b/src/TUnit.Reporting.Tool/Program.cs
index b05183e5dc0..82caaeb626e 100644
--- a/src/TUnit.Reporting.Tool/Program.cs
+++ b/src/TUnit.Reporting.Tool/Program.cs
@@ -153,8 +153,17 @@ private static (List Suites, int Skipped) LoadSidecars(string direct
// abort the walk — SearchOption.AllDirectories would throw mid-enumeration, before
// the per-file guard below ever ran.
var enumeration = new EnumerationOptions { RecurseSubdirectories = true, IgnoreInaccessible = true };
- foreach (var file in Directory.EnumerateFiles(directory, "*" + ReportDataJson.SidecarExtension, enumeration))
+ var effectiveSidecars = ReportDataJson.SelectEffectiveSidecars(
+ Directory.EnumerateFiles(directory, "*" + ReportDataJson.SidecarExtension, enumeration));
+ foreach (var file in effectiveSidecars)
{
+ // This command is the designated final merge, so no publisher should still
+ // be active. Skip any suite whose stable publication lock is still held.
+ if (ReportDataJson.IsSidecarPublicationInProgress(file))
+ {
+ continue;
+ }
+
byte[] bytes;
try
{
@@ -168,6 +177,11 @@ private static (List Suites, int Skipped) LoadSidecars(string direct
continue;
}
+ if (ReportDataJson.IsSidecarExcluded(file, bytes))
+ {
+ continue;
+ }
+
if (!seenDigests.Add(Convert.ToBase64String(sha.ComputeHash(bytes))))
{
continue;
diff --git a/tests/TUnit.Engine.Tests/GitHubReporterTests.cs b/tests/TUnit.Engine.Tests/GitHubReporterTests.cs
index 38295421c26..d8c31e7d26b 100644
--- a/tests/TUnit.Engine.Tests/GitHubReporterTests.cs
+++ b/tests/TUnit.Engine.Tests/GitHubReporterTests.cs
@@ -3,6 +3,7 @@
using Shouldly;
using TUnit.Engine.Exceptions;
using TUnit.Engine.Reporters;
+using TUnit.Engine.Reporters.Aggregation;
namespace TUnit.Engine.Tests;
@@ -114,6 +115,42 @@ public async Task IsEnabledAsync_Should_Return_False_When_GITHUB_ACTIONS_Is_Not_
result.ShouldBeFalse();
}
+ [Test]
+ public async Task ClearAggregatedSummary_Removes_Stale_Content()
+ {
+ var (reporter, outputFile) = await SetupReporter();
+ GitHubSummaryRegion.ReplaceOrAppend(outputFile, "stale aggregate");
+
+ reporter.ClearAggregatedSummary();
+
+ File.ReadAllText(outputFile).ShouldNotContain("stale aggregate");
+ }
+
+ [Test]
+ public async Task ResetSessionState_Clears_Test_And_Presentation_State()
+ {
+ var (reporter, outputFile) = await SetupReporter();
+ await FeedTestMessages(reporter,
+ CreatePassedTestMessage("retry", "CurrentTest", "Tests"),
+ CreatePassedTestMessage("retry", "CurrentTest", "Tests"),
+ CreatePassedTestMessage("stale", "StaleTest", "Tests"));
+ reporter.ArtifactUrl = "https://example.com/old-artifact";
+ reporter.ShowArtifactUploadTip = true;
+ reporter.SuppressPerSuiteSummary = true;
+
+ reporter.ResetSessionState();
+ await FeedTestMessages(reporter, CreatePassedTestMessage("retry", "CurrentTest", "Tests"));
+ await reporter.AfterRunAsync(0, CancellationToken.None);
+
+ var output = await File.ReadAllTextAsync(outputFile);
+ output.ShouldContain("**1 tests**");
+ output.ShouldNotContain("StaleTest");
+ output.ShouldNotContain("flaky");
+ reporter.ArtifactUrl.ShouldBeNull();
+ reporter.ShowArtifactUploadTip.ShouldBeFalse();
+ reporter.SuppressPerSuiteSummary.ShouldBeFalse();
+ }
+
[Test]
public async Task AfterRunAsync_Groups_Failures_By_Exception_Type()
{
diff --git a/tests/TUnit.Engine.Tests/HtmlReporterConfigurationTests.cs b/tests/TUnit.Engine.Tests/HtmlReporterConfigurationTests.cs
new file mode 100644
index 00000000000..3b4e53c6d28
--- /dev/null
+++ b/tests/TUnit.Engine.Tests/HtmlReporterConfigurationTests.cs
@@ -0,0 +1,755 @@
+using Microsoft.Testing.Platform.Extensions.Messages;
+using Microsoft.Testing.Platform.TestHost;
+using Shouldly;
+using TUnit.Core.Settings;
+using TUnit.Engine.Reporters;
+using TUnit.Engine.Reporters.Aggregation;
+using TUnit.Engine.Reporters.Html;
+
+namespace TUnit.Engine.Tests;
+
+[NotInParallel]
+public class HtmlReporterConfigurationTests
+{
+ private bool _htmlReportEnabled;
+ private bool _jsonReportEnabled;
+ private bool _artifactUploadEnabled;
+ private string? _disableHtmlReporter;
+ private string? _disableJsonReport;
+ private string? _disableArtifactUpload;
+ private string? _aggregateReports;
+ private string? _aggregateDirectory;
+
+ [Before(HookType.Test)]
+ public void SnapshotConfiguration()
+ {
+ _htmlReportEnabled = TUnitSettings.Default.Reporting.HtmlReportEnabled;
+ _jsonReportEnabled = TUnitSettings.Default.Reporting.JsonReportEnabled;
+ _artifactUploadEnabled = TUnitSettings.Default.Reporting.ArtifactUploadEnabled;
+ _disableHtmlReporter = Environment.GetEnvironmentVariable("TUNIT_DISABLE_HTML_REPORTER");
+ _disableJsonReport = Environment.GetEnvironmentVariable("TUNIT_DISABLE_JSON_REPORT");
+ _disableArtifactUpload = Environment.GetEnvironmentVariable("TUNIT_DISABLE_ARTIFACT_UPLOAD");
+ _aggregateReports = Environment.GetEnvironmentVariable("TUNIT_AGGREGATE_REPORTS");
+ _aggregateDirectory = Environment.GetEnvironmentVariable("TUNIT_AGGREGATE_DIR");
+
+ Environment.SetEnvironmentVariable("TUNIT_DISABLE_HTML_REPORTER", null);
+ Environment.SetEnvironmentVariable("TUNIT_DISABLE_JSON_REPORT", null);
+ Environment.SetEnvironmentVariable("TUNIT_DISABLE_ARTIFACT_UPLOAD", null);
+ Environment.SetEnvironmentVariable("TUNIT_AGGREGATE_REPORTS", null);
+ Environment.SetEnvironmentVariable("TUNIT_AGGREGATE_DIR", null);
+ }
+
+ [After(HookType.Test)]
+ public void RestoreConfiguration()
+ {
+ TUnitSettings.Default.Reporting.HtmlReportEnabled = _htmlReportEnabled;
+ TUnitSettings.Default.Reporting.JsonReportEnabled = _jsonReportEnabled;
+ TUnitSettings.Default.Reporting.ArtifactUploadEnabled = _artifactUploadEnabled;
+ Environment.SetEnvironmentVariable("TUNIT_DISABLE_HTML_REPORTER", _disableHtmlReporter);
+ Environment.SetEnvironmentVariable("TUNIT_DISABLE_JSON_REPORT", _disableJsonReport);
+ Environment.SetEnvironmentVariable("TUNIT_DISABLE_ARTIFACT_UPLOAD", _disableArtifactUpload);
+ Environment.SetEnvironmentVariable("TUNIT_AGGREGATE_REPORTS", _aggregateReports);
+ Environment.SetEnvironmentVariable("TUNIT_AGGREGATE_DIR", _aggregateDirectory);
+ }
+
+ [Test]
+ public async Task Programmatic_Settings_Can_Disable_Html_Reporting_Features()
+ {
+ TUnitSettings.Default.Reporting.HtmlReportEnabled = false;
+ TUnitSettings.Default.Reporting.JsonReportEnabled = false;
+ TUnitSettings.Default.Reporting.ArtifactUploadEnabled = false;
+
+ await Assert.That(HtmlReporter.IsHtmlReportEnabled()).IsFalse();
+ await Assert.That(HtmlReporter.IsJsonReportEnabled()).IsFalse();
+ await Assert.That(HtmlReporter.IsArtifactUploadEnabled()).IsFalse();
+ }
+
+ [Test]
+ public async Task Disable_Environment_Variables_Take_Precedence()
+ {
+ Environment.SetEnvironmentVariable("TUNIT_DISABLE_HTML_REPORTER", "true");
+ Environment.SetEnvironmentVariable("TUNIT_DISABLE_JSON_REPORT", "1");
+ Environment.SetEnvironmentVariable("TUNIT_DISABLE_ARTIFACT_UPLOAD", "yes");
+
+ await Assert.That(HtmlReporter.IsHtmlReportEnabled()).IsFalse();
+ await Assert.That(HtmlReporter.IsJsonReportEnabled()).IsFalse();
+ await Assert.That(HtmlReporter.IsArtifactUploadEnabled()).IsFalse();
+ }
+
+ [Test]
+ public async Task Disabled_Html_Report_Stops_Activity_Collection_After_Discovery(CancellationToken cancellationToken)
+ {
+ using var reporter = new HtmlReporter(new MockExtension());
+ cancellationToken.ThrowIfCancellationRequested();
+ await reporter.OnTestSessionStartingAsync(null!);
+ reporter.HasActivityCollector.ShouldBeTrue();
+
+ TUnitSettings.Default.Reporting.HtmlReportEnabled = false;
+ await reporter.ConsumeAsync(reporter, null!, cancellationToken);
+
+ reporter.HasActivityCollector.ShouldBeFalse();
+ }
+
+ [Test]
+ public async Task Html_Report_Setting_Is_Resolved_Per_Session(CancellationToken cancellationToken)
+ {
+ using var reporter = new HtmlReporter(new MockExtension());
+
+ cancellationToken.ThrowIfCancellationRequested();
+ await reporter.OnTestSessionStartingAsync(null!);
+ TUnitSettings.Default.Reporting.HtmlReportEnabled = false;
+ reporter.IsHtmlReportEnabledForRun().ShouldBeFalse();
+
+ cancellationToken.ThrowIfCancellationRequested();
+ await reporter.OnTestSessionStartingAsync(null!);
+ TUnitSettings.Default.Reporting.HtmlReportEnabled = true;
+ reporter.IsHtmlReportEnabledForRun().ShouldBeTrue();
+ }
+
+ [Test]
+ public async Task Activity_Collection_Starts_Before_Discovery_Reenables_Reporting(CancellationToken cancellationToken)
+ {
+ TUnitSettings.Default.Reporting.HtmlReportEnabled = false;
+ using var reporter = new HtmlReporter(new MockExtension());
+
+ cancellationToken.ThrowIfCancellationRequested();
+ await reporter.OnTestSessionStartingAsync(null!);
+ reporter.HasActivityCollector.ShouldBeTrue();
+
+ var activity = TUnitActivitySource.StartLifecycleActivity(TUnitActivitySource.SpanTestSession);
+ TUnitSettings.Default.Reporting.HtmlReportEnabled = true;
+ reporter.IsHtmlReportEnabledForRun().ShouldBeTrue();
+ TUnitActivitySource.StopActivity(activity);
+
+ reporter.StopActivityCollection();
+ var spans = reporter.BuildReportData().Spans;
+ spans.ShouldNotBeNull();
+ spans.ShouldContain(span => span.SpanType == TUnitActivitySource.SpanTestSession);
+ }
+
+ [Test]
+ public async Task Activity_Collection_Is_Recreated_Between_Sessions(CancellationToken cancellationToken)
+ {
+ TUnitSettings.Default.Reporting.HtmlReportEnabled = true;
+ using var reporter = new HtmlReporter(new MockExtension());
+
+ cancellationToken.ThrowIfCancellationRequested();
+ await reporter.OnTestSessionStartingAsync(null!);
+ reporter.HasActivityCollector.ShouldBeTrue();
+
+ cancellationToken.ThrowIfCancellationRequested();
+ await reporter.OnTestSessionFinishingAsync(null!);
+ reporter.HasActivityCollector.ShouldBeFalse();
+
+ cancellationToken.ThrowIfCancellationRequested();
+ await reporter.OnTestSessionStartingAsync(null!);
+ reporter.HasActivityCollector.ShouldBeTrue();
+
+ var activity = TUnitActivitySource.StartLifecycleActivity(TUnitActivitySource.SpanTestSession);
+ TUnitActivitySource.StopActivity(activity);
+ reporter.StopActivityCollection();
+ var spans = reporter.BuildReportData().Spans;
+ spans.ShouldNotBeNull();
+ spans.ShouldContain(span => span.SpanType == TUnitActivitySource.SpanTestSession);
+ }
+
+ [Test]
+ public async Task Test_Updates_Are_Cleared_Between_Sessions(CancellationToken cancellationToken)
+ {
+ using var reporter = new HtmlReporter(new MockExtension());
+
+ await reporter.OnTestSessionStartingAsync(null!);
+ await reporter.ConsumeAsync(reporter, CreatePassedUpdate("first"), cancellationToken);
+ reporter.BuildReportData().Groups.SelectMany(x => x.Tests).Single().Id.ShouldBe("first");
+
+ await reporter.OnTestSessionStartingAsync(null!);
+ await reporter.ConsumeAsync(reporter, CreatePassedUpdate("second"), cancellationToken);
+
+ reporter.BuildReportData().Groups.SelectMany(x => x.Tests).Single().Id.ShouldBe("second");
+ }
+
+ [Test]
+ public async Task GitHub_Report_State_Is_Reset_Between_Sessions(CancellationToken cancellationToken)
+ {
+ var tempDirectory = Path.Combine(Path.GetTempPath(), $"tunit-report-test-{Guid.NewGuid():N}");
+ Environment.SetEnvironmentVariable("TUNIT_AGGREGATE_REPORTS", "true");
+ Environment.SetEnvironmentVariable("TUNIT_AGGREGATE_DIR", Path.Combine(tempDirectory, "aggregate"));
+
+ try
+ {
+ Directory.CreateDirectory(tempDirectory);
+ using var reporter = new HtmlReporter(new MockExtension());
+ var githubReporter = new GitHubReporter(new MockExtension());
+ reporter.SetGitHubReporter(githubReporter);
+
+ await reporter.TryWriteSidecarAndAggregateAsync(
+ CreateReportData(),
+ Path.Combine(tempDirectory, "suite-report.html"),
+ cancellationToken);
+ githubReporter.SuppressPerSuiteSummary.ShouldBeTrue();
+ githubReporter.ArtifactUrl = "https://example.com/old-artifact";
+ githubReporter.ShowArtifactUploadTip = true;
+
+ await reporter.OnTestSessionStartingAsync(null!);
+
+ githubReporter.SuppressPerSuiteSummary.ShouldBeFalse();
+ githubReporter.ArtifactUrl.ShouldBeNull();
+ githubReporter.ShowArtifactUploadTip.ShouldBeFalse();
+ }
+ finally
+ {
+ if (Directory.Exists(tempDirectory))
+ {
+ Directory.Delete(tempDirectory, recursive: true);
+ }
+ }
+ }
+
+ [Test]
+ public async Task Disabled_Artifact_Upload_Does_Not_Publish_Session_File_Artifact(CancellationToken cancellationToken)
+ {
+ TUnitSettings.Default.Reporting.ArtifactUploadEnabled = false;
+ var reporter = new HtmlReporter(new MockExtension());
+ var messageBus = new CapturingMessageBus();
+ reporter.SetMessageBus(messageBus);
+
+ await reporter.PublishArtifactAsync("report.html", new SessionUid("session"), cancellationToken);
+
+ messageBus.Published.ShouldBeEmpty();
+ }
+
+ [Test]
+ public async Task Disabled_Json_Report_Removes_Stale_Aggregation_Outputs(CancellationToken cancellationToken)
+ {
+ var tempDirectory = Path.Combine(Path.GetTempPath(), $"tunit-report-test-{Guid.NewGuid():N}");
+ var aggregationDirectory = Path.Combine(tempDirectory, "aggregate");
+ var removedHtmlPath = Path.Combine(tempDirectory, "removed-report.html");
+ var remainingHtmlPath = Path.Combine(tempDirectory, "remaining-report.html");
+ var mergedReportPath = Path.Combine(aggregationDirectory, ReportDataJson.MergedReportFileName);
+ Environment.SetEnvironmentVariable("TUNIT_AGGREGATE_REPORTS", "true");
+ Environment.SetEnvironmentVariable("TUNIT_AGGREGATE_DIR", aggregationDirectory);
+
+ try
+ {
+ Directory.CreateDirectory(tempDirectory);
+ var reporter = new HtmlReporter(new MockExtension());
+ var aggregator = ReportAggregator.TryCreateFromEnvironment(Environment.GetEnvironmentVariable)!;
+
+ TUnitSettings.Default.Reporting.JsonReportEnabled = true;
+ await reporter.TryWriteSidecarAndAggregateAsync(CreateReportData("RemovedSuiteMarker"), removedHtmlPath, cancellationToken);
+ await reporter.TryWriteSidecarAndAggregateAsync(CreateReportData("RemainingSuiteMarker"), remainingHtmlPath, cancellationToken);
+ Directory.GetFiles(aggregationDirectory, $"*{ReportDataJson.SidecarExtension}").Length.ShouldBe(2);
+
+ TUnitSettings.Default.Reporting.JsonReportEnabled = false;
+ await reporter.TryWriteSidecarAndAggregateAsync(CreateReportData("RemovedSuiteMarker"), removedHtmlPath, cancellationToken);
+
+ File.Exists(HtmlReporter.GetSidecarPath(removedHtmlPath)).ShouldBeFalse();
+ Directory.GetFiles(aggregationDirectory, $"*{ReportDataJson.SidecarExtension}").Length.ShouldBe(2);
+ aggregator.ReadAllSidecars().Single().AssemblyName.ShouldBe("RemainingSuiteMarker");
+ var mergedReport = File.ReadAllText(mergedReportPath);
+ mergedReport.ShouldNotContain("RemovedSuiteMarker");
+ mergedReport.ShouldContain("RemainingSuiteMarker");
+
+ await reporter.TryWriteSidecarAndAggregateAsync(CreateReportData("RemainingSuiteMarker"), remainingHtmlPath, cancellationToken);
+
+ File.Exists(HtmlReporter.GetSidecarPath(remainingHtmlPath)).ShouldBeFalse();
+ Directory.GetFiles(aggregationDirectory, $"*{ReportDataJson.SidecarExtension}").Length.ShouldBe(2);
+ aggregator.ReadAllSidecars().ShouldBeEmpty();
+ File.Exists(mergedReportPath).ShouldBeFalse();
+ }
+ finally
+ {
+ if (Directory.Exists(tempDirectory))
+ {
+ Directory.Delete(tempDirectory, recursive: true);
+ }
+ }
+ }
+
+ [Test]
+ public async Task Cancelled_Session_Still_Removes_Disabled_Report_Sidecars(CancellationToken cancellationToken)
+ {
+ var tempDirectory = Path.Combine(Path.GetTempPath(), $"tunit-report-test-{Guid.NewGuid():N}");
+ var aggregationDirectory = Path.Combine(tempDirectory, "aggregate");
+ var htmlPath = Path.Combine(tempDirectory, "cancelled-report.html");
+ Environment.SetEnvironmentVariable("TUNIT_AGGREGATE_REPORTS", "true");
+ Environment.SetEnvironmentVariable("TUNIT_AGGREGATE_DIR", aggregationDirectory);
+
+ try
+ {
+ Directory.CreateDirectory(tempDirectory);
+ using var reporter = new HtmlReporter(new MockExtension());
+ var aggregator = ReportAggregator.TryCreateFromEnvironment(Environment.GetEnvironmentVariable)!;
+
+ TUnitSettings.Default.Reporting.JsonReportEnabled = true;
+ await reporter.TryWriteSidecarAndAggregateAsync(CreateReportData(), htmlPath, cancellationToken);
+
+ using var cancelled = CancellationTokenSource.CreateLinkedTokenSource(cancellationToken);
+ cancelled.Cancel();
+ TUnitSettings.Default.Reporting.JsonReportEnabled = false;
+ await reporter.TryWriteSidecarAndAggregateAsync(CreateReportData(), htmlPath, cancelled.Token);
+
+ File.Exists(HtmlReporter.GetSidecarPath(htmlPath)).ShouldBeFalse();
+ Directory.GetFiles(aggregationDirectory, $"*{ReportDataJson.SidecarExtension}").Length.ShouldBe(1);
+ aggregator.ReadAllSidecars().ShouldBeEmpty();
+ }
+ finally
+ {
+ if (Directory.Exists(tempDirectory))
+ {
+ Directory.Delete(tempDirectory, recursive: true);
+ }
+ }
+ }
+
+ [Test]
+ public async Task Disabled_Marker_Excludes_Stale_Shared_Sidecar(CancellationToken cancellationToken)
+ {
+ cancellationToken.ThrowIfCancellationRequested();
+ var tempDirectory = Path.Combine(Path.GetTempPath(), $"tunit-report-test-{Guid.NewGuid():N}");
+ Environment.SetEnvironmentVariable("TUNIT_AGGREGATE_REPORTS", "true");
+ Environment.SetEnvironmentVariable("TUNIT_AGGREGATE_DIR", tempDirectory);
+
+ try
+ {
+ var aggregator = ReportAggregator.TryCreateFromEnvironment(Environment.GetEnvironmentVariable)!;
+ var reportData = CreateReportData("DisabledSuiteMarker");
+ aggregator.WriteSidecar(ReportDataJson.SerializeToBytes(reportData), reportData.AssemblyName, "suite");
+ var staleGeneration = aggregator.ReadEffectiveSidecarGeneration(reportData.AssemblyName, "suite");
+
+ var replacement = CreateReportData(reportData.AssemblyName, "replacement-machine");
+ aggregator.WriteSidecar(ReportDataJson.SerializeToBytes(replacement), replacement.AssemblyName, "suite");
+ aggregator.ExcludeSidecarIfGenerationMatches(reportData.AssemblyName, "suite", staleGeneration);
+ Directory.GetFiles(tempDirectory, $"*{ReportDataJson.SidecarExclusionExtension}").ShouldBeEmpty();
+ aggregator.ReadAllSidecars().Single().MachineName.ShouldBe("replacement-machine");
+
+ aggregator.ExcludeSidecar(reportData.AssemblyName, "suite");
+ aggregator.ReadAllSidecars().ShouldBeEmpty();
+
+ var latest = CreateReportData(reportData.AssemblyName, "latest-machine");
+ aggregator.WriteSidecar(ReportDataJson.SerializeToBytes(latest), latest.AssemblyName, "suite");
+ aggregator.ReadAllSidecars().Single().MachineName.ShouldBe("latest-machine");
+
+ aggregator.IncludeSidecar(reportData.AssemblyName, "suite");
+ aggregator.ReadAllSidecars().Single().MachineName.ShouldBe("latest-machine");
+ }
+ finally
+ {
+ if (Directory.Exists(tempDirectory))
+ {
+ Directory.Delete(tempDirectory, recursive: true);
+ }
+ }
+ }
+
+ [Test]
+ public async Task Disabled_Html_Report_Removes_Stale_Aggregation_Outputs(CancellationToken cancellationToken)
+ {
+ var tempDirectory = Path.Combine(Path.GetTempPath(), $"tunit-report-test-{Guid.NewGuid():N}");
+ var aggregationDirectory = Path.Combine(tempDirectory, "aggregate");
+ var disabledHtmlPath = Path.Combine(tempDirectory, "disabled-report.html");
+ var remainingHtmlPath = Path.Combine(tempDirectory, "remaining-report.html");
+ var mergedReportPath = Path.Combine(aggregationDirectory, ReportDataJson.MergedReportFileName);
+ Environment.SetEnvironmentVariable("TUNIT_AGGREGATE_REPORTS", "true");
+ Environment.SetEnvironmentVariable("TUNIT_AGGREGATE_DIR", aggregationDirectory);
+
+ try
+ {
+ Directory.CreateDirectory(tempDirectory);
+ using var reporter = new HtmlReporter(new MockExtension());
+ var aggregator = ReportAggregator.TryCreateFromEnvironment(Environment.GetEnvironmentVariable)!;
+ reporter.SetOutputPath(disabledHtmlPath);
+ var disabledAssemblyName = reporter.BuildReportData().AssemblyName;
+
+ TUnitSettings.Default.Reporting.JsonReportEnabled = true;
+ await reporter.TryWriteSidecarAndAggregateAsync(CreateReportData(disabledAssemblyName), disabledHtmlPath, cancellationToken);
+ await reporter.TryWriteSidecarAndAggregateAsync(CreateReportData("RemainingSuiteMarker"), remainingHtmlPath, cancellationToken);
+
+ TUnitSettings.Default.Reporting.HtmlReportEnabled = false;
+ await reporter.OnTestSessionFinishingAsync(null!);
+
+ File.Exists(HtmlReporter.GetSidecarPath(disabledHtmlPath)).ShouldBeFalse();
+ Directory.GetFiles(aggregationDirectory, $"*{ReportDataJson.SidecarExtension}").Length.ShouldBe(2);
+ aggregator.ReadAllSidecars().Single().AssemblyName.ShouldBe("RemainingSuiteMarker");
+ var mergedReport = File.ReadAllText(mergedReportPath);
+ mergedReport.ShouldNotContain(disabledAssemblyName);
+ mergedReport.ShouldContain("RemainingSuiteMarker");
+ }
+ finally
+ {
+ if (Directory.Exists(tempDirectory))
+ {
+ Directory.Delete(tempDirectory, recursive: true);
+ }
+ }
+ }
+
+ [Test]
+ public async Task Shared_Sidecar_Stays_Hidden_Until_Lock_Wait_Completes(CancellationToken cancellationToken)
+ {
+ var tempDirectory = Path.Combine(Path.GetTempPath(), $"tunit-report-test-{Guid.NewGuid():N}");
+ var aggregationDirectory = Path.Combine(tempDirectory, "aggregate");
+ var htmlPath = Path.Combine(tempDirectory, "suite-report.html");
+ Environment.SetEnvironmentVariable("TUNIT_AGGREGATE_REPORTS", "true");
+ Environment.SetEnvironmentVariable("TUNIT_AGGREGATE_DIR", aggregationDirectory);
+
+ try
+ {
+ Directory.CreateDirectory(tempDirectory);
+ using var reporter = new HtmlReporter(new MockExtension());
+ var aggregator = ReportAggregator.TryCreateFromEnvironment(Environment.GetEnvironmentVariable);
+ var aggregationLock = await aggregator!.AcquireLockAsync(cancellationToken);
+ aggregationLock.ShouldNotBeNull();
+
+ Task writeTask;
+ using (aggregationLock!)
+ {
+ writeTask = reporter.TryWriteSidecarAndAggregateAsync(CreateReportData(), htmlPath, cancellationToken);
+
+ File.Exists(HtmlReporter.GetSidecarPath(htmlPath)).ShouldBeTrue();
+ Directory.GetFiles(aggregationDirectory, $"*{ReportDataJson.SidecarExtension}").Length.ShouldBe(1);
+ Directory.GetFiles(aggregationDirectory, $"*{ReportDataJson.SidecarPublishingExtension}").Length.ShouldBe(1);
+ aggregator.ReadAllSidecars().ShouldBeEmpty();
+ }
+
+ await writeTask;
+
+ Directory.GetFiles(aggregationDirectory, $"*{ReportDataJson.SidecarExtension}").Length.ShouldBe(1);
+ File.ReadAllText(Path.Combine(aggregationDirectory, ReportDataJson.MergedReportFileName)).ShouldContain("Tests");
+ }
+ finally
+ {
+ if (Directory.Exists(tempDirectory))
+ {
+ Directory.Delete(tempDirectory, recursive: true);
+ }
+ }
+ }
+
+ [Test]
+ public async Task Empty_Session_With_Disabled_Json_Removes_Stale_Sidecars(CancellationToken cancellationToken)
+ {
+ var tempDirectory = Path.Combine(Path.GetTempPath(), $"tunit-report-test-{Guid.NewGuid():N}");
+ var aggregationDirectory = Path.Combine(tempDirectory, "aggregate");
+ var htmlPath = Path.Combine(tempDirectory, "suite-report.html");
+ Environment.SetEnvironmentVariable("TUNIT_AGGREGATE_REPORTS", "true");
+ Environment.SetEnvironmentVariable("TUNIT_AGGREGATE_DIR", aggregationDirectory);
+
+ try
+ {
+ Directory.CreateDirectory(tempDirectory);
+ using var reporter = new HtmlReporter(new MockExtension());
+ var aggregator = ReportAggregator.TryCreateFromEnvironment(Environment.GetEnvironmentVariable)!;
+ reporter.SetOutputPath(htmlPath);
+ var assemblyName = reporter.BuildReportData().AssemblyName;
+
+ await reporter.TryWriteSidecarAndAggregateAsync(CreateReportData(assemblyName), htmlPath, cancellationToken);
+ TUnitSettings.Default.Reporting.JsonReportEnabled = false;
+
+ await reporter.OnTestSessionFinishingAsync(null!);
+
+ File.Exists(HtmlReporter.GetSidecarPath(htmlPath)).ShouldBeFalse();
+ Directory.GetFiles(aggregationDirectory, $"*{ReportDataJson.SidecarExtension}").Length.ShouldBe(1);
+ aggregator.ReadAllSidecars().ShouldBeEmpty();
+ }
+ finally
+ {
+ if (Directory.Exists(tempDirectory))
+ {
+ Directory.Delete(tempDirectory, recursive: true);
+ }
+ }
+ }
+
+ [Test]
+ public async Task Publication_Lock_File_Is_Stable_And_Reused(CancellationToken cancellationToken)
+ {
+ cancellationToken.ThrowIfCancellationRequested();
+ var tempDirectory = Path.Combine(Path.GetTempPath(), $"tunit-report-test-{Guid.NewGuid():N}");
+ Environment.SetEnvironmentVariable("TUNIT_AGGREGATE_REPORTS", "true");
+ Environment.SetEnvironmentVariable("TUNIT_AGGREGATE_DIR", tempDirectory);
+
+ try
+ {
+ var aggregator = ReportAggregator.TryCreateFromEnvironment(Environment.GetEnvironmentVariable)!;
+ var reportData = CreateReportData();
+ using var publicationLock = aggregator.BeginSidecarPublication(reportData.AssemblyName, "suite");
+ var sidecarPath = aggregator.WriteSidecar(ReportDataJson.SerializeToBytes(reportData), reportData.AssemblyName, "suite");
+
+ aggregator.ReadAllSidecars().ShouldBeEmpty();
+ publicationLock.Dispose();
+
+ aggregator.ReadAllSidecars().Single().AssemblyName.ShouldBe("Tests");
+ File.Exists(sidecarPath + ReportDataJson.SidecarPublishingExtension).ShouldBeTrue();
+
+ using (aggregator.BeginSidecarPublication(reportData.AssemblyName, "suite"))
+ {
+ aggregator.ReadAllSidecars().ShouldBeEmpty();
+ }
+ }
+ finally
+ {
+ if (Directory.Exists(tempDirectory))
+ {
+ Directory.Delete(tempDirectory, recursive: true);
+ }
+ }
+ }
+
+ [Test]
+ public async Task Cancelled_Lock_Wait_Keeps_Per_Suite_Summary(CancellationToken cancellationToken)
+ {
+ var tempDirectory = Path.Combine(Path.GetTempPath(), $"tunit-report-test-{Guid.NewGuid():N}");
+ var aggregationDirectory = Path.Combine(tempDirectory, "aggregate");
+ Environment.SetEnvironmentVariable("TUNIT_AGGREGATE_REPORTS", "true");
+ Environment.SetEnvironmentVariable("TUNIT_AGGREGATE_DIR", aggregationDirectory);
+
+ try
+ {
+ using var reporter = new HtmlReporter(new MockExtension());
+ var githubReporter = new GitHubReporter(new MockExtension());
+ reporter.SetGitHubReporter(githubReporter);
+ var aggregator = ReportAggregator.TryCreateFromEnvironment(Environment.GetEnvironmentVariable)!;
+ using var aggregationLock = await aggregator.AcquireLockAsync(cancellationToken);
+ using var cancelled = new CancellationTokenSource();
+ cancelled.Cancel();
+ var htmlPath = Path.Combine(tempDirectory, "suite-report.html");
+ aggregator.ExcludeSidecar("Tests", htmlPath);
+
+ await reporter.TryWriteSidecarAndAggregateAsync(
+ CreateReportData(),
+ htmlPath,
+ cancelled.Token);
+
+ githubReporter.SuppressPerSuiteSummary.ShouldBeFalse();
+ aggregator.ReadAllSidecars().Single().AssemblyName.ShouldBe("Tests");
+ Directory.GetFiles(aggregationDirectory, $"*{ReportDataJson.SidecarExclusionExtension}").ShouldBeEmpty();
+ File.Exists(Path.Combine(aggregationDirectory, ReportDataJson.MergedReportFileName)).ShouldBeFalse();
+ }
+ finally
+ {
+ if (Directory.Exists(tempDirectory))
+ {
+ Directory.Delete(tempDirectory, recursive: true);
+ }
+ }
+ }
+
+ [Test]
+ public async Task Cancelled_Defer_Lock_Wait_Suppresses_Per_Suite_Summary(CancellationToken cancellationToken)
+ {
+ var tempDirectory = Path.Combine(Path.GetTempPath(), $"tunit-report-test-{Guid.NewGuid():N}");
+ var aggregationDirectory = Path.Combine(tempDirectory, "aggregate");
+ Environment.SetEnvironmentVariable("TUNIT_AGGREGATE_REPORTS", "defer");
+ Environment.SetEnvironmentVariable("TUNIT_AGGREGATE_DIR", aggregationDirectory);
+
+ try
+ {
+ using var reporter = new HtmlReporter(new MockExtension());
+ var githubReporter = new GitHubReporter(new MockExtension());
+ reporter.SetGitHubReporter(githubReporter);
+ var aggregator = ReportAggregator.TryCreateFromEnvironment(Environment.GetEnvironmentVariable)!;
+ using var aggregationLock = await aggregator.AcquireLockAsync(cancellationToken);
+ using var cancelled = new CancellationTokenSource();
+ cancelled.Cancel();
+ var htmlPath = Path.Combine(tempDirectory, "suite-report.html");
+
+ await reporter.TryWriteSidecarAndAggregateAsync(
+ CreateReportData(),
+ htmlPath,
+ cancelled.Token);
+
+ githubReporter.SuppressPerSuiteSummary.ShouldBeTrue();
+ aggregator.ReadAllSidecars().Single().AssemblyName.ShouldBe("Tests");
+ File.Exists(Path.Combine(aggregationDirectory, ReportDataJson.MergedReportFileName)).ShouldBeFalse();
+ }
+ finally
+ {
+ if (Directory.Exists(tempDirectory))
+ {
+ Directory.Delete(tempDirectory, recursive: true);
+ }
+ }
+ }
+
+ [Test]
+ [Timeout(30_000)]
+ public async Task Disabled_Cleanup_Does_Not_Exclude_Active_Publication(CancellationToken cancellationToken)
+ {
+ cancellationToken.ThrowIfCancellationRequested();
+ var tempDirectory = Path.Combine(Path.GetTempPath(), $"tunit-report-test-{Guid.NewGuid():N}");
+ var aggregationDirectory = Path.Combine(tempDirectory, "aggregate");
+ var htmlPath = Path.Combine(tempDirectory, "suite-report.html");
+ Environment.SetEnvironmentVariable("TUNIT_AGGREGATE_REPORTS", "true");
+ Environment.SetEnvironmentVariable("TUNIT_AGGREGATE_DIR", aggregationDirectory);
+
+ try
+ {
+ using var reporter = new HtmlReporter(new MockExtension());
+ var aggregator = ReportAggregator.TryCreateFromEnvironment(Environment.GetEnvironmentVariable)!;
+ var reportData = CreateReportData();
+ aggregator.WriteSidecar(ReportDataJson.SerializeToBytes(reportData), reportData.AssemblyName, htmlPath);
+ TUnitSettings.Default.Reporting.JsonReportEnabled = false;
+
+ using (aggregator.BeginSidecarPublication(reportData.AssemblyName, htmlPath))
+ {
+ var replacement = CreateReportData(reportData.AssemblyName, "active-publisher");
+ aggregator.WriteSidecar(ReportDataJson.SerializeToBytes(replacement), replacement.AssemblyName, htmlPath);
+ await reporter.TryWriteSidecarAndAggregateAsync(reportData, htmlPath, cancellationToken);
+ }
+
+ aggregator.ReadAllSidecars().Single().MachineName.ShouldBe("active-publisher");
+ Directory.GetFiles(aggregationDirectory, $"*{ReportDataJson.SidecarExclusionExtension}").ShouldBeEmpty();
+ }
+ finally
+ {
+ if (Directory.Exists(tempDirectory))
+ {
+ Directory.Delete(tempDirectory, recursive: true);
+ }
+ }
+ }
+
+ [Test]
+ [Timeout(30_000)]
+ public async Task Enabled_Publication_Contention_Preserves_Shared_Sidecar(CancellationToken cancellationToken)
+ {
+ cancellationToken.ThrowIfCancellationRequested();
+ var tempDirectory = Path.Combine(Path.GetTempPath(), $"tunit-report-test-{Guid.NewGuid():N}");
+ var aggregationDirectory = Path.Combine(tempDirectory, "aggregate");
+ var htmlPath = Path.Combine(tempDirectory, "suite-report.html");
+ Environment.SetEnvironmentVariable("TUNIT_AGGREGATE_REPORTS", "true");
+ Environment.SetEnvironmentVariable("TUNIT_AGGREGATE_DIR", aggregationDirectory);
+
+ try
+ {
+ using var reporter = new HtmlReporter(new MockExtension());
+ var aggregator = ReportAggregator.TryCreateFromEnvironment(Environment.GetEnvironmentVariable)!;
+ var reportData = CreateReportData("ContendedSuiteMarker", "timeout-publisher");
+
+ using (aggregator.BeginSidecarPublication(reportData.AssemblyName, htmlPath))
+ {
+ var activeReport = CreateReportData(reportData.AssemblyName, "active-publisher");
+ aggregator.WriteSidecar(ReportDataJson.SerializeToBytes(activeReport), activeReport.AssemblyName, htmlPath);
+ await reporter.TryWriteSidecarAndAggregateAsync(reportData, htmlPath, cancellationToken);
+
+ Directory.GetFiles(aggregationDirectory, $"*{ReportDataJson.SidecarExtension}").Length.ShouldBe(2);
+ aggregator.ReadAllSidecars().Single().MachineName.ShouldBe("timeout-publisher");
+ File.Exists(Path.Combine(aggregationDirectory, ReportDataJson.MergedReportFileName)).ShouldBeTrue();
+ }
+
+ aggregator.ReadAllSidecars().Single().MachineName.ShouldBe("timeout-publisher");
+
+ var newerReport = CreateReportData(reportData.AssemblyName, "newer-publisher");
+ await reporter.TryWriteSidecarAndAggregateAsync(newerReport, htmlPath, cancellationToken);
+ Directory.GetFiles(aggregationDirectory, $"*{ReportDataJson.SidecarExtension}").Length.ShouldBe(1);
+ aggregator.ReadAllSidecars().Single().MachineName.ShouldBe("newer-publisher");
+ }
+ finally
+ {
+ if (Directory.Exists(tempDirectory))
+ {
+ Directory.Delete(tempDirectory, recursive: true);
+ }
+ }
+ }
+
+ [Test]
+ [Timeout(30_000)]
+ public async Task Enabled_Publication_Supersedes_In_Flight_Disabled_Cleanup(CancellationToken cancellationToken)
+ {
+ cancellationToken.ThrowIfCancellationRequested();
+ var tempDirectory = Path.Combine(Path.GetTempPath(), $"tunit-report-test-{Guid.NewGuid():N}");
+ var aggregationDirectory = Path.Combine(tempDirectory, "aggregate");
+ var htmlPath = Path.Combine(tempDirectory, "suite-report.html");
+ Environment.SetEnvironmentVariable("TUNIT_AGGREGATE_REPORTS", "true");
+ Environment.SetEnvironmentVariable("TUNIT_AGGREGATE_DIR", aggregationDirectory);
+
+ try
+ {
+ Directory.CreateDirectory(tempDirectory);
+ using var reporter = new HtmlReporter(new MockExtension());
+ var aggregator = ReportAggregator.TryCreateFromEnvironment(Environment.GetEnvironmentVariable)!;
+ var oldReport = CreateReportData(machineName: "old-machine");
+ await reporter.TryWriteSidecarAndAggregateAsync(oldReport, htmlPath, cancellationToken);
+
+ var aggregationLock = await aggregator.AcquireLockAsync(cancellationToken);
+ aggregationLock.ShouldNotBeNull();
+ Task cleanupTask;
+ Task publicationTask;
+ using (aggregationLock!)
+ {
+ TUnitSettings.Default.Reporting.JsonReportEnabled = false;
+ cleanupTask = reporter.TryWriteSidecarAndAggregateAsync(oldReport, htmlPath, cancellationToken);
+ Directory.GetFiles(aggregationDirectory, $"*{ReportDataJson.SidecarExclusionExtension}").Length.ShouldBe(1);
+
+ TUnitSettings.Default.Reporting.JsonReportEnabled = true;
+ var newReport = CreateReportData(machineName: "new-machine");
+ publicationTask = reporter.TryWriteSidecarAndAggregateAsync(newReport, htmlPath, cancellationToken);
+ }
+
+ await Task.WhenAll(cleanupTask, publicationTask);
+
+ var publishedReport = aggregator.ReadAllSidecars().Single();
+ publishedReport.MachineName.ShouldBe("new-machine");
+ Directory.GetFiles(aggregationDirectory, $"*{ReportDataJson.SidecarExclusionExtension}").ShouldBeEmpty();
+ }
+ finally
+ {
+ if (Directory.Exists(tempDirectory))
+ {
+ Directory.Delete(tempDirectory, recursive: true);
+ }
+ }
+ }
+
+ [Test]
+ public async Task Enabled_Publication_Clears_Stale_Exclusion(CancellationToken cancellationToken)
+ {
+ var tempDirectory = Path.Combine(Path.GetTempPath(), $"tunit-report-test-{Guid.NewGuid():N}");
+ var aggregationDirectory = Path.Combine(tempDirectory, "aggregate");
+ var htmlPath = Path.Combine(tempDirectory, "suite-report.html");
+ Environment.SetEnvironmentVariable("TUNIT_AGGREGATE_REPORTS", "true");
+ Environment.SetEnvironmentVariable("TUNIT_AGGREGATE_DIR", aggregationDirectory);
+
+ try
+ {
+ using var reporter = new HtmlReporter(new MockExtension());
+ var aggregator = ReportAggregator.TryCreateFromEnvironment(Environment.GetEnvironmentVariable)!;
+ aggregator.ExcludeSidecar("Tests", htmlPath);
+
+ await reporter.TryWriteSidecarAndAggregateAsync(CreateReportData(), htmlPath, cancellationToken);
+
+ aggregator.ReadAllSidecars().Single().AssemblyName.ShouldBe("Tests");
+ Directory.GetFiles(aggregationDirectory, $"*{ReportDataJson.SidecarExclusionExtension}").ShouldBeEmpty();
+ }
+ finally
+ {
+ if (Directory.Exists(tempDirectory))
+ {
+ Directory.Delete(tempDirectory, recursive: true);
+ }
+ }
+ }
+
+ private static ReportData CreateReportData(string assemblyName = "Tests", string machineName = "machine") => new()
+ {
+ AssemblyName = assemblyName,
+ MachineName = machineName,
+ Timestamp = DateTimeOffset.UtcNow.ToString("O"),
+ TUnitVersion = "1.0.0",
+ OperatingSystem = "test",
+ RuntimeVersion = "test",
+ Summary = new ReportSummary(),
+ Groups = [],
+ };
+
+ private static TestNodeUpdateMessage CreatePassedUpdate(string id) => new(
+ new SessionUid("session"),
+ new TestNode
+ {
+ Uid = new TestNodeUid(id),
+ DisplayName = id,
+ Properties = new PropertyBag(PassedTestNodeStateProperty.CachedInstance),
+ });
+}
diff --git a/tests/TUnit.Engine.Tests/HtmlReporterTests.cs b/tests/TUnit.Engine.Tests/HtmlReporterTests.cs
index 5d7d646d22e..05123ed29cc 100644
--- a/tests/TUnit.Engine.Tests/HtmlReporterTests.cs
+++ b/tests/TUnit.Engine.Tests/HtmlReporterTests.cs
@@ -30,6 +30,23 @@ public void HtmlReporter_DataTypesProduced_Contains_SessionFileArtifact()
producer.DataTypesProduced.ShouldContain(typeof(SessionFileArtifact));
}
+ [Test]
+ public async Task Stopping_Activity_Collection_Preserves_Spans_For_Report_Data(CancellationToken cancellationToken)
+ {
+ using var reporter = new HtmlReporter(new MockExtension());
+ cancellationToken.ThrowIfCancellationRequested();
+ await reporter.OnTestSessionStartingAsync(null!);
+
+ var activity = TUnitActivitySource.StartLifecycleActivity(TUnitActivitySource.SpanTestSession);
+ TUnitActivitySource.StopActivity(activity);
+ reporter.StopActivityCollection();
+
+ var reportData = reporter.BuildReportData();
+
+ reportData.Spans.ShouldNotBeNull();
+ reportData.Spans.ShouldContain(span => span.SpanType == TUnitActivitySource.SpanTestSession);
+ }
+
[Test]
public async Task PublishArtifactAsync_Publishes_SessionFileArtifact_When_SessionContext_Set_And_File_Exists()
{
diff --git a/tests/TUnit.Engine.Tests/ReportingSettingsTests.cs b/tests/TUnit.Engine.Tests/ReportingSettingsTests.cs
new file mode 100644
index 00000000000..1ca19d4ecfe
--- /dev/null
+++ b/tests/TUnit.Engine.Tests/ReportingSettingsTests.cs
@@ -0,0 +1,45 @@
+using Shouldly;
+using TUnit.Engine.Tests.Enums;
+
+namespace TUnit.Engine.Tests;
+
+public class ReportingSettingsTests(TestMode testMode) : InvokableTestBase(testMode)
+{
+ [Test]
+ public async Task Discovery_Hook_Can_Disable_Reporting(CancellationToken cancellationToken)
+ {
+ var tempDirectory = Path.Combine(Path.GetTempPath(), $"tunit-report-settings-{Guid.NewGuid():N}");
+ var reportPath = Path.Combine(tempDirectory, "report.html");
+ var aggregationDirectory = Path.Combine(tempDirectory, "aggregate");
+
+ try
+ {
+ var options = new RunOptions()
+ .WithArgument("--report-html-filename")
+ .WithArgument(reportPath)
+ .WithEnvironmentVariable("TUNIT_DISABLE_HTML_REPORTER", "false")
+ .WithEnvironmentVariable("TUNIT_DISABLE_JSON_REPORT", "false")
+ .WithEnvironmentVariable("TUNIT_DISABLE_ARTIFACT_UPLOAD", "false")
+ .WithEnvironmentVariable("TUNIT_AGGREGATE_REPORTS", "true")
+ .WithEnvironmentVariable("TUNIT_AGGREGATE_DIR", aggregationDirectory)
+ .WithEnvironmentVariable("TUNIT_TEST_DISABLE_REPORTING_FROM_DISCOVERY_HOOK", "true")
+ .WithGracefulCancellationToken(cancellationToken);
+
+ await RunTestsWithFilter(
+ "/*/*/ReportingSettingsTests/*",
+ [
+ result => result.ResultSummary.Counters.Passed.ShouldBe(1),
+ _ => File.Exists(reportPath).ShouldBeFalse(),
+ _ => Directory.Exists(aggregationDirectory).ShouldBeFalse(),
+ ],
+ options);
+ }
+ finally
+ {
+ if (Directory.Exists(tempDirectory))
+ {
+ Directory.Delete(tempDirectory, recursive: true);
+ }
+ }
+ }
+}
diff --git a/tests/TUnit.PublicAPI/Tests.Core_Library_Has_No_API_Changes.DotNet10_0.verified.txt b/tests/TUnit.PublicAPI/Tests.Core_Library_Has_No_API_Changes.DotNet10_0.verified.txt
index 863682c5d02..2c000ddceac 100644
--- a/tests/TUnit.PublicAPI/Tests.Core_Library_Has_No_API_Changes.DotNet10_0.verified.txt
+++ b/tests/TUnit.PublicAPI/Tests.Core_Library_Has_No_API_Changes.DotNet10_0.verified.txt
@@ -3046,11 +3046,18 @@ namespace .Settings
{
public int? MaximumParallelTests { get; set; }
}
+ public sealed class ReportingSettings
+ {
+ public bool ArtifactUploadEnabled { get; set; }
+ public bool HtmlReportEnabled { get; set; }
+ public bool JsonReportEnabled { get; set; }
+ }
public sealed class TUnitSettings
{
public . Display { get; }
public . Execution { get; }
public . Parallelism { get; }
+ public . Reporting { get; }
public . Timeouts { get; }
}
public sealed class TimeoutSettings
diff --git a/tests/TUnit.PublicAPI/Tests.Core_Library_Has_No_API_Changes.DotNet8_0.verified.txt b/tests/TUnit.PublicAPI/Tests.Core_Library_Has_No_API_Changes.DotNet8_0.verified.txt
index df987e2c496..7829d687fc6 100644
--- a/tests/TUnit.PublicAPI/Tests.Core_Library_Has_No_API_Changes.DotNet8_0.verified.txt
+++ b/tests/TUnit.PublicAPI/Tests.Core_Library_Has_No_API_Changes.DotNet8_0.verified.txt
@@ -3046,11 +3046,18 @@ namespace .Settings
{
public int? MaximumParallelTests { get; set; }
}
+ public sealed class ReportingSettings
+ {
+ public bool ArtifactUploadEnabled { get; set; }
+ public bool HtmlReportEnabled { get; set; }
+ public bool JsonReportEnabled { get; set; }
+ }
public sealed class TUnitSettings
{
public . Display { get; }
public . Execution { get; }
public . Parallelism { get; }
+ public . Reporting { get; }
public . Timeouts { get; }
}
public sealed class TimeoutSettings
diff --git a/tests/TUnit.PublicAPI/Tests.Core_Library_Has_No_API_Changes.DotNet9_0.verified.txt b/tests/TUnit.PublicAPI/Tests.Core_Library_Has_No_API_Changes.DotNet9_0.verified.txt
index 8657b96496c..11d504d7622 100644
--- a/tests/TUnit.PublicAPI/Tests.Core_Library_Has_No_API_Changes.DotNet9_0.verified.txt
+++ b/tests/TUnit.PublicAPI/Tests.Core_Library_Has_No_API_Changes.DotNet9_0.verified.txt
@@ -3046,11 +3046,18 @@ namespace .Settings
{
public int? MaximumParallelTests { get; set; }
}
+ public sealed class ReportingSettings
+ {
+ public bool ArtifactUploadEnabled { get; set; }
+ public bool HtmlReportEnabled { get; set; }
+ public bool JsonReportEnabled { get; set; }
+ }
public sealed class TUnitSettings
{
public . Display { get; }
public . Execution { get; }
public . Parallelism { get; }
+ public . Reporting { get; }
public . Timeouts { get; }
}
public sealed class TimeoutSettings
diff --git a/tests/TUnit.PublicAPI/Tests.Core_Library_Has_No_API_Changes.Net4_7.verified.txt b/tests/TUnit.PublicAPI/Tests.Core_Library_Has_No_API_Changes.Net4_7.verified.txt
index bc6b0a77b30..fb14bdc8ccc 100644
--- a/tests/TUnit.PublicAPI/Tests.Core_Library_Has_No_API_Changes.Net4_7.verified.txt
+++ b/tests/TUnit.PublicAPI/Tests.Core_Library_Has_No_API_Changes.Net4_7.verified.txt
@@ -2968,11 +2968,18 @@ namespace .Settings
{
public int? MaximumParallelTests { get; set; }
}
+ public sealed class ReportingSettings
+ {
+ public bool ArtifactUploadEnabled { get; set; }
+ public bool HtmlReportEnabled { get; set; }
+ public bool JsonReportEnabled { get; set; }
+ }
public sealed class TUnitSettings
{
public . Display { get; }
public . Execution { get; }
public . Parallelism { get; }
+ public . Reporting { get; }
public . Timeouts { get; }
}
public sealed class TimeoutSettings
diff --git a/tests/TUnit.TestProject/ReportingSettingsTests.cs b/tests/TUnit.TestProject/ReportingSettingsTests.cs
new file mode 100644
index 00000000000..5bba1a327da
--- /dev/null
+++ b/tests/TUnit.TestProject/ReportingSettingsTests.cs
@@ -0,0 +1,28 @@
+namespace TUnit.TestProject;
+
+public static class ReportingSettingsHooks
+{
+ internal const string DisableReportingEnvironmentVariable = "TUNIT_TEST_DISABLE_REPORTING_FROM_DISCOVERY_HOOK";
+
+ [Before(TestDiscovery)]
+ public static void ConfigureReporting(BeforeTestDiscoveryContext context, CancellationToken cancellationToken)
+ {
+ cancellationToken.ThrowIfCancellationRequested();
+
+ if (Environment.GetEnvironmentVariable(DisableReportingEnvironmentVariable) is not "true")
+ {
+ return;
+ }
+
+ context.Settings.Reporting.HtmlReportEnabled = false;
+ context.Settings.Reporting.JsonReportEnabled = false;
+ context.Settings.Reporting.ArtifactUploadEnabled = false;
+ }
+}
+
+public class ReportingSettingsTests
+{
+ [Test]
+ public void Test(CancellationToken cancellationToken)
+ => cancellationToken.ThrowIfCancellationRequested();
+}
diff --git a/tests/TUnit.UnitTests/ReportAggregationTests.cs b/tests/TUnit.UnitTests/ReportAggregationTests.cs
index c739508d64c..dd5a38bf963 100644
--- a/tests/TUnit.UnitTests/ReportAggregationTests.cs
+++ b/tests/TUnit.UnitTests/ReportAggregationTests.cs
@@ -17,6 +17,7 @@ public async Task Serialize_Then_Deserialize_RoundTrips_AllFields()
await Assert.That(restored).IsNotNull();
await Assert.That(restored!.AssemblyName).IsEqualTo("My.Tests");
+ await Assert.That(restored.PublicationGeneration).IsNotNull();
await Assert.That(restored.MachineName).IsEqualTo(original.MachineName);
await Assert.That(restored.Timestamp).IsEqualTo(original.Timestamp);
await Assert.That(restored.TUnitVersion).IsEqualTo(original.TUnitVersion);
diff --git a/tests/TUnit.UnitTests/TUnitSettingsTests.cs b/tests/TUnit.UnitTests/TUnitSettingsTests.cs
index 287fdbd63b5..054ba8ff403 100644
--- a/tests/TUnit.UnitTests/TUnitSettingsTests.cs
+++ b/tests/TUnit.UnitTests/TUnitSettingsTests.cs
@@ -18,6 +18,9 @@ public class TUnitSettingsTests
private int? _savedMaximumParallelTests;
private bool _savedDetailedStackTrace;
private bool _savedFailFast;
+ private bool _savedHtmlReportEnabled;
+ private bool _savedJsonReportEnabled;
+ private bool _savedArtifactUploadEnabled;
[Before(HookType.Test)]
public void SnapshotSettings()
@@ -29,6 +32,9 @@ public void SnapshotSettings()
_savedMaximumParallelTests = TUnitSettings.Default.Parallelism.MaximumParallelTests;
_savedDetailedStackTrace = TUnitSettings.Default.Display.DetailedStackTrace;
_savedFailFast = TUnitSettings.Default.Execution.FailFast;
+ _savedHtmlReportEnabled = TUnitSettings.Default.Reporting.HtmlReportEnabled;
+ _savedJsonReportEnabled = TUnitSettings.Default.Reporting.JsonReportEnabled;
+ _savedArtifactUploadEnabled = TUnitSettings.Default.Reporting.ArtifactUploadEnabled;
}
[After(HookType.Test)]
@@ -41,6 +47,9 @@ public void RestoreSettings()
TUnitSettings.Default.Parallelism.MaximumParallelTests = _savedMaximumParallelTests;
TUnitSettings.Default.Display.DetailedStackTrace = _savedDetailedStackTrace;
TUnitSettings.Default.Execution.FailFast = _savedFailFast;
+ TUnitSettings.Default.Reporting.HtmlReportEnabled = _savedHtmlReportEnabled;
+ TUnitSettings.Default.Reporting.JsonReportEnabled = _savedJsonReportEnabled;
+ TUnitSettings.Default.Reporting.ArtifactUploadEnabled = _savedArtifactUploadEnabled;
}
[Test]
@@ -53,6 +62,9 @@ public async Task Defaults_Are_Correct()
await Assert.That(TUnitSettings.Default.Parallelism.MaximumParallelTests).IsNull();
await Assert.That(TUnitSettings.Default.Display.DetailedStackTrace).IsFalse();
await Assert.That(TUnitSettings.Default.Execution.FailFast).IsFalse();
+ await Assert.That(TUnitSettings.Default.Reporting.HtmlReportEnabled).IsTrue();
+ await Assert.That(TUnitSettings.Default.Reporting.JsonReportEnabled).IsTrue();
+ await Assert.That(TUnitSettings.Default.Reporting.ArtifactUploadEnabled).IsTrue();
}
[Test]
@@ -62,6 +74,18 @@ public async Task Settings_Can_Be_Modified()
await Assert.That(TUnitSettings.Default.Timeouts.DefaultTestTimeout).IsEqualTo(TimeSpan.FromMinutes(10));
}
+ [Test]
+ public async Task Reporting_Settings_Can_Be_Modified()
+ {
+ TUnitSettings.Default.Reporting.HtmlReportEnabled = false;
+ TUnitSettings.Default.Reporting.JsonReportEnabled = false;
+ TUnitSettings.Default.Reporting.ArtifactUploadEnabled = false;
+
+ await Assert.That(TUnitSettings.Default.Reporting.HtmlReportEnabled).IsFalse();
+ await Assert.That(TUnitSettings.Default.Reporting.JsonReportEnabled).IsFalse();
+ await Assert.That(TUnitSettings.Default.Reporting.ArtifactUploadEnabled).IsFalse();
+ }
+
// Covers TestCoordinator's `test.Timeout ?? TUnitSettings...ExplicitDefaultTestTimeout` fallback:
// when the user never assigns DefaultTestTimeout, tests without [Timeout] skip the
// TimeoutHelper wrapper entirely (the right-hand side of the coalesce is null).