Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
26 commits
Select commit Hold shift + click to select a range
83d6f87
feat: add programmatic reporting settings
thomhurst Sep 1, 2026
4412d1d
fix: honor reporting settings end to end
thomhurst Sep 1, 2026
79f1ccf
fix: preserve report trace data
thomhurst Sep 1, 2026
40ee362
fix: scope reporting state to sessions
thomhurst Sep 1, 2026
a3e0410
fix: clear stale report state
thomhurst Sep 1, 2026
39e3ac7
fix: refresh aggregate after sidecar cleanup
thomhurst Sep 1, 2026
b44648a
fix: recreate tracing per test session
thomhurst Sep 1, 2026
731cf02
fix: harden reporter session lifecycle
thomhurst Sep 1, 2026
c47ac63
fix: preserve aggregation across sessions
thomhurst Sep 1, 2026
b641c94
fix: wait for complete report aggregation
thomhurst Sep 1, 2026
7d1937c
fix: bound report aggregation waits
thomhurst Sep 1, 2026
c3b5a31
fix: clean reporter state between sessions
thomhurst Sep 1, 2026
3bc6607
fix: persist aggregation timeout state
thomhurst Sep 1, 2026
9975f69
fix: preserve timed-out aggregation sidecars
thomhurst Sep 1, 2026
7fd08a7
fix(reporting): harden sidecar recovery
thomhurst Sep 3, 2026
27e5bfa
fix(reporting): revive enabled sidecars
thomhurst Sep 3, 2026
4a0cbe0
fix(reporting): serialize sidecar cleanup
thomhurst Sep 3, 2026
dd1c6f6
fix(reporting): persist sidecars before merge
thomhurst Sep 3, 2026
f359c63
fix(reporting): keep publication lock stable
thomhurst Sep 3, 2026
8078210
fix(reporting): preserve fallback ordering
thomhurst Sep 3, 2026
b52e7d6
fix(reporting): preserve contended sidecars
thomhurst Sep 3, 2026
30accde
fix: preserve newer report publications
thomhurst Sep 3, 2026
b2e60ba
fix: release cleanup lock before merge
thomhurst Sep 3, 2026
4e52379
fix: protect active report publications
thomhurst Sep 3, 2026
4eb30d4
fix: isolate timed-out report writes
thomhurst Sep 3, 2026
e969ae2
fix: embed report publication generation
thomhurst Sep 3, 2026
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 6 additions & 2 deletions docs/docs/guides/html-report.md
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,7 @@ The OS and runtime version are included automatically so that matrix builds (mul

Open it in any modern browser. The report is fully self-contained (single HTML file) and works offline.

A machine-readable JSON sidecar (`{AssemblyName}-{os}-{tfm}.tunit-report.json`) is written alongside the HTML report. It powers [report aggregation](/docs/guides/report-aggregation) — merging reports from multiple test projects into one — and can be disabled with `TUNIT_DISABLE_JSON_REPORT=true`.
A machine-readable JSON sidecar (`{AssemblyName}-{os}-{tfm}.tunit-report.json`) is written alongside the HTML report. It powers [report aggregation](/docs/guides/report-aggregation) — merging reports from multiple test projects into one — and can be disabled with `TUNIT_DISABLE_JSON_REPORT=true` or `context.Settings.Reporting.JsonReportEnabled = false`.

Running many test projects and want **one combined report instead of one per project**? See [Aggregated Reports](/docs/guides/report-aggregation).

Expand Down Expand Up @@ -53,6 +53,8 @@ export TUNIT_DISABLE_HTML_REPORTER=true

Accepts: `true`, `1`, `yes` (case-insensitive).

For version-controlled project configuration, set `context.Settings.Reporting.HtmlReportEnabled = false` in a `[Before(HookType.TestDiscovery)]` hook instead.

### Deprecated: `--report-html` Flag

The `--report-html` flag is deprecated since the report is now generated by default. Using it will show a deprecation warning but will not cause an error.
Expand Down Expand Up @@ -135,6 +137,8 @@ This is useful if you:

The report file and the `GITHUB_STEP_SUMMARY` are still generated.

This can also be configured in code with `context.Settings.Reporting.ArtifactUploadEnabled = false`.

### Viewing the Report

After the workflow run completes:
Expand Down Expand Up @@ -234,7 +238,7 @@ The collector uses **smart sampling**: spans from known test traces are fully re

### Report Not Generated

- Check that `TUNIT_DISABLE_HTML_REPORTER` is not set in your environment
- Check that `TUNIT_DISABLE_HTML_REPORTER` is not set and `context.Settings.Reporting.HtmlReportEnabled` is not `false`
- Verify that the `TestResults/` directory is writable
- Check the console output for any warning messages about report generation failures

Expand Down
6 changes: 3 additions & 3 deletions docs/docs/guides/report-aggregation.md
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@ Report aggregation merges all of that into **one combined HTML report and one Gi

## How It Works

1. Alongside every HTML report, TUnit writes a machine-readable sidecar: `{AssemblyName}-{os}-{tfm}.tunit-report.json`. This is on by default (disable with `TUNIT_DISABLE_JSON_REPORT=true`).
1. Alongside every HTML report, TUnit writes a machine-readable sidecar: `{AssemblyName}-{os}-{tfm}.tunit-report.json`. This is on by default (disable with `TUNIT_DISABLE_JSON_REPORT=true` or `context.Settings.Reporting.JsonReportEnabled = false`).
2. With aggregation enabled, each test process also copies its sidecar into a directory shared by all sibling processes.
3. As each process finishes, it takes a cross-process lock, reads *all* sidecars present so far, and regenerates the merged HTML report and the summary block. The last process to finish naturally leaves the complete aggregate — no process ever needs to know whether it is the last one.

Expand Down Expand Up @@ -166,11 +166,11 @@ tunit-report merge --directory <dir> [options]
| --- | --- |
| `TUNIT_AGGREGATE_REPORTS` | Unset (default) — cooperative merge wherever a shared directory is resolvable (GitHub Actions, or explicit `TUNIT_AGGREGATE_DIR`); silently off otherwise. `defer` — persist sidecars + merged HTML only; no summary blocks (multi-step scenarios). `off` (also `false`/`0`/`no`/`disabled`/`none`) — no aggregation. |
| `TUNIT_AGGREGATE_DIR` | Shared directory for sidecars and the merged report. Required outside GitHub Actions; optional override on GitHub Actions. |
| `TUNIT_DISABLE_JSON_REPORT` | Disables the JSON sidecar written next to the HTML report. Note: sidecars are what aggregation and `tunit-report` consume. |
| `TUNIT_DISABLE_JSON_REPORT` | Disables the JSON sidecar written next to the HTML report. Programmatic equivalent: `context.Settings.Reporting.JsonReportEnabled = false`. Note: sidecars are what aggregation and `tunit-report` consume. |

## Notes & Limitations

- Aggregation is driven by the HTML reporter's data pipeline — if you set `TUNIT_DISABLE_HTML_REPORTER`, no sidecars are produced and there is nothing to merge.
- Aggregation is driven by the HTML reporter's data pipeline — if you set `TUNIT_DISABLE_HTML_REPORTER` or `context.Settings.Reporting.HtmlReportEnabled = false`, no sidecars are produced and there is nothing to merge.
- With cooperative mode (the default) across *multiple steps in the same job*, each step appends its own progressively-larger block (earlier steps' blocks can't be rewritten). Use `defer` + the tool for that layout, or `off` to restore per-suite blocks.
- Suites are identified per assembly + OS + TFM, so multi-targeted projects appear as separate rows (e.g. `MyTests (.NET 8.0.x)` / `MyTests (.NET 9.0.x)`).
- The GitHub step summary is capped at 1 MB by GitHub; the aggregated block replaces N per-suite blocks, so it usually *reduces* summary size.
Expand Down
3 changes: 2 additions & 1 deletion docs/docs/reference/command-line-flags.md
Original file line number Diff line number Diff line change
Expand Up @@ -137,7 +137,8 @@ Please note that for the coverage and trx report, you need to install [additiona

--report-html
(Deprecated) The HTML report is now generated by default.
Disable it with the TUNIT_DISABLE_HTML_REPORTER environment variable.
Disable it with TUNIT_DISABLE_HTML_REPORTER or set
context.Settings.Reporting.HtmlReportEnabled = false.

--report-html-filename
Path for the HTML test report file
Expand Down
23 changes: 20 additions & 3 deletions docs/docs/reference/environment-variables.md
Original file line number Diff line number Diff line change
Expand Up @@ -60,9 +60,11 @@ Accepts truthy values: `true`, `1`, `yes` (case-insensitive).

**Use case:** When you don't need the HTML report or want to reduce disk I/O. The report is written to `TestResults/{AssemblyName}-report.html` by default.

**Programmatic equivalent:** `context.Settings.Reporting.HtmlReportEnabled = false`

### TUNIT_DISABLE_ARTIFACT_UPLOAD

Skips the autmoatic upload of the html report but still generates the files.
Skips automatic upload of the HTML report but still generates the files.

```bash
export TUNIT_DISABLE_ARTIFACT_UPLOAD=true
Expand All @@ -77,6 +79,20 @@ GitHub changed the server-side behaviour of `upload-artifacts` since v4 and hasn
Forgejo and gitea don't implement this new artifact endpoint but the runners set `GITHUB_ACTIONS=true`.
In this case it attempts to upload the report a few times until it eventually backs off after 30s.

**Programmatic equivalent:** `context.Settings.Reporting.ArtifactUploadEnabled = false`

### TUNIT_DISABLE_JSON_REPORT

Disables the machine-readable JSON sidecar written alongside the HTML report.

```bash
export TUNIT_DISABLE_JSON_REPORT=true
```

Accepts truthy values: `true`, `1`, `yes` (case-insensitive).

**Programmatic equivalent:** `context.Settings.Reporting.JsonReportEnabled = false`

### TUNIT_DISABLE_JUNIT_REPORTER

Disables the JUnit XML reporter.
Expand Down Expand Up @@ -284,8 +300,9 @@ When the same setting is configured in multiple places, TUnit follows this prior
| `TUNIT_DISABLE_GITHUB_REPORTER` | - | Disables GitHub reporter |
| `TUNIT_DISABLE_JUNIT_REPORTER` | - | Disables JUnit reporter |
| `TUNIT_ENABLE_JUNIT_REPORTER` | - | Enables JUnit reporter |
| `TUNIT_DISABLE_HTML_REPORTER` | - | Disables HTML report generation |
| `TUNIT_DISABLE_ARTIFACT_UPLOAD` | - | Keeps the HTML report file but skips the GitHub Actions artifact upload |
| `TUNIT_DISABLE_HTML_REPORTER` | - | Disables HTML report generation (`context.Settings.Reporting.HtmlReportEnabled = false`) |
| `TUNIT_DISABLE_JSON_REPORT` | - | Disables the machine-readable JSON sidecar (`context.Settings.Reporting.JsonReportEnabled = false`) |
| `TUNIT_DISABLE_ARTIFACT_UPLOAD` | - | Keeps the HTML report file but skips the GitHub Actions artifact upload (`context.Settings.Reporting.ArtifactUploadEnabled = false`) |
| `JUNIT_XML_OUTPUT_PATH` | - | JUnit output path |
| `TUNIT_MAX_PARALLEL_TESTS` | `--maximum-parallel-tests` | Max parallel tests |
| `TUNIT_EXECUTION_MODE` | `--reflection` | Selects source-generation (`sourcegeneration`/`aot`) or `reflection` execution mode |
Expand Down
12 changes: 12 additions & 0 deletions docs/docs/reference/programmatic-configuration.md
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,7 @@ Settings are organized into logical groups:
- `Parallelism` — concurrent test execution limits
- `Execution` — runtime behavior such as fail-fast
- `Display` — output and display options
- `Reporting` — HTML report generation and publishing
- `Mocks` — defaults for TUnit.Mocks when the package is referenced

## Usage
Expand All @@ -33,6 +34,7 @@ public class TestSetup
context.Settings.Timeouts.DefaultTestTimeout = TimeSpan.FromMinutes(5);
context.Settings.Timeouts.DefaultHookTimeout = TimeSpan.FromMinutes(2);
context.Settings.Execution.FailFast = true;
context.Settings.Reporting.HtmlReportEnabled = false;
context.Settings.Mocks.DefaultMode = MockBehavior.Strict;

return Task.CompletedTask;
Expand Down Expand Up @@ -73,6 +75,16 @@ Settings are accessed exclusively through `context.Settings` in the discovery ho
|---|---|---|---|
| `FailFast` | `bool` | `false` | Cancels the remaining test run after the first test failure. |

### `context.Settings.Reporting`

| Property | Type | Default | Description |
|---|---|---|---|
| `HtmlReportEnabled` | `bool` | `true` | Generates the HTML test report. |
| `JsonReportEnabled` | `bool` | `true` | Generates the machine-readable JSON sidecar used by report aggregation. |
| `ArtifactUploadEnabled` | `bool` | `true` | Uploads the HTML report as an artifact when supported by the CI environment. |

The corresponding `TUNIT_DISABLE_HTML_REPORTER`, `TUNIT_DISABLE_JSON_REPORT`, and `TUNIT_DISABLE_ARTIFACT_UPLOAD` environment variables take precedence over these values.

### `context.Settings.Mocks`

Available when `TUnit.Mocks` is referenced.
Expand Down
28 changes: 28 additions & 0 deletions src/TUnit.Core/Settings/ReportingSettings.cs
Original file line number Diff line number Diff line change
@@ -0,0 +1,28 @@
namespace TUnit.Core.Settings;

/// <summary>
/// Controls built-in report generation and publishing.
/// </summary>
public sealed class ReportingSettings
{
internal ReportingSettings() { }

/// <summary>
/// Whether to generate the HTML test report. Default: <c>true</c>.
/// Precedence: <c>TUNIT_DISABLE_HTML_REPORTER</c> → TUnitSettings → built-in default.
/// </summary>
public bool HtmlReportEnabled { get; set; } = true;

/// <summary>
/// Whether to generate the machine-readable JSON report sidecar. Default: <c>true</c>.
/// Precedence: <c>TUNIT_DISABLE_JSON_REPORT</c> → TUnitSettings → built-in default.
/// </summary>
public bool JsonReportEnabled { get; set; } = true;

/// <summary>
/// Whether to upload the HTML report as an artifact when supported by the CI environment.
/// Default: <c>true</c>.
/// Precedence: <c>TUNIT_DISABLE_ARTIFACT_UPLOAD</c> → TUnitSettings → built-in default.
/// </summary>
public bool ArtifactUploadEnabled { get; set; } = true;
}
5 changes: 5 additions & 0 deletions src/TUnit.Core/Settings/TUnitSettings.cs
Original file line number Diff line number Diff line change
Expand Up @@ -38,4 +38,9 @@ internal TUnitSettings() { }
/// Controls test run behavior.
/// </summary>
public ExecutionSettings Execution { get; } = new();

/// <summary>
/// Controls report generation and publishing.
/// </summary>
public ReportingSettings Reporting { get; } = new();
}
Loading
Loading