Skip to content

fix(html-report): extract categories using MTP Key=name convention#5946

Merged
thomhurst merged 1 commit into
mainfrom
fix/html-report-category-extraction
May 17, 2026
Merged

fix(html-report): extract categories using MTP Key=name convention#5946
thomhurst merged 1 commit into
mainfrom
fix/html-report-category-extraction

Conversation

@thomhurst
Copy link
Copy Markdown
Owner

Summary

Fixes a latent bug in the HTML report where [Category] attributes never made it into the report's categories array — leaving the category-pill filter UI shipped in #5486 permanently dark.

Root cause

Two coordinated emissions, one inverted check:

File Emits Result
TestExtensions.cs:71 new TestMetadataProperty(category) — single-arg ctor Key=""Async"", Value=""""
HtmlReporter.cs:521 if (IsNullOrEmpty(meta.Key)) … categories.Add(meta.Value) branches to else, category lands in customProperties

MTP's single-arg TestMetadataProperty(string) constructor stores the argument in Key and defaults Value to """" (verified via reflection on Microsoft.Testing.Platform.dll 2.2.3). Microsoft's own Microsoft.Testing.Extensions.VSTestBridge uses the same shape — new TestMetadataProperty(category, string.Empty) (ObjectModelConverters.cs lines 72-76). So the emitter is right; the extractor's field check is inverted.

Before the fix, decoding the embedded gzipped data blob from a report shows:

""customProperties"":[{""key"":""Async"",""value"":""""},{""key"":""DataSource"",""value"":""""}]

After:

""categories"":[""Async"",""DataSource""]

buildCatPills() (HtmlReportGenerator.cs:1353) only renders when catNames.length > 0, so with no categories ever populated the #categoryPills row stayed display:none.

Fix

Swap the field check — IsNullOrEmpty(meta.Value) distinguishes the category shape (name in Key, empty Value) from the trait/property shape (non-empty Value). One-file change.

TestFilterService is unaffected — it builds its own PropertyBag from TestDetails.Categories directly, which is why CLI --treenode-filter ""/*/*/*/*[Category=X]"" always worked.

Side-effect: [Property(""X"", """")]

A user-defined [Property(""X"", """")] (empty value) will now classify as a category named ""X"" rather than a custom property with empty value. This is consistent with the convention (empty value = tag) and matches Microsoft's bridge behaviour.

Test plan

  • New unit test ExtractTestResult_SortsTestMetadataProperty_Into_Categories_And_CustomProperties pins the split between the two TestMetadataProperty shapes.
  • All 14 HtmlReporterTests pass locally (dotnet test --treenode-filter ""/*/*/HtmlReporterTests/*"").
  • Verified end-to-end: ran TUnit.TestProject filtered to CategoryTests (which uses [Category(""ClassCategory"")] [Category(""MethodCategory"")] etc.) and confirmed the decoded report JSON now contains ""categories"":[""ClassCategory2"",""ClassCategory"",""MethodCategory2"",""MethodCategory""].
  • (Reviewer) Spot-check a real report in a browser: category pills should appear under the toolbar for any test class with [Category].

Refs #5912 — discovered while investigating why categories weren't filterable from the HTML report despite being well-supported in the code.

…convention

`HtmlReporter.ExtractTestResult` was checking `IsNullOrEmpty(meta.Key)` to
distinguish categories from key/value custom properties, but Microsoft's
own VSTestBridge convention (and TUnit's own `TestExtensions`) emits
categories as `TestMetadataProperty(name, "")` — name in Key, Value empty.
The check was inverted, so every `[Category]` silently landed in
`customProperties` as `{key:"Async", value:""}` instead of populating the
`categories` array. As a result the HTML report's category-pill UI
(`buildCatPills()` / `#categoryPills`) had nothing to render and stayed
hidden, even for projects that tag every test class.

Switch the field check: `IsNullOrEmpty(meta.Value)` → category (name in
Key), otherwise → custom property. Matches the MTP/VSTestBridge convention
already used elsewhere in TUnit (`TestFilterService.cs:248`,
`TestExtensions.cs:71`) and lets the existing pill UI shipped in #5486
finally light up.

Adds a unit test pinning the split between the two TestMetadataProperty
shapes so the extraction can't silently regress again.
Copy link
Copy Markdown
Contributor

@claude claude Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

Overall: Clean, well-motivated fix. Approve.

The Fix (HtmlReporter.cs:521)

The inverted field check is the correct diagnosis. MTP's single-arg ctor stores the value in with defaulting to , so the previous branch was never entered for real categories — they silently fell through to customProperties. The swap to IsNullOrEmpty(meta.Value) matches the actual MTP/VSTestBridge convention and is well-evidenced by the referenced source.

Visibility change (private → internal static)

Safe — TUnit.Engine.Tests is already in InternalsVisibleTo in the csproj, so this doesn't expand the public surface area.

Test coverage

The new test ExtractTestResult_SortsTestMetadataProperty_Into_Categories_And_CustomProperties directly exercises all three cases (two categories + one trait) and pins the split. The comment explaining the MTP convention is justified given how non-obvious the ctor behavior is.

Minor: edge case worth noting

A TestMetadataProperty("", "") (both empty) would now add an empty string to categories. This was already broken before (it would have added empty string to custom properties), so it's not a regression — but it might be worth a guard like !string.IsNullOrEmpty(meta.Key) in the category branch if defensive hygiene is desired. Not a blocker.

Side-effect documented in PR

The behavior change for [Property("X", "")] (reclassified from custom property to category) is intentional and consistent with the convention. The PR description calls this out clearly — no action needed.

The fix is minimal, targeted, and well-tested. Ship it.

Copy link
Copy Markdown
Contributor

@claude claude Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

Overall: Clean, well-motivated fix. Approve.

The Fix (HtmlReporter.cs:521)

The inverted field check is the correct diagnosis. MTP's single-arg TestMetadataProperty(string) ctor stores the value in Key with Value defaulting to "", so the previous IsNullOrEmpty(meta.Key) branch was never entered for real categories — they silently fell through to customProperties. The swap to IsNullOrEmpty(meta.Value) matches the actual MTP/VSTestBridge convention and is well-evidenced by the referenced source.

Visibility change (private → internal static)

Safe — TUnit.Engine.Tests is already in InternalsVisibleTo in the csproj, so this doesn't expand the public surface area.

Test coverage

The new test ExtractTestResult_SortsTestMetadataProperty_Into_Categories_And_CustomProperties directly exercises all three cases (two categories + one trait) and pins the split. The comment explaining the MTP convention is justified given how non-obvious the ctor behavior is.

Minor: edge case worth noting

A TestMetadataProperty("", "") (both empty) would now add an empty string to categories. This was already broken before (it would have added empty string to custom properties instead), so it's not a regression — but a guard like !string.IsNullOrEmpty(meta.Key) inside the category branch would be a cheap defensive improvement. Not a blocker.

Side-effect documented in PR

The behavior change for [Property("X", "")] (reclassified from custom property to category) is intentional and consistent with the convention. The PR description calls this out clearly — no action needed.

The fix is minimal, targeted, and well-tested. Ship it.

@codacy-production
Copy link
Copy Markdown

Up to standards ✅

🟢 Issues 0 issues

Results:
0 new issues

View in Codacy

🟢 Metrics 0 complexity

Metric Results
Complexity 0

View in Codacy

NEW Get contextual insights on your PRs based on Codacy's metrics, along with PR and Jira context, without leaving GitHub. Enable AI reviewer
TIP This summary will be updated as you push new changes.

@thomhurst thomhurst enabled auto-merge (squash) May 17, 2026 21:42
@thomhurst thomhurst merged commit 7819c7a into main May 17, 2026
15 checks passed
@thomhurst thomhurst deleted the fix/html-report-category-extraction branch May 17, 2026 21:52
@claude claude Bot mentioned this pull request May 18, 2026
1 task
BenjaminMichaelis pushed a commit to BenjaminMichaelis/DotnetTemplates that referenced this pull request May 18, 2026
Updated [TUnit](https://github.com/thomhurst/TUnit) from 1.44.39 to
1.45.0.

<details>
<summary>Release notes</summary>

_Sourced from [TUnit's
releases](https://github.com/thomhurst/TUnit/releases)._

## 1.45.0

<!-- Release notes generated using configuration in .github/release.yml
at v1.45.0 -->

## What's Changed
### Other Changes
* fix(generator): place CallerArgumentExpression before params in
[GenerateAssertion] emit by @​JohnVerheij in
thomhurst/TUnit#5940
* fix(sourcegen): drop covariant TActual when [GenerateAssertion] method
has its own type parameters by @​JohnVerheij in
thomhurst/TUnit#5935
* feat(assertions): add CancellationToken overload to WaitsFor and
Eventually by @​JohnVerheij in
thomhurst/TUnit#5938
* fix(html-report): extract categories using MTP Key=name convention by
@​thomhurst in thomhurst/TUnit#5946
* feat(html-report): rewrite as split-pane design template by
@​thomhurst in thomhurst/TUnit#5947
### Dependencies
* chore(deps): update microsoft.testing to 2.2.3 by @​thomhurst in
thomhurst/TUnit#5927
* chore(deps): update mstest to 4.2.3 by @​thomhurst in
thomhurst/TUnit#5928
* chore(deps): update tunit to 1.44.39 by @​thomhurst in
thomhurst/TUnit#5929
* chore(deps): update aspire to 13.3.3 by @​thomhurst in
thomhurst/TUnit#5933
* chore(deps): update dependency dompurify to v3.4.4 by @​thomhurst in
thomhurst/TUnit#5944
* chore(deps): update dependency qs to v6.15.2 by @​thomhurst in
thomhurst/TUnit#5941


**Full Changelog**:
thomhurst/TUnit@v1.44.39...v1.45.0

Commits viewable in [compare
view](thomhurst/TUnit@v1.44.39...v1.45.0).
</details>

[![Dependabot compatibility
score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=TUnit&package-manager=nuget&previous-version=1.44.39&new-version=1.45.0)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)

Dependabot will resolve any conflicts with this PR as long as you don't
alter it yourself. You can also trigger a rebase manually by commenting
`@dependabot rebase`.

[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)

---

<details>
<summary>Dependabot commands and options</summary>
<br />

You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits
that have been made to it
- `@dependabot show <dependency name> ignore conditions` will show all
of the ignore conditions of the specified dependency
- `@dependabot ignore this major version` will close this PR and stop
Dependabot creating any more for this major version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop
Dependabot creating any more for this minor version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop
Dependabot creating any more for this dependency (unless you reopen the
PR or upgrade to it yourself)


</details>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
github-actions Bot pushed a commit to IntelliTect/CodingGuidelines that referenced this pull request May 19, 2026
Updated [TUnit.Core](https://github.com/thomhurst/TUnit) from 1.44.0 to
1.45.8.

<details>
<summary>Release notes</summary>

_Sourced from [TUnit.Core's
releases](https://github.com/thomhurst/TUnit/releases)._

## 1.45.8

<!-- Release notes generated using configuration in .github/release.yml
at v1.45.8 -->

## What's Changed
### Other Changes
* fix(aspire): route CreateHttpClient through IHttpClientFactory by
@​thomhurst in thomhurst/TUnit#5957
### Dependencies
* chore(deps): update tunit to 1.45.0 by @​thomhurst in
thomhurst/TUnit#5949
* chore(deps): update dependency dompurify to v3.4.5 by @​thomhurst in
thomhurst/TUnit#5951
* chore(deps): update dependency
microsoft.testing.extensions.codecoverage to 18.7.0 by @​thomhurst in
thomhurst/TUnit#5953
* chore(deps): update dependency coverlet.collector to 10.0.1 by
@​thomhurst in thomhurst/TUnit#5952
* chore(deps): update dependency polyfill to 10.6.0 by @​thomhurst in
thomhurst/TUnit#5955
* chore(deps): update dependency polyfill to 10.6.0 by @​thomhurst in
thomhurst/TUnit#5954


**Full Changelog**:
thomhurst/TUnit@v1.45.0...v1.45.8

## 1.45.0

<!-- Release notes generated using configuration in .github/release.yml
at v1.45.0 -->

## What's Changed
### Other Changes
* fix(generator): place CallerArgumentExpression before params in
[GenerateAssertion] emit by @​JohnVerheij in
thomhurst/TUnit#5940
* fix(sourcegen): drop covariant TActual when [GenerateAssertion] method
has its own type parameters by @​JohnVerheij in
thomhurst/TUnit#5935
* feat(assertions): add CancellationToken overload to WaitsFor and
Eventually by @​JohnVerheij in
thomhurst/TUnit#5938
* fix(html-report): extract categories using MTP Key=name convention by
@​thomhurst in thomhurst/TUnit#5946
* feat(html-report): rewrite as split-pane design template by
@​thomhurst in thomhurst/TUnit#5947
### Dependencies
* chore(deps): update microsoft.testing to 2.2.3 by @​thomhurst in
thomhurst/TUnit#5927
* chore(deps): update mstest to 4.2.3 by @​thomhurst in
thomhurst/TUnit#5928
* chore(deps): update tunit to 1.44.39 by @​thomhurst in
thomhurst/TUnit#5929
* chore(deps): update aspire to 13.3.3 by @​thomhurst in
thomhurst/TUnit#5933
* chore(deps): update dependency dompurify to v3.4.4 by @​thomhurst in
thomhurst/TUnit#5944
* chore(deps): update dependency qs to v6.15.2 by @​thomhurst in
thomhurst/TUnit#5941


**Full Changelog**:
thomhurst/TUnit@v1.44.39...v1.45.0

## 1.44.39

<!-- Release notes generated using configuration in .github/release.yml
at v1.44.39 -->

## What's Changed
### Other Changes
* fix(tests): retry trx read to dodge MTP post-exit flush race on
Windows by @​thomhurst in thomhurst/TUnit#5888
* fix(pipeline): timeout + retry InstallPlaywrightModule so a hung
download fails fast by @​thomhurst in
thomhurst/TUnit#5889
* fix(otel): require two consecutive idle windows in DrainAsync to catch
in-transit POSTs by @​thomhurst in
thomhurst/TUnit#5890
* test(assertions): drop flaky wall-clock upper bound on WaitsFor
timeout test by @​thomhurst in
thomhurst/TUnit#5886
* fix(sourcegen): drop spurious ')' in MethodAssertionGenerator
Task<bool> emit by @​JohnVerheij in
thomhurst/TUnit#5920
* fix(sourcegen): merge generic parameter lists in [AssertionExtension]
emit by @​JohnVerheij in thomhurst/TUnit#5921
* fix(aspnetcore): scope correlation processor per-factory to stop
cross-factory tag leak by @​thomhurst in
thomhurst/TUnit#5891
* Changed FSharp.Core version to 10.1.300 by @​licon4812 in
thomhurst/TUnit#5909
* feat(mocks): add Mock.HttpClientFactory() helper by @​thomhurst in
thomhurst/TUnit#5894
* Harden WaitsFor timeout test by @​thomhurst in
thomhurst/TUnit#5926
* fix(sourcegen): emit `default` literal for value-type assertion
parameters by @​JohnVerheij in
thomhurst/TUnit#5919
### Dependencies
* chore(deps): update dependency nunit to 4.6.0 by @​thomhurst in
thomhurst/TUnit#5826
* chore(deps): update tunit to 1.44.0 by @​thomhurst in
thomhurst/TUnit#5882
* chore(deps): update dependency mockolate to 3.2.0 by @​thomhurst in
thomhurst/TUnit#5892
* chore(deps): update dependency yaml to v2.9.0 by @​thomhurst in
thomhurst/TUnit#5887
* chore(deps): update dependency nuget.protocol to 7.6.0 by @​thomhurst
in thomhurst/TUnit#5897
* chore(deps): update dependency microsoft.entityframeworkcore to 10.0.8
by @​thomhurst in thomhurst/TUnit#5898
* chore(deps): update dependency microsoft.templateengine.authoring.cli
to v10.0.300 by @​thomhurst in
thomhurst/TUnit#5899
* chore(deps): update microsoft.extensions by @​thomhurst in
thomhurst/TUnit#5905
* chore(deps): update microsoft.aspnetcore to 10.0.8 by @​thomhurst in
thomhurst/TUnit#5904
* chore(deps): update dependency
microsoft.templateengine.authoring.templateverifier to 10.0.300 by
@​thomhurst in thomhurst/TUnit#5902
* chore(deps): update aspire to 13.3.1 by @​thomhurst in
thomhurst/TUnit#5900
* chore(deps): update dependency system.commandline to 2.0.8 by
@​thomhurst in thomhurst/TUnit#5903
* chore(deps): update dependency azure.storage.blobs to 12.28.0 by
@​thomhurst in thomhurst/TUnit#5910
* chore(deps): update dependency dotnet-sdk to v10.0.300 by @​thomhurst
in thomhurst/TUnit#5901
* chore(deps): update dependency stackexchange.redis to 2.13.1 by
@​thomhurst in thomhurst/TUnit#5906
* chore(deps): update aspire to 13.3.2 by @​thomhurst in
thomhurst/TUnit#5924
* chore(deps): bump mermaid from 11.12.2 to 11.15.0 in /docs by
@​dependabot[bot] in thomhurst/TUnit#5893
* chore(deps): update dependency streamjsonrpc to 2.24.92 by @​thomhurst
in thomhurst/TUnit#5915
* chore(deps): update dependency dompurify to v3.4.3 by @​thomhurst in
thomhurst/TUnit#5913
* chore(deps): update microsoft.build to 18.6.3 by @​thomhurst in
thomhurst/TUnit#5914


**Full Changelog**:
thomhurst/TUnit@v1.44.0...v1.44.39

Commits viewable in [compare
view](thomhurst/TUnit@v1.44.0...v1.45.8).
</details>

[![Dependabot compatibility
score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=TUnit.Core&package-manager=nuget&previous-version=1.44.0&new-version=1.45.8)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)

Dependabot will resolve any conflicts with this PR as long as you don't
alter it yourself. You can also trigger a rebase manually by commenting
`@dependabot rebase`.

[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)

---

<details>
<summary>Dependabot commands and options</summary>
<br />

You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits
that have been made to it
- `@dependabot show <dependency name> ignore conditions` will show all
of the ignore conditions of the specified dependency
- `@dependabot ignore this major version` will close this PR and stop
Dependabot creating any more for this major version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop
Dependabot creating any more for this minor version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop
Dependabot creating any more for this dependency (unless you reopen the
PR or upgrade to it yourself)


</details>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant