Skip to content

Dependencies: Upgrade NUnit and related test dependencies to latest major versions#22155

Merged
AndyButland merged 7 commits into
v18/devfrom
v18/task/update-test-server-side-dependencies
Mar 19, 2026
Merged

Dependencies: Upgrade NUnit and related test dependencies to latest major versions#22155
AndyButland merged 7 commits into
v18/devfrom
v18/task/update-test-server-side-dependencies

Conversation

@NguyenThuyLan

Copy link
Copy Markdown
Contributor

Description

Upgrade NUnit from 3.x to 4.5.1 and update related test dependencies and test code to align with NUnit 4.x breaking changes and deprecations.

Package upgrades

  • NUnit: 3.x → 4.5.1
  • AutoFixture.NUnit3AutoFixture.NUnit4 (to resolve compatibility constraint with NUnit 4.x)

Test code migrations

  • Remove CollectionAssert and use Constraint model (Assert.That)
  • Remove Timeout and use CancelAfter

References

@NguyenThuyLan NguyenThuyLan changed the base branch from main to v18/dev March 17, 2026 08:34
@NguyenThuyLan NguyenThuyLan marked this pull request as draft March 17, 2026 08:34
LanThuyNguyen and others added 5 commits March 18, 2026 10:30
  - Use Task.Run in CreateHost to escape NUnit 4's SynchronizationContext
    which deadlocks sync-over-async calls from async test methods.
  - Use await using for factory disposal to avoid same deadlock on shutdown
  - Remove WithWebHostBuilder which wraps the factory in a
    DelegatedWebApplicationFactory that bypasses the CreateHost override.
  - Add ContentRoot property to UmbracoWebApplicationFactory so content
    root can be set without WithWebHostBuilder.
  - Set ModelsBuilder mode to Nothing to prevent BootFailedException.
  - Add AddTestServices for infrastructure test doubles (MainDom, etc.).
@AndyButland

Copy link
Copy Markdown
Contributor

I'll revert the changes to the build pipeline now we've (hopefully) diagnosed the issue with the hanging test, but note them here just in case we have a need to do similar again and remember this issue:

- arguments: '--filter "$(testFilter)" --configuration $(buildConfiguration) --no-build ${{parameters.integrationReleaseTestFilter}}'
+ arguments: '--filter "$(testFilter)" --configuration $(buildConfiguration) --no-build --blame-hang --blame-hang-timeout 3m --blame-hang-dump-type mini ${{parameters.integrationReleaseTestFilter}}'

@AndyButland AndyButland enabled auto-merge (squash) March 19, 2026 11:03
@AndyButland AndyButland merged commit 7014f9a into v18/dev Mar 19, 2026
26 of 27 checks passed
@AndyButland AndyButland deleted the v18/task/update-test-server-side-dependencies branch March 19, 2026 11:49
andr317c added a commit that referenced this pull request Apr 21, 2026
Reverts commit 7014f9a on this branch only, as a diagnostic experiment
to validate the root-cause hypothesis for the Part3Of4 SQL Server nightly
hang on v18/dev.

Blame-hang dump showed the test host deadlocking inside
ContentCacheRefresher.HandleMemoryCache on a sync-over-async
.GetAwaiter().GetResult() call. Main and v17/dev have the identical
production code but do not hang, because they still run NUnit 3.14.0
whose pumping synchronization context masks sync-over-async deadlocks.
v18/dev's NUnit 4.5.1 removes that mask.

If the nightly passes with this revert, the hypothesis is confirmed and
the proper fix is to make ContentCacheRefresher async (separate change,
not included here).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
andr317c added a commit that referenced this pull request Apr 21, 2026
Two test files used NUnit 4 APIs added by later commits (outside #22155's
scope, so untouched by the revert in a50f89b):

- UdiTests.cs: removed unused `using NUnit.Framework.Legacy;` (namespace
  introduced in NUnit 4).
- BackOfficeAuthorizationInitializationMiddlewareTests.cs: swapped
  `[CancelAfter(5000)]` (NUnit 4) for its NUnit 3 equivalent
  `[Timeout(5000)]`. Same intent: fail fast on semaphore deadlock.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
andr317c added a commit that referenced this pull request Apr 21, 2026
Sets ThreadPool.SetMinThreads to max(ProcessorCount * 4, 32) in the
assembly-level [OneTimeSetUp]. Prevents thread-pool starvation deadlocks
on low-core CI agents when production sync-over-async paths (e.g.
ContentCacheRefresher.HandleMemoryCache) hold worker threads while
awaiting continuations.

Why: NUnit 4 (PR #22155) dropped the single-threaded pumping
synchronization context that NUnit 3 used, so sync-over-async that used
to complete via the test thread now strictly requires a free thread-pool
thread for its continuation. On Azure Pipelines hosted agents with small
core counts, the pool can be fully blocked by the waiting threads,
producing the 180-minute Part3Of4 SQL Server nightly timeouts observed
on v18/dev since 2026-04-10.

This is a test-only mitigation. The underlying sync-over-async pattern
in ContentCacheRefresher remains a latent production issue and should be
fixed separately by making Refresh async end-to-end.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
andr317c added a commit that referenced this pull request Apr 21, 2026
andr317c added a commit that referenced this pull request Apr 22, 2026
…2562)

Revert "Dependencies: Upgrade NUnit and related test dependencies to latest major versions (#22155)"

This reverts commit 7014f9a on v18/dev
to resolve the Part3Of4 SQL Server integration test nightly hangs that
started around 2026-04-10.

Root cause was confirmed by hang-dump analysis: a sync-over-async call
in ContentCacheRefresher.HandleMemoryCache
(.GetAwaiter().GetResult() on an async cache method) that NUnit 3's
pumping synchronization context had been quietly completing on the test
thread. NUnit 4 dropped that behaviour, so the continuation now requires
a free thread-pool thread; under CI conditions it deadlocks.

Reverting #22155 on a branch was verified to make the nightly pass.
This is a temporary rollback to unblock v18; the proper fix is to make
ContentCacheRefresher.Refresh async end-to-end, tracked separately.

Additional adjustments beyond the pure revert to keep the branch
compiling:

- CoreConfigurationHttpTests.cs: added `using Umbraco.Cms.Core.Services;`
  for IUserService (referenced by post-#22155 code unaffected by the
  revert).
- ContentVersionCleanupServiceTest.cs: merged imports so both
  AutoFixture.NUnit3 (from revert) and Microsoft.Extensions.Options
  (from unrelated later commit) stay.
- UdiTests.cs and ContentPermissionResourceTests.cs: removed unused
  `using NUnit.Framework.Legacy;` (namespace introduced in NUnit 4).
- BackOfficeAuthorizationInitializationMiddlewareTests.cs: replaced
  `[CancelAfter(5000)]` with its NUnit 3 equivalent `[Timeout(5000)]`.

Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants