Skip to content

Limit the number of concurrent tests - #1305

Merged
meziantou merged 1 commit into
mainfrom
feature/limit-concurrent-tests-f5cc0d
Aug 19, 2026
Merged

meziantou merged 1 commit into
mainfrom
feature/limit-concurrent-tests-f5cc0d

Conversation

@meziantou

Copy link
Copy Markdown
Owner

Every test compiles code with Roslyn, so running one test per CPU thread uses a lot of memory. On a 15 thread machine, a single test project peaks at 20 GB, and dotnet test at the solution level runs the 5 Roslyn version projects in parallel, which multiplies it.

Changes

  • tests/Meziantou.Analyzer.Test/testconfig.json: limits xUnit.maxParallelThreads to 8. The file is shared by all the Roslyn versions, as their projects are in the same folder, and Microsoft.Testing.Platform.MSBuild copies it to the output folder of each project as Meziantou.Analyzer.Test.testconfig.json.
  • AGENTS.md: documents the new file, and dotnet test --max-parallel-test-modules to limit how many test projects run at the same time (there is no configuration file equivalent for that one).

Measurements

Roslyn 5.9 test project (3663 tests), Release, 15 CPU threads:

Max parallel threads Peak RSS Duration
default (1 per CPU thread) 20.2 GB 57s
8 3.0 GB 56s
4 1.5 GB 64s

The memory grows much faster than the throughput, so 8 divides the memory by ~7 for the same duration.

Notes for the reviewer

  • The value is a fixed number rather than a multiplier such as "0.5x", because the CI machines have 4 threads: a multiplier would halve their current parallelism and slow the jobs down, while 8 leaves them unchanged and only caps the bigger development machines.
  • --max-parallel-test-modules bounds the number of test projects, not the memory of one project, so it does not replace this setting: a single project would still peak at 20 GB.

Verification

  • dotnet test tests/Meziantou.Analyzer.Test/Meziantou.Analyzer.Test.roslyn4.8.csproj --configuration Release: 3533/3533 passed
  • Roslyn 5.9 test project: 3663/3663 passed, peaking at 3.7 GB instead of 20 GB
  • The generated Meziantou.Analyzer.Test.testconfig.json is in the output folder of both the roslyn4.8 and the roslyn5.9 projects

Every test compiles code with Roslyn, so running one test per CPU thread
uses a lot of memory: a single test project peaks at 20 GB on a 15 thread
machine, and the test projects of the Roslyn versions run in parallel.

Add a testconfig.json limiting 'xUnit.maxParallelThreads' to 8. It is
shared by all the Roslyn versions as they are in the same folder, and
Microsoft.Testing.Platform.MSBuild copies it to the output folder of each
project. The peak memory of a test project goes from 20 GB to 3 GB, for
the same duration (57s vs 58s).

The value is not a multiplier such as '0.5x' so that the 4 threads CI
machines keep their current parallelism.
@meziantou
meziantou enabled auto-merge (squash) August 19, 2026 20:08
@meziantou
meziantou merged commit 407da45 into main Aug 19, 2026
12 checks passed
@meziantou
meziantou deleted the feature/limit-concurrent-tests-f5cc0d branch August 19, 2026 20:11
This was referenced Aug 19, 2026
This was referenced Sep 21, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant