Implement parallel executor service without `ForkJoinPool` #5060

marcphilipp · 2025-10-13T11:13:12Z

This PR introduces a new implementation of HierarchicalTestExecutorService that runs rests in parallel and has limited work stealing capabilities but is not based on ForkJoinPool. It avoids its pitfalls (such as #3945) and #3108 but may require additional threads because its work stealing is limited to direct children. Contrary to the ForkJoinPool implementation, the new executor service guarantees that no more than parallelism test nodes are executed in parallel.

My intention is to initially ship this implementation as an opt-in feature (via the new junit.jupiter.execution.parallel.executor configuration parameter) in 6.1, make it an opt-out feature in 6.2, and drop support for the ForkJoinPool-based implementation in a later to-be-determined release.

The PR is not yet finished but feedback is already welcome! If you use parallel test execution in your projects (or other test engines), it would be great if you could try out the new implementation and report your observations.

Resolves #3108.

I hereby agree to the terms of the JUnit Contributor License Agreement.

Definition of Done

There are no TODOs left in the code
Method preconditions are checked and documented in the method's Javadoc
Coding conventions (e.g. for logging) have been followed
Change is covered by automated tests including corner cases, errors, and exception handling
Public API has Javadoc and @API annotations
Change is documented in the User Guide and Release Notes

* More precise clock * Fixed width thread names * Abbreviated package names

To avoid race conditions with other workers that lead to stalling.

mpkorstanje · 2025-10-14T12:42:21Z

...rg/junit/platform/engine/support/hierarchical/ConcurrentHierarchicalTestExecutorService.java

+			var allForkedChildren = forkConcurrentChildren(testTasks, isolatedTasks::add, sameThreadTasks);
+			executeAll(sameThreadTasks);
+			var remainingForkedChildren = stealWork(allForkedChildren);
+			waitFor(remainingForkedChildren);


The name remainingForkedChildren isn't quite right. These can be children that are picked up by other threads or children that are currently blocked by a locked resource.

So before waiting this thread could execute any remaining unexecuted children in a blocking fashion.

Do you have a proposal? concurrentChildren?

Not immediately. I'd rework the logic first then see what falls out.

The logic would have to make a distinction between children that executed by another thread and children that were skipped over to due to resource locks. The latter can be executed blockingly before calling waitFor. While looping through the resourceLockedChildren(?) the remainingForkedChildren would also have to be updated as other threads pick them up.

So maybe name remainingForkedChildren to childrenExecutedOnAnotherThread, but that is unwieldy. 😅

And I should have prefaced this by noting that the current thread starts to wait to early, there is potentially still work left to be done. But the naming is hiding that.

A good suggestion fell out in 1ffca7b, not included on this branch yet.

...rg/junit/platform/engine/support/hierarchical/ConcurrentHierarchicalTestExecutorService.java

mpkorstanje · 2025-10-16T16:31:03Z

...rg/junit/platform/engine/support/hierarchical/ConcurrentHierarchicalTestExecutorService.java

+				if (result == 0) {
+					result = Boolean.compare(this.isContainer(), that.isContainer());
+					if (result == 0) {
+						result = Integer.compare(that.attempts, this.attempts);


I think attempt should be sorted on first.

--- config: layout: elk --- erDiagram A ||--o{ AA: "" A ||--o{ AB: "" A ||--o{ AC: "" AA ||--o{ AAA: "" AA ||--o{ AAB: "" AB ||--o{ ABA: "" AB ||--o{ ABB: ""

Loading

Given this tree. Suppose

thread 1 is working on AA, claiming resource A,

thread 2 is working on AB and ABA, no resources needed.

thread 3 wants to work on AC, and would need resource A.

Item ABB would be available for work stealing, but isn't picked up by thread 3 because every time item AC is considered it readded to the front of the queue with an incremented attempt counter.

Ah. No it won't because the work stealer will use executeTask which blocks. So it hangs on ABB. This doesn't seem efficient.

I need to think about this a bit. It feels like there should be a more efficient way.

We could work steal across subtrees but then would have to make sure the resource locks are compatible. My intention was to avoid that complication at the expense of needing an extra thread in these cases. I think that tradeoff is acceptable as long as we guarantee that no more than parallelism tests are running concurrently which is now accomplished via worker leases. Happy to discuss ideas, though, of course!

I wasn't thinking about stealing work across subtrees. That's probably not necessary (yet?). Rather, I'm looking to keep the maximum number of created threads low and reuse existing once as much as possible - or when the maximum pool size is limited, optimize utilization.

This is a useful property, for example when using a web driver, it is reused between tests by using a thread locals. So fewer threads means fewer drivers are created.

mpkorstanje · 2025-10-17T15:56:32Z

...rg/junit/platform/engine/support/hierarchical/ConcurrentHierarchicalTestExecutorService.java

+			if (!queueEntries.isEmpty()) {
+				if (sameThreadTasks.isEmpty()) {
+					// hold back one task for this thread
+					sameThreadTasks.add(queueEntries.poll().task);


I'm observing that invoking poll() does not preserve queueEntries insertion order. This means that we put children, a, b and c in the queue they'll come out a, c, b.

To ensure our tests are stable and understandable is probably not what we want to happen. And while there is no guaranteed ordering during test execution, when users do order their tests, I would expect that we pick them up in that order if possible.

Perhaps each entry needs an index in addition to its level?

I'll make a test for this too. There are at least a few (2+) entries needed to make the flip happen.

mpkorstanje · 2025-10-17T16:31:30Z

...rg/junit/platform/engine/support/hierarchical/ConcurrentHierarchicalTestExecutorService.java

+			var claimed = workQueue.remove(entry);
+			if (claimed) {
+				LOGGER.trace(() -> "stole work: " + entry);
+				var executed = tryExecute(entry);


I have an intuition that the current thread should await any locks while the queue processing threads should skip over locked tasks. I.e. the reverse of the current setup.

If implemented correctly, the invariant of the current thread is it will finish all nodes of the sub-tree it working on unless they're stolen away. Because the current thread can't steal any work, it is not wasted if it has to await a lock and there is nothing else to do. The queue processing nodes however are wasted, they could be stealing something else.

We can then refine that further by making the current thread also skip over locked tasks and come back to them later.

Would you like to give that a try on another branch?

Yes, though I don't know how much work you've already got in progress.

I've now pushed everything.

Sorry, pushed a few more commits. I hope they are not throwing a wrench in your plans.

No problem.

This delays blocking until there is nothing else to do #5078

...rg/junit/platform/engine/support/hierarchical/ConcurrentHierarchicalTestExecutorService.java

marcphilipp added 30 commits October 13, 2025 11:57

Add support for executing a single task

892492a

Verify that invokeAll() is only called internally

5c9f0f6

Add support for executing children concurrently

a71f232

Add support for executing children in same thread

1c703f0

Always execute single child in same thread as its parent

c0a9d42

Polishing

df84bf6

Use fixed thread pool

bd426ec

Implement basic work stealing

0947a89

Polishing

58ac3d1

Configure timeout for all tests

a11039c

Introduce ResourceLock.tryAcquire

58198ae

Acquire resource locks for tasks

bb835e1

Polish tests

67db7ba

Fix race condition

4a231aa

Change thread pool configuration to achieve more parallelism

2dc1f3f

Polishing

b869706

Introduce worker leases to limit parallelism

f70433b

Add constructor needed by Jupiter

ef2c49d

Add support for blocking inside of worker thread

ff783e9

Add support for submitting SAME_THREAD child tasks dynamically

752d565

Run isolated tasks last to maximize parallelism

0223336

Polish logging

332abe1

Stop workers sooner (without waiting for queue entries or worker lease)

2df6b28

Use new implementation

c3032be

Improve logging pattern

5f3fd13

* More precise clock * Fixed width thread names * Abbreviated package names

Delete debug printing code

d33beb5

Prioritize children of started containers

2264989

Improve naming

8c9db08

Improve logging

539ef1a

Poll queue only if worker lease was available

86dfea4

To avoid race conditions with other workers that lead to stalling.

marcphilipp added 5 commits October 14, 2025 12:41

Polishing

cf923fc

Simplify work-stealing Future implementation

afbb9f6

Restore max pool size limit in test

a8cec38

Avoid starting an excessive number of threads

6103e4d

Polishing

855880f

mpkorstanje reviewed Oct 14, 2025

View reviewed changes

mpkorstanje reviewed Oct 15, 2025

View reviewed changes

...rg/junit/platform/engine/support/hierarchical/ConcurrentHierarchicalTestExecutorService.java Show resolved Hide resolved

marcphilipp added 2 commits October 16, 2025 12:23

Add test for WorkerLeaseManager and WorkerLease

d47f9fc

Avoid race during worker startup

ba98383

mpkorstanje reviewed Oct 16, 2025

View reviewed changes

...rg/junit/platform/engine/support/hierarchical/ConcurrentHierarchicalTestExecutorService.java Outdated Show resolved Hide resolved

mpkorstanje reviewed Oct 16, 2025

View reviewed changes

Add test for race condition when starting workers

d111712

mpkorstanje reviewed Oct 17, 2025

View reviewed changes

marcphilipp added 12 commits October 17, 2025 18:49

Avoid recursively calling maybeStartWorker

55d8b71

Repeat test to increase likelihood of triggering its flakiness

c8ddb46

Temporarily disable stacktrace pruning

4de2d9a

Temporarily enable logging to have more info when tests fail

f081fde

Execute unclaimed children in blocking mode prior to joining forked work

ed9b22d

Ignore rejected worker starts if there's at least one active worker

163b72c

Reinstate max-pool-size limit

be1ab01

Use unique ID as key

7c74974

Simplify forking and work stealing

03a1dd8

Yield worker lease when blocking thread can continue

82a2d7f

Add TODO

8cf7f47

fixup! Yield worker lease when blocking thread can continue

2cc5311

mpkorstanje reviewed Oct 19, 2025

View reviewed changes

...rg/junit/platform/engine/support/hierarchical/ConcurrentHierarchicalTestExecutorService.java Outdated Show resolved Hide resolved

Use EnumMap for queueEntriesByResult

08986fb

mpkorstanje reviewed Oct 19, 2025

View reviewed changes

...rg/junit/platform/engine/support/hierarchical/ConcurrentHierarchicalTestExecutorService.java Show resolved Hide resolved

Vampire mentioned this pull request Oct 23, 2025

Issue with Spock Parallel execution after upgrade to Selenium 4 spockframework/spock#1822

Closed

Uh oh!

Implement parallel executor service without ForkJoinPool #5060

Are you sure you want to change the base?

Implement parallel executor service without ForkJoinPool #5060

Conversation

marcphilipp commented Oct 13, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Definition of Done

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

mpkorstanje Oct 19, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

mpkorstanje Oct 16, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

mpkorstanje Oct 17, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Implement parallel executor service without `ForkJoinPool` #5060

Implement parallel executor service without `ForkJoinPool` #5060

marcphilipp commented Oct 13, 2025 •

edited

Loading

mpkorstanje Oct 19, 2025 •

edited

Loading

mpkorstanje Oct 16, 2025 •

edited

Loading

mpkorstanje Oct 17, 2025 •

edited

Loading