Add support for multi-threaded test execution

[cschreib]

The framework is written to be thread-safe, but there is currently no way to run tests in parallel. This should be easy to add, but perhaps not without using heap memory. In which case, this will have to be guarded by macros, so it can be disabled if necessary. The heap allocation should be limited to the creation of the threads; everything else we should be able to implement with atomics.

[AdelKS]

Another possibility is to implement what gtest calls "sharding" which is a multi processing approach. You tell the test executable when running it, its index i and N, which is how many you are going to run in parallel, and it will run a specific subset of the tests.

I am writing it here because it didn't see that it's supported in the readme.

[cschreib]

Hi and thanks for the tip! You are correct, this is not currently supported. I know Catch2 supports this too. Do you know how the different processes get spawned? I.e., is this meant to be supported by an external scheduler / test runner? And do you know how it meshes with test filtering?

[AdelKS]

Do you know how the different processes get spawned? I.e., is this meant to be supported by an external scheduler / test runner?

Yes, the user has to run the test executable N times, and tells each executable their index i and how many subdivisions, N he wants on the tests.

And do you know how it meshes with test filtering?

I do not unfortunately. Although I think it's best to filter first and then subdivide the subset that remains, so it probably does that.