Tests 51 - 54 fail, even with -LE not_reproducible -LE git_required

[Andreas-Schniertshauer]

I run the tests with the following flags:

ctest -C Release --output-on-failure -LE not_reproducible -LE git_required

but get

The following tests FAILED:
         51 - cmake_import_configure (Failed)
         52 - cmake_import_build (Not Run)
         53 - cmake_import_minver_configure (Failed)
         54 - cmake_import_minver_build (Not Run)
Errors while running CTest

51 und 53 failed because of:

CMake Error at json/build/nlohmann_jsonConfig.cmake:6 (include):
  include could not find load file:
    json/build/nlohmann_jsonTargets.cmake
Call Stack (most recent call first):
  CMakeLists.txt:5 (find_package)

52 and 54 Not Run because of:

Failed test dependencies: cmake_import_configure

Are there other flags to set?
Thanks for the help,
Andreas.

[nlohmann]

What were the exact steps? Did you download the test data manually?

[Andreas-Schniertshauer]

I used the json-3.9.1.zip, unziped it, opened a Visual Studio 2019 Command Prompt,

cd json-3.9.1
mkdir build
cd build
cmake .. -DJSON_BuildTests=On -D JSON_Install:BOOL=OFF -D JSON_ImplicitConversions:BOOL=OFF
cmake --build . --config Release
ctest -C Release --output-on-failure -LE not_reproducible -LE git_required

Test data was downloaded in the last step automatically, I got:

      Start  1: download_test_data
 1/58 Test  #1: download_test_data ...........................   Passed   34.60 sec

and the folder json_test_data was created.

[Andreas-Schniertshauer]

Are the test data properly downloaded? The folder should contain some 358 MB.

The size displayed in Total Commander is 365.905.663 -> ~349 MB.

[nlohmann]

I'm currently looking into this one. It seems the bug is that I try to give two labels to a test like

  set_tests_properties(cmake_fetch_content_configure PROPERTIES
    FIXTURES_SETUP cmake_fetch_content
    LABELS git_required
    LABELS not_reproducible
  )

though it has to be like this:

  set_tests_properties(cmake_fetch_content_configure PROPERTIES
    FIXTURES_SETUP cmake_fetch_content
    LABELS "git_required;not_reproducible"
  )

(see https://stackoverflow.com/a/24500217).

I'll create a branch for you to check.

[nlohmann]

Branch issue2596 contains a fix. Note you need to call

cmake -LE "not_reproducible|git_required"

to exclude multiple labels. I tried it locally.

[Andreas-Schniertshauer]

ctest -C Release --output-on-failure -LE "not_reproducible|git_required"

did work 100% tests passed, 0 tests failed out of 50

[nlohmann]

Cool! I'll merge the branch.