Run MacOS tests in correct directory. #3718

fruffy · 2022-11-22T16:44:24Z

Turns out MacOS tests have not run in a while. ctest was not executed in the right folder.

fruffy · 2022-11-22T19:35:57Z

Anyone with a MacOS setup able to debug the CI failure? I am not sure what to do here.

rst0git · 2022-11-23T11:24:34Z

Anyone with a MacOS setup able to debug the CI failure? I am not sure what to do here.

1375/1919 Test    #1: driver_inputs_test_1 ............................................................***Failed    0.08 sec
/Users/runner/work/p4c/p4c/tools/driver/test_scripts/driver_inputs_test_1: line 7: realpath: command not found
/Users/runner/work/p4c/p4c/tools/driver/test_scripts/driver_inputs_test_1: line 7: ./driver_inputs_test___shared_code.bash: No such file or directory
/Users/runner/work/p4c/p4c/tools/driver/test_scripts/driver_inputs_test_1: line 9: check_for_inadvisable_sourcing: command not found
/Users/runner/work/p4c/p4c/tools/driver/test_scripts/driver_inputs_test_1: line 10: return: can only `return' from a function or sourced script
/Users/runner/work/p4c/p4c/tools/driver/test_scripts/driver_inputs_test_1: line 14: resolve_symlink_only_of_basename: command not found
/Users/runner/work/p4c/p4c/tools/driver/test_scripts/driver_inputs_test_1: line 18: try_to_find_the_driver: command not found
Unable to find the driver of the P4 compiler.  Aborting the test '''' with a non-zero exit code.  This test failed.

It looks like we need to add coreutils to the list of dependencies in install_mac_deps.sh

fruffy · 2022-11-23T18:57:38Z

Anyone with a MacOS setup able to debug the CI failure? I am not sure what to do here.

1375/1919 Test    #1: driver_inputs_test_1 ............................................................***Failed    0.08 sec
/Users/runner/work/p4c/p4c/tools/driver/test_scripts/driver_inputs_test_1: line 7: realpath: command not found
/Users/runner/work/p4c/p4c/tools/driver/test_scripts/driver_inputs_test_1: line 7: ./driver_inputs_test___shared_code.bash: No such file or directory
/Users/runner/work/p4c/p4c/tools/driver/test_scripts/driver_inputs_test_1: line 9: check_for_inadvisable_sourcing: command not found
/Users/runner/work/p4c/p4c/tools/driver/test_scripts/driver_inputs_test_1: line 10: return: can only `return' from a function or sourced script
/Users/runner/work/p4c/p4c/tools/driver/test_scripts/driver_inputs_test_1: line 14: resolve_symlink_only_of_basename: command not found
/Users/runner/work/p4c/p4c/tools/driver/test_scripts/driver_inputs_test_1: line 18: try_to_find_the_driver: command not found
Unable to find the driver of the P4 compiler.  Aborting the test '''' with a non-zero exit code.  This test failed.

It looks like we need to add coreutils to the list of dependencies in install_mac_deps.sh

Do you have an idea what is causing the remaining JSON CI failure? The grep does not produce an output it seems.

davidbolvansky · 2022-11-25T13:31:36Z

2022-11-23T18:35:54.7364120Z 1597/1921 Test #1968: gtestp4c ........................................................................***Failed   41.50 sec
2022-11-23T18:35:54.7365390Z [==========] Running 132 tests from 23 test cases.
2022-11-23T18:35:54.7366170Z [----------] Global test environment set-up.
2022-11-23T18:35:54.7366800Z [----------] 9 tests from P4CArchitecture
2022-11-23T18:35:54.7367300Z [ RUN      ] P4CArchitecture.packet_out
2022-11-23T18:35:54.7367810Z [       OK ] P4CArchitecture.packet_out (3 ms)
2022-11-23T18:35:54.7368350Z [ RUN      ] P4CArchitecture.duplicatedDeclarationBug
2022-11-23T18:35:54.7369070Z (string)(19): [--Werror=duplicate] error: Deparser: multiple matching declarations
2022-11-23T18:35:54.7369600Z         PSA(Deparser()) main;
2022-11-23T18:35:54.7370010Z             ^^^^^^^^
2022-11-23T18:35:54.7370630Z (string)(7): [--Werror=duplicate] error: Candidate: control Deparser
2022-11-23T18:35:54.7371170Z         control Deparser<H> (packet_out b, in H hdr);
2022-11-23T18:35:54.7371620Z                 ^^^^^^^^
2022-11-23T18:35:54.7372230Z (string)(14): [--Werror=duplicate] error: Candidate: control Deparser
2022-11-23T18:35:54.7372780Z         control Deparser(packet_out b, in ParsedHeaders h){
2022-11-23T18:35:54.7373240Z                 ^^^^^^^^
2022-11-23T18:35:54.7373980Z [       OK ] P4CArchitecture.duplicatedDeclarationBug (0 ms)
2022-11-23T18:35:54.7374570Z [ RUN      ] P4CArchitecture.instantiation
2022-11-23T18:35:54.7375090Z [       OK ] P4CArchitecture.instantiation (3 ms)
2022-11-23T18:35:54.7375420Z [ RUN      ] P4CArchitecture.psa_package_with_body
2022-11-23T18:35:54.7375720Z (string)(5):syntax error, unexpected {, expecting ;
2022-11-23T18:35:54.7376220Z         package PSA<H, M> (Ingress<H, M> ig) {

rst0git · 2022-11-25T14:50:59Z

@davidbolvansky load_ir_from_json is the test that currently fails:

[ RUN      ] FromJSONTest.load_ir_from_json
105,112d104
<       "name" : "X",
<       "entries" : [
<         ["Field_0", 0],
<         ["Field_1", 1],
<         ["Field_2", 2]
<       ]
<     },
<     {
118a111,118
>     },
>     {
>       "name" : "X",
>       "entries" : [
>         ["Field_0", 0],
>         ["Field_1", 1],
>         ["Field_2", 2]
>       ]
/Users/runner/work/p4c/p4c/test/gtest/load_ir_from_json.cpp:51: Failure
Value of: exitCode
  Actual: true
Expected: false
[  FAILED  ] FromJSONTest.load_ir_from_json (1352 ms)

All commands have been installed with the prefix "g".
If you need to use these commands with their normal names, you
can add a "gnubin" directory to your PATH from your bashrc like:
  PATH="/usr/local/opt/grep/libexec/gnubin:$PATH"

It looks like we need to add the following line to make sure that the tests are using the GNU version of grep.

export PATH="/usr/local/opt/grep/libexec/gnubin:$PATH"

rst0git · 2022-11-28T13:31:36Z

When comparing the output of outputTO.json and outputFROM.json for the load_ir_from_json test running in CI, it looks like the order of "X" and "Y" has been swapped. This is causing the diff test to fail:

outputTO.json:

  "enums" : [
    {
      "name" : "X",
      "entries" : [
        ["Field_0", 0],
        ["Field_1", 1],
        ["Field_2", 2]
      ]
    },
    {
      "name" : "Y",
      "entries" : [
        ["Field_00", 0],
        ["Field_01", 1],
        ["Field_02", 2]
      ]
    }
  ],

outputFROM.json:

  "enums" : [
    {
      "name" : "Y",
      "entries" : [
        ["Field_00", 0],
        ["Field_01", 1],
        ["Field_02", 2]
      ]
    },
    {
      "name" : "X",
      "entries" : [
        ["Field_0", 0],
        ["Field_1", 1],
        ["Field_2", 2]
      ]
    }
  ],

fruffy · 2022-11-28T17:23:21Z

Hm could this be some non-determinism in the JSONGenerator?

mihaibudiu · 2022-11-28T17:26:05Z

Ideally it would generate fields in alphabetical order.

fruffy · 2022-11-28T19:40:22Z

It could either be the loader or generator, but I do not understand that code well enough yet to tell.

davidbolvansky · 2022-12-11T00:51:05Z

If this is hard to debug - maybe just disable this test for now to unblock this PR?

fruffy · 2022-12-15T14:35:15Z

Done.

davidbolvansky · 2022-12-15T15:55:09Z

.github/workflows/ci-test.yml

      run: |
-        ctest --output-on-failure -j 2 --schedule-random -LE "bpf$"
+        ctest --output-on-failure -j 2 --schedule-random -LE "bpf|gtest|ubpf"


drop -j2.

CTEST_PARALLEL_LEVEL is defined.

rst0git · 2022-12-15T18:54:39Z

.github/workflows/ci-test.yml

@@ -141,5 +141,7 @@ jobs:
          ./bootstrap.sh -DENABLE_GC=ON -DCMAKE_BUILD_TYPE=RELEASE -DENABLE_UNIFIED_COMPILATION=ON && cd build && make -j2

    - name: Run tests (MacOS)
+      # TODO: Renable gtest. Unit tests currently fail on MacOS.


@fruffy Would you be able to open a GitHub issue describing the problem and add a link in the comment, or we could add a link to this pull request?

fruffy added 4 commits December 15, 2022 12:20

Run MacOS tests in correct directory.

8bc920d

Add coreutils to Mac dependencies.

4e567df

Add grep to Mac dependencies.

641d708

Brew grep.

2673b88

fruffy force-pushed the fruffy/macos_tests branch from 11db6fd to 93d7cec Compare December 15, 2022 11:20

Ignore gtest.

a4381b8

fruffy force-pushed the fruffy/macos_tests branch from 93d7cec to a4381b8 Compare December 15, 2022 13:27

davidbolvansky reviewed Dec 15, 2022

View reviewed changes

davidbolvansky approved these changes Dec 15, 2022

View reviewed changes

Drop.

6725e14

rst0git approved these changes Dec 15, 2022

View reviewed changes

rst0git reviewed Dec 15, 2022

View reviewed changes

fruffy merged commit b8e80bf into main Dec 16, 2022

fruffy deleted the fruffy/macos_tests branch December 19, 2022 15:16

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Run MacOS tests in correct directory. #3718

Run MacOS tests in correct directory. #3718

fruffy commented Nov 22, 2022 •

edited

Loading

fruffy commented Nov 22, 2022

rst0git commented Nov 23, 2022

fruffy commented Nov 23, 2022

davidbolvansky commented Nov 25, 2022

rst0git commented Nov 25, 2022 •

edited

Loading

rst0git commented Nov 28, 2022 •

edited

Loading

fruffy commented Nov 28, 2022

mihaibudiu commented Nov 28, 2022

fruffy commented Nov 28, 2022

davidbolvansky commented Dec 11, 2022

fruffy commented Dec 15, 2022

davidbolvansky Dec 15, 2022

rst0git Dec 15, 2022

Run MacOS tests in correct directory. #3718

Run MacOS tests in correct directory. #3718

Conversation

fruffy commented Nov 22, 2022 • edited Loading

fruffy commented Nov 22, 2022

rst0git commented Nov 23, 2022

fruffy commented Nov 23, 2022

davidbolvansky commented Nov 25, 2022

rst0git commented Nov 25, 2022 • edited Loading

rst0git commented Nov 28, 2022 • edited Loading

fruffy commented Nov 28, 2022

mihaibudiu commented Nov 28, 2022

fruffy commented Nov 28, 2022

davidbolvansky commented Dec 11, 2022

fruffy commented Dec 15, 2022

davidbolvansky Dec 15, 2022

Choose a reason for hiding this comment

rst0git Dec 15, 2022

Choose a reason for hiding this comment

fruffy commented Nov 22, 2022 •

edited

Loading

rst0git commented Nov 25, 2022 •

edited

Loading

rst0git commented Nov 28, 2022 •

edited

Loading