[tensorpipe] create a new port

[luncliff]

What does your PR fix?

There was no port request for this project.

This is one of the 3rd party libraries for the PyTorch project. The PR will be used for future support of the libtorch port.

Which triplets are supported/not supported? Have you updated the CI baseline?

According to the project's CI configuration, its target is linux, osx. And there are no sources for windows, uwp triplets.

• x64-osx
• x64-linux

Does your PR follow the maintainer guide?

• Make separate Pull Requests per port. Requires [libnop] create a new port #16471
• "version-date" instead of "version-string", formatted as YYYY-MM-DD.

[luncliff]

The port will be rebased when #16471 is merged. To prevent merge, it will remain in the draft mode.

[NancyLi1013]

Hi @luncliff

Thanks for your PR.

Have you tested these features?

[luncliff]

@NancyLi1013 not yet. Doing so requires a Linux device with CUDA but I don't have one in mine. Asking my friends now :(

[luncliff]

I'd like to try vcpkg_cmake_configure and vcpkg_cmake_install for this port. Any suggestions?

[NancyLi1013]

I'd like to try vcpkg_cmake_configure and vcpkg_cmake_install for this port. Any suggestions?

Currently, there is no need to do any actions for the function changes. Just keep them as before. Maybe we will update them to new functions in the near future, but not now.

[NancyLi1013]

Hi @luncliff

Could you please help confirm the test status of features?

[luncliff]

Sadly, the tensorpipe_test keeps failing. (For those Linux features...)

• pybind11: needs an additional patch for its build. I will update the fix-cmakelists.patch within days.
• cma, ibv, cuda: build passed, but tensorpipe_test failes with infinite loop
• shm, benchmark: seems like these are OK

Does every test executable in the port must report green? 🤔 If so, I will change this to draft again then check the mainstream's CI.

[NancyLi1013]

Generally, we should keep all features work normally for every port. If one or more of these features cannot work. it would be better to fix them. If they can only support the specific platform, we should post some messages to describe it. Otherwise, you can choose to remove them from feature lists until they can work fine.

[luncliff]

I see. I'd better remove cma and ibv which are not passing the test. I want to support cuda, but it may not that easy to debug with current state.. :(
I will try more after fixing pybind11 related works.

[luncliff]

After 97bb90b, tensorpipe's CUDA test passes.

user@host:~/vcpkg$ ./vcpkg install tensorpipe[cuda,test]
Computing installation plan...
The following packages will be built and installed:
    tensorpipe[core,cuda,test]:x64-linux -> 2021-04-26
Detecting compiler hash for triplet x64-linux...
Could not locate cached archive: /home/luncliff/.cache/vcpkg/archives/5f/5fbca65575504f45776aebee0cf1f263fa52ac5d.zip
Starting package 1/1: tensorpipe:x64-linux
Building package tensorpipe[core,cuda,test]:x64-linux...
-- Using cached /home/luncliff/vcpkg/downloads/pytorch-tensorpipe-c5a21994bc766659f7f85edb75478e13f429f46c.tar.gz
-- Cleaning sources at /home/luncliff/vcpkg/buildtrees/tensorpipe/src/13f429f46c-d111f7299b.clean. Use --editable to skip cleaning for the packages you specify.
-- Extracting source /home/luncliff/vcpkg/downloads/pytorch-tensorpipe-c5a21994bc766659f7f85edb75478e13f429f46c.tar.gz
-- Applying patch fix-cmakelists.patch
-- Applying patch support-test.patch
-- Applying patch support-pybind11.patch
-- Using source at /home/luncliff/vcpkg/buildtrees/tensorpipe/src/13f429f46c-d111f7299b.clean
...
Installing package tensorpipe[core,cuda,test]:x64-linux... done
Elapsed time for package tensorpipe:x64-linux: 1.606 min

Total elapsed time: 1.612 min

The package tensorpipe:x64-linux provides CMake targets:

    find_package(tensorpipe CONFIG REQUIRED)
    target_link_libraries(main PRIVATE tensorpipe)

The test programs are located in each of the build directories.

user@host:~/vcpkg$ pushd ./buildtrees/tensorpipe/x64-linux-dbg/tensorpipe/test/
~/vcpkg/buildtrees/tensorpipe/x64-linux-dbg/tensorpipe/test ~/vcpkg
user@host:~/vcpkg/buildtrees/tensorpipe/x64-linux-dbg/tensorpipe/test$ ./tensorpipe_test
...
[  SKIPPED ] CudaIpc/CudaMultiGPUChannelTestSuite.SendReverseAcrossDevices/0
[  SKIPPED ] CudaIpc/CudaMultiGPUChannelTestSuite.SendAcrossNonDefaultDevices/0

  YOU HAVE 4 DISABLED TESTS

[luncliff]

For x64-osx failure, this will be rebased after #17610

[luncliff]

Test execution in Darwin also passed

user@host$ pushd /Users/user/dev/vcpkg/buildtrees/tensorpipe/x64-osx-dbg/tensorpipe/test
user@host$ ./tensorpipe_test 
...
[----------] 1 test from Mpt/MptChannelTestSuite
[ RUN      ] Mpt/MptChannelTestSuite.ContextIsNotJoined/0
[       OK ] Mpt/MptChannelTestSuite.ContextIsNotJoined/0 (1 ms)
[----------] 1 test from Mpt/MptChannelTestSuite (1 ms total)

[----------] Global test environment tear-down
[==========] 52 tests from 15 test suites ran. (6289 ms total)
[  PASSED  ] 52 tests.

  YOU HAVE 2 DISABLED TESTS

[NancyLi1013]

LGTM now, thanks for adding this port @luncliff.