[libtorch] create a new port #17199
Merged
[libtorch] create a new port #17199
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
This PR will remain as a shortcut to related PRs until some necessary ports become ready.
|
JackBoosY
added
the
category:new-port
JackBoosY
reviewed
Apr 12, 2021
Sure. I'll leave the part until some dependencies are ready. 👍 |
This was referenced May 23, 2021
|
@luncliff Any news? |
|
First, how I can help? |
Great. I will update note about related works within days. ONNX users can check #18073 |
|
Please ping me if you need any help. |
JackBoosY
added
the
depends:different-pr
![github-actions[bot]](https://avatars.githubusercontent.com/in/15368?s=60&v=4){kind=small}
There was a problem hiding this comment.
This is a new experimental fast check for PR issues. Please let us know if this bot is helpful!
After committing all other changes, the version database must be updated
git add -u && git commit
git checkout 1909fbc4eb8ef30017a79dce50c0f232d149a051 -- versions
./vcpkg x-add-version --allDiff
diff --git a/versions/baseline.json b/versions/baseline.json
index 5bf9f84..59a9d13 100644
--- a/versions/baseline.json
+++ b/versions/baseline.json
@@ -4646,7 +4646,7 @@
},
"onnx": {
"baseline": "1.9.0",
- "port-version": 0
+ "port-version": 1
},
"onnx-optimizer": {
"baseline": "0.2.6",
diff --git a/versions/o-/onnx.json b/versions/o-/onnx.json
index 5f4fc6e..44c1b4f 100644
--- a/versions/o-/onnx.json
+++ b/versions/o-/onnx.json
@@ -1,5 +1,10 @@
{
"versions": [
+ {
+ "git-tree": "ea25f2179598a99237671a57e2b57d0554c7ecb0",
+ "version-semver": "1.9.0",
+ "port-version": 1
+ },
{
"git-tree": "b53c9c9e969928def925c57ea5ddcdfb09293693",
"version-semver": "1.9.0",
* https://github.com/houseroad/foxi * install the project's copyright (MIT) * pytorch requires `foxi_loader` The CMake target will be renamed to `onnxifi_*` for convenience.
This was referenced Sep 11, 2021
Checked the protject's CI logs. It turned out onnx/onnx_proto are ALWAYS static. Specify it in CMakeLists.txt because vcpkg configures `BUILD_SHARED_LIBS=ON` If the triplet requires it. There are no `ONNXIFI_ENABLE_EXT=ON` case. Removed the misused build options in portfile. Add port feature `protobuf-lite` which is in build option.
* remove SHARED for `onnxifi_wrapper` and `onnxifi_dummy`
* also remove redundant part in patch files
* use `link_libraries` to vcpkg installed folder * future work may use library names without `find_library` * update versions JSON to use `version-semver`
* Caffe2 is exported when BUILD_SHARED_LIBS
Cheney-W
added
info:reviewed
vicroms
added
the
depends:vm-update
|
@luncliff thanks for all the effort you've put on this PR. Next VM update will be around (2023/01/10) and we'll be able to test this in CI and hopefully merge it. |
|
Waiting for #28895 merged. |
BillyONeal
removed
the
depends:vm-update
|
/azp run |
|
Azure Pipelines successfully started running 1 pipeline(s). |
vicroms
requested changes
Jan 17, 2023
vicroms
dismissed stale reviews from Cheney-W and github-actions[bot]
via
027b927
BillyONeal
requested changes
Jan 17, 2023
There was a problem hiding this comment.
The caffe2 conflict is still present. I think just making caffe2 an empty port that depends on libtorch should work now that you made it work on Windows?
Thanks! ❤️
Installing 26/26 libtorch:x64-linux...
Building libtorch[core,xnnpack]:x64-linux...
-- Note: libtorch only supports dynamic library linkage. Building dynamic library.
-- Downloading https://github.com/pytorch/pytorch/archive/v1.12.1.tar.gz -> pytorch-pytorch-v1.12.1.tar.gz...
-- Extracting source /home/bion/vcpkg/downloads/pytorch-pytorch-v1.12.1.tar.gz
-- Applying patch pytorch-pr-85958.patch
-- Applying patch fix-cmake.patch
-- Applying patch fix-fbgemm-include.patch
-- Applying patch fix-c10-glog.patch
-- Applying patch use-flatbuffers2.patch
-- Applying patch fix-windows.patch
-- Using source at /home/bion/vcpkg/buildtrees/libtorch/src/v1.12.1-d15308d103.clean
-- Using flatc: /home/bion/vcpkg/installed/x64-linux/tools/flatbuffers/flatc
-- Using protoc: /home/bion/vcpkg/installed/x64-linux/tools/protobuf/protoc
-- Setting up python virtual environmnent...
-- Installing python packages: typing-extensions;pyyaml;numpy
-- Setting up python virtual environmnent...finished.
-- Using Python3: /home/bion/vcpkg/buildtrees/libtorch/x64-linux-venv/bin/python
-- Configuring x64-linux-dbg
-- Configuring x64-linux-rel
-- Building x64-linux-dbg
-- Building x64-linux-rel
-- Building x64-linux-dbg
-- Building x64-linux-rel
-- Building x64-linux-dbg
-- Building x64-linux-rel
-- Installing: /home/bion/vcpkg/packages/libtorch_x64-linux/share/libtorch/copyright
-- Performing post-build validation
Stored binary cache: "/home/bion/.cache/vcpkg/archives/21/211408e045c75aac65a724e61051d81797b74bbf68df490cb6e0e1da3e1b2725.zip"
error: The following files are already installed in /home/bion/vcpkg/installed/x64-linux and are in conflict with libtorch:x64-linux
Installed by caffe2:x64-linux
include/caffe2/operators/shape_op.h
include/caffe2/operators/reducer_functors.h
include/caffe2/operators/reduction_ops.h
include/caffe2/operators/relu_op.h
include/caffe2/operators/remove_data_blocks_op.h
include/caffe2/operators/replace_nan_op.h
include/caffe2/operators/reshape_op.h
include/caffe2/operators/resize_op.h
include/caffe2/operators/reverse_packed_segs_op.h
include/caffe2/operators/roi_pool_op.h
include/caffe2/operators/rowmul_op.h
include/caffe2/operators/scale_op.h
include/caffe2/operators/segment_reduction_op.h
include/caffe2/operators/sequence_ops.h
include/caffe2/operators/rank_loss_op.h
include/caffe2/operators/sinusoid_position_encoding_op.h
include/caffe2/operators/softmax_op.h
include/caffe2/operators/softmax_with_loss_op.h
include/caffe2/operators/softplus_op.h
include/caffe2/operators/space_batch_op.h
include/caffe2/operators/sparse_to_dense_mask_op.h
include/caffe2/operators/sparse_to_dense_op.h
include/caffe2/operators/spatial_batch_norm_op.h
include/caffe2/operators/spatial_softmax_with_loss_op.h
include/caffe2/operators/square_root_divide_op.h
include/caffe2/operators/stop_gradient.h
include/caffe2/operators/string_ops.h
include/caffe2/operators/one_hot_ops.h
include/caffe2/operators/lengths_top_k_op.h
include/caffe2/operators/load_save_op.h
include/caffe2/operators/local_response_normalization_op.h
include/caffe2/operators/loss_op.h
include/caffe2/operators/lpnorm_op.h
include/caffe2/operators/lstm_unit_op.h
include/caffe2/operators/map_ops.h
include/caffe2/operators/margin_ranking_criterion_op.h
include/caffe2/operators/matmul_op.h
include/caffe2/operators/merge_id_lists_op.h
include/caffe2/operators/multi_class_accuracy_op.h
include/caffe2/operators/no_default_engine_op.h
include/caffe2/operators/normalize_op.h
include/caffe2/operators/summarize_op.h
include/caffe2/operators/operator_fallback_gpu.h
include/caffe2/operators/order_switch_ops.h
include/caffe2/operators/pack_rnn_sequence_op.h
include/caffe2/operators/pack_segments.h
include/caffe2/operators/pad_op.h
include/caffe2/operators/partition_ops.h
include/caffe2/operators/perplexity_op.h
include/caffe2/operators/piecewise_linear_transform_op.h
include/caffe2/operators/pool_op.h
include/caffe2/operators/prefetch_op.h
include/caffe2/operators/prelu_op.h
include/caffe2/operators/prepend_dim_op.h
include/caffe2/utils/proto_utils.h
include/caffe2/transforms/common_subexpression_elimination.h
include/caffe2/transforms/conv_to_nnpack_transform.h
include/caffe2/transforms/pattern_net_transform.h
include/caffe2/transforms/single_op_transform.h
include/caffe2/utils/cast.h
include/caffe2/utils/cblas.h
include/caffe2/utils/conversions.h
include/caffe2/utils/cpu_neon.h
include/caffe2/utils/cpuid.h
include/caffe2/utils/fixed_divisor.h
include/caffe2/utils/math-detail.h
include/caffe2/utils/math.h
include/caffe2/utils/murmur_hash3.h
include/caffe2/sgd/rmsprop_op.h
include/caffe2/utils/signal_handler.h
include/caffe2/utils/simple_queue.h
include/caffe2/utils/smart_tensor_printer.h
include/caffe2/utils/string_utils.h
include/caffe2/utils/threadpool/ThreadPool.h
include/caffe2/utils/threadpool/ThreadPoolCommon.h
include/caffe2/utils/threadpool/WorkersPool.h
include/caffe2/utils/threadpool/pthreadpool.h
include/caffe2/utils/zmq_helper.h
include/caffe2/video/video_decoder.h
include/caffe2/video/video_input_op.h
include/caffe2/video/video_io.h
include/caffe2/proto/caffe2.pb.h
include/caffe2/operators/tensor_protos_db_input.h
include/caffe2/operators/text_file_reader_utils.h
include/caffe2/operators/tile_op.h
include/caffe2/operators/top_k.h
include/caffe2/operators/transpose_op.h
include/caffe2/operators/tt_linear_op.h
include/caffe2/operators/utility_ops.h
include/caffe2/operators/while_op.h
include/caffe2/operators/zero_gradient_op.h
include/caffe2/perfkernels/common.h
include/caffe2/perfkernels/cvtsh_ss_bugfix.h
include/caffe2/perfkernels/embedding_lookup.h
include/caffe2/perfkernels/typed_axpy.h
include/caffe2/operators/lengths_tile_op.h
include/caffe2/queue/blobs_queue.h
include/caffe2/queue/blobs_queue_db.h
include/caffe2/queue/queue_ops.h
include/caffe2/queue/rebatching_queue.h
include/caffe2/queue/rebatching_queue_ops.h
include/caffe2/sgd/adagrad_op.h
include/caffe2/sgd/adam_op.h
include/caffe2/sgd/ftrl_op.h
include/caffe2/sgd/iter_op.h
include/caffe2/sgd/learning_rate_functors.h
include/caffe2/sgd/learning_rate_op.h
include/caffe2/sgd/momentum_sgd_op.h
include/caffe2/distributed/redis_store_handler.h
include/caffe2/core/scope_guard.h
include/caffe2/core/static_tracepoint.h
include/caffe2/core/static_tracepoint_elfx86.h
include/caffe2/core/stats.h
include/caffe2/core/tensor.h
include/caffe2/core/timer.h
include/caffe2/core/transform.h
include/caffe2/core/types.h
include/caffe2/core/workspace.h
include/caffe2/cuda_rtc/common_rtc.h
include/caffe2/db/create_db_op.h
include/caffe2/distributed/file_store_handler.h
include/caffe2/distributed/file_store_handler_op.h
include/caffe2/core/qtensor_serialization.h
include/caffe2/distributed/redis_store_handler_op.h
include/caffe2/distributed/store_handler.h
include/caffe2/distributed/store_ops.h
include/caffe2/experiments/operators/fully_connected_op_decomposition.h
include/caffe2/experiments/operators/fully_connected_op_prune.h
include/caffe2/experiments/operators/fully_connected_op_sparse.h
include/caffe2/experiments/operators/funhash_op.h
include/caffe2/experiments/operators/sparse_funhash_op.h
include/caffe2/experiments/operators/sparse_matrix_reshape_op.h
include/caffe2/experiments/operators/tt_contraction_op.h
include/caffe2/experiments/operators/tt_pad_op.h
include/caffe2/image/image_input_op.h
include/caffe2/core/graph.h
include/caffe2/core/blob.h
include/caffe2/core/blob_serialization.h
include/caffe2/core/blob_serializer_base.h
include/caffe2/core/blob_stats.h
include/caffe2/core/common.h
include/caffe2/core/common_cudnn.h
include/caffe2/core/common_gpu.h
include/caffe2/core/common_omp.h
include/caffe2/core/context.h
include/caffe2/core/context_gpu.h
include/caffe2/core/db.h
include/caffe2/core/event.h
include/caffe2/core/flags.h
include/caffe2/image/transform_gpu.h
include/caffe2/core/init.h
include/caffe2/core/logging.h
include/caffe2/core/macros.h
include/caffe2/core/memonger.h
include/caffe2/core/net.h
include/caffe2/core/net_simple.h
include/caffe2/core/observer.h
include/caffe2/core/operator.h
include/caffe2/core/operator_gradient.h
include/caffe2/core/operator_schema.h
include/caffe2/core/plan_executor.h
include/caffe2/core/qtensor.h
include/caffe2/operators/free_op.h
include/caffe2/operators/cross_entropy_op.h
include/caffe2/operators/dataset_ops.h
include/caffe2/operators/distance_op.h
include/caffe2/operators/do_op.h
include/caffe2/operators/dropout_op.h
include/caffe2/operators/elementwise_linear_op.h
include/caffe2/operators/elementwise_logical_ops.h
include/caffe2/operators/elementwise_op_test.h
include/caffe2/operators/elu_op.h
include/caffe2/operators/feed_blob_op.h
include/caffe2/operators/filler_op.h
include/caffe2/operators/find_duplicate_elements_op.h
include/caffe2/operators/find_op.h
include/caffe2/operators/counter_ops.h
include/caffe2/operators/fully_connected_op.h
include/caffe2/operators/given_tensor_fill_op.h
include/caffe2/operators/gru_unit_op.h
include/caffe2/operators/h_softmax_op.h
include/caffe2/operators/half_float_ops.h
include/caffe2/operators/if_op.h
include/caffe2/operators/im2col_op.h
include/caffe2/operators/index_hash_ops.h
include/caffe2/operators/instance_norm_op.h
include/caffe2/operators/layer_norm_op.h
include/caffe2/operators/leaky_relu_op.h
include/caffe2/operators/lengths_reducer_ops.h
include/caffe2/operators/clip_op.h
include/caffe2/mpi/mpi_common.h
include/caffe2/mpi/mpi_ops.h
include/caffe2/operators/accumulate_op.h
include/caffe2/operators/accuracy_op.h
include/caffe2/operators/apmeter_op.h
include/caffe2/operators/batch_box_cox_op.h
include/caffe2/operators/batch_gather_ops.h
include/caffe2/operators/batch_matmul_op.h
include/caffe2/operators/boolean_mask_ops.h
include/caffe2/operators/boolean_unmask_ops.h
include/caffe2/operators/cast_op.h
include/caffe2/operators/channel_shuffle_op.h
include/caffe2/core/allocator.h
include/caffe2/operators/concat_split_op.h
include/caffe2/operators/conv_op.h
include/caffe2/operators/conv_op_cache_cudnn.h
include/caffe2/operators/conv_op_impl.h
include/caffe2/operators/conv_op_shared.h
include/caffe2/operators/conv_pool_op_base.h
include/caffe2/operators/conv_transpose_op.h
include/caffe2/operators/conv_transpose_op_impl.h
include/caffe2/operators/conv_transpose_op_mobile.h
include/caffe2/operators/conv_transpose_op_mobile_impl.h
include/caffe2/operators/conv_transpose_unpool_op_base.h
include/caffe2/operators/cosine_embedding_criterion_op.h
error: Please ensure you're using the latest port files with `git pull` and `vcpkg update`.
Then check for known issues at:
https://github.com/microsoft/vcpkg/issues?q=is%3Aissue+is%3Aopen+in%3Atitle+libtorch
You can submit a new issue at:
https://github.com/microsoft/vcpkg/issues/new?template=report-package-build-failure.md&title=[libtorch]+Build+error
Include '[libtorch] Build error' in your bug report title, the following version information in your bug description, and attach any relevant failure logs from above.
vcpkg-tool version: 2022-12-14-7ae0d8527fb488fde10a89c2813802dc9b03b6f9
vcpkg-scripts version: 3fd29f0d3 2023-01-17 (29 minutes ago)
* update baseline
Cheney-W
approved these changes
Jan 18, 2023
Cheney-W
added
info:reviewed
vicroms
approved these changes
Jan 18, 2023
BillyONeal
approved these changes
Jan 18, 2023
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
What does your PR fix? Fixes
Resolves #16319
Resolves #15763
Resolves #5209
Resolves #6345
This is a rework of #8318. It tried to download the existing releases at the moment, but this PR will build from the source.
References
Related Works
Some required PRs are still open and in progress, but I think it's time to start the draft.
Wish I can get more feedback and opinions for the
libtorchport.find_package[fbgemm] fix find_package #20111onnx[onnx] create a new port #18073foxi[onnx] update ONNXIFI build output and host dependencies #20112foxi[foxi] create a new port #21786Which triplets are supported/not supported? Have you updated the CI baseline?
Supports 2 triplets. Out of source build is not available in
x64-windows.x64-osxx64-linuxDoes your PR follow the maintainer guide?
Yes, but I will check it regularly.