k2 source code build error in Nvidia ngc docker

[yjzhong89]

When I build k2 source code in Nvidia ngc docker 24.08, I get the error below

#54 38.93 #endif // K2_CSRC_VERSION_H_
#54 38.99 [ 1%] Building CUDA object k2/csrc/CMakeFiles/k2_log.dir/log.cu.o
#54 39.00 [ 2%] Building CXX object _deps/kaldifeat-build/kaldifeat/csrc/CMakeFiles/kaldifeat_core.dir/feature-fbank.cc.o
#54 39.00 [ 3%] Building CXX object _deps/kaldifeat-build/kaldifeat/csrc/CMakeFiles/kaldifeat_core.dir/feature-mfcc.cc.o
#54 39.00 [ 4%] Building CXX object _deps/kaldifeat-build/kaldifeat/csrc/CMakeFiles/kaldifeat_core.dir/feature-functions.cc.o
#54 52.80 [ 5%] Linking CUDA shared library ../../lib/libk2_log.so
#54 52.92 [ 5%] Built target k2_log
#54 52.95 [ 6%] Building CXX object _deps/kaldifeat-build/kaldifeat/csrc/CMakeFiles/kaldifeat_core.dir/feature-plp.cc.o
#54 53.41 [ 6%] Building CXX object _deps/kaldifeat-build/kaldifeat/csrc/CMakeFiles/kaldifeat_core.dir/feature-spectrogram.cc.o
#54 62.28 [ 7%] Building CXX object _deps/kaldifeat-build/kaldifeat/csrc/CMakeFiles/kaldifeat_core.dir/feature-window.cc.o
#54 63.32 [ 8%] Building CXX object _deps/kaldifeat-build/kaldifeat/csrc/CMakeFiles/kaldifeat_core.dir/matrix-functions.cc.o
#54 76.11 [ 9%] Building CXX object _deps/kaldifeat-build/kaldifeat/csrc/CMakeFiles/kaldifeat_core.dir/mel-computations.cc.o
#54 76.56 [ 10%] Building CXX object _deps/kaldifeat-build/kaldifeat/csrc/CMakeFiles/kaldifeat_core.dir/online-feature.cc.o
#54 77.90 [ 11%] Building CXX object k2/csrc/host/CMakeFiles/fsa.dir/arcsort.cc.o
#54 79.56 [ 12%] Building CXX object k2/csrc/host/CMakeFiles/fsa.dir/aux_labels.cc.o
#54 81.63 [ 12%] Building CXX object k2/csrc/host/CMakeFiles/fsa.dir/connect.cc.o
#54 83.38 [ 13%] Building CXX object k2/csrc/host/CMakeFiles/fsa.dir/determinize.cc.o
#54 85.54 [ 13%] Building CXX object _deps/kaldifeat-build/kaldifeat/csrc/CMakeFiles/kaldifeat_core.dir/whisper-fbank.cc.o
#54 87.88 [ 14%] Building CXX object k2/csrc/host/CMakeFiles/fsa.dir/determinize_pruned.cc.o
#54 92.10 [ 15%] Building CXX object k2/csrc/host/CMakeFiles/fsa.dir/fsa.cc.o
#54 93.01 [ 16%] Building CXX object k2/csrc/host/CMakeFiles/fsa.dir/fsa_equivalent.cc.o
#54 95.62 [ 17%] Building CXX object k2/csrc/host/CMakeFiles/fsa.dir/fsa_renderer.cc.o
#54 96.69 [ 17%] Building CXX object k2/csrc/host/CMakeFiles/fsa.dir/fsa_util.cc.o
#54 99.17 [ 18%] Building CXX object k2/csrc/host/CMakeFiles/fsa.dir/intersect.cc.o
#54 100.3 [ 19%] Building CXX object k2/csrc/host/CMakeFiles/fsa.dir/properties.cc.o
#54 100.6 [ 20%] Building CXX object k2/csrc/host/CMakeFiles/fsa.dir/rmepsilon_pruned.cc.o
#54 100.9 [ 21%] Building CXX object k2/csrc/host/CMakeFiles/fsa.dir/topsort.cc.o
#54 101.7 [ 21%] Building CXX object k2/csrc/host/CMakeFiles/fsa.dir/util.cc.o
#54 102.2 [ 22%] Building CXX object k2/csrc/host/CMakeFiles/fsa.dir/weights.cc.o
#54 104.1 [ 23%] Linking CXX shared library ../../../lib/libk2fsa.so
#54 104.2 [ 23%] Built target fsa
#54 104.3 [ 23%] Building CUDA object k2/csrc/CMakeFiles/context.dir/array_of_ragged.cu.o
#54 104.3 [ 24%] Building CUDA object k2/csrc/CMakeFiles/context.dir/array_ops.cu.o
#54 104.3 [ 25%] Building CUDA object k2/csrc/CMakeFiles/context.dir/algorithms.cu.o
#54 104.9 In file included from /usr/include/nvtx3/nvtx3.hpp:629,
#54 104.9 from /usr/local/cuda/include/cub/detail/nvtx.cuh:47,
#54 104.9 from /usr/local/cuda/include/cub/device/device_adjacent_difference.cuh:41,
#54 104.9 from /usr/local/cuda/include/cub/cub.cuh:62,
#54 104.9 from /tmp/k2/k2/csrc/cub.h:24,
#54 104.9 from /tmp/k2/k2/csrc/array_inl.h:33,
#54 104.9 from /tmp/k2/k2/csrc/array.h:969,
#54 104.9 from /tmp/k2/k2/csrc/algorithms.h:25,
#54 104.9 from /tmp/k2/k2/csrc/algorithms.cu:22:
#54 104.9 /usr/include/nvtx3/nvToolsExt.h:150:2: error: #error "Trying to #include NVTX version 3 in a source file where an older NVTX version has already been included. If you are not directly using NVTX (the NVIDIA Tools Extension library), you are getting this error because libraries you are using have included different versions of NVTX. Suggested solutions are: (1) reorder #includes so the newest NVTX version is included first, (2) avoid using the conflicting libraries in the same .c/.cpp file, or (3) update the library using the older NVTX version to use the newer version instead."
#54 104.9 150 | #error "Trying to #include NVTX version 3 in a source file where an older NVTX version has already been included. If you are not directly using NVTX (the NVIDIA Tools Extension library), you are getting this error because libraries you are using have included different versions of NVTX. Suggested solutions are: (1) reorder #includes so the newest NVTX version is included first, (2) avoid using the conflicting libraries in the same .c/.cpp file, or (3) update the library using the older NVTX version to use the newer version instead."

so what should I do?

[csukuangfj]

please post the complete logs.

[yjzhong89]

This is the complete log

RUN cd /tmp/k2 ; python3 setup.py install
2.945 running install
2.946 /usr/local/lib/python3.10/dist-packages/setuptools/_distutils/cmd.py:66: SetuptoolsDeprecationWarning: setup.py install is deprecated.
2.946 !!
2.946
2.946 ********************************************************************************
2.946 Please avoid running setup.py directly.
2.946 Instead, use pypa/build, pypa/installer or other
2.946 standards-based tools.
2.946
2.946 See https://blog.ganssle.io/articles/2021/10/setup-py-deprecated.html for details.
2.946 ********************************************************************************
2.946
2.946 !!
2.946 self.initialize_options()
3.196 /usr/local/lib/python3.10/dist-packages/setuptools/_distutils/cmd.py:66: EasyInstallDeprecationWarning: easy_install command is deprecated.
3.196 !!
3.196
3.196 ********************************************************************************
3.196 Please avoid running setup.py and easy_install.
3.196 Instead, use pypa/build, pypa/installer or other
3.196 standards-based tools.
3.196
3.196 See pypa/setuptools#917 for details.
3.196 ********************************************************************************
3.196
3.196 !!
3.196 self.initialize_options()
3.453 running bdist_egg
3.540 running egg_info
3.540 creating k2.egg-info
3.567 writing k2.egg-info/PKG-INFO
3.568 writing dependency_links to k2.egg-info/dependency_links.txt
3.569 writing requirements to k2.egg-info/requires.txt
3.570 writing top-level names to k2.egg-info/top_level.txt
3.570 writing manifest file 'k2.egg-info/SOURCES.txt'
3.626 reading manifest file 'k2.egg-info/SOURCES.txt'
3.626 reading manifest template 'MANIFEST.in'
3.626 adding license file 'LICENSE'
3.627 writing manifest file 'k2.egg-info/SOURCES.txt'
3.628 installing library code to build/bdist.linux-x86_64/egg
3.628 running install_lib
3.628 running build_py
3.628 creating build
3.629 creating build/lib.linux-x86_64-cpython-310
3.629 creating build/lib.linux-x86_64-cpython-310/k2
3.629 copying k2/python/k2/init.py -> build/lib.linux-x86_64-cpython-310/k2
3.629 copying k2/python/k2/autograd.py -> build/lib.linux-x86_64-cpython-310/k2
3.629 copying k2/python/k2/autograd_utils.py -> build/lib.linux-x86_64-cpython-310/k2
3.630 copying k2/python/k2/ctc_loss.py -> build/lib.linux-x86_64-cpython-310/k2
3.630 copying k2/python/k2/decode.py -> build/lib.linux-x86_64-cpython-310/k2
3.630 copying k2/python/k2/dense_fsa_vec.py -> build/lib.linux-x86_64-cpython-310/k2
3.630 copying k2/python/k2/fsa.py -> build/lib.linux-x86_64-cpython-310/k2
3.630 copying k2/python/k2/fsa_algo.py -> build/lib.linux-x86_64-cpython-310/k2
3.631 copying k2/python/k2/fsa_properties.py -> build/lib.linux-x86_64-cpython-310/k2
3.631 copying k2/python/k2/mutual_information.py -> build/lib.linux-x86_64-cpython-310/k2
3.631 copying k2/python/k2/mwer_loss.py -> build/lib.linux-x86_64-cpython-310/k2
3.631 copying k2/python/k2/nbest.py -> build/lib.linux-x86_64-cpython-310/k2
3.631 copying k2/python/k2/online_dense_intersecter.py -> build/lib.linux-x86_64-cpython-310/k2
3.632 copying k2/python/k2/ops.py -> build/lib.linux-x86_64-cpython-310/k2
3.632 copying k2/python/k2/rnnt_decode.py -> build/lib.linux-x86_64-cpython-310/k2
3.632 copying k2/python/k2/rnnt_loss.py -> build/lib.linux-x86_64-cpython-310/k2
3.632 copying k2/python/k2/symbol_table.py -> build/lib.linux-x86_64-cpython-310/k2
3.632 copying k2/python/k2/utils.py -> build/lib.linux-x86_64-cpython-310/k2
3.633 creating build/lib.linux-x86_64-cpython-310/k2/ragged
3.633 copying k2/python/k2/ragged/init.py -> build/lib.linux-x86_64-cpython-310/k2/ragged
3.633 creating build/lib.linux-x86_64-cpython-310/k2/sparse
3.633 copying k2/python/k2/sparse/init.py -> build/lib.linux-x86_64-cpython-310/k2/sparse
3.634 copying k2/python/k2/sparse/autograd.py -> build/lib.linux-x86_64-cpython-310/k2/sparse
3.634 creating build/lib.linux-x86_64-cpython-310/k2/version
3.634 copying k2/python/k2/version/init.py -> build/lib.linux-x86_64-cpython-310/k2/version
3.635 copying k2/python/k2/version/main.py -> build/lib.linux-x86_64-cpython-310/k2/version
3.635 copying k2/python/k2/version/version.py -> build/lib.linux-x86_64-cpython-310/k2/version
3.636 running build_ext
3.786 -- CMAKE_VERSION: 3.30.2
3.786 CMake Deprecation Warning at CMakeLists.txt:20 (cmake_policy):
3.786 The OLD behavior for policy CMP0111 will be removed from a future version
3.786 of CMake.
3.786
3.786 The cmake-policies(7) manual explains that the OLD behaviors of all
3.786 policies are deprecated and that a policy should be set to OLD only under
3.786 specific short-term circumstances. Projects should be ported to the NEW
3.786 behavior and not rely on setting a policy to OLD.
3.786
3.786
3.786 -- Enabled languages: CXX;CUDA
3.869 -- The CXX compiler identification is GNU 11.4.0
5.498 -- The CUDA compiler identification is NVIDIA 12.6.20
5.520 -- Detecting CXX compiler ABI info
5.643 -- Detecting CXX compiler ABI info - done
5.677 -- Check for working CXX compiler: /usr/bin/c++ - skipped
5.678 -- Detecting CXX compile features
5.679 -- Detecting CXX compile features - done
5.687 -- Detecting CUDA compiler ABI info
7.152 -- Detecting CUDA compiler ABI info - done
7.291 -- Check for working CUDA compiler: /usr/local/cuda/bin/nvcc - skipped
7.294 -- Detecting CUDA compile features
7.295 -- Detecting CUDA compile features - done
7.297 -- K2_OS:
7.301 -- Found Git: /usr/bin/git (found version "2.34.1")
7.318 -- C++ Standard version: 17
7.319 -- Looking for C++ include cxxabi.h
7.452 -- Looking for C++ include cxxabi.h - found
7.453 -- Looking for C++ include execinfo.h
7.577 -- Looking for C++ include execinfo.h - found
7.580 -- Performing Test K2_COMPILER_SUPPORTS_CXX17
7.700 -- Performing Test K2_COMPILER_SUPPORTS_CXX17 - Success
9.142 -- Automatic GPU detection failed. Building for common architectures.
9.142 -- Autodetected CUDA architecture(s): 5.0;5.2;6.0;6.1;7.0;7.5;8.0;8.6;8.6+PTX
9.143 -- K2_COMPUTE_ARCH_FLAGS: -gencode;arch=compute_50,code=sm_50;-gencode;arch=compute_52,code=sm_52;-gencode;arch=compute_60,code=sm_60;-gencode;arch=compute_61,code=sm_61;-gencode;arch=compute_70,code=sm_70;-gencode;arch=compute_75,code=sm_75;-gencode;arch=compute_80,code=sm_80;-gencode;arch=compute_86,code=sm_86;-gencode;arch=compute_86,code=compute_86
9.143 -- K2_COMPUTE_ARCH_CANDIDATES 50;60;61;70;75;80;86;90
9.143 -- Adding arch 50
9.143 -- Adding arch 60
9.143 -- Adding arch 61
9.143 -- Adding arch 70
9.143 -- Adding arch 75
9.143 -- Adding arch 80
9.143 -- Adding arch 86
9.143 -- Skipping arch 90
9.143 -- K2_COMPUTE_ARCHS: 50;60;61;70;75;80;86
9.146 -- Found Valgrind: /usr/bin
9.146 -- Found Valgrind: /usr/bin/valgrind
9.146 CMake Warning (dev) at CMakeLists.txt:299 (include):
9.146 Policy CMP0145 is not set: The Dart and FindDart modules are removed. Run
9.146 "cmake --help-policy CMP0145" for policy details. Use the cmake_policy
9.146 command to set the policy and suppress this warning.
9.146
9.146 This warning is for project developers. Use -Wno-dev to suppress it.
9.146
9.146 CMake Warning (dev) at /usr/local/lib/python3.10/dist-packages/cmake/data/share/cmake-3.30/Modules/Dart.cmake:47 (message):
9.146 Policy CMP0145 is not set: The Dart and FindDart modules are removed. Run
9.146 "cmake --help-policy CMP0145" for policy details. Use the cmake_policy
9.146 command to set the policy and suppress this warning.
9.146 Call Stack (most recent call first):
9.146 CMakeLists.txt:299 (include)
9.146 This warning is for project developers. Use -Wno-dev to suppress it.
9.146
9.175 -- To check memory, run ctest -R <NAME> -D ExperimentalMemCheck
9.177 -- Downloading pybind11 from file:///tmp/pybind11-5bc0943ed96836f46489f53961f6c438d2935357.zip
9.764 -- pybind11 is downloaded to /tmp/k2/build/temp.linux-x86_64-cpython-310/_deps/pybind11-src
9.765 CMake Deprecation Warning at build/temp.linux-x86_64-cpython-310/_deps/pybind11-src/CMakeLists.txt:8 (cmake_minimum_required):
9.765 Compatibility with CMake < 3.5 will be removed from a future version of
9.765 CMake.
9.765
9.765 Update the VERSION argument value or use a ... suffix to tell
9.765 CMake that the project does not need compatibility with older versions.
9.765
9.765
9.777 -- pybind11 v2.11.0 dev1
9.780 CMake Warning (dev) at build/temp.linux-x86_64-cpython-310/_deps/pybind11-src/tools/FindPythonLibsNew.cmake:98 (find_package):
9.780 Policy CMP0148 is not set: The FindPythonInterp and FindPythonLibs modules
9.780 are removed. Run "cmake --help-policy CMP0148" for policy details. Use
9.780 the cmake_policy command to set the policy and suppress this warning.
9.780
9.780 Call Stack (most recent call first):
9.780 build/temp.linux-x86_64-cpython-310/_deps/pybind11-src/tools/pybind11Tools.cmake:50 (find_package)
9.780 build/temp.linux-x86_64-cpython-310/_deps/pybind11-src/tools/pybind11Common.cmake:180 (include)
9.780 build/temp.linux-x86_64-cpython-310/_deps/pybind11-src/CMakeLists.txt:208 (include)
9.780 This warning is for project developers. Use -Wno-dev to suppress it.
9.780
9.818 -- Found PythonInterp: /usr/bin/python3 (found suitable version "3.10.12", minimum required is "3.6")
9.860 -- Found PythonLibs: /usr/lib/x86_64-linux-gnu/libpython3.10.so
9.862 -- Performing Test HAS_FLTO
10.03 -- Performing Test HAS_FLTO - Success
10.04 -- Python executable: /usr/bin/python3
12.51 -- Found CUDA: /usr/local/cuda (found version "12.6")
12.51 -- Found CUDAToolkit: /usr/local/cuda/include (found version "12.6.20")
12.52 -- Performing Test CMAKE_HAVE_LIBC_PTHREAD
12.66 -- Performing Test CMAKE_HAVE_LIBC_PTHREAD - Success
12.66 -- Found Threads: TRUE
12.79 -- Caffe2: CUDA detected: 12.6
12.79 -- Caffe2: CUDA nvcc is: /usr/local/cuda/bin/nvcc
12.79 -- Caffe2: CUDA toolkit directory: /usr/local/cuda
12.95 -- Caffe2: Header version is: 12.6
12.95 CMake Warning at /usr/local/lib/python3.10/dist-packages/torch/share/cmake/Caffe2/public/cuda.cmake:143 (message):
12.95 Failed to compute shorthash for libnvrtc.so
12.95 Call Stack (most recent call first):
12.95 /usr/local/lib/python3.10/dist-packages/torch/share/cmake/Caffe2/Caffe2Config.cmake:86 (include)
12.95 /usr/local/lib/python3.10/dist-packages/torch/share/cmake/Torch/TorchConfig.cmake:68 (find_package)
12.95 cmake/torch.cmake:15 (find_package)
12.95 CMakeLists.txt:309 (include)
12.95
12.95
12.95 -- USE_CUDNN is set to 0. Compiling without cuDNN support
12.95 -- USE_CUSPARSELT is set to 0. Compiling without cuSPARSELt support
12.95 CMake Warning at /usr/local/lib/python3.10/dist-packages/torch/share/cmake/Caffe2/public/utils.cmake:382 (message):
12.95 In the future we will require one to explicitly pass TORCH_CUDA_ARCH_LIST
12.95 to cmake instead of implicitly setting it as an env variable. This will
12.95 become a FATAL_ERROR in future version of pytorch.
12.95 Call Stack (most recent call first):
12.95 /usr/local/lib/python3.10/dist-packages/torch/share/cmake/Caffe2/public/cuda.cmake:290 (torch_cuda_get_nvcc_gencode_flag)
12.95 /usr/local/lib/python3.10/dist-packages/torch/share/cmake/Caffe2/Caffe2Config.cmake:86 (include)
12.95 /usr/local/lib/python3.10/dist-packages/torch/share/cmake/Torch/TorchConfig.cmake:68 (find_package)
12.95 cmake/torch.cmake:15 (find_package)
12.95 CMakeLists.txt:309 (include)
12.95
12.95
12.95 -- Added CUDA NVCC flags for: -gencode;arch=compute_52,code=sm_52;-gencode;arch=compute_60,code=sm_60;-gencode;arch=compute_61,code=sm_61;-gencode;arch=compute_70,code=sm_70;-gencode;arch=compute_72,code=sm_72;-gencode;arch=compute_75,code=sm_75;-gencode;arch=compute_80,code=sm_80;-gencode;arch=compute_86,code=sm_86;-gencode;arch=compute_87,code=sm_87;-gencode;arch=compute_90,code=sm_90;-gencode;arch=compute_90,code=compute_90
12.96 CMake Warning at /usr/local/lib/python3.10/dist-packages/torch/share/cmake/Torch/TorchConfig.cmake:22 (message):
12.96 static library kineto_LIBRARY-NOTFOUND not found.
12.96 Call Stack (most recent call first):
12.96 /usr/local/lib/python3.10/dist-packages/torch/share/cmake/Torch/TorchConfig.cmake:120 (append_torchlib_if_found)
12.96 cmake/torch.cmake:15 (find_package)
12.96 CMakeLists.txt:309 (include)
12.96
12.96
12.96 -- Found Torch: /usr/local/lib/python3.10/dist-packages/torch/lib/libtorch.so
12.97 CMake Warning at cmake/torch.cmake:37 (message):
12.97 CMAKE_CUDA_FLAGS is -Wno-deprecated-gpu-targets -lineinfo --expt-extended-lambda -use_fast_math -Xptxas=-w --expt-extended-lambda -gencode arch=compute_50,code=sm_50 -lineinfo --expt-extended-lambda -use_fast_math -Xptxas=-w --expt-extended-lambda -gencode arch=compute_60,code=sm_60 -lineinfo --expt-extended-lambda -use_fast_math -Xptxas=-w --expt-extended-lambda -gencode arch=compute_61,code=sm_61 -lineinfo --expt-extended-lambda -use_fast_math -Xptxas=-w --expt-extended-lambda -gencode arch=compute_70,code=sm_70 -lineinfo --expt-extended-lambda -use_fast_math -Xptxas=-w --expt-extended-lambda -gencode arch=compute_75,code=sm_75 -lineinfo --expt-extended-lambda -use_fast_math -Xptxas=-w --expt-extended-lambda -gencode arch=compute_80,code=sm_80 -lineinfo --expt-extended-lambda -use_fast_math -Xptxas=-w --expt-extended-lambda -gencode arch=compute_86,code=sm_86 -DONNX_NAMESPACE=onnx_c2 -gencode arch=compute_52,code=sm_52 -gencode arch=compute_60,code=sm_60 -gencode arch=compute_61,code=sm_61 -gencode arch=compute_70,code=sm_70 -gencode arch=compute_72,code=sm_72 -gencode arch=compute_75,code=sm_75 -gencode arch=compute_80,code=sm_80 -gencode arch=compute_86,code=sm_86 -gencode arch=compute_87,code=sm_87 -gencode arch=compute_90,code=sm_90 -gencode arch=compute_90,code=compute_90 -Xcudafe --diag_suppress=cc_clobber_ignored,--diag_suppress=field_without_dll_interface,--diag_suppress=base_class_has_different_dll_interface,--diag_suppress=dll_interface_conflict_none_assumed,--diag_suppress=dll_interface_conflict_dllexport_assumed,--diag_suppress=bad_friend_decl --expt-relaxed-constexpr --expt-extended-lambda -D_GLIBCXX_USE_CXX11_ABI=1
12.97 Call Stack (most recent call first):
12.97 CMakeLists.txt:309 (include)
12.97
12.97
17.91 -- K2_TORCH_VERSION: 2.5
20.35 -- PyTorch version: 2.5.0a0+872d972e41.nv24.08
22.91 -- PyTorch cuda version: 12.6
22.91 -- Generated /tmp/k2/build/temp.linux-x86_64-cpython-310/torch_version.py
22.91 -- Downloading moderngpu from file:///tmp/moderngpu-8ec9ac0de8672de7217d014917eedec5317f75f3.zip
23.44 -- moderngpu is downloaded to /tmp/k2/build/temp.linux-x86_64-cpython-310/_deps/moderngpu-src
23.44 -- CMAKE_CUDA_FLAGS: -Wno-deprecated-gpu-targets -lineinfo --expt-extended-lambda -use_fast_math -Xptxas=-w --expt-extended-lambda -gencode arch=compute_50,code=sm_50 -lineinfo --expt-extended-lambda -use_fast_math -Xptxas=-w --expt-extended-lambda -gencode arch=compute_60,code=sm_60 -lineinfo --expt-extended-lambda -use_fast_math -Xptxas=-w --expt-extended-lambda -gencode arch=compute_61,code=sm_61 -lineinfo --expt-extended-lambda -use_fast_math -Xptxas=-w --expt-extended-lambda -gencode arch=compute_70,code=sm_70 -lineinfo --expt-extended-lambda -use_fast_math -Xptxas=-w --expt-extended-lambda -gencode arch=compute_75,code=sm_75 -lineinfo --expt-extended-lambda -use_fast_math -Xptxas=-w --expt-extended-lambda -gencode arch=compute_80,code=sm_80 -lineinfo --expt-extended-lambda -use_fast_math -Xptxas=-w --expt-extended-lambda -gencode arch=compute_86,code=sm_86 -DONNX_NAMESPACE=onnx_c2 -gencode arch=compute_52,code=sm_52 -gencode arch=compute_60,code=sm_60 -gencode arch=compute_61,code=sm_61 -gencode arch=compute_70,code=sm_70 -gencode arch=compute_72,code=sm_72 -gencode arch=compute_75,code=sm_75 -gencode arch=compute_80,code=sm_80 -gencode arch=compute_86,code=sm_86 -gencode arch=compute_87,code=sm_87 -gencode arch=compute_90,code=sm_90 -gencode arch=compute_90,code=compute_90 -Xcudafe --diag_suppress=cc_clobber_ignored,--diag_suppress=field_without_dll_interface,--diag_suppress=base_class_has_different_dll_interface,--diag_suppress=dll_interface_conflict_none_assumed,--diag_suppress=dll_interface_conflict_dllexport_assumed,--diag_suppress=bad_friend_decl --expt-relaxed-constexpr --expt-extended-lambda -D_GLIBCXX_USE_CXX11_ABI=1 --compiler-options -Wall --compiler-options -Wno-strict-overflow --compiler-options -Wno-unknown-pragmas
23.44 -- CMAKE_CXX_FLAGS: -D_GLIBCXX_USE_CXX11_ABI=1 -Wno-unused-variable -Wno-strict-overflow
23.44 -- CMAKE_CUDA_FLAGS: -Wno-deprecated-gpu-targets -lineinfo --expt-extended-lambda -use_fast_math -Xptxas=-w --expt-extended-lambda -gencode arch=compute_50,code=sm_50 -lineinfo --expt-extended-lambda -use_fast_math -Xptxas=-w --expt-extended-lambda -gencode arch=compute_60,code=sm_60 -lineinfo --expt-extended-lambda -use_fast_math -Xptxas=-w --expt-extended-lambda -gencode arch=compute_61,code=sm_61 -lineinfo --expt-extended-lambda -use_fast_math -Xptxas=-w --expt-extended-lambda -gencode arch=compute_70,code=sm_70 -lineinfo --expt-extended-lambda -use_fast_math -Xptxas=-w --expt-extended-lambda -gencode arch=compute_75,code=sm_75 -lineinfo --expt-extended-lambda -use_fast_math -Xptxas=-w --expt-extended-lambda -gencode arch=compute_80,code=sm_80 -lineinfo --expt-extended-lambda -use_fast_math -Xptxas=-w --expt-extended-lambda -gencode arch=compute_86,code=sm_86 -DONNX_NAMESPACE=onnx_c2 -gencode arch=compute_52,code=sm_52 -gencode arch=compute_60,code=sm_60 -gencode arch=compute_61,code=sm_61 -gencode arch=compute_70,code=sm_70 -gencode arch=compute_72,code=sm_72 -gencode arch=compute_75,code=sm_75 -gencode arch=compute_80,code=sm_80 -gencode arch=compute_86,code=sm_86 -gencode arch=compute_87,code=sm_87 -gencode arch=compute_90,code=sm_90 -gencode arch=compute_90,code=compute_90 -Xcudafe --diag_suppress=cc_clobber_ignored,--diag_suppress=field_without_dll_interface,--diag_suppress=base_class_has_different_dll_interface,--diag_suppress=dll_interface_conflict_none_assumed,--diag_suppress=dll_interface_conflict_dllexport_assumed,--diag_suppress=bad_friend_decl --expt-relaxed-constexpr --expt-extended-lambda -D_GLIBCXX_USE_CXX11_ABI=1 --compiler-options -Wall --compiler-options -Wno-strict-overflow --compiler-options -Wno-unknown-pragmas
23.45 -- Generated /tmp/k2/build/temp.linux-x86_64-cpython-310/k2/csrc/version.h
23.45 -- PYTHON_LIBRARY: /usr/lib/x86_64-linux-gnu/libpython3.10.so
23.46 -- Including k2/torch. K2_TORCH_VERSION is 2.5
23.46 -- Downloading kaldifeat from https://github.com/csukuangfj/kaldifeat/archive/refs/tags/v1.25.4.tar.gz
26.73 -- kaldifeat is downloaded to /tmp/k2/build/temp.linux-x86_64-cpython-310/_deps/kaldifeat-src
26.73 -- kaldifeat's binary dir is /tmp/k2/build/temp.linux-x86_64-cpython-310/_deps/kaldifeat-build
26.79 -- The C compiler identification is GNU 11.4.0
26.80 -- Detecting C compiler ABI info
26.91 -- Detecting C compiler ABI info - done
26.94 -- Check for working C compiler: /usr/bin/cc - skipped
26.94 -- Detecting C compile features
26.94 -- Detecting C compile features - done
26.95 -- C++ Standard version: 17
26.95 -- BUILD_SHARED_LIBS ON
26.95 -- Python executable: /usr/bin/python3
29.47 CMake Warning at cmake/torch.cmake:37 (message):
29.47 CMAKE_CUDA_FLAGS is -Wno-deprecated-gpu-targets -lineinfo --expt-extended-lambda -use_fast_math -Xptxas=-w --expt-extended-lambda -gencode arch=compute_50,code=sm_50 -lineinfo --expt-extended-lambda -use_fast_math -Xptxas=-w --expt-extended-lambda -gencode arch=compute_60,code=sm_60 -lineinfo --expt-extended-lambda -use_fast_math -Xptxas=-w --expt-extended-lambda -gencode arch=compute_61,code=sm_61 -lineinfo --expt-extended-lambda -use_fast_math -Xptxas=-w --expt-extended-lambda -gencode arch=compute_70,code=sm_70 -lineinfo --expt-extended-lambda -use_fast_math -Xptxas=-w --expt-extended-lambda -gencode arch=compute_75,code=sm_75 -lineinfo --expt-extended-lambda -use_fast_math -Xptxas=-w --expt-extended-lambda -gencode arch=compute_80,code=sm_80 -lineinfo --expt-extended-lambda -use_fast_math -Xptxas=-w --expt-extended-lambda -gencode arch=compute_86,code=sm_86 -DONNX_NAMESPACE=onnx_c2 -gencode arch=compute_52,code=sm_52 -gencode arch=compute_60,code=sm_60 -gencode arch=compute_61,code=sm_61 -gencode arch=compute_70,code=sm_70 -gencode arch=compute_72,code=sm_72 -gencode arch=compute_75,code=sm_75 -gencode arch=compute_80,code=sm_80 -gencode arch=compute_86,code=sm_86 -gencode arch=compute_87,code=sm_87 -gencode arch=compute_90,code=sm_90 -gencode arch=compute_90,code=compute_90 -Xcudafe --diag_suppress=cc_clobber_ignored,--diag_suppress=field_without_dll_interface,--diag_suppress=base_class_has_different_dll_interface,--diag_suppress=dll_interface_conflict_none_assumed,--diag_suppress=dll_interface_conflict_dllexport_assumed,--diag_suppress=bad_friend_decl --expt-relaxed-constexpr --expt-extended-lambda -D_GLIBCXX_USE_CXX11_ABI=1 --compiler-options -Wall --compiler-options -Wno-strict-overflow --compiler-options -Wno-unknown-pragmas -D_GLIBCXX_USE_CXX11_ABI=1
29.47 Call Stack (most recent call first):
29.47 build/temp.linux-x86_64-cpython-310/_deps/kaldifeat-src/CMakeLists.txt:62 (include)
29.47
29.47
34.37 -- K2_TORCH_VERSION: 2.5
36.89 -- PyTorch version: 2.5.0a0+872d972e41.nv24.08
39.33 -- PyTorch cuda version: 12.6
39.33 -- CMAKE_CXX_FLAGS: -D_GLIBCXX_USE_CXX11_ABI=1 -Wno-unused-variable -Wno-strict-overflow -D_GLIBCXX_USE_CXX11_ABI=1
39.33 -- CMAKE_INSTALL_PREFIX: /tmp/k2/build/lib.linux-x86_64-cpython-310/k2
39.34 -- All headers: /tmp/k2/build/temp.linux-x86_64-cpython-310/_deps/kaldifeat-src/kaldifeat/csrc/feature-common-inl.h;/tmp/k2/build/temp.linux-x86_64-cpython-310/_deps/kaldifeat-src/kaldifeat/csrc/feature-common.h;/tmp/k2/build/temp.linux-x86_64-cpython-310/_deps/kaldifeat-src/kaldifeat/csrc/feature-fbank.h;/tmp/k2/build/temp.linux-x86_64-cpython-310/_deps/kaldifeat-src/kaldifeat/csrc/feature-functions.h;/tmp/k2/build/temp.linux-x86_64-cpython-310/_deps/kaldifeat-src/kaldifeat/csrc/feature-mfcc.h;/tmp/k2/build/temp.linux-x86_64-cpython-310/_deps/kaldifeat-src/kaldifeat/csrc/feature-plp.h;/tmp/k2/build/temp.linux-x86_64-cpython-310/_deps/kaldifeat-src/kaldifeat/csrc/feature-spectrogram.h;/tmp/k2/build/temp.linux-x86_64-cpython-310/_deps/kaldifeat-src/kaldifeat/csrc/feature-window.h;/tmp/k2/build/temp.linux-x86_64-cpython-310/_deps/kaldifeat-src/kaldifeat/csrc/log.h;/tmp/k2/build/temp.linux-x86_64-cpython-310/_deps/kaldifeat-src/kaldifeat/csrc/matrix-functions.h;/tmp/k2/build/temp.linux-x86_64-cpython-310/_deps/kaldifeat-src/kaldifeat/csrc/mel-computations.h;/tmp/k2/build/temp.linux-x86_64-cpython-310/_deps/kaldifeat-src/kaldifeat/csrc/online-feature-itf.h;/tmp/k2/build/temp.linux-x86_64-cpython-310/_deps/kaldifeat-src/kaldifeat/csrc/online-feature.h;/tmp/k2/build/temp.linux-x86_64-cpython-310/_deps/kaldifeat-src/kaldifeat/csrc/pitch-functions.h;/tmp/k2/build/temp.linux-x86_64-cpython-310/deps/kaldifeat-src/kaldifeat/csrc/whisper-fbank.h;/tmp/k2/build/temp.linux-x86_64-cpython-310/deps/kaldifeat-src/kaldifeat/csrc/whisper-mel-bank.h;/tmp/k2/build/temp.linux-x86_64-cpython-310/deps/kaldifeat-src/kaldifeat/csrc/whisper-v3-mel-bank.h
39.35 -- Configuring done (35.6s)
39.51 -- Generating done (0.2s)
39.51 -- Build files have been written to: /tmp/k2/build/temp.linux-x86_64-cpython-310
39.55 /**
39.55 * @brief
39.55 * This file contains information for building k2, which is helpful
39.55 * for describing the environment in which k2 is used when creating
39.55 * an issue on GitHub.
39.55 *
39.55 * @copyright
39.55 * Copyright (c) 2020 Mobvoi Inc. (authors: Fangjun Kuang)
39.55 *
39.55 * @copyright
39.55 * See LICENSE for clarification regarding multiple authors
39.55 *
39.55 * @caution
39.55 * This file is auto-generated by CMake. Do NOT edit!
39.55 * You can find it at build_dir/k2/csrc/version.h
39.55 */
39.55
39.55 #ifndef K2_CSRC_VERSION_H
39.55 #define K2_CSRC_VERSION_H
39.55
39.55 namespace k2 {
39.55
39.55 // Version of k2 in "major.minor.patch" format
39.55 static constexpr const char *kVersion = "1.24.4";
39.55
39.55 // The commit used to build k2
39.55 static constexpr const char *kGitSha1 = "65713bc93140d63cfec3ffeec17d7f603fb74a71";
39.55
39.55 // Date of the commit used to build k2
39.55 static constexpr const char *kGitDate = "Wed Nov 27 10:34:15 2024";
39.55
39.55 // Version of CUDA used to build k2.
39.55 // Its format is "major.minor", e.g., 10.1
39.55 static constexpr const char *kCudaVersion = "12.6";
39.55
39.55 // cuDNN version, e.g., 8.0.2
39.55 static constexpr const char *kCudnnVersion = "";
39.55
39.55 // clang-format off
39.55 // Version of Python used to build k2 Python bindings.
39.55 static constexpr const char *kPythonVersion = "3.10";
39.55
39.55 // clang-format on
39.55
39.55 // CMake build type, e.g., Release or Debug.
39.55 static constexpr const char *kBuildType = "Release";
39.55
39.55 // The operating system that is used to build k2, e.g., Ubuntu 16.04 LTS
39.55 static constexpr const char *kOS = R"os()os";
39.55
39.55 // e.g., 3.18.0
39.55 static constexpr const char *kCMakeVersion = "3.30.2";
39.55
39.55 // Version of the compiler, e.g., 5.4.0
39.55 static constexpr const char *kGCCVersion = "11.4.0";
39.55
39.55 // CUDA flags used to compile k2
39.55 static constexpr const char *kCMakeCudaFlags = R"cuda_flags(-Wno-deprecated-gpu-targets -lineinfo --expt-extended-lambda -use_fast_math -Xptxas=-w --expt-extended-lambda -gencode arch=compute_50,code=sm_50 -lineinfo --expt-extended-lambda -use_fast_math -Xptxas=-w --expt-extended-lambda -gencode arch=compute_60,code=sm_60 -lineinfo --expt-extended-lambda -use_fast_math -Xptxas=-w --expt-extended-lambda -gencode arch=compute_61,code=sm_61 -lineinfo --expt-extended-lambda -use_fast_math -Xptxas=-w --expt-extended-lambda -gencode arch=compute_70,code=sm_70 -lineinfo --expt-extended-lambda -use_fast_math -Xptxas=-w --expt-extended-lambda -gencode arch=compute_75,code=sm_75 -lineinfo --expt-extended-lambda -use_fast_math -Xptxas=-w --expt-extended-lambda -gencode arch=compute_80,code=sm_80 -lineinfo --expt-extended-lambda -use_fast_math -Xptxas=-w --expt-extended-lambda -gencode arch=compute_86,code=sm_86 -DONNX_NAMESPACE=onnx_c2 -gencode arch=compute_52,code=sm_52 -gencode arch=compute_60,code=sm_60 -gencode arch=compute_61,code=sm_61 -gencode arch=compute_70,code=sm_70 -gencode arch=compute_72,code=sm_72 -gencode arch=compute_75,code=sm_75 -gencode arch=compute_80,code=sm_80 -gencode arch=compute_86,code=sm_86 -gencode arch=compute_87,code=sm_87 -gencode arch=compute_90,code=sm_90 -gencode arch=compute_90,code=compute_90 -Xcudafe --diag_suppress=cc_clobber_ignored,--diag_suppress=field_without_dll_interface,--diag_suppress=base_class_has_different_dll_interface,--diag_suppress=dll_interface_conflict_none_assumed,--diag_suppress=dll_interface_conflict_dllexport_assumed,--diag_suppress=bad_friend_decl --expt-relaxed-constexpr --expt-extended-lambda -D_GLIBCXX_USE_CXX11_ABI=1 --compiler-options -Wall --compiler-options -Wno-strict-overflow --compiler-options -Wno-unknown-pragmas )cuda_flags";
39.55
39.55 // CXX flags used to compile k2
39.55 static constexpr const char *kCMakeCxxFlags = R"cxx_flags( -D_GLIBCXX_USE_CXX11_ABI=1 -Wno-unused-variable -Wno-strict-overflow )cxx_flags";
39.55
39.55 // Which PyTorch version k2 is using, e.g., 1.6.0+cu101
39.55 static constexpr const char *kTorchVersion = "2.5.0a0+872d972e41.nv24.08";
39.55
39.55 // Which CUDA version PyTorch is using, e.g., 10.1
39.55 static constexpr const char *kTorchCudaVersion = "12.6";
39.55
39.55 #ifndef K2_WITH_CUDA
39.55 #define K2_WITH_CUDA
39.55 #endif
39.55
39.55 #ifdef K2_WITH_CUDA
39.55 static constexpr bool kWithCuda = true;
39.55 #else
39.55 static constexpr bool kWithCuda = false;
39.55 #endif
39.55
39.55 // Indicate whether NVTX is enabled or not
39.55 #ifndef K2_ENABLE_NVTX
39.55 #define K2_ENABLE_NVTX
39.55 #endif
39.55
39.55 #ifdef K2_ENABLE_NVTX
39.55 static constexpr bool kEnableNvtx = true;
39.55 #else
39.55 static constexpr bool kEnableNvtx = false;
39.55 #endif
39.55 } // namespace k2
39.55
39.55 #endif // K2_CSRC_VERSION_H
39.61 [ 1%] Building CUDA object k2/csrc/CMakeFiles/k2_log.dir/log.cu.o
39.61 [ 2%] Building CXX object _deps/kaldifeat-build/kaldifeat/csrc/CMakeFiles/kaldifeat_core.dir/feature-fbank.cc.o
39.61 [ 4%] Building CXX object _deps/kaldifeat-build/kaldifeat/csrc/CMakeFiles/kaldifeat_core.dir/feature-mfcc.cc.o
39.61 [ 4%] Building CXX object _deps/kaldifeat-build/kaldifeat/csrc/CMakeFiles/kaldifeat_core.dir/feature-functions.cc.o
53.04 [ 5%] Linking CUDA shared library ../../lib/libk2_log.so
53.17 [ 5%] Built target k2_log
53.19 [ 6%] Building CXX object _deps/kaldifeat-build/kaldifeat/csrc/CMakeFiles/kaldifeat_core.dir/feature-plp.cc.o
54.31 [ 6%] Building CXX object _deps/kaldifeat-build/kaldifeat/csrc/CMakeFiles/kaldifeat_core.dir/feature-spectrogram.cc.o
63.33 [ 7%] Building CXX object _deps/kaldifeat-build/kaldifeat/csrc/CMakeFiles/kaldifeat_core.dir/feature-window.cc.o
64.10 [ 8%] Building CXX object _deps/kaldifeat-build/kaldifeat/csrc/CMakeFiles/kaldifeat_core.dir/matrix-functions.cc.o
76.85 [ 9%] Building CXX object _deps/kaldifeat-build/kaldifeat/csrc/CMakeFiles/kaldifeat_core.dir/mel-computations.cc.o
77.68 [ 10%] Building CXX object _deps/kaldifeat-build/kaldifeat/csrc/CMakeFiles/kaldifeat_core.dir/online-feature.cc.o
78.62 [ 10%] Building CXX object _deps/kaldifeat-build/kaldifeat/csrc/CMakeFiles/kaldifeat_core.dir/whisper-fbank.cc.o
86.65 [ 11%] Building CXX object k2/csrc/host/CMakeFiles/fsa.dir/arcsort.cc.o
88.22 [ 12%] Building CXX object k2/csrc/host/CMakeFiles/fsa.dir/aux_labels.cc.o
90.28 [ 12%] Building CXX object k2/csrc/host/CMakeFiles/fsa.dir/connect.cc.o
92.02 [ 13%] Building CXX object k2/csrc/host/CMakeFiles/fsa.dir/determinize.cc.o
96.53 [ 14%] Building CXX object k2/csrc/host/CMakeFiles/fsa.dir/determinize_pruned.cc.o
100.5 [ 15%] Building CXX object k2/csrc/host/CMakeFiles/fsa.dir/fsa.cc.o
100.8 [ 16%] Building CXX object k2/csrc/host/CMakeFiles/fsa.dir/fsa_equivalent.cc.o
101.3 [ 17%] Building CXX object k2/csrc/host/CMakeFiles/fsa.dir/fsa_renderer.cc.o
101.4 [ 17%] Building CXX object k2/csrc/host/CMakeFiles/fsa.dir/fsa_util.cc.o
101.6 [ 18%] Linking CXX shared library ../../../../lib/libkaldifeat_core.so
102.2 [ 18%] Built target kaldifeat_core
102.2 [ 19%] Building CXX object k2/csrc/host/CMakeFiles/fsa.dir/intersect.cc.o
102.4 [ 20%] Building CXX object k2/csrc/host/CMakeFiles/fsa.dir/properties.cc.o
103.5 [ 21%] Building CXX object k2/csrc/host/CMakeFiles/fsa.dir/rmepsilon_pruned.cc.o
103.7 [ 22%] Building CXX object k2/csrc/host/CMakeFiles/fsa.dir/topsort.cc.o
103.9 [ 22%] Building CXX object k2/csrc/host/CMakeFiles/fsa.dir/util.cc.o
104.8 [ 23%] Building CXX object k2/csrc/host/CMakeFiles/fsa.dir/weights.cc.o
105.1 [ 24%] Building CXX object k2/torch/csrc/CMakeFiles/k2_fbank.dir/features.cc.o
107.1 [ 25%] Linking CXX shared library ../../../lib/libk2fsa.so
107.2 [ 25%] Built target fsa
107.3 [ 25%] Building CUDA object k2/csrc/CMakeFiles/context.dir/array_of_ragged.cu.o
107.3 [ 26%] Building CUDA object k2/csrc/CMakeFiles/context.dir/algorithms.cu.o
107.3 [ 27%] Building CUDA object k2/csrc/CMakeFiles/context.dir/array_ops.cu.o
107.9 In file included from /usr/include/nvtx3/nvtx3.hpp:629,
107.9 from /usr/local/cuda/include/cub/detail/nvtx.cuh:47,
107.9 from /usr/local/cuda/include/cub/device/device_adjacent_difference.cuh:41,
107.9 from /usr/local/cuda/include/cub/cub.cuh:62,
107.9 from /tmp/k2/k2/csrc/cub.h:24,
107.9 from /tmp/k2/k2/csrc/array_inl.h:33,
107.9 from /tmp/k2/k2/csrc/array.h:969,
107.9 from /tmp/k2/k2/csrc/array_of_ragged.h:27,
107.9 from /tmp/k2/k2/csrc/array_of_ragged.cu:21:
107.9 /usr/include/nvtx3/nvToolsExt.h:150:2: error: #error "Trying to #include NVTX version 3 in a source file where an older NVTX version has already been included. If you are not directly using NVTX (the NVIDIA Tools Extension library), you are getting this error because libraries you are using have included different versions of NVTX. Suggested solutions are: (1) reorder #includes so the newest NVTX version is included first, (2) avoid using the conflicting libraries in the same .c/.cpp file, or (3) update the library using the older NVTX version to use the newer version instead."
107.9 150 | #error "Trying to #include NVTX version 3 in a source file where an older NVTX version has already been included. If you are not directly using NVTX (the NVIDIA Tools Extension library), you are getting this error because libraries you are using have included different versions of NVTX. Suggested solutions are: (1) reorder #includes so the newest NVTX version is included first, (2) avoid using the conflicting libraries in the same .c/.cpp file, or (3) update the library using the older NVTX version to use the newer version instead."
107.9 | ^~~~~
107.9 In file included from /usr/include/nvtx3/nvtx3.hpp:629,
107.9 from /usr/local/cuda/include/cub/detail/nvtx.cuh:47,
107.9 from /usr/local/cuda/include/cub/device/device_adjacent_difference.cuh:41,
107.9 from /usr/local/cuda/include/cub/cub.cuh:62,
107.9 from /tmp/k2/k2/csrc/cub.h:24,
107.9 from /tmp/k2/k2/csrc/array_inl.h:33,
107.9 from /tmp/k2/k2/csrc/array.h:969,
107.9 from /tmp/k2/k2/csrc/array_ops.cu:22:
107.9 /usr/include/nvtx3/nvToolsExt.h:150:2: error: #error "Trying to #include NVTX version 3 in a source file where an older NVTX version has already been included. If you are not directly using NVTX (the NVIDIA Tools Extension library), you are getting this error because libraries you are using have included different versions of NVTX. Suggested solutions are: (1) reorder #includes so the newest NVTX version is included first, (2) avoid using the conflicting libraries in the same .c/.cpp file, or (3) update the library using the older NVTX version to use the newer version instead."
107.9 150 | #error "Trying to #include NVTX version 3 in a source file where an older NVTX version has already been included. If you are not directly using NVTX (the NVIDIA Tools Extension library), you are getting this error because libraries you are using have included different versions of NVTX. Suggested solutions are: (1) reorder #includes so the newest NVTX version is included first, (2) avoid using the conflicting libraries in the same .c/.cpp file, or (3) update the library using the older NVTX version to use the newer version instead."
107.9 | ^~~~~
107.9 In file included from /usr/include/nvtx3/nvtx3.hpp:629,
107.9 from /usr/local/cuda/include/cub/detail/nvtx.cuh:47,
107.9 from /usr/local/cuda/include/cub/device/device_adjacent_difference.cuh:41,
107.9 from /usr/local/cuda/include/cub/cub.cuh:62,
107.9 from /tmp/k2/k2/csrc/cub.h:24,
107.9 from /tmp/k2/k2/csrc/array_inl.h:33,
107.9 from /tmp/k2/k2/csrc/array.h:969,
107.9 from /tmp/k2/k2/csrc/algorithms.h:25,
107.9 from /tmp/k2/k2/csrc/algorithms.cu:22:
107.9 /usr/include/nvtx3/nvToolsExt.h:150:2: error: #error "Trying to #include NVTX version 3 in a source file where an older NVTX version has already been included. If you are not directly using NVTX (the NVIDIA Tools Extension library), you are getting this error because libraries you are using have included different versions of NVTX. Suggested solutions are: (1) reorder #includes so the newest NVTX version is included first, (2) avoid using the conflicting libraries in the same .c/.cpp file, or (3) update the library using the older NVTX version to use the newer version instead."
107.9 150 | #error "Trying to #include NVTX version 3 in a source file where an older NVTX version has already been included. If you are not directly using NVTX (the NVIDIA Tools Extension library), you are getting this error because libraries you are using have included different versions of NVTX. Suggested solutions are: (1) reorder #includes so the newest NVTX version is included first, (2) avoid using the conflicting libraries in the same .c/.cpp file, or (3) update the library using the older NVTX version to use the newer version instead."
107.9 | ^~~~~
108.2 make[2]: *** [k2/csrc/CMakeFiles/context.dir/build.make:92: k2/csrc/CMakeFiles/context.dir/array_of_ragged.cu.o] Error 1
108.2 make[2]: *** Waiting for unfinished jobs....
108.2 make[2]: *** [k2/csrc/CMakeFiles/context.dir/build.make:107: k2/csrc/CMakeFiles/context.dir/array_ops.cu.o] Error 1
108.2 make[2]: *** [k2/csrc/CMakeFiles/context.dir/build.make:77: k2/csrc/CMakeFiles/context.dir/algorithms.cu.o] Error 1
108.2 make[1]: *** [CMakeFiles/Makefile2:1114: k2/csrc/CMakeFiles/context.dir/all] Error 2
108.2 make[1]: *** Waiting for unfinished jobs....
127.6 [ 27%] Linking CXX shared library ../../../lib/libk2_fbank.so
128.1 [ 27%] Built target k2_fbank
128.1 make: *** [Makefile:146: all] Error 2
128.1 cmake_path: /usr/local/bin/cmake
128.1 For fast compilation, run:
128.1 export K2_MAKE_ARGS="-j"; python setup.py install
128.1 Setting make_args to '-j4'
128.1 Setting PYTHON_EXECUTABLE to /usr/bin/python3
128.1 build command is:
128.1
128.1 cd build/temp.linux-x86_64-cpython-310
128.1
128.1 cmake -DCMAKE_BUILD_TYPE=Release -DPYTHON_EXECUTABLE=/usr/bin/python3 -DK2_ENABLE_BENCHMARK=OFF -DK2_ENABLE_TESTS=OFF -DCMAKE_INSTALL_PREFIX=/tmp/k2/build/lib.linux-x86_64-cpython-310/k2 -DCMAKE_CXX_STANDARD=17 /tmp/k2
128.1
128.1 cat k2/csrc/version.h
128.1
128.1 make -j4 install
128.1
128.1 Traceback (most recent call last):
128.1 File "/tmp/k2/setup.py", line 240, in
128.1 setuptools.setup(
128.1 File "/usr/local/lib/python3.10/dist-packages/setuptools/init.py", line 103, in setup
128.1 return distutils.core.setup(**attrs)
128.1 File "/usr/local/lib/python3.10/dist-packages/setuptools/_distutils/core.py", line 184, in setup
128.1 return run_commands(dist)
128.1 File "/usr/local/lib/python3.10/dist-packages/setuptools/_distutils/core.py", line 200, in run_commands
128.1 dist.run_commands()
128.1 File "/usr/local/lib/python3.10/dist-packages/setuptools/_distutils/dist.py", line 970, in run_commands
128.1 self.run_command(cmd)
128.1 File "/usr/local/lib/python3.10/dist-packages/setuptools/dist.py", line 974, in run_command
128.1 super().run_command(command)
128.1 File "/usr/local/lib/python3.10/dist-packages/setuptools/_distutils/dist.py", line 989, in run_command
128.1 cmd_obj.run()
128.1 File "/usr/local/lib/python3.10/dist-packages/setuptools/command/install.py", line 87, in run
128.1 self.do_egg_install()
128.1 File "/usr/local/lib/python3.10/dist-packages/setuptools/command/install.py", line 139, in do_egg_install
128.1 self.run_command('bdist_egg')
128.1 File "/usr/local/lib/python3.10/dist-packages/setuptools/_distutils/cmd.py", line 316, in run_command
128.1 self.distribution.run_command(command)
128.1 File "/usr/local/lib/python3.10/dist-packages/setuptools/dist.py", line 974, in run_command
128.1 super().run_command(command)
128.1 File "/usr/local/lib/python3.10/dist-packages/setuptools/_distutils/dist.py", line 989, in run_command
128.1 cmd_obj.run()
128.1 File "/usr/local/lib/python3.10/dist-packages/setuptools/command/bdist_egg.py", line 167, in run
128.1 cmd = self.call_command('install_lib', warn_dir=False)
128.1 File "/usr/local/lib/python3.10/dist-packages/setuptools/command/bdist_egg.py", line 153, in call_command
128.1 self.run_command(cmdname)
128.1 File "/usr/local/lib/python3.10/dist-packages/setuptools/_distutils/cmd.py", line 316, in run_command
128.1 self.distribution.run_command(command)
128.1 File "/usr/local/lib/python3.10/dist-packages/setuptools/dist.py", line 974, in run_command
128.1 super().run_command(command)
128.1 File "/usr/local/lib/python3.10/dist-packages/setuptools/_distutils/dist.py", line 989, in run_command
128.1 cmd_obj.run()
128.1 File "/usr/local/lib/python3.10/dist-packages/setuptools/command/install_lib.py", line 12, in run
128.1 self.build()
128.1 File "/usr/local/lib/python3.10/dist-packages/setuptools/_distutils/command/install_lib.py", line 110, in build
128.1 self.run_command('build_ext')
128.1 File "/usr/local/lib/python3.10/dist-packages/setuptools/_distutils/cmd.py", line 316, in run_command
128.1 self.distribution.run_command(command)
128.1 File "/usr/local/lib/python3.10/dist-packages/setuptools/dist.py", line 974, in run_command
128.1 super().run_command(command)
128.1 File "/usr/local/lib/python3.10/dist-packages/setuptools/_distutils/dist.py", line 989, in run_command
128.1 cmd_obj.run()
128.1 File "/usr/local/lib/python3.10/dist-packages/setuptools/command/build_ext.py", line 93, in run
128.1 _build_ext.run(self)
128.1 File "/usr/local/lib/python3.10/dist-packages/setuptools/_distutils/command/build_ext.py", line 359, in run
128.1 self.build_extensions()
128.1 File "/usr/local/lib/python3.10/dist-packages/setuptools/_distutils/command/build_ext.py", line 479, in build_extensions
128.1 self._build_extensions_serial()
128.1 File "/usr/local/lib/python3.10/dist-packages/setuptools/_distutils/command/build_ext.py", line 505, in _build_extensions_serial
128.1 self.build_extension(ext)
128.1 File "/tmp/k2/setup.py", line 212, in build_extension
128.1 raise Exception("Failed to build k2")
128.1 Exception: Failed to build k2
ERROR: process "/bin/sh -c cd /tmp/k2 ; python3 setup.py install" did not complete successfully: exit code: 1

[csukuangfj]

Please find out how many nvToolsExt.h is contained in your docker container.

[yjzhong89]

Here are all the nvToolsExt.h files

/opt/pytorch/pytorch/third_party/nccl/nccl/src/include/nvtx3/nvToolsExt.h
/usr/include/nvtx3/nvToolsExt.h
/usr/local/lib/python3.10/dist-packages/nvidia/nvtx/include/nvToolsExt.h
/usr/local/lib/python3.10/dist-packages/nvidia/nvtx/include/nvtx3/nvToolsExt.h
/usr/local/cuda-12.6/targets/x86_64-linux/include/nvToolsExt.h
/usr/local/cuda-12.6/targets/x86_64-linux/include/nvtx3/nvToolsExt.h
/usr/local/cuda-12.6/NsightSystems-cli-2024.4.2/target-linux-x64/nvtx/include/nvtx3/nvToolsExt.h

[csukuangfj]

Please keep only one of them, delete the build directory of k2, and retry.

[yjzhong89]

I solve the problem by deleting the file /usr/include/nvtx3/nvToolsExt.h