[Bug] Windows: `'ProcessSession': is not a member of 'tvm::runtime::Session' [C:\repo\mlc-llm\build\mlc_llm_objs.vcxproj]`

[jparismorgan]

🐛 Bug

I am trying to build MLC Runtime from source on Windows (https://mlc.ai/mlc-llm/docs/deploy/cli.html#id5) but getting:

C:\repo\mlc-llm\cpp\llm_chat.cc(158,29): error C2039: 'ProcessSession': is not a member of 'tvm::runtime::Session' [C:\repo\mlc-llm\build\mlc_llm_objs.vcxproj]
C:\repo\mlc-llm\3rdparty\tvm\include\tvm/runtime/disco/session.h(240,7): message : see declaration of 'tvm::runtime::Session' [C:\repo\mlc-llm\build\mlc_llm_objs.vcxproj]
C:\repo\mlc-llm\cpp\llm_chat.cc(158,29): error C3861: 'ProcessSession': identifier not found [C:\repo\mlc-llm\build\mlc_llm_objs.vcxproj]
C:\repo\mlc-llm\cpp\llm_chat.cc(159,19): error C2039: 'InitCCL': is not a member of 'tvm::runtime::SessionObj' [C:\repo\mlc-llm\build\mlc_llm_objs.vcxproj]
C:\repo\mlc-llm\3rdparty\tvm\include\tvm/runtime/disco/session.h(165,7): message : see declaration of 'tvm::runtime::SessionObj' [C:\repo\mlc-llm\build\mlc_llm_objs.vcxproj]
  Generating Code...

To Reproduce

build\config.cmake contains:

set(TVM_HOME 3rdparty/tvm)
set(CMAKE_BUILD_TYPE RelWithDebInfo)
set(USE_CUDA ON)
set(USE_CUTLASS OFF)
set(USE_CUBLAS OFF)
set(USE_ROCM OFF)
set(USE_VULKAN OFF)
set(USE_METAL OFF)
set(USE_OPENCL OFF)

I then build:

paris@DESKTOP-Q28GDF3 MINGW64 /c/repo/mlc-llm/build (main)
$ cmake .. && cmake --build .
-- Selecting Windows SDK version 10.0.22621.0 to target Windows 10.0.19045.
-- Hide private symbols
-- TVM_HOME: 3rdparty/tvm
-- Didn't find the path to CCACHE, disabling ccache
-- VTA build is skipped in Windows..
CMake Warning (dev) at 3rdparty/tvm/cmake/utils/FindCUDA.cmake:44 (find_package):
  Policy CMP0146 is not set: The FindCUDA module is removed.  Run "cmake
  --help-policy CMP0146" for policy details.  Use the cmake_policy command to
  set the policy and suppress this warning.

Call Stack (most recent call first):
  3rdparty/tvm/cmake/modules/CUDA.cmake:19 (find_cuda)
  3rdparty/tvm/CMakeLists.txt:532 (include)
This warning is for project developers.  Use -Wno-dev to suppress it.

-- Found CUDA_TOOLKIT_ROOT_DIR=C:/Program Files/NVIDIA GPU Computing Toolkit/CUDA/v11.1
-- Found CUDA_CUDA_LIBRARY=C:/Program Files/NVIDIA GPU Computing Toolkit/CUDA/v11.1/lib/x64/cuda.lib
-- Found CUDA_CUDART_LIBRARY=C:/Program Files/NVIDIA GPU Computing Toolkit/CUDA/v11.1/lib/x64/cudart.lib
-- Found CUDA_NVRTC_LIBRARY=C:/Program Files/NVIDIA GPU Computing Toolkit/CUDA/v11.1/lib/x64/nvrtc.lib
-- Found CUDA_CUDNN_INCLUDE_DIRS=
-- Found CUDA_CUDNN_LIBRARY=
-- Found CUDA_CUBLAS_LIBRARY=C:/Program Files/NVIDIA GPU Computing Toolkit/CUDA/v11.1/lib/x64/cublas.lib
-- Found CUDA_CURAND_LIBRARY=
-- Found CUDA_CUBLASLT_LIBRARY=C:/Program Files/NVIDIA GPU Computing Toolkit/CUDA/v11.1/lib/x64/cublasLt.lib
-- Build with CUDA 11.1 support
-- Build with contrib.random
-- Build with contrib.sort
-- Build with contrib.hybriddump
-- Git found: C:/Program Files/Git/mingw64/bin/git.exe
-- Found TVM_GIT_COMMIT_HASH=631f37b6bf8b101d16ecc55de7e6a749a3588570
-- Found TVM_GIT_COMMIT_TIME=2023-09-13 12:22:20 -0400
-- Building with TVM Map...
-- Build with thread support...
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD - Failed
-- Looking for pthread_create in pthreads
-- Looking for pthread_create in pthreads - not found
-- Looking for pthread_create in pthread
-- Looking for pthread_create in pthread - not found
-- Found Threads: TRUE  
-- Performing Test FILE_PREFIX_MAP_SUPPORTED
-- Performing Test FILE_PREFIX_MAP_SUPPORTED - Failed
-- system-nameWindows
CMake Deprecation Warning at 3rdparty/tokenizers-cpp/msgpack/CMakeLists.txt:1 (CMAKE_MINIMUM_REQUIRED):
  Compatibility with CMake < 3.5 will be removed from a future version of
  CMake.

  Update the VERSION argument <min> value or use a ...<max> suffix to tell
  CMake that the project does not need compatibility with older versions.


-- Could NOT find Doxygen (missing: DOXYGEN_EXECUTABLE) 
CMake Deprecation Warning at 3rdparty/tokenizers-cpp/sentencepiece/CMakeLists.txt:15 (cmake_minimum_required):
  Compatibility with CMake < 3.5 will be removed from a future version of
  CMake.

  Update the VERSION argument <min> value or use a ...<max> suffix to tell
  CMake that the project does not need compatibility with older versions.


-- VERSION: 0.2.00
-- Configuring done (4.7s)
-- Generating done (0.6s)
-- Build files have been written to: C:/repo/mlc-llm/build
MSBuild version 17.7.2+d6990bcfa for .NET Framework

  Checking File Globs
  1>Checking Build System
  Building Custom Rule C:/repo/mlc-llm/CMakeLists.txt
  cli_main.cc
C:\repo\mlc-llm\3rdparty\tvm\include\tvm/runtime/logging.h(353,9): warning C4068: unknown pragma 'disagnostic' [C:\repo\mlc-llm\build\mlc_cli_objs.vcxproj]
C:\repo\mlc-llm\3rdparty\tvm\include\tvm/runtime/logging.h(358,9): warning C4068: unknown pragma 'disagnostic' [C:\repo\mlc-llm\build\mlc_cli_objs.vcxproj]
C:\repo\mlc-llm\3rdparty\tvm\include\tvm/runtime/logging.h(563,9): warning C4068: unknown pragma 'GCC' [C:\repo\mlc-llm\build\mlc_cli_objs.vcxproj]
C:\repo\mlc-llm\3rdparty\tvm\include\tvm/runtime/logging.h(564,9): warning C4068: unknown pragma 'GCC' [C:\repo\mlc-llm\build\mlc_cli_objs.vcxproj]
C:\repo\mlc-llm\3rdparty\tvm\include\tvm/runtime/logging.h(571,9): warning C4068: unknown pragma 'GCC' [C:\repo\mlc-llm\build\mlc_cli_objs.vcxproj]
  mlc_cli_objs.vcxproj -> C:\repo\mlc-llm\build\mlc_cli_objs.dir\Debug\mlc_cli_objs.lib
  Building Custom Rule C:/repo/mlc-llm/CMakeLists.txt
  conv_templates.cc
C:\repo\mlc-llm\3rdparty\tvm\include\tvm/runtime/logging.h(353,9): warning C4068: unknown pragma 'disagnostic' [C:\repo\mlc-llm\build\mlc_llm_objs.vcxproj]
C:\repo\mlc-llm\3rdparty\tvm\include\tvm/runtime/logging.h(358,9): warning C4068: unknown pragma 'disagnostic' [C:\repo\mlc-llm\build\mlc_llm_objs.vcxproj]
C:\repo\mlc-llm\3rdparty\tvm\include\tvm/runtime/logging.h(563,9): warning C4068: unknown pragma 'GCC' [C:\repo\mlc-llm\build\mlc_llm_objs.vcxproj]
C:\repo\mlc-llm\3rdparty\tvm\include\tvm/runtime/logging.h(564,9): warning C4068: unknown pragma 'GCC' [C:\repo\mlc-llm\build\mlc_llm_objs.vcxproj]
C:\repo\mlc-llm\3rdparty\tvm\include\tvm/runtime/logging.h(571,9): warning C4068: unknown pragma 'GCC' [C:\repo\mlc-llm\build\mlc_llm_objs.vcxproj]
  conversation.cc
C:\repo\mlc-llm\3rdparty\tvm\include\tvm/runtime/logging.h(353,9): warning C4068: unknown pragma 'disagnostic' [C:\repo\mlc-llm\build\mlc_llm_objs.vcxproj]
C:\repo\mlc-llm\3rdparty\tvm\include\tvm/runtime/logging.h(358,9): warning C4068: unknown pragma 'disagnostic' [C:\repo\mlc-llm\build\mlc_llm_objs.vcxproj]
C:\repo\mlc-llm\3rdparty\tvm\include\tvm/runtime/logging.h(563,9): warning C4068: unknown pragma 'GCC' [C:\repo\mlc-llm\build\mlc_llm_objs.vcxproj]
C:\repo\mlc-llm\3rdparty\tvm\include\tvm/runtime/logging.h(564,9): warning C4068: unknown pragma 'GCC' [C:\repo\mlc-llm\build\mlc_llm_objs.vcxproj]
C:\repo\mlc-llm\3rdparty\tvm\include\tvm/runtime/logging.h(571,9): warning C4068: unknown pragma 'GCC' [C:\repo\mlc-llm\build\mlc_llm_objs.vcxproj]
  image_embed.cc
C:\repo\mlc-llm\3rdparty\tvm\include\tvm/runtime/logging.h(353,9): warning C4068: unknown pragma 'disagnostic' [C:\repo\mlc-llm\build\mlc_llm_objs.vcxproj]
C:\repo\mlc-llm\3rdparty\tvm\include\tvm/runtime/logging.h(358,9): warning C4068: unknown pragma 'disagnostic' [C:\repo\mlc-llm\build\mlc_llm_objs.vcxproj]
C:\repo\mlc-llm\3rdparty\tvm\include\tvm/runtime/logging.h(563,9): warning C4068: unknown pragma 'GCC' [C:\repo\mlc-llm\build\mlc_llm_objs.vcxproj]
C:\repo\mlc-llm\3rdparty\tvm\include\tvm/runtime/logging.h(564,9): warning C4068: unknown pragma 'GCC' [C:\repo\mlc-llm\build\mlc_llm_objs.vcxproj]
C:\repo\mlc-llm\3rdparty\tvm\include\tvm/runtime/logging.h(571,9): warning C4068: unknown pragma 'GCC' [C:\repo\mlc-llm\build\mlc_llm_objs.vcxproj]
  llm_chat.cc
C:\repo\mlc-llm\3rdparty\tvm\include\tvm/runtime/logging.h(353,9): warning C4068: unknown pragma 'disagnostic' [C:\repo\mlc-llm\build\mlc_llm_objs.vcxproj]
C:\repo\mlc-llm\3rdparty\tvm\include\tvm/runtime/logging.h(358,9): warning C4068: unknown pragma 'disagnostic' [C:\repo\mlc-llm\build\mlc_llm_objs.vcxproj]
C:\repo\mlc-llm\3rdparty\tvm\include\tvm/runtime/logging.h(563,9): warning C4068: unknown pragma 'GCC' [C:\repo\mlc-llm\build\mlc_llm_objs.vcxproj]
C:\repo\mlc-llm\3rdparty\tvm\include\tvm/runtime/logging.h(564,9): warning C4068: unknown pragma 'GCC' [C:\repo\mlc-llm\build\mlc_llm_objs.vcxproj]
C:\repo\mlc-llm\3rdparty\tvm\include\tvm/runtime/logging.h(571,9): warning C4068: unknown pragma 'GCC' [C:\repo\mlc-llm\build\mlc_llm_objs.vcxproj]
C:\repo\mlc-llm\cpp\llm_chat.cc(158,29): error C2039: 'ProcessSession': is not a member of 'tvm::runtime::Session' [C:\repo\mlc-llm\build\mlc_llm_objs.vcxproj]
C:\repo\mlc-llm\3rdparty\tvm\include\tvm/runtime/disco/session.h(240,7): message : see declaration of 'tvm::runtime::Session' [C:\repo\mlc-llm\build\mlc_llm_objs.vcxproj]
C:\repo\mlc-llm\cpp\llm_chat.cc(158,29): error C3861: 'ProcessSession': identifier not found [C:\repo\mlc-llm\build\mlc_llm_objs.vcxproj]
C:\repo\mlc-llm\cpp\llm_chat.cc(159,19): error C2039: 'InitCCL': is not a member of 'tvm::runtime::SessionObj' [C:\repo\mlc-llm\build\mlc_llm_objs.vcxproj]
C:\repo\mlc-llm\3rdparty\tvm\include\tvm/runtime/disco/session.h(165,7): message : see declaration of 'tvm::runtime::SessionObj' [C:\repo\mlc-llm\build\mlc_llm_objs.vcxproj]
  Generating Code...

Expected behavior

I can build MLC Runtime for Windows.

Environment

• Platform (e.g. WebGPU/Vulkan/IOS/Android/CUDA): CUDA
• Operating system (e.g. Ubuntu/Windows/MacOS/...): Windows 10 Pro
• Device (e.g. iPhone 12 Pro, PC+RTX 3090, ...): x64, NVIDIA GeForce RTX 2070 SUPER
• How you installed MLC-LLM (conda, source): No - I believe we don't need this, correct?
• How you installed TVM-Unity (pip, source): No - I believe we don't need this, correct?
• Python version (e.g. 3.10): 3.8.3
• GPU driver version (if applicable): 31.0.15.2756
• CUDA/cuDNN version (if applicable): cuda_11.1.relgpu_drvr455TC455_06.29190527_0
• TVM Unity Hash Tag (python -c "import tvm; print('\n'.join(f'{k}: {v}' for k, v in tvm.support.libinfo().items()))", applicable if you compile models):
• Any other relevant information: cmake version 3.27.5

Additional context

Thank you for any help!

[junrushao]

It's because the git submodule 3rdparty/tvm doesn't contain the latest commit of TVM. I updated it just now. Would you mind trying again?

[jparismorgan]

Hmm I just did /c/repo/mlc-llm git submodule update --recursive and then tried again, but got the same error. Would you mind sharing the commit that fixed it so that I can make sure I have it locally?

[junrushao]

Oh I mean I committed an update: 84a5e97

[jparismorgan]

Thanks, that worked!

[jparismorgan]

@junrushao actually I still do have an error, though it's different from the error above:

C:\repo\mlc-llm\3rdparty\tvm\src\runtime\disco\../../support/pipe.h(52,16): error C2440: 'static_cast': cannot convert from 'int64_t' to 'tvm::support::Pipe::PipeHandle' (compiling source file C:\repo\mlc-llm\3rdparty\tvm\src\runtime\disco\process_session.cc) [C:\repo\mlc-llm\build-windows\tvm\tvm_ 
runtime_objs.vcxproj]
C:\repo\mlc-llm\3rdparty\tvm\src\runtime\disco\../../support/pipe.h(52,43): message : Conversion from integral type to pointer type requires reinterpret_cast, C-style cast or function-style cast (compiling source file C:\repo\mlc-llm\3rdparty\tvm\src\runtime\disco\process_session.cc) [C:\repo\mlc-l 
lm\build-windows\tvm\tvm_runtime_objs.vcxproj]
C:\repo\mlc-llm\3rdparty\tvm\src\runtime\disco\../../support/pipe.h(67,1): error C2660: 'ReadFile': function does not take 4 arguments (compiling source file C:\repo\mlc-llm\3rdparty\tvm\src\runtime\disco\process_session.cc) [C:\repo\mlc-llm\build-windows\tvm\tvm_runtime_objs.vcxproj]
C:\Program Files (x86)\Windows Kits\10\Include\10.0.22621.0\um\fileapi.h(979,1): message : see declaration of 'ReadFile' (compiling source file C:\repo\mlc-llm\3rdparty\tvm\src\runtime\disco\process_session.cc) [C:\repo\mlc-llm\build-windows\tvm\tvm_runtime_objs.vcxproj]
C:\repo\mlc-llm\3rdparty\tvm\src\runtime\disco\../../support/pipe.h(86,1): error C2660: 'WriteFile': function does not take 4 arguments (compiling source file C:\repo\mlc-llm\3rdparty\tvm\src\runtime\disco\process_session.cc) [C:\repo\mlc-llm\build-windows\tvm\tvm_runtime_objs.vcxproj]
C:\Program Files (x86)\Windows Kits\10\Include\10.0.22621.0\um\fileapi.h(1139,1): message : see declaration of 'WriteFile' (compiling source file C:\repo\mlc-llm\3rdparty\tvm\src\runtime\disco\process_session.cc) [C:\repo\mlc-llm\build-windows\tvm\tvm_runtime_objs.vcxproj]

Do you also get this?

[junrushao]

I guess we somehow misused some windows API for pipe handling

[junrushao]

let me double check and get back to your asap

[junrushao]

Meanwhile, you may comment out those lines causing compilation error and it should be fine for single-gpu usecases

[junrushao]

Could you do me a favor trying the following changes on 3rdparty/tvm/src/support/pipe.h?

Line 52:

- explicit Pipe(int64_t handle) : handle_(static_cast<PipeHandle>(handle)) {}
+ explicit Pipe(int64_t handle) : handle_(reinterpret_cast<PipeHandle>(handle)) {}

Line 67:

-    ICHECK(ReadFile(handle_, static_cast<TCHAR*>(ptr), &nread, nullptr))
+    ICHECK(ReadFile(handle_, static_cast<TCHAR*>(ptr), size, &nread, nullptr))

Line 86:

-    ICHECK(WriteFile(handle_, static_cast<const TCHAR*>(ptr), &nwrite, nullptr) &&
+    ICHECK(WriteFile(handle_, static_cast<const TCHAR*>(ptr), size, &nwrite, nullptr) &&

[junrushao]

I submitted a fix for this windows compilation error: mlc-ai/relax@8f5954d. There's a Windows CI on it, meaning if it compiles on CI it should work on your machine. Let's see how it goes