[Bug] Could not convert TVM object of type runtime.Closure to a string.

[scottorly]

🐛 Bug

To Reproduce

Steps to reproduce the behavior:

Calling chat.reload(self.modelLib, modelPath: modelPath, appConfigJson: "")

produces the following crash:

Check failed: (IsObjectRef<tvm::runtime::String>()) is false: Could not convert TVM object of type runtime.Closure to a string.
Stack trace:
0x000000010009ecc4 tvm::runtime::detail::LogFatal::Entry::Finalize() + 68
0x000000010009ec80 tvm::runtime::detail::LogFatal::Entry::Finalize() + 0
0x000000010009dcf4 __clang_call_terminate + 0
0x00000001000acbfc tvm::runtime::TVMArgValue::operator std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>() const + 716
0x00000001000ac318 tvm::runtime::PackedFuncValueConverter<tvm::runtime::String>::From(tvm::runtime::TVMArgValue const&) + 104
0x00000001000b5928 mlc::llm::LLMChatModule::GetFunction(tvm::runtime::String const&, tvm::runtime::ObjectPtr<tvm::runtime::Object> const&)::'lambda'(tvm::runtime::TVMArgs, tvm::runtime::TVMRetValue*)::operator()(tvm::runtime::TVMArgs, tvm::runtime::TVMRetValue*) const + 724
0x00000001000b5648 tvm::runtime::PackedFuncObj::Extractor<tvm::runtime::PackedFuncSubObj<mlc::llm::LLMChatModule::GetFunction(tvm::runtime::String const&, tvm::runtime::ObjectPtr<tvm::runtime::Object> const&)::'lambda'(tvm::runtime::TVMArgs, tvm::runtime::TVMRetValue*)>>::Call(tvm::runtime::PackedFuncObj const*, tvm::runtime::TVMArgs, tvm::runtime::TVMRetValue*) + 40
tvm::runtime::TVMRetValue tvm::runtime::PackedFunc::operator()<tvm::runtime::Module&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>&>(tvm::runtime::Module&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>&) const + 260
-[ChatModule reload:modelPath:appConfigJson:] + 408

Environment

• Platform (e.g. WebGPU/Vulkan/IOS/Android/CUDA):iOS
• Operating system (e.g. Ubuntu/Windows/MacOS/...):MacOS
• Device (e.g. iPhone 12 Pro, PC+RTX 3090, ...) iPhone14 Pro
• How you installed MLC-LLM (conda, source): source
• How you installed TVM-Unity (pip, source): pip
• Python version (e.g. 3.10): 3.8
• GPU driver version (if applicable):
• CUDA/cuDNN version (if applicable):
• TVM Unity Hash Tag (python -c "import tvm; print('\n'.join(f'{k}: {v}' for k, v in tvm.support.libinfo().items()))", applicable if you compile models):
• Any other relevant information:

Additional context

[junrushao]

It's a bug I introduced in #867. Will have a fix later this week

[changun]

Hi @junrushao ,

Thanks for the confirmation. Is there any way we can get the previous version of wheel file? The latest file on the mlc/wheels are not working anymore as mentioned by @scottorly.

These two files are the ones we used that were working.
Downloading https://github.com/mlc-ai/package/releases/download/v0.9.dev0/mlc_ai_nightly_cu121-0.12.dev1569-cp311-cp311-manylinux_2_28_x86_64.whl (98.4 MB)
Downloading https://github.com/mlc-ai/package/releases/download/v0.9.dev0/mlc_chat_nightly_cu121-0.1.dev413-cp311-cp311-manylinux_2_28_x86_64.whl (21.1 MB)

[leavelet]

Encountered the exact problem on both ROCm and CUDA platforms

[sudeepag]

Not sure about the wheels, but if you are compiling locally, the following worked for me as a temporary workaround:

In mlc-ai/relax,
git reset --hard 6e2ca87cf8fdce10227c6b4349fe77a3cda01537

In mlc-ai/mlc-llm
git reset --hard 48602de00f7d02dc665b8f31bd4e5bcd60d9c0e6

[MasterJH5574]

Hi @scottorly @changun @leavelet, we just updated the wheels with the latest fix. Could you please update the pip package with the same pip install command in https://mlc.ai/package/ and retry?

[ali-libertify]

Hello,

I am still experiencing the issue:

line 103, in lifespan chat_mod = ChatModule( File "/usr/local/lib/python3.10/dist-packages/mlc_chat/chat_module.py", line 599, in __init__ self._reload(self.lib_path, self.model_path, user_chat_config_json_str) File "/usr/local/lib/python3.10/dist-packages/mlc_chat/chat_module.py", line 780, in _reload self._reload_func(lib, model_path, app_config_json) File "tvm/_ffi/_cython/./packed_func.pxi", line 332, in tvm._ffi._cy3.core.PackedFuncBase.__call__ File "tvm/_ffi/_cython/./packed_func.pxi", line 263, in tvm._ffi._cy3.core.FuncCall File "tvm/_ffi/_cython/./packed_func.pxi", line 252, in tvm._ffi._cy3.core.FuncCall3 File "tvm/_ffi/_cython/./base.pxi", line 182, in tvm._ffi._cy3.core.CHECK_CALL File "/usr/local/lib/python3.10/dist-packages/tvm/_ffi/base.py", line 476, in raise_last_ffi_error raise py_err File "/workspace/mlc-llm/cpp/llm_chat.cc", line 1164, in mlc::llm::LLMChatModule::GetFunction(tvm::runtime::String const&, tvm::runtime::ObjectPtr<tvm::runtime::Object> const&)::{lambda(tvm::runtime::TVMArgs, tvm::runtime::TVMRetValue*)#1}::operator()(tvm::runtime::TVMArgs, tvm::runtime::TVMRetValue*) const tvm.error.InternalError: Traceback (most recent call last): 3: mlc::llm::LLMChatModule::GetFunction(tvm::runtime::String const&, tvm::runtime::ObjectPtr<tvm::runtime::Object> const&)::{lambda(tvm::runtime::TVMArgs, tvm::runtime::TVMRetValue*)#1}::operator()(tvm::runtime::TVMArgs, tvm::runtime::TVMRetValue*) const at /workspace/mlc-llm/cpp/llm_chat.cc:1164 2: tvm::runtime::PackedFuncValueConverter<tvm::runtime::String>::From(tvm::runtime::TVMArgValue const&) 1: tvm::runtime::TVMArgValue::operator std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >() const 0: _ZN3tvm7runtime6deta File "/workspace/tvm/include/tvm/runtime/packed_func.h", line 683 InternalError: Check failed: (IsObjectRef<tvm::runtime::String>()) is false: Could not convert TVM object of type runtime.Closure to a string.

I am using the pip install --pre --force-reinstall mlc-ai-nightly-cu118 mlc-chat-nightly-cu118 -f https://mlc.ai/wheels command

[robert-b-hatch]

I'm also still seeing the issue using the above pip install command. However, I notice that in https://mlc.ai/wheels and when I run that command, it installs chat dev419, but over here the latest mlc-chat is dev421 https://github.com/mlc-ai/package/releases/tag/v0.9.dev0 ?

[changun]

We can retry shortly too.

Also, just a suggestion: I am wondering if it is possible to keep the older versions in the wheel . It will be pretty beneficial to the stability of our CI/CD pipeline if we can pin to a version that we know is working. We probably can do so if we build from the source, but it will be much easier to be based on the pre-built wheel.

I understand it will need more storage and you are definitely all super busy improving the core MLC, so no worry if it is not a priority. Thanks again for the fast response!

[robert-b-hatch]

I'm getting a new error with manually updating to dev421
$ pip uninstall mlc-chat-nightly-rocm
$ pip install --pre mlc-chat-nightly-rocm -f https://github.com/mlc-ai/package/releases/download/v0.9.dev0/mlc_chat_nightly_rocm-0.1.dev421-cp310-cp310-manylinux_2_28_x86_64.whl
$ python sample_mlc_chat.py
Edit: then see issue 913

Will try fix for 913 and test again.

[robert-b-hatch]

Yeah, with the below commands, it works for me(*):

pip install --pre --force-reinstall mlc-ai-nightly-cu118 mlc-chat-nightly-cu118 -f https://mlc.ai/wheels
pip uninstall mlc-chat-nightly-rocm
pip install --pre mlc-chat-nightly-rocm -f https://github.com/mlc-ai/package/releases/download/v0.9.dev0/mlc_chat_nightly_rocm-0.1.dev421-cp310-cp310-manylinux_2_28_x86_64.whl

• Then I had to update my prebuilt model weights to fix issue 913.
• Then I still hit issue 727, but that should be unrelated and expected for my GPU. I won't be able to use the prebuilt weights.

[junrushao]

Commit 330a726 should fix this issue

[scottorly]

Still seeing the issue on mac/ios using 421

[junrushao]

#421 is likely not relevant to this issue. @scottorly do you mind opening a new thread? Thanks!