Skip to content

Conversation

@Ubospica
Copy link
Collaborator

@Ubospica Ubospica commented Mar 15, 2024

Currently the ABI of tvm::runtime::Map will change if TVM_LOG_DEBUG is enabled or not:

// In MapNode::iterator
#if TVM_LOG_DEBUG
    uint64_t state_marker;
    ...
#else
    ...
#endif  // TVM_LOG_DEBUG

When TVM_LOG_DEBUG is inconsistent between different libraries (e.g. in TVM and in MLC-LLM), the inconsistency in ABI will lead to an invalid read/write.

This PR provides a workaround by changing the usage of tvm::runtime::Map into std::unordered_map.

Thanks to @tqchen @MasterJH5574


model_path = "dist/Llama-2-7b-chat-hf-q0f16-MLC"
model_lib_path = "dist/Llama-2-7b-chat-hf-q0f16-MLC/Llama-2-7b-chat-hf-q0f16-MLC-cuda.so"

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we revert the changes in this file? Given this test is not rather standard (like used in CI), I personally prefer a finer control of the paths in each function.

@tqchen
Copy link
Contributor

tqchen commented Mar 16, 2024

Closing as it is being fixed in upstream

@tqchen tqchen closed this Mar 16, 2024
@Ubospica
Copy link
Collaborator Author

Plz check out apache/tvm#16728

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants