[Bug] problem with reshaping the input new shape

[Vinaysukhesh98]

mlc_chat convert_weight --model-type ${MODEL_TYPE} ./dist/models/${MODEL_NAME}/ --quantization $QUANTIZATION -o dist/$MODEL_NAME/
[2024-02-15 11:19:53] INFO auto_config.py:115: Found model configuration: dist/models/MiniCPM-V/config.json
Error Message:

raise_last_ffi_error
raise py_err
tvm._ffi.base.TVMError: Traceback (most recent call last):
4: _ZN3tvm7runtime13PackedFun
3: tvm::runtime::TypedPackedFunc<tvm::RelayExpr (tvm::RelayExpr, tvm::runtime::ObjectRef)>::AssignTypedLambda<tvm::RelayExpr ()(tvm::RelayExpr, tvm::runtime::ObjectRef)>(tvm::RelayExpr ()(tvm::RelayExpr, tvm::runtime::ObjectRef), std::__cxx11::basic_string<char, std::char_traits, std::allocator >)::{lambda(tvm::runtime::TVMArgs const&, tvm::runtime::TVMRetValue*)#1}::operator()(tvm::runtime::TVMArgs const&, tvm::runtime::TVMRetValue*) const
2: tvm::relax::reshape(tvm::RelayExpr, tvm::runtime::ObjectRef)
1: tvm::relax::ConvertNewShapeToExpr(tvm::RelayExpr const&, tvm::runtime::ObjectRef const&)
0: _ZN3tvm7runtime6deta
File "/workspace/tvm/src/relax/op/tensor/manipulate.cc", line 675
TVMError: Check failed: (_len != nullptr) is false: Reshape only expects the input new shape to be either an Expr or an Array of PrimExprs. However, the given new shape is [[1, 1152, 1, 1]]
[11:19:55] /workspace/tvm/src/relax/ir/block_builder.cc:65: Warning: BlockBuilder destroyed with remaining blocks!

[shreygupta2809]

Could you please provide the exact sequence of commands including values for MODEL_TYPE, QUANTIZATION, and the command using which you downloaded the exact hugging face model.

[Vinaysukhesh98]

MODEL_NAME=MiniCPM-V
QUANTIZATION=q4f16_1
MODEL_TYPE=minicpm_v
mlc_chat convert_weight --model-type ${MODEL_TYPE} ./dist/models/${MODEL_NAME}/ --quantization $QUANTIZATION -o dist/$MODEL_NAME/

[MasterJH5574]

Hey @Vinaysukhesh98 thanks for following up. By reading the error message, I suppose the error should be something like:

It is expected to call reshape in the way of reshape(1, 1152, 1, 1), while reshape is called as reshape((1, 1152, 1, 1)). In this case it might produce this error.

So if you can help share the line where the reshape is called in Python, that would be very helpful. Thanks!

[Vinaysukhesh98]

Hey @Vinaysukhesh98 thanks for following up. By reading the error message, I suppose the error should be something like:

It is expected to call reshape in the way of reshape(1, 1152, 1, 1), while reshape is called as reshape((1, 1152, 1, 1)). In this case it might produce this error.

So if you can help share the line where the reshape is called in Python, that would be very helpful. Thanks!

hey @MasterJH5574

def forward(self, x: relax.Expr) -> relax.Var:
from tvm.relax.op import ( # pylint: disable=import-outside-toplevel
reshape,
take,
)

    ndim = x.struct_info.ndim
    if ndim == 1:
        return nn.emit(take(self.weight, x, axis=0))
    else:
        x_shape = x.struct_info.shape.values
        emb_size = self.weight.struct_info.shape.values[-1]
        x = nn.emit(reshape(x, shape=[-1]))
        embedding = nn.emit(take(self.weight, x, axis=0))
        return nn.emit(reshape(embedding, [*x_shape, emb_size]))

[Vinaysukhesh98]

i'm getting below error if i tried with different model:
Traceback (most recent call last):
File "/.local/bin/mlc_chat", line 33, in
sys.exit(load_entry_point('mlc-chat', 'console_scripts', 'mlc_chat')())
File "Documents/mlc-MiniCPM/python/mlc_chat/main.py", line 28, in main
cli.main(sys.argv[2:])
File "mlc-MiniCPM/python/mlc_chat/cli/convert_weight.py", line 87, in main
convert_weight(
File "Documents/mlc-MiniCPM/python/mlc_chat/interface/convert_weight.py", line 156, in convert_weight
_convert_args(args)
File "/Documents/mlc-MiniCPM/python/mlc_chat/interface/convert_weight.py", line 107, in _convert_args
for name, param in LOADER[args.source_format](
File "/Documents/mlc-MiniCPM/python/mlc_chat/loader/huggingface_loader.py", line 97, in init
check_parameter_usage(extern_param_map, set(self.torch_to_path.keys()))
File "mlc-MiniCPM/python/mlc_chat/loader/utils.py", line 32, in check_parameter_usage
raise ValueError(
ValueError: The following extern parameters do not exist in the weight files:

[shinerdeng]

the same problem

[shreygupta2809]

MODEL_NAME=MiniCPM-V QUANTIZATION=q4f16_1 MODEL_TYPE=minicpm_v mlc_chat convert_weight --model-type ${MODEL_TYPE} ./dist/models/${MODEL_NAME}/ --quantization $QUANTIZATION -o dist/$MODEL_NAME/

MODEL_TYPE=minicpm_v is not supported. Only the models in this list are currently supported https://github.com/mlc-ai/mlc-llm/blob/main/python/mlc_chat/model/model.py#L65 to run using MLC.
Running the same command on my system gives the below error Code Link where error is raised:

$ mlc_chat convert_weight ./dist/MiniCPM-V/ --model-type minicpm_v --quantization q4f16_1 -o ./dist/MiniCPM-V-q4f16_1-MLC/

`ValueError: Unknown model type: minicpmv. Available ones: ['llama', 'mistral', 'gpt2', 'mixtral', 'gpt_neox', 'gpt_bigcode', 'phi-msft', 'phi', 'qwen', 'qwen2', 'stablelm_epoch', 'baichuan']`

Could you try pulling the latest code from GitHub and trying again once?

[Vinaysukhesh98]

yes but there is another repo :
https://huggingface.co/openbmb/MiniCPM-V

could you check above link which replicates the mlc_llm

[shreygupta2809]

yes but there is another repo :
https://huggingface.co/openbmb/MiniCPM-V

could you check above link which replicates the mlc_llm

I used this model only. In its config.json it's model_type=minicpmv which is not valid. You can set model_type flag to auto which would allow automatic inference of model type from the config file which would be minicpmv, which should be an error.