Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

transformers bert 模型 onnx 格式推理正常，转为mnn 之后推理值固定为index =1 #2283

Closed
dshwei opened this issue Mar 14, 2023 · 16 comments
Closed

transformers bert 模型 onnx 格式推理正常，转为mnn 之后推理值固定为index =1 #2283

dshwei opened this issue Mar 14, 2023 · 16 comments
Assignees
@jxt1234
Labels
User The user ask question about how to use. Or don't use MNN correctly and cause bug.

Comments

@dshwei
Copy link

dshwei commented Mar 14, 2023 •
edited
Loading

平台:

Processor 12th Gen Intel(R) Core(TM) i7-12700 2.10 GHz
Installed RAM 16.0 GB (31.7 GB usable)
System type Linux bxt301 4.14.48 #1 SMP Wed Jul 14 23:34:49 CST 2021 x86_64 x86_64 x86_64 GNU/Linux

Github版本:

Github Version:

2.4.0

inputs = interpreter.getSessionInputAll(session)
print("【before resize inputs 】",inputs)

# 输入 resizeTensor
interpreter.resizeTensor(inputs["attention_mask"], (1,128))
interpreter.resizeTensor(inputs["token_type_ids"], (1,128))
interpreter.resizeTensor(inputs["input_ids"], (1,128))

# 重新计算形状分配内存
interpreter.resizeSession(session)

print("after resize inputs 】",inputs)
@jxt1234
Copy link
Collaborator

jxt1234 commented Mar 14, 2023

?? 报什么错呢?

@dshwei
Copy link
Author

dshwei commented Mar 14, 2023 •
edited
Loading

没有报错，推理结果一直是固定在了index =1的标签

没有报错，推理结果一直是固定在了index =1的标签
onnx 模型怎么传给你

@jxt1234
Copy link
Collaborator

jxt1234 commented Mar 14, 2023

完整代码是?

@dshwei
Copy link
Author

dshwei commented Mar 14, 2023 •
edited
Loading

输入数据
{'input_ids': array([[ 101, 1155, 2548, 1290, 3300, 3766, 3300, 4696, 3633, 2692, 721,
677, 2247, 754, 5632, 2346, 712, 4028, 4638, 4510, 2512, 8043,
102, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0]]), 'token_type_ids': array([[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]]), 'attention_mask': array([[1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1,
1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]])}

{'input_ids': array([[ 101, 3946, 3419, 4802, 6371, 4906, 3172, 1147, 2209, 2225, 1728,
839, 828, 2773, 1288, 2399, 6890, 2742, 3187, 5357, 686, 4518,
3344, 102, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0]]), 'token_type_ids': array([[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]]), 'attention_mask': array([[1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1,
1, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]])}

上面的代码不完整：完整代码如下：
import MNN
import numpy as np
from transformers import AutoTokenizer

MNN_GPU_TUNING_FAST = 1 << 4
MNN_GPU_MEMORY_BUFFER = 1 << 6
MNN_GPU_MEMORY_IMAGE = 1 << 7

def mode_run(model_path):

np.random.seed(0)
interpreter = MNN.Interpreter(model_path)

interpreter.setSessionMode(8)
interpreter.setSessionHint(0, 20)

config = {}
config['backend']   = "OPENCL"
config['precision'] = "high"
config['numThread'] = MNN_GPU_TUNING_FAST | MNN_GPU_MEMORY_BUFFER # Buffer Mode
session = interpreter.createSession(config)

inputs = interpreter.getSessionInputAll(session)
print("【before resize inputs 】",inputs)

# 输入 resizeTensor
interpreter.resizeTensor(inputs["attention_mask"], (1,128))
interpreter.resizeTensor(inputs["token_type_ids"], (1,128))
interpreter.resizeTensor(inputs["input_ids"], (1,128))

# 重新计算形状分配内存
interpreter.resizeSession(session)

print("after resize inputs 】",inputs)

tokenizer = AutoTokenizer.from_pretrained("bert_pretrain_model")
token_inputs = tokenizer("刘德华有没有真正意义上属于自己主演的电影？", return_tensors="np",max_length=128,truncation=True,padding="max_length")
print(token_inputs)


attention_mask_tensor = MNN.Tensor(inputs["attention_mask"].getShape(), MNN.Halide_Type_Int,token_inputs.get("attention_mask").astype(np.int32), MNN.Tensor_DimensionType_Caffe)
inputs["attention_mask"].copyFrom(attention_mask_tensor)


token_type_ids_tensor = MNN.Tensor(inputs["token_type_ids"].getShape(), MNN.Halide_Type_Int,token_inputs.get("attention_mask").astype(np.int32), MNN.Tensor_DimensionType_Caffe)
inputs["token_type_ids"].copyFrom(token_type_ids_tensor)


input_ids_tensor = MNN.Tensor(inputs["input_ids"].getShape(), MNN.Halide_Type_Int,token_inputs.get("attention_mask").astype(np.int32), MNN.Tensor_DimensionType_Caffe)
inputs["input_ids"].copyFrom(input_ids_tensor)

tmp_tensor = np.zeros((1,15),dtype=np.float32)
interpreter.runSession(session)

outputs = interpreter.getSessionOutput(session)

print("【outputs shape】",outputs.getShape())

out_tensor = MNN.Tensor(outputs.getShape(), MNN.Halide_Type_Float, tmp_tensor, MNN.Tensor_DimensionType_Caffe)

outputs.copyToHostTensor(out_tensor)

data = out_tensor.getData()
print("【out_tensor 】",data)
index  = np.argmax(data)
print("【index 】",index)

if name == "main":
model_path = "bert_cls.mnn"
mode_run(model_path)

@dshwei
Copy link
Author

dshwei commented Mar 14, 2023

完整代码是?

onnx 模型怎么传给你

@jxt1234
Copy link
Collaborator

jxt1234 commented Mar 14, 2023

google 云盘，或者发我邮箱

@dshwei
Copy link
Author

dshwei commented Mar 14, 2023

或者发我邮箱

邮箱吧
你的邮箱发我一下

@jxt1234
Copy link
Collaborator

jxt1234 commented Mar 14, 2023

[email protected]

@dshwei
Copy link
Author

dshwei commented Mar 14, 2023

[email protected]

已发送

@jxt1234
Copy link
Collaborator

jxt1234 commented Mar 17, 2023

你调用 onnxruntime 推理 onnx 模型的代码也发一下吧

@jxt1234
Copy link
Collaborator

jxt1234 commented Mar 17, 2023

另外，把 onnx 模型发我邮箱，mnn 文件发我没什么用

@dshwei
Copy link
Author

dshwei commented Mar 20, 2023

两个onnx 文件都发给你了，一个固定shape输入，一个是动态shape 输入

@jxt1234
Copy link
Collaborator

jxt1234 commented Mar 23, 2023

调用 onnxruntime 推理 onnx 模型的代码呢？需要确保下 onnx 本身是可用的

@jxt1234
Copy link
Collaborator

jxt1234 commented Mar 23, 2023

demo 里面代码这一句执行不了，没有 bert_pretrain_model 这个目录
AutoTokenizer.from_pretrained("bert_pretrain_model")

@LawGoswell
Copy link

@dshwei 你代码里的token_type_ids_tensor和input_ids_tensor好像写错了，按现在的写法这两个值都是token_inputs["attention_mask"]？

@dshwei
Copy link
Author

dshwei commented Mar 24, 2023

问题已解决，是这里的问题，谢谢，

@jxt1234 jxt1234 added User The user ask question about how to use. Or don't use MNN correctly and cause bug. and removed Converter labels Mar 24, 2023
@jxt1234 jxt1234 closed this as completed Mar 24, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@jxt1234 jxt1234

Labels
User The user ask question about how to use. Or don't use MNN correctly and cause bug.
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
4 participants
@jxt1234 @wangzhaode @LawGoswell @dshwei