Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

请问下,rknn-llm可以指定某个npu核进行推理么? #215

Open
pxrgod opened this issue Mar 7, 2025 · 4 comments
Open

请问下,rknn-llm可以指定某个npu核进行推理么? #215

pxrgod opened this issue Mar 7, 2025 · 4 comments

Comments

@pxrgod
Copy link

pxrgod commented Mar 7, 2025

如题

@waydong
Copy link
Collaborator

waydong commented Mar 7, 2025

你好,目前只能配置单核,推理时自动匹配一个核心进行推理。

@wohaiaini
Copy link

我监控NPU负载的时候,发现两个core利用率都是70%,不是都用了?

@waydong
Copy link
Collaborator

waydong commented Mar 7, 2025

我监控NPU负载的时候,发现两个core利用率都是70%,不是都用了?

rk3576,如果转模型时候num_npu_core设置为2,就会调用两个核心,如果设置为1,就会自动匹配一个核心计算。

@fungizhang
Copy link

rk3588,批量推理图片,npu使用率情况大概如下:
步骤1,当读取1张图片时:
core1 99% core2 0% core3 0%
步骤2,当这张图片进行模型推理时:
core1 40% core2 40% core3 20%

请问如何进一步提升npu使用率,尝试过修改src/main.cpp并行3张图片推理,步骤1npu满载了,但是步骤2发生段错误

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

4 participants