[v0.10.0rc1] FAQ / Feedback | 问题/反馈

[MengqingCao]

Release checklist for v0.10.0rc1: #2210

Please use doc: https://vllm-ascend.readthedocs.io/

请使用 https://vllm-ascend.readthedocs.io/ 安装

[MaoJianwei]

什么时候能出官方的0.10.0容器镜像？

[wangxiyuan]

@MaoJianwei today or tomorrow. It's under testing now.

[MaoJianwei]

checklist还有好多没完成

[miaomiaoguaiOvO]

#!/bin/sh

this obtained through ifconfig

nic_name is the network interface name corresponding to local_ip

nic_name="xxxx"
local_ip="xxxx"

export HCCL_IF_IP=$local_ip
export GLOO_SOCKET_IFNAME=$nic_name
export TP_SOCKET_IFNAME=$nic_name
export HCCL_SOCKET_IFNAME=$nic_name
export OMP_PROC_BIND=false
export OMP_NUM_THREADS=100
export HCCL_BUFFSIZE=1024

The w8a8 weight can obtained from https://www.modelscope.cn/models/vllm-ascend/DeepSeek-V3-W8A8

If you want to the quantization manually, please refer to https://vllm-ascend.readthedocs.io/en/latest/user_guide/feature_guide/quantization.html

vllm serve /root/.cache/ds_v3
--host 0.0.0.0
--port 8004
--data-parallel-size 4
--data-parallel-size-local 2
--data-parallel-address $local_ip
--data-parallel-rpc-port 13389
--tensor-parallel-size 4
--seed 1024
--served-model-name deepseek_v3
--enable-expert-parallel
--max-num-seqs 16
--max-model-len 32768
--quantization ascend
--max-num-batched-tokens 4096
--trust-remote-code
--no-enable-prefix-caching
--gpu-memory-utilization 0.9
--additional-config '{"ascend_scheduler_config":{"enabled":true},"torchair_graph_config":{"enabled":true}}'

使用上面的命令进行推理，上下文超过--max-num-batched-tokens设置的值后就不再返回内容了。日志中有：WARNING 08-11 02:09:39 [scheduler.py:150] Input prompt (5894 tokens) is too long and exceeds limit of 4096
，接口也不会返回数据一直卡住

[q963377805-blip]

还有微信交流群吗，链接找不到了

[VittorioYan]

请问镜像TAG里面的a3是啥意思？300I DUO是下载这个镜像吗

[MengqingCao]

请问镜像TAG里面的a3是啥意思？300I DUO是下载这个镜像吗

Nope, actually we have no 300I DUO images released

[VittorioYan]

请问镜像TAG里面的a3是啥意思？300I DUO是下载这个镜像吗

Nope, actually we have no 300I DUO images released

所以我用这个卡的话，就只能自己使用物理机安装环境了是吗？
谢谢

[MengqingCao]

所以我用这个卡的话，就只能自己使用物理机安装环境了是吗？ 谢谢

yes, or you can install vllm and vllm-ascend in your own image, plz refer to https://vllm-ascend.readthedocs.io/en/latest/installation.html#setup-vllm-and-vllm-ascend

[VittorioYan]

所以我用这个卡的话，就只能自己使用物理机安装环境了是吗？ 谢谢

yes, or you can install vllm and vllm-ascend in your own image, plz refer to https://vllm-ascend.readthedocs.io/en/latest/installation.html#setup-vllm-and-vllm-ascend

我需要部署Qwen3-235B-A30B ，在kylin系统/arm64下使用这种方法安装vllm-ascend是可以部署的吗？
谢谢