Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Can't launch QwQ-32B with 'Parsing Reasoning Content' switch on. #3023

Open
1 of 3 tasks
tacnaci opened this issue Mar 10, 2025 · 1 comment · May be fixed by #3024
Open
1 of 3 tasks

Can't launch QwQ-32B with 'Parsing Reasoning Content' switch on. #3023

tacnaci opened this issue Mar 10, 2025 · 1 comment · May be fixed by #3024
Labels
Milestone

Comments

@tacnaci
Copy link

tacnaci commented Mar 10, 2025

System Info / 系統信息

cuda: 12.4, vllm 0.7.3, python 3.11.11, ubuntu 22.04.5

Running Xinference with Docker? / 是否使用 Docker 运行 Xinfernece?

  • docker / docker
  • pip install / 通过 pip install 安装
  • installation from source / 从源码安装

Version info / 版本信息

1.3.1

The command used to start Xinference / 用以启动 xinference 的命令

xinference-local --host 0.0.0.0 --port 9997 --auth-config auth_config.json

Reproduction / 复现过程

Image

Image

It seems that no reason_parser has been registered for QwQ, causing a startup error.

Expected behavior / 期待表现

When the 'Parsing Reasoning Content' switch is turned on, QwQ-32B can start normally using vllm.

@XprobeBot XprobeBot added the gpu label Mar 10, 2025
@XprobeBot XprobeBot added this to the v1.x milestone Mar 10, 2025
@qinxuye
Copy link
Contributor

qinxuye commented Mar 10, 2025

This is a known issue and will be fixed soon. @amumu96

@qinxuye qinxuye linked a pull request Mar 10, 2025 that will close this issue
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants