Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

xinference 使用 qwen 模型,出现 PydanticSchemaGenerationError 错误 #2795

Open
1 of 3 tasks
luageng opened this issue Feb 1, 2025 · 0 comments
Open
1 of 3 tasks
Labels
Milestone

Comments

@luageng
Copy link

luageng commented Feb 1, 2025

System Info / 系統信息

Redhat Linux 8.4 x64 系统,docker-ce-26.1.3, 使用 CPU。
模型 qwen1.5-chart, qwen2.5-instruct
参数:llama.cpp, ggufv2, 0_5, q4_k_m, N GPU Layers

Running Xinference with Docker? / 是否使用 Docker 运行 Xinfernece?

  • docker / docker
  • pip install / 通过 pip install 安装
  • installation from source / 从源码安装

Version info / 版本信息

xprobe/xinference:v1.1.1-cpu
xprobe/xinference:v1.2.0-cpu

The command used to start Xinference / 用以启动 xinference 的命令

docker run --name xinference -d
-p 9997:9997
-e XINFERENCE_MODEL_SRC=modelscope
-e XINFERENCE_HOME=/root/.xinference
-v /home/opt/models/xinference:/root/.xinference
xprobe/xinference:v1.1.1-cpu
xinference-local -H 0.0.0.0 --log-level debug

Reproduction / 复现过程

1、选择 Running Models,在 qwen2.5-instruct 模型上点击 Launch WEB UI,然后输入消息,点击 submit 。
2、网页出现,could not parse server SyntaxError Unexpected token 'l', "Internal S" is not valid json 错误。
3、xinference 容器日志出现 pydantic.errors.PydanticSchemaGenerationError: Unable to generate pydantic-core schema for <class 'starlette.requests.Request'>. Set arbitrary_types_allowed=True in the model_config to ignore this error or implement __get_pydantic_core_schema__ on your type to fully support it. 错误。

Expected behavior / 期待表现

怎么避免这种错误,正常使用 qwen 模型。

@XprobeBot XprobeBot added the gpu label Feb 1, 2025
@XprobeBot XprobeBot added this to the v1.x milestone Feb 1, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

2 participants