Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

执行./build/bin/main -m chatglm-ggml.bin 卡住 #182

Open
JPChen2000 opened this issue Nov 9, 2023 · 0 comments
Open

执行./build/bin/main -m chatglm-ggml.bin 卡住 #182

JPChen2000 opened this issue Nov 9, 2023 · 0 comments

Comments

@JPChen2000
Copy link

JPChen2000 commented Nov 9, 2023 •
edited
Loading

模型：chatGLM3-6b-22k
模型量化： python3 chatglm_cpp/convert.py -i ZhipuAI/chatglm3-6b-32k -t q4_0 -o chatglm-ggml.bin
环境：WSL2 CUDA11.8 RTX4070
编译：CUBLAS=ON

求助：执行./build /bin/main -m chatglm-ggml.bin -p 过程中有概率出现卡住的情况，文字输出不完整，但是GPU还在跑，程序无法退出，nvidia-smi被阻塞，此时通过kill 能杀掉进程，但是无法释放占用的GPU内存，
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@JPChen2000