Bump exllamav2 to 0.0.4 and use pre-built wheels#4095
Conversation
| https://github.com/PanQiWei/AutoGPTQ/releases/download/v0.4.2/auto_gptq-0.4.2+cu117-cp310-cp310-linux_x86_64.whl; platform_system == "Linux" and platform_machine == "x86_64" | ||
| https://github.com/jllllll/exllama/releases/download/0.0.17/exllama-0.0.17+cu117-cp310-cp310-win_amd64.whl; platform_system == "Windows" | ||
| https://github.com/jllllll/exllama/releases/download/0.0.17/exllama-0.0.17+cu117-cp310-cp310-linux_x86_64.whl; platform_system == "Linux" and platform_machine == "x86_64" | ||
| https://github.com/turboderp/exllamav2/releases/download/v0.0.4/exllamav2-0.0.4+cu117-cp310-cp310-win_amd64.whl; platform_system == "Windows" |
There was a problem hiding this comment.
Out of curiosity, why is a and platform_machine == "x86_64" not needed here? Also, can this line and line 11 apply simultaneously?
There was a problem hiding this comment.
Glad you caught that. I forgot that Windows uses amd64 instead of x86_64 for some reason.
This line will need and platform_machine == "amd64" and line 11 will need and platform_machine != "amd64"
This will also need to be applied to requirements_noavx2.txt.
That said, I think the wheel may take precedence as it is specified later in the file. I'm testing that now.
Edit: Just tested it and, fortunately, it does use the wheel. May need the conditions anyway just to ensure it doesn't stop working in the future.
It seems to prefer the wheel simply because it sees 0.0.4+cu117 as a newer version to 0.0.4.
|
This will make things easier for many people. |
2 participants
Checklist:
turboderp has added pre-compiled wheels for exllamav2 0.0.4. This PR adds them to the appropriate requirements.txt while also specifying conditions for the PyPI distribution to be installed on systems not covered by the wheels.