-
Notifications
You must be signed in to change notification settings - Fork 1k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Publish all wheels to PyPI #741
Comments
Hey @simonw! Big fan of your datasette project. I hear you and I would like to make the setup process a little easier and less error-prone. Currently llama.cpp supports a number of optional accelerations including several BLAS libraries, CUDA versions, OpenCL, and Metal. In theory I could build a pre-built wheel that just includes a version of llama.cpp with no real accelerations enabled but I feel like this is counterintuitive to the goal of providing users with the fastest local inference for their hardware. I'm open to suggestions though, and I'll try to think of some possible solutions. |
Two approaches I can think of trying that might work are:
For that first option, one way that could work is to have a How large are the different binaries? If all of them could be bundled in a single wheel that was less than 50MB then that could be a neat solution, if you can write code that can detect which one to use. You could even distribute that as it's a tricky problem though! I bet there are good options I've missed here. |
Hey @simonw it took a while but this is finally possible through a self-hosted PEP503 repository on Github Pages (see #1247) You should now be able to specify pip install llama-cpp-python --extra-index-url https://abetlen.github.io/llama-cpp-python/whl/cpu on the CLI or --extra-index-url https://abetlen.github.io/llama-cpp-python/whl/cpu
llama-cpp-python in a The PR also includes initial support for Metal and CUDA wheels though I had to limit the number of supported Python and CUDA versions to avoid a combinatorial explosion in the number of builds. |
It looks like PyPI only has the source distribution for each release: https://pypi.org/project/llama-cpp-python/0.2.6/#files
But the GitHub release at https://github.com/abetlen/llama-cpp-python/releases/tag/v0.2.6 lists many more files than that:
Would it be possible to push those wheels to PyPI as well?
I'd love to be able to
pip install llama-cpp-python
and get a compiled wheel for my platform.The text was updated successfully, but these errors were encountered: