-
Notifications
You must be signed in to change notification settings - Fork 1.4k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
llama_model_load: loading model from 'models/7B/ggml-model-q4_0.bin' #251
Comments
I have the same problem on my 30B and 65B installations. 30B Installation Directory
65B Installation Directory
I'm not experienced in this area to know what to do with these. I found the ./quantize tutorial here for 7B and 13B because I keep getting a error of - './quantize' is not recognized as the name of a cmdlet, function, script file, or operable program - so I used this tutorial to quantize those files but I don't have the files needed in my 30B and 65B directories so I can't run those. Any help or guidance would be appreciated. Thanks! |
So Following the directions in the ReadMe, I got it working in terminal which honestly is good enough for me! Directions: Alpaca.cppRun a fast ChatGPT-like model locally on your device. The screencast below is not sped up and running on an M2 Macbook Air with 4GB of weights. This combines the LLaMA foundation model with an open reproduction of Stanford Alpaca a fine-tuning of the base model to obey instructions (akin to the RLHF used to train ChatGPT) and a set of modifications to llama.cpp to add a chat interface. Get Started (7B)Download the zip file corresponding to your operating system from the latest release. On Windows, download Download Once you've downloaded the model weights and placed them into the same directory as the
The weights are based on the published fine-tunes from Building from Source (MacOS/Linux)git clone https://github.com/antimatter15/alpaca.cpp
cd alpaca.cpp
make chat
./chat Building from Source (Windows)
cmake .
cmake --build . --config Release
.\Release\chat.exe
CreditThis combines Facebook's LLaMA, Stanford Alpaca, alpaca-lora and corresponding weights by Eric Wang (which uses Jason Phang's implementation of LLaMA on top of Hugging Face Transformers), and llama.cpp by Georgi Gerganov. The chat implementation is based on Matvey Soloviev's Interactive Mode for llama.cpp. Inspired by Simon Willison's getting started guide for LLaMA. Andy Matuschak's thread on adapting this to 13B, using fine tuning weights by Sam Witteveen. DisclaimerNote that the model weights are only to be used for research purposes, as they are derivative of LLaMA, and uses the published instruction data from the Stanford Alpaca project which is generated by OpenAI, which itself disallows the usage of its outputs to train competing models. |
Inside my "dalai\alpaca\models\7B" folder I just have 3 files "checklist.chk" "consolidated.00.pth" and "params.json". I really do more with Java and web development so I'm very out of my league. I assume I'm missing some files, but any help would be greatly appreciated!
The text was updated successfully, but these errors were encountered: