chore(model gallery): add qwen3-coder-30b-a3b-instruct based on model request #8082
Conversation
… request Signed-off-by: rampa3 <68955305+rampa3@users.noreply.github.com>
✅ Deploy Preview for localai ready!
To edit notification comments on pull requests, go to your Netlify project configuration. |
| uri: huggingface://mradermacher/boomerang-qwen3-4.9B-GGUF/boomerang-qwen3-4.9B.Q4_K_M.gguf | ||
| - !!merge <<: *qwen3 | ||
| name: "qwen3-coder-30b-a3b-instruct" | ||
| icon: https://cdn-avatars.huggingface.co/v1/production/uploads/620760a26e3b7210c2ff1943/-s1gyJfvbE1RgO5iBeNOi.png |
There was a problem hiding this comment.
a model config url is missing here. Take a look at #8088 for an example of a model import
There was a problem hiding this comment.
Whoops! Let me fix it quickly.
There was a problem hiding this comment.
Added url: "github:mudler/LocalAI/gallery/qwen3.yaml@master".
There was a problem hiding this comment.
I thought this carries over from the main Qwen 3 entry, since it is a merged entry. My bad.
There was a problem hiding this comment.
ouch. you are actually right. I have been misleaded when I saw the description - I thought you were adding the model without the anchor. You are right, if there is an anchor everything is carried over, and other entries are overridden, sorry!
There was a problem hiding this comment.
Should I undo it then?
Signed-off-by: rampa3 <68955305+rampa3@users.noreply.github.com>
|
Thank you @rampa3 ! |
|
@rampa3 Thank you very much!! Is the awesome Qwen3-Next in the model-requests channel by any chance? 😄 MLX 4-bit is only 45 GB for example, runs smoothly as butter. |
Not sure... Though you will have to ask someone else to add this one even in GGUF Q4_K_M if it is not, as that thing has 48.5 GB in that quantization, so I won't be able to test it if I wrote the model config for it due to my system RAM being 32 GB. Already for qwen3-coder-30b-a3b-instruct I was waiting for kernel to just pull the plug on it due to it almost 100% RAM together with rest of stuff I run in the background on my laptop. Also, I am PC user, so I cannot do anything MLX. If I test models, I do so against |
|
Just checked - it is not there. @reneleonhardt |
3 participants
Description
This PR adds GGUF for Q4_K_M quantization of Qwen3-Coder-30B-A3B-Instruct from Unsloth to the model gallery, based on request in the
#model-requestschannel of LocalAI Discord.Notes for Reviewers
Model installation was tested on locally hosted gallery, and syntax of merge was compared to other Qwen 3 merges structure-wise before PR.
Signed commits