Skip to content

convert : add --local-model and --local-llama options - #23

Merged
danbev merged 3 commits into
ggml-org:masterfrom
danbev:local-model-option
Aug 28, 2026
Merged

convert : add --local-model and --local-llama options#23
danbev merged 3 commits into
ggml-org:masterfrom
danbev:local-model-option

Conversation

@danbev

@danbev danbev commented Aug 25, 2026

Copy link
Copy Markdown
Member

This commit adds two new options to the convertion script to enable specifying a local model directory instead the models being downloaded form HuggingFace. And the other option is to enable a local llama.cpp directory to be used instead of upstream llama.cpp

The motivation for these options is to enable the convert repo/tool to be used with models that are not yet available on HF, for example unreleased models. These models sometimes requires additions/changes to llama.cpp as well and by introducing the local-llama option a local/private llama.cpp can be used to convert the models.


example usage
#!/bin/bash

set -e

if [ ! -x venv/bin/python ]; then
      bash create-venv.sh
fi


local_model=~/work/models/qwen/Qwen3.8-27B
local_llama=~/work/llama.cpp-some-local-dev-repo

convert_models() {
	source venv/bin/activate
	./convert.sh --owner danbev --one qwen3.8-27b --no-upload --keep --force \
		--local-model ${local_model} \
		--local-llama ${local_llama}
	deactivate
}
convert_models

Example upload: https://huggingface.co/danbev/Qwen3.8-27B-GGUF

Note

This model repo is private and should not be visible which is actually what we want so we don't disclose any information about new models.

danbev added 2 commits August 25, 2026 10:34
This commit adds two new options to the convertion script to enable
specifying a local model directory instead the models being downloaded
form HuggingFace. And the other option is to enable a local llama.cpp
directory to be used instead of upstream llama.cpp

The motivation for these options is to enable the convert repo/tool to
be used with models that are not yet available on HF, for example
unreleased models. These models sometimes requires additions/changes to
llama.cpp as well and it by introducing the local-llama option a
local/private llama.cpp can be used to convert the models.
This commit make --no-upload also apply to the model card to prevent
anything from being uploaded to HF.

The --private flag is also added when --local-model is specified which
make the model repository private on HF.

With these in place developing a new private model can be done and it is
possible to verify the uploaded artifact using a HF user separate from
the ggml-org namespace. I've tested this and the following repository
should not be accessible:
https://huggingface.co/danbev/Qwen3.8-27B-GGUF
@danbev
danbev marked this pull request as ready for review August 27, 2026 08:38
@danbev
danbev requested a review from ggerganov August 28, 2026 07:07
@danbev
danbev merged commit 47a1374 into ggml-org:master Aug 28, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants