Skip to content

llama : add llama_model_ftype_name() - #25134

Merged
angt merged 6 commits into
ggml-org:masterfrom
angt:llama-add-llama_model_ftype_name
Jul 2, 2026
Merged

llama : add llama_model_ftype_name()#25134
angt merged 6 commits into
ggml-org:masterfrom
angt:llama-add-llama_model_ftype_name

Conversation

@angt

@angt angt commented Jun 29, 2026

Copy link
Copy Markdown
Member

Overview

Add llama_model_ftype_name()

Additional information

Expose the model file type (quantization) name, e.g. "Q8_0" or "Q4_K - Medium", through a new public C API. The returned pointer is valid for the lifetime of the model and nullptr when the model is invalid or the file type is unknown.

$ ./build/bin/llama cli -hf unsloth/Qwen3.5-0.8B-GGUF
Downloading Qwen3.5-0.8B-Q4_K_M.gguf ─────────────────────────────── 100%
Downloading mmproj-BF16.gguf ─────────────────────────────────────── 100%

Loading model...


▄▄ ▄▄
██ ██
██ ██  ▀▀█▄ ███▄███▄  ▀▀█▄    ▄████ ████▄ ████▄
██ ██ ▄█▀██ ██ ██ ██ ▄█▀██    ██    ██ ██ ██ ██
██ ██ ▀█▄██ ██ ██ ██ ▀█▄██ ██ ▀████ ████▀ ████▀
                                    ██    ██
                                    ▀▀    ▀▀

build      : b9841-e1bbcdc25
model      : unsloth/Qwen3.5-0.8B-GGUF
ftype      : Q4_K - Medium
modalities : text, vision

available commands:
  /exit or Ctrl+C     stop or exit
  /regen              regenerate the last response
  /clear              clear the chat history
  /read <file>        add a text file
  /glob <pattern>     add text files using globbing pattern
  /image <file>       add an image file
  /video <file>       add a video file
  

See #25020

Requirements

Expose the model file type (quantization) name, e.g. "Q8_0" or
"Q4_K - Medium", through a new public C API. The returned pointer is
valid for the lifetime of the model and nullptr when the model is
invalid or the file type is unknown.

Signed-off-by: Adrien Gallouët <angt@huggingface.co>
@angt
angt requested review from a team, CISC and ngxson as code owners June 29, 2026 14:58
@ngxson

ngxson commented Jun 29, 2026

Copy link
Copy Markdown
Collaborator

no strong opinion about this feature, but IMO the more important thing is to get the :tag of the running model. IIRC I had that logic before the HF migration, but seems like it's no longer there

for example: if I run without the tag: -hf user/model, then common_params_model::get_name() currently have no info about the tag being used. would be useful if you can fix that @angt

@angt

angt commented Jun 29, 2026

Copy link
Copy Markdown
Member Author

for example: if I run without the tag: -hf user/model, then common_params_model::get_name() currently have no info about the tag being used. would be useful if you can fix that @angt

I'll make a PR for that, but I still think this one is useful for the mentioned issue

@ngxson ngxson left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

sounds ok to me, not sure if there are any edge cases, but let's see

Comment thread src/llama-model.cpp Outdated
Signed-off-by: Adrien Gallouët <angt@huggingface.co>
@angt
angt requested a review from ggerganov as a code owner June 29, 2026 21:13
Comment thread include/llama.h Outdated
Signed-off-by: Adrien Gallouët <angt@huggingface.co>
Comment thread src/llama-model-loader.cpp Outdated
Prepend the "(guessed)" label instead of appending it. This allows removing
the non-thread-safe static std::string, making the function allocation-free.

Signed-off-by: Adrien Gallouët <angt@huggingface.co>
Comment thread src/llama-model-loader.cpp Outdated
Signed-off-by: Adrien Gallouët <angt@huggingface.co>
@angt
angt requested review from ServeurpersoCom and ngxson July 2, 2026 07:40
Comment thread src/llama-model.cpp
Signed-off-by: Adrien Gallouët <angt@huggingface.co>
@angt
angt merged commit fdb1db8 into ggml-org:master Jul 2, 2026
25 checks passed
zengde added a commit to zengde/llama.cpp that referenced this pull request Jul 3, 2026
llama : add llama_model_ftype_name() (ggml-org#25134)
fewtarius added a commit to fewtarius/CachyLLama that referenced this pull request Jul 4, 2026
30 commits from ggml-org/llama.cpp master. Two conflicts resolved by
combining our changes with upstream work that did not deprecate them:

ggml/src/ggml-cuda/gated_delta_net.cu
  Upstream (ggml-org#23940) fused the post-GDN cpy into the kernel itself by
  passing state_d/state_slot_stride through the kernel signature so
  recurrent-state snapshots land directly in the cache during MTP
  decode. Our RDNA3.5 tuning (num_warps=8 for S_v=128 non-KDA paths)
  generalized the kernel template on num_warps and dispatched in
  launch_gated_delta_net. Both coexist: the upstream signature is
  preserved, the RDNA3.5 dispatch still routes to the 8-warp variant
  on gfx1151.

tools/server/server-context.cpp
  Upstream (ggml-org#25241) promoted sse_ping_interval to task_params so the
  WebUI can request a 1s ping cadence per request while CLI clients
  keep the 30s default. Our per-user concurrency cap
  (max_concurrent_per_user, 429 fast-fail) lives in the same function
  but in disjoint code paths; both retained.

Also carried (no conflicts):
  - MCP server recommendations UI + first-time opt-in
  - AVX2 NVFP4 dot product on CPU
  - OpenCL Q1_0 GEMM/GEMV kernels + precompiled-binary support
  - Hexagon flash-attn rework (kernels split into .h headers)
  - Vulkan bk-loop matmul for Asahi, flops-based heuristic
  - WebGPU NVFP4 support
  - chat: trim messages sent to StepFun parser (ggml-org#25238)
  - llama_model_ftype_name() (ggml-org#25134)
  - HTTP/URL bracketed IPv6 literal handling
  - 25+ other backend, model-arch, and tooling changes

Test results: 51/51 main+model tests pass after re-fetching LFS
vocab files (test-tokenizers-ggml-vocabs was a stale-pointer issue
unrelated to the merge). test-jinja-py / test-quant-type-selection
failures are pre-existing environment/snapshot issues.

Co-authored-by: Gaurav Garg <gaugarg@nvidia.com>
Co-authored-by: Pascal <admin@serveurperso.com>
fewtarius added a commit to fewtarius/CachyLLama that referenced this pull request Jul 4, 2026
30 commits from ggml-org/llama.cpp master. Two conflicts resolved by
combining our changes with upstream work that did not deprecate them:

ggml/src/ggml-cuda/gated_delta_net.cu
  Upstream (ggml-org#23940) fused the post-GDN cpy into the kernel itself by
  passing state_d/state_slot_stride through the kernel signature so
  recurrent-state snapshots land directly in the cache during MTP
  decode. Our RDNA3.5 tuning (num_warps=8 for S_v=128 non-KDA paths)
  generalized the kernel template on num_warps and dispatched in
  launch_gated_delta_net. Both coexist: the upstream signature is
  preserved, the RDNA3.5 dispatch still routes to the 8-warp variant
  on gfx1151.

tools/server/server-context.cpp
  Upstream (ggml-org#25241) promoted sse_ping_interval to task_params so the
  WebUI can request a 1s ping cadence per request while CLI clients
  keep the 30s default. Our per-user concurrency cap
  (max_concurrent_per_user, 429 fast-fail) lives in the same function
  but in disjoint code paths; both retained.

Also carried (no conflicts):
  - MCP server recommendations UI + first-time opt-in
  - AVX2 NVFP4 dot product on CPU
  - OpenCL Q1_0 GEMM/GEMV kernels + precompiled-binary support
  - Hexagon flash-attn rework (kernels split into .h headers)
  - Vulkan bk-loop matmul for Asahi, flops-based heuristic
  - WebGPU NVFP4 support
  - chat: trim messages sent to StepFun parser (ggml-org#25238)
  - llama_model_ftype_name() (ggml-org#25134)
  - HTTP/URL bracketed IPv6 literal handling
  - 25+ other backend, model-arch, and tooling changes

Test results: 51/51 main+model tests pass after re-fetching LFS
vocab files (test-tokenizers-ggml-vocabs was a stale-pointer issue
unrelated to the merge). test-jinja-py / test-quant-type-selection
failures are pre-existing environment/snapshot issues.
fewtarius added a commit to fewtarius/CachyLLama that referenced this pull request Jul 4, 2026
32 commits from ggml-org/llama.cpp master. Two conflicts resolved by
combining our changes with upstream work that did not deprecate them:

ggml/src/ggml-cuda/gated_delta_net.cu
  Upstream (ggml-org#23940) fused the post-GDN cpy into the kernel itself by
  passing state_d/state_slot_stride through the kernel signature so
  recurrent-state snapshots land directly in the cache during MTP
  decode. Our RDNA3.5 tuning (num_warps=8 for S_v=128 non-KDA paths)
  generalized the kernel template on num_warps and dispatched in
  launch_gated_delta_net. Both coexist: the upstream signature is
  preserved, the RDNA3.5 dispatch still routes to the 8-warp variant
  on gfx1151.

tools/server/server-context.cpp
  Upstream (ggml-org#25241) promoted sse_ping_interval to task_params so the
  WebUI can request a 1s ping cadence per request while CLI clients
  keep the 30s default. Our per-user concurrency cap
  (max_concurrent_per_user, 429 fast-fail) lives in the same function
  but in disjoint code paths; both retained.

Also carried (no conflicts):
  - MCP server recommendations UI + first-time opt-in
  - AVX2 NVFP4 dot product on CPU
  - OpenCL Q1_0 GEMM/GEMV kernels + precompiled-binary support
  - Hexagon flash-attn rework (kernels split into .h headers)
  - Vulkan bk-loop matmul for Asahi, flops-based heuristic
  - WebGPU NVFP4 support
  - chat: trim messages sent to StepFun parser (ggml-org#25238)
  - llama_model_ftype_name() (ggml-org#25134)
  - HTTP/URL bracketed IPv6 literal handling
  - 25+ other backend, model-arch, and tooling changes

Test results: 52/52 main+model tests pass after re-fetching LFS
vocab files (test-tokenizers-ggml-vocabs was a stale-pointer issue
unrelated to the merge). test-jinja-py / test-quant-type-selection
failures are pre-existing environment/snapshot issues.

Co-authored-by: Gaurav Garg <gaugarg@nvidia.com>
Co-authored-by: Pascal <admin@serveurperso.com>
fewtarius added a commit to fewtarius/CachyLLama that referenced this pull request Jul 4, 2026
32 commits from ggml-org/llama.cpp master. Two conflicts resolved by
combining our changes with upstream work that did not deprecate them:

ggml/src/ggml-cuda/gated_delta_net.cu
  Upstream ggml-org#23940 fused the post-GDN cpy into the kernel itself by
  passing state_d/state_slot_stride through the kernel signature so
  recurrent-state snapshots land directly in the cache during MTP
  decode. Our RDNA3.5 tuning (num_warps=8 for S_v=128 non-KDA paths)
  generalized the kernel template on num_warps and dispatched in
  launch_gated_delta_net. Both coexist: the upstream signature is
  preserved, the RDNA3.5 dispatch still routes to the 8-warp variant
  on gfx1151.

tools/server/server-context.cpp
  Upstream ggml-org#25241 promoted sse_ping_interval to task_params so the
  WebUI can request a 1s ping cadence per request while CLI clients
  keep the 30s default. Our per-user concurrency cap
  (max_concurrent_per_user, 429 fast-fail) lives in the same function
  but in disjoint code paths; both retained.

Also carried (no conflicts):
  - MCP server recommendations UI + first-time opt-in
  - AVX2 NVFP4 dot product on CPU
  - OpenCL Q1_0 GEMM/GEMV kernels + precompiled-binary support
  - Hexagon flash-attn rework (kernels split into .h headers)
  - Vulkan bk-loop matmul for Asahi, flops-based heuristic
  - WebGPU NVFP4 support
  - chat: trim messages sent to StepFun parser (ggml-org#25238)
  - llama_model_ftype_name() (ggml-org#25134)
  - HTTP/URL bracketed IPv6 literal handling
  - 25+ other backend, model-arch, and tooling changes

Test results: 52/52 main+model tests pass after re-fetching LFS
vocab files (test-tokenizers-ggml-vocabs was a stale-pointer issue
unrelated to the merge). test-jinja-py / test-quant-type-selection
failures are pre-existing environment/snapshot issues.
adrianhoehne pushed a commit to adrianhoehne/llama.cpp that referenced this pull request Jul 5, 2026
* llama : add llama_model_ftype_name()

Expose the model file type (quantization) name, e.g. "Q8_0" or
"Q4_K - Medium", through a new public C API. The returned pointer is
valid for the lifetime of the model and nullptr when the model is
invalid or the file type is unknown.

Signed-off-by: Adrien Gallouët <angt@huggingface.co>

* Export enum

Signed-off-by: Adrien Gallouët <angt@huggingface.co>

* s/llama_model_ftype_name/llama_ftype_name/

Signed-off-by: Adrien Gallouët <angt@huggingface.co>

* Move "(guessed)" to the front in llama_ftype_name

Prepend the "(guessed)" label instead of appending it. This allows removing
the non-thread-safe static std::string, making the function allocation-free.

Signed-off-by: Adrien Gallouët <angt@huggingface.co>

* Add LLAMA_FTYPE_PREFIX

Signed-off-by: Adrien Gallouët <angt@huggingface.co>

* Dont check for model

Signed-off-by: Adrien Gallouët <angt@huggingface.co>

---------

Signed-off-by: Adrien Gallouët <angt@huggingface.co>
CowboyTim pushed a commit to aardbeiplantje/llama.cpp that referenced this pull request Jul 21, 2026
* llama : add llama_model_ftype_name()

Expose the model file type (quantization) name, e.g. "Q8_0" or
"Q4_K - Medium", through a new public C API. The returned pointer is
valid for the lifetime of the model and nullptr when the model is
invalid or the file type is unknown.

Signed-off-by: Adrien Gallouët <angt@huggingface.co>

* Export enum

Signed-off-by: Adrien Gallouët <angt@huggingface.co>

* s/llama_model_ftype_name/llama_ftype_name/

Signed-off-by: Adrien Gallouët <angt@huggingface.co>

* Move "(guessed)" to the front in llama_ftype_name

Prepend the "(guessed)" label instead of appending it. This allows removing
the non-thread-safe static std::string, making the function allocation-free.

Signed-off-by: Adrien Gallouët <angt@huggingface.co>

* Add LLAMA_FTYPE_PREFIX

Signed-off-by: Adrien Gallouët <angt@huggingface.co>

* Dont check for model

Signed-off-by: Adrien Gallouët <angt@huggingface.co>

---------

Signed-off-by: Adrien Gallouët <angt@huggingface.co>
zommiommy pushed a commit to zommiommy/llama.cpp that referenced this pull request Aug 18, 2026
* llama : add llama_model_ftype_name()

Expose the model file type (quantization) name, e.g. "Q8_0" or
"Q4_K - Medium", through a new public C API. The returned pointer is
valid for the lifetime of the model and nullptr when the model is
invalid or the file type is unknown.

Signed-off-by: Adrien Gallouët <angt@huggingface.co>

* Export enum

Signed-off-by: Adrien Gallouët <angt@huggingface.co>

* s/llama_model_ftype_name/llama_ftype_name/

Signed-off-by: Adrien Gallouët <angt@huggingface.co>

* Move "(guessed)" to the front in llama_ftype_name

Prepend the "(guessed)" label instead of appending it. This allows removing
the non-thread-safe static std::string, making the function allocation-free.

Signed-off-by: Adrien Gallouët <angt@huggingface.co>

* Add LLAMA_FTYPE_PREFIX

Signed-off-by: Adrien Gallouët <angt@huggingface.co>

* Dont check for model

Signed-off-by: Adrien Gallouët <angt@huggingface.co>

---------

Signed-off-by: Adrien Gallouët <angt@huggingface.co>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants