Skip to content

common: sort safetensor files to prevent illegal split error at loadi… - #25183

Closed
adrianhoehne wants to merge 1 commit into
ggml-org:masterfrom
adrianhoehne:fix-hf-split-file-order-causing-crash-at-model-load
Closed

common: sort safetensor files to prevent illegal split error at loadi…#25183
adrianhoehne wants to merge 1 commit into
ggml-org:masterfrom
adrianhoehne:fix-hf-split-file-order-causing-crash-at-model-load

Conversation

@adrianhoehne

Copy link
Copy Markdown

…ng model.

fixes(#25181)

Overview

When hf is set then the files of the specific folder are read in, this may happen unordered, so at the end the wrong safetensor file is given to the main logic and will cause an "illegal split file idx error.

This code just sorts the list by index before returning it to the caller.

Additional information

Requirements

It is reproduce and tested with the following configuration:

017.551 I srv          load: spawning server instance with args:
0.17.017.552 I srv          load:   /home/adrian/llama.cpp/build/bin/llama-server
0.17.017.553 I srv          load:   --cache-reuse
0.17.017.554 I srv          load:   256
0.17.017.555 I srv          load:   --host
0.17.017.556 I srv          load:   127.0.0.1
0.17.017.557 I srv          load:   --jinja
0.17.017.557 I srv          load:   --no-mmproj-auto
0.17.017.558 I srv          load:   --no-mmproj-offload
0.17.017.559 I srv          load:   --offline
0.17.017.560 I srv          load:   --port
0.17.017.561 I srv          load:   46073
0.17.017.561 I srv          load:   --tags
0.17.017.562 I srv          load:   qwen3next,coder-next
0.17.017.563 I srv          load:   --temperature
0.17.017.564 I srv          load:   0.2
0.17.017.564 I srv          load:   --alias
0.17.017.566 I srv          load:   qwen3-coder-next-q6k-default-llama
0.17.017.567 I srv          load:   --ctx-size
0.17.017.567 I srv          load:   131072
0.17.017.568 I srv          load:   --device
0.17.017.569 I srv          load:   CUDA0
0.17.017.570 I srv          load:   --flash-attn
0.17.017.570 I srv          load:   on
0.17.017.571 I srv          load:   --fit
0.17.017.572 I srv          load:   off
0.17.017.615 I srv          load:   --hf-repo
0.17.017.616 I srv          load:   unsloth/Qwen3-Coder-Next-GGUF
0.17.017.616 I srv          load:   --hf-file
0.17.017.618 I srv          load:   UD-Q6_K/Qwen3-Coder-Next-UD-Q6_K-00001-of-00003.gguf
0.17.017.619 I srv          load:   --kv-offload
0.17.017.619 I srv          load:   --main-gpu
0.17.017.620 I srv          load:   0
0.17.017.621 I srv          load:   --n-cpu-moe
0.17.017.622 I srv          load:   55
0.17.017.622 I srv          load:   --n-gpu-layers
0.17.017.623 I srv          load:   999
0.17.017.624 I srv          load:   --parallel
0.17.017.625 I srv          load:   1
0.17.017.625 I srv          load:   --reasoning
0.17.017.626 I srv          load:   off
0.17.017.627 I srv          load:   --split-mode
0.17.017.628 I srv          load:   none
  • I have read and agree with the contributing guidelines
  • AI usage disclosure: YES, to explain code, as this is not my main language.

@adrianhoehne
adrianhoehne requested a review from a team as a code owner June 30, 2026 19:30
@ngxson

ngxson commented Jun 30, 2026

Copy link
Copy Markdown
Collaborator

not 100% if this could be a duplicated code path, cc @angt if you can have a look

@ngxson ngxson closed this in #25194 Jul 1, 2026
@adrianhoehne
adrianhoehne deleted the fix-hf-split-file-order-causing-crash-at-model-load branch July 1, 2026 16:35
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants