mtmd: add pos_0 to mtmd_image_tokens_get_decoder_pos (breaking change)#22082
Merged
Conversation
ggerganov
approved these changes
Apr 18, 2026
CISC
approved these changes
Apr 18, 2026
Member
CISC
left a comment
CISC
left a comment
There was a problem hiding this comment.
You need to fix the failing test obviously. :)
| pos[i + batch.n_tokens ] = pos_0 + i; | ||
| pos[i + batch.n_tokens * 2] = pos_0 + i; | ||
| pos[i + batch.n_tokens * 3] = 0; // last pos dim is unused | ||
| pos[i + batch.n_tokens * 3] = pos_0 + i; |
Contributor
Author
There was a problem hiding this comment.
yes, it should not change anything because mrope_sections is always configured such that last dim = 0, so backend will always skip the positional data being set here.
CISC
approved these changes
Apr 19, 2026
3 tasks
wendadawen
pushed a commit
to ManaEstras/llama.cpp
that referenced
this pull request
Apr 20, 2026
- decoder_pos: move HunyuanVL BOI/EOI/newline layout into mtmd_image_tokens_get_decoder_pos (matches ggml-org#22082) - remove set_position_mrope_hunyuanvl and mtmd_decode_use_mrope_hunyuanvl; mtmd-helper.cpp now identical to master - image_tokens: replace n_tokens_total with n_boi/n_eoi/n_newline/image_idx - convert: drop hardcoded vit.perceive.* remapping, use standard tensor mapping - clip: temporarily use ggml_interpolate (without custom sf)
ArberSephirotheca
pushed a commit
to ArberSephirotheca/llama.cpp
that referenced
this pull request
Apr 21, 2026
ggml-org#22082) * mtmd: add pos_0 to mtmd_image_tokens_get_decoder_pos * fix build
arthw
pushed a commit
to arthw/llama.cpp
that referenced
this pull request
Apr 23, 2026
ggml-org#22082) * mtmd: add pos_0 to mtmd_image_tokens_get_decoder_pos * fix build
rsenthilkumar6
pushed a commit
to rsenthilkumar6/llama.cpp
that referenced
this pull request
May 1, 2026
ggml-org#22082) * mtmd: add pos_0 to mtmd_image_tokens_get_decoder_pos * fix build
ljubomirj
pushed a commit
to ljubomirj/llama.cpp
that referenced
this pull request
May 6, 2026
ggml-org#22082) * mtmd: add pos_0 to mtmd_image_tokens_get_decoder_pos * fix build
my-other-github-account
pushed a commit
to my-other-github-account/llama.cpp
that referenced
this pull request
May 15, 2026
ggml-org#22082) * mtmd: add pos_0 to mtmd_image_tokens_get_decoder_pos * fix build
my-other-github-account
pushed a commit
to my-other-github-account/llama.cpp
that referenced
this pull request
May 15, 2026
ggml-org#22082) * mtmd: add pos_0 to mtmd_image_tokens_get_decoder_pos * fix build
fewtarius
pushed a commit
to fewtarius/llama.cpp
that referenced
this pull request
May 30, 2026
ggml-org#22082) * mtmd: add pos_0 to mtmd_image_tokens_get_decoder_pos * fix build
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Overview
Add
pos_0parameter tomtmd_image_tokens_get_decoder_pos, this allow model to have total control over other dimensions of the RoPE positionsTested on Qwen3 and confirmed that it doesn't break anything
Requirements