Skip to content

ggml: add ggml_rope_set_offset (+ metal support) - #27120

Merged
ngxson merged 9 commits into
masterfrom
xsn/ggml_rope_off
Aug 19, 2026
Merged

ggml: add ggml_rope_set_offset (+ metal support)#27120
ngxson merged 9 commits into
masterfrom
xsn/ggml_rope_off

Conversation

@ngxson

@ngxson ngxson commented Aug 15, 2026 •
edited
Loading

Copy link
Copy Markdown
Collaborator

Overview

Ref #27091 (comment)

Only CPU + Metal is supported, changes to other backends are to disable it

CUDA PR: #27121

2 use cases for this feature:

  • DSA indexer rope: [nope | rope] style, so offset = n_dims_nope
  • 2D rope (many mtmd models use this), rope each section [ X | Y ] without using concat

Requirements

@ngxson
ngxson requested review from a team and ggerganov as code owners August 15, 2026 10:23
@github-actions github-actions Bot added testing Everything test related ggml changes relating to the ggml tensor library for machine learning Apple Metal https://en.wikipedia.org/wiki/Metal_(API) labels Aug 15, 2026
@ngxson ngxson mentioned this pull request Aug 15, 2026
Merged
@ngxson
ngxson requested review from a team and marty1885 as code owners August 15, 2026 10:47
@github-actions github-actions Bot added Vulkan Issues specific to the Vulkan backend SYCL https://en.wikipedia.org/wiki/SYCL - GPU programming language Ascend NPU issues specific to Ascend NPUs OpenCL Issues specific to the OpenCL backend Hexagon CUDA Related to the CUDA backend OpenVINO WebGPU labels Aug 15, 2026
@ngxson
ngxson removed request for a team August 15, 2026 10:47
@ngxson ngxson removed SYCL https://en.wikipedia.org/wiki/SYCL - GPU programming language Ascend NPU issues specific to Ascend NPUs OpenCL Issues specific to the OpenCL backend Hexagon CUDA Related to the CUDA backend OpenVINO WebGPU labels Aug 15, 2026
@ngxson ngxson mentioned this pull request Aug 15, 2026
Merged
@github-actions github-actions Bot added Vulkan Issues specific to the Vulkan backend SYCL https://en.wikipedia.org/wiki/SYCL - GPU programming language Ascend NPU issues specific to Ascend NPUs OpenCL Issues specific to the OpenCL backend Hexagon CUDA Related to the CUDA backend OpenVINO WebGPU labels Aug 15, 2026
@ngxson
ngxson force-pushed the xsn/ggml_rope_off branch from 1f11035 to 5f4001e Compare August 15, 2026 15:01
@fairydreaming fairydreaming mentioned this pull request Aug 16, 2026
Merged
@ngxson

ngxson commented Aug 18, 2026

Copy link
Copy Markdown
Collaborator Author

@ggerganov Could you have a quick look on this? Small modification to RoPE kernel. Thanks in advance!

@ngxson ngxson mentioned this pull request Aug 18, 2026
Closed
@ggerganov ggerganov self-assigned this Aug 18, 2026
This was referenced Aug 18, 2026
Merged
Merged
ngxson added 2 commits August 19, 2026 10:38
@ngxson
ggml: support ggml_rope_set_offset on vulkan (#27344)
8847298 
* ggml: support ggml_rope_set_offset on vulkan

* remove inplace optimization
@ngxson
Merge branch 'master' into xsn/ggml_rope_off
6415eee
@ngxson
ngxson merged commit fe8156f into master Aug 19, 2026
54 of 58 checks passed
@ngxson ngxson mentioned this pull request Aug 21, 2026
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@ggerganov ggerganov ggerganov approved these changes

Assignees

@ggerganov ggerganov

Labels

Apple Metal https://en.wikipedia.org/wiki/Metal_(API) Ascend NPU issues specific to Ascend NPUs CUDA Related to the CUDA backend ggml changes relating to the ggml tensor library for machine learning Hexagon OpenCL Issues specific to the OpenCL backend OpenVINO SYCL https://en.wikipedia.org/wiki/SYCL - GPU programming language testing Everything test related Vulkan Issues specific to the Vulkan backend WebGPU

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@ngxson @ggerganov