model: GraniteSWAForCausalLM / GraniteMoeSWAForCausalLM by gabe-l-hart · Pull Request #25505 · ggml-org/llama.cpp

gabe-l-hart · 2026-07-09T19:02:19Z

Overview

This Pull Request mirrors huggingface/transformers#47179 to support forthcoming Granite models that will incorporate interleaved Sliding Window Attention with Attention Sinks.

Additional information

Model weights are not available just yet, but they exist internally and will eventually appear here
Some of the models in this family will have per-layer RoPE / NoPE. There is adjacent work to support this in the Granite Switch PR (Granite-Switch Architecture #25107). For the sake of keeping this encapsulated, I replicated the function signature for llama_hparams.has_rope. The logic behind the answer is different than the Granite Switch logic, so this will need to be rectified depending on merge order.
This has been verified on internal checkpoints.

Requirements

I have read and agree with the contributing guidelines
AI usage disclosure: YES

AI Disclosure

AI was used to create this PR. I manually reviewed all outputs and authored all commits myself. See stats below:

git-ai-stats

╔══════════════════════════════════════════════════════════╗
║           GIT AI USAGE ANALYSIS                          ║
╚══════════════════════════════════════════════════════════╝

📊 COMMITS BY AGENT

--- Aggregate ---
Commits                        |      Count
---------------------------------------------
IBM Bob                        |          5
IBM Bob, OpenCode + Qwen3.6-35b |          2
---------------------------------------------
TOTAL                          |          7

📊 COMMITS BY USAGE TYPE

--- Aggregate ---
Commits                        |      Count
---------------------------------------------
none                           |          0
draft                          |          0
full                           |          7
---------------------------------------------
TOTAL                          |          7

📈 LINES OF CODE BY AGENT

--- Aggregate ---
Agent                     |    Commits |  Additions |  Deletions
------------------------------------------------------------
IBM Bob                   |          5 |         85 |          8
IBM Bob, OpenCode + Qwen3.6-35b |          2 |        415 |          0
------------------------------------------------------------
TOTAL                     |          7 |        500 |          8

📈 LINES OF CODE BY USAGE TYPE

--- Aggregate ---
Usage Type           |    Commits |  Additions |  Deletions
-------------------------------------------------------
none                 |          0 |          0 |          0
draft                |          0 |          0 |          0
full                 |          7 |        500 |          8
-------------------------------------------------------
TOTAL                |          7 |        500 |          8

pwilkin · 2026-07-09T22:03:06Z

@gabe-l-hart and here I was about to ask you about https://huggingface.co/ibm-granite/granite-swash-2b/tree/main :) seems like at least one model from the new family made it through :)

gabe-l-hart · 2026-07-09T22:27:56Z

@pwilkin I didn't even catch that one, so it was even stealth within the house. I'll make sure everything is lined up for that model too (and get red tests green)

gabe-l-hart · 2026-07-09T22:41:39Z

The saga of Alex and Max is alive and well with granite-swash-2b:

model      : /Users/ghart/models/ibm-granite/granite-swash-2b/granite-swash-2B-BF16.gguf
ftype      : BF16
modalities : text

available commands:
  /exit or Ctrl+C     stop or exit
  /regen              regenerate the last response
  /clear              clear the chat history
  /read <file>        add a text file
  /glob <pattern>     add text files using globbing pattern



> Tell me a story about a developer and their dog
Once upon a time in a quaint little town, there lived a developer named Alex. Alex was an avid dog lover, and his dog, Max, was always by his side. Max was a golden retriever with a heart of gold and a spirit that could light up any room.

One sunny afternoon, while Alex was working on his latest project, he noticed that Max had been barking non-stop. He called his assistant, Sarah, over and asked if she had seen anything unusual. Sarah shrugged and said, "I don't know, maybe Max's hearing is a bit off today."
...

Branch: GraniteSWAForCausalLM AI-usage: full (Bob, OpenCode + Qwen3.6-35b) Signed-off-by: Gabe Goodhart <ghart@us.ibm.com>

NOTE: There is other work also targeting this, so this may be removed depending on merge order. Branch: GraniteSWAForCausalLM AI-usage: full (Bob) Signed-off-by: Gabe Goodhart <ghart@us.ibm.com>

Branch: GraniteSWAForCausalLM AI-usage: full (Bob) Signed-off-by: Gabe Goodhart <ghart@us.ibm.com>

NOTE: This shadows the work done for Granite Speech ggml-org#25107 Branch: GraniteSWAForCausalLM AI-usage: full (Bob) Signed-off-by: Gabe Goodhart <ghart@us.ibm.com>

Branch: GraniteSWAForCausalLM AI-usage: full (Bob) Signed-off-by: Gabe Goodhart <ghart@us.ibm.com>

Branch: GraniteSWAForCausalLM AI-usage: none Signed-off-by: Gabe Goodhart <ghart@us.ibm.com>

Branch: GraniteSWAForCausalLM AI-usage: full (OpenCode + Qwen3.6-35b) Signed-off-by: Gabe Goodhart <ghart@us.ibm.com>

Branch: GraniteSWAForCausalLM AI-usage: none Signed-off-by: Gabe Goodhart <ghart@us.ibm.com>

Branch: GraniteSWAForCausalLM AI-usage: draft (OpenCode + Qwen3.6-35b) Signed-off-by: Gabe Goodhart <ghart@us.ibm.com>

branch: GraniteSWAForCausalLM AI-usage: full (OpenCode + Qwen3.6-35b) Signed-off-by: Gabe Goodhart <ghart@us.ibm.com>

gabe-l-hart · 2026-07-14T19:12:03Z

We also have an MoE variant up now: https://huggingface.co/ibm-granite/granite-swash-3b-a600m. Needed a few changes, but this is also supported now.

Signed-off-by: Gabe Goodhart <ghart@us.ibm.com> * origin/master: (383 commits) cmake : introduce semantic versioning (ggml-org#26839) gguf : harden loader against malformed tensor dims and metadata types (ggml-org#25596) kleidiai: Add runtime feature detection mechanism for aarch64/kleidiai (ggml-org#26076) model : disallow integer dflash sliding_window_pattern (ggml-org#26900) sync : ggml cmake : add config version support (ggml/1582) server : support slot save/restore with media inputs (ggml-org#26640) ui: add read_media tool (ggml-org#25877) opencl: default FA c8 cluster width to 16 on X1E (ggml-org#26433) tests : update speculative params (ggml-org#26925) vulkan: add TQ2_0 (ternary) support (ggml-org#25850) wavtokenizer-dec : bound posnet/convnext block_count against n_layer_all (ggml-org#26892) convert : handle per_layer_config in Gemma4 (transformers 5.15) (ggml-org#26882) opencl: use flat mv q5_k when weight exceeds image1d_buffer_t limit (ggml-org#26880) chat : fix muse-glimmer detection of tool calls after EOM (ggml-org#26879) ci : add missing release check (ggml-org#26923) CUDA: only disable CUDA graphs when mul_mat_id actually needs a stream sync (ggml-org#26802) cuda : add warp-per-row wkv7 kernel for single-token decode (ggml-org#26111) spec : update speculative-simple (ggml-org#26904) chat : tighten bare function parsing for Qwen models (ggml-org#26793) ...

AI-usage: none Signed-off-by: Gabe Goodhart <ghart@us.ibm.com>

Branch: GraniteSWAForCausalLM AI-usage: none Signed-off-by: Gabe Goodhart <ghart@us.ibm.com>

gabe-l-hart · 2026-08-12T13:52:30Z

+    }
+
+    // Granite uses rope_finetuned as a switch for rope, so default to true
+    bool rope_finetuned = true;


Same point here: This should probably live in a common granite init utility

This isn't strictly necessary, but it will allow other models to rely on hparams.has_rope(il) without needting to prepopulate. Branch: GraniteSWAForCausalLM AI-usage: none Signed-off-by: Gabe Goodhart <ghart@us.ibm.com>

Now that we have a proper hparam for this, it's better to use it and not require a hacky fallback in the hparam method itself. Branch: GraniteSWAForCausalLM AI-usage: none Signed-off-by: Gabe Goodhart <ghart@us.ibm.com>

Branch: GraniteSWAForCausalLM AI-usage: none Signed-off-by: Gabe Goodhart <ghart@us.ibm.com>

There are no granitemoe models that use NoPE (it's not actually used in the layer building below), so this was just dead code. Branch: GraniteSWAForCausalLM AI-usage: none Signed-off-by: Gabe Goodhart <ghart@us.ibm.com>

pwilkin

No issues from my side.

Signed-off-by: Gabe Goodhart <ghart@us.ibm.com> Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@huggingface.co> Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@huggingface.co>

Branch: GraniteSWAForCausalLM AI-usage: full (Claude + Sonnet 5) Signed-off-by: Gabe Goodhart <ghart@us.ibm.com>

* origin/master: (86 commits) ui: enforce alphabetical enum member ordering (ggml-org#27272) ui: Refactor Built-In Tools naming (Server/Browser) (ggml-org#27271) ci: more optimizations (ggml-org#26983) doc: document MCP stdio servers and CORS defaults in the server README [no release] [no ci] (ggml-org#26847) server: save processed mtmd chunks as placeholder (ggml-org#27278) mtmd: use sha256 for input hashing (ggml-org#27274) vocab : support integer tokenizer scores (ggml-org#27260) mtmd : skip thumbnail for non-tiled LFM2 images (ggml-org#27246) cuda : skip UMA override for HIP builds (ggml-org#27083) vendor: move hash to vendor (ggml-org#27262) ci : push release tag explicitly in release.yml (ggml-org#27261) ui: move get_datetime tool to frontend (ggml-org#27255) ci : reduce builds in build-xcframework.sh (ggml-org#27252) model: support speculators-format checkpoints for DSpark (ggml-org#26275) ui: add browser get_info tool (ggml-org#27251) ci : restore release.yml check during make-release.yml (ggml-org#27247) mtmd: harden preprocessor_granite (ggml-org#27235) ci : allow make-release to target a specific commit (ggml-org#27234) ci : make release workflows use a deploy key (ggml-org#27229) convert: add @ModelBase.example (ggml-org#27208) ...

ggml-org#25505 (comment) Keeping is_swa_impl in the saver can break other models. Branch: GraniteSWAForCausalLM AI-usage: none Signed-off-by: Gabe Goodhart <ghart@us.ibm.com>

CISC

https://github.com/ggml-org/llama.cpp/actions/runs/32110372829/job/95628269167?pr=25505

Branch: GraniteSWAForCausalLM AI-usage: none Signed-off-by: Gabe Goodhart <ghart@us.ibm.com>

Thanks Claude! Branch: GraniteSWAForCausalLM AI-usage: none Signed-off-by: Gabe Goodhart <ghart@us.ibm.com>

Branch: GraniteSWAForCausalLM AI-usage: full (Claude + Sonnet 5) Signed-off-by: Gabe Goodhart <ghart@us.ibm.com>

CISC

nit

CISC · 2026-08-19T12:39:21Z

Sigh, this again?
https://github.com/ggml-org/llama.cpp/actions/runs/32253177999/job/96068698150?pr=25505

Edit: (I had honestly thought this was fixed, sooo annoying)

gabe-l-hart · 2026-08-19T12:41:25Z

Sigh, this again?
https://github.com/ggml-org/llama.cpp/actions/runs/32253177999/job/96068698150?pr=25505

Edit: (I had honestly thought this was fixed, sooo annoying)

Ooooh, so it wasn't Claude's fault! These lines both happened when I used GH's review suggestion batching.

CISC · 2026-08-19T12:42:43Z

Sigh, this again?
https://github.com/ggml-org/llama.cpp/actions/runs/32253177999/job/96068698150?pr=25505
Edit: (I had honestly thought this was fixed, sooo annoying)

Ooooh, so it wasn't Claude's fault! These lines both happened when I used GH's review suggestion batching.

Yes, it's some kind of random GH UI bug.

Signed-off-by: Gabe Goodhart <ghart@us.ibm.com> Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@huggingface.co>

gabe-l-hart · 2026-08-19T12:43:59Z

Ok, hopefully clean now. I promise to properly review whitespace myself next time! Too much juggling agents and human children at the same time...

CISC · 2026-08-19T12:44:53Z

Merge when CI is done.

* feat(convert): Add conversion for GraniteSWAForCausalLM Branch: GraniteSWAForCausalLM AI-usage: full (Bob, OpenCode + Qwen3.6-35b) Signed-off-by: Gabe Goodhart <ghart@us.ibm.com> * feat(llama): Add granite_swa support Branch: GraniteSWAForCausalLM AI-usage: full (Bob, OpenCode + Qwen3.6-35b) Signed-off-by: Gabe Goodhart <ghart@us.ibm.com> * feat(conversion): Add conversion infra for rope_pattern array NOTE: There is other work also targeting this, so this may be removed depending on merge order. Branch: GraniteSWAForCausalLM AI-usage: full (Bob) Signed-off-by: Gabe Goodhart <ghart@us.ibm.com> * fix(conversion): Fix SWA pattern logic and support for non-rope layers Branch: GraniteSWAForCausalLM AI-usage: full (Bob) Signed-off-by: Gabe Goodhart <ghart@us.ibm.com> * feat(conversion): Add support for GraniteMoeSWA Branch: GraniteSWAForCausalLM AI-usage: full (Bob) Signed-off-by: Gabe Goodhart <ghart@us.ibm.com> * feat: Add llama_hparams::has_rope and arch constants NOTE: This shadows the work done for Granite Speech ggml-org/llama.cpp#25107 Branch: GraniteSWAForCausalLM AI-usage: full (Bob) Signed-off-by: Gabe Goodhart <ghart@us.ibm.com> * feat: Add support for per-layer rope determination Branch: GraniteSWAForCausalLM AI-usage: full (Bob) Signed-off-by: Gabe Goodhart <ghart@us.ibm.com> * style: Fix failing flake8 for extra newlines Branch: GraniteSWAForCausalLM AI-usage: none Signed-off-by: Gabe Goodhart <ghart@us.ibm.com> * test: Write out SLIDING_WINDOW_PATTERN in llama-model-saver Branch: GraniteSWAForCausalLM AI-usage: full (OpenCode + Qwen3.6-35b) Signed-off-by: Gabe Goodhart <ghart@us.ibm.com> * fix(convert): Fix missing registration for GraniteMoeSWAForCausalLM Branch: GraniteSWAForCausalLM AI-usage: none Signed-off-by: Gabe Goodhart <ghart@us.ibm.com> * fix: Load MoE params as optional Branch: GraniteSWAForCausalLM AI-usage: draft (OpenCode + Qwen3.6-35b) Signed-off-by: Gabe Goodhart <ghart@us.ibm.com> * feat: Handle MoE params in conversion branch: GraniteSWAForCausalLM AI-usage: full (OpenCode + Qwen3.6-35b) Signed-off-by: Gabe Goodhart <ghart@us.ibm.com> * style: Remove unnecessary newline AI-usage: none Signed-off-by: Gabe Goodhart <ghart@us.ibm.com> * fix: Remove unnecessary tensor additions to GRANITE architecture Branch: GraniteSWAForCausalLM AI-usage: none Signed-off-by: Gabe Goodhart <ghart@us.ibm.com> * fix: Correctly handle naming for ffn gate inp Branch: GraniteSWAForCausalLM AI-usage: none Signed-off-by: Gabe Goodhart <ghart@us.ibm.com> * fix: Always default hparams.rope_pattern to 1s This isn't strictly necessary, but it will allow other models to rely on hparams.has_rope(il) without needting to prepopulate. Branch: GraniteSWAForCausalLM AI-usage: none Signed-off-by: Gabe Goodhart <ghart@us.ibm.com> * feat: Move to has_rope for all granite model architectures Now that we have a proper hparam for this, it's better to use it and not require a hacky fallback in the hparam method itself. Branch: GraniteSWAForCausalLM AI-usage: none Signed-off-by: Gabe Goodhart <ghart@us.ibm.com> * feat: No hacky rope_finetuned fallback in has_rope Branch: GraniteSWAForCausalLM AI-usage: none Signed-off-by: Gabe Goodhart <ghart@us.ibm.com> * fix: Fully remove rope hparam filling in granitemoe There are no granitemoe models that use NoPE (it's not actually used in the layer building below), so this was just dead code. Branch: GraniteSWAForCausalLM AI-usage: none Signed-off-by: Gabe Goodhart <ghart@us.ibm.com> * fix: Save out rope_pattern in model-saver Branch: GraniteSWAForCausalLM AI-usage: none Signed-off-by: Gabe Goodhart <ghart@us.ibm.com> * fix: Set hparams.rope_finetuned for round trip Since the value is _read_ from rope_finetuned, we need to persist it when the model is saved with the saver. Branch: GraniteSWAForCausalLM AI-usage: none Signed-off-by: Gabe Goodhart <ghart@us.ibm.com> * fix: Code review cleanup Signed-off-by: Gabe Goodhart <ghart@us.ibm.com> Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@huggingface.co> Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@huggingface.co> * refactor: Keep gate/up fused for MoE path Branch: GraniteSWAForCausalLM AI-usage: full (Claude + Sonnet 5) Signed-off-by: Gabe Goodhart <ghart@us.ibm.com> * fix: Skip GRANITE_SWA in model saver ggml-org/llama.cpp#25505 (comment) Keeping is_swa_impl in the saver can break other models. Branch: GraniteSWAForCausalLM AI-usage: none Signed-off-by: Gabe Goodhart <ghart@us.ibm.com> * add sliding window pattern for model in test * style: Fix indentation Branch: GraniteSWAForCausalLM AI-usage: none Signed-off-by: Gabe Goodhart <ghart@us.ibm.com> * fix: Fix \r\n Thanks Claude! Branch: GraniteSWAForCausalLM AI-usage: none Signed-off-by: Gabe Goodhart <ghart@us.ibm.com> * feat: Keep shared expert fused Branch: GraniteSWAForCausalLM AI-usage: full (Claude + Sonnet 5) Signed-off-by: Gabe Goodhart <ghart@us.ibm.com> * style: More indentation fixes Signed-off-by: Gabe Goodhart <ghart@us.ibm.com> Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@huggingface.co> --------- Signed-off-by: Gabe Goodhart <ghart@us.ibm.com> Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@huggingface.co>

Brings 218 upstream commits into CachyLLama without losing any of our features. Key carried-over changes from upstream: - llama.cpp v0.2.0 / ggml v0.21.0 version bumps - Vulkan FA MMQ fp32 scaling (ggml-org#27413), PAD_REFLECT_1D (ggml-org#26586), tiled transpose (ggml-org#26585), null checks in queue command pools cleanup (ggml-org#27353) - ggml: rope_set_offset on multiple backends, recurrent state rollback - Vulkan coopmat1 SHMEM_STRIDE_PAD/APPLY_SLM_A_RESHAPE for Intel Xe - server: LLAMA_SERVER_SLOTS_N_DIFF (ggml-org#27600), /metrics during llama_decode (ggml-org#27041), index.html no-cache (ggml-org#27006), make-release workflow - model: MiniMax-M1/Text01 (ggml-org#27018), Kimi-K3 (ggml-org#26185), BailingMoE3 (ggml-org#26608), GraniteSWA (ggml-org#25505), GLM-4.5-Air MTP, DSV4 tensor split (-sm tensor) - ui: Chat Conversation Tabbed navigation, settings refactor - common: --models-dir loading MTP assistant models (ggml-org#24431), --load-mode replacing --mmap (ggml-org#26934), json.h abstraction (ggml-org#27511) - vendor: cpp-httplib 0.53.1, BoringSSL 0.20260813.0, vendor/hash CachyLLama features preserved through conflict resolution: - Persistent SSD-backed KV cache (3-tier hot/warm/cold + system prompt cache) - Per-user isolation (user_id, per-user concurrency cap, slot affinity) - MoE expert residency + co-activation tracking - CachyLLama Vulkan Lightning Indexer (108/108 on Strix Halo) + DSV4 hyper-connection fused ops + DSV4 sparse FA + coopmat shaders - FA quant-KV dequant-once + f16 contiguize (with host-RAM safety gate) - DFlash framework + Laguna-S-2.1 model support - DFlash d2t reduced-vocab draft support (upstream merge) - Context checkpoint ring buffer + SWA skip + memory budget scaling - Stable-prefix LCP gate + prompt_stable_prefix_tokens param - conv_hash conversation-boundary detection - All CachyLLama Vulkan shaders (concat_transpose, lightning_indexer, mmid_row_lists, flash_attn_top_k, dequant_f16_transpose) - common::host_available_ram() utility - llama-moe-residency + llama-moe-coact modules Manual conflict resolution touches: src/models/dflash.cpp (DFlash d2t + aux_norm), src/llama-kv-cache-dsv4.cpp (state snapshot fix), src/llama- memory-recurrent.cpp (rs_idx bounds check), src/llama-model-saver.cpp (DSV4 compress_ratios + swiglu_clamp sizing), ggml/src/ggml-vulkan/ {ggml-vulkan.cpp,vulkan-shaders-gen.cpp,vulkan-shaders/dequant_q8_0. comp,vulkan-shaders/flash_attn.comp,vulkan-shaders/copy_transpose_02. comp} (CachyLLama shader registration + FA scratch gate), ggml/src/ ggml-cuda/mmvq.cu (RDNA3_5 + GB10 enum), gguf-py/gguf/constants.py (DFlash ENC_AUX_NORM + D2T tensors), tests/{CMakeLists.txt,test-backend- ops.cpp,test-llama-archs.cpp,test-recurrent-state-rollback.cpp} (test additions), tools/{CMakeLists.txt,server/*} (server_batch embd support + spec_is_replay + user_id routing + MCP servers + CORS), and docs/{AGENTS.md,README.md} (kept CachyLLama branding). Verified: full build succeeds, test-backend-ops Vulkan LIGHTNING_INDEXER + FLASH_ATTN pass on Strix Halo. Based on a re-merge from the 20260824 (pristine pre-merge) branch after a previous agent's merge attempt produced an unbuildable state from -X ours that wiped shader float-typing and broke the dequant_q8_0 + flash_attn shaders with redefinition errors.

github-actions Bot added model Model specific conversion labels Jul 9, 2026

gabe-l-hart added 12 commits July 14, 2026 13:00

feat(convert): Add conversion for GraniteSWAForCausalLM

4bbf9ab

Branch: GraniteSWAForCausalLM AI-usage: full (Bob, OpenCode + Qwen3.6-35b) Signed-off-by: Gabe Goodhart <ghart@us.ibm.com>

feat(llama): Add granite_swa support

c74049a

Branch: GraniteSWAForCausalLM AI-usage: full (Bob, OpenCode + Qwen3.6-35b) Signed-off-by: Gabe Goodhart <ghart@us.ibm.com>

feat(conversion): Add conversion infra for rope_pattern array

1f3741a

NOTE: There is other work also targeting this, so this may be removed depending on merge order. Branch: GraniteSWAForCausalLM AI-usage: full (Bob) Signed-off-by: Gabe Goodhart <ghart@us.ibm.com>

fix(conversion): Fix SWA pattern logic and support for non-rope layers

f7d057b

Branch: GraniteSWAForCausalLM AI-usage: full (Bob) Signed-off-by: Gabe Goodhart <ghart@us.ibm.com>

feat(conversion): Add support for GraniteMoeSWA

44215bb

Branch: GraniteSWAForCausalLM AI-usage: full (Bob) Signed-off-by: Gabe Goodhart <ghart@us.ibm.com>

feat: Add llama_hparams::has_rope and arch constants

cf31517

NOTE: This shadows the work done for Granite Speech ggml-org#25107 Branch: GraniteSWAForCausalLM AI-usage: full (Bob) Signed-off-by: Gabe Goodhart <ghart@us.ibm.com>

feat: Add support for per-layer rope determination

f5d4e9f

Branch: GraniteSWAForCausalLM AI-usage: full (Bob) Signed-off-by: Gabe Goodhart <ghart@us.ibm.com>

style: Fix failing flake8 for extra newlines

9d75ea8

Branch: GraniteSWAForCausalLM AI-usage: none Signed-off-by: Gabe Goodhart <ghart@us.ibm.com>

test: Write out SLIDING_WINDOW_PATTERN in llama-model-saver

ef14e1c

Branch: GraniteSWAForCausalLM AI-usage: full (OpenCode + Qwen3.6-35b) Signed-off-by: Gabe Goodhart <ghart@us.ibm.com>

fix(convert): Fix missing registration for GraniteMoeSWAForCausalLM

c32fe0b

Branch: GraniteSWAForCausalLM AI-usage: none Signed-off-by: Gabe Goodhart <ghart@us.ibm.com>

fix: Load MoE params as optional

36aa2e0

Branch: GraniteSWAForCausalLM AI-usage: draft (OpenCode + Qwen3.6-35b) Signed-off-by: Gabe Goodhart <ghart@us.ibm.com>

feat: Handle MoE params in conversion

94d2643

branch: GraniteSWAForCausalLM AI-usage: full (OpenCode + Qwen3.6-35b) Signed-off-by: Gabe Goodhart <ghart@us.ibm.com>

gabe-l-hart force-pushed the GraniteSWAForCausalLM branch from 7be415a to 94d2643 Compare July 14, 2026 19:00

gabe-l-hart added 4 commits August 12, 2026 07:18

style: Remove unnecessary newline

90c3d64

AI-usage: none Signed-off-by: Gabe Goodhart <ghart@us.ibm.com>

fix: Remove unnecessary tensor additions to GRANITE architecture

0b1df4c

Branch: GraniteSWAForCausalLM AI-usage: none Signed-off-by: Gabe Goodhart <ghart@us.ibm.com>

fix: Correctly handle naming for ffn gate inp

8865f1b

Branch: GraniteSWAForCausalLM AI-usage: none Signed-off-by: Gabe Goodhart <ghart@us.ibm.com>

gabe-l-hart commented Aug 12, 2026

View reviewed changes

Comment thread src/models/granite-swa.cpp

gabe-l-hart commented Aug 12, 2026

View reviewed changes

gabe-l-hart added 3 commits August 12, 2026 11:18

fix: Always default hparams.rope_pattern to 1s

8e5bcad

This isn't strictly necessary, but it will allow other models to rely on hparams.has_rope(il) without needting to prepopulate. Branch: GraniteSWAForCausalLM AI-usage: none Signed-off-by: Gabe Goodhart <ghart@us.ibm.com>

feat: Move to has_rope for all granite model architectures

977437b

Now that we have a proper hparam for this, it's better to use it and not require a hacky fallback in the hparam method itself. Branch: GraniteSWAForCausalLM AI-usage: none Signed-off-by: Gabe Goodhart <ghart@us.ibm.com>

feat: No hacky rope_finetuned fallback in has_rope

e400df6

Branch: GraniteSWAForCausalLM AI-usage: none Signed-off-by: Gabe Goodhart <ghart@us.ibm.com>

gabe-l-hart commented Aug 12, 2026

View reviewed changes

Comment thread src/models/granite-hybrid.cpp

fix: Fully remove rope hparam filling in granitemoe

e7695cf

There are no granitemoe models that use NoPE (it's not actually used in the layer building below), so this was just dead code. Branch: GraniteSWAForCausalLM AI-usage: none Signed-off-by: Gabe Goodhart <ghart@us.ibm.com>

pwilkin approved these changes Aug 12, 2026

View reviewed changes

CISC reviewed Aug 13, 2026

View reviewed changes

Comment thread conversion/granite.py Outdated

Comment thread src/llama-model-saver.cpp Outdated

Comment thread src/models/granite-swa.cpp Outdated

Comment thread src/models/granite-swa.cpp

Comment thread conversion/granite.py Outdated

gabe-l-hart and others added 4 commits August 17, 2026 12:05

fix: Code review cleanup

600615d

Signed-off-by: Gabe Goodhart <ghart@us.ibm.com> Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@huggingface.co> Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@huggingface.co>

refactor: Keep gate/up fused for MoE path

94fc69f

Branch: GraniteSWAForCausalLM AI-usage: full (Claude + Sonnet 5) Signed-off-by: Gabe Goodhart <ghart@us.ibm.com>

fix: Skip GRANITE_SWA in model saver

c94d5ac

ggml-org#25505 (comment) Keeping is_swa_impl in the saver can break other models. Branch: GraniteSWAForCausalLM AI-usage: none Signed-off-by: Gabe Goodhart <ghart@us.ibm.com>

gabe-l-hart commented Aug 17, 2026

View reviewed changes

Comment thread src/llama-model-saver.cpp

add sliding window pattern for model in test

b5b7d1d

CISC requested a review from JohannesGaessler as a code owner August 18, 2026 07:12

CISC removed the request for review from JohannesGaessler August 18, 2026 07:13

github-actions Bot added the testing Everything test related label Aug 18, 2026

CISC approved these changes Aug 18, 2026

View reviewed changes

Comment thread conversion/granite.py Outdated

Comment thread src/models/models.h Outdated

gabe-l-hart added 3 commits August 18, 2026 09:18

style: Fix indentation

f7b1311

Branch: GraniteSWAForCausalLM AI-usage: none Signed-off-by: Gabe Goodhart <ghart@us.ibm.com>

fix: Fix \r\n

eeb83f3

Thanks Claude! Branch: GraniteSWAForCausalLM AI-usage: none Signed-off-by: Gabe Goodhart <ghart@us.ibm.com>

feat: Keep shared expert fused

d7a74f1

Branch: GraniteSWAForCausalLM AI-usage: full (Claude + Sonnet 5) Signed-off-by: Gabe Goodhart <ghart@us.ibm.com>

CISC approved these changes Aug 19, 2026

View reviewed changes

CISC reviewed Aug 19, 2026

View reviewed changes

Comment thread src/llama-arch.cpp Outdated

CISC approved these changes Aug 19, 2026

View reviewed changes

style: More indentation fixes

de5f6fd

Signed-off-by: Gabe Goodhart <ghart@us.ibm.com> Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@huggingface.co>

gabe-l-hart force-pushed the GraniteSWAForCausalLM branch from 1ff0c6f to de5f6fd Compare August 19, 2026 12:42

CISC approved these changes Aug 19, 2026

View reviewed changes

CISC merged commit 7221e24 into ggml-org:master Aug 19, 2026
20 of 28 checks passed

Uh oh!

Conversation

gabe-l-hart commented Jul 9, 2026

Overview

Additional information

Requirements

AI Disclosure

Uh oh!

pwilkin commented Jul 9, 2026

Uh oh!

gabe-l-hart commented Jul 9, 2026

Uh oh!

gabe-l-hart commented Jul 9, 2026

Uh oh!

gabe-l-hart commented Jul 14, 2026

Uh oh!

Uh oh!

gabe-l-hart Aug 12, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

pwilkin left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

CISC left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

CISC left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

CISC commented Aug 19, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

gabe-l-hart commented Aug 19, 2026

Uh oh!

CISC commented Aug 19, 2026

Uh oh!

gabe-l-hart commented Aug 19, 2026

Uh oh!

CISC commented Aug 19, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

CISC commented Aug 19, 2026 •

edited

Loading