model: Muse Glimmer Support - #26841
Merged
Merged
Conversation
The pattern `self._set_vocab_gpt2()` seems preferred throughout the codebase, and it allows `set_vocab()` to be called from a different part of the Python class hierarchy: the drafter model converter that we may need eventually.
Another option would be to store it in the gguf file itself.
Note: some fields to be renamed after the implementation works. We are keeping compatibility with the reference Meta gguf for testing purposes.
Otherwise our generations are worse. Transformers does not use them. We need to trace inputs to verify whether they are equivalent.
lol, forgot from a previous commit
Co-authored-by: Young Han <younghan@fb.com>
common/chat.cpp on this branch has no Onyx handling, so a converted model
serves malformed chat: the assistant preamble leaks into content
("to=self<|message|>...") and tool calls fail with
HTTP 500 "The model produced output that does not match the expected
peg-native format"
common_chat_params_init_onyx exists on onyx-fair-patch, added there by
8bb73dd3d. It was never on this branch, so this is not a regression --
the two lines developed independently.
The code here is taken verbatim from that commit. It is the clean side of
`git merge origin/onyx-fair-patch`: chat.cpp is one of the files that
merges without conflict. The full merge is not viable -- it produces 13
conflicts, including add/add on conversion/onyx.py and src/models/onyx.cpp
where the q_norm-folding and metadata-scale approaches contradict each
other, and ggml-org#4/ggml-org#7 are stacked on this branch's side of that.
Verified on this branch: builds with 0 errors, converts an Onyx checkpoint,
and serving it gives "4" for "What is 2+2?" plus a correct
get_weather {"city":"Paris"} tool call, where the unported branch gives the
two failures above.
No converter or runtime changes are included, so this should not interact
with the q_norm work.
Co-authored-by: Beto de Paola <betodepaola@meta.com>
crusaderky
pushed a commit
to crusaderky/llama.cpp
that referenced
this pull request
Aug 10, 2026
* Get started with Onyx * Add architecture * Skip keys handled in super() * Loading tensors * Shorten * Graph * Apply suggestion from @pcuenca * Remove norm now embedding in transformers weights * Add eot * Explicit output_multiplier * Handle post_norm_eps * No super call; unhardcode eot. The pattern `self._set_vocab_gpt2()` seems preferred throughout the codebase, and it allows `set_vocab()` to be called from a different part of the Python class hierarchy: the drafter model converter that we may need eventually. * Register for drafting * DFlash: inherit rope type from the linked target. Another option would be to store it in the gguf file itself. * mmproj conversion Note: some fields to be renamed after the implementation works. We are keeping compatibility with the reference Meta gguf for testing purposes. * "clip" header declarations * Load mmproj * Pre-processing * Graph * Go back to using delimiters. Otherwise our generations are worse. Transformers does not use them. We need to trace inputs to verify whether they are equivalent. * downsample_factor -> merge_size * Add vision graph lol, forgot from a previous commit * Additional renames, align with llama.cpp / transformers * Prefer _size instead of independent _h and _w * Fix token layout Co-authored-by: Young Han <younghan@fb.com> * onyx: bring the chat parser onto the onyx branch common/chat.cpp on this branch has no Onyx handling, so a converted model serves malformed chat: the assistant preamble leaks into content ("to=self<|message|>...") and tool calls fail with HTTP 500 "The model produced output that does not match the expected peg-native format" common_chat_params_init_onyx exists on onyx-fair-patch, added there by 8bb73dd3d. It was never on this branch, so this is not a regression -- the two lines developed independently. The code here is taken verbatim from that commit. It is the clean side of `git merge origin/onyx-fair-patch`: chat.cpp is one of the files that merges without conflict. The full merge is not viable -- it produces 13 conflicts, including add/add on conversion/onyx.py and src/models/onyx.cpp where the q_norm-folding and metadata-scale approaches contradict each other, and TheTom#4/TheTom#7 are stacked on this branch's side of that. Verified on this branch: builds with 0 errors, converts an Onyx checkpoint, and serving it gives "4" for "What is 2+2?" plus a correct get_weather {"city":"Paris"} tool call, where the unported branch gives the two failures above. No converter or runtime changes are included, so this should not interact with the q_norm work. Co-authored-by: Beto de Paola <betodepaola@meta.com> * Less params, bilinear pos-emb interpolation as a graph op instead of CPU * Map to symbolic V_MMPROJ instead of strings * Make a couple params explicit * Patchify via build_inp() * No param for rope_theta * Small cleanup * Restore blank line * Unpermute, to adapt to the latest transformers checkpoint * Apply norm after token embeddings This follows the latest transformers approach. * Remove duplicated function * build_vit * onyx: use the model rope theta on sliding-window layers * DFlash: conversion from transformers drafter * Revert rope_type derivation from target NOTE: this breaks compatibility with Meta's distributed DFlash GGUFs, as the Q/K are stored in "NEOX" (rotated half) format, like in transformers. * Apply suggestion from @pcuenca * Set model type * Remove comment that will become obsolete * Hardcode post_norm_rms_eps instead of new param * Derive SWA+RoPE pattern from gguf array or scalar * Fix model type <-> number of layers * Reorder * Rename * Fix typo * DFlash: seed the draft KV cache from multimodal embedding batches `common_speculative_impl_draft_dflash::process()` returned early on any batch carrying embeddings, so an image prefill never had its target-layer features fused through the DFlash encoder and injected into the draft's KV cache. That left a hole spanning the image's positions, and the next injection at a post-image position failed to initialize its batch: ``` decoding image batch 1/1, n_tokens_batch = 256 decode: failed to initialize batch llama_decode: failed to decode, ret = -1 process: llama_decode(ctx_dft) failed rc=-1 (n_tokens=17, offset=0) srv decode: failed to process speculative batch ``` Every image request with `--spec-type draft-dflash` failed with HTTP 500. Text-only was unaffected, since those batches carry token ids and were let through. Restore the earlier condition, which admits a batch that is either tokens or embeddings and skips only the degenerate neither/both cases. The rest of `process()` is already layout-agnostic -- it gathers features via `llama_get_embeddings_layer_inp()` and indexes `batch_in.pos[]` / `batch_in.seq_id[]`, none of which assume token ids -- so this is the whole fix. Validated against `muse-glimmer-30B-bf16.gguf` + `mmproj-muse-glimmer-30B-bf16.gguf` + a DFlash draft head, on an image describe-the-shapes request: - before: HTTP 500, `failed to process speculative batch` - after: HTTP 200, draft acceptance 0.34012 (167 accepted / 491 generated), mean len 3.04 Output equivalence holds, which is the property that matters: at temperature 0 the drafted response is byte-identical to the same request served with no draft attached (1213/1213 chars), so the draft is drafting correctly through the image context rather than merely not crashing. * Conversion: prefer rewrite to mapping * Revert "Conversion: prefer rewrite to mapping" This reverts commit a92d0ac. * fix lint * sliding_window metadata is not optional * disable state save/load * Apply suggestion from @pcuenca --------- Co-authored-by: Young Han <younghan@fb.com> Co-authored-by: Beto de Paola <betodepaola@meta.com> Co-authored-by: Daniel Han <michaelhan2050@gmail.com> Co-authored-by: ruanrms <ruanslv@gmail.com> Co-authored-by: Xuan Son Nguyen <son@huggingface.co> Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@huggingface.co>
crusaderky
pushed a commit
to crusaderky/llama.cpp
that referenced
this pull request
Aug 10, 2026
* Get started with Onyx * Add architecture * Skip keys handled in super() * Loading tensors * Shorten * Graph * Apply suggestion from @pcuenca * Remove norm now embedding in transformers weights * Add eot * Explicit output_multiplier * Handle post_norm_eps * No super call; unhardcode eot. The pattern `self._set_vocab_gpt2()` seems preferred throughout the codebase, and it allows `set_vocab()` to be called from a different part of the Python class hierarchy: the drafter model converter that we may need eventually. * Register for drafting * DFlash: inherit rope type from the linked target. Another option would be to store it in the gguf file itself. * mmproj conversion Note: some fields to be renamed after the implementation works. We are keeping compatibility with the reference Meta gguf for testing purposes. * "clip" header declarations * Load mmproj * Pre-processing * Graph * Go back to using delimiters. Otherwise our generations are worse. Transformers does not use them. We need to trace inputs to verify whether they are equivalent. * downsample_factor -> merge_size * Add vision graph lol, forgot from a previous commit * Additional renames, align with llama.cpp / transformers * Prefer _size instead of independent _h and _w * Fix token layout Co-authored-by: Young Han <younghan@fb.com> * onyx: bring the chat parser onto the onyx branch common/chat.cpp on this branch has no Onyx handling, so a converted model serves malformed chat: the assistant preamble leaks into content ("to=self<|message|>...") and tool calls fail with HTTP 500 "The model produced output that does not match the expected peg-native format" common_chat_params_init_onyx exists on onyx-fair-patch, added there by 8bb73dd3d. It was never on this branch, so this is not a regression -- the two lines developed independently. The code here is taken verbatim from that commit. It is the clean side of `git merge origin/onyx-fair-patch`: chat.cpp is one of the files that merges without conflict. The full merge is not viable -- it produces 13 conflicts, including add/add on conversion/onyx.py and src/models/onyx.cpp where the q_norm-folding and metadata-scale approaches contradict each other, and TheTom#4/TheTom#7 are stacked on this branch's side of that. Verified on this branch: builds with 0 errors, converts an Onyx checkpoint, and serving it gives "4" for "What is 2+2?" plus a correct get_weather {"city":"Paris"} tool call, where the unported branch gives the two failures above. No converter or runtime changes are included, so this should not interact with the q_norm work. Co-authored-by: Beto de Paola <betodepaola@meta.com> * Less params, bilinear pos-emb interpolation as a graph op instead of CPU * Map to symbolic V_MMPROJ instead of strings * Make a couple params explicit * Patchify via build_inp() * No param for rope_theta * Small cleanup * Restore blank line * Unpermute, to adapt to the latest transformers checkpoint * Apply norm after token embeddings This follows the latest transformers approach. * Remove duplicated function * build_vit * onyx: use the model rope theta on sliding-window layers * DFlash: conversion from transformers drafter * Revert rope_type derivation from target NOTE: this breaks compatibility with Meta's distributed DFlash GGUFs, as the Q/K are stored in "NEOX" (rotated half) format, like in transformers. * Apply suggestion from @pcuenca * Set model type * Remove comment that will become obsolete * Hardcode post_norm_rms_eps instead of new param * Derive SWA+RoPE pattern from gguf array or scalar * Fix model type <-> number of layers * Reorder * Rename * Fix typo * DFlash: seed the draft KV cache from multimodal embedding batches `common_speculative_impl_draft_dflash::process()` returned early on any batch carrying embeddings, so an image prefill never had its target-layer features fused through the DFlash encoder and injected into the draft's KV cache. That left a hole spanning the image's positions, and the next injection at a post-image position failed to initialize its batch: ``` decoding image batch 1/1, n_tokens_batch = 256 decode: failed to initialize batch llama_decode: failed to decode, ret = -1 process: llama_decode(ctx_dft) failed rc=-1 (n_tokens=17, offset=0) srv decode: failed to process speculative batch ``` Every image request with `--spec-type draft-dflash` failed with HTTP 500. Text-only was unaffected, since those batches carry token ids and were let through. Restore the earlier condition, which admits a batch that is either tokens or embeddings and skips only the degenerate neither/both cases. The rest of `process()` is already layout-agnostic -- it gathers features via `llama_get_embeddings_layer_inp()` and indexes `batch_in.pos[]` / `batch_in.seq_id[]`, none of which assume token ids -- so this is the whole fix. Validated against `muse-glimmer-30B-bf16.gguf` + `mmproj-muse-glimmer-30B-bf16.gguf` + a DFlash draft head, on an image describe-the-shapes request: - before: HTTP 500, `failed to process speculative batch` - after: HTTP 200, draft acceptance 0.34012 (167 accepted / 491 generated), mean len 3.04 Output equivalence holds, which is the property that matters: at temperature 0 the drafted response is byte-identical to the same request served with no draft attached (1213/1213 chars), so the draft is drafting correctly through the image context rather than merely not crashing. * Conversion: prefer rewrite to mapping * Revert "Conversion: prefer rewrite to mapping" This reverts commit a92d0ac. * fix lint * sliding_window metadata is not optional * disable state save/load * Apply suggestion from @pcuenca --------- Co-authored-by: Young Han <younghan@fb.com> Co-authored-by: Beto de Paola <betodepaola@meta.com> Co-authored-by: Daniel Han <michaelhan2050@gmail.com> Co-authored-by: ruanrms <ruanslv@gmail.com> Co-authored-by: Xuan Son Nguyen <son@huggingface.co> Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@huggingface.co>
TheTom
added a commit
to TheTom/llama-cpp-turboquant
that referenced
this pull request
Aug 10, 2026
* Get started with Onyx * Add architecture * Skip keys handled in super() * Loading tensors * Shorten * Graph * Apply suggestion from @pcuenca * Remove norm now embedding in transformers weights * Add eot * Explicit output_multiplier * Handle post_norm_eps * No super call; unhardcode eot. The pattern `self._set_vocab_gpt2()` seems preferred throughout the codebase, and it allows `set_vocab()` to be called from a different part of the Python class hierarchy: the drafter model converter that we may need eventually. * Register for drafting * DFlash: inherit rope type from the linked target. Another option would be to store it in the gguf file itself. * mmproj conversion Note: some fields to be renamed after the implementation works. We are keeping compatibility with the reference Meta gguf for testing purposes. * "clip" header declarations * Load mmproj * Pre-processing * Graph * Go back to using delimiters. Otherwise our generations are worse. Transformers does not use them. We need to trace inputs to verify whether they are equivalent. * downsample_factor -> merge_size * Add vision graph lol, forgot from a previous commit * Additional renames, align with llama.cpp / transformers * Prefer _size instead of independent _h and _w * Fix token layout * onyx: bring the chat parser onto the onyx branch common/chat.cpp on this branch has no Onyx handling, so a converted model serves malformed chat: the assistant preamble leaks into content ("to=self<|message|>...") and tool calls fail with HTTP 500 "The model produced output that does not match the expected peg-native format" common_chat_params_init_onyx exists on onyx-fair-patch, added there by 8bb73dd3d. It was never on this branch, so this is not a regression -- the two lines developed independently. The code here is taken verbatim from that commit. It is the clean side of `git merge origin/onyx-fair-patch`: chat.cpp is one of the files that merges without conflict. The full merge is not viable -- it produces 13 conflicts, including add/add on conversion/onyx.py and src/models/onyx.cpp where the q_norm-folding and metadata-scale approaches contradict each other, and #4/#7 are stacked on this branch's side of that. Verified on this branch: builds with 0 errors, converts an Onyx checkpoint, and serving it gives "4" for "What is 2+2?" plus a correct get_weather {"city":"Paris"} tool call, where the unported branch gives the two failures above. No converter or runtime changes are included, so this should not interact with the q_norm work. * Less params, bilinear pos-emb interpolation as a graph op instead of CPU * Map to symbolic V_MMPROJ instead of strings * Make a couple params explicit * Patchify via build_inp() * No param for rope_theta * Small cleanup * Restore blank line * Unpermute, to adapt to the latest transformers checkpoint * Apply norm after token embeddings This follows the latest transformers approach. * Remove duplicated function * build_vit * onyx: use the model rope theta on sliding-window layers * DFlash: conversion from transformers drafter * Revert rope_type derivation from target NOTE: this breaks compatibility with Meta's distributed DFlash GGUFs, as the Q/K are stored in "NEOX" (rotated half) format, like in transformers. * Apply suggestion from @pcuenca * Set model type * Remove comment that will become obsolete * Hardcode post_norm_rms_eps instead of new param * Derive SWA+RoPE pattern from gguf array or scalar * Fix model type <-> number of layers * Reorder * Rename * Fix typo * DFlash: seed the draft KV cache from multimodal embedding batches `common_speculative_impl_draft_dflash::process()` returned early on any batch carrying embeddings, so an image prefill never had its target-layer features fused through the DFlash encoder and injected into the draft's KV cache. That left a hole spanning the image's positions, and the next injection at a post-image position failed to initialize its batch: ``` decoding image batch 1/1, n_tokens_batch = 256 decode: failed to initialize batch llama_decode: failed to decode, ret = -1 process: llama_decode(ctx_dft) failed rc=-1 (n_tokens=17, offset=0) srv decode: failed to process speculative batch ``` Every image request with `--spec-type draft-dflash` failed with HTTP 500. Text-only was unaffected, since those batches carry token ids and were let through. Restore the earlier condition, which admits a batch that is either tokens or embeddings and skips only the degenerate neither/both cases. The rest of `process()` is already layout-agnostic -- it gathers features via `llama_get_embeddings_layer_inp()` and indexes `batch_in.pos[]` / `batch_in.seq_id[]`, none of which assume token ids -- so this is the whole fix. Validated against `muse-glimmer-30B-bf16.gguf` + `mmproj-muse-glimmer-30B-bf16.gguf` + a DFlash draft head, on an image describe-the-shapes request: - before: HTTP 500, `failed to process speculative batch` - after: HTTP 200, draft acceptance 0.34012 (167 accepted / 491 generated), mean len 3.04 Output equivalence holds, which is the property that matters: at temperature 0 the drafted response is byte-identical to the same request served with no draft attached (1213/1213 chars), so the draft is drafting correctly through the image context rather than merely not crashing. * Conversion: prefer rewrite to mapping * Revert "Conversion: prefer rewrite to mapping" This reverts commit a92d0ac. * fix lint * sliding_window metadata is not optional * disable state save/load * Apply suggestion from @pcuenca --------- Co-authored-by: Pedro Cuenca <pedro@huggingface.co> Co-authored-by: Young Han <younghan@fb.com> Co-authored-by: Beto de Paola <betodepaola@meta.com> Co-authored-by: Daniel Han <michaelhan2050@gmail.com> Co-authored-by: ruanrms <ruanslv@gmail.com> Co-authored-by: Xuan Son Nguyen <son@huggingface.co> Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@huggingface.co>
wanghqc
added a commit
to qualcomm/llama.cpp
that referenced
this pull request
Aug 10, 2026
28 upstream commits since 687e778. One touches ggml-opencl: 689e227 (ggml-org#26428, FA prefill K-tile transpose) -- our own PR, merged upstream this window, so its four files conflicted against the x2ue originals they were carved from. Resolved to ours in all four: x2ue is a strict superset (FA_Q_HALF / FA_O_HALF, the FA*_PROBE_NO_LDS diagnostics and FA_V_LDS_T all landed on top). Verified line-by-line that no ggml-org#26428 content was lost -- the seven upstream lines absent from the merged tree are the same code reshaped by our later commits (#elif rather than #if because our probe branch precedes, mad() rather than *, Q_PRIV_TO_ACC4() around q_priv), all still reading through FA_LK/FA_LK_PAIR, plus one reworded comment. Also verified the automerge kept upstream intact: of the 659 files upstream touched, the merged tree is byte-identical to upstream/master on all but the eight we also modified, and all 247 upstream-added lines in the four auto-merged files (clip.cpp, clip-model.h, llama-context.cpp, test-backend-ops.cpp -- Muse Glimmer ggml-org#26841, multi-output sampling ggml-org#25532, Granite-Switch ggml-org#25107) are present.
alpaca-matrix
added a commit
to alpaca-matrix/llama.cpp
that referenced
this pull request
Aug 11, 2026
…e Ornith requant Three unrelated candidates.md updates, all dated 2026-08-11. Ling-3.0-flash, re-checked at the 2-4 week mark section 1 set for itself. Verdict holds but the reasons moved: the multi-argument tool-calling bug is fixed (PR ggml-org#26608, 2026-08-07, with regression tests), while the Vulkan freezes turn out to be reported on AMD Strix Halo - gfx1151, the same radv/amdgpu path this box runs on gfx1103. That was filed as generic risk and is not generic; it is now the binding blocker, made worse by MTP still being impossible to disable, which removes the first diagnostic step this box would take. Also new: every GGUF published before 2026-08-07 is missing the trained per-layer SwiGLU clamp metadata, so a stale download would silently misattribute quality to the quant tier. Still do not download. Muse Glimmer 30B, desk-rejected on throughput. Arch is merged upstream (PR ggml-org#26841, day-0) so it would only cost a rebase rather than a fork build, and almost everything else fits: 13 of 52 layers carry a growing KV, ~21 GiB resident with mmproj and drafter, a pre-quantized DFlash drafter that avoids the Laguna BF16 trap, and benchmarks that clear the deep slot's harder-discriminator bar. But config.json has no expert fields - it is a 28B dense decoder, so the whole 16.8 GB file is read every token. Ceiling 4.4 t/s, ~3.7-4.0 base against this box's measured dense datapoint, and ~6-7 t/s served taking Meta's Apple Silicon speculation figure rather than its 5090 one. Fourth dense rejection here. Recorded rather than dropped because it is the cheapest experiment this file has proposed, if the dense penalty is ever worth measuring instead of extrapolating. Requantizing Ornith, measured: a closed avenue. The deployed APEX I-Quality file is 0.26% from its own BF16 weights, which bounds every future requant of this model - quantization is not a quality lever on fast. The Laguna byte-budget play returned +2.0% of a predicted +7.5% and gave away 9.6% of prefill at depth, so IQ4_XS does not transfer from ordinary attention tensors to a GDN hybrid's projections. Includes a correction: fast is 10 of 40 full-attention at interval 4, not 41 of 41, which moves its arithmetic to the 60.5 GB/s hybrid constant and undercuts the desk-rejection of Qwen3.6-35B-A3B vanilla. Assisted-by: Claude Opus 5
danielhanchen
added a commit
to unslothai/llama.cpp
that referenced
this pull request
Aug 11, 2026
…he base (#90) Upstream squash-merged ggml-org#26841 as 62bf73d, which is an ancestor of b10356. The pinned commit 56b32b3 is not, so resolve re-applies the same Muse Glimmer code the base already carries and additive_merge.py correctly refuses it as one change made twice. The nightly stops there. Replayed the resolve merge locally on b10356 with the pin still in place: refused common/speculative.cpp: merge base is not empty, so at least one side edited existing text refused conversion/muse_glimmer.py: both sides add the same line(s), so this is one change made twice refused src/models/muse-glimmer.cpp: both sides add the same line(s) ggml-org#26841 (56b32b3) does not merge cleanly onto b10356 + the PRs listed before it Nothing is lost by dropping it. b10356 carries Muse Glimmer at upstream's final state, which is two lines ahead of the pinned commit, not behind. The _doc said a merged pin becomes an empty no-op. That only holds when upstream takes the PR as a merge commit, and upstream usually squashes, so the note now says to delete the entry once a base tag carries the work.
mlogix
added a commit
to SynoriAI/llama.cpp
that referenced
this pull request
Aug 11, 2026
…, multi-output backend sampling, pocket-tts Merges 45 upstream commits (0865990..ebb546b, b10362-16-gebb546b7e) into synori/llama-update-mtp-fit. Zero conflicts; all vendored patches carried over untouched. Primary motivation — new Meta architecture: * 62bf73d model: Muse Glimmer Support (ggml-org#26841) LLM_ARCH_MUSE_GLIMMER + src/models/muse-glimmer.cpp + the mtmd vision tower in tools/mtmd/models/muse-glimmer.cpp and conversion/muse_glimmer.py. Other notable changes that touch our public API surface: * dd1ea52 llama : support multi-output backend sampling (ggml-org#25532) llama_context_params gains n_outputs_max_per_seq; llama_sampler_i.backend_init takes it as a third argument; new backend_reset / copy_state vtable slots and llama_sampler_copy(). * 153d324 llama : default load-mode auto, avoids mmap on iGPUs (ggml-org#26081) llama_load_mode gains LLAMA_LOAD_MODE_AUTO = -1 (enum is now signed). * 6e62ba5 mtmd: support pocket-tts (ggml-org#26871) mtmd_gen_inp/mtmd_gen_out gain seed/temp/feats/is_eos; new mtmd_gen_inp_default(); mtmd_helper_gen_audio_step_gen() takes out_stop. * 157b81f model : Granite-Switch Architecture (ggml-org#25107) * 7a20b41 model: MTP support for Nemotron (ggml-org#26725) and cc078b4 Dflash support for nemotron-3.5 (ggml-org#26905) * e23e944 vendor : cpp-httplib 0.53.0, 4c6766f vendor : subprocess.h sync Vendored patches preserved: * 919fde3 feat(rpc): thread-local last_error accessor — intact, upstream touched ggml-rpc.cpp by one unrelated line. * 3679b23 Fixes (RPC) — intact. * 2aa76c7 fix(metal): drop stray kernel_pad_f32 — still applies; upstream has since refactored pad into a templated kernel_pad_impl<T>, so the duplicate definition that referenced the nonexistent kargs_pad.s0..s3 is gone on both sides and nothing had to be re-applied.
marchah
added a commit
to marchah/Proxmox
that referenced
this pull request
Aug 11, 2026
* chore: bump llama.cpp b10308 -> b10361 on both V620 containers Driver: Muse Glimmer's architecture (muse-glimmer, ggml-org/llama.cpp#26841) is absent from b10308, which rejects the model with "unknown model architecture". b10358 was the first release containing it; b10361 was newest at bump time. Tag + tarball SHA-256 bumped together in both scripts. The SHA (f98377a3a3ae7a541894aa512ab0610fdf6d1735a1ad13d9474e67b80d130e9a) was verified three ways: captured at install, re-downloaded independently from a different machine, and cross-checked by hashing the installed llama-server binary on BOTH containers - identical, so the two independent downloads produced the same artifact. Live state after the bump: - CT 120: current -> b10361, fingerprint b10361-14e78ddef, 77.7 tok/s, and --reasoning off still in effect (0 reasoning chars) - the flag survived. - CT 123: current -> b10361, all 8 models resolve. The temporary per-entry LLAMACPP_DIR override on the muse-glimmer entry is REMOVED, since the shared build now suffices; its stale comment was rewritten too. - b10308 kept on both containers for rollback (re-point 'current' + restart). TWO STANDING TODOS RE-CHECKED AT THIS BUMP - BOTH STILL OPEN: 1. --cache-ram 0 on CT 120 STAYS. The prompt-cache tickets are all still open: #25755 (Qwen3.6 + Vulkan, tensor-write-out-of-bounds crash in server_prompt_cache::load), #26207 and #26529. Worth noting #25755 cites #13194, where a contributor observes Qwen 3.6 models "consistently fail to reuse cache" while Gemma 4 works - i.e. a Qwen3.6-specific weakness in the SWA/checkpoint path rather than a generic bug, which makes a near-term fix less likely. #26529 independently arrives at --cache-ram 0 as the workaround. 2. The DFlash n>=8 cliff did NOT lift. Re-ran the sweep on b10361: n=4 43.0, n=6 45.1, n=8 17.6, n=16 18.7 - within noise of the b10308 numbers. So it is not a transient upstream bug that this bump fixed. CLAUDE.md now records that, so the next person does not re-run the same experiment expecting a change. n-max stays 4. * docs: record what CT 123 actually serves, incl. Muse Glimmer's quant + required settings CLAUDE.md described GPU 2 as hot-swapping "a coder model and a reviewer model". It serves EIGHT. Nothing about Muse Glimmer existed anywhere in this repo - the model, its quant, its drafter and its three required settings lived only in the knowledge base, so anyone reading this repo to understand the box got a stale picture and none of the gotchas. Adds: - The full live model list (8 aliases, grouped by role) with an explicit warning that they are live-only in /etc/llama-swap/config.yaml and a rebuild from the script yields NONE of them. That was already the convention - it just was not written down, and the two-model prose implied otherwise. - The deployed Muse Glimmer quant: Meta's own muse-glimmer-30B-kquant-dynamic.gguf (19.65 GB) plus Meta's dflash-kquant.gguf drafter (1.63 GB). Records WHY that build over Unsloth's wider ladder: it is the only Muse Glimmer quant with a published degradation figure (Meta: 0.2% average across 15 benchmarks; their 17 GB build is 1.0%). Unsloth publishes no accuracy numbers for any of its Muse quants, so switching would trade a measured build for an unmeasured one. Also notes the label is misleading: Meta calls it "4-bit" but the effective rate is ~5.64 bpw (derived from the BF16 total: 55.73 GB / 16 bpw = 27.86B params). It is mixed-precision, which is what makes 0.2% plausible. - The three required settings, each of which produced broken output before being found: * --reasoning-format auto (with `none`, the wrapper default, the model's channel format leaks raw into content) * --spec-draft-n-max 3 - the measured optimum FOR THIS DRAFTER (2 -> 37.4, 3 -> 41.8, 4 -> 41.0, 6 -> 39.4). qwen3.6-27b-dflash's optimum is 4, so the value does not transfer; sweep any new drafter. * a generous client max_tokens - at 300 the reply returns with content COMPLETELY EMPTY and all of it in reasoning_content. A low cap yields empty responses, not errors, which is easy to misdiagnose as a broken model. - The speed context: 18.8 tok/s bare, 41.8 with DFlash (2.2x, reproducing Meta's own 2.21x claim), against 63.1 for the 3B-active MoE unaccelerated. - That DFlash and vision are mutually exclusive upstream (llama.cpp #26108, open), which is why the mmproj projector is not deployed. --------- Co-authored-by: Marchah <hugo@manifestfinancial.com>
Nathanw1014
pushed a commit
to Nathanw1014/llama.cpp
that referenced
this pull request
Aug 12, 2026
* Get started with Onyx * Add architecture * Skip keys handled in super() * Loading tensors * Shorten * Graph * Apply suggestion from @pcuenca * Remove norm now embedding in transformers weights * Add eot * Explicit output_multiplier * Handle post_norm_eps * No super call; unhardcode eot. The pattern `self._set_vocab_gpt2()` seems preferred throughout the codebase, and it allows `set_vocab()` to be called from a different part of the Python class hierarchy: the drafter model converter that we may need eventually. * Register for drafting * DFlash: inherit rope type from the linked target. Another option would be to store it in the gguf file itself. * mmproj conversion Note: some fields to be renamed after the implementation works. We are keeping compatibility with the reference Meta gguf for testing purposes. * "clip" header declarations * Load mmproj * Pre-processing * Graph * Go back to using delimiters. Otherwise our generations are worse. Transformers does not use them. We need to trace inputs to verify whether they are equivalent. * downsample_factor -> merge_size * Add vision graph lol, forgot from a previous commit * Additional renames, align with llama.cpp / transformers * Prefer _size instead of independent _h and _w * Fix token layout Co-authored-by: Young Han <younghan@fb.com> * onyx: bring the chat parser onto the onyx branch common/chat.cpp on this branch has no Onyx handling, so a converted model serves malformed chat: the assistant preamble leaks into content ("to=self<|message|>...") and tool calls fail with HTTP 500 "The model produced output that does not match the expected peg-native format" common_chat_params_init_onyx exists on onyx-fair-patch, added there by 8bb73dd3d. It was never on this branch, so this is not a regression -- the two lines developed independently. The code here is taken verbatim from that commit. It is the clean side of `git merge origin/onyx-fair-patch`: chat.cpp is one of the files that merges without conflict. The full merge is not viable -- it produces 13 conflicts, including add/add on conversion/onyx.py and src/models/onyx.cpp where the q_norm-folding and metadata-scale approaches contradict each other, and ggml-org#4/ggml-org#7 are stacked on this branch's side of that. Verified on this branch: builds with 0 errors, converts an Onyx checkpoint, and serving it gives "4" for "What is 2+2?" plus a correct get_weather {"city":"Paris"} tool call, where the unported branch gives the two failures above. No converter or runtime changes are included, so this should not interact with the q_norm work. Co-authored-by: Beto de Paola <betodepaola@meta.com> * Less params, bilinear pos-emb interpolation as a graph op instead of CPU * Map to symbolic V_MMPROJ instead of strings * Make a couple params explicit * Patchify via build_inp() * No param for rope_theta * Small cleanup * Restore blank line * Unpermute, to adapt to the latest transformers checkpoint * Apply norm after token embeddings This follows the latest transformers approach. * Remove duplicated function * build_vit * onyx: use the model rope theta on sliding-window layers * DFlash: conversion from transformers drafter * Revert rope_type derivation from target NOTE: this breaks compatibility with Meta's distributed DFlash GGUFs, as the Q/K are stored in "NEOX" (rotated half) format, like in transformers. * Apply suggestion from @pcuenca * Set model type * Remove comment that will become obsolete * Hardcode post_norm_rms_eps instead of new param * Derive SWA+RoPE pattern from gguf array or scalar * Fix model type <-> number of layers * Reorder * Rename * Fix typo * DFlash: seed the draft KV cache from multimodal embedding batches `common_speculative_impl_draft_dflash::process()` returned early on any batch carrying embeddings, so an image prefill never had its target-layer features fused through the DFlash encoder and injected into the draft's KV cache. That left a hole spanning the image's positions, and the next injection at a post-image position failed to initialize its batch: ``` decoding image batch 1/1, n_tokens_batch = 256 decode: failed to initialize batch llama_decode: failed to decode, ret = -1 process: llama_decode(ctx_dft) failed rc=-1 (n_tokens=17, offset=0) srv decode: failed to process speculative batch ``` Every image request with `--spec-type draft-dflash` failed with HTTP 500. Text-only was unaffected, since those batches carry token ids and were let through. Restore the earlier condition, which admits a batch that is either tokens or embeddings and skips only the degenerate neither/both cases. The rest of `process()` is already layout-agnostic -- it gathers features via `llama_get_embeddings_layer_inp()` and indexes `batch_in.pos[]` / `batch_in.seq_id[]`, none of which assume token ids -- so this is the whole fix. Validated against `muse-glimmer-30B-bf16.gguf` + `mmproj-muse-glimmer-30B-bf16.gguf` + a DFlash draft head, on an image describe-the-shapes request: - before: HTTP 500, `failed to process speculative batch` - after: HTTP 200, draft acceptance 0.34012 (167 accepted / 491 generated), mean len 3.04 Output equivalence holds, which is the property that matters: at temperature 0 the drafted response is byte-identical to the same request served with no draft attached (1213/1213 chars), so the draft is drafting correctly through the image context rather than merely not crashing. * Conversion: prefer rewrite to mapping * Revert "Conversion: prefer rewrite to mapping" This reverts commit a92d0ac. * fix lint * sliding_window metadata is not optional * disable state save/load * Apply suggestion from @pcuenca --------- Co-authored-by: Young Han <younghan@fb.com> Co-authored-by: Beto de Paola <betodepaola@meta.com> Co-authored-by: Daniel Han <michaelhan2050@gmail.com> Co-authored-by: ruanrms <ruanslv@gmail.com> Co-authored-by: Xuan Son Nguyen <son@huggingface.co> Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@huggingface.co> (cherry picked from commit 62bf73d)
crusaderky
pushed a commit
to crusaderky/llama.cpp
that referenced
this pull request
Aug 12, 2026
* Get started with Onyx * Add architecture * Skip keys handled in super() * Loading tensors * Shorten * Graph * Apply suggestion from @pcuenca * Remove norm now embedding in transformers weights * Add eot * Explicit output_multiplier * Handle post_norm_eps * No super call; unhardcode eot. The pattern `self._set_vocab_gpt2()` seems preferred throughout the codebase, and it allows `set_vocab()` to be called from a different part of the Python class hierarchy: the drafter model converter that we may need eventually. * Register for drafting * DFlash: inherit rope type from the linked target. Another option would be to store it in the gguf file itself. * mmproj conversion Note: some fields to be renamed after the implementation works. We are keeping compatibility with the reference Meta gguf for testing purposes. * "clip" header declarations * Load mmproj * Pre-processing * Graph * Go back to using delimiters. Otherwise our generations are worse. Transformers does not use them. We need to trace inputs to verify whether they are equivalent. * downsample_factor -> merge_size * Add vision graph lol, forgot from a previous commit * Additional renames, align with llama.cpp / transformers * Prefer _size instead of independent _h and _w * Fix token layout Co-authored-by: Young Han <younghan@fb.com> * onyx: bring the chat parser onto the onyx branch common/chat.cpp on this branch has no Onyx handling, so a converted model serves malformed chat: the assistant preamble leaks into content ("to=self<|message|>...") and tool calls fail with HTTP 500 "The model produced output that does not match the expected peg-native format" common_chat_params_init_onyx exists on onyx-fair-patch, added there by 8bb73dd3d. It was never on this branch, so this is not a regression -- the two lines developed independently. The code here is taken verbatim from that commit. It is the clean side of `git merge origin/onyx-fair-patch`: chat.cpp is one of the files that merges without conflict. The full merge is not viable -- it produces 13 conflicts, including add/add on conversion/onyx.py and src/models/onyx.cpp where the q_norm-folding and metadata-scale approaches contradict each other, and TheTom#4/TheTom#7 are stacked on this branch's side of that. Verified on this branch: builds with 0 errors, converts an Onyx checkpoint, and serving it gives "4" for "What is 2+2?" plus a correct get_weather {"city":"Paris"} tool call, where the unported branch gives the two failures above. No converter or runtime changes are included, so this should not interact with the q_norm work. Co-authored-by: Beto de Paola <betodepaola@meta.com> * Less params, bilinear pos-emb interpolation as a graph op instead of CPU * Map to symbolic V_MMPROJ instead of strings * Make a couple params explicit * Patchify via build_inp() * No param for rope_theta * Small cleanup * Restore blank line * Unpermute, to adapt to the latest transformers checkpoint * Apply norm after token embeddings This follows the latest transformers approach. * Remove duplicated function * build_vit * onyx: use the model rope theta on sliding-window layers * DFlash: conversion from transformers drafter * Revert rope_type derivation from target NOTE: this breaks compatibility with Meta's distributed DFlash GGUFs, as the Q/K are stored in "NEOX" (rotated half) format, like in transformers. * Apply suggestion from @pcuenca * Set model type * Remove comment that will become obsolete * Hardcode post_norm_rms_eps instead of new param * Derive SWA+RoPE pattern from gguf array or scalar * Fix model type <-> number of layers * Reorder * Rename * Fix typo * DFlash: seed the draft KV cache from multimodal embedding batches `common_speculative_impl_draft_dflash::process()` returned early on any batch carrying embeddings, so an image prefill never had its target-layer features fused through the DFlash encoder and injected into the draft's KV cache. That left a hole spanning the image's positions, and the next injection at a post-image position failed to initialize its batch: ``` decoding image batch 1/1, n_tokens_batch = 256 decode: failed to initialize batch llama_decode: failed to decode, ret = -1 process: llama_decode(ctx_dft) failed rc=-1 (n_tokens=17, offset=0) srv decode: failed to process speculative batch ``` Every image request with `--spec-type draft-dflash` failed with HTTP 500. Text-only was unaffected, since those batches carry token ids and were let through. Restore the earlier condition, which admits a batch that is either tokens or embeddings and skips only the degenerate neither/both cases. The rest of `process()` is already layout-agnostic -- it gathers features via `llama_get_embeddings_layer_inp()` and indexes `batch_in.pos[]` / `batch_in.seq_id[]`, none of which assume token ids -- so this is the whole fix. Validated against `muse-glimmer-30B-bf16.gguf` + `mmproj-muse-glimmer-30B-bf16.gguf` + a DFlash draft head, on an image describe-the-shapes request: - before: HTTP 500, `failed to process speculative batch` - after: HTTP 200, draft acceptance 0.34012 (167 accepted / 491 generated), mean len 3.04 Output equivalence holds, which is the property that matters: at temperature 0 the drafted response is byte-identical to the same request served with no draft attached (1213/1213 chars), so the draft is drafting correctly through the image context rather than merely not crashing. * Conversion: prefer rewrite to mapping * Revert "Conversion: prefer rewrite to mapping" This reverts commit a92d0ac. * fix lint * sliding_window metadata is not optional * disable state save/load * Apply suggestion from @pcuenca --------- Co-authored-by: Young Han <younghan@fb.com> Co-authored-by: Beto de Paola <betodepaola@meta.com> Co-authored-by: Daniel Han <michaelhan2050@gmail.com> Co-authored-by: ruanrms <ruanslv@gmail.com> Co-authored-by: Xuan Son Nguyen <son@huggingface.co> Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@huggingface.co>
huaxel
pushed a commit
to huaxel/CachyLLama
that referenced
this pull request
Aug 12, 2026
* Get started with Onyx * Add architecture * Skip keys handled in super() * Loading tensors * Shorten * Graph * Apply suggestion from @pcuenca * Remove norm now embedding in transformers weights * Add eot * Explicit output_multiplier * Handle post_norm_eps * No super call; unhardcode eot. The pattern `self._set_vocab_gpt2()` seems preferred throughout the codebase, and it allows `set_vocab()` to be called from a different part of the Python class hierarchy: the drafter model converter that we may need eventually. * Register for drafting * DFlash: inherit rope type from the linked target. Another option would be to store it in the gguf file itself. * mmproj conversion Note: some fields to be renamed after the implementation works. We are keeping compatibility with the reference Meta gguf for testing purposes. * "clip" header declarations * Load mmproj * Pre-processing * Graph * Go back to using delimiters. Otherwise our generations are worse. Transformers does not use them. We need to trace inputs to verify whether they are equivalent. * downsample_factor -> merge_size * Add vision graph lol, forgot from a previous commit * Additional renames, align with llama.cpp / transformers * Prefer _size instead of independent _h and _w * Fix token layout Co-authored-by: Young Han <younghan@fb.com> * onyx: bring the chat parser onto the onyx branch common/chat.cpp on this branch has no Onyx handling, so a converted model serves malformed chat: the assistant preamble leaks into content ("to=self<|message|>...") and tool calls fail with HTTP 500 "The model produced output that does not match the expected peg-native format" common_chat_params_init_onyx exists on onyx-fair-patch, added there by 8bb73dd3d. It was never on this branch, so this is not a regression -- the two lines developed independently. The code here is taken verbatim from that commit. It is the clean side of `git merge origin/onyx-fair-patch`: chat.cpp is one of the files that merges without conflict. The full merge is not viable -- it produces 13 conflicts, including add/add on conversion/onyx.py and src/models/onyx.cpp where the q_norm-folding and metadata-scale approaches contradict each other, and fewtarius#4/ggml-org#7 are stacked on this branch's side of that. Verified on this branch: builds with 0 errors, converts an Onyx checkpoint, and serving it gives "4" for "What is 2+2?" plus a correct get_weather {"city":"Paris"} tool call, where the unported branch gives the two failures above. No converter or runtime changes are included, so this should not interact with the q_norm work. Co-authored-by: Beto de Paola <betodepaola@meta.com> * Less params, bilinear pos-emb interpolation as a graph op instead of CPU * Map to symbolic V_MMPROJ instead of strings * Make a couple params explicit * Patchify via build_inp() * No param for rope_theta * Small cleanup * Restore blank line * Unpermute, to adapt to the latest transformers checkpoint * Apply norm after token embeddings This follows the latest transformers approach. * Remove duplicated function * build_vit * onyx: use the model rope theta on sliding-window layers * DFlash: conversion from transformers drafter * Revert rope_type derivation from target NOTE: this breaks compatibility with Meta's distributed DFlash GGUFs, as the Q/K are stored in "NEOX" (rotated half) format, like in transformers. * Apply suggestion from @pcuenca * Set model type * Remove comment that will become obsolete * Hardcode post_norm_rms_eps instead of new param * Derive SWA+RoPE pattern from gguf array or scalar * Fix model type <-> number of layers * Reorder * Rename * Fix typo * DFlash: seed the draft KV cache from multimodal embedding batches `common_speculative_impl_draft_dflash::process()` returned early on any batch carrying embeddings, so an image prefill never had its target-layer features fused through the DFlash encoder and injected into the draft's KV cache. That left a hole spanning the image's positions, and the next injection at a post-image position failed to initialize its batch: ``` decoding image batch 1/1, n_tokens_batch = 256 decode: failed to initialize batch llama_decode: failed to decode, ret = -1 process: llama_decode(ctx_dft) failed rc=-1 (n_tokens=17, offset=0) srv decode: failed to process speculative batch ``` Every image request with `--spec-type draft-dflash` failed with HTTP 500. Text-only was unaffected, since those batches carry token ids and were let through. Restore the earlier condition, which admits a batch that is either tokens or embeddings and skips only the degenerate neither/both cases. The rest of `process()` is already layout-agnostic -- it gathers features via `llama_get_embeddings_layer_inp()` and indexes `batch_in.pos[]` / `batch_in.seq_id[]`, none of which assume token ids -- so this is the whole fix. Validated against `muse-glimmer-30B-bf16.gguf` + `mmproj-muse-glimmer-30B-bf16.gguf` + a DFlash draft head, on an image describe-the-shapes request: - before: HTTP 500, `failed to process speculative batch` - after: HTTP 200, draft acceptance 0.34012 (167 accepted / 491 generated), mean len 3.04 Output equivalence holds, which is the property that matters: at temperature 0 the drafted response is byte-identical to the same request served with no draft attached (1213/1213 chars), so the draft is drafting correctly through the image context rather than merely not crashing. * Conversion: prefer rewrite to mapping * Revert "Conversion: prefer rewrite to mapping" This reverts commit a92d0ac. * fix lint * sliding_window metadata is not optional * disable state save/load * Apply suggestion from @pcuenca --------- Co-authored-by: Young Han <younghan@fb.com> Co-authored-by: Beto de Paola <betodepaola@meta.com> Co-authored-by: Daniel Han <michaelhan2050@gmail.com> Co-authored-by: ruanrms <ruanslv@gmail.com> Co-authored-by: Xuan Son Nguyen <son@huggingface.co> Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@huggingface.co>
junebug-junie
pushed a commit
to junebug-junie/Orion-Sapienform
that referenced
this pull request
Aug 13, 2026
…r's real drafter Corrects a gap in the previous commit: DFlash is Muse Glimmer's actual shipped drafter (HF README, verbatim: "Muse Glimmer ships with a lightweight 'drafter' model based on DFlash..."), not a hand-picked example. It's a real, merged upstream mechanism (ggml-org/llama.cpp#22105) needing --spec-type/--spec-draft-n-max, which app/main.py's CLI builder didn't emit. Also corrects a bigger gap the previous commit missed entirely: this model's llama-server support (target architecture + vision encoder + DFlash drafter, ALL of it, not just DFlash) only exists as of ggml-org/llama.cpp#26841 "model: Muse Glimmer Support", merged 2026-08-10. The stack's current LLAMACPP_IMAGE_TAG=server-cuda-b8740 pin (~April 2026) cannot load this model at all, on any code path. Previous commit's claim that "chat/tools/vision already work on the current pin" was wrong -- it verified the generic mmproj mechanism against other models, not architecture recognition for this one. app/profiles.py: adds LlamaCppConfig.spec_type (llama-server --spec-type; Literal of the values in docs/speculative.md as of the DFlash merge) and spec_draft_n_max (--spec-draft-n-max). app/main.py: build_llama_server_cmd_and_env now emits --spec-type + --spec-draft-n-max when spec_type is set, alongside --model-draft for the draft GGUF path. Classic --draft-min/--draft-max only apply when spec_type is unset (they don't mean anything for block-drafting types). Treats --spec-type support as a hard prerequisite when spec_type is requested: if the pinned binary doesn't advertise it, the draft model is skipped ENTIRELY rather than falling back to plain --model-draft, since a dflash-arch draft GGUF is not safely interpretable via the classic small-LM draft path on an older binary. config/llm_profiles.yaml: corrects the profile's own header/notes to document the real blocker (PR #26841, not just PR #22105), wires hf_draft_repo_id/hf_draft_filename/spec_type/spec_draft_n_max for dflash-kquant.gguf (spec_draft_n_max=16 matches the drafter's card- documented block_size), and flags an open upstream issue (ggml-org/llama.cpp#25116, bug-unconfirmed) about a GGUF arch-name/ key-scheme mismatch that has broken DFlash draft loading for a different model on the same mechanism -- not confirmed to affect this model's own dflash-kquant.gguf, but the failure mode is a hard load error, not silent degradation, so it will surface immediately on first boot if it does apply. Does NOT bump the repo-wide LLAMACPP_IMAGE_TAG default: that fallback is shared across every atlas-workers lane (chat/metacog/fast/agent), and config/biometrics/node_catalog.yaml confirms Circe already runs the chat lane continuously -- if Circe's workers share one compose env, bumping the shared default would silently rebuild that already-stable lane onto 3-day-old master with no bench pass. The profile documents the required LLAMACPP_IMAGE_TAG=server-cuda-b10398 as an explicit override, not a default change, and flags that Circe's actual compose/env layout needs confirming before deploy. Tests: two new cases replace the old "not wired" assertion -- (1) supporting build emits --model-draft/--spec-type/--spec-draft-n-max correctly and omits classic draft-min/draft-max; (2) old (b8740-era) build advertising --model-draft but not --spec-type skips the draft model entirely (no flags emitted at all) rather than falling back unsafely, and logs the reason. UNVERIFIED: no live boot has been run against server-cuda-b10398 from this environment (no Docker/GPU access here). Schema validation and CLI-flag-emission logic are covered by tests; actual model load (target arch + vision encoder + DFlash drafter, and whether upstream issue #25116 applies to this GGUF) is not verified and needs a real Circe smoke test before this profile is trusted in production.
shedrachokonofua
added a commit
to shedrachokonofua/aether
that referenced
this pull request
Aug 15, 2026
- llama.cpp b10423 for Muse Glimmer arch support (ggml-org/llama.cpp#26841). - llama-swap: muse-glimmer-30b (unsloth/Muse-Glimmer-30B-GGUF:Q8_0, 131k ctx, Meta default sampling); exclusive swap-in, outside the co-residency matrix. - litellm: aether/muse-glimmer-30b{,:think}; omp key allowlist.
RepnikovPavel
pushed a commit
to RepnikovPavel/llama.cpp
that referenced
this pull request
Aug 15, 2026
shedrachokonofua
added a commit
to shedrachokonofua/aether
that referenced
this pull request
Aug 16, 2026
- llama.cpp b10423 for Muse Glimmer arch support (ggml-org/llama.cpp#26841). - llama-swap: muse-glimmer-30b (unsloth/Muse-Glimmer-30B-GGUF:Q8_0, 131k ctx, Meta default sampling); exclusive swap-in, outside the co-residency matrix. - litellm: aether/muse-glimmer-30b{,:think}; omp key allowlist.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
8 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Overview
As discussed offline with team, details to be communicated later
Requirements
skills/add-new-model, human validated