Skip to content

feat(models): add Gemma 4 QAT local models - #1268

Merged
gabrielste1n merged 1 commit into
OpenWhispr:mainfrom
xAlcahest:feat/gemma4-qat-models
Jul 22, 2026
Merged

feat(models): add Gemma 4 QAT local models#1268
gabrielste1n merged 1 commit into
OpenWhispr:mainfrom
xAlcahest:feat/gemma4-qat-models

Conversation

@xAlcahest

Copy link
Copy Markdown
Contributor

Summary

Google published official QAT builds of the Gemma 4 GGUFs, smaller than the bartowski Q4_K_M quants we ship and closer in quality to the unquantized weights. This adds them as four new local registry entries (31B, 26B MoE, E4B, E2B) pointing at the ungated google/gemma-4-*-qat-q4_0-gguf repos, which the app's plain unauthenticated downloader can fetch. Repo ids, filenames and byte sizes come straight from the HuggingFace tree API, since the upstream filename shape (gemma-4-31B_q4_0-it.gguf) does not follow the bartowski naming.

The existing q4_k_m entries are untouched, so nobody's downloaded model gets orphaned or force re-downloaded. The QAT builds are a separate opt-in download. The only edit to an existing entry is moving the "Recommended" badge from E4B Q4_K_M to the E4B QAT entry (the flag only renders a badge in the picker, nothing selects a model by it). One caveat: llama.cpp has an open report (ggml-org/llama.cpp#24311) of garbage output with Gemma 4 QAT on Vulkan with GPU offload, seen with unsloth's UD quants rather than these official q4_0 files. The default CPU path and Metal are unaffected.

Fixes #918

Changes

  • src/models/modelRegistryData.json: four new QAT entries (31B, 26B MoE, E4B, E2B) inserted next to their Q4_K_M counterparts, recommended badge moved to E4B QAT
  • src/locales/*/translation.json: descriptions for the four new entries in all 10 languages

@gabrielste1n
gabrielste1n merged commit e1eca67 into OpenWhispr:main Jul 22, 2026
9 checks passed
@gabrielste1n
gabrielste1n deleted the feat/gemma4-qat-models branch July 22, 2026 20:14
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Switch Gemma 4 to use the newly released QAT checkpoint models

2 participants