Add diffusion-gemma block-diffusion support - #24427

Draft

lnigam wants to merge 40 commits into

ggml-org:masterfrom

lnigam:nvidia-diffusion-gemma

Commits on Jun 24, 2026

convert: add diffusion_gemma4 (block-diffusion Gemma 4) conversion support
lnigam
committed
0d751f5
model: register diffusion-gemma4 arch (reuses gemma4 graph)
lnigam
committed
8d0feb8
model: diffusion-gemma4 bidirectional graph + self-conditioning input
lnigam
committed
b329cf3
model: wire self-conditioning MLP into the diffusion-gemma4 graph
lnigam
committed
03e53ea
examples: block-diffusion generation CLI for diffusion-gemma4
lnigam
committed
dfe6e77
diffusion-gemma4: self-conditioning input channel (feedback)
lnigam
committed
9b7d75b
diffusion-gemma4: prompt conditioning via prefix attention
lnigam
committed
3f12024
diffusion-gemma4: apply the chat template to the prompt
lnigam
committed
16c4d37
diffusion-gemma4: fix prompt conditioning (keep prompt tokens active)
lnigam
committed
191e8dc
diffusion-gemma4: greedy read-out of the final canvas
lnigam
committed
589fc2f
diffusion-gemma4: extract the clean final answer
lnigam
committed
f231fd6
diffusion-gemma4: offload to GPU by default
lnigam
committed
d718e8f
diffusion_gemma4: KV-cache reuse for block-diffusion generation
lnigam
committed
b014b82
diffusion-gemma (v7): rename arch + multimodal (gemma4 vision) support
lnigam
committed
c01ee56
diffusion-gemma cli: -n controls block count, generation timing, inline-argmax output
lnigam
committed
2e49867
diffusion-gemma: place precomputed transposed embedding on the offloaded backend
lnigam
committed
cbb6987
diffusion-gemma cli: report encoder-phase prefill timing
lnigam
committed
dbb2002
diffusion-gemma: top-k host sampling (top-k softmax/entropy/self-cond) with k-annealing
lnigam
committed
bbd8abd
diffusion-gemma: sparse top-k self-conditioning via on-device F16 gather
lnigam
committed
e8b1b22
diffusion-gemma: OpenAI-compatible HTTP server (llama-server analogue)
lnigam
committed
8b1d4a1
diffusion-gemma: add CUDA graph GPU sampling server path
lnigam
committed
7df4b25
diffusion-gemma: keep self-cond sampling on device
lnigam
committed
61e43a1
diffusion-gemma: keep denoising loop on device
lnigam
committed
e5aabf4
diffusion-gemma: use output flag for persistent inputs
lnigam
committed
ed78362
diffusion-gemma: checkpoint device-loop early stop
lnigam
committed
5b15dfd
diffusion-gemma: make device early stop every step
lnigam
committed
27fe91a
examples: update diffusion gemma mtmd bitmap loading
lnigam
committed
df9a66f
incorporating unsloth model related changes
lnigam
committed
b7f4bfc
diffusion-gemma: add tuning options and server accounting
lnigam
committed
66a924e
diffusion: add max denoising loop switch
lnigam
committed
500ce4e
diffusion: wire CUDA sampling optimization controls
lnigam
committed
7d362cc
ggml-cuda: add fused diffusion sampling kernels
lnigam
committed
0a098ec
diffusion: keep final denoise token copy
lnigam
committed
4627948
diffusion: default to fused CUDA full softmax
lnigam
committed
fbf9b61
diffusion-gemma: enable fast CUDA defaults
lnigam
committed
a969058
diffusion-gemma: add CUDA MMQ tile override flag
lnigam
committed
2e6695d
llama : scope diffusion graph paths to diffusion gemma
lnigam
committed
3ecb5b4
fix flake8_lint failure
lnigam
committed
f9538b7
avoid no-repack
lnigam
committed
273a992
fix chat template issues impacting the mean denoising steps in GGUF to be way higher compared to pytorch
lnigam
committed
dd0cf04