Add static KV cache (TensorScatter) support for Gemma4 - #246

Merged

justinchuby merged 17 commits into

mainfrom

gemma4-static-cache

Jul 25, 2026

Commits on Jul 25, 2026

Add static KV cache (TensorScatter) support for Gemma4

justinchuby
and
Copilot
committed
d08360c
Fix review findings: input ordering, tests, CLI duplication

justinchuby
and
Copilot
committed
1e36b62
Support static cache on multimodal Gemma4Task (decoder only)

justinchuby
and
Copilot
committed
b11fecf
Fix review: whitelist validation, sliding window warning, explicit static detection

justinchuby
and
Copilot
committed
15d2831
Reject static cache for models with sliding window

justinchuby
and
Copilot
committed
5ce76d0
Implement hybrid per-layer static/dynamic cache for Gemma4

justinchuby
and
Copilot
committed
a01fc33
Add design comment explaining hybrid static/dynamic cache rationale

justinchuby
and
Copilot
committed
a3fa2aa
Fix lint: remove trailing newline
justinchuby
committed
22dc6e9
Fix lint errors

justinchuby
and
Copilot
committed
40af5eb
Fix lint: prefix unused config variable
justinchuby
committed
4069215
Fix test: use _config instead of unbound config variable
justinchuby
committed
0e47d31
Fix static cache build: guard attention_mask usage

justinchuby
and
Copilot
committed
bed5c5a
Fix KV-shared layers with static cache sources

justinchuby
and
Copilot
committed
469dc8d
Fix static cache: is_causal=0 for nonpad_kv_seqlen, dynamic path for KV-shared

justinchuby
and
Copilot
committed
0c96310
Fix PR 246 lint and unittest compatibility

justinchuby
and
Copilot
committed
09ae65e
Align static cache test with rebased attention semantics

justinchuby
and
Copilot
committed
5f24f98
Merge branch 'main' into gemma4-static-cache
justinchuby
authored
12781d9

Navigation Menu

Uh oh!

Add static KV cache (TensorScatter) support for Gemma4 - #246

Add static KV cache (TensorScatter) support for Gemma4#246
justinchuby merged 17 commits into
mainfrom
gemma4-static-cache

Commits on Jul 25, 2026

Add static KV cache (TensorScatter) support for Gemma4

Fix review findings: input ordering, tests, CLI duplication

Support static cache on multimodal Gemma4Task (decoder only)

Fix review: whitelist validation, sliding window warning, explicit static detection

Reject static cache for models with sliding window

Implement hybrid per-layer static/dynamic cache for Gemma4

Add design comment explaining hybrid static/dynamic cache rationale

Fix lint: remove trailing newline

Fix lint errors

Fix lint: prefix unused config variable

Fix test: use _config instead of unbound config variable

Fix static cache build: guard attention_mask usage

Fix KV-shared layers with static cache sources

Fix static cache: is_causal=0 for nonpad_kv_seqlen, dynamic path for KV-shared

Fix PR 246 lint and unittest compatibility

Align static cache test with rebased attention semantics

Merge branch 'main' into gemma4-static-cache