Skip to content

fix(gemma4): only apply swa_full_tokens_ratio=0.15 to MoE variants#8

Open
pyc96 wants to merge 1 commit into
pyc/sota-gemma4-mtp-revert-clampfrom
pyc/sota-gemma4-mtp-swa-ratio-moe-only
Open

fix(gemma4): only apply swa_full_tokens_ratio=0.15 to MoE variants#8
pyc96 wants to merge 1 commit into
pyc/sota-gemma4-mtp-revert-clampfrom
pyc/sota-gemma4-mtp-swa-ratio-moe-only

Commits

Commits on May 23, 2026