Skip to content

perf(gemma4): default swa_full_tokens_ratio=0.15 for the 25:5 SWA:full split#2

Open
pyc96 wants to merge 6 commits into
pyc/sota-gemma4-mtp-fused-routingfrom
pyc/sota-gemma4-mtp-swa-ratio
Open

perf(gemma4): default swa_full_tokens_ratio=0.15 for the 25:5 SWA:full split#2
pyc96 wants to merge 6 commits into
pyc/sota-gemma4-mtp-fused-routingfrom
pyc/sota-gemma4-mtp-swa-ratio