Skip to content

Introduce new 64x384 kernel - #3630

Merged
JohnNikolay84 merged 1 commit into
mainfrom
64x384_fmoe_blockscale_1x128
Jun 9, 2026
Merged

JohnNikolay84 merged 1 commit into
mainfrom
64x384_fmoe_blockscale_1x128

Conversation

@JohnNikolay84

Copy link
Copy Markdown
Contributor

Motivation

We can optimize models such as minimax and glm for prefill if we switch from 32x384 to 64x384 kernel. More tokens per thread group will reduce weights re-load and also will reduce the number of thread groups we have to process.

Technical Details

A new kernel will be supporting 64 tokens per expert per thread group.

Test Plan

unit tests for minimax shapes.

Test Result

image image

Submission Checklist

@JohnNikolay84 JohnNikolay84 self-assigned this Jun 9, 2026
@JohnNikolay84
JohnNikolay84 requested a review from a team June 9, 2026 09:12
@github-actions

github-actions Bot commented Jun 9, 2026

Copy link
Copy Markdown
Contributor

🏷️ CI Guide

Runs automatically on every PR:

  • ✅ Pre-checks (submodule verification, code formatting)
  • ✅ Aiter op tests (gfx942 + gfx950)
  • ✅ Triton tests on MI35X (only when aiter/ops/triton/** or related paths are changed)

Extended tests (opt-in via labels):

Label Tests
ci:triton-300x Run an additional Triton test job on MI300X in PRs; main branch always runs both MI35X and MI300X
ci:sglang SGLang integration tests: DeepSeek-R1-MXFP4 accuracy, Qwen 3.5 accuracy
ci:atom ATOM benchmark: DeepSeek-R1-0528, GPT-OSS-120B
ci:atom_full ATOM accuracy suite for PR and main models from ATOM models_accuracy.json
ci:vllm vLLM benchmark: GPT-OSS-120B, DeepSeek-R1-0528, Kimi-K2.5
ci:all All standard extended tests (excludes ci:atom_full)

Only add ci:atom_full for FlyDSL or Triton upgrades.
Add labels via the sidebar or gh pr edit 3630 --add-label <label>

@JohnNikolay84
JohnNikolay84 merged commit 883aff7 into main Jun 9, 2026
40 checks passed
@JohnNikolay84
JohnNikolay84 deleted the 64x384_fmoe_blockscale_1x128 branch June 9, 2026 11:23
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants