Skip to content

[FlyDSL] Add gfx950 FHMoE tuner on gemm_moe_tune.py #5751

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft
a-canadasruiz wants to merge 14 commits into
base: main
Choose a base branch
from
Draft

[FlyDSL] Add gfx950 FHMoE tuner on gemm_moe_tune.py #5751

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
14 commits
Select commit Hold shift + click to select a range
030f49f
[FlyDSL] Draft FHMoE tuner: catalogue, --fhmoe, and fused_moe smoke
a-canadasruiz Sep 16, 2026
1f0509c
[FlyDSL] Run FHMoE tuner tasks through mp_tuner with host-side weight…
a-canadasruiz Sep 16, 2026
37317b9
[FlyDSL] Sweep fused FHMoE kernel pairs and emit native tuned CSV
a-canadasruiz Sep 17, 2026
71e77e1
[FlyDSL] Wire FHMoE into op_tune.sh and replay fused_moe against a ge…
a-canadasruiz Sep 17, 2026
a7e7f82
[FlyDSL] Gate FHMoE tune on torch cosine and time public fused_moe
a-canadasruiz Sep 18, 2026
0b2f8cd
[FlyDSL] Cover DSV4 M-tiers in the FHMoE producer replay
a-canadasruiz Sep 18, 2026
6659ae1
[FlyDSL] Skip FHMoE tune on non-gfx950 and drop --all from op_tune.sh
a-canadasruiz Sep 21, 2026
442113b
[FlyDSL] Cover FHMoE in Tuning Tests like other tuner families
a-canadasruiz Sep 21, 2026
8356338
[FlyDSL] Drop FHMoE accuracy_gate mock test
a-canadasruiz Sep 21, 2026
e854f4f
[FlyDSL] Format FHMoE tuner files for Black and Ruff
a-canadasruiz Sep 22, 2026
e2f9395
[FlyDSL] Publish FHMoE tuned rows under padded token
a-canadasruiz Sep 22, 2026
e4a7884
[FlyDSL] Match FMoE stage1 k_batch filter in FHMoE
a-canadasruiz Sep 22, 2026
bdac6e9
[FlyDSL] Honor --warmup/--iters in the FHMoE search
a-canadasruiz Sep 23, 2026
6d19303
[FlyDSL] Time both fused and non-fused stage1 quant in FHMoE
a-canadasruiz Sep 25, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
11 changes: 11 additions & 0 deletions .github/scripts/op_tune.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -24,6 +24,7 @@ declare -a tune_jobs=(
"ck_gemm_a8w8_blockscale_bpreshuffle:csrc/ck_gemm_a8w8_blockscale_bpreshuffle:op_tests/test_gemm_a8w8_blockscale.py:python3 csrc/ck_gemm_a8w8_blockscale/gemm_a8w8_blockscale_tune.py --preshuffle -i aiter/configs/a8w8_blockscale_bpreshuffle_untuned_gemm.csv -o aiter/configs/a8w8_blockscale_bpreshuffle_tuned_gemm.csv"
"ck_gemm_a8w8_bpreshuffle:csrc/ck_gemm_a8w8_bpreshuffle:op_tests/test_gemm_a8w8.py:python3 csrc/ck_gemm_a8w8_bpreshuffle/gemm_a8w8_bpreshuffle_tune.py -i aiter/configs/a8w8_bpreshuffle_untuned_gemm.csv -o aiter/configs/a8w8_bpreshuffle_tuned_gemm.csv"
"ck_gemm_moe_2stages_codegen:csrc/ck_gemm_moe_2stages_codegen:op_tests/test_moe.py:python3 csrc/ck_gemm_moe_2stages_codegen/gemm_moe_tune.py -i aiter/configs/untuned_fmoe.csv -o aiter/configs/tuned_fmoe.csv"
"fhmoe:csrc/ck_gemm_moe_2stages_codegen:op_tests/test_fhmoe.py:python3 csrc/ck_gemm_moe_2stages_codegen/gemm_moe_tune.py --fhmoe -i aiter/configs/untuned_fhmoe.csv -o aiter/configs/tuned_fhmoe.csv"
#"ck_gemm_a4w4_blockscale:csrc/ck_gemm_a4w4_blockscale:op_tests/test_gemm_a4w4_blockscale.py:python3 csrc/ck_gemm_a4w4_blockscale/gemm_a4w4_blockscale_tune.py -i aiter/configs/a4w4_blockscale_untuned_gemm.csv -o aiter/configs/a4w4_blockscale_tuned_gemm.csv"
)

Expand All @@ -46,6 +47,16 @@ for job in "${tune_jobs[@]}"; do
continue
fi
fi
if [ "$shape" = "fhmoe" ]; then
gfx=$(python3 -c "from aiter.jit.utils.chip_info import get_gfx; print(get_gfx())") || gfx=""
if [ "$gfx" != "gfx950" ]; then
echo "============================================================"
echo "Skipping fhmoe: requires gfx950, got ${gfx:-unknown}"
echo "==============================================="
echo
continue
fi
fi
echo "============================================================"
echo "🧪 Processing shape: $shape under directory: $dir"
echo "------------------------------------------------------------"
Expand Down
14 changes: 13 additions & 1 deletion aiter/configs/untuned_fhmoe.csv
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1 +1,13 @@
gfx,cu_num,token,model_dim,inter_dim,expert,topk,act_type,dtype,q_dtype_a,q_dtype_w,q_type,use_g1u1,doweight_stage1
token,model_dim,inter_dim,expert,topk,shared_expert_id,act_type,dtype,q_dtype_a,q_dtype_w,q_type,use_g1u1,doweight_stage1,hidden_pad,intermediate_pad,gate_mode
1,7168,384,385,7,384,ActivationType.Silu,torch.bfloat16,torch.float8_e4m3fn,torch.float4_e2m1fn_x2,QuantType.per_1x32,1,0,0,0,GateMode.INTERLEAVE
2,7168,384,385,7,384,ActivationType.Silu,torch.bfloat16,torch.float8_e4m3fn,torch.float4_e2m1fn_x2,QuantType.per_1x32,1,0,0,0,GateMode.INTERLEAVE
4,7168,384,385,7,384,ActivationType.Silu,torch.bfloat16,torch.float8_e4m3fn,torch.float4_e2m1fn_x2,QuantType.per_1x32,1,0,0,0,GateMode.INTERLEAVE
8,7168,384,385,7,384,ActivationType.Silu,torch.bfloat16,torch.float8_e4m3fn,torch.float4_e2m1fn_x2,QuantType.per_1x32,1,0,0,0,GateMode.INTERLEAVE
16,7168,384,385,7,384,ActivationType.Silu,torch.bfloat16,torch.float8_e4m3fn,torch.float4_e2m1fn_x2,QuantType.per_1x32,1,0,0,0,GateMode.INTERLEAVE
32,7168,384,385,7,384,ActivationType.Silu,torch.bfloat16,torch.float8_e4m3fn,torch.float4_e2m1fn_x2,QuantType.per_1x32,1,0,0,0,GateMode.INTERLEAVE
64,7168,384,385,7,384,ActivationType.Silu,torch.bfloat16,torch.float8_e4m3fn,torch.float4_e2m1fn_x2,QuantType.per_1x32,1,0,0,0,GateMode.INTERLEAVE
128,7168,384,385,7,384,ActivationType.Silu,torch.bfloat16,torch.float8_e4m3fn,torch.float4_e2m1fn_x2,QuantType.per_1x32,1,0,0,0,GateMode.INTERLEAVE
256,7168,384,385,7,384,ActivationType.Silu,torch.bfloat16,torch.float8_e4m3fn,torch.float4_e2m1fn_x2,QuantType.per_1x32,1,0,0,0,GateMode.INTERLEAVE
512,7168,384,385,7,384,ActivationType.Silu,torch.bfloat16,torch.float8_e4m3fn,torch.float4_e2m1fn_x2,QuantType.per_1x32,1,0,0,0,GateMode.INTERLEAVE
1024,7168,384,385,7,384,ActivationType.Silu,torch.bfloat16,torch.float8_e4m3fn,torch.float4_e2m1fn_x2,QuantType.per_1x32,1,0,0,0,GateMode.INTERLEAVE
2048,7168,384,385,7,384,ActivationType.Silu,torch.bfloat16,torch.float8_e4m3fn,torch.float4_e2m1fn_x2,QuantType.per_1x32,1,0,0,0,GateMode.INTERLEAVE
Loading
Loading