Skip to content

[FLYDSL] Rebase flydsl hgemm kernels with mixed policies - #3721

Closed
xytpai wants to merge 37 commits into
mainfrom
xyt/hgemm_ht
Closed

xytpai wants to merge 37 commits into
mainfrom
xyt/hgemm_ht

Conversation

@xytpai

@xytpai xytpai commented Jun 15, 2026

Copy link
Copy Markdown
Contributor
python3 csrc/gemm_a16w16/gemm_a16w16_tune.py \
  --tuned_file aiter/configs/model_configs/gptoss_bf16_tuned_gemm.csv \
  --input_file aiter/configs/model_configs/gptoss_bf16_untuned_gemm.csv \
  --libtype flydsl,torch,triton,skinny,opus

python3 csrc/gemm_a16w16/gemm_a16w16_tune.py \
  --tuned_file aiter/configs/model_configs/dsv3_bf16_tuned_gemm.csv \
  --input_file aiter/configs/model_configs/dsv3_bf16_untuned_gemm.csv \
  --libtype flydsl,torch,triton,asm,skinny

python3 csrc/gemm_a16w16/gemm_a16w16_tune.py \
  --tuned_file aiter/configs/model_configs/dsv4_bf16_tuned_gemm.csv \
  --input_file aiter/configs/model_configs/dsv4_bf16_untuned_gemm.csv \
  --libtype flydsl,torch,triton,asm,skinny,opus

python3 csrc/gemm_a16w16/gemm_a16w16_tune.py \
  --tuned_file aiter/configs/model_configs/glm5_bf16_tuned_gemm.csv \
  --input_file aiter/configs/model_configs/glm5_untuned_gemm_bf16.csv \
  --libtype flydsl,torch,triton,asm,skinny,opus

python3 csrc/gemm_a16w16/gemm_a16w16_tune.py \
  --tuned_file aiter/configs/model_configs/kimi_bf16_tuned_gemm.csv \
  --input_file aiter/configs/model_configs/kimi_bf16_untuned_gemm.csv \
  --libtype flydsl,torch,triton,asm,skinny

python3 csrc/gemm_a16w16/gemm_a16w16_tune.py \
  --tuned_file aiter/configs/model_configs/kimik2_bf16_tuned_gemm.csv \
  --input_file aiter/configs/model_configs/kimik2_bf16_untuned_gemm.csv \
  --libtype flydsl,torch,triton,asm,skinny

python3 csrc/gemm_a16w16/gemm_a16w16_tune.py \
  --tuned_file aiter/configs/model_configs/kimik3_bf16_tuned_gemm.csv \
  --input_file aiter/configs/model_configs/kimik3_bf16_untuned_gemm.csv \
  --libtype flydsl,torch,triton,asm,skinny,opus

python3 csrc/gemm_a16w16/gemm_a16w16_tune.py \
  --tuned_file aiter/configs/model_configs/llama70B_bf16_tuned_gemm.csv \
  --input_file aiter/configs/model_configs/llama70B_untuned_gemm_bf16.csv \
  --libtype flydsl,torch,triton,asm,skinny

python3 csrc/gemm_a16w16/gemm_a16w16_tune.py \
  --tuned_file aiter/configs/model_configs/llama405B_bf16_tuned_gemm.csv \
  --input_file aiter/configs/model_configs/llama405B_untuned_gemm_bf16.csv \
  --libtype flydsl,torch,triton,asm,skinny

python3 csrc/gemm_a16w16/gemm_a16w16_tune.py \
  --tuned_file aiter/configs/model_configs/qwen3_5_397b_bf16_tuned_gemm.csv \
  --input_file aiter/configs/model_configs/qwen3_5_397b_untuned_gemm.csv \
  --libtype flydsl,torch,triton,asm,skinny,opus

python3 csrc/gemm_a16w16/gemm_a16w16_tune.py \
  --tuned_file aiter/configs/model_configs/qwen32B_bf16_tuned_gemm.csv \
  --input_file aiter/configs/model_configs/qwen32B_untuned_gemm_bf16.csv \
  --libtype flydsl,torch,triton,asm,skinny

python3 csrc/gemm_a16w16/gemm_a16w16_tune.py \
  --tuned_file aiter/configs/model_configs/minimax_m3_eagle_bf16_tuned_gemm.csv \
  --input_file aiter/configs/model_configs/minimax_m3_eagle_bf16_untuned_gemm.csv \
  --libtype flydsl,torch,triton,asm,skinny

@xytpai
xytpai requested a review from a team June 15, 2026 06:32
@github-actions

Copy link
Copy Markdown
Contributor

🏷️ CI Guide

Runs automatically on every PR:

  • ✅ Pre-checks (submodule verification, code formatting)
  • ✅ Aiter op tests (gfx942 + gfx950)
  • ✅ Triton tests on MI35X (only when aiter/ops/triton/** or related paths are changed)

Extended tests (opt-in via labels):

Label Tests
ci:triton-300x Run an additional Triton test job on MI300X in PRs; main branch always runs both MI35X and MI300X
ci:sglang SGLang integration tests: DeepSeek-R1-MXFP4 accuracy, Qwen 3.5 accuracy
ci:atom ATOM benchmark: DeepSeek-R1-0528, GPT-OSS-120B
ci:atom_full ATOM accuracy suite for PR and main models from ATOM models_accuracy.json
ci:vllm vLLM benchmark: GPT-OSS-120B, DeepSeek-R1-0528, Kimi-K2.5
ci:all All standard extended tests (excludes ci:atom_full)

Only add ci:atom_full for FlyDSL or Triton upgrades.
Add labels via the sidebar or gh pr edit 3721 --add-label <label>

@xytpai xytpai changed the title [FLYDSL][HGEMM] Add hipKitten style half-tile sync generalized hgemm kernel [FLYDSL][HGEMM] Add generalized half-tile sync kernel with hipKitten style Jun 15, 2026
@xytpai xytpai changed the title [FLYDSL][HGEMM] Add generalized half-tile sync kernel with hipKitten style [FLYDSL][HGEMM-PROTOTYPE] Add generalized half-tile sync kernel with hipKitten style Jun 15, 2026
@xytpai xytpai changed the title [FLYDSL][HGEMM-PROTOTYPE] Add generalized half-tile sync kernel with hipKitten style [FLYDSL][HGEMM-PROTOTYPE] Add generalized half-tile schedule kernel with hipKitten style Jun 15, 2026
@xytpai xytpai added the ci:atom label Jun 15, 2026
@zufayu
zufayu requested a review from yifehuan June 26, 2026 08:54
@xytpai xytpai changed the title [FLYDSL][HGEMM-PROTOTYPE] Add generalized half-tile schedule kernel with hipKitten style [FLYDSL][HGEMM-PROTOTYPE] Adopting mixed policy for flydsl hgemm Jul 8, 2026
@xytpai xytpai changed the title [FLYDSL][HGEMM-PROTOTYPE] Adopting mixed policy for flydsl hgemm [FLYDSL] Rebase flydsl hgemm kernels Jul 8, 2026
@xytpai xytpai changed the title [FLYDSL] Rebase flydsl hgemm kernels [FLYDSL] Rebase flydsl hgemm kernels with mixed policies Jul 8, 2026
GROUP_M=kwargs["GROUP_M"],
USE_HALF_TILE_INTERLEAVED=USE_HALF_TILE_INTERLEAVED,
)
except Exception as e:

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ [ruff] <F841> reported by reviewdog 🐶
Local variable e is assigned to but never used

Suggested change
except Exception as e:
except Exception:

Comment on lines 18 to 19
from aiter.ops.flydsl.kernels.tensor_shim import ptr_arg

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ [ruff] <F401> reported by reviewdog 🐶
aiter.ops.flydsl.kernels.tensor_shim.ptr_arg imported but unused

Suggested change
from aiter.ops.flydsl.kernels.tensor_shim import ptr_arg

@xytpai

xytpai commented Aug 31, 2026

Copy link
Copy Markdown
Contributor Author

Move to #5145

@xytpai xytpai closed this Aug 31, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant