feat(gemm_a8w8_blockscale_bpreshuffle): add GLM-5.2 tuned configs (gfx950) - #4243
Merged
yifehuan merged 1 commit intoJul 21, 2026
Merged
Conversation
Contributor
🏷️ CI GuideRuns automatically on every PR:
Extended tests (opt-in via labels):
|
yifehuan
reviewed
Jul 16, 2026
Oseltamivir
added a commit
to SemiAnalysisAI/InferenceX
that referenced
this pull request
Jul 18, 2026
The cookbook-verified v0.5.13.post1-rocm720-mi35x-20260618 build segfaults in AITER gemm_a8w8_blockscale_bpreshuffle mid-profiling under the agentic corpus (KV usage 0.95+, 5.3M pending tokens, untuned GLM-5.2 GEMM shapes falling back to default configs - gfx950 tuned configs are not upstream yet, ROCm/aiter#4243). Move to the newest rocm720 build for the latest AITER fallback fixes. Cap the conc list at 32 (was 64): the B300 sibling frontier showed no-offload points beyond ~c48 are KV-cliff-dominated on the 131k-token mean corpus, and MI355X prefill is slower still - c48/c64 only added crash surface and sweep time for dominated points. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…ig (gfx950) Adds tuned a8w8_blockscale_bpreshuffle configs for GLM-5.2's o_proj (N=6144, K=4096) on gfx950 — the one 128-aligned MLA dense-projection shape missing from the GLM5 block-scale configs merged in ROCm#4236 (which covers q_b_proj 4096,2048 and other GLM5 shapes but not o_proj 6144,4096). Config coverage only, no code changes; merges via the existing *a8w8_blockscale_bpreshuffle_tuned_gemm*.csv glob. 60 rows, gfx950/cu_num=256, M = prefill (1024-65536) + decode/cuda-graph (1-512).
Raiden-Makoto
force-pushed
the
glm52-blockscale-bpreshuffle-gfx950
branch
from
July 20, 2026 15:50
889a79f to
6aa9329
Compare
Contributor
Author
|
@yifehuan following up on your review — the duplicated This is the same tuned-config pattern you already reviewed and merged in #4236 (and #4264), just the remaining GLM-5.2 |
yifehuan
approved these changes
Jul 21, 2026
yifehuan
left a comment
Contributor
There was a problem hiding this comment.
LGTM, no more duplicate shape now
3 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Motivation
The
a8w8_blockscale_bpreshuffleGEMM has per-model tuned configs underaiter/configs/model_configs/for dsv3, mm2.5, and qwen — but none for GLM-5.2.GLM-5.2's two 128-aligned MLA dense projections on gfx950 —
q_b_proj(N=4096, K=2048) and
o_proj(N=6144, K=4096) — therefore miss the tuned-configlookup and fall back to the default solution (aiter logs
not found tuned config in ... will use default config).Note aiter already ships
a8w8_bpreshuffle_tuned_gemm_glm5.2.csv, but that is theper-tensor bpreshuffle path (schema has
q_dtype_w). These projections use theblock-scale (128x128) path (
a8w8_blockscale_bpreshuffle), which has no GLM-5.2config. This PR adds one. Config coverage only — no code changes.
Technical Details
aiter/configs/model_configs/a8w8_blockscale_bpreshuffle_tuned_gemm_glm5.2.csvgfx950/cu_num=256covering both shapes(
q_b_proj4096,2048 ando_proj6144,4096).sizes (1–512).
a8w8_blockscale_bpreshuffle_tuned_gemm.csv:gfx,cu_num,M,N,K,libtype,kernelId,splitK,us,kernelName,tflops,bw,errRatio(libtype ∈ {asm, ck, cktile}).
AiterConfigs.get_config_file(...)globsmodel_configs/*a8w8_blockscale_bpreshuffle_tuned_gemm*.csvandupdate_config_files(...)merges + dedups (lowestusper shape) with the baseat runtime, so the new file is picked up automatically.
(
csrc/ck_gemm_a8w8_blockscale/gemm_a8w8_blockscale_tune.py --preshuffle), nomanual edits.
Test Plan
fallback warning (i.e.
get_config_filemerges the new model_config).op_tests/test_gemm_a8w8_blockscale.pyon the added shapes.Test Result
not found tuned config … will use defaultwarningsfor
4096,2048and6144,4096no longer appear at load; both resolve to theirtuned entries.
Submission Checklist
black/ruff).