[Triton/Gluon] Satya/unify gluon a8w8 blockscale config resolution by Boss2002n · Pull Request #4947 · ROCm/aiter

Boss2002n · 2026-08-23T14:03:39Z

#4927
satya/migrate-configs-gemm-a8w8-blockscale must land before this

github-actions · 2026-08-23T14:04:05Z

🏷️ CI Guide

Runs automatically on every PR:

✅ Pre-checks (submodule verification, code formatting)
✅ Aiter op tests (gfx942 + gfx950)
✅ Triton tests on MI35X (only when aiter/ops/triton/** or related paths are changed)

Extended tests (opt-in via labels):

Label	Tests
`ci:gfx1250-ffm-triton`	Run the five-shard gfx1250 FFM Triton test suite
`ci:triton-300x`	Run an additional Triton test job on MI300X in PRs; main branch always runs both MI35X and MI300X
`ci:sglang`	SGLang integration tests: DeepSeek-R1-MXFP4 accuracy, Qwen 3.5 accuracy
`ci:atom`	ATOM benchmark: DeepSeek-R1-0528, GPT-OSS-120B
`ci:atom_full`	ATOM accuracy suite for PR and main models from ATOM `models_accuracy.json`
`ci:vllm`	vLLM benchmark: GPT-OSS-120B, DeepSeek-R1-0528, Kimi-K2.5
`ci:all`	All standard extended tests (excludes `ci:atom_full`)

Only add ci:atom_full for FlyDSL or Triton upgrades.
Add labels via the sidebar or gh pr edit 4947 --add-label <label>

PR title tags:
Component tags ([Triton/Gluon], [HIP], [CK], [ASM], ...) are added to the PR title automatically from the changed files and re-synced on every push — change-type tags like [fix]/[Perf] and op tags like [MLA] are left untouched. Add the no-auto-title label to opt this PR out of title tagging.

Copilot

Pull request overview

This PR centralizes GEMM config directory resolution across Triton/Gluon backends and migrates Gluon A8W8 blockscale config lookup toward the nested <arch>/<backend>/gemm/<dtype_dir>/ layout, adding new nested config JSONs for gfx950/gfx1250.

Changes:

Added a shared resolve_config_dir() helper and refactored get_gemm_config() to use it for nested-vs-legacy probing.
Updated Gluon gemm_a8w8_blockscale to use the shared resolver and load DEFAULT.json in the nested layout.
Added new nested Gluon config JSONs for GEMM-A8W8_BLOCKSCALE and GEMM-A8W8_BLOCKSCALE_PRESHUFFLED for gfx950/gfx1250.

Reviewed changes

Copilot reviewed 2 out of 27 changed files in this pull request and generated 1 comment.

Show a summary per file

File	Description
aiter/ops/triton/utils/gemm_config_utils.py	Adds `resolve_config_dir()` and refactors GEMM config probing to reuse it.
aiter/ops/triton/gluon/gemm_a8w8_blockscale.py	Switches Gluon A8W8 blockscale config resolution to `resolve_config_dir()` and nested `DEFAULT.json`.
aiter/ops/triton/configs/gfx950/gluon/gemm/gemm_a8w8_blockscale/DEFAULT.json	Adds nested default config for gfx950 Gluon A8W8 blockscale.
aiter/ops/triton/configs/gfx950/gluon/gemm/gemm_a8w8_blockscale/GEMM-A8W8_BLOCKSCALE-N=7168-K=256.json	Adds gfx950 Gluon specialized config (N=7168, K=256).
aiter/ops/triton/configs/gfx950/gluon/gemm/gemm_a8w8_blockscale/GEMM-A8W8_BLOCKSCALE-N=7168-K=2048.json	Adds gfx950 Gluon specialized config (N=7168, K=2048).
aiter/ops/triton/configs/gfx950/gluon/gemm/gemm_a8w8_blockscale/GEMM-A8W8_BLOCKSCALE-N=512-K=7168.json	Adds gfx950 Gluon specialized config (N=512, K=7168).
aiter/ops/triton/configs/gfx950/gluon/gemm/gemm_a8w8_blockscale/GEMM-A8W8_BLOCKSCALE-N=4608-K=7168.json	Adds gfx950 Gluon specialized config (N=4608, K=7168).
aiter/ops/triton/configs/gfx950/gluon/gemm/gemm_a8w8_blockscale/GEMM-A8W8_BLOCKSCALE-N=3072-K=1536.json	Adds gfx950 Gluon specialized config (N=3072, K=1536).
aiter/ops/triton/configs/gfx950/gluon/gemm/gemm_a8w8_blockscale/GEMM-A8W8_BLOCKSCALE-N=2112-K=7168.json	Adds gfx950 Gluon specialized config (N=2112, K=7168).
aiter/ops/triton/configs/gfx1250/gluon/gemm/gemm_a8w8_blockscale/DEFAULT.json	Adds nested default config for gfx1250 Gluon A8W8 blockscale.
aiter/ops/triton/configs/gfx1250/gluon/gemm/gemm_a8w8_blockscale_preshuffled/DEFAULT.json	Adds nested default config for gfx1250 Gluon A8W8 blockscale preshuffled.
aiter/ops/triton/configs/gfx1250/gluon/gemm/gemm_a8w8_blockscale_preshuffled/GEMM-A8W8_BLOCKSCALE_PRESHUFFLED-N=8192-K=1536.json	Adds gfx1250 Gluon preshuffled specialized config (N=8192, K=1536).
aiter/ops/triton/configs/gfx1250/gluon/gemm/gemm_a8w8_blockscale_preshuffled/GEMM-A8W8_BLOCKSCALE_PRESHUFFLED-N=8192-K=1024.json	Adds gfx1250 Gluon preshuffled specialized config (N=8192, K=1024).
aiter/ops/triton/configs/gfx1250/gluon/gemm/gemm_a8w8_blockscale_preshuffled/GEMM-A8W8_BLOCKSCALE_PRESHUFFLED-N=7168-K=768.json	Adds gfx1250 Gluon preshuffled specialized config (N=7168, K=768).
aiter/ops/triton/configs/gfx1250/gluon/gemm/gemm_a8w8_blockscale_preshuffled/GEMM-A8W8_BLOCKSCALE_PRESHUFFLED-N=7168-K=4096.json	Adds gfx1250 Gluon preshuffled specialized config (N=7168, K=4096).
aiter/ops/triton/configs/gfx1250/gluon/gemm/gemm_a8w8_blockscale_preshuffled/GEMM-A8W8_BLOCKSCALE_PRESHUFFLED-N=7168-K=3072.json	Adds gfx1250 Gluon preshuffled specialized config (N=7168, K=3072).
aiter/ops/triton/configs/gfx1250/gluon/gemm/gemm_a8w8_blockscale_preshuffled/GEMM-A8W8_BLOCKSCALE_PRESHUFFLED-N=7168-K=16384.json	Adds gfx1250 Gluon preshuffled specialized config (N=7168, K=16384).
aiter/ops/triton/configs/gfx1250/gluon/gemm/gemm_a8w8_blockscale_preshuffled/GEMM-A8W8_BLOCKSCALE_PRESHUFFLED-N=65536-K=1536.json	Adds gfx1250 Gluon preshuffled specialized config (N=65536, K=1536).
aiter/ops/triton/configs/gfx1250/gluon/gemm/gemm_a8w8_blockscale_preshuffled/GEMM-A8W8_BLOCKSCALE_PRESHUFFLED-N=6144-K=7168.json	Adds gfx1250 Gluon preshuffled specialized config (N=6144, K=7168).
aiter/ops/triton/configs/gfx1250/gluon/gemm/gemm_a8w8_blockscale_preshuffled/GEMM-A8W8_BLOCKSCALE_PRESHUFFLED-N=4096-K=8192.json	Adds gfx1250 Gluon preshuffled specialized config (N=4096, K=8192).
aiter/ops/triton/configs/gfx1250/gluon/gemm/gemm_a8w8_blockscale_preshuffled/GEMM-A8W8_BLOCKSCALE_PRESHUFFLED-N=4096-K=4096.json	Adds gfx1250 Gluon preshuffled specialized config (N=4096, K=4096).
aiter/ops/triton/configs/gfx1250/gluon/gemm/gemm_a8w8_blockscale_preshuffled/GEMM-A8W8_BLOCKSCALE_PRESHUFFLED-N=4096-K=2048.json	Adds gfx1250 Gluon preshuffled specialized config (N=4096, K=2048).
aiter/ops/triton/configs/gfx1250/gluon/gemm/gemm_a8w8_blockscale_preshuffled/GEMM-A8W8_BLOCKSCALE_PRESHUFFLED-N=32768-K=1024.json	Adds gfx1250 Gluon preshuffled specialized config (N=32768, K=1024).
aiter/ops/triton/configs/gfx1250/gluon/gemm/gemm_a8w8_blockscale_preshuffled/GEMM-A8W8_BLOCKSCALE_PRESHUFFLED-N=2048-K=7168.json	Adds gfx1250 Gluon preshuffled specialized config (N=2048, K=7168).
aiter/ops/triton/configs/gfx1250/gluon/gemm/gemm_a8w8_blockscale_preshuffled/GEMM-A8W8_BLOCKSCALE_PRESHUFFLED-N=16384-K=1536.json	Adds gfx1250 Gluon preshuffled specialized config (N=16384, K=1536).
aiter/ops/triton/configs/gfx1250/gluon/gemm/gemm_a8w8_blockscale_preshuffled/GEMM-A8W8_BLOCKSCALE_PRESHUFFLED-N=1536-K=7168.json	Adds gfx1250 Gluon preshuffled specialized config (N=1536, K=7168).
aiter/ops/triton/configs/gfx1250/gluon/gemm/gemm_a8w8_blockscale_preshuffled/GEMM-A8W8_BLOCKSCALE_PRESHUFFLED-N=1536-K=4096.json	Adds gfx1250 Gluon preshuffled specialized config (N=1536, K=4096).

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Copilot

Pull request overview

Copilot reviewed 2 out of 27 changed files in this pull request and generated 1 comment.

Copilot

Pull request overview

Copilot reviewed 2 out of 27 changed files in this pull request and generated 1 comment.

Copilot

Pull request overview

Copilot reviewed 3 out of 28 changed files in this pull request and generated 1 comment.

Copilot

Pull request overview

Copilot reviewed 3 out of 28 changed files in this pull request and generated no new comments.

Suppressed comments (1)

aiter/ops/triton/gluon/gemm_a8w8_blockscale.py:995

The device-availability error message still says "requires CDNA4", but arch_info.is_gluon_avail() returns true for both gfx950 and gfx1250. This message will mislead users running on gfx1250 (which is now documented as supported).

            "Gluon implementation is not supported on this device (requires CDNA4)."
        )

    # This family's configs live in the nested layout, so its directory is the
    # only candidate and the returned name prefix is always empty: nested files

Copilot

Pull request overview

Copilot reviewed 3 out of 28 changed files in this pull request and generated 1 comment.

Suppressed comments (1)

aiter/ops/triton/gluon/gemm_a8w8_blockscale.py:991

The error message is out of date: arch_info.is_gluon_avail() currently allows both gfx950 and gfx1250, so saying it "requires CDNA4" is misleading and makes triage harder on non-gfx950 devices.

            "Gluon implementation is not supported on this device (requires CDNA4)."

…onfig resolution Add resolve_config_dir(), the shared op-parameterized directory probe from configs/CLAUDE.md section 5 step 1, and refactor _get_gemm_config_cached onto it behavior-identically. Rewrite the last hand-built config path in gluon/gemm_a8w8_blockscale.py to use the probe (the custom _SUPPORTED_TILES bucket walk is preserved), move the 25 gluon GEMM-A8W8_BLOCKSCALE(_PRESHUFFLED) configs from configs/gemm/gluon/ into configs/<arch>/gluon/gemm/<d_type>/, and drop the flat-layout fallback from the caller: with its own family nested, the nested directory is the only candidate it needs. Includes the gluon README update from review.

brunomazzottiamd

@Boss2002n, I was deliberately very strict with the resolve_config_dir function. I think it's a very important function and it should have a solid implementation and documentation. So, I started many related discussions... please don't get me wrong, it's for our own good!

Once this PR is merged, we can proceed faster with the following ones.

Boss2002n · 2026-08-26T15:27:30Z

This PR is just an intermediary PR so that the other Config move PRs do not fail CI
I will address all these comments in my utils cleanup PR

utils/ reorganized for reuse: ONE config_utils.py absorbs core.py and the gemm/conv/mhc/tuned config_utils modules, laid out in five clearly-banded sections (shared core, GEMM, conv, MHC, tuned kernel entries) so each op family stays visibly separate inside the single import surface. The duplicate utils/mha_kernel_utils.py is dropped in favor of the byte-equal utils/_triton copy; every importer tree-wide is updated. gmm_common stays put under utils/ so tests and benchmarks keep importing a public module. The MHC section is absorbed verbatim and still reads the legacy flat layout -- the MHC config-file migration later flips only this section's path builders. resolve_config_dir() gets the contract the #4947/#5019 reviews asked for, hardened: an Args/Returns/Raises/Example docstring; every argument validated against a whitelist with assertions (op, config_name and the programmer-written arch= override must match strict identifier patterns, the driver-derived running arch tolerates vendor formats but must stay path-safe, backend is checked against ("triton", "gluon")) so a bad value fails closed instead of resolving an escaped or wrong directory; a single str return value (the unused name_prefix tuple element is gone, no caller unpacks a tuple); and fully deterministic behavior -- the directory is built, never probed, so nothing silently returns a bogus last candidate: a missing required table fails loudly in load_config_json() naming the exact nested path, and optional tables pass required=False. The MOE dispatch loaders now declare their backend per dispatch path (a8w4's triton path resolves backend="triton", the gluon paths "gluon"; a4w4 is gluon-only), and gfx1250 gains byte-identical triton seeds copied from gfx950 (GEMM-AFP4WFP4, moe A8W4, HSTU fwd/bwd) so the one resolution the caller-declared policy changes -- triton GEMM-AFP4WFP4 on gfx1250 no longer borrowing gluon tables -- stays launchable until gfx1250 is tuned natively. Gluon configs are never copied across arches. Conv's section carries the variant-aware walk from #4869 unchanged (its M-bounds renamed CONV_STANDARD_M_BOUNDS to coexist with GEMM's).

utils/ reorganized for reuse: ONE config_utils.py absorbs core.py and the gemm/conv/mhc/tuned config_utils modules, laid out in five clearly-banded sections (shared core, GEMM, conv, MHC, tuned kernel entries) so each op family stays visibly separate inside the single import surface. The duplicate utils/mha_kernel_utils.py is dropped in favor of the byte-equal utils/_triton copy; every importer tree-wide is updated. gmm_common stays put under utils/ so tests and benchmarks keep importing a public module. resolve_config_dir() gets the contract the #4947/#5019 reviews asked for, hardened: an Args/Returns/Raises/Example docstring; every argument validated against a whitelist with assertions (op, config_name and the programmer-written arch= override must match strict identifier patterns, the driver-derived running arch tolerates vendor formats but must stay path-safe, backend is checked against ("triton", "gluon")) so a bad value fails closed instead of resolving an escaped or wrong directory; a single str return value (the unused name_prefix tuple element is gone, no caller unpacks a tuple); and fully deterministic behavior -- the directory is built, never probed, and the legacy escape hatches are deleted, so nothing silently returns a bogus last candidate: a missing required table fails loudly in load_config_json() naming the exact nested path, and optional tables pass required=False. The MOE dispatch loaders now declare their backend per dispatch path (a8w4's triton path resolves backend="triton", the gluon paths "gluon"; a4w4 is gluon-only), and gfx1250 gains byte-identical triton seeds copied from gfx950 (GEMM-AFP4WFP4, moe A8W4, HSTU fwd/bwd) so the one resolution the caller-declared policy changes -- triton GEMM-AFP4WFP4 on gfx1250 no longer borrowing gluon tables -- stays launchable until gfx1250 is tuned natively. Gluon configs are never copied across arches. Conv's section carries the variant-aware walk from #4869 unchanged (its M-bounds renamed CONV_STANDARD_M_BOUNDS to coexist with GEMM's).

Boss2002n requested review from a team and brunomazzottiamd and a lite review from Copilot August 23, 2026 14:03

github-actions Bot changed the title ~~Satya/unify gluon a8w8 blockscale config resolution~~ [Triton/Gluon] Satya/unify gluon a8w8 blockscale config resolution Aug 23, 2026

Copilot started reviewing on behalf of Boss2002n August 23, 2026 14:04 View session

Boss2002n changed the base branch from main to satya/migrate-configs-gemm-a8w8-blockscale August 23, 2026 14:04

Boss2002n changed the base branch from satya/migrate-configs-gemm-a8w8-blockscale to main August 23, 2026 14:05

Boss2002n mentioned this pull request Aug 23, 2026

[Triton/Gluon] Remove legacy flat-layout fallback from GEMM config resolution #4948

Merged

Copilot AI reviewed Aug 23, 2026

View reviewed changes

Comment thread aiter/ops/triton/utils/gemm_config_utils.py

Copilot AI review requested due to automatic review settings August 24, 2026 19:30

Boss2002n force-pushed the satya/unify-gluon-a8w8-blockscale-config-resolution branch from 409dde5 to d9ccb7d Compare August 24, 2026 19:30

Copilot started reviewing on behalf of Boss2002n August 24, 2026 19:31 View session

Copilot AI reviewed Aug 24, 2026

View reviewed changes

Comment thread aiter/ops/triton/gluon/gemm_a8w8_blockscale.py Outdated

Copilot AI mentioned this pull request Aug 24, 2026

Update Gluon GEMM-A8W8_BLOCKSCALE README to match nested config layout and supported archs #4973

Merged

Copilot AI review requested due to automatic review settings August 24, 2026 23:39

Boss2002n force-pushed the satya/unify-gluon-a8w8-blockscale-config-resolution branch from d9ccb7d to 79c2560 Compare August 24, 2026 23:40

Copilot started reviewing on behalf of Boss2002n August 24, 2026 23:40 View session

Copilot AI reviewed Aug 24, 2026

View reviewed changes

Comment thread aiter/ops/triton/utils/gemm_config_utils.py

Copilot AI review requested due to automatic review settings August 25, 2026 05:10

Copilot started reviewing on behalf of Boss2002n August 25, 2026 05:11 View session

Copilot AI reviewed Aug 25, 2026

View reviewed changes

Comment thread aiter/ops/triton/gluon/gemm_a8w8_blockscale.py

Copilot AI review requested due to automatic review settings August 25, 2026 05:41

Copilot started reviewing on behalf of Boss2002n August 25, 2026 05:41 View session

Copilot AI reviewed Aug 25, 2026

View reviewed changes

Copilot AI review requested due to automatic review settings August 26, 2026 04:55

Boss2002n force-pushed the satya/unify-gluon-a8w8-blockscale-config-resolution branch from e38d981 to 7d078b3 Compare August 26, 2026 04:55

Copilot started reviewing on behalf of Boss2002n August 26, 2026 04:56 View session

Copilot AI reviewed Aug 26, 2026

View reviewed changes

Comment thread aiter/ops/triton/gluon/gemm_a8w8_blockscale.py

Boss2002n force-pushed the satya/unify-gluon-a8w8-blockscale-config-resolution branch from 7d078b3 to 34e855c Compare August 26, 2026 11:05

brunomazzottiamd reviewed Aug 26, 2026

View reviewed changes

Comment thread aiter/ops/triton/utils/gemm_config_utils.py

Comment thread aiter/ops/triton/utils/gemm_config_utils.py

Comment thread aiter/ops/triton/utils/gemm_config_utils.py

Boss2002n requested a review from brunomazzottiamd August 26, 2026 16:15

brunomazzottiamd approved these changes Aug 26, 2026

View reviewed changes

This was referenced Aug 26, 2026

[Triton/Gluon] Migrate the MHC tuned configs to the nested layout #5021

Merged

[Triton/Gluon] Consolidate and reorganize ops/triton utils #5023

Closed

Boss2002n merged commit 948fee0 into main Aug 26, 2026
69 checks passed

Boss2002n deleted the satya/unify-gluon-a8w8-blockscale-config-resolution branch August 26, 2026 19:26

Uh oh!

Conversation

Boss2002n commented Aug 23, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions Bot commented Aug 23, 2026

🏷️ CI Guide

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

brunomazzottiamd left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Boss2002n commented Aug 26, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Boss2002n commented Aug 23, 2026 •

edited

Loading