Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
92 commits
Select commit Hold shift + click to select a range
0f922e6
[Config] Add Qwen3.8-Flash-Next PTPC-FP8 gfx942 a8w8 bpreshuffle and …
zovonoir Sep 30, 2026
b1a81bf
[ASM] [HIP] Add gfx950 MXFP4 SiTUv2 FLAT 16x192/16x128/16x32 kernels …
JohnNikolay84 Sep 30, 2026
1e04738
add fmoe tunings (#5904)
cpersson-amd Sep 30, 2026
d0c104d
Tune a8w8 bpreshuffle for Kimi-K3's KDA gate at tp4 (#5783)
XiaobingSuper Sep 30, 2026
5fc771e
[FlyDSL] Paged mla indexer (#4221)
fhuizing Sep 30, 2026
5240e88
[Triton/Gluon] [GFX950] Fix LDS OOM on mxfp8 triton gemm (#5963)
k50112113 Sep 30, 2026
e0cb2dd
[CI] Triton board: drive PR status from the workflow (#5880)
Boss2002n Sep 30, 2026
301f09d
feat(flydsl): add fused KDA decode kernels on gfx950
yuychang Sep 22, 2026
910e4d2
[Triton/Gluon] Move the chunk_delta_attn kernel directories to kimi_d…
Boss2002n Sep 30, 2026
48b1365
[Triton/Gluon] [Kernel] Add fused SiLU-and-multiply backward (#5542)
DaiXindi-AMD Sep 30, 2026
b68b0e5
[Triton/Gluon] Unified attention sglang compatability fix (#6009)
cagrikymk Sep 30, 2026
84061e3
Merge branch 'main' into feat/kimi-k3-kda-decode-flydsl
yuychang Oct 1, 2026
e69c977
fix(flydsl): resolve KDA decode CI lint errors
yuychang Oct 1, 2026
b67e185
fix(flydsl): use 64-bit slot offsets for KDA decode caches
yuychang Oct 1, 2026
88959d7
refactor(flydsl): drop vendored vector module from KDA decode
yuychang Oct 1, 2026
d11a9f6
fix(flydsl): bound KDA decode cache slots and specialization caches
yuychang Oct 1, 2026
82c4f74
[OPUS] Speed up the gfx1250 32mx1 MLA sparse-prefill kernels (#5805)
kaiyang-1 Oct 1, 2026
f1a2bad
configs: add DSR1 gfx1250 bf16 GEMM tuning (#5939)
demonsan Oct 1, 2026
44863a6
[HIP] [FlyDSL] [JIT] [gfx1250][GEMM] add MXFP8 1x32 bpreshuffle GEMM …
aoli26 Oct 1, 2026
4ee3874
[ASM] [HIP] [FlyDSL] Add pin gpr(nhead96/128) and round-robin DCP to …
xiangM99 Oct 1, 2026
45c5795
[HIP] [CK] Add relu2 (Squared ReLU) activation support to CK-Tile fus…
shantipriya-amd Oct 1, 2026
e78293d
[Triton] [Config] Add gfx950 preshuffled AFP4WFP4 GEMM config for N=4…
mjkvaak-amd Oct 1, 2026
fb32199
[Triton/Gluon] Fix pa_decode compile error with newer Triton: don't i…
zhanglx13 Oct 1, 2026
03aa566
[Triton/Gluon] [Config] Add gfx1250 A16W16 GEMM configs for Wan2.2 sh…
sogalin Oct 1, 2026
7b68813
[Triton/Gluon] Assert SPLIT_UNMASKED_LOOP is not used with sliding wi…
amd-xavierwang Oct 1, 2026
fc4f597
[Triton/Gluon] Fix fp8 KV correctness reference in bench_unified_atte…
mpashkovskii Oct 1, 2026
9cb2629
[Triton/Gluon] [Config] Tune gfx1201 unified attention 2d for D=512 f…
GodRishUniverse Oct 1, 2026
c0ce943
[Triton/Gluon] Account for sliding_window in bench_unified_attention …
mpashkovskii Oct 1, 2026
0178bd3
Skip MXFP4 logits tests on unsupported GPUs (#5958)
brunomazzottiamd Oct 1, 2026
569d981
Pin Conv3D CPU test inputs to the CPU device (#5984)
whx-sjtu Oct 1, 2026
fbbd9f6
[Triton] Use a zero default for masked scales in gemm_afp8wfp8 (#6007)
cagrikymk Oct 1, 2026
3412918
[Triton/Gluon] gemm_a16w16_atomic accumulate into an existing output …
nsusanto Oct 1, 2026
9d56f8b
[Triton/Gluon] MHA fwd: gfx950 mid_head config for 64 < head_dim <= 1…
NimitPtl Oct 1, 2026
282b311
[Triton/Gluon] gmm: gfx950 large-K/N config with JSON-driven shape di…
NimitPtl Oct 1, 2026
cd06839
[Triton/Gluon] Move Triton unit tests into their wrapper's folder (#5…
Boss2002n Oct 1, 2026
aaafddd
[Triton/Gluon] Expert-parallel support for moe_gemm_a16w4 (#5863)
Rohan138 Oct 1, 2026
52298dd
[Triton/Gluon] Add 3D neighborhood flash attention (na3d_flash) (#5230)
jjuvonen-amd Oct 2, 2026
635c10f
[Bugfix][CK] Keep two K tiles per split in blockscale MoE stage-1 spl…
jin-amd Oct 2, 2026
674a43a
[Triton/Gluon] tuning k3 shapes for a16w16 (#5875)
omuhamma Oct 2, 2026
0369423
gemm_a16w16: gfx950 config for GLM-5.3-Flash router GEMM (N=288, K=40…
Jacob0226 Oct 2, 2026
332b34f
[FlyDSL] Dev/a4w4 prefill v2 yadai and tuned config for dsr1 (#5934)
yadaish Oct 2, 2026
a1455f1
[CK] [FlyDSL] add mxfp8 a8w8 blockscale for gfx950 (#4951)
solinzby1 Oct 2, 2026
f9e5492
[FlyDSL] topk_select: allow gfx1250 (#6080)
HaonanWang98 Oct 2, 2026
9c15779
[Triton/Gluon] [HIP] gemm_a16w16: gfx950 config for MiniMax-M3 lm_hea…
valarLip Oct 2, 2026
3eca6e4
[Triton/Gluon] fix gluon import and benchmark + tune fused_clamp_act_…
nidal567 Oct 2, 2026
edb9107
[Triton/Gluon] [Config] gfx950 unified attention prefill tile for hea…
prashant182 Oct 2, 2026
847ebdc
[Triton/Gluon] spec decode support for gluon kda files (#6034)
omuhamma Oct 2, 2026
446b8e9
[Triton/Gluon] Gluon Chunked Kimi Delta Attention (950/1250) (#5866)
omuhamma Oct 2, 2026
01ad7da
[Triton/Gluon] Changes related to gla for Kimi K3 (#6056)
omuhamma Oct 2, 2026
c847570
[Triton/Gluon] enable TDM fusion (#6097)
Boss2002n Oct 2, 2026
2964cf9
[Triton/Gluon] [Bugfix] batched_gemm_a8w8: keep scale loads in bounds…
akii96 Oct 2, 2026
3276d64
[CI] Remove committed scratch files and stop them coming back (#5544)
Boss2002n Oct 2, 2026
394cc86
[Triton/Gluon] Add a16w4 MoE tuned dispatch and gfx942 DSv4.1 tables …
akii96 Oct 3, 2026
dbf4f3a
[ASM][flydsl] opt mla kernel for dspr1(gfx1250) (#6087)
xiangM99 Oct 3, 2026
c99c361
[Triton/Gluon] [Perf] MXFP8 MoE: keep the MFMA in FP8 when MX_SCALE_B…
akii96 Oct 3, 2026
e076dd9
[CI] [JIT] Fix ATOM GPU isolation and startup failures (#5989)
leo-automation Oct 3, 2026
45c5ad0
[FlyDSL] [Config] gfx950 A8W4 fused_moe configs for DSv4.1 TP-sharded…
valarLip Oct 4, 2026
56187c4
fix(custom_all_reduce): allocate IPC handle tensors on the host (#6129)
valarLip Oct 4, 2026
7bfe279
[ASM] [HIP] [MLA v4 nm] Split plan helper and persistent decode kerne…
yhl-amd Oct 4, 2026
c7f5d8c
Merge branch 'main' into feat/kimi-k3-kda-decode-flydsl
yuychang Oct 5, 2026
f4be7f1
[FlyDSL] MegaMoEV2: decode/prefill optimizations and Stage1 LDS broad…
yhl-amd Oct 5, 2026
11e8354
Revert "[HIP] [CK] Add relu2 (Squared ReLU) activation support to CK-…
shantipriya-amd Oct 5, 2026
474fc6b
fix(gluon): harden paged MQA logits prefetch masks (context_length + …
fallow5 Oct 5, 2026
2211f24
[Triton/Gluon] DSR1 GEMM tuning (#5867)
azaidy Oct 5, 2026
4c25224
[Triton/Gluon] add multicast support to gfx1250 MXFP4 (#6118)
Boss2002n Oct 5, 2026
1eb7de5
[Triton/Gluon] Fix Triton Conv2D/Conv3D caching for inference tensor …
saeid-rostami Oct 5, 2026
6c17917
[Triton/Gluon] Add lds-pipelined MLA variant (#6098)
AlexAUT Oct 5, 2026
741d3ea
[Triton/Gluon] Add per token scale to a8w8 moe gemm (#6106)
nsusanto Oct 5, 2026
b83cf64
[Triton/Gluon] Add gluon support for MXFP4 and MXFP8 quant kernel in …
NimitPtl Oct 5, 2026
aed3058
[Triton] [Bench] Fix bench_moe_gemm_int8_smoothquant and bench_moe_ge…
zhanglx13 Oct 5, 2026
eacab04
[Triton] [Bench] Remove bench_gemm_afp4wfp4_pre_quant_atomic (#6073)
zhanglx13 Oct 5, 2026
326cffa
[Triton/Gluon] [Config] : Add Inkling Model GEMM Configs (#5922)
rahulbatra85 Oct 5, 2026
356c53d
[CK] Add dsv41 some MoE tuning configs (#5967)
kkHuang-amd Oct 6, 2026
6a5ac94
[Triton/Gluon] Satya/dsr1 tuning oct4th (#6157)
Boss2002n Oct 6, 2026
5b2f7d1
[HIP] [FlyDSL] feat(flydsl): row-group FP4 paged MQA logits for 8- an…
valarLip Oct 6, 2026
addda10
perf(ci): init each multi-GPU test's process group once, not per case…
valarLip Oct 6, 2026
4b12dc5
[Triton/Gluon] [gfx942] Enable sparse_mla_fwd on gfx942 (#5721)
jin-amd Oct 6, 2026
1d447f9
feat(configs): DeepSeek-V4.1-Flash TP4 bf16 GEMMs in the dsv41 bf16 t…
valarLip Oct 6, 2026
d0029e8
[Triton/Gluon] Remove bad moe gemm wrapper code (#6035)
lburzawa Oct 6, 2026
f32a197
[Triton/Gluon] [CI] [Kimi-K3] Optimize speculative KDA decode on gfx9…
xiaohuguo2023 Oct 6, 2026
5b2fd3b
[Triton/Gluon] [Config] k3 gfx950 tuning bf16 tp8 shapes (#6019)
nsusanto Oct 6, 2026
8cfa090
[Triton/Gluon] Gluon PA decode: fix VGPR blow-up on Triton 3.8 (#5909)
sshlyapn Oct 6, 2026
1a37aa0
[CI] Run full AITER releases weekly at 1 a.m. Central (#6182)
vgokhale Oct 6, 2026
4da9350
[Triton/Gluon] Fix MXFP4 and MXFP8 fallback for older Triton (#6181)
vgokhale Oct 6, 2026
dd6b890
[Triton/Gluon] Fused Qwen3-Next GDN decode op for gfx950 (#5520)
rbrugaro-amd Oct 6, 2026
72020c0
[Triton/Gluon] unified_attention: mask V load and output store by val…
reger-men Oct 6, 2026
e8048a1
[Test] Add SDXL 1.0 conv2d shapes to conv_shapes.json (#4444)
Ragua1 Oct 6, 2026
42c7e4a
[Triton/Gluon] Add gluon support for MXFP4 and MXFP8 quant kernel in …
NimitPtl Oct 7, 2026
e4ceb9d
[CI] Add stale pull request label (#5540)
Boss2002n Oct 7, 2026
1f1983b
Merge main and preserve FlyDSL KDA and FP4 MQA exports
yuychang Oct 7, 2026
2d207f2
fix(flydsl): validate KDA int32 strides and byte spans
yuychang Oct 7, 2026
ecdcca5
fix(flydsl): reject overlapping KDA cache and output layouts
yuychang Oct 7, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
The table of contents is too big for display.
Diff view
Diff view
  •  
  •  
  •  
58 changes: 51 additions & 7 deletions .github/instructions/aiter-ops-triton.instructions.md
Original file line number Diff line number Diff line change
Expand Up @@ -45,9 +45,10 @@ that duplicates functionality already in the tree, even partially — the fix is
to extend or import the existing implementation, not to add a parallel copy.

`utils/` is layered on purpose: `config_utils.py` holds the shared core
(`resolve_config_dir`, `load_config_json`, the path constants) and each family
keeps its own loader module (`gemm_config_utils`, `conv_config_utils`,
`mhc_config_utils`, `moe_config_utils`, `tuned_config_utils`) on top of it.
(`resolve_config_dir`, `load_config_json`, `select_leq_config`, the path
constants) and each family keeps its own loader module (`gemm_config_utils`,
`conv_config_utils`, `mhc_config_utils`, `moe_config_utils`,
`quant_config_utils`, `tuned_config_utils`) on top of it.
Flag a function given a second home — a re-export, a wrapper that only
forwards to another module, or a copy of a core helper inside a family module.

Expand All @@ -69,6 +70,19 @@ and tuned JSON in `configs/`. Flag:
Gluon bodies under `_gluon_kernels/<arch>/`, mirroring the wrapper's
category path. JIT-decorated device helpers that are called only from
another kernel may remain with the entry kernel they support.
- A kernel module or directory whose path under `_triton_kernels/`
(`_gluon_kernels/<arch>/`) differs from the folder of the wrapper that
launches it. A kernel directory is named after the wrapper folder, never
after the kernel family or a topic: `normalization/`, not `norm/`. One
existing directory predates this rule and is grandfathered until a
dedicated follow-up moves it and all its importers: the vendored
`_triton_kernels/flash_attn_triton_amd/` (launched by `attention/mha.py`).
Do not flag code in it; flag any new mismatched directory. A kernel that
wrappers in several folders reuse lives with the wrapper that owns it, or in
`_triton_kernels/common/` when none does (`common/splitk_reduce.py`);
importing it across folders is fine. A PR that moves a wrapper into another
folder moves its kernels too and updates every importer of the old module
path (`grep -rn "_triton_kernels.<old>"`).
- Generic helpers (config loading, shuffling, arch detection, logging)
re-implemented inside a kernel file instead of imported from `utils/`.
- New code importing via the legacy flat paths
Expand Down Expand Up @@ -104,7 +118,7 @@ their tuned configs can be imported by a framework that is not PyTorch
Every tuned config lives in one nested layout:
`configs/<arch>/<backend>/<op>/<d_type>/`, e.g.
`configs/gfx950/triton/gemm/gemm_afp4wfp4/DEFAULT.json`. `<op>` is `gemm`,
`moe`, `conv`, `mhc`, `attention`, `gmm` or `fusions`; `<d_type>` is
`moe`, `conv`, `mhc`, `attention`, `gmm`, `fusions` or `quant`; `<d_type>` is
`config_name.lower().replace("-", "_")`. The flat arch-prefixed directories
and every fallback that reached them are gone. Flag:

Expand Down Expand Up @@ -215,14 +229,19 @@ values for either backend live in JSON, never in Python. Flag:
family loader or `resolve_config_dir()` would work — a hand-built path is a
second place the layout is encoded, and it skips the argument validation
that makes a wrong value fail closed.
- A hand-written loop selecting the smallest matching `N_LEQ_*` (or another
upper-bound prefix) entry, or a hand-written multi-axis bucket walk — use
`select_leq_config()` (`axes=` for several axes) so threshold ordering,
fallback, and copying semantics have one implementation.
- A second MOE config reader. `utils/moe_config_utils.py::get_moe_dispatch` is
the only MOE fetcher; flag any new MOE path built by hand, any direct
`load_config_json` on a `moe/` file, and any reintroduced per-wrapper MOE
loader.
- A new arch- or backend-fallback chain inside a loader (try this arch, then
that one; try triton, then gluon). Resolution is deterministic. MHC's gfx942
fallback is the one documented exception and it goes through the `arch=`
override, not through a probe.
that one; try triton, then gluon). Resolution is deterministic. The
documented compatibility exceptions are MHC's gfx942 fallback and Triton
`fused_clamp_act_mul`'s legacy gfx950 fallback; both use the `arch=` override
instead of a probe.
- A raw config list handed to `@triton.autotune`. Route it through
`autotune_configs` from `aiter.ops.triton.utils.tuned_config_utils`:

Expand Down Expand Up @@ -349,6 +368,31 @@ All weight/scale pre-shuffle helpers are unified in
inside the category folder, shared helpers in the existing
`*_test_utils.py` / `utils/` modules. Flag tests added flat at the
`op_tests/triton_tests/` root or as one-off scripts.
- A test's folder is the folder of the wrapper it imports. The path under
`op_tests/triton_tests/` mirrors the wrapper's path under
`aiter/ops/triton/`: a test of `aiter/ops/triton/attention/mla.py` is
`op_tests/triton_tests/attention/test_mla.py`, a test of
`gemm/basic/gemm_a8w8.py` is `gemm/basic/test_gemm_a8w8.py`, a test of
`gated_delta_net/fused_kda_decode.py` is
`gated_delta_net/test_fused_kda_decode.py`. The wrapper's folder decides
the test folder, not the kernel directory: `attention/mha.py` keeps its
tests in `triton_tests/attention/` even though its kernels sit in the
vendored `_triton_kernels/flash_attn_triton_amd/`; a sub-package counts as
a folder (`moe/moe_routing/routing.py` → `triton_tests/moe/moe_routing/`).
The few wrappers that still sit flat at `aiter/ops/triton/<op>.py`
(`activation.py`, `topk.py`, ...) keep their tests flat at the
`triton_tests/` root; `utils/` helpers are tested under
`triton_tests/utils/`; `torch_compile/` and `triton_metadata_redirect/`
are infrastructure suites, not op folders. Flag:
- A test whose folder differs from the folder of the wrapper it imports
(`from aiter.ops.triton.gated_delta_net...` in a test under
`triton_tests/attention/` or at the root).
- A test placed by kernel directory or by topic instead of by wrapper
folder, and a new test folder that does not match a folder under
`aiter/ops/triton/` (every op test folder also carries an `__init__.py`).
- A PR that moves a wrapper into another folder without moving its kernels
and its test, or that moves a test without updating every importer of the old module
path (`grep -rn "triton_tests.<old>"`).
- No kernel tuning configs in test files: flag test code that hardcodes
config dicts (`BLOCK_SIZE_*`, `num_warps`, `waves_per_eu`, ...) or passes
literal `config=` overrides to a wrapper. Tests exercise the wrapper's own
Expand Down
53 changes: 53 additions & 0 deletions .github/scripts/prebuild_atom_kimi_k3.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,53 @@
#!/usr/bin/env python3
# SPDX-License-Identifier: MIT
# Copyright (C) 2026, Advanced Micro Devices, Inc. All rights reserved.

"""Build Kimi-K3's slow GEMM dependency before the ATOM startup deadline."""

import importlib
import sys
from pathlib import Path

MODULE = "module_gemm_a8w8_bpreshuffle_cktile"
EXPECTED_ARCH = "gfx950"


def main() -> None:
root = Path(__file__).resolve().parents[2]
# Use the build-time import path without initializing AITER's runtime.
sys.path.insert(0, str(root / "aiter"))
core = importlib.import_module("jit.core")

args = core.get_args_of_build(MODULE)
core.build_module(
md_name=MODULE,
srcs=args["srcs"],
flags_extra_cc=args["flags_extra_cc"],
flags_extra_hip=args["flags_extra_hip"],
blob_gen_cmd=args["blob_gen_cmd"],
extra_include=args["extra_include"],
extra_ldflags=args["extra_ldflags"],
verbose=args["verbose"],
is_python_module=args["is_python_module"],
is_standalone=args["is_standalone"],
torch_exclude=args["torch_exclude"],
third_party=args.get("third_party", []),
hipify=args.get("hipify", False),
flags_extra_hip_per_source=args.get("flags_extra_hip_per_source", {}),
)

target = Path(core.get_user_jit_dir()) / f"{MODULE}.so"
expected = root / "aiter" / "jit" / target.name
if target.resolve() != expected.resolve() or not target.is_file():
raise RuntimeError(f"Prebuild did not create {expected}: got {target}")

arches = core._so_offload_archs(target)
if arches != {EXPECTED_ARCH}:
raise RuntimeError(f"Unexpected GPU architectures: {sorted(arches)}")
print(
f"Prebuilt {MODULE}: {target.stat().st_size} bytes, {EXPECTED_ARCH}", flush=True
)


if __name__ == "__main__":
main()
2 changes: 1 addition & 1 deletion .github/scripts/split_tests.sh
Original file line number Diff line number Diff line change
Expand Up @@ -334,6 +334,7 @@ elif [[ "$TEST_TYPE" == "triton" ]]; then
FILE_TIMES[op_tests/triton_tests/moe/test_moe_routing_herd.py]=14
FILE_TIMES[op_tests/triton_tests/test_fused_rearrange_sigmoid_gdr.py]=14
FILE_TIMES[op_tests/triton_tests/attention/test_pa_prefill_sparse.py]=12
FILE_TIMES[op_tests/triton_tests/gated_delta_net/test_fused_kda_decode.py]=12
FILE_TIMES[op_tests/triton_tests/gemm/basic/test_gemm_a16wfp4.py]=12
FILE_TIMES[op_tests/triton_tests/gemm/basic/test_gemm_a8wfp4.py]=12
FILE_TIMES[op_tests/triton_tests/chunk_delta_attn/test_fast_launch.py]=10
Expand All @@ -357,7 +358,6 @@ elif [[ "$TEST_TYPE" == "triton" ]]; then
FILE_TIMES[op_tests/triton_tests/torch_compile/test_compile_activation.py]=5
FILE_TIMES[op_tests/triton_tests/attention/test_sparse_attention_dsv4_bwd.py]=4
FILE_TIMES[op_tests/triton_tests/fusions/test_fused_rmsnorm_indexed_adaln.py]=4
FILE_TIMES[op_tests/triton_tests/test_fused_kda_decode.py]=4
FILE_TIMES[op_tests/triton_tests/test_softmax.py]=4
FILE_TIMES[op_tests/triton_tests/torch_compile/test_compile_fused_mul_add.py]=4
FILE_TIMES[op_tests/triton_tests/torch_compile/test_compile_quant_per_token.py]=4
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/actionlint.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -26,7 +26,7 @@ jobs:

- name: Install actionlint
env:
ACTIONLINT_VERSION: "1.7.7"
ACTIONLINT_VERSION: "1.7.12"
run: |
set -euo pipefail
curl -fsSL \
Expand Down
Loading
Loading